Cost models for LookupCoin, ValueContains, ValueData, UnValueData builtins

[Unisay]

Summary

This PR implements cost modeling for Value-related builtins: lookupCoin, valueContains, valueData, and unValueData.

Implementation

Complete cost modeling pipeline:

• Cost model infrastructure and parameter definitions
• Benchmarking framework with realistic Cardano constraints
• Statistical analysis with R models (linear/constant based on performance characteristics)
• Updated JSON cost model configurations across all versions

Cost models:

• valueData: Uses constant cost model based on uniform performance analysis
• lookupCoin: Linear cost model with dimension reduction for 3+ parameters
• valueContains: Linear cost model for container/contained size dependency
• unValueData: Linear cost model for size-dependent deserialization

All functions now have proper cost models instead of unimplemented placeholders.

[github-actions]

PR Preview Action v1.6.2
🚀 View preview at https://IntersectMBO.github.io/plutus/pr-preview/pr-7344/
Built to branch gh-pages at 2025-09-19 08:01 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[kwxm]

Here are some initial comments. I'll come back and add some more later. I need to look at the benchmarks properly though.

[zliu41]

If the keys are completely random, then lookupCoin will probably never hit an existing entry, right?

[kwxm]

lookupCoin will probably never hit an existing entry,

Maybe that's what we want? Do we know if finding out that something's not in the map is the worst case? Naively you might think that the time taken to discover that some key is not in the map is always greater or equal to the time taken to find a key that is in the map.

[ana-pantilie]

I don't agree. I think we should actively include both the case when the map contains the key and when it doesn't. Otherwise we're not really measuring this case, and that's the whole point of benchmarking, right? Otherwise we would just use the, analytically discovered, worst-time complexity of the algorithm and pick a function from that category for its cost, right?

[ana-pantilie]

Also, as I mentioned above, you won't have a good idea of the actual size of the Value if you don't enforce uniqueness of the keys.

[Unisay]

I have simplified the generators (less fixed values, more randomly generated samples, quantities are all maxBound :: Int64)

After that I've re-benchmarked and re-generated cost models. This is how I view them:

LookupCoin

ValueContains

ValueData

UnValueData

CC: @kwxm

[zliu41]

In order to benchmark the worst case, I think you should also ensure that lookupCoin always hits the largest inner map (or at least, such cases should be well-represented).

Also, we'll need to re-run benchmarking for unValueData after adding the enforcement of integer range.

[kwxm]

GitHub seeems to think that the data for all of the BLS functions has changed, but I don't think they have.

[Unisay]

The file on master contains Windows-style line terminators (\r\n) for BLS lines:

git show master:plutus-core/cost-model/data/benching-conway.csv | grep "Bls12_381_G1_multiScalarMul/1/1" | od -c | grep -C1 "\r"
0000000   B   l   s   1   2   _   3   8   1   _   G   1   _   m   u   l
0000020   t   i   S   c   a   l   a   r   M   u   l   /   1   /   1   ,
0000040   8   .   2   3   2   1   3   4   7   0   4   7   1   2   0   4
--
0000200   8   7   1   1   e   -   8   ,   1   .   8   4   3   1   0   7
0000220   3   4   2   9   1   7   5   0   2   e   -   7  \r  \n

This PR changes \r\n to \n .

[zliu41]

you can simply generate 4 random bytes, instead of generating an integer and doing these bitwise operations.

[kwxm]

Maps are implemented as balanced binary trees with (key, value) pairs at the internal nodes, so if you've got a node containing key k then the left subtree of the node will only contain keys less than k and the right subtree keys greater than k. The functions that operate on maps are supposed to keepthe tree balanced, so the left subtree should be (approximately) the same size as the right one. When you've got 2^n-1 nodes the tree should be prefectly balanced and every path from the root to a leaf should have length n (ie, you should pass through n nodes as you travel from the root to an entry both of whose subtrees are empty (Tip)).

I think that to get the worst case behaviour for lookupCoin you can generate an outer map with 2^a - 1 unique keys for a in some range like 1..10 or 1..15, so you get a full tree. Looking for the entry with the largest key should then require searching all the way to the bottom of the tree (always branching to the right), which should be the worst case (and also make sure that all of the keys have a long common prefix to maximise the comparison time). The inner map for this longest case should also be a full binary tree, with 2^b - 1 entries for some b; we want the worst case to search the inner map as well, and in this case that should happen when you look for a key that's bigger than all of the keys in the inner map, since again you'll have to search all the way down the right hand side of the tree; I think that if you search for the biggest key it'll take pretty much the same time though. The total time taken should be proportional to a+b, since you'll have so examine a nodes in the outer map and b in the inner map, and since the keys are of the same type for inner and outer maps the time taken per node should be about the same for both (if the key types in the inner and outer maps were different then you might be looking at something of the form ra+sb, but here r and s should be the same, so you've got r(a+b)).

Note that a and b are the depths of the trees, which wil be integerLog2 of the number of entries, so I think here you want to use a size measure which is integerLog2(outer size) + integerLog2(maximum inner size).

[kwxm]

I guess you'll need to benchmark this over some set of pairs of depths (a,b) where a and b both vary over [1..15] or something, but that might take a long time since for that range you'd be running 225 benchmarks. I think that in fact the time will only depend on a+b, but initially we should check that that's true by looking at different values of a and b with the same sum.

[zliu41]

It might be an overkill to bother generating full trees. In any balanced binary tree the depths of any two leaves differ by at most 1, so as long as we make sure that we hit a leaf node (using either the smallest key or the largest key), 🤷

It's more important to make sure the outer key hits the largest inner map. I think the worst case is: there's a large inner map with N/2 keys, together with N/2 singleton inner maps, and the outer key hits that large inner map. Whether the outer map and the inner map are full trees shouldn't matter that much.

[zliu41]

So in summary, I would do this:

• Given total size N, let there be roughly N/2 inner maps: one big inner map whose size is roughly N/2, and the rest are singletons.
• Outer key should hit the big inner map.
• Both the outer key and the inner key should hit leaf nodes. So both keys should be either the min or the max key in the respective map (or the inner key may be absent in the inner map).

This should be very close to the worst case, if not the worst case. I wouldn't bother with varying a and b, or generating full trees, or anything like that.

[kwxm]

I wouldn't bother with varying a and b

I think it's worth doing that at least once just to make sure that the time taken depends only on the sum of a and b . If we can show that it does then we can just restrict the benchmarking to the case when a = 1, so you only have one entry in the outer map; alternatively, you can take b = 1 and just worry about the size of the outer map. I think we can effectively regard the entire map as one big tree: when you get to the tip of the outer map you move into one of the inner maps and continue searching there, so it's like the inner maps are glued onto the leaves of the outer map. Then all that matters is the total depth, a+b. I'd like to check that assumption before going any further though: it's better to have some evidence than just to guess what's going on.

[zliu41]

If we can show that it does then we can just restrict the benchmarking to the case when a = 1, so you only have one entry in the outer map; alternatively, you can take b = 1 and just worry about the size of the outer map.

You can't. lookupCoin is O(log max(m, k)). So it's better to balance the size of the outer map and the size of the largest inner map.

[zliu41]

In other words, given a size measure of 10, you can:

• Make the outer map have 1024 entries, and the largest inner map have 1024 entries
• Or make the outer map have 1024 entries, and the largest inner map have 1 entry
• Or make the outer map have 1 entry, and the largest inner map have 1024 entries

Obviously the first is the worst case

[kwxm]

You can't. lookupCoin is O(log max(m, k))

No, I don't think it is. I think it's O(log a + log b), or at least that log a + log b is a size measure that will give a more precise result than log(max(a, b) , and that's what my proposed experiment is trying to confirm. The O's are obscuring what's actually going on since they're hiding the details of the constants. For the sum the actual costing function will be of the form r + s(log a+ log b) and for the maximum it'll be of the form u + v*log(max(a, b)). Now log a+log b <= 2*max(log a, log b), with equality when a=b, so r+s*(log a + log b) <= r + 2*s*max(log a, log b), and when you put an O round them they become the same, but the right hand one can actually be almost twice the left hand one.

In other words, given a size measure of 10, you can:

Make the outer map have 1024 entries, and the largest inner map have 1024 entries
Or make the outer map have 1024 entries, and the largest inner map have 1 entry
Or make the outer map have 1 entry, and the largest inner map have 1024 entries

Obviously the first is the worst case

With the sum size measure the first map has a size of 20 and the other ones have a size of 11, but with the maximum they all have size 10, so the sum version picks out the worst case and the maximum doesn't. A stragiht line fitted to the benchmarking results using the sum measure should give us a more precise bound for the execution time than if we use the maximum measure. The sum is a more accurate measure of the total depth of a value than the maximum, and I think that the total depth is what we need to worry about.

[Unisay]

I made benchmarking and here is what it shows (LLM summary):

Two-Level Map Lookup Performance: Experimental Analysis

  Experimental Setup

  Data structure: Map ByteString (Map ByteString Natural)

  Key characteristics:
  - 32-byte ByteStrings with 28-byte common prefix (simulating hash-like keys)
  - Perfect binary tree structure (keys = 2^n - 1)
  - Worst-case lookup: deepest key in both outer and inner maps

  Test parameters:
  - Outer map depth: a (containing 2^a - 1 keys)
  - Inner map depth: b (containing 2^b - 1 keys)
  - Total depth: N = a + b

  ---
  Experiment 1: Distribution Impact (Constant Total Depth)

  Hypothesis: When N is held constant, how does the distribution of depth between outer and inner maps
  affect performance?

  Test configuration: N = 17, varying distributions from (1,16) to (16,1)

  Results: Distribution Has Minimal Impact

  | Distribution   | Outer Keys | Inner Keys | Lookup Time | Deviation from Mean |
  |----------------|------------|------------|-------------|---------------------|
  | (10,7)         | 1,023      | 127        | 1.657 μs    | +1.5%               |
  | (6,11)         | 63         | 2,047      | 1.655 μs    | +1.4%               |
  | (8,9) balanced | 255        | 511        | 1.644 μs    | +0.7%               |
  | (1,16) extreme | 1          | 65,535     | 1.607 μs    | -1.6%               |
  | (16,1) extreme | 65,535     | 1          | 1.598 μs    | -2.1%               |

  Range: 1.598 μs to 1.657 μs (3.6% variation)

  Key finding: Distribution choice has negligible impact on performance when total depth is constant.

  ---
  Experiment 2: Linear Scaling with Total Depth

  Hypothesis: Lookup time scales linearly with total depth N = a + b, regardless of distribution.

  Test configuration: N ∈ {10, 12, 14, 16, 18, 20}, three representative distributions per N

  Results: Strong Linear Relationship

  | N (Total Depth) | Representative Samples  | Min Time | Max Time | Average Time | Cost per Level |
  |-----------------|-------------------------|----------|----------|--------------|----------------|
  | 10              | (1,9), (5,5), (9,1)     | 125.6 ns | 128.3 ns | 127.0 ns     | —              |
  | 12              | (1,11), (6,6), (11,1)   | 132.9 ns | 142.9 ns | 136.9 ns     | +4.95 ns       |
  | 14              | (1,13), (7,7), (13,1)   | 140.1 ns | 154.0 ns | 144.5 ns     | +3.80 ns       |
  | 16              | (1,15), (8,8), (15,1)   | 145.3 ns | 160.4 ns | 151.1 ns     | +3.30 ns       |
  | 18              | (1,17), (9,9), (17,1)   | 152.7 ns | 167.6 ns | 158.6 ns     | +3.75 ns       |
  | 20              | (1,19), (10,10), (19,1) | 158.8 ns | 173.9 ns | 168.8 ns     | +5.10 ns       |

  Average cost per level: 4.18 ns

  Linear model: Time ≈ 80 ns + 4.2 ns × N (R² ≈ 0.99)

  ---
  Scaling Analysis

  | Test                   | Expected (if linear) | Observed    | Result     |
  |------------------------|----------------------|-------------|------------|
  | N=10 → N=20 (2x depth) | 2.00x time           | 1.33x time  | Sub-linear |
  | Per-level increment    | Constant             | 4.18 ns avg | ✅ Constant |

  Note: The sub-linear scaling (1.33x instead of 2x) suggests the baseline overhead is significant
  relative to per-level cost at small N values.

  ---
  Key Experimental Tendencies

  1. Distribution Independence (Constant N)
    - Variation between distributions: <4%
    - Expensive key comparisons dominate performance
    - Tree shape/cache effects are negligible
  2. Linear Depth Scaling
    - Each additional tree level: +4.2 ns
    - Baseline overhead: ~80 ns
    - Strong linear correlation (R² ≈ 0.99)
  3. Distribution Variation by Depth

  | Total Depth (N) | Max-Min Spread | % Variation |
  |-----------------|----------------|-------------|
  | 10              | 2.7 ns         | 2.1%        |
  | 12              | 10.0 ns        | 7.3%        |
  | 14              | 13.9 ns        | 9.6%        |
  | 16              | 15.1 ns        | 10.0%       |
  | 18              | 14.9 ns        | 9.4%        |
  | 20              | 15.1 ns        | 8.9%        |

  Pattern: Variation increases slightly at deeper depths but remains <10%

  ---
  Conclusions

  1. For expensive key comparisons (ByteStrings with common prefixes):
    - Lookup time is primarily determined by total depth (a + b)
    - Distribution choice (split between outer/inner) has minimal impact
  2. Performance model:
  Lookup Time ≈ 80 ns + 4.2 ns × (a + b)
  3. Practical implication:
    - When designing nested map structures with expensive keys, optimize for total depth minimization
  rather than specific distribution patterns
    - Tree balancing and cache optimization are secondary concerns
  4. Cost breakdown:
    - ~80 ns: Fixed overhead (function calls, setup)
    - ~4.2 ns per level: Key comparison + tree traversal

[zliu41]

I'm fine with using log m + log k instead of log (max m k). I don't think it really matters either way, but let's just make a decision and move forward with getting the costing done.

We need to stay on schedule for HF by EOY, and it is a firm deadline. If both @kwxm and @Unisay prefers log m + log k then go ahead with it.