Invalid transactions missing inputs and outputs (collateral operations)

[linconvidal]

Context & versions

• rosetta-java v1.3.2
• Network: preprod
• Transaction example: 2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8 (Block 3422820)
• Issue: rosetta-java only shows invalid operations (failed script attempts), missing the valid collateral operations that actually succeeded

Steps to reproduce

1. Start rosetta-java on preprod
2. Query invalid transaction via /block/transaction:

   curl -X POST http://localhost:8083/block/transaction \
     -H "Content-Type: application/json" \
     -d '{
       "network_identifier": {"blockchain": "cardano", "network": "preprod"},
       "block_identifier": {"index": 3422820, "hash": "5d8c7e399374b40693ec8c05e5003dc3b7c3ef7c7e04d68507371a1e262281be"},
       "transaction_identifier": {"hash": "2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8"}
     }'

3. Observe response shows only invalid script input operation
4. Verify script UTXO still exists using cardano-cli (wasn't actually consumed)
5. Note collateral operations are completely missing

Actual behavior

rosetta-java response (INCOMPLETE)

{
  "operations": [
    {
      "type": "input",
      "status": "invalid",  // ✓ Correct status - this operation failed
      "account": {"address": "addr_test1wr3cmj7uzdl89zkrgdqk55sn9ka3368stgaky3xm59elfrg49uvlm"},
      "amount": {"value": "-5000000"},  // ✓ Shows attempted amount
      "coin_change": {
        "coin_identifier": {"identifier": "39bdb7ca1f83ab998811a1b29c57e3bb24336e8c8e10a6fd19e662c2c4a0f29f:0"},
        "coin_action": "coin_spent"
      }
    }
  ]
}

Problem: Only shows invalid operations. Missing the VALID collateral operations that succeeded.

UTXO Search for 39bdb7ca...#0 (the script input shown with invalid status):

curl -X POST http://localhost:8083/search/transactions \
  -H "Content-Type: application/json" \
  -d '{"network_identifier": {"blockchain": "cardano", "network": "preprod"}, 
       "coin_identifier": {"identifier": "39bdb7ca1f83ab998811a1b29c57e3bb24336e8c8e10a6fd19e662c2c4a0f29f:0"}}'

Response:

{
  "total_count": 1,  // ONLY ONE TRANSACTION - THE CREATION!
  "transactions": [
    {
      "block_identifier": {"index": 3413492},
      "transaction": {
        "transaction_identifier": {"hash": "39bdb7ca1f83ab998811a1b29c57e3bb24336e8c8e10a6fd19e662c2c4a0f29f"},
        "operations": [
          // ... shows this UTXO being CREATED as output #0 with 5 ADA
        ]
      }
    }
  ]
}

But via cardano-cli:

cardano-cli query utxo --address addr_test1wr3cmj7uzdl89zkrgdqk55sn9ka3368stgaky3xm59elfrg49uvlm --testnet-magic 1

Result:

                           TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
39bdb7ca1f83ab998811a1b29c57e3bb24336e8c8e10a6fd19e662c2c4a0f29f     0        5000000 lovelace + TxOutDatumInline

THIS UTXO STILL EXISTS! Confirms the operation correctly has status="invalid" (wasn't consumed).

Collateral UTXO fe079cee...#2 (what rosetta-ts shows but rosetta-java misses):

curl -X POST http://localhost:8083/search/transactions \
  -H "Content-Type: application/json" \
  -d '{"network_identifier": {"blockchain": "cardano", "network": "preprod"},
       "coin_identifier": {"identifier": "fe079ceee24e55afccdb6662c8b22ea228348f56b57587fb8e95a867969d31c3:2"}}'

Response:

{
  "total_count": 2,  // TWO TRANSACTIONS - CREATED AND SPENT!
  "transactions": [
    "2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8",  // SPENT HERE (our invalid tx)
    "fe079ceee24e55afccdb6662c8b22ea228348f56b57587fb8e95a867969d31c3"   // CREATED HERE
  ]
}

Collateral return verification via cardano-cli:

cardano-cli query utxo --address addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr --testnet-magic 1

Result:

                           TxHash                                 TxIx        Amount
--------------------------------------------------------------------------------------
2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8     2        13165743 lovelace

Collateral was consumed and returned! This is what rosetta should show.

rosetta-ts response (partially correct)

{
  "operations": [
    {
      "type": "input", 
      "status": "invalid",  // ❌ Should be "success" - this WAS consumed!
      "account": {"address": "addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr"},
      "amount": {"value": "-13720235"},  // 13.72 ADA - COLLATERAL
      "coin_change": {
        "coin_identifier": {"identifier": "fe079ceee24e55afccdb6662c8b22ea228348f56b57587fb8e95a867969d31c3:2"},
        "coin_action": "coin_spent"
      }
    },
    {
      "type": "output",
      "status": "invalid",  // ❌ Should be "success" - this WAS created!
      "account": {"address": "addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr"},
      "amount": {"value": "13165743"},  // 13.16 ADA - COLLATERAL RETURN  
      "coin_change": {
        "coin_identifier": {"identifier": "2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8:2"},
        "coin_action": "coin_created"
      }
    }
    // ❌ MISSING: Script input operation with status "invalid" (the actual failed operation)
  ]
}

Problems with rosetta-ts:

• Shows the RIGHT UTXOs (collateral) but WRONG status (should be "success" for collateral ops)
• Missing the actual invalid operation (script input that failed to be consumed)

What Actually Happened

graph LR
    subgraph "BEFORE Transaction"
        U1["Contract Wallet<br/>addr_test1wr3cmj7..."] 
        U1 --> UTXO1["39bdb7ca...#0<br/>5 ADA<br/>(script input)"]
        
        U2["User Wallet<br/>addr_test1vz8dlkd..."] 
        U2 --> UTXO3["fe079cee...#2<br/>13.72 ADA<br/>(COLLATERAL)"]
    end
    
    subgraph "Transaction Attempt"
        TX["TX: 2bb415e...<br/>Script Execution FAILS<br/>Phase 2 Validation ❌"]
        UTXO1 -.->|"TRIED to spend<br/>(but failed)"|TX
        UTXO3 ==>|"ACTUALLY SPENT<br/>(collateral)"|TX
    end
    
    subgraph "AFTER Transaction"
        TX ==>|"Collateral Return<br/>13.16 ADA"|UTXO4["2bb415e...#2<br/>13.16 ADA<br/>(collateral return)"]
        TX ==>|"Fee to Network<br/>0.554 ADA"|FEE["Network Fee Pot"]
        
        UTXO1 -->|"STILL EXISTS<br/>(unspent)"|U1B["Contract Wallet<br/>addr_test1wr3cmj7..."]
        UTXO4 -->|"NEW UTXO"|U2B["User Wallet<br/>addr_test1vz8dlkd..."]
    end
    
    style UTXO1 fill:#f5f5f5,stroke:#666,stroke-width:1px
    style UTXO3 fill:#fff,stroke:#000,stroke-width:2px
    style UTXO4 fill:#fff,stroke:#000,stroke-width:2px
    style FEE fill:#fff,stroke:#333,stroke-width:1px
    style TX fill:#fafafa,stroke:#000,stroke-width:2px
    style U1 fill:#fff,stroke:#333,stroke-width:1px
    style U2 fill:#fff,stroke:#333,stroke-width:1px
    style U1B fill:#fff,stroke:#333,stroke-width:1px
    style U2B fill:#fff,stroke:#333,stroke-width:1px

Loading

• Script input (39bdb7ca...#0): ATTEMPTED but FAILED - rosetta-java correctly shows this as invalid ✓
• Collateral (fe079cee...#2): SUCCESSFULLY consumed, 13.16 ADA returned - rosetta-java doesn't show this ✗
• Fee: 0.554 ADA taken from collateral
• Core problem: Incomplete representation showing only failures, not successes

Expected behavior

For phase 2 validation failures, should show ALL operations (both intended and actual) with correct statuses, network_index, and related_operations:

{
  "operations": [
    {
      "operation_identifier": {"index": 0, "network_index": 0},
      "type": "input",
      "status": "invalid",  // Script execution failed - UTXO NOT consumed
      "account": {"address": "addr_test1wr3cmj7uzdl89zkrgdqk55sn9ka3368stgaky3xm59elfrg49uvlm"},
      "amount": {"value": "-5000000"},
      "coin_change": {
        "coin_identifier": {"identifier": "39bdb7ca1f83ab998811a1b29c57e3bb24336e8c8e10a6fd19e662c2c4a0f29f:0"},
        "coin_action": "coin_spent"
      }
    },
    {
      "operation_identifier": {"index": 1, "network_index": 0},
      "related_operations": [{"index": 0}],  // References the script input that would have funded it
      "type": "output",
      "status": "invalid",  // Intended output NOT created (script failed)
      "account": {"address": "addr_test1wr3cmj7uzdl89zkrgdqk55sn9ka3368stgaky3xm59elfrg49uvlm"},
      "amount": {"value": "1611940"}  // 1.61 ADA
    },
    {
      "operation_identifier": {"index": 2, "network_index": 1},
      "related_operations": [{"index": 0}],  // References the script input that would have funded it
      "type": "output",
      "status": "invalid",  // Intended output NOT created (script failed)
      "account": {"address": "addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr"},
      "amount": {"value": "3018399"}  // 3.01 ADA
    },
    {
      "operation_identifier": {"index": 3, "network_index": 0},  // Position in body.collateralInputs[]
      "type": "collateralInput",  // Different type for collateral!
      "status": "success",  // Collateral WAS consumed
      "account": {"address": "addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr"},
      "amount": {"value": "-13720235"},
      "coin_change": {
        "coin_identifier": {"identifier": "fe079ceee24e55afccdb6662c8b22ea228348f56b57587fb8e95a867969d31c3:2"},
        "coin_action": "coin_spent"
      }
    },
    {
      "operation_identifier": {"index": 4, "network_index": 0},  // network_index=0 for consistency
      "related_operations": [{"index": 3}],  // References the collateral input that funded it
      "type": "collateralReturn",  // Different type for collateral return!
      "status": "success",  // Collateral return WAS created
      "account": {"address": "addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr"},
      "amount": {"value": "13165743"},
      "coin_change": {
        "coin_identifier": {"identifier": "2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8:2"},
        "coin_action": "coin_created"
      }
    }
  ]
}

Note on implementation: The complete fix with correct network_index values ideally requires implementing separate operation types for collateral operations (collateralInput and collateralReturn), as detailed in the sub-issue proposal. This ensures each operation's network_index correctly represents its position within its respective transaction body array.

[linconvidal]

Database Investigation - How Invalid Transactions are Stored in Yaci

Appendix: SQL queries to understand data representation

1. Database structure discovery

-- Connect to rosetta-java database (port forwarded to localhost:5432)
PGPASSWORD=postgres psql -h localhost -p 5432 -U postgres -d rosetta-java

-- List schemas
\dn
-- Result: preprod and public schemas exist

-- List tables in preprod schema
\dt preprod.*
-- Key tables: transaction, invalid_transaction, address_utxo, tx_input

2. Transaction basic data

-- Query invalid transaction basic info
SELECT tx_hash, invalid, fee, total_collateral, block 
FROM preprod.transaction 
WHERE tx_hash = '2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8';

-- Result:
-- invalid = true
-- fee = 369661 (intended fee, not actually paid)
-- total_collateral = 554492 (0.554 ADA - actually paid)
-- block = 3422820

3. Finding the phantom outputs

-- Query inputs and outputs from transaction table
SELECT inputs, outputs 
FROM preprod.transaction 
WHERE tx_hash = '2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8';

-- inputs: [{"tx_hash": "39bdb7ca...", "output_index": 0}]  -- Script input (NOT consumed)
-- outputs: [{"tx_hash": "2bb415e...", "output_index": 0},  -- Phantom output #0
--           {"tx_hash": "2bb415e...", "output_index": 1}]  -- Phantom output #1

4. Collateral operations (what actually happened)

-- Query collateral fields
SELECT collateral_inputs, collateral_return 
FROM preprod.transaction 
WHERE tx_hash = '2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8';

-- collateral_inputs: [{"tx_hash": "fe079cee...", "output_index": 2}]  -- WAS consumed
-- collateral_return: {"tx_hash": "2bb415e...", "output_index": 2}      -- WAS created

5. Actual UTXOs created (only collateral return)

-- Query actual UTXOs created
SELECT output_index, owner_addr, lovelace_amount, is_collateral_return 
FROM preprod.address_utxo 
WHERE tx_hash = '2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8' 
ORDER BY output_index;

-- Result: Only output #2 exists (collateral return)
-- output_index = 2
-- owner_addr = addr_test1vz8dlkd8zg4rxzs26fq95fheupph8mmgpf78rh3c5nyalsq5rgkvr
-- lovelace_amount = 13165743 (13.16 ADA)
-- is_collateral_return = true

6. Phantom outputs details from invalid_transaction table

-- Extract intended outputs from invalid_transaction CBOR
SELECT (transaction::json->'body'->'outputs')::jsonb 
FROM preprod.invalid_transaction 
WHERE tx_hash = '2bb415e28c4bbc81b3be24499ba8fa216fe728b1e828ebc35b5a58c079f9fec8';

-- Result: Two intended outputs that were never created:
-- [
--   {
--     "address": "addr_test1wr3cmj7...",
--     "amounts": [{"quantity": 1611940, ...}]  -- 1.61 ADA (phantom output #0)
--   },
--   {
--     "address": "addr_test1vz8dlkd...", 
--     "amounts": [{"quantity": 3018399, ...}]  -- 3.01 ADA (phantom output #1)
--   }
-- ]

7. Comparison with successful smart contract transaction

-- Query successful transaction with collateral
SELECT tx_hash, invalid, fee, total_collateral 
FROM preprod.transaction 
WHERE tx_hash = '5f39e514928bc08bbcb4305099b1a0af5b60f08032951a1ffc9034e6268e3cd9';

-- Result:
-- invalid = false (transaction succeeded)
-- total_collateral = NULL (no collateral paid)

-- Check collateral fields
SELECT collateral_inputs, collateral_return 
FROM preprod.transaction 
WHERE tx_hash = '5f39e514928bc08bbcb4305099b1a0af5b60f08032951a1ffc9034e6268e3cd9';

-- collateral_inputs: [{"tx_hash": "072973e2...", "output_index": 1}]
-- collateral_return: {"tx_hash": "5f39e514...", "output_index": 2}
-- BUT: collateral wasn't actually consumed since transaction succeeded

-- Check actual UTXOs created
SELECT output_index, owner_addr, lovelace_amount, is_collateral_return 
FROM preprod.address_utxo 
WHERE tx_hash = '5f39e514928bc08bbcb4305099b1a0af5b60f08032951a1ffc9034e6268e3cd9';

-- Result: Only outputs #0 and #1 exist (regular outputs)
-- No output #2 (collateral return doesn't exist when tx succeeds)

Key findings from database investigation

1. Two separate data representations exist:

   • transaction table: Contains transaction structure as submitted (including phantom outputs for failed txs)
   • address_utxo table: Contains only actual UTXOs created

2. Invalid transaction storage pattern (phase 2 failures):

   • Script inputs stored in inputs (marked as consumed but weren't)
   • Intended outputs stored in outputs (never created as UTXOs)
   • Collateral operations stored in separate collateral_inputs/collateral_return fields
   • Only collateral return appears in address_utxo table as real UTXO

3. Successful smart contract transaction pattern:

   • Collateral fields populated in DB but collateral NOT consumed
   • Same UTXO can appear as both regular input AND collateral input
   • Only regular outputs created in address_utxo table
   • Collateral return doesn't exist even though DB has the field

4. Data completeness:

   • Database has ALL the information needed to show complete operations
   • rosetta-java is only using partial data (script inputs) and ignoring:
     • Intended outputs from transaction body
     • Collateral operations (both successful and unused)