Problem

To test JSON-RPC conformance, we are currently using the tool provided by the Ethereum Execution APIs repository:
https://github.com/ethereum/execution-apis/tree/main/tests

This test suite is designed to validate client compatibility using a set of predefined scenarios, which are executed via the Hive test simulator rpc-compat.

However, a subset of the test case scenarios is currently being skipped, which means the tests from this repository cannot be fully executed or validated against our client implementation.

Why are these tests skipped?

The test framework relies on three essential chain-related files provided in the tests directory:

• genesis.json: A standard genesis configuration file in Go-Ethereum format.
• chain.rlp: A newline-delimited list of blocks that make up the test chain.
• bad.rlp: A list of sealed blocks that conduct invalid transitions (used in negative test scenarios).

The expected test flow involves ingesting genesis.json and chain.rlp, then generating and running tests against the head of that chain.

Our issue is that we are unable to load block data (and its transactions) from chain.rlp into our local Hedera node. As a result, we cannot recreate the expected chain state, and the affected test scenarios fail or must be skipped.

Root Cause

It would, in theory, be possible to "shadow" or replay the transactions from chain.rlp on our local node to emulate the required chain state. However, this is currently not possible due to a key limitation:

The test chain in the suite uses a chain ID of 3503995874084926, which exceeds the maximum value allowed by Hedera, where chain IDs are constrained to standard 32-bit integers.

All of the transactions listed in the chain.rlp are already signed and have this chain id value preset. Updating chain id in them would require a new signature which which would require the knownledge of what was their initial private key.

Response validation limitation

Another issue is that when running the rpc-compat tests (designed to be executed via Hive), we are currently only validating responses against the response schema using a schema validator. This means the full response structure is not being thoroughly checked.

As a result, we cannot currently use the Hive tool directly to execute the full test suite against our JSON-RPC API. However, once this task is completed, it will become possible to run Hive-based tests. For our custom/overriden test cases, because some Ethereum-specific features (e.g., blob support) are not yet available in our implementation and cannot be fully addressed at this time.

Solution

Although we cannot run all transactions from the original test branch due to compatibility issues, we can still recreate the required smart contracts and preset the balances of certain accounts on our local node before executing the tests.

This would allow at least some of the test cases to be executed successfully. For example, it’s possible to preset account balances for those tests that simply verify balance values.

Some of the currently skipped test cases may have been replaced by files in the same format, but adapted to reflect the actual behavior of the Hedera JSON-RPC relay. These overridden test cases can be placed in a dedicated override directory, allowing users to see where Hedera’s behavior differs from the reference Ethereum implementation.

This approach would make it possible to run hive compat-rpc against a modified test suite that is tailored for Hedera. The override directory could also include documentation explaining which tests were skipped and why, helping developers understand current limitations and expected differences in behavior.

We could also include in this directory tests for custom methods that are not currently part of the OpenRPC JSON-RPC API schema.

Alternatives

One possible workaround would be to support chain ids beyond the standard integer range and replay all transactions from the chain.rlp file on a local node. However, this would require significant modifications to the consensus node itself. Given the complexity and effort involved, this approach is likely not justifiable at this stage.