Methods of Running Tests

EEST has two commands, consume and execute, that run test cases against EL clients:

1. consume runs JSON test fixtures against a client - the client is said to "consume" the test case fixture.
2. execute runs test cases from Python source against a client - the test case is "executed" against the client.

Top-Level Comparison

Both consume and execute provide sub-commands which correspond to different methods of testing EL clients using EEST test cases:

Command	Description	Components tested	Environment	Scope
consume direct	Client consume tests via a statetest interface	EVM	None	Module test
consume direct	Client consume tests via a blocktest interface	EVM, block processing	None	Module test, Integration test
consume engine	Client imports blocks via Engine API EngineNewPayload in Hive	EVM, block processing, Engine API	Staging, Hive	System test
consume rlp	Client imports RLP-encoded blocks upon start-up in Hive	EVM, block processing, RLP import (sync*)	Staging, Hive	System test
execute hive	Tests executed against a client via JSON RPC eth_sendRawTransaction in Hive	EVM, JSON RPC, mempool	Staging, Hive	System test
execute remote	Tests executed against a client via JSON RPC eth_sendRawTransaction on a live network	EVM, JSON RPC, mempool, EL-EL/EL-CL interaction (indirectly)	Production	System Test

*sync: Depending on code paths used in the client implementation, see the RLP vs Engine Simulator section below.

The following sections describe the different methods in more detail.

./hive --sim=eest/consume-engine vs consume engine

EEST simulators can be ran either standalone using the ./hive command or via an EEST command against a ./hive --dev backend, more details are provided below.

Direct

Nomenclature
Command	consume direct
Simulator	None
Fixture Formats	state_test, blockchain_test

The direct method provides the fastest way to test EVM functionality by executing tests directly through a client's dedicated test interface (e.g. statetest or blocktest). This method requires clients to implement a custom interface to read tests and pass their inputs through appropriate code paths; implementation guides available for state tests and blockchain tests.

The EEST consume direct command is a small wrapper around client direct interfaces that allows fast and easy selection of test subsets to execute via test ID regex match (thanks to an index file). See Consume Direct and the Cache and Fixture Inputs and Useful Pytest Options pages for help with options.

Rapid EVM development

The direct method with the StateTest format should be used for the fastest EVM development feedback loop. Additionally, EVM traces can be readily generated and compared to other implementations.

Engine

Nomenclature
Command	consume engine
Simulator	eest/consume-engine
Fixture format	blockchain_test_engine

The consume engine method tests execution clients via the Engine API by sending block payloads and verifying the response (post-merge forks only). This method provides the most realistic testing environment for production Ethereum client behavior, covering consensus integration, payload validation, and state synchronization.

The consume engine command:

1. Initializes the execution client with genesis state.
2. Connects via Engine API (port 8551), primitively mocking a consensus client.
3. Sends a forkchoice update to establish the chain head.
4. Submits payloads using engine_newPayload calls.
5. Validates responses against expected results.
6. Tests error conditions and exception handling.

RLP

Nomenclature
Command	consume rlp
Simulator	eest/consume-rlp
Fixture format	blockchain_test

The RLP consumption method tests execution clients by providing them with RLP-encoded blocks to load upon startup, similar to the block import process during historical synchronization. This method tests the client's core block processing logic without the overhead of network protocols.

The consume rlp command:

1. Reads blockchain test fixtures from the specified input source.
2. Extracts RLP-encoded blocks from the fixture files.
3. Copies blocks to the client's container via files in the /blocks/ directory.
4. Starts the client with the genesis state and block files.
5. Validates the client's final blockHash via JSON RPC against the test's expectations.

This method simulates how clients import blocks during historical sync, testing the complete block validation and state transition pipeline, see below for more details and a comparison to consumption via the Engine API.

Engine vs RLP Simulator

The RLP Simulator (eest/consume-rlp) and the Engine Simulator (eest/consume-engine) should be seen as complimentary to one another. Although they execute the same underlying EVM test cases, the block validation logic is executed via different client code paths (using different fixture formats). Therefore, ideally, both simulators should be executed for full coverage.

Code Path Choices

Clients consume fixtures in the eest/consume-engine simulator via the Engine API's EngineNewPayloadv* endpoint; a natural way to validate, respectively invalidate, block payloads. In this case, there is no flexibility in the choice of code path - it directly harnesses mainnet client functionality. The eest/consume-rlp Simulator, however, allows clients more freedom, as the rlp-encoded blocks are imported upon client startup. Clients are recommended to try and hook the block import into the code path used for historical syncing.

Differences

	eest/consume-rlp	eest/consume-engine
Fixture Format Used	BlockchainTest	BlockchainTestEngine
Fork support	All forks (including pre-merge)	Post-merge forks only (Paris+)
Client code path	Historical sync / block import pipeline	Engine API / consensus integration
Real-world analogy	Blocks received during sync	Blocks received from consensus client
Interface	Block import upon start-up via RLP files	Engine API calls (newPayload, forkchoiceUpdated)
Exception testing	Basic exception handling	Advanced exception verification with client-specific mappers

Running both simulators adds some redundancy that can assist test debugging

If Engine tests fail but RLP tests pass, the issue is likely in your Engine API implementation rather than core EVM logic.

Execute

See Execute Command.

Two Methods to Run EEST Simulators

Many of the methods use the Hive Testing Environment to interact clients and run tests against them. These methods are also called Hive simulators. While Hive is always necessary to run simulators, they can be called in two different ways. Both of these commands execute the same simulator code, but in different environments, we take the example of the eest/consume-engine simulator:

1. ./hive --sim=eest/consume-engine is a standalone command that installs EEST and the consume command in a dockerized container managed by Hive. This is the standard method to execute EEST fixture releases against clients in CI environments and is the method to generate the results at hive.ethpandaops.io. See Hive and its Common Options for help with this method.
2. uv run consume engine requires the user to clone and install EEST and start a Hive server in development mode. In this case, the simulator runs on the native system and communicate to the client via the Hive API. This is particularly useful during test development as fixtures on the local disk can be specified via --input=fixtures/. As the simulator runs natively, it is easy to drop into a debugger and inspect the simulator or client container state. See Hive Developer Mode for help with this method.