This directory holds the Python side of FROST's simulation tests: the cocotb
benches in cocotb_tests/ and the models and encoders they share.
Block benches drive one RTL module and check it against independent models
and assertions. Application tests compile a program from sw/apps/, run it
on the whole SoC, and read its result from the UART.
TestRIG runs random instruction sequences on the SoC and
on the Sail RISC-V model and compares them one retired instruction at a time.
The runners in tests/ build the RTL with Verilator and
launch these benches; that README covers commands, memory tiers, the ISA
suites, and CI.
flowchart LR
Blocks[Block benches] --> RTL[Verilator RTL]
Programs[Compiled applications] --> RTL
ISA[ISA and torture cases] --> RTL
RTL --> Checks[Assertions and directed checks]
RTL --> UART[UART output]
RTL --> Memory[Low-BRAM contents]
UART --> Pass[Application pass/fail]
UART --> Signatures[Signature comparison]
Memory --> Signatures
Spike[Committed Spike references] --> Signatures
Engine[TestRIG instruction sequences] --> RTL
Engine --> Sail[Sail RISC-V model]
RTL --> Compare[Per-instruction comparison]
Sail --> Compare
Signature suites compare architectural results with Spike references committed to the repository, so Spike is needed only to regenerate them. The architecture tests of misaligned loads and stores in the DDR tier, where FROST performs those accesses in hardware, compare with references from the Sail model instead.
| Path | Purpose |
|---|---|
cocotb_tests/ |
Block benches, directed CPU tests, application harness, TestRIG's RVFI-DII server |
models/ |
Integer, floating-point, and memory models |
encoders/ |
Instruction encoders and operation tables |
utils/ |
Memory-access helpers, data types, struct packing, assertions |
testrig/ |
TestRIG support: RVFI-DII packets, the reset preamble, the Sail configuration, the A-extension test generator, and saved counterexamples |
config.py |
Constants (XLEN=64) and configurable DUT signal paths |
verification_types.py |
Shared NewType wrappers |
./scripts/frost.py cocotb --list-tests # every target
./scripts/frost.py cocotb directed_traps # one target
./scripts/frost.py cocotb directed_atomics --testcase test_directed_lr_sc # one test function
./scripts/frost.py cocotb hello_world # an application, from BRAM
FROST_COCOTB_MEM_CONFIG=ddr ./scripts/frost.py cocotb hello_world # the same, from cached DDRTEST_REGISTRY in tests/test_run_cocotb.py defines the targets. Always use
the wrapper, which cleans tests/ and runs the pinned image. Running make
in tests/ with no arguments runs the directed_traps suite. --testcase
selects test functions by regex through COCOTB_TEST_FILTER. The
environment table lists the
cycle budgets and run counts for applications.
frost_reset_release checks the two i_rst_n reset synchronizers in the
default frost clock configuration. Both have three stages. The main-clock
path includes an output register and releases on the fourth rising edge.
The divided-clock path releases on the third rising edge. The first
divided-clock rise after i_rst_n is released occurs while main-clock reset
is asserted. The test checks the release edge and reset levels at both
outputs and at the reset ports of the CPU and memory subsystem, the MMIO
FIFOs, the UART transmitter and receiver, and both sides of the UART
clock-crossing FIFOs.
The directed suites drive instruction words into cpu_tb, the core with a
fetch driver and a data memory model, and check results by waiting on commit
events and reading the architectural register files through DUTInterface.
They follow the out-of-order core's own timing rather than fixed pipeline
offsets. (cpu_tb fills the upper half of every fetch window with an ECALL,
which cannot pair as slot 2, so each 32-bit instruction is fetched alone.)
Random instruction streams run as compiled programs instead: the
riscv-torture suite checks them against Spike.
| CPU harness target | Status |
|---|---|
directed_traps |
Runs in CI |
directed_atomics, compressed |
Run from the command line only |
| Component | Role |
|---|---|
TestState |
Register and PC state, counter shadows, LR/SC reservation, expected-store queues |
DUTInterface |
Signal access through DUTSignalPaths |
| Memory monitor | Checks each store's address and data against the expected queues (the byte mask only marks that a store happened); it does not drive reads or update model memory |
encoders/op_tables.py pairs an encoder with a reference evaluator for each
modeled instruction. Atomic modeling covers word operations; supervisor
instructions are tested by other suites. TestConfig in
cocotb_tests/test_common.py supplies the reset, clock, and coverage options.
cocotb_tests/test_real_program.py runs every application, and the
architecture-test, riscv-tests, and torture runners use it too. It holds
reset, runs the program COCOTB_NUM_RUNS times with a reset between runs, and
watches the UART for the program's success or failure marker. Three
coroutines wake on every rising edge of the core clock, always in the same
order: the run loop, the divided-clock generator, and the UART monitor, which
records each byte the CPU writes. Opt-in diagnostics and some applications'
drivers add work of their own.
Verilator's evaluation of the model takes most of the simulation time.
cocotb's own clocks deposit their edges, and Verilator applies a deposit only
after the evaluation that opens the time step, so every clock edge would cost
an evaluation that changes nothing. start_clocks writes the core clock's and
the NIC clocks' edges without delay instead, which lets that evaluation take
the edge. Applications that never use the NIC run without its MAC clocks, as
on a board with no link clock, and fail if they access the NIC's registers
(hold_nic_without_mac_clocks). The architecture, riscv-tests ISA and
benchmark, and riscv-torture runners select this mode by default.
FROST_NIC_MAC_CLOCKS=1 starts the MAC clocks for a comparison run.
The debug and BRAM-reload benches start their
core clock the same way and leave the MAC clocks off. These tests run under
takes_last_edge, so that the model takes the edge a test returns or fails in
before the simulation stops. The docstring of start_clocks gives the
conditions under which the simulation stays the same as with cocotb's clocks:
no combinational logic reads the simulation time or a sampled value, and no
timer callback reads a clock or changes an input at an edge time.
The TestRIG bench also holds the MAC clocks and clock-OK inputs low by default. Its device reads use the harness RAM, and its access guard watches both NIC write enables, which the platform holds low. Loads and stores to the RAM at addresses in the NIC's window are valid TestRIG stimuli. The same environment override starts the MAC clocks in this bench.
- Add encoder and evaluator pairs to
encoders/op_tables.py. - Add monitors as coroutines started by the test.
- In a whole-SoC bench that uses
start_clocks, change inputs from clock-edge callbacks, not timers, and run the test undertakes_last_edge(see Application bench). - Override
DUTSignalPathsfor hierarchy differences instead of hardcoding paths in test logic. - Keep shared constants in
config.pyand per-run behavior inTestConfig. - Reuse CPU port layouts from
cocotb_tests/cpu_structs.pyand serialization helpers fromutils/packed_structs.py; keep stimulus defaults in each bench. - Register new benches in
TEST_REGISTRYas described in the test guide.