Skip to content

Latest commit

 

History

History
154 lines (133 loc) · 8.01 KB

README.md

File metadata and controls

154 lines (133 loc) · 8.01 KB

Verification

This directory holds the Python side of FROST's simulation tests: the cocotb benches in cocotb_tests/ and the models and encoders they share. Block benches drive one RTL module and check it against independent models and assertions. Application tests compile a program from sw/apps/, run it on the whole SoC, and read its result from the UART. TestRIG runs random instruction sequences on the SoC and on the Sail RISC-V model and compares them one retired instruction at a time. The runners in tests/ build the RTL with Verilator and launch these benches; that README covers commands, memory tiers, the ISA suites, and CI.

Architecture

flowchart LR
    Blocks[Block benches] --> RTL[Verilator RTL]
    Programs[Compiled applications] --> RTL
    ISA[ISA and torture cases] --> RTL
    RTL --> Checks[Assertions and directed checks]
    RTL --> UART[UART output]
    RTL --> Memory[Low-BRAM contents]
    UART --> Pass[Application pass/fail]
    UART --> Signatures[Signature comparison]
    Memory --> Signatures
    Spike[Committed Spike references] --> Signatures
    Engine[TestRIG instruction sequences] --> RTL
    Engine --> Sail[Sail RISC-V model]
    RTL --> Compare[Per-instruction comparison]
    Sail --> Compare
Loading

Signature suites compare architectural results with Spike references committed to the repository, so Spike is needed only to regenerate them. The architecture tests of misaligned loads and stores in the DDR tier, where FROST performs those accesses in hardware, compare with references from the Sail model instead.

Directory Structure

Path Purpose
cocotb_tests/ Block benches, directed CPU tests, application harness, TestRIG's RVFI-DII server
models/ Integer, floating-point, and memory models
encoders/ Instruction encoders and operation tables
utils/ Memory-access helpers, data types, struct packing, assertions
testrig/ TestRIG support: RVFI-DII packets, the reset preamble, the Sail configuration, the A-extension test generator, and saved counterexamples
config.py Constants (XLEN=64) and configurable DUT signal paths
verification_types.py Shared NewType wrappers

Running Tests

./scripts/frost.py cocotb --list-tests                                      # every target
./scripts/frost.py cocotb directed_traps                                    # one target
./scripts/frost.py cocotb directed_atomics --testcase test_directed_lr_sc   # one test function
./scripts/frost.py cocotb hello_world                                       # an application, from BRAM
FROST_COCOTB_MEM_CONFIG=ddr ./scripts/frost.py cocotb hello_world           # the same, from cached DDR

TEST_REGISTRY in tests/test_run_cocotb.py defines the targets. Always use the wrapper, which cleans tests/ and runs the pinned image. Running make in tests/ with no arguments runs the directed_traps suite. --testcase selects test functions by regex through COCOTB_TEST_FILTER. The environment table lists the cycle budgets and run counts for applications.

frost_reset_release checks the two i_rst_n reset synchronizers in the default frost clock configuration. Both have three stages. The main-clock path includes an output register and releases on the fourth rising edge. The divided-clock path releases on the third rising edge. The first divided-clock rise after i_rst_n is released occurs while main-clock reset is asserted. The test checks the release edge and reset levels at both outputs and at the reset ports of the CPU and memory subsystem, the MMIO FIFOs, the UART transmitter and receiver, and both sides of the UART clock-crossing FIFOs.

Directed CPU tests

The directed suites drive instruction words into cpu_tb, the core with a fetch driver and a data memory model, and check results by waiting on commit events and reading the architectural register files through DUTInterface. They follow the out-of-order core's own timing rather than fixed pipeline offsets. (cpu_tb fills the upper half of every fetch window with an ECALL, which cannot pair as slot 2, so each 32-bit instruction is fetched alone.) Random instruction streams run as compiled programs instead: the riscv-torture suite checks them against Spike.

CPU harness target Status
directed_traps Runs in CI
directed_atomics, compressed Run from the command line only
Component Role
TestState Register and PC state, counter shadows, LR/SC reservation, expected-store queues
DUTInterface Signal access through DUTSignalPaths
Memory monitor Checks each store's address and data against the expected queues (the byte mask only marks that a store happened); it does not drive reads or update model memory

encoders/op_tables.py pairs an encoder with a reference evaluator for each modeled instruction. Atomic modeling covers word operations; supervisor instructions are tested by other suites. TestConfig in cocotb_tests/test_common.py supplies the reset, clock, and coverage options.

Application bench

cocotb_tests/test_real_program.py runs every application, and the architecture-test, riscv-tests, and torture runners use it too. It holds reset, runs the program COCOTB_NUM_RUNS times with a reset between runs, and watches the UART for the program's success or failure marker. Three coroutines wake on every rising edge of the core clock, always in the same order: the run loop, the divided-clock generator, and the UART monitor, which records each byte the CPU writes. Opt-in diagnostics and some applications' drivers add work of their own.

Verilator's evaluation of the model takes most of the simulation time. cocotb's own clocks deposit their edges, and Verilator applies a deposit only after the evaluation that opens the time step, so every clock edge would cost an evaluation that changes nothing. start_clocks writes the core clock's and the NIC clocks' edges without delay instead, which lets that evaluation take the edge. Applications that never use the NIC run without its MAC clocks, as on a board with no link clock, and fail if they access the NIC's registers (hold_nic_without_mac_clocks). The architecture, riscv-tests ISA and benchmark, and riscv-torture runners select this mode by default. FROST_NIC_MAC_CLOCKS=1 starts the MAC clocks for a comparison run. The debug and BRAM-reload benches start their core clock the same way and leave the MAC clocks off. These tests run under takes_last_edge, so that the model takes the edge a test returns or fails in before the simulation stops. The docstring of start_clocks gives the conditions under which the simulation stays the same as with cocotb's clocks: no combinational logic reads the simulation time or a sampled value, and no timer callback reads a clock or changes an input at an edge time.

The TestRIG bench also holds the MAC clocks and clock-OK inputs low by default. Its device reads use the harness RAM, and its access guard watches both NIC write enables, which the platform holds low. Loads and stores to the RAM at addresses in the NIC's window are valid TestRIG stimuli. The same environment override starts the MAC clocks in this bench.

Extending the Framework

  • Add encoder and evaluator pairs to encoders/op_tables.py.
  • Add monitors as coroutines started by the test.
  • In a whole-SoC bench that uses start_clocks, change inputs from clock-edge callbacks, not timers, and run the test under takes_last_edge (see Application bench).
  • Override DUTSignalPaths for hierarchy differences instead of hardcoding paths in test logic.
  • Keep shared constants in config.py and per-run behavior in TestConfig.
  • Reuse CPU port layouts from cocotb_tests/cpu_structs.py and serialization helpers from utils/packed_structs.py; keep stimulus defaults in each bench.
  • Register new benches in TEST_REGISTRY as described in the test guide.