The supported boards are Alveo X3522PV (X3) and VCU118. Both have 256 KiB
BRAM and 128 KiB L1D. X3 has a 16 KiB L1I and a 2 MiB URAM L2 and exposes
1 GiB of DDR4; VCU118 has a 32 KiB L1I and a 4 MiB URAM L2 and exposes 2 GiB
of DDR4. Both L2s are
four-way with 64-byte lines. X3 has 10GBASE-R Ethernet; VCU118 has a 1 Gb/s
RJ-45 interface and 10GBASE-R on QSFP1 lane 0. Both NIC interfaces use
coherent DMA. Both builds perform plain misaligned loads and stores to cached
DDR in hardware by default (MISALIGNED_HW). VCU118 also has SPI flash boot
into OpenSBI and U-Boot. See the
FPGA guide for X3 usage
and the VCU118 guide for its board integration and usage.
The X3 has eight L1D read slots and eight L1D request endpoints, so up to seven load misses and one store or LR acquisition are in flight to the L2 at once, and eight load queue entries. Its L2 coherence home has eight fills with three demand seats each. A demand fill uses one seat for its initiating request; a prefetch fill starts with all three seats free. The home has two writeback slots and a stream prefetcher that reads ahead of the hart's data and instruction miss streams.
| Module | Role |
|---|---|
xilinx_frost_subsystem.sv |
CPU and BRAM, JTAG software loader, BSCAN debug, reset timers |
x3/x3_frost.sv |
Board clocks, DDR and NIC integration, reset sequencing |
x3/x3_ddr_init.sv |
Initializes the exposed DDR region with valid ECC |
x3/x3_ecc_reader.sv |
Polls the DDR4 controller's ECC registers for FROST's memory error unit |
x3/x3_nic_gty.sv |
Ethernet transceiver, MAC clocks, transceiver reset supervisor |
x3/x3_cpu_clock_gty.sv |
CPU clock from a dedicated GTY transmitter |
x3/constr/x3.xdc |
Pin and timing constraints |
x3/constr/cpu_clock_boundary.tcl |
Debug transport clock/CDC and optional CPU clock-boundary constraints, applied after synthesis |
x3/x3_frost.f lists the board RTL. The Vivado flow generates the loader,
DDR, and transceiver IP from fpga/build/build_step.tcl, x3_ddr_bd.tcl, and
x3_gty_ip.tcl.
The cache hierarchy's bridge issues 512-bit AXI transactions, one 64-byte L2
line each, with region-relative addresses. A SmartConnect merges them with
the DDR JTAG master and the ECC reader and crosses the CPU, CPU/4, and DDR
clocks. The CPU port
is as wide as the controller's 512-bit interface; only the JTAG master's
transfers are widened. CPU address 0x80000000 maps the first 1 GiB.
ECC management sits at region offset 0x40000000, reachable from the DDR
JTAG master and the ECC reader.
After calibration, x3_ddr_init zeroes the region so every word has valid ECC
(about 0.1 seconds). It writes whole 512-bit controller words, one per beat:
a partial-word write would make the controller read the uninitialized rest of
the word to recompute ECC. The CPU and the DDR loader leave
reset once the CPU clock runs and calibration and initialization are complete;
the loader, which runs on the CPU/4 clock, takes that reset through a
synchronizer on it. Check for ECC errors with fpga/ddr_ecc/ddr_ecc_status.py.
Once initialization is done, x3_ecc_reader polls the ECC registers about
once a millisecond and reports corrected and uncorrectable errors to FROST's
memory error unit (hw/rtl/cpu_and_mem/memory_error_unit.sv), with the CPU
physical address of the 64-byte line the controller captured. It translates
the controller's row, column and bank back to an address with the
controller's mapping for this board; the controller's ECC words interleave
across the line's 8-byte words, so the line is as precise as a capture gets.
It clears only the controller's CE status bit, so the controller captures
the next address, and writes CE_CNT back to 1 once it reaches 128, before the
count saturates at 255. It never writes CE_CNT to 0 and never clears the UE
bit, so ddr_ecc_status.py still fails a run that had an error. The
controller gives a register write priority over its own update in that
cycle, so an uncorrectable error in the cycle of the reader's CE clear is
lost to the controller's status, though the run's CE_CNT still fails the
check and a CPU read of the word still reports an error response.
| Domain | Clock |
|---|---|
| CPU | 161.1328125 MHz Ethernet reference × 2 = 322.265625 MHz (GTY X0Y29 TXOUTCLK) |
| Loader, UART, reset timers | CPU/4 = 80.56640625 MHz |
| DDR reference | Independent 300 MHz |
| Ethernet TX / recovered RX | GTY user clocks, about 161.13 MHz |
| GTY reset controllers | 300 MHz input / 2 = 150 MHz, independent of both transceivers and the link |
The CPU clock comes from its own GTY channel, X0Y29 in the NIC's quad: a CPLL at 3.22265625 GHz from the Ethernet reference clock, a 6.4453125 Gb/s transmitter that carries no data (held in electrical idle), and TXOUTCLK from the transmitter's programmable divider (the CPLL clock ÷ 10). Two BUFG_GTs make the CPU and CPU/4 clocks. The XDC roots them at X2Y7 (Vivado would otherwise root them at the transceiver, on the device's right edge), and the guided placement candidate moves the root to X1Y9. The channel shares only the reference clock buffer with the NIC, so a NIC PHY reset, which resets the NIC's QPLL0 and channel, leaves the CPU clock running. A small supervisor on the 150 MHz clock restarts the channel's reset sequence if it has not finished 100 ms after power good or if the CPLL loses lock, and holds the CPU in reset until the clock is back.
build.py --cpu-clock-div N sets the board top's CPU_CLK_DIV generic,
which divides the CPU and CPU/4 clocks by N; the DDR and Ethernet clocks do
not change. N = 2 divides in the BUFG_GTs. N = 3 and 4 need CPU/4 dividers of
12 and 16, beyond a BUFG_GT's 8, so those builds take the CPU clock from an
MMCM on the 300 MHz input instead (300 MHz / 8 × 34.375 / 4N); the CPU clock
channel still runs. Load software with the matching clock. The
PERF_COUNTERS generic includes the profiling counters (--perf-counters).
The X3 build includes the vector unit (VECTOR, at a VLEN of 256 bits); the
loader's board list for each vector test program names the boards whose builds
include it.
On X3, Linux reboot requests SYSRESET and re-enters the retained DDR images;
they can be stale or hang in OpenSBI, so reload them for a fresh boot.
poweroff halts with power still applied.
The loader writes images through two JTAG-to-AXI masters, one for low BRAM
and one for DDR (jtag_axi_ddr):
- A first BRAM write puts the CPU in reset.
- The DDR image (
sw_ddr.txt), if any, is written to DDR. Periodic BRAM writes keep the CPU in reset meanwhile. - The BRAM image (
sw.txt) is written.
Every BRAM write restarts a 27-bit counter on the CPU/4 clock, and the CPU
leaves reset when the counter runs out: about 1.67 seconds after the last
write at 322.265625 MHz. A separate 16-bit counter holds the loader and CPU in
reset briefly after board reset, until the clocks are stable. The board reset
reaches the loader, its counters, and the BRAM programming port through a
synchronizer on the CPU/4 clock, and frost synchronizes resets into both
clock domains.
Debug shares the FPGA's own TAP. BSCANE2 USER3 carries dtmcs, USER4 carries
dmi, and the Vivado debug hub keeps USER1. The subsystem sets
DEBUG_JTAG_TAP=0. The six-bit IR uses IDCODE 0x09, DTMCS 0x22, and DMI
0x23; the OpenOCD configurations are in fpga/debug/. Only one process can
own the cable, so stop Vivado's hardware server before starting OpenOCD, and
the reverse.
| Signal | Direction | Pin | Description |
|---|---|---|---|
i_sysclk_p |
Input | AK23 | 300 MHz differential clock (positive) |
i_sysclk_n |
Input | AL23 | 300 MHz differential clock (negative) |
o_uart_tx |
Output | AP24 | UART transmit for debug console |
i_uart_rx |
Input | AR24 | UART receive for debug console input |
i_nic_refclk_p / i_nic_refclk_n |
Input | P9 / P8 | 161.1328125 MHz Ethernet reference clock (MGTREFCLK0, quad 231) |
o_nic_txp / o_nic_txn |
Output | J7 / J6 | NIC transceiver TX (GTY X0Y28, DSFP28 cage labelled 2, lane 1) |
i_nic_rxp / i_nic_rxn |
Input | K4 / K3 | NIC transceiver RX (GTY X0Y28) |
o_cpu_clock_txp / o_cpu_clock_txn |
Output | H5 / H4 | CPU clock transceiver TX (GTY X0Y29), electrical idle |
i_cpu_clock_rxp / i_cpu_clock_rxn |
Input | J2 / J1 | CPU clock transceiver RX (GTY X0Y29), unused |
The UART console uses 115200 baud, 8N1.
To support another Xilinx FPGA board:
- Create a subdirectory named after the board.
- Start from the
x3_frost.svwrapper. - Adapt the clock generation: set the MMCM parameters for the board's input clock, drive the CPU clock from CLKOUT0, and derive the /4 clock with a global clock divider suitable for the FPGA family.
- Instantiate
xilinx_frost_subsystem. For a board with DDR, also instantiate itsddr_subsysblock design, wire the cache bridge's AXI port to it, and passENABLE_CACHED_TIER=1andUSE_BEHAVIORAL_DDR=0. Hold the CPU in reset until DDR calibration (mem_ok) and, if the memory has ECC, until the region has been written. The cached tier includes a configurable UltraRAM L2 (L2_CACHE_BYTES, 2 MiB by default), so the FPGA needs enough UltraRAM for its selected capacity. A BRAM-only board leaves the cached tier off and needs no DDR block design. - Write a constraint file with the board's pin assignments, including the
DDR pins unless a MIG
.prjor board interface supplies them. - Add the subsystem to the board's
.ffile list. - Add the board's FPGA part and its
has_ddrandhas_gtycapabilities toboard_build_configsinfpga/build/build_step.tcl. The Tcl flow derives the board's wrapper, file-list, constraint, and IP-creation procedure names from its name. A board with DDR also needsfpga/build/<board>_ddr_bd.tclfor itsddr_subsysblock design; the CPU port'sS00_AXIrange sets the memory size advertised to Linux. A board with a NIC transceiver needsfpga/build/<board>_gty_ip.tclfor its transceiver wizard core. - Register the board in the FPGA tools:
BOARD_CONFIGinfpga/build/build.py: clock, family, and synthesis directive.BOARD_INFOinfpga/build/extract_timing_and_util_summary.py.BOARD_CONFIGinfpga/load_software/load_software.py: clock, CoreMark iterations, and DDR support.BOARD_VENDOR_INFOinfpga/common/hw_target.py, andDEFAULT_TARGETS,DEFAULT_SERIALS, andDEFAULT_TIMEOUTSinfpga/common/hw_defaults.py.supported_boardsinfpga/program_bitstream/program_bitstream.tcl, for direct Tcl use. The Python programmer, regression, soak, and sweep tools take their board lists from the registries above.
- Calibrate the board's CoreMark-PRO
hardware_iterationsinsw/apps/software_registry.py, and sethardware_timeout_minimumsif setup takes longer than the board's default timeout. Optionally record score gates inBASELINE_SCORESinfpga/hw_regression.py. - Add the board to this README.
Before the first build, check that:
- the MMCM VCO frequency and dividers produce the target CPU clock;
- the timing constraints match the input clock period;
- the I/O standards match the board's bank voltages;
- the DDR controller IP matches the board's soldered or SODIMM memory.
A non-Xilinx FPGA would need a different subsystem, because
xilinx_frost_subsystem uses Xilinx IP and BSCANE2 primitives.
frost exposes three compile-time integration parameters (all zero by default,
which xilinx_frost_subsystem keeps for X3 and VCU118) together with the
core-domain synchronous reset output o_soc_rst:
EXT_DMA_PORTS(withEXT_DMA_ID_BITS) appends coherent 256-bit AXI4 DMA slave ports todma_interconnect, each isolated by anaxi_register_slice.EXTERNAL_MAC=1omits the internalnic_mac_10gblock and exports the primarynic_engine's core-clock 64-bit AXI4-Stream TX/RX and MAC sideband ports so a board wrapper can attach a different MAC.EXT_AXIL_WINDOW=1exposes a 16 KiB 64-bit AXI4-Lite master window at0x4002_4000-0x4002_7FFFbehind anaxil_register_slicein both cached and BRAM-only builds.
See hw/rtl/lib/axi/README.md for port widths, coherence aperture rules, and reset contracts.