Skip to content

Latest commit

 

History

History
483 lines (414 loc) · 30.6 KB

README.md

File metadata and controls

483 lines (414 loc) · 30.6 KB

FROST Linux

FROST boots Debian 13 on the Alveo X3 with Debian's own riscv64 kernel, unmodified. OpenSBI starts the kernel in S-mode, where it runs with Sv39 paging. The kernel's initramfs loads the FROST NIC driver as a module and mounts the root filesystem over NFS through that NIC. Without an NFS root, the image boots a small Buildroot test initramfs instead, which CI also boots under QEMU.

To set up a board, follow the Debian setup guide. To build the test image, see the Buildroot tree.

Boot chain and entry state

flowchart LR
    BRAM[BRAM boot shim] --> SBI[OpenSBI in DDR]
    SBI --> Kernel[Debian kernel]
    Kernel --> Init[Initramfs loads NIC]
    Init --> Root[Debian NFS root]
    Init --> Test[Buildroot test userspace]
Loading

The shim at address zero enters OpenSBI at 0x8000_0000 in M-mode, with a0 = 0 (the hart ID) and a1 pointing to the DTB. The packer places the DTB after the payload, as described under Memory map.

The firmware is OpenSBI v1.9's generic platform from the linux/opensbi submodule, built by opensbi_build.py with:

  • FW_TEXT_START=0x80000000 and FW_JUMP_OFFSET=0x200000.
  • An empty FW_JUMP_FDT_OFFSET, so the DTB address in a1 passes through unchanged.
  • The drivers in opensbi_frost_defconfig (the ns16550 UART, the APLIC, the IMSIC, and the CLINT) and FDT_ASSUME_MASK=7, which lets libfdt trust the packer's DTB.

It builds with the Bootlin riscv64-linux- toolchain by default, found on PATH or else in Buildroot's output, linux/build-mmu/host/bin. Another toolchain, selected with FROST_LINUX_CROSS_COMPILE or --cross, must be Linux-targeted with a PIE-capable linker: OpenSBI links position-independent.

Every invocation checks a content stamp for the configuration, OpenSBI revision and local source edits, build options, and toolchain prefix and versions. A missing or different stamp causes a build in an empty output directory; matching firmware is reused without running make. If Git cannot identify the source directory, every invocation builds from scratch without recording a reusable stamp. Only empty directories, directories with a helper stamp, and recognised OpenSBI output directories can be replaced. The last component of --out must not be a symlink; links in its parent path are resolved.

OpenSBI enters the payload in S-mode with satp Bare and sstatus.SIE=0. It delegates the SSI, STI, SEI, and local counter-overflow (LCOFI) interrupts and the misaligned-fetch, breakpoint, U-mode ecall, and page-fault exceptions. It sets mcounteren to all ones (0x7FF after WARL), scounteren=0x2, and menvcfg.STCE=1, so S-mode can read every counter and U-mode starts with access to time only. The X3, VCU118 and simulation builds enable MISALIGNED_HW by default, so plain misaligned loads and stores to cached DDR complete in hardware. With the engine disabled those accesses trap. Misaligned accesses to low BRAM and misaligned LR, SC and AMOs also trap. OpenSBI handles address-misaligned exceptions in M-mode until the kernel requests their delegation through FWFT.

Memory map

The board and simulation share one physical map.

Range Contents
[0x0000_0000, 256 KiB) Uncached boot BRAM, outside Linux's memory node. Code runs only from its first 128 KiB. [0x17C00, 0x18000) is reserved for the debug module.
[0x4000_0000, 0x4003_1000) Native MMIO: UART, FIFOs, timer, DMA test engine, memory error unit, SYSRESET, and NIC. See the software memory map.
[0x4000_1000, +0x100) ns16550a UART alias, APLIC source 1, with reg-shift=2 and reg-io-width=4
[0x4001_0000, +0xC000) SiFive CLINT alias: msip at +0, mtimecmp at +0x4000, mtime at +0xBFF8
[0x4002_1000, +0x100) Memory error unit, APLIC source 5, bound by the frost_memerr driver (device-tree binding)
[0x4002_F000, +4 KiB) SYSRESET: OpenSBI uses the DT's syscon-reboot node to implement cold and warm SBI SRST requests. A word store of 0x7777 resets the core domain and returns to the boot address.
[0x4003_0000, +4 KiB) DMA-coherent NIC, bound by the frost_net10g driver (device-tree binding)
[0x4400_0000, +4 MiB) AIA APLIC with direct and MSI delivery, with M and S domains at 0x4400_0000 and 0x4400_8000, each 32 KiB and each with one IDC for hart 0. Sources 1 to 5 are the UART, external pin (RJ-45 NIC on VCU118), DMA test engine, QSFP NIC, and memory error unit. OpenSBI delegates all sources to S mode and disables the M-domain node in Linux's DT.
[0x4800_0000, +2 MiB) IMSIC device window. Machine files start at 0x4800_0000 and supervisor files at 0x4810_0000, with 4 KiB per hart and 63 identities per file.
[0x8000_0000, +1 GiB) Cached DDR. The DTB advertises the board's DDR size: 1 GiB on X3.

SYSRESET has the same memory contract as debug ndmreset: it preserves BRAM and DDR storage while discarding dirty cache lines. On the X3 JTAG boot path, the BRAM shim jumps back into the existing OpenSBI image without reloading firmware, the kernel or the DTB. Those retained images may be stale; OpenSBI also keeps boot state in .data, so re-entry can hang in its warm-boot path. Reload the images to obtain a fresh X3 boot. On a VCU118 booting from flash, the surviving BRAM ROM reloads the FSBL, which checks and reloads OpenSBI, the DTB and U-Boot; U-Boot then follows its configured boot command. Linux uses SBI SRST when it detects this firmware service. Only cold and warm reboot are supported; there is no shutdown device. Any supervisor-mode code can write SYSRESET directly: the platform has no PMP enforcement, and OpenSBI exposes the syscon region with S-mode read/write permissions. Root can also reach it through /dev/mem. poweroff cannot remove power: the SBI shutdown request returns unsupported and Linux halts.

The PMA map has five regions: the BRAM, the MMIO register window [0x4000_0000, 0x4003_1000), the APLIC window [0x4400_0000, 0x4440_0000), the IMSIC window [0x4800_0000, 0x4820_0000), and the configured cached DDR aperture. An access outside these regions, including the gaps in the device quadrant [0x4000_0000, 0x8000_0000), raises a precise access fault (cause 1, 5, or 7 for a fetch, load, or store) with the faulting address in the trap value register. Under Sv39 the data MMU checks alignment before translation and page permissions before the physical address. For an untranslated access, the access fault takes priority over misalignment. Device windows permit loads and stores. Fetch, AMO, LR and SC to those windows raise causes 1, 7, 5 and 7, respectively. Addresses outside the map never alias onto it.

The image packer lays out DDR as follows, with offsets from 0x8000_0000:

Component Offset Size and alignment
OpenSBI fw_jump.bin +0 At most 1 MiB. OpenSBI reserves its runtime regions in the DTB.
Kernel Image or other S-mode payload +2 MiB 2 MiB aligned
DTB The first 2 MiB boundary at or above both +16 MiB and the payload's end 64 KiB slot, with room for OpenSBI's in-place fixups
Initramfs, if any Right after the DTB slot Must fit in the advertised memory

A Linux Image ends at its header's image_size, which includes BSS; a raw payload ends with its file. Linux reserves its image up to the next 2 MiB boundary and drops an initramfs that overlaps a reservation, so the DTB starts on that boundary and the initramfs follows it. Payloads up to 14 MiB put the DTB at +16 MiB and the initramfs at +16 MiB + 64 KiB; the Debian kernel is larger, so its DTB lands on a later boundary.

By default the DTB advertises 64 MiB, the size of the simulation DDR model, which leaves limited room for an initramfs behind the Debian kernel. Board loading advertises the board's full DDR instead, and simulation uses DDR_MODEL_BYTES when it is set. The packer rejects an image that does not fit.

Interrupts and time

The DT connects the CLINT to the hart's cpu-intc for machine software (cause 3) and machine timer (cause 7) interrupts. Machine and supervisor IMSIC files connect to causes 11 and 9, with 63 identities per file. Each APLIC domain uses an msi-parent link to its IMSIC node, with two interrupt cells per source: identity and level-high type (4). OpenSBI selects its aia-imsic IPI device and disables the machine APLIC and IMSIC nodes in the tree passed to Linux. Linux uses the supervisor IMSIC for IPIs and APLIC MSIs for wired sources. The APLIC's destination registers are locked at reset. Programs that leave eidelivery at its reset value 0x40000000 continue to receive direct APLIC lines. The CLINT registers are dword-aligned and support 64-bit access: an ld reads mtime atomically, with no rv32-style hi/lo/hi loop, and an 8-byte store to mtimecmp lands atomically.

timebase-frequency equals the CPU clock: mtime counts every core cycle, with no divider. (Simulation builds can scale it with the SIM_TIMER_SPEEDUP parameter.) The packer writes the timebase and the UART's clock-frequency into the DTB from FPGA_CPU_CLK_FREQ. The loader supplies 322.265625 MHz by default, and FROST_CPU_CLK_HZ overrides it for divided-clock bitstreams. Because OpenSBI leaves menvcfg.STCE=1, the kernel programs its timer directly through Sstc (stimecmp) instead of an SBI call.

Advertised ISA

The generated DTB declares the implemented extensions and profile properties, including Sstc, Smaia, Ssaia, Sscofpmf, Sstvala, Svade, Svinval, Svpbmt and Svvptc, 64-byte riscv,cbom-block-size, riscv,cboz-block-size and riscv,cbop-block-size, and mmu-type = "riscv,sv39". Userspace uses RV64 ELF and the LP64D ABI. The ISA list includes Zihintntl, Zimop, Zcb and Zcmop. Svvptc tells Linux that a PTE changed from invalid to valid becomes visible to the walker without sfence.vma, so the kernel can omit that synchronization for those mapping updates. Other PTE changes retain their existing address-translation synchronization requirements. No FROST translation structure retains an invalid PTE indefinitely: the TLBs install only fault-free walks, the walker reads page tables coherently through the L1D probe, and the instruction MMU expires both its fault memo and completed registered refusals every 64 clocks. Even a refusal copied from the memo on its expiry edge is dropped within 128 clocks. Trap, xRET and FENCE-class full flushes also drop memo entries and verdicts, including a response on the flush edge.

Svpbmt: OpenSBI sets menvcfg.PBMTE when the DTB lists svpbmt, and Linux then maps ioremap regions with PBMT = IO and write-combining or non-coherent-DMA remaps with PBMT = NC (_PAGE_IO_SVPBMT, _PAGE_NOCACHE_SVPBMT, patched in by the kernel's alternatives). Device pages are uncached by their PMA already, so every MMIO driver behaves as before; an NC or IO page over DDR uses the configured uncached alias through the coherent DMA port. Loads issue at the ROB head, and alias stores retire after their bus response. Memory loads can be flushed and repeated after an interrupt or debug halt; the CPU memory-type description details the ordering and retry behavior for each physical address range. A 2 GiB DDR aperture has no alias in the current physical map and rejects those DDR mappings. For wider apertures, the top 2 GiB also has no alias. Switching between cacheable and uncached views requires fence iorw, iorw; cbo.flush; fence iorw, iorw for the location. The NIC's dma-coherent property keeps its normal DMA buffers cacheable; its IO register mappings exercise PBMT without using the DDR alias. /proc/cpuinfo lists svpbmt in its isa line.

Ssnpm: the DTB lists ssnpm, so Linux 6.13 and later (CONFIG_RISCV_ISA_SUPM) offer user pointer masking (Supm): prctl(PR_SET_TAGGED_ADDR_CTRL, PR_TAGGED_ADDR_ENABLE | PMLEN << 24) sets senvcfg.PMM for the process, and the kernel grants the smallest of FROST's PMLEN 7 and 16 that covers the request (HWASAN's 8 tag bits get 16). U-mode loads, stores, AMOs, LR/SC and CBOs then ignore the top PMLEN address bits; instruction fetch and the kernel's own accesses are never masked. riscv_hwprobe reports RISCV_HWPROBE_EXT_SUPM and /proc/cpuinfo lists ssnpm. The pinned Debian 6.12 kernel predates this interface; the supplied Linux workloads do not request tagged addresses.

Svnapot: the DTB lists svnapot. With CONFIG_RISCV_ISA_SVNAPOT and hugetlb enabled, Linux offers 64 KiB hugetlb pages (hugepages-64kB). A workload must allocate and map such pages to use NAPOT PTEs; ordinary boot and the supplied Linux stress workload do not exercise them. The walker translates the requested 4 KiB page, so a resident 64 KiB range occupies sixteen TLB entries. Linux invalidates a changed range with an address fence for each 4 KiB page or a fence covering the address space. A single address fence removes that page's entry; ASID filters spare global entries.

Counters and mcounteren

FROST implements cycle, time, and instret as 64-bit Zicntr CSRs; time reads the CLINT's mtime. The rv32-only *h aliases are illegal instructions at every privilege level.

Two WARL registers gate counter access below M-mode. S-mode needs the counter's bit set in mcounteren (0x306); U-mode needs it set in both mcounteren and scounteren (0x106). M-mode is never gated. A blocked access is an illegal instruction (mcause 2, mtval the instruction's bits).

CSR Behavior
mcounteren, scounteren CY, TM, IR and HPM3..10 occupy bits 10:0; bits 31:11 read zero and ignore writes. Both reset to 0x7ff.
mcountinhibit (0x320) CY (bit 0), IR (bit 2), and HPM3..10 (bits 10:3) inhibit their counters. TM and bits 31:11 read zero.
mcycle (0xB00), minstret (0xB02) Accept full 64-bit M-mode writes
mhpmcounter3-10 (0xB03-0xB0A), mhpmevent3-10 (0x323-0x32A) Writable 64-bit machine counters. An event field (bits 55:0) of N selects counter N's fixed event; counters 8, 7 and 4 also accept the translation selectors 16, 17 and 18, and counters 3, 6 and 5 the cache refill selectors 19, 20 and 21 (see below); any other value selects zero and disables counting. Bits 63:60 are the Sscofpmf OF, MINH, SINH and UINH bits; VSINH, VUINH and bits 57:56 read zero.
mhpmcounter11-31, mhpmevent11-31 Machine-mode reads return zero and writes are ignored.
hpmcounter3-10 (0xC03-0xC0A) Read-only aliases, gated by counter enables in S/U modes.
hpmcounter11-31 (0xC0B-0xC1F) Read-only zero in M-mode; accesses raise illegal instruction on writes or below M-mode (mcounteren bits 11..31 are hardwired zero).
scountovf (0xDA0) Sscofpmf: read-only OF bits 3..10. M reads all of them; S reads those set in mcounteren; U access is illegal.

Counters 9 and 10 count cycles and retired instructions like mcycle and minstret, with Sscofpmf overflow and privilege filtering. OpenSBI v1.9 with Sscofpmf but without Smcntrpmf does not fall back to the fixed counters for these events, so the DTB maps cycles to counter 9 and instructions to counter 10.

Sscofpmf

A counter whose 64-bit count wraps sets its mhpmevent OF bit. If OF was clear, the wrap also sets mip.LCOFIP (interrupt 13), which mideleg bit 13 routes to S-mode, where it appears in sip/sie and stopi. LCOFI has the lowest default priority, below STI. OpenSBI detects Sscofpmf by reading scountovf, delegates LCOFI, and programs MINH and the requested SINH/UINH filters; the kernel's riscv-pmu-sbi driver takes the overflow interrupt through the AIA stopi path, reads scountovf, and clears sip.LCOFIP. This is the mechanism perf record and perf top sample with. Retirement events (branches, mispredictions, instructions) count in the privilege the instruction retired in; the other events count in the privilege of the cycle they occur in. Debug Mode counts as M. Smcntrpmf is not implemented, so the fixed counters mcycle and minstret count in every privilege.

OpenSBI's SBI PMU uses mcountinhibit, mcycle, and minstret to stop, start, and preload the fixed counters, and its privileged-version probe requires mcountinhibit before it sets menvcfg.STCE. OpenSBI hands the kernel scounteren=0x2, so userspace starts with only rdtime; the kernel decides whether to allow direct rdcycle and rdinstret.

Userspace counts cycles and instructions through perf_event_open (PERF_COUNT_HW_CPU_CYCLES, PERF_COUNT_HW_INSTRUCTIONS), which Linux serves through the SBI PMU. The DTB's riscv,pmu node maps both events to counters 9 and 10. Standard events map retired branches, branch mispredictions, ITLB read misses, and L1D permission misses. Permission misses include atomic requests that require cache permission. Raw selectors 3 through 10 select branches, mispredictions, L1 ITLB misses, L1 DTLB misses, L1D permission misses, store-queue capacity stalls, cycles, and retired instructions, respectively. L1 TLB misses include requests answered by the second-level TLB. DTLB misses include loads and stores; SQ stalls cover one source of backend stalls. These two events are exposed only through riscv,raw-event-to-mhpmcounters. Three raw translation selectors share those counters: 16 counts page-walker busy cycles on counter 8, 17 counts SFENCE.VMA and SINVAL.VMA TLB invalidates on counter 7, and 18 counts CSR-requested translation-state flushes (any satp access, or an mstatus write that changes MPRV, or MPP while MPRV is set) on counter 4. Walker busy cycles divided by ITLB plus DTLB misses give walker busy cycles per L1 TLB miss, including second-level TLB hits in the denominator. CSR-requested flushes can preserve ASID-tagged TLB entries. Three raw cache refill selectors follow: 19 counts L1I refills (lines the L1I requests from the L2, including the fetch provider's next-line prefetches) on counter 3, 20 counts L2 demand misses (lines the coherence home reads from memory for CPU and DMA requests) on counter 6, and 21 counts L2 prefetch misses (lines the L2 prefetcher reads from memory) on counter 5. The generic L1-icache-load-misses, LLC-load-misses and LLC-prefetch-misses events map to them; perf stat -e r13,r14,r15 names them as raw events. A counter counts either its fixed event or its alternate selector, so selector 19 replaces branches on counter 3, and 20 and 21 replace the L1 DTLB and ITLB misses. The L1I event counts allocations that fetch a line, including speculative fetches that are later discarded. L2 events count accepted memory line reads, including reads that return an error. A prefetch fill stays a prefetch event when a demand joins it. A zero fill sends no read and contributes no event. LLC-load-misses includes instruction reads, page-table walks, data ownership fills and coherent DMA or NIC traffic. Multiple requests sharing a fill contribute one memory read. The coherent L1D leaves its refill taps zero; its permission-miss event remains separate from these L2 line reads.

The cache observer and cpu_ooo each register the refill pulse before the CSR logic samples it. Privilege filtering uses the hart's mode at that sample, including for DMA traffic. A refill caused before a privilege transition can therefore count in the mode after the transition.

HPM storage and event counting are independent of the optional profiling counters. frost_stress --counters counts a child process running a fixed workload, from its exec to its exit, and the boot stress payload reports cycle, instruction, time, and IPC deltas for its own workload.

Kernel

debian_kernel.py downloads Debian's riscv64 kernel and headers from snapshot.debian.org and checks their sizes and SHA-256 sums. It also requires built-in APLIC, APLIC MSI and IMSIC drivers in the packaged kernel configuration, including when reusing a cached extraction. The pinned release is 6.12.113+deb13-riscv64; the snapshot, versions, and checksums are in the script. FROST runs this kernel unmodified, with its NIC and memory error drivers built as separate modules.

Its subcommands, fetch, release, image, module, and initramfs, print their result on stdout and progress on stderr. module and initramfs are described under Driver modules. FROST_DEBIAN_KERNEL_CACHE overrides the default linux/debian-kernel cache. Native and container builds can use the cache concurrently, and an incomplete entry is rebuilt.

To move to a new kernel, change the pinned values in debian_kernel.py, rebuild the images, and install the same release on each board's NFS root as the Debian guide describes. Then rerun the Linux boot checks.

A replacement kernel selected with FROST_LINUX_KERNEL must provide:

Option Why
CONFIG_MMU, CONFIG_ARCH_RV64I, CONFIG_64BIT Sv39 RV64 kernel in S-mode. Leave CONFIG_RISCV_M_MODE unset.
CONFIG_RISCV_SBI Runs under OpenSBI and calls the SBI interface
CONFIG_RISCV_APLIC, CONFIG_RISCV_APLIC_MSI, CONFIG_RISCV_IMSIC Built-in APLIC MSI delivery and IMSIC interrupt files for wired interrupts and IPIs. All three are enabled in the pinned kernel.
CONFIG_RISCV_PMU, CONFIG_RISCV_PMU_SBI Cycle and instruction counters through the SBI PMU
CONFIG_RISCV_MISALIGNED Handles misaligned accesses once the kernel takes FWFT delegation. The pin selects it through CONFIG_RISCV_PROBE_UNALIGNED_ACCESS; CONFIG_RISCV_EMULATED_UNALIGNED_ACCESS also works.
CONFIG_BINFMT_ELF, CONFIG_FPU ELF userspace with LP64D hard float
CONFIG_BLK_DEV_INITRD External initramfs through linux,initrd-*. A compressed initramfs also needs its CONFIG_RD_* decompressor; the test initramfs is uncompressed.
CONFIG_DEVTMPFS, CONFIG_TMPFS, CONFIG_PROC_FS, CONFIG_SYSFS Mounted by the test initramfs's inittab. It mounts devtmpfs itself, so CONFIG_DEVTMPFS_MOUNT is not needed (the pin leaves it unset).
CONFIG_SERIAL_8250, CONFIG_SERIAL_8250_CONSOLE, CONFIG_SERIAL_OF_PLATFORM Console on the ns16550a alias, found through the DT. Build these in: the test initramfs has no serial modules.
CONFIG_OF, CONFIG_OF_EARLY_FLATTREE Device-tree probing and the early console (earlycon=uart8250,mmio32,0x40001000)
CONFIG_NET, CONFIG_INET, CONFIG_PACKET Packet sockets for frost_nettest, and IPv4 for an NFS root
CONFIG_MODULES, or the drivers built in (CONFIG_NETDEVICES, CONFIG_ETHERNET, CONFIG_NET_VENDOR_FROST, CONFIG_FROST_NET10G, CONFIG_FROST_MEMERR) The frost_net10g driver for the frost,net10g node and frost_memerr for frost,memory-error. The pin loads both as modules (Driver modules).
CONFIG_DEBUG_FS The memory error driver's manual scrub and self-test files. The hardware regression requires the self-test.
CONFIG_NFS_FS, CONFIG_NFS_V3 The kernel's NFS client, needed for either kind of NFS root: the kernel performs the mount even when an initramfs requests it. The pin builds both as modules, which its initramfs carries.
CONFIG_ROOT_NFS, CONFIG_IP_PNP (CONFIG_IP_PNP_DHCP for ip=dhcp) Only for a kernel that mounts the NFS root itself from ip= and nfsroot=. That kernel also needs the NFS client and the NIC driver built in. The pin has neither option.
CONFIG_CGROUPS, CONFIG_UNIX Required by systemd on the NFS root, which needs the cgroup v2 hierarchy but no particular controller (the pin enables the usual ones). CONFIG_AUTOFS_FS, CONFIG_TMPFS_POSIX_ACL, and CONFIG_TMPFS_XATTR are recommended. systemd's other requirements, and the seccomp support it recommends, are kernel defaults.

The test image boots with ipv6.disable=1: Debian's kernel builds IPv6 in, and its autoconfiguration frames would loop back through the NIC and fail frost_nettest's idle checks. IPv4 cannot be turned off on an NFS root, and the kernel may send ARP requests for the server while the interface is looped back. frost_nettest captures every frame of the interface and accepts such a frame only when the packet counters, the frames the kernel sent and their looped copies all agree; see the package README for the rules.

Driver modules

debian_kernel.py module builds the NIC driver and the memory error driver against the pinned kernel's headers and checks each module's name, vermagic release and symbol CRCs against that kernel. Linux would not catch a mismatch: the pin sets CONFIG_MODVERSIONS, under which the kernel ignores the release in vermagic and loads the module into any ABI-compatible kernel. The image build compiles the modules with Buildroot's cross toolchain; a standalone build uses FROST_LINUX_CROSS_COMPILE or --cross.

debian_kernel.py initramfs appends the modules and a startup script for each to Buildroot's rootfs.cpio as a second cpio archive, which the kernel unpacks after the first, so Buildroot does not rebuild. For the same vermagic reason, each script checks uname -r itself before it loads its module, then prints FROST_NET10G_MODULE_PASS <release> or FROST_NET10G_MODULE_FAIL <reason>, and FROST_MEMERR_MODULE_PASS <release> or FROST_MEMERR_MODULE_FAIL <reason>, before login. FROST_NET10G_MODULE and FROST_MEMERR_MODULE name prebuilt modules to use instead of building them.

For Debian NFS boots, install the NIC driver through DKMS and include it in Debian's initramfs, as the driver guide describes. The memory error driver is not needed to mount the root: the hardware regression installs and loads its own build, and a DKMS install (its guide) loads it at every boot.

The memory error driver also rewrites lines with corrected errors and reads all of memory in the background to find more; its guide describes both and the hardware regression's checks.

NFS root

Follow the Debian setup guide to build and export a riscv64 root filesystem. Debian's kernel builds its NFS client as a module and has no ip= autoconfiguration, so it mounts the root from an initramfs that carries the NFS client and the NIC driver. The export must allow read-write access from the board without root squashing.

sw/apps/linux_boot reads these environment variables, which the loader passes through:

Variable Purpose
FROST_LINUX_NFSROOT Export as <server-ip>:/<path>. Unset, the image boots the test initramfs.
FROST_LINUX_IP Kernel ip= value; defaults to dhcp and requires FROST_LINUX_NFSROOT
FROST_LINUX_KERNEL Absolute path to a replacement kernel Image
FROST_LINUX_INITRD Absolute path to a replacement initramfs
FROST_LINUX_DTB Absolute path to a compiled device tree blob (.dtb) packed instead of generating one
FROST_LINUX_MAC Board MAC address; defaults to 02:11:22:33:44:55

For Debian, pack the root's /boot/vmlinux-<version> with the matching /boot/initrd.img-<version>. The hardware regression requires the pinned release and picks the kernel and initramfs itself; see its setup.

A static IP uses <client>::<gateway>:<netmask>:<hostname>:<device>:off, for example 192.0.2.2::192.0.2.1:255.255.255.0:frost:eth0:off. Each board on a shared network needs its own locally administered MAC. The packer rejects malformed, multicast, and all-zero addresses.

With both FROST_LINUX_NFSROOT and FROST_LINUX_INITRD set, the packer writes bootargs that select initramfs-tools' NFS boot (boot=nfs) and an NFSv3 mount over TCP:

earlycon console=ttyS0 boot=nfs root=/dev/nfs nfsroot=<server-ip>:/<path>,vers=3,tcp,hard rw ip=<FROST_LINUX_IP>

With FROST_LINUX_NFSROOT alone, the packer packs no initramfs and the kernel mounts the root itself. That needs a replacement kernel with CONFIG_IP_PNP, CONFIG_ROOT_NFS, the NFS client, and the NIC driver built in; Debian's kernel cannot do it. Both paths mount with nolock, so file locks are local to the board.

Without FROST_LINUX_NFSROOT, the kernel runs the initramfs's /sbin/init, so a replacement initramfs must provide it. Debian's initramfs-tools image starts at /init and works only with the NFS boot above.

Images and testing

sw/apps/linux_boot builds the images the FPGA loader writes: sw.{mem,txt} is the boot shim for low BRAM, sw_ddr.{mem,txt} is the DDR image, and sw64.mem is the shim in dword rows for the 64-bit data BRAM. By default the DDR image holds OpenSBI, the pinned Debian kernel, and the test initramfs from the Buildroot flow.

Linux boots are tested on X3 hardware and under QEMU. RTL simulation covers OpenSBI and directed bare-metal tests, but not a full Linux boot. On the board, the hardware regression boots Debian from its NFS root, and fpga/linux_boot_soak.py boots the test image repeatedly.

The NFS-root stage accepts systemd running, or degraded when all failed units are session-N.scope units with Result=resources, and logs the tolerated scopes. It runs the static NIC loopback test from /dev/shm after dropping dentry and inode caches. The test opens counter and carrier files while the link is up, then holds the interface down for 65 seconds to expire NFS's default directory attribute cache. It reads through those descriptors during loopback. The shell restores the link through files opened before the test, then checks NFS writes and sync.

CI boots the packaged kernel and initramfs on QEMU's virt machine with 256 MiB using both bundled OpenSBI and FROST's firmware. A third boot uses FROST's firmware and 3 GiB of RAM starting at 0x8000_0000. It requires Linux's NODE_DATA allocation above 4 GiB, successful module loads, the userspace stress token, and a login prompt. This checks firmware and Linux use of high physical memory on QEMU. It does not exercise FROST's RTL, DDR controller, or NIC traffic above 4 GiB.