Operational reliability: hardware-timed chirp bursts, FPGA heartbeat, and RX/TX synchronization

[alvarosamudio]

Summary

This PR addresses three operational reliability issues on the AERIS-10:

1. Chirp sequence jitter — the STM32 generated per-chirp timing in software (GPIO toggles + delay_us/delay_ns busy-waits), injecting interrupt jitter into the chirp windows and corrupting Doppler phase coherence. The FPGA already had a hardware-timed chirp controller; the MCU now fires a complete burst with a single toggle.
2. No FPGA liveness — the STM32 had no way to detect an FPGA hang (ERROR_FPGA_COMM existed in the enum but was never triggered). A 1 Hz heartbeat on DIG_7 with automatic recovery was added.
3. Receiver drift — the receiver auto-scan ran free-running, drifting from the MCU-gated TX bursts (gaps for SPI beam-pattern writes). RX frames are now locked 1:1 to TX burst starts.

Along the way we fixed a latent chirp controller bug that broke every burst after the first, and a scan counter shadowing bug that froze the state shown in the GUI.

Changes by area

FPGA — 9_Firmware/9_2_FPGA

plfm_chirp_controller.v — burst fix (latent bug)

• chirp_counter is reset to 0 on entering DONE.
• Why: the counter stayed at CHIRP_MAX after the first burst; every subsequent burst ran 48+ long chirps before reaching the GUARD transition (only corrected by the 6-bit wrap at 64). With single-trigger bursts from the MCU, each burst must start at 0: exactly 16 long + 16 short.

radar_system_top.v — FPGA heartbeat on DIG_7

• gpio_dig7 (previously reserved, tied low) now toggles at ~1 Hz (27-bit counter in clk_100m, 100 MHz / 50 M = 1 Hz).
• Why: the STM32 needs to detect an FPGA hang/crash. With a dead FPGA and TX mixers enabled, RF can be left in an uncontrolled state.

radar_mode_controller.v + radar_receiver_final.v — gated auto-scan

• New frame_gate port (1-cycle pulse). In mode 01 (auto-scan), the FSM waits in S_IDLE for the TX gate instead of starting immediately; after frame completion it returns to S_IDLE. The next frame's first mc_new_chirp toggle only happens on the gate.
• Elevation/azimuth tracking is preserved (cosmetic counters).
• Wiring: .frame_gate(tx_frame_start) in radar_receiver_final.v — the gate is the TX's new_chirp_frame (toggle-CDC into clk_100m).
• Modes 00 (STM32 pass-through) and 10 (single-chirp) are unchanged.
• Why: the auto-scan receiver ran continuously from reset; with MCU-gated TX bursts (pauses for SPI beam-pattern writes), RX chirp windows drifted from the actual bursts, shifting the matched-filter correlation peak and degrading Doppler. The gate eliminates the drift and prevents processing "chirps" with no TX between bursts.

formal/fv_radar_mode_controller.v — formal wrapper updated

• frame_gate added as an anyseq input; Property 6 changed from "never stalls in S_IDLE" to "starts only with the gate and stays in S_IDLE without it".

FPGA testbenches

tb/tb_chirp_contract.v — C7 contract updated: the counter now auto-resets at DONE (previously "holds at CHIRP_MAX"); C7b/C7d verify the per-burst auto-reset. C1 comments updated.

tb/tb_radar_mode_controller.v — frame_gate signal + race-free pulse_gate task (deassert with #1 after the sampling edge, avoiding a race with the DUT's always block); gates pulsed in the mode-01 groups; scan_complete counting moved before the re-gate (the pulse lasts 1-2 cycles and pulse_gate consumes 3 edges); new Group 17 verifies the gating contract (no scanning without a gate, one frame per gate, no activity between gates, second gate works). A full-scan testbench bug (elevation/azimuth reset on every gate) was fixed in the RTL: the gate only resets chirp_count.

tb/tb_radar_receiver_final.v — tx_frame_start (the gate) is now generated independently: an initial pulse after reset release plus a pulse on each scanning falling edge (frame end). Why: the previous derivation (from the mode controller's own chirp_count wrap) was circular with the gate — the RX would never have started.

tb/tb_system_e2e.v — G8.3 now verifies gated idle (previously: elevation counter ≥ 1); G9 fires one chirp toggle after reset to generate the gate; G11 restructured with its own reset, short timing, and a single toggle (latency measurement against a gated frame).

Golden regenerated — tb/golden/golden_doppler.mem and tb/cosim/rx_final_doppler_out.csv. Why: with the gate, the frame starts ~2 cycles later; the frame content changes and the old golden was stale. Regenerated against the merged receiver chain (including develop's ddc_400m.v). Exact comparison: 2048/2048 within tolerance (±2 LSB), 0 mismatches.

Test infrastructure

run_regression.sh — run_test accepts a per-test timeout (default 120 s); the golden tests now use 480 s. Why: the golden tests simulate ~1.3 ms with a 400 MHz ADC clock under iverilog; the fixed 120 s timeout was insufficient even before this PR (~5.5 min on the dev machine).

STM32 firmware — 9_Firmware/9_1_Microcontroller/9_1_3_C_Cpp_Code/main.cpp

executeChirpSequence() → burst mode with hardware timing

• Sequence per block: (1) pulseTXMode() → TR_SOURCE=1 (bit 2 of REG_SW_CONTROL 0x031) so the ADAR1000 TR pins control TX/RX; (2) one GPIO toggle starts the whole burst (16 long + guard + 16 short); (3) deterministic wait of num_chirps*PRI1 + Guard + num_chirps*PRI2 ≈ 5.65 ms (not timing-critical: chirp boundaries come from the FPGA); (4) pulseRXMode() → back to SPI RX (safe idle).
• Why: per-chirp toggles + busy-waits injected interrupt jitter into the chirp windows (kills Doppler phase coherence), and SPI-based TR switching inside the short chirp (0.5 µs) was physically impossible (an SPI transaction outlasts the chirp). The FPGA already drives the TR pins with hardware timing; what was missing was enabling the ADAR1000 pin override (verified in the datasheet Rev. B, Table 76: bit 2 = TR_SOURCE, 0 = SPI, 1 = TR pin; pin high = TX).

runRadarPulseSequence() → runRadarScanStep() (per-burst state machine)

• One burst (~6 ms) per call; the scan protocol is identical (elevation toggle per position, 3 patterns matrix1/broadside/matrix2 of 16+16 chirps each, azimuth toggle + stepper step per sweep).
• Why: the old loop blocked ~4 s per sweep; health checks, GPS, USB, and the heartbeat only ran between sweeps. They now run every ~6 ms: fault-response latency goes from ~4 s to ~6 ms, with much more watchdog margin.
• Counter fix: m/n/y now update the globals (local shadowing previously left the GUI stuck at BeamPos:1, Azimuth:1, ChirpCount:1).
• The outer AGC now updates per burst instead of per full scan.

checkFpgaHeartbeat() — liveness supervision with automatic recovery

• Edge detection on DIG_7 (PD15); 4 s timeout; cold-start seed on the first call (also detects an FPGA that never configures at boot); first failure → safe automatic re-init (TX mixers off → FPGA reset pulse 10 ms → mixers back on) and re-arm; persistent failure → ERROR_FPGA_COMM (12, inside the critical range 9-13 → Emergency_Stop, because an unresponsive FPGA can leave the PA in an uncontrolled TX state).

Verification

Test	Result
Full FPGA regression (27 tests: lint, chirp contract, mode controller 89 checks, golden gen 17/17 + cmp 18/18, E2E in both FT601 and FT2232H USB modes)	27/27 PASS
Quick FPGA regression	21/21 PASS
MCU tests (make test)	75/75 PASS
Host functional heartbeat test (4 scenarios: normal boot, dead at boot, transient hang with recovery, persistent hang with escalation)	12/12 PASS
Golden compare	2048/2048 exact (±2 LSB)
C++ syntax of the modified functions	0 errors

Risks / hardware pending

• TR pin override not bench-verified: TR_SOURCE=1 was validated against the datasheet (REG 0x031, bit 2) and the FPGA wiring (Bank 34, 1.8 V, adar_tr_1..4 pins), but final confirmation requires testing on hardware. If the pin override is not active on the bench, pin-based TX/RX switching will not work (the previous SPI-based behavior remains available as a fallback).
• CI has never run on GitHub: .github/workflows/ci-tests.yml is in the local repo but has never executed on the remote; opening this PR will run it for the first time (the full regression takes ~12 min on the dev machine).

Commits included

1. fix(fpga): reset chirp_counter at burst end for correct multi-burst operation (2026-07-03 01:39)
2. feat(fpga): add 1 Hz FPGA heartbeat on DIG_7 (2026-07-08 01:16)
3. feat(fpga): gate receiver auto-scan on transmitter frame start (2026-07-13 03:14)
4. test(fpga): update testbenches for gated auto-scan and regenerate golden (2026-07-19 02:13)
5. test(fpga): raise golden test timeout in regression runner (2026-07-25 01:39)
6. feat(mcu): FPGA-timed chirp bursts, responsive scan loop and FPGA heartbeat supervision (2026-08-01 00:46)