RogueTcp Stream/Memory/SideBand: GHDL VHPIDIRECT cosim + cocotb CI coverage by ruck314 · Pull Request #1450 · slaclab/surf

ruck314 · 2026-07-05T17:26:30Z

Description

Adds GHDL/VHPIDIRECT co-simulation backends for the Rogue TCP Stream, Memory, and SideBand models with cocotb regression coverage in CI. The simlink tree is organized into vcs/, ghdl/, and shared simulator-independent C cores so GHDL and VCS use the same ZeroMQ wire protocols and model state machines.

Each elaborated GHDL model owns independent native state through an explicit positive integer handle, including model creation, lookup, port reservation, destroy, and process-exit cleanup. This supports multi-link Stream users and concurrent Stream, Memory, and SideBand instances while preserving the existing public VHDL interfaces.

The change also fixes three shared C-model defects found during regression development: an uninitialized Memory read-frame copy, sparse Stream tKeep byte over-counting, and mismatched Memory socket NULL guards.

Details

Reorganizes the existing VCS/VHPI sources and extracts RogueTcpStreamCore.h, RogueTcpMemoryCore.h, and RogueSideBandCore.h as the shared model implementation.
Adds thin GHDL/VHPIDIRECT VHDL and C adapters plus cocotb-facing wrappers.
Adds deterministic Rogue-TCP/ZeroMQ peers, single-model round trips, an active mixed regression with four Stream, two Memory, and two SideBand instances, repeated-reset coverage, duplicate/invalid-handle checks, destroy/rebind coverage, and Linux Valgrind lifecycle checking.
Wires GHDL, ZeroMQ, pyzmq, and Valgrind dependencies into CI and keeps the simlink suite in the change selector.

The shared simulator-thread transport policy is intentionally unchanged: receives poll nonblocking, sends remain synchronous, and the peer must be connected and draining before outbound HDL traffic. No-peer, saturation, send-timeout, linger, retry-queue, or worker-thread changes belong in a later cross-backend hardening pull request covering VCS, GHDL, and xsim together.

Validation completed on implementation head 989731d754e729d3866308e7529c0c2842a73695; GitHub Actions run 29461564882 passed lint, documentation, and regression checks. Linux reported 813 passed and 21 skipped, including active multi-instance traffic, lifecycle Valgrind, duplicate-port, and invalid-handle tests. VCS execution was not available locally, but the VCS source path and shared-core consumers were reviewed and the established wire formats and transport policy were preserved.

Pull request overview

This PR brings the Rogue TCP sim-link stream, memory, and sideband simulation models under GHDL using VHPIDIRECT (while keeping the existing VCS/VHPI backend), adds shared simulator-agnostic C cores, and introduces cocotb/valgrind-driven CI coverage to prevent regressions.

Changes:

Reorganize axi/simlink into shared/ cores plus thin vcs/ (VHPI) and ghdl/ (VHPIDIRECT) backends.
Add always-run selection for tests/axi/simlink/ in the selective test resolver, plus new cocotb round-trip/smoke tests and a pyzmq peer.
Wire CI dependencies (libzmq, valgrind, pyzmq) and add flat VHDL wrappers for cocotbext.axi.

Reviewed changes

Copilot reviewed 43 out of 49 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/common/test_dep_map.py	Adds unit test ensuring simlink tests are always-run in selective resolution.
tests/common/dep_map.py	Adds directory-scoped always-run rule for `tests/axi/simlink/`.
tests/axi/simlink/uninit_read_recv_harness.c	Standalone harness to reproduce/guard uninitialized read bug under valgrind.
tests/axi/simlink/test_RogueTcpStreamWrap.py	Adds cocotb round-trip + sparse-tkeep regression for stream flat wrapper.
tests/axi/simlink/test_RogueTcpStream.py	Adds GHDL/VHPIDIRECT elaborate/bind smoke test for stream model.
tests/axi/simlink/test_RogueTcpMemoryWrap.py	Adds cocotb round-trip test for memory flat wrapper + valgrind repro.
tests/axi/simlink/test_RogueTcpMemory.py	Adds GHDL/VHPIDIRECT elaborate/bind smoke test for memory model.
tests/axi/simlink/test_RogueSideBandWrap.py	Adds cocotb round-trip test for sideband flat wrapper.
tests/axi/simlink/test_RogueSideBand.py	Adds GHDL/VHPIDIRECT elaborate/bind smoke test for sideband model.
tests/axi/simlink/simlink_test_utils.py	Adds helper to build/stage VHPIDIRECT `.so` and ensure `LD_LIBRARY_PATH` works in CI.
tests/axi/simlink/rogue_tcp_peer.py	Adds separate-process pyzmq peer implementing the simlink wire protocols for tests.
tests/axi/simlink/init.py	Marks simlink tests as a Python package.
ruckus.tcl	Bumps required ruckus submodule version.
pip_requirements.txt	Adds `pyzmq` for the new peer/tests.
conda.yml	Adds `pyzmq` to conda environment.
axi/simlink/wrappers/RogueTcpStreamWrapFlatWrapper.vhd	Adds cocotb-facing flat AXI-stream wrapper for `surf.RogueTcpStreamWrap`.
axi/simlink/wrappers/RogueTcpMemoryWrapFlatWrapper.vhd	Adds cocotb-facing flat AXI-Lite wrapper for `surf.RogueTcpMemoryWrap`.
axi/simlink/wrappers/RogueSideBandWrapFlatWrapper.vhd	Adds cocotb-facing flat wrapper for `surf.RogueSideBandWrap`.
axi/simlink/vcs/VhpiGeneric.h	Moves/hosts VHPI generic interface header under `vcs/`.
axi/simlink/vcs/VhpiGeneric.c	Moves/hosts VHPI generic interface implementation under `vcs/`.
axi/simlink/vcs/RogueTcpStream.vhd	Moves VHPI-bound VHDL entity into `vcs/`.
axi/simlink/vcs/RogueTcpStream.h	Updates VHPI backend header to use shared core via snapshot seam.
axi/simlink/vcs/RogueTcpStream.c	New thin VHPI backend linking shared stream core.
axi/simlink/vcs/RogueTcpMemory.vhd	Moves VHPI-bound VHDL entity into `vcs/`.
axi/simlink/vcs/RogueTcpMemory.h	Updates VHPI backend header to use shared core via snapshot seam.
axi/simlink/vcs/RogueTcpMemory.c	New thin VHPI backend linking shared memory core.
axi/simlink/vcs/RogueSideBand.vhd	Moves VHPI-bound VHDL entity into `vcs/`.
axi/simlink/vcs/RogueSideBand.h	Updates VHPI backend header to use shared core via snapshot seam.
axi/simlink/vcs/RogueSideBand.c	New thin VHPI backend linking shared sideband core.
axi/simlink/vcs/Makefile	Updates includes to build against `shared/` core headers.
axi/simlink/src/RogueTcpStream.c	Removed (logic moved into `shared/` + thin `vcs/` backend).
axi/simlink/src/RogueTcpMemory.c	Removed (logic moved into `shared/` + thin `vcs/` backend).
axi/simlink/src/RogueSideBand.c	Removed (logic moved into `shared/` + thin `vcs/` backend).
axi/simlink/shared/RogueTcpStreamCore.h	New shared stream transport+FSM core used by both backends.
axi/simlink/shared/RogueTcpMemoryCore.h	New shared memory transport+FSM core used by both backends.
axi/simlink/shared/RogueSideBandCore.h	New shared sideband transport+FSM core used by both backends.
axi/simlink/ruckus.tcl	Switches sim-only VHPI sources from `sim/` to `vcs/` when not using GHDL.
axi/simlink/ghdl/RogueTcpStream.vhd	Adds GHDL VHPIDIRECT binding for stream model.
axi/simlink/ghdl/RogueTcpStream.h	Adds GHDL backend header/shims for stream model.
axi/simlink/ghdl/RogueTcpStream.c	Adds GHDL VHPIDIRECT backend implementation for stream model.
axi/simlink/ghdl/RogueTcpMemory.vhd	Adds GHDL VHPIDIRECT binding for memory model.
axi/simlink/ghdl/RogueTcpMemory.h	Adds GHDL backend header/shims for memory model.
axi/simlink/ghdl/RogueTcpMemory.c	Adds GHDL VHPIDIRECT backend implementation for memory model.
axi/simlink/ghdl/RogueSideBand.vhd	Adds GHDL VHPIDIRECT binding for sideband model.
axi/simlink/ghdl/RogueSideBand.h	Adds GHDL backend header/shims for sideband model.
axi/simlink/ghdl/RogueSideBand.c	Adds GHDL VHPIDIRECT backend implementation for sideband model.
axi/simlink/ghdl/Makefile	Adds GHDL-only Makefile to build VHPIDIRECT `.so` modules.
.gitignore	Broadens ignore pattern to include `sim_build*` dirs.
.github/workflows/surf_ci.yml	Adds CI deps (libzmq, valgrind) and runs simlink tests under xdist.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 43 out of 49 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 43 out of 49 changed files in this pull request and generated no new comments.

Reorganize the Rogue simlink sources to support multiple simulator backends. The VCS/Vivado VHPI-specific C shims, headers, Makefile, and VHDL move from src/ and sim/ into vcs/, while the backend-agnostic protocol logic (ZMQ peering, memory/stream/side-band framing) is lifted into shared/ core headers (RogueTcpMemoryCore.h, RogueTcpStreamCore.h, RogueSideBandCore.h) that any backend can include. Update axi/simlink/ruckus.tcl to reference the new paths.

Add a GHDL VHPIDIRECT backend for the Rogue simlink modules alongside the existing VCS/Vivado path. New ghdl/ C shims bind the shared core to GHDL's VHPIDIRECT ABI, and the paired VHDL exposes foreign subprograms for RogueTcpMemory, RogueTcpStream, and RogueSideBand. Add flat-port wrappers under wrappers/ (RogueTcpMemoryWrapFlatWrapper, RogueTcpStreamWrapFlatWrapper, RogueSideBandWrapFlatWrapper) so records are flattened to std_logic_vector for cocotb DUT access. Fail fast if a VHPIDIRECT backend is instantiated more than once, since the C core keeps per-module global state that a second instance would corrupt.

Add a cocotb test suite for the GHDL VHPIDIRECT simlink backend covering RogueTcpMemory, RogueTcpStream, and RogueSideBand plus their flat Wrap variants. rogue_tcp_peer.py provides a Rogue-TCP ZMQ peer that drives the DUT over the same wire protocol Rogue uses, and simlink_test_utils.py builds the per-test .so, serializing that build under a file lock so pytest-xdist workers do not race.

Add pyzmq to conda.yml and pip_requirements.txt for the Rogue-TCP ZMQ peer, add libzmq3-dev and valgrind to the regression job's apt install so the Rogue-TCP VHPIDIRECT shims link and test_RogueTcpMemoryWrap.py can run under valgrind, and widen the .gitignore sim_build rule to sim_build* for the generated sim build artifacts.

Stop loading the simulation-only wrappers/ directory from protocols/jesd204b/ruckus.tcl so it stays out of the synthesized surf library. The flattened surf library imported for the cocotb regression then no longer contains the wrapper entities and GHDL fails to elaborate them (jesdrxlanewrapper, jesd204brxwrapper, jesdscramblerwrapper, jesdtxlanewrapper, jesd204btxwrapper, jesd204bloopbackwrapper). Add a jesd_wrapper_sources() helper to jesd204b_test_utils.py and pass each bench its wrapper through run_surf_vhdl_test(extra_vhdl_sources=...), so the wrapper compiles on top of the already-imported RTL. Keeps the simulation-only wrappers out of the synthesized library while restoring test coverage.

bengineerd

Review found three areas to address: fixed-buffer bounds safety, stale incremental GHDL builds, and regression-environment portability. Details are inline.

Guard both directions of the shared AXI-Stream core against writing past the fixed MAX_FRAME buffers. RogueTcpStreamRecv() copied the peer-declared size of the fourth ZMQ frame straight into obData[MAX_FRAME]; reject the frame first. The inbound RogueTcpStreamStep() loop grew ibSize per kept lane with no cap, so a frame that never asserted tLast overran ibData[MAX_FRAME]; guard ibSize before each byte write.

RogueTcpMemoryRecv() trusted the peer-declared size: a write copied it into the fixed data[MAX_DATA] buffer, and the AXI-Lite FSM advances curr four bytes per beat and completes only when curr == size. An oversized size overran data[], and a non-word-sized size made curr step past size so the transaction never terminated. Reject size > MAX_DATA or not a multiple of four before the size is used.

The GHDL Makefile pattern rule tracked only the .c file, so editing a shared shared/RogueTcp*Core.h (or a local RogueTcp*.h / RogueVhpiDirect.h) left the previously built .so in place and pytest loaded the stale library, producing false-green local runs. Emit per-module dependency files with -MMD -MP and -include them so any compiled-through header is a prerequisite.

test_RogueTcpMemory_uninitialized_read invokes valgrind unconditionally, which errors on macOS and on a clean conda env without valgrind. Skip it unless running on Linux with valgrind on PATH. Add zeromq and pkg-config to conda.yml so the simlink .so actually builds in the conda environment (the apt step only covers the Ubuntu CI runner).

ruck314 force-pushed the ci-RogueTcpMemoryWrap-RogueTcpStreamWrap branch from 7a1feed to b996467 Compare July 5, 2026 23:16

ruck314 changed the title ~~RogueTcp Stream/Memory: GHDL VHPIDIRECT cosim + cocotb CI coverage~~ RogueTcp Stream/Memory/SideBand: GHDL VHPIDIRECT cosim + cocotb CI coverage Jul 5, 2026

ruck314 requested a review from Copilot July 5, 2026 23:24

Copilot started reviewing on behalf of ruck314 July 5, 2026 23:24 View session

Copilot AI reviewed Jul 5, 2026

View reviewed changes

ruck314 requested a review from Copilot July 5, 2026 23:49

Copilot started reviewing on behalf of ruck314 July 5, 2026 23:50 View session

Copilot AI reviewed Jul 5, 2026

View reviewed changes

Comment thread tests/axi/simlink/simlink_test_utils.py

Comment thread axi/simlink/wrappers/RogueTcpStreamWrapFlatWrapper.vhd

ruck314 requested a review from Copilot July 6, 2026 00:05

Copilot started reviewing on behalf of ruck314 July 6, 2026 00:05 View session

Copilot AI reviewed Jul 6, 2026

View reviewed changes

ruck314 force-pushed the ci-RogueTcpMemoryWrap-RogueTcpStreamWrap branch 2 times, most recently from 0d83efe to 1e19314 Compare July 6, 2026 00:21

ruck314 marked this pull request as ready for review July 6, 2026 00:35

ruck314 requested review from bengineerd and slacrherbst as code owners July 6, 2026 00:35

ruck314 mentioned this pull request Jul 6, 2026

Release Candidate v2.73.0 #1441

Open

ruck314 force-pushed the ci-RogueTcpMemoryWrap-RogueTcpStreamWrap branch from 1e19314 to 58f111b Compare July 6, 2026 17:33

ruck314 changed the base branch from ci-test-cherry-pick-using-GHDL-cf-file to pre-release July 6, 2026 17:34

ruck314 force-pushed the ci-RogueTcpMemoryWrap-RogueTcpStreamWrap branch 3 times, most recently from 2411d68 to 1fa1284 Compare July 6, 2026 19:42

ruck314 added 6 commits July 8, 2026 11:44

chore: update ruckus submodule lock to 4.29.0

a1e7ee7

ruck314 force-pushed the ci-RogueTcpMemoryWrap-RogueTcpStreamWrap branch from 2c97b87 to a1e7ee7 Compare July 8, 2026 18:45

Merge branch 'pre-release' into ci-RogueTcpMemoryWrap-RogueTcpStreamWrap

2774551

bengineerd reviewed Jul 15, 2026

View reviewed changes

Comment thread axi/simlink/shared/RogueTcpStreamCore.h

Comment thread axi/simlink/shared/RogueTcpMemoryCore.h

Comment thread axi/simlink/ghdl/Makefile

Comment thread tests/axi/simlink/test_RogueTcpMemoryWrap.py

ruck314 added 4 commits July 15, 2026 13:48

ruck314 requested a review from bengineerd July 15, 2026 20:51

bengineerd added 5 commits July 15, 2026 14:45

fix(simlink): handle zero-size and sparse boundary frames

c930783

docs: add VHPIDIRECT merge-readiness plan

8cd2ba8

fix(simlink): add per-instance VHPIDIRECT state

a65bbc4

Add better testing.

989731d

Update plan readme.

8d845e6

bengineerd mentioned this pull request Jul 16, 2026

Add Vivado xsim DPI-C co-simulation backend for RogueTcpStream, RogueTcpMemory, and RogueSideBand #1452

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

RogueTcp Stream/Memory/SideBand: GHDL VHPIDIRECT cosim + cocotb CI coverage#1450

RogueTcp Stream/Memory/SideBand: GHDL VHPIDIRECT cosim + cocotb CI coverage#1450
ruck314 wants to merge 16 commits into
pre-releasefrom
ci-RogueTcpMemoryWrap-RogueTcpStreamWrap

ruck314 commented Jul 5, 2026 •

edited by bengineerd

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

bengineerd left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

ruck314 commented Jul 5, 2026 • edited by bengineerd Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Details

Related

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

bengineerd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ruck314 commented Jul 5, 2026 •

edited by bengineerd

Loading