Reduce Basilisk cold-start overhead by consolidating native bindings by schaubh · Pull Request #1597 · AVSLab/basilisk

schaubh · 2026-10-04T18:36:43Z

Review: By commit
Merge strategy: Merge (no squash)

Description

Reduce startup overhead by consolidating native extensions into separate libraries for messaging, core FSW, core simulation, MuJoCo, and OpenCV. Existing Python import paths, constructors, messages, and recorders remain compatible. Bindings initialize on demand, and individual source files and SWIG wrappers still compile separately.

Grouped bindings are the standard packaging layout. Optional dependencies remain optional, and core simulation does not acquire MuJoCo or OpenCV dependencies.

Prototype measurements on macOS arm64, using Release builds and Python 3.14.6, showed these cold-import improvements:

Binding group	Separate libraries	Grouped library	Speedup
Core FSW	12.544 s	0.770 s	16.3×
Core simulation	12.432 s	1.212 s	10.3×
MuJoCo	2.004 s	0.603 s	3.3×
OpenCV	0.557 s	0.241 s	2.3×

These are medians of three trials, measuring group imports after Basilisk initialization. “Cold” means the first process after rebuilding or relinking the native files; no reboot or disk-cache flush was performed.

One 16-worker pytest collection comparison improved from 27.78 s to 21.11 s, approximately 24%, when only the simulation binding layout changed. This was a single comparison, and timings depend on the platform and cache state.

FSW native binaries also decreased from 32.65 MiB to 16.63 MiB. Binding-group rebuild times remained similar overall; simulation’s median increased approximately 4.2%, with overlapping trial ranges. Individual edits still rebuild one object and relink its group. Sparse imports can incur additional loading cost from the larger library.

The 18 commits separate packaging, benchmarks, and test organization from individual fixes. Independent fixes cover module defects, invalid configuration handling, build-generator compatibility, and test reliability. The robotic grappling example also records plotting data at 10 Hz.

Verification

Development CI passed on macOS, Linux, and Windows, including separate optional-wheel validation.
Additional checks passed for Visual Studio Release and Debug, Ninja Multi-Config, Unix Makefiles, and Visual Studio 2022. Local checks also exercised Xcode.
The latest local routine suite passed 8,614 tests, with two prerequisite-related skips. The separate build-integration suite passed 19 tests.
Regression coverage includes import compatibility, recorders, concurrent imports, binding-group transitions, SWIG directors, optional wheels, and incremental builds.
History cleanup passed 158 focused checks across selected commit boundaries. Final file contents exactly match the version before cleanup, and repository hooks passed.

Temporary diagnostic workflows were removed. Permanent build-integration tests run separately in CI to keep routine test runs faster. The linked CI runs precede the history rewrite; normal PR checks should complete on the submitted commits before merging.

Documentation

Updated developer benchmark instructions, build-integration test guidance, release notes, and affected module documentation. Archived benchmark reports and raw measurements document the prototype comparisons and their limitations.

Known-issue entries describe pre-existing defects addressed by this branch. Focused Sphinx validation passed with warnings treated as errors.

Future work

Optionally extend cold-start measurements to Linux and Windows and profile the remaining pytest collection overhead.

chatgpt-codex-connector · 2026-10-04T18:36:57Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-04T18:42:41.458553Z	`7e2a243`	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Register the Basilisk DLL directory in the isolated retry subprocess without importing Basilisk eagerly. Retain the directory handle across failed imports and retries, and surface early import errors instead of waiting for a misleading initialization timeout. Validation: 29 related tests passed; all six retry cases passed with simulated Windows DLL-directory preconditions, and a negative control reported the underlying ImportError immediately.

Use a 0.1-second recorder interval for the spacecraft and arm outputs. Keep the simulation task rate unchanged while reducing plot data volume.

Report invalid parent types before computing forces and torques instead of continuing with uninitialized parent state. Document the validation behavior.

Use the state-sized scratch vector for covariance downdates. The previous measurement-sized buffer could overflow when copying a full state column. Record the pre-existing defect in the release notes and known issues.

Scale standard-normal samples by the configured noise levels so zero noise does not violate the distribution precondition. Preserve random-generator progression and cover noiseless output and repeatability in regression tests.

Apply a supported limb-message planet ID before selecting the radius, retain the configured target for a zero ID, and reject unsupported IDs. Document planet selection and add regression coverage for valid and invalid targets.

Initialize scenarioSmallBodyNav with the filter's 12-state vector. Reject incorrect state, covariance, and noise matrix dimensions during reset and update, with Python regression tests and module documentation.

Stop marking configure_file outputs as generated build outputs. This avoids invalid Ninja Multi-Config dependencies and prevents native clean targets from deleting C message interfaces. Test build, clean, and rebuild behavior.

Use a relative logical solution folder instead of an absolute filesystem path. Extend the C-message build fixture to reject invalid IDE folder names.

…builds Query CMake targets through the File API and normalize fixture paths for Windows. Build benchmark smoke targets in the active configuration, capture native warnings through OS pipes, and skip allocation-failure sweeps that terminate inside noexcept MSVC Debug STL constructors. Retain generator and warning regression coverage.

Allow cold Python and native-library startup before beginning the bounded thread-lifecycle check. Preserve hang detection with readiness signaling, traceback diagnostics, captured output, and cleanup of timed-out children.

Link separately compiled message wrappers into one native library to reduce first-import overhead. Preserve existing imports, recorder behavior, and message build targets through an on-demand loader. Isolate wrapper helper symbols and verify compatibility, initializer regeneration, and single-message incremental rebuilds.

Generate private import shims and load each extension under Python's import locking so concurrent imports cannot expose partially initialized bindings. Track shim outputs in CMake and add concurrent-import regression coverage.

Build core FSW wrapper objects into one native library and retain lazy imports, public module identity, and convenience build targets. Isolate colliding C symbols and leave custom dependency bindings separate. Make this the standard layout and test imports, callbacks, cleanup, and incremental compilation.

Group core simulation, MuJoCo, and OpenCV bindings into separate native libraries while preserving optional features and Python imports. Keep Vizard and unrecognized custom dependencies separate, share the loader, and prepare SWIG directors for combined linkage. Cover all groups, stale-layout cleanup, and optional wheel selection with regression tests.

Accept both SWIG director-mutex definition forms without altering director RTTI or callback behavior. Check the generated syntax and compile real SWIG wrappers across threading and Python API configurations.

Write a shared inventory of active bindings so cleanup from the previous group cannot delete the destination group's shim. Cover moves in both orders, parallel cleanup, retirement, and native CMake reconfiguration. Account for whole-second Make timestamps in the reconfiguration fixture.

Measure import and collection latency, no-change builds, incremental builds, and native library sizes. Keep reproducible benchmark tooling and explicitly label archived comparisons as prototype measurements. Document current usage and test benchmark command construction and reporting.

Share native fixtures across binding groups and avoid redundant compiler discovery and source inventory scans. Keep full SWIG API and threading coverage on the primary generator and representative checks on others. Mark native build integration tests separately and run them in dedicated platform CI steps so routine pytestCI runs retain fast runtime regressions.