ci: halve the work, cache more, add sanitizer and hardened builds - #60
Conversation
Stop building everything twice ----------------------------- "on: [push, pull_request]" ran the full matrix twice for every branch with an open PR, once per event. Now push only builds main, and everything else goes through pull_request. Added a concurrency group so a new push to a PR cancels the run it just obsoleted; main is never cancelled since that is what the README badge points at. Speed ----- * windows had no compiler cache at all and was the slowest job at 108s. ccache doesn't support MSVC, so use sccache there. * macos spent most of its 74s in "brew install gcc", and nothing used gcc: the build runs with CXX=c++, which is AppleClang. Dropped it, along with brew's meson/ninja in favour of pip like the other jobs. ccache is now installed by the ccache action rather than by brew. * Cache subprojects/packagecache keyed on the wrap files, so the fmt, doctest and abseil tarballs aren't re-downloaded every run. * Give each job its own ccache key so they stop evicting each other. * meson test -v printed the full output of every passing test. Use --print-errorlogs instead, which only prints logs for failures. New jobs -------- * sanitizers: address + undefined, built with clang++ at --buildtype=debugoptimized, plus detect_stack_use_after_return and halt_on_error. clang rather than g++ because __has_feature is what the bad_alloc test uses to skip its deliberate huge allocation. * hardened: the flags a distribution build would use. _FORTIFY_SOURCE=3, _GLIBCXX_ASSERTIONS, -fstack-protector-strong, -fstack-clash-protection, -fcf-protection=full, format-security, and full RELRO + BIND_NOW + noexecstack at link time. _GLIBCXX_ASSERTIONS is the interesting one for a container, it turns on libstdc++'s precondition checks. These only apply to the CI builds. svector is header only, so the default build keeps its existing flags and users pick their own. Also make the sanitizer detection in bad_alloc.cpp work for g++, which only gained __has_feature in version 14. Without the __SANITIZE_ADDRESS__ fallback a g++ sanitizer build aborts in asan instead of catching std::bad_alloc. Verified locally with the exact commands the workflow runs: both new configurations pass (Ok: 2, Fail: 0), and the hardened binary really does come out with full RELRO, BIND_NOW, a non-executable stack and stack protector symbols. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The wrap was dead weight. A wrap is only consulted as a fallback, and meson does not fall back for an optional dependency, so every CI run resolved it as: Run-time dependency absl_container found: NO (tried pkg-config and cmake) and no subproject was ever built. The abseil sources only got compiled under --wrap-mode=forcefallback, which nothing in CI uses. So this does not speed up CI, contrary to what I claimed earlier. What it does do is remove a stale 2023 pin and a dependency lookup that can never succeed, and it takes forcefallback builds from 160 abseil objects down to 0 (3.22s -> 1.82s locally on a full clean build). All abseil use in the tests is behind ANKERL_SVECTOR_HAS_ABSL(), which is driven by __has_include, so the benchmarks and show_comparison still build unchanged. Installing abseil system wide brings the comparison back; boost already works exactly this way. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Correction: abseil was never being built in CII claimed earlier that the abseil subproject was the largest remaining compile cost. That was wrong, and I want to be explicit about it since it was the stated reason for this change. A wrap is only consulted as a fallback, and meson does not fall back for an optional dependency. Every CI run has actually been resolving it as: with no subproject built. The abseil sources only ever compiled under So this commit does not speed up CI. What it does:
It removes a stale 2023 pin and a dependency lookup that can never succeed. Real, but cleanup rather than a speedup. All abseil use in the tests is behind |
Per-step timings showed the remaining cost was not compilation at all: linux apt-get install libboost-dev 6s .. 59s windows pip install meson ninja 12s .. 32s boost is only there for the benchmark comparisons and show_comparison, which meson test never runs, and it is guarded by __has_include exactly like abseil was. So stop installing it: no apt on linux, no brew on macos, and no dependency() lookup. The comparison comes back on its own for anyone who has the headers installed system wide, verified locally. pip now installs from a pinned requirements.txt with setup-python's cache enabled. Pinning keeps the cache key stable and saves pip from resolving against PyPI on every run. The lint job stays uncached, it installs nothing. macos no longer runs brew at all, since ccache comes from the ccache action. Verified both directions: with the __has_include guards forced to 0 to emulate a runner without the headers, everything still builds and passes including the bench suite; with boost present, show_sizeof still prints the boost::container::small_vector row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
I pinned ninja==1.13.2 by reading my distro's ninja version instead of what PyPI actually ships. The PyPI ninja package tops out at 1.13.0, so every job that installs from requirements.txt failed with: ERROR: No matching distribution found for ninja==1.13.2 Verified both pins now install into a clean venv. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stop building everything twice
on: [push, pull_request]ran the full matrix twice for every branch with an open PR — once for the push, once for the pull_request event. You can see it in the recent runs: each of #57/#58/#59 had two of every job.Now
pushonly buildsmain, everything else goes throughpull_request. Added aconcurrencygroup so a new push to a PR cancels the run it just obsoleted —mainis never cancelled, since that's what the README badge points at.Speed
Measured from the last
mainrun: windows 108s, macos 74s, linux 47s, lint 9s.brew install gccon macosCXX=c++, which is AppleClang. This was the single most expensive step in that job.subprojects/packagecache--print-errorlogsinstead of-v-vprinted the full output of every passing test. Failures still print inline, and the log artifact is unchanged.New:
sanitizersjobAddress + undefined behaviour,
clang++at--buildtype=debugoptimized, withdetect_stack_use_after_return=1andhalt_on_error=1.clang rather than g++ deliberately:
unit/bad_alloc.cppdeliberately allocatesmax_size()to check it throwsstd::bad_alloc, and has to be skipped under asan. It detects this via__has_feature, which g++ only gained in version 14.I also fixed that detection to fall back to
__SANITIZE_ADDRESS__/__SANITIZE_THREAD__, so a g++ sanitizer build works too — without it, asan aborts on the huge allocation instead of letting it throw.New:
hardenedjobThe flags a distribution build would use:
_GLIBCXX_ASSERTIONSis the one that actually matters for a container — it turns on libstdc++'s precondition checks (bounds, iterator validity).These apply only to the CI builds. svector is header-only, so
meson.buildkeeps its existing flags and users pick their own.Verification
Ran the exact commands the workflow produces, verbatim:
61 test cases passed, 617011 assertions,Ok: 2, Fail: 0Ok: 2, Fail: 0And confirmed the hardening isn't silently dropped — checked
compile_commands.jsonfor every flag, and the resulting binary:One thing I could not test locally
CXX: sccache clon windows. I have no MSVC here. sccache + MSVC historically needs/Z7rather than/Zifor debug info, and meson's default buildtype uses/Zi— so this may need--buildtype=releaseor a/Z7override. If the windows job goes red, that's the reason, and it's a one-line follow-up. Everything else is verified.🤖 Generated with Claude Code