Add opt-in MI_SINGLE_THREADED specialization (idiomatic + safe) with analysis

[angelica-moreira]

Summary

Opt-in, off-by-default MI_SINGLE_THREADED switch that removes the per-free
thread-ownership check (_mi_prim_thread_id() == segment->thread_id) from the
free fast path. This is the idiomatic + safety-hardened version of the idea in
microsoft#1322, plus a full reproducible analysis under
bench/single-threaded/.

What's in here

Code (src/free.c, include/mimalloc/types.h, CMakeLists.txt)

• Documented default in types.h (#if !defined(MI_SINGLE_THREADED) #define MI_SINGLE_THREADED 0)
  and a CMake option -DMI_SINGLE_THREADED=ON, matching the existing MI_* switches.
• Used as #if MI_SINGLE_THREADED (not an inline #if defined(...) && (...)).
• Debug safety net: in debug builds the ownership invariant is kept as
  mi_assert_internal, so multithreaded misuse aborts immediately instead of
  silently corrupting the heap. Zero cost in release. The default build is
  byte-identical to baseline.

Analysis (bench/single-threaded/) — scripts, microbenchmarks, reference
data and REPORT.md.

Results (AMD EPYC 7742, pinned core, interleaved, perf counters)

The eliminated branch is perfectly predicted, so this only helps when the
allocator fast path is actually on the critical path. On allocator-bound
microbenchmarks (bench/single-threaded/microbench/churn.cpp), cycles, n=21:

workload	Δ cycles	Mann-Whitney p	95% CI
pure LIFO churn	-4.61%	2.4e-3	excludes 0
windowed, 4K live	-4.15%	4.4e-6	excludes 0
windowed, 64K live	-3.45%	2.9e-8	excludes 0

On workloads where the allocator is not the bottleneck (e.g. alloc-test,
where 78% of branch-misses are in the benchmark's own RNG) the effect is ~0;
instruction count is not a proxy for speed on an out-of-order core. Full
discussion, codegen verification, and the rejected ExGen-style single-list
experiment (PR #2 / single-threaded-sota branch) are in
bench/single-threaded/REPORT.md.

Reproduce

bench/single-threaded/scripts/build_variants.sh /tmp/mi-st
BASE=$(ls /tmp/mi-st/baseline/libmimalloc.so.*.* | grep -E '\.[0-9]+$' | head -1)
SINGLE=$(ls /tmp/mi-st/single/libmimalloc.so.*.* | grep -E '\.[0-9]+$' | head -1)
CORE=4 REPEATS=21 OUT=micro.csv \
  bench/single-threaded/scripts/run_micro.sh "$BASE" baseline "$SINGLE" single
bench/single-threaded/scripts/analyze_micro.py micro.csv --baseline baseline

[Copilot]

Pull request overview

Adds an opt-in MI_SINGLE_THREADED build switch to specialize mimalloc’s free fast path for strictly single-threaded programs, plus a dedicated benchmark/analysis bundle under bench/single-threaded/ to reproduce and document the impact.

Changes:

• Introduces MI_SINGLE_THREADED (off by default) and wires it through the public config header and CMake options.
• Special-cases mi_free_ex to skip the per-free segment ownership check when MI_SINGLE_THREADED is enabled.
• Adds reproducible microbench tooling, analysis scripts, and reference result datasets under bench/single-threaded/.

Show a summary per file

File	Description
src/free.c	Adds #if MI_SINGLE_THREADED fast-path specialization for is_local.
include/mimalloc/types.h	Documents and provides a default value for MI_SINGLE_THREADED.
CMakeLists.txt	Adds MI_SINGLE_THREADED CMake option and forwards it as a compile define.
bench/single-threaded/scripts/run_micro.sh	Runs pinned/interleaved perf-based microbench sweeps comparing prebuilt mimalloc variants.
bench/single-threaded/scripts/build_variants.sh	Builds baseline vs MI_SINGLE_THREADED variants into a chosen output directory.
bench/single-threaded/scripts/analyze_micro.py	Computes summary stats, MWU significance, and bootstrap CI from raw CSV samples.
bench/single-threaded/results/real_benchmarks_n51.csv	Reference dataset for real-benchmark sweep runs.
bench/single-threaded/results/pgo_base_on_n21.csv	Reference dataset for PGO “base vs on” runs.
bench/single-threaded/results/microbench_base_pron_sota_n21.csv	Reference dataset for microbench churn comparisons.
bench/single-threaded/results/ipc_sh6bench_n11.csv	Reference dataset for IPC-focused sh6bench runs.
bench/single-threaded/REPORT.md	Full write-up of rationale, methodology, and results (delivered + rejected experiment).
bench/single-threaded/README.md	Top-level reproduction guide and directory layout.
bench/single-threaded/microbench/reuse.c	Small helper program demonstrating reuse behavior.
bench/single-threaded/microbench/churn.cpp	Allocator-bound churn microbenchmark used by the scripts.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 14/14 changed files
• Comments generated: 2
• Review effort level: Low

[angelica-moreira]

Thanks @copilot — cross-checked both comments; both were valid and are now fixed in 4e0a0b6.

1. mi_assert_internal compiled out at MI_DEBUG=1 — confirmed and fixed.
Verified against include/mimalloc/internal.h: mi_assert is active at MI_DEBUG>=1 (L333) while mi_assert_internal requires MI_DEBUG>1 (L338), and CMakeLists.txt maps -DMI_DEBUG=ON to MI_DEBUG=1 (L314). So the ownership safety net was indeed dropped in a plain -DMI_DEBUG=ON build (my earlier check only passed because it used MI_DEBUG_FULL=3). Switched to mi_assert. Re-verified: a -DMI_DEBUG=ON -DMI_SINGLE_THREADED=ON build now aborts on multithreaded misuse (free.c:160, SIGABRT) while the single-threaded api suite still passes 43/43.

2. Bootstrap CI non-deterministic — fixed.
analyze_micro.py now seeds the RNG (random.seed(), default 12345, --seed override). Re-running now yields byte-identical CIs.

[Copilot]

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 4
• Review effort level: Low

[angelica-moreira]

Round 2 — thanks @copilot. Three of the four are fixed in 8b10c88; one is a false positive.

✅ build_variants.sh hard-codes make — switched to cmake --build . -j (generator-agnostic, works with Ninja).

✅ REPORT.md still said mi_assert_internal (both the snippet and the narrative) — good catch, that was stale after the round-1 change. Both now read mi_assert and the narrative notes it is active from MI_DEBUG>=1.

❌ run_micro.sh:36 — 2>&1 >/dev/null leaves the CSV on the terminal → empty fields. This one is a false positive. Because the command is piped, stdout already points at the awk pipe before the redirections run, so 2>&1 points perf's stderr (its -x, CSV) at the pipe, and >/dev/null then sends only the benchmark's stdout to /dev/null. awk does receive the CSV. Verified directly:

$ perf stat -x, -e cycles,instructions ./churn 1 2000000 2>&1 >/dev/null \
    | awk -F, '$3=="cycles"{c=$1}$3=="instructions"{i=$1}END{print "cycles="c" instructions="i}'
cycles=83239137 instructions=235853227

and every results/*.csv in the PR has non-empty cycles,instructions columns produced by exactly this line. Left as-is.

[Copilot]

Review details

• Files reviewed: 14/14 changed files
• Comments generated: 0 new
• Review effort level: Low