feat: dependency-driven build scheduler (no per-region barrier)#1297
Open
krystophny wants to merge 1 commit into
Open
feat: dependency-driven build scheduler (no per-region barrier)#1297krystophny wants to merge 1 commit into
krystophny wants to merge 1 commit into
Conversation
Replace the per-region OpenMP barrier loop in build_package with a ready-queue scheduler: a target becomes ready as soon as its own queued dependencies are built, with no barrier between schedule regions. Workers pop ready targets, build them, then release dependents whose remaining dependency count reaches zero. build_dependency_graph computes per-target unmet-dependency counts and a CSR reverse-adjacency over the queued targets. Acyclicity is guaranteed upstream by sort_target; each target builds exactly once; on failure the queue still drains (skipped) and releases dependents, so the loop always terminates.
This was referenced Jun 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
build_packagewith adependency-driven ready-queue scheduler
remaining dependency count reaches zero
Context
Today the backend groups targets into schedule regions by dependency depth and
builds each region with
!$omp parallel do, so there is an implicit barrierbetween regions: region
i+1cannot start until every target in regionifinishes. One slow target (a straggler) stalls the whole next region.
Change
build_dependency_graphcomputes, over the queued targets, each target'sunmet-dependency count and a compressed-sparse-row reverse adjacency
(dependents of each target)
!$omp parallelregion: pop a ready target undercritical(fpm_sched), build it, then release its dependents and bump thecompletion count
sort_target; each target ispushed and built exactly once; on failure the queue drains and still releases
dependents, so the loop always terminates
Merge order
Independent PRs (any order):
Stacked chain (merge in order):
Fork-safety fix:
After the above:
Verification
runs), including with
OMP_NUM_THREADS=1busy-wait) builds correctly
vs 0.42s (best of 5, 3 reps)
The intended win is on unbalanced graphs where a straggler stalls a region
boundary; the balanced bigmod and the pure chain only show parity (as expected).