Skip to content

feat: dependency-driven build scheduler (no per-region barrier)#1297

Open
krystophny wants to merge 1 commit into
fortran-lang:mainfrom
krystophny:feat/dependency-scheduler
Open

feat: dependency-driven build scheduler (no per-region barrier)#1297
krystophny wants to merge 1 commit into
fortran-lang:mainfrom
krystophny:feat/dependency-scheduler

Conversation

@krystophny

@krystophny krystophny commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Replace the per-region OpenMP barrier in build_package with a
    dependency-driven ready-queue scheduler
  • Each target builds as soon as its own dependencies are done, with no barrier
  • Workers pop ready targets, build them, and release any dependent whose
    remaining dependency count reaches zero

Context

Today the backend groups targets into schedule regions by dependency depth and
builds each region with !$omp parallel do, so there is an implicit barrier
between regions: region i+1 cannot start until every target in region i
finishes. One slow target (a straggler) stalls the whole next region.

Change

  • build_dependency_graph computes, over the queued targets, each target's
    unmet-dependency count and a compressed-sparse-row reverse adjacency
    (dependents of each target)
  • The build loop runs one !$omp parallel region: pop a ready target under
    critical(fpm_sched), build it, then release its dependents and bump the
    completion count
  • Acyclicity is already guaranteed upstream by sort_target; each target is
    pushed and built exactly once; on failure the queue drains and still releases
    dependents, so the loop always terminates

Merge order

Independent PRs (any order):

Stacked chain (merge in order):

  1. Fortran compiles via argv (feat: exec Fortran compiles via argv (posix_spawn/CreateProcess) #1299)
  2. Parallel link/archive via argv (feat: run link and archive in parallel via argv spawn #1300)
  3. C/C++ compiles via argv (feat: exec C/C++ compiles via argv, close the shell-fork gap #1301)

Fork-safety fix:

After the above:

Verification

$ fpm test --flag -fopenmp
TALLY;TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT
PASSED: all 35 tests passed
  • A generated 200-module project builds successfully and deterministically (3/3
    runs), including with OMP_NUM_THREADS=1
  • A 24-module dependency chain (width-1 graph, worst case for the scheduler's
    busy-wait) builds correctly
  • Not slower than the barrier: bigmod clean build 3.25s vs 3.29s; chain 0.42s
    vs 0.42s (best of 5, 3 reps)

The intended win is on unbalanced graphs where a straggler stalls a region
boundary; the balanced bigmod and the pure chain only show parity (as expected).

Replace the per-region OpenMP barrier loop in build_package with a
ready-queue scheduler: a target becomes ready as soon as its own queued
dependencies are built, with no barrier between schedule regions. Workers
pop ready targets, build them, then release dependents whose remaining
dependency count reaches zero.

build_dependency_graph computes per-target unmet-dependency counts and a
CSR reverse-adjacency over the queued targets. Acyclicity is guaranteed
upstream by sort_target; each target builds exactly once; on failure the
queue still drains (skipped) and releases dependents, so the loop always
terminates.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant