Skip to content

WIP Unify the build pipelines on one orchestration core#10007

Open
Anton-4 wants to merge 9 commits into
mainfrom
unify-build-pipelines
Open

WIP Unify the build pipelines on one orchestration core#10007
Anton-4 wants to merge 9 commits into
mainfrom
unify-build-pipelines

Conversation

@Anton-4

@Anton-4 Anton-4 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Every entry path — check, build, run, test, docs, glue, bundle — now drives the same orchestration core (BuildEnv), which owns exactly four things: package registration and graph construction, cache wiring, report accumulation and rendering, and the platform relation. Entry paths differ only in their back half.

Core (compile_build.zig):

  • BuildEnv gains allow_user_errors: with executable_artifacts publication, user errors no longer block finalization when every erroring module still permits lowering (the run family's contract).
  • compileDiscovered records executable_lowering_ready before result transfer drains the Coordinator's reports, so back halves consult one recorded outcome instead of re-querying drained state.
  • Hard compilation errors now still move accumulated reports into the ordered sink so callers can drain and render them.
  • renderDiagnosticsWithConfig is the single drain-and-render path; draining moves reports out of the sink, so rendering the same report twice is structurally impossible (PR 9759's bug class).

Run path (cli/main.zig):

  • lowerLirWithCoordinator's hand-rolled front half — app-header parse, resolvePackages, ensurePackage/ensurePackageWithUrl loops, shorthand wiring, version-note plumbing, parse-task enqueueing — is deleted; lowerLirWithBuildEnv is now only the LIR-image back half on the core. grep -rn ensurePackage src/cli src/glue matches nothing.
  • renderCoordinatorReports and its version-note fetchRemove dance are gone; Coordinator.version_notes is deleted (BuildEnv's is the one mechanism).
  • Dead setupSharedMemoryWithCoordinator (the remaining literal-null cache manager surface) is deleted; buildLirImageWithCoordinator is renamed buildLirImage and takes no_cache like every other path.
  • The hot-reload dev worker now wires a real cache manager, with --no-cache propagated on the worker command line (the last PR 9811-class divergence).

One place for each policy:

  • initCliBuildEnv is the only place a cache manager is created and attached: every pipeline caches unless the user passed an explicit --no-cache. Glue gains cache wiring plus a --no-cache flag; bundle gains caching through the shared constructor; roc test passes build_env.cache_manager straight through instead of re-deriving it.
  • exitWithWarningCode is the only warning exit-code logic; check, build, run (shim, default-app, and build-and-exec), and test all use it. roc test now exits 2 on warnings like the other pipelines.
  • renderBuildEnvReports is the single terminal rendering path.

Tests:

  • In-process pipeline parity: the same multi-package project (app ->
    package -> transitive package) driven through the check front half and the run front half must agree on diagnostics counts, and the run path lowers exactly when check reports no errors.
  • Structural single-render test: a drained warning cannot be re-rendered or re-counted.
  • Non-app roots are rejected by the run front half with the expected CLI problem.
  • Core-level issue 9509 repro: app -> local package -> transitive URL package compiles from a warm package cache with no network.
  • CLI suite: warning exit-code parity across check/build/run/test on one fixture; issue 9788 repro (second roc run reuses the checked-module cache); issue 9509 repro (transitive package dependency checks, builds, runs, and tests identically); roc test warning exit 2.

zig build minici: all jobs pass except run-coverage-parser, whose kcov tracer hangs on this machine identically on the unmodified base branch (environmental; that job compiles only from src/parse, which this change does not touch).

rtfeldman and others added 9 commits July 1, 2026 19:48
Mark the app package in BuildEnv's compileDiscovered and require the mark
whenever the hosted transform is enabled, so app modules can never have
annotation-only defs silently rewritten into hosted lambdas (and check/run
can no longer populate the shared cache with divergent artifacts for one
key); appRootCheckedArtifact likewise requires the mark instead of guessing
the synthetic identity. Set the synthetic root and platform identities on
the default-app check path so staged temp-dir roots key by "app"/"pf"
like the run path, restoring cache hits across runs and identical
identities across pipelines.

Identity derivation for local paths now falls back to the lexically
normalized path when the file has no on-disk resolution, so missing
package paths keep producing parse-time file-not-found diagnostics,
editor-overlay-only files still build, and mock-filesystem contexts work
(the testing CoreCtx canonicalize now normalizes instead of panicking).
Docs output renders display names instead of identity keys: package labels
go through BuildEnv.displayNameForPackage and local type refs render the
display-qualified module path rather than qualified_module_ident.

Also: doc comments on every pub declaration in package_identity.zig,
packageIdentityFor returns a mutable slice so frees need no casts,
buildPackageKeys detects platforms via the resolver's package kind instead
of a bitset over root deps, versionBumpNotes is parameterized by the key
mapping instead of duplicated, and the unused synthetic-identity plumbing
in discoverAppFromPath and the platform registration wrappers is deleted.
The semantic audit gate rejects recovery vocabulary in compiler comments,
so the local-path identity comment now states the rule affirmatively (and
spells out what an unresolvable path is: a file that does not exist on disk
yet, like a typo'd package path or a never-saved editor buffer).

Checking a platform, package, or plain module root panicked because the
hosted transform required a marked app package while BuildEnv enables the
transform for every build. App-less builds now record that state explicitly
via markNoAppPackage, and the transform applies to every module in them;
the debug panic remains only for the genuinely unmarked case.
Every entry path — check, build, run, test, docs, glue, bundle — now
drives the same orchestration core (BuildEnv), which owns exactly four
things: package registration and graph construction, cache wiring,
report accumulation and rendering, and the platform relation. Entry
paths differ only in their back half.

Core (compile_build.zig):
- BuildEnv gains allow_user_errors: with executable_artifacts
  publication, user errors no longer block finalization when every
  erroring module still permits lowering (the run family's contract).
- compileDiscovered records executable_lowering_ready before result
  transfer drains the Coordinator's reports, so back halves consult one
  recorded outcome instead of re-querying drained state.
- Hard compilation errors now still move accumulated reports into the
  ordered sink so callers can drain and render them.
- renderDiagnosticsWithConfig is the single drain-and-render path;
  draining moves reports out of the sink, so rendering the same report
  twice is structurally impossible (PR 9759's bug class).

Run path (cli/main.zig):
- lowerLirWithCoordinator's hand-rolled front half — app-header parse,
  resolvePackages, ensurePackage/ensurePackageWithUrl loops, shorthand
  wiring, version-note plumbing, parse-task enqueueing — is deleted;
  lowerLirWithBuildEnv is now only the LIR-image back half on the core.
  `grep -rn ensurePackage src/cli src/glue` matches nothing.
- renderCoordinatorReports and its version-note fetchRemove dance are
  gone; Coordinator.version_notes is deleted (BuildEnv's is the one
  mechanism).
- Dead setupSharedMemoryWithCoordinator (the remaining literal-null
  cache manager surface) is deleted; buildLirImageWithCoordinator is
  renamed buildLirImage and takes no_cache like every other path.
- The hot-reload dev worker now wires a real cache manager, with
  --no-cache propagated on the worker command line (the last PR
  9811-class divergence).

One place for each policy:
- initCliBuildEnv is the only place a cache manager is created and
  attached: every pipeline caches unless the user passed an explicit
  --no-cache. Glue gains cache wiring plus a --no-cache flag; bundle
  gains caching through the shared constructor; roc test passes
  build_env.cache_manager straight through instead of re-deriving it.
- exitWithWarningCode is the only warning exit-code logic; check, build,
  run (shim, default-app, and build-and-exec), and test all use it.
  roc test now exits 2 on warnings like the other pipelines.
- renderBuildEnvReports is the single terminal rendering path.

Tests:
- In-process pipeline parity: the same multi-package project (app ->
  package -> transitive package) driven through the check front half and
  the run front half must agree on diagnostics counts, and the run path
  lowers exactly when check reports no errors.
- Structural single-render test: a drained warning cannot be re-rendered
  or re-counted.
- Non-app roots are rejected by the run front half with the expected
  CLI problem.
- Core-level issue 9509 repro: app -> local package -> transitive URL
  package compiles from a warm package cache with no network.
- CLI suite: warning exit-code parity across check/build/run/test on one
  fixture; issue 9788 repro (second roc run reuses the checked-module
  cache); issue 9509 repro (transitive package dependency checks,
  builds, runs, and tests identically); roc test warning exit 2.

zig build minici: all jobs pass except run-coverage-parser, whose kcov
tracer hangs on this machine identically on the unmodified base branch
(environmental; that job compiles only from src/parse, which this
change does not touch).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Anton-4 Anton-4 changed the base branch from projects to main July 7, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants