feat: support inline package definitions in package dependency tables#6523
Draft
Hofer-Julian wants to merge 17 commits into
Draft
feat: support inline package definitions in package dependency tables#6523Hofer-Julian wants to merge 17 commits into
Hofer-Julian wants to merge 17 commits into
Conversation
Hofer-Julian
changed the title
Parse `package = {...}` inline definitions in
`[package.run-dependencies]`, `[package.host-dependencies]`,
`[package.build-dependencies]` and `[package.extra-dependencies.*]`
(including their `if(...)` conditional sub-tables), mirroring the
workspace-level dependency tables:
- `InheritablePackageMap` peels the inline definition off each entry
before spec parsing. The surrounding spec must be a `git`, `path` or
`url` source, and combining a definition with `workspace = true` is
rejected pointing at `[workspace.dependencies]`.
- `PackageTarget` gains a name-keyed `inline_packages` map, drained
from all dependency tables of the target (one definition per name),
converted through the same `TomlPackage::into_manifest` path as
workspace-level definitions and fingerprinted the same way (recipe
now shared via `InlinePackageManifest::from_named_manifest`).
`run-constraints` rejects inline definitions; constraints are
binary-only.
- The stable hash of `PackageTarget` folds in `(name, content hash)`
pairs; empty maps are skipped so existing hashes are unchanged.
- Inline definitions now nest: a definition's own dependency tables are
parsed by the same code, so a manifest-less source dependency of
another inline definition can itself be defined inline.
Definitions parsed here are not yet honored by the solve; that plumbing
follows in later commits.
…] pool
Pool entries may attach an inline package definition to a source spec.
A `{ workspace = true }` package dependency inherits the definition
together with the spec — a manifest-less source location without its
definition would fail backend discovery at build time.
- `WorkspaceDependencyMap` peels `package = {...}` off each entry and
requires a `git`, `path` or `url` source location next to it.
- Definitions convert once at workspace assembly (inheriting the
workspace's external package properties, like the workspace-level
dependency-table definitions do) and are stored on
`Workspace::dependency_inline_packages` /
`WorkspacePackageProperties::dependency_inline_packages`.
- `into_package_target` adopts the pool definition for inherited
entries. The same pool entry inherited in several tables is fine
(identical content); clashing with a direct definition of the same
name errors as a duplicate.
- Backend inheritance from a definition-carrying pool entry keeps
erroring via the existing 'build backends cannot be defined as source
dependencies' rejection (such entries are always source specs).
Inline package definitions declared in a package's dependency tables now take effect for every source package pixi resolves, uniformly for run-, host-, build- and extra-dependencies: - `PackageManifest::combined_inline_packages` merges the definitions across the default and conditional targets. Which conditional target applies is decided by the backend, so matching is by name alone; parsing rejects a name carrying different definitions in different targets. - `DiscoveredBackend` carries the merged map (plus the workspace manifest that gives definitions their channel context) off backend discovery. The map is not serialized: it feeds only the in-process pipeline, never the metadata disk cache. Discovery runs on cache hits too, so `BuildBackendMetadata` and `SourceOutputs` expose the map on both paths. - `assemble_source_record` passes the package's map into its nested build/host environment solves; the existing seed-attachment logic matches definitions by name against the backend-reported deps. - The transitive walk in `walk_and_resolve` attaches the parent package's definition when pushing a child source dependency (run deps and extras flow through the same loop). The seen-set now records the inline content hash and the pushing parent: a direct dependency of the environment keeps overriding package-level definitions (seed-first, as before), while two package-level parents disagreeing about the same (package, source location) fail the solve with a `ConflictingInlineDefinitions` error naming both parents. - `ResolveSourcePackageKey` now returns `ResolvedSourcePackage` (records + the package's inline map) so the walk can do the attachment; cache identity of children is unchanged since the child key already hashes its inline definition.
The solve resolves records with package-level inline definitions; the build/install pipeline and lock-file verification need the same definitions to reconstruct backends without an on-disk manifest: - `SourceBuildKey` discovers its record's backend once (in-memory cached, no backend spawn) and uses the resulting map to (a) attach definitions to nested source-dependency builds (previously hardcoded `inline: None`) and (b) pass them into the build/host prefix installs, which recurse through the same env-install path. - The env install (`install_pixi_environment`) extends the caller-supplied (consumer-level, higher-precedence) definitions with package-level ones collected from each source record's manifest. Inline-defined records have no on-disk manifest, so collection iterates until no new definitions turn up, resolving one link of an A→B→C definition chain per round. - `verify_partial_source_record_against_backend` falls back to the workspace's own `[package]` definitions when the environment-level lookup misses. Definitions declared by transitive packages' manifests are not found there; the fresh metadata query then fails discovery and forces a re-lock, which resolves them through the regular walk.
…file checks in inline definitions
Hofer-Julian
force-pushed
the
claude/inline-manifest-dependencies-e4z9w0
branch
from
1a5f954 to
3b689b2
Compare
…function-scope imports
Hofer-Julian
force-pushed
the
claude/inline-manifest-dependencies-e4z9w0
branch
from
7915c60 to
b305473
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR extends inline package definition support to package dependency tables (
[package.run-dependencies],[package.host-dependencies],[package.build-dependencies], and[package.extra-dependencies.*], including theirif(...)conditional sub-tables), and adds support for inheriting inline definitions through the[workspace.dependencies]pool via{ workspace = true }markers.Key changes:
Package dependency table support: Inline
package = { ... }definitions are now accepted in a package's own dependency tables, allowing manifest-less source dependencies to be described inline at the point of consumption. This works in any package manifest pixi builds — the workspace's own[package]as well as the manifests ofpath/git/urlsource dependencies — and definitions now nest, so a chain of manifest-less repositories (A→B→C) can be described from one place.Workspace pool inheritance: Entries in
[workspace.dependencies]can now carry inline definitions. When a package dependency uses{ workspace = true }, it inherits both the source spec and the inline definition together from the pool. Combiningworkspace = truewith a use-sitepackage = {...}table is rejected, pointing at the pool.Solve integration: Backend discovery collects each package's inline definitions (never serialized into the metadata disk cache; discovery runs on cache hits too). They feed the nested build/host environment solves and are attached to transitive source dependencies during the environment walk, matched by name against the backend-reported dependencies.
Conflict rules: A direct dependency of the environment overrides package-level definitions (the existing seed-first behavior, now documented). Two packages disagreeing about the definition for the same
(package, source location)fail the solve with aConflictingInlineDefinitionserror naming both parents. Inline definitions stay rejected in[constraints]and[package.run-constraints](binary-only), and a name may carry at most one definition per target.Build & verify integration:
SourceBuildKeyand the environment install re-derive the definition maps via cached discovery so inline-defined packages build at install time, and the mutable-source satisfiability path falls back to the workspace's own[package]definitions (transitive declarers force a re-lock, resolved through the regular walk).Documentation:
docs/build/inline_packages.mdnow documents where inline definitions are accepted, pool inheritance, nesting, and the conflict rules. The JSON schema already admittedpackageon these tables (InheritableMatchspecTableinherits it fromMatchspecTable, and the pool uses the same spec type), so noschema/model.pychange is needed — the parser now matches the schema.Test data: Added
tests/data/pixi-build/inline_package_run_dep/, wherepackage_b's on-disk manifest names a backend that cannot exist and only the inline definition inpackage_a's[package.run-dependencies]can make the build succeed — a discriminating end-to-end case.How Has This Been Tested?
crates/pixi_manifest(parse-layer validation, pool inheritance, duplicate/conflict rejection, nesting, hash stability); the fullpixi_manifestandpixi_command_dispatchersuites pass.tests/integration_python/pixi_build/test_inline_packages.py:run-constraintsrejects it, pool inheritance resolves,{ workspace = true, package = {...} }errors;test_inline_in_package_run_dependencies_builds) mirroringtest_recursive_source_run_dependencies; it needs channel access and runs in CI.pixi run lint-fastjobs verified: cargo fmt, ruff lint/format, taplo, typos, dprint, ty, and cargo-shear all pass.AI Disclosure
Tools: Claude Code
Checklist:
schema/model.py.https://claude.ai/code/session_019UVuWiTRdhz5PokX91rcU4