[2/2] feat: embed component WIT in the compiled Miden package

CHANGELOG.md

@@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### `cargo-miden`
+
+- Added `cargo miden package-cache`, which prints the project's fingerprinted package-cache
+  directory, the number of direct dependencies resolved from that cache, and the input paths a
+  contract build script must watch #1298
+- Contract templates and the repository examples now include a `build.rs` that populates the
+  package cache for builds `midenc` does not drive: outside a midenc-driven build it locates
+  the cache with `cargo miden package-cache`, fills it with a nested
+  `cargo miden build --release` when the project has source dependencies, and exports
+  `MIDENC_PACKAGE_CACHE` to macro expansion. Plain `cargo check` and IDE analysis now resolve
+  dependency packages instead of reporting missing packages (#1215). The script uses
+  `cargo miden` from `PATH`, or the binary named by the `CARGO_MIDEN` environment variable
+  #1298
+- Fixed the contract templates' `miden-project.toml` manifests, which were missing the
+  `[lib].path` key the VM v0.25 project model requires; projects generated from the templates
+  failed both `cargo miden build` and macro expansion with "unable to parse project manifest:
+  missing field `path`"
+
 ### Rust SDK
 
 - The FPI macro diagnostic for a dependency package missing from a midenc-driven build now names
@@ -18,6 +36,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - FPI expansions record `option_env!("MIDENC_PACKAGE_CACHE")`, so a consumer crate recompiles —
   and re-reads its dependency procedure roots — whenever the compiler's fingerprinted package
   cache path rotates, even if a stale cache directory survives on disk #1302
+- BREAKING: The component WIT generated by `#[component]` is now embedded in the compiled Miden
+  package (a `wit` section of the `.masp`) instead of being written to `target/generated-wit/`,
+  and the SDK macros read dependency WIT from the dependency's compiled package. The
+  `wit = "..."` keys in `miden-project.toml` are now only a fallback for dependency packages
+  without embedded WIT (e.g. produced by other toolchains): setting the key for a package that
+  embeds WIT is an error, and packages built by older Miden toolchains are rejected unless the
+  key supplies their WIT. See the [migration guide](./sdk/sdk/MIGRATION.md) for the manifest
+  edits and rebuild steps #1248
+- BREAKING: The SDK macros now read dependency packages only from the `MIDENC_PACKAGE_CACHE`
+  directory (or from a manifest path that names a `.masp` file directly). The previous search
+  of `target/miden/<profile>` output directories — the dependency's own, surrounding
+  workspaces', and ambient (`CARGO_TARGET_DIR`, `OUT_DIR`, working-directory) targets — was
+  removed, along with its freshest-first selection and macro-side package id and version
+  checks; the fingerprinted cache is rewritten by every build and its contents are trusted.
+  Builds driven by `cargo miden build` export the variable already; plain `cargo build`,
+  `cargo check`, and IDE analysis need the contract `build.rs`. An expansion without a
+  configured cache now fails with instructions instead of searching the filesystem #1298
 
 ## [0.10.0-rc.1]
 

README.md

@@ -58,11 +58,11 @@ This will run all of the unit tests in the workspace, as well as all of our `lit
   (comma-delimited), where `PATH` is treated either as folder e.g. `MIDENC_EMIT=ir=target/emit` or file `MIDENC_EMIT=hir=my_name.hir`.
 - `MIDENC_EMIT_MACRO_EXPAND[=<dir>]`: When set, integration tests dump `cargo expand`
   output for Rust fixtures to `<fixture>.expanded.rs` files in `<dir>` (or the CWD if empty/`1`).
-- `MIDENC_EMIT_WIT[=<dir>]`: When set, integration tests emit public component WIT as
-  `<fixture>.wit` and resolved macro-generated inline worlds as `<package>.<world>.inline.wit` in
-  `<dir>` (or the CWD if empty/`1`). Resolved FPI worlds include their injected synthetic packages
-  and `fpi-*` functions. Generated SDK integration fixtures enable the internal WIT-printer
-  feature in their Cargo manifests.
+- `MIDENC_EMIT_WIT[=<dir>]`: When set, integration tests emit the public component WIT embedded
+  in each compiled package as `<fixture>.wit` and resolved macro-generated inline worlds as
+  `<package>.<world>.inline.wit` in `<dir>` (or the CWD if empty/`1`). Resolved FPI worlds include
+  their injected synthetic packages and `fpi-*` functions. Generated SDK integration fixtures
+  enable the internal WIT-printer feature in their Cargo manifests.
 
 ## Docs
 

examples/auth-component-no-auth/build.rs

@@ -0,0 +1,121 @@
+//! Populates the Miden package cache for builds that `midenc` does not drive.
+//!
+//! Plain `cargo check`, `cargo build`, and IDE analysis expand the Miden SDK macros without a
+//! surrounding `cargo miden build`. Those macros read compiled dependency packages from the
+//! directory named by `MIDENC_PACKAGE_CACHE`. This script asks `cargo-miden` for that
+//! directory, populates it with a nested build when the project has source dependencies, and
+//! exports the variable to the compilation of this crate.
+//!
+//! See <https://github.com/0xMiden/compiler/issues/1298>.
+
+use std::{
+    env, fs,
+    path::{Path, PathBuf},
+    process::{Command, Output},
+};
+
+fn main() {
+    // Re-evaluate this script when the build mode or the tool selection changes.
+    println!("cargo:rerun-if-env-changed=MIDENC_PACKAGE_CACHE");
+    println!("cargo:rerun-if-env-changed=CARGO_MIDEN");
+    // These inputs shape the compiler's package-cache fingerprint.
+    println!("cargo:rerun-if-env-changed=RUSTFLAGS");
+    println!("cargo:rerun-if-env-changed=RUSTUP_TOOLCHAIN");
+
+    // Inside a midenc-driven build the compiler owns the package cache, macro expansion
+    // already sees the variable, and a nested build would recurse into this script forever.
+    if env::var_os("MIDENC_PACKAGE_CACHE").is_some() {
+        return;
+    }
+
+    let manifest_dir = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+
+    // Ask the compiler where this project's package cache lives and which inputs shape it.
+    let query = run_cargo_miden(&manifest_dir, "package-cache");
+    if !query.status.success() {
+        panic!(
+            "`cargo miden package-cache` failed ({}).\nInstall cargo-miden (`cargo install \
+             cargo-miden`) or point the CARGO_MIDEN environment variable at a cargo-miden \
+             binary.\n--- stderr ---\n{}",
+            query.status,
+            String::from_utf8_lossy(&query.stderr),
+        );
+    }
+
+    let stdout =
+        String::from_utf8(query.stdout).expect("cargo miden package-cache output is UTF-8");
+    let mut cache_dir = None;
+    let mut source_dependencies = 0usize;
+    for line in stdout.lines() {
+        if let Some(value) = line.strip_prefix("cache-dir=") {
+            cache_dir = Some(PathBuf::from(value));
+        } else if let Some(value) = line.strip_prefix("source-dependencies=") {
+            source_dependencies = value.parse().expect("source-dependencies is a number");
+        } else if let Some(value) = line.strip_prefix("watch=") {
+            println!("cargo:rerun-if-changed={value}");
+        }
+    }
+    let cache_dir = cache_dir.expect("cargo miden package-cache printed no cache-dir");
+
+    if source_dependencies > 0 {
+        // Populate the cache. Dependency packages publish before the root target compiles, so
+        // even a failing build (for example, this crate is mid-edit) usually leaves the
+        // dependency packages usable; the macros report anything that is genuinely missing.
+        let build = run_cargo_miden(&manifest_dir, "build");
+        if !build.status.success() {
+            println!(
+                "cargo:warning=`cargo miden build --release` failed ({}); dependency packages \
+                 may be stale or missing: {}",
+                build.status,
+                last_stderr_line(&build.stderr),
+            );
+        }
+    }
+
+    // The macros treat a missing directory as an empty cache; create it so the exported
+    // variable always points at a real location. Watching the directory re-runs this script
+    // when another build rewrites or prunes the cache (cargo re-runs unconditionally while a
+    // watched path is missing), which keeps the exported path and its packages live.
+    fs::create_dir_all(&cache_dir).expect("failed to create the Miden package cache directory");
+    println!("cargo:rerun-if-changed={}", cache_dir.display());
+    println!("cargo:rustc-env=MIDENC_PACKAGE_CACHE={}", cache_dir.display());
+}
+
+/// Runs `cargo miden <subcommand> --release` for the project in `manifest_dir`.
+///
+/// `CARGO_MIDEN` selects a specific `cargo-miden` binary; otherwise the `cargo miden` plugin
+/// is resolved through the `cargo` that drives this build. The nested build gets its own
+/// cargo target directory: the outer cargo holds a lock on this build's target directory
+/// while build scripts run, and a nested build against the same directory would deadlock.
+/// `--release` keeps the cache fingerprint identical to a `cargo miden build --release` run
+/// by hand, so both share one cache.
+fn run_cargo_miden(manifest_dir: &Path, subcommand: &str) -> Output {
+    let mut command = match env::var_os("CARGO_MIDEN") {
+        Some(cargo_miden) => Command::new(cargo_miden),
+        None => Command::new(env::var_os("CARGO").unwrap_or_else(|| "cargo".into())),
+    };
+    command
+        .args(["miden", subcommand, "--release"])
+        .current_dir(manifest_dir)
+        .env(
+            "CARGO_TARGET_DIR",
+            manifest_dir.join("target").join("miden").join("build-script"),
+        );
+    command.output().unwrap_or_else(|err| {
+        panic!(
+            "failed to run `cargo miden {subcommand}`: {err}.\nInstall cargo-miden (`cargo \
+             install cargo-miden`) or point the CARGO_MIDEN environment variable at a \
+             cargo-miden binary."
+        )
+    })
+}
+
+/// Returns the last non-empty stderr line for a compact warning.
+fn last_stderr_line(stderr: &[u8]) -> String {
+    String::from_utf8_lossy(stderr)
+        .lines()
+        .rev()
+        .find(|line| !line.trim().is_empty())
+        .unwrap_or("no error output")
+        .to_string()
+}

examples/auth-component-rpo-falcon512/build.rs

@@ -0,0 +1,121 @@
+//! Populates the Miden package cache for builds that `midenc` does not drive.
+//!
+//! Plain `cargo check`, `cargo build`, and IDE analysis expand the Miden SDK macros without a
+//! surrounding `cargo miden build`. Those macros read compiled dependency packages from the
+//! directory named by `MIDENC_PACKAGE_CACHE`. This script asks `cargo-miden` for that
+//! directory, populates it with a nested build when the project has source dependencies, and
+//! exports the variable to the compilation of this crate.
+//!
+//! See <https://github.com/0xMiden/compiler/issues/1298>.
+
+use std::{
+    env, fs,
+    path::{Path, PathBuf},
+    process::{Command, Output},
+};
+
+fn main() {
+    // Re-evaluate this script when the build mode or the tool selection changes.
+    println!("cargo:rerun-if-env-changed=MIDENC_PACKAGE_CACHE");
+    println!("cargo:rerun-if-env-changed=CARGO_MIDEN");
+    // These inputs shape the compiler's package-cache fingerprint.
+    println!("cargo:rerun-if-env-changed=RUSTFLAGS");
+    println!("cargo:rerun-if-env-changed=RUSTUP_TOOLCHAIN");
+
+    // Inside a midenc-driven build the compiler owns the package cache, macro expansion
+    // already sees the variable, and a nested build would recurse into this script forever.
+    if env::var_os("MIDENC_PACKAGE_CACHE").is_some() {
+        return;
+    }
+
+    let manifest_dir = PathBuf::from(env::var_os("CARGO_MANIFEST_DIR").unwrap());
+
+    // Ask the compiler where this project's package cache lives and which inputs shape it.
+    let query = run_cargo_miden(&manifest_dir, "package-cache");
+    if !query.status.success() {
+        panic!(
+            "`cargo miden package-cache` failed ({}).\nInstall cargo-miden (`cargo install \
+             cargo-miden`) or point the CARGO_MIDEN environment variable at a cargo-miden \
+             binary.\n--- stderr ---\n{}",
+            query.status,
+            String::from_utf8_lossy(&query.stderr),
+        );
+    }
+
+    let stdout =
+        String::from_utf8(query.stdout).expect("cargo miden package-cache output is UTF-8");
+    let mut cache_dir = None;
+    let mut source_dependencies = 0usize;
+    for line in stdout.lines() {
+        if let Some(value) = line.strip_prefix("cache-dir=") {
+            cache_dir = Some(PathBuf::from(value));
+        } else if let Some(value) = line.strip_prefix("source-dependencies=") {
+            source_dependencies = value.parse().expect("source-dependencies is a number");
+        } else if let Some(value) = line.strip_prefix("watch=") {
+            println!("cargo:rerun-if-changed={value}");
+        }
+    }
+    let cache_dir = cache_dir.expect("cargo miden package-cache printed no cache-dir");
+
+    if source_dependencies > 0 {
+        // Populate the cache. Dependency packages publish before the root target compiles, so
+        // even a failing build (for example, this crate is mid-edit) usually leaves the
+        // dependency packages usable; the macros report anything that is genuinely missing.
+        let build = run_cargo_miden(&manifest_dir, "build");
+        if !build.status.success() {
+            println!(
+                "cargo:warning=`cargo miden build --release` failed ({}); dependency packages \
+                 may be stale or missing: {}",
+                build.status,
+                last_stderr_line(&build.stderr),
+            );
+        }
+    }
+
+    // The macros treat a missing directory as an empty cache; create it so the exported
+    // variable always points at a real location. Watching the directory re-runs this script
+    // when another build rewrites or prunes the cache (cargo re-runs unconditionally while a
+    // watched path is missing), which keeps the exported path and its packages live.
+    fs::create_dir_all(&cache_dir).expect("failed to create the Miden package cache directory");
+    println!("cargo:rerun-if-changed={}", cache_dir.display());
+    println!("cargo:rustc-env=MIDENC_PACKAGE_CACHE={}", cache_dir.display());
+}
+
+/// Runs `cargo miden <subcommand> --release` for the project in `manifest_dir`.
+///
+/// `CARGO_MIDEN` selects a specific `cargo-miden` binary; otherwise the `cargo miden` plugin
+/// is resolved through the `cargo` that drives this build. The nested build gets its own
+/// cargo target directory: the outer cargo holds a lock on this build's target directory
+/// while build scripts run, and a nested build against the same directory would deadlock.
+/// `--release` keeps the cache fingerprint identical to a `cargo miden build --release` run
+/// by hand, so both share one cache.
+fn run_cargo_miden(manifest_dir: &Path, subcommand: &str) -> Output {
+    let mut command = match env::var_os("CARGO_MIDEN") {
+        Some(cargo_miden) => Command::new(cargo_miden),
+        None => Command::new(env::var_os("CARGO").unwrap_or_else(|| "cargo".into())),
+    };
+    command
+        .args(["miden", subcommand, "--release"])
+        .current_dir(manifest_dir)
+        .env(
+            "CARGO_TARGET_DIR",
+            manifest_dir.join("target").join("miden").join("build-script"),
+        );
+    command.output().unwrap_or_else(|err| {
+        panic!(
+            "failed to run `cargo miden {subcommand}`: {err}.\nInstall cargo-miden (`cargo \
+             install cargo-miden`) or point the CARGO_MIDEN environment variable at a \
+             cargo-miden binary."
+        )
+    })
+}
+
+/// Returns the last non-empty stderr line for a compact warning.
+fn last_stderr_line(stderr: &[u8]) -> String {
+    String::from_utf8_lossy(stderr)
+        .lines()
+        .rev()
+        .find(|line| !line.trim().is_empty())
+        .unwrap_or("no error output")
+        .to_string()
+}