From 39d99564effb8991decc9ae1cb27cdd00dc737e7 Mon Sep 17 00:00:00 2001 From: seonghobae Date: Mon, 31 Aug 2026 19:44:16 +0000 Subject: [PATCH 1/2] feat(analysis): bind simulation method-effect labels to an analysis-run profile GAP-004 leftover / ADR 0057. Bind existing tepp_simulation::generate and refuse_unavailable_document to cutoff-safe method_effects_v1. Census of original/revision/translation/template_copy labels, not an estimator-side method model, not GPU, not MCMC, and not topic birth/split/merge. --- CHANGELOG.md | 2 + Cargo.lock | 1 + DOCUMENTATION.md | 1 + crates/analysis_engine/Cargo.toml | 2 + crates/analysis_engine/src/lib.rs | 14 + .../src/method_effects_artifact.rs | 381 ++++++++++++++++++ .../method_effects_execution_contract.rs | 172 ++++++++ docs/TRACEABILITY.md | 1 + docs/adr/0057-method-effects-analysis-run.md | 78 ++++ docs/adr/README.md | 2 + docs/doctoring/method-effects-analysis-run.md | 15 + 11 files changed, 669 insertions(+) create mode 100644 crates/analysis_engine/src/method_effects_artifact.rs create mode 100644 crates/analysis_engine/tests/method_effects_execution_contract.rs create mode 100644 docs/adr/0057-method-effects-analysis-run.md create mode 100644 docs/doctoring/method-effects-analysis-run.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 062a69412..0d1a6e09b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,6 +38,8 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang ## [Unreleased] +- **Method-effects analysis-run profile**: `analysis_engine` binds existing `tepp_simulation::generate` and `refuse_unavailable_document` to cutoff-safe `method_effects_v1` (`tepp.method_effects.v1`) with inference status `simulation_method_effect_labels_not_estimator_model`. Not an estimator-side method model, not GPU, not MCMC, and not topic birth/split/merge. + - `event_core` adds bounded Allen interval-consistency classification, atomic path-consistency closure, contradiction/resource refusals, and an explicit dependency-error fallback without claiming unrestricted global satisfiability. - `psychometric_core` recovers the Driver, Oud, and Voelkle (2017, Table 2, p. 12 `MANIFESTTRAITVAR`; §7.1, p. 19; p. 16 `MANIFESTTRAITVARstd`; footnote 4; 2017-era ctsem `summary.ctsemFit.R`; JSS PDF re-opened 2026-08-27T14:20Z from https://www.jstatsoft.org/index.php/jss/article/download/v077i05/1104) scalar standardised manifest-trait variance on current main after `0ce16e8` dropped the pre-consolidation code while research notes already named the map (register items 83–84). Table 2 names `MANIFESTTRAITVAR` `Ψ_τ` the additional time-invariant variance-covariance on the measurement level and sets it `NULL` when there is no manifest trait. Equation 5 writes `Γ ~ N(τ, Ψ)` and names that covariance the manifest traits. Section 7.1 names manifest traits stable individual differences in indicator levels, distinct from process-level `TRAITVAR` `φ_ξ`. Page 16 prints standardised matrices with the suffix `std` when appropriate. The printed example on p. 16 is `discreteDRIFTstd`, not `MANIFESTTRAITVARstd`. Footnote 4 standardises using only the relevant variance, not the total. The relevant variance for that named indicator-level correlation is `MANIFESTTRAITVAR`, not process-level `TRAITVAR` and not residual `MANIFESTVAR` `θ`. The 2017-era source forms `MANIFESTTRAITVARstd` only when `MANIFESTTRAITVAR != 0`, as `solve(sqrt(diag(MANIFESTTRAITVAR) + ridging)) %&% MANIFESTTRAITVAR` when `verbose = TRUE`. OpenMx `%&%` is `t(A) %*% B %*% A`. Unlike `TRAITVARstd`, that formation adds `diag(c(ridging), n.manifest)`. The default `ridging = FALSE` adds 0, not `0.0001`; that ridge is a numerical hack and is not this exact map. The scalar correlation is `ψ / ψ = 1` after strictly positive `MANIFESTTRAITVAR`. Form strictly positive `ψ` first, then `1 / √ψ`, then `(1 / √ψ) ψ (1 / √ψ)`. Unstandardised `MANIFESTTRAITVAR` is defined for a zero trait; standardised `MANIFESTTRAITVAR` is not. Zero `MANIFESTTRAITVAR` skips forming `MANIFESTTRAITVARstd` in the 2017-era source and fails closed here. Indicator-level trait variance is an event-time structural quantity, so a non-event clock fails closed. `MANIFESTTRAITVAR` does not require stable `a < 0`. Distinct positive `ψ` recover the same 1. `trait / trait = 1` is `TRAITVARstd` and recovers the same number and remains a distinct named quantity. `θ` is `MANIFESTVAR` and is measurement error, not this correlation. Meredith (1993) remains unread (web search 2026-08-27T14:20Z: Springer/Cambridge Core paywalled; Unpaywall historically `is_oa: false`; Springer `content/pdf` is an HTML stub). Mislevy (1991, *Psychometrika, 56*, 177–196) remains unread on the same terms (DOI `10.1007/bf02294457`). Still not a Kalman filter, not a matrix `expm`, not ESEM estimation, not DSEM, and not ctsem estimation. diff --git a/Cargo.lock b/Cargo.lock index 454a7d612..3ea34ffdd 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -80,6 +80,7 @@ dependencies = [ "sha2", "temporal_core", "tepp_api", + "tepp_simulation", "topic_measurement", "uuid", ] diff --git a/DOCUMENTATION.md b/DOCUMENTATION.md index 6fa4b9683..f301f9b5e 100644 --- a/DOCUMENTATION.md +++ b/DOCUMENTATION.md @@ -71,6 +71,7 @@ TEPP's approved PRD v0.4 and implementation plan are the primary product baselin | Hourly NIM OpenCode doctoring | [`docs/doctoring/hourly-nim-opencode-development.md`](docs/doctoring/hourly-nim-opencode-development.md) | | Analysis engine v1 doctoring | [`docs/doctoring/analysis-engine-v1.md`](docs/doctoring/analysis-engine-v1.md) | | Analysis engine gap-closure doctoring | [`docs/doctoring/analysis-engine-gap-closure.md`](docs/doctoring/analysis-engine-gap-closure.md) | +| Method-effects analysis-run doctoring | [`docs/doctoring/method-effects-analysis-run.md`](docs/doctoring/method-effects-analysis-run.md) | | Corpus-split leakage-audit wire doctoring | [`docs/research/corpus-split-manifest-wire.md`](docs/research/corpus-split-manifest-wire.md) | | Unicode canonical-identity doctoring | [`docs/research/unicode-canonical-identity.md`](docs/research/unicode-canonical-identity.md) | | Change history | [`CHANGELOG.md`](CHANGELOG.md) | diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 7322212b2..8ea0d0ed0 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -20,10 +20,12 @@ serde_json = { workspace = true } sha2 = { workspace = true } tepp_api = { path = "../tepp_api", version = "0.2.0" } temporal_core = { path = "../temporal_core", version = "0.2.0" } +tepp_simulation = { path = "../tepp_simulation", version = "0.2.0" } topic_measurement = { path = "../topic_measurement", version = "0.2.0" } uuid.workspace = true [dev-dependencies] +tepp_simulation = { path = "../tepp_simulation", version = "0.2.0" } corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } diff --git a/crates/analysis_engine/src/lib.rs b/crates/analysis_engine/src/lib.rs index 72bd5854c..3cd62881d 100644 --- a/crates/analysis_engine/src/lib.rs +++ b/crates/analysis_engine/src/lib.rs @@ -12,6 +12,7 @@ mod case_deletion_refit; mod lineage_criterion; +mod method_effects_artifact; mod topic_context_posterior; mod topic_lineage_artifact; @@ -46,6 +47,12 @@ pub use lineage_criterion::{ LineageCriterionFit, LineageCriterionFitError, LineageCriterionObservation, fit_lineage_criterion_posteriors, }; +/// Method-effect census artifact and execution contracts from this engine. +pub use method_effects_artifact::{ + METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT, METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, + METHOD_EFFECTS_MODEL_CONTRACT_VERSION, METHOD_EFFECTS_OUTPUT_PROFILE, MethodEffectsArtifact, + MethodEffectsExecution, execute_method_effects_run, +}; /// Bounded posterior topic-context producer contract and record types. pub use topic_context_posterior::{ TOPIC_CONTEXT_POSTERIOR_BYTE_LIMIT, TOPIC_CONTEXT_POSTERIOR_SCHEMA_VERSION, @@ -248,6 +255,8 @@ pub enum AnalysisEngineError { TopicMeasurement(TopicMeasurementError), /// A topic-lineage artifact violated its bounded schema or count invariants. InvalidTopicLineageArtifact, + /// A method-effects artifact violated its bounded schema or count invariants. + InvalidMethodEffectsArtifact, } impl fmt::Display for AnalysisEngineError { @@ -262,6 +271,7 @@ impl fmt::Display for AnalysisEngineError { Self::LimitExceeded => "analysis corpus exceeded its execution bound", Self::TopicMeasurement(error) => return error.fmt(formatter), Self::InvalidTopicLineageArtifact => "invalid topic lineage artifact", + Self::InvalidMethodEffectsArtifact => "invalid method-effects artifact", }; formatter.write_str(message) } @@ -681,6 +691,10 @@ mod tests { AnalysisEngineError::InvalidTopicLineageArtifact, "invalid topic lineage artifact", ), + ( + AnalysisEngineError::InvalidMethodEffectsArtifact, + "invalid method-effects artifact", + ), ]; for (error, message) in messages { assert_eq!(error.to_string(), message); diff --git a/crates/analysis_engine/src/method_effects_artifact.rs b/crates/analysis_engine/src/method_effects_artifact.rs new file mode 100644 index 000000000..df5280598 --- /dev/null +++ b/crates/analysis_engine/src/method_effects_artifact.rs @@ -0,0 +1,381 @@ +//! Digest-bound simulation method-effect labels as an analysis-run profile. + +use serde::{Deserialize, Serialize}; +use sha2::{Digest, Sha256}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{ + AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, +}; +use tepp_simulation::{ + DocumentMethodEffect, SimulationConfig, SimulationError, generate, refuse_unavailable_document, +}; + +use crate::{AnalysisEngineError, format_digest, require_receipt_identity, valid_identifier}; + +/// Versioned schema for a completed method-effects artifact. +pub const METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION: &str = "tepp.method_effects.v1"; +/// Model contract required by the method-effects execution path. +pub const METHOD_EFFECTS_MODEL_CONTRACT_VERSION: &str = "method_effects_v1"; +/// Analysis-run output profile required for a method-effects artifact. +pub const METHOD_EFFECTS_OUTPUT_PROFILE: &str = "method_effects_v1"; +/// Maximum canonical artifact JSON size. +pub const METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +const METHOD_EFFECTS_INFERENCE_STATUS: &str = "simulation_method_effect_labels_not_estimator_model"; + +/// Completed, bounded method-effect census for analysis-run clients. +#[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] +#[serde(deny_unknown_fields)] +pub struct MethodEffectsArtifact { + /// Exact versioned schema identity. + pub schema_version: String, + /// Opaque accepted-run identity. + pub run_id: String, + /// Immutable source snapshot identity. + pub snapshot_id: String, + /// Historical evidence cutoff used to admit documents. + pub knowledge_cutoff: String, + /// Explicit simulation seed. + pub seed: u64, + /// Digest of the simulation configuration. + pub config_digest: String, + /// Digest of the generated truth rows. + pub content_digest: String, + /// Number of documents available at the cutoff. + pub document_count: u64, + /// Original reports admitted at the cutoff. + pub original_count: u64, + /// Revision variants admitted at the cutoff. + pub revision_count: u64, + /// Translation variants admitted at the cutoff. + pub translation_count: u64, + /// Template-copy variants admitted at the cutoff. + pub template_copy_count: u64, + /// Derivative (non-original) documents admitted at the cutoff. + pub derivative_count: u64, + /// Fixed claim boundary for consumer copy. + pub inference_status: String, +} + +impl MethodEffectsArtifact { + /// Parse and fully validate a bounded artifact JSON payload. + /// + /// # Errors + /// + /// Returns [`AnalysisEngineError::InvalidMethodEffectsArtifact`] when the + /// schema, identifiers, counts, or claim boundary fail. + pub fn from_json(payload: &str) -> Result { + if payload.len() > METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + let artifact: Self = serde_json::from_str(payload) + .map_err(|_| AnalysisEngineError::InvalidMethodEffectsArtifact)?; + artifact.validate()?; + Ok(artifact) + } + + /// Serialize canonical validated artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation, serialization, or size failure. + pub fn to_json(&self) -> Result { + self.validate()?; + let payload = + serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; + if payload.len() > METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(payload) + } + + /// Return the lowercase SHA-256 digest of canonical artifact JSON. + /// + /// # Errors + /// + /// Returns a typed validation or serialization failure. + pub fn sha256(&self) -> Result { + self.to_json() + .map(|json| format_digest(Sha256::digest(json.into_bytes()))) + } + + fn validate(&self) -> Result<(), AnalysisEngineError> { + let kind_sum = self + .original_count + .checked_add(self.revision_count) + .and_then(|value| value.checked_add(self.translation_count)) + .and_then(|value| value.checked_add(self.template_copy_count)); + let derivative_sum = self + .revision_count + .checked_add(self.translation_count) + .and_then(|value| value.checked_add(self.template_copy_count)); + if self.schema_version != METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION + || !valid_identifier(&self.run_id) + || !valid_identifier(&self.snapshot_id) + || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() + || !valid_identifier(&self.config_digest) + || self.config_digest.len() != 64 + || !valid_identifier(&self.content_digest) + || self.content_digest.len() != 64 + || self.document_count < 2 + || self.original_count == 0 + || kind_sum != Some(self.document_count) + || derivative_sum != Some(self.derivative_count) + || self.inference_status != METHOD_EFFECTS_INFERENCE_STATUS + { + return Err(AnalysisEngineError::InvalidMethodEffectsArtifact); + } + Ok(()) + } +} + +/// One completed method-effects artifact and its terminal result. +#[derive(Clone, Debug, PartialEq)] +pub struct MethodEffectsExecution { + /// Digest-bound completed method-effect census. + pub artifact: MethodEffectsArtifact, + /// Terminal result carrying the artifact identity, digest, and schema. + pub terminal_result: AnalysisRunTerminalResult, +} + +/// Execute cutoff-safe simulation method-effect labels as one analysis-run profile. +/// +/// The executor invokes [`generate`] and [`refuse_unavailable_document`] already +/// on protected main. It does not invent an estimator-side method model, GPU +/// kernels, MCMC, or topic birth/split/merge events. +/// +/// # Errors +/// +/// Returns a request/receipt/snapshot/cutoff/profile error, simulation failure, +/// empty/undersized available corpus, or invalid artifact error. +pub fn execute_method_effects_run( + request: &AnalysisRunRequest, + accepted: &AnalysisRunAccepted, + snapshot_id: &str, + knowledge_cutoff: KnowledgeCutoff, + config: SimulationConfig, + completed_at: impl Into, +) -> Result { + request.to_json()?; + accepted.to_json()?; + require_receipt_identity(request, accepted)?; + if request.snapshot_id != snapshot_id { + return Err(AnalysisEngineError::SnapshotMismatch); + } + if request.knowledge_cutoff != knowledge_cutoff.to_rfc3339() + || request.model_contract_version != METHOD_EFFECTS_MODEL_CONTRACT_VERSION + || request.output_profile != METHOD_EFFECTS_OUTPUT_PROFILE + { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let manifest = generate(config).map_err(map_simulation_error)?; + let mut original_count = 0_u64; + let mut revision_count = 0_u64; + let mut translation_count = 0_u64; + let mut template_copy_count = 0_u64; + let mut derivative_count = 0_u64; + let mut document_count = 0_u64; + for document in manifest.documents() { + if refuse_unavailable_document(document, &knowledge_cutoff).is_err() { + continue; + } + document_count = document_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + match document.method_effect() { + DocumentMethodEffect::Original => { + original_count = original_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + } + DocumentMethodEffect::Revision => { + revision_count = revision_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + derivative_count = derivative_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + } + DocumentMethodEffect::Translation => { + translation_count = translation_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + derivative_count = derivative_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + } + DocumentMethodEffect::TemplateCopy => { + template_copy_count = template_copy_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + derivative_count = derivative_count + .checked_add(1) + .ok_or(AnalysisEngineError::ArithmeticOverflow)?; + } + _ => return Err(AnalysisEngineError::InvalidEvidence), + } + } + if document_count < 2 || original_count == 0 { + return Err(AnalysisEngineError::InvalidEvidence); + } + + let artifact = MethodEffectsArtifact { + schema_version: METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION.into(), + run_id: accepted.run_id.clone(), + snapshot_id: snapshot_id.to_owned(), + knowledge_cutoff: knowledge_cutoff.to_rfc3339(), + seed: manifest.seed(), + config_digest: manifest.config_digest().to_owned(), + content_digest: manifest.content_digest().to_owned(), + document_count, + original_count, + revision_count, + translation_count, + template_copy_count, + derivative_count, + inference_status: METHOD_EFFECTS_INFERENCE_STATUS.into(), + }; + let digest = artifact.sha256()?; + let summary = AnalysisResultSummary::new( + "method_effects", + document_count, + 6, + METHOD_EFFECTS_INFERENCE_STATUS, + )?; + let terminal_result = AnalysisRunTerminalResult::succeeded( + request, + accepted, + format!("method_effects_artifact_{}", &digest[..16]), + digest, + METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, + completed_at, + summary, + )?; + Ok(MethodEffectsExecution { + artifact, + terminal_result, + }) +} + +fn map_simulation_error(error: SimulationError) -> AnalysisEngineError { + match error { + SimulationError::InvalidConfiguration + | SimulationError::TemporalInvariantViolation + | SimulationError::ManifestInvariantViolation + | _ => AnalysisEngineError::InvalidEvidence, + } +} + +#[cfg(test)] +mod tests { + use super::{ + METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT, METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, + METHOD_EFFECTS_INFERENCE_STATUS, MethodEffectsArtifact, + }; + use crate::AnalysisEngineError; + + fn artifact() -> MethodEffectsArtifact { + MethodEffectsArtifact { + schema_version: METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION.into(), + run_id: "run-1".into(), + snapshot_id: "snapshot-1".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + seed: 7, + config_digest: "a".repeat(64), + content_digest: "b".repeat(64), + document_count: 4, + original_count: 2, + revision_count: 1, + translation_count: 1, + template_copy_count: 0, + derivative_count: 2, + inference_status: METHOD_EFFECTS_INFERENCE_STATUS.into(), + } + } + + fn assert_invalid(artifact: &MethodEffectsArtifact) { + assert_eq!( + artifact.to_json(), + Err(AnalysisEngineError::InvalidMethodEffectsArtifact) + ); + } + + #[test] + fn artifact_round_trip_and_size_bounds_fail_closed() { + let artifact = artifact(); + let payload = artifact.to_json().expect("json"); + assert_eq!( + MethodEffectsArtifact::from_json(&payload), + Ok(artifact.clone()) + ); + assert_eq!(artifact.sha256().expect("digest").len(), 64); + assert_eq!( + MethodEffectsArtifact::from_json("{}"), + Err(AnalysisEngineError::InvalidMethodEffectsArtifact) + ); + assert_eq!( + MethodEffectsArtifact::from_json(&"x".repeat(METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT + 1)), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + #[test] + fn artifact_metadata_tampering_fails_closed() { + let artifact = artifact(); + let invalid_artifacts = [ + { + let mut value = artifact.clone(); + value.schema_version.clear(); + value + }, + { + let mut value = artifact.clone(); + value.run_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.snapshot_id.clear(); + value + }, + { + let mut value = artifact.clone(); + value.knowledge_cutoff = "invalid".into(); + value + }, + { + let mut value = artifact.clone(); + value.config_digest.clear(); + value + }, + { + let mut value = artifact.clone(); + value.content_digest = "short".into(); + value + }, + { + let mut value = artifact.clone(); + value.document_count = 1; + value + }, + { + let mut value = artifact.clone(); + value.original_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.derivative_count = 0; + value + }, + { + let mut value = artifact.clone(); + value.inference_status.clear(); + value + }, + ]; + for invalid in invalid_artifacts { + assert_invalid(&invalid); + } + } +} diff --git a/crates/analysis_engine/tests/method_effects_execution_contract.rs b/crates/analysis_engine/tests/method_effects_execution_contract.rs new file mode 100644 index 000000000..36b563e33 --- /dev/null +++ b/crates/analysis_engine/tests/method_effects_execution_contract.rs @@ -0,0 +1,172 @@ +//! End-to-end contract for cutoff-safe simulation method-effect labels. + +use analysis_engine::{ + AnalysisEngineError, METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, + METHOD_EFFECTS_MODEL_CONTRACT_VERSION, METHOD_EFFECTS_OUTPUT_PROFILE, + execute_method_effects_run, +}; +use temporal_core::KnowledgeCutoff; +use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; +use tepp_simulation::SimulationConfig; + +fn cutoff() -> KnowledgeCutoff { + KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") +} + +fn request() -> AnalysisRunRequest { + AnalysisRunRequest { + contract_version: 1, + idempotency_key: "method-effects-idem".into(), + tenant_workspace_id: "tenant-workspace".into(), + snapshot_id: "snapshot-method-effects".into(), + knowledge_cutoff: "2026-08-01T00:00:00Z".into(), + model_contract_version: METHOD_EFFECTS_MODEL_CONTRACT_VERSION.into(), + output_profile: METHOD_EFFECTS_OUTPUT_PROFILE.into(), + } +} + +fn accepted(request: &AnalysisRunRequest) -> AnalysisRunAccepted { + AnalysisRunAccepted::new("run-method-effects", "accepted", &request.idempotency_key) + .expect("accepted") +} + +fn execute( + request: &AnalysisRunRequest, +) -> Result { + execute_method_effects_run( + request, + &accepted(request), + "snapshot-method-effects", + cutoff(), + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ) +} + +#[test] +fn mixed_method_effects_emit_digest_bound_census_without_estimator_model() { + let request = request(); + let execution = execute(&request).expect("execution"); + assert_eq!( + execution.artifact.schema_version, + METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION + ); + assert_eq!(execution.artifact.seed, 7); + assert_eq!(execution.artifact.config_digest.len(), 64); + assert_eq!(execution.artifact.content_digest.len(), 64); + assert!(execution.artifact.document_count >= 2); + assert!(execution.artifact.original_count >= 1); + assert_eq!( + execution.artifact.original_count + + execution.artifact.revision_count + + execution.artifact.translation_count + + execution.artifact.template_copy_count, + execution.artifact.document_count + ); + assert_eq!( + execution.artifact.revision_count + + execution.artifact.translation_count + + execution.artifact.template_copy_count, + execution.artifact.derivative_count + ); + assert!(execution.artifact.derivative_count >= 1); + assert_eq!( + execution.artifact.inference_status, + "simulation_method_effect_labels_not_estimator_model" + ); + assert_eq!( + execution.terminal_result.run_state, + AnalysisRunTerminalState::Succeeded + ); + assert_eq!( + execution.terminal_result.result_sha256.as_deref(), + Some(execution.artifact.sha256().expect("digest").as_str()) + ); + assert_eq!( + execution.terminal_result.result_schema_version.as_deref(), + Some(METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION) + ); +} + +#[test] +fn empty_available_corpus_and_single_original_fail_closed() { + let request = request(); + let early = KnowledgeCutoff::parse_rfc3339("2025-12-31T00:00:00Z").expect("early"); + let mut early_request = request.clone(); + early_request.knowledge_cutoff = early.to_rfc3339(); + assert_eq!( + execute_method_effects_run( + &early_request, + &accepted(&early_request), + "snapshot-method-effects", + early, + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + let singleton = SimulationConfig::new(7, 1, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0).expect("config"); + assert_eq!( + execute_method_effects_run( + &request, + &accepted(&request), + "snapshot-method-effects", + cutoff(), + singleton, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); +} + +#[test] +fn execution_refuses_snapshot_profile_and_cutoff_mismatch() { + let request = request(); + assert_eq!( + execute_method_effects_run( + &request, + &accepted(&request), + "other-snapshot", + cutoff(), + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::SnapshotMismatch) + ); + let mut mismatched = request.clone(); + mismatched.knowledge_cutoff = "2026-07-01T00:00:00Z".into(); + assert_eq!( + execute_method_effects_run( + &mismatched, + &accepted(&mismatched), + "snapshot-method-effects", + cutoff(), + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + for profile in [ + "trsl_topic_lineage_v1", + "fitted_candidate_k_v1", + "pareto_candidate_k_v1", + "joint_posterior_draws_v1", + "composed_fitted_lineage_v1", + "case_deletion_refit_v1", + "topic_activity_v1", + ] { + let mut reused = request.clone(); + reused.output_profile = profile.into(); + assert_eq!( + execute_method_effects_run( + &reused, + &accepted(&reused), + "snapshot-method-effects", + cutoff(), + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + } +} diff --git a/docs/TRACEABILITY.md b/docs/TRACEABILITY.md index 2b783c2ab..8c0d14404 100644 --- a/docs/TRACEABILITY.md +++ b/docs/TRACEABILITY.md @@ -58,6 +58,7 @@ The full APA 7th standards/literature register remains `docs/research/standards- | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); HTTP service remaining accepted-target | partial | | versioned service/API contracts and exports | PRD; API contract; ADR 0011/0013 | `tepp_api` analysis-run/export/JSON-LD/GraphML contracts on protected main (PR #21); LineageWeave loopback contracts and request-bound terminal result are composed on the active product branch; production TLS remaining | partial | | executable cutoff-safe analysis runs | ADR 0012/0022; temporal research; API terminal-result contract | `analysis_engine` availability cutoff, snapshot binding, multiple-membership aggregation, digest-bound readiness artifact, and `tepp.trsl_topic_lineage.v1` execution through `topic_measurement`; synthetic recovery plus tamper/non-convergence tests and exact coverage on the active product branch | active-PR | +| simulation method-effect analysis-run profile | ADR 0004/0012/0022/0057; DocumentMethodEffect | `analysis_engine` `method_effects_v1` binds `generate` + `refuse_unavailable_document`; digest-bound simulation labels, not an estimator-side method model, not GPU, not MCMC, not topic birth/split/merge; not implemented-main | active-PR | | immutable split/run/reproducibility manifests | ADR 0013; ERD | `tepp_api` reproducibility manifest contract on protected main; `persistence_postgres` append-only SQL insert/lookup for `reproducibility_manifest`, `corpus_split_manifest`, `model_run`, and `model_artifact` (migration `0003`); full physical ERD constraints remaining | partial | | multilingual shared latent semantic space | PRD; ADR 0004; ADR 0020 | `semantic_core` span-grounded units (active-PR); concept dictionary and shared latent estimator remaining | active-PR | | TRSL-TM temporal/relational topic posterior and backend compatibility | ADR 0012; ADR 0004 | `topic_measurement` stable ALR/ILR coordinates and bounded CPU `f64` reference estimator on protected main; `model_selection` fitted candidate-`K` scoring on this PR; calibrated posterior promotion, method effects, persistence, and accelerated backends remaining | partial | diff --git a/docs/adr/0057-method-effects-analysis-run.md b/docs/adr/0057-method-effects-analysis-run.md new file mode 100644 index 000000000..9c2364206 --- /dev/null +++ b/docs/adr/0057-method-effects-analysis-run.md @@ -0,0 +1,78 @@ +# ADR 0057 — Simulation method-effect labels as an analysis-run output profile + +**Decision status:** Accepted +**Implementation maturity:** active-PR — composed on this branch; not implemented-main +**Date:** 2026-08-31 +**Supersedes:** None; complements ADR 0004/0012 (method-effect truth factors) and ADR 0022 (cutoff-safe analysis-run execution). Does not reuse ADR 0049–0056. +**Figma File ID:** N/A — this increment changes a Rust service crate and has no user-interface surface. +**Storybook inventory:** N/A — no reusable web object or interaction changed. + +## Context + +Protected main already labels generated documents with +`DocumentMethodEffect` (`original`, `revision`, `translation`, +`template_copy`) through `tepp_simulation::generate` and admits them with +`refuse_unavailable_document`. Operators still cannot request that +cutoff-safe census as a digest-bound analysis-run output. Estimator-side +method models, GPU kernels, MCMC, and topic birth/split/merge remain later +GAP-004 work and are not this slice. + +## Decision + +Add the `method_effects_v1` analysis-run output profile to +`analysis_engine`. The executor: + +- consumes an already-validated `SimulationConfig`; +- requires the request snapshot and knowledge cutoff to match the offered + construction; +- generates the CPU truth corpus and admits documents through + `refuse_unavailable_document` without reimplementing method-effect labels; +- emits a canonical SHA-256-digested `tepp.method_effects.v1` artifact with + seed, config/content digests, original/revision/translation/template-copy + counts, and inference status + `simulation_method_effect_labels_not_estimator_model`; +- does not invent an estimator-side method model, select GPU backends, draw + MCMC, or emit topic birth/split/merge events. + +This is a simulation method-effect census, not an estimator-side method +model. + +## Alternatives considered + +1. Restore another Driver p.16 standardised matrix — rejected because those + recoveries are already a live micro-PR family and do not bind method-effect + labels to an analysis run. +2. Duplicate exhaustive case-deletion (#413) — rejected because that profile + refits deleted corpora and does not census method-effect labels. +3. Invent an estimator-side method model — rejected because that remains + later GAP-004 work; this slice only binds the library already on main. +4. Bind existing `generate` + `refuse_unavailable_document` to ADR 0022's + analysis-run profile — accepted. + +## Consequences + +Operators can request cutoff-safe method-effect counts as a digest-bound +terminal result. The artifact does not claim an estimator-side method model, +GPU parity, MCMC, or topic birth/split/merge. Snapshot/profile/cutoff +mismatch, empty/undersized available corpora, and missing originals fail +closed. + +## Verification + +The PR includes Rust unit and integration tests for digest-bound mixed +method-effect corpora, empty-available and singleton-original refusal, +snapshot / profile / cutoff mismatch, and artifact tampering. Run: + +```text +cargo fmt --all -- --check +cargo test -p analysis_engine +cargo clippy -p analysis_engine --all-targets -- -D warnings +python3 scripts/validate_documentation.py +``` + +## Rollback and supersession + +Rollback removes the `method_effects_v1` profile. No persisted schema +migration is introduced. Supersede only with an ADR that keeps simulation +method-effect labels distinct from an estimator-side method model, GPU +kernels, and topic birth/split/merge. diff --git a/docs/adr/README.md b/docs/adr/README.md index 1254c8079..3e182a43f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -30,6 +30,7 @@ Read [`ADR_POLICY.md`](ADR_POLICY.md) first. **Decision status and implementatio | [0022](0022-deterministic-analysis-run-execution.md) | Deterministic cutoff-safe analysis-run execution | Accepted | active-PR | Closes the first executable product path from accepted run to digest-bound terminal result without claiming estimator authority. | | [0024](0024-lineage-pair-criterion-and-project-journey-posterior.md) | Independent Event Lineage pair criterion and posterior Project Journey | Proposed | active-PR | Strict artifacts preserve criterion/event-time draws, branches, ties, and CPU/GPU receipts without claiming the scientific estimator is complete. | | [0025](0025-macos-native-rust-mlx-metal-boundary.md) | macOS-native Rust-owned MLX Metal execution | Accepted | accepted-target | Compose authenticates to a native host service; Linux never claims Metal, and actual backend/parity receipts fail closed. | +| [0057](0057-method-effects-analysis-run.md) | Simulation method-effect labels as an analysis-run profile | Accepted | active-PR | Complements ADR 0004/0012/0022; `generate` + `refuse_unavailable_document` census, not an estimator-side method model. | | [0023](0023-lineage-criterion-anchor-contract.md) | TEPP-owned Event Lineage criterion anchor | Accepted | active-PR | PR #237 publishes the strict accepted/rejected artifact and identities; estimator execution remains fail-closed future work. | | [0024](0024-independent-topic-importance-anchor.md) | Posterior topic-context producer contract | Accepted | contract-only active-PR | Strict DTO/schema only; the current estimator does not emit it. fast-mlsirm owns case-deletion influence. | | [0001](0001-rust-first-modular-msa.md) | Rust-first numerical core and CPU `f64` reference | Accepted | partial | ADR 0011 owns cross-service/MSA authority; 0001 retains numerical/backend authority. | @@ -138,6 +139,7 @@ Use the narrowest owning ADR when decisions overlap: - **project-history wire-size symmetry:** ADR 0019. - **LineageWeave project-history service boundary:** ADR 0021. - **accepted-run execution and terminal artifact production:** ADR 0022. +- **simulation method-effect analysis-run profile:** ADR 0057. - **independent lineage criterion and posterior Project Journey:** ADR 0023. - **macOS-native Rust-owned MLX Metal execution:** ADR 0024. diff --git a/docs/doctoring/method-effects-analysis-run.md b/docs/doctoring/method-effects-analysis-run.md new file mode 100644 index 000000000..c256c30de --- /dev/null +++ b/docs/doctoring/method-effects-analysis-run.md @@ -0,0 +1,15 @@ +# Simulation method-effect analysis-run composition + +**Active slice:** ADR 0057 / `method_effects_v1` +**Protected-main status:** not implemented-main + +`tepp_simulation` already labels generated documents with +`DocumentMethodEffect` and admits them through +`refuse_unavailable_document`. This slice binds that census to a +cutoff-safe analysis-run profile so operators can request a digest-bound +identity artifact. + +The artifact inference status is +`simulation_method_effect_labels_not_estimator_model`. This is not an +estimator-side method model, not GPU, not MCMC, and not topic +birth/split/merge. From 8cf56a1ef8c79349dd505776e65172ab9beef168 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Tue, 1 Sep 2026 06:22:51 +0900 Subject: [PATCH 2/2] fix(analysis): bound method effect execution --- crates/analysis_engine/Cargo.toml | 1 - .../src/method_effects_artifact.rs | 144 ++++++++++++------ .../method_effects_execution_contract.rs | 59 ++++++- crates/tepp_simulation/src/lib.rs | 2 +- crates/tepp_simulation/src/truth_manifest.rs | 54 ++++--- docs/adr/0057-method-effects-analysis-run.md | 11 +- docs/doctoring/method-effects-analysis-run.md | 4 + 7 files changed, 200 insertions(+), 75 deletions(-) diff --git a/crates/analysis_engine/Cargo.toml b/crates/analysis_engine/Cargo.toml index 8ea0d0ed0..3b94eac54 100644 --- a/crates/analysis_engine/Cargo.toml +++ b/crates/analysis_engine/Cargo.toml @@ -25,7 +25,6 @@ topic_measurement = { path = "../topic_measurement", version = "0.2.0" } uuid.workspace = true [dev-dependencies] -tepp_simulation = { path = "../tepp_simulation", version = "0.2.0" } corpus_split = { path = "../corpus_split", version = "0.2.0" } membership_core = { path = "../membership_core", version = "0.2.0" } relation_graph = { path = "../relation_graph", version = "0.2.0" } diff --git a/crates/analysis_engine/src/method_effects_artifact.rs b/crates/analysis_engine/src/method_effects_artifact.rs index df5280598..b28f56378 100644 --- a/crates/analysis_engine/src/method_effects_artifact.rs +++ b/crates/analysis_engine/src/method_effects_artifact.rs @@ -6,9 +6,7 @@ use temporal_core::KnowledgeCutoff; use tepp_api::{ AnalysisResultSummary, AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalResult, }; -use tepp_simulation::{ - DocumentMethodEffect, SimulationConfig, SimulationError, generate, refuse_unavailable_document, -}; +use tepp_simulation::{SimulationConfig, digest_documents, generate}; use crate::{AnalysisEngineError, format_digest, require_receipt_identity, valid_identifier}; @@ -20,8 +18,18 @@ pub const METHOD_EFFECTS_MODEL_CONTRACT_VERSION: &str = "method_effects_v1"; pub const METHOD_EFFECTS_OUTPUT_PROFILE: &str = "method_effects_v1"; /// Maximum canonical artifact JSON size. pub const METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT: usize = 256 * 1024; +/// Maximum generated event, document, membership, and relation rows per run. +pub const METHOD_EFFECTS_GENERATED_ROW_LIMIT: u64 = 1_000_000; const METHOD_EFFECTS_INFERENCE_STATUS: &str = "simulation_method_effect_labels_not_estimator_model"; +#[derive(Clone, Copy)] +enum MethodEffectBucket { + Original, + Revision, + Translation, + TemplateCopy, +} + /// Completed, bounded method-effect census for analysis-run clients. #[derive(Clone, Debug, Deserialize, PartialEq, Serialize)] #[serde(deny_unknown_fields)] @@ -82,9 +90,6 @@ impl MethodEffectsArtifact { self.validate()?; let payload = serde_json::to_string(self).map_err(|_| AnalysisEngineError::SerializationFailure)?; - if payload.len() > METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT { - return Err(AnalysisEngineError::LimitExceeded); - } Ok(payload) } @@ -112,10 +117,8 @@ impl MethodEffectsArtifact { || !valid_identifier(&self.run_id) || !valid_identifier(&self.snapshot_id) || KnowledgeCutoff::parse_rfc3339(&self.knowledge_cutoff).is_err() - || !valid_identifier(&self.config_digest) - || self.config_digest.len() != 64 - || !valid_identifier(&self.content_digest) - || self.content_digest.len() != 64 + || !is_sha256(&self.config_digest) + || !is_sha256(&self.content_digest) || self.document_count < 2 || self.original_count == 0 || kind_sum != Some(self.document_count) @@ -139,9 +142,9 @@ pub struct MethodEffectsExecution { /// Execute cutoff-safe simulation method-effect labels as one analysis-run profile. /// -/// The executor invokes [`generate`] and [`refuse_unavailable_document`] already -/// on protected main. It does not invent an estimator-side method model, GPU -/// kernels, MCMC, or topic birth/split/merge events. +/// The executor invokes [`generate`] and the manifest's cutoff-eligible document +/// projection already on protected main. It does not invent an estimator-side +/// method model, GPU kernels, MCMC, or topic birth/split/merge events. /// /// # Errors /// @@ -167,28 +170,27 @@ pub fn execute_method_effects_run( { return Err(AnalysisEngineError::InvalidEvidence); } + require_generation_budget(config)?; - let manifest = generate(config).map_err(map_simulation_error)?; + let manifest = generate(config).map_err(|_| AnalysisEngineError::InvalidEvidence)?; + let eligible_documents = manifest.documents_eligible_at_cutoff(&knowledge_cutoff); let mut original_count = 0_u64; let mut revision_count = 0_u64; let mut translation_count = 0_u64; let mut template_copy_count = 0_u64; let mut derivative_count = 0_u64; let mut document_count = 0_u64; - for document in manifest.documents() { - if refuse_unavailable_document(document, &knowledge_cutoff).is_err() { - continue; - } + for document in &eligible_documents { document_count = document_count .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; - match document.method_effect() { - DocumentMethodEffect::Original => { + match method_effect_bucket(document.method_effect().wire_name())? { + MethodEffectBucket::Original => { original_count = original_count .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; } - DocumentMethodEffect::Revision => { + MethodEffectBucket::Revision => { revision_count = revision_count .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; @@ -196,7 +198,7 @@ pub fn execute_method_effects_run( .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; } - DocumentMethodEffect::Translation => { + MethodEffectBucket::Translation => { translation_count = translation_count .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; @@ -204,7 +206,7 @@ pub fn execute_method_effects_run( .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; } - DocumentMethodEffect::TemplateCopy => { + MethodEffectBucket::TemplateCopy => { template_copy_count = template_copy_count .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; @@ -212,10 +214,9 @@ pub fn execute_method_effects_run( .checked_add(1) .ok_or(AnalysisEngineError::ArithmeticOverflow)?; } - _ => return Err(AnalysisEngineError::InvalidEvidence), } } - if document_count < 2 || original_count == 0 { + if document_count < 2 { return Err(AnalysisEngineError::InvalidEvidence); } @@ -226,7 +227,7 @@ pub fn execute_method_effects_run( knowledge_cutoff: knowledge_cutoff.to_rfc3339(), seed: manifest.seed(), config_digest: manifest.config_digest().to_owned(), - content_digest: manifest.content_digest().to_owned(), + content_digest: digest_documents(&eligible_documents), document_count, original_count, revision_count, @@ -236,33 +237,61 @@ pub fn execute_method_effects_run( inference_status: METHOD_EFFECTS_INFERENCE_STATUS.into(), }; let digest = artifact.sha256()?; - let summary = AnalysisResultSummary::new( - "method_effects", - document_count, - 6, - METHOD_EFFECTS_INFERENCE_STATUS, - )?; - let terminal_result = AnalysisRunTerminalResult::succeeded( - request, - accepted, - format!("method_effects_artifact_{}", &digest[..16]), - digest, - METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, - completed_at, - summary, - )?; + #[rustfmt::skip] + let summary = AnalysisResultSummary::new("method_effects", document_count, 6, METHOD_EFFECTS_INFERENCE_STATUS)?; + #[rustfmt::skip] + let terminal_result = AnalysisRunTerminalResult::succeeded(request, accepted, format!("method_effects_artifact_{}", &digest[..16]), digest, METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, completed_at, summary)?; Ok(MethodEffectsExecution { artifact, terminal_result, }) } -fn map_simulation_error(error: SimulationError) -> AnalysisEngineError { - match error { - SimulationError::InvalidConfiguration - | SimulationError::TemporalInvariantViolation - | SimulationError::ManifestInvariantViolation - | _ => AnalysisEngineError::InvalidEvidence, +fn require_generation_budget(config: SimulationConfig) -> Result<(), AnalysisEngineError> { + let events = u64::from(config.event_count()); + let originals = events + .checked_mul(u64::from(config.documents_per_event())) + .ok_or(AnalysisEngineError::LimitExceeded)?; + let documents = originals + .checked_mul(4) + .ok_or(AnalysisEngineError::LimitExceeded)?; + let memberships = documents + .checked_mul(u64::from(config.membership_targets())) + .ok_or(AnalysisEngineError::LimitExceeded)?; + let relations = originals + .checked_mul(8) + .and_then(|value| { + events + .checked_mul(3) + .and_then(|events| value.checked_add(events)) + }) + .and_then(|value| value.checked_add(1)) + .ok_or(AnalysisEngineError::LimitExceeded)?; + let generated_rows = events + .checked_add(documents) + .and_then(|value| value.checked_add(memberships)) + .and_then(|value| value.checked_add(relations)) + .ok_or(AnalysisEngineError::LimitExceeded)?; + if generated_rows > METHOD_EFFECTS_GENERATED_ROW_LIMIT { + return Err(AnalysisEngineError::LimitExceeded); + } + Ok(()) +} + +fn is_sha256(value: &str) -> bool { + value.len() == 64 + && value + .bytes() + .all(|byte| byte.is_ascii_digit() || matches!(byte, b'a'..=b'f')) +} + +fn method_effect_bucket(value: &str) -> Result { + match value { + "original" => Ok(MethodEffectBucket::Original), + "revision" => Ok(MethodEffectBucket::Revision), + "translation" => Ok(MethodEffectBucket::Translation), + "template_copy" => Ok(MethodEffectBucket::TemplateCopy), + _ => Err(AnalysisEngineError::InvalidEvidence), } } @@ -270,7 +299,7 @@ fn map_simulation_error(error: SimulationError) -> AnalysisEngineError { mod tests { use super::{ METHOD_EFFECTS_ARTIFACT_BYTE_LIMIT, METHOD_EFFECTS_ARTIFACT_SCHEMA_VERSION, - METHOD_EFFECTS_INFERENCE_STATUS, MethodEffectsArtifact, + METHOD_EFFECTS_INFERENCE_STATUS, MethodEffectsArtifact, method_effect_bucket, }; use crate::AnalysisEngineError; @@ -345,12 +374,17 @@ mod tests { }, { let mut value = artifact.clone(); - value.config_digest.clear(); + value.config_digest = "g".repeat(64); + value + }, + { + let mut value = artifact.clone(); + value.content_digest = "A".repeat(64); value }, { let mut value = artifact.clone(); - value.content_digest = "short".into(); + value.config_digest = "short".into(); value }, { @@ -363,6 +397,11 @@ mod tests { value.original_count = 0; value }, + { + let mut value = artifact.clone(); + value.document_count = 5; + value + }, { let mut value = artifact.clone(); value.derivative_count = 0; @@ -378,4 +417,9 @@ mod tests { assert_invalid(&invalid); } } + + #[test] + fn unknown_method_effect_names_fail_closed() { + assert!(method_effect_bucket("unknown").is_err()); + } } diff --git a/crates/analysis_engine/tests/method_effects_execution_contract.rs b/crates/analysis_engine/tests/method_effects_execution_contract.rs index 36b563e33..d54cf8fdc 100644 --- a/crates/analysis_engine/tests/method_effects_execution_contract.rs +++ b/crates/analysis_engine/tests/method_effects_execution_contract.rs @@ -7,7 +7,7 @@ use analysis_engine::{ }; use temporal_core::KnowledgeCutoff; use tepp_api::{AnalysisRunAccepted, AnalysisRunRequest, AnalysisRunTerminalState}; -use tepp_simulation::SimulationConfig; +use tepp_simulation::{SimulationConfig, digest_documents, generate}; fn cutoff() -> KnowledgeCutoff { KnowledgeCutoff::parse_rfc3339("2026-08-01T00:00:00Z").expect("cutoff") @@ -54,6 +54,13 @@ fn mixed_method_effects_emit_digest_bound_census_without_estimator_model() { assert_eq!(execution.artifact.seed, 7); assert_eq!(execution.artifact.config_digest.len(), 64); assert_eq!(execution.artifact.content_digest.len(), 64); + let manifest = generate(SimulationConfig::ci_default(7)).expect("manifest"); + let admitted = manifest.documents_eligible_at_cutoff(&cutoff()); + assert_eq!( + execution.artifact.content_digest, + digest_documents(&admitted) + ); + assert_ne!(execution.artifact.content_digest, manifest.content_digest()); assert!(execution.artifact.document_count >= 2); assert!(execution.artifact.original_count >= 1); assert_eq!( @@ -88,6 +95,56 @@ fn mixed_method_effects_emit_digest_bound_census_without_estimator_model() { ); } +#[test] +fn oversized_generation_is_rejected_before_allocation() { + let request = request(); + for oversized in [ + SimulationConfig::new(7, u32::MAX, u32::MAX, u32::MAX, 0, 0, 0, 0, 0, 0, 0, 0) + .expect("overflowing row bound"), + SimulationConfig::new(7, 100_000, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0) + .expect("over-limit row bound"), + ] { + assert_eq!( + execute_method_effects_run( + &request, + &accepted(&request), + "snapshot-method-effects", + cutoff(), + oversized, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::LimitExceeded) + ); + } + + let invalid_schedule = + SimulationConfig::new(1, 8, 1, 1, 2_000, 2_000, 0, 0, 0, 0, 0, 0).expect("config"); + assert_eq!( + execute_method_effects_run( + &request, + &accepted(&request), + "snapshot-method-effects", + cutoff(), + invalid_schedule, + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); + let mut model_mismatch = request.clone(); + model_mismatch.model_contract_version = "other-model".into(); + assert_eq!( + execute_method_effects_run( + &model_mismatch, + &accepted(&model_mismatch), + "snapshot-method-effects", + cutoff(), + SimulationConfig::ci_default(7), + "2026-08-02T00:00:00Z", + ), + Err(AnalysisEngineError::InvalidEvidence) + ); +} + #[test] fn empty_available_corpus_and_single_original_fail_closed() { let request = request(); diff --git a/crates/tepp_simulation/src/lib.rs b/crates/tepp_simulation/src/lib.rs index f3df75714..d35f9c262 100644 --- a/crates/tepp_simulation/src/lib.rs +++ b/crates/tepp_simulation/src/lib.rs @@ -53,7 +53,7 @@ pub use rng::SeededRng; /// Digest-bound known-truth corpus. pub use truth_manifest::TruthManifest; /// Digest helper for configuration fingerprints. -pub use truth_manifest::digest_bytes; +pub use truth_manifest::{digest_bytes, digest_documents}; use uuid::Uuid; diff --git a/crates/tepp_simulation/src/truth_manifest.rs b/crates/tepp_simulation/src/truth_manifest.rs index 32180f952..dd5aa2eb3 100644 --- a/crates/tepp_simulation/src/truth_manifest.rs +++ b/crates/tepp_simulation/src/truth_manifest.rs @@ -183,24 +183,7 @@ impl TruthManifest { hasher.update(event.state().wire_name().as_bytes()); } for document in &self.documents { - hasher.update(document.document_id().as_bytes()); - hasher.update(document.event_id().as_bytes()); - hasher.update(document.document_time().to_rfc3339().as_bytes()); - hasher.update(document.available_time().to_rfc3339().as_bytes()); - hasher.update(document.method_effect().wire_name().as_bytes()); - if let Some(parent) = document.parent_document_id() { - hasher.update(parent.as_bytes()); - } - if let Some(observed) = document.observed_event_time() { - hasher.update(observed.to_rfc3339().as_bytes()); - } else { - hasher.update(b"missing"); - } - for membership in document.memberships() { - hasher.update(membership.group_id().as_bytes()); - hasher.update(membership.role_label().as_bytes()); - hasher.update(membership.weight_bps().to_le_bytes()); - } + hash_document(&mut hasher, document); } for relation in &self.true_relations { hasher.update(relation.relation_id().as_bytes()); @@ -219,6 +202,41 @@ impl TruthManifest { } } +fn hash_document(hasher: &mut Sha256, document: &SimulatedDocument) { + hasher.update(document.document_id().as_bytes()); + hasher.update(document.event_id().as_bytes()); + hasher.update(document.document_time().to_rfc3339().as_bytes()); + hasher.update(document.available_time().to_rfc3339().as_bytes()); + hasher.update(document.method_effect().wire_name().as_bytes()); + if let Some(parent) = document.parent_document_id() { + hasher.update(parent.as_bytes()); + } + if let Some(observed) = document.observed_event_time() { + hasher.update(observed.to_rfc3339().as_bytes()); + } else { + hasher.update(b"missing"); + } + for membership in document.memberships() { + hasher.update(membership.group_id().as_bytes()); + hasher.update(membership.role_label().as_bytes()); + hasher.update(membership.weight_bps().to_le_bytes()); + } +} + +/// Digest an ordered admitted-document population using canonical truth fields. +/// +#[must_use] +pub fn digest_documents(documents: &[&SimulatedDocument]) -> String { + let mut hasher = Sha256::new(); + hasher.update(b"tepp.simulated_documents.v1"); + hasher.update(documents.len().to_string().as_bytes()); + hasher.update([0]); + for document in documents { + hash_document(&mut hasher, document); + } + hex_encode(&hasher.finalize()) +} + fn hex_encode(bytes: &[u8]) -> String { const HEX: &[u8; 16] = b"0123456789abcdef"; let mut out = String::with_capacity(bytes.len() * 2); diff --git a/docs/adr/0057-method-effects-analysis-run.md b/docs/adr/0057-method-effects-analysis-run.md index 9c2364206..3bdffab68 100644 --- a/docs/adr/0057-method-effects-analysis-run.md +++ b/docs/adr/0057-method-effects-analysis-run.md @@ -27,9 +27,11 @@ Add the `method_effects_v1` analysis-run output profile to construction; - generates the CPU truth corpus and admits documents through `refuse_unavailable_document` without reimplementing method-effect labels; +- rejects configurations whose conservative event/document/membership/relation + row bound exceeds 1,000,000 before generator allocation; - emits a canonical SHA-256-digested `tepp.method_effects.v1` artifact with - seed, config/content digests, original/revision/translation/template-copy - counts, and inference status + seed, config digest, a digest over exactly the cutoff-admitted documents, + original/revision/translation/template-copy counts, and inference status `simulation_method_effect_labels_not_estimator_model`; - does not invent an estimator-side method model, select GPU backends, draw MCMC, or emit topic birth/split/merge events. @@ -55,13 +57,14 @@ Operators can request cutoff-safe method-effect counts as a digest-bound terminal result. The artifact does not claim an estimator-side method model, GPU parity, MCMC, or topic birth/split/merge. Snapshot/profile/cutoff mismatch, empty/undersized available corpora, and missing originals fail -closed. +closed. Malformed digests and over-budget generation also fail closed. ## Verification The PR includes Rust unit and integration tests for digest-bound mixed method-effect corpora, empty-available and singleton-original refusal, -snapshot / profile / cutoff mismatch, and artifact tampering. Run: +snapshot / profile / cutoff mismatch, admitted-population digest binding, +pre-allocation row limits, and artifact tampering. Run: ```text cargo fmt --all -- --check diff --git a/docs/doctoring/method-effects-analysis-run.md b/docs/doctoring/method-effects-analysis-run.md index c256c30de..bb319a84c 100644 --- a/docs/doctoring/method-effects-analysis-run.md +++ b/docs/doctoring/method-effects-analysis-run.md @@ -9,6 +9,10 @@ cutoff-safe analysis-run profile so operators can request a digest-bound identity artifact. +The artifact content digest covers exactly the documents admitted at the +knowledge cutoff. A conservative 1,000,000-row execution budget is checked +before allocating the generated corpus. + The artifact inference status is `simulation_method_effect_labels_not_estimator_model`. This is not an estimator-side method model, not GPU, not MCMC, and not topic