Skip to content

fix(attestation): resync 3 missing v1.3.1 hardware rows for Chutes - #881

Open
henrypark133 wants to merge 1 commit into
mainfrom
fix/chutes-resync-v131-hardware-rows-20260711
Open

fix(attestation): resync 3 missing v1.3.1 hardware rows for Chutes#881
henrypark133 wants to merge 1 commit into
mainfrom
fix/chutes-resync-v131-hardware-rows-20260711

Conversation

@henrypark133

Copy link
Copy Markdown
Contributor

Summary

  • blackbox-attestation probe alerted probe_failed on https://cloud-api.near.ai/v1/attestation/report?model=z-ai/glm-5.2 — a genuine GLM-5.2-TEE instance on 8xb200 [10.2.1, XEON6, SNC3] was rejected with observed measurements match no accepted Chutes config.
  • Confirmed via GET https://api.chutes.ai/servers/tee/measurements: the failing row's MRTD/RTMR1/RTMR2/RTMR3 match our already-trusted v1.3.1 software identity byte-for-byte — only RTMR0 (the per-hardware register) was missing from the pinned allow-list in vetted_golden_measurements().
  • The same snapshot pull also surfaced two sibling rows from the identical v1.3.1 family that were missing: 8xRTX_PRO_6000 [10.2.1, NUMA2] and 8xRTX_PRO_6000 [10.2.1, FLAT]. Added all three per the file's own guidance to resync the full published family, not just the observed-failing row.
  • No new software identity is trusted — same MRTD/RTMR1/RTMR2/RTMR3 as the existing pinned v1.3.1 rows; this only adds hardware-variant register rows Chutes had already published for that release.

Test plan

  • cargo test --lib -p services attestation::chutes — 15/15 pass, including new accepts_the_2026_07_11_resynced_rows
  • cargo fmt --all -- --check
  • cargo clippy -p services --lib — clean
  • Confirm blackbox-attestation probe clears after deploy

🤖 Generated with Claude Code

blackbox-attestation probe alerted probe_failed: GLM-5.2-TEE on
8xb200 [10.2.1, XEON6, SNC3] failed register-pin verification with
"observed measurements match no accepted Chutes config" — the
previous snapshot never pinned this hardware row.

Pulled GET /servers/tee/measurements and confirmed the row shares
the exact MRTD/RTMR1/RTMR2/RTMR3 already trusted for the v1.3.1
family (only RTMR0, the per-hardware register, differs), along with
two sibling rows from the same pull that were also missing:
8xRTX_PRO_6000 [10.2.1, NUMA2] and 8xRTX_PRO_6000 [10.2.1, FLAT].

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings July 11, 2026 20:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds three missing v1.3.1 hardware measurement rows to the vetted golden measurements for Chutes attestation, resolving an issue where certain instances failed closed due to missing configurations. It also updates the corresponding test assertions and adds a regression test to ensure these newly added rows are correctly accepted. I have no feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@claude

claude Bot commented Jul 11, 2026

Copy link
Copy Markdown

Review — fix(attestation): resync 3 missing v1.3.1 hardware rows

Focused on the security-critical aspect: does this change trust any new software identity, or only additional per-hardware register rows?

No critical issues found. The change is correctly scoped and safe:

  • The 3 new entries are appended as (name, rtmr0) tuples inside the existing version: "1.3.1" Family, so vetted_golden_measurements() builds them via flat_map with the identical mrtd/rtmr1/rtmr2/rtmr3 as the other final-v1.3.1 rows. The data model makes it structurally impossible for these additions to introduce a new software identity — they can only enumerate new per-hardware RTMR0 boot-config values, exactly as the PR body claims. ✅
  • All three new RTMR0 values are well-formed 96-hex (48-byte) registers (verified across the whole file). ✅
  • Count assertion is arithmetically correct: 6 (v1.3.0) + 3 (v1.3.1-rc1) + 14 (v1.3.1 final) = 23, matching the updated assert_eq!(..., 23). ✅
  • New regression test accepts_the_2026_07_11_resynced_rows covers all three rows and pins them to the shared v1.3.1 identity constants. ✅
  • Fail-closed behavior is unchanged; this only widens the allow-list by the three published rows. The crypto root of trust remains the Intel DCAP signature (per the file's own documentation), so the marginal trust added is "accept the already-trusted v1.3.1 software on these additional Chutes-published hardware configs." Low risk.

Notes (non-blocking):

  • The genuineness of the RTMR0 values is inherently out-of-band (pulled from GET /servers/tee/measurements) and cannot be verified from the diff — this matches the documented resync workflow and the PR body records the byte-for-byte cross-check.
  • I could not execute cargo test in this environment (build/network sandboxed); relying on the author's reported 15/15 pass plus the structural/arithmetic verification above.

✅ Approved

@henrypark133
henrypark133 requested a review from lloydmak99 July 11, 2026 20:54

@lloydmak99 lloydmak99 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

Requesting changes: 1 High and 1 Nit across five review lenses.

The three added rows are well-formed and reuse the already trusted final-v1.3.1 MRTD/RTMR1/RTMR2/RTMR3. However, a live refresh of the public Chutes measurements on 2026-07-16 shows another final-v1.3.1 hardware row that this “full family” resync omits. Because the policy requires an exact five-register match, that genuine configuration will still fail closed with NoMatch.

Findings

  • Correctness (High): add the currently published 8xb200 [10.2.1, XEON6, 272CPU] row and regression coverage before merging.
  • Conventions (Nit): correct the regression comment so the pinned snapshot, rather than Chutes, is described as having missed the published row.

Validation

  • cargo test --lib -p services attestation::chutes: 15 passed.
  • cargo fmt --all -- --check: passed.
  • cargo clippy -p services --lib -- -D warnings: passed.
  • Security, performance/concurrency, and test-coverage lenses found no additional issues.
  • GitHub lint, unit, and integration checks are green. The E2E job is red on an unrelated admin_list_models database-connection failure and should be rerun before merge.

("8xb300", "91adf9667ba4c65bec5345a8c9b98010708d903847bf838c4526c3ebbc35561719e2127e48a3f6f77f651d71d2cbc8d4"),
// Resynced 2026-07-11 (probe_failed/blackbox-attestation incident):
// three more v1.3.1 rows Chutes had published that the previous
// snapshot missed. Same family MRTD/RTMR1/RTMR2/RTMR3 as above.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

High — the resync still omits a published v1.3.1 row. I refreshed the public Chutes measurement endpoint at review time (2026-07-16). Its final-v1.3.1 family now also contains 8xb200 [10.2.1, XEON6, 272CPU] with RTMR0 9673907ceb0c9ca79337437bb91695e7a3d19e82df1e41de1b0d2db8081fccb5d82f26d479a5016553cb20964d5948b9, under the same MRTD/RTMR1/RTMR2/RTMR3 already trusted here. This value is absent from the policy, and ChutesMeasurementPolicy::verify requires an exact RTMR0 match, so a genuine instance scheduled on that configuration will still fail with NoMatch and disable the fallback path in exactly the way this PR intends to fix. Please add this row, cover it in the regression test, and update the count assertion.

// Regression guard for the 2026-07-11 incident: GLM-5.2-TEE scheduled
// on `8xb200 [10.2.1, XEON6, SNC3]` was rejected with "observed
// measurements match no accepted Chutes config" (probe_failed on
// blackbox-attestation) because the previous snapshot never published

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: “the previous snapshot never published this row” reverses ownership: Chutes publishes the row; our pinned snapshot missed it. Please use “the previous snapshot never pinned this published row” (or “never included this published row”) so the next resync is not misled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants