feat(judge): materialize agent judge context

[roark47]

Summary

• Add prompt delivery for built-in CLI agents so large prompts are written to runtime-readable files and sent through stdin instead of argv.
• Add judge.context for agent_judge, including standard and minimal profiles, per-field modes, limits, attachments, context manifests, and report metadata.
• Update docs, the skill-upper eval template, and tests for the new materialized context behavior.

Why

Large agent/judge prompts could exceed OS argv limits when transcripts or workspace diffs became large. Materializing judge context and switching large prompts away from argv avoids that failure mode while keeping criteria authoring unchanged.

Validation

• make fmt
• make verify
• make test

The commit hook also ran incremental go vet, revive, and golangci-lint before commit.

[chatgpt-codex-connector]

💡 Codex Review

skill-up/.github/workflows/skill-upper-self-eval.yml

Line 93 in 271acf1

ENGINE_KEY: ${{ matrix.engine == 'qodercli' && secrets.QODER_ACCESS_TOKEN || secrets.DASHSCOPE_API_KEY }}

Skip qoder self-eval when its PAT is absent

When DASHSCOPE_API_KEY is set but QODER_ACCESS_TOKEN is not, this && || expression falls through to the DashScope key because GitHub Actions treats an empty string as falsy (per the expressions docs), so the qodercli matrix row is marked available and line 127 passes a DashScope API key as the Qoder PAT. In that environment the workflow fails instead of cleanly skipping qodercli; split the qoder secret check from the fallback value.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

This PR introduces a safer, scalable delivery path for large agent and judge prompts by materializing agent_judge review context to files and switching engine prompt transport away from argv when prompts are large.

Changes:

• Add judge.context (profiles, per-field modes, limits, attachments) and materialize judge inputs into runtime-readable artifacts referenced from a small “materials table” in the judge prompt.
• Add shared prompt delivery helper for built-in CLI engines to avoid OS ARG_MAX by sending large prompts via runtime files/stdin instead of argv.
• Update docs/templates/tests and CI workflows to reflect the new context/materialization and prompt-delivery behavior.

Reviewed changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
skills/skill-upper/assets/eval.yaml.tmpl	Update eval template comments to show agent_judge.context usage.
proposals/zh/0004-agent-judge-context-delivery.md	Add Chinese SUP-0004 proposal documenting context delivery/materialization design.
proposals/README.md	Add SUP-0004 to the proposals index.
proposals/0004-agent-judge-context-delivery.md	Add English SUP-0004 proposal documenting context delivery/materialization design.
internal/report/grading_test.go	Add test coverage for new judge_context grading metadata.
internal/report/grading.go	Include judge_context metadata in Anthropic grading output.
internal/judge/judge.go	Extend judge result schema with report-facing ContextMetadata.
internal/judge/factory_test.go	Add tests for context propagation and merge behavior.
internal/judge/factory.go	Wire JudgeContextConfig into AgentJudge and merge context config.
internal/judge/context_materializer_test.go	Add unit tests for context materialization modes, limits, and attachments.
internal/judge/context_materializer.go	Implement judge context materialization (files + manifest + runtime uploads).
internal/judge/agent_judge_test.go	Update prompt-building tests to assert file references and new metadata behavior.
internal/judge/agent_judge.go	Materialize context before prompting; render “materials table” instead of inlining huge blocks.
internal/evaluator/evaluator_test.go	Update tests to assert prompt references to materialized artifacts rather than inline diff/transcript.
internal/config/validator_test.go	Add validator tests for judge.context fields and invalid values.
internal/config/validator.go	Validate judge.context schema (profiles, modes, limits, attachments).
internal/config/schema.go	Add JudgeContextConfig/JudgeContextLimits/JudgeContextAttachment to config schema.
internal/agent/qwen_code.go	Route Qwen Code engine prompt construction through shared prompt delivery.
internal/agent/qodercli.go	Route Qoder CLI engine prompt construction through shared prompt delivery.
internal/agent/prompt_delivery_test.go	Add tests for inline-vs-file prompt delivery behavior and artifact writes.
internal/agent/prompt_delivery.go	Implement prompt delivery helper (inline vs runtime file/stdin) + metadata.
internal/agent/codex.go	Route Codex engine prompt construction through shared prompt delivery.
internal/agent/claude_code.go	Route Claude Code engine prompt construction through shared prompt delivery; refactor provider error handling.
internal/agent/agent.go	Persist prompt-delivery metadata on agent session results.
e2e/agent_test.go	Update E2E mock judge script to read workspace.diff from referenced path when present.
docs/zh/guide/writing-evals.md	Document judge.context profiles/modes/limits in Chinese guide.
docs/guide/writing-evals.md	Document judge.context profiles/modes/limits in English guide.
action/entry.sh	Add exit trap to restore workspace report ownership/permissions for Docker action runs.
action/Dockerfile	Update workflow reference comment for runner-image build.
action.yml	Update workflow reference comment for runner-image build.
CHANGELOG.md	Document default behavior change for agent_judge + new prompt delivery mechanism.
.github/workflows/skill-eval-self.yml	Make self-eval manual-only; adjust runner-image workflow reference and report ownership fix.
.github/workflows/extended-ci.yml	Add extended CI workflow covering e2e variants and release checks.
.github/workflows/ci.yml	Slim default CI, add e2e smoke suite, adjust concurrency behavior.
.github/workflows/build-runner-image.yml	Rename/extend runner image workflow to support run-scoped tags and optional latest publishing.
.github/dependabot.yml	Reduce open PR limit and group GitHub Actions updates.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[roark47]

Addressed the Codex review item about qoder self-eval secret handling in 4a767b9: the workflow now checks DASHSCOPE_API_KEY and QODER_ACCESS_TOKEN separately, skips qodercli when its PAT is absent, and passes the selected key through steps.secrets.outputs.api_key instead of relying on the && || fallback expression.

Validation: make verify and make test both passed.

[hittyt]

LGTM