Skip to content

docs(benchmark): document LongMemEval context_answer/answer_judge steps#332

Open
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:docs/lme-benchmark
Open

docs(benchmark): document LongMemEval context_answer/answer_judge steps#332
RerankerGuo wants to merge 2 commits into
agentscope-ai:mainfrom
RerankerGuo:docs/lme-benchmark

Conversation

@RerankerGuo

Copy link
Copy Markdown
Contributor

Summary

  • Adds end-user docs for the two LME benchmark steps that landed in feat(benchmark): add lme benchmark steps #326 (context_answer_step, answer_judge_step).
  • New pages: docs/en/lme_benchmark.md and docs/zh/lme_benchmark.md; both _toc.yml get a new 'Benchmark' section that lists lme_benchmark.
  • Documents the per-step input/output contract, the question_type to judge prompt mapping, the answer_judgement normalization rules, and a self-contained job stub plus reme app invocation.
  • Calls out that the shipped jinli_lme.yaml does not currently wire the steps into a runnable job, and shows the YAML stanza a benchmark author would add on top.

Why docs only

The runtime code and example config were already merged in #326. grep -r context_answer/answer_judge on docs/ returns nothing, so this PR closes that gap without changing code or example YAML - keeps it orthogonal to whatever follow-ups are planned for #326 / #298.

Verification

  • pre-commit run --files docs/en/lme_benchmark.md docs/zh/lme_benchmark.md docs/en/_toc.yml docs/zh/_toc.yml: all hooks green.
  • Both _toc.yml files load as valid YAML and reference lme_benchmark.

@RerankerGuo

Copy link
Copy Markdown
Contributor Author

Pushed a small follow-up commit a0cb046 on top of the existing branch:

  • docs/en/index.md and docs/zh/index.md — adds an 'LME Benchmark' card to the 'Start Reading' grid so the new page is discoverable from the docs landing without having to expand the 'Benchmark' toc section by hand.
  • docs/zh/quick_start.md — adds a one-liner right after the auto-memory/auto-resource pointer so users who follow the quick start end-to-end see the benchmark entry point as a follow-up option.

No source code or YAML touched; pre-commit run --files docs/en/index.md docs/zh/index.md docs/zh/quick_start.md stays green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant