Skip to content

Fix hidden PDDL grader dependencies#16

Open
parkavenue9639 wants to merge 3 commits into
agentscope-ai:mainfrom
parkavenue9639:codex/fix-pddl-grader-dependencies
Open

Fix hidden PDDL grader dependencies#16
parkavenue9639 wants to merge 3 commits into
agentscope-ai:mainfrom
parkavenue9639:codex/fix-pddl-grader-dependencies

Conversation

@parkavenue9639

Copy link
Copy Markdown
Contributor

Summary

  • Declare T136's unified-planning and up-pyperplan dependencies in the top-level grader requirements.
  • Pin planner versions so Docker-produced plan pickles and host-side grading use compatible implementations.
  • Check installed dependencies before attempting the T136 compatibility install.
  • Add regression coverage for embedded grader dependencies and a real pyperplan solve using the bundled T136 fixture.

Problem

T136 is marked as a closed task with external_dependency: none, but its embedded grader previously depended on a runtime PyPI installation:

  • unified-planning
  • up-pyperplan

These packages were missing from the top-level requirements.txt. If the dynamic installation failed because of network, permissions, or environment restrictions, the checker returned all-zero automated scores. This conflated grader setup failures with agent performance.

The supplied solve.py also uses these packages inside the agent runtime. All PawBench Dockerfiles install the top-level requirements, so declaring them there keeps the runtime and host grader environments aligned.

Changes

  • Add:
    • unified-planning==1.3.0
    • up-pyperplan==1.1.0
  • Change T136 grading to import-check dependencies first and only use dynamic installation as a compatibility fallback.
  • Add a test that:
    • Ensures packages explicitly installed by embedded graders are declared in requirements.txt.
    • Verifies pyperplan can solve the bundled task01.pddl fixture.

Validation

ruff check tests/test_grader_dependencies.py
PYTHONPATH=. python -m pytest tests -q

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant