[Docs] Document environment variables and internal-* SLURM configs in the README - #77
aaronsmulktis wants to merge 6 commits into
Conversation
… the README
Cloners running OpenApps on a cluster had to read the scripts to find the knobs.
Add two README sections:
- Environment variables: the .env / load_dotenv path, the vars read by configs
and code (USER, GPT55_API_KEY, AWS_*, WANDB_*, EXPERIMENT_CONFIG_PATH,
OPENAPPS_MCP_*), how to point an agent's api_key at any env var via Hydra
interpolation, and the env knobs for scripts/conduct.sh and conduct_slurm.sh.
- Running on SLURM: the gitignored internal-* convention (copy
config/mode/slurm_cluster.yaml to config/mode/internal-slurm_cluster.yaml and
select it with mode=internal-slurm_cluster), a placeholder example of that
file, vLLM host wiring, and submit-time SBATCH overrides.
Also fixes the stale quickstart example: agent=GPT-5-1 no longer exists (renamed
to the GPT-5.5-* configs), so the command as written fails to resolve in Hydra.
Verified: 'agent.api_key=${oc.env:OPENAI_API_KEY}' resolves via --cfg job
--resolve, and mode=internal-slurm_cluster composes.
dc788e5 to
1c097f5
Compare
Adds a commented .env.example covering the variables load_dotenv() actually feeds into the configs (GPT55_API_KEY / any api_key via Hydra interpolation, the AWS trio for Bedrock, WANDB_*), so 'cp .env.example .env' is the whole setup step. README follow-ups for accuracy: - point at .env.example instead of an inline heredoc; - state explicitly that conduct.sh / conduct_slurm.sh and the MCP server read their variables from the shell, not from .env -- only launch_agent.py and launch_parallel_agents.py call load_dotenv() -- with the 'set -a; source .env' idiom for people who want to keep them in one file; - move OPENAPPS_APP / OPENAPPS_MCP_HOST / OPENAPPS_MCP_PORT into that shell-level table, with their defaults and CLI equivalents; - drop the EXPERIMENT_CONFIG_PATH row: load_config() in configs.py has no callers, so documenting it invites people to set a variable that does nothing.
marksibrahim
left a comment
There was a problem hiding this comment.
Looks good!
nice to have: some of the details / SLURM launcher can probably go in the docs with only a brief reference in the README, but this isn't a big deal. Feel free to ignore or make the change and merge.
…nts.md Per review: the README's 'Running on SLURM' section duplicated docs/agents.md 'Running Evals on a Cluster'. Keep the .env table in the README (quickstart-adjacent) and move the rest to the docs site: - internal-* config convention + internal-slurm_cluster.yaml template - conduct.sh / conduct_slurm.sh env-var table - sbatch --account/--qos/--partition override at submit time README keeps a 6-line pointer. Also documents the MCP server's env vars where they belong (src/open_apps/mcp/README.md), noting that --app/--host/ --port always win because __main__ writes them unconditionally. mkdocs.yml gains exclude_docs: internal-*.md — mkdocs renders every page under docs/ whether or not it is in the nav, so a force-added internal note would publish to GitHub Pages.
Good call. Updated! |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved moderate issues affect environment guidance, SLURM examples and defaults, and vLLM port wiring.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds documentation for environment variables, SLURM cluster setup, MCP configuration, and the GPT-5.5 quickstart.
Changes:
- Documents environment variables,
.env.example, and Hydra overrides. - Adds
internal-*SLURM guidance, vLLM discovery, and submission options. - Clarifies MCP environment handling and MkDocs exclusions.
File summaries
| File | Changes and final findings |
|---|---|
src/open_apps/mcp/README.md |
Documents MCP environment wiring. No final comments. |
README.md |
Adds environment and cluster guidance. Findings: add EXPERIMENT_CONFIG_PATH (nit, 3 votes); instruct users to edit copied SLURM placeholders (moderate, 1); update stale GPT-5-1 examples in published docs (nit, 1); clarify credential exposure in logs and W&B configs (moderate, 1). |
mkdocs.yml |
Excludes internal documentation files. No final comments. |
docs/agents.md |
Expands SLURM and vLLM guidance. Findings: pass VLLM_PORT through to the launched agent (moderate, 1); add placeholder-edit instructions (moderate, 1); correct the .env.example completeness claim (nit, 1); fix or document the unavailable default agent (moderate, 1); show actual vLLM defaults (nit, 1). |
.env.example |
Provides environment configuration examples. Findings: scope the automatic-loading header correctly (moderate, 3); add EXPERIMENT_CONFIG_PATH (nit, 1); add assignments or remove the sourcing alternative for shell knobs (nit, 1). |
Review details
Suppressed comments (12)
.env.example:41
- The template omits
EXPERIMENT_CONFIG_PATH, althoughsrc/open_apps/configs.py:14reads it from the environment and the agents documentation calls this file the full set of.envvariables. Add an optional entry so users can configure that supported override without discovering the variable from source.
# --- Experiment tracking -----------------------------------------------------
# `wandb.entity` and `logs_dir` interpolate ${oc.env:USER}; your shell already
# sets USER, override it here only if the W&B account differs from the login.
# USER=
# WANDB_API_KEY=
# WANDB_BASE_URL= # only for a self-hosted W&B server
# WANDB_MODE=offline # skip online logging entirely
.env.example:26
AWS_REGIONis listed as if it can be configured through.env, butconfig/agent/claude_4_sonnet.yamlhard-codesaws_region: us-west-2and passes that value explicitly toAnthropicBedrock; changing this environment variable has no effect. Remove the setting or wire the config's region through Hydra interpolation.
# AWS_REGION=us-west-2
.env.example:37
load_dotenv()defaults tooverride=False, andUSERis normally already set by the shell, so uncommentingUSER=here will not override the login name as this comment promises. Document exportingUSERat the call site, or change the entrypoints to load dotenv with override enabled.
# `wandb.entity` and `logs_dir` interpolate ${oc.env:USER}; your shell already
# sets USER, override it here only if the W&B account differs from the login.
# USER=
.env.example:56
- As copied, this file contains no
AGENTS,COUNT,MAX_PARALLEL, orVLLM_*assignments—only commented command examples—so sourcing.envhere cannot configure those knobs. Add actual assignments for values users should keep in.env, or remove this sourcing alternative.
# ...or source this file into the shell first, if you prefer keeping them here:
#
# set -a; source .env; set +a
README.md:122
- After this
cp, the private file still contains/example/dirandexample_replace_mevalues, so the command immediately below is not runnable and will submit invalid SLURM settings. Add an explicit edit step (or tell the reader which fields must be replaced) before selecting the mode.
cp config/mode/slurm_cluster.yaml config/mode/internal-slurm_cluster.yaml
uv run launch_parallel_agents.py mode=internal-slurm_cluster agent=dummy \
tasks=longer_horizon parallel_tasks.task_names=all use_wandb=True
README.md:68
- This updates only the root README, but the linked public docs still contain the removed
GPT-5-1example indocs/index.md,docs/tasks.md, anddocs/agents.md. Update those published examples too, otherwise users following the docs site still get a nonexistent agent config.
uv run launch_agent.py agent=GPT-5.5-computer-use task_name=add_meeting_with_dennis
README.md:86
- A gitignored
.envkeeps credentials out of the repository, but it does not keep them out of runtime artifacts: Hydra resolvesagent.api_key,save_config()writes the resolved config underlogs_dir, and W&B receives the resolved config when enabled. Clarify this sentence and warn users to protect those logs/W&B configs rather than implying the keys stay out of all configs.
Copy [`.env.example`](.env.example) and fill in what you need — `launch_agent.py` and
`launch_parallel_agents.py` call `load_dotenv()`, so a `.env` at the repo root is picked up
automatically, and `.env` is git-ignored so keys stay out of the configs:
docs/agents.md:240
- Documenting
VLLM_PORTas an override is misleading:scripts/conduct_slurm.shuses it for discovery, but forwards onlyagent.hostnametoconduct.sh(line 105), so the launched agent still uses its configured port8000. A non-default port can therefore be discovered successfully and then fail every request; passagent.port=$VLLM_PORTthrough the wrapper as well, or remove this as a supported override.
| `VLLM_MODEL`, `VLLM_PORT` | `scripts/conduct_slurm.sh` | the `served_model_name` and port to look for |
docs/agents.md:144
- The example is explicitly placeholder-only, but the mode-selection command follows without telling the reader to replace
/your/checkpoint/pathand theyour_*scheduler values. Following this walkthrough as written still submits an invalid configuration; add a clear edit instruction before the command.
# config/mode/internal-slurm_cluster.yaml (untracked)
# @package _global_
docs/agents.md:128
- This new statement calls
.env.examplethe full set of variables read through.env, butEXPERIMENT_CONFIG_PATHis read bysrc/open_apps/configs.pyand is absent from that example. Add it there or narrow this sentence to the model/tracking variables that the example currently contains.
`launch_agent.py` calls `load_dotenv()`, so `.env` is picked up automatically. See
[`.env.example`](https://github.com/facebookresearch/OpenApps/blob/main/.env.example) for the
full set of variables read through `.env`.
docs/agents.md:234
- The SLURM wrapper's default is
AGENTS=gemma-4-e2b-it, but the repository has noconfig/agent/gemma-4-e2b-it.yaml(the available file isgemma-4-computer-use.yaml). Consequently, the documented smoke-test commands fail unless callers overrideAGENTS; update the wrapper/examples consistently or document the required override here.
| `AGENTS` | `scripts/conduct.sh` | `dummy` — space-separated `config/agent/<name>` stems, used round-robin |
docs/agents.md:241
- The
Defaultcolumn does not give the actual defaults for these new SLURM knobs:scripts/conduct_slurm.shusesgoogle/gemma-4-E2B-itforVLLM_MODELand8000forVLLM_PORT. Replace the descriptive text with those values so users know what discovery will probe without overrides.
| `VLLM_MODEL`, `VLLM_PORT` | `scripts/conduct_slurm.sh` | the `served_model_name` and port to look for |
| `VLLM_HOST` | `scripts/conduct_slurm.sh` | unset — pin a node to skip auto-discovery |
- Files reviewed: 5/5 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…tis/OpenApps into aaronsmulktis/readme-env-vars
What
Two new README sections for those running on a cluster, plus one stale-example fix.
Environment variables
.env/load_dotenv()path (bothlaunch_agent.pyandlaunch_parallel_agents.pycall it).USER(W&Bentity+logs_dir),GPT55_API_KEY, the AWS trio forclaude_4_sonnet(client_type: aws),WANDB_API_KEY/WANDB_BASE_URL/WANDB_MODE,EXPERIMENT_CONFIG_PATH, andOPENAPPS_APP/OPENAPPS_MCP_HOST/OPENAPPS_MCP_PORT.api_keyat whatever variable you use, via Hydra interpolation.scripts/conduct.sh(AGENTS,COUNT,MAX_PARALLEL,HEADLESS,LOG_DIR,WANDB_GROUP) andscripts/conduct_slurm.sh(VLLM_MODEL,VLLM_PORT,VLLM_HOST).Running on SLURM
config/mode/slurm_cluster.yamlships placeholders thatsbatchrejects. Documents the existing gitignoredinternal-*convention: copy it toconfig/mode/internal-slurm_cluster.yaml, fill in real paths/account/QOS, and select it withmode=internal-slurm_cluster. Includes a placeholder-only example of that file.agent.hostname/agent.port), whatconduct_slurm.shdiscovery does, and overriding account/QOS/partition at submit time rather than editing the#SBATCHlines.Note: the quickstart used
agent=GPT-5-1, which was renamed to theGPT-5.5-*in the configs — the command as written fails. Now documentingagent=GPT-5.5-computer-usewith the matchingGPT55_API_KEY.Tests