Skip to content

[Docs] Document environment variables and internal-* SLURM configs in the README - #77

Open
aaronsmulktis wants to merge 6 commits into
facebookresearch:mainfrom
aaronsmulktis:aaronsmulktis/readme-env-vars
Open

aaronsmulktis wants to merge 6 commits into
facebookresearch:mainfrom
aaronsmulktis:aaronsmulktis/readme-env-vars

Conversation

@aaronsmulktis

@aaronsmulktis aaronsmulktis commented Sep 1, 2026 •

Copy link
Copy Markdown
Contributor

What

Two new README sections for those running on a cluster, plus one stale-example fix.

Environment variables

  • The .env / load_dotenv() path (both launch_agent.py and launch_parallel_agents.py call it).
  • Table of the vars actually read by configs and code: USER (W&B entity + logs_dir), GPT55_API_KEY, the AWS trio for claude_4_sonnet (client_type: aws), WANDB_API_KEY / WANDB_BASE_URL / WANDB_MODE, EXPERIMENT_CONFIG_PATH, and OPENAPPS_APP / OPENAPPS_MCP_HOST / OPENAPPS_MCP_PORT.
  • How to point an agent's api_key at whatever variable you use, via Hydra interpolation.
  • The env knobs for scripts/conduct.sh (AGENTS, COUNT, MAX_PARALLEL, HEADLESS, LOG_DIR, WANDB_GROUP) and scripts/conduct_slurm.sh (VLLM_MODEL, VLLM_PORT, VLLM_HOST).

Running on SLURM

  • config/mode/slurm_cluster.yaml ships placeholders that sbatch rejects. Documents the existing gitignored internal-* convention: copy it to config/mode/internal-slurm_cluster.yaml, fill in real paths/account/QOS, and select it with mode=internal-slurm_cluster. Includes a placeholder-only example of that file.
  • vLLM host wiring (agent.hostname / agent.port), what conduct_slurm.sh discovery does, and overriding account/QOS/partition at submit time rather than editing the #SBATCH lines.

Note: the quickstart used agent=GPT-5-1, which was renamed to the GPT-5.5-* in the configs — the command as written fails. Now documenting agent=GPT-5.5-computer-use with the matching GPT55_API_KEY.

Tests

# api_key interpolation override resolves
GPT55_API_KEY=x OPENAI_API_KEY=testkey uv run launch_agent.py \
  agent=GPT-5.5-computer-use 'agent.api_key=${oc.env:OPENAI_API_KEY}' --cfg job --resolve
#   api_key: testkey

# internal-* mode composes
uv run launch_parallel_agents.py mode=internal-slurm_cluster agent=dummy --cfg job
#   cluster: slurm, logs_dir/slurm_account from the untracked file

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Sep 1, 2026
… the README

Cloners running OpenApps on a cluster had to read the scripts to find the knobs.
Add two README sections:

- Environment variables: the .env / load_dotenv path, the vars read by configs
  and code (USER, GPT55_API_KEY, AWS_*, WANDB_*, EXPERIMENT_CONFIG_PATH,
  OPENAPPS_MCP_*), how to point an agent's api_key at any env var via Hydra
  interpolation, and the env knobs for scripts/conduct.sh and conduct_slurm.sh.
- Running on SLURM: the gitignored internal-* convention (copy
  config/mode/slurm_cluster.yaml to config/mode/internal-slurm_cluster.yaml and
  select it with mode=internal-slurm_cluster), a placeholder example of that
  file, vLLM host wiring, and submit-time SBATCH overrides.

Also fixes the stale quickstart example: agent=GPT-5-1 no longer exists (renamed
to the GPT-5.5-* configs), so the command as written fails to resolve in Hydra.

Verified: 'agent.api_key=${oc.env:OPENAI_API_KEY}' resolves via --cfg job
--resolve, and mode=internal-slurm_cluster composes.
@aaronsmulktis
aaronsmulktis force-pushed the aaronsmulktis/readme-env-vars branch from dc788e5 to 1c097f5 Compare September 1, 2026 00:38
Adds a commented .env.example covering the variables load_dotenv() actually
feeds into the configs (GPT55_API_KEY / any api_key via Hydra interpolation,
the AWS trio for Bedrock, WANDB_*), so 'cp .env.example .env' is the whole
setup step.

README follow-ups for accuracy:
- point at .env.example instead of an inline heredoc;
- state explicitly that conduct.sh / conduct_slurm.sh and the MCP server read
  their variables from the shell, not from .env -- only launch_agent.py and
  launch_parallel_agents.py call load_dotenv() -- with the 'set -a; source .env'
  idiom for people who want to keep them in one file;
- move OPENAPPS_APP / OPENAPPS_MCP_HOST / OPENAPPS_MCP_PORT into that
  shell-level table, with their defaults and CLI equivalents;
- drop the EXPERIMENT_CONFIG_PATH row: load_config() in configs.py has no
  callers, so documenting it invites people to set a variable that does
  nothing.
@aaronsmulktis aaronsmulktis self-assigned this Sep 8, 2026

@marksibrahim marksibrahim left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

nice to have: some of the details / SLURM launcher can probably go in the docs with only a brief reference in the README, but this isn't a big deal. Feel free to ignore or make the change and merge.

…nts.md

Per review: the README's 'Running on SLURM' section duplicated
docs/agents.md 'Running Evals on a Cluster'. Keep the .env table in the
README (quickstart-adjacent) and move the rest to the docs site:

- internal-* config convention + internal-slurm_cluster.yaml template
- conduct.sh / conduct_slurm.sh env-var table
- sbatch --account/--qos/--partition override at submit time

README keeps a 6-line pointer. Also documents the MCP server's env vars
where they belong (src/open_apps/mcp/README.md), noting that --app/--host/
--port always win because __main__ writes them unconditionally.

mkdocs.yml gains exclude_docs: internal-*.md — mkdocs renders every page
under docs/ whether or not it is in the nav, so a force-added internal
note would publish to GitHub Pages.
@aaronsmulktis

Copy link
Copy Markdown
Contributor Author

Looks good!

nice to have: some of the details / SLURM launcher can probably go in the docs with only a brief reference in the README, but this isn't a big deal. Feel free to ignore or make the change and merge.

Good call. Updated!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved moderate issues affect environment guidance, SLURM examples and defaults, and vLLM port wiring.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds documentation for environment variables, SLURM cluster setup, MCP configuration, and the GPT-5.5 quickstart.

Changes:

  • Documents environment variables, .env.example, and Hydra overrides.
  • Adds internal-* SLURM guidance, vLLM discovery, and submission options.
  • Clarifies MCP environment handling and MkDocs exclusions.
File summaries
File Changes and final findings
src/open_apps/mcp/README.md Documents MCP environment wiring. No final comments.
README.md Adds environment and cluster guidance. Findings: add EXPERIMENT_CONFIG_PATH (nit, 3 votes); instruct users to edit copied SLURM placeholders (moderate, 1); update stale GPT-5-1 examples in published docs (nit, 1); clarify credential exposure in logs and W&B configs (moderate, 1).
mkdocs.yml Excludes internal documentation files. No final comments.
docs/agents.md Expands SLURM and vLLM guidance. Findings: pass VLLM_PORT through to the launched agent (moderate, 1); add placeholder-edit instructions (moderate, 1); correct the .env.example completeness claim (nit, 1); fix or document the unavailable default agent (moderate, 1); show actual vLLM defaults (nit, 1).
.env.example Provides environment configuration examples. Findings: scope the automatic-loading header correctly (moderate, 3); add EXPERIMENT_CONFIG_PATH (nit, 1); add assignments or remove the sourcing alternative for shell knobs (nit, 1).
Review details

Suppressed comments (12)

.env.example:41

  • The template omits EXPERIMENT_CONFIG_PATH, although src/open_apps/configs.py:14 reads it from the environment and the agents documentation calls this file the full set of .env variables. Add an optional entry so users can configure that supported override without discovering the variable from source.
# --- Experiment tracking -----------------------------------------------------

# `wandb.entity` and `logs_dir` interpolate ${oc.env:USER}; your shell already
# sets USER, override it here only if the W&B account differs from the login.
# USER=

# WANDB_API_KEY=
# WANDB_BASE_URL=            # only for a self-hosted W&B server
# WANDB_MODE=offline         # skip online logging entirely

.env.example:26

  • AWS_REGION is listed as if it can be configured through .env, but config/agent/claude_4_sonnet.yaml hard-codes aws_region: us-west-2 and passes that value explicitly to AnthropicBedrock; changing this environment variable has no effect. Remove the setting or wire the config's region through Hydra interpolation.
# AWS_REGION=us-west-2

.env.example:37

  • load_dotenv() defaults to override=False, and USER is normally already set by the shell, so uncommenting USER= here will not override the login name as this comment promises. Document exporting USER at the call site, or change the entrypoints to load dotenv with override enabled.
# `wandb.entity` and `logs_dir` interpolate ${oc.env:USER}; your shell already
# sets USER, override it here only if the W&B account differs from the login.
# USER=

.env.example:56

  • As copied, this file contains no AGENTS, COUNT, MAX_PARALLEL, or VLLM_* assignments—only commented command examples—so sourcing .env here cannot configure those knobs. Add actual assignments for values users should keep in .env, or remove this sourcing alternative.
# ...or source this file into the shell first, if you prefer keeping them here:
#
#   set -a; source .env; set +a

README.md:122

  • After this cp, the private file still contains /example/dir and example_replace_me values, so the command immediately below is not runnable and will submit invalid SLURM settings. Add an explicit edit step (or tell the reader which fields must be replaced) before selecting the mode.
cp config/mode/slurm_cluster.yaml config/mode/internal-slurm_cluster.yaml
uv run launch_parallel_agents.py mode=internal-slurm_cluster agent=dummy \
    tasks=longer_horizon parallel_tasks.task_names=all use_wandb=True

README.md:68

  • This updates only the root README, but the linked public docs still contain the removed GPT-5-1 example in docs/index.md, docs/tasks.md, and docs/agents.md. Update those published examples too, otherwise users following the docs site still get a nonexistent agent config.
uv run launch_agent.py agent=GPT-5.5-computer-use task_name=add_meeting_with_dennis

README.md:86

  • A gitignored .env keeps credentials out of the repository, but it does not keep them out of runtime artifacts: Hydra resolves agent.api_key, save_config() writes the resolved config under logs_dir, and W&B receives the resolved config when enabled. Clarify this sentence and warn users to protect those logs/W&B configs rather than implying the keys stay out of all configs.
Copy [`.env.example`](.env.example) and fill in what you need — `launch_agent.py` and
`launch_parallel_agents.py` call `load_dotenv()`, so a `.env` at the repo root is picked up
automatically, and `.env` is git-ignored so keys stay out of the configs:

docs/agents.md:240

  • Documenting VLLM_PORT as an override is misleading: scripts/conduct_slurm.sh uses it for discovery, but forwards only agent.hostname to conduct.sh (line 105), so the launched agent still uses its configured port 8000. A non-default port can therefore be discovered successfully and then fail every request; pass agent.port=$VLLM_PORT through the wrapper as well, or remove this as a supported override.
| `VLLM_MODEL`, `VLLM_PORT` | `scripts/conduct_slurm.sh` | the `served_model_name` and port to look for |

docs/agents.md:144

  • The example is explicitly placeholder-only, but the mode-selection command follows without telling the reader to replace /your/checkpoint/path and the your_* scheduler values. Following this walkthrough as written still submits an invalid configuration; add a clear edit instruction before the command.
# config/mode/internal-slurm_cluster.yaml  (untracked)
# @package _global_

docs/agents.md:128

  • This new statement calls .env.example the full set of variables read through .env, but EXPERIMENT_CONFIG_PATH is read by src/open_apps/configs.py and is absent from that example. Add it there or narrow this sentence to the model/tracking variables that the example currently contains.
`launch_agent.py` calls `load_dotenv()`, so `.env` is picked up automatically. See
[`.env.example`](https://github.com/facebookresearch/OpenApps/blob/main/.env.example) for the
full set of variables read through `.env`.

docs/agents.md:234

  • The SLURM wrapper's default is AGENTS=gemma-4-e2b-it, but the repository has no config/agent/gemma-4-e2b-it.yaml (the available file is gemma-4-computer-use.yaml). Consequently, the documented smoke-test commands fail unless callers override AGENTS; update the wrapper/examples consistently or document the required override here.
| `AGENTS` | `scripts/conduct.sh` | `dummy` — space-separated `config/agent/<name>` stems, used round-robin |

docs/agents.md:241

  • The Default column does not give the actual defaults for these new SLURM knobs: scripts/conduct_slurm.sh uses google/gemma-4-E2B-it for VLLM_MODEL and 8000 for VLLM_PORT. Replace the descriptive text with those values so users know what discovery will probe without overrides.
| `VLLM_MODEL`, `VLLM_PORT` | `scripts/conduct_slurm.sh` | the `served_model_name` and port to look for |
| `VLLM_HOST` | `scripts/conduct_slurm.sh` | unset — pin a node to skip auto-discovery |
  • Files reviewed: 5/5 changed files
  • Comments generated: 2
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .env.example Outdated
Comment thread README.md
aaronsmulktis and others added 3 commits September 11, 2026 11:36
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…tis/OpenApps into aaronsmulktis/readme-env-vars

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants