Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Copy to `.env` and fill in the values you need:
#
# cp .env.example .env
#
# `.env` is git-ignored. `launch_agent.py` and `launch_parallel_agents.py` call
# `load_dotenv()`, so the model-provider and experiment-tracking variables below
# are picked up automatically by those two entrypoints (including Hydra `${oc.env:...}` interpolation in the configs).


# --- Model providers ---------------------------------------------------------

# Key for the OpenAI-compatible endpoint used by config/agent/GPT-5.5-*.yaml
# (`api_key: ${oc.env:GPT55_API_KEY}`).
GPT55_API_KEY=

# Any variable name works -- point an agent's `api_key` at the one you use:
# uv run launch_agent.py agent=GPT-5.5-computer-use \
# 'agent.api_key=${oc.env:OPENAI_API_KEY}'
# OPENAI_API_KEY=

# Bedrock, for config/agent/claude_4_sonnet.yaml (`client_type: aws`). The
# config leaves the credentials null, so the Anthropic SDK falls back to these.
# AWS_ACCESS_KEY_ID=
# AWS_SECRET_ACCESS_KEY=
# AWS_SESSION_TOKEN=
# AWS_REGION=us-west-2

# Self-hosted models need no key -- point the agent at the serving host instead:
# uv run launch_agent.py agent=Qwen3.6-27B-computer-use \
# agent.hostname=<node> agent.port=8000


# --- Experiment tracking -----------------------------------------------------

# `wandb.entity` and `logs_dir` interpolate ${oc.env:USER}; your shell already
# sets USER, override it here only if the W&B account differs from the login.
# USER=

# WANDB_API_KEY=
# WANDB_BASE_URL= # only for a self-hosted W&B server
# WANDB_MODE=offline # skip online logging entirely


# --- Shell-level variables ---------------------------------------------------
#
# NOT read from this file: `scripts/conduct.sh` and `scripts/conduct_slurm.sh`
# are shell scripts, and `src/open_apps/mcp/` is launched separately, so none of
# them go through `load_dotenv()`. Export them at the call site:
#
# AGENTS="dummy" COUNT=4 MAX_PARALLEL=2 ./scripts/conduct.sh
# VLLM_HOST=<node> AGENTS=gemma-4-computer-use COUNT=1 sbatch scripts/conduct_slurm.sh
# uv run python -m open_apps.mcp --app todo --host 127.0.0.1 --port 8000
#
# ...or source this file into the shell first, if you prefer keeping them here:
#
# set -a; source .env; set +a
63 changes: 55 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<div align="center">

# <img width="45" height="45" alt="image" src="https://github.com/user-attachments/assets/6c409d42-6f3a-4a62-be7f-57793d9dad9d" /> OpenApps

*Building Blocks for Computer-Use Agents Research*

🏆 ICLR Oral, Top 1%
Expand Down Expand Up @@ -41,7 +41,7 @@ see [docs](https://facebookresearch.github.io/OpenApps/) for details.
Simply run:

```bash
uv run launch.py
uv run launch.py
```
<img width="1440" height="822" alt="image" src="https://github.com/user-attachments/assets/46024c36-9f6d-462b-acb7-b6c148ed1754" />

Expand All @@ -52,7 +52,7 @@ Each app can be modified with variables available in `config/apps`. You can over
uv run launch.py app.todo.title='Super Todo'
```

Learn more about to customize the content and appearance of apps in the [docs](https://facebookresearch.github.io/OpenApps/).
Learn more about to customize the content and appearance of apps in the [docs](https://facebookresearch.github.io/OpenApps/).



Expand All @@ -64,8 +64,8 @@ Launch an agent to perform a task of *adding a meeting with Dennis to the calend


```
# export OPENAI_API_KEY=""
uv run launch_agent.py agent=GPT-5-1 task_name=add_meeting_with_dennis
# export GPT55_API_KEY=""
uv run launch_agent.py agent=GPT-5.5-computer-use task_name=add_meeting_with_dennis
```

To see the agent solving the task live, add the headless argument:
Expand All @@ -79,6 +79,53 @@ You can specify the agent of your choice with the `agent=` argument. For example

Learn more about launching with OpenAI, Claude, and VLLM models such as UI-Tars in our [docs](https://facebookresearch.github.io/OpenApps/).

## Environment variables

Copy [`.env.example`](.env.example) and fill in what you need — `launch_agent.py` and
`launch_parallel_agents.py` call `load_dotenv()`, so a `.env` at the repo root is picked up
automatically, and `.env` is git-ignored so keys stay out of the configs:

```bash
cp .env.example .env
```

| Variable | Read by | Purpose |
| --- | --- | --- |
| `USER` | `config/config*.yaml`, `config/mode/*` | W&B `entity` and the `logs_dir` path |
| `GPT55_API_KEY` | `config/agent/GPT-5.5-*.yaml` | key for the OpenAI-compatible endpoint |
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN` | `config/agent/claude_4_sonnet.yaml` (`client_type: aws`) | Bedrock credentials, when left null in the config |
| `WANDB_API_KEY`, `WANDB_BASE_URL`, `WANDB_MODE` | `wandb` | auth, self-hosted server, and `WANDB_MODE=offline` to skip online logging |
| `EXPERIMENT_CONFIG_PATH` | `src/open_apps/configs.py` | optional path loaded by `load_config()` instead of the default config |

Agent API keys are read through Hydra interpolation, so any variable name works — point the
agent's `api_key` at the one you use:

```bash
uv run launch_agent.py agent=GPT-5.5-computer-use 'agent.api_key=${oc.env:OPENAI_API_KEY}'
```

The batch scripts take environment variables too (`AGENTS`, `COUNT`, `MAX_PARALLEL`,
`VLLM_HOST`, …), but they are read by the shell, **not** through `.env` — export them at the
call site, or `set -a; source .env; set +a` first. They are listed in the
[agents docs](https://facebookresearch.github.io/OpenApps/agents/); the MCP server's
variables are in [`src/open_apps/mcp/README.md`](src/open_apps/mcp/README.md).

## Running on a cluster

`config/mode/slurm_cluster.yaml` and the `#SBATCH` lines in `scripts/conduct_slurm.sh` ship
with placeholder accounts and paths that `sbatch` will reject. Copy the mode to an
`internal-` twin — `.gitignore` keeps any `internal-*` file untracked, so your site's paths
and account names can't be committed by accident:

```bash
cp config/mode/slurm_cluster.yaml config/mode/internal-slurm_cluster.yaml
uv run launch_parallel_agents.py mode=internal-slurm_cluster agent=dummy \
tasks=longer_horizon parallel_tasks.task_names=all use_wandb=True
```

See the [agents docs](https://facebookresearch.github.io/OpenApps/agents/) for the full
SLURM + vLLM + W&B walkthrough.

## OpenApps in action


Expand All @@ -103,7 +150,7 @@ To build docs:
```
mkdocs build
mkdocs serve
```
```

this will launch docs available at https://facebookresearch.github.io/OpenApps/

Expand All @@ -121,13 +168,13 @@ uv run -m pytest tests/

## Attribution

Our apps are built on top of several excellent frameworks:
Our apps are built on top of several excellent frameworks:

- FastHTML [framework](https://github.com/AnswerDotAI/fasthtml) and [examples](https://github.com/AnswerDotAI/fasthtml-example) which allowed us to build fully functional apps in Python, the language most familiar to AI researchers.
- [Browser Gym](https://github.com/ServiceNow/BrowserGym/blob/main/LICENSE) and [AgentLab](https://github.com/ServiceNow/AgentLab/blob/main/LICENSE):
- [Spacy](https://github.com/innoq/spacy/blob/main/LICENSE): for natural language processing
- Open Street Maps: https://www.openstreetmap.org/copyright for our Maps apps.
- (and for the optional webshop) we rely on [WebShop](https://github.com/princeton-nlp/WebShop/blob/master/LICENSE.md) developed by Princeton
- (and for the optional webshop) we rely on [WebShop](https://github.com/princeton-nlp/WebShop/blob/master/LICENSE.md) developed by Princeton

Some icons are have been designed using resources from Flaticon.com

Expand Down
78 changes: 74 additions & 4 deletions docs/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,55 @@ WANDB_API_KEY=...
EOF
```

`launch_agent.py` calls `load_dotenv()`, so `.env` is picked up automatically.
`launch_agent.py` calls `load_dotenv()`, so `.env` is picked up automatically. See
[`.env.example`](https://github.com/facebookresearch/OpenApps/blob/main/.env.example) for the
full set of variables read through `.env`.

### Cluster config: the `internal-*` convention

`config/mode/slurm_cluster.yaml` ships with placeholder values (`logs_dir: /example/dir`,
`slurm_account: example_replace_me`, …) that `sbatch` will reject. Rather than editing it and
risking committing your site's paths and account names, `.gitignore` carries an `internal-*`
rule: **any file named `internal-*` stays untracked**. The convention is to keep a private
twin next to the public one:

```bash
cp config/mode/slurm_cluster.yaml config/mode/internal-slurm_cluster.yaml
```

```yaml
# config/mode/internal-slurm_cluster.yaml (untracked)
# @package _global_
project: open_apps

logs_dir: /your/checkpoint/path/${oc.env:USER}/logs/${project}/${now:%Y-%m-%d_%H-%M-%S}-${agent.model_name}/${job_id}
databases_dir: ${logs_dir}/databases

cluster: slurm

slurm_sweep_launcher:
gpus_per_node: 0
nodes: 1
tasks_per_node: 1
cpus_per_task: 2
timeout_min: 400
slurm_account: your_account
slurm_qos: your_qos
slurm_partition: your_partition
mem_gb: 10
slurm_srun_args: ["-vv", "--cpu-bind", "none"]
slurm_comment: "parallel agent tasks"
```

Select it like any other Hydra mode:

```bash
uv run launch_parallel_agents.py mode=internal-slurm_cluster agent=dummy \
tasks=longer_horizon parallel_tasks.task_names=all use_wandb=True
```

The same pattern applies elsewhere — e.g. `docs/internal-notes.md` for cluster-specific
instructions alongside the public `docs/`.

### 1. Launch the persistent vLLM serve job

Expand Down Expand Up @@ -168,9 +216,31 @@ model*, so it doesn't matter how the vLLM job was started (e.g. a `bash`-named
VLLM_HOST=example_host AGENTS=gemma-4-e2b-it COUNT=1 sbatch scripts/conduct_slurm.sh
```

Other env overrides: `VLLM_MODEL`, `VLLM_PORT`, and `WANDB_MODE` (set
`WANDB_MODE=offline` to skip online logging). Extra CLI args are forwarded
verbatim to `launch_agent.py` as Hydra overrides.
**Account/QOS/partition:** the `#SBATCH` lines in `scripts/conduct_slurm.sh` are
placeholders. Override them at submit time rather than editing the script — the
command line takes precedence:

```bash
AGENTS=gemma-4-e2b-it COUNT=1 \
sbatch --account=... --qos=... --partition=... scripts/conduct_slurm.sh
```

The worker pool and its wrapper are configured entirely through the environment.
These are read by the shell, **not** through `.env` — export them at the call site,
or `set -a; source .env; set +a` first:

| Variable | Read by | Default |
| --- | --- | --- |
| `AGENTS` | `scripts/conduct.sh` | `dummy` — space-separated `config/agent/<name>` stems, used round-robin |
| `COUNT` | `scripts/conduct.sh` | number of agents — total runs to launch |
| `MAX_PARALLEL` | `scripts/conduct.sh` | `4` — concurrent runs |
| `HEADLESS` | `scripts/conduct.sh` | `True` |
| `LOG_DIR`, `WANDB_GROUP` | `scripts/conduct.sh` | `log_outputs`, `batch-<timestamp>` |
| `WANDB_MODE` | `wandb` | unset — `offline` skips online logging |
| `VLLM_MODEL`, `VLLM_PORT` | `scripts/conduct_slurm.sh` | the `served_model_name` and port to look for |
| `VLLM_HOST` | `scripts/conduct_slurm.sh` | unset — pin a node to skip auto-discovery |

Extra CLI args are forwarded verbatim to `launch_agent.py` as Hydra overrides.

**Fallback if compute→compute `:8000` is firewalled:** run the eval inside the
vLLM job's own allocation and talk to it over localhost:
Expand Down
6 changes: 6 additions & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@ theme:
site_url: https://facebookresearch.github.io/OpenApps/
repo_url: https://github.com/facebookresearch/OpenApps

# mkdocs renders every page under docs/, nav or not. `internal-*` files are
# git-ignored so CI never checks them out, but exclude them here too — a
# force-added one would otherwise publish straight to GitHub Pages.
exclude_docs: |
internal-*.md

nav:
- Start: index.md
- Manual Installation: installation.md
Expand Down
7 changes: 7 additions & 0 deletions src/open_apps/mcp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,13 @@ uv run python -m open_apps.mcp --app todo --transport http --host 127.0.0.1 --po
Flags: `--app {todo,calendar,messages,map,codeeditor}` (default `todo`),
`--transport {stdio,http,sse}` (default `stdio`), `--host`, `--port`.

`__main__` publishes those flags as `OPENAPPS_APP`, `OPENAPPS_MCP_HOST` and
`OPENAPPS_MCP_PORT` before importing the server, which is where `server.py`
reads them (defaults `todo`, `127.0.0.1`, `8000`). Because it writes them
unconditionally, the flags always win over anything already in the environment —
set these variables only when embedding `open_apps.mcp.server` directly rather
than launching via `python -m`.

On startup you'll see the apps initialize ("Setting environment for ...");
the server is then ready for tool calls.

Expand Down
Loading