ZJLi2013 · Oct 20, 2026 · Oct 20, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -13,12 +13,14 @@ All notable user-facing changes are documented here. This project follows Semant
 - Contribution guide, code of conduct, security policy, AI contribution policy, and issue / PR templates.
 - `scripts/assets/import_objaverse.py`, which produced the Objaverse assets in `assets/objects/`.
 - `scripts/train_smolvla.py` for SmolVLA post-training on a recorded dataset, and a `lerobot` extra.
+- `docs/design.md` (design and key concepts) and `docs/quick-start.md` (fresh GPU node to a first recorded run).
 
 ### Changed
 
 - `scripts/setup/run_scenario.sh` takes host mounts from `REPO`, `GRASPGEN_DIR`, `GRASPGEN_MODELS`, and
   `HF_CACHE`.
 - The `pick_place_into_drawer` camera looks at the drawer opening.
+- `docker/README.md` builds from `docker/Dockerfile.gfx942` / `docker/Dockerfile.gfx950`.
 
 ### Known issues
 

diff --git a/README-en.md b/README-en.md
@@ -55,6 +55,9 @@ Example scenario: [`scenarios/pick_place_into_drawer.py`](scenarios/pick_place_i
 
 ## Quick start
 
+Step-by-step setup from a fresh GPU node (image, GraspGen weights, rocRobo sidecar, first run):
+[`docs/quick-start.md`](docs/quick-start.md). Design and key concepts: [`docs/design.md`](docs/design.md).
+
 ```bash
 pip install -e .
 

diff --git a/README.md b/README.md
@@ -55,6 +55,9 @@ SDG 的价值直接:在仿真里制造演示,物理 / 随机化 / 标签全部
 
 ## 快速开始
 
+从一台全新 GPU 节点跑通(镜像、GraspGen 权重、rocRobo sidecar、首次运行)见 [`docs/quick-start.md`](docs/quick-start.md);
+设计与关键概念见 [`docs/design.md`](docs/design.md)。
+
 ```bash
 pip install -e .
 

diff --git a/docker/Dockerfile.gfx950 b/docker/Dockerfile.gfx950
@@ -3,7 +3,7 @@
 # RoboSmith ROCm runtime.
 #
 # Build from the repository root:
-#   docker build -f docker/Dockerfile -t robotsmith:rocm-headless .
+#   docker build -f docker/Dockerfile.gfx950 -t robotsmith:rocm-headless .
 #
 # The important invariant is headless rendering through Xvfb/GLX.  Do not set
 # PYOPENGL_PLATFORM=egl for Genesis runs; EGL has been brittle on the cluster.

diff --git a/docker/README.md b/docker/README.md
@@ -1,30 +1,20 @@
 # RoboSmith Docker
 
-Build from the repository root:
+One image per GPU architecture. Build from the repository root:
 
-```bash
-docker build -f docker/Dockerfile -t robotsmith:rocm-headless .
-```
+| GPU | Command |
+|---|---|
+| MI300 (gfx942) | `docker build -f docker/Dockerfile.gfx942 -t robotsmith:gfx942-rocm6.4.3-genesis0.4.5 .` |
+| MI350 (gfx950) | `docker build -f docker/Dockerfile.gfx950 -t robotsmith:rocm-headless .` |
 
-Run on an AMD GPU node:
-
-```bash
-docker run --rm -it \
-  --device=/dev/kfd --device=/dev/dri --group-add video \
-  --ipc=host --shm-size=24g \
-  -e HIP_VISIBLE_DEVICES=0 \
-  -v "$PWD":/workspace/robotsmith \
-  -v /data/GraspGen:/workspace/GraspGen:ro \
-  -v /data/GraspGenModels:/workspace/GraspGenModels:ro \
-  robotsmith:rocm-headless \
-  bash
-```
+The gfx942 image includes the GraspGen dependencies; the gfx950 image needs the GraspGen layer described in
+[`docs/quick-start.md`](../docs/quick-start.md), which also covers weights, the rocRobo sidecar, and the first run.
 
 Headless rendering policy:
 
-- Use `Xvfb`/GLX for Genesis camera rendering.
+- Use `Xvfb`/GLX for Genesis camera rendering; the entrypoint starts Xvfb.
 - Do not set `PYOPENGL_PLATFORM=egl`.
-- Do not set `HSA_OVERRIDE_GFX_VERSION` unless a specific node requires it and the override has been validated.
+- The gfx942 image keeps `HSA_OVERRIDE_GFX_VERSION=9.4.2` set; the gfx950 image unsets it.
 - Keep large asset meshes outside git; copy or mount them into `assets/objects/...` when running experiments.
 
 For GraspGen on ROCm, install `spconv_rocm` from the ROCm branch and use the repo `torch_scatter` shim rather than PyG ROCm wheels.
diff --git a/docs/design.md b/docs/design.md
@@ -0,0 +1,155 @@
+# RoboSmith design
+
+RoboSmith turns **assets plus a declarative task** into **successful expert episodes recorded as a LeRobot
+dataset**. You write *what* should happen; RoboSmith decides *how*: it builds the scene, plans a grasp for every
+episode, generates a joint trajectory, rolls it out in Genesis, and keeps the success verdict next to the data.
+
+It is data infrastructure, not a policy-training framework. The one training entry, `scripts/train_smolvla.py`,
+consumes the dataset RoboSmith writes.
+
+## Data flow
+
+```text
+asset library ──► scenario (CAP: scene + task + skill intents + success)
+                      │  cap/adapters.py lowers it to the runtime
+                      ▼
+                  SimEnv (Genesis + Franka) ── reset, settle
+                      │
+     per skill:   grasp planner ──► GraspPlan ──► MotionExecutor ──► MotionPlanner (Genesis IK or rocRobo)
+                      │
+                      ▼
+                  rollout + success tree ──► LeRobot dataset, videos, run summary
+```
+
+Each arrow is one typed contract, and each layer has one owner in the code:
+
+| Contract | Type / module |
+|---|---|
+| Asset → scene | `AssetMetadata` (`robotsmith/assets/`) |
+| Scenario → runtime | `robotsmith/cap/adapters.py` |
+| Grasp → execution | `GraspPlan`, including its waypoints |
+| Execution → motion backend | the `MotionPlanner` protocol (`robotsmith/motion/`) |
+| Simulator / recorder | `robotsmith/sim/sim_env.py`, `robotsmith/sim/recorder.py` (the only adapters) |
+
+## Key concepts
+
+### Asset and affordance
+
+An asset is a directory `assets/objects/<name>/` (`model.urdf`, `metadata.json`, `visual.*`, `collision.obj`,
+`provenance.json`); articulated assets live in `assets/articraft/<name>/`. Assets come from
+[RocRecon](https://github.com/ZJLi2013/RocRecon) or `scripts/assets/import_objaverse.py`.
+
+What an asset can do follows from its metadata, not from its category:
+
+| Asset kind | Unlocked by | Actions | Success predicates |
+|---|---|---|---|
+| rigid | a graspable mesh with a verified upright pose and a metric scale | `pick`, `place` | `object_in_container`, `object_above`, `stacked`, `objects_aligned` |
+| articulated | `task_joints` + `handles` | `open`, `close` | `joint_opened`, `joint_closed` |
+| supporter / fixture | `role="fixture"` or any movable joint | none (it is a place target or an obstacle) | — |
+
+Three metadata fields carry most of the physics:
+
+- **`metric_scale`** is the single owner of scale. Point clouds, grasp gates, and support clearances are all computed
+  in metres from it; a wrong scale makes all of them wrong without an error.
+- **Frames.** Mesh frame → object frame (canonical, `+Z` up) → scene frame. `canonical_frame` is the import
+  correction; `task_poses.upright` is the pose used at runtime and is written only after human visual QA
+  (`scripts/assets/render_asset_table.py`, `verified=true`); `stable_poses` only says the object rests stably.
+- **Gate.** An asset enters default experiments only with `upright.verified=true` and an empty
+  `metadata.mesh.issues`. The audit includes an 8 cm gripper-width check and recommends a `metric_scale`. Fixtures are
+  exempt: they load with `fixed=True`, are never grasped, and use their bounding box for placement collision.
+
+Articulated assets keep their kinematics in the URDF. Metadata only adds task semantics: `task_joints`, `handles`,
+`joint_init`, and `place_targets`.
+
+### Scenario and CAP
+
+A scenario is `scene + task + skill intents + success`, written in Python with the CAP API
+(`robotsmith/cap/`: `SceneSpec`, `RegionSpec`, `ObjectSpec`, `TargetPositionSpec`, `TaskSpec`; helpers in
+`cap/scene_api.py` and `cap/task_api.py`). Examples live in [`scenarios/`](../scenarios/); the authoring contract for
+agents is [`scenarios/AGENT_PROMPT.md`](../scenarios/AGENT_PROMPT.md).
+
+- **Skill intents** are limited to `pick`, `place`, `open`, `close`. A scenario never writes waypoints, controls the
+  gripper, or calls Genesis.
+- **Success** is a tree of `all_of` / `any_of` / `negate` over predicates. A new predicate is one function in
+  `cap/predicates.py` plus its registration.
+- **Target positions** are named points, not physical objects; place targets on an articulated fixture
+  (`on_articulated_opening(...)`) follow the joint as it opens.
+- `ObjectSpec.joint_init` sets a scene-level initial joint value that overrides the asset's.
+- Scenario loading executes Python, so only load trusted scenario files.
+
+### Grasp: generate, then only subtract
+
+Each episode generates candidates on the object as it actually settled, then filters them down to one. Later steps
+never add candidates back:
+
+1. **Generate**: GraspGen diffusion on a 2000-point cloud in metres (`grasp/graspgen_wrapper.py`).
+2. **Support clearance**: drop grasps whose gripper mesh comes within 1 cm of the object's own support surface
+   (`grasp/collision_screen.py::support_clearance_mask`). This runs before top-k.
+3. **Geometric rejection**: the grasp point must sit above the support surface, the approach must not come from
+   below; keep the top `GRASP_TOP_K` (100).
+4. **Expand to waypoints**: pre-grasp (backed off along the approach axis) → grasp → close → retreat.
+5. **Feasibility gate** (`grasp/feasibility.py`): the fingers must land in the support band and IK must succeed on
+   every segment without a joint jump above 0.8 rad. This gate does not check collisions.
+6. **Select**: the best-scoring survivor, or — when the asset's metadata names a `grasp_policy_bucket` — the best
+   survivor in that bucket. If nothing survives, the episode records `no_feasible_grasp` instead of executing an
+   ungated grasp.
+
+The support surface is the target object's own base, not the table, so objects on shelves or inside drawers reject
+grasps that reach in from below. Grasp pose axes: local X opens the fingers, Y is palm height, Z is the approach.
+
+### Motion: segment-level routing
+
+Every skill compiles into a sequence of waypoints executed by `motion/chain.py::execute_motion_chain`. Each segment
+is routed by what it is, not by which skill it belongs to:
+
+| Segment | Route | Collision-aware |
+|---|---|---|
+| finger-only (close / open gripper) | arm locked, fingers move | — |
+| contact (final approach, drag handle, descend, side insertion) | Cartesian straight line | no, by design |
+| free (approach, lift, transport, retreat) | `plan_motion` when a collision world exists, otherwise IK + interpolation | yes, with rocRobo |
+
+Contact segments are never motion-planned: near contact, a planner's detours are worse than a short controlled line.
+A pick whose Cartesian segment jumps IK branches is rejected rather than executed.
+
+### Motion backends and the collision world
+
+`ROCROBO_BACKEND` selects the backend:
+
+- **off** → `GenesisBackend`: IK only; `plan_motion` always fails, so free segments use interpolation.
+- **on** → `RocRoboBackend`: collision-free IK and trajectory optimisation from
+  [rocRobo](https://github.com/ZJLi2013/rocRobo) (JAX). JAX and PyTorch do not share a ROCm process well, so rocRobo
+  runs in a sidecar container and RoboSmith talks to it over JSONL through `docker exec`.
+
+With a collision-aware backend, `motion/rocrobo_world.py::build_obstacle_world` builds the world from live poses:
+fixture collision boxes plus a ground half-space, with articulated parts placed at their current joint value
+(prismatic joints only). During `place`, the held object is attached to the hand as a set of spheres. rocRobo plans
+the 7 arm joints; the client adds the two finger joints back.
+
+If `plan_motion` fails, the segment falls back to a straight line **without collision checks**; the log line
+`straight-line fallback` marks it.
+
+### Articulation
+
+Articulated assets load as URDF joint bodies, reset to `joint_init`, and expose joint values in
+`env_state.joint_positions`. `open` / `close` drag the handle along the joint axis ("open" means the joint value
+increases), with travel scaled by `metric_scale`. Long-horizon tasks such as `open → pick → place → close` re-anchor
+every segment on the live simulation through `skills/frames.py::resolve_frame`, so a place target inside a drawer
+moves with the drawer.
+
+### Rollout and data
+
+Per episode: reset objects to their resolved positions, settle for `settle_steps` (30), record the settled poses as
+the initial state, run the skills, then evaluate the success tree. The run writes:
+
+- a LeRobot dataset with the `up` camera; articulated joint values are appended to `observation.environment_state`;
+- per-episode videos and `scenario_run_summary.json` (success per episode, diagnostics, skill traces).
+
+## Current limits
+
+- **Collision awareness covers free segments only.** Contact segments are straight lines, and a failed plan falls back
+  to an unchecked straight line.
+- Articulation supports prismatic joints only; revolute open / close, push / press, and bimanual tasks are not
+  implemented.
+- One asset per process: `SimEnv.build` initialises Genesis every time, so a second build in the same process fails.
+- The first `plan_motion` of a new scene shape compiles for about 10 minutes; later runs reuse the JAX disk cache.
+- Data comes from scripted IK replay only; there is no DAgger-style or closed-loop evaluation backend yet.
diff --git a/docs/quick-start.md b/docs/quick-start.md
@@ -0,0 +1,140 @@
+# Quick start
+
+From a fresh AMD GPU node to a recorded drawer episode (`open → pick → place → close`). For what each piece does,
+see [design.md](design.md).
+
+## 0. Without a GPU
+
+The package and its unit tests run anywhere:
+
+```shell
+python -m pip install -e ".[dev]"
+python -m pytest tests -q
+```
+
+Everything below needs an AMD GPU node with ROCm and Docker.
+
+## 1. Layout on the host
+
+`scripts/setup/run_scenario.sh` expects these directories under `$HOME` (override with `REPO`, `GRASPGEN_DIR`,
+`GRASPGEN_MODELS`, `HF_CACHE`):
+
+```text
+$HOME/RoboSmith         this repository
+$HOME/rocRobo           motion backend (sidecar)
+$HOME/GraspGen          GraspGen source
+$HOME/GraspGenModels    GraspGen checkpoints
+$HOME/.cache/huggingface  LeRobot datasets are written here
+```
+
+```bash
+cd $HOME
+git clone https://github.com/ZJLi2013/RoboSmith.git
+git clone https://github.com/ZJLi2013/rocRobo.git
+git clone --depth 1 https://github.com/NVlabs/GraspGen.git
+```
+
+## 2. Build the RoboSmith image
+
+Build from the repository root (`cd $HOME/RoboSmith`) with the image for your GPU architecture:
+
+| GPU | Build | Tag |
+|---|---|---|
+| MI300 (gfx942) | `docker build -f docker/Dockerfile.gfx942 -t robotsmith:gfx942-rocm6.4.3-genesis0.4.5 .` | includes GraspGen deps |
+| MI350 (gfx950) | `docker build -f docker/Dockerfile.gfx950 -t robotsmith:rocm-headless .`, then the GraspGen layer below | `robotsmith:rocm-headless-graspgen` |
+
+gfx942 needs its own older stack (ROCm 6.4.3, Genesis 0.4.5) because newer Genesis crashes while compiling collision
+kernels on that architecture. On gfx950, add the GraspGen layer:
+
+```dockerfile
+FROM robotsmith:rocm-headless
+RUN python -m pip install --no-cache-dir git+https://github.com/ZJLi2013/spconv_rocm.git@rocm \
+ && python -m pip install --no-cache-dir transformers==4.45.2 tensordict diffusers==0.11.1 \
+      huggingface-hub==0.25.2 sharedarray
+```
+
+```bash
+docker build -f Dockerfile.graspgen -t robotsmith:rocm-headless-graspgen .
+```
+
+Use `spconv_rocm`, not GraspGen's upstream `spconv-cu120`.
+
+## 3. GraspGen weights
+
+GraspGen is under an NVIDIA research / evaluation license and is not shipped with RoboSmith. Download the checkpoints
+to the host and convert them to the `spconv_rocm` weight layout once:
+
+```bash
+IMAGE=robotsmith:gfx942-rocm6.4.3-genesis0.4.5   # or robotsmith:rocm-headless-graspgen
+docker run --rm -v $HOME:/host $IMAGE bash -lc '
+  python -c "from huggingface_hub import snapshot_download; snapshot_download(repo_id=\"adithyamurali/GraspGenModels\", local_dir=\"/host/GraspGenModels\")"
+  cd /host/RoboSmith && python scripts/setup/_convert_spconv_ckpt.py \
+    /host/GraspGenModels/checkpoints/graspgen_franka_panda_gen.pth \
+    /host/GraspGenModels/checkpoints/graspgen_franka_panda_dis.pth'
+```
+
+This writes `*_rocm.pth` next to the originals; the grasp wrapper prefers them automatically. The run points
+`GRASPGEN_CONFIG` at `GraspGenModels/checkpoints/graspgen_franka_panda.yml`.
+
+## 4. Start the rocRobo sidecar
+
+Once per machine. The container must be named `rocrobo_dev` (or set `ROCROBO_CONTAINER`) and mount rocRobo at
+`/rocrobo`:
+
+```bash
+docker run -d --name rocrobo_dev \
+  --device=/dev/kfd --device=/dev/dri --group-add video --ipc=host \
+  -e AMD_COMGR_NAMESPACE=1 -e HIP_VISIBLE_DEVICES=0 \
+  -v $HOME/rocRobo:/rocrobo \
+  rocrobo/pyroki:rocm7.2.4-jax0.8.2 sleep infinity
+docker exec rocrobo_dev pip install -e /rocrobo/pyroki --no-deps
+```
+
+## 5. Run the drawer scenario
+
+```bash
+cd $HOME/RoboSmith
+bash scripts/setup/run_scenario.sh                                     # MI300
+IMAGE=robotsmith:rocm-headless-graspgen bash scripts/setup/run_scenario.sh   # MI350
+docker logs -f f1_m1
+```
+
+The launcher starts a detached container (default name `f1_m1`; set `NAME`, `SCENARIO`, `SEED`, `EPISODES` to
+change the run). It mounts the Docker socket so RoboSmith can `docker exec` into the sidecar.
+
+Outputs:
+
+- `output/<NAME>/scenario_run_summary.json`: per-episode success and diagnostics;
+- `output/<NAME>/videos/*.mp4`;
+- the LeRobot dataset under `$HOME/.cache/huggingface/lerobot/local/robotsmith-scenario-<task>/`.
+
+To render a layout without rolling out (inside the same image, which provides Genesis):
+
+```bash
+docker run --rm --device=/dev/kfd --device=/dev/dri --group-add video \
+  -v $HOME/RoboSmith:/workspace/robotsmith robotsmith:gfx942-rocm6.4.3-genesis0.4.5 \
+  bash -lc 'cd /workspace/robotsmith && python scripts/datagen/snapshot_scenario.py --scenario scenarios/pick_place_into_drawer.py'
+```
+
+## 6. Write your own scenario
+
+Copy a file from [`scenarios/`](../scenarios/), change the scene, skill intents, and success tree, and pass it with
+`SCENARIO=scenarios/<your_file>.py`. To have an agent draft one, give it
+[`scenarios/AGENT_PROMPT.md`](../scenarios/AGENT_PROMPT.md) explicitly. To validate a single asset's picks, use
+`scripts/datagen/validate_asset_picks.py --assets <name>`.
+
+## Troubleshooting
+
+- **Stuck after `starting warm serve` for ~10 minutes on the first run.** This is the JAX compile of the first
+  `plan_motion`, not a hang. The compile is cached in `/rocrobo/.jax_cache` (override with `ROCROBO_JAX_CACHE`);
+  later runs with the same scene geometry start in seconds. While waiting,
+  `docker exec rocrobo_dev pgrep -f rocrobo.serve` should print one process. If it prints none and the run is still
+  stuck, the serve died during startup; restart the run.
+- **A GPU is held by a dead serve.** `docker exec rocrobo_dev pkill -9 -f python`, then rerun.
+- **`collision-blind straight-line fallback` in the log.** `plan_motion` failed for that segment and it ran without
+  collision checks. Treat it as a failed segment, not a success.
+- **`Genesis already initialized`.** One process can build only one scene; run each asset or scenario in its own
+  process.
+- **Missing mesh files.** Some large meshes are not in git; copy them into `assets/objects/<name>/`
+  (`visual.*`, `collision.obj`, `material.mtl`).
+- **Changed rocRobo code has no effect.** The sidecar bind-mounts `/rocrobo`; restart the run so a new serve starts.