feat: share local dev infrastructure across worktrees

.github/workflows/dev-workflow-tests.yml

@@ -0,0 +1,60 @@
+name: Dev Workflow Tests
+
+on:
+  pull_request:
+    paths:
+      - "start-dev.sh"
+      - "stop-dev.sh"
+      - "scripts/setup-worktree.sh"
+      - "scripts/dev-infra.sh"
+      - "scripts/dev-infra.test.sh"
+      - "packages/plugins/lib/meiliIndexName.ts"
+      - "packages/plugins/lib/meiliIndexName.test.ts"
+      - "packages/plugins/search-meilisearch/**"
+      - "packages/plugins/vectorstore-meilisearch/**"
+      - "package.json"
+      - ".github/workflows/dev-workflow-tests.yml"
+  push:
+    branches:
+      - main
+    paths:
+      - "start-dev.sh"
+      - "stop-dev.sh"
+      - "scripts/setup-worktree.sh"
+      - "scripts/dev-infra.sh"
+      - "scripts/dev-infra.test.sh"
+      - "packages/plugins/lib/meiliIndexName.ts"
+      - "packages/plugins/lib/meiliIndexName.test.ts"
+      - "packages/plugins/search-meilisearch/**"
+      - "packages/plugins/vectorstore-meilisearch/**"
+      - "package.json"
+      - ".github/workflows/dev-workflow-tests.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  shared-dev-infra:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+        with:
+          persist-credentials: false
+
+      - name: Setup
+        uses: ./tooling/github/setup
+
+      - name: Check Bash syntax
+        run: |
+          for script in start-dev.sh stop-dev.sh scripts/setup-worktree.sh scripts/dev-infra.sh scripts/dev-infra.test.sh; do
+            bash -n "$script"
+          done
+
+      - name: Run shared dev infrastructure tests
+        run: bash scripts/dev-infra.test.sh
+
+      - name: Run Meilisearch namespace tests
+        run: pnpm --filter @karakeep/plugins test --run
+
+      - name: Typecheck plugins
+        run: pnpm --filter @karakeep/plugins typecheck

AGENTS.md

@@ -15,8 +15,12 @@
 - Monorepo: Next.js, React, TypeScript, Hono, tRPC, Drizzle, SQLite, Meilisearch, pnpm, and Turborepo.
 - Runtime: Node 24 through `mise exec node@24 --`; pnpm 11.2.1 through Corepack.
 - Install with `pnpm install`, create the documented `.env` symlinks, then run `pnpm db:migrate`.
-- Start local development with `pnpm dev:start`. Use `pnpm dev:start -d` for detached mode and `pnpm dev:stop` to stop it.
+- Start local development with `pnpm dev:start`. Use `pnpm dev:start -d` for detached mode and `pnpm dev:stop` to stop only that workspace.
+- Shared local infrastructure is machine-level: one Meilisearch at `http://localhost:7700` and one Chrome/CDP at `http://localhost:9222`. Manage it explicitly with `pnpm dev:infra:up`, `pnpm dev:infra:status`, and `pnpm dev:infra:down`.
+- Parallel worktrees keep separate SQLite/assets data and unique web ports. `scripts/setup-worktree.sh` assigns each worktree a unique `MEILI_INDEX_PREFIX`; both `bookmarks` and `bookmarks_vectors` use that namespace on the shared Meilisearch server.
+- `pnpm dev:start` defaults the main workspace namespace to `main_`. Outside the fork dev launcher, an unset `MEILI_INDEX_PREFIX` preserves the original `bookmarks` and `bookmarks_vectors` index names.
 - Run focused checks before broad checks when practical. Standard checks are `pnpm format:fix`, `pnpm lint`, `pnpm typecheck`, and `pnpm test`.
+- Validate shared-dev shell behavior with `bash scripts/dev-infra.test.sh`.
 - Root `.env` is canonical. Do not print or commit secrets.
 - Guided Docker self-hosting: `docs/docs/02-installation/11-guided-docker-setup.md`.
 
@@ -114,21 +118,41 @@ pnpm dev:start
 Useful variants:
 - `pnpm dev:start` - foreground
 - `pnpm dev:start -d` - detached
-- `pnpm dev:stop` - stop detached services
-
-What it does:
-- runs `web` + `workers` natively
-- runs Meilisearch + headless Chrome in Docker
+- `pnpm dev:stop` - stop only this workspace's web/workers processes
+- `pnpm dev:infra:up` - explicitly start/reuse shared Meilisearch + Chrome
+- `pnpm dev:infra:status` - inspect shared dev infrastructure
+- `pnpm dev:infra:down` - explicitly remove shared containers while preserving Meilisearch data
+
+Local-dev ownership model:
+- `web` + `workers` run natively per workspace
+- one machine-level Meilisearch container is shared at `http://localhost:7700`
+- one machine-level Chrome container is shared at `http://localhost:9222`
+- `pnpm dev:start` automatically ensures those shared containers exist
+- `pnpm dev:stop` never stops shared infrastructure because other worktrees may still use it
+- the shared Chrome image is `ghcr.io/karakeep-app/karakeep-chrome:release`
+
+Parallel-worktree isolation:
+- every worktree keeps its own `.data/local` SQLite/assets state and unique web port
+- `scripts/setup-worktree.sh` points all worktrees at shared Meilisearch/Chrome endpoints
+- every worktree receives a safe unique `MEILI_INDEX_PREFIX` derived from its normalized workspace name plus `WT_PORT_BASE`
+- both `bookmarks` and `bookmarks_vectors` use that prefix, so separate SQLite states never share Meilisearch documents
+- `pnpm dev:start` defaults the main workspace prefix to `main_`
+- outside this fork's dev launcher, unset `MEILI_INDEX_PREFIX` preserves the original `bookmarks` / `bookmarks_vectors` names
 
 ### Direct commands
 
+When bypassing `pnpm dev:start`, manual starts **must** set an explicit unique `MEILI_INDEX_PREFIX` for that workspace before starting web or workers. Use `main_` only for the main workspace; parallel worktrees need distinct prefixes.
+
 ```bash
+export MEILI_INDEX_PREFIX=main_
+pnpm dev:infra:up
 pnpm web
 pnpm workers
 ```
 
 Notes:
 - Meilisearch and headless Chrome are optional for booting the app, but required for full search/crawling behavior.
+- shared infra binds only to localhost; if ports `7700` or `9222` are occupied by something else, the helper fails rather than silently reusing an unknown service
 - If `next dev` crashes with a stale Turbopack/instrumentation issue, clear `apps/web/.next`.
 
 ### Pull prod state to local dev
@@ -145,7 +169,7 @@ Optional root `.env` keys:
 - `KARAKEEP_PROD_COMPOSE_SERVICE`
 - `KARAKEEP_PROD_EXPORT_IMAGE`
 
-Every pull restores the full `/data` volume because SQLite rows can reference stored assets. Do not use DB-only pulls or print `.env` secrets.
+Every pull restores the full `/data` volume because SQLite rows can reference stored assets. Do not use DB-only pulls or print `.env` secrets. Meilisearch remains derived local state in that workspace's own index namespace.
 
 ## Deploy model for this fork
 
@@ -175,6 +199,7 @@ Additional tooling used in this fork:
 - `pnpm doctor` - React health scan via react.doctor
 - `pnpm doctor:staged` - staged-file React scan
 - `bash scripts/install.test.sh` - guided installer shell-level validation
+- `bash scripts/dev-infra.test.sh` - shared worktree-dev infrastructure validation
 
 Notes:
 - `react.doctor` is advisory in pre-commit and can emit noisy temp-package errors.
@@ -209,6 +234,10 @@ pnpm test
 pnpm knip
 pnpm doctor
 bash scripts/install.test.sh
+bash scripts/dev-infra.test.sh
+pnpm dev:infra:up
+pnpm dev:infra:status
+pnpm dev:infra:down
 pnpm db:generate --name <description>
 pnpm db:migrate
 pnpm web

docs/fork-setup.md

@@ -48,24 +48,73 @@ pnpm dev:start
 Variants:
 - `pnpm dev:start` — foreground
 - `pnpm dev:start -d` — detached (`.dev/` logs, shell returns immediately)
-- `pnpm dev:stop` — stop detached services
+- `pnpm dev:stop` — stop only this workspace's detached web/workers processes
+- `pnpm dev:infra:status` — show the shared Chrome/Meilisearch container status
+- `pnpm dev:infra:up` — explicitly start/reuse shared Chrome + Meilisearch
+- `pnpm dev:infra:down` — explicitly remove the shared containers while preserving the Meilisearch data volume
+
+`pnpm dev:start` automatically ensures the machine-level dev infrastructure is running, then starts only this workspace's native processes:
 
-What this starts:
 - `web`
 - `workers`
-- Meilisearch in Docker
-- headless Chrome in Docker
+
+The machine-level infrastructure is shared across all local worktrees:
+
+- `karakeep-dev-meilisearch` on `127.0.0.1:7700`
+- `karakeep-dev-chrome` on `127.0.0.1:9222`
+
+The Chrome helper uses `ghcr.io/karakeep-app/karakeep-chrome:release`, which is published for both `linux/amd64` and `linux/arm64`. The Meilisearch container uses the named volume `karakeep-dev-meilisearch-data`, which survives `pnpm dev:infra:down`.
+
+`pnpm dev:stop` never stops the shared containers. This is intentional: another worktree may still be using them.
+
+### Parallel worktrees
+
+Worktrees share the physical Chrome and Meilisearch containers, but application state stays isolated.
+
+`scripts/setup-worktree.sh` keeps these values per worktree:
+
+- unique `KARAKEEP_PORT`
+- unique `DATA_DIR` (`<worktree>/.data/local`)
+- unique `API_URL` / `NEXTAUTH_URL`
+- unique `MEILI_INDEX_PREFIX`
+
+Every generated worktree points at the same local infrastructure endpoints:
+
+```text
+MEILI_ADDR=http://localhost:7700
+BROWSER_WEB_URL=http://localhost:9222
+```
+
+The Meilisearch plugins prepend `MEILI_INDEX_PREFIX` to both index UIDs. For example, a worktree prefix `issue-123-7_` produces:
+
+```text
+issue-123-7_bookmarks
+issue-123-7_bookmarks_vectors
+```
+
+The generated prefix is based on the normalized worktree name plus `WT_PORT_BASE`, so two configured worktrees do not share search/vector state even though they use the same Meilisearch server.
+
+The main workspace uses `main_` when `pnpm dev:start` does not find an explicit `MEILI_INDEX_PREFIX`. When `MEILI_INDEX_PREFIX` is entirely unset outside this fork's dev launcher, the plugins retain the original production-compatible index names `bookmarks` and `bookmarks_vectors`.
+
+Do not share a worktree's `.data/local` directory with another worktree. SQLite rows and stored assets are authoritative per workspace; Meilisearch remains derived state and can be rebuilt into that workspace's namespace.
 
 ### Direct/manual start
 
+If you intentionally bypass `pnpm dev:start`, manual starts **must** set an explicit unique `MEILI_INDEX_PREFIX` for that workspace. Use `main_` only for the main workspace; parallel worktrees need distinct prefixes.
+
 ```bash
+export MEILI_INDEX_PREFIX=main_
+pnpm dev:infra:up
 pnpm web
 pnpm workers
 ```
 
+Direct commands do not synthesize a namespace for you. Leaving the prefix unset selects the backward-compatible unprefixed indexes and can mix search/vector state when multiple manual worktrees share the same Meilisearch server.
+
 Notes:
 - Meilisearch and headless Chrome are optional for booting the app, but required for full search/crawling behavior.
 - If `next dev` crashes with a stale Turbopack / `instrumentation.ts` parse issue, clear `apps/web/.next` and restart.
+- If port `7700` is occupied by something other than `karakeep-dev-meilisearch`, or port `9222` by something other than `karakeep-dev-chrome`, `pnpm dev:infra:up` fails instead of silently reusing an unknown service.
 
 ### Verify the offline iPhone PWA
 
@@ -86,7 +135,9 @@ The root `.env` is the source of truth, but several processes load `.env` from t
 The most important variables for local development are:
 - `DATA_DIR`
 - `NEXTAUTH_SECRET`
-- `MEILI_ADDR` (if search should work)
+- `MEILI_ADDR` (shared dev default: `http://localhost:7700`)
+- `MEILI_INDEX_PREFIX` (per-worktree search/vector namespace; empty remains backward-compatible outside the dev launcher)
+- `BROWSER_WEB_URL` (shared dev default: `http://localhost:9222`)
 - `OPENAI_API_KEY` (if AI tagging/summarization should work)
 
 ### Pull production state into local development
@@ -110,6 +161,8 @@ Optional root `.env` keys:
 - `KARAKEEP_PROD_COMPOSE_SERVICE`
 - `KARAKEEP_PROD_EXPORT_IMAGE`
 
+A production-state pull still populates only that workspace's SQLite/assets state. Its local search/vector data belongs to the workspace's own `MEILI_INDEX_PREFIX` namespace in the shared local Meilisearch container.
+
 ## CI
 
 Primary workflow:
@@ -125,8 +178,9 @@ It runs:
 Fork-specific notes:
 - this fork does **not** use Turbo remote cache
 - some CI jobs reclaim disk space before heavy steps because typecheck/tests can otherwise exhaust hosted-runner storage
-- CI reads the exact Node runtime from `.nvmrc`; keep the temporary 24.18.1 pin until the Node v24 cleanup-hook fix tracked in [nodejs/node#65042](https://github.com/nodejs/node/pull/65042) ships in a usable Node 24 release
+- local development and production use Node 24.18.1 from `.nvmrc`; the combined CI `tests` job temporarily overrides setup to Node 22.21.1 because Vitest + `better-sqlite3` can abort during Node 24 worker teardown; remove that override once the Node fix tracked in [nodejs/node#65042](https://github.com/nodejs/node/pull/65042) ships in a usable Node 24 release
 - `knip` and `react-doctor` run as **non-blocking** report jobs
+- `.github/workflows/dev-workflow-tests.yml` validates the shared local-dev Bash lifecycle and Meilisearch namespace behavior when relevant files change
 
 ## Extra quality tooling