Fix runtime issues

.ai/component-port-types.md

@@ -0,0 +1,91 @@
+# Component Port Type System (v2)
+
+Last updated: 2024-07-06
+
+## Goals
+
+- Replace the ad-hoc string enum (`'string' | 'array' | …'`) with an explicit, self-documenting type contract.
+- Make coercion rules visible and enforceable so builders know when conversions occur.
+- Create a path for structured outputs without leaking arbitrary objects through ports.
+- Keep ergonomics high for component authors (helpers + registry-backed contracts).
+
+## Core Concepts
+
+### Primitive Types
+
+| Name    | Description                                     | Default coercion (`coerceFrom`)             |
+|---------|-------------------------------------------------|---------------------------------------------|
+| `text`  | UTF-8 string payloads                           | `['number', 'boolean']`                     |
+| `secret`| Masked string values (never coerced)            | `[]`                                        |
+| `number`| Numeric values (int/float)                      | `['text']` (via `parseFloat`)               |
+| `boolean`| Boolean values                                 | `['text']` (truthy string check)            |
+| `file`  | Structured file handle (id + metadata)          | `[]`                                        |
+| `json`  | Arbitrary JSON payload                          | configurable per port                       |
+
+### Collections
+
+- `list<primitive|contract>`: Homogeneous arrays. Metadata is persisted as `{ kind: 'list', element: … }`.
+- `map<primitive>`: String-keyed dictionaries with primitive values (`{ kind: 'map', value: … }`).
+
+### Contracts
+
+Structured data exits the component via a **named contract**:
+
+```ts
+registerContract({
+  name: 'dnsx.v1',
+  schema: z.object({
+    host: z.string(),
+    answers: z.record(z.string(), z.array(z.string())),
+  }),
+  summary: 'ProjectDiscovery dnsx response',
+});
+```
+
+Ports reference the contract using `{ kind: 'contract', name: 'dnsx.v1' }`.
+When an output references a contract, the workflow runner parses the payload with the registered Zod schema before the result is stored.
+
+### Coercion Rules
+
+- Defined on the *target* primitive (`coercion.from: PrimitiveTypeName[]`).
+- Applied during input resolution before Zod validation.
+- Conversions:
+  - `text` ⇐ `number`/`boolean` via `.toString()`.
+  - `number` ⇐ `text` via `parseFloat` (rejects `NaN`).
+  - `boolean` ⇐ `text` via `['true','false']` (case insensitive).
+  - Additional rules can be declared per port (`port.number({ coerceFrom: ['boolean'] })`).
+
+## Authoring Helpers (`@shipsec/component-sdk`)
+
+```ts
+import { port, registerContract } from '@shipsec/component-sdk';
+
+const definition = {
+  metadata: {
+    inputs: [
+      { id: 'items', label: 'Items', dataType: port.list(port.text()) },
+      { id: 'separator', label: 'Separator', dataType: port.text({ coerceFrom: [] }) },
+    ],
+    outputs: [
+      { id: 'text', label: 'Joined Text', dataType: port.text() },
+      { id: 'count', label: 'Item Count', dataType: port.number() },
+    ],
+  },
+};
+```
+
+Helpers return fresh descriptors, so authors do not mutate shared instances accidentally.
+
+## Validation Flow
+
+1. **Connection validation (frontend)** uses the serialized `PortDataType` to check compatibility and planned coercions.
+2. **Input resolver (worker)** resolves upstream values, applies coercions defined on the target port, and raises errors if conversion fails.
+3. **Zod validation (component)** still runs (`component.inputSchema` + `component.outputSchema`).
+4. **Contract enforcement** ensures structured outputs match registered schemas.
+
+## Migration Notes
+
+- Ports now use `dataType` instead of `type`.
+- `ComponentPortMetadata` is shared between SDK, backend API, and UI.
+- Existing workflows must be rewritten or migrated because the serialized metadata changes shape (no backwards compatibility promised for Phase 2).
+- Update `docs/execution-contract.md` and component READMEs when introducing new contracts.

.ai/visual-execution-notes.md

@@ -33,6 +33,7 @@
 ## Replay Mode
 - Historical runs selectable from timeline; playback re-applies captured events to animate the DAG.
 - Scrubber jumps to a timestamp; canvas + console reflect state at that moment.
+- Timeline rows include `workflowVersionId`/`workflowVersion` so replays pin to the exact saved DAG; surface the version badge in the run details panel.
 - Diff view highlights behavioral changes between runs (new nodes, altered outputs).
 
 ## Trace Event Schema (concept)

.gitignore

@@ -60,3 +60,6 @@ vite.config.ts.timestamp-*
 # Optional: Generated files (if using codegen later)
 # src/generated/
 .vscode/settings.json
+
+# Playground testing directory
+.playground/

ARCHITECTURE.md

@@ -34,7 +34,7 @@ shipsec-studio/
 │       │   └── trace.adapter.ts         # In-memory trace collector
 │       └── temporal/
 │           ├── workflows/      # Temporal workflow definitions
-│           ├── activities/     # runWorkflowActivity
+│           ├── activities/     # runComponentActivity, setRunMetadataActivity, finalizeRunActivity
 │           └── workers/        # dev.worker.ts
 │
 ├── backend/                    # 🌐 REST API (NestJS on Bun)
@@ -151,13 +151,24 @@ const secretsAdapter = new SecretsAdapter(db);
 const logAdapter = new LokiLogAdapter(new LokiLogClient({ baseUrl: process.env.LOKI_URL! }), db);
 
 // Inject into activities
-initializeActivityServices(storageAdapter, traceAdapter, logAdapter, secretsAdapter);
+initializeComponentActivityServices({
+  storage: storageAdapter, 
+  trace: traceAdapter, 
+  logs: logAdapter, 
+  secrets: secretsAdapter
+});
 
 // Start worker
 const worker = await Worker.create({
   connection,
+  namespace,
+  taskQueue,
   workflowsPath,
-  activities: { runWorkflowActivity },
+  activities: {
+    runComponentActivity,
+    setRunMetadataActivity,
+    finalizeRunActivity,
+  },
 });
 ```
 
@@ -201,17 +212,17 @@ const worker = await Worker.create({
    └─> Picks up workflow task
    └─> Executes shipsecWorkflowRun() workflow function
 
-5. Workflow calls runWorkflowActivity()
-   └─> Activity receives DSL definition
-   └─> For each action in order:
-       ├─> Looks up component in registry
-       ├─> Creates ExecutionContext with injected services
-       ├─> Runs component.execute(params, context)
-       └─> Component uses context.storage.downloadFile(...)
+5. Workflow orchestrates component execution by calling runComponentActivity() for each component
+   └─> Each component execution uses the same runComponentActivity
+   └─> Activity receives componentId and parameters
+   └─> Looks up component in registry by componentId
+   └─> Creates ExecutionContext with injected services
+   └─> Runs component.execute(params, context)
+   └─> Component uses context.storage.downloadFile(...)
 
 6. Results flow back
    └─> Activity completes with outputs
-   └─> Workflow completes
+   └─> Workflow continues to next component or completes
    └─> Backend polls Temporal for result
    └─> Frontend displays result
 ```
@@ -249,6 +260,11 @@ const context = createExecutionContext({
 });
 ```
 
+**How injection happens during execution**:
+- Activities receive service adapters during worker initialization via `initializeComponentActivityServices`
+- Each component execution gets an ExecutionContext with injected services
+- Single activity (`runComponentActivity`) handles all component types dynamically using the componentId to look up the right component in the registry
+
 **Benefits**:
 - ✅ Components are portable and testable (mock interfaces)
 - ✅ Adapters can be swapped (MinIO → S3, PostgreSQL → MongoDB)
@@ -257,17 +273,15 @@ const context = createExecutionContext({
 ## Running the System
 
 ```bash
-# Start Temporal cluster (docker-compose)
-cd temporal && docker-compose up -d
+# Start infrastructure (Temporal, PostgreSQL, MinIO, Loki via docker-compose)
+docker compose up -d
 
-# Start PostgreSQL + MinIO
-cd .. && docker-compose up -d postgres minio
+# Start backend + worker + frontend (via PM2)
+bun run dev:stack
 
-# Start backend + worker (via PM2)
-pm2 start pm2.config.cjs
-
-# Start frontend
-cd frontend && bun dev
+# Or start services individually
+pm2 start pm2.config.cjs  # starts backend and worker
+cd frontend && bun dev    # starts frontend dev server
 ```
 
 ## Development Workflow
@@ -300,17 +314,20 @@ backend
 
 worker
   ├─> @shipsec/component-sdk (SDK types + registry)
-  └─> (MinIO, PostgreSQL, Temporal SDKs)
+  └─> @shipsec/shared (execution schemas and types)
 
 component-sdk
   └─> (zod only - zero runtime dependencies)
+
+shared
+  └─> (zod, TypeScript types for execution contracts)
 ```
 
 ## Future Enhancements
 
 - [ ] Docker runner implementation (currently stubbed)
 - [ ] Remote runner for distributed execution
-- [ ] Secrets management service
-- [ ] Artifact storage service
-- [ ] Real-time trace streaming via WebSockets
+- [x] Secrets management service (completed: SecretsAdapter with PostgreSQL backend)
+- [x] Artifact storage service (completed: IArtifactService interface implemented)
+- [x] Real-time trace streaming via WebSockets (completed: ITraceService with TraceAdapter)
 - [ ] Component marketplace

backend/drizzle/0007_add-workflow-versions.sql

@@ -0,0 +1,40 @@
+CREATE TABLE "workflow_versions" (
+  "id" uuid PRIMARY KEY DEFAULT gen_random_uuid(),
+  "workflow_id" uuid NOT NULL,
+  "version" integer NOT NULL,
+  "graph" jsonb NOT NULL,
+  "compiled_definition" jsonb DEFAULT NULL,
+  "created_at" timestamp with time zone NOT NULL DEFAULT now()
+);
+
+CREATE UNIQUE INDEX "workflow_versions_workflow_version_uidx"
+  ON "workflow_versions" ("workflow_id", "version");
+
+ALTER TABLE "workflow_runs" ADD COLUMN "workflow_version_id" uuid;
+ALTER TABLE "workflow_runs" ADD COLUMN "workflow_version" integer;
+
+INSERT INTO "workflow_versions" (
+  "workflow_id",
+  "version",
+  "graph",
+  "compiled_definition",
+  "created_at"
+)
+SELECT
+  w."id",
+  1 AS "version",
+  w."graph",
+  w."compiled_definition",
+  COALESCE(w."updated_at", w."created_at")
+FROM "workflows" w;
+
+UPDATE "workflow_runs"
+SET "workflow_version" = 1
+WHERE "workflow_version" IS NULL;
+
+UPDATE "workflow_runs" wr
+SET "workflow_version_id" = v."id"
+FROM "workflow_versions" v
+WHERE wr."workflow_id" = v."workflow_id"
+  AND wr."workflow_version" = v."version"
+  AND wr."workflow_version_id" IS NULL;