feat(memory): append-only audit log with history reads, revert, and undelete - #212
Conversation
Record every memory mutation as an immutable row in a new per-space memory_event table (optionally a TimescaleDB hypertable), via a single AFTER insert/update/delete trigger. Each event snapshots the resulting (or, for deletes, removed) state plus the physical operation, an app-level cause, a bulk-correlation operation_id, and the actor. The actor comes from a transaction-local me.event_context GUC set by the request layer (withEventContext), so the space SQL functions stay untouched. Principal and API-key display names are threaded through authenticateSpace -> SpaceRpcContext to populate the actor; the router projection is now compile-time checked against the context shape. Unlike an archive-prior-state model, delete is just another event, so every version's author is preserved — the deleter never overwrites the final author.
Expose the append-only memory_event log through a new read path across every layer: a get_memory_history SQL function (read-gated per event tree, nullable memoryId/tree/operation/operationId filters, bounded + ordered), the memory.history RPC, engine store getMemoryHistory, the client method, a `me memory history` CLI command (top-level alias `me history`), and the me_memory_history MCP tool. At least one scope (memoryId/tree/operationId) is required. Each event is gated by read access to its own tree, so a moved memory's history may be partial; deleted memories stay readable by id. Presentation trims the snapshot via --select/select while always keeping the audit envelope (who/when/what). Adds docs for the CLI command and MCP tool (+ nav), and integration coverage for per-event-tree gating, filters/order, and the end-to-end insert/update/delete → three-event history with the updater preserved.
Make the audit log serve both a per-memory revision log and a date-oriented activity feed, and reach deleted memories: - Indexes: explicit (at desc) for the time feed (created outside the hypertable block so it exists with or without timescaledb), plus (operation_id) and (tree, name); keep (memory_id, version desc) for a future `me memory revert`. create_default_indexes => false. - Retention: a 30-day drop_after policy on the hypertable (per-memory history is bounded to the window). - get_memory_history gains since/until (chunk-pruned) and a keyset cursor on event_id (uuidv7 is unique and co-monotonic with `at`, so a single-column seek is exact — no timestamp-precision loss). The scope rule relaxes to memoryId | path | tree | operationId | since, enabling a space-wide since-based feed. - resolve_memory_id_from_history + a `path` scope: memory.history resolves a path live-first, then via the log, so a deleted memory's history is reachable by its old path (live id wins on slot reuse). - Result carries nextCursor. CLI gains --since/--until/--cursor and a "more:" hint; the MCP tool gains path/since/until/cursor. Docs updated.
The `(_since is null or e.at >= _since)` disjunction is opaque to timescaledb chunk exclusion under a generic plan (the null branch qualifies every row, so it isn't a bound on `at`). Normalize the bounds once (`coalesce(_since, '-infinity')` / `coalesce(_until, 'infinity')`) and compare directly — `e.at >= _since` / `e.at < _until`. That keeps null-means-unbounded semantics while leaving a bare `at OP $param` qual the constraint-aware Append prunes chunks with at executor startup.
Restore a memory's current state to a version-N snapshot from the audit log, applied as a new forward version (a logged `revert` event) — not a history rewrite. Full snapshot (content/meta/tree/name/temporal) is restored, so a revert can move the memory back or hit a (tree, name) CONFLICT; access requires write on the current and snapshot trees. Undelete falls out of the same path: `memory_before_insert` no longer forces version = 1 (ordinary inserts keep the column default), so revert_memory re-inserts a deleted memory continuing BOTH sequences from history — `version` (logical payload) and `content_version` (the embedding-queue guard token). Continuing content_version matters: a reset to 1 could collide with a pre-delete version still in flight in the embedding worker, letting a stale embedding win the version-guarded write-back. A deleted memory is reachable by its old path (via the log). Deliberate override by default; optional expectedVersionHash guards a concurrent change on a live memory (ME002). Retention still bounds which versions are revertable. Adds memory.revert across protocol/engine/ server/client, `me memory revert <id-or-path> <version>` (TTY confirm, top-level `me revert` alias), the me_memory_revert MCP tool, docs, and migration + RPC integration coverage.
There was a problem hiding this comment.
Pull request overview
Adds an append-only memory_event audit log for all memory mutations, and builds user-facing history + revert/undelete capabilities on top of it, including actor attribution (principal + api key display names) propagated from request auth down into space mutations.
Changes:
- Introduces the
memory_eventaudit table (optionally a TimescaleDB hypertable), trigger-based event capture, and SQL read/write helpers (get_memory_history,resolve_memory_id_from_history,revert_memory). - Adds new RPC surface
memory.historyandmemory.revert, plus protocol + engine + client wiring to expose these features. - Adds CLI + MCP tools + docs for history browsing and version revert/undelete, and updates docs-site navigation.
Reviewed changes
Copilot reviewed 31 out of 31 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/server/wiring.test.ts | Adds a wiring test ensuring authenticated space context reaches a memory RPC handler. |
| packages/server/rpc/memory/types.ts | Extends Space RPC context to include principal/api-key display names for audit attribution. |
| packages/server/rpc/memory/memory.ts | Implements memory.history and memory.revert; wraps mutations with transaction-local audit event context. |
| packages/server/rpc/memory/memory.integration.test.ts | Adds integration coverage for history logging, paging, deleted-path resolution, and revert/undelete behavior. |
| packages/server/rpc/memory/management.integration.test.ts | Updates management RPC test contexts to include the new attribution fields. |
| packages/server/router.ts | Threads new attribution fields into the constructed Space RPC handler context. |
| packages/server/middleware/authenticate-space.ts | Captures principal/api-key display names during auth for downstream audit attribution. |
| packages/server/middleware/authenticate-space.integration.test.ts | Verifies authenticateSpace returns principalName/apiKeyName in both session and api-key flows. |
| packages/protocol/memory.ts | Adds Zod schemas/types for memory.history / memory.revert plus event result shapes. |
| packages/engine/space/types.ts | Introduces engine types for memory audit events, filters, and mutation event context. |
| packages/engine/space/index.ts | Re-exports the new audit-log-related engine types. |
| packages/engine/space/db.ts | Implements DB calls for history reads, deleted-path resolution, revert, and event-context transactions. |
| packages/engine/core/types.ts | Extends validated API key type with API key display name for attribution. |
| packages/engine/core/db.ts | Reads api_key_name from validate_api_key for attribution. |
| packages/docs-site/lib/nav.ts | Adds docs-site nav entries for the new MCP tool docs pages. |
| packages/database/space/version.ts | Bumps space schema version for the new audit-log migration. |
| packages/database/space/migrate/migrate.ts | Registers the new incremental + idempotent space migrations for memory_event. |
| packages/database/space/migrate/migrate.integration.test.ts | Expands migration assertions to cover memory_event, triggers, hypertable/retention, gating, and revert semantics. |
| packages/database/space/migrate/incremental/008_memory_event.sql | Adds memory_event table + indexes and optionally converts it to a Timescale hypertable with retention. |
| packages/database/space/migrate/idempotent/004_memory_event.sql | Adds trigger + helper functions for logging, history reads, deleted-path resolution, and revert logic. |
| packages/database/space/migrate/idempotent/001_memory.sql | Adjusts insert trigger behavior so undelete can reinsert with a continued version sequence. |
| packages/database/core/version.ts | Bumps core schema version for the validate_api_key return-shape update. |
| packages/database/core/migrate/idempotent/008_api_key.sql | Updates validate_api_key signature/returns to include API key display name. |
| packages/client/memory.ts | Adds client methods for memory.history and memory.revert. |
| packages/cli/memory-projection.ts | Adds projection utilities to trim event snapshot fields for CLI/MCP outputs. |
| packages/cli/mcp/server.ts | Registers MCP tools me_memory_history and me_memory_revert. |
| packages/cli/mcp/server.test.ts | Updates MCP server tests for the increased number of memory tools. |
| packages/cli/commands/memory.ts | Adds me memory history and me memory revert CLI commands (with paging + confirmation). |
| docs/mcp/me_memory_revert.md | Documents the MCP revert tool behavior and parameters. |
| docs/mcp/me_memory_history.md | Documents the MCP history tool behavior, paging, and event shape. |
| docs/cli/me-memory.md | Adds CLI reference sections for me memory history and me memory revert. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…tribution Align the me_memory_history tool + doc and me_memory_revert doc with the actual contract (history scopes: memoryId|path|tree|operationId|since; revert takes id or path, path wins). Emit the audit actor's api_key_name only when present instead of coercing a missing name to "".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (3)
packages/server/rpc/memory/memory.ts:277
eventContextconditionally includesapi_key_nameusing a truthiness check. Sincecore.api_key.nameistext not nullwithout an empty-string constraint, an API key name can legally be""— in that case this code drops the name and the audit log loses attribution. Check fornullinstead so empty strings are preserved while session-auth remains null/absent.
// Include the name only when present — keep attribution null/absent
// rather than coercing a missing name to an ambiguous "".
...(ctx.apiKeyName ? { api_key_name: ctx.apiKeyName } : {}),
packages/database/space/migrate/idempotent/004_memory_event.sql:87
- The comment describing required history scopes is out of date: the RPC/protocol layer allows a
since-only scope (space-wide activity feed) in addition to memoryId/tree/operationId. Updating this comment avoids future confusion when maintaining the SQL layer.
-- may therefore look partial. Filters are all nullable; the caller supplies at
-- least one of _memory_id / _tree / _operation_id (enforced at the RPC layer).
-- Deleted memories remain visible (their tombstone events outlive the row).
packages/database/space/migrate/idempotent/004_memory_event.sql:165
get_memory_historyorders by(at, event_id)but the keyset cursor predicate only comparesevent_id. Keyset pagination is only guaranteed correct when the seek predicate matches the full ORDER BY; with UUIDv7 (ms time component) andat(microsecond resolution), multiple events within the same millisecond can legitimately reorder byatwhileevent_idordering remains effectively random inside that ms. That can cause skipped/duplicated rows across pages under bulk writes/high throughput. Consider switching to a composite cursor that includes bothatandevent_id(or ordering solely byevent_idand indexing it) so the cursor is exact.
-- keyset cursor on event_id: uuidv7 is unique and co-monotonic with `at`, so a
-- single-column seek is exact (no timestamp-precision loss) and matches the
-- (at, event_id) sort order below.
and (
_cursor_event_id is null
or (_order = 'desc' and e.event_id < _cursor_event_id)
or (_order = 'asc' and e.event_id > _cursor_event_id)
)
order by
case when _order = 'asc' then e.at end asc
, case when _order = 'desc' then e.at end desc
, case when _order = 'asc' then e.event_id end asc
, case when _order = 'desc' then e.event_id end desc
limit _limit;
…omment Order get_memory_history solely by event_id (a unique, chronological uuidv7 total order) so the seek predicate matches ORDER BY exactly — removing the (at, event_id) ordering/cursor mismatch that could skip or duplicate rows under clock skew. at still drives the since/until window and chunk pruning. Also preserve an empty-string api_key_name (check != null, not truthiness) and update the stale scope comment to include _since. Addresses Copilot's 3 low-confidence review comments.
|
Addressed Copilot's 3 suppressed (low-confidence) comments in ec659d4:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 31 out of 31 changed files in this pull request and generated no new comments.
Suppressed comments (2)
packages/database/space/migrate/migrate.integration.test.ts:105
- The integration test pins TimescaleDB to a very specific minimum version (2.29.1). The repo’s Postgres image install isn’t version-pinned, so this makes tests brittle (a perfectly valid environment with TimescaleDB 2.x could fail the suite for no functional reason). If the goal is simply to ensure the extension exists, use a broad minimum (e.g. 2.0.0) and let the migration itself fail if it requires newer APIs.
beforeAll(async () => {
sql = connect(12);
await sql.begin((tx) => ensureExtension(tx, "timescaledb", "2.29.1"));
await bootstrapSpaceDatabase(sql);
[canonical, dim768, customIdx] = await Promise.all([
docs/mcp/me_memory_revert.md:6
- The audit log’s
operationis onlyinsert|update|delete; a revert is represented via the event’scause(set torevert) while the physical operation will beupdate(orinsertfor undelete). Saying this “records a newrevertevent” is misleading for tool users reading history output.
Restore a memory to an earlier version's state, applied as a new forward version.
Reverting does not rewrite history — it reproduces the version-N snapshot as the memory's current state, which bumps the version and records a new `revert` event. Look up the target version with [me_memory_history](me_memory_history.md).
Summary
Adds a permanent, append-only audit log for memories and builds the read/write surface on top of it — history queries, an activity feed, and version
revert(including undelete). Every memory mutation is recorded as an immutable event with the actor, so we can answer who changed a memory, when, how, and who deleted it — and roll a memory back to (or restore) any prior version within the retention window.Built on top of the TimescaleDB base image (
main); the audit table is an optional hypertable that degrades to a plain table without TimescaleDB.What's included (commit by commit)
memory_eventaudit log — one immutable row per insert/update/delete via a singleAFTERtrigger, snapshotting the resulting (or, for deletes, removed) state plus a flatactor(principal/API-key names threaded throughauthenticateSpace→SpaceRpcContext), an app-levelcause, and a bulk-correlationoperation_id. The actor flows through a transaction-localme.event_contextGUC set by the request layer, so the space SQL functions stay untouched. Delete is just another event, so no version's author is ever lost.memory.historyread surface — RPC + engine + client +me memory history(aliasme history) +me_memory_historyMCP tool. Read-gated per event tree; deleted memories remain readable by id or path.since/until(chunk-excludable bounds → TimescaleDB chunk pruning), keyset pagination onevent_id(uuidv7, exact — no timestamp-precision loss), and history-aware(tree,name)resolution so a deleted memory's history is reachable by its old path. A 30-day retention policy on the hypertable.memory.revert— restore a memory's current state to a version-N snapshot as a new forward version (git-restoresemantics, not a history rewrite). Undelete falls out of the same path:memory_before_insertno longer forcesversion = 1, so a deleted memory is re-inserted continuing bothversionandcontent_versionfrom history. Continuingcontent_versionis load-bearing — it's the embedding-queue guard token, and resetting it could let a stale in-flight embedding win the version-guarded write-back. Deliberate override by default; optionalexpectedVersionHashguards a concurrent change.Surface added
memory.history,memory.revertme memory history/me history,me memory revert/me revertme_memory_history,me_memory_revertdocs/cli/me-memory.mdsections +docs/mcp/me_memory_history.md,docs/mcp/me_memory_revert.md+ sidebar navNotes / deferred
memory.get'screatedByis still hardcodednull; the event log now makes real creator/last-modifier attribution possible as a follow-up.Testing
./bun run checkgreen (1252 pass). Integration coverage against local Postgres+TimescaleDB: append-only event semantics (author preserved across delete), per-event-tree gating, filters/order,since/untilwindowing, keyset paging, deleted-path resolution, index + retention-policy assertions, and revert (live restore, undelete with version+content_version continuation, no-op,NOT_FOUND, read-only →FORBIDDEN, stale-hash →ME002).