Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/fuzzy-turns-rollback.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"eve": patch
---

Hooks can inspect a completed conversation response with `beforeResponseRelease` and return `"skip"` to suppress terminal channel delivery. Earlier events, model history, and external side effects remain unchanged.
28 changes: 27 additions & 1 deletion docs/guides/hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,33 @@ The slug is the path-relative basename. `agent/hooks/audit.ts` becomes `"audit"`

`defineHook`, `HookDefinition`, and `HookContext` live on `eve/hooks`.

A hook file declares stream-event subscribers under the `events` map, keyed by event type, with `*` matching every event. Subscribe to any event in the runtime stream vocabulary documented in [Sessions, runs and streaming](../concepts/sessions-runs-and-streaming), including the lifecycle events `session.started`, `turn.completed`, `message.completed`, `action.partial`, and `action.result`. Handlers are observe-only. They cannot inject model context. To contribute runtime model messages, use `defineDynamic` and `defineInstructions` in `agent/instructions/`.
A hook file declares stream-event subscribers under the `events` map, keyed by event type, with `*` matching every event. Subscribe to any event in the runtime stream vocabulary documented in [Sessions, runs and streaming](../concepts/sessions-runs-and-streaming), including the lifecycle events `session.started`, `turn.completed`, `message.completed`, `action.partial`, and `action.result`. Stream-event handlers are observe-only. They cannot inject model context. To contribute runtime model messages, use `defineDynamic` and `defineInstructions` in `agent/instructions/`.

## Gate terminal response release

Use `beforeResponseRelease` when application policy must inspect a completed response before eve
releases its terminal completion to the channel:

```ts title="agent/hooks/review.ts"
import { defineHook } from "eve/hooks";

export default defineHook({
beforeResponseRelease(candidate) {
return shouldSuppress(candidate.history, candidate.output) ? "skip" : undefined;
},
});
```

The hook receives candidate model history, terminal output, and `turnId`. Returning `"skip"`
suppresses the withheld terminal `message.completed` event. Returning `undefined` releases it. If
several hooks are present, eve stops at the first `"skip"` decision.

This is a response-release boundary, not a private execution environment. Earlier events
have already run through channel handlers, the durable stream, memory, instrumentation, and ordinary
hooks. Model providers, tools, external systems, sandboxes, subagents, and background tasks may also
retain or continue acting on candidate content. The hook itself cannot durably pause for HITL; use a
tool or workflow for durable human input, then inspect its model-visible record before response
release.

## Scope side effects to a channel

Expand Down
9 changes: 9 additions & 0 deletions packages/eve/extension-contracts/compatibility/hook/v22.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import { defineHook } from "#public/hooks/index.js";

export default defineHook({
events: {
"turn.started"(event, ctx) {
console.info(event.meta.id, event.data.turnId, ctx.session.id);
},
},
});
7 changes: 7 additions & 0 deletions packages/eve/extension-contracts/reports/hook/v23.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"kind": "eve-extension-capability-contract",
"capability": "hook",
"epoch": 23,
"sha256": "bdc1bf4bf929bdd09c72c2863c2ff9ab4655338b779f117832fc6de9c8400f91",
"exports": ["defineHook"]
}
4 changes: 2 additions & 2 deletions packages/eve/src/compiler/extension-compatibility.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,8 @@ const EXTENSION_CAPABILITY_CONTRACTS = {
},
},
hook: {
current: 22,
supported: [10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22],
current: 23,
supported: [10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23],
dropped: {
1: "Model identity moved from session.started runtime metadata to step.started call attribution.",
2: "Model identity moved from session.started runtime metadata to step.started call attribution.",
Expand Down
7 changes: 7 additions & 0 deletions packages/eve/src/compiler/normalize-hook.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ export async function compileHookEntry(
}),
`Expected the hook export "${source.exportName ?? "default"}" from "${source.logicalPath}" to return an object.`,
);
const beforeResponseRelease = loaded.beforeResponseRelease;
if (beforeResponseRelease !== undefined) {
expectFunction(
beforeResponseRelease,
`Expected the hook export "${source.exportName ?? "default"}" from "${source.logicalPath}" to provide a function for beforeResponseRelease.`,
);
}
const events =
loaded.events === undefined
? {}
Expand Down
28 changes: 27 additions & 1 deletion packages/eve/src/context/hook-lifecycle.integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
import { stampTestEvent } from "#internal/testing/events.js";
import { mockSandbox } from "#internal/testing/mocks/mock-sandbox.js";
import { ContextContainer, contextStorage } from "./container.js";
import { dispatchStreamEventHooks } from "./hook-lifecycle.js";
import { dispatchBeforeResponseReleaseHooks, dispatchStreamEventHooks } from "./hook-lifecycle.js";
import {
BundleKey,
ChannelKey,
Expand Down Expand Up @@ -55,6 +55,7 @@ function buildCtx(): ContextContainer {

function hook(slug: string, hooks: Partial<ResolvedHookDefinition>): ResolvedHookDefinition {
return {
beforeResponseRelease: hooks.beforeResponseRelease,
events: hooks.events ?? {},
exportName: undefined,
logicalPath: `hooks/${slug}.ts`,
Expand All @@ -64,6 +65,31 @@ function hook(slug: string, hooks: Partial<ResolvedHookDefinition>): ResolvedHoo
};
}

describe("dispatchBeforeResponseReleaseHooks", () => {
it("runs every pre-release hook in order", async () => {
const calls: string[] = [];
const registry = createRuntimeHookRegistry([
hook("first", { beforeResponseRelease: async () => void calls.push("first") }),
hook("second", { beforeResponseRelease: async () => void calls.push("second") }),
]);
const ctx = buildCtx();

await contextStorage.run(ctx, () =>
dispatchBeforeResponseReleaseHooks({
candidate: {
history: [],
output: "candidate",
turnId: "turn_0",
},
ctx,
registry,
}),
);

expect(calls).toEqual(["first", "second"]);
});
});

describe("dispatchStreamEventHooks", () => {
it("invokes typed then wildcard subscribers and propagates errors", async () => {
const calls: string[] = [];
Expand Down
21 changes: 20 additions & 1 deletion packages/eve/src/context/hook-lifecycle.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { BoundaryHookError } from "#shared/boundary-hook-error.js";
import { getAdapterKind } from "#channel/adapter.js";
import type { MessageStreamEvent } from "#protocol/message.js";
import type { HookContext } from "#public/definitions/hook.js";
import type { HookContext, ResponseReleaseCandidate } from "#public/definitions/hook.js";
import type { RuntimeHookRegistry } from "#runtime/hooks/registry.js";
import { buildCallbackContext } from "#context/build-callback-context.js";
import type { ContextContainer } from "./container.js";
Expand Down Expand Up @@ -42,6 +42,25 @@ export async function dispatchStreamEventHooks(input: {
}
}

/** Runs ordered pre-release hooks. */
export async function dispatchBeforeResponseReleaseHooks(input: {
readonly candidate: ResponseReleaseCandidate;
readonly ctx: ContextContainer;
readonly registry: RuntimeHookRegistry;
}): Promise<"release" | "skip"> {
const hookCtx = buildHookContext(input.ctx);
for (const entry of input.registry.beforeResponseRelease) {
const decision: unknown = await entry.handler(input.candidate, hookCtx);
if (decision !== undefined && decision !== "skip") {
throw new Error(
`Hook "${entry.slug}" returned ${JSON.stringify(decision)} from beforeResponseRelease; expected undefined or "skip".`,
);
}
if (decision === "skip") return "skip";
}
return "release";
}

/** Builds the {@link HookContext} surfaced to one handler. */
function buildHookContext(ctx: ContextContainer): HookContext {
const bundle = ctx.require(BundleKey);
Expand Down
4 changes: 4 additions & 0 deletions packages/eve/src/execution/node-step.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ export interface CreateExecutionNodeStepInput {
*/
readonly createRuntime: CreateRuntime;
readonly handleEvent?: HandleEventFn;
readonly beforeResponseRelease?: Parameters<
typeof createToolLoopHarness
>[0]["beforeResponseRelease"];
readonly historyProjector?: HistoryViewProjector;
readonly historyView?: PreparedHistoryView;
readonly instrumentation: ExecutionInstrumentation | undefined;
Expand Down Expand Up @@ -108,6 +111,7 @@ export function createExecutionNodeStep(input: CreateExecutionNodeStepInput): St
compactOnly: input.compactOnly,
workflow: input.node.agent.workflowTool !== undefined,
workflowMaxSubagents: input.workflowMaxSubagents,
beforeResponseRelease: input.beforeResponseRelease,
handleEvent: input.handleEvent,
historyProjector: input.historyProjector,
historyView: input.historyView,
Expand Down
78 changes: 78 additions & 0 deletions packages/eve/src/execution/response-release-event-gate.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import { describe, expect, it, vi } from "vitest";

import { ContextContainer } from "#context/container.js";
import { ResponseReleaseEventGate } from "#execution/response-release-event-gate.js";
import type { RuntimeHookRegistry } from "#runtime/hooks/registry.js";

const { dispatchBeforeResponseReleaseHooks } = vi.hoisted(() => ({
dispatchBeforeResponseReleaseHooks: vi.fn(),
}));

vi.mock("#context/hook-lifecycle.js", () => ({ dispatchBeforeResponseReleaseHooks }));

const terminal = {
data: {
finishReason: "stop" as const,
message: "candidate",
sequence: 0,
stepIndex: 0,
turnId: "turn_0",
},
type: "message.completed" as const,
};

const registry: RuntimeHookRegistry = {
beforeResponseRelease: [{ handler: vi.fn(), slug: "review" }],
streamEventsByType: new Map(),
streamEventsWildcard: [],
};

describe("ResponseReleaseEventGate", () => {
it("withholds then releases a terminal completion when history is retained", async () => {
const gate = new ResponseReleaseEventGate(new ContextContainer(), registry);
const release = vi.fn().mockResolvedValue(undefined);

expect(gate.intercept(terminal)).toBe(true);
await expect(
gate.beforeRelease(release)!({ history: [], output: "candidate", turnId: "turn_0" }),
).resolves.toBeUndefined();
expect(release).toHaveBeenCalledWith(terminal);
});

it("does not intercept task-mode terminal completions", () => {
const gate = new ResponseReleaseEventGate(new ContextContainer(), registry, false);

expect(gate.intercept(terminal)).toBe(false);
expect(gate.beforeRelease(vi.fn())).toBeUndefined();
});

it("does not intercept a response that parks on tool calls", () => {
const gate = new ResponseReleaseEventGate(new ContextContainer(), registry);

expect(
gate.intercept({
...terminal,
data: { ...terminal.data, finishReason: "tool-calls" },
}),
).toBe(false);
});

it("drops a terminal completion when a hook skips release", async () => {
dispatchBeforeResponseReleaseHooks.mockResolvedValueOnce("skip");
const gate = new ResponseReleaseEventGate(new ContextContainer(), registry);
const release = vi.fn().mockResolvedValue(undefined);

expect(gate.intercept(terminal)).toBe(true);
await expect(
gate.beforeRelease(release)!({
history: [
{ content: "keep", role: "user" },
{ content: "remove", role: "assistant" },
],
output: "candidate",
turnId: "turn_0",
}),
).resolves.toBe("skip");
expect(release).not.toHaveBeenCalled();
});
});
70 changes: 70 additions & 0 deletions packages/eve/src/execution/response-release-event-gate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
import type { ContextContainer } from "#context/container.js";
import { dispatchBeforeResponseReleaseHooks } from "#context/hook-lifecycle.js";
import type { ToolLoopHarnessConfig } from "#harness/types.js";
import type { UnstampedMessageStreamEvent } from "#protocol/message.js";
import type { RuntimeHookRegistry } from "#runtime/hooks/registry.js";

/** Holds terminal content while authored hooks inspect the settling turn. */
export class ResponseReleaseEventGate {
private readonly ctx: ContextContainer;
private readonly registry: RuntimeHookRegistry;
private readonly supported: boolean;
private releasing = false;
private terminalEvent: UnstampedMessageStreamEvent | undefined;

constructor(ctx: ContextContainer, registry: RuntimeHookRegistry, supported = true) {
this.ctx = ctx;
this.registry = registry;
this.supported = supported;
}

get enabled(): boolean {
return this.supported && this.registry.beforeResponseRelease.length > 0;
}

/** Returns true when the terminal event was withheld from ordinary delivery. */
intercept(event: UnstampedMessageStreamEvent): boolean {
if (
this.releasing ||
!this.enabled ||
event.type !== "message.completed" ||
event.data.finishReason === "tool-calls"
) {
return false;
}
this.terminalEvent = event;
return true;
}

beforeRelease(
release: (event: UnstampedMessageStreamEvent) => Promise<void>,
): NonNullable<ToolLoopHarnessConfig["beforeResponseRelease"]> | undefined {
if (!this.enabled) return undefined;
return async (candidate) => {
const decision = await dispatchBeforeResponseReleaseHooks({
candidate: {
history: candidate.history,
output: candidate.output,
turnId: candidate.turnId,
},
ctx: this.ctx,
registry: this.registry,
});
if (decision === "skip") {
this.terminalEvent = undefined;
return "skip";
}
if (this.terminalEvent !== undefined) {
const terminalEvent = this.terminalEvent;
this.terminalEvent = undefined;
this.releasing = true;
try {
await release(terminalEvent);
} finally {
this.releasing = false;
}
}
return undefined;
};
}
}
11 changes: 9 additions & 2 deletions packages/eve/src/execution/workflow-steps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ import { resolveEffectiveAgentRuntime } from "#execution/effective-agent-config.
import { reconcileSessionContinuationToken } from "#execution/reconcile-session-continuation-token.js";
import { hydrateDurableSession, refreshSessionFromTurnAgent } from "#execution/session.js";
import { createExecutionHistoryView } from "#execution/history-view.js";
import { ResponseReleaseEventGate } from "#execution/response-release-event-gate.js";
import { resolveRuntimeCompiledArtifactsVersionedCacheKey } from "#runtime/cache-key.js";
import { createWorkflowRuntime } from "#execution/workflow-runtime.js";
import { bindDynamicConnections } from "#execution/dynamic-connections.js";
Expand Down Expand Up @@ -381,6 +382,12 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu
}

const writer = input.parentWritable.getWriter();
const mode = ctx.require(ModeKey);
const responseReleaseGate = new ResponseReleaseEventGate(
ctx,
hookRegistry,
mode === "conversation",
);

// Persisted chunks and hooks must agree on the stamped id.
const emit = async (event: UnstampedMessageStreamEvent): Promise<MessageStreamEvent> => {
Expand All @@ -391,6 +398,7 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu
return stamped;
};
const handleEvent: HandleEventFn = async (event, messages): Promise<void> => {
if (responseReleaseGate.intercept(event)) return;
// A remote task's parent owns its HITL. Forward blocking events over
// the task callback and keep them out of the child's local channel;
// otherwise two TUIs can present and answer the same request.
Expand Down Expand Up @@ -448,8 +456,6 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu
});
};

const mode = ctx.require(ModeKey);

let stepResult: StepResult;
try {
// A signal already aborted at entry (cancellation during an in-line
Expand Down Expand Up @@ -532,6 +538,7 @@ export async function turnStep(rawInput: TurnStepInput): Promise<DurableStepResu

const step = createExecutionNodeStep({
abortSignal: input.abortSignal,
beforeResponseRelease: responseReleaseGate.beforeRelease(handleEvent),
capabilities,
clearOnly: input.input?.kind === "clear",
compactOnly: input.input?.kind === "compact",
Expand Down
Loading