Skip to content

engine: the audit trail grows with every task of every loop sweep, even in an untraced run #373

Closed

Description

@codetiger

Summary

Every executed task pushes an AuditTrail entry onto message.audit_trail (dataflow-rs 3.14.0 workflow_executor.rs:2518), and nothing prunes it — including in an untraced run (task_details: false, capture_changes off), where nothing reads it before the run ends.

In a loop workflow that is O(tasks × sweeps) entries held for the life of the run: a 1000-sweep turn loop with ~7 + 2·seats tasks per sweep holds ~23k entries at 8 seats. Each is small (two Arc<str> and a timestamp), but it is also deep-cloned into every for_each element (#372), so it multiplies.

Proposal

When the run is untraced, keep a bounded audit trail (e.g. the last N entries, or counts only), or none; or clear the per-sweep entries at each loop sweep the way scratch is reset. #350 made capture opt-in for the same reason — the audit trail is the remaining per-sweep growth in an untraced loop.

Activity

  1. codetiger commented on Oct 1, 2026

    @codetiger
    MemberAuthor

    Fixed on branch fix/issues-369-377 (not yet pushed; it lands after PR #368) via dataflow-rs 3.15's AuditMode (GoPlasmatic/dataflow-rs#68). execute_admitted builds every untraced serving message with AuditMode::Off; nothing on that path reads the trail. A traced run keeps Full because its per-step diff is read from the entries, and the offline test runner keeps it for case-file assertions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions