Skip to content

fix(aiops): remove implicit 15-second tool execution deadline #523

Closed

Description

@singchia

Problem

On the v3 test build (v0.18.2-v3-test.8b1c1db8), disk diagnosis reports that du scanning timed out and falls back to host_bash.

Code inspection confirms that the shared tool decorator defaults to 15 seconds and the chat composition root explicitly applies that budget to the startup tool catalog, including du_summary. Workflow tools repeat the same default. This silently overrides longer tool-specific execution budgets. The v3 Graph Config also advertises a 15-second default although its ToolTimeout field is not wired to execution.

Reproduction

  1. Ask the agent to diagnose disk usage on a sufficiently large directory tree.
  2. Invoke du_summary where scanning takes more than 15 seconds.
  3. The generic wrapper expires before the filesystem tool's own budget; the agent may retry through host_bash.

The reported user symptom is confirmed; the exact affected live invocation has not been traced. This is a code-confirmed timeout defect, not a claim that every scan failure has the same cause.

Expected behavior / fix

  • No implicit global 15-second timeout around tool execution.
  • Zero/unset decorator timeout adds no deadline; explicitly configured positive timeouts remain supported.
  • Preserve parent cancellation/deadlines, approvals, auditing, rate limits and metrics.
  • Remove the chat/workflow hardcoded 15-second wrapper and the misleading v3 Graph default. Keep archived v1/v2 source unchanged.
  • Cover default behavior, explicit timeout, caller cancellation/deadline, and a tool execution lasting over 15 seconds with race tests.

Scope and remaining limits

This fix removes the generic 15-second restriction, not all operational deadlines. The filesystem tool currently has a 60-second Manager round-trip limit, 30-second Edge per-path limit and 60-second Edge batch limit. host_bash has independent Manager/Edge/sandbox budgets. These must not be represented as unlimited execution after this fix.

Tracked on the current runtime requirements branch refactor/unified-agent-runtime, related to #503. No merge or production release is requested.

Activity

  1. singchia commented on Oct 16, 2026

    @singchia
    MemberAuthor

    Follow-up on the same branch: extend built-in execution budgets before introducing configurable/asynchronous long-task execution.

    • du_summary / find_large_files: 5 minutes per path; 20 minutes for four waves of 16 paths; Manager waits 21 minutes including the separate 30-second filesystem snapshot and response headroom. Shared Manager/Edge constants prevent mismatches.
    • host_bash: default sandbox execution 5 minutes (existing maximum stays 300 seconds); Manager per-device deadline 5m30s and batch deadline accounts for all four waves. Shorter requested deadlines still apply. Operator policy overrides remain authoritative.
    • cloud_bash: explicitly pass a 10-minute budget to the runner instead of inheriting its 2-minute default.
    • Ordinary diagnostic tool dispatch budgets: 60 seconds; edge-summary batches sized for all waves.
    • Built-in journal reads: 2 minutes; network namespace inspection and web search: 60 seconds. Manifest subprocess default: 5 minutes; explicit manifest timeouts remain respected.
    • Connection/probe deadlines, approval expiry and specialized capture/action budgets are unchanged. Backend API clients can still impose their own deadlines; this change does not promise unlimited execution.

    Validation includes a real shell command completing after 31 seconds, a requested 1-second deadline stopping a command, and Manager deadline propagation/parent deadline preservation. Edge changes require an Edge rollout; updating Manager alone does not activate the new scan/sandbox limits. Rollback is restoring the previous Manager/Edge binaries; no database or wire schema changes.

  2. RyoKusnadi commented on Oct 17, 2026

    @RyoKusnadi
    Contributor
  3. RyoKusnadi commented on Oct 17, 2026

    @RyoKusnadi
    Contributor

    Status update:

    This issue stays open until #525 lands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions