feat(OPEN-11243): add GitHub Copilot SDK tracing

[viniciusdsmello]

Traces agents built on the GitHub Copilot SDK (github-copilot-sdk on PyPI, imported as copilot) to Openlayer.

from openlayer.lib import trace_copilot
trace_copilot()

patches CopilotClient.create_session, so every session is traced with no change to the code that builds them. openlayer_event_handler() is the explicit alternative for callers who build sessions themselves. Both compose with a user-supplied on_event and never replace it; an exception in either handler cannot break the other.

Trace shape

One trace per send():

AGENT  "GitHub Copilot"              user prompt in, final answer out
├─ CHAT_COMPLETION  "turn 0"         model, provider, tokens, latency
├─ TOOL  "bash"                      arguments in, result out
├─ AGENT "subagent: Explore Agent"   a `task` dispatch
│   ├─ CHAT_COMPLETION "turn 0"
│   └─ TOOL "view"
└─ CHAT_COMPLETION  "turn 1"

Every trace carries the Copilot session id, so multi-turn conversations group into one session rather than unrelated rows.

Why buffer instead of building live

Two wire facts pull in opposite directions:

• assistant.usage — which carries every token count — is ephemeral and absent from get_events(). A 3-turn session emits 153 live events but replays only 19, so we have to subscribe live.
• Copilot fires tool calls concurrently: three tool.execution_start arrive before any completion, and completions come back out of order. Building steps from the live callbacks would nest siblings inside one another.

So we buffer live and build the whole trace in one deterministic, correctly-nested pass at session.idle. This is the one place the Claude Agent SDK integration's pattern is deliberately not copied — its _ToolStepHandle docstring notes it assumes serial pre/post tool pairs, which does not hold here.

Cost

Copilot's cost field is premium-request units, not dollars — a flat per-model multiplier, identical on every call regardless of size — so it is recorded as metadata rather than published as cost. We emit provider + model and let Openlayer price it, mapping the model prefix to the real underlying vendor. llm-costs has no github provider at all, so labelling it that way would silently yield $0.

That price is not an approximation. GitHub meters each call in AIU and ships its own per-token rates on the wire, and those rates are the vendor's list prices scaled by exactly 100 (1 AIU = $0.01):

Model	GitHub AIU/1M (in / out / cache-read)	Vendor list USD/1M
claude-haiku-4.5	100 / 500 / 10	Anthropic $1.00 / $5.00 / $0.10
gpt-5-mini	25 / 200 / 2.5	OpenAI $0.25 / $2.00 / $0.025

Each chat step records GitHub's own figure as copilot_metered_cost_usd, so the priced cost is independently checkable on every row — on live traces the two agree to twelve decimal places. A model whose prefix we cannot map falls back to that figure rather than landing at $0.

Tokens: input_tokens is a superset already containing cache reads and writes, so usage_details is emitted as a non-overlapping partition — which reproduces Copilot's own _token_details breakdown exactly.

Testing

33 unit tests driven by two real captured sessions (55 and 93 events), covering concurrent tools, subagent nesting, the apiCallId usage join, the token partition, handler composition, and the monkey-patch itself.

Plus a live end-to-end test, gated on OPENLAYER_COPILOT_LIVE_TEST=1 and Python 3.11+ — the Copilot SDK's own floor, while this SDK still supports 3.9, which is why the unit tests run off fixtures rather than the real package.

Two live-only shape bugs that fixtures structurally cannot catch have explicit regression tests: the binding hands back SessionEventType.SESSION_START (an Enum) rather than a string, and nests copilot_usage as a dataclass rather than a dict. Both silently produced empty output before being found by the live gate.

Verified against real ingest — a published row comes back with provider: anthropic, a populated per-category costDetails, the user's actual prompt, and the final answer as output.

No new failures against baseline; lint clean.

Example

examples/tracing/copilot_sdk/copilot_sdk_tracing.ipynb — basic session, client-side tool with a subagent dispatch, and multi-turn session grouping.

Related

• feat(OPEN-11243): add GitHub Copilot SDK tracing openlayer-ts#223 — TypeScript parity
• openlayer-ai/openlayer-docs#248 — customer-facing docs page

Deliberately does not use closes, since OPEN-11243 spans three PRs.

🤖 Generated with Claude Code

https://claude.ai/code/session_01S48tdMHz7rjd7aJeZCkVic