Add AI Tools plugin detection to the user-agent#5924
Open
parthban-db wants to merge 2 commits into
Open
Conversation
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: 20147d1
8 interesting tests: 4 SKIP, 2 RECOVERED, 2 flaky
Top 10 slowest tests (at least 2 minutes):
|
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
f0bf565 to
20e717d
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
20e717d to
7cb15f0
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
changed the title
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
7cb15f0 to
72cf29c
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
72cf29c to
20147d1
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
renaudhartert-db
approved these changes
Jul 14, 2026
Comment on lines
+18
to
+26
| func withAiToolsInUserAgent(ctx context.Context) context.Context { | ||
| // Each tool appends one aitools/<tool>_<version> pair to the same context; | ||
| // these accumulate, so this is not the accidental context nesting fatcontext | ||
| // flags. | ||
| for _, tool := range installer.InstalledTools(ctx) { | ||
| ctx = useragent.InContext(ctx, aiToolsKey, tool) //nolint:fatcontext | ||
| } | ||
| return ctx | ||
| } |
Contributor
There was a problem hiding this comment.
Could we also test this function as this the single top level call point for new functionality? Maybe move it in installer as a public function?
Contributor
Author
There was a problem hiding this comment.
I am not sure which tests you want to add.
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
20147d1 to
2780658
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
parthban-db
force-pushed
the
parthban-db/stack/skills-status
branch
from
2780658 to
7afc61b
Compare
parthban-db
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR
Use this link to review incremental changes.
Changes
Emit an
aitools/<tool>pair in the CLI user-agent for each coding tool that has the Databricks plugin installed (e.g.aitools/claude-code aitools/codex), so plugin adoption is measurable per tool from request telemetry. When no tool has it installed, noaitools/pair is added at all. A tool counts as installed if either signal says so: the plugin is present in the tool's own plugin cache (<configDir>/plugins/cache/*/databricks), or the plugin is recorded in the CLI install state. Detection runs over every agent in the registry that ships a plugin (Claude Code, Codex, Copilot); skills-only agents are not reported.Why
Coding-agent usage is already visible in the user-agent (the SDK's
agent/dimension), but whether Databricks AI Tools are actually installed is not, so adoption cannot be measured. Reading only the CLI install state would undercount badly: the Databricks plugin is published in the agents' own plugin marketplaces, so an install done directly through the agent (not viadatabricks aitools install) leaves nothing in the CLI state. The plugin cache on disk is the ground truth that captures those direct installs. Taking the union of the on-disk cache and the CLI state maximizes coverage and only ever over-counts a stale record, never hides a real install — the right bias for an adoption signal.Tests
Unit tests for on-disk plugin detection (
HasDatabricksPlugin, including the varying marketplace segment and wrong-plugin-id cases) and forInstalledToolscovering disk-only (direct-marketplace install), state-only across global and project scope, the disk+state union with dedup, and skipping records for agents no longer in the registry. Acmd/roottest asserts the user-agent emission contract (oneaitools/<tool>pair per installed tool, nothing when none). The existing user-agent acceptance test is unaffected: the test harness runs with an isolated empty home directory, so the dimension stays silent.