-
Notifications
You must be signed in to change notification settings - Fork 3.6k
feat: add the snapshot and dom tap commands for reading AUT DOM #34255
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
davidr-cy
wants to merge
10
commits into
davidr/feat/tap-cli/tap-status
Choose a base branch
from
davidr/feat/tap-cli/tap-snapshots
base: davidr/feat/tap-cli/tap-status
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
cd1bb41
feat: add the snapshot and dom tap commands for reading AUT DOM
davidr-cy ea386a5
refactor: replace app-side snapshot/dom with CDP-native tap frame com…
davidr-cy b9eb410
feat: add the tap frame ax command (accessibility tree over CDP)
davidr-cy 34dc1d7
feat: add the pin tap command to make a past command's DOM live
davidr-cy 014b9d3
feat: add tap frame inspect and report the pinned snapshot in status
davidr-cy 159786e
refactor: rename the tap frame ax command to aria
davidr-cy 11723e2
fix: auto-release a tap pin when the run changes
davidr-cy 1939fd5
feat: reflect the tap pin in the runner UI and move it with --at
davidr-cy 4c1b2ea
refactor: flatten the tap frame subcommands into top-level dom, aria,…
davidr-cy 7c7484c
fix: adapt the tap pin command to the attempt-based test-state helpers
davidr-cy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| import Debug from 'debug' | ||
| import type CRI from 'chrome-remote-interface' | ||
|
|
||
| const debug = Debug('cypress:cli:tap') | ||
|
|
||
| // The app names the AUT iframe deterministically (packages/app/src/runner/ | ||
| // aut-iframe.ts): its name is `Your project: '<name>'`. That prefix is how we | ||
| // pick it out of the runner page's frame tree — distinct from the snapshot | ||
| // double-buffer frames (`AUT Snapshot - N`) and the spec bridge (`Your Spec`). | ||
| const AUT_FRAME_NAME_PREFIX = 'Your project:' | ||
|
|
||
| /** | ||
| * A failure in a frame command (dom/aria/inspect), surfaced to the user as `{ code, message }` | ||
| * via `renderFailure`. The frame commands are CLI-native and never cross the | ||
| * binding's `exec` envelope, so this mirrors the binding's `TapCommandError` | ||
| * on the CLI side. | ||
| */ | ||
| export class FrameCommandError extends Error { | ||
| code: string | ||
|
|
||
| constructor (code: string, message: string) { | ||
| super(message) | ||
| this.name = 'FrameCommandError' | ||
| this.code = code | ||
| } | ||
| } | ||
|
|
||
| export interface AutFrame { | ||
| /** CDP frameId — scopes `Accessibility.getFullAXTree` and `Page.createIsolatedWorld`. */ | ||
| frameId: string | ||
| url: string | ||
| } | ||
|
|
||
| interface FrameNode { | ||
| frame: { id: string, name?: string, url?: string } | ||
| childFrames?: FrameNode[] | ||
| } | ||
|
|
||
| const findAutFrame = (node: FrameNode): { id: string, url: string } | undefined => { | ||
| if ((node.frame.name ?? '').startsWith(AUT_FRAME_NAME_PREFIX)) { | ||
| return { id: node.frame.id, url: node.frame.url ?? '' } | ||
| } | ||
|
|
||
| for (const child of node.childFrames ?? []) { | ||
| const found = findAutFrame(child) | ||
|
|
||
| if (found) { | ||
| return found | ||
| } | ||
| } | ||
|
|
||
| return undefined | ||
| } | ||
|
|
||
| /** | ||
| * Locates the app-under-test frame within the runner page. Verified empirically: | ||
| * the AUT is a same-process child frame of the runner-page target, so one | ||
| * attached session reaches it — no separate target attach. A pinned snapshot is | ||
| * always same-super-domain, so it is reliably this child frame too. | ||
| */ | ||
| export const resolveAutFrame = async (client: CRI.Client, sessionId: string): Promise<AutFrame> => { | ||
| await client.Page.enable({}, sessionId) | ||
|
|
||
| // getFrameTree takes no params, so its typed signature is (sessionId?); CRI | ||
| // routes the string arg to sessionId by type, not position. | ||
| const { frameTree } = await client.Page.getFrameTree(sessionId) | ||
| const found = findAutFrame(frameTree as FrameNode) | ||
|
|
||
| if (!found) { | ||
| throw new FrameCommandError('NO_AUT_FRAME', 'no app under test is loaded — run a spec first (and, to read a past command, pin it with the pin command)') | ||
| } | ||
|
|
||
| debug('resolved AUT frame %o', found) | ||
|
|
||
| return { frameId: found.id, url: found.url } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,206 @@ | ||
| import type { TapSession } from '../tap-session' | ||
| import type { AutFrame } from '../aut-frame' | ||
| import { FrameCommandError } from '../aut-frame' | ||
|
|
||
| // The accessibility tree of a real app is deep; cap the projection so it stays | ||
| // affordable for an LLM. A selector roots it at a subtree for finer reads. | ||
| export const DEFAULT_MAX_NODES = 200 | ||
|
|
||
| // Structural/text roles carry no semantic signal on their own — dropping them | ||
| // yields the compact role/name tree DevTools shows, not the raw render tree. | ||
| const NOISE_ROLES = new Set(['InlineTextBox', 'StaticText', 'LineBreak', 'generic', 'none', 'GenericContainer', 'paragraph']) | ||
|
|
||
| // The boolean states worth reporting when true; the rest are rarely actionable. | ||
| const REPORTED_STATES = new Set(['focused', 'disabled', 'required', 'invalid', 'checked', 'expanded', 'selected', 'pressed', 'readonly', 'hidden', 'modal', 'busy']) | ||
|
|
||
| interface AXValue { value?: unknown } | ||
| interface AXProperty { name: string, value?: AXValue } | ||
| interface AXNode { | ||
| nodeId: string | ||
| parentId?: string | ||
| role?: AXValue | ||
| name?: AXValue | ||
| value?: AXValue | ||
| properties?: AXProperty[] | ||
| childIds?: string[] | ||
| ignored?: boolean | ||
| backendDOMNodeId?: number | ||
| } | ||
|
|
||
| export interface AriaNodeOut { | ||
| depth: number | ||
| role: string | ||
| name?: string | ||
| value?: string | ||
| states?: string[] | ||
| } | ||
|
|
||
| export interface FrameAriaResult { | ||
| url?: string | ||
| nodes: AriaNodeOut[] | ||
| nodeCount: number | ||
| truncated?: true | ||
| } | ||
|
|
||
| const projectNode = (node: AXNode, depth: number): AriaNodeOut | undefined => { | ||
| const role = node.role?.value | ||
|
|
||
| if (typeof role !== 'string' || node.ignored || NOISE_ROLES.has(role)) { | ||
| return undefined | ||
| } | ||
|
|
||
| const out: AriaNodeOut = { depth, role } | ||
| const name = node.name?.value | ||
|
|
||
| if (typeof name === 'string' && name.length > 0) { | ||
| out.name = name | ||
| } | ||
|
|
||
| const value = node.value?.value | ||
|
|
||
| if (value !== undefined && value !== null && value !== '') { | ||
| out.value = String(value) | ||
| } | ||
|
|
||
| const states = (node.properties ?? []) | ||
| .filter((property) => REPORTED_STATES.has(property.name) && property.value?.value === true) | ||
| .map((property) => property.name) | ||
|
|
||
| if (states.length) { | ||
| out.states = states | ||
| } | ||
|
|
||
| return out | ||
| } | ||
|
|
||
| /** | ||
| * Walks the AX forest from `rootId` depth-first. A node that projects to a | ||
| * meaningful role deepens the indent for its descendants; a noise node is | ||
| * skipped but its children keep flowing, so the output is a clean semantic | ||
| * tree rather than the raw render tree. Stops at `maxNodes`. | ||
| */ | ||
| const projectTree = (byId: Map<string, AXNode>, rootId: string, maxNodes: number): { nodes: AriaNodeOut[], truncated: boolean } => { | ||
| const nodes: AriaNodeOut[] = [] | ||
| let truncated = false | ||
|
|
||
| const walk = (id: string, depth: number): void => { | ||
| if (truncated) { | ||
| return | ||
| } | ||
|
|
||
| const node = byId.get(id) | ||
|
|
||
| if (!node) { | ||
| return | ||
| } | ||
|
|
||
| const projected = projectNode(node, depth) | ||
|
|
||
| if (projected) { | ||
| if (nodes.length >= maxNodes) { | ||
| truncated = true | ||
|
|
||
| return | ||
| } | ||
|
|
||
| nodes.push(projected) | ||
| } | ||
|
|
||
| const childDepth = projected ? depth + 1 : depth | ||
|
|
||
| for (const childId of node.childIds ?? []) { | ||
| walk(childId, childDepth) | ||
| } | ||
| } | ||
|
|
||
| walk(rootId, 0) | ||
|
|
||
| return { nodes, truncated } | ||
| } | ||
|
|
||
| const resolveSelectorBackendNodeId = async ( | ||
| session: TapSession, | ||
| frame: AutFrame, | ||
| selector: string, | ||
| ): Promise<number | undefined> => { | ||
| const { client, sessionId } = session | ||
|
|
||
| const { executionContextId } = await client.Page.createIsolatedWorld({ | ||
| frameId: frame.frameId, | ||
| worldName: 'cypress-tap', | ||
| }, sessionId) | ||
|
|
||
| const { result, exceptionDetails } = await client.Runtime.callFunctionOn({ | ||
| functionDeclaration: 'function (selector) { return document.querySelector(selector) }', | ||
| executionContextId, | ||
| arguments: [{ value: selector }], | ||
| }, sessionId) | ||
|
|
||
| if (exceptionDetails) { | ||
| throw new FrameCommandError('INVALID_SELECTOR', `"${selector}" is not a valid CSS selector`) | ||
| } | ||
|
|
||
| // querySelector returned null — a real "nothing matched" answer, not an error. | ||
| if (!result.objectId || result.subtype === 'null') { | ||
| return undefined | ||
| } | ||
|
|
||
| const { node } = await client.DOM.describeNode({ objectId: result.objectId }, sessionId) | ||
|
|
||
| return node.backendNodeId | ||
| } | ||
|
|
||
| export const extractAria = async ( | ||
| session: TapSession, | ||
| frame: AutFrame, | ||
| selector: string | undefined, | ||
| maxNodes: number, | ||
| ): Promise<FrameAriaResult> => { | ||
| const { client, sessionId } = session | ||
|
|
||
| await client.DOM.enable({}, sessionId) | ||
| await client.Accessibility.enable(sessionId) | ||
|
|
||
| const { nodes: axNodes } = await client.Accessibility.getFullAXTree({ frameId: frame.frameId }, sessionId) | ||
|
|
||
| const byId = new Map<string, AXNode>() | ||
|
|
||
| for (const node of axNodes as AXNode[]) { | ||
| byId.set(node.nodeId, node) | ||
| } | ||
|
|
||
| const base: FrameAriaResult = { ...(frame.url ? { url: frame.url } : {}), nodes: [], nodeCount: 0 } | ||
|
|
||
| let rootId: string | undefined | ||
|
|
||
| if (selector !== undefined) { | ||
| const backendNodeId = await resolveSelectorBackendNodeId(session, frame, selector) | ||
|
|
||
| if (backendNodeId === undefined) { | ||
| // Selector matched nothing, or the match is absent from the a11y tree. | ||
| return base | ||
| } | ||
|
|
||
| rootId = (axNodes as AXNode[]).find((node) => node.backendDOMNodeId === backendNodeId)?.nodeId | ||
|
|
||
| if (rootId === undefined) { | ||
| return base | ||
| } | ||
| } else { | ||
| // The frame root is the sole node with no parent (the RootWebArea). | ||
| rootId = (axNodes as AXNode[]).find((node) => node.parentId === undefined)?.nodeId ?? (axNodes as AXNode[])[0]?.nodeId | ||
| } | ||
|
|
||
| if (rootId === undefined) { | ||
| return base | ||
| } | ||
|
|
||
| const { nodes, truncated } = projectTree(byId, rootId, maxNodes) | ||
|
|
||
| return { | ||
| ...base, | ||
| nodes, | ||
| nodeCount: nodes.length, | ||
| ...(truncated ? { truncated: true } : {}), | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.