Skip to content

AI Workspace: show a screen built from A2UI messages (experiment) - #1332

Draft
whyisjake wants to merge 1 commit into
feat/ai-workspacefrom
experiment/ai-workspace-a2ui
Draft

whyisjake wants to merge 1 commit into
feat/ai-workspacefrom
experiment/ai-workspace-a2ui

Conversation

@whyisjake

@whyisjake whyisjake commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

What?

Related: #1004

An experiment, not meant to merge as is. It lets the AI Workspace model show the person a small screen (a summary card, a comparison, a short form) built from A2UI messages. The screen is rendered with the WordPress component library through Pascal Birchler's a2ui-wp renderer (@swissspidy/a2ui-wp 0.2.0). A button press on the screen goes back to the model as the person's next message.

Why?

Pascal's renderer draws agent-described screens with @wordpress/components. Its README lists "tools for the agent" from the Abilities API as the next step, and that is what the workspace already has. This PR checks how the two fit together, and what it would cost to ship.

How?

  • ai/show-screen (Show_Screen.php), modelled on ai/ask-user:

    • It takes A2UI v0.9.1 messages, writes nothing, and ends the turn.
    • It is hidden from REST and MCP, and refuses outside a workspace turn.
  • What the server allows:

    • the basic catalog only, without Image, Video and AudioPlayer, whose URLs would let the model make the browser fetch any address;
    • exactly one createSurface;
    • at most 20 messages and 32 KB.

    Each refusal tells the model what to change.

  • The system instruction gets a short A2UI guide when the tool is offered. It is adapted from a2ui-wp's agent instructions, with credit. It tells the model to use only data it looked up, and that a screen never changes the site: changes still go through a proposal and its approval card.

  • The transcript (Screen.tsx) renders each screen with its own renderer instance.

    • The client also removes the media components, and limits openUrl to this site.
    • Once the conversation moves on, the screen is made inert and dimmed, like a question card.
  • A button press is sent as [screen] <event> "<button text>" {context}, so the model gets the event and its data. The person sees Chose "<button text>" and the readable values, not the JSON.

  • The ai/show-screen tool is added to the workspace's default tool list (Tool_Selector) and to the abilities that end a turn (Turn_Runner).

Two problems came up while building it:

  • PHP turns {} into [] when it decodes the model's tool call, and the renderer refuses "context": []. The client restores empty objects under context, args and theme. Any ability that passes free-form JSON through the workspace would hit the same thing.
  • Version mismatch: the renderer needs @wordpress/icons 17 or later, and the plugin has 12.2. A peer dependency cannot have its own nested copy, so wp-icons-for-a2ui (an npm alias for @wordpress/icons@19.0.0) is installed. A webpack rule points the renderer's @wordpress/icons import at it, and only inside that package.

Use of AI Tools

AI assistance: Yes. Tool(s): Claude Code (Claude Opus 5.5). Used for: implementing the experiment and its tests, the live and browser checks, and writing this description.

I've reviewed the changes, understand them and can explain them under review. They have all been tested locally.

Testing Instructions

  1. Turn on the AI Workspace, connect a provider that supports function calling, and add a few posts in a few categories.
  2. In Tools → AI Workspace, ask: "Give me an overview of what is on the site: published posts and drafts by category. Show it as a screen."
  3. Ask: "Compare the two drafts side by side so I can pick which to finish first." Press a button on the screen. Your message reads Chose "…" with the post title, the screen dims, and the model answers.
  4. Ask for a planning form ("Show a screen where I pick which categories to focus on and how many posts per week"), tick a category, move the slider and submit. The picks stay ticked after the press. The model's next step is a draft proposal for you to approve; nothing is written until then.

Tests run (all passing):

  • PHPUnit:

    • full suite on WordPress latest: 3219 tests, 10901 assertions, 0 failures, 57 skipped;
    • after the last PHP change, the AI Workspace tests again: 179 tests, 653 assertions.
    • New: tests/Integration/Includes/Experiments/AI_Workspace/Show_ScreenTest.php:
      • test_returns_a_valid_screen_unchanged;
      • test_refuses_an_invalid_screen, with ten cases: an image, a video, an unknown component, another catalog, no createSurface, two createSurface, a deleteSurface, two message types in one message, another version, and too large;
      • test_refuses_outside_a_workspace_turn;
      • test_is_hidden_from_rest;
      • test_ends_the_turn.
    • Tool_SelectorTest and Tool_PolicyTest now list ai/show-screen among the default workspace tools.
  • Live check (Anthropic claude-sonnet-5, through the workspace REST route and in the browser):

    • an overview card, a two-draft comparison and a planning form;
    • every screen passed @a2ui/web_core's own validation, and the counts matched the site;
    • a plain question ("How many published posts are there?") got the question card, not a screen;
    • after a press on the planning form, the model proposed drafts, and the site's post count was unchanged.
  • Browser (headless Chromium, full UI):

    • presses render as Chose "…", and the screen goes inert after a press;
    • checkbox state matches what is sent, by box or by label;
    • no console or page errors.

    This caught a bug: the screen reset to its starting values after a press, because a new renderer was made on every render. That is fixed by keying on the messages' content.

  • Other checks: tsc, lint-js, phpcs and PHPStan (full project) are clean, and the build succeeds.

  • Not run:

    • no e2e spec was added;
    • multisite;
    • providers other than Anthropic, whose stricter tool schemas may not accept free-form objects.

Screenshots

1-overview-screen 2-compare-after 3-plan-screen 3-plan-after-crop
  1. An overview screen built from the site's posts and categories.
  2. The draft comparison after a press: the screen is dimmed, the press reads Chose "Finish this first" with the post's title, and the model answers.
  3. The planning form: a category picker, a slider and a submit button.
  4. The press on the form, and the model's next step: a draft proposal waiting for approval.

Known limits

  • Bundle size: the AI Workspace script grows by roughly 105 KB gzipped (515 KB against 407 KB for a build made shortly before the base commit), mostly zod (required by @a2ui/web_core) and the second icons copy.
  • License: @a2ui/web_core and @swissspidy/a2ui-wp are Apache-2.0. Bundling them makes the built script GPL-3.0-or-later, and the plugin ships as GPL-2.0-or-later. This needs a decision before anything like this merges.
  • Reopened chats: a screen redraws with its starting values. What was typed or picked is not saved, though the press message keeps what was sent.
  • Press text: values from the context are shown as sent, so a category can show as its slug (remote-work).
  • Protocol version: A2UI v0.9.1 only. The renderer does not implement the v1.0 candidate.

Changelog Entry

Experimental: the AI Workspace can show a small screen built from A2UI messages and rendered with WordPress components. Not for release.


🤖 Generated with Claude Code

Open WordPress Playground Preview

Adds ai/show-screen, which takes A2UI v0.9.1 messages and ends the
turn, like ai/ask-user. The transcript renders the screen with
@swissspidy/a2ui-wp (WordPress components), and a button press is sent
as the person's next message, shown as the button's text.

The server allows the basic catalog only, without Image, Video and
AudioPlayer, and limits a screen to 20 messages and 32 KB. The client
removes the media components as well and restricts openUrl to this site.
A screen never writes: changes still go through a proposal and its
approval card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 8, 2026

Copy link
Copy Markdown

✅ WordPress Plugin Check Report

✅ Status: Passed

📊 Report

All checks passed! No errors or warnings found.


🤖 Generated by WordPress Plugin Check Action • Learn more about Plugin Check

@codecov

codecov Bot commented Nov 8, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.89189% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.71%. Comparing base (fa40f00) to head (48242a1).

Files with missing lines Patch % Lines
includes/Experiments/AI_Workspace/Show_Screen.php 92.59% 8 Missing ⚠️
includes/Experiments/AI_Workspace/Turn_Runner.php 50.00% 1 Missing ⚠️
Additional details and impacted files
@@                   Coverage Diff                   @@
##             feat/ai-workspace    #1332      +/-   ##
=======================================================
+ Coverage                85.67%   85.71%   +0.03%     
- Complexity                5873     5904      +31     
=======================================================
  Files                      217      217              
  Lines                    21466    21558      +92     
=======================================================
+ Hits                     18391    18478      +87     
- Misses                    3075     3080       +5     
Flag Coverage Δ
unit 85.71% <91.89%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jeffpaul jeffpaul added this to the 1.6.0 milestone Nov 9, 2026
@jeffpaul jeffpaul modified the milestones: 1.6.0, 1.7.0 Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants