Skip to content

Python: preserve multimodal messages in AG-UI chat client requests - #8116

Merged
Evan Mattson (moonbox3) merged 8 commits into
microsoft:mainfrom
CoralGarden52:fix/ag-ui-multimodal-input-8114
Sep 9, 2026
Merged

Python: preserve multimodal messages in AG-UI chat client requests#8116
Evan Mattson (moonbox3) merged 8 commits into
microsoft:mainfrom
CoralGarden52:fix/ag-ui-multimodal-input-8114

Conversation

@CoralGarden52

Copy link
Copy Markdown
Contributor

Summary

  • Preserve ordered text, URI, and inline data content when Python AGUIChatClient serializes user messages for AG-UI.
  • Emit current AG-UI InputContent source fields (source.type, source.value, and mimeType) with MIME-aware image/audio/video/document part types.
  • Keep text-only messages, tool calls/results, and assistant message string content backward compatible.
  • Add adapter regressions plus a real httpx.MockTransport request JSON regression test.

Validation

  • PYTHONPATH="$PWD/python/packages/core:$PWD/python/packages/ag-ui:$PWD/python/packages/orchestrations" /home/nyc/agent-framework/python/.venv/bin/pytest python/packages/ag-ui/tests/ag_ui -q — 1100 passed, 1 skipped.
  • uvx --from ruff==0.16.4 ruff check ... — passed.
  • uvx --from ruff==0.16.4 ruff format --check ... — 3 files already formatted.
  • Generated user payload was validated with ag_ui.core.UserMessage; the captured HTTP JSON contains the ordered URL and data parts.

Related issue

Fixes #8114

This is the outbound Agent Framework-to-AG-UI conversion path. It is distinct from #8083, which addresses multimodal content during orchestrator handoff.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@CoralGarden52

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree

@CoralGarden52
CoralGarden52 deployed to github-app-auth September 7, 2026 11:25 — with GitHub Actions Active
@moonbox3 Evan Mattson (moonbox3) added the ag-ui Usage: [Issues, PRs], Target: AG-UI protocol integration label Sep 8, 2026
Comment thread python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py Outdated
@CoralGarden52
CoralGarden52 deployed to github-app-auth September 8, 2026 07:11 — with GitHub Actions Active
Comment thread python/packages/ag-ui/agent_framework_ag_ui/_client.py Outdated
@moonbox3

Copy link
Copy Markdown
Contributor

One file conflict, too.

@CoralGarden52
CoralGarden52 force-pushed the fix/ag-ui-multimodal-input-8114 branch from 884cfd2 to e562c95 Compare September 8, 2026 08:14
@CoralGarden52
CoralGarden52 deployed to github-app-auth September 8, 2026 08:14 — with GitHub Actions Active
@agent-framework-automation agent-framework-automation Bot added the documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs label Sep 8, 2026
@CoralGarden52

Copy link
Copy Markdown
Contributor Author

Thanks for the heads-up, Evan Mattson (@moonbox3). I rebased the branch onto the current upstream main and resolved the conflict. The PR should now be conflict-free.

Comment thread python/packages/ag-ui/agent_framework_ag_ui/_client.py Outdated
Comment thread python/packages/ag-ui/agent_framework_ag_ui/_client.py Outdated
@CoralGarden52
CoralGarden52 deployed to github-app-auth September 8, 2026 09:50 — with GitHub Actions Active
Comment thread python/packages/ag-ui/agent_framework_ag_ui/_client.py
@CoralGarden52
CoralGarden52 deployed to github-app-auth September 8, 2026 11:11 — with GitHub Actions Active
@moonbox3

Copy link
Copy Markdown
Contributor

CoralGarden52 please fix the failing code quality checks. thanks.

@CoralGarden52
CoralGarden52 deployed to github-app-auth September 9, 2026 01:17 — with GitHub Actions Active
@CoralGarden52

Copy link
Copy Markdown
Contributor Author

Evan Mattson (@moonbox3) Thanks for pointing this out. I fixed the failing code quality check and pushed a follow-up commit. The AG-UI test suite and all package-scoped typing checks pass locally. Please let me know if CI reports any further issues.

@moonbox3
Evan Mattson (moonbox3) added this pull request to the merge queue Sep 9, 2026
Merged via the queue into microsoft:main with commit 3312026 Sep 9, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ag-ui Usage: [Issues, PRs], Target: AG-UI protocol integration documentation Usage: [Issues, PRs], Target: documentation in the code base and learn docs python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: AG-UI chat client drops multimodal message content

3 participants