fix(sglang): copy caller extra_body before merging structured outputs#1937
Conversation
|
📚 Documentation preview: https://dottxt-ai.github.io/outlines/pr-preview/pr-1937/ Preview updates automatically with each commit. |
RobinPicard
left a comment
There was a problem hiding this comment.
Thanks for opening a PR! The change looks good, but the tests are not working because you're using a fixture that does not exist.
_pop("extra_body", {}) returns the caller's dict when present, so
.update() leaked structured_outputs/regex into subsequent unconstrained
calls. Mirror the VLLM fix (dottxt-ai#1933) with dict(...).
Fixes dottxt-ai#1931
54703c3 to
074c0e8
Compare
|
Thanks for the review — fixed the test to use the module-level mock |
|
Re-requesting review: the fixture issue is fixed on the latest commit (module-level |
Summary
#1931 reported that
VLLM/AsyncVLLM._build_client_argsmutated the caller'sextra_body. That path was fixed in #1933. The samepop+ in-placeupdatepattern remains inSGLang/AsyncSGLang._build_client_args, so a reusedextra_bodydict still leaks structured-output keys (regex/ebnf) into later unconstrained calls.Change
extra_bodywithdict(...)before merging (sync + async)regexTest plan
pop/updatebehaviortest_sglang_build_client_args_does_not_mutate_caller_extra_body<3.14; CI on supported versions should run the new testFixes #1931