Docker: a COPY --from carries an image reference in the grammar - #8604
Open
timtebeek wants to merge 3 commits into
Open
Docker: a COPY --from carries an image reference in the grammar#8604timtebeek wants to merge 3 commits into
timtebeek wants to merge 3 commits into
Conversation
#8590 gave `FROM` an `IMAGE_REF` lexer mode, but `COPY --from=` never reached it: the lexer emits `--name=value` as a single `FLAG` token, so `COPY --from=host:5000/img:1.2` stayed one flat literal and `ImageReferences.split(List, Space)` had to re-implement the registry-port rule the lexer already knows. A `FROM_FLAG` token now pushes a `FLAG_IMAGE_REF` mode, which is `IMAGE_REF` ending at the whitespace before the paths rather than at `AS` or the end of the line, and `fromFlag : FROM_FLAG imageReference? FLAG_END?` runs the value through the same `imageReference` rule a `FROM` uses. `ImageReferences.split(List, Space)` is deleted; `split(String, Space)` stays for the reference a recipe supplies as text.
timtebeek
force-pushed
the
tim/docker-copy-from-image-ref
branch
from
August 22, 2026 00:26
01f91c9 to
ebe9940
Compare
A semantic predicate reachable without consuming anything sits in the ATN start-state closure of its mode, and ANTLR refuses to cache a start state whose closure carried a semantic context. `decisionToDFA[DEFAULT_MODE].s0` therefore stayed null and every token re-ran a closure over all of the default mode's rules: lexing a 12,001 line Dockerfile holding no `--from` at all took 318ms rather than 3ms. Both flag rules already begin with '--', so the predicate can sit after it. Only a token starting with '-' pays an uncached edge, and lexing that same file is back to 5ms.
`ImageReferences` was narrowed to text, which left the trait layer holding a private copy of the same partition and a split-then-rejoin round trip through a representation it did not want. It now owns both directions -- parts to contents and contents back to parts -- so `DockerCopyFrom` is two thin calls, `DockerParserVisitor` shares one `imageReference` walk between `FROM` and a `COPY`'s `--from`, and a second consumer of a flag's reference has something to call. In the lexer, the reset of the flags scoped to a logical line is one method rather than four inline copies that had already drifted, the two image reference modes share the text fragment they had a copy of each of, and that fragment reuses `ESCAPE_SEQUENCE`, which it had been a fourth spelling of.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
FROManIMAGE_REFlexer mode.COPY --from=never reached it, so it is the last of the "what could the grammar have owned" items about image references.What changed
The lexer emits
--name=valueas a singleFLAGtoken, soCOPY --from=host:5000/img:1.2was one flat literal andImageReferences.split(List, Space)had to decide for itself which colon was a tag separator and which belonged to a registry port — the ruleIMAGE_REFalready knows. This is live: the docker-library corpus holds 72--from=<image>:<tag>.A
FROM_FLAGtoken now pushes aFLAG_IMAGE_REFmode, and the value goes through the sameimageReferencerule aFROMdoes:Three things the mode needs:
:always loses toFLAG. The predicate has to sit at the very start of the longer rule, turning it off:atFromFlag()is acopyAddFlagsmember set atCOPY/ADD, cleared atUNQUOTED_TEXTandNEWLINE, plus an_input.LA()peek for--from=.--chown=1:2,RUN echo --from=a:band--mount=type=cache,from=buildertherefore stay singleFLAGtokens.FLAG_IMAGE_REFdiffers fromIMAGE_REFonly in where it ends: at the whitespace before the paths rather than atASor the end of the line.imageName : textElement+happily consumed past the pop, making--linkpart of the image name ofCOPY --from=build --link /target/ /. The pop has to emit a visible token the parser rule names, henceFLAG_END : ( WS_CHAR+ | LINE_CONT ) -> popMode. Retyping the mode's tokens back toUNQUOTED_TEXTwould erase the boundary the mode created.ImageReferences.split(List, Space)is deleted.DockerCopyFromreads the parts off the separators the mode left in the flag value's contents, the same convention--mount=type=bindalready uses for its=; the value prints as before, and a colon in a registry port, a quoted name or a variable reference never becomes a content of its own.split(String, Space)stays for the reference a recipe supplies as text (withImageReference), and shrinks to a single loop now that it has one caller shape.One fix came out of
FLAG_IMAGE_REFbeing modelled onIMAGE_REF:ESCAPED_CHAR('\\' .) swallowed the newline of a line continuation, soFROM nginx:1.25\+ newline parsed the tag as1.25\␊. Both modes now use'\\' ~[\r\n], which also keepsCOPY --from=nginx:1.25\+ newline from regressing against the oldFLAGtoken.Verification
:rewrite-docker:testis green (539 tests). Four LST-shape cases inCopyTest(separators split out, only--fromsplits, a following flag is not part of the value, a line continuation ends the value) and two inDockerCopyFromTest(a variable tag split from a variable image name, a recipe-written reference read back as its parts) — the corpus holds no flag value containing a$VAR, so it can only prove "no regression", never "the fix works".The corpus diff: 713 Dockerfiles from 29 docker-library repos, LST parts + round-trip result + syntax errors (via
ExecutionContext.getOnError()) dumped atorigin/mainand with this change. Every one of the 144 changed lines is a--fromvalue splitting as intended; no other LST changed, no round-trip mismatch appeared or disappeared, and the syntax-error set is identical.