Xml: reject attributes that only exist because of ANTLR error recovery

[timtebeek]

What's wrong

XMLParser.g4 applies attribute : Name '=' STRING to whatever the lexer produces. On malformed markup ANTLR error recovery reaches that rule with tokens that were never written, and XmlParserVisitor.visitAttribute built an Xml.Attribute from the result without checking. That goes wrong three different ways:

Input	Recovery	Result
<t k="" abc "v"/>	inserts the missing =	XmlPrinter re-emits it — the document reprints as <t k="" abc="v"/>
<t k="" a b "v"/>	drops the rule's children	c.EQUALS() is null → NullPointerException in convert
<t k="" x = 1 "v"/>	resyncs past the 1	silently accepted as k="" plus an invented x="v"

The third is the one that matters. The dropped source survives in the neighbouring prefix, so the document reprints byte-for-byte and requirePrintEqualsInput never fires. The file is parsed, cached, and searched with an attribute value that is not what the source says.

Why it shows up in practice

The usual trigger is an embedded expression containing unescaped double quotes — common in generated or hand-written config that carries a scripting snippet in an attribute:

<set-variable name="allowedIps" value="@{
    string ips = "";
    ips += "10.0.0.1,";
    ips += "10.0.0.2";
    return ips;
}" />

STRING is '"' ~[<"]* '"', so this value is not representable — XML 1.0 §2.3 does not permit a literal " inside a "-delimited value, and xmllint rejects the file too. The lexer therefore re-tokenises the value into alternating STRING runs with stray tokens in the gaps, and the table above decides which symptom you get. Which one is essentially arbitrary: it depends only on the token shape of whatever sits inside the expression's string literals.

On main today, that document parses without error, reprints byte-for-byte, and yields:

[set-variable name=<allowedIps>]
[set-variable value=<@{
    string ips = >]

The value stops at the first quote — the addresses are not in the LST at all. A search recipe reading value finds nothing and reports a clean result. Worse, a recipe that writes the attribute writes the truncated value back: ChangeTagAttribute("set-variable", "value", "REDACTED") currently emits

<set-variable name="allowedIps" value="REDACTED"";
    ips += "10.0.0.1,";
    ips += "10.0.0.2";
    return ips;
}" />

which is no longer valid XML.

The fix

Require that an attribute is built only from tokens that actually appeared in the source, and that nothing but whitespace separates them:

• each of Name, EQUALS, STRING must be present and not an ANTLR ErrorNode (recovery marks inserted tokens as error nodes, so this catches the invented = and the null dereference)
• the attribute's prefix, beforeEquals, and the value's prefix must be whitespace only (this catches tokens that recovery dropped, which is what silently truncates the value)
• the same whitespace requirement applies to the trailing prefix of each rule that accepts attributes — an element's beforeTagDelimiterPrefix, and the equivalents in visitXmldecl and visitJspdirective — for the case where recovery produces no attribute at all and the stray tokens are swallowed before the closing delimiter

Otherwise the parse fails with a message naming the actual problem:

Malformed attribute in policy.xml at line 3, column 9. The markup here is not a well-formed
name="value" attribute; common causes are a literal '"' inside a double-quoted value, which
XML 1.0 section 2.3 does not permit (write it as " or delimit the value with single
quotes), an unquoted value, or an HTML-style attribute written without a value.

Both halves are needed, and neither alone is sufficient. The whitespace checks catch tokens recovery dropped — that is what silently truncates the value. The ErrorNode check catches tokens recovery inserted; there is nothing wrong with any prefix in <t k="" abc "v"/>, so a whitespace check cannot see it. Disabling either half locally lets two of the six regression cases through.

XmlPrinter.visitAttribute still appends '=' unconditionally. With this check in place no Xml.Attribute can reach it without a real = in the source, so it is no longer reachable as a bug; making the = optional in the LST would be a much larger change for no gain today.

Behaviour change

Documents matching the third row above previously parsed into an Xml.Document and now become a ParseError. That is the point of the change — they were being parsed into a tree that did not describe the source — but it does mean files that used to ingest "successfully" will start reporting. The first two rows already failed, just less legibly (is not print idempotent, or an NPE); they now carry an actionable message.

Well-formed spellings are unaffected and keep their full value, so there is a clear path for anyone hitting this:

<set-variable value='@{ string ips = ""; ips += "10.0.0.1,"; return ips; }'/>
<set-variable value="@{ string ips = &quot;10.0.0.1&quot;; return ips; }"/>

Notes on provenance

The invented = and the NullPointerException date to the original XML module and only became visible as parse failures once requirePrintEqualsInput landed in 8.1.14 (#3459).

The silent truncation is newer. Bisecting released versions (8.1.14 → 8.91.0-SNAPSHOT) puts it at 8.84.4: before that the same input failed loudly, after it the file is quietly accepted with a truncated value. The only rewrite-xml change in that range is #7906, which left-factored the element rule to support HTML void elements. The feature itself is correctly gated behind htmlMode, but the grammar restructuring it needed is not, so it changed ANTLR's error recovery for the attribute* loop in strict XML too. Worth keeping in mind for future grammar refactors — a change that looks purely structural can move which malformed inputs get rejected.

Testing

Added to XmlParserTest:

• malformedAttributeIsNotSilentlyAccepted — six recovery shapes (including an orphan string in a tag and in an xmldecl) become a ParseError, still print verbatim, and carry the new message
• attributeValueWithUnescapedQuotes — the embedded-expression document above, asserting the message
• wellFormedEmbeddedExpressionsKeepTheirFullValue — " and single-quoted delimiters keep the whole value

:rewrite-xml:test and :rewrite-maven:test pass. The pre-existing malformedBareAmpersandDoesNotThrow / malformedUnterminatedEndTagDoesNotThrow / malformedMissingRootCloseDoesNotThrow cases still pass, so lenient handling of other malformed input is unchanged.

Two checks on the blast radius of the added strictness:

• Parsing 18,181 XML and POM files from a local ~/.m2/repository with the patched parser produces zero parse errors, so the guards do not fire on ordinary well-formed XML.
• Differentially probing the old and new visitor over shapes the guards newly reject — HTML boolean attributes, unquoted values, JSP dynamic attributes, <root a/> — shows every one of them was already a ParseError before this change. For those the PR replaces an NPE or an idempotency failure with a readable message. The only previously-successful shapes that now fail are genuinely malformed: <t a=="1"/>, and orphan strings inside a tag.