Conversation
added 6 commits
August 31, 2026 15:02
FilterByRegex supports two match modes: by default a value must match the regex in full to be kept, and setting the contains parameter relaxes that to keep any value the regex matches anywhere within. The plugin description, the related-plugin text, and the parameter docs cover both modes. Test coverage for FilterByRegex lives in filter/FilterByRegexTest.scala, extending TransformerTest. Seven TransformExamples cover the same input/output facts the generated documentation renders. Three hand-written tests cover properties a single example can't express: negate producing the exact complement of its non-negated result, and full match always landing inside the contains match, both checked with contains on and off.
FilterByRegex requires at least one connected input; without one, apply throws a ValidationException that names the problem. Two TransformExamples cover the edges this guards: a connected input with no values returns an empty result, and no connected input at all throws.
Four new examples: a malformed regex throws PatternSyntaxException; a backslash-escaped metacharacter is matched literally rather than as its regex meaning; a whitespace-only value and a Unicode value both match a literal pattern correctly. PatternSyntaxException is added to the existing java.util.regex import.
FilterByRegex has a Markdown documentation file covering the two match modes and how negate composes with them, escaping regex metacharacters, the zero-width-match behavior under contains, and the two failure modes for an invalid regex and missing input. Wired via documentationFile on the Plugin annotation.
RegexSelectTransformer's cross-reference to FilterByRegex said "based on full-string matching" — stale since FilterByRegex gained contains-mode. Now reads "based on regex matching," matching the correction FilterByRegex's own reciprocal entry already carries.
…formers FilterByRegex now cross-references ValidateRegex, RegexReplaceTransformer, RegexExtractionTransformer, IfMatchesRegexTransformer, FilterByLength, and RemoveValues in its relatedPlugins array, with each of the six pointing back. The descriptions call out the actual behavioral difference in each pair rather than a generic compatibility note. FilterByLength and RemoveValues also pick up some incidental cleanup on apply, matching FilterByRegex's style.
robertisele
reviewed
Sep 8, 2026
|
|
||
| ## Notes on regular expressions | ||
|
|
||
| Attention: regex metacharacters in the pattern have to be escaped to be matched literally, e.g. a literal dot needs `\\.`, not `.`, which otherwise means "any character." |
Member
There was a problem hiding this comment.
While \\. is the correct way to escape \. in Java/Scala strings, this will confuse UI users. In the UI, users need to enter \. (single slash)
Contributor
Author
There was a problem hiding this comment.
Good catch. I wrote that in "programming-mode". The double-escape is indeed wrong there.
robertisele
reviewed
Sep 8, 2026
|
|
||
| A pattern that can match an empty string, such as `a*`, matches every value under `contains`, since an empty match always "occurs somewhere" in any string. | ||
|
|
||
| A regex that fails to compile throws a `PatternSyntaxException` rather than silently doing nothing. |
Member
There was a problem hiding this comment.
I think we should clarify that it only throws that during execution (I can save an operator with an invalid regex just fine)
robertisele
reviewed
Sep 8, 2026
| ### Note for advanced users | ||
|
|
||
| A compilation of the available constructs for building regular expressions is available in the | ||
| [API of the Java `Pattern`](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html#sum). |
Member
There was a problem hiding this comment.
We are using JDK 25, so we might just as well update that link :)
Contributor
Author
There was a problem hiding this comment.
Updated to https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/util/regex/Pattern.html (the #sum part was wrong).
robertisele
requested changes
Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add contains-mode matching to FilterByRegex (CMEM-8042)
Description
FilterByRegex now matches on substring containment as well as full-value equality, through a single
containsflag that composes cleanly with the existingnegateinversion.Behavior
By default, a value is kept only when the regex matches it in full. Setting
containsrelaxes that: a value is kept if the pattern matches anywhere within it.negatestill inverts whichever of the two decisions is in effect — the same value can be excluded on a full match, excluded on a partial match, or either, from the same two flags.Implementation
The match check is a single predicate, chosen by
contains: match the whole value, or find the pattern anywhere within it.negatewraps that predicate rather than branching around it separately, so the same logic handles all four flag combinations without special-casing any of them.Tests
Coverage comes from the plugin's own worked examples rather than a single hand-written check, covering both matching modes together with negation and the sharper edges around substring matching — an anchored pattern, a zero-width pattern that matches everywhere, a regex that fails to compile. Three additional checks establish the algebraic relationships directly: negation produces the exact complement of a match set whether or not contains is also set, and contains-mode always keeps a superset of what full-match keeps for the same input.
Additional changes
Missing-input crash
Connecting no input to FilterByRegex used to crash with an unguarded
NoSuchElementException. It now throws aValidationExceptionwith a message naming the problem. A connected input that carries zero values is unaffected — it already just produced an empty result.Documentation
FilterByRegex only had a one-line description. It now has a real documentation page, walking through the regex-escaping rules and why a missing input and an empty one behave differently.
Related plugins
FilterByRegex cross-referenced one plugin, and that description had gone stale the moment
containslanded — it still described full-string-only matching. It's now cross-referenced with six related transformers, each side naming the behavioral difference instead of a generic note.