Skip to content

fix: unnested array columns don't expand into rows and crash on cross-filter click #1065

Merged
domoritz merged 4 commits into
uwdata:mainfrom
SuMayaBee:fix/unnest-quoting-977
Jul 15, 2026
Merged

fix: unnested array columns don't expand into rows and crash on cross-filter click #1065
domoritz merged 4 commits into
uwdata:mainfrom
SuMayaBee:fix/unnest-quoting-977

Conversation

@SuMayaBee

Copy link
Copy Markdown
Contributor

Description

I've been exploring Mosaic (love that it can push millions of rows straight to DuckDB and still feel snappy) and ran into this bug while building a chart on an array column. Turns out there was already an attempt at a fix in #978, closed by the author because of the time issue, so I picked it up, brought it up to date with current main, fixed a couple of things.

Summary of changes:

  • Mark.js: Wraps unnested fields in UNNEST() in query(), so arrays expand into individual rows instead of one stringified bar. isUnnested() only matches real column references, not just anything with a .column property.

  • Toggle.js & Nearest.js: Unnested fields now use clauseList() instead of clausePoints(), generating list_has_any("tags", ['Roads']) instead of an unquoted IN. Both take a listMatch option ('any' | 'all'), same as Menu.

  • SelectionClause.ts: clauseList() now passes arrays straight to listHasAny/listHasAll instead of through literal(), which was coercing them to unquoted strings. Also fixed empty-array clears filtering out every row.

  • selection-clause.test.ts (new): Tests for all of the above, including the exact values from the issue that used to crash.

Testing

Ran the new tests, they pass. Also checked it by hand against the repro from the issue: tag bars now render individually instead of as one combined array, and clicking a bar cross-filters cleanly with no console errors.

Before.online-video-cutter.com.mp4
  • After:
After.online-video-cutter.com.mp4

Closes #977.

cc @derekperkins for review.

@SuMayaBee
SuMayaBee requested review from domoritz and jheer as code owners July 5, 2026 11:25

@jheer jheer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, this is looking good to me.

@derekperkins if you can take a quick look as well to confirm changes to clauseList behavior, that would be great! 🙏

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes vgplot cross-filtering and query generation for array columns when a source is configured with unnest, ensuring arrays expand into rows (via UNNEST(...)) and that interactive filtering uses properly-quoted DuckDB list predicates (list_has_any / list_has_all) instead of invalid IN (...) SQL.

Changes:

  • Wrap unnested channel fields in UNNEST() during mark query construction and tighten isUnnested() to only match real column references / names.
  • Update Toggle and Nearest interactors to generate list-membership predicates for unnested fields and thread through a listMatch: 'any' | 'all' option.
  • Fix clauseList() to correctly handle array values (quote as list literals) and treat empty arrays as “clear selection”, plus add regression tests.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
packages/vgplot/plot/src/marks/Mark.js Ensures unnest-declared fields are selected as UNNEST(...) to expand arrays into rows; adds isUnnested() helper for interactors.
packages/vgplot/plot/src/interactors/Toggle.js Uses clauseList() for single-field unnested selections and supports listMatch.
packages/vgplot/plot/src/interactors/Nearest.js Uses clauseList() for single-field unnested selections and supports listMatch.
packages/mosaic/core/src/SelectionClause.ts Correctly builds list predicates from array values and clears selection on empty arrays.
packages/mosaic/core/test/selection-clause.test.ts Adds regression coverage for quoting/spacing and empty-array clearing behavior in clauseList().

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@derekperkins

Copy link
Copy Markdown
Collaborator

SelectionClause.ts: clauseList() now passes arrays straight to listHasAny/listHasAll instead of through literal(), which was coercing them to unquoted strings. Also fixed empty-array clears filtering out every row.

this is a good fix

@SuMayaBee

Copy link
Copy Markdown
Contributor Author

Thanks for the reviews @jheer @derekperkins @domoritz! Just pulled in the latest main and sorted out the conflict, so this should be all clear now.

@domoritz
domoritz merged commit 59e1216 into uwdata:main Jul 15, 2026
5 checks passed
@domoritz

Copy link
Copy Markdown
Member

This change introduced a regression: #1080. I sent a fix in #1081. It seems to be limited to toggling in legends so I will not deploy an emergency release until I have a review from Jeff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Filtering on unnested data doesn't get quoted properly

5 participants