Skip to content

Fold single-table filter subqueries into joins on SQLite - #38692

Open
Aykuttonpc wants to merge 1 commit into
dotnet:mainfrom
Aykuttonpc:relational-lift-join-predicate
Open

Fold single-table filter subqueries into joins on SQLite#38692
Aykuttonpc wants to merge 1 commit into
dotnet:mainfrom
Aykuttonpc:relational-lift-join-predicate

Conversation

@Aykuttonpc

Copy link
Copy Markdown
Contributor

When a query filters a joined table (an optional navigation with a Where, a SelectMany with a predicate, etc.), EF currently emits a join over a subquery even when the subquery does nothing but filter a single table:

LEFT JOIN (
    SELECT "l0"."Id", "l0"."Name"
    FROM "LevelThree" AS "l0"
    WHERE "l0"."Id" > 5
) AS "l1" ON "l"."Id" = "l1"."OwnerId"

Since the filter references only the inner table, it can move onto the join condition and the subquery can go away:

LEFT JOIN "LevelThree" AS "l1" ON "l"."Id" = "l1"."OwnerId" AND "l1"."Id" > 5

This adds SqliteSubqueryToJoinRewriter, run from SqliteQueryTranslationPostprocessor, which does exactly that: rewrites a LEFT/INNER JOIN over a single-table filter subquery into a join over the table, folding the filter into the ON condition and inlining the subquery's projection.

Where it deliberately does nothing

Working through the test suite surfaced several cases where folding the filter out would change results, so the rewrite skips them:

  • Another join references the subquery's alias — a later join keying off this one (nested collection includes) relies on it having filtered its rows first.
  • The join key depends on a sibling join — a correlated join chain (e.g. keying off a preceding optional/LEFT JOIN) relies on the current join order.
  • A LEFT JOIN projection isn't purely columns — the projected columns must be made nullable under a LEFT JOIN, which only works for bare column references.
  • The predicate is only IS NOT NULL checks — that's an optional-entity existence test (table splitting), not a value filter; the containing query uses it to decide whether the entity is present.

Scope

This is implemented in the SQLite provider only. The underlying reasoning is provider-agnostic (it's about relational join semantics), so it could move to the relational layer to benefit all providers, but I kept it local to SQLite so it could be validated against the full SQLite functional suite. Happy to move it up if that's the preferred direction.

Tests

Full SQLite functional suite passes: 38367 tests, 0 failures. 97 query baselines lost their subqueries (net −334 lines in the affected test files, e.g. GearsOfWarQuerySqliteTest shrank noticeably). No data or nullability regressions — each skip condition above was added specifically because a test showed the rewrite was otherwise changing results.

Part of #33745

EF translates a query that joins onto a filtered table (e.g. an optional
navigation with a Where, or a SelectMany with a predicate) into a LEFT/INNER
JOIN over a subquery: LEFT JOIN (SELECT ... WHERE ...) AS t ON .... When the
subquery only filters a single table, that filter can move onto the join
condition instead, dropping the subquery.

Add SqliteSubqueryToJoinRewriter, run from SqliteQueryTranslationPostprocessor,
which rewrites such joins into LEFT/INNER JOIN <table> ON <key> AND <filter>,
inlining the subquery's projection into the containing SELECT.

The rewrite is skipped whenever it could change results: when another join
references the subquery's alias, when the join key depends on a sibling join,
when a LEFT JOIN projection isn't made purely of columns (can't be made
nullable), or when the predicate is only IS NOT NULL checks (an optional-entity
existence test rather than a value filter).

Part of dotnet#33745

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes SQL generation in the SQLite provider by removing unnecessary derived-table subqueries used solely for filtering on the joined (inner) table, folding those filters directly into the JOIN predicate. This addresses cases like DefaultIfEmpty().Where(...) patterns where EF previously emitted JOIN (SELECT ... WHERE ...) even though an ... JOIN ... ON ... AND <filter> is semantically equivalent.

Changes:

  • Introduces SqliteSubqueryToJoinRewriter, which rewrites LEFT/INNER joins over single-table, filter-only subqueries into direct joins with the filter folded into the ON predicate.
  • Hooks the rewriter into SqliteQueryTranslationPostprocessor so the transformation runs during SQLite query postprocessing.
  • Updates multiple SQLite functional test SQL baselines to reflect the simplified JOIN SQL (subqueries removed, aliases/orderings adjusted).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/EFCore.Sqlite.FunctionalTests/Query/QueryFilterFuncletizationSqliteTest.cs Updates expected SQL to reflect folded join filters and changed aliases/orderings.
test/EFCore.Sqlite.FunctionalTests/Query/GearsOfWarQuerySqliteTest.cs Updates many expected SQL baselines where join subqueries are replaced by direct joins with ANDed predicates.
test/EFCore.Sqlite.FunctionalTests/Query/ComplexTypeQuerySqliteTest.cs Adjusts expected SQL for a left join where the subquery filter is now in the join condition.
test/EFCore.Sqlite.FunctionalTests/BulkUpdates/NorthwindBulkUpdatesSqliteTest.cs Updates expected SQL for bulk update/delete statements affected by join-subquery folding.
src/EFCore.Sqlite.Core/Query/Internal/SqliteSubqueryToJoinRewriter.cs Adds the new rewriting visitor implementing the subquery-to-join folding logic with multiple safety skip conditions.
src/EFCore.Sqlite.Core/Query/Internal/SqliteQueryTranslationPostprocessor.cs Runs the new rewriter during SQLite query translation postprocessing.

@AndriySvyryd AndriySvyryd added this to the 12.0.0 milestone Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants