Rust PG server: fix-one batch snapshot of pg-batch (42577120, 3 commits)

.github/workflows/test.yml

@@ -1831,6 +1831,11 @@ jobs:
           # 3-minute test on its own: no shard of this lane gets below that.
           if [ "${{ matrix.shard }}" = "server" ]; then
           files="tests/test_rust_pgserver_slice.py"
+          # The catalog's storage key log (the statement after DDL patches
+          # the catalogs from it) and relation-name resolution across
+          # schemas. Binary only.
+          files="$files tests/test_rust_pg_catalog_key_log.py"
+          files="$files tests/test_rust_pgserver_relation_names.py"
           else
           files=""
           # The DIFFERENTIAL suite, whole: 836 cases against the PostgreSQL 14

changelog.d/pg-left-join-anti-join.md

@@ -0,0 +1,21 @@
+### EXPLAIN shows the anti join a LEFT join's IS NULL makes
+
+A LEFT join whose WHERE keeps only the rows with no match -- `left join e2
+on e.a = e2.a where e2.a is null` -- is an anti join to PostgreSQL's
+planner: it stops looking at the first match and drops the `IS NULL`
+check. The Rust PostgreSQL server now explains it that way, as a `Hash
+Anti Join` over an equality or a `Nested Loop Anti Join` over a `<>`, with
+PostgreSQL's own costs and row estimate, where it printed a `Left Join`
+with a `Filter` above it and zero costs. The rows returned were already
+right.
+
+#### Fixed
+
+- EXPLAIN of a LEFT join whose WHERE tests the nullable side's join key
+  `IS NULL` prints PostgreSQL 15's `Hash Anti Join` / `Nested Loop Anti
+  Join`, priced by `final_cost_hashjoin` / `final_cost_nestloop`'s
+  JOIN_ANTI arms over `compute_semi_anti_join_factors`
+  (`eqjoinsel_semi`, and `neqjoinsel` for JOIN_ANTI).
+  Fixes RP-908 (tests/backlog/test_rp_explain_costs.py::test_plan[a left join where the nullable side is null is an anti join], tests/backlog/test_rp_explain_costs.py::test_plan[a left join on an equality where the nullable side is null is an anti join])
+- Filed RP-980 (NOT EXISTS as a Hash Anti Join) and RP-981 (an anti join on
+  an expression key).

changelog.d/pg-prepare-error-position.md

@@ -0,0 +1,13 @@
+### A PREPARE's error points into the PREPARE
+
+When the query inside a `PREPARE` fails analysis, the Rust PostgreSQL server
+now reports the error's position in the statement the client sent, as
+PostgreSQL does. It planned a deparsed copy of the query, so the position was
+an offset into that copy: `prepare p1(int) as select 12 = true` pointed at the
+`(` of `p1(int)` rather than at the `=`.
+
+#### Fixed
+
+- Rust PG server: an analysis error in a `PREPARE`'s query is positioned in
+  the `PREPARE` text.
+  Fixes RP-593 (tests/backlog/test_rp_found_rp530.py::test_prepare_error_position)

changelog.d/rp-390-chain-into-memoize.md

@@ -0,0 +1,15 @@
+### EXPLAIN ANALYZE counts a nested-loop chain that ends in a Memoize
+
+When a chain of nested loops ended in a Memoize over a parameterized index
+scan, EXPLAIN ANALYZE printed the lower join, its scans and the Memoize
+bare. They now carry the counts PostgreSQL 15.19 prints: the lower join's
+rows and its Join Filter's removed pairs, the Memoize's rows per loop and
+its `Hits / Misses / Memory Usage` line, and the index scan's loops, each
+taken from the rows the lower join actually produced.
+
+#### Fixed
+
+- EXPLAIN ANALYZE of `z join z z2 on z.a < z2.a join kz on z2.a < kz.a`
+  annotates every node below the top Nested Loop.
+
+Fixes RP-390, part (tests/backlog/test_rp_explain_costs.py::test_plan[analyze of a nested loop of three relations into a memoized index])

changelog.d/rp-390-chain-over-hash-join.md

@@ -0,0 +1,12 @@
+### EXPLAIN ANALYZE counts a nested-loop chain over a Hash Join
+
+A nested loop whose outer side is a lower Hash Join now prints that join's
+counts as PostgreSQL does: the Hash Join's rows, each scan's, the Hash's
+buckets, batches and memory, and the top join's rows removed by its filter.
+They printed bare. The lower join's hash table is sized from its inner scan
+priced on its own, as the plan above it carries no estimate.
+
+#### Fixed
+
+- EXPLAIN ANALYZE annotates a Nested Loop over a Hash Join (RP-390, part).
+  Fixes RP-390, part (tests/backlog/test_rp_explain_costs.py::test_a_chains_lower_hash_join_is_counted)

changelog.d/rp-390-char-n-left-join-filter.md

@@ -0,0 +1,15 @@
+### EXPLAIN ANALYZE counts a LEFT join filter over a char(n) column
+
+A LEFT Nested Loop whose Join Filter reads a `char(n)` column now prints
+its children's actual rows and loops, as PostgreSQL 15.19 does. The
+per-pair evaluation of the filter used to refuse any `char(n)` column, so
+the scans below the join printed bare; it now reads the column
+blank-padded to its declared width, as PostgreSQL stores it.
+
+#### Fixed
+
+- `explain (analyze) select * from cc left join s on cc.b < s.c` over a
+  `char(4)` column annotates `Seq Scan on cc` and the Materialize below
+  the join.
+
+Fixes RP-390, part (tests/backlog/test_rp_explain_costs.py::test_plan[analyze of a left nested loop over a char(n) join filter]; crates/secantus-pgplan/src/tests.rs::a_pair_expression_pads_a_char_n_column)

changelog.d/rp-390-nested-loop-chain.md

@@ -0,0 +1,16 @@
+### EXPLAIN ANALYZE counts every join of a nested-loop chain
+
+A chain of three or more tables joined by plain Nested Loops now prints
+actual rows, loops and `Rows Removed by Join Filter` at every level, as
+PostgreSQL 15.19 does; before, only the top join was counted and the joins
+below it printed bare. Each lower join's rows are counted by running its
+Join Filter over its pairs, and the join above reads them as its outer
+side.
+
+#### Fixed
+
+- `explain (analyze) select * from z join z z2 on z.a < z2.a join z z3 on
+  z2.a < z3.a` prints the lower Nested Loop's `(actual rows=190 loops=1)`
+  and its Materialize's `loops=20`.
+
+Fixes RP-390, part (tests/backlog/test_rp_explain_costs.py::test_plan[analyze of a nested loop of three relations]; crates/secantus-pgplan/src/tests.rs::a_relations_expression_reads_each_aliass_row)

changelog.d/rp-393-397-quals-through-nested-joins.md

@@ -0,0 +1,22 @@
+### A filter inside a join nested in an outer join runs in its table's scan
+
+On the Rust PostgreSQL server, a condition that reads one table of an inner
+join nested inside a LEFT or RIGHT join now runs in that table's scan, as
+PostgreSQL's `distribute_qual_to_rels` places it: both the inner join's own
+one-table ON conjuncts and the outer join's ON conjuncts over one table of
+the inner join it null-extends. EXPLAIN prints them as that scan's
+`Filter`, not as a `Join Filter`. The rows returned are unchanged.
+
+#### Fixed
+
+- An inner join under an outer join runs its one-table ON conjuncts in
+  their tables' scans (RP-393).
+- A LEFT or RIGHT join's ON conjunct over one table of an inner join on its
+  nullable side runs in that table's scan (RP-397). Not through a lower
+  outer join's nullable side, where PostgreSQL keeps it at the join.
+
+The plan around them, its join order and method and its estimates, is filed
+as RP-978.
+
+Fixes RP-393 (tests/backlog/test_rp_explain_costs.py::test_plan[an inner join's one-table ON conjunct under an outer join]).
+Fixes RP-397 (tests/backlog/test_rp_explain_costs.py::test_plan[a left join's ON conjunct over the hashed table of an inner join]).

changelog.d/rp-398-joined-scan-filter-order.md

@@ -0,0 +1,13 @@
+### A joined scan's Filter is in PostgreSQL's qual order
+
+A scan under a join now orders its own Filter the way a lone scan already
+did: an equality to a constant after the other conjuncts, then by cost.
+That holds too for an ON conjunct over a LEFT join's nullable side once the
+WHERE reduces the join to an inner one, which now sorts among that scan's
+WHERE conjuncts.
+
+#### Fixed
+
+- `select * from s join e on s.a < e.a where e.b = 'v3' and e.a > 5`
+  prints `Filter: ((a > 5) AND (b = 'v3'::text))` on `e`'s scan.
+  Fixes RP-398 (tests/backlog/test_rp_explain_costs.py::test_plan[a joined scan's filter puts its equality to a constant last], tests/backlog/test_rp_explain_costs.py::test_plan[a reduced left join's scan filter puts its equality to a constant last])

changelog.d/rp-400-join-contradictory-class.md

@@ -0,0 +1,14 @@
+### A join whose WHERE pins one column to two constants plans no scan
+
+On the Rust PostgreSQL server, EXPLAIN of an inner join whose WHERE equates
+one column to two different constants now prints what PostgreSQL 15.19 does:
+a bare `Result` with `One-Time Filter: false`. The column's equivalence
+class is contradictory, so its relation is empty, and so is the join.
+
+#### Fixed
+
+- EXPLAIN of an inner (or comma) join whose WHERE equates a column to two
+  distinct constants is a `Result` with `One-Time Filter: false`, not the
+  join's scans. The same in a LEFT join's ON is still open (RP-400).
+
+Fixes RP-400 in part (tests/backlog/test_rp_explain_costs.py::test_plan[a join whose where equates a column to two constants], tests/backlog/test_rp_explain_costs.py::test_plan[a comma join of aliases whose where equates a column to two constants]).

changelog.d/rp-400-left-join-on-const-gate.md

@@ -0,0 +1,8 @@
+### A LEFT join's contradictory ON constants gate its nullable side
+
+When a LEFT join's ON equates a column of the nullable side to two different constants, PostgreSQL's equivalence class keeps the first as that side's filter and turns the rest into a `false` pseudoconstant on a `Result` over its scan. EXPLAIN on the Rust PostgreSQL server now prints the same gated plan instead of a scan filtering on both constants.
+
+#### Fixed
+
+- EXPLAIN of `select * from s left join e on s.a < e.a and e.b = 'v1' and e.b = 'v2'` prints a `Result` with `One-Time Filter: false` over `e`'s scan, filtered by `b = 'v1'` only.
+  Fixes RP-400 (tests/backlog/test_rp_explain_costs.py::test_plan[a left join's on equating its nullable column to two constants])

changelog.d/rp-402-repeated-equality.md

@@ -0,0 +1,14 @@
+### A repeated WHERE equality is planned once
+
+PostgreSQL's `process_equivalence` finds an equality that repeats an
+earlier one redundant, so `where e.b = k.b and e.b = k.b` is one hash
+clause and a repeated `e.a = 5` one scan filter, each priced once. The Rust
+PostgreSQL server now drops such a repeat before it plans a join, so its
+clauses, estimates and costs match.
+
+#### Fixed
+
+- A repeated join equality is one `Hash Cond`, estimated once
+  (`rows=100000`, not `10000`), and a repeated `column = constant` one
+  `Filter`.
+  Fixes RP-402 (tests/backlog/test_rp_explain_costs.py::test_plan[comma join with a repeated join equality], tests/backlog/test_rp_explain_costs.py::test_plan[comma join with a repeated constant equality])

changelog.d/rp-406-scan-filter-cost-order.md

@@ -0,0 +1,8 @@
+### A scan's Filter lists an expression's conjunct after a cheaper one
+
+PostgreSQL orders a scan's quals cheapest first, pricing each by the operators it evaluates: an expression such as `a + 1 > 996`, a NOT BETWEEN's OR, or a column coerced by a call (`(a)::numeric`, an integer's cast to text) comes after a plain comparison. The Rust PostgreSQL server now prices those clauses the same way, alone or as a join's scan, where before it printed them in the order written.
+
+#### Fixed
+
+- EXPLAIN orders a scan Filter's expression, OR and coerced-column conjuncts by `cost_qual_eval`'s operator cost.
+  Fixes RP-406 (tests/backlog/test_rp_explain_costs.py::test_plan[a scan filter's conjuncts in cost order], tests/backlog/test_rp_explain_costs.py::test_plan[a scan filter's or after a cheaper conjunct])

changelog.d/rp-407-coerced-constant-folded.md

@@ -0,0 +1,8 @@
+### EXPLAIN folds a constant coerced to its operator's type
+
+PostgreSQL's planner folds the implicit coercion of a constant before EXPLAIN prints it: `a::numeric > 4` filters on `'4'::numeric`, and a constant made double precision prints in `float8out`'s form (`'1e-05'::double precision`). The Rust PostgreSQL server now prints the folded constant, alone or as a join's scan, and places an expression equal to such a constant after the scan's other quals, as its equivalence class is.
+
+#### Fixed
+
+- EXPLAIN prints a constant coerced to an integer, numeric or double precision operator folded, and orders an expression's equality with a constant last.
+  Fixes RP-407 (tests/backlog/test_rp_explain_costs.py::test_plan[a scan filter's coerced constant is folded])

changelog.d/rp-412-not-pushed-down.md

@@ -0,0 +1,8 @@
+### A NOT is pushed onto the clause it negates
+
+PostgreSQL's planner never keeps a NOT over something it can negate: `negate_clause` applies De Morgan to an AND or an OR, turns a BETWEEN into the OR of its negated comparisons, and replaces a comparison, LIKE, regex match or IN list by its negator and a null test by its opposite. The AND that comes out joins the scan's other quals, where it can become index conditions. The Rust PostgreSQL server's EXPLAIN now plans a NOT the same way, including one whose pushed-down AND meets a NULL arm and becomes a constant-false `One-Time Filter`.
+
+#### Fixed
+
+- EXPLAIN pushes a WHERE's NOT onto its arms and operators, as `negate_clause` does.
+  Fixes RP-412 (tests/backlog/test_rp_explain_costs.py::test_plan[a not over an or is the and of its negated arms], tests/backlog/test_rp_explain_costs.py::test_plan[a not over a between is the or of its negators], tests/backlog/test_rp_explain_costs.py::test_plan[a not over an or with a null arm is a false one-time filter])

changelog.d/rp-413-function-in-from-hint.md

@@ -0,0 +1,12 @@
+### A missing column's hint searches a function in FROM
+
+A misspelt column in a query with a function in FROM now gets PostgreSQL's
+`Perhaps you meant to reference the column ...` hint: the search for the
+closest column reads the function's columns from its result type, as
+PostgreSQL's range table holds them, instead of giving up.
+
+#### Fixed
+
+- Rust PG server: `select * from generate_series(1,2) g where gg = 1` hints
+  `"g.g"`, and a function beside a table no longer suppresses the table's
+  hint. Fixes RP-413 (tests/backlog/test_rp_column_errors.py::test_function_in_from_hint)

changelog.d/rp-421-coalesce-folded.md

@@ -0,0 +1,8 @@
+### A COALESCE's constant arguments are folded
+
+PostgreSQL's planner simplifies a COALESCE before it plans a scan: NULL constant arguments are dropped, the list ends at the first non-NULL constant, and a COALESCE whose first argument is a constant is that constant. `coalesce(null, a) = 1` filters on `COALESCE(a) = 1`, and `coalesce(1, a) = 1` folds to true, a scan with no filter at all. The Rust PostgreSQL server's EXPLAIN now plans it the same way.
+
+#### Fixed
+
+- EXPLAIN folds a COALESCE's constant arguments as `eval_const_expressions` does, and a comparison of two literals that folding leaves.
+  Fixes RP-421 (tests/backlog/test_rp_explain_costs.py::test_plan[a coalesce drops its null arguments], tests/backlog/test_rp_explain_costs.py::test_plan[a coalesce of a constant first folds])

changelog.d/rp-428-explain-names-before-options.md

@@ -0,0 +1,14 @@
+### EXPLAIN resolves names before it reads its options
+
+PostgreSQL analyses the statement under EXPLAIN before it reads a single
+option, so `explain (bogus) select * from nosuch` reports the missing
+relation, not the unknown option. The Rust PostgreSQL server now does the
+same for a missing relation, a missing function and a bad literal, while an
+error that only planning raises, such as a constant division by zero, still
+stays behind the option's.
+
+#### Fixed
+
+- EXPLAIN with a bad option reported 42601 ahead of a 42P01, 42883 or 22P02
+  its statement raises in parse analysis.
+  Fixes RP-428 (tests/backlog/test_rp_analysis_order.py::test_explain_resolves_names_before_options)

changelog.d/rp-429-target-list-before-where.md

@@ -0,0 +1,11 @@
+### A missing output column is reported before the WHERE's errors
+
+PostgreSQL analyses a SELECT's target list before its WHERE clause, so
+`select nosuch from e where 'x'` reports the missing column, not the
+WHERE's bad boolean. The Rust PostgreSQL server now does the same.
+
+#### Fixed
+
+- Rust PG server: a missing column in the target list is 42703 ahead of a
+  22P02, 42804 or 42883 raised by the WHERE clause.
+  Fixes RP-429 (tests/backlog/test_rp_where_types.py::test_target_list_is_analysed_before_where)

changelog.d/rp-442-cast-literal-before-operator.md

@@ -0,0 +1,8 @@
+### A cast literal's bad input is reported before the operator around it
+
+PostgreSQL reads a constant cast such as `'x'::int` as the cast is analysed, before it looks up the operator the cast sits under. So `select 'x'::int = true` is the integer input error, not "operator does not exist: integer = boolean". The Rust PostgreSQL server now reports the same error at the same position, in an operator and in a simple CASE's argument.
+
+#### Fixed
+
+- Rust PG server: `select 'x'::int = true`, `select 1 = 'x'::uuid`, `select '2020-13-01'::date = 1` and `case 'x'::uuid when 1` are 22P02 / 22008 at the literal, not 42883.
+  Fixes RP-442 (tests/backlog/test_rp_fromless_analysis.py::test_cast_literal_input_error_before_operator)

changelog.d/rp-447-operator-after-reference.md

@@ -0,0 +1,17 @@
+### An operator's type error no longer jumps ahead of a missing column to its left
+
+In a SELECT with no FROM, `select nosuch + (1 + true)` answered 42883
+`operator does not exist: integer + boolean`, because the operator-type
+check ran over the whole statement before any column reference was looked
+at. PostgreSQL analyses an operator's operands before it looks the operator
+up, so the missing column to its left fails first: 42703 `column "nosuch"
+does not exist` at 8. The FROM-less reference walk now resolves each
+operator with no reference under it as it meets it, and runs ahead of the
+operator-type check, so the first error in parse-analysis order is the one
+reported, whichever side of the operator it is on.
+
+#### Fixed
+
+- Rust PG server: in a FROM-less SELECT, a missing column to an operator's
+  left is reported before the operator's 42883.
+  Fixes RP-447 (tests/backlog/test_rp_fromless_analysis.py::test_missing_column_before_operator_to_its_right)

changelog.d/rp-459-exists-nested-subquery.md

@@ -0,0 +1,15 @@
+### A subquery in an EXISTS subquery's select list is analysed
+
+PostgreSQL parse-analyses the whole select list of an EXISTS subquery before
+its planner throws the list away, so a subquery inside it that names a
+column or table that does not exist is an error, while one that would only
+fail when run is never run. The select list's subqueries are now planned
+without running before the list is dropped, and the errors parse analysis
+raises are the statement's.
+
+#### Fixed
+
+- Rust PG server: `select exists (select (select nosuch))` is 42703 at 31
+  and `select exists (select 1 + (select nosuch))` 42703 at 35 (each
+  answered `true`); `select exists (select (select 1/0))` stays `true`.
+- Fixes RP-459 (tests/backlog/test_rp_fromless_analysis.py::test_exists_target_subquery_is_analysed)

changelog.d/rp-460-from-subquery-sibling-hint.md

@@ -0,0 +1,19 @@
+### A FROM-subquery sees its siblings as PostgreSQL's range table has them
+
+When PostgreSQL analyses a FROM-subquery, only the FROM items to its left are
+in the range table, and unless the subquery is LATERAL none of them is
+visible. A subquery inside one that names a sibling is therefore told the
+entry cannot be referenced from there, never offered an alias it could not
+use, and a name of an item to its right is simply a missing FROM-clause
+entry.
+
+#### Fixed
+
+- Rust PG server: `select * from fe x, (select (select fe.a)) s` and the
+  `exists (select fe.a)` form are 42P01 with the hint `There is an entry for
+  table "x", but it cannot be referenced from this part of the query.` (the
+  hint offered the alias `x`); `select * from (select (select y.a)) s, fe y`
+  is `missing FROM-clause entry for table "y"` (an "invalid reference" with
+  the cannot-be-referenced hint); a FROM-subquery on a join's right side
+  sees its left items the same way.
+- Fixes RP-460 (tests/backlog/test_rp_subquery_scope.py::test_from_subquery_sibling_is_not_visible)

changelog.d/rp-469-hidden-join-name-reuse.md

@@ -0,0 +1,15 @@
+### A table an aliased JOIN hides can be named again beside it
+
+An aliased JOIN hides the names of the tables inside it, so PostgreSQL lets
+another FROM item reuse one: `select * from (fs join fe using (a)) as j, fs`
+reads `fs` twice, once inside `j` and once on its own. The Rust PostgreSQL
+server refused this with "not supported", because a joined row's columns
+were keyed by the table's name and the two `fs` would have shared keys. Each
+table leaf of such a name is now keyed apart, so the query answers as
+PostgreSQL does, whole-row references (`select fs from ...`) included.
+
+#### Fixed
+
+- A FROM item named as a table an aliased JOIN hides is no longer 0A000.
+  Fixes RP-469 (tests/backlog/test_rp_subquery_scope.py::test_hidden_join_name_reused)
+- Filed RP-972: a subquery or function reusing such a name is still refused.