Skip to content

fix variables - #26762

Open
daviszhen wants to merge 16 commits into
matrixorigin:mainfrom
daviszhen:0806-fix-var
Open

fix variables#26762
daviszhen wants to merge 16 commits into
matrixorigin:mainfrom
daviszhen:0806-fix-var

Conversation

@daviszhen

@daviszhen daviszhen commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #25123

#24492

What this PR does / why we need it:

  • 支持未赋值用户变量读取返回 NULL,不再报 “user variable does not exist”。

  • 支持 SELECT ... INTO @var,包括多变量赋值、空结果不覆盖旧值、多行结果报错。

  • 修复用户变量数值表达式:

    • SET @A = 1, @b = 2;
    • SELECT @A + @b;
    • 现在返回 3.0,不再报 TEXT TEXT 类型错误。
  • 修复 prepared statement 参数数值上下文:

    • prepare ps_count from 'select ? + ? as sum_val';
    • execute ps_count using @c1, @c2;
    • 现在可正常返回 3.0。
  • 补充了 planner 单测和 BVT case:

    • pkg/sql/plan/user_variable_numeric_test.go
    • test/distributed/cases/expression/mysql_compat_user_variables.sql
    • test/distributed/cases/expression/mysql_compat_user_variables.result

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@iamlinjunhong iamlinjunhong left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete diff from merge-base 6798bd63884c3fb363589565f925fd16f94eccbe to head 2b290fc5f5c05f3be1105316f42a4c82df23b04b, including parser generation, frontend compile/execute paths, prepared reuse, planner binding, and tests. Requesting changes for two P1 correctness issues; one P2 performance issue is also recorded inline. No P0 or P3 findings.

Comment thread pkg/frontend/status_stmt.go
Comment thread pkg/frontend/select_into_user_variables.go Outdated
Comment thread pkg/frontend/select_into_user_variables.go

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes on exact head 2b290fc5f5c05f3be1105316f42a4c82df23b04b.

I independently traced the parser → planner → frontend/status execution paths and found three blocking correctness gaps:

  1. SELECT ... INTO @vars validates expression/variable cardinality only after receiving a non-empty batch. select 1 where false into @a, @b therefore succeeds and leaves the variables untouched, while MySQL 8.4 rejects it with error 1222 regardless of row count. I reproduced the silent success through MatrixOne's embedded SQL path; the structural check must be independent of runtime result cardinality.
  2. The background execution path installs and fills selectIntoUserVariables, but executeStatusStmtInBack only calls runner.Run and never calls apply. Stored-procedure SQL uses this path, so SELECT ... INTO @var can report success without assigning the variable.
  3. Capture keeps only []any and assignment calls SetUserDefinedVar, which hard-codes IsBin=false. Binary-string metadata from the result vector is therefore lost. Later prepared execution (EXECUTE ... USING @v) consults ResolveVariableIsBin, so values assigned by this new syntax can change type/lookup semantics compared with the existing SET path.

There is also an avoidable unhappy-path cost: the collector detects a second row but does not stop execution, and reports the error only after the entire query has completed. Large inputs continue scanning and transporting rows after the outcome is already known.

Focused parser, planner, frontend, and collector tests pass, but they do not cover these execution/metadata/zero-row boundaries. The zero-row counterexample fails against this head exactly because MatrixOne returns nil error.

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the exact head. Requesting changes for three P1 correctness gaps. First, pkg/frontend/select_into_user_variables.go:61 returns before validating expression and variable arity for a zero-row result, so SELECT 1 WHERE FALSE INTO @A,@b silently succeeds. Validate structural arity independently of runtime batches. Second, line 91 stores only the extracted value and loses the source vector binary flag; preserve per-column IsBin metadata and use the binary-aware setter because EXECUTE USING depends on it. Third, the background and stored-procedure path installs the collector but executeStatusStmtInBack never calls apply, so SELECT INTO reports success without assigning variables. Also return the too-many-rows error as soon as rowCount exceeds one instead of scanning the remaining result.

@daviszhen

Copy link
Copy Markdown
Contributor Author

Request changes on exact head 2b290fc5f5c05f3be1105316f42a4c82df23b04b.

I independently traced the parser → planner → frontend/status execution paths and found three blocking correctness gaps:

  1. SELECT ... INTO @vars validates expression/variable cardinality only after receiving a non-empty batch. select 1 where false into @a, @b therefore succeeds and leaves the variables untouched, while MySQL 8.4 rejects it with error 1222 regardless of row count. I reproduced the silent success through MatrixOne's embedded SQL path; the structural check must be independent of runtime result cardinality.
  2. The background execution path installs and fills selectIntoUserVariables, but executeStatusStmtInBack only calls runner.Run and never calls apply. Stored-procedure SQL uses this path, so SELECT ... INTO @var can report success without assigning the variable.
  3. Capture keeps only []any and assignment calls SetUserDefinedVar, which hard-codes IsBin=false. Binary-string metadata from the result vector is therefore lost. Later prepared execution (EXECUTE ... USING @v) consults ResolveVariableIsBin, so values assigned by this new syntax can change type/lookup semantics compared with the existing SET path.

There is also an avoidable unhappy-path cost: the collector detects a second row but does not stop execution, and reports the error only after the entire query has completed. Large inputs continue scanning and transporting rows after the outcome is already known.

Focused parser, planner, frontend, and collector tests pass, but they do not cover these execution/metadata/zero-row boundaries. The zero-row counterexample fails against this head exactly because MatrixOne returns nil error.

  • 每条 SELECT ... INTO 语句重新初始化 collector,修复同一批次多条语句之间状态复用的问题。
  • 保存用户变量赋值时的实际数据类型,数值绑定优先使用变量自身类型,避免被旁边的整数常量错误窄化;支持小数、大整数及数值字符串场景。
  • 扩展 SELECT ... INTO @var 语法,支持 SELECT col INTO @var FROM ... 的 pre-FROM 形式。
  • 零行结果保留原变量值,并增加 MySQL 兼容的 1329 No data warning。
  • 增加多语句、数值类型、prepared statement、pre-FROM、无数据 warning 等 UT/BVT 覆盖。
  • 更新 mysql_compat_user_variables.result,@A + @b 结果由 3.0 修正为 3。

@daviszhen

Copy link
Copy Markdown
Contributor Author

Reviewed the exact head. Requesting changes for three P1 correctness gaps. First, pkg/frontend/select_into_user_variables.go:61 returns before validating expression and variable arity for a zero-row result, so SELECT 1 WHERE FALSE INTO @A,@b silently succeeds. Validate structural arity independently of runtime batches. Second, line 91 stores only the extracted value and loses the source vector binary flag; preserve per-column IsBin metadata and use the binary-aware setter because EXECUTE USING depends on it. Third, the background and stored-procedure path installs the collector but executeStatusStmtInBack never calls apply, so SELECT INTO reports success without assigning variables. Also return the too-many-rows error as soon as rowCount exceeds one instead of scanning the remaining result.

  • 每条 SELECT ... INTO 语句重新初始化 collector,修复同一批次多条语句之间状态复用的问题。
  • 保存用户变量赋值时的实际数据类型,数值绑定优先使用变量自身类型,避免被旁边的整数常量错误窄化;支持小数、大整数及数值字符串场景。
  • 扩展 SELECT ... INTO @var 语法,支持 SELECT col INTO @var FROM ... 的 pre-FROM 形式。
  • 零行结果保留原变量值,并增加 MySQL 兼容的 1329 No data warning。
  • 增加多语句、数值类型、prepared statement、pre-FROM、无数据 warning 等 UT/BVT 覆盖。
  • 更新 mysql_compat_user_variables.result,@A + @b 结果由 3.0 修正为 3。

@iamlinjunhong iamlinjunhong left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the complete diff from merge-base a4b0ce286d182c24efd5a349620ae37016262301 to exact head ce302fc95481492fd2714acb65cdc34251ffc42c, including parser generation, SELECT-INTO normal/background/prepared execution, user-variable type binding and value reconstruction, diagnostics, lifecycle/Q1-Q3 paths, and tests. The author explicitly replied to the previous P2 comments in the PR conversation, and those prior findings are addressed on this head.

This pass confirms three new P1 correctness defects: array-valued user variables can be reconstructed with invalid raw bytes and panic, TIMESTAMP user variables can shift across session/process time zones, and INTO clauses nested in UNION/parenthesized query trees can be silently dropped. No P0, P2, or P3 findings. Requesting changes because P1 blockers remain.

All 26 GitHub checks are terminal with no failures, and git diff --check is clean. A PR-specific targeted-test worktree could not be created because this isolated repository exposes .git/worktrees read-only; no code or worktree files were modified.

Comment thread pkg/sql/util/eval_expr_util.go
Comment thread pkg/sql/plan/base_binder.go
Comment thread pkg/sql/parsers/tree/select.go

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed exact head ce302fc after the successful CI rollup. The follow-up closes the earlier blockers: zero-row arity is validated before execution, second rows fail during capture, binary/type metadata is retained, and frontend/background execution both apply the collected variables.

@daviszhen

Copy link
Copy Markdown
Contributor Author

Reviewed the complete diff from merge-base a4b0ce286d182c24efd5a349620ae37016262301 to exact head ce302fc95481492fd2714acb65cdc34251ffc42c, including parser generation, SELECT-INTO normal/background/prepared execution, user-variable type binding and value reconstruction, diagnostics, lifecycle/Q1-Q3 paths, and tests. The author explicitly replied to the previous P2 comments in the PR conversation, and those prior findings are addressed on this head.

This pass confirms three new P1 correctness defects: array-valued user variables can be reconstructed with invalid raw bytes and panic, TIMESTAMP user variables can shift across session/process time zones, and INTO clauses nested in UNION/parenthesized query trees can be silently dropped. No P0, P2, or P3 findings. Requesting changes because P1 blockers remain.

All 26 GitHub checks are terminal with no failures, and git diff --check is clean. A PR-specific targeted-test worktree could not be created because this isolated repository exposes .git/worktrees read-only; no code or worktree files were modified.

  • array/vector 用户变量

    • 用户变量读回时识别 T_array_* / vector 类型。
    • 对真实 Go slice 直接编码成 MatrixOne 内部 array bytes。
    • 避免走 fmt.Sprint(value) 生成 [1 2 3] 这类非法 payload,防止 panic。
    • 覆盖 vecf32/vecf64/vecbf16/vecf16/vecint8/vecuint8。
  • TIMESTAMP 用户变量

    • TIMESTAMP 重建时使用 session timezone,不再依赖进程全局 time.Local。
    • 补充非本地时区和切换 time_zone 后读回的回归测试,避免跨时区漂移。
  • nested / UNION SELECT ... INTO @var

    • AST 层递归提取 SELECT INTO,支持 parenthesized SELECT。
    • UNION 中只允许最后 query block 携带 INTO。
    • 非最后位置的 INTO 明确报 Misplaced INTO clause,不再静默忽略。
    • nested final query block 的 INTO 会产生 MySQL 兼容的 deprecated warning。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/XXL Denotes a PR that changes 2000+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants