Skip to content

add check constraints - #26785

Merged
mergify[bot] merged 26 commits into
matrixorigin:mainfrom
daviszhen:0807-add-check-constraints
Aug 11, 2026
Merged

add check constraints#26785
mergify[bot] merged 26 commits into
matrixorigin:mainfrom
daviszhen:0807-add-check-constraints

Conversation

@daviszhen

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #24730

What this PR does / why we need it:

主要完成 information_schema.CHECK_CONSTRAINTS 的兼容支持:

  • 新增 CHECK_CONSTRAINTS 视图及 v4.0.6 租户升级注册。
  • 新增 mo_check_constraints() table function,从 mo_tables.extra_info 解码 SchemaExtra.Checks。
  • 增加租户隔离、临时表过滤、异常元数据处理和结果排序。
  • 添加 planner/executor 单元测试、升级测试及 BVT case/result。
  • 未修改 pb.go;复用了已有的 SchemaExtra.Checks 字段。

@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for 2 P1 and 2 P2 findings at head f3c4991.

[P1] Gate the new information_schema views behind a fresh protocol generation after rebasing latest main.

This PR persists CHECK_CONSTRAINTS and TABLE_CONSTRAINTS definitions that reference the new mo_check_constraints table function:

// CHECK_CONSTRAINTS is backed by a table function because CHECK metadata is
// stored in the serialized SchemaExtra of each table. The function decodes
// that metadata at query time and applies the current tenant's visibility.
InformationSchemaCheckConstraintsDDL = "CREATE VIEW information_schema.CHECK_CONSTRAINTS AS " +
"SELECT " +
"cc.constraint_catalog AS CONSTRAINT_CATALOG, " +
"cc.constraint_schema AS CONSTRAINT_SCHEMA, " +
"cc.constraint_name AS CONSTRAINT_NAME, " +
"cc.check_clause AS CHECK_CLAUSE " +
"FROM mo_check_constraints() cc"

func upgradeInformationSchemaCheckConstraints() versions.UpgradeEntry {
return versions.UpgradeEntry{
Schema: sysview.InformationDBConst,
TableName: "CHECK_CONSTRAINTS",
UpgType: versions.CREATE_VIEW,
UpgSql: sysview.InformationSchemaCheckConstraintsDDL,
CheckFunc: checkViewDefinition("CHECK_CONSTRAINTS",
sysview.InformationSchemaCheckConstraintsDDL),
PreSql: fmt.Sprintf("DROP VIEW IF EXISTS %s.%s;",
sysview.InformationDBConst, "CHECK_CONSTRAINTS"),
}
}
func upgradeInformationSchemaTableConstraints() versions.UpgradeEntry {
return versions.UpgradeEntry{
Schema: sysview.InformationDBConst,
TableName: "TABLE_CONSTRAINTS",
UpgType: versions.MODIFY_VIEW,
UpgSql: sysview.InformationSchemaTableConstraintsDDL,
CheckFunc: checkViewDefinition("TABLE_CONSTRAINTS",
sysview.InformationSchemaTableConstraintsDDL),
PreSql: fmt.Sprintf("DROP VIEW IF EXISTS %s.%s;",
sysview.InformationDBConst, "TABLE_CONSTRAINTS"),
}

The upgrade framework explicitly permits a same-version CN with a lower version offset to remain in the cluster:

checker := func() (bool, error) {
if v.Version == final.Version && v.VersionOffset >= final.VersionOffset {
return true, nil
}

That older binary has no mo_check_constraints dispatch and returns table function not supported:
case "mo_cache":
nodeId, err = builder.buildMoCache(tbl, ctx, exprs, nil)
case "fulltext_index_scan":
nodeId, err = builder.buildFullTextIndexScan(tbl, ctx, exprs, nil)
case "fulltext_index_tokenize":
inputNodeID := int32(-1)
if input != nil {
inputNodeID = input.nodeID
}
nodeId, err = builder.buildFullTextIndexTokenize(tbl, ctx, exprs, nil, inputNodeID)
case "stage_list":
nodeId, err = builder.buildStageList(tbl, ctx, exprs, nil)
case "moplugin_table":
nodeId, err = builder.buildPluginExec(tbl, ctx, exprs, nil)
case "parse_jsonl_data":
nodeId, err = builder.buildParseJsonlData(tbl, ctx, exprs, nil)
case "parse_jsonl_file":
nodeId, err = builder.buildParseJsonlFile(tbl, ctx, exprs, nil)
case "table_stats":
nodeId = builder.buildTableStats(tbl, ctx, exprs, nil)
case "load_file_chunks":
nodeId = builder.buildLoadFileChunks(tbl, ctx, exprs, nil)
default:
err = moerr.NewNotSupportedf(builder.GetContext(), "table function '%s' not supported", id)

After this offset upgrade, mixed-cluster queries routed to an older CN will fail. Rebase latest main, preserve its existing protocol assignments, allocate a fresh MORPCVersion13 for this view/function contract, and make the tenant upgrade wait until the common protocol reaches v13. Add a mixed-CN boundary regression.

[P1] Do not parse non-table rel_createsql payloads as legacy CREATE TABLE SQL.

The catalog scan includes all non-temporary objects:

var checkConstraintCatalogQuery = "SELECT tbl.reldatabase, tbl.relname, tbl.rel_createsql, tbl.extra_info " +
"FROM mo_catalog.mo_tables tbl " +
"WHERE tbl.account_id = current_account_id() AND " +
catalog.NonTemporaryTableSQLPredicate("tbl") +
" ORDER BY tbl.reldatabase, tbl.relname"

The fallback then parses any payload containing the substring CHECK:
if strings.TrimSpace(createSQL) == "" ||
!strings.Contains(strings.ToUpper(createSQL), "CHECK") {
return nil, nil
}
stmt, err := parsers.ParseOneWithSQLMode(ctx, dialect.MYSQL, createSQL, 1, "")
if err != nil {
return nil, err
}
defer stmt.Free()
createStmt, ok := stmt.(*tree.CreateTable)
if !ok {
return nil, nil

Generic external tables store JSON in rel_createsql. A valid filepath such as stage://bucket/check.csv therefore enters the SQL parser, returns an error, and aborts the whole metadata stream. Both CHECK_CONSTRAINTS and TABLE_CONSTRAINTS then fail for that tenant because of an unrelated external table. Select/filter relkind and only run legacy parsing for eligible base tables; add external envelope, view, and source regressions.

[P2] Preserve the SQL mode of legacy CHECK definitions.

Legacy rel_createsql preserves the creating session SQL text, but the fallback always parses with the empty/default SQL mode:

if strings.TrimSpace(createSQL) == "" ||
!strings.Contains(strings.ToUpper(createSQL), "CHECK") {
return nil, nil
}
stmt, err := parsers.ParseOneWithSQLMode(ctx, dialect.MYSQL, createSQL, 1, "")
if err != nil {
return nil, err
}
defer stmt.Free()
createStmt, ok := stmt.(*tree.CreateTable)
if !ok {
return nil, nil

For create table source_t(a int, check ("a" > 0)), default mode and ANSI_QUOTES produce different CHECK clauses. The current code silently reports the default interpretation, so legacy valid tables can expose incorrect CHECK_CLAUSE metadata. Use source-preserving extraction or explicit SQL-mode ambiguity handling; do not silently select one mode. Cover ANSI_QUOTES, NO_BACKSLASH_ESCAPES, and PIPES_AS_CONCAT.

[P2] Avoid blocking full scans for LIMIT queries.

The catalog query imposes a global ORDER BY before streaming:

var checkConstraintCatalogQuery = "SELECT tbl.reldatabase, tbl.relname, tbl.rel_createsql, tbl.extra_info " +
"FROM mo_catalog.mo_tables tbl " +
"WHERE tbl.account_id = current_account_id() AND " +
catalog.NonTemporaryTableSQLPredicate("tbl") +
" ORDER BY tbl.reldatabase, tbl.relname"

and fillBatch always builds up to 8192 rows without honoring TableFunction.Limit:
func (s *checkConstraintsState) fillBatch(tf *TableFunction, proc *process.Process) error {
positions := checkConstraintOutputPositions(s.batch.Attrs)
rowCount := 0
for rowCount < checkConstraintBatchSize {
if len(s.pending) == 0 {
if s.streamEnded || !s.streaming {
break
}
if err := s.readStreamResult(proc); err != nil {
return err
}
continue
}
space := checkConstraintBatchSize - rowCount
count := len(s.pending)
if count > space {
count = space
}
for i := 0; i < count; i++ {
if err := appendCheckConstraintRow(s.batch.Vecs, positions, s.pending[i], proc); err != nil {
return err
}
}
rowCount += count
s.pending = s.pending[count:]
}
if rowCount == 0 && s.streamEnded {
return nil
}
s.batch.SetRowCount(rowCount)

Thus SELECT ... FROM information_schema.check_constraints LIMIT 1 may still scan the whole tenant before returning. Remove the unnecessary sort and honor pushed limits so the producer can be cancelled early.

The focused parser counterexamples and git diff --check pass. The full table-function package was blocked by an unchanged Darwin CGo baseline failure; the same command failed at the exact PR base, so this is not attributed to the PR.

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head f3c4991 after the successful CI rollup.

[P1] pkg/sql/colexec/table_function/check_constraints.go:220: stopStreaming cancels and drains only streamCh before waiting for streamDone. The streaming SQL executor can publish an error to errCh and return it, after which this producer publishes the same error again. Because errCh has capacity 1, reset/free after cancellation can leave the first error buffered and block the producer on the second send; streamCh is then never closed and stopStreaming waits forever. Give the error channel explicit producer ownership and drain it while shutting down, or otherwise ensure the producer can never block on error publication. Please add a deterministic executor-error plus cancellation/reset lifecycle regression.

@daviszhen

Copy link
Copy Markdown
Contributor Author

Requesting changes for 2 P1 and 2 P2 findings at head f3c4991.

[P1] Gate the new information_schema views behind a fresh protocol generation after rebasing latest main.

This PR persists CHECK_CONSTRAINTS and TABLE_CONSTRAINTS definitions that reference the new mo_check_constraints table function:

// CHECK_CONSTRAINTS is backed by a table function because CHECK metadata is
// stored in the serialized SchemaExtra of each table. The function decodes
// that metadata at query time and applies the current tenant's visibility.
InformationSchemaCheckConstraintsDDL = "CREATE VIEW information_schema.CHECK_CONSTRAINTS AS " +
"SELECT " +
"cc.constraint_catalog AS CONSTRAINT_CATALOG, " +
"cc.constraint_schema AS CONSTRAINT_SCHEMA, " +
"cc.constraint_name AS CONSTRAINT_NAME, " +
"cc.check_clause AS CHECK_CLAUSE " +
"FROM mo_check_constraints() cc"

func upgradeInformationSchemaCheckConstraints() versions.UpgradeEntry {
return versions.UpgradeEntry{
Schema: sysview.InformationDBConst,
TableName: "CHECK_CONSTRAINTS",
UpgType: versions.CREATE_VIEW,
UpgSql: sysview.InformationSchemaCheckConstraintsDDL,
CheckFunc: checkViewDefinition("CHECK_CONSTRAINTS",
sysview.InformationSchemaCheckConstraintsDDL),
PreSql: fmt.Sprintf("DROP VIEW IF EXISTS %s.%s;",
sysview.InformationDBConst, "CHECK_CONSTRAINTS"),
}
}
func upgradeInformationSchemaTableConstraints() versions.UpgradeEntry {
return versions.UpgradeEntry{
Schema: sysview.InformationDBConst,
TableName: "TABLE_CONSTRAINTS",
UpgType: versions.MODIFY_VIEW,
UpgSql: sysview.InformationSchemaTableConstraintsDDL,
CheckFunc: checkViewDefinition("TABLE_CONSTRAINTS",
sysview.InformationSchemaTableConstraintsDDL),
PreSql: fmt.Sprintf("DROP VIEW IF EXISTS %s.%s;",
sysview.InformationDBConst, "TABLE_CONSTRAINTS"),
}

The upgrade framework explicitly permits a same-version CN with a lower version offset to remain in the cluster:

checker := func() (bool, error) {
if v.Version == final.Version && v.VersionOffset >= final.VersionOffset {
return true, nil
}

That older binary has no mo_check_constraints dispatch and returns table function not supported:

case "mo_cache":
nodeId, err = builder.buildMoCache(tbl, ctx, exprs, nil)
case "fulltext_index_scan":
nodeId, err = builder.buildFullTextIndexScan(tbl, ctx, exprs, nil)
case "fulltext_index_tokenize":
inputNodeID := int32(-1)
if input != nil {
inputNodeID = input.nodeID
}
nodeId, err = builder.buildFullTextIndexTokenize(tbl, ctx, exprs, nil, inputNodeID)
case "stage_list":
nodeId, err = builder.buildStageList(tbl, ctx, exprs, nil)
case "moplugin_table":
nodeId, err = builder.buildPluginExec(tbl, ctx, exprs, nil)
case "parse_jsonl_data":
nodeId, err = builder.buildParseJsonlData(tbl, ctx, exprs, nil)
case "parse_jsonl_file":
nodeId, err = builder.buildParseJsonlFile(tbl, ctx, exprs, nil)
case "table_stats":
nodeId = builder.buildTableStats(tbl, ctx, exprs, nil)
case "load_file_chunks":
nodeId = builder.buildLoadFileChunks(tbl, ctx, exprs, nil)
default:
err = moerr.NewNotSupportedf(builder.GetContext(), "table function '%s' not supported", id)

After this offset upgrade, mixed-cluster queries routed to an older CN will fail. Rebase latest main, preserve its existing protocol assignments, allocate a fresh MORPCVersion13 for this view/function contract, and make the tenant upgrade wait until the common protocol reaches v13. Add a mixed-CN boundary regression.

[P1] Do not parse non-table rel_createsql payloads as legacy CREATE TABLE SQL.

The catalog scan includes all non-temporary objects:

var checkConstraintCatalogQuery = "SELECT tbl.reldatabase, tbl.relname, tbl.rel_createsql, tbl.extra_info " +
"FROM mo_catalog.mo_tables tbl " +
"WHERE tbl.account_id = current_account_id() AND " +
catalog.NonTemporaryTableSQLPredicate("tbl") +
" ORDER BY tbl.reldatabase, tbl.relname"

The fallback then parses any payload containing the substring CHECK:

if strings.TrimSpace(createSQL) == "" ||
!strings.Contains(strings.ToUpper(createSQL), "CHECK") {
return nil, nil
}
stmt, err := parsers.ParseOneWithSQLMode(ctx, dialect.MYSQL, createSQL, 1, "")
if err != nil {
return nil, err
}
defer stmt.Free()
createStmt, ok := stmt.(*tree.CreateTable)
if !ok {
return nil, nil

Generic external tables store JSON in rel_createsql. A valid filepath such as stage://bucket/check.csv therefore enters the SQL parser, returns an error, and aborts the whole metadata stream. Both CHECK_CONSTRAINTS and TABLE_CONSTRAINTS then fail for that tenant because of an unrelated external table. Select/filter relkind and only run legacy parsing for eligible base tables; add external envelope, view, and source regressions.

[P2] Preserve the SQL mode of legacy CHECK definitions.

Legacy rel_createsql preserves the creating session SQL text, but the fallback always parses with the empty/default SQL mode:

if strings.TrimSpace(createSQL) == "" ||
!strings.Contains(strings.ToUpper(createSQL), "CHECK") {
return nil, nil
}
stmt, err := parsers.ParseOneWithSQLMode(ctx, dialect.MYSQL, createSQL, 1, "")
if err != nil {
return nil, err
}
defer stmt.Free()
createStmt, ok := stmt.(*tree.CreateTable)
if !ok {
return nil, nil

For create table source_t(a int, check ("a" > 0)), default mode and ANSI_QUOTES produce different CHECK clauses. The current code silently reports the default interpretation, so legacy valid tables can expose incorrect CHECK_CLAUSE metadata. Use source-preserving extraction or explicit SQL-mode ambiguity handling; do not silently select one mode. Cover ANSI_QUOTES, NO_BACKSLASH_ESCAPES, and PIPES_AS_CONCAT.

[P2] Avoid blocking full scans for LIMIT queries.

The catalog query imposes a global ORDER BY before streaming:

var checkConstraintCatalogQuery = "SELECT tbl.reldatabase, tbl.relname, tbl.rel_createsql, tbl.extra_info " +
"FROM mo_catalog.mo_tables tbl " +
"WHERE tbl.account_id = current_account_id() AND " +
catalog.NonTemporaryTableSQLPredicate("tbl") +
" ORDER BY tbl.reldatabase, tbl.relname"

and fillBatch always builds up to 8192 rows without honoring TableFunction.Limit:

func (s *checkConstraintsState) fillBatch(tf *TableFunction, proc *process.Process) error {
positions := checkConstraintOutputPositions(s.batch.Attrs)
rowCount := 0
for rowCount < checkConstraintBatchSize {
if len(s.pending) == 0 {
if s.streamEnded || !s.streaming {
break
}
if err := s.readStreamResult(proc); err != nil {
return err
}
continue
}
space := checkConstraintBatchSize - rowCount
count := len(s.pending)
if count > space {
count = space
}
for i := 0; i < count; i++ {
if err := appendCheckConstraintRow(s.batch.Vecs, positions, s.pending[i], proc); err != nil {
return err
}
}
rowCount += count
s.pending = s.pending[count:]
}
if rowCount == 0 && s.streamEnded {
return nil
}
s.batch.SetRowCount(rowCount)

Thus SELECT ... FROM information_schema.check_constraints LIMIT 1 may still scan the whole tenant before returning. Remove the unnecessary sort and honor pushed limits so the producer can be cancelled early.

The focused parser counterexamples and git diff --check pass. The full table-function package was blocked by an unchanged Darwin CGo baseline failure; the same command failed at the exact PR base, so this is not attributed to the PR.

  1. information_schema.CHECK_CONSTRAINTS 兼容旧表

读取 mo_tables.rel_createsql,当旧表没有 SchemaExtra.Checks 时,从 legacy CREATE TABLE SQL 解析 CHECK 定义,避免升级后旧 CHECK 约束丢失。

  1. 加协议版本门禁

新增/使用 MORPCVersion13,CHECK_CONSTRAINTS 和 TABLE_CONSTRAINTS 的 upgrade entry 会等所有服务协议版本达到 v13 后再创建视图;planner 构建 mo_check_constraints 时也检查本机协议版本,避免 rolling upgrade 混合集群中老 CN 无法识别 table function。

  1. 避免误解析非表对象

catalog 查询限制 relkind='r',decode 层也跳过 external/view/source 等非 ordinary table,避免这些对象的 rel_createsql payload 含 CHECK 时被当作 CREATE TABLE 解析并中断查询。

  1. 处理 legacy SQL mode 歧义

legacy CHECK 解析会尝试 parser 相关 SQL mode 组合;如果不同 SQL mode 得到不同 CHECK clause,返回明确 ambiguity error,不再静默按 default mode 解释。

  1. 优化 LIMIT 路径

移除 catalog 查询里的 ORDER BY,支持把 plain LIMIT 下推到 mo_check_constraints table function;但不再把 LIMIT 拼到 mo_tables 源查询,而是在输出 CHECK 行层面计数,达到 LIMIT 后 cancel stream,避免前面无 CHECK 表导致漏结果。

  1. 修复 streaming error/cancel/reset 死锁

RunStreamingSql 改用内部 error channel,外层 producer 只向 public errCh 非阻塞发布一次错误;stopStreaming 在 cancel/reset/free 时 drain error channel,避免 executor error 后 reset/free 卡死。

@daviszhen

Copy link
Copy Markdown
Contributor Author

Reviewed exact head f3c4991 after the successful CI rollup.

[P1] pkg/sql/colexec/table_function/check_constraints.go:220: stopStreaming cancels and drains only streamCh before waiting for streamDone. The streaming SQL executor can publish an error to errCh and return it, after which this producer publishes the same error again. Because errCh has capacity 1, reset/free after cancellation can leave the first error buffered and block the producer on the second send; streamCh is then never closed and stopStreaming waits forever. Give the error channel explicit producer ownership and drain it while shutting down, or otherwise ensure the producer can never block on error publication. Please add a deterministic executor-error plus cancellation/reset lifecycle regression.

  1. information_schema.CHECK_CONSTRAINTS 兼容旧表

读取 mo_tables.rel_createsql,当旧表没有 SchemaExtra.Checks 时,从 legacy CREATE TABLE SQL 解析 CHECK 定义,避免升级后旧 CHECK 约束丢失。

  1. 加协议版本门禁

新增/使用 MORPCVersion13,CHECK_CONSTRAINTS 和 TABLE_CONSTRAINTS 的 upgrade entry 会等所有服务协议版本达到 v13 后再创建视图;planner 构建 mo_check_constraints 时也检查本机协议版本,避免 rolling upgrade 混合集群中老 CN 无法识别 table function。

  1. 避免误解析非表对象

catalog 查询限制 relkind='r',decode 层也跳过 external/view/source 等非 ordinary table,避免这些对象的 rel_createsql payload 含 CHECK 时被当作 CREATE TABLE 解析并中断查询。

  1. 处理 legacy SQL mode 歧义

legacy CHECK 解析会尝试 parser 相关 SQL mode 组合;如果不同 SQL mode 得到不同 CHECK clause,返回明确 ambiguity error,不再静默按 default mode 解释。

  1. 优化 LIMIT 路径

移除 catalog 查询里的 ORDER BY,支持把 plain LIMIT 下推到 mo_check_constraints table function;但不再把 LIMIT 拼到 mo_tables 源查询,而是在输出 CHECK 行层面计数,达到 LIMIT 后 cancel stream,避免前面无 CHECK 表导致漏结果。

  1. 修复 streaming error/cancel/reset 死锁

RunStreamingSql 改用内部 error channel,外层 producer 只向 public errCh 非阻塞发布一次错误;stopStreaming 在 cancel/reset/free 时 drain error channel,避免 executor error 后 reset/free 卡死。

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 9a1a9f9.

[P1] Make new-tenant information_schema initialization protocol-aware.

InitInformationSchemaSysTables now unconditionally contains both view definitions that reference mo_check_constraints() (pkg/util/sysview/sysview.go:58,67). Every CREATE ACCOUNT executes this list (pkg/frontend/authenticate.go:10489-10497), but building either view reaches buildCheckConstraints, whose protocol gate rejects a deployment-wide version below v14.

During a v13/v14 rolling upgrade, account creation routed to an upgraded CN therefore aborts when it reaches these views, while account creation routed to an old CN installs the legacy schema. UpgradeEntry.RequiredProtocolVersion protects existing-tenant upgrades but does not cover this initialization path.

Make tenant initialization version-aware: preserve or omit the old definitions while the common protocol is below v14, and guarantee that tenants created during that window receive the v14 views after rollout. Add a mixed-version CREATE ACCOUNT regression.

The previous legacy recovery, relation filtering, SQL-mode ambiguity, LIMIT, and stream-shutdown findings are addressed. I also closed the streaming ownership, wait, and buffer audit with no additional lifecycle finding.

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-reviewed exact head 9a1a9f9.

[P1] Make new-tenant information_schema initialization protocol-aware.

InitInformationSchemaSysTables unconditionally includes CHECK_CONSTRAINTS and the new TABLE_CONSTRAINTS definition (pkg/util/sysview/sysview.go:58,67), and every CREATE ACCOUNT executes the full list (pkg/frontend/authenticate.go:10489-10497). Both views reference mo_check_constraints(), whose planner path rejects a deployment protocol below v14 (pkg/sql/plan/check_constraints.go:76-89 and pkg/sql/plan/build_constraint_util.go:205-223).

The RequiredProtocolVersion on the v4.0.6 upgrade entries protects existing-tenant upgrades only. During a v13/v14 rolling upgrade, CREATE ACCOUNT on a v14 CN therefore fails while planning these views; routing it to an older CN creates the legacy schema instead. The result depends on routing and the newly created tenant is not guaranteed to receive the v14 definitions after rollout.

Gate initial schema construction on the common protocol and ensure tenants created during the mixed-version window are upgraded after v14 becomes common. Please add a mixed-version CREATE ACCOUNT regression. The previous legacy recovery, relation filtering, SQL-mode ambiguity, LIMIT, and streaming shutdown findings are addressed on this head.

@daviszhen

Copy link
Copy Markdown
Contributor Author

Reviewed exact head 9a1a9f9.

[P1] Make new-tenant information_schema initialization protocol-aware.

InitInformationSchemaSysTables now unconditionally contains both view definitions that reference mo_check_constraints() (pkg/util/sysview/sysview.go:58,67). Every CREATE ACCOUNT executes this list (pkg/frontend/authenticate.go:10489-10497), but building either view reaches buildCheckConstraints, whose protocol gate rejects a deployment-wide version below v14.

During a v13/v14 rolling upgrade, account creation routed to an upgraded CN therefore aborts when it reaches these views, while account creation routed to an old CN installs the legacy schema. UpgradeEntry.RequiredProtocolVersion protects existing-tenant upgrades but does not cover this initialization path.

Make tenant initialization version-aware: preserve or omit the old definitions while the common protocol is below v14, and guarantee that tenants created during that window receive the v14 views after rollout. Add a mixed-version CREATE ACCOUNT regression.

The previous legacy recovery, relation filtering, SQL-mode ambiguity, LIMIT, and stream-shutdown findings are addressed. I also closed the streaming ownership, wait, and buffer audit with no additional lifecycle finding.

  • CREATE ACCOUNT 初始化 information_schema 时改为 protocol-aware。

  • 新增 InitInformationSchemaSysTablesForProtocol(protocol):

    • protocol < MORPCVersion14:不创建 CHECK_CONSTRAINTS,TABLE_CONSTRAINTS 使用不依赖 mo_check_constraints() 的 legacy 定义。
    • protocol >= MORPCVersion14:使用完整新版视图。
  • CREATE ACCOUNT 路径传入 ses.GetService(),读取同一 service runtime 的 MOProtocolVersion,与 planner gate 口径一致。

  • protocol 取不到时 fail-safe 到 legacy 初始化,避免 mixed-version rolling upgrade 中旧 CN 解析失败。

  • 保留 v4.0.6 upgrade entry 的 RequiredProtocolVersion = MORPCVersion14,并补测试证明 mixed-window 创建的新租户在 rollout 后会被识别为需要升级到新版视图。

  • 补充 UT 覆盖 v13/v14 两种初始化结果和 legacy view 后续升级判断。

@daviszhen

Copy link
Copy Markdown
Contributor Author

Deep-reviewed exact head 9a1a9f9.

[P1] Make new-tenant information_schema initialization protocol-aware.

InitInformationSchemaSysTables unconditionally includes CHECK_CONSTRAINTS and the new TABLE_CONSTRAINTS definition (pkg/util/sysview/sysview.go:58,67), and every CREATE ACCOUNT executes the full list (pkg/frontend/authenticate.go:10489-10497). Both views reference mo_check_constraints(), whose planner path rejects a deployment protocol below v14 (pkg/sql/plan/check_constraints.go:76-89 and pkg/sql/plan/build_constraint_util.go:205-223).

The RequiredProtocolVersion on the v4.0.6 upgrade entries protects existing-tenant upgrades only. During a v13/v14 rolling upgrade, CREATE ACCOUNT on a v14 CN therefore fails while planning these views; routing it to an older CN creates the legacy schema instead. The result depends on routing and the newly created tenant is not guaranteed to receive the v14 definitions after rollout.

Gate initial schema construction on the common protocol and ensure tenants created during the mixed-version window are upgraded after v14 becomes common. Please add a mixed-version CREATE ACCOUNT regression. The previous legacy recovery, relation filtering, SQL-mode ambiguity, LIMIT, and streaming shutdown findings are addressed on this head.

  • CREATE ACCOUNT 初始化 information_schema 时改为 protocol-aware。

  • 新增 InitInformationSchemaSysTablesForProtocol(protocol):

    • protocol < MORPCVersion14:不创建 CHECK_CONSTRAINTS,TABLE_CONSTRAINTS 使用不依赖 mo_check_constraints() 的 legacy 定义。
    • protocol >= MORPCVersion14:使用完整新版视图。
  • CREATE ACCOUNT 路径传入 ses.GetService(),读取同一 service runtime 的 MOProtocolVersion,与 planner gate 口径一致。

  • protocol 取不到时 fail-safe 到 legacy 初始化,避免 mixed-version rolling upgrade 中旧 CN 解析失败。

  • 保留 v4.0.6 upgrade entry 的 RequiredProtocolVersion = MORPCVersion14,并补测试证明 mixed-window 创建的新租户在 rollout 后会被识别为需要升级到新版视图。

  • 补充 UT 覆盖 v13/v14 两种初始化结果和 legacy view 后续升级判断。

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-reviewed the complete current diff and all review history. The latest head closes the prior blocking gaps: legacy CHECK metadata remains recoverable without parsing non-table payloads, SQL-mode-dependent legacy definitions fail explicitly, LIMIT stops the streaming catalog scan, result/error/cancel ownership is bounded and reset-safe, and protocol v15 gates both planning and mixed-version tenant view installation/upgrade. I also checked the latest main merge (clean), ran the focused reset race case 16x, the full table_function package under race, and the affected upgrade/frontend/planner/sysview packages; all passed. No blocking correctness, lifecycle, compatibility, or general-case performance issue found.

@aunjgr aunjgr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 698efde with all 26 checks terminal and passing. The previous blocker is closed: new-tenant information_schema initialization now selects legacy definitions below protocol v15, both upgraded views are gated on v15, and the upgrade entries detect and replace mixed-window legacy or missing definitions after rollout. The earlier legacy metadata, relation filtering, SQL-mode, LIMIT, and streaming shutdown fixes remain intact. No remaining correctness, compatibility, lifecycle, or boundedness blocker found.

@LeftHandCold LeftHandCold left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head d91ee27. No blocking issue found. Rechecked the protocol-v16 gate after the latest main merge, legacy metadata recovery, account/relation/temp filtering, SQL-mode ambiguity handling, LIMIT-driven stream cancellation, Reset/Free ownership, protocol-aware tenant initialization, and idempotent v4.0.6 view upgrades. Focused upgrade/sysview/planner tests and build/vet passed. Full local table_function/frontend validation hits the same pkg/common/docfilter CGo compile failure on exact base d01c859, so it is not introduced by this PR.

@mergify

mergify Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-08-11 03:26 UTC · Rule: main · triggered by rule Automatic queue on approval for main
  • Checks passed · in-place
  • Merged2026-08-11 04:13 UTC · at a8598e3cd4cde4a96eeadced1303bc8512b4b2a7 · squash

This pull request spent 47 minutes 31 seconds in the queue, including 47 minutes 4 seconds running CI.

Required conditions to merge
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-approved [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / UT Test on Ubuntu/x86
    • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
    • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / SCA Test on Linux/arm64
    • check-neutral = Matrixone CI / SCA Test on Linux/arm64
    • check-skipped = Matrixone CI / SCA Test on Linux/arm64
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Utils CI / Coverage
    • check-neutral = Matrixone Utils CI / Coverage
    • check-skipped = Matrixone Utils CI / Coverage
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-neutral = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-skipped = Matrixone UT Coverage / UT Coverage on Ubuntu/x86

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/XXL Denotes a PR that changes 2000+ lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants