Skip to content

fix: preserve binary literal semantics across expressions - #26750

Draft
ck89119 wants to merge 26 commits into
matrixorigin:mainfrom
ck89119:issue-25295-main
Draft

fix: preserve binary literal semantics across expressions#26750
ck89119 wants to merge 26 commits into
matrixorigin:mainfrom
ck89119:issue-25295-main

Conversation

@ck89119

@ck89119 ck89119 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #25295

What this PR does / why we need it:

This PR completes the binary-literal contract across expression, materialization,
session-variable, and distributed execution boundaries.

The implementation separates two independent concepts:

  • Literal.IsBin remains local to raw hexadecimal/bit literals and preserves
    their big-endian numeric-context behavior. It is not propagated through
    batches, user variables, prepared statements, or RPC.
  • a separate runtime binary-string marker preserves byte-string semantics for
    dynamically typed values. It is carried through local batch copies, user
    variables, text PREPARE parameters, and remote process parameters, then
    materialized as a static binary string type at projection boundaries.

This preserves the existing numeric and bitwise rules for raw literals while
fixing byte-oriented behavior for CHAR_LENGTH, CONCAT, SUBSTR,
LOWER/UPPER, REPEAT, IF, CASE, and COALESCE. CTAS and set-operation
outputs now receive the correct binary type and byte width, and BINARY(N)
comparisons retain their padding semantics.

The regression matrix includes:

  • raw hex/bit numeric, unary, and bitwise expressions;
  • user-variable and text-PREPARE binary/text/binary reuse;
  • local and remote parameter metadata transport, including legacy decoding;
  • direct and nested string functions and control flow;
  • CTAS, UNION, BINARY(N) comparison/filter behavior, empty/NULL/invalid UTF-8;
  • COM_STMT parameter behavior.

Validation on the final head:

  • full unit tests:
    ./pkg/container/batch ./pkg/sql/plan/function ./pkg/sql/plan ./pkg/sql/colexec ./pkg/sql/compile ./pkg/frontend ./pkg/vm/process;
  • focused race tests for batch, function, projection, and process metadata paths;
  • make build;
  • func_string_char_length.test: 141/141;
  • func_cast.test: 243/243 value checks;
  • mysql_compat_bitwise_unsigned.test: 28/28 value checks;
  • real MatrixOne/MySQL 8.4.8 differential checks for user variables, text
    PREPARE, numeric insertion, bitwise/unary operations, binary padding,
    CTAS/UNION, and COM_STMT;
  • git diff --check and complete-diff self-review.

@ck89119 ck89119 changed the title fix: count binary literal char length in bytes fix: preserve binary literal semantics across expressions Aug 6, 2026
@matrix-meow matrix-meow added size/L Denotes a PR that changes [500,999] lines and removed size/S Denotes a PR that changes [10,99] lines labels Aug 6, 2026
ck89119 added 2 commits August 9, 2026 22:54
# Conflicts:
#	pkg/container/batch/batch.go
#	pkg/container/batch/batch_test.go
#	pkg/container/pSpool/copy.go
#	pkg/container/vector/vector.go
#	pkg/defines/const.go
#	pkg/frontend/back_exec.go
#	pkg/frontend/compiler_context.go
#	pkg/frontend/computation_wrapper.go
#	pkg/frontend/computation_wrapper_test.go
#	pkg/frontend/mysql_cmd_executor.go
#	pkg/frontend/session.go
#	pkg/frontend/util.go
#	pkg/frontend/variables.go
#	pkg/sql/colexec/evalExpression.go
#	pkg/sql/colexec/evalExpression_test.go
#	pkg/sql/compile/remoterunServer.go
#	pkg/sql/compile/remoterunServer_test.go
#	pkg/sql/plan/rule/constant_fold_test.go
#	pkg/vm/process/process.go
#	pkg/vm/process/process_codec.go
#	pkg/vm/process/process_codec_test.go
#	pkg/vm/process/types.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/XL Denotes a PR that changes [1000, 1999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants