Skip to content

Implement SCYLLA_USE_METADATA_ID protocol extention negotiat… - #590

Merged
dkropachev merged 5 commits into
scylladb:masterfrom
nikagra:support-scylla-use-metadata-id-extention
Aug 11, 2026
Merged

dkropachev merged 5 commits into
scylladb:masterfrom
nikagra:support-scylla-use-metadata-id-extention

Conversation

@nikagra

@nikagra nikagra commented Oct 15, 2025 •

Copy link
Copy Markdown

Issue gocql#527

Implements driver-side negotiation and use of Scylla's SCYLLA_USE_METADATA_ID protocol extension so that prepared-statement result-metadata invalidation works correctly (see scylladb/scylladb#20860, fixed server-side by scylladb/scylladb#23292).

When the extension is negotiated, on prepare the server hands out a result-metadata id; on execute the driver sends that id back, and the server responds with METADATA_CHANGED + a new id + fresh metadata whenever the cached metadata is stale. The driver updates its prepared-statement cache before deserializing, fixing outdated-metadata errors after schema changes (e.g. ALTER TABLE).

What it does

  • Negotiate SCYLLA_USE_METADATA_ID: parse it from the SUPPORTED frame and send it in STARTUP (scyllaUseMetadataIDExt implementing cqlProtocolExtension, registered in parseCQLProtocolExtensions).
  • Thread the negotiated flag through frame.go/framer.go/conn.go, broadening the existing v5 metadata-id read/write gates from proto > v4 to proto > v4 || scyllaUseMetadataID — no duplicated primitives; the v5 read/write paths are reused.
  • Track result-metadata ids for prepared statements and handle the METADATA_CHANGED flag, updating the cached prepared statement.
  • Skip result metadata once a metadata id is in hand, on native v5 as well as on v4 with the extension (see Behaviour changes).

v5 is not auto-negotiated — discoverProtocol() caps at protoVersion4. Against Scylla the capability is reached via this extension on v4; native v5 is only reachable with an explicit ProtoVersion: 5.

Behaviour changes

Skip-metadata becomes the effective default wherever a result metadata id is exchanged.

NewCluster() sets DisableSkipMetadata: true, so today gocql never sends skip_metadata. That default is not upstream's — apache/cassandra-gocql-driver leaves it false and skips on every protocol version. It was flipped in f292aaf ("Disable skipping metadata by default", Ref: scylladb/scylladb#20860) precisely because prepared-statement metadata could not be invalidated safely. This PR retires that workaround: where the driver holds a result metadata id, the server reports staleness via METADATA_CHANGED, skipping is safe, and DisableSkipMetadata is ignored — including when it was set to true explicitly.

This follows the rule stated in the parent task, scylla-drivers#81:

Default value of skip metadata flag should be treated as safest option. Which means that if SCYLLA_USE_METADATA_ID was negotiated or CQL v5 is used then it is true, result metadata is skipped.

Conn.tracksResultMetadataID() reports either mechanism. Scoping the override to the extension alone would leave gocql with opposite defaults for two encodings of the same mechanism, and the losing one would be the one where the id is guaranteed by the protocol rather than negotiated — since #593 landed native v5, a ProtoVersion: 5 session would carry full result metadata on every response for no reason.

The sibling drivers split on this, which is worth recording:

driver knob behaviour once a metadata id is in play
python-driver#770 none skip_meta = bool(result_metadata) and result_metadata_id is not None; _should_skip_metadata() emits the flag only when the connection negotiated the extension, deliberately not on native v5 ("a separate behavior change out of scope")
java-driver#599 + #758 (4.x), #663 + #757 (3.x) advanced.prepared-statements.skip-cql4-metadata-resolve-method (smart|enabled|disabled) DefaultPreparedStatement.resolveSkipMetadata() returns true before ever reading the option once resultMetadataId is non-empty — which native v5 always supplies, so java skips there too

gocql follows #81 and java-driver here rather than python-driver. Consequences:

  • Per-query Query.NoSkipMetadata() still forces metadata, and is now the only way to do so — there is deliberately no gocql equivalent of java-driver's session-level disabled. ScanCAS/MapScanCAS set it internally, so conditional statements are unaffected either way; and in practice a prepared conditional statement's result metadata is empty, so the column-set gate below already prevents skipping for them.
  • The Validate() warning from 5e07ffd fires on !DisableSkipMetadata, so after this change the population that actually gets skipping (default config) is unwarned while those who opted in explicitly still are. The protocol version is negotiated per connection, and the extension long after Validate runs, so the trigger can't be narrowed; the text now names the case that is still risky — a connection that exchanges no result metadata id at all.

A metadata id is required, not just a capable connection. Both siblings gate on a non-empty result metadata id on the statement being executed, not only on the connection. That distinction is reachable here: the prepared cache is keyed (hostID, keyspace, statement) and is evicted only on prepare failure or UNPREPARED, never on connection close — so a statement prepared before the extension was negotiated survives a reconnect to a now-extension-enabled connection. Without the gate the driver would request skip_metadata while sending an empty id. Such a statement now asks for metadata for one more round trip, acquires an id from the resulting METADATA_CHANGED response, and skips from then on.

The non-empty-column-set gate is load-bearing, not an optimization. A statement whose RESULT/Prepared carries no result metadata is handed an id hashed from empty metadata. Current Scylla compares the client-supplied id against that same empty-metadata id, always matches, and so never sets METADATA_CHANGED — leaving a driver that asked to skip with a response it has no columns to decode. LIST ROLES OF is the motivating case. The server-side fixes for this, scylladb#29233 and scylladb#29275, are both closed unmerged, so no released Scylla has them. gocql never skips without cached columns, which is what keeps such statements working; java-driver checks the same thing first of all, and python-driver's bool(result_metadata) does the same. It is now documented and unit-tested as such rather than left to look incidental.

dialer.GetFrameHash gained a parameter. GetFrameHash(frame []byte, useMetadataID bool) is a source-incompatible change to an exported symbol. The v4 EXECUTE resultMetadataID field cannot be inferred from the frame bytes, so the negotiated state has to be plumbed in. The break is deliberate rather than papered over with a defaulting wrapper: a hash used to match recorded frames is better served by a compile error than by a silently-wrong default. dialer is the record/replay and single-connection benchmark harness the driver runs against itself, has no consumers outside its own subpackages, and now carries a package doc saying its signatures track the wire handling they mirror and may change without a major bump. Record gains a use_metadata_id field, which defaults to false and so leaves existing recordings valid.

Commits

commit what
scylla,conn,frame: negotiate SCYLLA_USE_METADATA_ID and skip result metadata under it the feature; the skip-metadata rule and its three gates; one derivation of the negotiated flag (newFramerWithExts, which had no non-test callers, is deleted rather than kept in sync); the METADATA_CHANGED-without-metadata guard
dialer: derive the header shift once, bound the metadata-id read, parse the option map one masked headerShift(); bounds the now-live v4 length reads; walks the STARTUP [string map] instead of substring-scanning it; package doc
tests,CI: cover the v4 read path, the malformed response and proto v5 v4 read-side unit coverage; rolling-upgrade integration test; the malformed-METADATA_CHANGED regression test, which needed the mock server to speak the extension and so gives the negotiated v4 path its first unit-level coverage end to end; a proto-5 CI step; one existing test strengthened

The frame: bound pkeyCount allocation in parsePreparedMetadata commit that used to ride along here is an unrelated pre-existing defect and an upstream backport candidate. It has been split into #976 so it can merge on its own timeline.

Notable review outcomes

  • METADATA_CHANGED with a new id but no column metadata is rejected as an error. Two things had to be prevented, not one. Adopting the id while keeping the old columns is unrecoverable — the server would match the id from then on and stop sending metadata, leaving the driver decoding against stale columns indefinitely; python-driver guards that identically. But returning the response's own rows is the same misdecode one execute earlier, since the server has just declared those columns stale and the skip-metadata path would reuse them anyway. An earlier revision logged and carried on, which meant a malformed response produced silently wrong rows instead of an error. The query now fails with the old id still cached, so a retry resends it and the server gets another chance to answer with the metadata it owes.
  • Conn and its framers can no longer disagree. The flag was derived independently in three places, one of which (newFramerWithExts) had no non-test callers. A Conn that believed the extension was on while its framers did not would request skip_metadata while writing no id. newFramerWithExts is now deleted and its scylla_test.go call sites go through connFramers.initCache, so there is one derivation rather than two plus a comment asking future authors to keep them aligned. One framer is still not built from framerConfig — framerPool.get falls back to newFramer when the pool is disabled — but that is correct during the handshake and unreachable from the request path afterwards, because execInternal takes its framer before addCall rejects a closed connection. The accessor's doc comment says exactly that rather than claiming an invariant that holds by call ordering rather than by construction; #982 tracks the fix, which also covers the pre-existing flagLWT case (zero makes meta.lwt unconditionally true).
  • The dialer EXECUTE parser's resultMetadataID read was dead code before this series and is now live on v4, where loadResponseFramesFromFiles feeds it bytes straight out of a recording file. The two length reads are bounded and the final slice range validated, falling back to raw-bytes hashing like the existing v5 guard. Recorded hashes are unchanged wherever the parse was already in bounds. Note the limit: addQueryParams, which runs after them, is still unbounded — pre-existing and shared with the opQuery branch.
  • StartupNegotiatesMetadataID no longer substring-scans the frame. startupOptions puts caller-supplied DRIVER_NAME, DRIVER_VERSION and ApplicationInfo values into the same STARTUP [string map], so a caller could latch the flag by naming their application after the extension — and a spuriously latched flag makes GetFrameHash skip a field that isn't on the wire, surfacing as silent replay hash mismatches rather than an error. The map is now walked properly and only keys are compared.
  • The integration tests no longer skip unconditionally. The extension case skips only when the server does not advertise the capability, and fails when the server advertises it but negotiation did not happen. It is the only end-to-end coverage of the feature, so a negotiation regression must not be able to turn it green.

Tests

  • Unit: extension negotiation/registration/serialize; framer propagation; Conn.tracksResultMetadataID over both mechanisms and both protocol versions, including that it masks the direction bit and that usesMetadataID stays narrower so a v5 connection cannot pass for a negotiated extension; EXECUTE encoding on v4+extension, v4 without it, v5, and with a nil id (zero-length short bytes); the skip-metadata decision including the nil/zero-length-id, id-without-exchange, explicit-opt-in and empty-column-set cases; a truncated resultMetadataID in RESULT/Prepared reported as an error rather than a serve-goroutine panic; the read side of METADATA_CHANGED on v4 as well as v5; dialer truncation fallbacks, the v2 header shape, option-map parsing (key-only matching, truncation, overstated count) and a SUPPORTED response not latching the flag.
  • Unit, end to end over a negotiated extension: TestExecuteMetadataChangedWithoutColumns drives a METADATA_CHANGED+NO_METADATA response through a TestServer that advertises SCYLLA_USE_METADATA_ID, so the driver reads a result metadata id out of RESULT/Prepared and writes one back on EXECUTE — the negotiated v4 path's first coverage outside the integration suite. The mock server rejects that execute unless skip_metadata was requested, which pins the skip-metadata decision on a live connection rather than only against shouldSkipResultMetadata's arguments. Reverting the rejection makes the test fail by returning rows, which was the defect.
  • Integration: TestPrepareExecuteMetadataChangedFlag is now table-driven over {native v5, v4 + SCYLLA_USE_METADATA_ID} rather than carrying a second near-verbatim copy of its ~150-line flow, and drops/recreates its table so it is rerunnable against the same keyspace. TestPrepareExecuteScyllaEmptyMetadataID covers the rolling-upgrade sentinel, mirroring java-driver#758's should_handle_empty_metadata_id_when_executing_statement_when_supported and python-driver#770's test_empty_sentinel_id_triggers_metadata_changed.
  • The native-v5 case now runs in CI. Makefile pins TEST_CQL_PROTOCOL ?= 4, so every ProtoVersion >= 5 integration test skipped on every leg — including the v5 half of the skip-metadata default this PR adds, which rested on unit tests only. The Cassandra 5-LATEST job gains one step at TEST_CQL_PROTOCOL=5, scoped with -run to TestPrepareExecuteMetadataChangedFlag. TEST_COMPRESSOR=no-compression goes with it out of necessity, not tidiness: the default is snappy, and Validate() rejects any compressor that is not a SegmentCompressor once ProtoVersion >= 5, which would fail every session in the suite. A full v5 lane would also light up ~8 unrelated tests that have never run, and a gap in any of them would block this work — that lane stays #960.
  • make test-unit green (root and the nested lz4 module, -race); make check reports 0 lint issues and no go.mod drift; gofmt -l ., go vet ./... and go vet -tags "integration gocql_debug" ./... clean; each of the three commits builds and vets standalone.

Fixes

@dkropachev

Copy link
Copy Markdown
Collaborator

@nikagra , please rebase and update PR description

@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 3 times, most recently from 1f2fe11 to 4999fb6 Compare November 4, 2025 17:39
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 10 times, most recently from e2ebb22 to 8484cdc Compare March 31, 2026 17:39
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch from 8484cdc to c2f729d Compare April 13, 2026 11:35
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch from c2f729d to 4c5fc24 Compare June 19, 2026 12:17
@coderabbitai

coderabbitai Bot commented Jun 19, 2026 •

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The change adds SCYLLA_USE_METADATA_ID support for protocol v4. It propagates negotiated state through framers, prepared execution, metadata caching, dialer recording, and replay hashing. It adds bounded parsing and malformed-frame validation. Tests cover metadata refresh, stale and empty IDs, segmented reads, deadlines, and protocol-v5 execution.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Conn
  participant Framer
  participant Server
  participant MetadataCache
  Client->>Conn: execute prepared statement
  Conn->>Framer: encode metadata ID and skip decision
  Framer->>Server: send EXECUTE request
  Server-->>Framer: return metadata response
  Framer-->>Conn: decode metadata ID and columns
  Conn->>MetadataCache: replace valid cached metadata
  Conn-->>Client: return decoded rows
Loading

Possibly related PRs

Suggested labels: area/Driver_-_gocql

Suggested reviewers: dkropachev, sylwiaszunejko

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 68.33% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies implementation of SCYLLA_USE_METADATA_ID protocol-extension negotiation, which is the main change.
Description check ✅ Passed The description directly and comprehensively explains the protocol-extension negotiation, metadata handling, tests, and behavior changes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Comment @coderabbitai help to get the list of available commands.

@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 2 times, most recently from d088322 to 1abf28b Compare June 26, 2026 21:00
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 2 times, most recently from b39d8ce to eae7353 Compare June 30, 2026 22:29
@nikagra nikagra changed the title [#527] Implement SCYLLA_USE_METADATA_ID protocol extention negotiat… Implement SCYLLA_USE_METADATA_ID protocol extention negotiat… Jul 1, 2026

@dkropachev dkropachev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one inline comment about a test reliability issue. The protocol behavior itself looks aligned with how the Rust driver handles SCYLLA_USE_METADATA_ID, including ignoring the cached-metadata opt-out when the extension is available.

Comment thread cassandra_test.go Outdated
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 3 times, most recently from 63d85e6 to 91b744d Compare July 13, 2026 18:13
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch from 63facdd to 656999c Compare August 3, 2026 12:29
@nikagra
nikagra force-pushed the support-scylla-use-metadata-id-extention branch 2 times, most recently from 278546d to 6de9131 Compare August 3, 2026 21:12
@nikagra
nikagra marked this pull request as ready for review August 3, 2026 22:13
@nikagra
nikagra requested a review from dkropachev August 3, 2026 22:14

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
dialer/utils.go (1)

317-353: 🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

Make all EXECUTE parsing bounds-aware before indexing.

GetFrameHash reads stream-ID bytes before it validates a complete legacy header. It also calls addQueryParams before the check at Lines 351-353. A short v1-v4 frame, or an EXECUTE frame truncated in query parameters, can still panic while loadResponseFramesFromFiles loads a damaged recording. Validate the header before stream normalization and make query-parameter parsing return failure before indexing.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dialer/utils.go` around lines 317 - 353, Make EXECUTE parsing in GetFrameHash
fully bounds-aware before any frame indexing. Validate that the complete legacy
header, including stream-ID bytes, is present before stream normalization, and
update addQueryParams to signal truncated input so callers return the raw-byte
Murmur3H1 fallback instead of indexing invalid offsets. Ensure the EXECUTE path
handles short v1–v4 frames and truncated query parameters without panicking.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dialer/recorder/recorder.go`:
- Around line 97-108: The recorder currently evaluates
StartupNegotiatesMetadataID against incomplete data after f.record is reset
across Write calls. Update the framing logic in Write to retain and append the
in-progress STARTUP frame until its declared length is complete, then evaluate
dialer.StartupNegotiatesMetadataID and latch f.useMetadataID; preserve the
existing f.record.UseMetadataID assignment for subsequent records.

---

Outside diff comments:
In `@dialer/utils.go`:
- Around line 317-353: Make EXECUTE parsing in GetFrameHash fully bounds-aware
before any frame indexing. Validate that the complete legacy header, including
stream-ID bytes, is present before stream normalization, and update
addQueryParams to signal truncated input so callers return the raw-byte
Murmur3H1 fallback instead of indexing invalid offsets. Ensure the EXECUTE path
handles short v1–v4 frames and truncated query parameters without panicking.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Pro Plus

Run ID: d92ba146-b4e2-48dc-9020-83f19862e22b

📥 Commits

Reviewing files that changed from the base of the PR and between 4bee517 and 6de9131.

📒 Files selected for processing (15)
  • cassandra_test.go
  • cluster.go
  • conn.go
  • conn_test.go
  • dialer/recorder/recorder.go
  • dialer/replayer/replayer.go
  • dialer/replayer/replayer_test.go
  • dialer/utils.go
  • dialer/utils_test.go
  • frame.go
  • frame_test.go
  • framer.go
  • scylla.go
  • scylla_test.go
  • session.go

Comment thread dialer/recorder/recorder.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
dialer/utils.go (1)

278-299: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Validate the full header and short-bytes payloads before indexing.

A nonempty v4 frame shorter than 8 + headerShift(frame) panics on header access. A truncated resultMetadataID payload also moves endIndex beyond the slice before addQueryParams runs. The same payload check is missing after preparedIDLen.

Fix
 p := headerShift(frame)
+if len(frame) < 8+p {
+	return murmur.Murmur3H1(frame)
+}
 if p == 1 {
@@
 preparedIDLen := int(frame[index])<<8 | int(frame[index+1])
 endIndex = endIndex + 2 + preparedIDLen
+if endIndex > len(frame) {
+	return murmur.Murmur3H1(frame)
+}
@@
 resultMetadataIDLen := int(frame[endIndex])<<8 | int(frame[endIndex+1])
 endIndex = endIndex + 2 + resultMetadataIDLen
+if endIndex > len(frame) {
+	return murmur.Murmur3H1(frame)
+}

Also applies to: 322-343

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dialer/utils.go` around lines 278 - 299, In the frame-hashing/parsing flow
around headerShift, validate that nonempty v4 frames contain at least 8 +
headerShift(frame) bytes before accessing header fields. Before advancing
endIndex or calling addQueryParams, validate the resultMetadataID payload
length, and add the equivalent bounds check after preparedIDLen so truncated
payloads return safely instead of indexing past the slice.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@dialer/utils.go`:
- Around line 278-299: In the frame-hashing/parsing flow around headerShift,
validate that nonempty v4 frames contain at least 8 + headerShift(frame) bytes
before accessing header fields. Before advancing endIndex or calling
addQueryParams, validate the resultMetadataID payload length, and add the
equivalent bounds check after preparedIDLen so truncated payloads return safely
instead of indexing past the slice.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Pro Plus

Run ID: a4257ecd-8245-4ea8-9490-a05d40543e49

📥 Commits

Reviewing files that changed from the base of the PR and between 6de9131 and 32ff7ef.

📒 Files selected for processing (16)
  • .github/workflows/main.yml
  • cassandra_test.go
  • cluster.go
  • conn.go
  • conn_test.go
  • dialer/recorder/recorder.go
  • dialer/replayer/replayer.go
  • dialer/replayer/replayer_test.go
  • dialer/utils.go
  • dialer/utils_test.go
  • frame.go
  • frame_test.go
  • framer.go
  • scylla.go
  • scylla_test.go
  • session.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dialer/utils.go`:
- Around line 279-283: Update GetFrameHash and the legacy-frame parsing path
around headerShift and addQueryParams to validate bounds before masking, reading
opcodes, or indexing frame data. Reject v4 frames shorter than the complete
header, and verify every variable-length field—including EXECUTE
resultMetadataID—fits within the remaining bytes before passing control to
addQueryParams or continuing parsing.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Pro Plus

Run ID: 65f09c61-a97f-4edc-a6f0-0d8c557d73a7

📥 Commits

Reviewing files that changed from the base of the PR and between 32ff7ef and f90cdab.

📒 Files selected for processing (16)
  • .github/workflows/main.yml
  • cassandra_test.go
  • cluster.go
  • conn.go
  • conn_test.go
  • dialer/recorder/recorder.go
  • dialer/replayer/replayer.go
  • dialer/replayer/replayer_test.go
  • dialer/utils.go
  • dialer/utils_test.go
  • frame.go
  • frame_test.go
  • framer.go
  • scylla.go
  • scylla_test.go
  • session.go

Comment thread dialer/utils.go
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dialer/recorder/recorder.go`:
- Around line 104-118: Update the frame-recording logic around the to_record
handling so each CQL frame becomes a separate record. Limit the current append
to the remaining bytes in the frame, then loop over surplus bytes to initialize
and record subsequent frames, preserving their lengths and metadata handling
through StartupNegotiatesMetadataID and UseMetadataID.

In `@dialer/utils.go`:
- Around line 148-150: Update fits to validate index is nonnegative and no
greater than len(frame) before performing subtraction, then validate n is
nonnegative and at most len(frame)-index; avoid calculating index+n so malformed
lengths cannot overflow.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Pro Plus

Run ID: a009dbd2-7078-4e64-9ce3-66e697690a35

📥 Commits

Reviewing files that changed from the base of the PR and between 0fdab60 and 5eb5170.

📒 Files selected for processing (17)
  • .github/workflows/main.yml
  • cassandra_test.go
  • cluster.go
  • conn.go
  • conn_test.go
  • dialer/recorder/recorder.go
  • dialer/recorder/recorder_test.go
  • dialer/replayer/replayer.go
  • dialer/replayer/replayer_test.go
  • dialer/utils.go
  • dialer/utils_test.go
  • frame.go
  • frame_test.go
  • framer.go
  • scylla.go
  • scylla_test.go
  • session.go

Comment thread dialer/recorder/recorder.go Outdated
Comment thread dialer/utils.go
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (2)
dialer/utils_test.go-316-321 (1)

316-321: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

The "truncated at the metadata id length" case never reaches that check.

v4ExecuteFrame(true) is 21 bytes: header 0–8, preparedID length 9–10, preparedID payload 11–13, resultMetadataID length 14–15. full[:11] fails the preparedID payload check (fits(frame, 9, 2+3)), so both cases exercise the same guard. Use full[:15] to make the resultMetadataID length field the truncation point.

💚 Proposed fix
 		{
 			// preparedID parses, then the metadata-id length field runs off the end.
 			name:          "truncated at the metadata id length",
-			frame:         full[:11],
+			frame:         full[:15],
 			useMetadataID: true,
 		},
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dialer/utils_test.go` around lines 316 - 321, The test fixture named
“truncated at the metadata id length” uses a slice that truncates during the
preparedID payload instead. Update its frame slice to full[:15] so
v4ExecuteFrame(true) reaches the resultMetadataID length-field boundary, while
leaving the neighboring truncation case unchanged.
dialer/utils.go-460-468 (1)

460-468: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Bound the v1 values-count read.

The v1 branch reads frame[index] and frame[index+1] with no fits check. The only preceding guard is fits(frame, 0, 8+p), which for v1 (p == 0) proves only len(frame) >= 8. A v1 EXECUTE record with a header-length body panics here, which is the failure mode the rest of this function now avoids.

🛡️ Proposed guard
 		} else {
+			if !fits(frame, index, 2) {
+				return murmur.Murmur3H1(frame)
+			}
 			valuesLen := int(frame[index])<<8 | int(frame[index+1])
 			index = index + 2
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dialer/utils.go` around lines 460 - 468, In the v1 branch of the surrounding
parser, validate that two bytes are available at the current index before
reading frame[index] and frame[index+1] for valuesLen. If the bounds check
fails, return the existing Murmur3H1 fallback, preserving the current behavior
for valid records and preventing short v1 EXECUTE frames from panicking.
🧹 Nitpick comments (1)
dialer/utils.go (1)

291-298: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Advance index past the custom-payload count unconditionally.

index is advanced by 2 only when customPayloadLenght > 0. For a zero-length custom payload the count short stays in the range, so addQueryParams then reads the count bytes as the consistency field. The hash stays deterministic, so replay still matches, but the extracted range is wrong.

♻️ Proposed change
 	customPayloadLenght := int(frame[8+p])<<8 | int(frame[9+p])
-	if customPayloadLenght > 0 {
-		index = index + 2
-	}
+	index = index + 2
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dialer/utils.go` around lines 291 - 298, Update addCustomPayload so index is
advanced past the two-byte custom-payload count unconditionally after the bounds
check, regardless of customPayloadLenght. Preserve the existing length
evaluation and return behavior while ensuring subsequent addQueryParams parsing
starts after the count field.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Other comments:
In `@dialer/utils_test.go`:
- Around line 316-321: The test fixture named “truncated at the metadata id
length” uses a slice that truncates during the preparedID payload instead.
Update its frame slice to full[:15] so v4ExecuteFrame(true) reaches the
resultMetadataID length-field boundary, while leaving the neighboring truncation
case unchanged.

In `@dialer/utils.go`:
- Around line 460-468: In the v1 branch of the surrounding parser, validate that
two bytes are available at the current index before reading frame[index] and
frame[index+1] for valuesLen. If the bounds check fails, return the existing
Murmur3H1 fallback, preserving the current behavior for valid records and
preventing short v1 EXECUTE frames from panicking.

---

Nitpick comments:
In `@dialer/utils.go`:
- Around line 291-298: Update addCustomPayload so index is advanced past the
two-byte custom-payload count unconditionally after the bounds check, regardless
of customPayloadLenght. Preserve the existing length evaluation and return
behavior while ensuring subsequent addQueryParams parsing starts after the count
field.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: QUIET

Plan: Pro Plus

Run ID: c4d8b30f-9001-49db-ad91-9a42d0d9183e

📥 Commits

Reviewing files that changed from the base of the PR and between 0fdab60 and 3b42491.

📒 Files selected for processing (17)
  • .github/workflows/main.yml
  • cassandra_test.go
  • cluster.go
  • conn.go
  • conn_test.go
  • dialer/recorder/recorder.go
  • dialer/recorder/recorder_test.go
  • dialer/replayer/replayer.go
  • dialer/replayer/replayer_test.go
  • dialer/utils.go
  • dialer/utils_test.go
  • frame.go
  • frame_test.go
  • framer.go
  • scylla.go
  • scylla_test.go
  • session.go

@nikagra

nikagra commented Aug 6, 2026

Copy link
Copy Markdown
Author

@coderabbitai on the two grouped comments, which have no thread to reply in:

dialer/utils_test.go 316-321 — correct, fixed. full[:11] stopped at the preparedID payload check, so both cases hit the same guard; it is full[:15] now. Verified by deleting the resultMetadataID length guard and watching that case panic at frame[15].

dialer/utils.go 460-468 — false positive. index is unchanged between the fits(frame, index, 2) at line 430, which guards the preparedID length read, and the v1 valuesLen read at 461: the only thing that moves it is addCustomPayload, above both. Added a comment there saying so.

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ast-grep outline dialer/utils.go --items all --type function --match 'GetFrameHash|fits|addCustomPayload'
ast-grep outline dialer/utils_test.go --items all --type function --match 'Test.*Metadata|Test.*Execute|Test.*FrameHash'

printf '\n--- dialer/utils.go target region ---\n'
nl -ba dialer/utils.go | sed -n '400,485p'

printf '\n--- dialer/utils_test.go target region ---\n'
nl -ba dialer/utils_test.go | sed -n '285,340p'

Length of output: 1607


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- dialer/utils.go: GetFrameHash relevant region ---'
awk 'NR >= 400 && NR <= 485 { printf "%6d  %s\n", NR, $0 }' dialer/utils.go

printf '%s\n' '--- dialer/utils_test.go: truncated EXECUTE cases ---'
awk 'NR >= 302 && NR <= 360 { printf "%6d  %s\n", NR, $0 }' dialer/utils_test.go

Length of output: 7150


@nikagra Verified.

  • full[:15] reaches the result-metadata-ID length guard.
  • The valuesLen read at Line 463 is covered by the fits(frame, index, 2) check at Line 430. No code changes index between those lines.

✏️ Learnings added
Learnt from: nikagra
URL: https://github.com/scylladb/gocql/pull/590

Timestamp: 2026-08-06T20:55:05.078Z
Learning: In `dialer/utils.go`, the protocol-v1 `valuesLen` read in `GetFrameHash` uses the same `index` as the earlier prepared-ID length read. `fits(frame, index, 2)` guards both reads because only `endIndex` changes between them.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

You are interacting with an AI system.

@sylwiaszunejko sylwiaszunejko left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice change overall

Comment thread scylla.go Outdated
Comment thread dialer/replayer/replayer.go
nikagra and others added 5 commits August 10, 2026 17:40
…etadata under it

Prepared-statement result metadata could not be invalidated safely: after an
ALTER the server kept answering with the old column set, so a driver reusing the
metadata it cached at prepare time decoded rows against columns that no longer
described the response (scylladb/scylladb#20860). gocql's answer was to stop
reusing it — f292aaf ("Disable skipping metadata by default") flipped
DisableSkipMetadata to default true — at the cost of carrying result metadata on
every response.

scylladb/scylladb#23292 fixes the underlying problem for SELECT statements. A
server advertising SCYLLA_USE_METADATA_ID hands out a result metadata ID at
prepare time; the driver returns that ID with every EXECUTE, and a stale ID is
answered with the METADATA_CHANGED flag plus fresh metadata and a fresh ID. That
is native protocol v5's mechanism made available on v4, which is what Scylla
negotiates. Implement the driver half.

Negotiation and plumbing.

scyllaUseMetadataIDExt implements cqlProtocolExtension and is registered in
parseCQLProtocolExtensions, so it is sent in STARTUP whenever SUPPORTED lists the
key. The v5 metadata-ID gates in frame.go widen from `proto > protoVersion4` to
`proto > protoVersion4 || scyllaUseMetadataID`: the read in parseResultPrepared,
the read in parseResultMetadata behind METADATA_CHANGED, and the write in
writeExecuteFrame. A v4 connection that negotiated the extension therefore drives
the same primitives the v5 port already built, rather than a second
implementation of them.

Detection is consolidated onto the extension. parseSupported already keyed
isMetadataIDSupported — and through it the isScylla heuristic and the
IsMetadataIDSupported() getter — off a function-local SCYLLA_USE_METADATA_ID
const. That const moves to package scope and both the detector and the extension
read it, leaving one spelling and one detector for the capability.

One source of truth for the negotiated flag.

Two independently derived booleans governing the two halves of one wire contract
is a bug waiting to happen: a Conn that believed the extension was on while its
framers did not would ask the server to skip result metadata while writing no ID
for it to compare against, and the driver would then decode rows against whatever
metadata it had cached. So the negotiated state lives in framerConfig, populated
by connFramers.initCache during connection setup before any query can run, and
Conn reads it through the usesMetadataID() and tracksResultMetadataID()
accessors. There is no Conn-level copy to diverge from it.

newFramerWithExts derived the same flags a second time and is deleted. It had no
non-test callers — production framers come from the per-Conn pool — so keeping it
meant every future extension had to be handled in two places, with only a comment
to say so. Its call sites in scylla_test.go now go through initFramerCache and
getWriteFramer, i.e. the path production uses, which is what makes the claim above
true rather than aspirational.

One framer is still not built from framerConfig: framerPool.get falls back to
newFramer when the pool is disabled, which yields scyllaUseMetadataID false whatever
was negotiated. That is correct during the handshake, and unreachable from the
request path afterwards, because execInternal takes its framer before addCall
rejects a closed connection — so a framer taken after the pool closed belongs to a
call that never writes a frame. The accessor's doc comment says so rather than
claiming an invariant that holds by call ordering rather than by construction; scylladb#982
tracks the fix, which also covers flagLWT and tabletsRoutingV1.

Skipping result metadata.

shouldSkipResultMetadata replaces the inline skipMeta expression in
executeQueryWithMetrics, and metadataIDTracked gates it on both halves of the
mechanism: the connection exchanges result metadata IDs *and* the prepared
statement carries a non-empty one. Where both hold, the session-level
DisableSkipMetadata is ignored, including when it was set to true explicitly — the
flag is a workaround for the bug this mechanism fixes, so once the server reports
metadata changes there is nothing left to work around. Upstream gocql skips by
default on every protocol version, and there is deliberately no session-level knob
to force metadata back on; the java-driver's
skip-cql4-metadata-resolve-method has no equivalent here.

The ID exchange is active on native protocol v5, where the field is mandatory, as
well as on v4 with the extension, and Conn.tracksResultMetadataID reports either.
Scoping the override to the extension alone would leave gocql with opposite
defaults for two encodings of one mechanism, and the losing one would be the one
where the ID is guaranteed by the protocol rather than negotiated: a v5 connection
would carry full result metadata on every response for no reason.
scylladb/scylla-drivers#81 states the rule as "if SCYLLA_USE_METADATA_ID was
negotiated or CQL v5 is used", and the java-driver reaches it from the other
direction — DefaultPreparedStatement.resolveSkipMetadata returns true for any
non-empty result metadata ID, which v5 always supplies. The python-driver
implements the extension half only.

The second condition, a non-empty ID, is reachable and matters. The prepared cache
is keyed (hostID, keyspace, statement) and is evicted only on prepare failure or
UNPREPARED, never on connection close, so a statement prepared before the
extension was negotiated survives a reconnect onto a now-extension-enabled
connection. Without the gate the driver would request skip_metadata while sending
an empty ID. With it, such a statement asks for metadata for one more round trip,
acquires an ID from the resulting METADATA_CHANGED response, and skips from then
on — leaving no window in which the driver skips metadata it cannot recover. Both
sibling drivers gate on the same condition (scylladb/python-driver#770,
scylladb/java-driver#599 and follow-ups).

The remaining gate, a non-empty cached column set, is not an optimization either.
A statement whose RESULT/Prepared carries no result metadata is handed an ID
hashed from empty metadata; current Scylla compares the returned ID against that
same empty-metadata ID, always matches, and so never sets METADATA_CHANGED,
leaving a driver that asked to skip with a response it has no columns to decode.
LIST ROLES OF is the motivating case. The server-side fixes,
scylladb/scylladb#29233 and scylladb/scylladb#29275, are both closed unmerged, so
this gate is what keeps such statements working; document and test it as such.

Query.NoSkipMetadata wins in every case, and is now the only way to force metadata
where the ID exchange is active. Conditional statements are the case to keep in
mind, since their response column set depends on whether the condition applied —
something a result metadata ID cannot express, as it describes the statement and
not the outcome. In practice the column-set gate already covers them, because a
prepared conditional statement's result metadata is empty, and ScanCAS and
MapScanCAS set NoSkipMetadata internally regardless.

A RESULT/Rows that sets METADATA_CHANGED while also setting NO_METADATA is
rejected. The combination is malformed — METADATA_CHANGED obliges the server to
include the new metadata — and both ways of continuing are unrecoverable. Adopting
the ID while keeping the old columns lets the server match it from then on and stop
sending metadata, leaving the driver decoding against stale columns indefinitely;
the python-driver guards that identically. But returning the response's own rows is
the same misdecode one execute earlier: the server has just declared those columns
stale, and the skip-metadata path below would decode against them anyway. So do
neither, and fail the query with the old ID left in the cache — a retry resends it
and the server gets another chance to answer with the metadata it owes.

Record and replay.

The record/replay dialers hash EXECUTE frames at fixed offsets, and skipped the
resultMetadataID field only for protocol v5+. Under this extension that field also
appears on v4 EXECUTE frames, which the frame bytes alone cannot reveal, so the
negotiated state is plumbed through instead: StartupNegotiatesMetadataID detects
the opt-in on both the record and the replay path, the recorder latches it and
stamps each Record with UseMetadataID, and GetFrameHash takes it as an argument.

Documentation.

The DisableSkipMetadata comment said the driver "may still" send skip_metadata
under the extension, which understates it: the flag defaults to true, so the
override is the normal case rather than an exception, and "Default: true" is
misleading on its own. It now says plainly that the flag is ignored — explicit
values included — which connections that applies to, and why that is safe. The
Validate() warning fires on !DisableSkipMetadata, so after the override the
population that actually gets skipping is unwarned while those who opted in
explicitly still are. The protocol version is negotiated per connection, and the
extension long after Validate runs, so the trigger cannot be narrowed; the message
instead names the case that is still risky — a connection that exchanges no result
metadata ID at all.

Tests.

Unit coverage for the extension's negotiation, registration and serialization
(scylla_test.go); for initFramerCache and usesMetadataID against the framer
config, with a negative counterpart for the not-negotiated case; for
tracksResultMetadataID over both mechanisms and both protocol versions, including
that it masks the request/response direction bit, and that usesMetadataID stays
narrower so a v5 connection cannot pass for a negotiated extension; for
shouldSkipResultMetadata composed with metadataIDTracked the way the EXECUTE path
composes them, over the nil and zero-length ID cases, an ID without an ID
exchange, an explicit opt-in, and the empty-column-set gate; and for the
v4-plus-extension GetFrameHash skip. A regression test pins that a truncated
resultMetadataID in a RESULT/Prepared frame is reported as an error through
parseFrame's recover rather than panicking the serve goroutine — the extension
makes that short-bytes read live on protocol v4.

TestPrepareExecuteMetadataChangedFlag becomes table-driven over both ways the ID
exchange can be active, rather than growing a second near-verbatim copy of its
~150-line flow for the extension. It also drops and recreates its table instead of
CREATE IF NOT EXISTS, because the flow ALTERs that table and one left over from an
earlier run already has the added column; and it asserts the no-change case by
pointer identity on the cache entry, since comparing the entry's fields compares it
with itself and can never fail. The extension case stops skipping unconditionally:
it skips only when the server does not advertise the capability, and fails when the
server advertises it but negotiation did not happen, since it is the only
end-to-end coverage of the feature and a negotiation regression must not be able to
turn it green. The v5 case does not run at the suite's default protocol, since
TEST_CQL_PROTOCOL is pinned to 4; the tests commit later in this series gives it an
explicit run against Cassandra 5.

One occurrence of the Id spelling is deliberate, in frame_test.go's transcription
of Cassandra's ResultSet$ResultMetadata$Codec.encode — a verbatim quote of Java
source, which keeps its original naming.

Fixes: https://scylladb.atlassian.net/browse/DRIVER-152
Fixes: scylladb#527
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…option map

Four problems in the record/replay harness — three introduced or made reachable by
the SCYLLA_USE_METADATA_ID support, one pre-existing and surfaced by it — plus the
package's missing stability note.

StartupNegotiatesMetadataID picked the opcode offset with an unmasked
`frame[0] > 0x02`, immediately above the const block whose comment states that
version comparisons in this file must always mask with protoVersionMask so they
cannot disagree with each other. GetFrameHash derives the same offset masked. Pull
the derivation into headerShift() and use it from both, so there is one definition
of where a frame's opcode is.

This is a consistency fix, not a bug fix: for a request frame the direction bit is
clear, and for a v3+ response the masked and unmasked forms pick the same offset, so
no realistic input is misread today. The value is that the next person to add a
parser here cannot pick the third variant.

The same function then decided the opt-in by scanning the whole frame for the
literal SCYLLA_USE_METADATA_ID. gocql's startupOptions puts caller-influenced values
into the same STARTUP [string map] — DRIVER_NAME, DRIVER_VERSION, DRIVER_CONFIG,
SESSION_ID and whatever ApplicationInfo adds — so a caller could latch the flag by
naming their application after the extension, and a spuriously latched flag makes
GetFrameHash skip a field that is not on the wire, which surfaces as silent replay
hash mismatches rather than an error. Walk the map properly instead and compare keys
only, treating a malformed or truncated map as "not negotiated". That list has grown
twice while this branch was open, which is the argument for matching on keys: it does
not care what else ends up in the map.

Bound every read the frame parser makes. The resultMetadataID skip was dead code
before this series — the v5 guard diverted everything that could reach it — and is
now live on protocol v4, where loadResponseFramesFromFiles feeds GetFrameHash bytes
straight out of a recording file. A damaged or wrongly-stamped record made the parser
index past the end and panic the replayer.

The first gap is ahead of the parse: GetFrameHash blanks the stream id at frame[2..3]
and switches on the opcode at frame[3+p] having checked only that the frame is
non-empty and not v5, so GetFrameHash([]byte{0x04}, false) panicked two statements
into the function. The rest are lengths. The preparedID and resultMetadataID
short-bytes fields had their length words checked but not the payloads those words
announce, so a plausible length running off the end moved the cursor out of range and
handed it to addQueryParams — which, along with addBytes and addCustomPayload,
checked nothing at all. That also left the opQuery branch, which reaches
addQueryParams by a different route and slices on its result without a range check,
able to panic on any short QUERY.

So the three walking helpers return (int, bool) and refuse rather than index, a
fits() predicate states the check once, and every caller falls back to hashing the
raw bytes the way the v5 guard does. fits compares n against the space remaining
instead of adding it to index, because on a 32-bit int the largest length a [bytes]
field can encode overflows the obvious index+n and the negative sum then passes
every upper bound. addBytes also reads that length as signed, which the CQL spec
says it is: -1 encodes a null value, and reading it unsigned turns a null bind value
into a 4 GB payload that no bounds check can accept.

Recorded hashes do not change wherever the parse was already in bounds. The
checked-in tests/bench recordings replaying unchanged is the end-to-end form of that
claim: they are rehashed at load time and matched against live frames, so any shift
in the offset math surfaces as "unable to find a response to replay".

The recorder never framed anything. FrameWriter.new was set when a frame completed
and never cleared, so the reset at the top of Write ran on every call and no frame
ever accumulated: one delivered in two pieces became two records, the first cut short
and the second starting mid-frame. The metadata-ID latch is what makes that visible —
it looks for the opt-in key in a completed STARTUP, so a truncated one misses, every
later EXECUTE is stamped false, and replay fails as silent hash mismatches — but the
damage predates it. The v5 rejection reads its version byte from the same restarted
record, so a call boundary falling mid-body could read an arbitrary payload byte as a
protocol version and refuse a perfectly good v4 connection.

Clearing the flag would only have covered half of it, because neither side delivers
one frame per call in the other direction either: Conn wraps the dialed connection in
a bufio.Reader, which fills with reads of up to its buffer size, so a read can carry
several responses as easily as half of one. Write is therefore a loop over consume(),
which takes only the bytes belonging to the frame in progress — the header first,
since the declared body length is unknown until it is complete, then exactly that
many body bytes — records the frame once it is whole, and starts the next from what
is left.

to_record becomes bodyLeft and means what it says, and the old length arithmetic goes
with it: | binds looser than + and - in Go, so
`9 + b0<<24 | b1<<16 | b2<<8 | b3 - recorded_ealier` was never the sum it read as,
and agreed with one only because recorded_ealier was always zero. FrameWriter.new is
gone too — "the header is not complete yet" is len(record.Data) < headerLen, which
needs no flag and leaves the zero value usable.

The package gets the doc comment it never had. GetFrameHash's signature change is
source-incompatible for anything outside this repo that called it, and nothing said
whether that was allowed. It is: this package is the record/replay and
single-connection benchmark harness the driver uses on itself, its exported
identifiers exist so the recorder, replayer and benchmarks can share frame parsing,
and they change whenever the wire handling they mirror does — GetFrameHash needs
whatever protocol context a frame's layout depends on but its bytes do not reveal.
Say so, rather than leaving the break to look like an oversight.

Tests: cover both truncation points, and assert GetFrameHash leaves the caller's
frame unmutated. Every prefix of a header-only frame, in both header shapes, must
divert to the raw-bytes hash; so must a preparedID or resultMetadataID payload that
overruns its frame, an EXECUTE truncated at its query flags or at a bound value, and
a QUERY truncated at its flags or inside its custom payload — with a well-formed
QUERY as the control that bounding the walk did not turn parsing into a fallback.
Each of those panics against the parser as it stood. fits gets a direct test at
MaxInt, since the overflow it now avoids is unreachable through a real frame on a
64-bit target.

TestStartupNegotiatesMetadataID now builds real [string map] bodies — with the v2
header shape (1-byte stream id, so the opcode is at index 3) to pin the shift=0
branch, a SUPPORTED response advertising the key, since the recorder's read-side
FrameWriter calls this on every completed response frame and must not latch on it,
the key appearing only in option *values*, every truncation of a valid frame, and a
map count claiming more entries than the body holds. Also give the test's synthetic
frames a correct body length; they declared zero.

On the recorder side: a STARTUP split inside its header, and again inside its body,
comes back as one record byte-equal to the frame that was written; two responses
arriving in one read come back as two records with their own stream ids; a call
ending part-way into the following header completes it on the next one; and every
record carries UseMetadataID.

Refs: https://scylladb.atlassian.net/browse/DRIVER-152
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The read side of METADATA_CHANGED on protocol v4 had no unit coverage: the gate in
parseResultMetadata was exercised only by an integration test that skips unless a
capable server is present. Parameterise
TestParseResultMetadata_PagingStateBeforeNewMetadataID over native v5 and v4 with
the extension — it already synthesises the one frame shape that distinguishes the
field order (both HAS_MORE_PAGES and METADATA_CHANGED set), so the v4 case is a
two-line addition that pins the extension's read path against the same layout.
Removing `|| f.scyllaUseMetadataID` from the gate now fails the v4 subtest and
leaves the v5 one passing.

Add TestPrepareExecuteScyllaEmptyMetadataID for the rolling-upgrade case the
previous commits' skipMeta gate exists to protect: a statement cached before the
extension was negotiated, executed over an extension-enabled connection. It asserts
both halves — the row still decodes, so the server did send metadata, and a fresh ID
is adopted afterwards, so the statement heals. The same scenario is covered in
java-driver#758 and python-driver#770, both verified against a live server; this
gives gocql the equivalent.

Cover the malformed METADATA_CHANGED response the first commit now rejects, which
needed the mock server to speak the extension and so gives the negotiated v4 path its
first unit-level coverage end to end. TestServer advertises SCYLLA_USE_METADATA_ID
through the existing supportedFactory seam; process() then writes a result metadata ID
after the prepared ID in RESULT/Prepared and consumes the one the driver sends on
EXECUTE, both gated on the advertised key so every existing test keeps producing
exactly the frames it did. A canned statement replies with METADATA_CHANGED and
NO_METADATA together, and the server rejects that execute outright unless
skip_metadata was requested — which pins the skip-metadata decision on a live
connection rather than only against shouldSkipResultMetadata's arguments. The test
asserts the query fails and that the ID from the malformed response was not cached.
Reverting the rejection makes it fail by returning rows, which is the defect: the old
code logged and then decoded against the stale columns anyway.

Give the native-v5 half of the result-metadata-ID exchange a CI run. Every
ProtoVersion >= 5 integration test skips at the default TEST_CQL_PROTOCOL=4, so the
v5 side of the skip-metadata default the first commit adds rested on unit tests only.
Add one step to the Cassandra 5-LATEST job at TEST_CQL_PROTOCOL=5, scoped with -run to
TestPrepareExecuteMetadataChangedFlag: a full v5 lane is worth having, but it would
also light up ~8 unrelated tests that have never run, and a gap in any of them would
block this work. That lane stays scylladb#960. TEST_COMPRESSOR=no-compression is required
rather than tidy — the default is snappy, and ClusterConfig.Validate rejects any
compressor that is not a SegmentCompressor once ProtoVersion >= 5, which would fail
every session in the suite. cassandra-start short-circuits when the cluster is already
up, so the extra step costs only the test run.

One fix to an existing test: Test_framer_writeExecuteFrame compared
params.consistency against the value read back while both were the zero value, so
that assertion could not fail on the v4 subtests. Use Quorum, and consume the
trailing 1-byte query flags and require the buffer to be empty — which turns the
subtests into a check that every field was read at exactly the length it was
written, the resultMetadataID short bytes included.

Both new tests look their statement up in the prepared cache through
preparedLRU.keyFor, which takes the host id as a UUID since 31421d1 — so they pass
HostInfo.hostUUID() rather than the HostID() string every other call site here used
before that change.

Refs: https://scylladb.atlassian.net/browse/DRIVER-152
Refs: scylladb#960
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
parseCQLProtocolExtensions registered scyllaUseMetadataIDExt on the presence of the
key in SUPPORTED alone, and that list is both what startupCoordinator.startup
serializes into STARTUP and what connFramers.initCache turns into framer state. The
connection's protocol version never entered into it, but the extension's whole effect
is version-specific: an EXECUTE then carries a [short bytes] result metadata id and
RESULT/Prepared answers with one.

newFramer accepts v3 through v5 and a non-zero ClusterConfig.ProtoVersion bypasses
discoverProtocol, so `cluster.ProtoVersion = 3` against a Scylla advertising the key
made the driver opt in on a v3 connection and start writing and reading a field v3
never defined. Every doc comment in this series says v4 — ClusterConfig.DisableSkipMetadata,
Conn.tracksResultMetadataID, dialer.Record.UseMetadataID — so the code was broader
than the contract it documented.

Gate it in newScyllaUseMetadataIDExt, which is the one place both halves of the wire
contract are derived from: below v4 the field is undefined, and at v5 the protocol
already carries the id, so Conn.tracksResultMetadataID is true there without the
extension and asking for the backport again would be redundant. The version is
threaded through parseCQLProtocolExtensions rather than checked at the two consumers,
because a gate the STARTUP path and the framer config could apply differently is the
divergence this series took care to remove: announcing the opt-in while the framers
ignore it leaves the server sending ids the driver does not read.

parseSupported keeps setting ScyllaHostFeatures.isMetadataIDSupported unconditionally.
That is "the server advertises this", a host fact that also feeds the isScylla
heuristic, and it stays true whatever version a given connection speaks.

The `f.proto > protoVersion4 || f.scyllaUseMetadataID` conditions in frame.go are
unchanged and still correct; the second disjunct is simply now unreachable below v4.

Tests: the extension is refused on v3 and v5 while still returned for v4 with the
direction bit set, since the gate reads through protoVersionMask as initCache does;
parseCQLProtocolExtensions leaves it out of the list on v3 while a version-independent
extension in the same SUPPORTED map survives; and a v3 Conn taken through the parser
and initFramerCache reports false from usesMetadataID, tracksResultMetadataID, the
framer config and a pooled write framer.

The two integration gates that fail when the server advertises the extension and the
driver did not negotiate it now check the connection is v4 first. That assertion is
deliberate — it is the only end-to-end coverage of the extension, so a negotiation
regression must not be able to skip — but after this change a run pinned to another
protocol version says nothing about negotiation either way, and would fail for doing
the right thing.

Refs: https://scylladb.atlassian.net/browse/DRIVER-152
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
loadFramesFromFile read records with a bare bufio.NewScanner, whose default token
limit is bufio.MaxScanTokenSize — 64 KiB. Record.Data is a []byte, so encoding/json
base64-encodes it and inflates it by 4/3: any recorded frame over ~48 KiB produces a
line the scanner refuses, and the whole recording fails to load with
"bufio.Scanner: token too long".

The cap was always there but out of reach while the recorder wrote one record per
syscall, bounded on the read side by the bufio.Reader the driver fills from. Now that
a record holds one whole frame, a single large response — a page of a SELECT, a
prepared statement with wide metadata — reaches it. The checked-in recordings under
tests/bench top out at 2759 bytes per line, which is why nothing caught this.

Read the lines with a plain bufio.Reader instead. There is no size a fixed cap could
be set to that is right: a frame's length is the peer's to choose, up to the driver's
own maxFrameSize, and ReadBytes grows to the line in front of it and nothing larger.
A record that does not decode is still reported and skipped rather than failing the
file — a recording is a debugging artefact that can be truncated or hand-edited, and
the frames around a damaged one still replay — and a final record with no trailing
newline is still read.

Tests: a 128 KiB frame, asserted to encode past the old cap so the test cannot
quietly stop covering it, loads whole and does not swallow the record after it; and a
file with a damaged middle record still yields the two good ones, the last of them
without a trailing newline. The first fails against the scanner with the error above.

Refs: https://scylladb.atlassian.net/browse/DRIVER-152
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@sylwiaszunejko sylwiaszunejko left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this is a real issue, but worth verifying I guess

Comment thread frame.go
Comment thread dialer/replayer/replayer.go
Comment thread dialer/replayer/replayer.go

@dkropachev dkropachev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. The metadata-ID negotiation, wire framing, cache refresh, and focused coverage are sound. The two deeper replay limitations found during review are pre-existing and explicitly deferred to #990 and #991; neither blocks this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants