fix(nemesis): bound reliable RF by rack count under rf_rack_valid_keyspaces (SCT-717) - #15727
Draft
scylla-zeus-bot wants to merge 1 commit into
Draft
fix(nemesis): bound reliable RF by rack count under rf_rack_valid_keyspaces (SCT-717)#15727scylla-zeus-bot wants to merge 1 commit into
scylla-zeus-bot wants to merge 1 commit into
Conversation
…spaces (SCT-717) 'rf_rack_valid_keyspaces' in scylla.yaml makes Scylla reject CREATE KEYSPACE/ALTER KEYSPACE statements whose RF exceeds the DC's rack count. When both tablets and 'rf_rack_valid_keyspaces' are enabled for a run, nemeses computing an RF via 'reliable_replication_factor' (and the truncate-large-partition nemesis, which hardcoded RF=3) could still request an RF above the rack count and get rejected. Add ClusterTester._is_rf_rack_valid_keyspaces_enabled(), mirroring the existing 'force_gossip_topology_changes' idiom in cluster.py, and use it to additionally cap 'reliable_replication_factor' by 'max(db_cluster.racks_count, 1)' -- but only when tablets are enabled AND the gate is on. Non-tablets runs and tablets runs without 'rf_rack_valid_keyspaces' keep today's RF selection unchanged. Wire disrupt_truncate_large_partition's scylla-bench RF to the same property instead of a hardcoded RF=3, so it benefits from the same bound. Tests: unit_tests/unit/test_tester.py -- 47 passed, including new test_reliable_replication_factor (7 cases covering tablets x rf_rack_valid_keyspaces x racks_count x n_db_nodes) and test_is_rf_rack_valid_keyspaces_enabled (5 cases over append_scylla_yaml variants).
Contributor
how? and why? |
Contributor
In nemesis we have multiple calls to c-s, that need to assume what RF to set for the keyspace. It was adjusted for tablets and RF=rack is some why, but on a test case for RF=1, where we set only one rack, it breaks, see the Jira. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes SCT-717: SCT nemeses (RefreshMonkey, TruncateMonkey, RefreshBigMonkey, NoCorruptRepairMonkey, LoadAndStreamMonkey, TruncateLargeParititionMonkey) were creating test keyspaces with a replication factor that could exceed the cluster's rack count, causing Scylla to reject CREATE KEYSPACE with 'Replication factor N exceeds the number of racks (1)' whenever rf_rack_valid_keyspaces: true is set (which is the repo default).
Tester.reliable_replication_factor now also bounds RF by db_cluster.racks_count, but only when both:
For every other run (non-tablets, or rf_rack_valid_keyspaces disabled/absent), RF selection is byte-for-byte unchanged.
disrupt_truncate_large_partition also stops hardcoding -replication-factor=3 for its scylla-bench command and now uses self.tester.reliable_replication_factor, so it inherits the same gating automatically.
Changes
Verification
Process notes
Ran plan -> implementer -> reviewer + pragmatist. Both review passes returned only minor/non-blocking notes (a latent, pre-existing-pattern edge case if append_scylla_yaml were ever a non-dict, and a stylistic f-string nit) -- no changes were requested back.
Closes SCT-717.
Note
This draft PR was generated by Maia. The root-cause assessment is AI-generated and may be wrong; it requires human review and is never auto-merged.