Skip to content

mcs: reject registration reusing another instance's address - #11043

Open
bufferflies wants to merge 1 commit into
tikv:masterfrom
bufferflies:mcs-register-validation
Open

mcs: reject registration reusing another instance's address#11043
bufferflies wants to merge 1 commit into
tikv:masterfrom
bufferflies:mcs-register-validation

Conversation

@bufferflies

@bufferflies bufferflies commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: Close #11001, Close #10998

A microservice instance (TSO / Scheduling) started with an --advertise-listen-addr already used by another live instance could blindly overwrite that instance's etcd registry entry and then join the primary election with the same participant identity.

What is changed and how does it work?

Validate the registry key at registration time in pkg/mcs/discovery:

  • putWithTTL now claims the registry key with an atomic create-if-absent etcd transaction instead of an unconditional Put, so a duplicate advertised address can no longer overwrite another live instance's entry.
  • If the key already exists with this instance's own value (e.g. re-registering after a keepalive failure while the previous lease is still alive), it is taken over with the new lease via a value-guarded transaction; otherwise the registration is rejected with a clear error and the granted lease is revoked.
  • Register retries the claim within the lease TTL window, so a stale entry left by a crashed instance with the same address expires and the restart succeeds without manual intervention, while a genuine duplicate live instance keeps its lease refreshed and the orphan fails startup before ever entering the primary election (all MCS servers call Register before startServer).

Check List

Tests

  • Unit test: TestRegisterConflict verifies a duplicate live registration is rejected without overwriting the existing entry, and that a new instance can register after the stale lease expires.
  • Integration test: tests/integrations/mcs/discovery passes.

Release note

Reject microservice registration when the advertised address is already claimed by another live instance, preventing registry overwrite and unintended primary election takeover.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Bug Fixes
    • Improved service registration when another live instance is already using the same address.
    • Prevented duplicate registrations from replacing an active service entry.
    • Automatically retries registration until the existing lease expires or the operation is canceled.
    • Allows registration to succeed after a stale service entry is removed.

Claim the service registry key with an atomic create-if-absent
transaction so that an instance advertising a duplicate address
cannot overwrite the registry entry of another live instance and
further join the primary election with the same identity. A stale
entry left by a crashed instance expires with its lease, so the
registration retries within the lease TTL before giving up.

Signed-off-by: tongjian <1045931706@qq.com>
@ti-chi-bot ti-chi-bot Bot added dco-signoff: yes Indicates the PR's author has signed the dco. release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/needs-triage-completed labels Jul 23, 2026
@ti-chi-bot

ti-chi-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign overvenus for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 23, 2026
@coderabbitai

coderabbitai Bot commented Jul 23, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Service registration now uses lease-backed conditional etcd transactions, rejects live duplicate addresses, retries occupied registrations within a TTL-based deadline, and revokes failed leases. Tests cover conflict rejection, lease expiry, and subsequent successful registration.

Changes

Service registration conflict handling

Layer / File(s) Summary
Registration retry loop
pkg/mcs/discovery/register.go
Adds an occupied-address sentinel and retries registration until it succeeds, encounters another error, is canceled, or reaches the TTL-based deadline.
Lease-backed claim validation
pkg/mcs/discovery/register.go, pkg/mcs/discovery/register_test.go
Uses conditional etcd transactions with lease revocation on failure, supports same-value takeover, and tests duplicate registration rejection followed by registration after lease expiry.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ServiceRegister
  participant Etcd
  participant ServiceContext
  ServiceRegister->>Etcd: Grant lease and conditionally claim registry key
  Etcd-->>ServiceRegister: Return success or occupied error
  ServiceRegister->>ServiceContext: Wait for retry interval or cancellation
  ServiceRegister->>Etcd: Retry claim before TTL deadline
  Etcd-->>ServiceRegister: Register after previous lease expires
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: rejecting duplicate registration reusing another instance's address.
Description check ✅ Passed The description includes the required issue references, change summary, checklist, and release note.
Linked Issues check ✅ Passed The PR implements the requested uniqueness checks and stale-entry retry behavior for conflicting live registrations.
Out of Scope Changes check ✅ Passed The added retry logic, lease handling, and tests are all aligned with the stated registration-fix objectives.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot

ti-chi-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

@bufferflies: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-unit-test-next-gen-2 18f200d link true /test pull-unit-test-next-gen-2

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/mcs/discovery/register_test.go (1)

109-111: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Prefer errors.Is over substring matching for the sentinel error check.

Since this is an internal test (same package), asserting errors.Is(err, errServiceAddrOccupied) directly ties the test to the actual sentinel error instead of its message wording, which is more robust against future message-copy changes.

♻️ Proposed refactor
 	err := sr2.Register()
 	re.Error(err)
-	re.Contains(err.Error(), "occupied")
+	re.ErrorIs(err, errServiceAddrOccupied)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/mcs/discovery/register_test.go` around lines 109 - 111, Update the
Register test’s error assertion after sr2.Register() to use errors.Is with the
errServiceAddrOccupied sentinel instead of checking for the "occupied" message
substring, while retaining the existing error assertion.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/mcs/discovery/register_test.go`:
- Around line 109-111: Update the Register test’s error assertion after
sr2.Register() to use errors.Is with the errServiceAddrOccupied sentinel instead
of checking for the "occupied" message substring, while retaining the existing
error assertion.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b278b836-d3c2-4210-9736-a876500f995b

📥 Commits

Reviewing files that changed from the base of the PR and between f7db425 and 18f200d.

📒 Files selected for processing (2)
  • pkg/mcs/discovery/register.go
  • pkg/mcs/discovery/register_test.go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the dco. do-not-merge/needs-triage-completed release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

1 participant