Skip to content

fix(gossipsub): avoid quadratic send-queue scan on IDONTWANT - #6661

Merged
mergify[bot] merged 1 commit into
libp2p:masterfrom
jsdanielh:jsdanielh/gossipsub-idontwant-hashset
Oct 3, 2026
Merged

mergify[bot] merged 1 commit into
libp2p:masterfrom
jsdanielh:jsdanielh/gossipsub-idontwant-hashset

Conversation

@jsdanielh

@jsdanielh jsdanielh commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Description

Queue::remove_data_messages, which runs for every received IDONTWANT, checked each queued message against the received ids with message_ids.contains(..). Handling one IDONTWANT therefore cost O(queue × ids). This looks the ids up in a HashSet instead, so the cost is O(queue + ids). Behaviour is unchanged.

The linear search came in with the send-queue rewrite in #6175.

Also adds a test that a received IDONTWANT removes the listed messages from the peer's send queue, which no existing test covered.

AI Assistance Disclosure

Tools used: Claude Code

Attestation:

  • I have read every line of this diff, understand what it does, and can explain it in review.

Notes & open questions

In a local release build, one call with a full queue (5000, the default connection_handler_queue_len) and 5000 ids took ~47 ms before this change and ~0.16 ms after.

Two related changes we can open as follow-ups if you want them:

  • Remove the ids of all IDONTWANT entries in an RPC in one pass. Today each entry walks the whole queue.
  • Limit how many IDONTWANT per peer and heartbeat walk the queue, like go-libp2p's MaxIDontWantMessages. This would add a config option.

Change checklist

  • I have performed a self-review of my own code
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • A changelog entry has been made in the appropriate crates

@jxs jxs left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@jxs jxs added the send-it label Oct 3, 2026
@mergify mergify Bot added the queued label Oct 3, 2026
@mergify

mergify Bot commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • ✅ Entered queue — 2026-10-03 07:46 UTC · Rule: default · triggered by rule Add send-it labeled PRs to merge queue
  • ✅ Checks skipped · PR is already up-to-date
  • ✅ Merged — 2026-10-03 07:46 UTC · at c82d0235abb48308d7ac0751172fc7ddf609204c · squash

This pull request spent 13 seconds in the queue, including 2 seconds running CI.

Required conditions to merge
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-approved [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = Check rustdoc intra-doc links
    • check-neutral = Check rustdoc intra-doc links
    • check-skipped = Check rustdoc intra-doc links
  • any of [🛡 GitHub branch protection]:
    • check-success = rustfmt
    • check-neutral = rustfmt
    • check-skipped = rustfmt
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-metrics
    • check-neutral = Test libp2p-metrics
    • check-skipped = Test libp2p-metrics
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile on x86_64-apple-darwin
    • check-neutral = Compile on x86_64-apple-darwin
    • check-skipped = Compile on x86_64-apple-darwin
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile on x86_64-pc-windows-msvc
    • check-neutral = Compile on x86_64-pc-windows-msvc
    • check-skipped = Compile on x86_64-pc-windows-msvc
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-yamux
    • check-neutral = Test libp2p-yamux
    • check-skipped = Test libp2p-yamux
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-swarm
    • check-neutral = Test libp2p-swarm
    • check-skipped = Test libp2p-swarm
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-mdns
    • check-neutral = Test libp2p-mdns
    • check-skipped = Test libp2p-mdns
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-relay
    • check-neutral = Test libp2p-relay
    • check-skipped = Test libp2p-relay
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-uds
    • check-neutral = Test libp2p-uds
    • check-skipped = Test libp2p-uds
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-rendezvous
    • check-neutral = Test libp2p-rendezvous
    • check-skipped = Test libp2p-rendezvous
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p
    • check-neutral = Test libp2p
    • check-skipped = Test libp2p
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-noise
    • check-neutral = Test libp2p-noise
    • check-skipped = Test libp2p-noise
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-gossipsub
    • check-neutral = Test libp2p-gossipsub
    • check-skipped = Test libp2p-gossipsub
  • any of [🛡 GitHub branch protection]:
    • check-success = Test rw-stream-sink
    • check-neutral = Test rw-stream-sink
    • check-skipped = Test rw-stream-sink
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-dns
    • check-neutral = Test libp2p-dns
    • check-skipped = Test libp2p-dns
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-floodsub
    • check-neutral = Test libp2p-floodsub
    • check-skipped = Test libp2p-floodsub
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-swarm-derive
    • check-neutral = Test libp2p-swarm-derive
    • check-skipped = Test libp2p-swarm-derive
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-pnet
    • check-neutral = Test libp2p-pnet
    • check-skipped = Test libp2p-pnet
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-tls
    • check-neutral = Test libp2p-tls
    • check-skipped = Test libp2p-tls
  • any of [🛡 GitHub branch protection]:
    • check-success = Test multistream-select
    • check-neutral = Test multistream-select
    • check-skipped = Test multistream-select
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-kad
    • check-neutral = Test libp2p-kad
    • check-skipped = Test libp2p-kad
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-core
    • check-neutral = Test libp2p-core
    • check-skipped = Test libp2p-core
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-tcp
    • check-neutral = Test libp2p-tcp
    • check-skipped = Test libp2p-tcp
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-websocket
    • check-neutral = Test libp2p-websocket
    • check-skipped = Test libp2p-websocket
  • any of [🛡 GitHub branch protection]:
    • check-success = IPFS Integration tests
    • check-neutral = IPFS Integration tests
    • check-skipped = IPFS Integration tests
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-identify
    • check-neutral = Test libp2p-identify
    • check-skipped = Test libp2p-identify
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-dcutr
    • check-neutral = Test libp2p-dcutr
    • check-skipped = Test libp2p-dcutr
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile on wasm32-unknown-emscripten
    • check-neutral = Compile on wasm32-unknown-emscripten
    • check-skipped = Compile on wasm32-unknown-emscripten
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile with select features (mdns tcp dns tokio)
    • check-neutral = Compile with select features (mdns tcp dns tokio)
    • check-skipped = Compile with select features (mdns tcp dns tokio)
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-request-response
    • check-neutral = Test libp2p-request-response
    • check-skipped = Test libp2p-request-response
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile on wasm32-unknown-unknown
    • check-neutral = Compile on wasm32-unknown-unknown
    • check-skipped = Compile on wasm32-unknown-unknown
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-mplex
    • check-neutral = Test libp2p-mplex
    • check-skipped = Test libp2p-mplex
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-ping
    • check-neutral = Test libp2p-ping
    • check-skipped = Test libp2p-ping
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-plaintext
    • check-neutral = Test libp2p-plaintext
    • check-skipped = Test libp2p-plaintext
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-autonat
    • check-neutral = Test libp2p-autonat
    • check-skipped = Test libp2p-autonat
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-quic
    • check-neutral = Test libp2p-quic
    • check-skipped = Test libp2p-quic
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-webrtc
    • check-neutral = Test libp2p-webrtc
    • check-skipped = Test libp2p-webrtc
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-identity
    • check-neutral = Test libp2p-identity
    • check-skipped = Test libp2p-identity
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-allow-block-list
    • check-neutral = Test libp2p-allow-block-list
    • check-skipped = Test libp2p-allow-block-list
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-swarm-test
    • check-neutral = Test libp2p-swarm-test
    • check-skipped = Test libp2p-swarm-test
  • any of [🛡 GitHub branch protection]:
    • check-success = Check for changes in proto files
    • check-neutral = Check for changes in proto files
    • check-skipped = Check for changes in proto files
  • any of [🛡 GitHub branch protection]:
    • check-success = Ensure that `Cargo.lock` is up-to-date
    • check-neutral = Ensure that `Cargo.lock` is up-to-date
    • check-skipped = Ensure that `Cargo.lock` is up-to-date
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-connection-limits
    • check-neutral = Test libp2p-connection-limits
    • check-skipped = Test libp2p-connection-limits
  • any of [🛡 GitHub branch protection]:
    • check-success = examples
    • check-neutral = examples
    • check-skipped = examples
  • any of [🛡 GitHub branch protection]:
    • check-success = Run all WASM tests
    • check-neutral = Run all WASM tests
    • check-skipped = Run all WASM tests
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-memory-connection-limits
    • check-neutral = Test libp2p-memory-connection-limits
    • check-skipped = Test libp2p-memory-connection-limits
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-webtransport-websys
    • check-neutral = Test libp2p-webtransport-websys
    • check-skipped = Test libp2p-webtransport-websys
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-perf
    • check-neutral = Test libp2p-perf
    • check-skipped = Test libp2p-perf
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile with MSRV
    • check-neutral = Compile with MSRV
    • check-skipped = Compile with MSRV
  • any of [🛡 GitHub branch protection]:
    • check-success = manifest_lint
    • check-neutral = manifest_lint
    • check-skipped = manifest_lint
  • any of [🛡 GitHub branch protection]:
    • check-success = Test libp2p-server
    • check-neutral = Test libp2p-server
    • check-skipped = Test libp2p-server
  • any of [🛡 GitHub branch protection]:
    • check-success = Compile on wasm32-wasip1
    • check-neutral = Compile on wasm32-wasip1
    • check-skipped = Compile on wasm32-wasip1

@mergify
mergify Bot merged commit c82d023 into libp2p:master Oct 3, 2026
68 of 71 checks passed
@mergify mergify Bot removed the queued label Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants