test: analyzer bot test PR - #1
Open
Ariznawlll wants to merge 1 commit into
Open
Conversation
Owner
Author
|
/analyze-pr |
mergify Bot
pushed a commit
that referenced
this pull request
May 7, 2026
…origin#24117) Move postEvict callbacks (value.Release + metrics updates) outside the global `queueLock` in the FIFO cache to eliminate lock convoy under memory pressure. ### Root Cause When MemCache (12GB FIFO cache) is 100% full, every `Set()` triggers `Evict()` which holds the global `queueLock` while executing `postEvict` callbacks. Under GC pressure (STW pauses up to ~1s), this creates a lock convoy where all concurrent cache operations serialize through the single lock: - Each `Set()` takes **300-444ms** (normal: <0.1ms, 3000-4400x slower) - Queries with ~90 cache ops inflate from 1-5s to **60-107s** - Exceeds client-side timeouts → **connection disconnections** Evidence from fileservice slow event trace: ``` Single S3FS.Read (total: 798ms) breakdown: disk cache read: 35ms set memory cache entry #1: 318ms ← queueLock wait + eviction set memory cache entry #2: 444ms ← queueLock wait + eviction 762ms (95%) spent in "set memory cache entry" ``` ### Fix 1. Collect evicted items under `queueLock` into a pending list 2. Release `queueLock` 3. Execute `postEvict` callbacks (value.Release + metrics) outside the lock The `item.valueOK` flag is still set to `false` under the **shard lock** (not queueLock) to prevent data races with concurrent `Get()` calls. ### Impact In stability testing (commit 725b723): - **TPCC**: 40 "Communications link failure" events - **Fulltext**: 57 "Lost connection" events - **Sysbench**: 0 errors (point queries <5s skip tombstone transfer) Approved by: @gouhongshen, @XuPeng-SH, @fengttt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Testing the PR analyzer bot