xds: suppress the duplicate errors at the client side by mswierq · Pull Request #9185 · grpc/grpc-go

mswierq · 2026-06-15T12:24:08Z

This change suppresses duplicate errors on the xds client side by skipping calling watchers for the duplicates. This also prevents generating new child names for the same broken EDS resource (EDS resource has no localities) that is being re-send by the control plane.

RELEASE NOTES:

xds: handle receipt of duplicated errors at the client side by suppressing them in the same manner the duplicate updates are suppressed

codecov · 2026-06-15T12:27:29Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.22%. Comparing base (1a80fca) to head (bc6a797).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9185      +/-   ##
==========================================
- Coverage   83.23%   83.22%   -0.02%     
==========================================
  Files         420      420              
  Lines       34024    34035      +11     
==========================================
+ Hits        28321    28325       +4     
+ Misses       4275     4274       -1     
- Partials     1428     1436       +8

Files with missing lines	Coverage Δ
internal/xds/clients/xdsclient/authority.go	`79.90% <100.00%> (-0.90%)`	⬇️

... and 23 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

eshitachandwani · 2026-06-22T09:19:30Z

+			isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == uErr.Err.Error()
+			var errState *xdsresource.UpdateErrorMetadata
+			if md.ErrState != nil {
+				errState = &xdsresource.UpdateErrorMetadata{


Why do we need to do this ? Can't we directly do state.md.ErrState = md.ErrState like earlier ?

md.ErrState contains concatenated errors for all resources in the received update, so it's impossible to detect an error duplication right, if there was more than one error. It might be worth it to keep uErr.Err (which is a resource specific error) as a separate member in the state to avoid confusion? WDYT?

I dont think that is the right way to go. We should not change the error stored. We can instead compare the error in state with the ms error like :
isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == md.ErrState.Err.Error()
WDYT ?

We cannot compare state.md.ErrState.Err.Error() == md.ErrState.Err.Error() in order to detect a duplication, e.g. Let's assume that we got an update for three resources, where two of them got errors:

Resource A (Ok)

Resource B (Error B)

Resource C (Error C)

md.ErrState.Err.Error() returns a concatenated error "Error B; Error C". This concatenated error is stored in state.md.ErrState.

Then the next update brings a duplicate error for the Resource C only, because Resource B is fine, we did not get an update for that resource or the error is different:

Resource A (Ok)

Resource B (Ok)

Resource C (Error C)

md.ErrState.Err.Error() returns a concatenated error "Error C", which is different from the previously stored one "Error B; Error C". Even though the duplication is there a simple comparison will not work.

As an alternative I can try to check if the concatenated error in the md.ErrState contains a resource specific error to seek for a duplication.

I have changed the logic to look for a duplicate resource error in the concatenated error (md.ErrState)

eshitachandwani · 2026-06-22T09:21:14Z

 				})
 			}
-			state.md.ErrState = md.ErrState
+			isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == uErr.Err.Error()


Also instead of evaluating this as a variable and then checking can we instead do the assignment first(which I assume needs to be done for either case) and then directly in an if block we can evaluate the condition and call continue on true. WDYT ?

state works as an cache of the previously received error for the resource, so trying to find out if there was an error duplication after updating state would always return true

Right! but we can do the check first and if true call continue, because if we are getting the same error, we might not need to update the state? Or can the status be different ?

The stored metadata ErrState may differ, this is why we need to update it even if a duplicate error for a resource is detected. This is also the reason why the duplicate check has to be done before updating it. Please, look at the comment with an example that I made: #9185 (comment)

eshitachandwani · 2026-06-22T09:40:22Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces duplicate error suppression for ADS resource updates in the xDS client, skipping watcher notifications when duplicate errors are received, and adds a corresponding test to verify this behavior. The reviewer identified a critical issue in authority.go where a nil pointer dereference panic could occur if md.ErrState is nil but uErr.Err is non-nil, and provided a code suggestion to safely initialize errState.

…uplicate-errors

eshitachandwani · 2026-07-08T15:14:30Z

+// duplicate error notification to the watcher.
+//
+// Note: This behavior (suppress duplicate error notifications to watchers) was
+// originally introduced to handle invalid EDS resources (missing localities) as


I don't think we need this note here.

eshitachandwani · 2026-07-08T15:17:41Z

+			}
+			if req.GetErrorDetail() != nil {
+				if count := nackCount.Add(1); count >= 3 {
+					nackReceivedCh.Send(nil)


nit: Can we change the nackReceivedCh to be a go channel and close it here since it is only sent to once. We do not need a size one channel.

eshitachandwani · 2026-07-08T15:44:42Z

+			isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == uErr.Err.Error()
+			var errState *xdsresource.UpdateErrorMetadata
+			if md.ErrState != nil {
+				errState = &xdsresource.UpdateErrorMetadata{


I dont think that is the right way to go. We should not change the error stored. We can instead compare the error in state with the ms error like :
isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == md.ErrState.Err.Error()
WDYT ?

eshitachandwani · 2026-07-08T15:47:51Z

 				})
 			}
-			state.md.ErrState = md.ErrState
+			isDuplicateErr := state.md.ErrState != nil && state.md.ErrState.Err != nil && state.md.ErrState.Err.Error() == uErr.Err.Error()


Right! but we can do the check first and if true call continue, because if we are getting the same error, we might not need to update the state? Or can the status be different ?

…resource error

Suppress the duplicate errors at the client side

35d57a6

mbissa self-assigned this Jun 16, 2026

easwars requested a review from eshitachandwani June 17, 2026 18:20

easwars assigned eshitachandwani Jun 17, 2026

easwars added Type: Bug Area: xDS Includes everything xDS related, including LB policies used with xDS. labels Jun 17, 2026

easwars added this to the 1.83 Release milestone Jun 17, 2026

eshitachandwani reviewed Jun 22, 2026

View reviewed changes

eshitachandwani modified the milestone: 1.83 Release Jun 22, 2026

gemini-code-assist Bot reviewed Jun 22, 2026

View reviewed changes

Comment thread internal/xds/clients/xdsclient/authority.go Outdated

eshitachandwani assigned mswierq and unassigned eshitachandwani Jun 22, 2026

mswierq added 3 commits July 6, 2026 07:28

Merge remote-tracking branch 'origin/master' into fix/8994-suppress-d…

0e3ce43

…uplicate-errors

Move the test case to the low-level client tests

82cbfbb

Fix go fmt

0306ac0

easwars assigned eshitachandwani and unassigned mswierq and mbissa Jul 8, 2026

eshitachandwani self-requested a review July 8, 2026 15:12

eshitachandwani reviewed Jul 8, 2026

View reviewed changes

eshitachandwani assigned mswierq and unassigned eshitachandwani Jul 8, 2026

mswierq added 2 commits July 9, 2026 08:57

Check if the metadata concatenated error contains a duplicate of the …

9db08e2

…resource error

Address feedback for the test case

bc6a797

eshitachandwani assigned eshitachandwani and unassigned mswierq Jul 10, 2026

easwars removed this from the 1.83 Release milestone Jul 14, 2026

easwars added this to the 1.84 Release milestone Jul 14, 2026

Uh oh!

Conversation

mswierq commented Jun 15, 2026

Uh oh!

codecov Bot commented Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

eshitachandwani commented Jun 22, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Jun 15, 2026 •

edited

Loading