Skip to content

stats/opentelemetry: add e2e test for baggage propagation#9160

Open
ulascansenturk wants to merge 2 commits into
grpc:masterfrom
ulascansenturk:test/otel-baggage-propagation
Open

stats/opentelemetry: add e2e test for baggage propagation#9160
ulascansenturk wants to merge 2 commits into
grpc:masterfrom
ulascansenturk:test/otel-baggage-propagation

Conversation

@ulascansenturk

@ulascansenturk ulascansenturk commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #8520.

OpenTelemetry baggage already propagates correctly through the gRPC pipeline when the W3C Baggage propagator is configured — the tracing carriers (IncomingCarrier/OutgoingCarrier) are generic TextMapCarriers used by the configured propagator's Inject/Extract, so baggage rides in the baggage metadata header exactly like trace context. No production code change is needed; the gap was the lack of a test proving it.

This PR adds TestRPC_BaggagePropagation, which:

  • Sets baggage (key1=value1, key2=value2) on the client context.
  • Configures propagation.Baggage{} in the trace options.
  • Asserts the server handler observes the same baggage, for both a unary and a streaming RPC.

Verification

  • Passes with -race.
  • Confirmed meaningful: removing propagation.Baggage{} makes the test fail (server sees empty baggage), so it exercises propagation rather than passing vacuously.
  • go vet and gofmt clean.

RELEASE NOTES: none

@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.17%. Comparing base (adc97de) to head (8e96a8f).
⚠️ Report is 22 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9160      +/-   ##
==========================================
- Coverage   83.20%   83.17%   -0.03%     
==========================================
  Files         418      419       +1     
  Lines       33709    33824     +115     
==========================================
+ Hits        28048    28134      +86     
- Misses       4245     4264      +19     
- Partials     1416     1426      +10     

see 37 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

OpenTelemetry baggage already propagates through the gRPC pipeline when
the W3C Baggage propagator is configured, since the tracing carriers are
generic TextMapCarriers used for Inject/Extract. There was, however, no
test verifying this behavior.

Add TestRPC_BaggagePropagation which sets baggage on the client context
and asserts the server handler observes it for both unary and streaming
RPCs.

Fixes grpc#8520
@ulascansenturk ulascansenturk force-pushed the test/otel-baggage-propagation branch from b496c54 to d3232f3 Compare June 2, 2026 12:53
@easwars easwars requested a review from mbissa June 3, 2026 03:22
@easwars easwars added Type: Feature New features or improvements in behavior Area: Observability Includes Stats, Tracing, Channelz, Healthz, Binlog, Reflection, Admin, GCP Observability labels Jun 3, 2026
@easwars easwars added this to the 1.82 Release milestone Jun 3, 2026
@mbissa mbissa modified the milestones: 1.82 Release, 1.83 Release Jun 5, 2026
@mbissa

mbissa commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

/gemini review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new end-to-end test, TestRPC_BaggagePropagation, to verify that OpenTelemetry baggage is correctly propagated from the client to the server across both unary and streaming RPCs. Feedback on the changes points out a critical issue in the streaming RPC handler (FullDuplexCallF), where a non-EOF error returned by stream.Recv() would result in an infinite loop and high CPU usage. It is recommended to update the loop to handle and return any non-nil errors.

Comment on lines +2514 to +2518
for {
if _, err := stream.Recv(); err == io.EOF {
return nil
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If stream.Recv() returns an error other than io.EOF (such as a canceled context or a transport error), the loop will continue infinitely because there is no exit condition for other errors. This can cause a tight loop spinning at 100% CPU. We should return the error if it is not nil.

			for {
				if _, err := stream.Recv(); err != nil {
					if err == io.EOF {
						return nil
					}
					return err
				}
			}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed: the loop now returns the error when it isn't io.EOF, so a canceled context or transport error no longer spins at 100% CPU.

In TestRPC_BaggagePropagation, the FullDuplexCall server handler looped on
stream.Recv() and only exited on io.EOF, so any other error (canceled
context, transport error) would spin the loop at 100% CPU. Return the
error when it is not io.EOF.
@mbissa

mbissa commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Thanks @ulascansenturk for this contribution. Assigning to @easwars for second set of eyes.

@mbissa mbissa assigned easwars and unassigned mbissa Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Observability Includes Stats, Tracing, Channelz, Healthz, Binlog, Reflection, Admin, GCP Observability Type: Feature New features or improvements in behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add tests for OTel Baggage in metrics/traces

3 participants