OTEL_PYTHON_FASTAPI_EXCLUDED_URLS doesn't suppress child spans from being created

[bbeattie-phxlabs]

Consider the scenario:

• An environment in which opentelemetry-instrumentation-aiohttp-client and opentelemetry-instrumentation-fastapi are installed
• A FastAPI endpoint that makes a call via aiohttp
• This generates a span for the FastAPI endpoint and a span for the aiohttp call.

Now alter the environment such that OTEL_PYTHON_FASTAPI_EXCLUDED_URLS excludes the endpoint. Surprisingly, the aiohttp still generates a span!

I see conversation around open-telemetry/opentelemetry-specification#530 on the subject of "stop tracing at this point", but that issue has sat open for 3+ years now.

Is there a better way to tackle this?

[Mr0grog]

I am hitting a similar issue (in my case, Django + psycopg). The best solution I can think of there is to create a valid, non-recording span for the excluded URLs instead of creating no span at all (what everything here currently does). Then, child spans would inherit that non-recording context and also not be recorded.

You could implement your own middleware to do this, thought it feels like more work than should be necessary. This seems to work for me in Django, though you’d have to adjust for FastAPI or ASGI or whatever middleware format you need (and don’t forget to install it 😉):

from opentelemetry.util.http import get_excluded_urls
from opentelemetry.trace import NonRecordingSpan, Span, SpanContext, TraceFlags, TraceState, get_tracer, use_span


class NonRecordingDjangoMiddleware:
    """
    Django middleware that adds non-recording Open Telemetry spans when URLs
    specified in the ``OTEL_PYTHON_DJANGO_EXCLUDED_URLS`` or
    ``OTEL_PYTHON_EXCLUDED_URLS`` environment variables are requested.

    This ensures all requests have a span (the official OpenTelemetry Django
    instrumentation will create spans for non-excluded URLs). If other
    instrumentations create spans, they'll wind up as a child of these
    non-recording spans and thus also not get recorded. Without this, those
    other instrumentations wind up creating and sending spans of their own when
    you reqeust an excluded URL, so you haven't actually excluded the URL so
    much as created a mess of confusing low-level spans that show up as
    root-level spans.

    See also: https://github.com/open-telemetry/opentelemetry-python-contrib/discussions/1984
    """

    _environ_activation_key = 'nonrecording-django-middleware.activation_key'
    _excluded_urls = get_excluded_urls('DJANGO')

    @staticmethod
    def create_nonrecording_span() -> Span:
        tracer = get_tracer(__name__)

        # We have to create a new context with a valid trace_id and span_id,
        # otherwise it won't get treated as a parent span to inherit from.
        # The default `trace_flags` is what makes it non-recording.
        nonrecording_context = SpanContext(
            trace_id=tracer.id_generator.generate_trace_id(),
            span_id=tracer.id_generator.generate_span_id(),
            is_remote=False,
            trace_flags=TraceFlags(TraceFlags.DEFAULT),
            trace_state=TraceState.get_default(),
        )
        return NonRecordingSpan(nonrecording_context)

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        self.process_request(request)
        response = self.get_response(request)
        return self.process_response(request, response)

    def process_request(self, request):
        if self._excluded_urls.url_disabled(request.build_absolute_uri('?')):
            span = self.create_nonrecording_span()
            activation = use_span(span, end_on_exit=True)
            activation.__enter__()
            request.META[self._environ_activation_key] = activation

    def process_response(self, request, response):
        activation = request.META.pop(self._environ_activation_key, None)
        if activation:
            activation.__exit__(None, None, None)

        return response