GPU user annotations (GpuUserAnnotationEvent) - #6887
Conversation
Add a GpuUserAnnotationEvent packet (TracePacket field 136) describing a user-defined, named time slice on the GPU timeline. It carries an event_id for correlating host and GPU work, a duration, a name, a gpu_id and graphics context, an optional hw_queue_iid, and a repeated ExtraArg for typed key/value arguments. ExtraArg interns argument names via DebugAnnotationName and string values via debug_annotation_string_values, so no new interned data tables are needed. This adds only the proto definition and its TracePacket wiring; the trace_processor parser and stdlib support follow separately.
Include the gpu_user_annotation track type in the gpu_slice view's type filter so GpuUserAnnotationEvent slices appear alongside render stage, Vulkan and GPU log slices.
Add GpuEventParser::ParseGpuUserAnnotation, turning a GpuUserAnnotationEvent packet into a slice on a gpu_user_annotation track keyed by (gpu, upid). The graphics context is resolved to a process, and the annotation's ExtraArg entries are flattened into slice args with names interned via debug_annotation_names and string values via debug_annotation_string_values. When the event carries an event_id, the slice is registered under it so host track events can form a flow to the annotation.
…rack When a GpuUserAnnotationEvent carries a hw_queue_iid, intern it onto that hardware queue's render stage track (reusing the render stage blueprint) so it appears alongside the events on the queue. Without a resolvable specification it falls back to the stream-scoped gpu_user_annotation track.
|
So stupid question: why is this not TrackEvent + GPU scope? I'm struggling to understand why we should separate it from that, you will miss out all the features you get from TrackEvent... |
🎨 Perfetto UI Builds
|
@LalitMaganti Good question, honestly I'm not 100% sure TrackEvent + a GPU scope couldn't work. But a couple of specifics that feel relevant (though I'm not certain they fully motivate a separate message):
So this is basically the same reasoning as why GpuRenderStageEvent isn't a TrackEvent, and it keeps annotations on the same GPU tracks as render stages. Which raises the flip side: if GPU annotation events should be a TrackEvent extension, shouldn't render stage events be too? They share these properties, so it seems like they'd want to go the same way. The usual TrackEvent niceties (name interning, typed args, host correlation) are reused here via the same infra rather than lost. |
You can just emit each one on a different track though for this. A downside of this is an extra uint64 per event which we could hypothetically get rid of but it complicates the track event model so it's never been worth doing.
To me the reason is render stage events have a very well defined semantic across multiple operating systems, drivers and OEMs so it's worth modelling that as a concept explicitly. I'm not convinced gpu annotation events meet that mark In general this is how it goes for using track event vs custom proto: use track event if you are doing something which doesn't have specific semantics by itself and so it wouldn't be a big deal to use track event. Use custom protos where there is already a strong pre-existing shape and you want to simply model that in the proto. There's no hard line here, it's just a judgement call on a case by case basis. |
That's the strategy I've taken so far (using fake hw queues for annotations) but is confusing as with json traces and creating full hierarchies of processes/threads that represent GPU work these annotations are nicely nested at the top of a gpu queue track with kernels under. It doesn't have to be visualized exactly like this but I'm convinced that we need to do better than putting these on separate tracks.
Yes, the questioning this makes sense. I'll give the the track event route a try and use the gpu.track_event extension instead to see what that looks like. |
But we have track merging in the trackevent format which lets you describe how you want independent tracks in the trace to be merged together into a single UI track. I would be curious to understand the problem you're facing with this, it might help expose some missing feature in trackevent. |
#6923 is an alternative solution that introduces a GpuTrackDescriptor. A lot more powerful and allows us to more properly encode and visualize the logical queue concept for GPU workloads without relying on arbitrary render stage arguments and hardcoding GPU API concepts in the GpuByProcess plugin. |
|
Closing in favor of #6923 |
2 participants
Adds a
GpuUserAnnotationEventTracePacketmessage (field 136) foruser-defined, named spans on the GPU timeline, parsed in trace_processor as
ordinary GPU slices. Any GPU workload that emits
GpuRenderStageEvents can nowwrap or label that work with named annotations without overloading the
render-stage message.
It's a new message rather than a reuse of
GpuRenderStageEvent, but its encodingmirrors render stages so annotations behave like any other GPU slice, with no UI
special-casing:
gpu_id+contextidentify the GPU and graphics context (resolved to aprocess), as render stages do.
hw_queue_iid(optional): when set, the annotation is placed on that hardwarequeue's render-stage track; otherwise on a
gpu_user_annotationtrack keyed by(gpu, process). Either way it joins thegpu_sliceview.event_id(optional): correlates the annotation with host work, using the samemechanism as
GpuRenderStageEvent.repeated ExtraArg args: typed key/value arguments; names intern viaDebugAnnotationNameand string values viadebug_annotation_string_values,so no new interned-data tables are added.