fix(tracing): report spans that outlive their sent parent

[solnic]

A span that outlives its transaction root (ie async work via Tasks/Broadway/Oban continuing a trace after the root was reported) was either corrupted or lost:

• If it was still running when the root ended, it was serialized into the root transaction with "timestamp": null — Relay patches that into a zero-duration span with a bogus deadline_exceeded status (first screenshot; the span actually ran for 500ms and succeeded).
• Once it finished, it was silently dropped.

Unfinished children are now excluded from the transaction payload, and sent transaction roots leave a short-lived marker in span storage. A span ending with a marked parent is promoted to a follow-up transaction in the same trace - same trace_id, parent_span_id pointing at the sent root, tagged sentry.parent_span_already_sent: true.

Before

After

---

Fixes #1011

[cursor]

Finished grandchildren reported in two transactions

Medium Severity

get_child_spans returns all descendants, so a finished grandchild whose direct parent is still in flight is included in the root transaction even though its parent is filtered out, leaving it orphaned. That record is not removed (removal only touches direct children), so when the in-flight parent later ends and is promoted to a follow-up transaction, the same span is sent a second time.

 

^{Reviewed by Cursor Bugbot for commit b822b9b. Configure here.}

[cursor]

Deeper orphaned spans still silently dropped

Low Severity

Only transaction roots get a sent marker, so promotion works one level deep. A span whose direct parent was an ordinary child of the sent root finds neither a marker nor a stored parent when it ends, and falls through to the no-op branch. Such spans were previously emitted (with a null timestamp) inside the root transaction and are now lost entirely.

 

^{Reviewed by Cursor Bugbot for commit b822b9b. Configure here.}

[sentry]

Bug: Grandchild spans can be duplicated across transactions if their parent span outlives the root span, because the cleanup logic in remove_child_spans is not recursive.
_{Severity: MEDIUM}

Suggested Fix

Update remove_child_spans to recursively remove all descendant spans, not just direct children. This would ensure that when a root transaction is processed, all its finished descendants are cleaned up from storage, preventing them from being included in a subsequent follow-up transaction.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: lib/sentry/opentelemetry/span_processor.ex#L104-L108

Potential issue: In a nested span hierarchy (e.g., root `R`, child `P`, grandchild `G`),
if the grandchild `G` finishes but its parent `P` outlives the root `R`, a duplication
issue occurs. When `R` finishes, `get_child_spans` recursively finds and includes `G` in
the root transaction. However, the cleanup function `remove_child_spans` only removes
direct children, leaving `G` in storage. Later, when `P` finishes and is promoted to a
new transaction, `get_child_spans` is called for `P`, which finds and includes `G` a
second time. This results in the grandchild span appearing in both the root transaction
and the follow-up transaction.

Also affects:

• lib/sentry/opentelemetry/span_storage.ex:230~243

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 3 total unresolved issues (including 2 from previous reviews).

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 203b3c5. Configure here.}

[cursor]

Nested late spans lose promotion path

Medium Severity

Promotion only triggers when the immediate parent carries a sent marker. If that parent finished before the root, it gets deleted from storage when the root transaction is sent and is never marked, so a still-running grandchild ends up with neither a stored nor a marked parent and is silently discarded once it finishes — previously it was at least present in the root payload.

 

^{Reviewed by Cursor Bugbot for commit 203b3c5. Configure here.}