Repository navigation
oss_14_processor_export_lock_timeout #12
Description
Activity
🔒 Internal (dash0) — not for upstream.
Tracking for the span-processor export serialization + timeout fix.
- Linear: https://linear.app/dash0/issue/OSS-14/t2-t3-simplespanprocessor-concurrent-export-bsp-export-timeout-not (findings T2 + T3)
- Slug / branch / worktree:
oss_14_processor_export_lock_timeout - Fix commits:
1900fc9d8(T2+T3),07255d44f(suppression-context fix caught in validation) - PR: oss_14_processor_export_lock_timeout #15
- Validation: scope contained to
opentelemetry-sdk+ changelog; full SDK suite 834 passed (incl. new suppression regression test).
📣 Public-facing draft — to be opened upstream in
open-telemetry/opentelemetry-python. No internal references; copy verbatim.Title:
SimpleSpanProcessorexports concurrently, andBatchSpanProcessorexport timeout is never enforcedTwo related span-processor conformance bugs in
opentelemetry-sdk/.../trace/export/__init__.pyand the shared batch processor.T2 —
SimpleSpanProcessor.export()is not serializedSimpleSpanProcessor.on_endcallsself.span_exporter.export((span,))with no lock. Two threads ending sampled spans can therefore callexporton the same exporter concurrently. The spec (specification/trace/sdk.md) states that Export MUST NOT be called concurrently for the same exporter —BatchSpanProcessoralready serializes via an export lock, butSimpleSpanProcessordoes not.
Fix: guard theexportcall with a per-processor lock.T3 —
BatchSpanProcessorexport timeout is never enforcedOTEL_BSP_EXPORT_TIMEOUTis read into configuration but never applied. The code carried an explicit "No way currently to pass timeout to export" comment, and a hung exporter blocks the batch worker (andforce_flush) indefinitely. The spec states Export MUST NOT block indefinitely; Go and Java enforce a deadline.
Fix: enforce the configured timeout as a real deadline around the export call so a hung exporter cannot block forever.Notes / limitations
Exporters are synchronous and Python offers no way to cancel a running call. A sound approach is to run the export on a dedicated single-worker executor and await it with the configured timeout; on timeout, log, record the failure, and stop draining rather than blocking. The abandoned export keeps running on the executor thread (bounded, single-thread back-pressure), but the worker and
force_flushare unblocked, which is the guarantee the spec requires. Care is needed to preserve the instrumentation-suppression context across the executor thread (Pythoncontextvarsare not copied intoThreadPoolExecutorworkers), otherwise the exporter's own transport calls could generate telemetry that feeds back into the processor. Because the timeout lives in the shared batch processor, it also applies toBatchLogRecordProcessor.Testing
Add tests that (a) prove
SimpleSpanProcessornever invokesexportconcurrently under parallelon_end, (b) prove a hung export is bounded by the timeout rather than hanging, and (c) prove instrumentation suppression is active during the export call.
Tracks span-processor export serialization + timeout (Linear OSS-14, findings T2/T3).
T2:
SimpleSpanProcessor.export()is not serialized; concurrent on_end calls invoke the exporter concurrently (spec forbids). Fix: guard with a lock.T3: BSP export timeout (
OTEL_BSP_EXPORT_TIMEOUT) is read but never enforced; a hung export blocks forever. Fix: enforce a deadline around the export call.Branch / worktree:
oss_14_processor_export_lock_timeoutLinear issue: https://linear.app/dash0/issue/OSS-14/t2-t3-simplespanprocessor-concurrent-export-bsp-export-timeout-not