Skip to content

Fix scheduler lock inversion and add leak and thread sanitizer checks - #29

Open
YasenSS wants to merge 11 commits into
linker-bot:mainfrom
YasenSS:fix/scheduler-sanitizers-split
Open

YasenSS wants to merge 11 commits into
linker-bot:mainfrom
YasenSS:fix/scheduler-sanitizers-split

Conversation

@YasenSS

@YasenSS YasenSS commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

Problem

Destroying retained command resources under the scheduler mutex can acquire the runtime mutex in the opposite order to runtime send observers. Existing CI also leaves leak detection disabled and does not check data races.

Change

  • Release completed, replaced, invalidated and expired command resources after unlocking, with barrier-based regression coverage. Preserve public APIs and avoid new allocations on normal completion.
  • Enable LSan in the ASan/UBSan job and add an independent Clang 18 TSan job with diagnostic artifacts. Instrument the SDK and paired dependencies.
  • Isolate offline Catch2 test names to avoid stale mutex-lifetime lock graphs while retaining race and deadlock detection. Document requirements and coverage in English and Chinese.

Validation

  • Ubuntu 24.04: Clang 18 TSan passed 1,147/1,147 CTest entries; GCC 13 ASan/UBSan/LSan passed 21/21; no skips. Counts include discovered entries, not distinct logical cases.
  • The new regression fails on the original scheduler in all four sections and passes after the fix. Intentional leak and race probes confirm detector failures.
  • Installed find_package consumer passed with ASan/UBSan/LSan. Formatting, actionlint, documentation and whitespace checks passed.
  • Linux validation used Unix Makefiles and process-scoped ASLR disabling; GitHub uses Ninja and adjusts ASLR on its ephemeral runner. GitHub Actions is pending. No physical-device acceptance is claimed.

Retained callbacks can destroy a backend and acquire the runtime lock.
Release completed and discarded requests after unlocking to remove the
inverse order with runtime send observers. Keep completion allocation-free.
Use synchronization barriers to check completion, latest replacement,
invalidation and timeout cleanup. Scheduler inspection must remain
available while retained resources are being destroyed.
Discover communication and discovery cases through Catch2 when requested.
Separate test names to avoid stale lock graphs across reused mutex
addresses. Preserve default grouping and keep deadlock detection enabled.
Enable LeakSanitizer and use exit code 23 for leak findings alongside
existing address and undefined-behavior checks.
Fail when no tests are registered and produce JUnit results. Upload
ASan, UBSan and LSan diagnostics even when an earlier step fails.
Instrument the SDK and paired dependencies with Clang 18 ThreadSanitizer.
Use an independent build, compatible runner address-space settings and
offline test isolation while retaining race and deadlock checks. Upload
CTest results and failure diagnostics.
Explain both sanitizer builds, test isolation, runtime requirements and
diagnostic artifacts in English and Chinese. State the offline scope
and distinguish initialization errors from clean sanitizer results.
Clang 18 selects GCC 14 headers on the hosted runner and fails compiling
std::timed_mutex calls before tests start. Install GCC 13 explicitly and
select its installation for every TSan C++ compile and link invocation.

Validated toolchain selection, DeviceRuntime compilation and a TSan
timed-mutex probe with Clang 18 and GCC 13 on Linux.
Document the Clang 18 and GCC 14 timed-mutex compilation failure and the
explicit GCC 13 selection in both language guides.
The close regression ran REQUIRE_THROWS in an async worker concurrently
with the main thread assertion. TSan detects a race in Catch2 output
redirection because assertion reporting is not thread-safe.

Keep request.wait in the worker and assert its exception through
future.get on the main thread. Preserve the close and wakeup checks.

The corrected case passed 100 consecutive Clang 18 TSan runs on Linux.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant