Add an asynchronous cancellation API for in-flight operations#5882
Conversation
Long-running BLAS calls (a large gemm can run for minutes) cannot currently be interrupted: callers embedding OpenBLAS (e.g. the Julia runtime responding to a user's ^C) can only wait for completion or kill the process. Add a minimal cooperative cancellation protocol: Every thread owns a pointer-sized generation slot in thread-local storage, whose stable address is returned by openblas_cancel_token(). Instrumented compute drivers advance the slot to a fresh even generation at operation entry on the issuing thread (forwarding the slot and generation to worker threads through blas_arg_t) and poll it at block granularity. openblas_cancel(token, loaded_token) - callable from any thread - sets the cancel bit (bit 0) iff the slot still holds loaded_token, so a canceller that loaded the value while an operation was in flight stops exactly that operation, while stale or racing requests either miss or dirty an already-dead generation, both harmless. There is no object lifecycle: nothing to allocate, bind, reset, or free. A cancelled operation returns quickly, leaving its output buffer in an unspecified partially-updated state that the caller must discard; every synchronization point in the threaded driver is still executed, so sibling threads never stall and the library remains consistent for subsequent calls. Coverage: the level-3 gemm/symm/hemm drivers (level3.c and level3_thread.c). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012nCkyKUguncLLJrH9K5o7m
|
I wonder if this should be a compile-time option - given that nobody else expressed interest in this feature during all those years, and you're adding extra polling to implement it ? |
|
I think that would be ok, but if the extra comparison is the only concern, then I'd rather leave it enabled by default and switch to the non-cooperative cancellation version. That said, the poll can and should be at coarse enough granularity that it doesn't really matter. At least we don't need more than ~50ms responsiveness here or something at which point the extra instruction is completely meaningless (not saying the places chosen here achieve that, just making a general point). |
|
Could you please rebase this PR (or allow maintainer modifications for its branch so that I can resolve the merge conflict) ? |
2 participants
This implements a feature I first requested 12 years ago (#378). Better late than never.
Long-running BLAS calls (a large gemm can run for minutes) cannot currently be interrupted: callers embedding OpenBLAS (e.g. the Julia runtime responding to a user's ^C) can only wait for completion or kill the process. Add a minimal cooperative cancellation protocol:
Every thread owns a pointer-sized generation slot in thread-local storage, whose stable address is returned by openblas_cancel_token(). Instrumented compute drivers advance the slot to a fresh even generation at operation entry on the issuing thread (forwarding the slot and generation to worker threads through blas_arg_t) and poll it at block granularity. openblas_cancel(token, loaded_token) - callable from any thread - sets the cancel bit (bit 0) iff the slot still holds loaded_token, so a canceller that loaded the value while an operation was in flight stops exactly that operation, while stale or racing requests either miss or dirty an already-dead generation, both harmless.
A cancelled operation returns quickly, leaving its output buffer in an unspecified partially-updated state that the caller must discard; every synchronization point in the threaded driver is still executed, so sibling threads never stall and the library remains consistent for subsequent calls. Coverage: the level-3 gemm/symm/hemm drivers (level3.c and level3_thread.c).
The cooperative cancellation approach is the simplest. That said, even though the check is cheap it does of course execute instructions. If that's deemed too much for a niche feature, it would be possible to implement a non-cooperative cancellation approach. However, because that approach is significantly more complicated and imposes structural constraints on what can be executed inside cancellable regions, I went with the cooperative approach for now. Of course the API surface would be the same so we can still switch later.
AI Disclosure: The code was generated by AI (as part of a big project to fix cancellation in Julia) based on my description of the interface and tested in situ in that PR.