You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BuildBackendHookCaller.subprocess_runner() context manager is not thread-safeΒ #226
If two threads share the same BuildBackendHookCaller instance and one of them uses subprocess_runner() to temporarily swap the runner, the other thread sees the swapped runner and may invoke hooks with the wrong runner. The restore in finally also races with concurrent reads of self._subprocess_runner in _call_hook.
Impact
This prevents consumers (e.g. pypa/build) from safely sharing a BuildBackendHookCaller (or a wrapping ProjectBuilder) across threads when subprocess_runner() is used.
Discovered during a thread-safety audit of pypa/build for pypa/build#377.
BuildBackendHookCaller should be pretty cheap to create. Is it reasonable to document that multi-threaded callers should create a separate instance per thread, rather than adding something like a thread local? Or even just document that you shouldn't use the convenience context manager along with threads? You can set the context manager subprocess runner when instantiating the object.
FWIW, I generally don't assume that arbitrary Python objects are thread safe unless it's documented as such.
I checked this locally and can reproduce the cross-thread runner leak with a small regression test: while one thread is inside subprocess_runner(scoped_runner), another thread using the same caller also sees the scoped runner.
I also have a small ContextVar-based patch that keeps the temporary runner scoped to the current context while preserving the default runner on the shared caller.
Before opening a PR, would you prefer a code fix for this behavior, or would you rather document that callers should use a separate BuildBackendHookCaller per thread?
If the limitation is causing actual difficulties, then I would rather design an API that visibly doesn't rely on mutable state, instead of adding magic mechanisms around that state. That would look something like this:
new_hook_caller=hook_caller.with_subprocess_runner(my_runner) # Make a modified copynew_hook_caller.get_requires_for_build_wheel()
But it doesn't look like build actually uses the with hook_caller.subprocess_runner(...) API that this issue is about, so I'm not sure there actually is a problem at all.
Activity
fix: resolve thread-safety issues in API
π fix(api): resolve thread-safety races in build API (#1015)
takluyver commentedon Apr 9, 2026
BuildBackendHookCallershould be pretty cheap to create. Is it reasonable to document that multi-threaded callers should create a separate instance per thread, rather than adding something like a thread local? Or even just document that you shouldn't use the convenience context manager along with threads? You can set thecontext managersubprocess runner when instantiating the object.FWIW, I generally don't assume that arbitrary Python objects are thread safe unless it's documented as such.
hansu650 commentedon May 18, 2026
I checked this locally and can reproduce the cross-thread runner leak with a small regression test: while one thread is inside
subprocess_runner(scoped_runner), another thread using the same caller also sees the scoped runner.I also have a small ContextVar-based patch that keeps the temporary runner scoped to the current context while preserving the default runner on the shared caller.
Before opening a PR, would you prefer a code fix for this behavior, or would you rather document that callers should use a separate
BuildBackendHookCallerper thread?takluyver commentedon May 19, 2026
I would rather document this for now.
If the limitation is causing actual difficulties, then I would rather design an API that visibly doesn't rely on mutable state, instead of adding magic mechanisms around that state. That would look something like this:
But it doesn't look like
buildactually uses thewith hook_caller.subprocess_runner(...)API that this issue is about, so I'm not sure there actually is a problem at all.