Title: gh-155717: Determine multiprocessing's default start method lazily by gpshead · Pull Request #159254 · python/cpython · GitHub
Open Graph Title: gh-155717: Determine multiprocessing's default start method lazily by gpshead · Pull Request #159254 · python/cpython
X Title: gh-155717: Determine multiprocessing's default start method lazily by gpshead · Pull Request #159254 · python/cpython
Description: gh-155827 made import multiprocessing check whether the temporary directory is writeable, to choose between forkserver and spawn as the default start method. Doing that at import time is a startup regression, reported by Leandro Damascena in #155717 (comment): the import now also loads multiprocessing.util, subprocess, tempfile, shutil and random, even in programs that never start a process. Spawned children, the fork server and the resource tracker pay it too, as each imports multiprocessing.
This keeps the behaviour of gh-155827 but determines the default start method the first time it is needed, as that comment suggested.
Changes
DefaultContext resolves the platform default on first use (get_context(), get_start_method(), get_all_start_methods(), or starting a default Process) and caches it. get_start_method(allow_none=True) still returns None without resolving anything.
multiprocessing.context no longer imports multiprocessing.util at the top level, and multiprocessing.util imports tempfile only where it is used, as before gh-155827.
Includes the fix from gh-158410 by Vyron Vasileiadis, with a test adapted from that PR: _has_writeable_tempdir() returns false instead of raising FileNotFoundError when tempfile finds no usable temporary directory at all. With the lazy change alone that error would only move from import to first use, so both are needed. This PR supersedes gh-158410.
Documents the fallback in the forkserver start method description, with a versionchanged.
flowchart TB
subgraph before["Before (gh-155827)"]
direction TB
b1["import multiprocessing"] --> b2["import multiprocessing.util,
subprocess, tempfile, shutil, random"]
b2 --> b3{"can a temporary file
be created?"}
b3 -- "yes" --> b4["default = forkserver"]
b3 -- "no" --> b5["default = spawn"]
b3 -- "no usable temp
directory at all" --> b6["import fails with
FileNotFoundError"]
b4 --> b7["first get_context() /
Process.start()"]
b5 --> b7
b7 --> b8["use the default
chosen at import"]
end
subgraph after["After (this PR)"]
direction TB
a1["import multiprocessing"] --> a2["no extra imports,
no default chosen yet"]
a2 --> a3["first get_context() /
Process.start()"]
a3 --> a4["import multiprocessing.util
and tempfile"]
a4 --> a5{"can a temporary file
be created?"}
a5 -- "yes" --> a6["default = forkserver"]
a5 -- "no, or no usable
temp directory at all" --> a7["default = spawn"]
a6 --> a8["cache and use
the default"]
a7 --> a8
end
Loading
The diagram shows POSIX platforms that support forkserver, other than macOS. Windows and macOS always default to spawn and never run the check; that is unchanged.
Measurements
Linux x86_64, -O3 non-PGO build, whole-process wall time, median of 140 interleaved runs. "Before gh-155827" uses the Lib/multiprocessing package from the parent of that commit.
Before gh-155827
main
This PR
import multiprocessing
24.3 ms
35.9 ms
23.8 ms
from multiprocessing import cpu_count; cpu_count()
24.1 ms
36.0 ms
23.9 ms
modules in sys.modules after the import
67
101
67
Behaviour notes
The check now reflects the temporary directory configuration at first use instead of at import.
A process whose start method is set explicitly, which includes spawn and forkserver children of a parent using the default, does not run the check unless it asks for the default: get_all_start_methods(), or get_context() while no start method is set.
DefaultContext._default_context is now a functools.cached_property. It can still be read, assigned, deleted and patched like the plain attribute it was, and DefaultContext(context) still accepts an explicit default.
Backports
This needs backporting to 3.15: the 3.15 backport of gh-155827 (gh-155887) was merged on 2026-10-10, so the 3.15 branch currently has the import-time check, including the FileNotFoundError on import when there is no usable temporary directory. The 3.14 backport (gh-155886) is still open and should pick this up before it is merged.
There is no new NEWS entry: the entry from gh-155827 is still in Misc/NEWS.d/next/ on both main and 3.15 and still describes the user-visible change. If a release containing gh-155827 goes out before this lands on that branch, this needs its own NEWS entry there.
Authored using Claude, reviewed by me.
Issue: gh-155717
Open Graph Description: gh-155827 made import multiprocessing check whether the temporary directory is writeable, to choose between forkserver and spawn as the default start method. Doing that at import time is a startup ...
X Description: gh-155827 made import multiprocessing check whether the temporary directory is writeable, to choose between forkserver and spawn as the default start method. Doing that at import time is a startup ...
Opengraph URL: https://github.com/python/cpython/pull/159254
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:fcd937ca-d311-69e2-67bf-ff264eb0bc4b |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | BCE8:E0ADF:1B730CB:2291929:6B0233AD |
| html-safe-nonce | a11d6b12c2b109cafd224a59e14de7b98c5e667ccc8bab387b3f8d627a002bc2 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCQ0U4OkUwQURGOjFCNzMwQ0I6MjI5MTkyOTo2QjAyMzNBRCIsInZpc2l0b3JfaWQiOiI4NTk1NjE2MTQ0MDM4MTc5NzU3IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | c0b0dd57f3573ffce6ee3f3ffe90f5a43eaa0a4e64e0b75db985bafb2860698c |
| hovercard-subject-tag | pull_request:4834268218 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/159254/files |
| twitter:image | https://avatars.githubusercontent.com/u/68491?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/68491?s=400&v=4 |
| og:image:alt | gh-155827 made import multiprocessing check whether the temporary directory is writeable, to choose between forkserver and spawn as the default start method. Doing that at import time is a startup ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width