René's URL Explorer Experiment

Title: bpo-32146: multiprocessing freeze_support needed outside win32 by bbayles · Pull Request #5195 · python/cpython · GitHub

Open Graph Title: bpo-32146: multiprocessing freeze_support needed outside win32 by bbayles · Pull Request #5195 · python/cpython

X Title: bpo-32146: multiprocessing freeze_support needed outside win32 by bbayles · Pull Request #5195 · python/cpython

Description: This PR un-breaks "frozen" executables (e.g., those produced by PyInstaller or cx_Freeze that (a) run on Unix, and (b) use multiprocessing, and (c) use the spawn or forkserver start methods. I linked to this dissection of the problem in the issue; it is a good overview of what goes wrong currently. In a nutshell, the forkserver and semaphore_tracker modules try to launch new processes with command line arguments that the "frozen" executable doesn't support: When using argparse, the error is clear: error: unrecognized arguments: -S -E -c from multiprocessing.semaphore_tracker import main;main(5) error: unrecognized arguments: -S -E -c from multiprocessing.forkserver import main; main(8, 9, ['__main__'], **{'sys_path': [''/opt/freeze_support_test/lib/library.zip', '/opt/freeze_support_test/lib']}) On Windows this problem is avoided because freeze_support captures the arguments, parses them, executes the appropriate code, and then exits here. It recognizes this by way of a dummy command line argument, --multiprocessing-fork. This PR: Removes the win32 check that prevents freeze_support from having an effect on Unix Adds dummy --multiprocessing-forkserver and --multiprocessing-semaphore-tracker command line arguments, which are only used with "frozen" executables Extends freeze_support such that it captures and parses the command line arguments for the fork server and semaphore tracker, and launches them. This fixes the problem for me on both PyInstaller and cx_Freeze on Unix. I couldn't find that there were any tests for the existing freeze_support code. I added some basic ones for the command line manipulation that I've added here, but it may be useful to file an issue to add others. I will add a few more comments in-line. I'm a new-ish contributor, so if discussion of the approach to fixing this issue should take place elsewhere (in the bug tracker or on a mailing list), please let me know (gently). Many thanks! https://bugs.python.org/issue32146

Open Graph Description: This PR un-breaks "frozen" executables (e.g., those produced by PyInstaller or cx_Freeze that (a) run on Unix, and (b) use multiprocessing, and (c) use the spawn or forkserver start metho...

X Description: This PR un-breaks "frozen" executables (e.g., those produced by PyInstaller or cx_Freeze that (a) run on Unix, and (b) use multiprocessing, and (c) use the spawn or forkserver sta...

Opengraph URL: https://github.com/python/cpython/pull/5195

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width