René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`lib/python3.12: File exists` when doing `make -jN install`","articleBody":"# Bug report\n\n### Bug description:\n\nOn CPython 3.12.0rc3 I am reliably seeing a new apparent race condition in the build system.\n\nOn prior CPython releases, `make -jN install` would reliably work. On 3.12, it no longer does.\n\nHere is example build output:\n\n```\nTo find the necessary bits, look in configure.ac and config.log.\n\nChecked 111 modules (31 built-in, 73 shared, 4 n/a on macosx-13.5-x86_64, 0 disabled, 3 missing, 0 failed on import)\nCreating directory /tools/host/lib/python3.12\nCreating directory /tools/host/lib/python3.12/config-3.12-darwin\nCreating directory /tools/host/lib/python3.12\ninstall: mkdir /private/var/folders/dd/xb3jz0tj133_hgnvdttctwxc0000gn/T/tmpsgqb2dup/out/tools/host/lib/python3.12: File exists\nCreating directory /tools/host/lib/python3.12/lib-dynload\nmake: *** [libainstall] Error 71\nmake: *** Waiting for unfinished jobs....\nCreating directory /tools/host/lib/python3.12/asyncio\n```\n\nThe `Creating directory %s` log message comes from CPython's `Makefile.pre.in` and we can see from these logs that there appear to be 2 callers racing to create `lib/python3.12` around the same time.\n\nSteps to reproduce are just a simple `./configure` + `make -jN install`, where N is say 8.\n\nI can reliably reproduce this failure on macOS 13.5.2 and it _might_ be due to a buggy `/usr/bin/install -d` implementation on macOS. I think the problem is `/usr/bin/install -d` doesn't swallow/ignore EEXIST like `mkdir -p` does.\n\nI want to say I encountered limitations/bugs in macOS `/usr/bin/install` a few years back when I was a maintainer of Firefox's build system.\n\n### CPython versions tested on:\n\n3.12\n\n### Operating systems tested on:\n\nmacOS","author":{"url":"https://github.com/indygreg","@type":"Person","name":"indygreg"},"datePublished":"2023-09-24T03:05:30.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/109796/cpython/issues/109796"}