René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"make altinstall for PGO is not parallel-safe","articleBody":"**Bug report**\r\n\r\n``setup.py`` has a helper method [``add_multiarch_paths``](https://github.com/python/cpython/blob/70690c7233aaad4431660a64daef81b5220e2ac1/setup.py#L688-L696). The method creates a temporary file ``build/lib.{platform}/multiarch`` and unlinks it at the end of the function call. This is not parallel-safe.\r\n\r\nPGO builds of Python use ``$(MAKE)``, so called *recursive make*. [Recursive makes are considered harmful](https://accu.org/journals/overload/14/71/miller_2004/) because the main make process has no understanding what the child make process is doing. With heavy parallel makes this can cause race conditions.\r\n\r\nThe combination of unsafe ``add_multiarch_paths()``, recursive make and loooots of CPU cores can lead to build issues like https://github.com/python/cpython/issues/84461#issuecomment-1148672087.\r\n\r\n**Possible workarounds**\r\n\r\n1) Rewrite our Makefile to not use ``$(MAKE)``\r\n2) Move ``CC -print-multiarch`` and ``dpkg-architecture ... -qDEB_HOST_MULTIARCH`` calls to ``configure.ac``\r\n3) Fix ``_bootsubprocess.check_output()`` and ``add_multiarch_paths()`` to use a more safe tmp file\r\n\r\nOption (3) is the simplest approach. Even ``tmpfile = os.path.join(self.build_temp, f'multiarch-{os.getpid()}')`` would be good enough to avoid file conflicts in parallel builds.","author":{"url":"https://github.com/tiran","@type":"Person","name":"tiran"},"datePublished":"2022-06-07T14:04:40.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":7},"url":"https://github.com/93584/cpython/issues/93584"}