René's URL Explorer Experiment

Title: Pin Python 3.9.16 on Cygwin CI by EliahKagan · Pull Request #1814 · gitpython-developers/GitPython · GitHub

Open Graph Title: Pin Python 3.9.16 on Cygwin CI by EliahKagan · Pull Request #1814 · gitpython-developers/GitPython

X Title: Pin Python 3.9.16 on Cygwin CI by EliahKagan · Pull Request #1814 · gitpython-developers/GitPython

Description: The bug The latest currently packaged version of Python 3.9 for Cygwin is 3.9.18 (provided by the Cygwin package python39 at version 3.9.18-1). That version, at least as we are using it, has a problem where pip stalls indefinitely on some PyPI package downloads. This is the problem encountered around the time #1813 was reviewed and merged, and discussed in comments there, but it was not triggered by that or any other change in GitPython. Details In #1813 (comment) I said I couldn't produce it locally, but since then I have been able to do so. I believe the reason I couldn't produce it locally at that time was that I had not cleared cached pip downloads in ~/.cache. I am unsure of the exact condition under which the problem occurs, but it is possibly when a single run of pip must download two packages; the first downloads, and the second stalls forever. I have in some places, including some commit messages, described this as "blocking," but that was a guess, and a wrong one. When produced locally, the Cygwin python3.9 process uses a full CPU core until terminated (so it is "spinning" rather than "blocking"). I'm willing to rebase commits on request to correct that, but I lean toward not doing so for reasons of efficiency, in part because CI will run again on them. In Cygwin with the kill command, SIGTERM does not terminate it; SIGKILL does not seem to either, or at least not reliably. Right-clicking on the process in the "Details" tab of the Task Manager always terminates it immediately. Once terminated, pip can be attempted again, and it will succeed in downloading what it had just failed to download, but then fail on the next download if there is another one (which there usually is). This happens whether or not I upgrade the PyPA packages (pip, setuptools, and wheel). It sometimes happens for wheel when I do, which is how a number of the CI runs have failed. When it doesn't, that pip run succeeds. Whether or not PyPA packages were upgraded, the subsequent run of pip install -e ".[test]", which installs many more packages, always encounters the problem. More specifically, it used to happen with the coverage package, but a few hours ago started usually happening with pytest instead, which I believe is just due to a different order in which packages are downloaded, triggered by the very recent release of pytest 8. If pip is terminated and rerun, it will happen on every second package, as far as I have observed. This happens with multiple versions of the python39-pip Cygwin package, as well as with multiple versions of the PyPI pip package if requested explicitly. Therefore, while the problem happens when pip runs, I don't believe it is due to a bug in pip or even a bug in Cygwin's python39-pip package. It also does not seem to be affected by whether pip is run with pip or python3.9 -m pip, nor by whether or not pip is used in a virtual environment. It happens with multiple versions of the cygwin Cygwin package which provides cygwin1.dll. It can happen when attempting to download a source package or a wheel. It only happens on Cygwin, not native Windows or other platforms. Examining the CI logs for commits from before and after the problem began reveals that the problem did not occur when the python39 Cygwin package was at version 3.9.16-1, and always occurred once it was at version 3.9.18-1. For example, 9b7e15f used 3.9.16-1 while 987dbf4 used 3.8.18-1. Specific versions were not requested, so typically the latest stable versions of Cygwin packages are installed, and it was on 26 January that python39 version 3.9.18-1 was promoted to stable. This does not occur with Python 3.8. The workaround Pinning 3.9.16-1 This pull request downgrades Python on Cygwin to the latest available patch version of 3.9 packaged for Cygwin of those that strictly precede 3.9.18 where the problem occurs. That version is 3.9.16, provided by the Cygwin package python39 at version 3.9.16-1. This version may eventually no longer be available for download from Cygwin's repositories, so hopefully a real solution or better workaround will be found by then, or perhaps a future update to the package itself will fix the problem. Because 3.8 works, an existing backup workaround is to downgrade even further to 3.8. Switching actions to facilitate pinning Although GitHub code search finds repositories where the official cygwin/install-cygwin-action GitHub Action was used, or an attempt made to use it, with the package=version syntax for specifying Cygwin packages to be installed, that does not appear working. It doesn't report an error when I attempt it, but the version number seems always to be ignored. So in order to pin python39 at 3.9.16-1, I also switched to using egor-tensin/setup-cygwin. The dependency graph feature in GitHub reports them as being about equally popular (official, unofficial). However, I removed add-to-path: false; it seems egor-tensin/setup-cygwin doesn't have such a feature. So it may be worthwhile to switch back once we can, to use that again. The main benefit of add-to-path: false is clarity about the environment from which we are using Cygwin facilities, but it can also help with cleanup/finalization performed by actions that that are run before Cygwin is installed: actions/checkout issues a warning and seems like it may not be cleaning up fully, due to attempting to use the Cygwin git for cleanup. I think it is okay for now because the GitHub hosted runners are virtual machines that get deleted and recreated each time; only self-hosted runners are (potentially) reused. But if we have to keep using it for a long time then that should be fixed. Other things I tried I tried a bunch of other things while investigating this, as well as trying a few variations on the downgrades (corresponding to some of the things I said made no difference above). If each of my original commits were its own commit in this pull request, the pull request would have more than 40 commits. Although this may not be inherently excessive, it seems to me that it was better to squash them down into two commits, represented the two actually useful workarounds I found (downgrading to 3.8, and downgrading only to 3.9.16) done in what seemed like the best of the ways I tried. However, for future investigation, I preserved the full history by using GitHub itself to perform the squashes, from two fork-internal PRs, EliahKagan#2 and EliahKagan#3, which are linked and explained in the two commits. (I had to amend the second after squashing to fix a mistake in its title, which is why its hash differs from the hash the PR shows as merged.) The individual original commits can be examined in those PRs, though I don't believe it is at all necessary to look at that to review this PR (if I did, I wouldn't have squashed them). Note that some of those individual commits might be misleading individually because the changes did not always achieve what the messages described; in particular, attempts to pin packages before switching from cygwin/cygwin-install-action to egor-tensin/setup-cygwin did not actually do any pinning.

Open Graph Description: The bug The latest currently packaged version of Python 3.9 for Cygwin is 3.9.18 (provided by the Cygwin package python39 at version 3.9.18-1). That version, at least as we are using it, has a prob...

X Description: The bug The latest currently packaged version of Python 3.9 for Cygwin is 3.9.18 (provided by the Cygwin package python39 at version 3.9.18-1). That version, at least as we are using it, has a prob...

Opengraph URL: https://github.com/gitpython-developers/GitPython/pull/1814

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width