René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Installation test always fails on native Windows","articleBody":"GitPython supports and can be installed on native Windows systems (not just Cygwin), but the `TestInstallation.test_installation` test in `test/test_installation.py` always fails with a `FileNotFoundError`. This is not caught on CI because, for Windows, only Cygwin currently has CI coverage. The test failure happens for two reasons:\r\n\r\n- **The test relies on the virtual environment having a `bin` directory**, but on Windows this is called `Scripts`. This is the immediate cause of the error, but if only this is fixed, a second, closely related incompatibility will cause the test to fail.\r\n- **The test relies on the virtual environment having `python3` and `pip3` commands**, but Windows does not have these in virtual environments. Python virtual environments on all systems have `python` and `pip` commands (even on systems such as Debian and Ubuntu where those commands are not available globally or whose global versions are part of a legacy Python 2 installation).\r\n\r\nThe following output is produced on Windows 10 using Python 3.11.5, installing GitPython from the tip of the main branch (c8e303ff), but I believe the test would fail on all Windows systems with all versions of Python in the same way:\r\n\r\n```text\r\n(.venv) C:\\Users\\ek\\source\\repos\\GitPython [main ≡]\u003e pytest --no-cov -k test_installation\r\nTest session starts (platform: win32, Python 3.11.5, pytest 7.4.2, pytest-sugar 0.9.7)\r\nrootdir: C:\\Users\\ek\\source\\repos\\GitPython\r\nconfigfile: pyproject.toml\r\ntestpaths: test\r\nplugins: cov-4.1.0, sugar-0.9.7\r\ncollected 504 items / 503 deselected / 1 selected\r\n\r\n\r\n――――――――――――――――――――――――――――――――――――――――― TestInstallation.test_installation ――――――――――――――――――――――――――――――――――――――――――\r\n\r\nself = \u003ctest.test_installation.TestInstallation testMethod=test_installation\u003e\r\nrw_dir = 'C:\\\\Users\\\\ek\\\\AppData\\\\Local\\\\Temp\\\\test_installatione9rg39ma'\r\n\r\n    @with_rw_directory\r\n    def test_installation(self, rw_dir):\r\n        self.setUp_venv(rw_dir)\r\n\u003e       result = subprocess.run(\r\n            [self.pip, \"install\", \"-r\", \"requirements.txt\"],\r\n            stdout=subprocess.PIPE,\r\n            cwd=self.sources,\r\n        )\r\n\r\ntest\\test_installation.py:24:\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\nC:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.11_3.11.1520.0_x64__qbz5n2kfra8p0\\Lib\\subprocess.py:548: in run\r\n    with Popen(*popenargs, **kwargs) as process:\r\nC:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.11_3.11.1520.0_x64__qbz5n2kfra8p0\\Lib\\subprocess.py:1026: in __init__\r\n    self._execute_child(args, executable, preexec_fn, close_fds,\r\n_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _\r\n\r\nself = \u003cPopen: returncode: None args: ['C:\\\\Users\\\\ek\\\\AppData\\\\Local\\\\Temp\\\\test_i...\u003e\r\nargs = 'C:\\\\Users\\\\ek\\\\AppData\\\\Local\\\\Temp\\\\test_installatione9rg39ma\\\\bin/pip3 install -r requirements.txt'\r\nexecutable = None, preexec_fn = None, close_fds = False, pass_fds = ()\r\ncwd = 'C:\\\\Users\\\\ek\\\\AppData\\\\Local\\\\Temp\\\\test_installatione9rg39ma\\\\src', env = None\r\nstartupinfo = \u003csubprocess.STARTUPINFO object at 0x000001B4D625B390\u003e, creationflags = 0, shell = False\r\np2cread = Handle(1292), p2cwrite = -1, c2pread = 15, c2pwrite = Handle(1396), errread = -1, errwrite = Handle(1416)\r\nunused_restore_signals = True, unused_gid = None, unused_gids = None, unused_uid = None, unused_umask = -1\r\nunused_start_new_session = False, unused_process_group = -1\r\n\r\n    def _execute_child(self, args, executable, preexec_fn, close_fds,\r\n                       pass_fds, cwd, env,\r\n                       startupinfo, creationflags, shell,\r\n                       p2cread, p2cwrite,\r\n                       c2pread, c2pwrite,\r\n                       errread, errwrite,\r\n                       unused_restore_signals,\r\n                       unused_gid, unused_gids, unused_uid,\r\n                       unused_umask,\r\n                       unused_start_new_session, unused_process_group):\r\n        \"\"\"Execute program (MS Windows version)\"\"\"\r\n\r\n        assert not pass_fds, \"pass_fds not supported on Windows.\"\r\n\r\n        if isinstance(args, str):\r\n            pass\r\n        elif isinstance(args, bytes):\r\n            if shell:\r\n                raise TypeError('bytes args is not allowed on Windows')\r\n            args = list2cmdline([args])\r\n        elif isinstance(args, os.PathLike):\r\n            if shell:\r\n                raise TypeError('path-like args is not allowed when '\r\n                                'shell is true')\r\n            args = list2cmdline([args])\r\n        else:\r\n            args = list2cmdline(args)\r\n\r\n        if executable is not None:\r\n            executable = os.fsdecode(executable)\r\n\r\n        # Process startup details\r\n        if startupinfo is None:\r\n            startupinfo = STARTUPINFO()\r\n        else:\r\n            # bpo-34044: Copy STARTUPINFO since it is modified above,\r\n            # so the caller can reuse it multiple times.\r\n            startupinfo = startupinfo.copy()\r\n\r\n        use_std_handles = -1 not in (p2cread, c2pwrite, errwrite)\r\n        if use_std_handles:\r\n            startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES\r\n            startupinfo.hStdInput = p2cread\r\n            startupinfo.hStdOutput = c2pwrite\r\n            startupinfo.hStdError = errwrite\r\n\r\n        attribute_list = startupinfo.lpAttributeList\r\n        have_handle_list = bool(attribute_list and\r\n                                \"handle_list\" in attribute_list and\r\n                                attribute_list[\"handle_list\"])\r\n\r\n        # If we were given an handle_list or need to create one\r\n        if have_handle_list or (use_std_handles and close_fds):\r\n            if attribute_list is None:\r\n                attribute_list = startupinfo.lpAttributeList = {}\r\n            handle_list = attribute_list[\"handle_list\"] = \\\r\n                list(attribute_list.get(\"handle_list\", []))\r\n\r\n            if use_std_handles:\r\n                handle_list += [int(p2cread), int(c2pwrite), int(errwrite)]\r\n\r\n            handle_list[:] = self._filter_handle_list(handle_list)\r\n\r\n            if handle_list:\r\n                if not close_fds:\r\n                    warnings.warn(\"startupinfo.lpAttributeList['handle_list'] \"\r\n                                  \"overriding close_fds\", RuntimeWarning)\r\n\r\n                # When using the handle_list we always request to inherit\r\n                # handles but the only handles that will be inherited are\r\n                # the ones in the handle_list\r\n                close_fds = False\r\n\r\n        if shell:\r\n            startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW\r\n            startupinfo.wShowWindow = _winapi.SW_HIDE\r\n            if not executable:\r\n                # gh-101283: without a fully-qualified path, before Windows\r\n                # checks the system directories, it first looks in the\r\n                # application directory, and also the current directory if\r\n                # NeedCurrentDirectoryForExePathW(ExeName) is true, so try\r\n                # to avoid executing unqualified \"cmd.exe\".\r\n                comspec = os.environ.get('ComSpec')\r\n                if not comspec:\r\n                    system_root = os.environ.get('SystemRoot', '')\r\n                    comspec = os.path.join(system_root, 'System32', 'cmd.exe')\r\n                    if not os.path.isabs(comspec):\r\n                        raise FileNotFoundError('shell not found: neither %ComSpec% nor %SystemRoot% is set')\r\n                if os.path.isabs(comspec):\r\n                    executable = comspec\r\n            else:\r\n                comspec = executable\r\n\r\n            args = '{} /c \"{}\"'.format (comspec, args)\r\n\r\n        if cwd is not None:\r\n            cwd = os.fsdecode(cwd)\r\n\r\n        sys.audit(\"subprocess.Popen\", executable, args, cwd, env)\r\n\r\n        # Start the process\r\n        try:\r\n\u003e           hp, ht, pid, tid = _winapi.CreateProcess(executable, args,\r\n                                     # no special security\r\n                                     None, None,\r\n                                     int(not close_fds),\r\n                                     creationflags,\r\n                                     env,\r\n                                     cwd,\r\n                                     startupinfo)\r\nE                                    FileNotFoundError: [WinError 2] The system cannot find the file specified\r\n\r\nC:\\Program Files\\WindowsApps\\PythonSoftwareFoundation.Python.3.11_3.11.1520.0_x64__qbz5n2kfra8p0\\Lib\\subprocess.py:1538: FileNotFoundError\r\n\r\n test/test_installation.py ⨯                                                                             100% ██████████\r\n=============================================== short test summary info ===============================================\r\nFAILED test/test_installation.py::TestInstallation::test_installation - FileNotFoundError: [WinError 2] The system cannot find the file specified\r\n\r\nResults (7.00s):\r\n       1 failed\r\n         - test/test_installation.py:21 TestInstallation.test_installation\r\n     503 deselected\r\n```\r\n\r\nI have also verified that this occurs on Python 3.12.0rc2 on Windows. #1640 affects Windows as much as other systems, but this specific issue causes the test to fail first.\r\n\r\nI think it would make sense to fix this together with #1640, since a fix for that would include related changes to `test_installation`. I've opened #1654 for this (which also fixes #1652 and #1653).","author":{"url":"https://github.com/EliahKagan","@type":"Person","name":"EliahKagan"},"datePublished":"2023-09-10T17:55:06.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/1651/GitPython/issues/1651"}