René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Tests: error \"tests may fail, unable to create temporary directory (...): [WinError 3] The system cannot find the path specified\" on Windows","articleBody":"# Windows CI failure: `test_itertools` corrupts Unicode temp directory name under `mbcs` encoding, breaks subsequent tests\n\n## Summary\n\nOn Windows (official Azure CI), the CPython 3.15 test suite produces cascading failures originating from `test_itertools`. The test creates a temporary directory whose name contains a non-ASCII character. Because the Windows default filesystem encoding is `mbcs`, the Unicode byte sequence is misinterpreted and the directory name becomes corrupted on disk. The directory is never cleaned up, which causes all subsequent tests that create temporary directories to fail with `FileNotFoundError`.\n\n---\n\n## CPython Version\n\n```\nPython 3.15.0a7 (main branch)\n```\n\n## Operating System / Platform\n\n```\nWindows (Azure Pipelines Microsoft-hosted runner)\n```\n\n## Filesystem Encoding\n\n```python\n\u003e\u003e\u003e import sys\n\u003e\u003e\u003e sys.getfilesystemencoding()\n'mbcs'\n```\n\n---\n\n## CI Pipeline Context\n\n**Build step:**\n```\nPCbuild\\build.bat -e $(buildOpt)\n```\n\n**Test step:**\n```\nPCbuild\\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 -j0 \\\n  --junit-xml=\"$(Build.BinariesDirectory)\\test-results.xml\" \\\n  --tempdir=\"$(Build.BinariesDirectory)\\test\"\n```\n\n**Azure DevOps build log:**\nhttps://dev.azure.com/Python/cpython/_build/results?buildId=168864\u0026view=logs\u0026j=c8a71634-e5ec-54a0-3958-760f4148b765\u0026t=ddcdae4e-111a-5c2a-2289-6b784c553924\n\n---\n\n## Steps to Reproduce\n\n1. Check out the CPython `main` branch.\n2. Build on Windows:\n   ```\n   PCbuild\\build.bat -e \u003cbuildOpt\u003e\n   ```\n3. Run the test suite:\n   ```\n   PCbuild\\rt.bat -q -uall -u-cpu -rwW --slowest --timeout=1200 ^\n       -j0 --tempdir=\"%TEMP%\"\n   ```\n4. Observe a directory with a corrupted name (e.g. `packageæ/`) left behind in the temp directory.\n5. Observe that subsequent tests fail with `FileNotFoundError`.\n\n---\n\n## Expected Behavior\n\n- All tests pass cleanly on Windows during CI builds.\n- No test leaves behind artifacts that interfere with other tests.\n- Test isolation is preserved — temporary directories are created and removed correctly.\n- Unicode filenames are handled consistently regardless of platform filesystem encoding.\n\n---\n\n## Actual Behavior\n\n- `test_itertools` creates a directory whose name contains a non-ASCII character (e.g. `æ`).\n- Under the Windows default codepage (`mbcs`), the Unicode name is misinterpreted: `æ` (U+00E6) is stored as `æ`.\n- The corrupted directory is **not removed** after `test_itertools` completes.\n- Subsequent tests (e.g. `test_class`, `test_pdb`) fail when attempting to create temporary directories because the corrupted entry causes `os_helper.temp_dir` to raise `FileNotFoundError`.\n- The runner reports a warning for `test_itertools` and marks unrelated tests as failed, significantly increasing triage cost.\n\n---\n\n## Relevant Log Excerpt\n\nFrom Azure build 168864:\n\n```\nWarning -- files was modified by test_itertools\nAfter: ... 'packageæ/'\n\ntest_class failed (env changed)\nFileNotFoundError: [WinError 3] unable to create temporary directory:\n'D:\\a\\1\\s\\build\\test_python_9140æ'\n```\n\n---\n\n## Root Cause (Analysis)\n\nTwo independent problems combine to produce this failure:\n\n**1. Encoding mismatch in `test_itertools`**\n\nThe test constructs or receives a directory name containing a non-ASCII character. On Windows, `sys.getfilesystemencoding()` returns `'mbcs'` rather than `'utf-8'`. The UTF-8 byte sequence for `æ` (`0xC3 0xA6`) is reinterpreted under the ANSI code page as two separate characters, producing the visible corruption `æ`. This is a violation of cross-platform Unicode path handling requirements.\n\n**2. Missing cleanup in `test_itertools`**\n\nThe test creates the directory but does not register it for cleanup (e.g. via `addCleanup`, `self.addCleanup`, or a context manager such as `tempfile.TemporaryDirectory`). The corrupted directory persists in the shared `--tempdir` location for the duration of the entire test run, causing every subsequent call to `os_helper.temp_dir` to fail with `WinError 3`.\n\n---\n\n## Impact\n\n| Area | Description | Severity |\n|---|---|---|\n| CI pipeline stability | All Windows builds fail after `test_itertools` runs | **High** |\n| Test isolation | Subsequent tests inherit corrupted filesystem state | **High** |\n| Unicode correctness | Non-ASCII filenames silently corrupted on Windows | **Medium** |\n| Triage cost | Unrelated failures obscure the root cause | **Medium** |\n| Local Windows dev | Developers running the full suite locally may be affected | **Low** |\n\n---\n\n## Files Involved\n\n- `Lib/test/test_itertools.py` — primary location of the encoding issue and missing cleanup\n- `Lib/test/support/os_helper.py` — secondary impact point (`temp_dir` context manager)\n\n---\n\n## Linked Resources\n\n- Azure DevOps build 168864: https://dev.azure.com/Python/cpython/_build/results?buildId=168864\n- PEP 529 — Change Windows filesystem encoding to UTF-8: https://peps.python.org/pep-0529/\n- `sys.getfilesystemencoding()` docs: https://docs.python.org/3/library/sys.html#sys.getfilesystemencoding\n- `os.fsencode()` docs: https://docs.python.org/3/library/os.html#os.fsencode\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-146347\n* gh-146349\n* gh-146350\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/Aasyaco","@type":"Person","name":"Aasyaco"},"datePublished":"2026-03-20T11:15:29.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":10},"url":"https://github.com/146202/cpython/issues/146202"}