René's URL Explorer Experiment

Title: nvbug-6193808: Work around mojibake in nvml.system_get_process_name on WSL by mdboom · Pull Request #2118 · NVIDIA/cuda-python · GitHub

Open Graph Title: nvbug-6193808: Work around mojibake in nvml.system_get_process_name on WSL by mdboom · Pull Request #2118 · NVIDIA/cuda-python

X Title: nvbug-6193808: Work around mojibake in nvml.system_get_process_name on WSL by mdboom · Pull Request #2118 · NVIDIA/cuda-python

Description: Summary cuda.core.system.get_process_name(pid) raises UnicodeDecodeError under WSL whenever the calling process has a non-C locale (which is the default state for any CPython process, since the interpreter calls setlocale(LC_ALL, "") at startup). This is reproducible by running the cuda_core test suite with any seed that schedules tests/system/test_system_device.py::test_compute_running_processes before tests/system/test_system_system.py::test_get_process_name. The underlying defect is in NVML's WSL implementation (see Root cause below). This PR adds a scoped, defensive workaround in cuda_core so the public API returns a correct value on WSL. It also fixes a latent issue where get_process_name was effectively unusable from a fresh process because it never primed NVML's per-PID name cache. Root cause: the WSL mojibake NVML's nvmlSystemGetProcessName on WSL takes a different code path depending on the process's current locale. With the default "C" locale, the function returns the basename portion of /proc//exe correctly. With any other locale (including the typical en_US.UTF-8), it instead walks an internal UTF-16LE buffer holding the executable path but uses a 4-byte stride (as if the buffer were UTF-32LE). Each "code point" it pulls is therefore two adjacent ASCII bytes packed into the low and next-higher bytes of a single 24-bit value. That value is then emitted as an extended 5-byte UTF-8 sequence (the 0xF8-prefixed encoding used to represent code points beyond U+10FFFF). The net result for, say, a Python process whose /proc//exe resolves to: /home/mdboom/.local/share/uv/python/cpython-3.14.0-linux-x86_64-gnu/bin/python3.14 is that the returned buffer looks like ~180 bytes of f8 … chunks followed by the correctly-encoded trailing basename, e.g.: f8 9a 80 80 af f8 9b 90 81 af f8 8b b0 81 a5 f8 99 80 81 ad … /python3.14\0 Decoding the first chunk illustrates the pattern: bytes f8 9a 80 80 af decode as the extended-UTF-8 code point 0x68002F that 24-bit value packs 'h' (0x68) in the high byte and '/' (0x2F) in the low byte — i.e. the source ASCII bytes /h read as a little-endian 32-bit value padded with zeros Every chunk has this structure; together they spell out the prefix /home/mdboom/.local/share/uv/python/cpython-3.14.0-linux-x86_64-gnu/bin/ two characters at a time. The trailing /python3.14 is unaffected because of where the buggy stride leaves the cursor. Why the workaround needs to "re-prime" nvmlSystemGetProcessName is cache-driven: the per-PID name is populated the first time NVML enumerates compute processes that include the PID (typically via nvmlDeviceGetComputeRunningProcesses_v3). Critically: The mojibake is produced during the prime call, not during the read. Once a buggy entry is in NVML's cache, switching to the "C" locale and re-reading does not unscramble it — the cache survives the locale flip. Re-running the prime call under the "C" locale overwrites the cached entry with the correct UTF-8 string. Subsequent reads (in any locale) then return correctly. So the workaround must do prime + read together under "C". Behaviour after this PR Native Linux / Windows: behaviour is identical to before, except that get_process_name now primes the NVML cache automatically. This makes it usable from a fresh process (previously a caller had to have manually queried device.compute_running_processes first or accept NotFoundError). WSL: the locale flip is applied around the prime + read sequence, so the returned name is the correct UTF-8 string regardless of the caller's locale. Discussion Should we instead try to fix this at the cuda_bindings layer and fix it for all cuda_bindings users? In that case I guess cuda_core should raise an exception from get_process_name if on WSL and the cuda_bindings installed is too old? I do plan to file the underlying bug with NVML. Locale-sensitive APIs should generally be avoided.

Open Graph Description: Summary cuda.core.system.get_process_name(pid) raises UnicodeDecodeError under WSL whenever the calling process has a non-C locale (which is the default state for any CPython process, since the int...

X Description: Summary cuda.core.system.get_process_name(pid) raises UnicodeDecodeError under WSL whenever the calling process has a non-C locale (which is the default state for any CPython process, since the int...

Opengraph URL: https://github.com/NVIDIA/cuda-python/pull/2118

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width