René's URL Explorer Experiment

Title: fix(out): reconfigure stdout to UTF-8 on any non-UTF-8 terminal by bearomorphism · Pull Request #1975 · commitizen-tools/commitizen · GitHub

Open Graph Title: fix(out): reconfigure stdout to UTF-8 on any non-UTF-8 terminal by bearomorphism · Pull Request #1975 · commitizen-tools/commitizen

X Title: fix(out): reconfigure stdout to UTF-8 on any non-UTF-8 terminal by bearomorphism · Pull Request #1975 · commitizen-tools/commitizen

Description: Description Closes #956. Why Commitizen emits Unicode characters that fall outside the Latin-1 repertoire — most visibly the rocket emoji (\U0001f680) in out.success("Configuration complete 🚀") and the typographic apostrophe (\u2019) scattered throughout the conventional-commits info text. When Python's print() encodes those characters through sys.stdout, it uses whatever encoding the operating system assigned to the terminal. On a system whose LANG is set to de_CH.ISO8859-1 (or any other non-UTF-8 locale), that encoding is Latin-1, which cannot represent either character, so Python raises UnicodeEncodeError and crashes the command mid-output. The original guard in commitizen/out.py:7-9 only reconfigures sys.stdout to UTF-8 when sys.platform == "win32". Linux and macOS users on legacy ISO-8859-1 locales — the reporter ran AlmaLinux 8.9 and macOS Ventura 13.6.3, both with LANG=de_CH.ISO8859-1 — received no such reconfiguration, so every print() call that contained an out-of-range character exploded. Maintainer @Lee-W acknowledged the reproduction in the issue thread; a verification comment on the triage audit (#1964) confirmed the code path is unchanged in master (v4.15.1). The fix extracts the reconfiguration into a small _ensure_utf8_stdout() helper that is encoding-agnostic and platform-agnostic. It is called once at module-import time so that every subsequent print() in every command — write, line, success, info, warn — is already operating on a UTF-8 stream before the first byte is written. What changed File Change commitizen/out.py Drop the sys.platform == "win32" guard (lines 7–9); add _ensure_utf8_stdout() helper; call it unconditionally on sys.stdout at module import tests/test_out.py New file — 6 unit tests: UTF-8 no-op, dashless-alias no-op (UTF8), ISO-8859-1 reconfigure, cp1252 reconfigure, non-TextIOWrapper skip, and end-to-end emoji write-through How it works Encoding normalisation before comparison. The helper computes (stream.encoding or "").lower().replace("-", "").replace("_", "") before comparing against "utf8". This collapses the alias zoo — UTF-8, utf_8, UTF8 — into a single canonical string, so the no-op path is taken for all valid UTF-8 spelling variants without maintaining an allowlist. errors="replace" as a safety net. Even after reconfiguration, a terminal that physically can't render a byte sequence (e.g., a very old xterm with a broken locale) will receive a ? placeholder rather than raising a second UnicodeEncodeError. Crashing mid-output after a user has already answered several cz init prompts is a worse experience than a one-character substitution. Module-level call, not per-print wrapping. _ensure_utf8_stdout(sys.stdout) executes once when commitizen.out is first imported. Because every CLI command imports out before writing anything, the reconfiguration is guaranteed to be in place for the full lifetime of the process — no per-function guard is needed. isinstance(stream, io.TextIOWrapper) guard. Pipes, io.StringIO instances, and pytest's stdout-capture objects are not TextIOWrapper subclasses and do not expose reconfigure(). Checking the type before calling keeps the helper safe in test environments and in any context where sys.stdout has been replaced by a non-standard object. AttributeError / ValueError are swallowed silently. A stream whose underlying buffer is already closed raises ValueError; a stream backed by a read-only file descriptor raises AttributeError. Neither case should crash the import. The # pragma: no cover annotation marks these as genuine safety-net branches rather than untested logic. Backward compatibility Windows behaviour is unchanged: the new helper fires on any platform when encoding is not UTF-8, including cp1252 (the historical Windows case), so the reconfiguration that previously lived behind the win32 guard still happens. Callers that already have a UTF-8 sys.stdout (the common case on modern Linux/macOS) hit the early-return branch — no reconfigure() call, no observable difference. All existing tests pass; tests/test_out.py is a new file, so there are no test regressions. No CLI flags, exit codes, or configuration keys are added or removed. Checklist I have read the contributing guidelines Was generative AI tooling used to co-author this PR? Yes (please specify the tool below) Generated-by: Claude following the guidelines Code Changes Add test cases to all the changes you introduce Run uv run poe all locally to ensure this change passes linter check and tests Manually test the changes (see "Steps to Test" below) Update the documentation for the changes Expected Behavior Scenario Outcome LANG=de_CH.ISO8859-1 cz info on Linux/macOS Prints full info text without UnicodeEncodeError; \u2019 (typographic apostrophe) passes through or is replaced by ? LANG=de_CH.ISO8859-1 cz init completes successfully Prints Configuration complete 🚀 without crashing on \U0001f680 LANG=en_US.UTF-8 cz info (UTF-8 terminal) Behaviour unchanged — _ensure_utf8_stdout hits the early-return branch and calls no reconfigure() Non-TextIOWrapper stdout (e.g. piped, pytest capture) _ensure_utf8_stdout returns immediately; no AttributeError Steps to Test This Pull Request git fetch fork fix/956-cross-platform-stdout-utf8 git checkout fork/fix/956-cross-platform-stdout-utf8 # 1. Targeted regression test. uv run pytest tests/test_out.py -v # 2. Reproduce-the-bug-then-verify-the-fix sequence. # Simulate a Latin-1 stdout without needing to change the OS locale: python - << 'EOF' import io, sys # Replace stdout with a Latin-1 TextIOWrapper — exactly what the OS gives # you when LANG=de_CH.ISO8859-1. sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="latin-1") # Re-import out so _ensure_utf8_stdout() runs against our stub stream. import importlib, commitizen.out as _out importlib.reload(_out) # Before the fix this line crashed with UnicodeEncodeError. _out.write("Conventional commits info: \u2019") _out.success("Configuration complete \U0001f680") print("OK — no UnicodeEncodeError", file=sys.__stdout__) EOF # On a real Linux/macOS machine with a non-UTF-8 locale: # LANG=de_CH.ISO8859-1 cz info # previously crashed # LANG=de_CH.ISO8859-1 cz init # previously crashed at success message Additional Context This fix emerged from the cross-cutting audit in #1964, which confirmed the Windows-only guard in commitizen/out.py:7-9 was the sole cause and that the code path is unchanged in master (v4.15.1). The _ensure_utf8_stdout helper was designed to be testable without monkey-patching the real sys.stdout and reimporting the module, which is why it accepts an arbitrary stream argument rather than hard-coding sys.stdout inside the function body.

Open Graph Description: Description Closes #956. Why Commitizen emits Unicode characters that fall outside the Latin-1 repertoire — most visibly the rocket emoji (\U0001f680) in out.success("Configuration complete 🚀&...

X Description: Description Closes #956. Why Commitizen emits Unicode characters that fall outside the Latin-1 repertoire — most visibly the rocket emoji (\U0001f680) in out.success(&quot;Configuration complet...

Opengraph URL: https://github.com/commitizen-tools/commitizen/pull/1975

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width