René's URL Explorer Experiment

Title: Complete rustpython-unicode isolation: case mapping, casing predicates, sre parity by youknowone · Pull Request #8237 · RustPython/RustPython · GitHub

Open Graph Title: Complete rustpython-unicode isolation: case mapping, casing predicates, sre parity by youknowone · Pull Request #8237 · RustPython/RustPython

X Title: Complete rustpython-unicode isolation: case mapping, casing predicates, sre parity by youknowone · Pull Request #8237 · RustPython/RustPython

Description: Follow-up to #7560, completing the rustpython-unicode isolation begun in #8211. Implements #8236. What this does Moves the remaining case mapping and casing predicates out of rustpython-vm into rustpython-unicode, so the crate is the single authoritative Unicode path and crates/unicode becomes the only workspace member with a direct icu4x dependency. Also fixes two long-standing TODO: check with cpython quirks in the SRE engine. Commits Move str casing into rustpython-unicode::case and drop icu from vm — adds code-point simple mappings (simple_lowercase/uppercase/titlecase/fold), casing predicates (is_lowercase/is_uppercase/is_titlecase/is_cased/is_case_ignorable), and string-level capitalize/title/swapcase/casefold helpers. str.rs/anystr.rs route through the crate; icu_casemap/icu_locale/icu_properties/writeable leave crates/vm. str.lower/upper keep using Wtf8::to_lowercase/to_uppercase (no icu, and std already applies the final-sigma rule). Route sre is_uni_space through classify::is_space — replaces the hand-rolled BMP list. A full-range sweep confirms it agrees with the old list on every code point, and classify::is_space is differential-tested against CPython. Use simple case mappings and the Cased property for sre IGNORECASE — lower_unicode/upper_unicode took the first char of the full mapping, miscasing code points with a full but no simple mapping (e.g. upper_unicode('ß') returned 'S'). Now use simple_lowercase/simple_uppercase (Py_UNICODE_TOLOWER/TOUPPER). _sre.unicode_iscased derived casedness from those mappings, which broke the _casefix ligature equivalences (ſt/st) once simple mappings were used; it now queries the Cased property directly. Sweep casing predicates and simple lowercase mapping against CPython — extends the differential harness (full 0..0x110000 range) to the new casing predicates and the simple lowercase mapping, each with its own version-skew allow-list. Records the U+0295 Ll→Lo recategorization (Unicode 16.0.0 → 17.0.0) as a known reverse-direction divergence. Verification crates/unicode unit + differential tests pass; no_std target (thumbv7em-none-eabi) still builds. Regression gate green: test_unicodedata, test_str, test_re, test_pkgutil (run=381). re.IGNORECASE now matches CPython on ß/SS, the ſt/st ligatures, and Kelvin K. Acceptance: crates/unicode is the only workspace member with a direct icu dependency. 🤖 Generated with Claude Code Summary by CodeRabbit New Features Expanded Unicode-aware casing and casefolding for strings, including added one-to-one “simple” case mappings and predicates (lowercase/uppercase/titlecase/cased). Bug Fixes Improved CPython-style final-sigma and titlecasing behavior, plus updated non-ASCII casing checks. Preserves lone surrogate characters correctly in WTF-8 string processing. Tests Added and refreshed Unicode conformance coverage for case mappings and predicates.

Open Graph Description: Follow-up to #7560, completing the rustpython-unicode isolation begun in #8211. Implements #8236. What this does Moves the remaining case mapping and casing predicates out of rustpython-vm into rus...

X Description: Follow-up to #7560, completing the rustpython-unicode isolation begun in #8211. Implements #8236. What this does Moves the remaining case mapping and casing predicates out of rustpython-vm into rus...

Opengraph URL: https://github.com/RustPython/RustPython/pull/8237

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width