René's URL Explorer Experiment

Title: Match CPython's islower/isupper exactly by joshuamegnauth54 · Pull Request #7646 · RustPython/RustPython · GitHub

Open Graph Title: Match CPython's islower/isupper exactly by joshuamegnauth54 · Pull Request #7646 · RustPython/RustPython

X Title: Match CPython's islower/isupper exactly by joshuamegnauth54 · Pull Request #7646 · RustPython/RustPython

Description: This PR fixes a regression from my last islower/isupper patch. Python's Bytes doesn't assume an encoding, so methods like islower should only consider ASCII casing. I updated islower/isupper for UTF-8 and WTF-8 to match CPython exactly. The two functions now use the same properties as CPython and in the exact same way that CPython. I updated the unit tests to pass on Python 3.15. Unicode updates sometimes cause properties to shift. I previously tested everything on Python 3.14, but that lead to failures that I assumed were bugs but were actually due to Unicode differences. For example, U+0295 is a lower case letter in older Unicode versions but is NOT in newer versions. With Python 3.14: assert "\u0295".islower() assert not "\u0295".isupper() With Python 3.15 and RustPython: assert not "\u0295".islower() assert not "\u0295".isupper() assert not "\u0295".istitle() ^It's imperative to test with the latest Python because of Unicode differences. I, uh, spent like three hours trying to fix a bug that didn't exist because I was crosschecking the tests with Python 3.14... Oops. I bumped CI's Python version to 3.15 as well. Reverted because it's more complicated than I thought. Here is CPython's isupper implementation for comparison. CPython's core logic for islower (isupper is basically the same) /* Special case for empty strings */ if (length == 0) Py_RETURN_FALSE; cased = 0; for (i = 0; i < length; i++) { const Py_UCS4 ch = PyUnicode_READ(kind, data, i); if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch)) Py_RETURN_FALSE; else if (!cased && Py_UNICODE_ISLOWER(ch)) cased = 1; } return PyBool_FromLong(cased); RustPython's islower: let mut all_cased = false; for chunk in self.as_bytes().utf8_chunks().map(|c| c.valid()) { if chunk.chars().any(|c| { INVALID::for_char(c) || GeneralCategoryGroup::TitlecaseLetter.contains(GeneralCategory::for_char(c)) }) { return false; } if !all_cased && chunk.chars().any(VALID::for_char) { all_cased = true; } } all_cased They're the same now! 😁 Summary by CodeRabbit Bug Fixes Improved case-checking: text strings use Unicode casedness detection while byte/encoded checks use ASCII-only scanning, preventing misclassification for mixed/invalid byte sequences and certain rare or non-cased characters. Tests Updated and expanded tests for Unicode edge cases, version-dependent casing, non-cased scripts, and bytes case-checking with invalid/mixed sequences.

Open Graph Description: This PR fixes a regression from my last islower/isupper patch. Python's Bytes doesn't assume an encoding, so methods like islower should only consider ASCII casing. I updated islower/isuppe...

X Description: This PR fixes a regression from my last islower/isupper patch. Python&#39;s Bytes doesn&#39;t assume an encoding, so methods like islower should only consider ASCII casing. I updated islowe...

Opengraph URL: https://github.com/RustPython/RustPython/pull/7646

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width