Title: fix: Handle char expansion in islower, isupper by joshuamegnauth54 · Pull Request #7583 · RustPython/RustPython · GitHub
Open Graph Title: fix: Handle char expansion in islower, isupper by joshuamegnauth54 · Pull Request #7583 · RustPython/RustPython
X Title: fix: Handle char expansion in islower, isupper by joshuamegnauth54 · Pull Request #7583 · RustPython/RustPython
Description: Closes: #7526 py_islower and py_isupper need to handle expansions for letter casing. Comparing chars directly can miss edge cases in certain languages. Unfortunately, like the last PR, this allocates to handle potential expansions. I also had to add icu_casemap as a dependency. RustPython is already using parts of icu4x so this doesn't add many transitive dependencies. I figured out how to do it without allocation after working on CodeRabbit's suggestions. I also added more tests based on CodeRabbit which all pass. I cross checked the tests with CPython too. 🥳 The ICU crates use the Writeable trait internally which allows writing to a buffer. If you want to avoid the heap allocations, I can try wrapping a stack allocated buffer that falls back to heap allocations in my next patch. icu_casemap also handles title casing. It may be useful to drop unicode-casing in favor of icu_casemap. icu4x is the official Unicode consortium implementation, so it's robust and will be maintained indefinitely. ^ I wrote this comment earlier when I added icu_casemap. I still think it's relevant so I'll leave it. Currently, there are still title case bugs that may be easier to fix using icu_casemap over unicode-casing. I'll tackle that in another patch. Summary by CodeRabbit Bug Fixes Improved Unicode case detection and classification so string case checks (isupper/islower/istitle) behave correctly for a wider range of international characters and mixed emoji/text sequences. Tests Expanded string tests to cover Unicode edge cases: modifier letters, title/swapcase behavior, emoji with text, and digit-only strings to ensure correct case-reporting.
Open Graph Description: Closes: #7526 py_islower and py_isupper need to handle expansions for letter casing. Comparing chars directly can miss edge cases in certain languages. Unfortunately, like the last PR, this allocat...
X Description: Closes: #7526 py_islower and py_isupper need to handle expansions for letter casing. Comparing chars directly can miss edge cases in certain languages. Unfortunately, like the last PR, this allocat...
Opengraph URL: https://github.com/RustPython/RustPython/pull/7583
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:01aee7b0-39ee-ac1f-649b-51a02205d052 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | DC06:3609E5:2C01B50:3F85BA2:6A60D9ED |
| html-safe-nonce | 52153a6a808918b75c13ff69337227260b56a862ab956338c597f4a6fb7024b1 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEQzA2OjM2MDlFNToyQzAxQjUwOjNGODVCQTI6NkE2MEQ5RUQiLCJ2aXNpdG9yX2lkIjoiMzIyNDA5NzEzMTg2ODUxMDcwMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | f549e0fa00debd4f398130d5d0b2737e09efde0a84fce50f5d824f2dd0bea8ec |
| hovercard-subject-tag | pull_request:3520047168 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/7583/files |
| twitter:image | https://avatars.githubusercontent.com/u/48846352?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/48846352?s=400&v=4 |
| og:image:alt | Closes: #7526 py_islower and py_isupper need to handle expansions for letter casing. Comparing chars directly can miss edge cases in certain languages. Unfortunately, like the last PR, this allocat... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 8302bdfafed4cf34ac9c1d3593108fafa53bc3f86b893907e38fa6bdd74361ac |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | c3c2ff0c76b34772c83c9498a08cefee28f53da0 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width