Title: Use Unicode properties for alnum, alpha, etc. by joshuamegnauth54 · Pull Request #7626 · RustPython/RustPython · GitHub
Open Graph Title: Use Unicode properties for alnum, alpha, etc. by joshuamegnauth54 · Pull Request #7626 · RustPython/RustPython
X Title: Use Unicode properties for alnum, alpha, etc. by joshuamegnauth54 · Pull Request #7626 · RustPython/RustPython
Description: Rust and Python differ in which properties they use for alphanumeric, numeric, et cetera. Both languages list which properties are used which makes it easy to mimic Python's behavior in Rust. My previous patch was a bit shortsighted because I filtered out combining characters from is_alphanumeric. Using properties is exact and also much cleaner. It also covers edge cases that my initial approach missed. Besides isalnum, I also fixed isnumeric and isdigit in the same way by using properties. This contributes a bit to #7527. Unicode issues for isalnum etc. should be solved now (and permanently too!!). Title casing looks harder to fix, but the basic solution of using properties more should apply there too. Also, the RegEx engine can benefit from using properties more too, but I'm terrible at RegEx so I didn't really try to fix issues there. 😆 For extra reading, here is Python's docs which lists the categories used. Look at isalpha(), for example. Summary by CodeRabbit Bug Fixes Improved Unicode classification for string methods (isalnum, isalpha, isdigit, isnumeric) for more accurate behavior across international characters and special Unicode categories. Tests Added/expanded tests covering edge cases: superscript digits, fractions, Roman numerals, and additional non-Latin characters to validate classification behavior.
Open Graph Description: Rust and Python differ in which properties they use for alphanumeric, numeric, et cetera. Both languages list which properties are used which makes it easy to mimic Python's behavior in Rust. M...
X Description: Rust and Python differ in which properties they use for alphanumeric, numeric, et cetera. Both languages list which properties are used which makes it easy to mimic Python's behavior in Rus...
Opengraph URL: https://github.com/RustPython/RustPython/pull/7626
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:464213b0-c02c-3a95-2d0d-16c5c4a75009 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 8DD0:12CDD6:71BEEB:992F9E:6A610CCE |
| html-safe-nonce | 564ac8f9329e556122af6fad6691753290793ce5345718757b20d1586f9c9f2a |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4REQwOjEyQ0RENjo3MUJFRUI6OTkyRjlFOjZBNjEwQ0NFIiwidmlzaXRvcl9pZCI6IjYzODYyOTA0MTcyNzg3MTcxMzUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 5096105eb53f34672d62339aeebfd81103241b7b52dc043f5daab563ade5841c |
| hovercard-subject-tag | pull_request:3550553852 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/7626/files |
| twitter:image | https://avatars.githubusercontent.com/u/48846352?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/48846352?s=400&v=4 |
| og:image:alt | Rust and Python differ in which properties they use for alphanumeric, numeric, et cetera. Both languages list which properties are used which makes it easy to mimic Python's behavior in Rust. M... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 86be97fd69ac233b46e43a5eed04ceac6498abe1efaf1af3b9573eb216019c05 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | f80280886562a52fe1f6b9cab9f9c1dc3b9a78a2 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width