Title: tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char · Issue #53921 · python/cpython · GitHub
Open Graph Title: tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char · Issue #53921 · python/cpython
X Title: tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char · Issue #53921 · python/cpython
Description: BPO 9712 Nosy @terryjreedy, @benjaminp, @florentx Superseder bpo-24194: Make tokenize recognize Other_ID_Start and Other_ID_Continue chars Note: these values reflect the state of the issue at the time it was migrated and might not reflec...
Open Graph Description: BPO 9712 Nosy @terryjreedy, @benjaminp, @florentx Superseder bpo-24194: Make tokenize recognize Other_ID_Start and Other_ID_Continue chars Note: these values reflect the state of the issue at the t...
X Description: BPO 9712 Nosy @terryjreedy, @benjaminp, @florentx Superseder bpo-24194: Make tokenize recognize Other_ID_Start and Other_ID_Continue chars Note: these values reflect the state of the issue at the t...
Opengraph URL: https://github.com/python/cpython/issues/53921
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char","articleBody":"BPO | [9712](https://bugs.python.org/issue9712)\n--- | :---\nNosy | @terryjreedy, @benjaminp, @florentx\nSuperseder | \u003cli\u003ebpo-24194: Make tokenize recognize Other_ID_Start and Other_ID_Continue chars\u003c/li\u003e\n\n\u003csup\u003e*Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.*\u003c/sup\u003e\n\n\u003cdetails\u003e\u003csummary\u003eShow more details\u003c/summary\u003e\u003cp\u003e\n\nGitHub fields:\n```python\nassignee = None\nclosed_at = \u003cDate 2010-08-30.14:41:43.763\u003e\ncreated_at = \u003cDate 2010-08-30.07:42:55.735\u003e\nlabels = ['type-bug']\ntitle = 'tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char'\nupdated_at = \u003cDate 2018-03-15.00:18:48.250\u003e\nuser = 'https://github.com/florentx'\n```\n\nbugs.python.org fields:\n```python\nactivity = \u003cDate 2018-03-15.00:18:48.250\u003e\nactor = 'terry.reedy'\nassignee = 'none'\nclosed = True\nclosed_date = \u003cDate 2010-08-30.14:41:43.763\u003e\ncloser = 'benjamin.peterson'\ncomponents = []\ncreation = \u003cDate 2010-08-30.07:42:55.735\u003e\ncreator = 'flox'\ndependencies = []\nfiles = []\nhgrepos = []\nissue_num = 9712\nkeywords = []\nmessage_count = 5.0\nmessages = ['115201', '115218', '240544', '313846', '313847']\nnosy_count = 4.0\nnosy_names = ['terry.reedy', 'benjamin.peterson', 'flox', 'Joshua.Landau']\npr_nums = []\npriority = 'normal'\nresolution = 'fixed'\nstage = 'test needed'\nstatus = 'closed'\nsuperseder = '24194'\ntype = 'behavior'\nurl = 'https://bugs.python.org/issue9712'\nversions = ['Python 3.1', 'Python 3.2', 'Python 3.4']\n```\n\n\u003c/p\u003e\u003c/details\u003e\n","author":{"url":"https://github.com/florentx","@type":"Person","name":"florentx"},"datePublished":"2010-08-30T07:42:56.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/53921/cpython/issues/53921"}
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:92a35f7a-84a7-0544-1119-16ea2e87c998 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | A4B6:233E0B:22640D1:30E475F:6A50E7EC |
| html-safe-nonce | 86eba5eae605992f0d4453e33f7bcbcdd504b0d629bad20eb9c9a9478fb295c9 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBNEI2OjIzM0UwQjoyMjY0MEQxOjMwRTQ3NUY6NkE1MEU3RUMiLCJ2aXNpdG9yX2lkIjoiNjIxNDQ0NDEzMjkwNTA1MjE0MCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 4fae489a6e4e78fd2f77ff90f0d53a35cfce309262ee25923dd27ce0e66d3cab |
| hovercard-subject-tag | issue:1198859375 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/53921/issue_layout |
| twitter:image | https://opengraph.githubassets.com/dea784af2a6ea3e9a622bdc75d2e633fdb0af247ad1c7a09c6471f20053cff47/python/cpython/issues/53921 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/dea784af2a6ea3e9a622bdc75d2e633fdb0af247ad1c7a09c6471f20053cff47/python/cpython/issues/53921 |
| og:image:alt | BPO 9712 Nosy @terryjreedy, @benjaminp, @florentx Superseder bpo-24194: Make tokenize recognize Other_ID_Start and Other_ID_Continue chars Note: these values reflect the state of the issue at the t... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | florentx |
| hostname | github.com |
| expected-hostname | github.com |
| None | 72b13e0e8d0319acdd27a145cfe73f4f06c791713d0ac4690e41fb68ad28cac0 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | fb19d617b534959801b4b7453938ecf1dfa22131 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width