Title: Tokenizer/untokenizer seems to break when linting multiline strings with variables/emojis · Issue #120377 · python/cpython · GitHub
Open Graph Title: Tokenizer/untokenizer seems to break when linting multiline strings with variables/emojis · Issue #120377 · python/cpython
X Title: Tokenizer/untokenizer seems to break when linting multiline strings with variables/emojis · Issue #120377 · python/cpython
Description: Bug report Bug description: Linting this file with the latest versions of pyupgrade and docformatter on Python 3.12.4: def test(test_arg1): content = f"""
\n \u003ctr>\n \u003ctd style=\"width: 100%; height: 100%;\">🔗 This is a test {test_arg1} \u003c/td>\n \u003c/tr>\n \u003c/table>\n \"\"\"\n return content\n```\nseems to break:\n```\ndef test(test_arg1):\n content = f\"\"\"\n \u003ctable style=\"width: 100%; height: 100%; font-size: 1.1em;\">\n \u003ctr>\n \u003ctd style=\"width: 100%; height: 100%;\">🔗 This is a test {te{test_arg1} \u003c/td>\n \u003c/tr>\n \u003c/table>\n \"\"\"\n return content\n```\n\nI filed https://github.com/PyCQA/docformatter/issues/282 and https://github.com/asottile/pyupgrade/issues/949; the maintainer of pyupgrade directed me here. I wasn't able to find a duplicate f-string or tokenizer issue, so hoping this is of help - thanks!\n\n\n### CPython versions tested on:\n\n3.12\n\n### Operating systems tested on:\n\nLinux, macOS","author":{"url":"https://github.com/hemanthkini","@type":"Person","name":"hemanthkini"},"datePublished":"2024-06-12T00:13:59Z","url":"https://github.com/python/cpython/issues/120377"}
react-profiling 0 react-import-map react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 route-pattern /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) route-controller voltron_issues_fragments route-action issue_layout fetch-nonce v2:d90b628e-8db5-6c4b-ead4-3661d98f8b8f current-catalog-service-hash 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 request-id B03E:302FC:EFE49F:1413D29:6B005896 html-safe-nonce ee0ea2ec998e024ea8e91cf87c98f3714e4d93d335e210540fb716ce4eca6cfc visitor-payload eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMDNFOjMwMkZDOkVGRTQ5RjoxNDEzRDI5OjZCMDA1ODk2IiwidmlzaXRvcl9pZCI6IjkxODE4NzcwNjY4MTgxNDg1MDIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== visitor-hmac 5fe6aed3dfa62b3747710fde9c27c14d6400f2c297830e3abfbda6d71bcbf262 hovercard-subject-tag issue:2347514164 github-keyboard-shortcuts repository,issues,copilot google-site-verification Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I octolytics-url https://collector.github.com/github/collect analytics-location / fb:app_id 1401488693436528 apple-itunes-app app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/120377/issue_layout twitter:image https://opengraph.githubassets.com/dd2059d7d543bf04d3720ef2c3a12db3fbc57b785ea276d6231ac86c1b76face/python/cpython/issues/120377 twitter:card summary_large_image og:image https://opengraph.githubassets.com/dd2059d7d543bf04d3720ef2c3a12db3fbc57b785ea276d6231ac86c1b76face/python/cpython/issues/120377 og:image:alt Bug report Bug description: Linting this file with the latest versions of pyupgrade and docformatter on Python 3.12.4: def test(test_arg1): content = f"""