Title: `exec()`, `eval()` and `-c` keep a lone CR in string literals when the source is a `str` · Issue #9761 · RustPython/RustPython · GitHub
Open Graph Title: `exec()`, `eval()` and `-c` keep a lone CR in string literals when the source is a `str` · Issue #9761 · RustPython/RustPython
X Title: `exec()`, `eval()` and `-c` keep a lone CR in string literals when the source is a `str` · Issue #9761 · RustPython/RustPython
Description: Summary When source code is passed as a str to exec() or eval(), or with rustpython -c, a lone carriage return inside a string literal stays '\r'. CPython reads it as '\n', as it does for CRLF. So code saved with CR-only line endings get...
Open Graph Description: Summary When source code is passed as a str to exec() or eval(), or with rustpython -c, a lone carriage return inside a string literal stays '\r'. CPython reads it as '\n', as it does for CRLF. So ...
X Description: Summary When source code is passed as a str to exec() or eval(), or with rustpython -c, a lone carriage return inside a string literal stays '\r'. CPython reads it as '\n', as it do...
Opengraph URL: https://github.com/RustPython/RustPython/issues/9761
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`exec()`, `eval()` and `-c` keep a lone CR in string literals when the source is a `str`","articleBody":"## Summary\n\nWhen source code is passed as a `str` to `exec()` or `eval()`, or with `rustpython -c`, a lone carriage return inside a string literal stays `'\\r'`. CPython reads it as `'\\n'`, as it does for CRLF. So code saved with CR-only line endings gets different string values and docstrings in RustPython. Docstrings are also left uncleaned, because a docstring with only CR line breaks counts as one line. Within RustPython, the same text also gives different values depending on the call: `exec(text)` keeps the CR, but `exec(text.encode())` and `exec(compile(text, ...))` turn it into `'\\n'`.\n\nThe [language reference (\"Physical lines\")](https://docs.python.org/3/reference/lexical_analysis.html#physical-lines) says: \"Regardless of platform, each of these sequences is replaced by a single ASCII LF (linefeed) character. (This is done even inside string literals.)\"\n\n## Reproducer\n\n```python\n# source saved with old Mac (CR-only) line endings, e.g. read from a zip or with pkgutil.get_data()\ndata = b'\"\"\"Usage:\\r tool run\"\"\"\\rBANNER = \"\"\"line one\\rline two\"\"\"\\r'\ntext = data.decode()\n\nns = {}; exec(data, ns)\nprint(\"exec(bytes) \", ascii(ns[\"__doc__\"]), ascii(ns[\"BANNER\"]))\nns = {}; exec(compile(text, \"\u003cs>\", \"exec\"), ns)\nprint(\"exec(compile(str)) \", ascii(ns[\"__doc__\"]), ascii(ns[\"BANNER\"]))\nns = {}; exec(text, ns)\nprint(\"exec(str) \", ascii(ns[\"__doc__\"]), ascii(ns[\"BANNER\"]))\nprint(\"eval(str) \", ascii(eval('\"\"\"a\\rb\"\"\"')))\n```\n\n## Expected\n\nCPython 3.15.2 (3.13.18 prints the same; 3.12.13 differs only in not dedenting the docstring):\n\n```\nexec(bytes) 'Usage:\\ntool run' 'line one\\nline two'\nexec(compile(str)) 'Usage:\\ntool run' 'line one\\nline two'\nexec(str) 'Usage:\\ntool run' 'line one\\nline two'\neval(str) 'a\\nb'\n```\n\n## Actual\n\nRustPython main (b66534d), same on ef023bc:\n\n```\nexec(bytes) 'Usage:\\ntool run' 'line one\\nline two'\nexec(compile(str)) 'Usage:\\ntool run' 'line one\\nline two'\nexec(str) 'Usage:\\r tool run' 'line one\\rline two'\neval(str) 'a\\rb'\n```\n\n`rustpython -c $'x = \"\"\"a\\rb\"\"\"\\nprint(ascii(x))'` prints `'a\\rb'` (CPython `'a\\nb'`).\n\nOn the `exec(str)` / `eval(str)` / `-c` paths, every literal kind keeps the CR: plain, raw (`r\"\"\"a\u003cCR>b\"\"\"`), bytes (`b\"\"\"...\"\"\"`, `rb\"\"\"...\"\"\"`), f-strings and t-strings (literal parts and a format spec), and a raw string with a backslash before the CR (`r\"a\\\u003cCR>b\"` gives `'a\\\\\\rb'`, CPython `'a\\\\\\nb'`). CRLF is handled correctly on these paths. Everything else is already correct: `exec`/`eval` of bytes, `compile()` of str or bytes, `ast.parse`, `codeop`, importing a CR-only module, running a CR-only script file, and stdin. Statements, comments and line numbers already treat a lone CR as a line break, for both str and bytes source.\n\n## Cause\n\n#9429 added newline normalization (`\\r\\n` and `\\r` to `\\n`) to the source decoder ([compile.rs#L464-L470](https://github.com/RustPython/RustPython/blob/b66534d448f8b94e7874d44ca607c42803c2399f/crates/vm/src/vm/compile.rs#L464-L470)). `compile()` always goes through that decoder, and so do `exec`/`eval` of bytes. Its description already notes that \"direct string paths that bypass the decoder are not changed\". `exec()` and `eval()` with a `str` pass the text straight to `run_code` ([builtins.rs#L640-L650](https://github.com/RustPython/RustPython/blob/b66534d448f8b94e7874d44ca607c42803c2399f/crates/vm/src/stdlib/builtins.rs#L640-L650), [#L591-L601](https://github.com/RustPython/RustPython/blob/b66534d448f8b94e7874d44ca607c42803c2399f/crates/vm/src/stdlib/builtins.rs#L591-L601), [#L701-L708](https://github.com/RustPython/RustPython/blob/b66534d448f8b94e7874d44ca607c42803c2399f/crates/vm/src/stdlib/builtins.rs#L701-L708)), and `-c` goes through `vm.run_string`. Both end in `compile_with_syntax_warning_handler`, which only replaces `\"\\r\\n\"` ([lib.rs#L2409-L2421](https://github.com/RustPython/RustPython/blob/b66534d448f8b94e7874d44ca607c42803c2399f/crates/compiler/src/lib.rs#L2409-L2421)). The parser copies literal text from the source as is ([string.rs#L499-L518](https://github.com/RustPython/ruff/blob/9290a8849542134d83f77e9360e7f0b93897fd93/crates/ruff_python_parser/src/string.rs#L499-L518) in the ruff fork; upstream ruff tracks the same parser behavior as astral-sh/ruff#16318). So any CR that reaches it unnormalized ends up in the value.\n\nI tried changing line 2416 to `source.replace(\"\\r\\n\", \"\\n\").replace('\\r', \"\\n\")` locally. With that change, all the cases above match CPython 3.15. Running the str paths through the same normalization as the decoder would work too.\n\nCPython's own tests do not catch this: `test_source_encoding`'s `test_crlf`/`test_crcrlf` cases go through bytes and script files, and `test_compile.test_other_newlines` uses `compile()`. All of these pass.\n\n\u003csub>Drafted with help from Claude Code; I reviewed and checked it before posting.\u003c/sub>\n","author":{"url":"https://github.com/taxor03","@type":"Person","name":"taxor03"},"datePublished":"2026-11-09T05:50:22Z","url":"https://github.com/RustPython/RustPython/issues/9761"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:cb7de5c6-b085-8d55-f9bb-b3ba6779d8fd |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | BDE4:A71F9:287209E:3331172:6B01D411 |
| html-safe-nonce | e1796afed32c546ee53cc9624ff6999e1a04aa433ac30b898c994afea002e6f8 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCREU0OkE3MUY5OjI4NzIwOUU6MzMzMTE3Mjo2QjAxRDQxMSIsInZpc2l0b3JfaWQiOiI2MzA1NDM1MjY1NjQ1MjcwMDMzIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 43325b06c0b1e0961d60633ec9111b169ade769a5a0888e5cbfb4b0b1f66242e |
| hovercard-subject-tag | issue:6280745418 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/RustPython/RustPython/9761/issue_layout |
| twitter:image | https://opengraph.githubassets.com/0e844dce7172d392047a25cf0c5abb68878ca94c26882bd1abd2382eb593abfe/RustPython/RustPython/issues/9761 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/0e844dce7172d392047a25cf0c5abb68878ca94c26882bd1abd2382eb593abfe/RustPython/RustPython/issues/9761 |
| og:image:alt | Summary When source code is passed as a str to exec() or eval(), or with rustpython -c, a lone carriage return inside a string literal stays '\r'. CPython reads it as '\n', as it does for CRLF. So ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | taxor03 |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width