Title: Inconsistent `ast.parse` result for `python<3.9` and PEP614 · Issue #96427 · python/cpython · GitHub
Open Graph Title: Inconsistent `ast.parse` result for `python<3.9` and PEP614 · Issue #96427 · python/cpython
X Title: Inconsistent `ast.parse` result for `python<3.9` and PEP614 · Issue #96427 · python/cpython
Description: PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/ Right now, ast.parse with feature_version=(3, 8) and bellow behave differently from specified python versions themself. For example, main branch is abl...
Open Graph Description: PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/ Right now, ast.parse with feature_version=(3, 8) and bellow behave differently from specified python versions t...
X Description: PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/ Right now, ast.parse with feature_version=(3, 8) and bellow behave differently from specified python versions t...
Opengraph URL: https://github.com/python/cpython/issues/96427
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Inconsistent `ast.parse` result for `python\u003c3.9` and PEP614","articleBody":"PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/\r\n\r\nRight now, `ast.parse` with `feature_version=(3, 8)` and bellow behave differently from specified python versions themself.\r\n\r\nFor example, `main` branch is able to parse this code:\r\n\r\n```python\r\nPython 3.12.0a0 (heads/main-dirty:e860e521ec, Aug 28 2022, 21:36:18) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import ast\r\n>>> code = \"\"\"\r\n... @decs[0].func\r\n... def some(): pass\r\n... \"\"\"\r\n>>> ast.parse(code)\r\n\u003cast.Module object at 0x102db8150>\r\n>>> ast.parse(code, feature_version=(3, 8))\r\n\u003cast.Module object at 0x102d9d320>\r\n```\r\n\r\nWhile `python==3.8.x` is not able to:\r\n\r\n```python\r\nPython 3.8.9 (default, May 3 2021, 12:15:25) \r\n[Clang 11.0.0 (clang-1100.0.33.16)] on darwin\r\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\r\n>>> import ast\r\n>>> code = \"\"\"\r\n... @decs[0].func\r\n... def some(): pass\r\n... \"\"\"\r\n>>> ast.parse(code)\r\nTraceback (most recent call last):\r\n File \"\u003cstdin>\", line 1, in \u003cmodule>\r\n File \"/Users/sobolev/.pyenv/versions/3.8.9/lib/python3.8/ast.py\", line 47, in parse\r\n return compile(source, filename, mode, flags,\r\n File \"\u003cunknown>\", line 2\r\n @decs[0].func\r\n ^\r\nSyntaxError: invalid syntax\r\n```\r\n\r\nWhen looking at `we-love-parsers/pegen` - it does have a version check for this: https://github.com/we-like-parsers/pegen/blame/599255a5ba3930e86367cd3fc78fae91ee184dd5/data/python.gram#L660-L662\r\n\r\n## Pros and Cons\r\n\r\nI think that there are several key points to this, pros:\r\n1. Correct parsing of older versions\r\n2. Consistency with other parser like `we-love-parsers/pegen`\r\n\r\nHowever, there are some serious cons to it:\r\n1. Grammar is made quite significanly more complex\r\n2. There might be new bugs in new behaviour: locations / performance / etc\r\n\r\nI think it is up to @pablogsal and other core devs to decide.\r\n\r\n## Proposed solution\r\n\r\nI've modified an existing `we-love-parsers/pegen`'s solution a bit, here's what it looks like:\r\n\r\n```\r\ndecorators[asdl_expr_seq*]: a[asdl_expr_seq*]=decorator+\r\ndecorator[expr_ty]:\r\n | a=('@' f=dec_maybe_call NEWLINE { f }) { a }\r\n | a=('@' f=named_expression NEWLINE { f }) {\r\n CHECK_VERSION(expr_ty, 9, \"Relaxed decorator syntax is\", a) }\r\ndec_maybe_call[expr_ty]:\r\n | a=dec_primary '(' b=[arguments] ')' {\r\n _PyAST_Call(a,\r\n (b) ? ((expr_ty) b)->v.Call.args : NULL,\r\n (b) ? ((expr_ty) b)->v.Call.keywords : NULL,\r\n EXTRA) }\r\n | dec_primary\r\ndec_primary[expr_ty]:\r\n | a=dec_primary '.' b=NAME { _PyAST_Attribute(a, b->v.Name.id, Load, EXTRA) }\r\n | a=NAME\r\n```\r\n\r\nI will send a PR with all the changes / tests / auto-generated files, so we can take a closer look and decide whether we want this or not.","author":{"url":"https://github.com/sobolevn","@type":"Person","name":"sobolevn"},"datePublished":"2022-08-30T13:49:10Z","url":"https://github.com/python/cpython/issues/96427"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:81ee9067-fb2d-50c5-971d-14eef5405df2 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | C760:11BB24:2B41419:3AFCDE3:6B005198 |
| html-safe-nonce | 864e776616a74d90d7832abd0c902bb52cd34f97e2bf39f40ac2c35f85c97a61 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDNzYwOjExQkIyNDoyQjQxNDE5OjNBRkNERTM6NkIwMDUxOTgiLCJ2aXNpdG9yX2lkIjoiNTc3MDAzMjkxMDMzNzg1NTg5NiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 28c987e12159521797c0c7e64798293701c646224cedfd4b9bf491958a1f66a9 |
| hovercard-subject-tag | issue:1355804760 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/96427/issue_layout |
| twitter:image | https://opengraph.githubassets.com/5809f689a1a6861a90c9ada6c8bf53a79fc566b130b48777ae41a8e6088d7729/python/cpython/issues/96427 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/5809f689a1a6861a90c9ada6c8bf53a79fc566b130b48777ae41a8e6088d7729/python/cpython/issues/96427 |
| og:image:alt | PEP614 introduces relaxed decorator grammar in 3.9: https://peps.python.org/pep-0614/ Right now, ast.parse with feature_version=(3, 8) and bellow behave differently from specified python versions t... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | sobolevn |
| hostname | github.com |
| expected-hostname | github.com |
| None | f420f4996e6b1ff1ccd6d7683f8134f4b5f0f8a50124c210dbfe7490e5644abb |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 26e00df917aba0f128135dc073ea0c5e33a70d40 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width