Title: enumerate's are slow · Issue #121380 · python/cpython · GitHub
Open Graph Title: enumerate's are slow · Issue #121380 · python/cpython
X Title: enumerate's are slow · Issue #121380 · python/cpython
Description: Performance Improvement Using enumerate(...) in for loops are slow and there are at least +250 enumerate's in CPython Let's take for example this for loop with enumerate: cpython/Lib/genericpath.py Lines 116 to 118 in 5f660e8 for i, c in...
Open Graph Description: Performance Improvement Using enumerate(...) in for loops are slow and there are at least +250 enumerate's in CPython Let's take for example this for loop with enumerate: cpython/Lib/genericpath.py...
X Description: Performance Improvement Using enumerate(...) in for loops are slow and there are at least +250 enumerate's in CPython Let's take for example this for loop with enumerate: cpython/Lib/generi...
Opengraph URL: https://github.com/python/cpython/issues/121380
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"enumerate's are slow","articleBody":"# Performance Improvement\r\n\r\nUsing `enumerate(...)` in `for` loops are slow and there are at least +250 `enumerate`'s in CPython\r\n\r\nLet's take for example this `for` loop with `enumerate`:\r\n\r\nhttps://github.com/python/cpython/blob/5f660e8e2ca3acfb89ccbdd990f072149b6baa6a/Lib/genericpath.py#L116-L118\r\n\r\nIf you remove `enumerate`, this method will run 5% to 7% faster:\r\n\r\n```python\r\n i = 0\r\n for c in s1:\r\n if c != s2[i]:\r\n return s1[:i]\r\n i += 1\r\n```\r\n\r\n Complete performance test script, basically I copied twice the original `commonprefix(m)` and then I modified the last one:\r\n\r\n```python\r\nimport os\r\nimport time\r\n\r\n# From file CPython/Lib/genericpath.py\r\n\r\n# Return the longest prefix of all list elements.\r\ndef old_commonprefix(m):\r\n \"Given a list of pathnames, returns the longest common leading component\"\r\n if not m: return ''\r\n # Some people pass in a list of pathname parts to operate in an OS-agnostic\r\n # fashion; don't try to translate in that case as that's an abuse of the\r\n # API and they are already doing what they need to be OS-agnostic and so\r\n # they most likely won't be using an os.PathLike object in the sublists.\r\n if not isinstance(m[0], (list, tuple)):\r\n m = tuple(map(os.fspath, m))\r\n s1 = min(m)\r\n s2 = max(m)\r\n for i, c in enumerate(s1):\r\n if c != s2[i]:\r\n return s1[:i]\r\n return s1\r\n\r\n# Return the longest prefix of all list elements.\r\ndef commonprefix(m):\r\n \"Given a list of pathnames, returns the longest common leading component\"\r\n if not m: return ''\r\n # Some people pass in a list of pathname parts to operate in an OS-agnostic\r\n # fashion; don't try to translate in that case as that's an abuse of the\r\n # API and they are already doing what they need to be OS-agnostic and so\r\n # they most likely won't be using an os.PathLike object in the sublists.\r\n if not isinstance(m[0], (list, tuple)):\r\n m = tuple(map(os.fspath, m))\r\n s1 = min(m)\r\n s2 = max(m)\r\n i = 0\r\n for c in s1:\r\n if c != s2[i]:\r\n return s1[:i]\r\n i += 1\r\n return s1\r\n\r\ndef perfomance(func):\r\n start = time.time()\r\n for _ in range(1000000):\r\n func([\"/usr/lib\", \"/usr/local/lib\", \"/usr/local/bin\", \"/usr/bin\", \"/usr/local/share\"])\r\n end = time.time()\r\n return end - start\r\n\r\nold_perfomance = perfomance(old_commonprefix)\r\nnew_perfomance = perfomance(commonprefix)\r\nperc = (old_perfomance / new_perfomance) * 100\r\n\r\nprint(f\"with enumerate: {old_perfomance} secs\")\r\nprint(f\"without enumerate: {new_perfomance} secs\")\r\nprint(f\"New version is {perc:.2f}% faster\")\r\n```\r\n\r\nYou will get this performance improvement:\r\n\r\n```\r\nwith enumerate: 0.8863015174865723 secs\r\nwithout enumerate: 0.8414063453674316 secs\r\nNew version is 105.34% faster\r\n```\r\n\r\nRunning this commands, you can count and find all `for` loops with `enumerate` in `/Lib` for example\r\n```bash\r\n$ grep -r \"in enumerate\" ./Lib | wc -l\r\n$ grep -rl \"in enumerate\" ./Lib\r\n```\r\n\r\nIf this is useful for CPython, I can create specific issues+PR replacing `enumerate(...)`'s.\r\n\r\n### Has this already been discussed elsewhere?\r\n\r\nNo response given\r\n\r\n### Links to previous discussion of this feature:\r\n\r\n_No response_","author":{"url":"https://github.com/LimaBD","@type":"Person","name":"LimaBD"},"datePublished":"2024-07-04T20:53:20Z","url":"https://github.com/python/cpython/issues/121380"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:8bec6bff-18d1-c56a-3d9a-9ea4f0d96f70 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | DC66:16E4ED:1325689:17F4481:6B024A59 |
| html-safe-nonce | 50af181663b23a37273952af3c950405b4b5c8a9b046daa23b4b45d254700585 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEQzY2OjE2RTRFRDoxMzI1Njg5OjE3RjQ0ODE6NkIwMjRBNTkiLCJ2aXNpdG9yX2lkIjoiNDkyMDM5NjU2NTY3MzI5ODUyMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | d836650f0f32401cf4d37db1c76b6dac2158be487866f9122614de36a23f8887 |
| hovercard-subject-tag | issue:2391454115 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/121380/issue_layout |
| twitter:image | https://opengraph.githubassets.com/66058d131f04f2e4ec928c1e496123508813cd41db94f8646ccffa359265ae10/python/cpython/issues/121380 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/66058d131f04f2e4ec928c1e496123508813cd41db94f8646ccffa359265ae10/python/cpython/issues/121380 |
| og:image:alt | Performance Improvement Using enumerate(...) in for loops are slow and there are at least +250 enumerate's in CPython Let's take for example this for loop with enumerate: cpython/Lib/genericpath.py... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | LimaBD |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width