Title: gh-159184: Do not compare set elements when marshalling a set by gpshead · Pull Request #159196 · python/cpython · GitHub
Open Graph Title: gh-159184: Do not compare set elements when marshalling a set by gpshead · Pull Request #159196 · python/cpython
X Title: gh-159184: Do not compare set elements when marshalling a set by gpshead · Pull Request #159196 · python/cpython
Description: marshal writes the elements of a set in the order of their dumps, so that the output does not depend on hash order. It did that by sorting (dump, element) pairs, which compares the elements whenever two of them have the same dump. That happens for NaNs that are distinct objects, and complex numbers cannot be ordered:
>>> marshal.dumps(frozenset({complex('nan'), complex('nan')}))
TypeError: '<' not supported between instances of 'complex' and 'complex'
>>> code = compile("x in {(1e999-1e999)*1j, (1e999-1e999)*2j}", "m.py", "exec")
>>> marshal.dumps(code) # so no .pyc can be written for this module
TypeError: '<' not supported between instances of 'complex' and 'complex'
Change
flowchart TB
subgraph before["main"]
direction TB
b1["list of (dump, element)"] --> b2["PyList_Sort()"]
b2 --> b3{"equal dumps?"}
b3 -- no --> b4["ordered by dump"]
b3 -- yes --> b5["compares the elements"]
b5 -- complex --> b6["TypeError"]
b5 -- "float NaN" --> b7["neither is smaller:
left in set order"]
end
subgraph after["this PR"]
direction TB
a1["list of (dump, index, element)
index = position in the set"] --> a2["PyList_Sort()"]
a2 --> a3{"equal dumps?"}
a3 -- no --> a4["ordered by dump"]
a3 -- yes --> a5["compares the indexes,
which always differ:
left in set order"]
end
Loading
The output is unchanged wherever marshalling worked before: elements with different dumps are ordered by dump, and float NaNs with equal dumps stay in set order as they did. The only difference is that elements are never compared.
This is the small part of gh-159184 that can be backported. The other part, that the order of elements with equal dumps follows their addresses, needs #159192 and is a separate PR.
This touches the same block as the PR for gh-159185; whichever is merged second needs a one-line conflict resolved.
Tests
test_unorderable_set_items and test_unorderable_set_items_in_code replace test_error_in_set_item, which asserted the TypeError.
Open Graph Description: marshal writes the elements of a set in the order of their dumps, so that the output does not depend on hash order. It did that by sorting (dump, element) pairs, which compares the elements whenev...
X Description: marshal writes the elements of a set in the order of their dumps, so that the output does not depend on hash order. It did that by sorting (dump, element) pairs, which compares the elements whenev...
Opengraph URL: https://github.com/python/cpython/pull/159196
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:fffa1d5e-4094-c52b-6767-78e73036f834 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | 9BB0:5D487:170EBB1:1E8D85F:6B020AF2 |
| html-safe-nonce | 11e0c553c2e056eae883525e1406346a5f95dc3a7260812d0a77ff4f139daf17 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5QkIwOjVENDg3OjE3MEVCQjE6MUU4RDg1Rjo2QjAyMEFGMiIsInZpc2l0b3JfaWQiOiI3NjIxNTI3OTU5NDIxMTkyOTQ2IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 94ca3b4267c02c44f57465e3f59ae8139c41c7cf0a63378a7fdca72afbd47b45 |
| hovercard-subject-tag | pull_request:4822189388 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/python/cpython/pull/159196/files |
| twitter:image | https://avatars.githubusercontent.com/u/68491?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/68491?s=400&v=4 |
| og:image:alt | marshal writes the elements of a set in the order of their dumps, so that the output does not depend on hash order. It did that by sorting (dump, element) pairs, which compares the elements whenev... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width