René's URL Explorer Experiment


Title: gh-148286: Fix undefined behaviour in the ctypes bit field accessors by matthiasgoergens · Pull Request #154912 · python/cpython · GitHub

Open Graph Title: gh-148286: Fix undefined behaviour in the ctypes bit field accessors by matthiasgoergens · Pull Request #154912 · python/cpython

X Title: gh-148286: Fix undefined behaviour in the ctypes bit field accessors by matthiasgoergens · Pull Request #154912 · python/cpython

Description: BIT_MASK, GET_BITFIELD and SET in Modules/_ctypes/cfield.c do their bit manipulation in the bit field's own type. When that type is signed, both shifting a negative value left and shifting a one into the sign bit are undefined behaviour, which is why Tools/ubsan/suppressions.txt carries two file-level entries for this file. Removing those entries and running test_ctypes under UBSAN_OPTIONS=halt_on_error=1 aborts immediately at cfield.c:644 in i64_set. Collecting rather than halting shows diagnostics at three distinct lines, so the suppression was covering rather more than its comment documents: cfield.c:644: left shift of 1 by 63 places cannot be represented in type 'int64_t' cfield.c:644: left shift of negative value -9223372036854775793 cfield.c:640: signed integer overflow: -2147483648 - 1 cannot be represented in type 'int' cfield.c:640: left shift of 1 by 31 places cannot be represented in type 'int32_t' cfield.c:636: left shift of negative value -32768 This does the arithmetic in uint64_t — the widest type the accessors support — and converts back to the field's own type at the end. BIT_MASK becomes LOW_BITS_MASK(), which no longer needs a type argument now that the mask is always computed unsigned; GET_BITFIELD gains the field type so it can still sign-extend signed fields, and is wrapped in do { } while (0). Since this rewrites the arithmetic of every integer bit field getter and setter, I checked it exhaustively against the old macros rather than relying on review alone. For every fixed-width type from int8_t to uint64_t, every num_bits, every low_bit, ten storage bit patterns and twenty stored values — 1,167,600 cases — the old and new expressions produce identical results, under both gcc and clang. The same sweep under -fsanitize=undefined -fno-sanitize-recover=all is clean, where the old macros produce eleven distinct diagnostics. With the two Modules/_ctypes/cfield.c entries removed from the suppressions file, test_ctypes passes under UBSAN_OPTIONS=halt_on_error=1. Provenance This supersedes #96925, my 2022 attempt at the same bug, which I have now closed. That version worked around the undefined shifts with multiplication, and @mdickinson pushed back on it in review — both because the intent is to move bits rather than to do arithmetic, and because multiplication drags the usual arithmetic conversions in on top of the integer promotions, making the result harder to reason about. He proposed doing the bit manipulation in unsigned types and sign-extending afterwards, and sketched the (v ^ sign_bit) - sign_bit idiom used here. So the approach in this PR is his rather than mine, four years late. It is also a more complete fix than the old one: #96925 only added an assertion to BIT_MASK and left (type)1 << (NUM_BITS(size) - 1) in place, which is itself undefined for a full-width signed field — that is the left shift of 1 by 63 places diagnostic currently recorded in the suppressions file, so the original patch would not have cleared its own suppression. Thanks, Mark. Issue: gh-148286

Open Graph Description: BIT_MASK, GET_BITFIELD and SET in Modules/_ctypes/cfield.c do their bit manipulation in the bit field's own type. When that type is signed, both shifting a negative value left and shifting a on...

X Description: BIT_MASK, GET_BITFIELD and SET in Modules/_ctypes/cfield.c do their bit manipulation in the bit field&#39;s own type. When that type is signed, both shifting a negative value left and shifting ...

Opengraph URL: https://github.com/python/cpython/pull/154912

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:a769cb0c-c35d-cd9b-5b13-f23a9f3208d5
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idBA7E:1F82:665845:85C76E:6B024F41
html-safe-nonce941234e49b858095f4b0516e0ba9ac04f05317de1904729e09b5724ccbc29086
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCQTdFOjFGODI6NjY1ODQ1Ojg1Qzc2RTo2QjAyNEY0MSIsInZpc2l0b3JfaWQiOiIyMzQ5MjQ2OTcxMjQ5ODM5OTM3IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmacce6778f87ec661df19c64743e40b71df06379ad928f20ea5025999c49e2b79ba
hovercard-subject-tagpull_request:4167685166
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/154912/files
twitter:imagehttps://avatars.githubusercontent.com/u/57026?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/57026?s=400&v=4
og:image:altBIT_MASK, GET_BITFIELD and SET in Modules/_ctypes/cfield.c do their bit manipulation in the bit field's own type. When that type is signed, both shifting a negative value left and shifting a on...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/154912/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F154912%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F154912%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/154912/files
Reloadhttps://github.com/python/cpython/pull/154912/files
Reloadhttps://github.com/python/cpython/pull/154912/files
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
matthiasgoergenshttps://github.com/matthiasgoergens
python:mainhttps://github.com/python/cpython/tree/main
matthiasgoergens:ctypes-bitfield-ubhttps://github.com/matthiasgoergens/cpython/tree/ctypes-bitfield-ub
Conversation 2 https://github.com/python/cpython/pull/154912
Commits 3 https://github.com/python/cpython/pull/154912/commits
Checks 52 https://github.com/python/cpython/pull/154912/checks
Files changed https://github.com/python/cpython/pull/154912/files
Please reload this pagehttps://github.com/python/cpython/pull/154912/files
gh-148286: Fix undefined behaviour in the ctypes bit field accessors https://github.com/python/cpython/pull/154912/files#top
Show all changes 3 commits https://github.com/python/cpython/pull/154912/files
721f6ce gh-148286: Fix undefined behaviour in the ctypes bit field accessors matthiasgoergens Jul 30, 2026 https://github.com/python/cpython/pull/154912/commits/721f6ce70d027339f0f0da9352c2557d61720137
e291f3b Trim the NEWS entry to the user-visible statement matthiasgoergens Aug 1, 2026 https://github.com/python/cpython/pull/154912/commits/e291f3bc79ec7f35587dfb1aedeed640836429d1
58d58b0 Merge branch 'main' into ctypes-bitfield-ub matthiasgoergens Oct 21, 2026 https://github.com/python/cpython/pull/154912/commits/58d58b022e9a5bcfd39a625596fb99ca4f5a28e1
Clear filters https://github.com/python/cpython/pull/154912/files
Please reload this pagehttps://github.com/python/cpython/pull/154912/files
Please reload this pagehttps://github.com/python/cpython/pull/154912/files
2026-07-30-12-00-00.gh-issue-148286.Kx7Qm2.rst https://github.com/python/cpython/pull/154912/files#diff-8314d6f7836ffe3452781936f8c498c2d9315b2dba36dce3eccdb6389615f26b
cfield.c https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
suppressions.txt https://github.com/python/cpython/pull/154912/files#diff-702de60c74ca92f35f0fd6824405a53ef373113035944c92f2f9596e5a8b99e7
Misc/NEWS.d/next/Library/2026-07-30-12-00-00.gh-issue-148286.Kx7Qm2.rsthttps://github.com/python/cpython/pull/154912/files#diff-8314d6f7836ffe3452781936f8c498c2d9315b2dba36dce3eccdb6389615f26b
View file https://github.com/python/cpython/blob/58d58b022e9a5bcfd39a625596fb99ca4f5a28e1/Misc/NEWS.d/next/Library/2026-07-30-12-00-00.gh-issue-148286.Kx7Qm2.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/154912/{{ revealButtonHref }}
Modules/_ctypes/cfield.chttps://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
View file https://github.com/python/cpython/blob/58d58b022e9a5bcfd39a625596fb99ca4f5a28e1/Modules/_ctypes/cfield.c
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/154912/{{ revealButtonHref }}
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
https://github.com/python/cpython/pull/154912/files#diff-2952ec5c776412a52a7a69a6cf735e480d7454e3862d2b8e02f34c927f600aa1
Tools/ubsan/suppressions.txthttps://github.com/python/cpython/pull/154912/files#diff-702de60c74ca92f35f0fd6824405a53ef373113035944c92f2f9596e5a8b99e7
View file https://github.com/python/cpython/blob/58d58b022e9a5bcfd39a625596fb99ca4f5a28e1/Tools/ubsan/suppressions.txt
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/154912/{{ revealButtonHref }}
https://github.com/python/cpython/pull/154912/files#diff-702de60c74ca92f35f0fd6824405a53ef373113035944c92f2f9596e5a8b99e7
Please reload this pagehttps://github.com/python/cpython/pull/154912/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.