René's URL Explorer Experiment


Title: New sysconfig API 🤞 (meta-issue) · Issue #103480 · python/cpython · GitHub

Open Graph Title: New sysconfig API 🤞 (meta-issue) · Issue #103480 · python/cpython

X Title: New sysconfig API 🤞 (meta-issue) · Issue #103480 · python/cpython

Description: Feature or enhancement Provide new syconfig API that is aligned to how things have evolved. Fixes GH-99942 Fixes GH-99560 Pitch The sysconfig module was added in Python 3.2, over 10 years ago, and though there were some minor changes and...

Open Graph Description: Feature or enhancement Provide new syconfig API that is aligned to how things have evolved. Fixes GH-99942 Fixes GH-99560 Pitch The sysconfig module was added in Python 3.2, over 10 years ago, and ...

X Description: Feature or enhancement Provide new syconfig API that is aligned to how things have evolved. Fixes GH-99942 Fixes GH-99560 Pitch The sysconfig module was added in Python 3.2, over 10 years ago, and ...

Opengraph URL: https://github.com/python/cpython/issues/103480

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"New sysconfig API 🤞 (meta-issue)","articleBody":"# Feature or enhancement\r\n\r\nProvide new `syconfig` API that is aligned to how things have evolved.\r\n\r\nFixes GH-99942\r\nFixes GH-99560\r\n\u003c!--\r\nFixes GH-77620? (XXX: python-config.py should be part of the stdlib)\r\n-->\r\n\r\n# Pitch\r\n\r\nThe `sysconfig` module was added in Python 3.2, over 10 years ago, and though there were some minor changes and additions, its API stayed mostly the same. Since then things have evolved, with new platform support (eg. WASM) and features being added, and we are now at a point where the current design is not great and has been the source of several issues, especially since the `distutils` deprecation (see GH-99942 for eg.).\r\n\r\nChanging the existing API to better represent modern Python installations would cause a lot of breakage on existing code. Considering that, on top of several of its shortcomings, we think the best step at the moment would be to introduce a new API and mark part of the existing one as either deprecated, or pending deprecation, depending how development goes and on feedback from the impacted parties.\r\n\r\nHere are the main points we'd like to improve on:\r\n\r\n1) Users currently rely on [`sysconfig.get_config_vars`](https://docs.python.org/3/library/sysconfig.html) for **a lot** of things\r\n\r\nOn Posix, `sysconfig.get_config_vars` exports all the variables present in the installation's `Makefile`, and on Windows, it exports a handful of selected variables.\r\n\r\nhttps://github.com/python/cpython/blob/52f96d3ea39fea4c16e26b7b10bd2db09726bd7c/Lib/sysconfig.py#L540-L556\r\n\r\nThere are several issues with this, the main being that the exported variables are not documented, so there are no guarantees about what is information is gonna be available, or the variable names.\r\n\r\nWhile this is a good escape-hatch mechanism, it is a very poor option for the intended interface for key information. We should be exporting that information in a proper API.\r\n\r\n2) The installation paths model is outdated\r\n\r\nThere are a couple of issues and things to improve here here:\r\n- A scheme should be able to not provide non-critial paths (`scripts` for eg.)\r\n- We should introduce the concept of writable and read-only paths (users should not install to any path in the installed base, like `include`, for eg.)\r\n- We should introduce the concept of active location schemes (eg. `posix_user` should not be available on virtual environments)\r\n  - This would extend to vendor schemes outside the default environment, if we ever get around to standardize that\r\n\r\n**Discussion on this should go to GH-103482**\r\n\r\n3) Cross-compilation support is very poor\r\n\r\nCross-compilation at the moment is done by patching several aspects of Python, with the `sysconfig` module being the main thing. We provide several undocumented features to support this, like the allowing the project base path and sysconfig data module name to be overwritten via the `_PYTHON_PROJECT_BASE` and `_PYTHON_SYSCONFIGDATA_NAME` environment variables, respectively.\r\n\r\nWe believe one of the reasons that plays into users choosing to support cross-compilation by monkey patching Python, instead of directly adding cross-compilation support to the 3rd party building code, is because with the current `sysconfig` design, a lot of custom code would have to be added to properly support cross builds.\r\n\r\nOur proposal to solve this is to make the new API standalone, meaning it would provide all the information users may need for binary interfacing (eg. building extension modules) and similar applications. On top of this, we'd shift the design from using module functions to data holder objects, meaning all the relevant details of the Python installation would be represented by just a couple objects. Users that adopt this new API should then be able to support the cross build by simply changing the data objects for ones that represent the target installation.\r\n\r\nThis has the drawback that we'd be re-exporting information already present in other places (eg. `sys.version`, `sys.implementation`), but we believe it is worth.\r\n\r\n###### Future work\r\n\r\n*In a future step* we would also like to get Python to provide a static file detailing all the information required by the new `sysconfig` API. With this, we could then allow the new API to use an external installation/interpreter config file as the data source, making cross builds even simpler to support.\r\nSee the discussion at https://discuss.python.org/t/what-information-is-useful-to-know-statically-about-an-interpreter/25563 for examples of users that would benefit from this feature.\r\n\r\n## Migration plan\r\n\r\nTo help migration, and to make these improvements available on older Python versions, we plan to provide a backport package on PyPI, similarly with we already do with [`importlib_metadata`](https://github.com/python/importlib_metadata) (for [`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html)), [`importlib_resources`](https://github.com/python/importlib_resources) (for [`importlib.resources`](https://docs.python.org/3/library/importlib.resources.html)), and [`typing_extensions`](https://github.com/python/typing_extensions) (for [`typing`](https://docs.python.org/3/library/typing.html)).\r\n\r\n## Sub-issues\r\n\r\nThe proposal is quite big, and may require extensive discussions with different parties, so I am splitting it into a couple separate issues.\r\n\r\n- Install paths (GH-103481)\r\n- Build information\r\n  - Compilation (GH-103482)\r\n  - Compatibility (GH-103483)\r\n\r\n# Previous discussion\r\n\r\nThis has been mentioned in a couple places, here are some of the most relevant:\r\n\r\nhttps://discuss.python.org/t/building-extensions-modules-in-a-post-distutils-world/23938\r\nhttps://discuss.python.org/t/sysconfig-should-provide-more-information-to-help-packaging/22950\r\nhttps://discuss.python.org/t/pep-582-python-local-packages-directory/963/391\r\nhttps://discuss.python.org/t/linux-distro-patches-to-sysconfig-are-changing-pip-install-prefix-outside-virtual-environments/18240/28\r\nhttps://github.com/python/cpython/issues/102522#issuecomment-1497727808\r\nhttps://github.com/PyO3/maturin/blob/5d5b96a9974eac26b8cdf601cd2faf64f4999de9/src/python_interpreter/sysconfig-freebsd.json\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-110785\n* gh-110792\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/FFY00","@type":"Person","name":"FFY00"},"datePublished":"2023-04-12T18:48:07Z","url":"https://github.com/python/cpython/issues/103480"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:7afbf9c9-7129-7d8a-91a5-5632a1050b5f
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idA064:19AA62:3E32E48:4EE0109:6B020F4D
html-safe-nonce044e86010278afc4abf09352fcc2f5697ec6ef760bc048f47185f57fa88129d1
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMDY0OjE5QUE2MjozRTMyRTQ4OjRFRTAxMDk6NkIwMjBGNEQiLCJ2aXNpdG9yX2lkIjoiMjYxNzQ0NDE5ODI5ODgxNjMzMyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac2e2ff599c32ea854a9c3f6a15d1241e3e211e862b61d69dbac1f7a81e3e889a3
hovercard-subject-tagissue:1665071363
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/103480/issue_layout
twitter:imagehttps://opengraph.githubassets.com/c1d3e479809a97687c33d309bf7dfe675dbe4473ad27445fd44b92f81b0afe7c/python/cpython/issues/103480
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/c1d3e479809a97687c33d309bf7dfe675dbe4473ad27445fd44b92f81b0afe7c/python/cpython/issues/103480
og:image:altFeature or enhancement Provide new syconfig API that is aligned to how things have evolved. Fixes GH-99942 Fixes GH-99560 Pitch The sysconfig module was added in Python 3.2, over 10 years ago, and ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernameFFY00
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/python/cpython/issues/103480#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F103480
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fissues%2F103480
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/issues/103480
Reloadhttps://github.com/python/cpython/issues/103480
Reloadhttps://github.com/python/cpython/issues/103480
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
https://github.com/python/cpython
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/103480
New issuehttps://github.com/login?return_to=https://github.com/python/cpython/issues/103480
New sysconfig API 🤞 (meta-issue)https://github.com/python/cpython/issues/103480#top
topic-sysconfighttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-sysconfig%22
type-featureA feature request or enhancementhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-feature%22
https://github.com/FFY00
FFY00https://github.com/FFY00
on Apr 12, 2023https://github.com/python/cpython/issues/103480#issue-1665071363
GH-99942https://github.com/python/cpython/issues/99942
GH-99560https://github.com/python/cpython/issues/99560
GH-99942https://github.com/python/cpython/issues/99942
sysconfig.get_config_varshttps://docs.python.org/3/library/sysconfig.html
cpython/Lib/sysconfig.pyhttps://github.com/python/cpython/blob/52f96d3ea39fea4c16e26b7b10bd2db09726bd7c/Lib/sysconfig.py#L540-L556
52f96d3https://github.com/python/cpython/commit/52f96d3ea39fea4c16e26b7b10bd2db09726bd7c
GH-103482https://github.com/python/cpython/issues/103482
https://discuss.python.org/t/what-information-is-useful-to-know-statically-about-an-interpreter/25563https://discuss.python.org/t/what-information-is-useful-to-know-statically-about-an-interpreter/25563
importlib_metadatahttps://github.com/python/importlib_metadata
importlib.metadatahttps://docs.python.org/3/library/importlib.metadata.html
importlib_resourceshttps://github.com/python/importlib_resources
importlib.resourceshttps://docs.python.org/3/library/importlib.resources.html
typing_extensionshttps://github.com/python/typing_extensions
typinghttps://docs.python.org/3/library/typing.html
New sysconfig API: Install paths #103481https://github.com/python/cpython/issues/103481
New sysconfig API: Build information - Compilation #103482https://github.com/python/cpython/issues/103482
New sysconfig API: Build information - Compatibility #103483https://github.com/python/cpython/issues/103483
https://discuss.python.org/t/building-extensions-modules-in-a-post-distutils-world/23938https://discuss.python.org/t/building-extensions-modules-in-a-post-distutils-world/23938
https://discuss.python.org/t/sysconfig-should-provide-more-information-to-help-packaging/22950https://discuss.python.org/t/sysconfig-should-provide-more-information-to-help-packaging/22950
https://discuss.python.org/t/pep-582-python-local-packages-directory/963/391https://discuss.python.org/t/pep-582-python-local-packages-directory/963/391
https://discuss.python.org/t/linux-distro-patches-to-sysconfig-are-changing-pip-install-prefix-outside-virtual-environments/18240/28https://discuss.python.org/t/linux-distro-patches-to-sysconfig-are-changing-pip-install-prefix-outside-virtual-environments/18240/28
#102522 (comment)https://github.com/python/cpython/issues/102522#issuecomment-1497727808
https://github.com/PyO3/maturin/blob/5d5b96a9974eac26b8cdf601cd2faf64f4999de9/src/python_interpreter/sysconfig-freebsd.jsonhttps://github.com/PyO3/maturin/blob/5d5b96a9974eac26b8cdf601cd2faf64f4999de9/src/python_interpreter/sysconfig-freebsd.json
GH-103480: make sysconfig a package (GH-110785) #110785https://github.com/python/cpython/pull/110785
GH-103480: fix sysconfig package not being installed #110792https://github.com/python/cpython/pull/110792
FFY00https://github.com/FFY00
type-featureA feature request or enhancementhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-feature%22
topic-sysconfighttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-sysconfig%22
on Apr 12, 2023https://github.com/python/cpython/issues/103480#event-8988887108
https://github.com/encukou
encukouhttps://github.com/encukou
on Jul 10, 2023https://github.com/python/cpython/issues/103480#issuecomment-1628988769
https://github.com/FFY00
FFY00https://github.com/FFY00
on Jul 12, 2023https://github.com/python/cpython/issues/103480#issuecomment-1631700472
GH-77620https://github.com/python/cpython/issues/77620
https://github.com/FFY00
FFY00https://github.com/FFY00
on Aug 14, 2023https://github.com/python/cpython/issues/103480#issuecomment-1678223646
PEP 3149https://peps.python.org/pep-3149
(typing) protocolshttps://docs.python.org/3/library/typing.html#typing.Protocol
pypa/packaginghttps://github.com/pypa/packaging/
python-config.py should be part of the stdlib #77620https://github.com/python/cpython/issues/77620
Install a static installation description file as part of the Python installation #107956https://github.com/python/cpython/issues/107956
https://github.com/rgommers
rgommershttps://github.com/rgommers
on Aug 15, 2023https://github.com/python/cpython/issues/103480#issuecomment-1678480820
@FFY00https://github.com/FFY00
https://github.com/FFY00
FFY00https://github.com/FFY00
on Aug 15, 2023https://github.com/python/cpython/issues/103480#issuecomment-1679558842
https://github.com/zooba
zoobahttps://github.com/zooba
on Aug 15, 2023https://github.com/python/cpython/issues/103480#issuecomment-1679627318
FFY00https://github.com/FFY00
on Oct 12, 2023https://github.com/python/cpython/issues/103480#event-10637393472
pythonGH-103480: make sysconfig a packagehttps://github.com/FFY00/cpython/commit/aae3f5c56bf53e39bf5467a7c276ee3c6d8c4770
pythonGH-103480https://github.com/python/cpython/issues/103480
aae3f5chttps://github.com/FFY00/cpython/commit/aae3f5c56bf53e39bf5467a7c276ee3c6d8c4770
GH-103480: make sysconfig a package (GH-110785)https://github.com/python/cpython/commit/4a53a397c311567f05553bc25a28aebaba4f6f65
GH-103480https://github.com/python/cpython/issues/103480
GH-110785https://github.com/python/cpython/pull/110785
4a53a39https://github.com/python/cpython/commit/4a53a397c311567f05553bc25a28aebaba4f6f65
pythonGH-103480: fix sysconfig package not being installedhttps://github.com/FFY00/cpython/commit/59b1b3bbeaa737070b3cf74f27b5655fb5e68cff
pythonGH-103480https://github.com/python/cpython/issues/103480
59b1b3bhttps://github.com/FFY00/cpython/commit/59b1b3bbeaa737070b3cf74f27b5655fb5e68cff
pythonGH-103480: fix sysconfig package not being installedhttps://github.com/FFY00/cpython/commit/c27efe0aec189ebba72b364459a796556e2aa306
pythonGH-103480https://github.com/python/cpython/issues/103480
c27efe0https://github.com/FFY00/cpython/commit/c27efe0aec189ebba72b364459a796556e2aa306
GH-103480: fix sysconfig package not being installed (#110792)https://github.com/python/cpython/commit/b883cad06b12443014d57dcebd42d55f559b18f4
GH-103480https://github.com/python/cpython/issues/103480
#110792https://github.com/python/cpython/pull/110792
b883cadhttps://github.com/python/cpython/commit/b883cad06b12443014d57dcebd42d55f559b18f4
Glyphackhttps://github.com/Glyphack
on Sep 2, 2024https://github.com/python/cpython/issues/103480#event-14106069016
pythonGH-103480: make sysconfig a package (pythonGH-110785)https://github.com/Glyphack/cpython/commit/9fd4958805a5b020c45aaf8817197be0a4728925
pythonGH-103480https://github.com/python/cpython/issues/103480
pythonGH-110785https://github.com/python/cpython/pull/110785
9fd4958https://github.com/Glyphack/cpython/commit/9fd4958805a5b020c45aaf8817197be0a4728925
pythonGH-103480: fix sysconfig package not being installed (python#11…https://github.com/Glyphack/cpython/commit/b7af6ee3f0eaa2a6c708af9d92aea36a7f6dca55
pythonGH-103480https://github.com/python/cpython/issues/103480
python#11…https://github.com/python/cpython/pull/110792
b7af6eehttps://github.com/Glyphack/cpython/commit/b7af6ee3f0eaa2a6c708af9d92aea36a7f6dca55
Sign up for freehttps://github.com/signup?return_to=https://github.com/python/cpython/issues/103480
Sign in to commenthttps://github.com/login?return_to=https://github.com/python/cpython/issues/103480
topic-sysconfighttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22topic-sysconfig%22
type-featureA feature request or enhancementhttps://github.com/python/cpython/issues?q=state%3Aopen%20label%3A%22type-feature%22
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.