René's URL Explorer Experiment


Title: gh-93851: Add Tools/scripts/checkhtmllinks.py by arhadthedev · Pull Request #93856 · python/cpython · GitHub

Open Graph Title: gh-93851: Add Tools/scripts/checkhtmllinks.py by arhadthedev · Pull Request #93856 · python/cpython

X Title: gh-93851: Add Tools/scripts/checkhtmllinks.py by arhadthedev · Pull Request #93856 · python/cpython

Description: Broken links from a parent issue were found using this tool. C:\Users\oleg\Documents\dev\notmine\cpython>python Tools/scripts/checkhtmllinks.py -h usage: checkhtmllinks.py [-h] [-r] [-l LIMIT] path Check if specified HTML files have dead or redirected links. positional arguments: path a glob pattern of file paths to scan options: -h, --help show this help message and exit -r, --allow-redirects do not report HTTP 3xx links as kind-of-broken -l LIMIT, --limit LIMIT skip files that contain more links than specified Call this script on HTML files of the rendered documentation. Eventhough the script is multithreaded and findings for already processed pages are cached, a full run through the whole rendered documentation takes about an hour. Example output for python Tools/scripts/checkhtmllinks.py -l50 build/doc-html/using/*.html Note: [1/19] link to ... is not a bug; it's a manifestation of in a top bar, like this: 3.10.5 Documentation » The Python Standard Library » Debugging and Profiling » Audit events table ^^^^^^^^^^^^^^^^^^ the empty link collecting filenames to check... ======================================= [1/7] build\doc-html\using\cmdline.html ======================================= skipped; 158 links is above the --limit threshold ========================================= [2/7] build\doc-html\using\configure.html ========================================= skipped; 171 links is above the --limit threshold ======================================= [3/7] build\doc-html\using\editors.html ======================================= [1/19] link to ... [2/19] link to https://www.sphinx-doc.org/... [3/19] link to https://www.python.org/... [4/19] link to ../bugs.html... [5/19] link to https://github.com/python/cpython/blob/main/Doc/using/editors.rst... [6/19] link to ../index.html... [7/19] link to https://www.python.org/psf/donations/... [8/19] link to mac.html... [9/19] link to /license.html... [10/19] link to #editors-and-ides... skipped /license.html (absolute links are unsupported yet) [11/19] link to ../genindex.html... [13/19] link to ../copyright.html... [12/19] link to /bugs.html... skipped /bugs.html (absolute links are unsupported yet) [15/19] link to index.html... [14/19] link to ../py-modindex.html... [16/19] link to ../reference/index.html... [17/19] link to https://peps.python.org/pep-0008/... [18/19] link to https://wiki.python.org/moin/IntegratedDevelopmentEnvironments... [19/19] link to https://wiki.python.org/moin/PythonEditors... redirected https://www.sphinx-doc.org/ ===================================== [4/7] build\doc-html\using\index.html ===================================== skipped; 103 links is above the --limit threshold =================================== [5/7] build\doc-html\using\mac.html =================================== [1/47] link to ... [2/47] link to https://www.sphinx-doc.org/... [3/47] link to #running-scripts-with-a-gui... [4/47] link to https://www.python.org/... redirected https://www.sphinx-doc.org/ [6/47] link to ../bugs.html... [5/47] link to #other-resources... [8/47] link to #how-to-run-a-python-script... [7/47] link to http://aquamacs.org/... [11/47] link to http://macvim-dev.github.io/macvim/... [9/47] link to ../index.html... [13/47] link to https://www.python.org/psf/donations/... [14/47] link to http://www.hashcollision.org/hkn/python/idle_intro/index.html... [15/47] link to https://github.com/python/cpython/blob/main/Doc/using/mac.rst... [10/47] link to ../contents.html... [16/47] link to /license.html... [17/47] link to https://macromates.com/... [12/47] link to ../library/tkinter.html#module-tkinter... skipped /license.html (absolute links are unsupported yet) [18/47] link to mailto:bobsavage@mac.com... [19/47] link to #distributing-python-applications-on-the-mac... [20/47] link to ../genindex.html... [21/47] link to https://pypi.org/project/pyobjc/... [22/47] link to https://www.wxpython.org... [23/47] link to /bugs.html... redirected http://macvim-dev.github.io/macvim/ skipped /bugs.html (absolute links are unsupported yet) [24/47] link to https://pip.pypa.io/... [25/47] link to #getting-and-installing-macpython... [26/47] link to #using-python-on-a-mac... [28/47] link to https://riverbankcomputing.com/software/pyqt/intro... [27/47] link to ../copyright.html... [29/47] link to #ide... [30/47] link to ../py-modindex.html... [31/47] link to https://www.python.org... [32/47] link to #installing-additional-python-packages... [33/47] link to https://pypi.org/project/py2app/... [34/47] link to editors.html... [35/47] link to index.html... [36/47] link to http://www.barebones.com/products/bbedit/index.html... [37/47] link to https://www.activestate.com... [38/47] link to https://wiki.python.org/moin/MacPython... redirected https://pip.pypa.io/ [39/47] link to cmdline.html#envvar-PYTHONPATH... [40/47] link to windows.html... [41/47] link to #configuration... [42/47] link to https://www.python.org/community/sigs/current/pythonmac-sig/... [43/47] link to #the-ide... [44/47] link to #gui-programming-on-the-mac... [45/47] link to #mac-package-manager... [46/47] link to https://www.tcl.tk... [47/47] link to #... ==================================== [6/7] build\doc-html\using\unix.html ==================================== [1/36] link to ... [2/36] link to #getting-and-installing-the-latest-version-of-python... [3/36] link to https://www.sphinx-doc.org/... [4/36] link to https://www.python.org/... [5/36] link to ../bugs.html... redirected https://www.sphinx-doc.org/ [8/36] link to https://www.python.org/psf/donations/... [6/36] link to ../index.html... [7/36] link to ../contents.html... [10/36] link to /license.html... [9/36] link to #custom-openssl... [11/36] link to https://devguide.python.org/setup/#getting-the-source-code... [14/36] link to https://www.opencsw.org/... [16/36] link to ../genindex.html... [12/36] link to ../library/subprocess.html#module-subprocess... [17/36] link to https://github.com/python/cpython/blob/main/Doc/using/unix.rst... skipped /license.html (absolute links are unsupported yet) [15/36] link to https://www.debian.org/doc/manuals/maint-guide/first.en.html... [18/36] link to /bugs.html... [19/36] link to #using-python-on-unix-platforms... [13/36] link to https://github.com/python/cpython/tree/main/README.rst... skipped /bugs.html (absolute links are unsupported yet) [20/36] link to ../copyright.html... [21/36] link to ../py-modindex.html... [22/36] link to #on-freebsd-and-openbsd... [23/36] link to #on-opensolaris... [24/36] link to index.html... [25/36] link to #building-python... [26/36] link to https://www.python.org/downloads/source/... [27/36] link to #miscellaneous... [28/36] link to https://docs-old.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html... [29/36] link to http://www.slackbook.org/html/package-management-making-packages.html... failed https://devguide.python.org/setup/#getting-the-source-code [30/36] link to cmdline.html... [31/36] link to configure.html... [32/36] link to #on-linux... [33/36] link to configure.html#configure-options... [34/36] link to https://en.opensuse.org/Portal:Packaging... redirected https://github.com/python/cpython/tree/main/README.rst [35/36] link to #... [36/36] link to #python-related-paths-and-files... redirected https://docs-old.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html ======================================= [7/7] build\doc-html\using\windows.html ======================================= skipped; 117 links is above the --limit threshold ======================== Final report on problems ======================== build\doc-html\using\editors.html redirected https://www.sphinx-doc.org/ link; increased loading time build\doc-html\using\mac.html redirected https://www.sphinx-doc.org/ link; increased loading time redirected http://macvim-dev.github.io/macvim/ link; increased loading time redirected https://pip.pypa.io/ link; increased loading time build\doc-html\using\unix.html redirected https://www.sphinx-doc.org/ link; increased loading time broken https://devguide.python.org/setup/#getting-the-source-code link; check if #getting-the-source-code exists redirected https://github.com/python/cpython/tree/main/README.rst link; increased loading time redirected https://docs-old.fedoraproject.org/en-US/Fedora_Draft_Documentation/0.1/html/RPM_Guide/ch-creating-rpms.html link; increased loading time Issue: gh-93851

Open Graph Description: Broken links from a parent issue were found using this tool. C:\Users\oleg\Documents\dev\notmine\cpython>python Tools/scripts/checkhtmllinks.py -h usage: checkhtmllinks.py [-h] [-r] [-l LIMIT] p...

X Description: Broken links from a parent issue were found using this tool. C:\Users\oleg\Documents\dev\notmine\cpython>python Tools/scripts/checkhtmllinks.py -h usage: checkhtmllinks.py [-h] [-r] [-l LIMI...

Opengraph URL: https://github.com/python/cpython/pull/93856

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:8cfc0318-6f1c-d0b9-9a4d-717557f4dbbe
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idA3EA:11D50D:1540F94:1C644FE:6969B60C
html-safe-nonce1c705e73c834eb7adcef7d585335da1af56041dcdaccac6c371d1589775d37ac
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBM0VBOjExRDUwRDoxNTQwRjk0OjFDNjQ0RkU6Njk2OUI2MEMiLCJ2aXNpdG9yX2lkIjoiMjUxNDEyMzc2OTIxNDc3Njg0NCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac095b61c1de5ab8d5a095c1595abb0b3e2670ddb83285b0895ac2095bf5adafd1
hovercard-subject-tagpull_request:968133749
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/93856/files
twitter:imagehttps://avatars.githubusercontent.com/u/4881073?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/4881073?s=400&v=4
og:image:altBroken links from a parent issue were found using this tool. C:\Users\oleg\Documents\dev\notmine\cpython>python Tools/scripts/checkhtmllinks.py -h usage: checkhtmllinks.py [-h] [-r] [-l LIMIT] p...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneacedec8b5f975d9e3d494ddd8f949b0b8a0de59d393901e26f73df9dcba80056
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release83c08c21cdda978090dc44364b71aa5bc6dcea79
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/93856/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F93856%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F93856%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/93856/files
Reloadhttps://github.com/python/cpython/pull/93856/files
Reloadhttps://github.com/python/cpython/pull/93856/files
python https://github.com/python
cpythonhttps://github.com/python/cpython
Please reload this pagehttps://github.com/python/cpython/pull/93856/files
Notifications https://github.com/login?return_to=%2Fpython%2Fcpython
Fork 33.9k https://github.com/login?return_to=%2Fpython%2Fcpython
Star 71.1k https://github.com/login?return_to=%2Fpython%2Fcpython
Code https://github.com/python/cpython
Issues 5k+ https://github.com/python/cpython/issues
Pull requests 2.1k https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects 31 https://github.com/python/cpython/projects
Security Uh oh! There was an error while loading. Please reload this page. https://github.com/python/cpython/security
Please reload this pagehttps://github.com/python/cpython/pull/93856/files
Insights https://github.com/python/cpython/pulse
Code https://github.com/python/cpython
Issues https://github.com/python/cpython/issues
Pull requests https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
arhadthedevhttps://github.com/arhadthedev
python:mainhttps://github.com/python/cpython/tree/main
arhadthedev:script-checkhtmlhttps://github.com/arhadthedev/cpython/tree/script-checkhtml
Conversation 4 https://github.com/python/cpython/pull/93856
Commits 8 https://github.com/python/cpython/pull/93856/commits
Checks 0 https://github.com/python/cpython/pull/93856/checks
Files changed https://github.com/python/cpython/pull/93856/files
Please reload this pagehttps://github.com/python/cpython/pull/93856/files
gh-93851: Add Tools/scripts/checkhtmllinks.py https://github.com/python/cpython/pull/93856/files#top
Show all changes 8 commits https://github.com/python/cpython/pull/93856/files
8f0661d Initial revision of Tools/scripts/checkhtmllinks.py arhadthedev Jun 14, 2022 https://github.com/python/cpython/pull/93856/commits/8f0661d26186f60b703d203a425fc64880f95a21
a547a73 Clarify runtime estimations arhadthedev Jun 14, 2022 https://github.com/python/cpython/pull/93856/commits/a547a7397316489c189c95936e3927cf197e68f6
5a9e8d8 Hide an unimportant abstraction arhadthedev Jun 15, 2022 https://github.com/python/cpython/pull/93856/commits/5a9e8d86aaab522114d6e76f15a205ab998695f2
c57e264 Use more relevant abstractions arhadthedev Jun 15, 2022 https://github.com/python/cpython/pull/93856/commits/c57e2643078265b01e1bb9d6786071780e7953e7
9c4ab0f Use the same capitalization as in `-h` cmdline key arhadthedev Jun 15, 2022 https://github.com/python/cpython/pull/93856/commits/9c4ab0ff139d6d254a40aff2cebbb12d850371f9
0911e9d Merge branch 'main' into script-checkhtml arhadthedev Jul 21, 2022 https://github.com/python/cpython/pull/93856/commits/0911e9d8121d5963842707b449a7fc70f3a8bfc6
af53ada Fix test_target() so it always returns a list arhadthedev Aug 4, 2022 https://github.com/python/cpython/pull/93856/commits/af53adaec9abd25c3dcb54a38a446cf5fd5a1e07
937a72c Merge branch 'main' into script-checkhtml arhadthedev Oct 8, 2022 https://github.com/python/cpython/pull/93856/commits/937a72c9db293d4b7cb845a86ceb0cd0c81a1135
Clear filters https://github.com/python/cpython/pull/93856/files
Please reload this pagehttps://github.com/python/cpython/pull/93856/files
Please reload this pagehttps://github.com/python/cpython/pull/93856/files
ACKS https://github.com/python/cpython/pull/93856/files#diff-d1243956feb505c5459fc05387e194609efb5c350cee202942be47ed30d7d7d0
2022-06-14-10-15-17.gh-issue-93851.rL64WQ.rst https://github.com/python/cpython/pull/93856/files#diff-7c0b803787fd7db6b7e43680292a991ecb483e56bad1f63ba7c2063dd07752ac
README https://github.com/python/cpython/pull/93856/files#diff-98fa6c3ef02372369d7b8a0fbc2d5448e681bfd43096725223ae1c05ce357cdc
checkhtmllinks.py https://github.com/python/cpython/pull/93856/files#diff-ce00862f44188b9c1b659eb0e8ec81c58900765bd1ca6da369dcf1a774bc04b2
Misc/ACKShttps://github.com/python/cpython/pull/93856/files#diff-d1243956feb505c5459fc05387e194609efb5c350cee202942be47ed30d7d7d0
View file https://github.com/arhadthedev/cpython/blob/937a72c9db293d4b7cb845a86ceb0cd0c81a1135/Misc/ACKS
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/93856/{{ revealButtonHref }}
https://github.com/python/cpython/pull/93856/files#diff-d1243956feb505c5459fc05387e194609efb5c350cee202942be47ed30d7d7d0
https://github.com/python/cpython/pull/93856/files#diff-d1243956feb505c5459fc05387e194609efb5c350cee202942be47ed30d7d7d0
Misc/NEWS.d/next/Tools-Demos/2022-06-14-10-15-17.gh-issue-93851.rL64WQ.rsthttps://github.com/python/cpython/pull/93856/files#diff-7c0b803787fd7db6b7e43680292a991ecb483e56bad1f63ba7c2063dd07752ac
View file https://github.com/arhadthedev/cpython/blob/937a72c9db293d4b7cb845a86ceb0cd0c81a1135/Misc/NEWS.d/next/Tools-Demos/2022-06-14-10-15-17.gh-issue-93851.rL64WQ.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/93856/{{ revealButtonHref }}
Tools/scripts/READMEhttps://github.com/python/cpython/pull/93856/files#diff-98fa6c3ef02372369d7b8a0fbc2d5448e681bfd43096725223ae1c05ce357cdc
View file https://github.com/arhadthedev/cpython/blob/937a72c9db293d4b7cb845a86ceb0cd0c81a1135/Tools/scripts/README
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/93856/{{ revealButtonHref }}
https://github.com/python/cpython/pull/93856/files#diff-98fa6c3ef02372369d7b8a0fbc2d5448e681bfd43096725223ae1c05ce357cdc
https://github.com/python/cpython/pull/93856/files#diff-98fa6c3ef02372369d7b8a0fbc2d5448e681bfd43096725223ae1c05ce357cdc
Tools/scripts/checkhtmllinks.pyhttps://github.com/python/cpython/pull/93856/files#diff-ce00862f44188b9c1b659eb0e8ec81c58900765bd1ca6da369dcf1a774bc04b2
View file https://github.com/arhadthedev/cpython/blob/937a72c9db293d4b7cb845a86ceb0cd0c81a1135/Tools/scripts/checkhtmllinks.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/93856/{{ revealButtonHref }}
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.