René's URL Explorer Experiment


Title: fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host filtering by aryansk · Pull Request #2162 · apify/crawlee-python · GitHub

Open Graph Title: fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host filtering by aryansk · Pull Request #2162 · apify/crawlee-python

X Title: fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host filtering by aryansk · Pull Request #2162 · apify/crawlee-python

Description: Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included. - fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host filtering by aryansk · Pull Request #2162 · apify/crawlee-python

Open Graph Description: Summary Sitemap.from_xml_string only accepted content and hardcoded SitemapSource(type='raw', content=content) with no url, so host filtering — conditional on source.get('url') — wa...

X Description: Summary Sitemap.from_xml_string only accepted content and hardcoded SitemapSource(type='raw', content=content) with no url, so host filtering — conditional on source.get('ur...

Opengraph URL: https://github.com/apify/crawlee-python/pull/2162

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format)
route-controllervoltron_pull_requests_fragments
route-actionpull_request_layout
fetch-noncev2:fe1119e2-be51-782b-13aa-c05373879099
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idC62E:95C88:BF5D4A:FEB612:6B00F251
html-safe-noncefb25470c301a23463597214c900ae67dc355eb62c311a6ac80e03844203fe2fa
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDNjJFOjk1Qzg4OkJGNUQ0QTpGRUI2MTI6NkIwMEYyNTEiLCJ2aXNpdG9yX2lkIjoiNzI2NjA5MjUwMzQxMjA0NDM2OSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac90d180b1ae1fb1ae8752d47c28f9b744efd0055d4ff9c0a8efc10123cb19263e
hovercard-subject-tagpull_request:4285335632
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/pull_requests_fragments/pull_request_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/apify/crawlee-python/2162/pull_request_layout
twitter:imagehttps://opengraph.githubassets.com/03ae8ea24d251681e85f53fd103c5c4efc2ba5cf5e905abb2b88d49e39ca35f5/apify/crawlee-python/pull/2162
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/03ae8ea24d251681e85f53fd103c5c4efc2ba5cf5e905abb2b88d49e39ca35f5/apify/crawlee-python/pull/2162
og:image:altSummary Sitemap.from_xml_string only accepted content and hardcoded SitemapSource(type='raw', content=content) with no url, so host filtering — conditional on source.get('url') — wa...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamearyansk
hostnamegithub.com
expected-hostnamegithub.com
None4ef498cbabcbe3c50befc7a5b743e7072ee770b4daf0e15e54801180d1fa91fc
turbo-cache-controlno-cache
go-importgithub.com/apify/crawlee-python git https://github.com/apify/crawlee-python.git
octolytics-dimension-user_id24586296
octolytics-dimension-user_loginapify
octolytics-dimension-repository_id741365632
octolytics-dimension-repository_nwoapify/crawlee-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id741365632
octolytics-dimension-repository_network_root_nwoapify/crawlee-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release9983dd5da06943f833c971dd5b6bc988e2858be5
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-namepull-requests

Links:

Skip to contenthttps://github.com/apify/crawlee-python/pull/2162#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2162
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2162
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fpull_requests_fragments%2Fpull_request_layout&source=header-repo&source_repo=apify%2Fcrawlee-python
Reloadhttps://github.com/apify/crawlee-python/pull/2162
Reloadhttps://github.com/apify/crawlee-python/pull/2162
Reloadhttps://github.com/apify/crawlee-python/pull/2162
apifyhttps://github.com/apify
crawlee-pythonhttps://github.com/apify/crawlee-python
Codehttps://github.com/apify/crawlee-python
Issues72 (72)https://github.com/apify/crawlee-python/issues
Pull requests14 (14)https://github.com/apify/crawlee-python/pulls
Discussionshttps://github.com/apify/crawlee-python/discussions
Actionshttps://github.com/apify/crawlee-python/actions
Security and quality1 (1)https://github.com/apify/crawlee-python/security
Insightshttps://github.com/apify/crawlee-python/insights
https://github.com/apify/crawlee-python
vdusekhttps://github.com/vdusek
apify:masterhttps://github.com/apify/crawlee-python/tree/master
aryansk:fix/2118-sitemap-from-xml-host-filterhttps://github.com/aryansk/crawlee-python/tree/fix/2118-sitemap-from-xml-host-filter
Conversationhttps://github.com/apify/crawlee-python/pull/2162
Commits3 (3)https://github.com/apify/crawlee-python/pull/2162/commits
Checkshttps://github.com/apify/crawlee-python/pull/2162/checks
Files changedhttps://github.com/apify/crawlee-python/pull/2162/files
fix(sitemaps): allow Sitemap.from_xml_string to opt into enqueue host filteringhttps://github.com/apify/crawlee-python/pull/2162#top
vdusekhttps://github.com/vdusek
apify:masterhttps://github.com/apify/crawlee-python/tree/master
aryansk:fix/2118-sitemap-from-xml-host-filterhttps://github.com/aryansk/crawlee-python/tree/fix/2118-sitemap-from-xml-host-filter
https://github.com/aryansk
aryanskhttps://github.com/aryansk
Aug 15, 2026https://github.com/apify/crawlee-python/pull/2162#issue-5158280552
#2118https://github.com/apify/crawlee-python/issues/2118
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2162
https://github.com/aryansk
https://github.com/codebuff-team
fix(sitemap): let from_xml_string opt into enqueue host filteringhttps://github.com/apify/crawlee-python/pull/2162/commits/25628c47764501f9d292f96ab5153ea92150a00f
25628c4https://github.com/apify/crawlee-python/pull/2162/commits/25628c47764501f9d292f96ab5153ea92150a00f
apify#2118https://github.com/apify/crawlee-python/issues/2118
https://github.com/vdusek
vdusekhttps://github.com/vdusek
Aug 18, 2026https://github.com/apify/crawlee-python/pull/2162#event-29604936642
vdusekhttps://github.com/vdusek
August 18, 2026 10:01https://github.com/apify/crawlee-python/pull/2162#commits-pushed-6b2c6a0
https://github.com/vdusek
fix(sitemap): report the provided URL as origin_sitemap_url for raw s…https://github.com/apify/crawlee-python/pull/2162/commits/6b2c6a00eec3f95a526b42de4120f4e7fcb0a4d7
6b2c6a0https://github.com/apify/crawlee-python/pull/2162/commits/6b2c6a00eec3f95a526b42de4120f4e7fcb0a4d7
https://github.com/vdusek
test: cover the raw:// content-hash origin fallback for raw sitemap s…https://github.com/apify/crawlee-python/pull/2162/commits/c07c3a3c450524cacfa836ca682be3c7e7cc1ff4
c07c3a3https://github.com/apify/crawlee-python/pull/2162/commits/c07c3a3c450524cacfa836ca682be3c7e7cc1ff4
https://github.com/vdusek
vdusekhttps://github.com/vdusek
Aug 18, 2026 https://github.com/apify/crawlee-python/pull/2162#pullrequestreview-4958804844
View reviewed changes https://github.com/apify/crawlee-python/pull/2162/files/c07c3a3c450524cacfa836ca682be3c7e7cc1ff4
vdusekhttps://github.com/vdusek
https://github.com/apify/crawlee-python/pull/2162#pullrequestreview-4958804844
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2162
https://github.com/vdusek
vdusekhttps://github.com/vdusek
a3c20c1https://github.com/apify/crawlee-python/commit/a3c20c15a15f7ffa5644640adde6daf1dd4c3523
Aug 18, 2026https://github.com/apify/crawlee-python/pull/2162#event-29606462289
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2162
https://github.co/hiddenchars
https://github.com/apify/crawlee-python/pull/{{ revealButtonHref }}
Sign up for freehttps://github.com/join?source=comment-repo
Sign in to commenthttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2162
https://github.com/vdusek
vdusek https://github.com/vdusek
https://github.com/apify/crawlee-python/pull/2162/files/c07c3a3c450524cacfa836ca682be3c7e7cc1ff4
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2162
Sitemap.from_xml_string cannot opt into enqueue host filtering https://github.com/apify/crawlee-python/issues/2118
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2162
https://github.com/aryansk
https://github.com/vdusek
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.