René's URL Explorer Experiment


Title: fix(crawlers): Send cookies set during navigation with `send_request` in `PlaywrightCrawler` by rome-xi · Pull Request #2485 · apify/crawlee-python · GitHub

Open Graph Title: fix(crawlers): Send cookies set during navigation with `send_request` in `PlaywrightCrawler` by rome-xi · Pull Request #2485 · apify/crawlee-python

X Title: fix(crawlers): Send cookies set during navigation with `send_request` in `PlaywrightCrawler` by rome-xi · Pull Request #2485 · apify/crawlee-python

Description: Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract structured data with LLMs via Pydantic AI, or control the browser in natural language with Stagehand. Works with BeautifulSoup, Parsel, Playwright, and raw HTTP. Download HTML, PDF, images, and other files. Proxy rotation included. - fix(crawlers): Send cookies set during navigation with `send_request` in `PlaywrightCrawler` by rome-xi · Pull Request #2485 · apify/crawlee-python

Open Graph Description: Problem PlaywrightCrawler only copied browser cookies into context.session.cookies after the request handler returned. Custom HTTP clients (ImpitHttpClient, HttpxHttpClient, and similar) read the s...

X Description: Problem PlaywrightCrawler only copied browser cookies into context.session.cookies after the request handler returned. Custom HTTP clients (ImpitHttpClient, HttpxHttpClient, and similar) read the s...

Opengraph URL: https://github.com/apify/crawlee-python/pull/2485

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format)
route-controllervoltron_pull_requests_fragments
route-actionpull_request_layout
fetch-noncev2:9da487d1-e2ca-6b9f-a2ae-f55fb0145ee5
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idAC2E:19AA62:73CD18:94F33A:6B00E404
html-safe-nonce2849efd5afea78f0c959f194bb90345bf0d56f7b4832b90602672efd40dccb41
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQzJFOjE5QUE2Mjo3M0NEMTg6OTRGMzNBOjZCMDBFNDA0IiwidmlzaXRvcl9pZCI6IjM1MDcyNDI4NjgwNDc2NjgyMjgiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmac615958773b7bcb1ea58e80690be4059924d740917bbdfc32b12f7a253b6bc299
hovercard-subject-tagpull_request:5395929408
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/pull_requests_fragments/pull_request_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/apify/crawlee-python/2485/pull_request_layout
twitter:imagehttps://opengraph.githubassets.com/6db7087d03c6953733d6e078c01c1bde68fdc4028b548846e0d74323146776ca/apify/crawlee-python/pull/2485
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/6db7087d03c6953733d6e078c01c1bde68fdc4028b548846e0d74323146776ca/apify/crawlee-python/pull/2485
og:image:altProblem PlaywrightCrawler only copied browser cookies into context.session.cookies after the request handler returned. Custom HTTP clients (ImpitHttpClient, HttpxHttpClient, and similar) read the s...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamerome-xi
hostnamegithub.com
expected-hostnamegithub.com
None27088b6977484d0d3321d9a11c907c6d6637b315cfa359ef7952dbf6292f9b14
turbo-cache-controlno-cache
go-importgithub.com/apify/crawlee-python git https://github.com/apify/crawlee-python.git
octolytics-dimension-user_id24586296
octolytics-dimension-user_loginapify
octolytics-dimension-repository_id741365632
octolytics-dimension-repository_nwoapify/crawlee-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id741365632
octolytics-dimension-repository_network_root_nwoapify/crawlee-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release624179fb8b88d26ba5bba08894eb60a1c5a1d93d
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-namepull-requests

Links:

Skip to contenthttps://github.com/apify/crawlee-python/pull/2485#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2485
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2485
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fpull_requests_fragments%2Fpull_request_layout&source=header-repo&source_repo=apify%2Fcrawlee-python
Reloadhttps://github.com/apify/crawlee-python/pull/2485
Reloadhttps://github.com/apify/crawlee-python/pull/2485
Reloadhttps://github.com/apify/crawlee-python/pull/2485
apifyhttps://github.com/apify
crawlee-pythonhttps://github.com/apify/crawlee-python
Codehttps://github.com/apify/crawlee-python
Issues72 (72)https://github.com/apify/crawlee-python/issues
Pull requests14 (14)https://github.com/apify/crawlee-python/pulls
Discussionshttps://github.com/apify/crawlee-python/discussions
Actionshttps://github.com/apify/crawlee-python/actions
Security and quality1 (1)https://github.com/apify/crawlee-python/security
Insightshttps://github.com/apify/crawlee-python/insights
https://github.com/apify/crawlee-python
rome-xihttps://github.com/rome-xi
apify:masterhttps://github.com/apify/crawlee-python/tree/master
rome-xi:fix/2483-playwright-session-cookies-before-handlerhttps://github.com/rome-xi/crawlee-python/tree/fix/2483-playwright-session-cookies-before-handler
Conversationhttps://github.com/apify/crawlee-python/pull/2485
Commits2 (2)https://github.com/apify/crawlee-python/pull/2485/commits
Checkshttps://github.com/apify/crawlee-python/pull/2485/checks
Files changedhttps://github.com/apify/crawlee-python/pull/2485/files
fix(crawlers): Send cookies set during navigation with send_request in PlaywrightCrawlerhttps://github.com/apify/crawlee-python/pull/2485#top
rome-xihttps://github.com/rome-xi
apify:masterhttps://github.com/apify/crawlee-python/tree/master
rome-xi:fix/2483-playwright-session-cookies-before-handlerhttps://github.com/rome-xi/crawlee-python/tree/fix/2483-playwright-session-cookies-before-handler
https://github.com/rome-xi
rome-xihttps://github.com/rome-xi
Nov 19, 2026https://github.com/apify/crawlee-python/pull/2485#issue-6490797912
#2483https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
https://github.com/rome-xi
fix(playwright): sync browser cookies to the session before the handlerhttps://github.com/apify/crawlee-python/pull/2485/commits/854e10cc54dd6df0e0a637d68da951785356e32a
854e10chttps://github.com/apify/crawlee-python/pull/2485/commits/854e10cc54dd6df0e0a637d68da951785356e32a
apify#2483https://github.com/apify/crawlee-python/issues/2483
https://github.com/apps/github-actions
github-actionshttps://github.com/apps/github-actions
t-tooling https://github.com/apify/crawlee-python/issues?q=state%3Aopen%20label%3At-tooling
Nov 19, 2026https://github.com/apify/crawlee-python/pull/2485#event-36654952437
https://github.com/vdusek
vdusekhttps://github.com/vdusek
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#event-36746888865
https://github.com/vdusek
vdusekhttps://github.com/vdusek
Nov 20, 2026 https://github.com/apify/crawlee-python/pull/2485#pullrequestreview-5989887767
View reviewed changes https://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a
src/crawlee/crawlers/_playwright/_playwright_crawler.pyhttps://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a#diff-57030939af3f9613b8173fe2eb6d9f653d941685d521eba4a13d51c8e45fd8c4
vdusekhttps://github.com/vdusek
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640368543
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
rome-xihttps://github.com/rome-xi
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640902707
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
d3ec31ahttps://github.com/apify/crawlee-python/commit/d3ec31ab5a2ec3de79684e2d6c660ce4d5251933
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
src/crawlee/crawlers/_playwright/_playwright_crawler.pyhttps://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a#diff-57030939af3f9613b8173fe2eb6d9f653d941685d521eba4a13d51c8e45fd8c4
vdusekhttps://github.com/vdusek
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640368565
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
rome-xihttps://github.com/rome-xi
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640903390
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
d3ec31ahttps://github.com/apify/crawlee-python/commit/d3ec31ab5a2ec3de79684e2d6c660ce4d5251933
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
src/crawlee/crawlers/_playwright/_playwright_crawler.pyhttps://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a#diff-57030939af3f9613b8173fe2eb6d9f653d941685d521eba4a13d51c8e45fd8c4
vdusekhttps://github.com/vdusek
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640368579
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
#2483https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
rome-xihttps://github.com/rome-xi
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640904039
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
#2483https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
src/crawlee/crawlers/_playwright/_playwright_crawler.pyhttps://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a#diff-57030939af3f9613b8173fe2eb6d9f653d941685d521eba4a13d51c8e45fd8c4
vdusekhttps://github.com/vdusek
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640368587
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
rome-xihttps://github.com/rome-xi
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#discussion_r4640904583
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
#2483https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
https://github.com/vdusek
vdusekhttps://github.com/vdusek
apify-factoryhttps://github.com/apify-factory
November 20, 2026 08:56https://github.com/apify/crawlee-python/pull/2485#event-36750216268
https://github.com/apify-factory
apify-factoryhttps://github.com/apify-factory
Nov 20, 2026 https://github.com/apify/crawlee-python/pull/2485#pullrequestreview-5989942071
View reviewed changes https://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a
apify-factoryhttps://github.com/apify-factory
https://github.com/apify/crawlee-python/pull/2485#pullrequestreview-5989942071
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@apify-factoryhttps://github.com/apify-factory
Review details on the dashboardhttps://internal-apps--ai-developer-dashboard.apify.actor/runs/0mwoqwfzz256de1be8f57
workflow runhttps://github.com/apify/crawlee-python/actions/runs/44706552705
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
https://github.com/rome-xi
fix(playwright): rename cookie helper and wrap the session commenthttps://github.com/apify/crawlee-python/pull/2485/commits/d3ec31ab5a2ec3de79684e2d6c660ce4d5251933
d3ec31ahttps://github.com/apify/crawlee-python/pull/2485/commits/d3ec31ab5a2ec3de79684e2d6c660ce4d5251933
apify#2485https://github.com/apify/crawlee-python/pull/2485
https://github.com/rome-xi
rome-xihttps://github.com/rome-xi
Nov 20, 2026https://github.com/apify/crawlee-python/pull/2485#issuecomment-7036298717
@vdusekhttps://github.com/vdusek
d3ec31ahttps://github.com/apify/crawlee-python/commit/d3ec31ab5a2ec3de79684e2d6c660ce4d5251933
#2483https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
https://github.co/hiddenchars
https://github.com/apify/crawlee-python/pull/{{ revealButtonHref }}
Sign up for freehttps://github.com/join?source=comment-repo
Sign in to commenthttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapify%2Fcrawlee-python%2Fpull%2F2485
https://github.com/vdusek
vdusek https://github.com/vdusek
https://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a
https://github.com/apify-factory
apify-factory https://github.com/apify-factory
https://github.com/apify/crawlee-python/pull/2485/files/854e10cc54dd6df0e0a637d68da951785356e32a
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
t-tooling https://github.com/apify/crawlee-python/issues?q=state%3Aopen%20label%3At-tooling
PlaywrightCrawler: context.send_request with a custom http_client sends pre-navigation cookies https://github.com/apify/crawlee-python/issues/2483
Please reload this pagehttps://github.com/apify/crawlee-python/pull/2485
https://github.com/rome-xi
https://github.com/vdusek
https://github.com/apify-factory
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.