René's URL Explorer Experiment


Title: RequestOptions | API | Crawlee for Python · Fast, reliable Python web crawlers.

Open Graph Title: RequestOptions | API | Crawlee for Python · Fast, reliable Python web crawlers.

Description: Options that can be used to customize request creation.

Open Graph Description: Options that can be used to customize request creation.

Opengraph URL: https://crawlee.dev/python/api/class/RequestOptions

Generator: Docusaurus v3.10.2

direct link

Domain: crawlee.dev


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[]}

twitter:cardsummary_large_image
og:imagehttps://crawlee.dev/python/img/crawlee-python-og.png
twitter:imagehttps://crawlee.dev/python/img/crawlee-python-og.png
og:localeen
docusaurus_localeen
docsearch:languageen
docusaurus_version1.11
docusaurus_tagdocs-default-1.11
docsearch:version1.11
docsearch:docusaurus_tagdocs-default-1.11

Links:

Skip to main contenthttps://crawlee.dev/python/api/class/RequestOptions#__docusaurus_skipToContent_fallback
Crawlee for Pythonhttps://crawlee.dev/python/
https://crawlee.dev/js
https://crawlee.dev/python/
https://crawlee.dev
Docshttps://crawlee.dev/python/docs/quick-start
APIhttps://crawlee.dev/python/api
Changeloghttps://crawlee.dev/python/docs/changelog
Bloghttps://crawlee.dev/blog
1.11https://crawlee.dev/python/api
Nexthttps://crawlee.dev/python/api/next
1.11https://crawlee.dev/python/api
0.6https://crawlee.dev/python/api/0.6
Get startedhttps://crawlee.dev/python/docs/quick-start
Crawlee for Pythonhttps://crawlee.dev/python/
Overviewhttps://crawlee.dev/python/api
Autoscalinghttps://crawlee.dev/python/api/class/AutoscaledPool
Browser managementhttps://crawlee.dev/python/api/class/BrowserController
Configurationhttps://crawlee.dev/python/api/class/ConcurrencySettings
Crawlershttps://crawlee.dev/python/api/class/AbstractHttpCrawler
Crawling contextshttps://crawlee.dev/python/api/class/AdaptivePlaywrightCrawlingContext
Errorshttps://crawlee.dev/python/api/class/ContextPipelineFinalizationError
Event datahttps://crawlee.dev/python/api/enum/Event
Event managershttps://crawlee.dev/python/api/class/EventManager
Functionshttps://crawlee.dev/python/api/class/AddRequestsFunction
HTTP clientshttps://crawlee.dev/python/api/class/CurlImpersonateHttpClient
HTTP parsershttps://crawlee.dev/python/api/class/AbstractHttpParser
Request loadershttps://crawlee.dev/python/api/class/RequestList
Session managementhttps://crawlee.dev/python/api/class/CookieParam
Statisticshttps://crawlee.dev/python/api/class/AdaptivePlaywrightCrawlerStatisticState
Storage clientshttps://crawlee.dev/python/api/class/FileSystemStorageClient
Storage datahttps://crawlee.dev/python/api/class/AddRequestsResponse
Storageshttps://crawlee.dev/python/api/class/Dataset
Otherhttps://crawlee.dev/python/api/class/BasePydanticAiHtmlDistiller
BasePydanticAiHtmlDistillerhttps://crawlee.dev/python/api/class/BasePydanticAiHtmlDistiller
BasePydanticAiHtmlExtractorhttps://crawlee.dev/python/api/class/BasePydanticAiHtmlExtractor
BrowserforgeFingerprintGeneratorhttps://crawlee.dev/python/api/class/BrowserforgeFingerprintGenerator
ContextPipelinehttps://crawlee.dev/python/api/class/ContextPipeline
CrawlerInstrumentorhttps://crawlee.dev/python/api/class/CrawlerInstrumentor
DefaultRenderingTypePredictorhttps://crawlee.dev/python/api/class/DefaultRenderingTypePredictor
FingerprintGeneratorhttps://crawlee.dev/python/api/class/FingerprintGenerator
HeaderGeneratorhttps://crawlee.dev/python/api/class/HeaderGenerator
HttpHeadershttps://crawlee.dev/python/api/class/HttpHeaders
HttpResponsehttps://crawlee.dev/python/api/class/HttpResponse
PageSnapshothttps://crawlee.dev/python/api/class/PageSnapshot
ProxyInfohttps://crawlee.dev/python/api/class/ProxyInfo
PydanticAiCleanHtmlDistillerhttps://crawlee.dev/python/api/class/PydanticAiCleanHtmlDistiller
PydanticAiDirectExtractorhttps://crawlee.dev/python/api/class/PydanticAiDirectExtractor
PydanticAiHtmlExtractorhttps://crawlee.dev/python/api/class/PydanticAiHtmlExtractor
PydanticAiSelectorExtractorhttps://crawlee.dev/python/api/class/PydanticAiSelectorExtractor
PydanticAiSkeletonDistillerhttps://crawlee.dev/python/api/class/PydanticAiSkeletonDistiller
PydanticAiUsageStatshttps://crawlee.dev/python/api/class/PydanticAiUsageStats
RenderingTypePredictionhttps://crawlee.dev/python/api/class/RenderingTypePrediction
RenderingTypePredictorhttps://crawlee.dev/python/api/class/RenderingTypePredictor
RequestOptionshttps://crawlee.dev/python/api/class/RequestOptions
Routerhttps://crawlee.dev/python/api/class/Router
https://crawlee.dev/python/
https://crawlee.dev/python/api/class/RequestOptions#Index
always_enqueuehttps://crawlee.dev/python/api/class/RequestOptions#always_enqueue
enqueue_strategyhttps://crawlee.dev/python/api/class/RequestOptions#enqueue_strategy
headershttps://crawlee.dev/python/api/class/RequestOptions#headers
idhttps://crawlee.dev/python/api/class/RequestOptions#id
keep_url_fragmenthttps://crawlee.dev/python/api/class/RequestOptions#keep_url_fragment
labelhttps://crawlee.dev/python/api/class/RequestOptions#label
max_retrieshttps://crawlee.dev/python/api/class/RequestOptions#max_retries
methodhttps://crawlee.dev/python/api/class/RequestOptions#method
no_retryhttps://crawlee.dev/python/api/class/RequestOptions#no_retry
payloadhttps://crawlee.dev/python/api/class/RequestOptions#payload
session_idhttps://crawlee.dev/python/api/class/RequestOptions#session_id
unique_keyhttps://crawlee.dev/python/api/class/RequestOptions#unique_key
urlhttps://crawlee.dev/python/api/class/RequestOptions#url
use_extended_unique_keyhttps://crawlee.dev/python/api/class/RequestOptions#use_extended_unique_key
user_datahttps://crawlee.dev/python/api/class/RequestOptions#user_data
https://crawlee.dev/python/api/class/RequestOptions#Properties
https://crawlee.dev/python/api/class/RequestOptions#always_enqueue
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L138
https://crawlee.dev/python/api/class/RequestOptions#enqueue_strategy
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L141
EnqueueStrategyhttps://crawlee.dev/python/api#EnqueueStrategy
https://crawlee.dev/python/api/class/RequestOptions#headers
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L130
HttpHeadershttps://crawlee.dev/python/api/class/HttpHeaders
https://crawlee.dev/python/api/class/RequestOptions#id
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L135
https://crawlee.dev/python/api/class/RequestOptions#keep_url_fragment
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L136
https://crawlee.dev/python/api/class/RequestOptions#label
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L132
https://crawlee.dev/python/api/class/RequestOptions#max_retries
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L142
https://crawlee.dev/python/api/class/RequestOptions#method
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L129
HttpMethodhttps://crawlee.dev/python/api#HttpMethod
https://crawlee.dev/python/api/class/RequestOptions#no_retry
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L140
https://crawlee.dev/python/api/class/RequestOptions#payload
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L131
HttpPayloadhttps://crawlee.dev/python/api#HttpPayload
https://crawlee.dev/python/api/class/RequestOptions#session_id
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L133
https://crawlee.dev/python/api/class/RequestOptions#unique_key
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L134
https://crawlee.dev/python/api/class/RequestOptions#url
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L128
https://crawlee.dev/python/api/class/RequestOptions#use_extended_unique_key
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L137
https://crawlee.dev/python/api/class/RequestOptions#user_data
https://github.com/apify/crawlee-python/blob/ea492fa9b5377f44082dc51d582e85d686b23c75//src/crawlee/_request.py#L139
PreviousRenderingTypePredictorhttps://crawlee.dev/python/api/class/RenderingTypePredictor
NextRouterhttps://crawlee.dev/python/api/class/Router
always_enqueuehttps://crawlee.dev/python/api/class/RequestOptions#always_enqueue
enqueue_strategyhttps://crawlee.dev/python/api/class/RequestOptions#enqueue_strategy
headershttps://crawlee.dev/python/api/class/RequestOptions#headers
idhttps://crawlee.dev/python/api/class/RequestOptions#id
keep_url_fragmenthttps://crawlee.dev/python/api/class/RequestOptions#keep_url_fragment
labelhttps://crawlee.dev/python/api/class/RequestOptions#label
max_retrieshttps://crawlee.dev/python/api/class/RequestOptions#max_retries
methodhttps://crawlee.dev/python/api/class/RequestOptions#method
no_retryhttps://crawlee.dev/python/api/class/RequestOptions#no_retry
payloadhttps://crawlee.dev/python/api/class/RequestOptions#payload
session_idhttps://crawlee.dev/python/api/class/RequestOptions#session_id
unique_keyhttps://crawlee.dev/python/api/class/RequestOptions#unique_key
urlhttps://crawlee.dev/python/api/class/RequestOptions#url
use_extended_unique_keyhttps://crawlee.dev/python/api/class/RequestOptions#use_extended_unique_key
user_datahttps://crawlee.dev/python/api/class/RequestOptions#user_data
https://crawlee.dev
Quick starthttps://crawlee.dev/python/docs/quick-start
Guideshttps://crawlee.dev/python/docs/guides
API referencehttps://crawlee.dev/python/api
Changeloghttps://crawlee.dev/python/docs/changelog
Discordhttps://discord.com/invite/jyEM2PRvMU
Stack Overflowhttps://stackoverflow.com/questions/tagged/crawlee-python
Twitterhttps://twitter.com/apify
YouTubehttps://www.youtube.com/apify
Apify platformhttps://apify.com
Docusaurushttps://docusaurus.io
GitHubhttps://github.com/apify/crawlee-python

Viewport: width=device-width, initial-scale=1.0


URLs of crawlers that visited me.