René's URL Explorer Experiment


Title: CrawlOptions - Reader Docs

Open Graph Title: Reader - Open-source web scraping for LLMs

X Title: Reader - Open-source web scraping for LLMs

Description: Every field accepted by crawl() with type and default.

Open Graph Description: Production-grade web scraping engine. Two primitives. Clean markdown. Ready for your agents.

X Description: Production-grade web scraping engine. Two primitives. Clean markdown. Ready for your agents.

Keywords:

Opengraph URL: https://reader.dev

direct link

Domain: reader.dev

authorVakra Dev
og:site_nameReader
og:imagehttps://reader.dev/og-image.png
og:typewebsite
twitter:cardsummary_large_image
twitter:imagehttps://reader.dev/og-image.png

Links:

https://reader.dev/
Markdownhttps://reader.dev/markdown
Docshttps://reader.dev/docs
https://github.com/vakra-dev/reader
Introductionhttps://reader.dev/docs/getting-started/introduction
Installationhttps://reader.dev/docs/getting-started/installation
Quickstarthttps://reader.dev/docs/getting-started/quickstart
Exampleshttps://reader.dev/docs/getting-started/examples
Scrapinghttps://reader.dev/docs/concepts/scraping
Crawlinghttps://reader.dev/docs/concepts/crawling
Content Extractionhttps://reader.dev/docs/concepts/content-extraction
Browser Poolhttps://reader.dev/docs/concepts/browser-pool
Browser Sessionshttps://reader.dev/docs/concepts/browser-sessions
Engine Waterfallhttps://reader.dev/docs/concepts/engine-waterfall
Proxy Tiershttps://reader.dev/docs/concepts/proxy-tiers
Error Handlinghttps://reader.dev/docs/concepts/error-handling
Basic Scrapinghttps://reader.dev/docs/guides/basic-scraping
Batch Scrapinghttps://reader.dev/docs/guides/batch-scraping
Website Crawlinghttps://reader.dev/docs/guides/website-crawling
Proxy Configurationhttps://reader.dev/docs/guides/proxy-configuration
CLIhttps://reader.dev/docs/guides/cli
Daemon Modehttps://reader.dev/docs/guides/daemon-mode
Browser Sessionshttps://reader.dev/docs/guides/browser-sessions
Deploymenthttps://reader.dev/docs/guides/deployment
ReaderClienthttps://reader.dev/docs/api-reference/reader-client
scrape()https://reader.dev/docs/api-reference/scrape
crawl()https://reader.dev/docs/api-reference/crawl
ScrapeOptionshttps://reader.dev/docs/api-reference/scrape-options
ScrapeResulthttps://reader.dev/docs/api-reference/scrape-result
CrawlOptionshttps://reader.dev/docs/api-reference/crawl-options
CrawlResulthttps://reader.dev/docs/api-reference/crawl-result
BrowserSessionhttps://reader.dev/docs/api-reference/browser-session
Errorshttps://reader.dev/docs/api-reference/errors
CrawlResultThe return type for every crawl call.https://reader.dev/docs/api-reference/crawl-result
Crawling conceptHow BFS link discovery works.https://reader.dev/docs/concepts/crawling

Viewport: width=device-width, initial-scale=1

URLs of crawlers that visited me.