René's URL Explorer Experiment

{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://urlpipe.dev/#organization","name":"URLpipe","url":"https://urlpipe.dev","logo":"https://urlpipe.dev/icon.png","sameAs":["https://github.com/URLpipe"],"founder":{"@type":"Person","name":"Roger Campos","url":"https://rogercampos.com"}},{"@type":"WebSite","@id":"https://urlpipe.dev/#website","url":"https://urlpipe.dev","name":"URLpipe","publisher":{"@id":"https://urlpipe.dev/#organization"}}]}

{"@context":"https://schema.org","@graph":[{"@type":"SoftwareApplication","name":"URLpipe","applicationCategory":"DeveloperApplication","operatingSystem":"Web","url":"https://urlpipe.dev","description":"Markdown, screenshots, metadata, AI summaries and Lighthouse audits from any URL — one API, over HTTP or MCP, rendered in real Chrome. Start free, no card.","offers":[{"@type":"Offer","name":"Free","price":"0","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"0","priceCurrency":"USD","unitCode":"MON","billingIncrement":1,"billingDuration":"P1M"},"description":"1,000 credits per month — 1,000 page fetches or markdown conversions, or 58 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Starter","price":"19","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"19","priceCurrency":"USD","unitCode":"MON","billingIncrement":1,"billingDuration":"P1M"},"description":"20,000 credits per month — 20,000 page fetches or markdown conversions, or 1,176 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Starter, billed annually","price":"190","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"190","priceCurrency":"USD","unitCode":"ANN","billingIncrement":1,"billingDuration":"P1Y"},"description":"20,000 credits per month — 20,000 page fetches or markdown conversions, or 1,176 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Pro","price":"49","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"49","priceCurrency":"USD","unitCode":"MON","billingIncrement":1,"billingDuration":"P1M"},"description":"55,000 credits per month — 55,000 page fetches or markdown conversions, or 3,235 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Pro, billed annually","price":"490","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"490","priceCurrency":"USD","unitCode":"ANN","billingIncrement":1,"billingDuration":"P1Y"},"description":"55,000 credits per month — 55,000 page fetches or markdown conversions, or 3,235 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Scale","price":"149","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"149","priceCurrency":"USD","unitCode":"MON","billingIncrement":1,"billingDuration":"P1M"},"description":"175,000 credits per month — 175,000 page fetches or markdown conversions, or 10,294 AI summaries, or any mix. Repeat requests served from cache are free."},{"@type":"Offer","name":"Scale, billed annually","price":"1490","priceCurrency":"USD","priceSpecification":{"@type":"UnitPriceSpecification","price":"1490","priceCurrency":"USD","unitCode":"ANN","billingIncrement":1,"billingDuration":"P1Y"},"description":"175,000 credits per month — 175,000 page fetches or markdown conversions, or 10,294 AI summaries, or any mix. Repeat requests served from cache are free."}]},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Do I need to install anything on my site?","acceptedAnswer":{"@type":"Answer","text":"No. URLpipe calls your public URLs from our servers. No scripts, no DNS records, no server access required."}},{"@type":"Question","name":"Can I try it without signing up?","acceptedAnswer":{"@type":"Answer","text":"Yes. There are free, no-signup tools at /tools for every kind of data the API returns — paste a URL and see the result in your browser. It's the same API underneath: the example pages are free and don't count against your own runs, which are limited to a couple every ten minutes. When you're ready to build, the free plan's API key takes a minute."}},{"@type":"Question","name":"Can my AI agent use it?","acceptedAnswer":{"@type":"Answer","text":"Yes — URLpipe is also a remote MCP server. Point Claude, Cursor or any MCP client at https://urlpipe.dev/mcp with a bearer token and the agent can read pages as Markdown, take screenshots, pull metadata and run audits on its own, rendering JavaScript as it goes. There is nothing to install, and the tools cost the same credits as the HTTP endpoints."}},{"@type":"Question","name":"Is there an SDK?","acceptedAnswer":{"@type":"Answer","text":"Yes — official clients for Python (pip install urlpipe), JavaScript and TypeScript (npm install @urlpipe/sdk), Ruby (gem install urlpipe) and Go, with retries that never bill twice, typed errors and webhook verification. There are also packages for LangChain and LlamaIndex, an n8n node and a command-line tool (brew install urlpipe/tap/urlpipe). None of them is required: every endpoint is one POST with a bearer token, so curl or any HTTP library works just as well. The client libraries page in the docs has each one."}},{"@type":"Question","name":"Which endpoints use AI?","acceptedAnswer":{"@type":"Answer","text":"/summarize, /keywords, and /extract. The rest don't — /markdown converts the rendered page itself and /meta reads what the page declares, which is why each costs the same as a page fetch and gives the same answer every time. One credit allowance covers all of them, so you spend it on whatever mix your workload needs."}},{"@type":"Question","name":"Can I use URLpipe to feed web pages to an LLM?","acceptedAnswer":{"@type":"Answer","text":"Yes. /markdown returns the page's main content as clean Markdown, with the navigation and boilerplate left behind — the format LLMs and RAG pipelines work best with. Pages are rendered in real Chrome first, so JavaScript-heavy sites produce complete content too, and page_options can remove cookie banners and ads before the conversion."}},{"@type":"Question","name":"Can I get several things from one page at once?","acceptedAnswer":{"@type":"Answer","text":"Yes — /scrape runs any mix of the operations off a single page visit: a screenshot, the metadata and the Markdown for a link preview, say. You pay for each operation as usual, and the page is only loaded once, which is the slow part."}},{"@type":"Question","name":"How is usage priced?","acceptedAnswer":{"@type":"Answer","text":"In credits. Every successful, uncached call spends credits according to what it costs to run: /console, /html, /markdown, /meta, /pdf, /screenshot and /technologies cost 1 credit each; /lighthouse 2; /keywords 15; /summarize 17; /extract 24. Plans are a monthly credit allowance you spend on whatever mix you like, and there's a calculator on the pricing page. The one extra is a residential exit (25 credits per page visit), which is off unless you ask for it."}},{"@type":"Question","name":"Some sites serve a scraper less than they serve a browser. Can URLpipe get the real page?","acceptedAnswer":{"@type":"Answer","text":"Usually, yes. Every request is a real Chrome, which is enough for most of what stops a plain HTTP fetch, and we work through bot checks and retry before you hear about them — a check we can't clear costs you nothing. When a site goes further and reads the address the request came from, add residential: true and we fetch the page from a home broadband line instead of a data center. It works on every endpoint, costs 25 credits per page visit on top of the operation, and is off by default."}},{"@type":"Question","name":"Can URLpipe read pages behind a login?","acceptedAnswer":{"@type":"Answer","text":"Yes. Sign in once in a browser, save its state (Playwright's storageState — cookies and localStorage), and upload it as an auth profile. Name the profile on any request and the page is loaded signed in: no login to script, no password to hand over, and nothing extra to pay. What the site renews during each visit is kept, so the session lives as long as you use it. The state is encrypted, never returned by the API, and used only on the domains you choose."}},{"@type":"Question","name":"What happens when I run out of credits?","acceptedAnswer":{"@type":"Answer","text":"You hear about it first: we email your organization admins when the month's usage reaches 80% of the allowance. On a paid plan, nothing stops: calls keep being served, uncapped, and every credit past the allowance costs $1.50 per 1,000 credits, billed on an invoice of its own once the month closes. On the Free plan, which has no card to bill the extra to, the endpoint returns HTTP 429 with a JSON body giving your limit, what you've used, how many credits the refused request needed, and when the allowance resets (the 1st of next month, UTC). Every response carries X-Quota-Cost, X-Quota-Remaining and X-Quota-Overage, and the dashboard itemises the month by operation."}},{"@type":"Question","name":"Do repeated requests spend credits?","acceptedAnswer":{"@type":"Answer","text":"No. When you request the same URL and operation again within your freshness window, we serve the cached result — instantly, and without spending a credit. A repeat that arrives while the first request is still running waits for it, free as well. You control the window with the max_age parameter (default 7 days); set max_age=0 to always fetch fresh."}},{"@type":"Question","name":"Can I tag requests by customer or job?","acceptedAnswer":{"@type":"Answer","text":"Yes. Send labels — up to 16 of your own ids, like a client or a job — with any request. They come back with the result and the webhook, and the dashboard shows usage broken down by label, which is what an agency or a multi-tenant app needs to bill its own customers."}},{"@type":"Question","name":"Can my team use the same account?","acceptedAnswer":{"@type":"Answer","text":"Yes — every paid plan takes as many team members as you need. Invite them by email; everyone shares the projects, the API keys and the monthly credits, and admins manage billing. One login can belong to several organizations too, one per client or company, each billed on its own."}},{"@type":"Question","name":"What does async mode do?","acceptedAnswer":{"@type":"Answer","text":"Async is the default. URLpipe accepts the request immediately, hands you a token, and processes it in the background. Collect the result whichever way suits you: give us a webhook URL and we POST it to you — signed, if you turn signing on, and retried if your endpoint is down — or fetch it with GET /result/:token when you're ready. Send sync: true when you'd rather wait for the result in the response."}},{"@type":"Question","name":"Can I run requests in parallel?","acceptedAnswer":{"@type":"Answer","text":"Yes — each plan sets how many run at once, alongside its monthly credits. The free plan runs one at a time, and the paid plans raise it to 3, 8 and 20. Past that we size it with you. A crawl of 5,000 pages costs the same credits however fast you run it, so running several at once is the difference between minutes and hours. Every response carries X-Concurrency-Limit, so a worker pool can size itself from the plan. Go over and you get HTTP 429 with error \"concurrency_limit\" — nothing is spent, and the request goes through as soon as one of yours finishes."}},{"@type":"Question","name":"Can I keep everything in the EU?","acceptedAnswer":{"@type":"Answer","text":"Yes. Pages are fetched, rendered and stored on our own servers in Europe on every plan — that already covers /html, /screenshot, /markdown, /meta, /lighthouse, /console, /technologies and /pdf, which never call an AI model. The one step that can sit outside Europe is the model behind /summarize, /keywords and /extract, and an organization admin can switch that to an EU-only endpoint from the dashboard. No surcharge, no change to your integration. The data residency docs have the specifics."}},{"@type":"Question","name":"Where is my data stored?","acceptedAnswer":{"@type":"Answer","text":"Account data, request history and stored results live on our own infrastructure in Europe. Stored results are private to your organization and kept for 30 days. We don't sell or share any of it. See the privacy policy for the full picture."}}]}]}