René's URL Explorer Experiment

{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://urlpipe.dev/#organization","name":"URLpipe","url":"https://urlpipe.dev","logo":"https://urlpipe.dev/icon.png","sameAs":["https://github.com/URLpipe"],"founder":{"@type":"Person","name":"Roger Campos","url":"https://rogercampos.com"}},{"@type":"WebSite","@id":"https://urlpipe.dev/#website","url":"https://urlpipe.dev","name":"URLpipe","publisher":{"@id":"https://urlpipe.dev/#organization"}}]}

{"@context":"https://schema.org","@graph":[{"@type":"TechArticle","@id":"https://urlpipe.dev/code/node/rendered-html#article","headline":"Get JavaScript-rendered HTML in Node.js — working code","description":"Fetch the HTML of a JavaScript page after it rendered, in Node.js: sync, async with a webhook, and error handling. Tested code.","url":"https://urlpipe.dev/code/node/rendered-html","mainEntityOfPage":"https://urlpipe.dev/code/node/rendered-html","datePublished":"2026-09-24","dateModified":"2026-09-24","author":{"@type":"Person","name":"Roger Campos","url":"https://rogercampos.com"},"publisher":{"@id":"https://urlpipe.dev/#organization"},"image":"https://urlpipe.dev/og-image.png"},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"How is this different from fetching the URL myself?","acceptedAnswer":{"@type":"Answer","text":"A plain GET returns what the server sent before any JavaScript ran. /html returns the DOM after the page's scripts ran in real Chrome — the HTML a visitor's browser actually holds."}},{"@type":"Question","name":"Can I wait for content that loads late?","acceptedAnswer":{"@type":"Answer","text":"Yes: page_options.wait_for_selector waits up to 10 seconds for an element to appear, and delay adds a fixed wait. If the element never appears the request is a 422 and costs nothing."}},{"@type":"Question","name":"Does URLpipe follow robots.txt?","acceptedAnswer":{"@type":"Answer","text":"Yes, by default, for every project. You can turn it off per project, and then you are responsible for having the right to fetch those pages."}},{"@type":"Question","name":"Do I need an SDK to call URLpipe from Node.js?","acceptedAnswer":{"@type":"Answer","text":"No package is needed: Node 18 and later ship fetch, and the API is one POST per job. If you'd rather use one, the official client installs with npm install @urlpipe/sdk and adds retries that never bill twice, typed errors and webhook verification."}}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://urlpipe.dev/"},{"@type":"ListItem","position":2,"name":"Code recipes","item":"https://urlpipe.dev/code"},{"@type":"ListItem","position":3,"name":"Node.js","item":"https://urlpipe.dev/code/node"},{"@type":"ListItem","position":4,"name":"Get the rendered HTML of a JavaScript page in Node.js","item":"https://urlpipe.dev/code/node/rendered-html"}]},{"@type":"HowTo","name":"Get the rendered HTML of a JavaScript page in Node.js","description":"To get the HTML of a JavaScript-rendered page in Node.js, POST its URL to https://urlpipe.dev/html with sync set to true. The body is the DOM after the page's scripts ran in real Chrome, serialized as HTML — what a visitor's browser holds, not what the server first sent — read with await res.text(), for 1 credit.","url":"https://urlpipe.dev/code/node/rendered-html","tool":[{"@type":"HowToTool","name":"Node.js 18+"}],"step":[{"@type":"HowToStep","name":"Before you start","text":"Nothing to install: fetch is global from Node 18. The files end in .mjs so Node reads them as ES modules and top-level await works without a wrapper function.","url":"https://urlpipe.dev/code/node/rendered-html#setup"},{"@type":"HowToStep","name":"Save the rendered HTML and report its size","text":"sync: true keeps the request open until the result is ready. fetch resolves on any status, 4xx and 5xx included, so check res.ok yourself. Buffer.byteLength counts bytes; html.length would count UTF-16 code units and come out smaller on any page with non-ASCII text.","url":"https://urlpipe.dev/code/node/rendered-html#sync"},{"@type":"HowToStep","name":"The async variant: a token, a webhook and a poll","text":"Leave out sync and the answer is a token, straight away. setTimeout from node:timers/promises is the awaitable sleep, so the polling loop reads top to bottom.","url":"https://urlpipe.dev/code/node/rendered-html#async"},{"@type":"HowToStep","name":"Handle errors and retries","text":"Read the status before the body: a 401 answers in plain text, so res.json() would throw. .catch(() => ({})) turns any body that is not JSON into an empty object, and the status still says what happened.","url":"https://urlpipe.dev/code/node/rendered-html#errors"}]}]}