René's URL Explorer Experiment

{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://urlpipe.dev/#organization","name":"URLpipe","url":"https://urlpipe.dev","logo":"https://urlpipe.dev/icon.png","sameAs":["https://github.com/URLpipe"],"founder":{"@type":"Person","name":"Roger Campos","url":"https://rogercampos.com"}},{"@type":"WebSite","@id":"https://urlpipe.dev/#website","url":"https://urlpipe.dev","name":"URLpipe","publisher":{"@id":"https://urlpipe.dev/#organization"}}]}

{"@context":"https://schema.org","@graph":[{"@type":"TechArticle","@id":"https://urlpipe.dev/code/java/html-to-markdown#article","headline":"Convert a URL to Markdown in Java — working code","description":"Turn any URL into clean Markdown in Java, rendered in real Chrome first: sync, async with a webhook, and retries. Tested code.","url":"https://urlpipe.dev/code/java/html-to-markdown","mainEntityOfPage":"https://urlpipe.dev/code/java/html-to-markdown","datePublished":"2026-09-24","dateModified":"2026-09-24","author":{"@type":"Person","name":"Roger Campos","url":"https://rogercampos.com"},"publisher":{"@id":"https://urlpipe.dev/#organization"},"image":"https://urlpipe.dev/og-image.png"},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Does the Markdown conversion use an LLM?","acceptedAnswer":{"@type":"Answer","text":"No. It is a deterministic walk over the rendered DOM, so it is fast, costs 1 credit a page, and gives the same output for the same page. Only /summarize, /keywords, and /extract call a model."}},{"@type":"Question","name":"Does it work on pages built with JavaScript?","acceptedAnswer":{"@type":"Answer","text":"Yes. The page is loaded in real Chrome and its scripts run before the DOM is read, so React, Vue and other client-rendered pages convert like static ones."}},{"@type":"Question","name":"How do I keep the Markdown for later?","acceptedAnswer":{"@type":"Answer","text":"Write the body to a .md file, or rely on the cache: the same URL within max_age (7 days by default) is answered from storage for free, and any result can be fetched again by its token for 30 days."}},{"@type":"Question","name":"Do I need an SDK to call URLpipe from Java?","acceptedAnswer":{"@type":"Answer","text":"No SDK. HttpClient is in the JDK; the programs that read JSON add Gson, one jar, because the JDK has no parser."}}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://urlpipe.dev/"},{"@type":"ListItem","position":2,"name":"Code recipes","item":"https://urlpipe.dev/code"},{"@type":"ListItem","position":3,"name":"Java","item":"https://urlpipe.dev/code/java"},{"@type":"ListItem","position":4,"name":"Convert a web page to Markdown in Java","item":"https://urlpipe.dev/code/java/html-to-markdown"}]},{"@type":"HowTo","name":"Convert a web page to Markdown in Java","description":"To convert a web page to Markdown in Java, POST its URL to https://urlpipe.dev/markdown with sync set to true; the response body is the Markdown, read with HttpResponse.BodyHandlers.ofString(). The page is rendered in real Chrome first, then converted by a deterministic walk over the DOM, not a model, for 1 credit a page.","url":"https://urlpipe.dev/code/java/html-to-markdown","tool":[{"@type":"HowToTool","name":"Java 17+"}],"step":[{"@type":"HowToStep","name":"Before you start","text":"Java 17 or later: java.net.http is in the JDK. The async and error-handling programs read JSON, and the JDK has no parser, so those use Gson — one jar, no dependencies of its own. In a Maven or Gradle build add com.google.code.gson:gson:2.13.1; to try a file on its own, download the jar next to it.","url":"https://urlpipe.dev/code/java/html-to-markdown#setup"},{"@type":"HowToStep","name":"Print a page as Markdown","text":""sync": true keeps the request open until the result is ready. BodyHandlers.ofString() reads the whole body as text, decoded with the charset the response names. HttpClient does not throw on a 4xx or 5xx, so check statusCode(). The body is the Markdown itself, so the String is the whole job.","url":"https://urlpipe.dev/code/java/html-to-markdown#sync"},{"@type":"HowToStep","name":"The async variant: a token, a webhook and a poll","text":"Leave out sync and the answer is a token, straight away. One HttpClient is shared by every call — it holds the connection pool, so build it once — and a switch expression covers the answers GET /result/:token can give.","url":"https://urlpipe.dev/code/java/html-to-markdown#async"},{"@type":"HowToStep","name":"Handle errors and retries","text":"A checked URLpipeException makes the caller decide what a refusal means. firstValueAsLong reads Retry-After without a parse of your own, and a body that is not JSON (a proxy in front of the API can answer in HTML) falls back to an empty object.","url":"https://urlpipe.dev/code/java/html-to-markdown#errors"}]}]}