René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Serving HEAD requests from duckdb (CLI/WASM) read_csv() and read_json() calls","articleBody":"When using duckdb CLI or WASM as a client to an host with instance running the httpserver extension and reading CSV or JSON from a URL like \"https://some.httpserver.instance/api/?add_http_cors_header=1\u0026default_format=CSV\u0026q=select%2042\" it is possible to add basic auth to the request by doing\n\n```sql\n\n-- set up basic auth\ncreate secret http_auth_header(type HTTP, EXTRA_HTTP_HEADERS MAP {'Authorization': 'Basic \u003cbase64enc_of_user:pass\u003e'});\n\n-- retrieve data\nfrom read_csv('https://some.httpserver.instance/api/?add_http_cors_header=1\u0026default_format=CSV\u0026q=select 42');\n\n-- clean up\ndrop secret http_auth_header;\n\n```\n\nFor the duckdb CLI there is a setting `SET force_download = true` that can be used to stop HEAD requests to be sent (a setting which doesn't kick in when using duckdb WASM (yet), I think?) which avoids read_csv and read_json causing HEAD requests, since some servers just don't serve that verb but you can GET csv and/or json. \n\nI believe httpserver is one of those servers after a cursory glance at the code? I cannot find any mention of HEAD there and there or with regards to the CORS headers where GET, POST, OPTIONS and PUT are mentioned.\n\nWould it be possible to add support to httpserver for just the kind of HEAD requests that duckdb (CLI/WASM) makes when used for reading csv or json from a url served by httpserver?\n\nThere might even be fancy features such as serving range requests etc for (large, streaming) CSV content which read_csv() already makes use of that potentially could be supported by httpserver when it serves this content type?\n\n\n","author":{"url":"https://github.com/mskyttner","@type":"Person","name":"mskyttner"},"datePublished":"2025-10-23T09:40:18.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/47/httpserver/issues/47"}