René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"HTTP2 download of parallel requests is slower than HTTP1.1","articleBody":"### Version\r\n\r\nv20.17.0\r\n\r\n### Platform\r\n\r\n```text\r\nMacOS Sonoma 14.5, 2,3 GHz Quad-Core Intel Core i7\r\n```\r\n\r\n\r\n### Subsystem\r\n\r\n_No response_\r\n\r\n### What steps will reproduce the bug?\r\n\r\nDefine an http2 and http1.1 clients and fetch some data in parallel.\r\n\r\nYou can use the script below. If you run it using the following command, it will send 25 parallel requests using HTTP1.1\r\n```sh\r\nnode client.js\r\n```\r\n\r\nor you can run this command and it will send 25 parallel requests using HTTP2:\r\n```sh\r\nnode client.js http2\r\n```\r\n\r\n\u003cdetails>\r\n\u003csummary>client.js\u003c/summary>\r\n\u003cp>\r\n\r\n\r\n\r\n\r\n\r\n```javascript\r\nimport http2 from 'node:http2';\r\nimport https from 'node:https';\r\n\r\nconst isHttp2 = process.argv[2] === 'http2';\r\n\r\nconst MB = 1024 * 1024;\r\nconst WINDOW_SIZE = 32 * MB;\r\n\r\nconst url = new URL('https://nodejs.org/dist/v17.1.0/node-v17.1.0-x64.msi');\r\n\r\nconst client = http2.connect(url, {\r\n  settings: {\r\n    initialWindowSize: WINDOW_SIZE,\r\n  },\r\n});\r\n\r\nclient.on('connect', () => {\r\n  client.setLocalWindowSize(WINDOW_SIZE);\r\n});\r\n\r\nconst fetchHttp1 = (id) => {\r\n  return new Promise((resolve) => {\r\n    const req = https.request({\r\n        host: url.host,\r\n        path: url.pathname,\r\n        headers: {\r\n          connection: 'keep-alive',\r\n        },\r\n      }, (res) => {\r\n        res.on('data', () => {});\r\n        res.on('end', () => {\r\n          console.log(`Complete request with id: ${id}`);\r\n          resolve();\r\n        });\r\n      },\r\n    );\r\n\r\n    req.end();\r\n  });\r\n};\r\n\r\nconst fetchHttp2 = (id) => {\r\n  return new Promise((resolve) => {\r\n    const req = client.request({\r\n      ':path': url.pathname,\r\n    });\r\n    req.on('data', () => {});\r\n    req.on('end', () => {\r\n      console.log(`Complete request with id: ${id}`);\r\n      resolve();\r\n    });\r\n    req.end();\r\n  });\r\n};\r\n\r\nconst main = async () => {\r\n  console.log(`Starting requests using HTTP${isHttp2 ? '2' : '1.1'} protocol.`);\r\n\r\n  const startTime = Date.now();\r\n\r\n  await Promise.all(\r\n    Array.from({ length: 25 })\r\n      .fill(null)\r\n      .map((_, index) => isHttp2 ? fetchHttp2(index) : fetchHttp1(index)),\r\n  );\r\n\r\n  console.log(`Requests complete. Completion time: ${(Date.now() - startTime) / 1000}s`);\r\n};\r\n\r\nmain();\r\n``` \r\n\r\n\r\n\r\n\r\n\u003c/p>\r\n\u003c/details> \r\n\r\n\r\n### How often does it reproduce? Is there a required condition?\r\n\r\nIt's not quite stable, because sometimes http2 has the same performance or even better performance.\r\n\r\n### What is the expected behavior? Why is that the expected behavior?\r\n\r\nI expect http2 to be faster or at least have the same performance as http1.1.\r\n\r\n### What do you see instead?\r\n\r\nInstead, I often see that http1.1 requests are faster. Most of the times http2 took 25s to complete while http1.1 took 20s.\r\n\r\nThere's a problem, though, because those results are not consistent when using the URL I provided in the example. \r\n\r\nBut, the results are consistent with some internal APIs I'm working with. First, I thought there might be a problem with our internal APIs, but making 25 parallel requests to our internal APIs using `curl` has consistently the same performance on both http 1.1 and http2.\r\n\r\nI ran curl with the following command:\r\n```sh\r\ncurl --config ./config.txt --parallel --parallel-max 25 --http2\r\n```\r\n\r\nwhere config.txt looked like this:\r\n```txt\r\nurl = \"our-internal-api\"\r\noutput = \"/dev/null\"\r\n// repeated 25 times\r\n```\r\n\r\n### Additional information\r\n\r\nI was thinking the reason for such slowness would be TCP stall because http2 uses only 1 tcp connection, so I checked in wireshark how many tcp connections creates node vs curl when using http2, and both created only 1 tcp connection.\r\n\r\nAlso, I noticed node receives first 10 http2 requests randomly, but then sequentially, while http1 requests are all received randomly.\r\n\r\nhttp2:\r\n\r\n\u003cimg width=\"405\" alt=\"image\" src=\"https://github.com/user-attachments/assets/7b1b155e-656b-4266-aca9-64cb009f02cd\">\r\n\r\n---\r\n\r\nhttp1:\r\n\r\n\u003cimg width=\"424\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f317b751-ed98-44aa-b562-1854ccc88835\">\r\n","author":{"url":"https://github.com/stanislav-halyn","@type":"Person","name":"stanislav-halyn"},"datePublished":"2024-09-10T14:46:45Z","url":"https://github.com/nodejs/node/issues/54874"}