René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"zstd decoding complete frame in read stream halts stream","articleBody":"### Version\n\nv26.5.0\n\n### Platform\n\n```text\nDarwin Mac.lan 24.6.0 Darwin Kernel Version 24.6.0: Tue Apr 21 20:19:12 PDT 2026; root:xnu-11417.140.69.710.16~1/RELEASE_ARM64_T6041 arm64\n```\n\n### Subsystem\n\n_No response_\n\n### What steps will reproduce the bug?\n\n```js\nimport { Buffer } from 'node:buffer';\nimport * as consumers from 'node:stream/consumers';\nimport * as zlib from 'node:zlib';\n\nfunction consume(codec, writes) {\n\tconst inflate = codec.createDecompress();\n\tconst result = consumers.buffer(inflate);\n\twrites.forEach(write => inflate.write(write));\n\tinflate.end();\n\treturn result;\n}\n\nfunction describe(buffer) {\n\tconst aa = buffer.includes(0x41);\n\tconst bb = buffer.includes(0x42);\n\tconst which = function() {\n\t\tif (aa && bb) {\n\t\t\treturn 'frameA + frameB';\n\t\t} else if (aa) {\n\t\t\treturn 'frameA only';\n\t\t} else if (bb) {\n\t\t\treturn 'frameB only';\n\t\t} else {\n\t\t\treturn 'nothing';\n\t\t}\n\t}();\n\treturn `${buffer.length} bytes (${which})`;\n}\n\nasync function attempt(codec, writes) {\n\ttry {\n\t\treturn describe(await consume(codec, writes));\n\t} catch (error) {\n\t\treturn `threw: ${error.message}`;\n\t}\n}\n\nconst tests = [ {\n\tname: 'zstd',\n\tcompress: buffer => zlib.zstdCompressSync(buffer),\n\tcreateDecompress: () => zlib.createZstdDecompress(),\n}, {\n\tname: 'gzip',\n\tcompress: buffer => zlib.gzipSync(buffer),\n\tcreateDecompress: () => zlib.createGunzip(),\n} ];\n\nconsole.log(`node ${process.version}`);\nfor (const codec of tests) {\n\t// 'A' / 'B'\n\tconst frameA = codec.compress(Buffer.alloc(2000, 0x41));\n\tconst frameB = codec.compress(Buffer.alloc(2000, 0x42));\n\tconst concatenated = Buffer.concat([ frameA, frameB ]);\n\n\t// BUG: both frames in a single write — the boundary between them falls inside one input buffer.\n\tconst bug = await attempt(codec, [ concatenated ]);\n\t// COUNTER-EXAMPLE: the same two frames, one per write — the boundary lands at a write boundary.\n\tconst counter = await attempt(codec, [ frameA, frameB ]);\n\n\tconsole.log(`${codec.name}:`);\n\tconsole.log(`  stream, both frames in ONE write   -> ${bug}`);\n\tconsole.log(`  stream, one frame per write        -> ${counter}`);\n}\n```\n\n### How often does it reproduce? Is there a required condition?\n\nThis reproduces every time\n\n### What is the expected behavior? Why is that the expected behavior?\n\nDecoding concatenated zstd payloads should decode correctly as one stream. Indeed, `zstdcat` will decode this correctly (omitted from example but you can take my word for it).\n\n### What do you see instead?\n\n```\nmarcel[1:42:21PM] [~/xx] ~/Downloads/node-v26.5.0-darwin-arm64/bin/node a.mjs\nnode v26.5.0\nzstd:\n  stream, both frames in ONE write   -> 2000 bytes (frameA only)\n  stream, one frame per write        -> 4000 bytes (frameA + frameB)\ngzip:\n  stream, both frames in ONE write   -> 4000 bytes (frameA + frameB)\n  stream, one frame per write        -> 4000 bytes (frameA + frameB)\n```\n\n### Additional information\n\nThis will cause spurious failures while decoding zstd streams. The odds of it happening are a function of stream read size vs zstd chunk size. In my case I ran into this while decoding a stream after about 3GB and tracked it down to this bug.","author":{"url":"https://github.com/laverdet","@type":"Person","name":"laverdet"},"datePublished":"2026-07-25T18:45:40Z","url":"https://github.com/nodejs/node/issues/64741"}