René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`fs.openAsBlob()` does not work properly for files > 2GB","articleBody":"### Version\r\n\r\nv21.7.3 (and v20.12.2)\r\n\r\n### Platform\r\n\r\nDarwin bender.local 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T6000 arm64\r\n\r\n### Subsystem\r\n\r\nfs\r\n\r\n### What steps will reproduce the bug?\r\n\r\nSteps to reproduce bug:\r\n\r\n1. Generate a 3 GB file with `dd if=/dev/random of=random.bin bs=1048576 count=3072`\r\n2. Launch node (v21.7.3 and v20.12.2 both repro for me)\r\n3. `const b = await fs.openAsBlob(\"random.bin\")`\r\n4. `b.slice(2**31-2, 2**31-1)` \u003c-- works\r\n5. `b.slice(2**31-1, 2**31)` \u003c-- returns a slice of zero length\r\n6. `b.slice(2**31, 2**31+1)` \u003c-- crashes\r\n\r\nFull textual output of doing this given below.\r\n\r\nAlso, for files greater than 4GB, the size of the blob is wrong. It appears to be exactly `4*(2**30)` less than the actual size. For example, for a `5*(2**30)` byte file, the blob size is reported as `1073741824`, which is `(2**30)`. And, for a file that is exactly `4*(2**30)`, it reports a size of zero.\r\n\r\n### How often does it reproduce? Is there a required condition?\r\n\r\nAlways reproduces.\r\n\r\n### What is the expected behavior? Why is that the expected behavior?\r\n\r\nCan operate on the Blob normally.\r\n\r\n### What do you see instead?\r\n\r\nDescribed above. See output below.\r\n\r\n### Additional information\r\n\r\n```console\r\n$ dd if=/dev/random of=random.bin bs=1048576 count=3072\r\n3072+0 records in\r\n3072+0 records out\r\n3221225472 bytes transferred in 4.213743 secs (764457033 bytes/sec)\r\n$ ls -l random.bin\r\n-rw-r--r--  1 jbrandt  staff  3221225472 Apr 18 15:04 random.bin\r\n$ node -v\r\nv21.7.3\r\n$ node\r\nWelcome to Node.js v21.7.3.\r\nType \".help\" for more information.\r\n> const b = await fs.openAsBlob(\"random.bin\")\r\nundefined\r\n> b\r\nBlob { size: 3221225472, type: '' }\r\n> b.slice(2**31-2, 2**31-1)\r\nBlob { size: 1, type: '' }\r\n> b.slice(2**31-1, 2**31)\r\nBlob { size: 0, type: '' }\r\n> b.slice(2**31, 2**31+1)\r\n\r\n  #  node[96070]: static void node::Blob::ToSlice(const FunctionCallbackInfo\u003cv8::Value> &) at ../src/node_blob.cc:247\r\n  #  Assertion failed: args[0]->IsUint32()\r\n\r\n----- Native stack trace -----\r\n\r\n 1: 0x1041dfc0c node::Assert(node::AssertionInfo const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 2: 0x10589b4a4 node::Blob::ToSlice(v8::FunctionCallbackInfo\u003cv8::Value> const&) (.cold.2) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 3: 0x1041a8994 node::Blob::ToSlice(v8::FunctionCallbackInfo\u003cv8::Value> const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 4: 0x104c32a38 Builtins_CallApiCallbackGeneric [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 5: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 6: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 7: 0x104c2e8ac Builtins_JSEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 8: 0x104c2e594 Builtins_JSEntry [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n 9: 0x1044daef8 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n10: 0x1044db0e0 v8::internal::Execution::CallScript(v8::internal::Isolate*, v8::internal::Handle\u003cv8::internal::JSFunction>, v8::internal::Handle\u003cv8::internal::Object>, v8::internal::Handle\u003cv8::internal::Object>) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n11: 0x10439ef68 v8::Script::Run(v8::Local\u003cv8::Context>, v8::Local\u003cv8::Data>) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n12: 0x1041c8d1c node::contextify::ContextifyScript::EvalMachine(v8::Local\u003cv8::Context>, node::Environment*, long long, bool, bool, bool, v8::MicrotaskQueue*, v8::FunctionCallbackInfo\u003cv8::Value> const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n13: 0x1041c87fc node::contextify::ContextifyScript::RunInContext(v8::FunctionCallbackInfo\u003cv8::Value> const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n14: 0x104c32a38 Builtins_CallApiCallbackGeneric [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n15: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n16: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n17: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n18: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n19: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n20: 0x10a6531d0 \r\n21: 0x10a6ad6ac \r\n22: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n23: 0x104c30b84 Builtins_InterpreterEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n24: 0x10a66b508 \r\n25: 0x10a6767a8 \r\n26: 0x10a66cb7c \r\n27: 0x10a65307c \r\n28: 0x10a6ad6ac \r\n29: 0x10a6a73f0 \r\n30: 0x104c69e04 Builtins_GeneratorPrototypeNext [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n31: 0x10a65c654 \r\n32: 0x10a65307c \r\n33: 0x10a6ad6ac \r\n34: 0x10a66d390 \r\n35: 0x10a6754b8 \r\n36: 0x10a66e24c \r\n37: 0x10a675b48 \r\n38: 0x10a64f78c \r\n39: 0x104c2e8ac Builtins_JSEntryTrampoline [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n40: 0x104c2e594 Builtins_JSEntry [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n41: 0x1044daef8 v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n42: 0x1044da478 v8::internal::Execution::Call(v8::internal::Isolate*, v8::internal::Handle\u003cv8::internal::Object>, v8::internal::Handle\u003cv8::internal::Object>, int, v8::internal::Handle\u003cv8::internal::Object>*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n43: 0x1043b2404 v8::Function::Call(v8::Local\u003cv8::Context>, v8::Local\u003cv8::Value>, int, v8::Local\u003cv8::Value>*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n44: 0x104100fc0 node::InternalMakeCallback(node::Environment*, v8::Local\u003cv8::Object>, v8::Local\u003cv8::Object>, v8::Local\u003cv8::Function>, int, v8::Local\u003cv8::Value>*, node::async_context) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n45: 0x104117c00 node::AsyncWrap::MakeCallback(v8::Local\u003cv8::Function>, int, v8::Local\u003cv8::Value>*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n46: 0x1042af900 node::StreamBase::CallJSOnreadMethod(long, v8::Local\u003cv8::ArrayBuffer>, unsigned long, node::StreamBase::StreamBaseJSChecks) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n47: 0x1042b0f7c node::EmitToJSStreamListener::OnStreamRead(long, uv_buf_t const&) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n48: 0x1042b526c node::LibuvStreamWrap::OnUvRead(long, uv_buf_t const*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n49: 0x1042b59a0 node::LibuvStreamWrap::ReadStart()::$_1::__invoke(uv_stream_s*, long, uv_buf_t const*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n50: 0x104c1a390 uv__stream_io [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n51: 0x104c21da4 uv__io_poll [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n52: 0x104c0fda4 uv_run [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n53: 0x1041016f0 node::SpinEventLoopInternal(node::Environment*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n54: 0x10421f754 node::NodeMainInstance::Run(node::ExitCode*, node::Environment*) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n55: 0x10421f4d4 node::NodeMainInstance::Run() [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n56: 0x10419eb48 node::Start(int, char**) [/Users/jbrandt/.nvm/versions/node/v21.7.3/bin/node]\r\n57: 0x19e7060e0 start [/usr/lib/dyld]\r\n\r\n----- JavaScript stack trace -----\r\n\r\n1: slice (node:internal/blob:265:21)\r\n2: REPL5:1:3\r\n3: runInThisContext (node:vm:136:12)\r\n4: defaultEval (node:repl:598:22)\r\n5: bound (node:domain:432:15)\r\n6: runBound (node:domain:443:12)\r\n7: onLine (node:repl:927:10)\r\n8: emit (node:events:531:35)\r\n9: emit (node:domain:488:12)\r\n10: [_onLine] (node:internal/readline/interface:416:12)\r\n\r\n\r\nzsh: abort      node\r\n```","author":{"url":"https://github.com/joelrbrandt","@type":"Person","name":"joelrbrandt"},"datePublished":"2024-04-18T22:05:51Z","url":"https://github.com/nodejs/node/issues/52585"}