René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Racing immediately-resolving Promises leads to memory leak","articleBody":"### Version\r\n\r\n21.5.0\r\n\r\n### Platform\r\n\r\nLinux penguin 6.1.55-06877-gc83437f2949f #1 SMP PREEMPT_DYNAMIC Thu Dec 14 19:17:39 PST 2023 x86_64 GNU/Linux\r\n\r\n### Subsystem\r\n\r\nasync_hooks or async/await\r\n\r\n### What steps will reproduce the bug?\r\n\r\nRun the following code. Failure is quicker and less likely to interfere with system stability if you run with a low heap ceiling like this, but it will fail without...\r\n\r\n```\r\nnode --max-old-space-size=16 test/examples/ephemeralPromiseMemoryLeak.js\r\n```\r\n\r\n```js\r\n//ephemeralPromiseMemoryLeak.js\r\n\r\nasync function promiseValue(value) {\r\n  return value;\r\n}\r\n\r\nasync function run() {\r\n  for (;;) {\r\n    await Promise.race([promiseValue(\"foo\"), promiseValue(\"bar\")]);\r\n  }\r\n}\r\n\r\nrun();\r\n```\r\n\r\nAn equivalent OOM is created if you substitute `Promise.any` for `Promise.race`...\r\n\r\n```js\r\nasync function promiseValue(value) {\r\n  return value;\r\n}\r\n\r\nasync function run() {\r\n  for (;;) {\r\n    await Promise.any([promiseValue(\"foo\"), promiseValue(\"bar\")]);\r\n  }\r\n}\r\n\r\nrun();\r\n```\r\n\r\n### How often does it reproduce? Is there a required condition?\r\n\r\nIt always fails.\r\n\r\n### What is the expected behavior? Why is that the expected behavior?\r\n\r\nI would expect it not to accumulate references in memory and fail.\r\n\r\n### What do you see instead?\r\n\r\nFails with the following error\r\n\r\n```sh\r\n✗ node test/examples/ephemeralPromiseMemoryLeak.js\r\n\r\n\u003c--- Last few GCs ---\u003e\r\n\r\n[31511:0x6de4330]    39874 ms: Mark-Compact (reduce) 2048.2 (2083.6) -\u003e 2047.3 (2083.9) MB, 1308.67 / 0.00 ms  (average mu = 0.071, current mu = 0.001) allocation failure; scavenge might not succeed\r\n\r\n\r\n\u003c--- JS stacktrace ---\u003e\r\n\r\nFATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory\r\n----- Native stack trace -----\r\n\r\n 1: 0xcc062a node::OOMErrorHandler(char const*, v8::OOMDetails const\u0026) [node]\r\n 2: 0x104eb90 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const\u0026) [node]\r\n 3: 0x104ee77 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const\u0026) [node]\r\n 4: 0x126e0b5  [node]\r\n 5: 0x126e58e  [node]\r\n 6: 0x12837b6 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [node]\r\n 7: 0x12842d9  [node]\r\n 8: 0x12848e8  [node]\r\n 9: 0x19d4311  [node]\r\n[1]    31511 abort (core dumped)  node test/examples/ephemeralPromiseMemoryLeak.js\r\n```\r\n\r\n### Additional information\r\n\r\nIf the promiseValue call incorporates an explicit scheduling on the event loop, there is no memory leak...\r\n\r\n```js\r\n// setImmediateNoLeak.js\r\n\r\nfunction promiseValue(value) {\r\n  return new Promise((resolve) =\u003e {\r\n    setImmediate(() =\u003e resolve(value));\r\n  });\r\n}\r\n\r\nasync function run() {\r\n  for (;;) {\r\n    await Promise.race([promiseValue(\"foo\"), promiseValue(\"bar\")]);\r\n  }\r\n}\r\n\r\nrun();\r\n```\r\n\r\nIf the promiseValue call isn't composed via a Promise.race there is no leak...\r\n\r\n```js\r\n// noRaceNoLeak.js\r\n\r\nasync function promiseValue(value) {\r\n  return value;\r\n}\r\n\r\nasync function run() {\r\n  for (;;) {\r\n    await promiseValue(\"foo\");\r\n    await promiseValue(\"bar\");\r\n  }\r\n}\r\n\r\nrun();\r\n```\r\n\r\nMaybe obviously, but putting it here for completeness, if you don't use an async await loop, but compose the loop itself with setImmediate there is no leak...\r\n\r\n\r\n```js\r\n// noLoopNoLeak.js\r\n\r\nasync function promiseValue(value) {\r\n  return value;\r\n}\r\n\r\nasync function doRace() {\r\n  await Promise.race([promiseValue(\"foo\"), promiseValue(\"bar\")]);\r\n}\r\n\r\nfunction run() {\r\n  doRace().then(() =\u003e setImmediate(run));\r\n}\r\n\r\nrun();\r\n```","author":{"url":"https://github.com/cefn","@type":"Person","name":"cefn"},"datePublished":"2024-01-13T22:50:04.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":12},"url":"https://github.com/51452/node/issues/51452"}