René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`fs.rmSync(path, { recursive: true, maxRetries: manyManyRetries})` throws on `ENFILE`","articleBody":"### Version\n\nv22.8.0\n\n### Platform\n\n```text\nLinux tumba 6.11.4-gentoo-yuran-r7 #8 SMP Sat Oct 19 16:45:43 +08 2024 x86_64 Intel(R) Core(TM)2 Duo CPU     E6550 GenuineIntel GNU/Linux\n```\n\n\n### Subsystem\n\nfs\n\n### What steps will reproduce the bug?\n\nSet low limits or high `FILE_COUNT`, run the following:\r\n\r\n```mjs\r\nimport fs from 'node:fs';\r\nimport { readFile } from 'node:fs/promises';\r\n\r\nconst OUTPUT_DIRECTORY = \"/tmp/emfile-check\";\r\nconst FILE_COUNT = 150_000; // adjust this according to ulimit -n, ulimit -Hn, /proc/sys/fs/file-max, etc.\r\n\r\nfunction generateFiles() {\r\n  fs.rmSync(OUTPUT_DIRECTORY, { recursive: true, force: true, maxRetries: 8192 });\r\n  fs.mkdirSync(OUTPUT_DIRECTORY, { recursive: true });\r\n  for (let i = 0; i \u003c FILE_COUNT; i++) {\r\n    const fileName = `file_${i}.js`;\r\n    const fileContent = `// This is file ${i}`;\r\n    fs.writeFileSync(`${OUTPUT_DIRECTORY}/${fileName}`, fileContent);\r\n  }\r\n}\r\n\r\nfunction generateEmFileError() {\r\n  return Promise.all(\r\n    Array.from({ length: FILE_COUNT }, (_, i) => {\r\n      const fileName = `file_${i}.js`;\r\n      return readFile(`${OUTPUT_DIRECTORY}/${fileName}`);\r\n    })\r\n  );\r\n}\r\n\r\nconsole.log(`Generating ${FILE_COUNT} files in ${OUTPUT_DIRECTORY}...`);\r\ngenerateFiles();\r\n\r\nconsole.log(\"Checking that this number of files would cause an EMFILE error...\");\r\nawait generateEmFileError()\r\n  .then(() => {\r\n    throw new Error(\"Test failure: not enough files to encounter EMFILE.\");\r\n  })\r\n  .catch(error => {\r\n    if (error.code === \"EMFILE\") {\r\n      console.log(\"Successfully got intentional EMFILE:\", error.message);\r\n    } else if (error.code === \"ENFILE\") {\r\n      console.log(\"Successfully got intentional ENFILE:\", error.message);\r\n    } else {\r\n      console.error(\"Failed with unknown error:\", error.message);\r\n      throw error;\r\n    }\r\n  })\r\n  .finally(() => {\r\n    console.log(`Attempting to remove ${OUTPUT_DIRECTORY}`);\r\n    fs.rmSync(OUTPUT_DIRECTORY, { recursive: true, force: true, maxRetries: 8192 });\r\n    console.log(`Successfully removed ${OUTPUT_DIRECTORY}`);\r\n  });\r\n```\n\n### How often does it reproduce? Is there a required condition?\n\nMight be flaky depending on system configuration (under some conditions exhausting fd limits may render system unresponsive).\n\n### What is the expected behavior? Why is that the expected behavior?\n\n```\r\nGenerating 150000 files in /tmp/emfile-check...\r\nChecking that this number of files would cause an EMFILE error...\r\nSuccessfully got intentional ENFILE: ENFILE: file table overflow, open '/tmp/emfile-check/file_63275.js'\r\nAttempting to remove /tmp/emfile-check\r\nSuccessfully removed /tmp/emfile-check\r\n```\r\nWe should encounter the ENFILE via `generateEmFileError` function, catch the error, and remove the directory. Directory removal might encounter errors due to unfinished jobs, but it should retry with 100ms delay 8192 times (more than 13 minutes).\n\n### What do you see instead?\n\n```\r\nGenerating 150000 files in /tmp/emfile-check...\r\nChecking that this number of files would cause an EMFILE error...\r\nSuccessfully got intentional ENFILE: ENFILE: file table overflow, open '/tmp/emfile-check/file_63275.js'\r\nAttempting to remove /tmp/emfile-check\r\nnode:fs:1502\r\n  const result = binding.readdir(\r\n                         ^\r\n\r\nError: ENFILE: file table overflow, scandir '/tmp/emfile-check'\r\n    at readdirSync (node:fs:1502:26)\r\n    at _rmdirSync (node:internal/fs/rimraf:249:29)\r\n    at rimrafSync (node:internal/fs/rimraf:192:7)\r\n    at Object.rmSync (node:fs:1247:10)\r\n    at file:///tmp/repro/t.mjs:52:8\r\n    at \u003canonymous>\r\n    at async file:///tmp/repro/t.mjs:36:1 {\r\n  errno: -23,\r\n  code: 'ENFILE',\r\n  syscall: 'scandir',\r\n  path: '/tmp/emfile-check'\r\n}\r\n\r\nNode.js v22.8.0\r\n```\n\n### Additional information\n\nAccording to the [docs](https://nodejs.org/api/fs.html#fsrmsyncpath-options), it must continue in case of `ENFILE`.","author":{"url":"https://github.com/LiviaMedeiros","@type":"Person","name":"LiviaMedeiros"},"datePublished":"2024-10-26T18:01:19Z","url":"https://github.com/nodejs/node/issues/55555"}