René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Memory leak while iterating over commits and their diffs?","articleBody":"I have some code that iterates over a range of commits and tries to extract some information from each commit (which requires to look at the diff of each commit). For longer histories, this systematically ends up with a process out memory error or with a core dump.\n\nI made a modified examples program that triggers it (just put it in nodegit examples directory and run it).\nIt will iterate over the commits and trigger the error.\n\nHere's the code:\n\n``` javascript\nvar nodegit = require(\"../\");\nvar path = require(\"path\");\n\nvar repo;\n\nfunction process(commit) {\n  \"use strict\";\n  console.log(\"commit \" + commit.sha());\n  return commit.getDiff().then(function (diffList) {\n    return Promise.all(diffList.map(function (diff) {\n      return diff.patches().then(function (patches) {\n        return Promise.all(patches.map(function (patch) {\n          return patch.hunks().then(function (hunks) {\n            return Promise.all(hunks.map(function (hunk) {\n              return hunk.lines().then(function (lines) {\n                return true;\n              });\n            }));\n          });\n        }));\n      })\n    })).then(function () {\n      if(commit.parentcount() === 0) {\n        return false;\n      } else {\n        return repo.getCommit(commit.parentId(0));\n      }\n    });\n  });\n}\n\nfunction iterate(commit) {\n  \"use strict\";\n\n  return process(commit).then(function(childcommit) {\n    if(!childcommit) {\n      return true;\n    } else {\n      return iterate(childcommit);\n    }\n  });\n}\n\n// This code examines the diffs between a particular commit and all of its\n// parents. Since this commit is not a merge, it only has one parent. This is\n// similar to doing `git show`.\n\nnodegit.Repository.open(path.resolve(__dirname, \"../.git\"))\n  .then(function (re) {\n    repo = re;\n    return repo.getHeadCommit();\n  })\n  .then(function (commit) {\n    return iterate(commit);\n  })\n  .done(function () {\n    \"use strict\";\n    console.log(\"done\");\n  })\n```\n\nI get an error similar to the following:\n\n```\n\u003c--- Last few GCs ---\u003e\n\n   29305 ms: Scavenge 1398.7 (1457.0) -\u003e 1398.7 (1457.0) MB, 5.0 / 0 ms (+ 1.8 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].\n   30477 ms: Mark-sweep 1398.7 (1457.0) -\u003e 1398.7 (1457.0) MB, 1172.2 / 5 ms (+ 2.7 ms in 2 steps since start of marking, biggest step 1.8 ms) [last resort gc].\n   31407 ms: Mark-sweep 1398.7 (1457.0) -\u003e 1398.7 (1457.0) MB, 930.4 / 5 ms [last resort gc].\n\n\n\u003c--- JS stacktrace ---\u003e\n\n==== JS stack trace =========================================\n\nSecurity context: 0x72d20d36ae1 \u003cJS Object\u003e\n    1: /* anonymous */(aka /* anonymous */) [/home/jfremy/prog/nodejs/nodegit/node_modules/nodegit-promise/lib/node-extensions.js:24] [pc=0x28f3e3723a14] (this=0x72d20d04131 \u003cundefined\u003e,resolve=0x1460e70041d1 \u003cJS Function done (SharedFunctionInfo 0x30acef4d48d1)\u003e,reject=0x1460e7004189 \u003cJS Function (SharedFunctionInfo 0x30acef4d4979)\u003e)\n    2: tryCallTwo(aka tryCallTwo) [/home/jfremy/prog/nodejs/n...\n\nFATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory\nAbandon (core dumped)\n```\n","author":{"url":"https://github.com/jfremy","@type":"Person","name":"jfremy"},"datePublished":"2015-10-26T14:16:36.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":6},"url":"https://github.com/766/nodegit/issues/766"}