Title: The formatter allocates over 1,000 times its input, and the native binary spends most of its run collecting it · Issue #229 · openjavaformat/open-java-format · GitHub
Open Graph Title: The formatter allocates over 1,000 times its input, and the native binary spends most of its run collecting it · Issue #229 · openjavaformat/open-java-format
X Title: The formatter allocates over 1,000 times its input, and the native binary spends most of its run collecting it · Issue #229 · openjavaformat/open-java-format
Description: What happens Formatting allocates more than a thousand times the size of its input, and the native binary spends most of its run collecting it. 865 files from java.base of JDK 21, 15 MB of source, --replace with 21.0.2 on 10 cores: Wall ...
Open Graph Description: What happens Formatting allocates more than a thousand times the size of its input, and the native binary spends most of its run collecting it. 865 files from java.base of JDK 21, 15 MB of source, ...
X Description: What happens Formatting allocates more than a thousand times the size of its input, and the native binary spends most of its run collecting it. 865 files from java.base of JDK 21, 15 MB of source, ...
Opengraph URL: https://github.com/openjavaformat/open-java-format/issues/229
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"The formatter allocates over 1,000 times its input, and the native binary spends most of its run collecting it","articleBody":"### What happens\n\nFormatting allocates more than a thousand times the size of its input, and the native binary spends most of its run collecting it. 865 files from `java.base` of JDK 21, 15 MB of source, `--replace` with 21.0.2 on 10 cores:\n\n| | Wall | Of it in GC pauses | Peak RSS |\n|---|---|---|---|\n| Native binary | 6.9 s | 5.0 s: 17 full collections for 3.05 s, 23 incremental for 1.94 s | 2.4 GB |\n| `java -cp ... Main`, JDK 21 | 6.2 s | | 4.3 GB |\n\nThe native run's GC log adds up to at least 16.6 GB allocated, about 1,100 bytes for each byte of source. The image is built without `--gc`, so its serial collector stops all 20 threads for each collection, and they format on about 2 of the 10 cores (15 s of CPU in 6.9 s).\n\nInput and output are not where it goes: reading the 865 files takes 0.04 s, and `Main`'s reading and writing are 0.5% of the allocation, so memory-mapped files or other I/O changes cannot help.\n\n### Where it goes\n\nFlight Recorder on the JVM run, with `-XX:FlightRecorderOptions:stackdepth=2048` so that no stack was cut, 1,611 allocation samples and 469 CPU samples:\n\n| Step of `FormatFileCallable.formatFile` | Allocation | CPU |\n|---|---|---|\n| `Formatter.formatSource`, the layout | 65.5% | 56.1% |\n| `StringWrapper.wrap` | 16.8% | 23.5% |\n| `RemoveUnusedImports` | 12.9% | 19.8% |\n| `ImportOrderer` | 4.0% | 0.6% |\n| `Main`, reading and writing | 0.5% | |\n\n- **The file is parsed several times.** 42% of the CPU is in javac's parser and scanner: 16% for the layout, 16% for the parse of its own that `RemoveUnusedImports` makes, 10% in `StringWrapper`, whose rounds parse again and parse before and after to compare the trees.\n- **Each parse sets javac up from scratch: 18.1% of the allocation.** `Formatter.parseJcCompilationUnit`, `JavacTokens.getTokens` and `JavaInput.buildToks` each create a new `Context`, and with it a `SharedNameTable` (`NameImpl[0x8000]` and `byte[0x20000]`), the writers of `Log`, and the buffers of `ClassWriter` (through `Modules`), `ClassReader` and even `Gen`, which parsing does not use. javac keeps disposed name tables for reuse (`SharedNameTable.create`), but nothing here calls `Names.dispose()`.\n- **The top sites otherwise:** `OpsBuilder.build` 11.5%, the memoizing supplier of `HasUniqueId.ord` 4.6% plus 2.3% for its `Suppliers` objects, `Doc.\u003cinit>` 4.3%, `InputOutput.makeKToIJ` 3.1%, `JavaOutput.append` 2.7%. By class: `byte[]` 23.4%, `char[]` 8.3%, `Object[]` 8.1%, boxed `Integer` 7.2%, Guava `Range` 2.7%.\n\n### Proposal\n\nIn order of what each is likely to give for its work, each measured on the run above:\n\n1. **The native image's collector:** a larger young generation, another collector where GraalVM offers one for the platform, fewer threads under the serial one (#228).\n2. **javac set up once per thread, or reused:** call `Names.dispose()` after each parse so that javac hands the name table out again, and find what makes a parse create `Modules`, `ClassWriter`, `ClassReader` and `Gen`, and stop it.\n3. **One parse per file:** `RemoveUnusedImports` and `StringWrapper` work from the compilation unit and tokens of the formatting pass where the text did not change, as google-java-format's `Cleanups` reuse its parse (google/google-java-format@633ce6be, #224).\n4. **Fewer small objects in the layout:** token positions as `int`, not boxed `Integer` in `Range`s, and the unique ids of `Doc`s without a memoizing supplier for each.\n","author":{"url":"https://github.com/abashev","@type":"Person","name":"abashev"},"datePublished":"2026-11-14T06:54:56Z","url":"https://github.com/openjavaformat/open-java-format/issues/229"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:b2f0c6b7-e547-aba9-fa17-91554133da11 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D708:1189F5:29AB24:37751D:6AFBB791 |
| html-safe-nonce | b88e21854c0cee6b7bfcd290c5d6fa194e45d2b8c1a29a01f65eeb40631a7aee |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENzA4OjExODlGNToyOUFCMjQ6Mzc3NTFEOjZBRkJCNzkxIiwidmlzaXRvcl9pZCI6IjI1NTg0NjE4MjgwMDgzMjQwMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 2b443f3de214beb31ee56708a336d4f8aab77177ba8382dd4b50291ad16cb8a7 |
| hovercard-subject-tag | issue:6386584088 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/openjavaformat/open-java-format/229/issue_layout |
| twitter:image | https://opengraph.githubassets.com/ac12d09ab01da568490cfcf74e183dd1de1910375ddcc13c88293dae8ce9e747/openjavaformat/open-java-format/issues/229 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/ac12d09ab01da568490cfcf74e183dd1de1910375ddcc13c88293dae8ce9e747/openjavaformat/open-java-format/issues/229 |
| og:image:alt | What happens Formatting allocates more than a thousand times the size of its input, and the native binary spends most of its run collecting it. 865 files from java.base of JDK 21, 15 MB of source, ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | abashev |
| hostname | github.com |
| expected-hostname | github.com |
| None | 2b2ef9c1578c18d8ed27027940544c958743213cf4461659f14d3ccdf51e162c |
| turbo-cache-control | no-preview |
| go-import | github.com/openjavaformat/open-java-format git https://github.com/openjavaformat/open-java-format.git |
| octolytics-dimension-user_id | 325177626 |
| octolytics-dimension-user_login | openjavaformat |
| octolytics-dimension-repository_id | 1288055459 |
| octolytics-dimension-repository_nwo | openjavaformat/open-java-format |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | true |
| octolytics-dimension-repository_parent_id | 214273098 |
| octolytics-dimension-repository_parent_nwo | palantir/palantir-java-format |
| octolytics-dimension-repository_network_root_id | 214273098 |
| octolytics-dimension-repository_network_root_nwo | palantir/palantir-java-format |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | e4cce0e99cf61481a318ff12c2e8580153a6c7c6 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width