Title: fix: keep the base class of runtime-resolved types in large apps by edusperoni · Pull Request #2111 · NativeScript/android · GitHub
Open Graph Title: fix: keep the base class of runtime-resolved types in large apps by edusperoni · Pull Request #2111 · NativeScript/android
X Title: fix: keep the base class of runtime-resolved types in large apps by edusperoni · Pull Request #2111 · NativeScript/android
Description: Description The symptom. In a large app, iterating Java collections from JS stops working: const it = map.entrySet().iterator(); while (it.hasNext()) { … } // TypeError: it.hasNext is not a function jsonObject.keys().hasNext(); // TypeError: … hasNext is not a function unmodifiableList.iterator(); // TypeError: … iterator is not a function ArrayList.iterator() keeps working, and so does everything else in the app, which makes it look like something specific to iterators. Cause. These objects are instances of private JDK classes (java.util.HashMap$EntryIterator, java.util.LinkedHashMap$LinkedKeyIterator, java.util.Collections$UnmodifiableRandomAccessList, …) that are not in the static metadata, so the runtime resolves them on first use and appends a node for each to the metadata node table. For such a type the base class was stored as a 16-bit node id, the same slot a static class record uses. But appended nodes sit after the static nodes plus a 1000-slot gap, so once the static tree has more than about 64.5k nodes their ids no longer fit: the id wraps and the type is linked to an unrelated node as its base. The JS wrapper then has the members the class itself declares and nothing from its real base class: Runtime class Real base (also runtime-resolved) Member lost HashMap$KeyIterator / ValueIterator / EntryIterator HashMap$HashIterator hasNext() LinkedHashMap$Linked*Iterator (also JSONObject.keys()) LinkedHashMap$LinkedHashIterator hasNext() TreeMap$*Iterator, ConcurrentHashMap$*Iterator their private base iterators hasNext() Collections$UnmodifiableList, $SynchronizedList, List.of(…) their private base collections iterator() A type whose base class is in the static metadata (ArrayList$Itr extends Object) is linked correctly, which is why only some collections break. Nothing in this path changed recently; it is a size cliff. An app gets there by growing: a newer compile SDK (android-36 alone has 246 more classes than android-35) or more dependencies. The fix. A runtime-resolved type holds its base class as a node pointer (MetadataTreeNode::runtimeBaseClass), and GetBaseClassNode returns it. The value record keeps its layout, with the id slot unused. Node ids are 32-bit wherever a runtime-created node can appear. The other place is the element type of an array (ARRAY_OFFSET + id), which was narrowed to 16 bits the same way. Runtime-created nodes are numbered from 65536 in every app, in their own vector, instead of right after the static ones. Static ids are 16-bit by format, so the two ranges never overlap, and the wide-id path now runs in a small app too. That is what lets the test app cover this. The unused 1000-slot gap in the static table is gone. Thread safety. The metadata tree is shared by every runtime in the process, and auditing it for this change turned up two existing races around runtime-resolved nodes: A runtime-resolved type was appended to its parent's children vector under the reader's state lock, but MetadataNode walks those vectors with no lock in four places: building a package object, defining a class's inner types, looking up a package member, and creating the top-level namespaces. A worker resolving HashMap$KeyIterator while another runtime materializes java.util.HashMap could reallocate the vector mid-iteration. Two runtimes resolving the same unknown type at once could each publish a node for it: resolving the base class can release the lock, after the only re-check. That was benign (a duplicate node). Both are closed without adding a lock to any MetadataNode path: Nodes created at run time go into a table the reader owns, keyed by parent, and are never appended to MetadataTreeNode::children. The static tree is written only while it is built at startup, so walking it unlocked is safe, and MetadataNode.cpp ends up unchanged by this PR. (An intermediate commit guarded those walks with the state lock and a copy of the list; the last commit replaces that, because those paths run under the isolate lock.) The base class is resolved first, and the lookup is repeated right before the node is created and published, with no unlock in between. The base class link is written before its node is published and never changes, and the reader's node vectors and table are only touched under the state lock. Interface records. ReadInterfaceImplementationTypeName reads a prefix flag and a name offset that follow the type byte and base class id in a static interface record. A type resolved at run time only gets the type byte and the id slot, so for a runtime-resolved interface that read would take whatever bytes come next in the value buffer and use them as an offset into the names. Before this PR that was reachable by name, in the same timing-dependent way described under the behavior change below; with runtime nodes out of the static tree, tracing its two callers turned up none that can still get there. The function now returns an empty name for a runtime-resolved node, so it stays safe if a path to it is ever added. Behavior change. A type resolved at run time is no longer reachable by name from its parent package or class. It used to be, but only in a runtime that had not yet materialized that parent when the type was first seen, so it depended on timing. Such types are still wrapped normally when Java hands one to JS. Cost. Nothing is added to method calls, wrapper creation, or class and package materialization, apart from one branch when an interface's node is first created (the check above). Resolving a type that is in the static metadata looks it up in the static child list first, as before; the reader's own table is consulted only on a miss, which previously went straight to a Java call. Resolving a runtime type no longer does a linear search of the whole node table for its base class id. MetadataTreeNode stays at 64 bytes on 64-bit ABIs (the new pointer fits after reordering its members; a static_assert holds it), and there is one per metadata node. All of this state is process-wide, not per isolate. Not covered here. The static tree itself is limited to 65,535 nodes by the metadata format. The generator fails the build past that ("Metadata is too big and has overflown our current limit"), and this change does not raise the limit; it removes the cliff about 1,000 nodes below it. Does your commit message include the wording below to reference a specific issue in this repo? No issue. Related Pull Requests None. Does your pull request have unit tests? Yes. testJavaIterators.js (46 specs) drains iterators with hasNext()/next() over the JDK list, set and map implementations (key sets, values and entry sets), List.of, unmodifiable and synchronized wrappers, org.json.JSONObject.keys(), android.os.Bundle.keySet(), android.util.ArrayMap, an Enumeration, Kotlin collections, and app-defined iterator classes: package-private, private nested, anonymous, lambda-backed, one whose hasNext() is inherited from a non-public base class, and an array of a private class. There were no iterator specs before. Runs, all on an API 36 arm64 emulator: Build Result main, node table padded so runtime nodes start past 65535 (simulates a large app) 26 of these specs fail, everything else passes This branch 1598 / 1598 pass This branch with the 16-bit base id read restored the same 26 specs fail ./gradlew runtestsAndVerifyResults on the final tree: 1598 / 1598. The two races have no spec: each needs a runtime to resolve a type in the same instant another one walks its parent, and a type is only resolved once per process, so a test could not hit the window reliably. They were found by reading the accesses to the tree's child lists; BuildTree is now their only writer. The interface record read has no spec either: no path to it with a runtime-resolved interface was found to test. The array spec passes in all three: a wrapped element-type id still resolves to an object type, which is enough to read elements, so that narrowing is fixed by inspection rather than by a failing spec. Summary by CodeRabbit Bug Fixes Improved runtime handling of dynamically resolved Java types, including classes, interfaces, and array types. Improved compatibility with Java iterators and enumerations across collections and map views, including implementations with inherited or non-public methods. Tests Expanded coverage for Java and Kotlin collections, JSON objects, bundles, iterator exhaustion, and JavaScript-wrapped iterators. Added checks for iterator behavior across additional Java collection and iterable implementations.
Open Graph Description: Description The symptom. In a large app, iterating Java collections from JS stops working: const it = map.entrySet().iterator(); while (it.hasNext()) { … } // TypeError: it.hasNext is not ...
X Description: Description The symptom. In a large app, iterating Java collections from JS stops working: const it = map.entrySet().iterator(); while (it.hasNext()) { … } // TypeError: it.hasNext is not ...
Opengraph URL: https://github.com/NativeScript/android/pull/2111
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:d641346b-9890-4fb0-973f-e64bbe8f8fd4 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | AF64:1EC9BC:19847CA:208795B:6B014AE7 |
| html-safe-nonce | 77769ebfd724d6c388b2383b23ad73d65f217d4a857b9c9bdb46e48642b51dd6 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBRjY0OjFFQzlCQzoxOTg0N0NBOjIwODc5NUI6NkIwMTRBRTciLCJ2aXNpdG9yX2lkIjoiNjI5NzUwNzYyNjg3MjUzMjIzIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 6044cc21749739f139b5e08e69c28a0b37877566d47c2cbea2da43e48b378a3c |
| hovercard-subject-tag | pull_request:5340359192 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/NativeScript/android/pull/2111/files |
| twitter:image | https://avatars.githubusercontent.com/u/4061767?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/4061767?s=400&v=4 |
| og:image:alt | Description The symptom. In a large app, iterating Java collections from JS stops working: const it = map.entrySet().iterator(); while (it.hasNext()) { … } // TypeError: it.hasNext is not ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/NativeScript/android git https://github.com/NativeScript/android.git |
| octolytics-dimension-user_id | 7392261 |
| octolytics-dimension-user_login | NativeScript |
| octolytics-dimension-repository_id | 29137950 |
| octolytics-dimension-repository_nwo | NativeScript/android |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 29137950 |
| octolytics-dimension-repository_network_root_nwo | NativeScript/android |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width