Title: perf(runtime): compile large HTTP modules off the JS thread by edusperoni · Pull Request #587 · NativeScript/ios · GitHub
Open Graph Title: perf(runtime): compile large HTTP modules off the JS thread by edusperoni · Pull Request #587 · NativeScript/ios
X Title: perf(runtime): compile large HTTP modules off the JS thread by edusperoni · Pull Request #587 · NativeScript/ios
Description: Builds on #584 (merged), whose worker-teardown quiesce the drain below relies on. What ES modules fetched over HTTP by the async graph walk are compiled off the JS thread when their JavaScript body is 32 KiB or larger. Parse and bytecode generation run on a background thread through V8's streaming API (ScriptCompiler::StartStreaming / ScriptStreamingTask::Run), and the JS thread only finalizes the result (ScriptCompiler::CompileModule(context, StreamedSource*, ...)). Smaller bodies, JSON modules, failed fetches and the synchronous resolver fallback compile exactly as before. Why During a dev-server load the JS thread compiled every fetched module itself, serially, while other fetches were already complete. Large modules (bundles, modules carrying inline source maps) dominate that time. Measured with a prototype of this change (same design, a GCD queue instead of the dedicated threads below): iPhone 11 Pro over a USB tunnel, warm HTTP cache, 32 connections per host, medians of 4 interleaved runs: an Angular dev app (75 modules, 18 MB including inline source maps): 373 ms with the 32 KiB threshold vs 537 ms without; JS-thread time during the load 153 ms -> 31 ms; a synthetic app of 671 small modules: 570 ms vs 653 ms; streaming every module regardless of size did not help that app (671 ms), which is why there is a threshold. Simulator: three 0.5-1 MB bundles 32 ms -> 17 ms; 70 modules carrying inline source maps 76 ms -> 37 ms; graphs of hundreds of tiny modules were unchanged or slightly slower when everything was streamed. Background compilation costs about 25% more total CPU than compiling on the JS thread. Not measured: a cold download on a device. How it works All in ModuleInternalCallbacks.mm: Issue (JS thread). AsyncGraphEnqueue creates a StreamedModuleCompile (a StreamedSource over an ExternalSourceStream that blocks until it is fed) and calls StartStreaming(..., ScriptType::kModule, options). It is parked in the isolate's ModuleLoaderState (Caches::StateFor slot) under a ticket id; the transport only carries a StreamTicket, which owns no V8 state. options is kEagerCompile when the runtime runs with --no-lazy (non-debug), because streaming follows --lazy-streaming, not --lazy. Transport completes. An OK JavaScript response of at least 32 KiB is handed to a small pool of dedicated threads; everything else posts the completion as today. The pool job copies the body into a new uint8_t[] chunk (V8 delete[]s it), checks whether it is pure ASCII, feeds the stream, runs task->Run(), and only then posts the completion to the isolate's event loop. Finalize (JS thread). CompileStreamedModuleRegisterOnly builds the full source string and calls CompileModule(context, &streamedSource, fullSource, origin). A pure-ASCII body becomes an external one-byte string that takes ownership of the bytes (no UTF-8 decode, no copy into the heap); anything else uses NewFromUtf8 as before. Registry check, registration, walking GetModuleRequests(), completion accounting, the microtask checkpoint and error reporting are shared with the synchronous path through CompileAndRegisterModule, which CompileModuleForResolveRegisterOnly now also uses. If the key was registered meanwhile (two walks fetched the same URL), the existing module is reused and the streamed compile is discarded. Lifetime. A StreamedModuleCompile is created, finalized and destroyed on the JS thread only. A gate counts compiles in flight; a dead walk starts none. Isolate teardown (QuiesceModuleLoadsForIsolate, workers included) closes the gate, waits for the running compiles and destroys the unfinalized ones before the isolate is disposed. The wait keeps running the event loop's nestable V8 tasks: a background compile that needs a GC posts BackgroundCollectionInterruptTask to the isolate's foreground runner and parks until it runs. Threads and stack size. BackgroundCompileTask::Run (src/codegen/compiler.cc) sets the parser's stack limit to GetCurrentStackPosition() - v8_flags.stack_size * KB, and stack_size defaults to V8_DEFAULT_STACK_SIZE_KB, 984 KB on arm64 (src/common/globals.h; the runtime does not override it). GCD worker threads have a stack of about 512 KB (pthread_get_stacksize_np on the NSURLSession delegate thread, measured on the simulator: 536576 bytes), so on a GCD queue deeply nested source runs off the real stack before V8's check fires. Run() therefore uses a dedicated pool: at most min(4, cores - 1) detached pthreads, created on demand and kept for the life of the process, each with a 2 MB stack (about twice V8's assumed limit), at user-initiated QoS. Each job runs in an autorelease pool. External memory. V8 14.9 tracks external string payloads only for heap statistics (Heap::backing_store_bytes, ExternalStringTable::GetBytes), not for GC scheduling, so the external source string carries an ExternalMemoryCharge for its byte length, the same way native wrappers are charged; it is released when V8 disposes the string. Behaviour and limits Nothing JS-visible changes: same module identity, evaluation order, error texts and positions. A body starting with a UTF-8 BOM is not streamed: the stream skips a leading BOM that the full source string keeps, which would shift every reported position. Debugger: finalization reports the script with its real URL and the full source string; Script::SetSource initializes line ends when NeedsSourcePositions() (true while a debugger is attached), and FinalizeUnoptimizedCompilation adds source positions to the background-compiled functions; the debugger hears about the script from Compiler::PostInstantiation, as for any module. Checked end to end, see below. Every HTTP edge still pays a StartStreaming call at issue time, because the body size is unknown until the response arrives. The pool is shared by all isolates and does not grow past its cap. A compile parked on a GC request keeps its pool thread until that isolate's thread runs tasks; GCD would have added threads in that case. Worker termination does not block the terminating thread on the worker's teardown, so a drain cannot wait on a thread that is waiting on it. If no pool thread can be started, the module compiles on the JS thread as before. How it was verified Specs. ModuleTestServer gains /esm/big.mjs, a generated module of N real declarations whose evaluation folds into exported values (total, count, and bytes, which each spec asserts is above 32 KiB, so these modules are over the threshold by construction). It has options for non-ASCII string literals, a trailing syntax error, deeply nested array literals and a response delay. /esm/big-parent.mjs statically imports one of them. New specs under "HTTP ESM Loader / large modules": evaluates one loaded with import() evaluates one reached through a static import evaluates one reached during a pumped require() (local root -> import-mapped HTTP module, the blocking RunModuleGraphLoadPumped path) evaluates non-ASCII string literals (accents, CJK, an emoji) to the right values rejects a syntax error with the text a small module produces (same message and column; line = filler lines + 3) rejects pathologically deep nesting like a small module does (RangeError: Maximum call stack size exceeded on both paths) resolves one module for concurrent imports from several call sites (two import()s plus two parents that statically import it: one namespace object, evaluated once) survives workers terminated while their large (3 MB) modules compile, and the main isolate still loads one afterwards Streaming confirmed by temporary logging (removed before commit). In a suite run with logging, every large spec's module went through a background Run() and was finalized on the JS thread. The second finalization of the shared module took the registry-hit path. Two worker teardowns closed the gate with a compile still running (inFlight=1) and waited for it, running the loop's nestable tasks 18 and 36 times while waiting. That shows the wait itself ran. Whether it services a background GC request is reasoned from heap/collection-barrier.cc (the request is a PostTask, which the nestable lane runs) and was not observed. The pool threads reported 2109440 bytes of stack and the transport thread 536576. Stack-size control. With the pool's stack temporarily set to 512 KB, the deep-nesting spec crashed the app (SIGBUS, KERN_PROTECTION_FAILURE, on org.nativescript.esm.stream-compile inside ParserBase::ParsePrimaryExpression). With 2 MB it gets the RangeError. kEagerCompile. The suite runs a debug runtime, so the non-debug branch is not reached on its own. In one full run, kEagerCompile was forced on for every streamed compile, and all specs passed. The normal non-debug selection is covered by reasoning. Debugger. On the simulator, with the inspector attached over the CDP socket and Debugger.enable sent before the import, two generated modules served over HTTP were loaded through Runtime.evaluate: a 101 KB ASCII module (external string) and a 159 KB non-ASCII module. This ran on the build installed by the stack-size control run: the same code as the commit plus temporary logging, with the 512 KB pool stack, which these shallow modules do not approach. The logging confirmed both modules were streamed. Results: Debugger.scriptParsed reported each module's real URL; Debugger.getScriptSource returned text identical to the served file; a Debugger.setBreakpointByUrl breakpoint inside an exported function paused at the expected line; evaluation returned the expected values. Attaching mid-load and pausing during finalization were not exercised. Suite. ./run_tests.sh --temp-sim: 2165 test cases (2157 before plus the 8 above), 0 failures, 11 skipped. Under AddressSanitizer (-a): 2165 test cases, 0 failures, 16 skipped (the 5 extra are the worker QoS specs that skip themselves under ASan), and no ASan report. Release build of TestRunner for the simulator succeeds. Reasoned, not tested: invalid UTF-8 decodes the same way in the stream and in NewFromUtf8 (both use V8's UTF-8 decoder); the external-memory charge is released when V8 disposes the string; the real non-debug kEagerCompile selection; anything on a device. Open items Disk (file) modules are not streamed. No interaction with the code cache (HTTP modules have none today). Android is unchanged. Summary by CodeRabbit Performance Large JavaScript modules loaded over HTTP can now compile in the background, helping keep the app responsive during module loading. Bug Fixes Improved reliability when loading large modules through dynamic imports, static imports, and require(), including concurrent imports and worker shutdowns during compilation. Preserved source locations in syntax errors and behavior for deeply nested modules.
Open Graph Description: Builds on #584 (merged), whose worker-teardown quiesce the drain below relies on. What ES modules fetched over HTTP by the async graph walk are compiled off the JS thread when their JavaScript body...
X Description: Builds on #584 (merged), whose worker-teardown quiesce the drain below relies on. What ES modules fetched over HTTP by the async graph walk are compiled off the JS thread when their JavaScript body...
Opengraph URL: https://github.com/NativeScript/ios/pull/587
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:9c872674-b8e7-c765-aa37-f682b69829ac |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | B030:3F8A67:1AE84A4:23EC293:6B015FC5 |
| html-safe-nonce | cc0fcd807ee735442a350f324466cd8c2154021f1f6a760035498ed2c36f57cc |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMDMwOjNGOEE2NzoxQUU4NEE0OjIzRUMyOTM6NkIwMTVGQzUiLCJ2aXNpdG9yX2lkIjoiNjMyNjQ4MDY2MTIzMDIxNTEwOSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 1bfc945ddac3ce6d6b7889171e4b33c42f456b5b8093b17a4132f8af0bf8f888 |
| hovercard-subject-tag | pull_request:5407903044 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/NativeScript/ios/pull/587/files |
| twitter:image | https://avatars.githubusercontent.com/u/4061767?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/4061767?s=400&v=4 |
| og:image:alt | Builds on #584 (merged), whose worker-teardown quiesce the drain below relies on. What ES modules fetched over HTTP by the async graph walk are compiled off the JS thread when their JavaScript body... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/NativeScript/ios git https://github.com/NativeScript/ios.git |
| octolytics-dimension-user_id | 7392261 |
| octolytics-dimension-user_login | NativeScript |
| octolytics-dimension-repository_id | 172934359 |
| octolytics-dimension-repository_nwo | NativeScript/ios |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 172934359 |
| octolytics-dimension-repository_network_root_nwo | NativeScript/ios |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | canary-2 |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width