René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"iOS: ASWebAuthenticationSession cannot be automated — binding the session to `com.apple.SafariViewService` destroys the view, and the tree served afterwards is stale","articleBody":"# iOS: ASWebAuthenticationSession cannot be automated — binding the session to `com.apple.SafariViewService` destroys the view, and the tree served afterwards is stale\n\n## Summary\n\nAn app that signs in through OAuth/OIDC presents the identity provider in an\n`ASWebAuthenticationSession`. That view is hosted by `com.apple.SafariViewService`, out of the\napp's process. On the iOS Simulator this makes the sign-in form unreachable, in both possible\nsession bindings:\n\n| session bound to | auth screen readable | typing possible |\n|---|---|---|\n| the app under test | no — out of process | no — `TEXT_INPUT_NOT_FOCUSED` |\n| `com.apple.SafariViewService` | yes, full tree | moot — **the rebind destroys the view** |\n\nReading the screen requires the rebind; the rebind cancels the authentication session. There is no\nthird position.\n\nA second, independent problem makes this very hard to diagnose: **after the view is gone,\nagent-device keeps serving a stale accessibility tree.** `wait` succeeds, `get attrs` returns a\nplausible rect, and the screen is black.\n\nVersion: `agent-device` 0.20.10 (latest on npm at the time of writing).\nEnvironment: local iOS Simulator (iPhone 17, iOS 26), macOS 15.6, React Native app in debug via Metro.\nNot BrowserStack — this is the local simulator path, so it is not #1658.\n\n## Bug A: binding the session to `com.apple.SafariViewService` cancels the auth session\n\nReproduction, with an app whose sign-in button opens an `ASWebAuthenticationSession`:\n\n```bash\nagent-device open com.example.app --platform ios --device \"iPhone 17\" --relaunch\nagent-device press 'id=\"account-button\"' --settle\nagent-device press 'id=\"sign-in-button\"' --settle\n\n# system consent dialog: \"\u003cApp> Wants to Use \u003cdomain> to Sign In\"\nagent-device wait 'label=\"Continue\"' 25000\nagent-device press 'label=\"Continue\"' --settle\nagent-device wait stable 3000 60000\n\n# at this point a screenshot shows the provider's form, fully rendered,\n# with the email field focused and the keyboard up.\nagent-device screenshot            # => the sign-in form\n\nagent-device open com.apple.SafariViewService --platform ios\nagent-device screenshot            # => black screen, the view is gone\n```\n\nObserved with and without `--foreground`, and with and without `--relaunch`.\n\nThe view is verifiably alive immediately *before* the rebind (verified by `screenshot`) and gone\nimmediately *after*. Nothing else runs in between.\n\nThis looks specific to authentication views: #1296 shows the same\n`open com.apple.springboard` mechanism working fine against SpringBoard. The system presumably\ncancels an `ASWebAuthenticationSession` when its presenting context is disturbed by another bundle\nbeing activated.\n\n## Bug B: the accessibility tree served after the view is destroyed is stale\n\nWith the session bound to `com.apple.SafariViewService` and the view already destroyed (black\nscreen, confirmed by `screenshot`):\n\n```bash\nagent-device wait 'role=button label=\"LOGIN\"' 60000\n# => succeeds\n\nagent-device get attrs 'role=textfield'\n# => {\"rect\": {\"x\": 26, \"y\": 281, \"width\": 350, \"height\": 49}, ...}\n\nagent-device screenshot\n# => black screen\n```\n\nSelectors resolve against a screen that no longer exists. Every downstream command then fails in a\nway that points at the wrong cause:\n\n- `fill 'role=textfield' \"\u003cvalue>\"` → `no text input found at the provided coordinates to clear`\n  (it taps the rect of a field that is gone)\n- `press 'role=textfield'` then `type \"\u003cvalue>\"` → `TEXT_INPUT_NOT_FOCUSED`\n- sometimes `main thread execution timed out` or `RUNNER_BUSY`\n\nI spent a long time treating these as selector or focus problems. They are not: the targets were\ncorrect, they just described a destroyed screen. A stale tree that answers confidently is worse\nthan a capture failure that says so — this is close in spirit to #2273, but the failure mode is the\nopposite (false positives rather than indistinguishable negatives).\n\n## Impact\n\nAny iOS app using delegated authentication — OAuth2/OIDC via `ASWebAuthenticationSession`, which is\nApple's recommended approach and the default for expo-auth-session / AppAuth — cannot have its\nauthenticated flows automated. In our case that is 15 of 34 end-to-end flows: everything behind\nsign-in (paywall, bookmarks, comments, account, subscriber content).\n\nMaestro drives the same screen without trouble, because its XCTest driver reads the whole system\ntree rather than binding a session to one bundle.\n\n## What I ruled out\n\n- **Not a selector problem.** The form is fully semantic once the session is bound to\n  `com.apple.SafariViewService`: `[link] \"LOGIN\"`, `[text-field]`, `[securetextfield]`,\n  and the keyboard's own accessory bar exposes `[button] \"Previous\" / \"Next\" / \"Done\"` — so field\n  traversal needs no coordinates.\n- **Not a timing problem.** Waiting up to 5 s after a `press` does not make `type` find the focus;\n  the app-bound tree simply never contains the auth screen.\n- **Not runner exhaustion.** Reproduced on a freshly rebooted simulator with a fresh\n  `prepare ios-runner`.\n- **Not `--settle` alone.** Removing it avoids some `main thread execution timed out`, but the\n  underlying stale-tree behaviour is unchanged.\n\n## What would help\n\n1. A way to observe a system-hosted view **without changing the session's bundle binding** —\n   something like the `--surface frontmost-app` that exists on macOS, or an explicit\n   \"read the frontmost surface\" mode for iOS.\n2. Failing that: make the stale tree detectable. If the bound surface is no longer frontmost or has\n   been torn down, `wait` / `get` / `press` should fail with a distinct reason rather than answer\n   from a cached tree.\n\nItem 2 is valuable on its own even if item 1 is out of scope — it turns a silent wrong answer into\na diagnosable error.\n\n## Related\n\n- #1658 — `fill` no-ops / mis-navigates on WebView inputs (fixed, BrowserStack-only path)\n- #1298 — WKWebView iOS: web forms mis-tap under the keyboard (embedded webview, not a system auth view)\n- #1571 — first capture can see an unreadable or empty tree (empty, not stale)\n- #2273 / #2274 — iOS selector queries: false negatives and no fallback producer\n- #1296 — first-class system UI target (shows the rebind working against SpringBoard)\n","author":{"url":"https://github.com/Ge0ffreyS","@type":"Person","name":"Ge0ffreyS"},"datePublished":"2026-09-10T07:43:24Z","url":"https://github.com/callstack/agent-device/issues/2438"}