René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[Bug]: ui-automation element refs become unavailable after debugger-paused tap","articleBody":"### Bug Description\n\nWhen a `ui-automation tap --element-ref` action triggers an LLDB breakpoint and pauses the app, the runtime UI automation snapshot cache appears to be dropped. After continuing the app, a follow-up tap against a same-screen element ref immediately fails with `No runtime UI snapshot is available for this simulator.`\n\nThis blocks debugger-assisted reproduction workflows that need to issue rapid repeated interactions through element refs, such as opening and closing the same SwiftUI disclosure/card view quickly to reproduce an animation bug. A coordinate fallback can work for reproduction, but it bypasses the normal runtime snapshot/element-ref workflow.\n\nIf invalidating refs after a debugger pause is intentional, the current behavior still seems worth improving because the failure differs from the normal stale-ref path and does not explain that the snapshot was invalidated or that a fresh `snapshot-ui`/`wait-for-ui` is required.\n\n### Debug Output\n\nCommand:\n\n```bash\nxcodebuildmcp doctor doctor --output json --style minimal\n```\n\nOutput:\n\n```json\n{\n  \"schema\": \"xcodebuildmcp.output.doctor-report\",\n  \"schemaVersion\": \"2\",\n  \"didError\": false,\n  \"error\": null,\n  \"data\": {\n    \"serverVersion\": \"2.6.2\",\n    \"checks\": [\n      {\n        \"name\": \"xcode\",\n        \"status\": \"ok\",\n        \"message\": \"Xcode 26.5 - Build version 17F42 (/Applications/Xcode.app/Contents/Developer)\"\n      },\n      {\n        \"name\": \"process-tree\",\n        \"status\": \"ok\",\n        \"message\": \"Running under Xcode: No; 4 process entries\"\n      },\n      {\n        \"name\": \"axe\",\n        \"status\": \"ok\",\n        \"message\": \"Available: Yes; UI automation: Yes; Video capture: Yes\"\n      },\n      {\n        \"name\": \"xcodemake\",\n        \"status\": \"ok\",\n        \"message\": \"Enabled: No; Binary: No; Makefile: Not checked\"\n      },\n      {\n        \"name\": \"mise\",\n        \"status\": \"warning\",\n        \"message\": \"Running under mise: No; Available: No\"\n      },\n      {\n        \"name\": \"debugger-dap\",\n        \"status\": \"ok\",\n        \"message\": \"Selected backend: lldb-cli; lldb-dap available: Yes\"\n      },\n      {\n        \"name\": \"manifest-tools\",\n        \"status\": \"ok\",\n        \"message\": \"Total tools: 82; Workflows: 15\"\n      },\n      {\n        \"name\": \"runtime-registration\",\n        \"status\": \"warning\",\n        \"message\": \"Runtime registry unavailable.\"\n      },\n      {\n        \"name\": \"xcode-ide-bridge\",\n        \"status\": \"ok\",\n        \"message\": \"Workflow enabled: No; Connected: No; Proxied tools: 0\"\n      },\n      {\n        \"name\": \"sentry\",\n        \"status\": \"ok\",\n        \"message\": \"Enabled: Yes\"\n      }\n    ]\n  }\n}\n```\n\n### Editor/Client\n\nCodex desktop app, driving XcodeBuildMCP primarily through the CLI for this reproduction.\n\n### MCP Server Version\n\n2.6.2\n\n### LLM\n\nGPT-5 Codex\n\n### MCP Configuration\n\nN/A for the minimal repro path above. The issue was observed using the installed CLI at `/opt/homebrew/bin/xcodebuildmcp`.\n\n### Steps to Reproduce\n\n1. Launch an iOS simulator app and capture a runtime UI snapshot with verbose element refs:\n\n   ```bash\n   xcodebuildmcp ui-automation snapshot-ui \\\n     --simulator-id \u003cUDID\u003e \\\n     --output json \\\n     --style minimal \\\n     --verbose\n   ```\n\n2. Pick a tappable same-screen `elementRef` from the snapshot output, for example a SwiftUI disclosure/card view ref such as `e99`.\n\n3. Attach LLDB to the simulator app and set a breakpoint that will be hit by that tap action.\n\n4. Tap the element ref:\n\n   ```bash\n   xcodebuildmcp ui-automation tap \\\n     --simulator-id \u003cUDID\u003e \\\n     --element-ref e99 \\\n     --post-delay 0 \\\n     --output json \\\n     --style minimal\n   ```\n\n   The tap succeeds and the app stops at the LLDB breakpoint.\n\n5. Continue the app from LLDB.\n\n6. Immediately try to issue a second tap against the same visible element/ref, or another same-screen ref from the same snapshot:\n\n   ```bash\n   xcodebuildmcp ui-automation tap \\\n     --simulator-id \u003cUDID\u003e \\\n     --element-ref e99 \\\n     --pre-delay 0.05 \\\n     --post-delay 0 \\\n     --output json \\\n     --style minimal\n   ```\n\n### Expected Behavior\n\nOne of these would be expected:\n\n- same-screen refs remain usable after continuing from the debugger pause;\n- the cached snapshot remains present and fails through the normal stale/expired-ref path if it is no longer valid;\n- the error clearly explains that the debugger pause/app state transition invalidated the runtime snapshot and that the caller must refresh with `snapshot-ui`/`wait-for-ui`;\n- or there is a supported way to keep/restore refs for debugger-assisted rapid interaction workflows.\n\n### Actual Behavior\n\nThe second ref-based tap fails immediately because the simulator no longer has a runtime UI snapshot available. This happens even though the ref came from a recent snapshot and the app remains on the same visible screen after continuing from the breakpoint.\n\nThis is different from the normal stale-ref behavior. In the same debugging session, an older ref produced an expiration-style error, while the post-breakpoint follow-up produced a no-snapshot error.\n\n### Error Messages\n\n```shell\nNo runtime UI snapshot is available for this simulator.\n```\n\nA related stale-ref path observed separately in the same session produced:\n\n```shell\nThe runtime UI snapshot for this simulator has expired.\n```\n","author":{"url":"https://github.com/dpearson2699","@type":"Person","name":"dpearson2699"},"datePublished":"2026-06-07T23:10:56.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/445/XcodeBuildMCP/issues/445"}