René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Runtime snapshot does not expose SwiftUI TabView tab items as actionable refs","articleBody":"### Summary\n\nWith XcodeBuildMCP 2.6.0 runtime UI automation, SwiftUI `TabView` tab bar items can collapse into a single non-actionable `Tab Bar` element in `snapshot-ui`. The individual tabs are not exposed as targetable `elementRef`s, so agents still have to rely on an app-side UIKit accessibility/automation bridge or coordinates to switch tabs.\n\nThis may be an AXe capture limitation rather than a bug in the JS runtime-snapshot normalizer, but it shows up as an XcodeBuildMCP UI automation gap because `wait-for-ui` and `tap --element-ref` can only act on elements present in the runtime snapshot.\n\n### Environment\n\n- XcodeBuildMCP: 2.6.0 (Homebrew)\n- Xcode: 26.5 / build 17F42\n- Simulator profile: clean disposable iOS simulator\n- App under test: generic SwiftUI iOS app using `TabView(selection:)` with `.tabItem { Label(...) }`, `.tag(...)`, and `.accessibilityIdentifier(\"tab_reports\")` on the tab content view\n\n`xcodebuildmcp doctor doctor --output json --style minimal` reported:\n\n- `serverVersion`: `2.6.0`\n- `axe`: `Available: Yes; UI automation: Yes; Video capture: Yes`\n- `manifest-tools`: `Total tools: 82; Workflows: 15`\n\n### Reproduction\n\n1. Build and launch a SwiftUI app with a standard tab view:\n\n```swift\nTabView(selection: $selectedTab) {\n    NavigationStack { HomeView() }\n        .accessibilityIdentifier(\"screen_home\")\n        .tabItem { Label(\"Home\", systemImage: \"house\") }\n        .tag(Tab.home)\n        .accessibilityIdentifier(\"tab_home\")\n\n    NavigationStack { ReportsView() }\n        .accessibilityIdentifier(\"screen_reports\")\n        .tabItem { Label(\"Reports\", systemImage: \"chart.bar\") }\n        .tag(Tab.reports)\n        .accessibilityIdentifier(\"tab_reports\")\n}\n```\n\n2. Capture a verbose runtime snapshot:\n\n```bash\nxcodebuildmcp ui-automation snapshot-ui \\\n  --simulator-id \u003cUDID\u003e \\\n  --output json \\\n  --style minimal \\\n  --verbose\n```\n\n3. Try waiting for the tab identifier or a tab role/label:\n\n```bash\nxcodebuildmcp ui-automation wait-for-ui \\\n  --simulator-id \u003cUDID\u003e \\\n  --predicate exists \\\n  --identifier tab_reports \\\n  --timeout-ms 7000 \\\n  --output json \\\n  --style minimal \\\n  --verbose\n\nxcodebuildmcp ui-automation wait-for-ui \\\n  --simulator-id \u003cUDID\u003e \\\n  --predicate exists \\\n  --label Reports \\\n  --role tab \\\n  --timeout-ms 7000 \\\n  --output json \\\n  --style minimal \\\n  --verbose\n```\n\n### Actual result\n\nBoth waits time out with no candidates. The runtime snapshot contains a single `Tab Bar` element but no child tab controls and no actionable `Reports` target:\n\n```json\n{\n  \"elementCount\": 7,\n  \"targetCount\": 0,\n  \"screenHash\": \"1aztj8k\"\n}\n```\n\nRelevant elements from `data.capture.elements`:\n\n```json\n[\n  {\n    \"ref\": \"e1\",\n    \"role\": \"application\",\n    \"label\": \"SampleTabApp\",\n    \"frame\": { \"x\": 0, \"y\": 0, \"width\": 440, \"height\": 956 },\n    \"actions\": []\n  },\n  {\n    \"ref\": \"e3\",\n    \"role\": \"other\",\n    \"label\": \"Home\",\n    \"actions\": [\"longPress\", \"touch\"]\n  },\n  {\n    \"ref\": \"e7\",\n    \"role\": \"other\",\n    \"label\": \"Tab Bar\",\n    \"frame\": { \"x\": 0, \"y\": 873, \"width\": 440, \"height\": 83 },\n    \"actions\": [\"longPress\", \"touch\"]\n  }\n]\n```\n\nThe action hints confirm there is no tappable tab target:\n\n```json\n{\"action\":\"longPress\",\"elementRef\":\"e7\",\"label\":\"Tab Bar\"}\n{\"action\":\"touch\",\"elementRef\":\"e7\",\"label\":\"Tab Bar\"}\n```\n\n`wait-for-ui --identifier tab_reports` and `wait-for-ui --label Reports --role tab` both return:\n\n```json\n{\n  \"code\": \"WAIT_TIMEOUT\",\n  \"message\": \"Timed out after 7000ms waiting for UI predicate 'exists'.\",\n  \"candidates\": []\n}\n```\n\n### Expected result\n\nFor a standard SwiftUI `TabView`, the runtime snapshot should expose each visible tab item as a targetable element, ideally with:\n\n- role `tab` or an equivalent actionable role;\n- label such as `Reports`;\n- any available accessibility identifier from the tab item/content association, when iOS exposes it;\n- a `tap` action and stable `elementRef`.\n\nThat would let agents switch tabs using the v2.6 runtime workflow instead of coordinates or app-side UIKit overlays.\n\n### Workaround\n\nAn app-side UIKit overlay of transparent `UIButton`s at the tab bar frame works. In the same app/session, adding buttons with `accessibilityIdentifier` values like `tab_reports` produced runtime refs such as:\n\n```json\n{\n  \"ref\": \"e11\",\n  \"role\": \"button\",\n  \"label\": \"Reports\",\n  \"identifier\": \"tab_reports\",\n  \"actions\": [\"tap\", \"longPress\", \"touch\"]\n}\n```\n\nThen `tap --element-ref e11` successfully navigated to the Reports screen and the refreshed snapshot reported `tab_reports` as selected.\n\nThat workaround is undesirable for SwiftUI-first apps, so it would be useful if XcodeBuildMCP/AXe could expose standard SwiftUI tab items directly.\n","author":{"url":"https://github.com/dpearson2699","@type":"Person","name":"dpearson2699"},"datePublished":"2026-06-02T06:29:50.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/439/XcodeBuildMCP/issues/439"}