René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[FEATURE] Add robot coverage for REST page_number pagination and pluggable GraphQL pagination strategies","articleBody":"## Summary\n\nThe three features from the any-sdk GraphQL/pagination uplift are implemented and wired through stackql, but part of the planned robot coverage was never added. This issue tracks the remaining test work only; no feature code is expected to change.\n\nWhat already shipped:\n\n- GraphQL `response.transform` consumption: wired in `internal/stackql/primitivebuilder/graphql_single_select_acquire.go` via PR #658 (`d1f204e`), covered by any-sdk unit tests and the live `Trevorblades Continents GraphQL Response Transform` robot test in `test/robot/integration-traffic-lights/`.\n- Pluggable GraphQL cursor strategies (`cursor_after`, `keyset`, `offset`, `page_info`): plumbed through the same dispatch site; strategies are unit tested in any-sdk (`pkg/graphql/graphql_test.go`).\n- REST `algorithm: page_number` with `responseTerminator`: implemented entirely in any-sdk (`internal/anysdk/pagination.go`, paging loop in `public/providerinvokers/anysdkhttp/invoker.go`), consumed by stackql via the invoker call in `internal/stackql/execution/mono_valent_execution.go`. Closed stackql/any-sdk#91 via stackql/any-sdk#94. Consumed since any-sdk `v0.5.3-alpha05`; current dep is `v0.5.3-alpha11`.\n\nWhat is missing (this issue):\n\n1. There is no stackql robot test or fixture for `page_number` anywhere in the repo (zero occurrences of `page_number` / `responseTerminator` under `test/`).\n2. The mocked GraphQL pagination suite (`test/robot/functional/stackql_graphql_pagination.robot`) exercises only the default `cursor_after` strategy. `keyset`, `offset` and `page_info` have no SQL -\u003e HTTP -\u003e projection round-trip coverage.\n\nPer AGENTS.md, black-box robot regression tests are the required evidence for user-visible query behaviour; any-sdk unit tests alone do not satisfy that for these features.\n\n## Test set A: REST `page_number`\n\nAdd a synthetic fixture under `test/registry-mocked/src/` with a pagination block shaped like Cloudflare V4:\n\n~~~yaml\npagination:\n  algorithm: page_number\n  requestToken:\n    key: page\n    location: query\n  responseToken:\n    key: result_info.page\n    location: body\n  responseTerminator:\n    key: result_info.total_pages\n    location: body\n~~~\n\nScenarios (mock endpoints in the flask tooling under `test/python/stackql_test_tooling/flask/`):\n\n~~~robotframework\nSelect With Page Number Pagination Terminates When Page Equals Total Pages\n    # GET /things?page=1 -\u003e {result_info:{page:1, total_pages:3}, result:[A,B]}\n    # GET /things?page=2 -\u003e {result_info:{page:2, total_pages:3}, result:[C,D]}\n    # GET /things?page=3 -\u003e {result_info:{page:3, total_pages:3}, result:[E]}\n    # Assert: 5 rows, exactly 3 requests.\n\nPage Number Pagination Stops Immediately When Page Equals Total Pages At Start\n    # GET /things?page=1 -\u003e {result_info:{page:1, total_pages:1}, result:[A]}\n    # Assert: 1 row, exactly 1 request.\n\nPage Number Pagination Fails Cleanly When Terminator Field Absent\n    # GET /things?page=1 -\u003e {result:[A]} (no result_info)\n    # Assert: terminates (no infinite loop); row returned or explicit error.\n~~~\n\nThis test set doubles as verification of a residual risk: `internal/stackql/execution/mono_valent_execution.go` still carries its own token-only paging helpers (`inferNextPageResponseElement` / `inferNextPageRequestElement`, flagged `!!TECH_DEBT!!`) that do not know about `page_number`. If the mocked scenario pages and terminates correctly, the live path is confirmed to be the any-sdk invoker. If it loops or stops early, that is a real bug and this issue should be re-scoped.\n\n## Test set B: GraphQL pagination strategies\n\nOne mocked scenario per strategy, following the existing pattern in `stackql_graphql_pagination.robot` (native_test flask mock, `--http.log.enabled` wire assertions):\n\n~~~robotframework\nSelect With Keyset Pagination Drives Forward And Terminates\n    # Page 1: 2 rows; page 2: 1 row; page 3: 0 rows -\u003e terminate.\n    # Assert the _gt filter on the sort key advances per page.\n\nSelect With Offset Pagination Terminates On Short Page\n    # Page size 2. Pages: 2 rows, 2 rows, 1 row (short) -\u003e terminate. 5 rows total.\n\nSelect With PageInfo Pagination Terminates On hasNextPage False\n    # hasNextPage true/true/false across 3 pages -\u003e terminate after page 3.\n~~~\n\nThe existing `cursor_after` scenarios must keep passing unchanged (back-compat guard).\n\n## Out of scope\n\n- Any change to feature code in stackql or any-sdk (unless test set A exposes a live-path bug, which becomes its own issue).\n- A mocked variant of the GraphQL `response.transform` test (live trevorblades coverage exists; a mocked twin is nice-to-have, not required here).\n- REST `link_header` pagination improvements.\n\n## Acceptance criteria\n\n- [ ] Mocked fixture and flask endpoints for `page_number` added under the existing test tooling.\n- [ ] Test set A passes, including both negative scenarios.\n- [ ] Test set B passes for keyset, offset and page_info.\n- [ ] Existing `cursor_after` scenarios and the rest of the robot suite pass unchanged.\n\n## References\n\n- stackql PR #658 (`d1f204e`) - GraphQL transform + cursor config consumption.\n- stackql PRs #678 (`b51c514`), #679 (`902c151`) - any-sdk bumps to v0.5.3-alpha08/alpha11.\n- stackql/any-sdk#91 (feature request), closed by stackql/any-sdk#94 - `page_number` implementation.\n- `test/robot/functional/stackql_graphql_pagination.robot` - existing mocked GraphQL pagination suite.\n- `test/robot/integration-traffic-lights/stackql_traffic_light_integration_from_cmd_line.robot` - live transform coverage.\n","author":{"url":"https://github.com/jeffreyaven","@type":"Person","name":"jeffreyaven"},"datePublished":"2026-07-07T05:39:27.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/684/stackql/issues/684"}