René's URL Explorer Experiment

Title: SK-2954: common/v2/flowvault split + flowvault insert support by saileshwar-skyflow · Pull Request #269 · skyflowapi/skyflow-python · GitHub

Open Graph Title: SK-2954: common/v2/flowvault split + flowvault insert support by saileshwar-skyflow · Pull Request #269 · skyflowapi/skyflow-python

X Title: SK-2954: common/v2/flowvault split + flowvault insert support by saileshwar-skyflow · Pull Request #269 · skyflowapi/skyflow-python

Description: Why Splits skyflow-python from a single-variant SDK into three build variants sharing a bundled common/ module: common/ — shared credential resolution, vault-URL resolution, bearer-token fetch/cache/expiry, validation, logging, and client-composition logic. Never published independently; bundled into each variant's wheel at build time. v2/ — today's SDK, relocated from the repo root. Public API unchanged — same class names, signatures, import paths. flowvault/ (renamed from v3/) — new, built on the flowservice/flowdb API. Insert-only this round by design; every other operation is deliberately out of scope until a follow-up round (tracked in SK-2972). Publishes as skyflow-flowvault, importable as skyflow_flowvault, starting at 1.0.0. common/ — interfaces and shared logic ISkyflow/IVaultController/IVaultClient: pure interfaces (ABC + @abstractmethod, no implementation) declaring the contract each variant must satisfy. BaseSkyflow/BaseVaultController/BaseVaultClient implement them with the logic shared by every variant — protected helpers like _validate_table_name_if_present/_validate_field_values live on the base, per-variant behavior (insert/get/... , resolve_vault_url/initialize_api_client) stays abstract for concrete subclasses to fill in. make_skyflow_class(...): a factory function that builds each variant's concrete Skyflow class from injected collaborators (VaultClient, VaultController, its messages catalog, validators). Each variant's own client/skyflow.py is now a handful of lines calling this factory with its own classes — no variant-specific wiring logic lives in common, and common never imports v2 or flowvault. Unified validation/logging: validate_vault_config, validate_update_vault_config, validate_log_level, validate_credentials, LogLevel, Logger were confirmed field-for-field identical between v2 and flowvault and are now single implementations in common, using a messages= dependency-injection parameter so each variant's own error text/SDK version still renders correctly (common has no SDK version of its own). BaseInsertRequest/BaseInsertResponse: shared base types for insert's request/response shape. table/values are required (no silent default) on the base; upsert's concrete type is left generic on the base (Union[str, dict]) since it's a plain string for v2 and a dict for flowvault — each variant's own InsertRequest/InsertResponse subclass declares its exact type. BaseSkyflow and the interface classes guard against direct instantiation via a SkyflowError (not a bare NotImplementedError), matching how every other SDK error is raised. v2/ Structural relocation, zero behavior change to the public contract — same class names, signatures, import paths. Vault remains a backward-compatible alias for the internal VaultController name. InsertRequest now extends BaseInsertRequest; every controller method (insert, get, update, delete, query, detokenize, tokenize, upload_file) has full request/response type hints. Fixed a real bug surfaced during this work: v2's own Env enum was failing cross-class comparisons against common's Env because they were two structurally-identical-but-distinct enum classes — v2's is now a re-export of common's. flowvault/ — insert support Request shape: InsertRequest(values, table=None, upsert=None) — values is a plain list of dicts (each shaped {"values": {...}, "table": ..., "upsert": ...}), matching PDB's own naming/shape convention rather than a custom InsertRecord class. upsert (at either the request or per-record level) is a plain dict, typed via a TypedDict (Upsert) rather than a class — type(upsert) is exactly dict at runtime, matching how records are handled. InsertResponse shape was iterated against a real vault until it matched PDB's own shape exactly: no redundant data/table keys, tokens flattened directly onto each result dict (no nested tokens key), errors=None (not []) when there are none, each entry tagged with request_index (stable position in the original values list). Validation ported from Java's v3 Validations.java: table/upsert must live in exactly one place (request-level applying to every record, or per-record — never mixed, never both, confirmed against a real vault's 400 response), 10,000-record cap, and (post-review) empty/null field values are explicitly allowed — only empty/null keys are rejected. Error handling: parses the backend's structured per-record error body into individual entries, propagating x-request-id onto each. All records for a request go out in a single API call (no batching this round). Fixed a real, live-vault-confirmed bug: SDK_VERSION was hardcoded/stale (0.1.0) instead of matching setup.py's real version (1.0.0). Skyflow/VaultController/VaultClient are built via make_skyflow_class/extend common's base classes — no flowvault-specific wiring logic beyond supplying its own collaborators. CI/CD Every workflow (main, ci, beta-release, internal-release, release, and the two shared reusable workflows) now correctly targets flowvault/ (the workflows were still pointing at the pre-rename v3/ directory, which meant every flowvault CI job failed outright — ci-scripts/bump_version.sh and the shared workflows now also take a package-name input, since they'd hardcoded the skyflow package name too and would've silently bumped/measured the wrong files for flowvault). test-common now actually runs coverage and uploads to Codecov, matching v2/flowvault's own pattern — it previously ran unittest discover with no coverage tracking at all, making common/'s substantial growth this round invisible to Codecov's patch/project checks. Fixed .semgreprules/customRule.yml's check-sensitive-info regex: an optional quote-capture group let its own backreference match an empty string, so the rule fired on any keyword: value regardless of quoting — flagging things like token=OPTIONAL_TOKEN or CredentialField.TOKEN as "sensitive info". Tightened to require an actual quoted literal and exclude self-referential values (TOKEN = 'token'), eliminating 28 false positives with no inline suppressions. Also excludes **/generated/** (Fern-owned code) from semgrep, and fixed a real finding — a ${{ }} GitHub Actions context value was interpolated directly into a run: shell block instead of being routed through env: first. Testing common/tests, v2/tests, flowvault/tests, tests/contract/ all pass locally with no regressions from this round's changes. Manually verified end-to-end against a real flowdb/flowvault vault multiple times through this round (SSL, auth, vault-URL resolution, exact response/error shapes all confirmed live, not just mocked). Known follow-ups (intentionally out of scope here) flowvault's get/update/delete/query/detokenize are stubs (NotImplementedError) — tracked in SK-2972, one operation at a time, same pattern as this round's insert. CI: JFrog Artifactory publish target is reused as-is for both variants; worth a careful look before the first real flowvault- tag/branch push since some of the release automation couldn't be dry-run outside GitHub's own infrastructure.

Open Graph Description: Why Splits skyflow-python from a single-variant SDK into three build variants sharing a bundled common/ module: common/ — shared credential resolution, vault-URL resolution, bearer-token fetch/cac...

X Description: Why Splits skyflow-python from a single-variant SDK into three build variants sharing a bundled common/ module: common/ — shared credential resolution, vault-URL resolution, bearer-token fetch/cac...

Opengraph URL: https://github.com/skyflowapi/skyflow-python/pull/269

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width