René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Retire the opt-in strict_provenance runtime guardrail; enforce the make() contract by deploy-time code inspection","articleBody":"## Summary\n\nRemove `dj.config[\"strict_provenance\"]` (introduced in 2.3.0, #1425) in an upcoming 2.3.x release, along with its documentation. Keep `Diagram.trace()` (#1423) and `self.upstream` (#1424). Enforcement of the `make()` contract moves to static/agentic inspection of `make()` bodies at deployment (platform CI), backed by governed, commit-pinned execution. The open-source documentation will document the **reproducible `make()` contract** itself — the rules — rather than a runtime enforcement flag.\n\n## What shipped in 2.3.0\n\n`strict_provenance` is an opt-in, off-by-default, best-effort development guardrail. It wires read/write checks into the Python client via a per-call `contextvars` enforcement context, confining reads to `self.upstream[...]` and writes to `self` and Parts. It shipped alongside `Diagram.trace()` and `self.upstream` as the \"provenance trinity.\"\n\n## Why remove it\n\nThe shipped specification already concedes the core point: comprehensive enforcement of the `make()` contract is fundamentally a **code-inspection problem, not a runtime-interception one**. Against the flag's modest dev-only benefit, the costs are real:\n\n1. **Best-effort by construction.** The spec's own limits section enumerates the bypasses — existence/count idioms (`len`, `bool`, `in`), restriction-by-table, and any access outside the Python client. It also cannot see code paths a given run does not execute. A guardrail that misses these invites reliance it cannot support.\n2. **Wrong instrument, wrong time.** The `make()` contract is a static property of the method body (reads confined to the upstream trace cone; writes to self/Parts; one entity per call in isolation; no result-affecting side inputs). Static analysis sees all paths, catches the documented bypass idioms, and reports once in CI before deploy — not per-invocation in the populate hot path.\n3. **Complexity and maintenance burden.** Per-call `contextvars` context, client-path interception, and a documented-limits appendix are a standing support cost for a check that cannot be made sound in-process.\n4. **Redundant with the authoritative enforcer.** The platform inspects `make()` source at deploy and controls execution (commit-pinned, mediated connections). That is where a sound guarantee lives.\n\n## Why removal (not deprecation) is safe now\n\n- The flag is **off by default** and has **no schema, data, or query-result impact** — removal changes no pipeline's behavior for anyone who has not opted in.\n- It was introduced in 2.3.0 and has **no known adoption**: no platform customer has enabled it. Removing it now, before it gains use, is cheaper for everyone than carrying a deprecation cycle for a feature nobody depends on.\n\n## What we keep (and stabilize)\n\n`Diagram.trace()` and `self.upstream` — the provenance query API and the idiomatic read pattern. \"Read from `self.upstream`, write to `self` and Parts\" remains the documented convention, and it is precisely what makes a `make()` method statically verifiable.\n\n## Proposed changes\n\n- Remove the `strict_provenance` config key and the runtime interception machinery (enforcement context, client-path checks).\n- Remove all mentions from the documentation (2.3 release notes, provenance specification §3); reframe the \"trinity\" as the `trace` + `self.upstream` pair.\n- Replace the removed enforcement docs with documentation of the **reproducible `make()` contract**: populate-only insertion; one entity per call, computed in isolation; reads confined to the upstream trace cone; writes only to self and Parts; no other result-affecting inputs (populate kwargs orchestrate, never parameterize).\n- Optionally, publish the static `make()` verifier as a standalone dev tool / pre-commit hook, preserving the \"catch accidental drift locally\" purpose with strictly better coverage than the runtime gate.\n\n## Alternatives considered\n\n- **Harden the runtime gate** — rejected: closing the documented bypasses approaches whole-program analysis inside a running process.\n- **Deprecate first, remove at 2.4** — unnecessary ceremony given off-by-default, zero functional impact, and no known adoption; removing before the feature gains use is the cheapest path for users and maintainers.\n\nRefs: #1425, #1423, #1424, #1474, #1480, #1485. Docs affected: `about/whats-new-23.md`, `reference/specs/provenance.md`.\n","author":{"url":"https://github.com/dimitri-yatsenko","@type":"Person","name":"dimitri-yatsenko"},"datePublished":"2026-07-08T20:11:45.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/1488/datajoint-python/issues/1488"}