René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Provenance-first schema evolution: restructure dependencies and recompute safely","articleBody":"## Summary\n\nRoadmap capability from the strict-provenance adoption analysis ([#1474 discussion](https://github.com/datajoint/datajoint-python/pull/1474#issuecomment-4849784492), @ttngu207): **provenance-first schema evolution** — tooling to change a pipeline's dependency structure and recompute safely, instead of drop-and-repopulate.\n\n## Why\n\nStrict provenance surfaces undeclared dependencies (e.g. \"sideways\" reads to a cousin table). The remediation is always *restructure the schema* — declare the dependency, move a table, split a fat ingestion `make()`. Today there is no tooling for that: users fall back on dropping and repopulating, which destroys history. This is the primary adoption blocker for `strict_provenance` in existing pipelines: the alarm shipped in 2.3; this is the fire exit.\n\n## Shape\n\nBuilt from the primitives 2.3 already ships:\n- `Diagram.trace` answers *what is upstream of the change* (what must be preserved/re-read).\n- `Diagram.cascade` answers *what the change invalidates* (what must be recomputed).\n- A migration runner sequences: declare new FK structure → backfill/attribute the affected rows → recompute invalidated subgraph via the normal populate machinery → retire the old structure.\n\nNot alembic: the hard part is not DDL diffing but managing the **invalidation cascade** with provenance intact.\n\n## Scope notes\n\n- Pairs with #1481 (trace through merge boundaries) — versioned/merge pipelines are the heaviest users of restructuring.\n- The platform's code-deployment CI/CD (agentic review of `make()` source) identifies *what* to restructure; this capability is *how* to execute it safely.\n","author":{"url":"https://github.com/dimitri-yatsenko","@type":"Person","name":"dimitri-yatsenko"},"datePublished":"2026-07-04T20:40:38.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/1482/datajoint-python/issues/1482"}