René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Feature request: Mergable persistance in Postgres tabular","articleBody":"Hi, me again, another issue to track and get feedback on what improvements we're doing to our fork of TinyBase and what we'll raise PR to contribute back.\n\n## Problem / use-case\n_Is your feature request related to a problem? Please describe._\n\nAs docs mention using mergable data structure with tabular store isn't supported in any SQL backed database\n\u003e The following database-oriented [Persister](https://tinybase.org/api/persisters/interfaces/persister/persister/) types can be used to persist a [MergeableStore](https://tinybase.org/api/mergeable-store/interfaces/mergeable/mergeablestore/), but only in the 'JSON-serialization' mode:\n\u003e - PostgresPersister\n\nOur use-case for having this (related to [#236](https://github.com/tinyplex/tinybase/issues/236)) is that especially during migration towards TinyBase we have an existing application that is writing directly to postgres and we want to sync the changes to a store which is then later synced to DurableObjects and to clients.\n\n\n## Feature Request\n**Describe the solution you'd like**\n\ntl;dr: Allow to use `updated_at` column from the table to create approximate HLC during load from DB. ~~But to be able to do that we'd probably need to change HLC logic.~~\n\nI understand that full support is not possible because it would mean storing HLC and the hash for every cell which in tabular mode isn't really possible/practical. However I believe that we can achieve partial support that would be good in 99% of cases and better than having none.\n\n_disclaimer: following is how I understand how it works. And I'm probably wrong_\n\nIf I understand how HLC generally works it has combination of physical time and logical component. And if we could use `updated_at` timestamp from the database as source of physical time we could create an approximate HLC which could be used for merging data together. If we fix the logical counter for approx HLC to 0 we would have a predictable behaviour that in case of 2 writes within same millisecond TinyBase wins. That would work great for us.\n\n~~However, looking through Tinybase [hlc implementation](https://github.com/tinyplex/tinybase/blob/main/src/common/hlc.ts) it seems that it's not using physical time at all. I'm assuming that it's because we can't trust browsers to have correct time set and wrong time could cause completely wrong resolutions. But often HLC implementations have built in protection by checking for back-in-time jumps and adjusting physical time (from wall clock time) to handle that gracefully.~~ _As guessed at in the original write down, I was wrong. The Tinybase HLC implementation uses timestamp_\n\n## Alternatives\n**Describe alternatives you've considered**\nunfortunately we don't see an alternative that would work for us within current TinyBase capabilities. Our main alternative is to completely fork the persister and implement this.\n\n**Additional context**\nThis partially relates to my existing [PR for whereCondition](https://github.com/tinyplex/tinybase/pull/234) and [Feature Request #236](https://github.com/tinyplex/tinybase/issues/236) for partial load of postgres changes but it's not coupled with either of them.\n","author":{"url":"https://github.com/jakubriedl","@type":"Person","name":"jakubriedl"},"datePublished":"2025-04-29T14:55:24.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/237/tinybase/issues/237"}