René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"No store migration path: engine 0.22.1 cannot serve a 0.11.2 store; 15s readiness gate aborts; memory_export capped at 15 MiB","articleBody":"## What happened\n\nA long-running daemon (agentmemory 0.9.29 + iii-engine 0.11.2, file_based kv store) degraded into a wedge: REST worker routes stopped answering while the gateway kept accepting connections. Upgrading to 0.9.30 + the pinned engine 0.22.1 gives a healthy daemon — that cannot read the old store, so there is no way to migrate the 985 memories / 27k-node graph off it.\n\nWindows 11, native engine binaries (no Docker).\n\n## 1. The old stack wedges; symptoms are gateway-level\n\nStore `C:\\Users\\Leo\\data\\state_store.db` (348 MB, 331 shards), stack 0.9.29 + iii 0.11.2, ~60 min uptime:\n\n| probe | result |\n| --- | --- |\n| `agentmemory status` (CLI, direct kv read) | `v0.9.29 · degraded · Sessions 349 · Observations 8719 · Memories 985 · Graph 27256 nodes / 54396 edges · Heap 917 MB` |\n| `GET /agentmemory/livez` | 200 in 5.9 s |\n| `GET /agentmemory/health` | times out (>8 s, no response) |\n| `GET /agentmemory/sessions` | times out (>8 s, no response) |\n| `netstat` | ~10 sockets ESTABLISHED on 3111, never closing |\n\nConsumers time out: an extension hook that awaited `smart-search` was killed by the harness at 30 s, every prompt. Health went `degraded → critical` and heap sat at 770–917 MB within an hour of a fresh boot.\n\n## 2. Engine 0.22.1 does not surface the old store\n\n`agentmemory --data-dir C:\\Users\\Leo\\data` (0.9.30, pinned engine 0.22.1 installed to `~/.agentmemory/bin/iii.exe`):\n\n- engine boots, REST binds ~20 s\n- worker log: `Loaded persisted vector index (355 vectors)` → vector index is readable\n- then: `agentmemory worker did not become ready within 15s.` → `process.exit(1)` (hardcoded 15 s, no flag/env to raise it)\n- afterwards `status` reports `Sessions 0 · Observations 0 · Memories 0 · Graph 0 nodes, 0 edges · Provider ✗ noop (no key)` — memories/graph/bm25 keys are invisible, only the vector index loaded\n- in one run the engine grew to 1.3 GB RSS without ever serving REST\n\n## 3. No migration surface\n\n- `agentmemory --help` (0.9.30): no `export`, `backup`, `migrate`, or `restore` command.\n- MCP surface (54 tools, enumerated via `tools/list` on `agentmemory mcp`): no import/restore/bulk-write/seed tool — `memory_save` inserts one record at a time.\n- `memory_export` takes **no** arguments and refuses on a real store:\n  `{\"bytes\":320615835,\"error\":\"Response is 305.8 MiB, over the ~15 MiB limit\",\"limitBytes\":15728640,\"oversized\":true,\"success\":false}`\n  It also proxies over REST, so on a wedged daemon it is unreachable anyway.\n\nNet: 305.8 MiB of history is readable by the old (wedging) stack only, and there is no supported path to move it.\n\n## 4. Embeddings regressed across the upgrade\n\nSame `~/.agentmemory/.env`: 0.9.29 reports `Provider ✓ llm · Embeddings ✓ embeddings`; 0.9.30 on a fresh store reports `Provider ✓ llm · Embeddings bm25-only`. Silent recall-quality downgrade on upgrade; `status` shows it but nothing warns during `upgrade`.\n\n## Asks\n\n1. A documented store migration (0.11.x file_based kv → 0.22.x), or a `migrate --from \u003cdir>` command that reads the old shards and replays them.\n2. Make the worker readiness gate non-fatal and configurable (flag/env, default higher than 15 s). Aborting at 15 s on a 348 MB store leaves an engine running with no worker — indistinguishable from the wedge.\n3. Paginate/filter `memory_export` (`limit`, `offset`, `include: {memories, observations, graph}`) so a 300 MB store is exportable under the shim's response cap.\n4. Detect and warn on embedding-provider loss in `status`/`upgrade` output.\n\n## Workaround applied\n\nFresh store + pinned 0.22.1: healthy, 34 MB heap, `GET /agentmemory/health` 5 ms, save→recall round trip verified. History is kept as a cold archive at the old path. Separately, the consumer side was hardened: every extension request now has a bounded timeout (8 s default, 2 s health, 6 s smart-search), so a wedged daemon costs ~10 s per prompt instead of a 30 s harness timeout.\n","author":{"url":"https://github.com/leopold-dauvergne","@type":"Person","name":"leopold-dauvergne"},"datePublished":"2026-10-18T22:58:06Z","url":"https://github.com/rohitg00/agentmemory/issues/1584"}