René's URL Explorer Experiment

Title: feat(runtime-core,cli,mcp): typed runtime failures, fail-fast shutdown, and real toolkit integration tests by jamesbhobbs · Pull Request #519 · deepnote/deepnote · GitHub

Open Graph Title: feat(runtime-core,cli,mcp): typed runtime failures, fail-fast shutdown, and real toolkit integration tests by jamesbhobbs · Pull Request #519 · deepnote/deepnote

X Title: feat(runtime-core,cli,mcp): typed runtime failures, fail-fast shutdown, and real toolkit integration tests by jamesbhobbs · Pull Request #519 · deepnote/deepnote

Description: Stacked on #518 (interpreter resolution for #288); merge that first. Part of #162: for headless execution to be a first-class feature, the runtime has to fail loudly, fail fast, and be tested against the real toolkit. Warm server pooling for the MCP is deliberately not in this PR. It lands as a follow-up stacked on this one (#520), with real concurrency and crash-recovery tests, so the reliability work here can be reviewed and merged on its own. Problems Read against the runtime as it stands on main: Nothing in CI starts the toolkit. The "CLI E2E" job runs --version and --help; every engine and kernel test mocks the Jupyter client. A toolkit release or a @jupyterlab/services bump can break deepnote run with all checks green. A dead server hangs the run. If the Jupyter server dies mid-run, the websocket drops and the Jupyter client retries with backoff for about two minutes without ever marking the kernel dead; the in-flight execution waits forever. The server starter only watches for process exit during startup. A failed start delays exit by two minutes. When python -m deepnote_toolkit server exits immediately (toolkit not installed), the startup race rejects at once, but the health-check loop is never cancelled and keeps polling until the 120 s startup timeout. The CLI has already printed its error and sits there. Measured at 2:00 before, 0.3 s after. Every start failure says "install deepnote-toolkit". The CLI appends that hint to any engine.start() error, including port conflicts, timeouts and bad --python paths. Machine output has no failure category, and exit code 1 covers infra failures and user code alike. Interpreter fallback ignores a project .venv. #518 adds the extension sidecar and DEEPNOTE_PYTHON; a .venv next to the notebook was still ignored in favour of bare python. Timeouts are constants (30 s kernel idle, 120 s server start) and server logs are dropped unless the process dies during startup. Stopping the server can leak Jupyter. stopServer SIGKILLs the toolkit supervisor two seconds after SIGTERM. The supervisor terminates each child and waits up to five seconds for it, so under load it is still shutting Jupyter down when it is killed, and the Jupyter server, language server and kernel are orphaned. Found while adding a leaked-process check to the integration tests; it reproduced twice in full runs and never in isolation. Changes Typed failures (@deepnote/runtime-core). New RuntimeError family with a closed category: server-launch, kernel-launch, kernel-died, server-exited, execution-timeout, plus in-block for the block's own code. Errors carry an optional hint. startServer names the real cause (toolkit missing for that interpreter, a missing server dependency, process exit with stderr tail, health-check timeout). BlockExecutionResult and ExecutionSummary gain failureCategory. No more hangs. ServerInfo.exited settles when the process ends; the engine fails the in-flight execution with ServerExitedError when that happens mid-run, through a removable process listener it detaches on stop. The kernel client fails in-flight executions with KernelDiedError on dead/autorestarting status (and translates the client's opaque "Canceled future" rejection), and runs a connection watchdog: a dropped websocket that is not back within 10 s is probed; a dead server or missing kernel fails immediately, a server that still answers gets exactly one more grace period. After a fatal error every later execute() rejects immediately, and a client can connect() again from a clean slate. Integration tests measure a dead kernel reported in ~8 s and a killed Jupyter server in ~14 s. Fast startup failure. The health-check loop is abortable and is aborted when startup fails, and every health request is bounded by the time left in the startup timeout, so a server that accepts the connection but never answers cannot stall past --startup-timeout. Kernel disposal detaches the kernel's Lumino listeners when the connection is down, because @jupyterlab/services awaits its post-restart reconnect() without handling rejection; disposing mid-reconnect otherwise surfaces an unhandled rejection. Adds @lumino/signaling as a direct dependency (already in the tree). No more orphaned servers. The toolkit supervisor starts Jupyter and the language server in their own sessions, so nothing but its explicit cleanup ever reaches them. startServer records those child pids once the server is ready, and stopServer waits up to 10 s for the supervisor to finish its own cleanup, kills the children before force-killing a supervisor that ignores SIGTERM (also on a failed startup), and afterwards terminates any recorded child that outlived the supervisor. Every integration test file checks after each test that no toolkit, Jupyter, language-server or kernel process from the interpreter's environment was left behind, and prints the toolkit server's log when one is. Configurable timeouts and logs. RuntimeConfig gains serverStartupTimeoutMs, kernelStartupTimeoutMs, blockTimeoutMs and onServerLog. A block that exceeds the block timeout is interrupted and the run fails with execution-timeout. CLI: --startup-timeout (applied to server and kernel) and --block-timeout ; deepnote --debug run streams the toolkit server log as [server stdout]/[server stderr]. CLI reporting. The blanket install hint is gone; Failed to start server: is followed by the runtime's hint (and the interpreter hint from #518 when only a bare python was available). -o json/-o toon add failureCategory on the run and on the failed block, and hint; a startup failure emits { success: false, error, failureCategory, hint }. Exit codes are unchanged. MCP reporting. deepnote_run responses now report success: false when a block failed (previously hardcoded true), plus failureCategory and hint; a runtime that cannot start returns a structured error with the same fields. Each call still starts and stops its own server; reuse is the follow-up PR. Interpreter resolution. resolveProjectPython gains a tier between the IDE sidecar and the system default: a .venv/venv found from the notebook's directory upward whose interpreter can import deepnote_toolkit. A venv without the toolkit is skipped with a warning, so an unrelated project venv never shadows a working system Python (source: venv). Integration tests in CI. *.integration.test.ts files are excluded from pnpm test and run by pnpm test:integration (vitest.integration.config.ts). New job "Runtime Integration (deepnote-toolkit)" installs Python 3.12 + deepnote-toolkit[server], builds, and runs them with DEEPNOTE_PYTHON set. Eleven tests against the real server: engine success/in-block/kernel-died/server-exited/block-timeout/missing-toolkit, the built CLI for success/in-block/--block-timeout/missing toolkit (asserting the fast exit), and the pre-existing local-runner test, which now runs instead of skipping. A leak guard keyed on the interpreter's sys.prefix flags any process a test left behind. Docs skills/deepnote/references/cli-run.md (new flags, failure-category table, venv tier, cloud-ignored flags), cli-analysis.md, SKILL.md, packages/cli/README.md, packages/mcp/README.md (failure fields, venv tier), AGENTS.md and CONTRIBUTING.md (pnpm test:integration). Tests Unit: the full suite passes; new coverage for the server starter (exit tracking, abort, cause detection, log forwarding, child cleanup), kernel client (typed failures, timeout + interrupt, watchdog with bounded retry, reconnect after failure, listener detachment), engine (categories, server exit, listener detachment), the venv tier, the CLI (hints, timeouts, debug logs, JSON fields) and the MCP (structured failures, engine lifecycle). Integration: 11 tests pass locally against deepnote-toolkit 2.6.0 (pnpm test:integration, ~65 s) and in the new CI job, with no leaked processes. pnpm typecheck, pnpm lintAndFormat, pnpm spell-check clean. Not in this PR: warm server pooling for the MCP (follow-up); --kernel / non-Python kernels (see #154, #162); .venv discovery does not consult pyproject.toml; and if the toolkit supervisor itself is SIGKILLed (for example by the OOM killer) its already-orphaned children are cleaned up only when stopServer runs, which is only fully fixable on the toolkit side (a process group or PR_SET_PDEATHSIG for its children). 🤖 Generated with Claude Code Summary by CodeRabbit New Features Added startup and per-block timeout options for local runs. Added virtual-environment discovery and improved interpreter selection. Added structured runtime failure categories, actionable hints, and machine-readable results. Added debug-mode server log forwarding. Bug Fixes Improved detection, reporting, and cleanup for kernel, server, startup, and timeout failures. Improved execution lifecycle handling and process cleanup. Documentation Expanded CLI and integration-testing guidance, including CI fail-fast examples. Tests Added integration coverage for successful runs, failures, timeouts, and process cleanup.

Open Graph Description: Stacked on #518 (interpreter resolution for #288); merge that first. Part of #162: for headless execution to be a first-class feature, the runtime has to fail loudly, fail fast, and be tested again...

X Description: Stacked on #518 (interpreter resolution for #288); merge that first. Part of #162: for headless execution to be a first-class feature, the runtime has to fail loudly, fail fast, and be tested again...

Opengraph URL: https://github.com/deepnote/deepnote/pull/519

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width