René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"sqlite: `expandedSQL`, `sourceSQL`, column names, and `sqlite.db.query` tracing abort the process on text longer than `String::kMaxLength`","articleBody":"### Version\n\nv26.10.0 (all repros below). The `expandedSQL` blob repro also aborts on v24.21.0 and v22.23.2.\n\n### Platform\n\n```text\nLinux hostname 7.0.0-34-generic #34-Ubuntu SMP PREEMPT_DYNAMIC Wed Sep  2 14:29:37 UTC 2026 x86_64 GNU/Linux\n```\n\n### Subsystem\n\nsqlite\n\n### What steps will reproduce the bug?\n\n\n`statement.expandedSQL`:\n\n```js\nconst { DatabaseSync } = require('node:sqlite');\nconst { constants } = require('node:buffer');\n\nconst db = new DatabaseSync(':memory:');\nconst stmt = db.prepare('SELECT ?');\nstmt.run(Buffer.alloc((constants.MAX_STRING_LENGTH >>> 1) + 1));\nstmt.expandedSQL; // aborts\n```\n\n`sqlite.db.query` diagnostics channel (v26.8.0 and later):\n\n```js\nconst { DatabaseSync } = require('node:sqlite');\nconst dc = require('node:diagnostics_channel');\nconst { constants } = require('node:buffer');\n\ndc.subscribe('sqlite.db.query', () => {});\nconst db = new DatabaseSync(':memory:');\ndb.prepare('SELECT length(?)').get(\n  Buffer.alloc((constants.MAX_STRING_LENGTH >>> 1) + 1)); // aborts\n```\n\n`statement.sourceSQL`:\n\n```js\nconst { DatabaseSync } = require('node:sqlite');\nconst { constants } = require('node:buffer');\n\nconst n = Math.ceil((constants.MAX_STRING_LENGTH + 1) / 3);\nconst db = new DatabaseSync(':memory:');\nconst stmt = db.prepare(`SELECT '${'€'.repeat(n)}'`);\nstmt.sourceSQL; // aborts\n```\n\nIn the first two, the blob is 268,435,445 bytes. `sqlite3_expanded_sql()` renders a bound blob as `x'\u003chex>'`, so the expanded SQL is 536,870,900 bytes, which is more than `String::kMaxLength` (536,870,888 on 64-bit).\n\nIn the third, the SQL string has 178,956,972 characters, which is under `String::kMaxLength`, but its UTF-8 encoding is 536,870,898 bytes, which is over it.\n\n\n### How often does it reproduce? Is there a required condition?\n\nEvery run. The condition is SQL text whose UTF-8 encoding is longer than `String::kMaxLength` bytes:\n\n- `expandedSQL` and the trace callback: a bound parameter makes the expanded SQL that long. A blob does it at about 256 MiB. On v26.10.0, a text parameter of multi-byte characters also does it: `stmt.run('€'.repeat(178956963))` followed by `stmt.expandedSQL` aborts.\n- `sourceSQL`: the SQL passed to `prepare()` is that long in UTF-8.\n\nThe two blob repros take about 8 s and reach about 830 MB peak RSS. The `sourceSQL` repro takes 7.4 s and reaches 3.0 GB. All were measured with `/usr/bin/time`.\n\n\n### What is the expected behavior? Why is that the expected behavior?\n\n\nThe process should not abort.\n\n- `statement.expandedSQL` should throw `ERR_STRING_TOO_LONG`. Since c5b7a06d9b (#66209), column values and user-defined function arguments longer than `String::kMaxLength` throw `ERR_STRING_TOO_LONG`, and `buffer.toString()` throws the same code for the same limit.\n- A statement traced through `sqlite.db.query` should complete as it does without a subscriber. If the SQL text cannot be represented as a string, the event can be skipped.\n- `statement.sourceSQL` should return the string or throw `ERR_STRING_TOO_LONG`. On v24.21.0 the `sourceSQL` repro and the text-parameter `expandedSQL` variant both return the string.\n\n\n### What do you see instead?\n\n\n`expandedSQL` on v26.10.0 (exit code 133, SIGTRAP):\n\n```text\n#\n# Fatal error in , line 0\n# Check failed: String::kMaxLength >= len.\n#\n#\n#\n#FailureMessage Object: 0x7ffdf8743130\n----- Native stack trace -----\n 1: 0xaa1c41  [node]\n 2: 0x28ff6b1 V8_Fatal(char const*, ...) [node]\n 3: 0xcf20be  [node]\n 4: 0xbb1ff5 node::sqlite::StatementSync::ExpandedSQLGetter(v8::FunctionCallbackInfo\u003cv8::Value> const&) [node]\n 5: 0x73297fdcfae9\nTrace/breakpoint trap (core dumped)\n```\n\n`sqlite.db.query` on v26.10.0 (exit code 133):\n\n```text\n#\n# Fatal error in , line 0\n# Check failed: String::kMaxLength >= len.\n#\n#\n#\n#FailureMessage Object: 0x7ffd1b3ad090\n----- Native stack trace -----\n 1: 0xaa1c41  [node]\n 2: 0x28ff6b1 V8_Fatal(char const*, ...) [node]\n 3: 0xcf20be  [node]\n 4: 0xba679c node::sqlite::DatabaseSync::TraceCallback(unsigned int, void*, void*, void*) [node]\n 5: 0x22f1a46  [node]\n 6: 0x22eda9d  [node]\n 7: 0xbb07ac node::sqlite::StatementExecutionHelper::Get(node::Environment*, node::sqlite::StatementSync*) [node]\n 8: 0xbb1368 node::sqlite::StatementSync::Get(v8::FunctionCallbackInfo\u003cv8::Value> const&) [node]\n 9: 0x746b6f58fae9\nTrace/breakpoint trap (core dumped)\n```\n\n`sourceSQL` on v26.10.0 (exit code 133):\n\n```text\n#\n# Fatal error in , line 0\n# Check failed: String::kMaxLength >= len.\n#\n#\n#\n#FailureMessage Object: 0x7ffe08b2f620\n----- Native stack trace -----\n 1: 0xaa1c41  [node]\n 2: 0x28ff6b1 V8_Fatal(char const*, ...) [node]\n 3: 0xcf20be  [node]\n 4: 0xbb1ea6 node::sqlite::StatementSync::SourceSQLGetter(v8::FunctionCallbackInfo\u003cv8::Value> const&) [node]\nTrace/breakpoint trap (core dumped)\n```\n\nOn v24.21.0 and v22.23.2 the blob `expandedSQL` repro fails a different check, `Check failed: (location_) != nullptr.`, with the same `ExpandedSQLGetter` frame. The reason is below.\n\n\n### Additional information\n\nThis was discovered and repro'ed by claude opus 5.5.\n\nThese call sites pass SQLite's text to `String::NewFromUtf8()` without a length:\n\n- `Statement::ExpandedSQLGetter`: [src/node_sqlite.cc#L4437](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L4437)\n- `Statement::SourceSQLGetter`: [src/node_sqlite.cc#L4417](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L4417)\n- `Database::TraceCallback`, for the expanded SQL: [src/node_sqlite.cc#L3716](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L3716) (added in c7b962c222b, #62241)\n- `Database::TraceCallback`, for the source SQL fallback: [src/node_sqlite.cc#L3725](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L3725)\n\nEach one then checks `ToLocal()`, but that code is not reached. In V8's `NEW_STRING` macro ([deps/v8/src/api/api.cc#L7524-L7543](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/deps/v8/src/api/api.cc#L7524-L7543)), the check that returns an empty handle for `length > String::kMaxLength` runs only when `length > 0`. With the default length of -1, V8 measures the string itself:\n\n- V8 in v26.x and main (14.6): `StringLength()` runs `CHECK_GE(String::kMaxLength, len)` on the `strlen()` byte count ([api.cc#L7470-L7475](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/deps/v8/src/api/api.cc#L7470-L7475)) and aborts. This CHECK is present from v26.0.0. v25.9.0 (V8 14.1) checks only against `kMaxInt`.\n- V8 in v24.x (13.6) and v22.x (12.4): `StringLength()` checks only against `kMaxInt`. `Factory::NewStringFromUtf8()` then sizes the string by its UTF-16 length. If that is over `String::kMaxLength`, `NewRawStringWithMap()` throws `InvalidStringLength`, returns an empty `MaybeHandle`, and the `.ToHandleChecked()` in `NEW_STRING` ([v24.21.0 api.cc#L7624](https://github.com/nodejs/node/blob/955266bfdd854cd280dffd47548673914484e4c0/deps/v8/src/api/api.cc#L7624)) aborts. If it is under, as in the `sourceSQL` repro, the string is created.\n\nSo the blob `expandedSQL` repro aborts on v22.23.2, v24.21.0, and v26.10.0. The `sourceSQL` repro and the text-parameter `expandedSQL` variant abort on v26.10.0 and return the string on v24.21.0.\n\nWith an explicit length greater than `String::kMaxLength`, `NEW_STRING` returns an empty `MaybeLocal` without throwing and without aborting.\n\n`TraceCallback` uses the source SQL fallback when `sqlite3_expanded_sql()` returns NULL. That happens when the expansion would exceed `SQLITE_LIMIT_LENGTH` (1,000,000,000 bytes by default). This aborts v26.10.0 in `TraceCallback` (14.5 s, 2.7 GB peak RSS):\n\n```js\nconst { DatabaseSync } = require('node:sqlite');\nconst dc = require('node:diagnostics_channel');\nconst { constants } = require('node:buffer');\n\ndc.subscribe('sqlite.db.query', () => {});\nconst n = Math.ceil((constants.MAX_STRING_LENGTH + 1) / 3);\nconst db = new DatabaseSync(':memory:');\nconst stmt = db.prepare(\n  `SELECT length('${'€'.repeat(n)}') AS a, length(?) AS b`);\nstmt.get(Buffer.alloc(240_000_000)); // aborts\n```\n\nColumn names take the same path. `Statement::ColumnNameToName` ([src/node_sqlite.cc#L4007](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L4007)) creates row object keys with `NewFromUtf8(..., NewStringType::kInternalized)` and no length, and SQLite names an unaliased result column after its expression text. This aborts v26.10.0 in `StatementSync::GetCachedColumnNames` (8.4 s, 3.5 GB peak RSS). On v24.21.0 it returns a row whose key has 178,956,973 characters:\n\n```js\nconst { DatabaseSync } = require('node:sqlite');\nconst { constants } = require('node:buffer');\n\nconst n = Math.ceil((constants.MAX_STRING_LENGTH + 1) / 3);\nconst db = new DatabaseSync(':memory:');\ndb.prepare(`SELECT length('${'€'.repeat(n)}')`).get(); // aborts\n```\n\nThese calls also pass no length. I have not tested whether SQLite can produce text that long for them:\n\n- `NullableSQLiteStringToValue()`, used by `statement.columns()` and the authorizer callback: [src/node_sqlite.cc#L402](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L402)\n- `CreateSQLiteErrorImpl()`, for the error message and `errstr`: [src/node_sqlite.cc#L287](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L287) and [#L296](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L296)\n\nc5b7a06d9b added the `ERR_STRING_TOO_LONG` check in `Utf8StringMaybeOneByte()` ([src/node_sqlite.cc#L74-L93](https://github.com/nodejs/node/blob/2152942d52f9f841f7de79c7219742ed61e82b90/src/node_sqlite.cc#L74-L93)) for column values and function arguments. It did not change these call sites.\n\nSuggested fix (not built or tested against Node.js core):\n\n- `ExpandedSQLGetter` and `SourceSQLGetter`: pass the length, and throw `ERR_STRING_TOO_LONG` when V8 returns an empty handle. `Utf8StringMaybeOneByte(env->isolate(), std::string_view(...))` already does both.\n- `ColumnNameToName`: same, but it needs `kInternalized`, so it would pass the length to `NewFromUtf8()` directly and throw `ERR_STRING_TOO_LONG` itself. `GetCachedColumnNames()` already returns `false` when it gets an empty handle.\n- `TraceCallback`, both calls: pass `strlen()` as the length. The existing `return 0` on an empty handle then skips publishing and the traced statement completes, so subscribing to the channel does not change whether a statement succeeds. SQLite calls this callback from inside `sqlite3_step()`, `sqlite3_reset()`, or `sqlite3_finalize()`. Both strings are capped at 1,000,000,000 bytes by default (`SQLITE_LIMIT_LENGTH` and `SQLITE_LIMIT_SQL_LENGTH`), so the length fits in an `int`.\n\nV8 compares an explicit length against `String::kMaxLength` in bytes, not UTF-16 code units. With this fix, the `sourceSQL` repro, the text-parameter `expandedSQL` variant, and the column-name repro would throw `ERR_STRING_TOO_LONG`, where v24.21.0 returns the string or row. `Utf8StringMaybeOneByte()` already rejects column values by byte length the same way. Returning those strings would require computing the UTF-16 length and transcoding before creating the string.\n\n@photostructure/sqlite, a Node-API port of `node:sqlite`, hit the same abort through `napi_create_string_utf8(..., NAPI_AUTO_LENGTH, ...)`, which takes the same `length = -1` path. Its fix, [photostructure/node-sqlite@095efcd](https://github.com/photostructure/node-sqlite/commit/095efcdc38957c7d09da98634ab248e317c21c7a), takes the approach above for `expandedSQL` and the trace callback: pass `strlen()`, throw `ERR_STRING_TOO_LONG` from `expandedSQL`, and skip the trace event.\n","author":{"url":"https://github.com/mceachen","@type":"Person","name":"mceachen"},"datePublished":"2026-10-03T19:30:32Z","url":"https://github.com/nodejs/node/issues/66487"}