René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Replace better-sqlite3 with sql.js for cross-platform compatibility","articleBody":"## Problem\n\nAfter fixing bcrypt → bcryptjs in #91, the next blocker for Android/Termux compatibility is `better-sqlite3`, which also requires native C++ compilation.\n\n### Current Error on Termux\n\n```\ngyp: Undefined variable android_ndk_path in binding.gyp\ngyp ERR! configure error \ngyp ERR! cwd .../node_modules/better-sqlite3\nnpm ERR! code 1\n```\n\n### Platforms Affected\n\nSame platforms as the bcrypt issue:\n- ✅ **Termux/Android** - Fails with `gyp: Undefined variable android_ndk_path`\n- ✅ **Windows** - Common node-gyp compilation errors\n- ✅ **ARM devices** - Various build issues\n- ✅ **Restricted environments** - No build tools available\n\n## Proposed Solution\n\n**Switch to sql.js as the primary SQLite implementation.**\n\n### Why This Makes Sense\n\n1. **Already a Dependency**\n   - `sql.js` is already in package.json (line 41)\n   - Just need to use it instead of better-sqlite3\n   - No new dependencies required\n\n2. **Performance Impact is Minimal**\n   \n   JavaScriptSolidServer uses SQLite for:\n   - Nostr relay event storage (when `--nostr` enabled)\n   - ActivityPub/federation data (when ActivityPub enabled)\n   - **NOT used for main pod storage** (that's filesystem-based)\n\n   Performance difference:\n   - better-sqlite3 (C++): ~0.1-0.5ms per query\n   - sql.js (WASM): ~0.2-1ms per query\n   - Difference: Sub-millisecond for infrequent federation operations\n\n3. **Real-World Usage**\n   - SQLite is only used for optional federation features\n   - Main pod operations (GET/PUT/POST/DELETE files) use the filesystem\n   - Federation queries happen rarely compared to file operations\n   - The performance difference is negligible\n\n4. **Cross-Platform Benefits**\n   - Works everywhere JavaScript runs\n   - No compilation step = faster npm install\n   - Better user experience\n\n## Implementation\n\nSimilar to the bcrypt fix:\n\n**Before:**\n```javascript\nimport Database from 'better-sqlite3';\nconst db = new Database('data.db');\n```\n\n**After:**\n```javascript\nimport initSqlJs from 'sql.js';\nconst SQL = await initSqlJs();\nconst db = new SQL.Database();\n```\n\nNeed to update:\n- `package.json` - Remove better-sqlite3, keep sql.js\n- `src/ap/store.js` - ActivityPub storage\n- `src/nostr/relay.js` - Nostr event storage\n- Any other files using better-sqlite3\n\n## Testing Required\n\nTest that federation features still work:\n- Nostr relay event storage\n- ActivityPub inbox/outbox operations\n- Database persistence (sql.js needs file I/O wrapper)\n\n## Note on Persistence\n\n`sql.js` runs in-memory by default. For persistence, we need to:\n1. Load database from file on startup\n2. Save database to file periodically or on writes\n3. Use `fs.readFileSync()` / `fs.writeFileSync()` wrapper\n\nThis is straightforward but needs careful implementation to avoid data loss.\n\n## Alternative Approach\n\nCould also use a try/catch fallback like we originally had for bcrypt:\n\n```javascript\nlet Database;\ntry {\n  Database = (await import('better-sqlite3')).default;\n} catch {\n  // Use sql.js fallback\n  const initSqlJs = (await import('sql.js')).default;\n  Database = await createSqlJsWrapper(initSqlJs);\n}\n```\n\nBut based on the bcrypt experience, it's simpler to just use sql.js everywhere.\n\n## Impact\n\nThis would enable:\n- 📱 Running full Solid+Nostr+ActivityPub server on Android/Termux\n- 💻 Easier installation on Windows\n- 🌍 Universal compatibility\n- ⚡ Faster npm install (no compilation)\n\nCombined with the bcrypt fix, this would make JavaScriptSolidServer truly \"just works\" on all platforms.\n\n## Related\n\n- Closes after bcrypt fix #91\n- Builds on the cross-platform compatibility work\n\n## Testing on Termux\n\nAfter bcrypt fix (#91), installation now fails on better-sqlite3:\n\n```bash\n$ npx javascript-solid-server@latest --version\n# ✅ bcrypt passes (using bcryptjs)\n# ❌ better-sqlite3 fails (native compilation)\n```\n\nWith this fix, the entire stack would work.","author":{"url":"https://github.com/melvincarvalho","@type":"Person","name":"melvincarvalho"},"datePublished":"2026-01-16T12:14:34.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/92/JavaScriptSolidServer/issues/92"}