René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Feature: remoteStorage Protocol Support","articleBody":"## Summary\n\nAdd remoteStorage protocol support to JavaScriptSolidServer, enabling the server to serve both Solid and remoteStorage clients from the same storage backend. This would make JSS the first server to support both major per-user storage protocols.\n\n**Difficulty**: 45/100  \n**Estimated Effort**: 4-6 days  \n**Dependencies**: None (can be implemented as optional plugin)\n\n---\n\n## What is remoteStorage?\n\n[remoteStorage](https://remotestorage.io/) is an open protocol for per-user storage on the web, predating Solid. It enables \"unhosted\" web apps where:\n- Users own their data\n- Apps request scoped access via OAuth\n- Data syncs across devices automatically\n\nThe protocol is defined in [IETF draft-dejong-remotestorage](https://datatracker.ietf.org/doc/draft-dejong-remotestorage/).\n\n---\n\n## Protocol Comparison\n\n| Feature | Solid (current) | remoteStorage | Compatibility |\n|---------|-----------------|---------------|---------------|\n| **Discovery** | WebFinger → WebID | WebFinger → storage URL | Different format, same endpoint |\n| **Authentication** | Solid-OIDC + DPoP | OAuth 2.0 Bearer (simpler) | RS is subset |\n| **HTTP Methods** | GET/PUT/DELETE/PATCH | GET/PUT/DELETE/HEAD | RS is subset |\n| **Directory Listings** | Turtle/JSON-LD (LDP vocab) | JSON-LD (simple format) | Transform needed |\n| **ETags** | ✅ Supported | ✅ Required | Already works |\n| **Conditional Requests** | If-Match, If-None-Match | If-Match, If-None-Match | Already works |\n| **CORS** | ✅ Enabled | ✅ Required | Already works |\n| **Access Control** | WAC (ACL files) | OAuth scopes (category:rw) | Different model |\n| **Path Structure** | `/{pod}/path/to/file` | `/storage/{user}/{module}/` | Different |\n| **Public Data** | ACL-based | `/public/` folder | Map to ACL |\n\n---\n\n## Why Integrate?\n\n1. **Larger ecosystem** - Access to remoteStorage apps (100+ apps listed)\n2. **Simpler onboarding** - RS OAuth is simpler than Solid-OIDC for some apps\n3. **Migration path** - RS users can migrate to Solid gradually\n4. **Unified storage** - One server, two protocols, same data\n5. **Unique positioning** - First server supporting both protocols\n\n---\n\n## Architecture\n\n### Proposed Plugin Structure\n\n```\nsrc/\n├── remotestorage/\n│   ├── index.js          # Plugin registration\n│   ├── webfinger.js      # RS WebFinger format\n│   ├── oauth.js          # OAuth 2.0 implicit flow\n│   ├── storage.js        # GET/PUT/DELETE handlers\n│   ├── listings.js       # Directory listing formatter\n│   └── scopes.js         # Scope validation \u0026 mapping\n```\n\n### Request Flow\n\n```\n┌─────────────────────────────────────────────────────────────┐\n│                      JSS Server                             │\n├─────────────────────────────────────────────────────────────┤\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │              WebFinger Handler                       │   │\n│  │         /.well-known/webfinger                       │   │\n│  │                                                      │   │\n│  │   ?resource=acct:alice@example.com                   │   │\n│  │         │                                            │   │\n│  │         ├─► rel=remotestorage → RS client            │   │\n│  │         └─► rel=solid → Solid client                 │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │           remoteStorage Routes                       │   │\n│  │                                                      │   │\n│  │   GET  /rs/oauth/authorize  → OAuth dialog          │   │\n│  │   POST /rs/oauth/token      → Token exchange        │   │\n│  │   GET  /storage/{user}/*    → Read file/folder      │   │\n│  │   PUT  /storage/{user}/*    → Write file            │   │\n│  │   DELETE /storage/{user}/*  → Delete file           │   │\n│  │   HEAD /storage/{user}/*    → Get metadata          │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │              Solid Routes (existing)                 │   │\n│  │                                                      │   │\n│  │   /{pod}/*  → Solid LDP + WAC                       │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n│  ┌─────────────────────────────────────────────────────┐   │\n│  │           Shared Storage Backend                     │   │\n│  │              (filesystem.js)                         │   │\n│  └─────────────────────────────────────────────────────┘   │\n│                                                             │\n└─────────────────────────────────────────────────────────────┘\n```\n\n---\n\n## Implementation Plan\n\n### Phase 1: WebFinger Extension (10/100)\n\nAdd remoteStorage link relation to existing WebFinger response.\n\n**Current response:**\n```json\n{\n  \"subject\": \"acct:alice@example.com\",\n  \"links\": [\n    {\n      \"rel\": \"http://webfinger.net/rel/profile-page\",\n      \"href\": \"https://example.com/alice/profile/card\"\n    }\n  ]\n}\n```\n\n**Extended response:**\n```json\n{\n  \"subject\": \"acct:alice@example.com\",\n  \"links\": [\n    {\n      \"rel\": \"http://webfinger.net/rel/profile-page\",\n      \"href\": \"https://example.com/alice/profile/card\"\n    },\n    {\n      \"rel\": \"http://tools.ietf.org/id/draft-dejong-remotestorage\",\n      \"href\": \"https://example.com/storage/alice\",\n      \"properties\": {\n        \"http://remotestorage.io/spec/version\": \"draft-dejong-remotestorage-26\",\n        \"http://tools.ietf.org/html/rfc6749#section-4.2\": \"https://example.com/rs/oauth/authorize\",\n        \"http://tools.ietf.org/html/rfc6750#section-2.3\": \"Bearer\"\n      }\n    }\n  ]\n}\n```\n\n---\n\n### Phase 2: OAuth 2.0 Implicit Flow (25/100)\n\nImplement simplified OAuth for remoteStorage clients.\n\n**Authorization endpoint:**\n```\nGET /rs/oauth/authorize\n  ?client_id=https://app.example.com\n  \u0026redirect_uri=https://app.example.com/callback\n  \u0026response_type=token\n  \u0026scope=photos:rw documents:r\n```\n\n**Authorization dialog:**\n```\n┌─────────────────────────────────────────────┐\n│  App \"MyPhotos\" is requesting access        │\n│                                             │\n│  ☑ photos - Read and write                  │\n│  ☑ documents - Read only                    │\n│                                             │\n│  [Deny]                    [Allow Access]   │\n└─────────────────────────────────────────────┘\n```\n\n**Successful redirect:**\n```\nhttps://app.example.com/callback#access_token=abc123\u0026token_type=bearer\n```\n\n**Scope format:**\n- `photos:rw` - Read/write access to /storage/{user}/photos/\n- `documents:r` - Read-only access to /storage/{user}/documents/\n- `*:rw` - Full access (root scope)\n\n---\n\n### Phase 3: Storage API (20/100)\n\nImplement remoteStorage HTTP API.\n\n**GET document:**\n```http\nGET /storage/alice/photos/vacation.jpg\nAuthorization: Bearer abc123\n\n200 OK\nContent-Type: image/jpeg\nETag: \"a1b2c3\"\nContent-Length: 12345\n\n\u003cbinary data\u003e\n```\n\n**GET folder:**\n```http\nGET /storage/alice/photos/\nAuthorization: Bearer abc123\n\n200 OK\nContent-Type: application/ld+json\nETag: \"folder-etag\"\n\n{\n  \"@context\": \"http://remotestorage.io/spec/folder-description\",\n  \"items\": {\n    \"vacation.jpg\": {\n      \"ETag\": \"a1b2c3\",\n      \"Content-Type\": \"image/jpeg\",\n      \"Content-Length\": 12345\n    },\n    \"summer/\": {\n      \"ETag\": \"d4e5f6\"\n    }\n  }\n}\n```\n\n**PUT document:**\n```http\nPUT /storage/alice/photos/new.jpg\nAuthorization: Bearer abc123\nContent-Type: image/jpeg\nIf-None-Match: *\n\n\u003cbinary data\u003e\n\n201 Created\nETag: \"new-etag\"\n```\n\n**DELETE document:**\n```http\nDELETE /storage/alice/photos/old.jpg\nAuthorization: Bearer abc123\nIf-Match: \"old-etag\"\n\n200 OK\nETag: \"new-folder-etag\"\n```\n\n---\n\n### Phase 4: Access Control (15/100)\n\nMap OAuth scopes to storage paths.\n\n```javascript\n// src/remotestorage/scopes.js\n\nfunction validateScope(token, path, method) {\n  // Parse path: /storage/alice/photos/vacation.jpg\n  const [, , user, category, ...rest] = path.split('/');\n  \n  // Check token scopes\n  const scope = token.scopes.find(s =\u003e {\n    const [cat, perm] = s.split(':');\n    return (cat === '*' || cat === category) \u0026\u0026\n           (perm === 'rw' || (perm === 'r' \u0026\u0026 method === 'GET'));\n  });\n  \n  if (!scope) {\n    return { allowed: false, error: 'Insufficient scope' };\n  }\n  \n  return { allowed: true };\n}\n```\n\n**Public folder handling:**\n- `/storage/{user}/public/*` - Readable without auth\n- Maps to ACL with `acl:agentClass foaf:Agent` in Solid terms\n\n---\n\n### Phase 5: Testing \u0026 Compliance (15/100)\n\nRun the official [RS API Test Suite](https://github.com/remotestorage/api-test-suite).\n\n```bash\n# Run compliance tests\nnpx rs-api-test-suite https://localhost:3000/storage/testuser\n```\n\n**Test categories:**\n- [ ] WebFinger discovery\n- [ ] OAuth authorization flow\n- [ ] Document CRUD operations\n- [ ] Folder listings\n- [ ] Conditional requests (ETags)\n- [ ] CORS headers\n- [ ] Public folder access\n\n---\n\n## Configuration\n\n```javascript\n// Proposed config additions\n{\n  \"remotestorage\": {\n    \"enabled\": true,\n    \"basePath\": \"/storage\",       // RS storage path prefix\n    \"oauthPath\": \"/rs/oauth\",     // OAuth endpoints\n    \"publicFolder\": \"public\",     // Public folder name\n    \"tokenExpiry\": 86400,         // Token lifetime (seconds)\n    \"allowedOrigins\": [\"*\"]       // CORS origins for RS\n  }\n}\n```\n\n**CLI flag:**\n```bash\njss start --remotestorage        # Enable RS support\njss start --no-remotestorage     # Disable (default?)\n```\n\n**Environment variable:**\n```bash\nJSS_REMOTESTORAGE=true\n```\n\n---\n\n## Storage Mapping\n\n### Option A: Shared Storage (Recommended)\n\nRS and Solid share the same filesystem:\n\n```\ndata/\n└── alice/\n    ├── profile/          # Solid WebID profile\n    ├── photos/           # Accessible via both protocols\n    │   └── vacation.jpg\n    ├── public/           # RS public folder = Solid public\n    └── .acl              # Solid WAC (RS uses OAuth scopes)\n```\n\n**Access patterns:**\n- Solid: `GET /alice/photos/vacation.jpg` (with Solid-OIDC)\n- RS: `GET /storage/alice/photos/vacation.jpg` (with Bearer token)\n\n### Option B: Isolated Storage\n\nSeparate storage areas:\n\n```\ndata/\n└── alice/\n    ├── solid/            # Solid-only data\n    └── rs/               # RS-only data\n```\n\n**Pros:** Clean separation, no ACL conflicts  \n**Cons:** Data duplication, users manage two silos\n\n**Recommendation:** Option A (shared) with clear documentation on access model differences.\n\n---\n\n## Scope ↔ WAC Mapping\n\n| RS Scope | Solid WAC Equivalent |\n|----------|---------------------|\n| `photos:r` | `acl:Read` on `/photos/` |\n| `photos:rw` | `acl:Read, acl:Write` on `/photos/` |\n| `*:r` | `acl:Read` on root |\n| `*:rw` | `acl:Read, acl:Write, acl:Control` on root |\n| (public folder) | `acl:agentClass foaf:Agent; acl:Read` |\n\n**Note:** RS scopes are simpler than WAC. RS tokens can only access paths within their scopes, regardless of ACL files.\n\n---\n\n## Compatibility Notes\n\n### remoteStorage Apps\n\nPopular RS apps that would work:\n- [Litewrite](https://litewrite.net/) - Distraction-free writing\n- [Groovebasin](https://groovebasin.com/) - Music player\n- [Sharesome](https://sharesome.5apps.com/) - File sharing\n- [Webmarks](https://nickvergessen.github.io/webmarks/) - Bookmark manager\n\n### remoteStorage.js Library\n\nThe official [remotestorage.js](https://github.com/remotestorage/remotestorage.js) client library should work out of the box once the protocol is implemented.\n\n```javascript\nconst rs = new RemoteStorage();\nrs.access.claim('photos', 'rw');\nrs.connect('alice@example.com');\n```\n\n---\n\n## Difficulty Breakdown\n\n| Component | Difficulty | Notes |\n|-----------|------------|-------|\n| WebFinger extension | 10/100 | Add link relation to existing handler |\n| OAuth implicit flow | 25/100 | Simpler than Solid-OIDC |\n| Storage GET/PUT/DELETE | 15/100 | Reuse existing storage backend |\n| Directory listings | 10/100 | JSON-LD transformation |\n| Scope validation | 15/100 | Path-based access check |\n| HEAD requests | 5/100 | Trivial |\n| Public folder | 10/100 | Skip auth for /public/ |\n| Compliance testing | 15/100 | RS test suite |\n| Documentation | 10/100 | Usage guide |\n| **Total** | **45/100** | |\n\n---\n\n## Open Questions\n\n1. **Default state:** Should RS be enabled by default or opt-in?\n2. **Storage mapping:** Shared storage (Option A) or isolated (Option B)?\n3. **Token storage:** Reuse IdP accounts DB or separate RS tokens?\n4. **Scope persistence:** Store granted scopes in accounts DB?\n5. **Revocation UI:** Add RS token management to future admin panel?\n\n---\n\n## References\n\n- [remoteStorage Protocol](https://remotestorage.io/protocol)\n- [IETF draft-dejong-remotestorage-26](https://datatracker.ietf.org/doc/draft-dejong-remotestorage/)\n- [remoteStorage Servers](https://remotestorage.io/servers.html)\n- [RS API Test Suite](https://github.com/remotestorage/api-test-suite)\n- [remotestorage.js Library](https://github.com/remotestorage/remotestorage.js)\n- [RS App List](https://remotestorage.io/apps/)\n\n---\n\n## Related Issues\n\n- #100 - Production Readiness (foundation for RS integration)\n- #95 - Web Onboarding (could add RS setup option)\n- #96 - Admin Panel (RS token management)\n- #101 - ETag Strategy (shared with RS)\n","author":{"url":"https://github.com/melvincarvalho","@type":"Person","name":"melvincarvalho"},"datePublished":"2026-01-25T13:40:32.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/106/JavaScriptSolidServer/issues/106"}