René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG] `JSONReaderASCII` decodes field names as UTF-8, corrupting `nameLength` — valid JSON crashes","articleBody":"**Version:** 2.0.64 (reproduced against the published jar)\n**Classes:** `JSONReaderASCII`, `JSONReaderUTF8`\n**Related:** #3928 (closed, fixed) — same defect, fixed for the UTF-8 reader only\n\nValid JSON throws `ArrayIndexOutOfBoundsException`:\n\n```java\nJSON.parse(\"[{\\\"\\\\t\\u00e0\\u00aa\\u00ae\\\":1}]\");   // 13 bytes: [{\"\\tમ\":1}]\n```\n\n```\njava.lang.ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2\n    at com.alibaba.fastjson2.JSONReaderASCII.getFieldName(JSONReaderASCII.java:876)\n    at com.alibaba.fastjson2.reader.ObjectReaderImplObject.readObject(ObjectReaderImplObject.java:169)\n    at com.alibaba.fastjson2.JSONReader.read(JSONReader.java:3281)\n    at com.alibaba.fastjson2.JSON.parse(JSON.java:138)\n```\n\nThree ingredients are needed, and removing any one makes it work:\n\n| input | result |\n|---|---|\n| `[{\"\\tમ\":1}]` | **AIOOBE** |\n| `[{\"\\tabc\":1}]` | parses — no bytes ≥ 0x80 |\n| `[{\"aમ\":1}]` | parses — no escape |\n| `{\"\\tમ\":1}` | parses — not inside an array |\n\nSo: an object **inside an array** (which routes through `ObjectReaderImplObject` →\n`readFieldNameHashCode()` + `getFieldName()`), a field name with a **backslash escape**,\nand a field name byte **≥ 0x80**.\n\n## Cause\n\n`JSONReaderASCII` treats the buffer as latin1 — one byte is one character — but it does\nnot define `readFieldNameHashCode0()`. It inherits `JSONReaderUTF8`'s, which decodes\nUTF-8 (`JSONReaderUTF8.java:2700-2722`):\n\n```java\nif (ch >= 0) {\n    offset++;\n} else {\n    switch (ch >> 4) {\n        case 12: case 13:\n            ch = char2_utf8(ch, bytes[offset + 1], offset);\n            offset += 2;            // 2 bytes -> 1 character\n            break;\n        case 14:\n            ch = char2_utf8(ch, bytes[offset + 1], bytes[offset + 2], offset);\n            offset += 3;            // 3 bytes -> 1 character\n            break;\n        default:\n            throw new JSONException(\"malformed input around byte \" + offset);\n    }\n}\n```\n\nThe counter that becomes `nameLength` (line 2694) therefore counts **characters**, while\nthe buffer is latin1. For the name `\\tમ`, the bytes `E0 AA AE` are taken as one 3-byte\ncharacter, so `nameLength = 2` for a name spanning 5 bytes.\n\n`JSONReaderASCII.getFieldName()` then walks the same bytes as latin1\n(`JSONReaderASCII.java:828-877`):\n\n```java\nbyte[] chars = new byte[nameLength];   // 2\nfor (int i = 0, end = this.end; offset \u003c nameEnd; ++i) {\n    ...\n    chars[i] = b;                      // line 876 -> index 2 into a length-2 array\n    offset++;\n}\n```\n\nInstrumenting the two calls shows the disagreement directly:\n\n```\nreadFieldNameHashCode -> nameBegin=3 nameEnd=8 nameLength=2 escape=true\ngetFieldName          -> ArrayIndexOutOfBoundsException: Index 2 out of bounds for length 2\n```\n\n`nameEnd - nameBegin` is 5 bytes; `nameLength` claims 2.\n\nThe escape is required because it forces this slow path — without a backslash,\n`JSONReaderASCII.readFieldNameHashCode()` (line 156) uses `name_len = index - start`, a\nbyte count, which is correct for latin1.\n\n## The same cause also rejects valid JSON\n\nWhen the high bytes do not form a valid UTF-8 sequence, the inherited decoder throws\nrather than miscounting. Identical documents are then accepted at top level and rejected\ninside an array:\n\n| document | top level | inside an array |\n|---|---|---|\n| `{\"\\tà\":1}` | parses | `JSONException: malformed input around byte 5` |\n| `{\"a\\tbé\":1}` | parses | `JSONException: malformed input around byte 7` |\n\n## This is the `JSONReaderASCII` half of #3928\n\n[#3928](https://github.com/alibaba/fastjson2/issues/3928) — *\"Exception on parsing object\nkey with emoji and escape character\"* — reported the same thing: a field name containing\n**an escape plus non-ASCII characters** breaks name-length accounting and throws\n`JSONException: malformed input around byte 2`. It was fixed by #3929 and closed on\n2026-02-07.\n\nThat fix works, but only on the UTF-8 path. On 2.0.64:\n\n```java\n// #3928's own test case -- now passes\nMap\u003cString,String> m = Map.of(\"\\uD83D\\uDE07\\\\\", \"\");\nJSON.parseObject(JSON.toJSONBytes(m, StandardCharsets.UTF_8), Map.class);   // OK\n\n// same shape, latin1-backed String -> JSONReaderASCII -- still broken\nJSON.parse(\"[{\\\"\\\\t\\u00e0\\u00aa\\u00ae\\\":1}]\");   // ArrayIndexOutOfBoundsException\nJSON.parse(\"[{\\\"\\\\t\\u00e0\\\":1}]\");             // JSONException: malformed input\n```\n\nThe reason is structural: `JSONReaderASCII` does not define `readFieldNameHashCode0()`.\nIt inherits `JSONReaderUTF8`'s, which decodes UTF-8, so the fix that taught that method\nabout 4-byte sequences never made the method correct *for a latin1 buffer*. Whatever it\ncounts, it is counting characters of a different encoding than the one\n`JSONReaderASCII.getFieldName()` then walks.\n\n## Fix\n\nA conservative fix for the crash — size the buffer from the byte span and trim to what\nwas written (escapes only ever shrink the output):\n\n```diff\n--- a/core/src/main/java/com/alibaba/fastjson2/JSONReaderASCII.java\n+++ b/core/src/main/java/com/alibaba/fastjson2/JSONReaderASCII.java\n@@ -825,9 +825,14 @@\n         if (JDKUtils.STRING_CREATOR_JDK11 != null) {\n-            byte[] chars = new byte[nameLength];\n+            byte[] chars = new byte[nameEnd - nameBegin];\n+            int i = 0;\n             forStmt:\n-            for (int i = 0, end = this.end; offset \u003c nameEnd; ++i) {\n+            for (int end = this.end; offset \u003c nameEnd; ++i) {\n@@ -878,7 +883,8 @@\n             if (chars != null) {\n-                return STRING_CREATOR_JDK11.apply(chars, LATIN1);\n+                return STRING_CREATOR_JDK11.apply(\n+                        i == chars.length ? chars : java.util.Arrays.copyOf(chars, i), LATIN1);\n             }\n```\n\nVerified against 2.0.64: `[{\"\\tમ\":1}]` now parses to `[{\"\\tમ\":1}]`, and these are\nunchanged — `[{\"\\tabc\":1}]`, `[{\"aમ\":1}]`, `{\"\\tમ\":1}`, `[{\"a\":1},{\"b\":2}]`,\n`[{\"\\u00e9\":1}]`, `[{\"a\\\\b\":1}]`, `[{\"a\\\"b\":1}]`, `[{\"\":1}]`, `[{\"\\n\\t\\r\":1}]`.\n\n**This fixes the crash only.** The \"malformed input\" rejections above still happen,\nbecause they come from the inherited UTF-8 decoder itself. The real fix is for\n`JSONReaderASCII` to stop using UTF-8 name decoding — either an override of\n`readFieldNameHashCode0()` for latin1, or a branch on the reader's encoding in the shared\ncode. That seemed like your call rather than something to force in a patch, so it is not\nincluded.\n\n## Not a security issue\n\nRecording this so it is not mistaken for one. The out-of-range access is a *write* index\n(`chars[i] = b`), but the JVM bounds check stops it — no memory is corrupted or\ndisclosed. We also checked whether the UTF-8/latin1 split could make a key hash as one\nname and stringify as another (which would allow binding to an unintended field): it does\nnot. A key of bytes `C3 A9` does not bind to a field named `é`, and the map key comes\nback as `é`. The mismatch corrupts a length, not an identity.\n\nThe practical impact is that valid JSON either throws an unchecked exception past a\ncaller's `catch (JSONException)`, or is rejected as malformed.\n\n---\nFound by the CISPA Fandango Team","author":{"url":"https://github.com/Namron2000","@type":"Person","name":"Namron2000"},"datePublished":"2026-08-24T16:16:00Z","url":"https://github.com/alibaba/fastjson2/issues/7808"}