René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Lexer outputs confused reminder when the terminal NUMBER prohibits trailing dot","articleBody":"version: v0.4.12\n\nAccording to the JSON specification (RFC 8259), trailing dot is prohibited for floating point.\nHowever, when I define the new JSON grammar like below, I found the lexer's suspicious behavior.\n\n```\n// RFC 8259 without complex STRING definition\n?start: value\n\n?value: object\n| array\n| STRING\n| NUMBER\n| \"true\"             -\u003e true\n| \"false\"            -\u003e false\n| \"null\"             -\u003e null\n\nobject: \"{\" [member (\",\" member)*] \"}\"\nmember: STRING \":\" value\narray : \"[\" [value (\",\" value)*] \"]\"\n\nNUMBER: MINUS? INT FRAC? EXP?\nMINUS: \"-\"\nINT: \"0\" | (\"1\"..\"9\") DIGIT*\nDIGIT: \"0\"..\"9\"\nFRAC: \".\" DIGIT+\nEXP: (\"e\"|\"E\") [\"+\"|\"-\"] DIGIT+\n\nSTRING: /\\\"[^\"]*\\\"/\nWS: /[ \\t\\f\\r\\n]/+\n\n%ignore WS\n```\n\nThe observed behavior:\n```python\n\u003e\u003e\u003e grammar_engine._parse_partial_code(0, '{ \"cap\": 10.0', b'', accepted_generation=True)\n(remainder : b'10.0', remainder_state: RemainderState.MAYBE_COMPLETE, accept_sequences: {accept_terminals: ['NUMBER', 'COMMA'], accept_terminals: ['NUMBER', 'WS', 'COMMA'], accept_terminals: ['LBRACE'], accept_terminals: ['WS'], accept_terminals: ['NULL'], accept_terminals: ['STRING'], accept_terminals: ['NUMBER', 'WS', 'RBRACE'], accept_terminals: ['NUMBER', 'RBRACE'], accept_terminals: ['TRUE'], accept_terminals: ['FALSE'], accept_terminals: ['LSQB']}, next_ac_indents: None, False)\n\n# ↑ This looks correct.\n\n\u003e\u003e\u003e grammar_engine._parse_partial_code(0, '{ \"cap\": 10.', b'', accepted_generation=True)\n(remainder : b'.', remainder_state: RemainderState.INCOMPLETE, accept_sequences: {accept_terminals: ['COMMA'], accept_terminals: ['WS'], accept_terminals: ['RBRACE']}, next_ac_indents: None, False)\n\n# ↑ This reminder must be '10.' ?\n```\n\nIt seems that the lexer will be confused when its state moves along accepted (digits) -\u003e live-state (trailing dot) -\u003e accepted (digits).","author":{"url":"https://github.com/Yosshi999","@type":"Person","name":"Yosshi999"},"datePublished":"2025-04-23T07:26:55.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/195/syncode/issues/195"}