René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Another case where output does not follow grammar","articleBody":"I did some more testing and found another case where the output did not match my defined grammar:\n\n```py\ngrammar = \"\"\"\nstart: compound_statement\n\ncompound_statement: statement | (compound_statement \";\" statement)\n\nstatement: def_var | use_var\n\ndef_var: DEF_VAR_NAME \" = \" VAR_VALUE\n\nDEF_VAR_NAME: VAR_NAME\n\nuse_var: \"use(\"i VAR_NAME \")\"\n\nVAR_NAME: /[a-zA-Z_][a-zA-Z0-9_]*/\n\nVAR_VALUE: /[a-zA-Z0-9_]+/\n\"\"\"\n\nmodel_name = \"microsoft/phi-2\"\nsyn_llm = Syncode(model=model_name, grammar=grammar, max_new_tokens=200, mode=\"grammar_strict\")\n\nprompt = \"repeat after me: a=1;b=2;c=3;use(a)\\n\"\nprint(syn_llm.infer(prompt, debug=True)[0])\n```\n\nresults in the output:\n\n```\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use(a)\na = 1;b = 2;c = 3;use\n```\n\n\nSo it just ignores that there needs to be a `;` after the `use(a)` and I'm not sure why.\n\n\nIs there something wrong with my grammar? or is this somehow related to #208?","author":{"url":"https://github.com/5ila5","@type":"Person","name":"5ila5"},"datePublished":"2025-06-12T11:33:19.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/211/syncode/issues/211"}