René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Issues with built-in python, java, and go grammars","articleBody":"I am experiencing issues with the built-in grammars for Java, Python, and Go. The Python and Java grammars appear to be ignored, while the Go grammar produces mostly gibberish. Below is a script to reproduce the issue, along with example outputs. I installed with `pip install git+https://github.com/uiuc-focal-lab/syncode.git`.  syncode version = 0.1 .\r\n\r\n```python\r\nimport torch\r\nfrom syncode import SyncodeLogitsProcessor\r\nfrom syncode import Grammar\r\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\r\n\r\ndevice = 'cuda'\r\n# model_name = \"meta-llama/Llama-3.2-1B-Instruct\"\r\nmodel_name = \"meta-llama/Llama-3.1-8B-Instruct\"\r\ncache_dir = None\r\n\r\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.bfloat16, cache_dir=cache_dir).eval().to(device)\r\ntokenizer = AutoTokenizer.from_pretrained(model_name)\r\n\r\n# Initialize SynCode logits processor for the given grammar\r\n\r\n# grammar_str = \"\"\" start: month \" \" day \r\n              \r\n#               day: /[1-9]/ | /[1-2][0-9]/ | /3[0-1]/\r\n              \r\n#               month: \"January\" | \"February\" | \"March\" | \"April\" | \"May\" | \"June\" | \"July\" | \"August\" | \"September\" | \"October\" | \"November\" | \"December\"\r\n# \"\"\"\r\ngrammar_str = \"python\"\r\n# grammar_str = \"go\"\r\n# grammar_str = \"java\"\r\n\r\ndate_grammar = Grammar(grammar_str)\r\nsyncode_logits_processor = SyncodeLogitsProcessor(grammar=date_grammar, tokenizer=tokenizer, parse_output_only=True)\r\n\r\nprompt = f\"Write a {grammar_str} function that prints 'hello world' in reverse.\"\r\nmessages = [{\"role\": \"user\", \"content\": prompt}]\r\nprompt = tokenizer.apply_chat_template(\r\n                  messages, tokenize=False, add_generation_prompt=True\r\n            )\r\nprint(\"[PROMPT]\", prompt, \"\\n\")\r\n\r\nsyncode_logits_processor.reset(prompt)\r\n\r\ninputs = tokenizer(prompt, return_tensors='pt').input_ids.to(device)\r\n\r\nattention_mask = torch.ones_like(inputs)\r\noutput = model.generate(\r\n      inputs,\r\n      attention_mask=attention_mask,\r\n      max_length=512, \r\n      num_return_sequences=1, \r\n      pad_token_id=tokenizer.eos_token_id, \r\n      logits_processor=[syncode_logits_processor]\r\n      )\r\noutput_str = tokenizer.decode(output[0][len(inputs[0]):], skip_special_tokens=True)\r\nprint(\"[OUTPUT]\", output_str)\r\n```\r\n\r\nPython\r\n````\r\n[PROMPT] \u003c|begin_of_text|\u003e\u003c|start_header_id|\u003esystem\u003c|end_header_id|\u003e\r\n\r\nCutting Knowledge Date: December 2023\r\nToday Date: 26 Jul 2024\r\n\r\n\u003c|eot_id|\u003e\u003c|start_header_id|\u003euser\u003c|end_header_id|\u003e\r\n\r\nWrite a python function that prints 'hello world' in reverse.\u003c|eot_id|\u003e\u003c|start_header_id|\u003eassistant\u003c|end_header_id|\u003e\r\n\r\n \r\n\r\n[OUTPUT] **Reversing 'Hello World' Function**\r\n=====================================\r\n\r\nHere is a simple Python function that prints 'hello world' in reverse:\r\n\r\n```python\r\ndef print_reverse_hello_world():\r\n    \"\"\"\r\n    Prints 'hello world' in reverse.\r\n    \"\"\"\r\n    message = \"hello world\"\r\n    reversed_message = message[::-1]\r\n    print(reversed_message)\r\n\r\nprint_reverse_hello_world()\r\n```\r\n\r\n**\r\n````\r\n\r\nJava\r\n````\r\n[PROMPT] \u003c|begin_of_text|\u003e\u003c|start_header_id|\u003esystem\u003c|end_header_id|\u003e\r\n\r\nCutting Knowledge Date: December 2023\r\nToday Date: 26 Jul 2024\r\n\r\n\u003c|eot_id|\u003e\u003c|start_header_id|\u003euser\u003c|end_header_id|\u003e\r\n\r\nWrite a java function that prints 'hello world' in reverse.\u003c|eot_id|\u003e\u003c|start_header_id|\u003eassistant\u003c|end_header_id|\u003e\r\n\r\n \r\n\r\n[OUTPUT] interface\r\nHere is a simple Java function that prints 'hello world' in reverse:\r\n\r\n```java\r\npublic class HelloWorld {\r\n    public static void main(String[] args) {\r\n        System.out.println(\"Hello World\");\r\n    }\r\n}\r\n```\r\n\r\nExplanation:\r\n\r\n- The `System.out.println()` function is used to print the string \"Hello World\" to the console.\r\n- The `public static void\r\n````\r\n\r\nGo\r\n````\r\n[PROMPT] \u003c|begin_of_text|\u003e\u003c|start_header_id|\u003esystem\u003c|end_header_id|\u003e\r\n\r\nCutting Knowledge Date: December 2023\r\nToday Date: 26 Jul 2024\r\n\r\n\u003c|eot_id|\u003e\u003c|start_header_id|\u003euser\u003c|end_header_id|\u003e\r\n\r\nWrite a go function that prints 'hello world' in reverse.\u003c|eot_id|\u003e\u003c|start_header_id|\u003eassistant\u003c|end_header_id|\u003e\r\n\r\n \r\n\r\n[OUTPUT] \\ \r\n\\\r\n\r\n\\\r\n\r\n\\\r\n...\r\n````","author":{"url":"https://github.com/ivnle","@type":"Person","name":"ivnle"},"datePublished":"2024-12-19T22:44:42.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/137/syncode/issues/137"}