René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Enable definition for generated interpreter cases to be composed from multiple files","articleBody":"# Feature or enhancement\r\n\r\nAllow specifying multiple input files to the `generate_cases.py` script, making it behave mostly as if the input files were concatenated. Additionally allow existing definitions of instructions to be explicitly overridden using a new `override` keyword.\r\n\r\nI attach a PR with a proposed initial implementation.\r\n\r\n# Pitch\r\n\r\nIn [Cinder](https://github.com/facebookincubator/cinder) we add a number of new instructions to support our features like Static Python. We also currently have a few tweaks to existing instructions. When we migrate to the new upstream generated interpreter it would be preferable if we could avoid having to make changes to the core `bytecodes.c` and keep our own definitions/changes separate. As well as easing upstream merges, this would also avoid us having to copy/fork more than we need for Cinder features in a standalone module.\r\n\r\nI've made an initial implementation which allows extra files to be passed to `generate_cases.py` by repeated use of the `-i` argument. E.g.:\r\n```\r\n$ generate_cases.py -i bytecodes.c -i cinder-bytecodes.c -o generated_cases.c.h\r\n```\r\n\r\nThis mostly behaves as if the input files are concatenated but parsing only takes place between the `BEGIN/END BYTECODES` markers in each file. We also take advantage of mostly existing book-keeping features to track which input files definitions come from when producing errors.\r\n\r\nI've also added a new `override` keyword which can prefix instruction definitions to explicitly express the intent to override an existing definition. E.g.:\r\n\r\n```\r\ninst(NOP, (--)) {\r\n}\r\n\r\n// This is the definition which ends up being used in generation.\r\noverride inst(NOP, (--)) {\r\n  magic();\r\n}\r\n\r\n// Error - previous definition of NOP exists and \"override\" not specified.\r\ninst(NOP, (--)) {\r\n}\r\n\r\n// Error - requested override but no previous definition of ZOP exists.\r\noverride inst(ZOP, (--)) {\r\n}\r\n```\r\n\r\nThe goal of explicitly calling out overrides is to quickly reveal if either: something we modify is removed from upstream, or if a new opcode we add ends up with a name clash with a new upstream opcode.\r\n\r\n# Previous discussion\r\n\r\nThe idea of having multiple input files for interpreter generation was briefly [discussed around the faster-cpython project](https://github.com/faster-cpython/ideas/issues/479#issuecomment-1279598946).\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-102022\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/jbower-fb","@type":"Person","name":"jbower-fb"},"datePublished":"2023-02-18T04:14:47.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/102021/cpython/issues/102021"}