René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Bytecode positions seem way too broad","articleBody":"(Note that `dis` currently has a [bug](https://github.com/python/cpython/pull/93663) in displaying accurate location info in the presence of `CACHE`s. The correct information can be observed by working with `co_positions` directly or using the code from that PR.)\r\n\r\nWhile developing `specialist`, I realized that there are lots of common code patterns that produce bytecode with unexpectedly large source ranges. In addition to being unhelpful for both friendly tracebacks (the original motivation) and things like bytecode introspection, I suspect these huge ranges may also be bloating the size of our internal position tables as well.\r\n\r\nConsider the following function:\r\n\r\n```py\r\ndef analyze(path):                         #  1\r\n    upper = lower = total = 0              #  2\r\n    with open(path) as file:               #  3\r\n        for line in file:                  #  4\r\n            for character in line:         #  5\r\n                if character.isupper():    #  6\r\n                    upper += 1             #  7\r\n                elif character.islower():  #  8\r\n                    lower += 1             #  9\r\n                total += 1                 # 10\r\n    return lower / total, upper / total    # 11\r\n\r\n\r\nimport dis\r\nfrom pprint import pprint as pp\r\ndef pos(p):\r\n    return (p.lineno, p.end_lineno, p.col_offset, p.end_col_offset)\r\n\r\npp([(pos(x.positions), x.opname, x.argval) for x in dis.get_instructions(analyze)])\r\n```\r\n\r\nThings that should probably span one line at most:\r\n- The first `GET_ITER`/`FOR_ITER` pair span all of lines 4 through 10.\r\n- The second `GET_ITER`/`FOR_ITER` pair spans all of lines 5 through 10.\r\n- The first `POP_JUMP_FORWARD_IF_FALSE` spans all of lines 6 through 9.\r\n- The second `POP_JUMP_FORWARD_IF_FALSE` spans all of lines 8 through 9.\r\n- Ten instructions for `with` cleanup each span all of lines 3 through 10.\r\n\r\nThings that should probably be artificial:\r\n- A `JUMP_FORWARD` spans all of line 7.\r\n- The first `JUMP_BACKWARD` spans all of line 10.\r\n- The second `JUMP_BACKWARD` spans all of lines 5 through 10.\r\n\r\nThings I don't get:\r\n- A `NOP` spans all of lines 4 through 10.\r\n\r\nAs a result, over half of the generated bytecode for this function claims to span line 9, for instance. Also not shown here: the instructions for building functions and classes have similarly huge spans.\r\n\r\nI think this can be tightened up in the compiler by:\r\n- Being more aggressive in calling `SET_LOC` on child nodes.\r\n- Being more aggressive in calling `UNSET_LOC` before unconditional jumps.\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-120125\n* gh-120330\n* gh-120399\n* gh-120405\n* gh-123604\n* gh-123605\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/brandtbucher","@type":"Person","name":"brandtbucher"},"datePublished":"2022-06-10T18:12:47.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/93691/cpython/issues/93691"}