René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"sqlite3.iterdump() incompatible with binary data","articleBody":"# Bug report\n\n### Checklist\n\n- [X] I am confident this is a bug in CPython, not a bug in a third-party project\n- [X] I have searched the [CPython issue tracker](https://github.com/python/cpython/issues?q=is%3Aissue+sort%3Acreated-desc),\nand am confident this bug has not been reported before\n\n\n### CPython versions tested on:\n\n3.11\n\n### Operating systems tested on:\n\nLinux\n\n### Output from running 'python -VV' on the command line:\n\nPython 3.11.5 (main, Aug 25 2023, 13:19:53) [GCC 9.4.0]\n\n### A clear and concise description of the bug:\n\nApologies if I'm misunderstanding. Please advice if I should post elsewhere. But shouldn't iterdump() properly detect VARCHAR columns with binary data and output X'' strings instead of throwing an error? This is what `sqlite3 .dump` does.\n\n```python\nimport sqlite3\nwith sqlite3.connect(db_path) as conn:\n    with open(dump_path, 'w') as dump:\n        for line in conn.iterdump():\n            pass\n```\n\nThe above will throw an error:\n```\n  File \"foo.py\", line 79, in dump_sqlite_db\n    for line in conn.iterdump():\n  File \"/usr/lib/python3.11/sqlite3/dump.py\", line 63, in _iterdump\n    for row in query_res:\nsqlite3.OperationalError: Could not decode to UTF-8 column ''INSERT INTO \"sync_entities_metadata\" VALUES('||quote(\"storage_key\")||','||quote(\"metadata\")||')'' with text 'INSERT INTO \"sync_entities_metadata\" VALUES(1,'v10����\n```\n\nI tried enabling `conn.text_factory = bytes` as a workaround, but now get a different error.\n```\n  File \"foo.py\", line 79, in dump_sqlite_db\n    for line in conn.iterdump():\n  File \"/usr/lib/python3.11/sqlite3/dump.py\", line 43, in _iterdump\n    elif table_name.startswith('sqlite_'):\n         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nTypeError: startswith first arg must be bytes or a tuple of bytes, not str\n```\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-108657\n* gh-108673\n* gh-108674\n* gh-108683\n* gh-108686\n* gh-108694\n* gh-108695\n* gh-108699\n* gh-111324\n* gh-111325\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/dotysan","@type":"Person","name":"dotysan"},"datePublished":"2023-08-28T21:58:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":32},"url":"https://github.com/108590/cpython/issues/108590"}