Title: utf-8-sig: UnicodeDecodeError offsets do not count the BOM · Issue #159071 · python/cpython · GitHub
Open Graph Title: utf-8-sig: UnicodeDecodeError offsets do not count the BOM · Issue #159071 · python/cpython
X Title: utf-8-sig: UnicodeDecodeError offsets do not count the BOM · Issue #159071 · python/cpython
Description: Bug report When the input starts with a BOM, the utf-8-sig codec reports UnicodeDecodeError offsets relative to the input without the BOM, i.e. error.start is off-by-3: data = b"\xef\xbb\xbf\xff" # Invalid byte at position 3 try: data.de...
Open Graph Description: Bug report When the input starts with a BOM, the utf-8-sig codec reports UnicodeDecodeError offsets relative to the input without the BOM, i.e. error.start is off-by-3: data = b"\xef\xbb\xbf\xff" #...
X Description: Bug report When the input starts with a BOM, the utf-8-sig codec reports UnicodeDecodeError offsets relative to the input without the BOM, i.e. error.start is off-by-3: data = b"\xef\xbb\xbf\x...
Opengraph URL: https://github.com/python/cpython/issues/159071
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"utf-8-sig: UnicodeDecodeError offsets do not count the BOM","articleBody":"### Bug report\n\nWhen the input starts with a BOM, the utf-8-sig codec reports `UnicodeDecodeError` offsets relative to the input without the BOM, i.e. `error.start` is off-by-3:\n\n```python\ndata = b\"\\xef\\xbb\\xbf\\xff\" # Invalid byte at position 3\ntry:\n data.decode(\"utf-8-sig\")\nexcept UnicodeDecodeError as e:\n print(e) # 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte\n```\n\nThe input is being sliced in `utf_8_sig.py` to remove the BOM, but raised exceptions are not modified to correct the offsets and object attributes. This occurs in a few places, e.g. https://github.com/python/cpython/blob/5a22a62b96a68c2dd31784c5e3427ad7b365388a/Lib/encodings/utf_8_sig.py#L18-L24\n\nThe utf-16 and utf-32 codecs also remove the BOM, but it is included in the offsets and object attributes in their errors, so this looks like an oversight.\n\n### CPython versions tested on:\n\n3.14, 3.15rc2\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-159081\n* gh-159082\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/alicederyn","@type":"Person","name":"alicederyn"},"datePublished":"2026-10-09T13:27:44Z","url":"https://github.com/python/cpython/issues/159071"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:424e01d9-50ca-513b-43b3-1ea311699590 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 8A20:AB8A2:4A61F7:66B329:6AFDE57C |
| html-safe-nonce | f1b440cc4155ad7ae2acf32160657e6f49dd4ed176eaef76c0b23b1020129f40 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4QTIwOkFCOEEyOjRBNjFGNzo2NkIzMjk6NkFGREU1N0MiLCJ2aXNpdG9yX2lkIjoiMzQxNzMzMjI0MTcxNjM0MDA5MiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | a67188837ff6731089eec380935e1ca7fddb95ad5905568da925cdda9a9a0f78 |
| hovercard-subject-tag | issue:5778883975 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/159071/issue_layout |
| twitter:image | https://opengraph.githubassets.com/7e8aa987d9a1a84d533dd2cbc3bd0fccfce4d7328ae43d720d45ecd053578322/python/cpython/issues/159071 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/7e8aa987d9a1a84d533dd2cbc3bd0fccfce4d7328ae43d720d45ecd053578322/python/cpython/issues/159071 |
| og:image:alt | Bug report When the input starts with a BOM, the utf-8-sig codec reports UnicodeDecodeError offsets relative to the input without the BOM, i.e. error.start is off-by-3: data = b"\xef\xbb\xbf\xff" #... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | alicederyn |
| hostname | github.com |
| expected-hostname | github.com |
| None | 91ef1a291a064ac31ac459ab3f75261dbdd924c41ea5bd027e591c5b46cc9ce7 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | d62bc98ce2346f4006e4a2810e9e22f37e7e6422 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width