René's URL Explorer Experiment


Title: Fix UnicodeDecodeError when reading packed-refs with non-UTF8 characters by MirrorDNA-Reflection-Protocol · Pull Request #2091 · gitpython-developers/GitPython · GitHub

Open Graph Title: Fix UnicodeDecodeError when reading packed-refs with non-UTF8 characters by MirrorDNA-Reflection-Protocol · Pull Request #2091 · gitpython-developers/GitPython

X Title: Fix UnicodeDecodeError when reading packed-refs with non-UTF8 characters by MirrorDNA-Reflection-Protocol · Pull Request #2091 · gitpython-developers/GitPython

Description: Summary Fixes #2064 The packed-refs file can contain ref names that are not valid UTF-8 (e.g., Latin-1 encoded tag names created by older Git versions or systems with different locale settings). Previously, GitPython would fail with UnicodeDecodeError when reading such files. Reproduction As described in #2064: git clone https://github.com/ACRA/acra cd acra python -c 'import git; print(git.Repo(".").tags)' Before fix: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 6216: invalid continuation byte After fix: Successfully reads all 101 tags. Changes Add errors='surrogateescape' to the open() call in _iter_packed_refs() This allows reading files with arbitrary byte sequences while preserving valid UTF-8 as text Add test that verifies non-UTF8 packed-refs can be read successfully Technical Details The surrogateescape error handler is Python's standard approach for handling potentially non-UTF8 data in filesystem operations. It: Passes through valid UTF-8 unchanged Converts invalid byte sequences to Unicode surrogate characters (\uDC80-\uDCFF) Preserves the original bytes in a reversible way (can be re-encoded back to original bytes) This is the same approach used by Python's os.fsdecode() and is recommended for filesystem operations where encoding may be unknown or mixed.

Open Graph Description: Summary Fixes #2064 The packed-refs file can contain ref names that are not valid UTF-8 (e.g., Latin-1 encoded tag names created by older Git versions or systems with different locale settings). Pr...

X Description: Summary Fixes #2064 The packed-refs file can contain ref names that are not valid UTF-8 (e.g., Latin-1 encoded tag names created by older Git versions or systems with different locale settings). Pr...

Opengraph URL: https://github.com/gitpython-developers/GitPython/pull/2091

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/checks(.:format)
route-controllerpull_requests
route-actionchecks
fetch-noncev2:19476aa2-bf43-dcdc-ff55-c045155e938b
current-catalog-service-hash87dc3bc62d9b466312751bfd5f889726f4f1337bdff4e8be7da7c93d6c00a25a
request-id92DC:46801:149170:1BEB83:69696513
html-safe-noncec00f2bc482dfd1fdeee34e215dab6e744a74cb7d6a4934b01b8b590ea869c6d7
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5MkRDOjQ2ODAxOjE0OTE3MDoxQkVCODM6Njk2OTY1MTMiLCJ2aXNpdG9yX2lkIjoiMzEzMTA2ODc1ODY5NjIyNDAxOSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac861f08a8c69f36f984dd52f1bd22f00631fed6fb7a5337179d6f1df2ddf395e9
hovercard-subject-tagpull_request:3079227966
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,checks,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/checks
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/gitpython-developers/GitPython/pull/2091/checks
twitter:imagehttps://avatars.githubusercontent.com/u/239564541?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/239564541?s=400&v=4
og:image:altSummary Fixes #2064 The packed-refs file can contain ref names that are not valid UTF-8 (e.g., Latin-1 encoded tag names created by older Git versions or systems with different locale settings). Pr...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None48487c1ad776a7975b7132d95f4240ff3ae37cd5b8e3cb597102a4edb76738f1
turbo-cache-controlno-cache
go-importgithub.com/gitpython-developers/GitPython git https://github.com/gitpython-developers/GitPython.git
octolytics-dimension-user_id503709
octolytics-dimension-user_logingitpython-developers
octolytics-dimension-repository_id1126087
octolytics-dimension-repository_nwogitpython-developers/GitPython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id1126087
octolytics-dimension-repository_network_root_nwogitpython-developers/GitPython
turbo-body-classeslogged-out env-production page-responsive full-width full-width-p-0
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release669463fcc54773a88c1f5a44eef6b99a5504b9c7
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/gitpython-developers/GitPython/pull/2091/checks#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgitpython-developers%2FGitPython%2Fpull%2F2091%2Fchecks
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgitpython-developers%2FGitPython%2Fpull%2F2091%2Fchecks
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Fchecks&source=header-repo&source_repo=gitpython-developers%2FGitPython
Reloadhttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Reloadhttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Reloadhttps://github.com/gitpython-developers/GitPython/pull/2091/checks
gitpython-developers https://github.com/gitpython-developers
GitPythonhttps://github.com/gitpython-developers/GitPython
Please reload this pagehttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Notifications https://github.com/login?return_to=%2Fgitpython-developers%2FGitPython
Fork 964 https://github.com/login?return_to=%2Fgitpython-developers%2FGitPython
Star 5k https://github.com/login?return_to=%2Fgitpython-developers%2FGitPython
Code https://github.com/gitpython-developers/GitPython
Issues 169 https://github.com/gitpython-developers/GitPython/issues
Pull requests 8 https://github.com/gitpython-developers/GitPython/pulls
Discussions https://github.com/gitpython-developers/GitPython/discussions
Actions https://github.com/gitpython-developers/GitPython/actions
Security Uh oh! There was an error while loading. Please reload this page. https://github.com/gitpython-developers/GitPython/security
Please reload this pagehttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Insights https://github.com/gitpython-developers/GitPython/pulse
Code https://github.com/gitpython-developers/GitPython
Issues https://github.com/gitpython-developers/GitPython/issues
Pull requests https://github.com/gitpython-developers/GitPython/pulls
Discussions https://github.com/gitpython-developers/GitPython/discussions
Actions https://github.com/gitpython-developers/GitPython/actions
Security https://github.com/gitpython-developers/GitPython/security
Insights https://github.com/gitpython-developers/GitPython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fgitpython-developers%2FGitPython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fgitpython-developers%2FGitPython%2Fissues%2Fnew%2Fchoose
MirrorDNA-Reflection-Protocolhttps://github.com/MirrorDNA-Reflection-Protocol
gitpython-developers:mainhttps://github.com/gitpython-developers/GitPython/tree/main
MirrorDNA-Reflection-Protocol:fix-packed-refs-encodinghttps://github.com/MirrorDNA-Reflection-Protocol/GitPython/tree/fix-packed-refs-encoding
Conversation 1 https://github.com/gitpython-developers/GitPython/pull/2091
Commits 3 https://github.com/gitpython-developers/GitPython/pull/2091/commits
Checks 27 https://github.com/gitpython-developers/GitPython/pull/2091/checks
Files changed https://github.com/gitpython-developers/GitPython/pull/2091/files
Please reload this pagehttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Please reload this pagehttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Sign in for the full log viewhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fgitpython-developers%2FGitPython%2Fpull%2F2091%2Fchecks
Fix UnicodeDecodeError when reading packed-refs with non-UTF8 characters https://github.com/gitpython-developers/GitPython/pull/2091/checks#top
Please reload this pagehttps://github.com/gitpython-developers/GitPython/pull/2091/checks
Python package on: pull_request https://github.com/gitpython-developers/GitPython/actions/runs/20022247963
test (ubuntu, 3.7) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468668812?pr=2091
test (ubuntu, 3.8) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468656995?pr=2091
test (ubuntu, 3.9) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468668795?pr=2091
test (ubuntu, 3.10) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657073?pr=2091
test (ubuntu, 3.11) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657143?pr=2091
test (ubuntu, 3.12) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468656808?pr=2091
test (ubuntu, 3.13) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657211?pr=2091
test (ubuntu, 3.13t) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657085?pr=2091
test (macos, 3.8) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657191?pr=2091
test (macos, 3.9) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657393?pr=2091
test (macos, 3.10) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657382?pr=2091
test (macos, 3.11) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657327?pr=2091
test (macos, 3.12) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657453?pr=2091
test (macos, 3.13) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657668?pr=2091
test (windows, 3.7) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657250?pr=2091
test (windows, 3.8) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657523?pr=2091
test (windows, 3.9) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657418?pr=2091
test (windows, 3.10) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657607?pr=2091
test (windows, 3.11) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657376?pr=2091
test (windows, 3.12) https://github.com/gitpython-developers/GitPython/actions/runs/20022247963/job/57468657434?pr=2091
Lint on: pull_request https://github.com/gitpython-developers/GitPython/actions/runs/20022247969
lint https://github.com/gitpython-developers/GitPython/actions/runs/20022247969/job/57411548625?pr=2091
test-alpine on: pull_request https://github.com/gitpython-developers/GitPython/actions/runs/20022247973
test https://github.com/gitpython-developers/GitPython/actions/runs/20022247973/job/57411548588?pr=2091
test-cygwin on: pull_request https://github.com/gitpython-developers/GitPython/actions/runs/20022247987
test (fast) https://github.com/gitpython-developers/GitPython/actions/runs/20022247987/job/57411548651?pr=2091
test (perf) https://github.com/gitpython-developers/GitPython/actions/runs/20022247987/job/57411548643?pr=2091
CodeQL on: pull_request https://github.com/gitpython-developers/GitPython/actions/runs/20022248000
Analyze (actions) https://github.com/gitpython-developers/GitPython/actions/runs/20022248000/job/57411548610?pr=2091
Analyze (python) https://github.com/gitpython-developers/GitPython/actions/runs/20022248000/job/57411548598?pr=2091
CodeQL https://github.com/gitpython-developers/GitPython/pull/2091/checks?check_run_id=57411647713
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.