René's URL Explorer Experiment


Title: bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg by maxwellmckinnon · Pull Request #15326 · python/cpython · GitHub

Open Graph Title: bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg by maxwellmckinnon · Pull Request #15326 · python/cpython

X Title: bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg by maxwellmckinnon · Pull Request #15326 · python/cpython

Description: Important work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwise wonderful pathlib and PEP519. https://stackoverflow.com/questions/33625931/copy-file-with-pathlib-in-python. This adds the outstanding test request from that PR @vstinner (#5393). Test fails without the change, passes with it, along with every other test in test_shutil. Some variants were experimented with to make the one line change and the most performant one was picked. Added Test for PathLike directory destination, the current fail case Lib/test/test_shutil.py::TestMove::test_move_file_pathlike FAILED [100%] ============================================================== FAILURES =============================================================== __________________________________________________ TestMove.test_move_file_pathlike ___________________________________________________ self = def test_move_file_pathlike(self): # Move a file to another location on the same filesystem. src = pathlib.Path(self.src_file) > self._check_move_file(src, self.dst_dir, self.dst_file) Lib/test/test_shutil.py:1563: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ Lib/test/test_shutil.py:1545: in _check_move_file shutil.move(src, dst) /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:562: in move real_dst = os.path.join(dst, _basename(src)) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = PosixPath('/var/folders/r2/psq74t5x3nbfzlph8bh2pvdw0000gn/T/tmp9ie0wh9_/foo') def _basename(path): # A basename() variant which first strips the trailing slash, if present. # Thus we always get the last component of the path, even for directories. sep = os.path.sep + (os.path.altsep or '') > return os.path.basename(path.rstrip(sep)) E AttributeError: 'PosixPath' object has no attribute 'rstrip' /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/shutil.py:526: AttributeError ============================================== 1 failed, 102 deselected in 0.30 seconds =============================================== After change: ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items / 102 deselected / 1 selected Lib/test/test_shutil.py::TestMove::test_move_file_pathlike PASSED [100%] ============================================== 1 passed, 102 deselected in 0.06 seconds =============================================== Running all the tests in test_shutil.py ╰─ pytest Lib/test/test_shutil.py -v ========================================================= test session starts ========================================================= platform darwin -- Python 3.7.4, pytest-5.0.1, py-1.8.0, pluggy-0.12.0 -- /Users/maxwellmckinnon/.venvs/TA3.7/bin/python3.7 cachedir: .pytest_cache rootdir: /Users/maxwellmckinnon/dev/cpython plugins: cov-2.7.1, mock-1.10.4 collected 103 items Lib/test/test_shutil.py::TestShutil::test_chown PASSED [ 0%] Lib/test/test_shutil.py::TestShutil::test_copy PASSED [ 1%] ... Lib/test/test_shutil.py::TermsizeTests::test_stty_match SKIPPED [ 99%] Lib/test/test_shutil.py::PublicAPITests::test_module_all_attribute PASSED [100%] ================================================ 96 passed, 7 skipped in 1.25 seconds ================================================= Performance Considerations Is it considered poor form to get rid of _basename altogether and make use of pathlib in the move function? I'm not sure if the idea is for all these modules to strictly avoid circular dependencies. They are already using os.path which is just as much a citizen in 3.8 as pathlib right? e.g. real_dst = os.path.join(dst, _basename(src)) becomes real_dst = Path(dst) / Path(src).name I've looked around and familiarized myself, and I now think importing pathlib here is fine. My only remaining concern is that of performance. Here's the performance difference for this step. In [46]: %timeit real_dst = os.path.join("a/b/c", _basename('b/')) 2.71 µs ± 62.6 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) In [47]: %timeit real_dst = Path("a/b/c") / Path('b/').name 12.4 µs ± 65.3 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each) Is 10us significant or insignificant compared to the least expensive operation this function will do? I don't know. Let's find out. In [55]: %timeit os.rename('/tmp/a/a.txt', '/tmp/a/b.txt'); os.rename('/tmp/a/b.txt', '/tmp/a/a.txt') 124 µs ± 2.18 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each) 62us to rename. 10us seems significant enough that we wouldn't want to favor the Path sugar suggestion. 16% speed decrease from adding the 10us. What do people think? I was hoping to get to use pathlib.Path here, but I suspect for this low level move, it should be as fast as possible, and 16% is not worth one line of sugary code to me. https://bugs.python.org/issue32689 Automerge-Triggered-By: @gvanrossum

Open Graph Description: Important work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwi...

X Description: Important work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwi...

Opengraph URL: https://github.com/python/cpython/pull/15326

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:401cf7fa-2d76-fca3-b460-d5584fadb1f2
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idB61C:256537:2BEA1B:3C6C98:6B00B215
html-safe-noncedf2ee76d7c1c3a4b7bf818b12f1b147083c9c1d703051661677c995465287333
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNjFDOjI1NjUzNzoyQkVBMUI6M0M2Qzk4OjZCMDBCMjE1IiwidmlzaXRvcl9pZCI6IjY4MDg5OTQ5ODI1MjgwNzAxNjUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==
visitor-hmacc1b8376cc9f54305b497a4af93cf4e886c6b20c5aa732adba16cb0700a538d39
hovercard-subject-tagpull_request:308363292
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/15326/files
twitter:imagehttps://avatars.githubusercontent.com/u/5950299?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/5950299?s=400&v=4
og:image:altImportant work originally done by @emilyemorehouse two years ago and nearly ready to go in. This bug has affected many people and in some cases has been a dealbreaker to the adoption of the otherwi...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Nonec0dd5f5d6f515a591019ecd66df2c4eab3379c2d3b0777568a90d8c7ae10b79e
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release729d8cda69b3d267995675dfbf89ff27ccf61bf6
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/15326/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F15326%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F15326%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/15326/files
Reloadhttps://github.com/python/cpython/pull/15326/files
Reloadhttps://github.com/python/cpython/pull/15326/files
pythonhttps://github.com/python
cpythonhttps://github.com/python/cpython
Codehttps://github.com/python/cpython
Issues5k+ (5k+)https://github.com/python/cpython/issues
Pull requests2.4k (2.4k)https://github.com/python/cpython/pulls
Actionshttps://github.com/python/cpython/actions
Projectshttps://github.com/python/cpython/projects
Security and qualityhttps://github.com/python/cpython/security
Insightshttps://github.com/python/cpython/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
miss-islingtonhttps://github.com/miss-islington
python:masterhttps://github.com/python/cpython/tree/master
maxwellmckinnon:fix-issue-32689https://github.com/maxwellmckinnon/cpython/tree/fix-issue-32689
Conversation 16 https://github.com/python/cpython/pull/15326
Commits 16 https://github.com/python/cpython/pull/15326/commits
Checks 0 https://github.com/python/cpython/pull/15326/checks
Files changed https://github.com/python/cpython/pull/15326/files
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
bpo-32689: Updates shutil.move to allow for Path objects to be used as source arg https://github.com/python/cpython/pull/15326/files#top
Show all changes 16 commits https://github.com/python/cpython/pull/15326/files
cc5645d bpo-32689: Updates shutil.move function to allow for Path objects to … emilyemorehouse Jan 28, 2018 https://github.com/python/cpython/pull/15326/commits/cc5645d899e34936e5c67ddbef90ee56d3065ae3
61f1019 Adding news entry emilyemorehouse Feb 14, 2018 https://github.com/python/cpython/pull/15326/commits/61f1019d963343308f112751f6b96d0ad233c3fd
3f016ab Add test to show breakage for shutil.move where destination is a dire… Aug 18, 2019 https://github.com/python/cpython/pull/15326/commits/3f016ab1a04d78754b4b35cc686080aef252d2bc
a66757b Taking a guess at what the whitespace issue was. Aug 18, 2019 https://github.com/python/cpython/pull/15326/commits/a66757ba4bbcd15c8219cd7af824097177b19619
b071689 The results of running reindent.check on the file. Hope this fixes th… Aug 18, 2019 https://github.com/python/cpython/pull/15326/commits/b0716891b685da683aa084bc6b58b69431efd516
0c935fc remove whitespace maxwellmckinnon Aug 28, 2019 https://github.com/python/cpython/pull/15326/commits/0c935fc6d88a93bdc56c8bdcb85a61f4c4f92ad9
5380349 Update doc .rst files Aug 28, 2019 https://github.com/python/cpython/pull/15326/commits/538034961b9c367c744da4b183ba237dbde6bbb6
caac7fe Merge branch 'fix-issue-32689' of https://github.com/maxwellmckinnon/… Aug 28, 2019 https://github.com/python/cpython/pull/15326/commits/caac7fe8c71970901e1a356df4334257828203ae
090a1bd Merge branch 'master' into fix-issue-32689 maxwellmckinnon Aug 28, 2019 https://github.com/python/cpython/pull/15326/commits/090a1bdf5bdca9c3790dca0dbd2e9d5680a92c97
d1da9e3 Update shutil.py maxwellmckinnon Sep 5, 2019 https://github.com/python/cpython/pull/15326/commits/d1da9e3dc280b83081136da7e8f331d50cb4ad58
4777b4e Update shutil.rst maxwellmckinnon Sep 5, 2019 https://github.com/python/cpython/pull/15326/commits/4777b4eb8c6623cd68f75e16f32f6c0dc78d90ab
54c752f ran python reindent.py -v ~/dev/cpython/Lib/shutil.py to fix whitespa… Sep 11, 2019 https://github.com/python/cpython/pull/15326/commits/54c752f1ebc5f4cc547c036c8f1e882d3a3513ef
84832eb Update Misc/NEWS.d/next/Library/2018-02-13-17-58-30.bpo-32689.a-3SnH.rst maxwellmckinnon Sep 18, 2019 https://github.com/python/cpython/pull/15326/commits/84832ebc40f4352eaf7f856ee7dd22a77e4defc4
75f5d11 Update shutil.rst maxwellmckinnon Sep 18, 2019 https://github.com/python/cpython/pull/15326/commits/75f5d1177a90763155528fdb7038f84e27953985
821d510 Update test_shutil.py maxwellmckinnon Sep 18, 2019 https://github.com/python/cpython/pull/15326/commits/821d5103509088853855a6e6e54feb1e2270c0d5
446d971 Fixed another whitespace blocker I introduced via online editing -- s… Sep 19, 2019 https://github.com/python/cpython/pull/15326/commits/446d97153276f5979165d35e8e26c510cd6be138
Clear filters https://github.com/python/cpython/pull/15326/files
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
shutil.rst https://github.com/python/cpython/pull/15326/files#diff-f2e20586f33534d9cf4f058fffcedbe5cba1abf0f9e9813968846d7a8cccb719
shutil.py https://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
test_shutil.py https://github.com/python/cpython/pull/15326/files#diff-ed52359d591fff799c0b81414bdfd9a90541cbc3b97b50a7ebaca38ed98062e7
2018-02-13-17-58-30.bpo-32689.a-3SnH.rst https://github.com/python/cpython/pull/15326/files#diff-2b248e62eb21bdb91a39a0a417c582fee326011e389549faa40681f5677f8495
Doc/library/shutil.rsthttps://github.com/python/cpython/pull/15326/files#diff-f2e20586f33534d9cf4f058fffcedbe5cba1abf0f9e9813968846d7a8cccb719
View file https://github.com/maxwellmckinnon/cpython/blob/446d97153276f5979165d35e8e26c510cd6be138/Doc/library/shutil.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/15326/{{ revealButtonHref }}
https://github.com/python/cpython/pull/15326/files#diff-f2e20586f33534d9cf4f058fffcedbe5cba1abf0f9e9813968846d7a8cccb719
https://github.com/python/cpython/pull/15326/files#diff-f2e20586f33534d9cf4f058fffcedbe5cba1abf0f9e9813968846d7a8cccb719
Lib/shutil.pyhttps://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
View file https://github.com/maxwellmckinnon/cpython/blob/446d97153276f5979165d35e8e26c510cd6be138/Lib/shutil.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/15326/{{ revealButtonHref }}
https://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
https://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
https://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
https://github.com/python/cpython/pull/15326/files#diff-db8ac59326160713929e0e1973aef54f0280fe9f154ef24d14244909a0e0689b
Lib/test/test_shutil.pyhttps://github.com/python/cpython/pull/15326/files#diff-ed52359d591fff799c0b81414bdfd9a90541cbc3b97b50a7ebaca38ed98062e7
View file https://github.com/maxwellmckinnon/cpython/blob/446d97153276f5979165d35e8e26c510cd6be138/Lib/test/test_shutil.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/15326/{{ revealButtonHref }}
https://github.com/python/cpython/pull/15326/files#diff-ed52359d591fff799c0b81414bdfd9a90541cbc3b97b50a7ebaca38ed98062e7
Please reload this pagehttps://github.com/python/cpython/pull/15326/files
https://github.com/python/cpython/pull/15326/files#diff-ed52359d591fff799c0b81414bdfd9a90541cbc3b97b50a7ebaca38ed98062e7
Misc/NEWS.d/next/Library/2018-02-13-17-58-30.bpo-32689.a-3SnH.rsthttps://github.com/python/cpython/pull/15326/files#diff-2b248e62eb21bdb91a39a0a417c582fee326011e389549faa40681f5677f8495
View file https://github.com/maxwellmckinnon/cpython/blob/446d97153276f5979165d35e8e26c510cd6be138/Misc/NEWS.d/next/Library/2018-02-13-17-58-30.bpo-32689.a-3SnH.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/15326/{{ revealButtonHref }}
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.