René's URL Explorer Experiment


Title: Fix writing to local filesystem by kevinjqliu · Pull Request #301 · apache/iceberg-python · GitHub

Open Graph Title: Fix writing to local filesystem by kevinjqliu · Pull Request #301 · apache/iceberg-python

X Title: Fix writing to local filesystem by kevinjqliu · Pull Request #301 · apache/iceberg-python

Description: PyIceberg. Contribute to apache/iceberg-python development by creating an account on GitHub.

Open Graph Description: Issue #299 This PR changes the behavior of both PyArrow and FsSpec file systems. When writing to the local file system, parent directories will be created first before writing to the file. Previous...

X Description: Issue #299 This PR changes the behavior of both PyArrow and FsSpec file systems. When writing to the local file system, parent directories will be created first before writing to the file. Previous...

Opengraph URL: https://github.com/apache/iceberg-python/pull/301

X: @github

direct link

Domain: github.com

route-pattern/_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format)
route-controllervoltron_pull_requests_fragments
route-actionpull_request_layout
fetch-noncev2:01b2209a-8cba-988b-2bc4-07706dc9e075
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idA90E:22605D:49B9C:5FCF4:6991D7F7
html-safe-noncecf4b5acfd388f948a11055c073d7ffe9c2219fd843f701b136cdde4d1690ce2a
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBOTBFOjIyNjA1RDo0OUI5Qzo1RkNGNDo2OTkxRDdGNyIsInZpc2l0b3JfaWQiOiIzMDU2OTc1MTg2OTUzODg1Njg3IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0=
visitor-hmac6831f17ba7085c82f1ee52ae146cace15f2241788344c70daf5a49c8807885ac
hovercard-subject-tagpull_request:1694192740
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/pull_requests_fragments/pull_request_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/apache/iceberg-python/301/pull_request_layout
twitter:imagehttps://opengraph.githubassets.com/e05b59ac2c3871062e9967d80ee4dc53b77b96e3a698af4b4dc06592a271cb58/apache/iceberg-python/pull/301
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/e05b59ac2c3871062e9967d80ee4dc53b77b96e3a698af4b4dc06592a271cb58/apache/iceberg-python/pull/301
og:image:altIssue #299 This PR changes the behavior of both PyArrow and FsSpec file systems. When writing to the local file system, parent directories will be created first before writing to the file. Previous...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamekevinjqliu
hostnamegithub.com
expected-hostnamegithub.com
None42c603b9d642c4a9065a51770f75e5e27132fef0e858607f5c9cb7e422831a7b
turbo-cache-controlno-cache
go-importgithub.com/apache/iceberg-python git https://github.com/apache/iceberg-python.git
octolytics-dimension-user_id47359
octolytics-dimension-user_loginapache
octolytics-dimension-repository_id677260606
octolytics-dimension-repository_nwoapache/iceberg-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id677260606
octolytics-dimension-repository_network_root_nwoapache/iceberg-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release848bc6032dcc93a9a7301dcc3f379a72ba13b96e
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/apache/iceberg-python/pull/301#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapache%2Ficeberg-python%2Fpull%2F301
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub SparkBuild and deploy intelligent appshttps://github.com/features/spark
GitHub ModelsManage and compare promptshttps://github.com/features/models
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
GitHub SponsorsFund open source developershttps://github.com/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/accelerator
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapache%2Ficeberg-python%2Fpull%2F301
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fpull_requests_fragments%2Fpull_request_layout&source=header-repo&source_repo=apache%2Ficeberg-python
Reloadhttps://github.com/apache/iceberg-python/pull/301
Reloadhttps://github.com/apache/iceberg-python/pull/301
Reloadhttps://github.com/apache/iceberg-python/pull/301
apache https://github.com/apache
iceberg-pythonhttps://github.com/apache/iceberg-python
Notifications https://github.com/login?return_to=%2Fapache%2Ficeberg-python
Fork 440 https://github.com/login?return_to=%2Fapache%2Ficeberg-python
Star 1k https://github.com/login?return_to=%2Fapache%2Ficeberg-python
Code https://github.com/apache/iceberg-python
Issues 116 https://github.com/apache/iceberg-python/issues
Pull requests 137 https://github.com/apache/iceberg-python/pulls
Actions https://github.com/apache/iceberg-python/actions
Projects 0 https://github.com/apache/iceberg-python/projects
Security 0 https://github.com/apache/iceberg-python/security
Insights https://github.com/apache/iceberg-python/pulse
Code https://github.com/apache/iceberg-python
Issues https://github.com/apache/iceberg-python/issues
Pull requests https://github.com/apache/iceberg-python/pulls
Actions https://github.com/apache/iceberg-python/actions
Projects https://github.com/apache/iceberg-python/projects
Security https://github.com/apache/iceberg-python/security
Insights https://github.com/apache/iceberg-python/pulse
Fokkohttps://github.com/Fokko
apache:mainhttps://github.com/apache/iceberg-python/tree/main
kevinjqliu:kevinjqliu/fix-write-local-fshttps://github.com/kevinjqliu/iceberg-python/tree/kevinjqliu/fix-write-local-fs
Conversationhttps://github.com/apache/iceberg-python/pull/301
Commits5 (5)https://github.com/apache/iceberg-python/pull/301/commits
Checkshttps://github.com/apache/iceberg-python/pull/301/checks
Files changedhttps://github.com/apache/iceberg-python/pull/301/files
Fix writing to local filesystemhttps://github.com/apache/iceberg-python/pull/301#top
Fokkohttps://github.com/Fokko
apache:mainhttps://github.com/apache/iceberg-python/tree/main
kevinjqliu:kevinjqliu/fix-write-local-fshttps://github.com/kevinjqliu/iceberg-python/tree/kevinjqliu/fix-write-local-fs
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
Jan 24, 2024https://github.com/apache/iceberg-python/pull/301#issue-2098812294
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
Issue #299https://github.com/apache/iceberg-python/issues/299
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/kevinjqliu
add a test for writing data to diskhttps://github.com/apache/iceberg-python/pull/301/commits/369df6e8141a2c44a2b575d14215602b60100003
369df6ehttps://github.com/apache/iceberg-python/pull/301/commits/369df6e8141a2c44a2b575d14215602b60100003
https://github.com/Fokko
Fokkohttps://github.com/Fokko
Jan 24, 2024 https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1842235594
View reviewed changes https://github.com/apache/iceberg-python/pull/301/files
Fokkohttps://github.com/Fokko
https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1842235594
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
pyiceberg/io/pyarrow.pyhttps://github.com/apache/iceberg-python/pull/301/files#diff-8d5e63f2a87ead8cebe2fd8ac5dcf2198d229f01e16bb9e06e21f7277c328abd
Fokkohttps://github.com/Fokko
Jan 24, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1465458297
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
kevinjqliuhttps://github.com/kevinjqliu
Jan 25, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1465718521
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@Fokkohttps://github.com/Fokko
LocalOutputFile implementationhttps://github.com/apache/iceberg/blob/fd1cf49280bde07d67c6bc1a6ec60238e1e38f7f/api/src/main/java/org/apache/iceberg/Files.java#L59
iceberg-python/pyiceberg/io/__init__.pyhttps://github.com/apache/iceberg-python/blob/4cf1f35dfd3e7cfb2996887e861d740239746306/pyiceberg/io/__init__.py#L278
4cf1f35https://github.com/apache/iceberg-python/commit/4cf1f35dfd3e7cfb2996887e861d740239746306
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
Fokkohttps://github.com/Fokko
Jan 25, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1466412423
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@kevinjqliuhttps://github.com/kevinjqliu
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
Fokkohttps://github.com/Fokko
Jan 25, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1466602467
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
kevinjqliuhttps://github.com/kevinjqliu
Jan 26, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1467137427
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@Fokkohttps://github.com/Fokko
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
force-pushedhttps://github.com/apache/iceberg-python/compare/752b63f2711665ef211abbe2a1c7b37f735914fa..1c480a53d3a29351d244067db03d5a6ffe4fa114
752b63fhttps://github.com/apache/iceberg-python/commit/752b63f2711665ef211abbe2a1c7b37f735914fa
1c480a5https://github.com/apache/iceberg-python/commit/1c480a53d3a29351d244067db03d5a6ffe4fa114
Compare https://github.com/apache/iceberg-python/compare/752b63f2711665ef211abbe2a1c7b37f735914fa..1c480a53d3a29351d244067db03d5a6ffe4fa114
January 26, 2024 01:20https://github.com/apache/iceberg-python/pull/301#event-11603923862
https://github.com/kevinjqliu
create dir firsthttps://github.com/apache/iceberg-python/pull/301/commits/5751b804c649bae08221058c27351ac3a13d79b8
5751b80https://github.com/apache/iceberg-python/pull/301/commits/5751b804c649bae08221058c27351ac3a13d79b8
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
force-pushedhttps://github.com/apache/iceberg-python/compare/5ea2ceb1512279d03637432d191c8613b5bcdf2b..5c777969d956efdeb15c40bb02fd672084c1c5e6
5ea2cebhttps://github.com/apache/iceberg-python/commit/5ea2ceb1512279d03637432d191c8613b5bcdf2b
5c77796https://github.com/apache/iceberg-python/commit/5c777969d956efdeb15c40bb02fd672084c1c5e6
Compare https://github.com/apache/iceberg-python/compare/5ea2ceb1512279d03637432d191c8613b5bcdf2b..5c777969d956efdeb15c40bb02fd672084c1c5e6
January 26, 2024 02:12https://github.com/apache/iceberg-python/pull/301#event-11604159136
kevinjqliuhttps://github.com/kevinjqliu
January 25, 2024 18:23https://github.com/apache/iceberg-python/pull/301#commits-pushed-4101dd1
https://github.com/kevinjqliu
add fsspec testhttps://github.com/apache/iceberg-python/pull/301/commits/4101dd11d35ae5e49a59719e7f64120fe892af20
4101dd1https://github.com/apache/iceberg-python/pull/301/commits/4101dd11d35ae5e49a59719e7f64120fe892af20
https://github.com/kevinjqliu
make linthttps://github.com/apache/iceberg-python/pull/301/commits/57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
57d68f2https://github.com/apache/iceberg-python/pull/301/commits/57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
force-pushedhttps://github.com/apache/iceberg-python/compare/5c777969d956efdeb15c40bb02fd672084c1c5e6..57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
5c77796https://github.com/apache/iceberg-python/commit/5c777969d956efdeb15c40bb02fd672084c1c5e6
57d68f2https://github.com/apache/iceberg-python/commit/57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
Compare https://github.com/apache/iceberg-python/compare/5c777969d956efdeb15c40bb02fd672084c1c5e6..57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
January 26, 2024 02:23https://github.com/apache/iceberg-python/pull/301#event-11604208287
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
Jan 26, 2024 https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1845000036
View reviewed changes https://github.com/apache/iceberg-python/pull/301/files/57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c
pyiceberg/io/fsspec.pyhttps://github.com/apache/iceberg-python/pull/301/files/57d68f2f936b1bcc8b2ec8dfa4291d7b9bab940c#diff-24c3aa912b523fdb2afba6a0ea2dfe69fdcd05d9268e1e13ac1023ac26b54ccc
kevinjqliuhttps://github.com/kevinjqliu
Jan 26, 2024https://github.com/apache/iceberg-python/pull/301#discussion_r1467169574
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
iceberg-python/pyiceberg/io/pyarrow.pyhttps://github.com/apache/iceberg-python/blob/8464d713d69e6e2f4fa6f629362a27c940a097e6/pyiceberg/io/pyarrow.py#L322
8464d71https://github.com/apache/iceberg-python/commit/8464d713d69e6e2f4fa6f629362a27c940a097e6
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
Fokkohttps://github.com/Fokko
January 26, 2024 02:35https://github.com/apache/iceberg-python/pull/301#event-11604263412
https://github.com/kevinjqliu
add fs testshttps://github.com/apache/iceberg-python/pull/301/commits/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
242e365https://github.com/apache/iceberg-python/pull/301/commits/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
https://github.com/Fokko
Fokkohttps://github.com/Fokko
Jan 29, 2024 https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1849396603
View reviewed changes https://github.com/apache/iceberg-python/pull/301/files/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
Fokkohttps://github.com/Fokko
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1849396603
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@kevinjqliuhttps://github.com/kevinjqliu
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/HonahX
HonahXhttps://github.com/HonahX
Jan 29, 2024 https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1849529245
View reviewed changes https://github.com/apache/iceberg-python/pull/301/files/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
HonahXhttps://github.com/HonahX
https://github.com/apache/iceberg-python/pull/301#pullrequestreview-1849529245
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
@kevinjqliuhttps://github.com/kevinjqliu
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/Fokko
Fokkohttps://github.com/Fokko
f66e365https://github.com/apache/iceberg-python/commit/f66e3652fdf9720d6c63a6fcec7bcd08d5bb186c
Jan 29, 2024https://github.com/apache/iceberg-python/pull/301#event-11632096959
https://github.com/Fokko
Fokkohttps://github.com/Fokko
PyIceberg 0.6.0 releasehttps://github.com/apache/iceberg-python/milestone/1
Jan 29, 2024https://github.com/apache/iceberg-python/pull/301#event-11632097658
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
January 30, 2024 02:34https://github.com/apache/iceberg-python/pull/301#event-11635645803
https://github.com/kevinjqliu
kevinjqliuhttps://github.com/kevinjqliu
Feb 24, 2024 https://github.com/apache/iceberg-python/pull/301#ref-pullrequest-2152462417
Cleanup conftest, remove LocalOutputFile #468 https://github.com/apache/iceberg-python/pull/468
https://github.co/hiddenchars
https://github.com/apache/iceberg-python/pull/{{ revealButtonHref }}
Sign up for freehttps://github.com/join?source=comment-repo
Sign in to commenthttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fapache%2Ficeberg-python%2Fpull%2F301
https://github.com/Fokko
Fokko https://github.com/Fokko
https://github.com/apache/iceberg-python/pull/301/files/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
https://github.com/HonahX
HonahX https://github.com/HonahX
https://github.com/apache/iceberg-python/pull/301/files/242e365e75c3fa7fe631e1ad76a72f7bc459eaab
PyIceberg 0.6.0 release https://github.com/apache/iceberg-python/milestone/1
Please reload this pagehttps://github.com/apache/iceberg-python/pull/301
https://github.com/kevinjqliu
https://github.com/Fokko
https://github.com/HonahX
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.