René's URL Explorer Experiment


Title: Python codegen support by robholt · Pull Request #923 · sqlc-dev/sqlc · GitHub

Open Graph Title: Python codegen support by robholt · Pull Request #923 · sqlc-dev/sqlc

X Title: Python codegen support by robholt · Pull Request #923 · sqlc-dev/sqlc

Description: I've come to really like using sqlc for Go projects and wanted something similar for Python. There's a few existing Python libraries that load queries from sql files, but they lack the type information, models, and query validation that sqlc can do. Since sqlc already has support for multiple languages I figured adding Python support should be fairly straight forward, which turned out to be accurate. So basically this PR adds support for generating fully type-hinted Python code. I modeled my changes after the existing Kotlin support, although I think it ended up looking at lot more like the Go codegen. There's a bunch of code duplication between the Python and Go codegen, but I think that's unavoidable until some kind of general codegen interface can be developed. I wanted to keep my changes limited to just adding Python support. The generated code relies on a small runtime library: sqlc-python-runtime. This keeps the generated code simple and allows for one function to support synchronous and asynchronous executors. What's working: postgresql All the projects under examples/ (including tests I've added for them) and the features they use Dates Json fields Type overrides What doesn't work: mysql (I focused on postgresql because the project I'm using this on is postgresql only, but mysql support could definitely be added) json/db tag options (not a thing for Python) The emit_prepared_queries option (although the asyncpg executor automatically prepares and caches queries) As far as additional testing goes, I've started using this in an existing project with a couple dozen queries against a moderately complicated schema and it's been great so far. What's different: Like JDBC in Kotlin/Java, psycopg2 doesn't support numbered parameters. However asyncpg does, so the regex hack the Kotlin codegen uses is applied at runtime for only psycopg2. I'm planning on investigating psycopg2s named parameters as another potential solution. The generated query functions aren't part of a class or anything, so their first parameter is the database connection object. This made multiple python query files (instead of just one big one) and the sync/async support much simpler. Thanks again for sqlc! Hopefully this is something you're interested in for sqlc.

Open Graph Description: I've come to really like using sqlc for Go projects and wanted something similar for Python. There's a few existing Python libraries that load queries from sql files, but they lack the type...

X Description: I've come to really like using sqlc for Go projects and wanted something similar for Python. There's a few existing Python libraries that load queries from sql files, but they lack ...

Opengraph URL: https://github.com/sqlc-dev/sqlc/pull/923

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:8c626162-fe56-fdf0-266b-955b7f06151c
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idB436:2E9831:12A53A1:18ACE3A:6B01EB09
html-safe-nonce9f1b9b70e1fe029619eb5ab48c098a11154417f5bbb4ed91f4247c848256b5da
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCNDM2OjJFOTgzMToxMkE1M0ExOjE4QUNFM0E6NkIwMUVCMDkiLCJ2aXNpdG9yX2lkIjoiNjA3MjU1MTg4Njc2OTIyMDM2MSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmacfbe4ec4746f3b23c879d5e7d4073626faa20c5da1f7afde0ea2bb6454ff88bd4
hovercard-subject-tagpull_request:584277513
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/sqlc-dev/sqlc/pull/923/files
twitter:imagehttps://avatars.githubusercontent.com/u/794548?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/794548?s=400&v=4
og:image:altI've come to really like using sqlc for Go projects and wanted something similar for Python. There's a few existing Python libraries that load queries from sql files, but they lack the type...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/sqlc-dev/sqlc git https://github.com/sqlc-dev/sqlc.git
octolytics-dimension-user_id136738596
octolytics-dimension-user_loginsqlc-dev
octolytics-dimension-repository_id193160679
octolytics-dimension-repository_nwosqlc-dev/sqlc
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id193160679
octolytics-dimension-repository_network_root_nwosqlc-dev/sqlc
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/sqlc-dev/sqlc/pull/923/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsqlc-dev%2Fsqlc%2Fpull%2F923%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fsqlc-dev%2Fsqlc%2Fpull%2F923%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=sqlc-dev%2Fsqlc
Reloadhttps://github.com/sqlc-dev/sqlc/pull/923/files
Reloadhttps://github.com/sqlc-dev/sqlc/pull/923/files
Reloadhttps://github.com/sqlc-dev/sqlc/pull/923/files
sqlc-devhttps://github.com/sqlc-dev
sqlchttps://github.com/sqlc-dev/sqlc
Codehttps://github.com/sqlc-dev/sqlc
Issues600 (600)https://github.com/sqlc-dev/sqlc/issues
Pull requests132 (132)https://github.com/sqlc-dev/sqlc/pulls
Discussionshttps://github.com/sqlc-dev/sqlc/discussions
Actionshttps://github.com/sqlc-dev/sqlc/actions
Projectshttps://github.com/sqlc-dev/sqlc/projects
Security and qualityhttps://github.com/sqlc-dev/sqlc/security
Insightshttps://github.com/sqlc-dev/sqlc/insights
Sign up for GitHub https://github.com/signup?return_to=%2Fsqlc-dev%2Fsqlc%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fsqlc-dev%2Fsqlc%2Fissues%2Fnew%2Fchoose
kyleconroyhttps://github.com/kyleconroy
sqlc-dev:masterhttps://github.com/sqlc-dev/sqlc/tree/master
robholt:python-supporthttps://github.com/robholt/sqlc/tree/python-support
Conversation 13 https://github.com/sqlc-dev/sqlc/pull/923
Commits 2 https://github.com/sqlc-dev/sqlc/pull/923/commits
Checks 0 https://github.com/sqlc-dev/sqlc/pull/923/checks
Files changed https://github.com/sqlc-dev/sqlc/pull/923/files
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
Python codegen support https://github.com/sqlc-dev/sqlc/pull/923/files#top
Show all changes 2 commits https://github.com/sqlc-dev/sqlc/pull/923/files
8956e2a Initial python codegen support Jan 29, 2021 https://github.com/sqlc-dev/sqlc/pull/923/commits/8956e2aef9a46337e21935e8dd787348bdf58e7d
0a382c3 Add python tests CI job Mar 3, 2021 https://github.com/sqlc-dev/sqlc/pull/923/commits/0a382c3cb9a634e9f4929715849e98261533854d
Clear filters https://github.com/sqlc-dev/sqlc/pull/923/files
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
ci-python.yml https://github.com/sqlc-dev/sqlc/pull/923/files#diff-a7ec86144013d78c9f1d734aa65a898e66b609638aa7b872491ee3858ebebbeb
requirements.txt https://github.com/sqlc-dev/sqlc/pull/923/files#diff-c233c301c314607d2dfcee9a9ec0d5a7b78a67d2c8927808fd4c9d382bbb4a8e
sqlc.json https://github.com/sqlc-dev/sqlc/pull/923/files#diff-51ce5461fb9e627abb72ef91a05f41da8a7272d3d6d9101fe941b3b5ed7d99d9
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-86a97e55c4d2e8852a049aeae4633ceb4bf71e28ff76b2c13fd1f4cb5329ef6f
models.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-ee5ac580ad63bf714e44b9be5731416a492cf541f81038ad93aa944ad974a4c4
query.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-c89f8979af02e775ccddfaba91d614eddb3bab71498afba34cc43d6464cffad9
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-061db2e5fb7334d64138a2fe43043fb683e59518a3502c3e49e21c794cd3b343
models.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-96909b321988c2a971e29eaa178d76f6c13550c628fc9238525934947d95c7bf
query.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-ecc31cbe8cf7dd3168f2145777c647e51a41fed5f009a2ca5531136b75d3513c
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-cc2d8e57070e7a0eebb0533653b0498f66d8388446d39b78193dff15e072c583
migrations.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-085a2592d365b45529f54567683e6f86bcfcaacf6861c444c1f1587f6198e803
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-35ae09da843892ce84097110ba3b64f159384acfe95a99275925aab75bb847ac
models.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-bb527e6d8bb3ab4658d672ea3f63e9c5c583c927cc70cab2ec1066d5c2364e4a
query-building.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-509b3335e4fbca4d213186c60be3d59e4f1d53846d86dd576357315b09e8ce08
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-979f914d60011ac638919ea5621436e0c9a5c9cc42a8c62cf88c50f1d7094913
city.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-bdad798e6b8e85201714a8192ac42845523dbbb5a5a017d6b739ef66e5839fc1
models.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-4fe6296bd7df312e72842738a7944d256832948fbb1df1adeaab3a9391fc3eee
venue.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-40122bbc61cbbfbffce8e3eb9733a7701e4c3cb675a4190f93b76172ee6dc03c
__init__.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-7b97127e53bae5326e09568785799df53eb12d0c7b603d14394103be1260b8f7
conftest.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-6fb414c226d7f3b94d6748e88dbd6f5839d7843cd1b7c6e75ced2f014bc22b89
test_authors.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-25cbb374c04950655a28cbc365e7abb6c80108da5c564057c41f8bad903a84c3
test_booktest.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-2f6d7f5b3991bc5a950ba891a64ef574e2be28ff5506b892fc3a36bc355e0f89
test_ondeck.py https://github.com/sqlc-dev/sqlc/pull/923/files#diff-9472850ce0e42b3e9a2c736765bc700caeff7cae221c83866f7337d41d743e86
generate.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-2ccbd26a302b29c524a67aa623a57582264d19b24a40646065be500f2e15bbd3
go_type.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-9febf00618188ee666b7f8cc8382cd90e83a745379afbf307f106cdc6da10d9e
imports.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-5888848f9ffbddb7b5d76142e6226de61375df0b151360152edae46b25965fde
gen.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-4a1cf90ba2481aa2ceb7d4fa05de163b7af23afe883d14d85dac6f7bd031fe67
imports.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-ec691f29985038d8622fe3e038f5ef9ea3dab8bde745df1a7a201d1e5a35b3e5
postgresql_type.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-5962cb308fe6790725623571e3c275c44fef79e8f2f84d7e49bcc3fc55f29ece
config.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-54c7c1af5fa8d5db4dc49f0e8e80e93ba2b1183ba4d5c9e2e5729e6deae6a3cd
python_type.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-64dc351f540135df435f963f69e5696622108a199360d3d0517c0c7fbc2ec92a
v_two.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-5cbb844a4791fafa0b189ed613ce169fd9157f636b7767f35555c62d91206bd4
endtoend_test.go https://github.com/sqlc-dev/sqlc/pull/923/files#diff-05b57a482f9a1eab1ee8b30fa77f0daa2bd2e3f7b1e5031e477a851ae3ec7f5e
.github/workflows/ci-python.ymlhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-a7ec86144013d78c9f1d734aa65a898e66b609638aa7b872491ee3858ebebbeb
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/.github/workflows/ci-python.yml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
examples/python/requirements.txthttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-c233c301c314607d2dfcee9a9ec0d5a7b78a67d2c8927808fd4c9d382bbb4a8e
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/requirements.txt
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
examples/python/sqlc.jsonhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-51ce5461fb9e627abb72ef91a05f41da8a7272d3d6d9101fe941b3b5ed7d99d9
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/sqlc.json
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
examples/python/src/authors/__init__.pyhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-86a97e55c4d2e8852a049aeae4633ceb4bf71e28ff76b2c13fd1f4cb5329ef6f
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/src/authors/__init__.py
Open in desktop https://desktop.github.com
examples/python/src/authors/models.pyhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-ee5ac580ad63bf714e44b9be5731416a492cf541f81038ad93aa944ad974a4c4
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/src/authors/models.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
kyleconroyhttps://github.com/kyleconroy
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587037314
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
robholthttps://github.com/robholt
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587555718
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
examples/python/src/authors/query.pyhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-c89f8979af02e775ccddfaba91d614eddb3bab71498afba34cc43d6464cffad9
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/src/authors/query.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
kyleconroyhttps://github.com/kyleconroy
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587040318
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
robholthttps://github.com/robholt
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587564948
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
kyleconroyhttps://github.com/kyleconroy
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587039485
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
robholthttps://github.com/robholt
Mar 4, 2021https://github.com/sqlc-dev/sqlc/pull/923/files#r587561113
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
examples/python/src/booktest/__init__.pyhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-061db2e5fb7334d64138a2fe43043fb683e59518a3502c3e49e21c794cd3b343
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/src/booktest/__init__.py
Open in desktop https://desktop.github.com
examples/python/src/booktest/models.pyhttps://github.com/sqlc-dev/sqlc/pull/923/files#diff-96909b321988c2a971e29eaa178d76f6c13550c628fc9238525934947d95c7bf
View file https://github.com/robholt/sqlc/blob/0a382c3cb9a634e9f4929715849e98261533854d/examples/python/src/booktest/models.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/sqlc-dev/sqlc/pull/923/{{ revealButtonHref }}
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
Please reload this pagehttps://github.com/sqlc-dev/sqlc/pull/923/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.