Title: feat: Add batch commit mode for MySQL OnlineStore by chimeyrock999 · Pull Request #5699 · feast-dev/feast · GitHub
Open Graph Title: feat: Add batch commit mode for MySQL OnlineStore by chimeyrock999 · Pull Request #5699 · feast-dev/feast
X Title: feat: Add batch commit mode for MySQL OnlineStore by chimeyrock999 · Pull Request #5699 · feast-dev/feast
Description: What this PR does / why we need it: Adds optional batch insert and commit mode to the MySQLOnlineStore to improve materialization performance for TiDB and MySQL backends. By default, Feast writes each row individually and commits per record, which causes large overhead on MySQL compatible Raft-based databases like TiDB. This PR introduces a configurable batch mode to group inserts and commit per batch, improving write throughput. Changes Added config flags: batch_mode: true batch_size: 1000 Implemented batched insert using executemany() / multi-row SQL. Each batch is committed atomically. Backward compatible (default = per-row mode). Misc Verified on TiDB v7.5.0 and MySQL 8.0.44. Thanks @ntkathole confirming this direction.
Open Graph Description: What this PR does / why we need it: Adds optional batch insert and commit mode to the MySQLOnlineStore to improve materialization performance for TiDB and MySQL backends. By default, Feast writes e...
X Description: What this PR does / why we need it: Adds optional batch insert and commit mode to the MySQLOnlineStore to improve materialization performance for TiDB and MySQL backends. By default, Feast writes e...
Opengraph URL: https://github.com/feast-dev/feast/pull/5699
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:3f9bff50-b97e-edd9-63ed-0601c012ab81 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | CE50:5C4F6:4245A6C:596A75B:696E0429 |
| html-safe-nonce | b37db8bba3042a361b5a5a89f4593855771014b23edd9db76d0bb7ed338a5c29 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDRTUwOjVDNEY2OjQyNDVBNkM6NTk2QTc1Qjo2OTZFMDQyOSIsInZpc2l0b3JfaWQiOiIyNTY0NDc2MDk3NDgyNTg5MjI1IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 0897bea0ac4c8780e821f828eb972c52bcaf633bea86da896fc3b90c06b7737b |
| hovercard-subject-tag | pull_request:2953717965 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/feast-dev/feast/pull/5699/files |
| twitter:image | https://avatars.githubusercontent.com/u/57527992?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/57527992?s=400&v=4 |
| og:image:alt | What this PR does / why we need it: Adds optional batch insert and commit mode to the MySQLOnlineStore to improve materialization performance for TiDB and MySQL backends. By default, Feast writes e... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 9b5131b207ddd175abf059a848d5f4302ec0606b02211b989013be49cf08593e |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/feast-dev/feast git https://github.com/feast-dev/feast.git |
| octolytics-dimension-user_id | 57027613 |
| octolytics-dimension-user_login | feast-dev |
| octolytics-dimension-repository_id | 161133770 |
| octolytics-dimension-repository_nwo | feast-dev/feast |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 161133770 |
| octolytics-dimension-repository_network_root_nwo | feast-dev/feast |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | f8590a63bfc8093b241930ca57d536c9a50f9680 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width