Title: perf(pii): batch spaCy NER passes in redact_batch by TheodoreSpeaks · Pull Request #8772 · simstudioai/sim · GitHub
Open Graph Title: perf(pii): batch spaCy NER passes in redact_batch by TheodoreSpeaks · Pull Request #8772 · simstudioai/sim
X Title: perf(pii): batch spaCy NER passes in redact_batch by TheodoreSpeaks · Pull Request #8772 · simstudioai/sim
Description: Summary Presidio's BatchAnalyzerEngine.analyze_iterator defaults nlp.pipe to batch_size=1, so every /redact_batch request that runs spaCy NER does one forward pass per text. Prod chunks sit at the 2000-text cap almost always (median and p90 = 2000 texts on the full-NLP path), so each request was paying 2000 separate passes. This passes batch_size=256 through. Context: PII redaction latency in prod has been elevated since an org enabled 9-entity full-NLP redaction; uncontended 2000-text requests take ~12s (p10) and p50 is 26-36s even after scaling the fleet to 10 tasks. This halves the CPU cost per request without changing detections. Type of Change Bug fix New feature Breaking change Documentation Other: performance Testing Built docker/pii.Dockerfile locally and ran _analyze_many on 2000 synthetic prod-shaped texts (1 CPU, best of 2), asserting detected spans are identical to batch_size=1: Texts Entities bs=1 bs=64 bs=256 ~70 B 9-entity 4.21s 1.82s 1.84s ~70 B all 5.08s 4.11s 2.63s ~141 B 9-entity 6.14s 3.21s 3.03s ~141 B all 6.34s 3.74s 3.54s Detections were identical at every batch size. Memory stays bounded since requests are already capped at 256 KB by pii-batching.ts. Regex-only requests (blank-artifacts fast path) are unaffected. Checklist Code follows project style guidelines Self-reviewed my changes Tests added/updated and passing No new warnings introduced I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA) 🤖 Generated with Claude Code https://claude.ai/code/session_018cHfdA1zkn7otFDmHHqtRd
Open Graph Description: Summary Presidio's BatchAnalyzerEngine.analyze_iterator defaults nlp.pipe to batch_size=1, so every /redact_batch request that runs spaCy NER does one forward pass per text. Prod chunks sit at ...
X Description: Summary Presidio's BatchAnalyzerEngine.analyze_iterator defaults nlp.pipe to batch_size=1, so every /redact_batch request that runs spaCy NER does one forward pass per text. Prod chunks sit...
Opengraph URL: https://github.com/simstudioai/sim/pull/8772
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react-reconciler,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@d5750093facf |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:a886f106-9f19-f795-e667-f28465f04593 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | DBD4:27DCD7:DFC8A1:1318E95:6AFB5A9B |
| html-safe-nonce | 9d83325fbece1056ad31e2c46e3c72819666fc56dcb95ef3c8d806f60df223d2 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEQkQ0OjI3RENENzpERkM4QTE6MTMxOEU5NTo2QUZCNUE5QiIsInZpc2l0b3JfaWQiOiI2NTY3NzU4OTQ0NTI3OTMyMDU5IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | d9801d07e8d1941f87d1361cc1379aefe9790f4d5dce44d4fa31f9a25728365f |
| hovercard-subject-tag | pull_request:4780369509 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/simstudioai/sim/pull/8772/files |
| twitter:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/19564703?s=400&v=4 |
| og:image:alt | Summary Presidio's BatchAnalyzerEngine.analyze_iterator defaults nlp.pipe to batch_size=1, so every /redact_batch request that runs spaCy NER does one forward pass per text. Prod chunks sit at ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | ea33d13c96d0ff66bd8fa906b193d2213536b78cb06b0c50a4632c58b1418f94 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/simstudioai/sim git https://github.com/simstudioai/sim.git |
| octolytics-dimension-user_id | 199344406 |
| octolytics-dimension-user_login | simstudioai |
| octolytics-dimension-repository_id | 912559512 |
| octolytics-dimension-repository_nwo | simstudioai/sim |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 912559512 |
| octolytics-dimension-repository_network_root_nwo | simstudioai/sim |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 59ef97164c52cf59bd10558ddec9c4f482763988 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width