| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:eae02d2b-a3f7-1b64-8297-28adad4f9bec |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | B9B0:11EEA7:153B95:1C6C6C:6AFF6082 |
| html-safe-nonce | 544fea239ee18a0795cac4cef66e9210f1d15ed252a175a1a9ee2c48b3bd0283 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCOUIwOjExRUVBNzoxNTNCOTU6MUM2QzZDOjZBRkY2MDgyIiwidmlzaXRvcl9pZCI6IjI5ODU1NDYwMjM5NjQ3Mjk0NzQiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 2787becfe2634f9b2e5448fc6d5c085a0f3d914bacf7b7b6fb38e259b89cc9db |
| hovercard-subject-tag | pull_request:3712994056 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | ///pull_requests/show/files |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/UiPath/uipath-python/pull/1663/files |
| twitter:image | https://avatars.githubusercontent.com/u/277853788?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/277853788?s=400&v=4 |
| og:image:alt | Summary
Completes the classification evaluator feature shipped in #1397 by adding the three pieces that PR didn't carry:
Generated type schemas — BinaryClassificationEvaluator.json and Multic... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 2422ccd4bc783f932948cce4871cb9e0622bab749db07f387c9631b7e6834d1f |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/UiPath/uipath-python git https://github.com/UiPath/uipath-python.git |
| octolytics-dimension-user_id | 375663 |
| octolytics-dimension-user_login | UiPath |
| octolytics-dimension-repository_id | 925171912 |
| octolytics-dimension-repository_nwo | UiPath/uipath-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 925171912 |
| octolytics-dimension-repository_network_root_nwo | UiPath/uipath-python |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 2a39546baeb98186e651a01d5d7aa6465825eeb1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| Skip to content | https://github.com/UiPath/uipath-python/pull/1663/files#start-of-content |
| https://github.com/ |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FUiPath%2Fuipath-python%2Fpull%2F1663%2Ffiles |
| GitHubRepository hosting, code review, and collaboration | https://github.com/platform |
| CopilotAI coding agent for tasks, projects, and automated work | https://github.com/features/copilot |
| ActionsCI/CD and automation | https://github.com/features/actions |
| Code QualityQuality analysis, fixes, and enforcement | https://github.com/features/code-quality |
| CodespacesCloud development environments | https://github.com/features/codespaces |
| Code SecurityVulnerability detection and remediation | https://github.com/security/advanced-security/code-security |
| Secret ProtectionSecret scanning and leak prevention | https://github.com/security/advanced-security/secret-protection |
| All Features | https://github.com/features |
| Documentation | https://docs.github.com/en |
| Training | https://learn.github.com/skills |
| Certifications | https://learn.github.com/credentials |
| Build with Copilot | https://github.com/features/ai |
| Customize Copilot | https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview |
| Integrations | https://github.com/marketplace |
| Trending repos | https://github.com/trending |
| GitHub Sponsors | https://github.com/open-source/sponsors |
| Developer articles | https://github.com/resources/articles |
| ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place. | https://github.blog/changelog/ |
| Enterprises | https://github.com/enterprise |
| Team | https://github.com/team |
| Startups | https://github.com/enterprise/startups |
| Nonprofits | https://github.com/solutions/industry/nonprofits |
| Healthcare | https://github.com/solutions/industry/healthcare |
| Financial services | https://github.com/solutions/industry/financial-services |
| Manufacturing | https://github.com/solutions/industry/manufacturing |
| Government | https://github.com/solutions/industry/government |
| See all industries | https://github.com/solutions/industry |
| Why GitHub | https://github.com/why-github |
| Customer Stories | https://github.com/customer-stories |
| Business insights | https://github.com/solutions/executive-insights |
| Trust Center | https://github.com/trust-center |
| GitHub Advanced SecurityEnterprise-grade code security | https://github.com/security/advanced-security |
| Copilot for businessEnterprise-grade AI features | https://github.com/features/copilot/copilot-business |
| Premium SupportEnterprise-grade 24/7 support | https://github.com/enterprise/premium-support |
| Documentation | https://docs.github.com/en |
| Resource center | https://github.com/resources |
| Blog | https://github.blog/ |
| Events & webinars | https://github.com/resources/events |
| Community Forum | https://github.com/orgs/community/discussions |
| Customer Stories | https://github.com/customer-stories |
| Integrations | https://github.com/marketplace |
| Customer support | https://help.github.com/ |
| Training | https://learn.github.com/skills |
| GitHub Status | https://www.githubstatus.com/ |
| About | https://github.com/about |
| Partners | https://github.com/partners |
| Changelog | https://github.blog/changelog/ |
| GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub. | https://githubuniverse.com/ |
| Pricing | https://github.com/pricing |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FUiPath%2Fuipath-python%2Fpull%2F1663%2Ffiles |
| Sign up | https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=UiPath%2Fuipath-python |
| Reload | https://github.com/UiPath/uipath-python/pull/1663/files |
| Reload | https://github.com/UiPath/uipath-python/pull/1663/files |
| Reload | https://github.com/UiPath/uipath-python/pull/1663/files |
| UiPath | https://github.com/UiPath |
| uipath-python | https://github.com/UiPath/uipath-python |
| Code | https://github.com/UiPath/uipath-python |
| Issues81 (81) | https://github.com/UiPath/uipath-python/issues |
| Pull requests144 (144) | https://github.com/UiPath/uipath-python/pulls |
| Actions | https://github.com/UiPath/uipath-python/actions |
| Projects | https://github.com/UiPath/uipath-python/projects |
| Security and quality | https://github.com/UiPath/uipath-python/security |
| Insights | https://github.com/UiPath/uipath-python/insights |
| Sign up for GitHub
| https://github.com/signup?return_to=%2FUiPath%2Fuipath-python%2Fissues%2Fnew%2Fchoose |
| terms of service | https://docs.github.com/terms |
| privacy statement | https://docs.github.com/privacy |
| Sign in | https://github.com/login?return_to=%2FUiPath%2Fuipath-python%2Fissues%2Fnew%2Fchoose |
| ajay-kesavan | https://github.com/ajay-kesavan |
| main | https://github.com/UiPath/uipath-python/tree/main |
| feat/classification-evaluator-types | https://github.com/UiPath/uipath-python/tree/feat/classification-evaluator-types |
|
Conversation
38
| https://github.com/UiPath/uipath-python/pull/1663 |
|
Commits
34
| https://github.com/UiPath/uipath-python/pull/1663/commits |
|
Checks
174
| https://github.com/UiPath/uipath-python/pull/1663/checks |
|
Files changed
| https://github.com/UiPath/uipath-python/pull/1663/files |
| Please reload this page | https://github.com/UiPath/uipath-python/pull/1663/files |
|
feat(eval): dataset-level evaluators + classification schemas, samples, e2e
| https://github.com/UiPath/uipath-python/pull/1663/files#top |
|
Show all changes
34 commits
| https://github.com/UiPath/uipath-python/pull/1663/files |
|
6b11767
feat(eval): add evaluator type schemas for classification evaluators
ajay-kesavan May 20, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/6b11767d30fb08969146d4bb58ac8570cc20c34f |
|
037b60c
test(eval): add e2e tests + sample projects for classification evalua…
ajay-kesavan May 20, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/037b60cdb6e721c494b2b4fd173e6bf1bdb450ed |
|
5e574f1
feat(eval): add dataset-level evaluator framework with precision/reca…
ajay-kesavan May 20, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/5e574f1895feccb314fd929d57e15dd69580c5f0 |
|
d6b7ab5
docs(eval): add runnable dataset evaluator demo + bump uv.lock for 2.…
ajay-kesavan May 20, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/d6b7ab5566d07a9e34611358a4b7539912982936 |
|
e9ba8aa
Merge remote-tracking branch 'origin/main' into feat/eval-dataset-eva…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/e9ba8aa8b16f96e30e255b74b76ca294cdb6b04e |
|
46c24e1
Merge remote-tracking branch 'origin/main' into feat/classification-e…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/46c24e11129518f8d2adaab196cd6ef367c95ef0 |
|
fb091e4
refactor(eval): embed aggregator specs in per-datapoint evaluator con…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/fb091e46c686da88958aa002cbfdb34527fe08ab |
|
d4e06b1
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/d4e06b1019f44e8b15b40054c47f6a9764d2b34b |
|
77fcc10
feat(eval): wire sample classification evaluators to embedded aggrega…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/77fcc109777dd2ba943e4ff3c2d3745dbed7dc21 |
|
c0436a3
refactor(eval): apply ponytail-review cleanup
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/c0436a3da061146b61b117dbe885606b4fd52fef |
|
05f6697
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/05f669701a159cc556cde95c57eb20285db9a96c |
|
50c64f4
refactor(eval): apply ponytail-review cleanup (justification + demo)
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/50c64f4862c57834437b1dba59266106f29e3b66 |
|
ad32c22
fix(eval): address adversarial-review feedback on dataset evaluators
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/ad32c22c64e7ccb5aaf8446454cf8bc9408f6c30 |
|
cbbaf5f
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/cbbaf5f275437e379ce9a6b9ee8d271aadfc3dea |
|
027901c
fix(eval): address adversarial-review feedback on classification samples
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/027901c96be416d791e76d93c3b2ca9d4a470a95 |
|
4d6afcc
fix(eval): address codex P1 + lint failures on dataset evaluators
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/4d6afccafbdb4adb5af0365af69d6613953ddc31 |
|
c347fc7
Merge branch 'feat/eval-dataset-evaluators' into feat/classification-…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/c347fc7592926e7269cd5f9147431ec331d102e8 |
|
5d78205
test(eval): drop fscore-duplicate test that conflicts with #1663 H2 v…
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/5d782052d4cd1ee22966b5784e7a9f885192ec29 |
|
363855d
fix(eval): publish aggregators in classification evaluator type schemas
ajay-kesavan Jun 19, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/363855d4f2b86321ee933c6b1e382364257a6c44 |
|
2be26c9
Merge remote-tracking branch 'origin/main' into feat/classification-e…
ajay-kesavan Jun 24, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/2be26c9dd19be39fc92975dbe9ca1bce03fb7996 |
|
b872c8f
Merge remote-tracking branch 'origin/main' into feat/classification-e…
ajay-kesavan Jul 7, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/b872c8f641fe86067e9243240e701db7ad6d3133 |
|
dd910e4
refactor(eval): move aggregators to ExactMatch, revert on Binary/Mult…
ajay-kesavan Jul 8, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/dd910e4550eead1ba70aa197776fa574a3912b7a |
|
383cbb7
refactor(eval)!: trim PR to the ExactMatch aggregator contract only
ajay-kesavan Jul 8, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/383cbb788535833690482680f48e6054781c2490 |
|
2885c26
feat(eval): restore local dataset aggregation for `uipath eval`
ajay-kesavan Jul 8, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/2885c2644b079a69e5313dc1ddc10f7912fcb467 |
|
1ce1209
feat(eval)!: single aggregation implementation — SDK owns the math
ajay-kesavan Jul 9, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/1ce1209790ed9f1607df3f4a07d2822b5a690611 |
|
ea3672e
fix(eval): review-round hardening for dataset aggregators
ajay-kesavan Jul 9, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/ea3672e426a27622ce36d6b922d3481117aa9532 |
|
fa0d86c
fix(eval): round-2 review — validator gaps + test leanness
ajay-kesavan Jul 9, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/fa0d86c59dcc804f7c4ae48c1f09c3765e2a05b7 |
|
e746492
Merge remote-tracking branch 'origin/main' into feat/classification-e…
ajay-kesavan Jul 21, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/e746492e2a0553a06e27fe6013455681af3b62d8 |
|
1ea11cd
fix(eval): dedup per-datapoint results before aggregation; drop dead …
ajay-kesavan Jul 21, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/1ea11cde8448a5cdfeb0604535e9b8dba5b1260c |
|
603ba28
test: fix mypy type errors in dataset classification test files
ajay-kesavan Jul 21, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/603ba28e9133cf627fbb3ec4b2b9171b2dd83c54 |
|
c710a77
Merge remote-tracking branch 'origin/main' into feat/classification-e…
ajay-kesavan Jul 22, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/c710a77a5443aa8f85e3b7e4cd537cf9fbaf3d84 |
|
0d76fe8
chore(uipath): bump version to 2.13.15
ajay-kesavan Jul 23, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/0d76fe8848be376e5b3246499c77a7fbbeb4e2b1 |
|
5b95d05
fix(eval): count OOV predictions as recall misses; reject negated + a…
ajay-kesavan Jul 23, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/5b95d05112413fe8909243335897a94bdaedf983 |
|
d85dc44
Merge branch 'main' into feat/classification-evaluator-types
ajay-kesavan Jul 23, 2026
| https://github.com/UiPath/uipath-python/pull/1663/commits/d85dc44470eaf003403fdb02ddd8d6cb65166a69 |
|
Clear filters
| https://github.com/UiPath/uipath-python/pull/1663/files |
| Please reload this page | https://github.com/UiPath/uipath-python/pull/1663/files |
| Please reload this page | https://github.com/UiPath/uipath-python/pull/1663/files |
|
_aggregator_specs.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-49094fe4c46eb48c736a63cd0573ba58a227437f7381e88aa1b6a7bf99321d4d |
|
base_dataset_evaluator.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-a606f0a5654fc261b8030ba701a1eac4fa1b281d8696d312c2a18882203aa84c |
|
base_evaluator.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-34ead661d21eee7b1613ccdc64c834b97f2ab417f86164131ee412505356351f |
|
classification_dataset_evaluators.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-48d222a0b2a8d41e280cc9f055af28c64ac3e83438ad9f18dfc488cfc3fad7a9 |
|
dataset_evaluator_factory.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-7f720784bfbb9eefa17e7867dd63be2ccd71a2d8777d40c9802f36cb69cd537f |
|
exact_match_evaluator.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-6cec336548679d0a325cec6b421350a0080c1e808f3b12c84c7fc9d2ee013ae0 |
|
ExactMatchEvaluator.json
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-af69a8044bd5ee5aa2cdc36ce07e142e74f247191d17cbcdc95a0d1a045fc083 |
|
_types.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-8d46b76f9d671dd9c162388fa594a1ce588a2e4acb4a5ffe60bac0f92f0c2b59 |
|
runtime.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-4b41488cb098f634547e40dcc6c8a9c3c7e9fc4fe8a1f215b8334e9fd85e6da0 |
|
test_dataset_classification_evaluators.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-fe20e87e2ebc42bf25c2b19d430afd174ad0c2732213129d8a0b1faf355e47a4 |
|
test_exact_match_aggregators.py
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-5ed2a2296649ac61b6b6f0d4869914b5058c3364627179d971b97c875ca0da28 |
| packages/uipath/src/uipath/eval/evaluators/_aggregator_specs.py | https://github.com/UiPath/uipath-python/pull/1663/files#diff-49094fe4c46eb48c736a63cd0573ba58a227437f7381e88aa1b6a7bf99321d4d |
|
View file
| https://github.com/UiPath/uipath-python/blob/d85dc44470eaf003403fdb02ddd8d6cb65166a69/packages/uipath/src/uipath/eval/evaluators/_aggregator_specs.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/UiPath/uipath-python/pull/1663/{{ revealButtonHref }} |
| packages/uipath/src/uipath/eval/evaluators/base_dataset_evaluator.py | https://github.com/UiPath/uipath-python/pull/1663/files#diff-a606f0a5654fc261b8030ba701a1eac4fa1b281d8696d312c2a18882203aa84c |
|
View file
| https://github.com/UiPath/uipath-python/blob/d85dc44470eaf003403fdb02ddd8d6cb65166a69/packages/uipath/src/uipath/eval/evaluators/base_dataset_evaluator.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/UiPath/uipath-python/pull/1663/{{ revealButtonHref }} |
| packages/uipath/src/uipath/eval/evaluators/base_evaluator.py | https://github.com/UiPath/uipath-python/pull/1663/files#diff-34ead661d21eee7b1613ccdc64c834b97f2ab417f86164131ee412505356351f |
|
View file
| https://github.com/UiPath/uipath-python/blob/d85dc44470eaf003403fdb02ddd8d6cb65166a69/packages/uipath/src/uipath/eval/evaluators/base_evaluator.py |
|
Open in desktop
| https://desktop.github.com |
| https://github.co/hiddenchars |
| https://github.com/UiPath/uipath-python/pull/1663/{{ revealButtonHref }} |
|
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-34ead661d21eee7b1613ccdc64c834b97f2ab417f86164131ee412505356351f |
|
| https://github.com/UiPath/uipath-python/pull/1663/files#diff-34ead661d21eee7b1613ccdc64c834b97f2ab417f86164131ee412505356351f |
| Please reload this page | https://github.com/UiPath/uipath-python/pull/1663/files |
| Please reload this page | https://github.com/UiPath/uipath-python/pull/1663/files |
|
| https://github.com |
| Terms | https://docs.github.com/site-policy/github-terms/github-terms-of-service |
| Privacy | https://docs.github.com/site-policy/privacy-policies/github-privacy-statement |
| Security | https://github.com/security |
| Status | https://www.githubstatus.com/ |
| Community | https://github.community/ |
| Docs | https://docs.github.com/ |
| Contact | https://support.github.com?tags=dotcom-footer |