Title: Split up JUnit XML output per file · Issue #4233 · python/mypy · GitHub
Open Graph Title: Split up JUnit XML output per file · Issue #4233 · python/mypy
X Title: Split up JUnit XML output per file · Issue #4233 · python/mypy
Description: Hello! I've recently started running mypy with --junit-xml since our CI system supports parsing JUnit output (and it's been a really useful feature!). Currently, mypy's JUnit output lumps together all of the error output into one "testca...
Open Graph Description: Hello! I've recently started running mypy with --junit-xml since our CI system supports parsing JUnit output (and it's been a really useful feature!). Currently, mypy's JUnit output lumps together ...
X Description: Hello! I've recently started running mypy with --junit-xml since our CI system supports parsing JUnit output (and it's been a really useful feature!). Currently, mypy's JUnit output lum...
Opengraph URL: https://github.com/python/mypy/issues/4233
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Split up JUnit XML output per file","articleBody":"Hello! I've recently started running mypy with `--junit-xml` since our CI system supports parsing JUnit output (and it's been a really useful feature!).\r\n\r\nCurrently, mypy's JUnit output lumps together all of the error output into one \"testcase\", like so:\r\n```xml\r\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n\u003ctestsuite errors=\"0\" failures=\"1\" name=\"mypy\" skips=\"0\" tests=\"1\" time=\"0.826\">\r\n \u003ctestcase classname=\"mypy\" file=\"mypy\" line=\"1\" name=\"mypy\" time=\"0.826\">\r\n \u003cfailure message=\"mypy produced messages\">testmodule/bar.py:2: error: The return type of \"__init__\" must be None\r\ntestmodule/bar.py:2: error: Missing return statement\r\ntestmodule/foo.py:4: error: Self argument missing for a non-static method (or an invalid type for self)\r\ntestmodule/foo.py:5: error: Incompatible return value type (got \"str\", expected \"int\")\u003c/failure>\r\n \u003c/testcase>\r\n\u003c/testsuite>\r\n```\r\n\r\nWould anyone have any objection to breaking this output up into 1 testcase per file? I imagine it would look something like this:\r\n```xml\r\n\u003c?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n\u003ctestsuite errors=\"0\" failures=\"1\" name=\"mypy\" skips=\"0\" tests=\"3\" time=\"0.826\">\r\n \u003ctestcase classname=\"testmodule.bar\" file=\"testmodule/bar.py\" line=\"0\" name=\"testmodule.bar\">\r\n \u003cfailure message=\"mypy produced messages\">testmodule/bar.py:2: error: The return type of \"__init__\" must be None\r\ntestmodule/bar.py:2: error: Missing return statement\u003c/failure>\r\n \u003c/testcase>\r\n \u003ctestcase classname=\"testmodule.foo\" file=\"testmodule/foo.py\" line=\"0\" name=\"testmodule.foo\">\r\n \u003cfailure message=\"mypy produced messages\">testmodule/foo.py:4: error: Self argument missing for a non-static method (or an invalid type for self)\r\ntestmodule/foo.py:5: error: Incompatible return value type (got \"str\", expected \"int\")\u003c/failure>\r\n \u003c/testcase>\r\n \u003ctestcase classname=\"testmodule.file_with_no_issues\" file=\"testmodule/file_with_no_issues.py\" line=\"0\" name=\"testmodule.file_with_no_issues\">\u003c/testcase>\r\n\u003c/testsuite>\r\n```\r\n\r\nThere are a few benefits to splitting it up by file:\r\n- CI systems that support the JUnit format can nicely format each error message per file, making it easier for developers to see which files need fixing\r\n- CI systems that collect and aggregate metrics about which test cases fail most often / how many tests are run can have more detailed information to play with\r\n- We can account for files that don't have typing issues (like `testmodule/file_with_no_issues.py` in the example above).\r\n\r\nAs an alternative, I considered having each typing error surface as a failed test case, but that makes the total number of tests and the number of failing tests vary wildly, making it difficult for CI systems that try to aggregate test case data.\r\n\r\nI would love to implement this, just wanted to make sure no one had any objections to this before starting on it!\r\n\r\nFor implementation, I'm thinking of pulling in https://github.com/kyrus/python-junit-xml to actually generate the XML.","author":{"url":"https://github.com/kennydo","@type":"Person","name":"kennydo"},"datePublished":"2017-11-10T05:26:52Z","url":"https://github.com/python/mypy/issues/4233"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:cdf8f37e-26e2-30e0-6c9a-ac7e591b3a26 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | E98C:4D728:19DC2F4:21253D3:6AFFB834 |
| html-safe-nonce | ad5da9e066b3373ff9976dfe6980b779f7cdd738175070f7d30aab0b24e17f9c |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFOThDOjRENzI4OjE5REMyRjQ6MjEyNTNEMzo2QUZGQjgzNCIsInZpc2l0b3JfaWQiOiIyODY4MzEwMDY4MzcyMTU4NTE2IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 27f9ec672ec72c6237645c9417ddd64a3380f8c4ac45ba9dff7fd8249d2b8465 |
| hovercard-subject-tag | issue:272823323 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/mypy/4233/issue_layout |
| twitter:image | https://opengraph.githubassets.com/bb4778704373aa6dcbbd714de1a804cad9ef2ae82fbd67057e88c4268671e17b/python/mypy/issues/4233 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/bb4778704373aa6dcbbd714de1a804cad9ef2ae82fbd67057e88c4268671e17b/python/mypy/issues/4233 |
| og:image:alt | Hello! I've recently started running mypy with --junit-xml since our CI system supports parsing JUnit output (and it's been a really useful feature!). Currently, mypy's JUnit output lumps together ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | kennydo |
| hostname | github.com |
| expected-hostname | github.com |
| None | 7e461843db1c75e40d788c5cfd440d0e4ae1271cf19e06a798d0e7feb2d708f3 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/mypy git https://github.com/python/mypy.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 7053637 |
| octolytics-dimension-repository_nwo | python/mypy |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 7053637 |
| octolytics-dimension-repository_network_root_nwo | python/mypy |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | d29a6a99861646939301302fab465f9f123b4773 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width