Title: Browserstack and/or Sauce Labs should be required CI jobs · Issue #5407 · angular/angular · GitHub
Open Graph Title: Browserstack and/or Sauce Labs should be required CI jobs · Issue #5407 · angular/angular
X Title: Browserstack and/or Sauce Labs should be required CI jobs · Issue #5407 · angular/angular
Description: They are only optional today. but to get there, they need to be made reliable. We are currently running 19 browsers in SL and 15 in BS, and monitoring the output. Thanks to #4417 which landed yesterday, the 2 are both green if everything...
Open Graph Description: They are only optional today. but to get there, they need to be made reliable. We are currently running 19 browsers in SL and 15 in BS, and monitoring the output. Thanks to #4417 which landed yeste...
X Description: They are only optional today. but to get there, they need to be made reliable. We are currently running 19 browsers in SL and 15 in BS, and monitoring the output. Thanks to #4417 which landed yeste...
Opengraph URL: https://github.com/angular/angular/issues/5407
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Browserstack and/or Sauce Labs should be required CI jobs","articleBody":"They are only optional today. but to get there, they need to be made reliable. We are currently running 19 browsers in SL and 15 in BS, and monitoring the output.\n\nThanks to #4417 which landed yesterday, the 2 are both green if everything goes well. But it doesn't happen often... We are experiencing 2 kind of errors:\n- failure: browser does not start or disconnects or fails or is very slow, so the campaign is not fully run\n- flake: a test that should pass fails because it reaches the timeout\n\nTo improve the situation, I've looked at about 60 builds in the last 10 days (all builds in master and presubmit-\\* after #4417). That's about 120 jobs or 2000 campaigns or 4M+ unit tests.\nMost jobs failed (90%) due to 123 failures and 74 flakes. This is bad, but:\n- 70% of the failures come from iOS9. Our campaign doesn't work well in this simulator, be it in SL,in BS or locally. In 80% of the campaigns, Safari mobile freezes silently, fails to load resources (XHR error), or is very slow.\n- 80% of the flakes come from 5 unit tests which can \"randomly\" fail in any browser. They are:\n\n```\nPostMessageBusSink should send messages immediatly when run outside the zone\nelement probe should provide a global function to inspect elements\nMdButton button[md-button] should disable the button\nMdButton a[md-button] should remove disabled anchors from tab order\nMdButton button[md-button] should handle a click on the button\n```\n\nIt also interesting to note that there is a trend which shows that SL is better with Windows, while BS is better with Mac. This applies to both failures and flakes.\nSo an idea would be split browsers among the 2 providers to improve the overall reliability.\n\nNext steps:\n- remove iOS9 from CI until it is made reliable, if possible. There is no problem with the desktop version, so it shouldn't be an issue\n- increase the timeout of the 5 identified flaky tests\n- continue monitoring and apply the split if the trend is confirmed\n","author":{"url":"https://github.com/marclaval","@type":"Person","name":"marclaval"},"datePublished":"2015-11-20T14:59:06Z","url":"https://github.com/angular/angular/issues/5407"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:342739e1-8d30-2d3e-c468-f7f9c0c313c5 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 91BE:3A347:1AA6EE7:233C9F3:6B014D73 |
| html-safe-nonce | c860dea53af222e09313375e84f949e3f17bcf4a8328d28f7947274ff6cc91b3 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5MUJFOjNBMzQ3OjFBQTZFRTc6MjMzQzlGMzo2QjAxNEQ3MyIsInZpc2l0b3JfaWQiOiI3MzI3OTUyNjk0MDM4MDU2MzA3IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 1fe7eebd0154ef7429e81da26b4eef49c41da74edf481391332edd5ac6826450 |
| hovercard-subject-tag | issue:118057862 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/angular/angular/5407/issue_layout |
| twitter:image | https://opengraph.githubassets.com/0897137e6d6788ad831bd69f9cb0eae5baff88c236a1d1511a88b3ff31980b1a/angular/angular/issues/5407 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/0897137e6d6788ad831bd69f9cb0eae5baff88c236a1d1511a88b3ff31980b1a/angular/angular/issues/5407 |
| og:image:alt | They are only optional today. but to get there, they need to be made reliable. We are currently running 19 browsers in SL and 15 in BS, and monitoring the output. Thanks to #4417 which landed yeste... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | marclaval |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/angular/angular git https://github.com/angular/angular.git |
| octolytics-dimension-user_id | 139426 |
| octolytics-dimension-user_login | angular |
| octolytics-dimension-repository_id | 24195339 |
| octolytics-dimension-repository_nwo | angular/angular |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 24195339 |
| octolytics-dimension-repository_network_root_nwo | angular/angular |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width