René's URL Explorer Experiment


Title: runBenchmark setup needs to be awaited · Issue #36986 · angular/angular · GitHub

Open Graph Title: runBenchmark setup needs to be awaited · Issue #36986 · angular/angular

X Title: runBenchmark setup needs to be awaited · Issue #36986 · angular/angular

Description: 🐞 bug report Is this a regression? No Context perf_util.ts defines the function runBenchmark which is a wrapper for benchpress's sampler. The function does three key things: 1. Open the browser to a given url, 2. Allow for some initial s...

Open Graph Description: 🐞 bug report Is this a regression? No Context perf_util.ts defines the function runBenchmark which is a wrapper for benchpress's sampler. The function does three key things: 1. Open the browser to ...

X Description: 🐞 bug report Is this a regression? No Context perf_util.ts defines the function runBenchmark which is a wrapper for benchpress's sampler. The function does three key things: 1. Open the browser...

Opengraph URL: https://github.com/angular/angular/issues/36986

X: @github

direct link

Domain: github.com


Hey, it has json ld scripts:
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"runBenchmark setup needs to be awaited","articleBody":"# 🐞 bug report\r\n\r\n\r\n### Is this a regression?\r\n\r\nNo\r\n\r\n\r\n### Context\r\n\r\n[perf_util.ts](https://github.com/angular/angular/blob/master/modules/e2e_util/perf_util.ts) defines the function `runBenchmark` which is a wrapper for benchpress's sampler. The function does three key things: 1. Open the browser to a given url, 2. Allow for some initial setup via a callback method \"`setup`\", and 3. Run a benchpress sampler to run `prepare` and `work` over and over again to get performance metrics.\r\n\r\nThis bug is specifically surrounding the `setup` callback, step 2 as described above.\r\n\r\n\r\n### Description\r\n\r\nBecause we do not await `config.setup()` in [perf_util.ts L37](https://github.com/angular/angular/blob/master/modules/e2e_util/perf_util.ts#L37), if setup is an async function (which is very likely since setup code will probably be interacting with the dom) there is no guarantee the call to `config.setup()` will finish before the sampling begins.\r\n\r\n\r\n## 🔬 Minimal Reproduction\r\nHere is a simple test I wrote\r\n\r\n```ts\r\n  it('click (unchecked -> checked)', async() => {\r\n    await runBenchmark({\r\n      id: 'checkbox-overview-click-unchecked-to-checked',\r\n      url: '',\r\n      ignoreBrowserSynchronization: true,\r\n      params: [],\r\n      setup: async () => {\r\n        await $('#show').click();\r\n        console.log('setup before');\r\n        await $('mat-checkbox').click();\r\n        console.log('setup after');\r\n      },\r\n      prepare: async () => {\r\n        console.log('prepare before');\r\n        await $('mat-checkbox').click();\r\n        console.log('prepare after');\r\n      },\r\n      work: async () => {\r\n        console.log('execute before');\r\n        await $('mat-checkbox').click();\r\n        console.log('execute after');\r\n      },\r\n    });\r\n  });\r\n```\r\n\r\nAnd here are the logs from that test\r\n```\r\nRunning protractor tests\r\n-----------------------------------------------------------------------------\r\nversion     : 5.4.3\r\npwd         : /private/var/tmp/_bazel_wagnermaciel/7b9bf2897304b51bc8634c41fff3e07f/execroot/angular_material/bazel-out/darwin-fastbuild/bin/test/benchmarks/material/checkbox/benchmark_chromium.sh.runfiles/angular_material\r\nconf        : /private/var/tmp/_bazel_wagnermaciel/7b9bf2897304b51bc8634c41fff3e07f/execroot/angular_material/bazel-out/darwin-fastbuild/bin/test/benchmarks/material/checkbox/benchmark_chromium.sh.runfiles/angular_material/test/benchmarks/material/checkbox/benchmark_wrapped_test.conf.js\r\nnode_options:\r\n\r\nYour protractor configuration specifies capabilities for browser 'chrome'\r\nwhich will be merged with capabilities provided by Bazel resulting in: {\r\n  \"browserName\": \"chrome\",\r\n  \"chromeOptions\": {\r\n    \"args\": [\r\n      \"--js-flags=--expose-gc\",\r\n      \"--no-sandbox\",\r\n      \"--headless\",\r\n      \"--disable-dev-shm-usage\",\r\n      \"--no-sandbox\",\r\n      \"--headless\",\r\n      \"--disable-gpu\",\r\n      \"--disable-dev-shm-usage\"\r\n    ],\r\n    \"perfLoggingPrefs\": {\r\n      \"traceCategories\": \"v8,blink.console,devtools.timeline,disabled-by-default-devtools.timeline,blink.user_timing\"\r\n    },\r\n    \"binary\": \"/private/var/tmp/_bazel_wagnermaciel/7b9bf2897304b51bc8634c41fff3e07f/execroot/angular_material/bazel-out/darwin-fastbuild/bin/test/benchmarks/material/checkbox/benchmark_chromium.sh.runfiles/npm_dev_infra/benchmark/browsers/chromium_archive.out/chrome-mac/Chromium.app/Contents/MacOS/chromium\"\r\n  },\r\n  \"loggingPrefs\": {\r\n    \"performance\": \"ALL\",\r\n    \"browser\": \"ALL\"\r\n  }\r\n}\r\n[19:36:23] I/launcher - Running 1 instances of WebDriver\r\n[19:36:23] I/direct - Using ChromeDriver directly...\r\nServer listening on http://wagnermaciel-macbookpro.roam.corp.google.com:36947/\r\nStarted\r\n\r\n\r\n(node:30861) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.\r\n*\r\nsetup before\r\nBENCHMARK checkbox-overview-click-unchecked-to-checked\r\nDescription:\r\n- forceGc: false\r\n- regressionSlopeMetric: scriptTime\r\n- runId: 09423a40-909a-11ea-b84b-a70081dde6c8\r\n- sampleSize: 20\r\n- userAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/76.0.3809.0 Safari/537.36\r\nMetrics:\r\n- gcAmount: gc amount in kbytes\r\n- gcTime: gc time in ms\r\n- majorGcTime: time of major gcs in ms\r\n- pureScriptTime: script execution time in ms, without gc nor render\r\n- renderTime: render time in ms\r\n- scriptTime: script execution time in ms, including gc and render\r\n\r\n          gcAmount |             gcTime |        majorGcTime |     pureScriptTime |         renderTime |         scriptTime\r\n------------------ | ------------------ | ------------------ | ------------------ | ------------------ | ------------------\r\nprepare before\r\nsetup after\r\nprepare after\r\nexecute before\r\nexecute after\r\n              0.00 |               0.00 |               0.00 |               1.57 |               2.87 |               1.83\r\n```\r\n","author":{"url":"https://github.com/wagnermaciel","@type":"Person","name":"wagnermaciel"},"datePublished":"2020-05-07T19:46:35Z","url":"https://github.com/angular/angular/issues/36986"}

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format)
route-controllervoltron_issues_fragments
route-actionissue_layout
fetch-noncev2:44a36d83-c2cc-343a-9ad8-25ca7186c7f2
current-catalog-service-hash81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114
request-idCE88:25598F:188ECDC:2068CEB:6B0140F4
html-safe-nonce9ccf2cf5c9116fc845616c65611a342426f91d2bbcfd11eccb9a9080b7816110
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDRTg4OjI1NTk4RjoxODhFQ0RDOjIwNjhDRUI6NkIwMTQwRjQiLCJ2aXNpdG9yX2lkIjoiNjg2MjczOTA5MzcxMzkyODQzNiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac6ff7ba266a4d51e70a81ccf24451802e929436643f3e57f8829e697d76d40a1a
hovercard-subject-tagissue:614301066
github-keyboard-shortcutsrepository,issues,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///voltron/issues_fragments/issue_layout
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/angular/angular/36986/issue_layout
twitter:imagehttps://opengraph.githubassets.com/c90a596c2f04ff0bd83076def12dd489bc74eac0b1c0587fcc6e54eb0909dfdb/angular/angular/issues/36986
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/c90a596c2f04ff0bd83076def12dd489bc74eac0b1c0587fcc6e54eb0909dfdb/angular/angular/issues/36986
og:image:alt🐞 bug report Is this a regression? No Context perf_util.ts defines the function runBenchmark which is a wrapper for benchpress's sampler. The function does three key things: 1. Open the browser to ...
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
og:author:usernamewagnermaciel
hostnamegithub.com
expected-hostnamegithub.com
Nonecc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c
turbo-cache-controlno-preview
go-importgithub.com/angular/angular git https://github.com/angular/angular.git
octolytics-dimension-user_id139426
octolytics-dimension-user_loginangular
octolytics-dimension-repository_id24195339
octolytics-dimension-repository_nwoangular/angular
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id24195339
octolytics-dimension-repository_network_root_nwoangular/angular
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release8c77c9e8439031ad54c75a250694842892f693e1
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-nameissues

Links:

Skip to contenthttps://github.com/angular/angular/issues/36986#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fangular%2Fangular%2Fissues%2F36986
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fangular%2Fangular%2Fissues%2F36986
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fvoltron%2Fissues_fragments%2Fissue_layout&source=header-repo&source_repo=angular%2Fangular
Reloadhttps://github.com/angular/angular/issues/36986
Reloadhttps://github.com/angular/angular/issues/36986
Reloadhttps://github.com/angular/angular/issues/36986
angularhttps://github.com/angular
angularhttps://github.com/angular/angular
Codehttps://github.com/angular/angular
Issues836 (836)https://github.com/angular/angular/issues
Pull requests141 (141)https://github.com/angular/angular/pulls
Discussionshttps://github.com/angular/angular/discussions
Actionshttps://github.com/angular/angular/actions
Projectshttps://github.com/angular/angular/projects
Security and quality40 (40)https://github.com/angular/angular/security
Insightshttps://github.com/angular/angular/insights
https://github.com/angular/angular
New issuehttps://github.com/login?return_to=https://github.com/angular/angular/issues/36986
New issuehttps://github.com/login?return_to=https://github.com/angular/angular/issues/36986
runBenchmark setup needs to be awaitedhttps://github.com/angular/angular/issues/36986#top
https://github.com/wagnermaciel
area: benchpresshttps://github.com/angular/angular/issues?q=state%3Aopen%20label%3A%22area%3A%20benchpress%22
needsTriagehttps://github.com/angular/angular/milestone/83
https://github.com/wagnermaciel
wagnermacielhttps://github.com/wagnermaciel
on May 7, 2020https://github.com/angular/angular/issues/36986#issue-614301066
perf_util.tshttps://github.com/angular/angular/blob/master/modules/e2e_util/perf_util.ts
perf_util.ts L37https://github.com/angular/angular/blob/master/modules/e2e_util/perf_util.ts#L37
alxhubhttps://github.com/alxhub
area: benchpresshttps://github.com/angular/angular/issues?q=state%3Aopen%20label%3A%22area%3A%20benchpress%22
on May 8, 2020https://github.com/angular/angular/issues/36986#event-3317691396
ngbothttps://github.com/apps/ngbot
needsTriagehttps://github.com/angular/angular/milestone/83
on May 8, 2020https://github.com/angular/angular/issues/36986#event-3317691469
wagnermacielhttps://github.com/wagnermaciel
on May 8, 2020https://github.com/angular/angular/issues/36986#event-3317724055
atscotthttps://github.com/atscott
on Jun 8, 2020https://github.com/angular/angular/issues/36986#event-3420918091
fix(dev-infra): await setup in runBenchmark (#37428)https://github.com/angular/angular/commit/77efddcac3e68648dbffb439f2ab408709f1f44c
#37428https://github.com/angular/angular/pull/37428
77efddchttps://github.com/angular/angular/commit/77efddcac3e68648dbffb439f2ab408709f1f44c
fix(dev-infra): await setup in runBenchmark (#37428)https://github.com/angular/angular/commit/776c4afc0302ecbf76a4c077a6c70159cb764c0d
#37428https://github.com/angular/angular/pull/37428
776c4afhttps://github.com/angular/angular/commit/776c4afc0302ecbf76a4c077a6c70159cb764c0d
ngwattcoshttps://github.com/ngwattcos
on Jun 25, 2020https://github.com/angular/angular/issues/36986#event-3483789864
fix(dev-infra): await setup in runBenchmark (angular#37428)https://github.com/ngwattcos/angular/commit/165bbe4e3eda2e791a13c17b660c3a01cb15862e
angular#37428https://github.com/angular/angular/pull/37428
165bbe4https://github.com/ngwattcos/angular/commit/165bbe4e3eda2e791a13c17b660c3a01cb15862e
profanishttps://github.com/profanis
on Sep 5, 2020https://github.com/angular/angular/issues/36986#event-3732916000
fix(dev-infra): await setup in runBenchmark (angular#37428)https://github.com/profanis/angular/commit/296af21391e318096763e297bfd6853915cd735b
angular#37428https://github.com/angular/angular/pull/37428
296af21https://github.com/profanis/angular/commit/296af21391e318096763e297bfd6853915cd735b
wagnermacielhttps://github.com/wagnermaciel
completedhttps://github.com/angular/angular/issues?q=is%3Aissue%20state%3Aclosed%20archived%3Afalse%20reason%3Acompleted
on Sep 10, 2020https://github.com/angular/angular/issues/36986#event-3751573704
https://github.com/apps/angular-automatic-lock-bot
angular-automatic-lock-bothttps://github.com/apps/angular-automatic-lock-bot
on Oct 11, 2020https://github.com/angular/angular/issues/36986#issuecomment-706729674
automatic conversation locking policyhttps://github.com/angular/angular/blob/67d80f/docs/GITHUB_PROCESS.md#conversation-locking
angular-automatic-lock-bothttps://github.com/apps/angular-automatic-lock-bot
on Oct 11, 2020https://github.com/angular/angular/issues/36986#event-3863845277
Sign up for freehttps://github.com/signup?return_to=https://github.com/angular/angular/issues/36986
Sign in to commenthttps://github.com/login?return_to=https://github.com/angular/angular/issues/36986
wagnermacielhttps://github.com/wagnermaciel
area: benchpresshttps://github.com/angular/angular/issues?q=state%3Aopen%20label%3A%22area%3A%20benchpress%22
needsTriageNo due datehttps://github.com/angular/angular/milestone/83
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.