Title: Run e2e tests on each PR. by purple4reina · Pull Request #584 · DataDog/datadog-lambda-python · GitHub
Open Graph Title: Run e2e tests on each PR. by purple4reina · Pull Request #584 · DataDog/datadog-lambda-python
X Title: Run e2e tests on each PR. by purple4reina · Pull Request #584 · DataDog/datadog-lambda-python
Description: What does this PR do? Run the serverless-e2e-tests on PR, skipping anything that isn't a python test. First publishes the layer build to sandbox account region us-west-2. Then, after gathering the arn for the new layer, pass that to the e2e tests for testing. Motivation Better to get alerted right away to potential e2e bugs. Testing Guidelines Additional Notes Because this job is triggering job in a different repo, it is quite tough to access the results of that downstream job. As designed right now, a e2e-status job will be created in addition to the e2e-test job. This status job runs only after the e2e-test job completes. The reason this status job is necessary is so that we can post the results back to github. The problem with this is the e2e-status job can take hours to fully complete. That's because not only do all the other tests in this repo need to run (integration, unit, layer size, etc), it also needs to run the e2e tests, and it must also wait for the full teardown of aws resources. The latter is what can take so long because these teardown jobs are set to run after a 2 hour delay. So, we technically aren't able to get results posted back to github until all of that completes. The work around, which was too hard to implement with the time I had, is to use the gitlab api to poll the status of the e2e-test job. This might involve creating a Project Token which is more work that I think it's worth. An alternative solution is to set the e2e-status test as required by github. There are some major problems with this though. There are times in which we would expect that the e2e tests would fail. For example, if I'm implementing a new feature which will turn an e2e test from not working to working, then the e2e tests themselves will always fail in that case, which is what we want. To get around that, we could remove the xfail strict requirement on the e2e tests downstream. But this would mean we aren't able to keep our feature parity doc in lock step with reality. At this point, I think the best thing to do is to mark the e2e-status job as required. We are always able to override these failures in the UI, which shouldn't mean we're moving any slower. Types of Changes Bug fix New feature Breaking change Misc (docs, refactoring, dependency upgrade, etc.) Check all that apply This PR's description is comprehensive This PR contains breaking changes that are documented in the description This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future This PR impacts documentation, and it has been updated (or a ticket has been logged) This PR's changes are covered by the automated tests This PR collects user input/sensitive content into Datadog This PR passes the integration tests (ask a Datadog member to run the tests) https://datadoghq.atlassian.net/browse/APMSVLS-20
Open Graph Description: What does this PR do? Run the serverless-e2e-tests on PR, skipping anything that isn't a python test. First publishes the layer build to sandbox account region us-west-2. Then, after gathering ...
X Description: What does this PR do? Run the serverless-e2e-tests on PR, skipping anything that isn't a python test. First publishes the layer build to sandbox account region us-west-2. Then, after gather...
Opengraph URL: https://github.com/DataDog/datadog-lambda-python/pull/584
X: @github
Domain: github.com
| route-pattern | /:user_id/:repository/pull/:id/checks(.:format) |
| route-controller | pull_requests |
| route-action | checks |
| fetch-nonce | v2:fd757041-6dc3-2ed7-26a3-45b189c52e53 |
| current-catalog-service-hash | 87dc3bc62d9b466312751bfd5f889726f4f1337bdff4e8be7da7c93d6c00a25a |
| request-id | D1DA:12F068:4922D8:5D0CD9:69912D50 |
| html-safe-nonce | 6cd6a08b085bb14b5db17ef3f04e20ab1df4cc3fc01863cc2ce8c55d0ebc87d7 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMURBOjEyRjA2ODo0OTIyRDg6NUQwQ0Q5OjY5OTEyRDUwIiwidmlzaXRvcl9pZCI6Ijc0OTgyMjk4MjkyNjQ4MTMzOTIiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 04fc57d7d5bca8d7b62683f1dc54ba221f975e690a92d1890ca6a376f44e2c96 |
| hovercard-subject-tag | pull_request:2452269612 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,checks,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/DataDog/datadog-lambda-python/pull/584/checks |
| twitter:image | https://avatars.githubusercontent.com/u/1383216?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/1383216?s=400&v=4 |
| og:image:alt | What does this PR do? Run the serverless-e2e-tests on PR, skipping anything that isn't a python test. First publishes the layer build to sandbox account region us-west-2. Then, after gathering ... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 42c603b9d642c4a9065a51770f75e5e27132fef0e858607f5c9cb7e422831a7b |
| turbo-cache-control | no-preview |
| go-import | github.com/DataDog/datadog-lambda-python git https://github.com/DataDog/datadog-lambda-python.git |
| octolytics-dimension-user_id | 365230 |
| octolytics-dimension-user_login | DataDog |
| octolytics-dimension-repository_id | 183265904 |
| octolytics-dimension-repository_nwo | DataDog/datadog-lambda-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 183265904 |
| octolytics-dimension-repository_network_root_nwo | DataDog/datadog-lambda-python |
| turbo-body-classes | logged-out env-production page-responsive full-width full-width-p-0 |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 848bc6032dcc93a9a7301dcc3f379a72ba13b96e |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width