Title: Add support for coroutine-based concurrency · Issue #720 · allure-framework/allure-python · GitHub
Open Graph Title: Add support for coroutine-based concurrency · Issue #720 · allure-framework/allure-python
X Title: Add support for coroutine-based concurrency · Issue #720 · allure-framework/allure-python
Description: Currently, we don`t support running allure-python in an async function (e.g. through asyncio event loop or a similar coroutine execution engine). This is important, because Async-based frameworks and libraries are growing in popularity (...
Open Graph Description: Currently, we don`t support running allure-python in an async function (e.g. through asyncio event loop or a similar coroutine execution engine). This is important, because Async-based frameworks a...
X Description: Currently, we don`t support running allure-python in an async function (e.g. through asyncio event loop or a similar coroutine execution engine). This is important, because Async-based frameworks a...
Opengraph URL: https://github.com/allure-framework/allure-python/issues/720
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Add support for coroutine-based concurrency","articleBody":"Currently, we don\\`t support running allure-python in an async function (e.g. through asyncio event loop or a similar coroutine execution engine).\r\n\r\nThis is important, because\r\n1. Async-based frameworks and libraries are growing in popularity (`aiohttp`, `aiofile`, `aiopg`, etc.), so we need to support testing asynchronous code\r\n2. E2E tests are heavily IO-bound. The ability to run them concurrently may improve tests performance significantly.\r\n\r\n#### Coroutines and contexts\r\nA coroutine in python is created with a coroutine declaration, i.e. a function with the `async` keyword. Inside a function the `await` keyword can be used to wait for an asynchronous operation to complete. A coroutine execution engine can then pause the coroutine and resume execution of another one, increasing performance of IO-bound computations.\r\n\r\nWithout proper means a coroutine paused inside a context manager may leak unintended global state into another coroutine.\r\n\r\n#### Basic illustration\r\n\r\nSuppose we have the following abstract code:\r\n```python\r\nasync def f():\r\n with change_global_state():\r\n await long_io_operation()\r\n\r\nasync def g():\r\n use_global_state()\r\n```\r\n\r\nIf we create two coroutines (one with `f()` and the other one with `g()`) and schedule both coroutines for execution, it is possible that `g` will use the state, not expected to be visible outside `f`.\r\n\r\nBelow is an example, how both coroutines could be run with `asyncio`:\r\n\r\n```python\r\nimport asyncio \r\n\r\nasync def execute_all():\r\n await asyncio.gather(f(), g())\r\n\r\nasyncio.run(execute_all())\r\n```\r\n\r\nThere are two general scenarios we can face while generating allure report. They are described below.\r\n\r\n### Scenario 1: running multiple tests concurrently\r\nThis scenario requires a pytest plugin capable of executing async tests concurrently, like [pytest-asyncio-cooperative](https://github.com/willemt/pytest-asyncio-cooperative). Unfortunately, this particular plugin currently doesn\\`t work with allure-pytest, so the remaining content in this section describes more of a theoretical case, as if the plugin actually works.\r\n\r\nIf we assume the plugin works, the test may looks like this ([also, see here](https://github.com/allure-framework/allure-python/blob/a86a8f4172681257bc64eb3ff8feeb78592090c2/allure-pytest/test/acceptance/concurrency/async_concurrency_test.py#L51)):\r\n```python\r\nimport allure\r\nimport asyncio\r\nimport pytest\r\n\r\n@pytest.fixture(scope=\"session\")\r\nasync def fence():\r\n yield asyncio.Event()\r\n\r\n@pytest.mark.asyncio_cooperative\r\nasync def test_with_async_operation(fence):\r\n with allure.step(\"Wait for a fence\"):\r\n await fence.wait()\r\n\r\n@pytest.mark.asyncio_cooperative\r\nasync def test_without_async_operation(fence):\r\n with allure.step(\"Setting a fence\"):\r\n fence.set()\r\n```\r\n\r\nAfter `test_with_async_operation` is paused, the event loop starts executing `test_without_async_operation`. The step `\"Setting a fence\"` is created inside the `\"Wait for a fence\"\"` step context, emitting an invalid test result: the first test result contains two nested steps instead of one, while the second one contains no steps whatsoever.\r\n\r\n### Scenario 2: running multiple steps concurrently inside one test\r\nThis one is easily achieved either with a self-written fixture, or with [pytest-asyncio](https://github.com/pytest-dev/pytest-asyncio).\r\n\r\nGiven we have pytest-asyncio installed and the following code:\r\n```python\r\nimport pytest\r\nimport allure\r\nimport asyncio\r\n\r\n@pytest.mark.asyncio\r\nasync def test_with_concurrent_steps():\r\n fence = asyncio.Event()\r\n async def run_with_step(name):\r\n with allure.step(name):\r\n await fence.wait()\r\n async def release_fence():\r\n fence.set()\r\n await asyncio.gather(\r\n run_with_step(\"Step 1\"),\r\n run_with_step(\"Step 2\"),\r\n release_fence()\r\n )\r\n```\r\n\r\nNote: the steps could've been executed with `asyncio`'s `create_task` function instead with the same effect.\r\n\r\nThe following `*result.json` is generated (most unrelated fields are omitted):\r\n```json\r\n{\r\n \"name\": \"test_with_concurrent_steps\",\r\n \"status\": \"passed\",\r\n \"steps\": [{\r\n \"name\": \"Step 1\",\r\n \"status\": \"passed\",\r\n \"steps\": [{\r\n \"name\": \"Step 2\",\r\n \"status\": \"passed\"\r\n }]\r\n }]\r\n}\r\n```\r\n\r\nAnd it is shown in the report like this:\r\n\r\n\r\nThe steps are nested one into another instead of being independent as was intended.\r\n\r\n#### Workaround\r\nTo fix the report, change you code so the functions that create steps are executed synchronously:\r\n\r\n```python\r\nimport pytest\r\nimport allure\r\nimport asyncio\r\n\r\n@pytest.mark.asyncio\r\nasync def test_with_concurrent_steps():\r\n fence = asyncio.Event()\r\n async def run_with_step(name):\r\n with allure.step(name):\r\n await fence.wait()\r\n async def release_fence():\r\n fence.set()\r\n await release_fence()\r\n await run_with_step(\"Step 1\")\r\n await run_with_step(\"Step 2\")\r\n```\r\n\r\nThe downside is slower execution and the need to rewrite the logic in your code.\r\n\r\nAlso, see [this test](https://github.com/allure-framework/allure-python/blob/707c304677a2a5ac1b4753a2b219fb4b953b3de2/allure-pytest/test/acceptance/concurrency/async_concurrency_test.py#L7).\r\n\r\n### Thought on possible implementation\r\nWe probably should utilize ContextVars, they were introduced for the very this issue (see [PEP567](https://peps.python.org/pep-0567/)). Cons: they require python 3.7. If we decide to move this way, we may either declare async concurrency supported in python 3.7+ only or drop python 3.6 support completely (our latest decision was to focus on 3.7+ while trying to stay 3.6-compatible, if possible).\r\n\r\nCan\\`t think of another possible way for now.","author":{"url":"https://github.com/delatrie","@type":"Person","name":"delatrie"},"datePublished":"2022-12-19T20:07:11Z","url":"https://github.com/allure-framework/allure-python/issues/720"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:636346a0-e487-2a52-8c89-b797ef0faf44 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | CAF4:21AF62:8764C:BD6E9:6AFD4A47 |
| html-safe-nonce | 97ea93db4a4906472f8e690cffda22da5afd589d9fd88e752cd96a0bf9411836 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDQUY0OjIxQUY2Mjo4NzY0QzpCRDZFOTo2QUZENEE0NyIsInZpc2l0b3JfaWQiOiIxMDg1NDM1MzEyMzQ1MDA4NzExIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 4455fced1b75bf66d508dffb851f4855bd331f6ae0aed71cfdcacd0867c38081 |
| hovercard-subject-tag | issue:1503520344 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/allure-framework/allure-python/720/issue_layout |
| twitter:image | https://opengraph.githubassets.com/751d06dff60f9c877bad5580c23d78cf277e8e8e216bc490c5ac24070be8325b/allure-framework/allure-python/issues/720 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/751d06dff60f9c877bad5580c23d78cf277e8e8e216bc490c5ac24070be8325b/allure-framework/allure-python/issues/720 |
| og:image:alt | Currently, we don`t support running allure-python in an async function (e.g. through asyncio event loop or a similar coroutine execution engine). This is important, because Async-based frameworks a... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | delatrie |
| hostname | github.com |
| expected-hostname | github.com |
| None | 66653c2e55a6fff2930be1bdf07ee4f8aa137e3956d056c822563be6f878755e |
| turbo-cache-control | no-preview |
| go-import | github.com/allure-framework/allure-python git https://github.com/allure-framework/allure-python.git |
| octolytics-dimension-user_id | 5879127 |
| octolytics-dimension-user_login | allure-framework |
| octolytics-dimension-repository_id | 79346720 |
| octolytics-dimension-repository_nwo | allure-framework/allure-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 79346720 |
| octolytics-dimension-repository_network_root_nwo | allure-framework/allure-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | b6f01f7cb9f35f8c09949773241a3d49dd04b340 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width