Title: allure-karate: afterScenario hook failures can leave scenarios reported as passed · Issue #1446 · allure-framework/allure-java · GitHub
Open Graph Title: allure-karate: afterScenario hook failures can leave scenarios reported as passed · Issue #1446 · allure-framework/allure-java
X Title: allure-karate: afterScenario hook failures can leave scenarios reported as passed · Issue #1446 · allure-framework/allure-java
Description: What happened? A Karate scenario with a passing body and a failing configure afterScenario hook can be reported by Allure as passed, without status details or a reported teardown failure. The hook does execute, and Karate records its exc...
Open Graph Description: What happened? A Karate scenario with a passing body and a failing configure afterScenario hook can be reported by Allure as passed, without status details or a reported teardown failure. The hook ...
X Description: What happened? A Karate scenario with a passing body and a failing configure afterScenario hook can be reported by Allure as passed, without status details or a reported teardown failure. The hook ...
Opengraph URL: https://github.com/allure-framework/allure-java/issues/1446
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"allure-karate: afterScenario hook failures can leave scenarios reported as passed","articleBody":"### What happened?\n\nA Karate scenario with a passing body and a failing `configure afterScenario` hook can be reported by Allure as **passed**, without status details or a reported teardown failure.\n\nThe hook does execute, and Karate records its exception. The Allure adapter finalizes the scenario before that exception is added to Karate's result. This is a reporting correctness problem: scenario cleanup can fail without affecting the reported scenario outcome.\n\n### Steps to reproduce\n\nThe repository already contains this [reproduction feature](https://github.com/allure-framework/allure-java/blob/813f4b90b5e8a06865473af8bea5721900e1bfd6/allure-karate/src/test/resources/testdata/global-errors/scenario-and-feature-hooks-failure.feature):\n\n```gherkin\nFeature: Scenario and feature cleanup failures\n\nBackground:\n * configure afterScenario = function(){ throw new Error('scenario cleanup failed') }\n * configure afterFeature = function(){ throw new Error('feature cleanup failed') }\n\nScenario: Scenario with both cleanup hooks\n * match 1 == 1\n```\n\nRun the existing test through Allure agent mode:\n\n```sh\nnpx --yes allure@3.17.0 agent --report off \\\n --results-dir allure-karate/build/allure-results \\\n --expect-tests 1 \\\n --expect-prefix io.qameta.allure.karate. \\\n -- ./gradlew --no-daemon :allure-karate:cleanTest :allure-karate:test \\\n --tests io.qameta.allure.karate.AllureKarateGlobalErrorsTest.shouldReportOnlyFeatureHookFailuresAsGlobals\n```\n\nOpen the generated agent `index.md`, then inspect the test's attached inner Karate `TestResult` and globals payloads.\n\n**Important:** the existing outer JUnit test checks global-error scoping and therefore passes. It does not assert that the inner scenario's teardown failure is reported correctly.\n\n### Actual behavior\n\nThe captured Allure scenario result has these relevant fields:\n\n```json\n{\n \"name\": \"Scenario with both cleanup hooks\",\n \"status\": \"passed\"\n}\n```\n\nThere are no status details or failed teardown steps for `scenario cleanup failed`. The configuration steps and the ordinary `match` step are reported as passed.\n\nThe separate `afterFeature` exception is correctly present in globals as `feature cleanup failed`. It is included in the reproducer to demonstrate the distinction between scenario-scoped and feature-scoped cleanup failures.\n\n### Expected behavior\n\n- A failing `afterScenario` hook must be associated with its own scenario and produce an appropriate non-passing outcome, with its exception details available in the report.\n- If the scenario body also fails, the primary failure and the additional cleanup failure should both remain visible.\n- Per-scenario hook failures must not be converted into framework-wide global errors.\n- Shared `afterFeature` and `afterScenarioOutline` failures must remain separate from the scenario outcome.\n\n### Investigation\n\nIn Karate 2.1.2, the relevant order in `ScenarioRuntime.call()` is:\n\n1. Emit `SCENARIO_EXIT`.\n2. Enter the `finally` block.\n3. Invoke `afterScenario` and append its hook result, including any exception.\n\nIn [`AllureKarate.afterScenario()`](https://github.com/allure-framework/allure-java/blob/813f4b90b5e8a06865473af8bea5721900e1bfd6/allure-karate/src/main/java/io/qameta/allure/karate/AllureKarate.java#L284-L329), the adapter removes the scenario context, reads the outcome, and calls `stopTest` and `writeTest` while handling step 1. The later hook failure therefore arrives after the Allure result has been finalized.\n\nThe feature-completion global-error handling intentionally excludes per-scenario hook results. That is the correct scope distinction, but it does not repair the premature scenario finalization.\n\nA fix needs a completion point after scenario teardown, while still separating shared-hook errors from scenario errors. Merely copying the final Karate scenario status at feature completion would also be unsafe, because Karate appends shared-hook failures to scenario results too.\n\nRegression coverage should include a passing body with failing teardown, a failing body with failing teardown, report suppression (including suppressed callees), and parallel/nested execution. Any hook logs or attachments should also be checked when changing the completion boundary.\n\n### What Allure Integration are you using?\n\nallure-karate\n\n### What version of Allure Integration you are using?\n\n3.1-SNAPSHOT, development revision `813f4b90b5e8a06865473af8bea5721900e1bfd6`, with `io.karatelabs:karate-core:2.1.2`. Observed using JDK 25. This report does not claim that every released adapter version is affected.\n\n### What version of Allure Report you are using?\n\n3.17.0. The incorrect outcome is already present in the captured Allure result JSON, before report rendering.\n\n### Duplicate check\n\nSearched open and closed issues for Karate, `afterScenario`, `SCENARIO_EXIT`, and hook/status symptoms. No matching report of this lifecycle-ordering problem was found.\n\n### Code of Conduct\n\n- [x] I agree to follow this project's Code of Conduct\n","author":{"url":"https://github.com/andrcuns","@type":"Person","name":"andrcuns"},"datePublished":"2026-09-18T14:39:15Z","url":"https://github.com/allure-framework/allure-java/issues/1446"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:00851a2b-42f1-de60-0529-8194f76ea39a |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | D56C:9905:39EA9:51C27:6AFD481E |
| html-safe-nonce | 8ff64acb1b1d8e7fb83f5d13169dd02873cb0c2fdbc87efbe6ad4915e36091b5 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJENTZDOjk5MDU6MzlFQTk6NTFDMjc6NkFGRDQ4MUUiLCJ2aXNpdG9yX2lkIjoiMjMwOTgxNTYwNTYyNDM2MTI2IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | cd7e6b4d8cd8e79bb24484fea4722136f598ceb19c587ac9cabb9bf7ebf8827f |
| hovercard-subject-tag | issue:5500866750 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/allure-framework/allure-java/1446/issue_layout |
| twitter:image | https://opengraph.githubassets.com/21a8bfbe56d4133141ab4a217d3e58792eef292b1262df353d333a8102f2ebf1/allure-framework/allure-java/issues/1446 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/21a8bfbe56d4133141ab4a217d3e58792eef292b1262df353d333a8102f2ebf1/allure-framework/allure-java/issues/1446 |
| og:image:alt | What happened? A Karate scenario with a passing body and a failing configure afterScenario hook can be reported by Allure as passed, without status details or a reported teardown failure. The hook ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | andrcuns |
| hostname | github.com |
| expected-hostname | github.com |
| None | bdbacc25b921224ff0befcb9e629560a1255be51aec9991d6e3f95b5e9151474 |
| turbo-cache-control | no-preview |
| go-import | github.com/allure-framework/allure-java git https://github.com/allure-framework/allure-java.git |
| octolytics-dimension-user_id | 5879127 |
| octolytics-dimension-user_login | allure-framework |
| octolytics-dimension-repository_id | 80822758 |
| octolytics-dimension-repository_nwo | allure-framework/allure-java |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 80822758 |
| octolytics-dimension-repository_network_root_nwo | allure-framework/allure-java |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 556dd37bb0aceffb25724e151b5d858b3265b40b |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width