René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"A failing test doesn't fail the build","articleBody":"\u003c!--- Provide a general summary of the issue in the Title above --\u003e\r\n\r\n## Expected Behavior\r\n\u003c!--- Tell us what should happen --\u003e\r\n\r\nIf a browser fails to spawn or a test fails in one of the browsers, the process should exit with code different than 0.\r\n\r\n## Actual Behavior\r\n\u003c!--- Tell us what happens instead of the expected behavior --\u003e\r\n\r\nThe process always exits with code 0.\r\n\r\n## Steps to Reproduce the Problem\r\n\r\n  1. `git clone git@github.com:jquery/jquery-color.git`\r\n  2. `npm install`\r\n  3.  Now, modify one of the test cases in `test/unit/color.js` so that it fails, e.g. by changing one 255 to 254.\r\n  4. Run `npm run browserstack; echo $?`\r\n\r\nYou should see a failure \u0026 a number different than 0 as the last output. 0 is printed instead.\r\n\r\n## browserstack.json\r\n\u003c!-- Provide your browserstack.json, remember to exclude your browserstack username and access key.--\u003e\r\n\r\n## Platform details\r\n \r\n  1. browserstack-runner version: 0.9.1\r\n  2. node version: v12.16.3\r\n  3. os type and version: macOS 10.15.4 (19E287)\r\n\r\n## Details\r\n\u003c!--- Provide a detailed description of the issue that you are facing. --\u003e\r\n\r\nThis is quite a critical bug to me as it can mask errors by exiting the CI successfully. If one doesn't notice, this may lead to releasing a broken version of a project.","author":{"url":"https://github.com/mgol","@type":"Person","name":"mgol"},"datePublished":"2020-05-08T09:18:31.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/227/browserstack-runner/issues/227"}