ava assertions do not halt test

[andykais]

I wrote a test and was confused because despite ava telling me there was a failure at a particular assertion, the test kept execution. I would have expected the test to short circuit at a failed assertion and throw an error. Instead, it seems to just continue and record that a failure happened somewhere. I dont know if this is expected behavior or a bug. If it is expected, please tell me how to disable it.

Sample code:

import test from 'ava'

test('sample test', async t => {
  t.is('me', 'you')
  console.log('arrived here')
})

test output:

 andrew  project  npx ava test.ts 

arrived here
  sample test

  test.ts:4

   3: test('sample test', async t => {
   4:   t.is('me', 'you')             
   5:   console.log('arrived here')   

  Difference:

  - 'me'
  + 'you'

  › test.ts:4:5

  ─

  1 test failed

you can see the line arrived here is printed, though it shouldnt be.

ava config:

{
  "ava": {
    "extensions": [
      "ts"
    ],
    "require": [
      "ts-node/register"
    ],
    "ignoredByWatcher": [
      "scratchwork/*",
      "test/fixtures/*"
    ]
  }
}

ava version:

3.15.0

[novemberborn]

This is by design. Ultimately we want to report all failed assertions, not just the first one. It shouldn't typically be an issue but this behavior can be surprising when you first encounter it.

AVA 4 updates some assertions to return booleans, for those use cases (e.g. to help with type narrowing when using TypeScript) where it's helpful not to continue.

[andykais]

I dont think I can use ava if this is by design 😞. If this behavior were opt-in or even opt-out, then I could manage. The way I write all my tests though is assuming the previous line succeeded. Adding if statements all over tests is going to be too arduous and verbose

[novemberborn]

The way I write all my tests though is assuming the previous line succeeded. Adding if statements all over tests is going to be too arduous and verbose

Of course, I do the same. It's probably only some specific use cases where executing assertions conditionally is useful.

However I find that when you have a series of assertions it doesn't really matter whether the test tries to run a few more. Are you doing something more complicated in the test that it really matters for execution to stop?

[andykais]

I'll admit this poses a question I haven't given a lot of thought to. I would say conversely though, is there. any use in continuing to execute a test after a failure has occurred? I guess it helps with testing unexpected states.

The big reasons I think of are that my library does a lot of io which takes time. It also uses typescript, which adds to the setup time. When I run ava test.ts --watch, I want tests to start & finish as quickly as possible for fast iteration. When all my tests run on CI, I want the shortest amount of output that tells me what went wrong with a test. If I run 100 tests, failing fast might mean the difference in 3x the output.

[novemberborn]

is there. any use in continuing to execute a test after a failure has occurred?

Admittedly it'd be more useful if we showed all assertion failures. Node Tap has the same behavior.

There is a --fail-fast option which stops AVA from running new tests once there has been a failure. However due to AVA's concurrent and asynchronous nature this is not a "halt everything immediately" mode.