Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: --fail-on-incomplete doesn't cause exit code to be non-zero when snapshots are missing #1328

Open
tangrufus opened this issue Dec 14, 2024 · 1 comment · May be fixed by #1348
Open
Labels

Comments

@tangrufus
Copy link

tangrufus commented Dec 14, 2024

What Happened

--fail-on-incomplete doesn't cause exit code to be non-zero when snapshots are missing.

How to Reproduce

  1. Create a test with toMatchSnapshot
    test('example', function () {
        expect('foo')->toMatchSnapshot();
    });
  2. Delete the snapshots folder (if exists)
    $ rm -rf tests/.pest/snapshots
  3. Run the tests
    $ ./vendor/bin/pest --fail-on-incomplete
    
       WARN  Tests\Feature\ExampleTest
      … example → Snapshot created at [tests/.pest/snapshots/Feature/ExampleTest/example.snap]                                                                                                                   0.01s
      ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       INCOMPLETE  Tests\Feature\ExampleTest > example
      Snapshot created at [tests/.pest/snapshots/Feature/ExampleTest/example.snap]
    
    
      Tests:    1 incomplete (0 assertions)
      Duration: 0.07s
  4. Check the exit code
    $ echo $?
    0

Sample Repository

https://github.com/tangrufus/pest-fail-on-incomplete

Pest Version

3.7.1

PHP Version

8.4.1

Operation System

macOS

Notes

In the above example, exit code is 0 even with all of the --fail-on-xxx flags.

$ ./vendor/bin/pest --fail-on-empty-test-suite --fail-on-warning --fail-on-risky --fail-on-deprecation --fail-on-phpunit-deprecation --fail-on-notice --fail-on-skipped --fail-on-incomplete
@tangrufus tangrufus added the bug label Dec 14, 2024
@tangrufus tangrufus changed the title [Bug]: --fail-on-incomplete doesn't cause exit code to be non-zero when snapshot created [Bug]: --fail-on-incomplete doesn't cause exit code to be non-zero when snapshots created Dec 14, 2024
@tangrufus tangrufus changed the title [Bug]: --fail-on-incomplete doesn't cause exit code to be non-zero when snapshots created [Bug]: --fail-on-incomplete doesn't cause exit code to be non-zero when snapshots are missing Dec 14, 2024
@mdeboer mdeboer linked a pull request Jan 29, 2025 that will close this issue
2 tasks
@mdeboer
Copy link

mdeboer commented Jan 29, 2025

We ran into the same issue surprisingly. I took the liberty to open a PR which proposes a fix and also includes a test case for this scenario. Hopefully this gets fixed soon 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants