Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

flaky-test: bifrost::tests::trim_log_smoke_test panicked at crates/bifrost/src/bifrost.rs:104:14 #2857

Open
lsytj0413 opened this issue Mar 7, 2025 · 2 comments

Comments

@lsytj0413
Copy link
Contributor

When run test with cargo test -p restate-bifrost --lib -- --nocapture --test-threads=1, it failed:

test bifrost::tests::trim_log_smoke_test ... !!!! Runnning with test-util enabled !!!!
2025-03-07T09:32:12.849393Z TRACE restate_bifrost::service: Starting loglet provider local
2025-03-07T09:32:12.849465Z ERROR restate_bifrost::service: Failed to start loglet provider local: system is shutting down

thread 'bifrost::tests::trim_log_smoke_test' panicked at crates/bifrost/src/bifrost.rs:104:14:
in memory loglet must start: Failed to start loglet provider local: system is shutting down
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
@tillrohrmann
Copy link
Contributor

This is probably caused by cargo test not splitting individual tests into individual processes but reusing the process. We still have a few globals like the DB_MANAGER (

static DB_MANAGER: OnceLock<RocksDbManager> = OnceLock::new();
). That's why we use nextest to run our tests. Try cargo nextest run --all-features --target-dir target/tests -p restate-bifrost and let me know whether the tests are now passing.

@lsytj0413
Copy link
Contributor Author

This is probably caused by cargo test not splitting individual tests into individual processes but reusing the process. We still have a few globals like the DB_MANAGER (

restate/crates/rocksdb/src/db_manager.rs

Line 31 in 41e606e

static DB_MANAGER: OnceLock = OnceLock::new();
). That's why we use nextest to run our tests. Try cargo nextest run --all-features --target-dir target/tests -p restate-bifrost and let me know whether the tests are now passing.

Yep, it's caused by DB_MANAGER was a global variables and reused by multi testcase, so all the testcases interfered with each other. Use nextest didn't reproduced this issue.

Is there any plan to reform the use of global variables?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants