Improve performances when wait_before_drop is 0 #8925

Job	Run time
Typos Check	5s
Lints and doc tests on ubuntu-latest	8m 37s
Lints and doc tests on windows-latest	11m 2s
Unit tests on ubuntu-latest	20m 27s
Lints and doc tests on macos-13	12m 26s
Unit tests on windows-latest	28m 45s
Lints and doc tests on macos-14	5m 17s
Unit tests on macos-13	14m 0s
Unit tests on macos-14	11m 36s
Memory leak checks	2m 7s
CI status checks	1s
	1h 54m 23s

Provide feedback