Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Block on dropping SyncWrapper. #330

Open
xuxiaocheng0201 opened this issue Jun 7, 2024 · 3 comments
Open

Block on dropping SyncWrapper. #330

xuxiaocheng0201 opened this issue Jun 7, 2024 · 3 comments
Labels
A-sqlite Area: SQLite / deadpool-sqlite enhancement New feature or request

Comments

@xuxiaocheng0201
Copy link

When I removing the sqlite db file, I occurred an os error 32.
Error: 另一个程序正在使用此文件,进程无法访问。 (os error 32)
It means Another program is using this file and the process cannot access it.
This is a common error in windows.

I have found the code that caused this problem.
In SyncWrapper, the connection resource will be dropped in a background task. However, I have no way of knowing whether this task has been completed or not. So if I remove the db file immediately after dropping the pool, this error will occur.

So is there any way to ensure the connections is completely dropped? or block on when the SyncWrapper is dropping?

Reproduce:
Cargo.toml:

anyhow = "^1.0"
tokio = { version = "^1.0", features = ["macros", "rt-multi-thread"] }
rusqlite = { version = "~0.31", features = ["bundled"] }
r2d2_sqlite = "~0.24"
r2d2 = "~0.8"
deadpool-r2d2 = "~0.4"

main.rs:

use std::fs::remove_file;
use anyhow::Result;
use deadpool_r2d2::Runtime;
use r2d2_sqlite::SqliteConnectionManager;

pub type Manager = deadpool_r2d2::Manager<SqliteConnectionManager>;
pub type Pool = deadpool_r2d2::Pool<Manager>;

#[tokio::main]
async fn main() -> Result<()> {
    // Create the sqlite database pool.
    let pool = Pool::builder(Manager::new(SqliteConnectionManager::file("1.db"), Runtime::Tokio1)).build()?;

    // Perform a simple query.
    pool.get().await?.interact(|c| c.execute("CREATE TABLE db(id INTEGER);", [])).await.unwrap()?;

    // Close the pool.
    pool.close();
    drop(pool);

    // Remove the database file.
    remove_file("1.db")?; // Error here.
    Ok(())
}
@xuxiaocheng0201
Copy link
Author

I tried my best to resolve the question:

while let Err(e) = tokio::fs::remove_file("1.db").await {
    tokio::task::yield_now().await;
}

But it's not perfect.

@bikeshedder bikeshedder added enhancement New feature or request A-sqlite Area: SQLite / deadpool-sqlite labels Jun 8, 2024
@bikeshedder
Copy link
Owner

I've never thought about that but it makes perfect sense to have that feature. You basically want to wait for all objects returned from the pool to be dropped.

I'm currently thinking of adding a return value to Pool::close which gives you the ability to .await it.

I can think of two ways to implement this:

  • Adding a counter to the pool which reports how many objects are currently "ready to be used". This counter is increased by one whenever a new object is created and reduced by one after a object got dropped. Once the counter reaches 0 there should be no connections whatsoever.
  • Adding a tokio::sync::watch to all objects so they can report their state (Ready, Dropping, Dropped). For that to work the pool would need to keep a reference to all the objects it created. Similar to the way deadpool-postgres keeps track of the objects so that the StatementCaches can be empied via the manager.

Even though it's a lot more involved I'm almost leaning towards the second implementation. It always bugged me a bit that deadpool-postgres keeps track of the StatementCaches and not the entire object. e.g. It would be nice if it was also possible to access the Metrics of objects which are currently not living inside the pool.

@bikeshedder
Copy link
Owner

Related to:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-sqlite Area: SQLite / deadpool-sqlite enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants