Concurrent queries

[loicalleyne]

Very nice concept, was wondering if you've considered keeping the db connection open and implementing a connection pool to allow multiple queries to run at the same time?

[lmangani]

Thanks! Absolutely we could add a session mode with a file as coordinator to make this possible. What's your usecase? I assume you're working with actual db files as opposed to just querying remote parquets, etc?

[loicalleyne]

The use case I have in mind currently is a lightweight reverse ETL to make aggregated data available to internal tools used by multiple users at once.

[lmangani]

We can implement the same method we had in chdb where providing auth produces a semi-persistent query session allowing multiple interactions as opposed to running individual queries. Will add to the roadmap.

PS: If you'd like to PR this feature, please make yourself home! @loicalleyne

[lmangani]

@loicalleyne this is now partially supported through authenticated sessions which use filesystem storage so it should be fine for reading in parallel (not writing yet, WIP)