New in v3.1 brings streamlined and powerful SQL caching and storage options, enabling efficient data handling and retrieval in SQLite format alongside existing DataFrame support. This release optimizes workflows for handling large datasets by storing them in a structured SQL database, reducing memory usage and improving access times.
Highlights
-
SQL Save and Load Support:
Addedsave_df_as_sql
andload_from_sqlite
utility functions to easily save and retrieve data from SQLite databases. Now, users can seamlessly choose between DataFrame or SQL-based caching. -
Enhanced
Fetcher
Class:
Updated theFetcher
class to support SQL caching, allowing automatic storage and retrieval of data from SQL databases. This enhances flexibility, especially for larger datasets. -
Generalized Cache Handling:
Streamlined cache handling logic for both DataFrame and SQL formats, removing redundant methods and using a unified approach for efficient data management. -
Dependency Updates:
Updatedpoetry.lock
to includepackaging
version 24.2 for compatibility and performance improvements.
With these upgrades, bcrpy v3.1 is now more efficient for large-scale data projects, providing robust caching and storage options to meet diverse needs.
Upgrade Notes:
Existing projects using previous versions should update any custom caching workflows to leverage the new SQL functions and streamlined cache handling in Fetcher
.