Add a utility to get a local directory that can be used as a persistent cache #1053
Labels
part:core
Affects the SDK core components (data structures, etc.)
priority:❓
We need to figure out how soon this should be addressed
type:enhancement
New feature or enhancement visitble to users
Milestone
What's needed?
We need to be able to access some directory that can be used to store cache data that can persist application restarts or re-deployment.
Proposed solution
Add an utility function like
def get_cache_dir(unique_cache_key: str) -> pathlib.Path
. Where this utility function should live is still to be determine.For now the SDK should have some initialization parameter called
cache_dir_root
or similar, and/or being able to get this configuration from someSDK_CACHE_DIR_ROOT
(we could also have some fallback default using the XDG standard, like$XDG_CACHE_HOME
).Use cases
A typical use case is to cache data that actors or applications need for startup. For example, to do some predictions we need to collect data for some time and after enough data is collected, we can start predicting. If the application is restarted, this data collection period kicks in again, which can leave the app effectively not doing its job for a non trivial amount of time (warm-up period).
By using a cache, we can save the current status and pick up right were we left it when restarting, so we can skip this warm-up period.
Alternatives and workarounds
No response
Additional context
For cloud apps we might eventually want to use something else than a raw filesystem, like a object store (like AWS S3 compatible storage) for scalability reasons. If we move in that direction, this approach will not be enough, as apps/actors will need to act on a per-file basis, explicitly saving and retrieving from the object store.
The text was updated successfully, but these errors were encountered: