[ASoC 2022] Enable data caching cross jobs to boost job performance with high memory efficiency #252
Labels
asoc2022
Alibaba Summer of Code, 2022
community
Community discussions
enhancement
New feature or request
What would you like to be added:
Why is this needed:
Caching datasets in memory of the local cluster helps to accelerate the training jobs. Typically, popular and public datasets might be used by multiple jobs. Therefore, it helps improve the caching efficiency to make datasets sharable across training jobs with a well-designed caching policy.
The text was updated successfully, but these errors were encountered: