DataLoad Manager is a module of DataStor, which is a cloud-native local storage system acceleration solution in AI scenarios. It combines p2p technology to provide the ability to quickly load remote data.
DataloadManager supports multiple data loading protocols: s3, nfs, ftp, http, ssh
In AI data training scenarios, data can be loaded into local cache volumes faster. Especially when the data set supports s3 protocol pull, p2p technology can be combined to significantly improve data loading.
DataLoad Manager is a component of HwameiStor and must work with the DataLoad Manager module.