sync up (#205)

* Request changes from MLPerf Storage (#199)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* Fixed potential insufficient samples due to num_files is not divisible by comm.size (#200)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* recovered back dlio_profiler

* fixed potential not enough samples

* Update tf_reader.py

* Mlperf requests (#201)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* fixed issue with dlio_profiler

* bring back dlio_profiler_py

* Bring v1.0 to the most recent commit  (#202) (#203)

* Request changes from MLPerf Storage (#199)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* Fixed potential insufficient samples due to num_files is not divisible by comm.size (#200)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* recovered back dlio_profiler

* fixed potential not enough samples

* Update tf_reader.py

* Mlperf requests (#201)

* added au metric to the configuration file; set shuffling and shuffle buffer size to be 2 for cosmoflow

* removed dependencies on dlioprofiler

* fixed bugs

* fixed issue with dlio_profiler

* bring back dlio_profiler_py

* Fix requirements file (#204)

Signed-off-by: Johnu George <[email protected]>

---------

Signed-off-by: Johnu George <[email protected]>
Co-authored-by: Johnu George <[email protected]>

requirements.txt

@@ -20,7 +20,7 @@ Markdown>=3.4.1
 MarkupSafe>=2.1.1
 mpi4py>=3.1.4
 numpy>=1.23.5
---extra-index-url https://pypi.nvidia.com nvidia-dali-cuda110>=1.34.0
+nvidia-dali-cuda110>=1.34.0
 oauthlib>=3.2.2
 omegaconf>=2.2.3
 opt-einsum>=3.3.0