Skip to content

Commit

Permalink
Change max to abs for preprocess time (#240)
Browse files Browse the repository at this point in the history
`normal` result can be negative.
if we do `max(t, 0) then we will reduce the variance when the generated value is negative.
It is better to use `abs` in this case.
  • Loading branch information
rayandrew authored Oct 30, 2024
1 parent c9225fb commit 8614311
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions dlio_benchmark/reader/reader_handler.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ def __init__(self, dataset_type, thread_index):

@dlp.log
def preprocess(self, a=None):
if self._args.preprocess_time != 0. or self._args.preprocess_time_stdev != 0.:
if self._args.preprocess_time > 0. or self._args.preprocess_time_stdev > 0.:
t = np.random.normal(self._args.preprocess_time, self._args.preprocess_time_stdev)
sleep(max(t, 0.0))
sleep(abs(t))
return a
@abstractmethod
def open(self, filename):
Expand Down Expand Up @@ -151,4 +151,4 @@ def is_index_based(self):

@abstractmethod
def is_iterator_based(self):
return False
return False

0 comments on commit 8614311

Please sign in to comment.