Skip to content

Commit

Permalink
Make __getitem__ for raw imageseries threadsafe
Browse files Browse the repository at this point in the history
This ensures that for multithreaded situations (such as writing out
a frame cache), `__getitem__` will be thread-safe. The change ensures
that the frame will be obtained immediately after seeking, and no other
thread can seek until the frame is obtained.

When I test the example Darren gave us (#608) on the master branch with
no multithreading, it ran in 5m 36.826s. With this change, and using
multithreading, it ran in 5m 12.811s. So the multithreading produced
a minor speed increase.

Fixes: #608

Signed-off-by: Patrick Avery <[email protected]>
  • Loading branch information
psavery committed Jan 25, 2024
1 parent d60f265 commit cc00d7d
Showing 1 changed file with 8 additions and 2 deletions.
10 changes: 8 additions & 2 deletions hexrd/imageseries/load/rawimage.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
""" Adapter class for raw image reader"""
import os
import threading

import numpy as np
import yaml
Expand Down Expand Up @@ -30,6 +31,7 @@ def __init__(self, fname, **kwargs):
self._shape = tuple((int(si) for si in y['shape'].split()))
self._frame_size = self._shape[0] * self._shape[1]
self._frame_bytes = self._frame_size * self.dtype.itemsize
self._frame_read_lock = threading.Lock()
self.skipbytes = y['skip']
self._len = self._get_length()
self._meta = dict()
Expand Down Expand Up @@ -105,8 +107,12 @@ def __iter__(self):

def __getitem__(self, key):
count = key * self._frame_bytes + self.skipbytes
self.f.seek(count, 0)
frame = np.fromfile(self.f, self.dtype, count=self._frame_size)

# Ensure reading a frame the file is thread-safe
with self._frame_read_lock:
self.f.seek(count, 0)
frame = np.fromfile(self.f, self.dtype, count=self._frame_size)

return frame.reshape(self.shape)

@property
Expand Down

0 comments on commit cc00d7d

Please sign in to comment.