Skip to content

Commit

Permalink
[irods#124] Use "target" for redis_lock key in create_dirs
Browse files Browse the repository at this point in the history
The create_dirs function recursively creates all collections and
subcollections for a particular path. This is made safe from
concurrent collection creation through the use of redis_lock.Lock.
Each descent into the collection's subcollections means acquiring
a lock for that subcollection until it reaches a collection which
already exists. This is the base case.

If the number of elements in the object name "path" (including "/"
and the bucket name) is fewer than the number of subcollections to
check, it can get stuck because it runs out of path elements to use
for unique lock names.

This change uses the collection for the redis_lock key rather than
the path. In this way the path elements will never run out.
  • Loading branch information
alanking committed Oct 4, 2024
1 parent e23f3f4 commit 66b9a5d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion irods_capability_automated_ingest/irods/irods_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ def create_dirs(logger, session, meta, **options):
if target.startswith("/"):
r = get_redis(config)
if not session.collections.exists(target):
with redis_lock.Lock(r, "create_dirs:" + path):
with redis_lock.Lock(r, "create_dirs:" + target):
if not session.collections.exists(target):
meta2 = meta.copy()
# TODO(#250): This will not work on Windows.
Expand Down

0 comments on commit 66b9a5d

Please sign in to comment.