Skip to content

Commit

Permalink
Added logging to help monitor shadow run folderes.
Browse files Browse the repository at this point in the history
Added runfolder path to some of log messages so that it is
easier to search the logs.

Added comments to the staging monitor scripts to highlight
that the script is likely to discover and process shadow
runfolders.
  • Loading branch information
mgcam committed Jan 25, 2024
1 parent 5ad4b4e commit 54e21b1
Show file tree
Hide file tree
Showing 3 changed files with 22 additions and 5 deletions.
19 changes: 18 additions & 1 deletion bin/staging_area_monitor
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,24 @@ sub main {
while (1) {

_log_noindent($time->());


#######
# find_live finds all run folders in 'incoming and 'analysis'.
# It can return two run folders for the same run, one in each
# of 'incoming and 'analysis'. In this case the run folder in
# incoming is a 'shadow' run folder, which is sometimes created
# by the instrument well after the original run.
#
# It is also possible for a 'shadow' folder to exist in 'incoming',
# while the run folder with the original data is already in 'outgoing'.
#
# Depending on how much data is present in a 'shadow' run folder,
# the staging monitor might attempt to update the trackign database.
# Mostly the values will be the same as for teh main run folder.
# However, the actual cycle count might be different. Therefore, it is
# important never to upload to the database a cycle count that is
# lower than the one already stored in the database.
#
foreach my $run_path ( $monitor->find_live($staging_area) ) {
_log_noindent("Considering $run_path");
my $done;
Expand Down
2 changes: 1 addition & 1 deletion lib/Monitor/RunFolder.pm
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ sub update_cycle_count {
defined $latest_cycle or croak 'Latest cycle count not supplied';
my $actual_cycle = $self->tracking_run()->actual_cycle_count();
$actual_cycle ||= 0;
if ($latest_cycle > $actual_cycle) {
if ($latest_cycle > $actual_cycle) { # A very important condition!
$self->tracking_run()->update({actual_cycle_count => $latest_cycle});
return 1;
}
Expand Down
6 changes: 3 additions & 3 deletions lib/Monitor/RunFolder/Staging.pm
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ sub update_run_from_path { ##### Method to deal with one run folder
}

if ($folder->is_in_analysis) {
_log('Folder is in /analysis/');
_log("Folder $run_path is in /analysis/");

if( $run_status eq 'qc complete') {
_log('Moving run folder to /outgoing/');
Expand All @@ -377,7 +377,7 @@ sub update_run_from_path { ##### Method to deal with one run folder
return; # Nothing else to do for a folder in /analysis/
}

_log('Folder is in /incoming/');
_log("Folder $run_path is in /incoming/");

# If we don't remember seeing it before, set the folder name and glob;
# set staging tag, if appropriate, set/fix instrument side, workflow side.
Expand Down Expand Up @@ -411,7 +411,7 @@ sub update_run_from_path { ##### Method to deal with one run folder

my $latest_cycle = $folder->get_latest_cycle();
if ($folder->update_cycle_count($latest_cycle)) {
_log("Cycle count updated to $latest_cycle");
_log("Cycle count updated to $latest_cycle for $run_path");
}

if ( $run_status eq 'run pending' ) {
Expand Down

0 comments on commit 54e21b1

Please sign in to comment.