Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added logging to help monitor shadow run folderes. #804

Merged
merged 1 commit into from
Feb 7, 2024
Merged

Added logging to help monitor shadow run folderes. #804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion bin/staging_area_monitor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,24 @@ sub main {
while (1) {

_log_noindent($time->());


#######
# find_live finds all run folders in 'incoming and 'analysis'.
# It can return two run folders for the same run, one in each
# of 'incoming and 'analysis'. In this case the run folder in
# incoming is a 'shadow' run folder, which is sometimes created
# by the instrument well after the original run.
#
# It is also possible for a 'shadow' folder to exist in 'incoming',
# while the run folder with the original data is already in 'outgoing'.
#
# Depending on how much data is present in a 'shadow' run folder,
# the staging monitor might attempt to update the trackign database.
# Mostly the values will be the same as for teh main run folder.
# However, the actual cycle count might be different. Therefore, it is
# important never to upload to the database a cycle count that is
# lower than the one already stored in the database.
#
foreach my $run_path ( $monitor->find_live($staging_area) ) {
_log_noindent("Considering $run_path");
my $done;
Expand Down
2 changes: 1 addition & 1 deletion lib/Monitor/RunFolder.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ sub update_cycle_count {
defined $latest_cycle or croak 'Latest cycle count not supplied';
my $actual_cycle = $self->tracking_run()->actual_cycle_count();
$actual_cycle ||= 0;
if ($latest_cycle > $actual_cycle) {
if ($latest_cycle > $actual_cycle) { # A very important condition!
$self->tracking_run()->update({actual_cycle_count => $latest_cycle});
return 1;
}
Expand Down
6 changes: 3 additions & 3 deletions lib/Monitor/RunFolder/Staging.pm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ sub update_run_from_path { ##### Method to deal with one run folder
}

if ($folder->is_in_analysis) {
_log('Folder is in /analysis/');
_log("Folder $run_path is in /analysis/");

if( $run_status eq 'qc complete') {
_log('Moving run folder to /outgoing/');
Expand All @@ -377,7 +377,7 @@ sub update_run_from_path { ##### Method to deal with one run folder
return; # Nothing else to do for a folder in /analysis/
}

_log('Folder is in /incoming/');
_log("Folder $run_path is in /incoming/");

# If we don't remember seeing it before, set the folder name and glob;
# set staging tag, if appropriate, set/fix instrument side, workflow side.
Expand Down Expand Up @@ -411,7 +411,7 @@ sub update_run_from_path { ##### Method to deal with one run folder

my $latest_cycle = $folder->get_latest_cycle();
if ($folder->update_cycle_count($latest_cycle)) {
_log("Cycle count updated to $latest_cycle");
_log("Cycle count updated to $latest_cycle for $run_path");
}

if ( $run_status eq 'run pending' ) {
Expand Down
Loading