Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scream plugin: single Fic!=Ric error crashes everything #18

Open
filefolder opened this issue Nov 20, 2024 · 3 comments
Open

scream plugin: single Fic!=Ric error crashes everything #18

filefolder opened this issue Nov 20, 2024 · 3 comments

Comments

@filefolder
Copy link
Contributor

Hi all

Noticing that a single corrupted packet from a remote scream! server seems to be crashing the entire plugin, causing a gap of 60-100 seconds on every station rather than the individual culprit. Here is the error:

 Fic!=Ric
Wed Nov 20 07:16:32 2024 - seedlink: [scream0] unexpected eof
Wed Nov 20 07:16:33 2024 - seedlink: [scream0] terminated with error status 255
Wed Nov 20 07:17:33 2024 - seedlink: [scream0] starting shell

this is in various functions in gcf.c

  if (b->fic != b->ric)
    fatal (("Fic!=Ric"));

Is there a reasonable way to replace that fatal error, throw a warning, and drop that particular packet Instead of killing the whole plugin?

I might try removing the checks in the extract_## functions and moving them to the switch cases in gcf_dispatch, e.g.

          case 4:
            block.csize = 24 + block.samples;
            extract_8 (&block);
            if (b->fic != b->ric)
                return;
            break;
          break;

Would this work? Better ideas? I know this code is 20 years old now..

(n.b. I suspect this is from a packet from a sensor that is currently back-filling old data, if anyone has any experience mitigating these sort of things would also love to hear how you did it!)

Thanks as usual

@gempa-jabe
Copy link
Contributor

As you said, this code is very old and contributed. Maybe the fic != ric issue is unrecoverable in this particular code design and therefore needs a hard reset. As you are the only one so far reporting this issue since 20 years I doubt that there is a workaround on someones table. I personally do not have access to a Guralp datalogger and cannot test this case. Just give your ideas a try and see what happens.

Maybe @andres-h has more viable input as he is much more experienced with seedlink and its plugins than anybody else.

@filefolder
Copy link
Contributor Author

Thanks Jan. I will probably end up just going for it, but I only have "real" data to test it on and I certainly don't have a strong eye for such things.

@filefolder
Copy link
Contributor Author

Quick update but the above hack seems to work as intended-- big news for us at least as this has been a particularly diabolical issue.

WARNING: Fic!=Ric
Wed Nov 20 14:51:08 2024 - seedlink: S1.AUAPY : HHZ time gap 5 seconds (detected)
WARNING: Fic!=Ric
Wed Nov 20 14:54:38 2024 - seedlink: S1.AUAPY : HHZ time gap 5 seconds (detected)
WARNING: Fic!=Ric

Not sure exactly how to specify the offending serial number in the warning (though clearly AUAPY in this example), but instead of gapping the entire network it just lets dispatch handle the individual gap. I will write up a PR and keep testing for the next few weeks for any 2nd order timing effects.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants