Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function to drop/flag obviously bad data #10

Open
serbinsh opened this issue Mar 21, 2023 · 1 comment
Open

Function to drop/flag obviously bad data #10

serbinsh opened this issue Mar 21, 2023 · 1 comment
Assignees

Comments

@serbinsh
Copy link
Member

It would be good to remove some of the hard-coded assumptions in the processing functions to allow for flagging/dropping data where we have obviously broken/bad thermister data (0.9-1.0) during data load. This way we can avoid having to deal with those data in the data processing steps and hopefully avoid issues with erroneous data.

This would require flagging at the start then dealing with issues including when there is no usable data during a time window and skipping that tree in the processing steps.

@DavidsonKen
Copy link
Member

I have added functionality to the "sanity_check_fine" function so that it now sets all data which are negative to NA, and also flags data with no heat pulse and changes that whole pulse to NA.

We can still also build in a function to flag bad data from the start, but the automated function should cover most cases of bad data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants