-
-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
📊 WHO/Global Polio Eradication Initiative polio datasets #2502
Conversation
Staging server:
etl diff: ✅ No differences found+ Dataset garden/health/2024-04-12/polio_free_countries
+ + Table polio_free_countries
+ + Column latest_year_wild_polio_case
+ + Column status
+ Dataset garden/who/2024-04-08/polio
+ + Table polio
+ + Column total_cases
+ + Column afp_cases
+ + Column non_polio_afp_rate
+ + Column pct_adequate_stool_collection
+ + Column pending
+ + Column wild_poliovirus_cases
+ + Column cvdpv_cases
+ + Column compatibles
+ + Column footnote
+ + Column cvdpv1
+ + Column cvdpv2
+ + Column cvdpv3
+ + Column correction_factor
+ + Column estimated_cases
+ + Column polio_surveillance_status
+ + Column afp_cases_per_million
+ + Column wild_poliovirus_cases_per_million
+ + Column cvdpv_cases_per_million
+ + Column total_cases_per_million
+ + Column estimated_cases_per_million
+ + Column cvdpv1_per_million
+ + Column cvdpv2_per_million
+ + Column cvdpv3_per_million
+ Dataset garden/who/2024-04-09/polio_historical
+ + Table polio_historical
+ + Column cases
Legend: +New ~Modified -Removed =Identical Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile are not included Edited: 2024-04-17 13:42:27 UTC |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very good! I added several minor comments. Sorry for not sending the comments in one go, the pull request extension on VSCode is not working properly!
etl/steps/data/meadow/health/2024-04-12/polio_free_countries.py
Outdated
Show resolved
Hide resolved
etl/steps/data/garden/health/2024-04-12/polio_free_countries.py
Outdated
Show resolved
Hide resolved
etl/steps/data/garden/health/2024-04-12/polio_free_countries.py
Outdated
Show resolved
Hide resolved
etl/steps/data/garden/health/2024-04-12/polio_free_countries.countries.json
Outdated
Show resolved
Hide resolved
Thanks for the brilliant review! :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good! I just suggested a bunch of small things, feel free to ignore.
""" | ||
Some values for "Adequate stool collection" are over 100%, we should set these to NA. | ||
""" | ||
tb.loc[tb["pct_adequate_stool_collection"] > 100, "pct_adequate_stool_collection"] = pd.NA |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to happen only in three cases, with values 102, 104 and 113. I haven't looked into it, but I'm wondering if it would be better to set 102 and 104 to 100, and consider it a numerical issue with rounded numbers (and 113 to nan, as it possibly is an issue). Up to you.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will check with Saloni!
Thank you very much!! |
Hello!
This is a little bit of a monster, so I think it makes sense to split the review into two, by the namespaces I used.
@paarriagadap would you be able to review the following:
dag/health.yml
snapshots/health/2024-04-12/
meadow/health/2024-04-12/
garden/health/2024-04-12/
grapher/health/2024-04-12/
@pabloarosado you be able to review:
snapshots/who/2024-04-08/
snapshots/who/2024-04-09/
meadow/who/2024-04-08/
meadow/who/2024-04-09/
garden/who/2024-04-08/
garden/who/2024-04-09/
grapher/who/2024-04-08/