Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

πŸ“Š hmd update #3642

Merged
merged 38 commits into from
Dec 2, 2024
Merged

Conversation

lucasrodes
Copy link
Member

@lucasrodes lucasrodes commented Nov 27, 2024

Update HMD dataset + derived datasets.

Dataset Name Add new datasets Migrate charts & indicators Archive
HMD
data://garden/hmd/2023-09-19/hmd
βœ… βœ… βœ…
Survivor percentiles
data://garden/demography/2023-09-27/survivor_percentiles
βœ… βœ… βœ…

@owidbot
Copy link
Contributor

owidbot commented Nov 27, 2024

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-lucasrodes-data-hmd-update-beabe3-private

chart-diff: βœ…
  • 8/8 reviewed charts
  • Modified: 5/5
  • New: 0/0
  • Rejected: 3
data-diff: ❌ Found differences
~ Dataset garden/antibiotics/2024-11-12/antimicrobial_usage
-   - update_period_days: 365
    ?                      ^^
+   + update_period_days: 308
    ?                      ^^
- - Table class_aggregated
-   - Column ddd_anti_malarials
-   - Column ddd_antibacterials_and_antituberculosis
-   - Column ddd_antifungals
-   - Column ddd_antituberculosis
-   - Column ddd_antivirals
-   - Column did_anti_malarials
-   - Column did_antibacterials_and_antituberculosis
-   - Column did_antifungals
-   - Column did_antituberculosis
-   - Column did_antivirals
  = Table class
    ~ Column ddd (changed metadata)
-       -   Defined daily doses of <% if routeofadministration == "O" %> orally administered <% elif routeofadministration == "P" %> parentearally administered <% elif routeofadministration == "R" %> rectally administered4 <% elif routeofadministration == "I" %> inhaled <% endif %>  << antimicrobialclass.lower()>> - << atc4name.lower() >>  used
        ?                                                                                                                                                                                                                                                                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
+       +   Defined daily doses of <% if routeofadministration == "O" %> orally administered <% elif routeofadministration == "P" %> parentearally administered <% elif routeofadministration == "R" %> rectally administered4 <% elif routeofadministration == "I" %> inhaled <% endif %>  << antimicrobialclass>> - << atc4name.lower() >>  used
        ?                                                                                                                                                                                                                                                                                                        ++++++++++++++++        ^^^
-       - description_short: Total (#dod:defined-daily-doses) of antimicrobials used in a given year.
+       + description_short: Volume of antimicrobials used in a given year.
-       - display:
-       -   numDecimalPlaces: 0
    ~ Column did (changed metadata)
-       - description_short: Total (#dod:defined-daily-doses) of antimicrobials used per 1000 inhabitants per day.
+       + description_short: Volume of antimicrobials used per 1000 inhabitants per day.
-       - display:
-       -   numDecimalPlaces: 1
  = Table aware
    ~ Column ddd (changed metadata)
-       -   Total (#dod:defined-daily-doses) of AWaRe category: << awarelabel >> antibiotics used in a given year. <% if aware == "A" %> Access antibiotics have activity against a wide range of common pathogens and show lower resistance potential than antibiotics in the other groups. <% elif aware == "W" %> Watch antibiotic have higher resistance potential and include most of the highest priority agents among the Critically Important Antimicrobials for Human Medicine and/or antibiotics that are at relatively high risk of bacterial resistance. <% elif aware == "R" %> Reserve antibiotics  should be reserved for treatment of confirmed or suspected infections due to multi-drug-resistant organisms. Reserve group antibiotics should be treated as β€œlast resort” options. <% elif aware == "O" %> The use of the Not classified/Not recommended antibiotics is not evidence-based, nor recommended in high-quality international guidelines.  WHO does not recommend the use of these antibiotics in clinical practice. <% endif %>
        ?   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+       +   Volume of AWaRe category: << awarelabel >> antibiotics used in a given year. <% if aware == "A" %> Access antibiotics have activity against a wide range of common pathogens and show lower resistance potential than antibiotics in the other groups. <% elif aware == "W" %> Watch antibiotic have higher resistance potential and include most of the highest priority agents among the Critically Important Antimicrobials for Human Medicine and/or antibiotics that are at relatively high risk of bacterial resistance. <% elif aware == "R" %> Reserve antibiotics  should be reserved for treatment of confirmed or suspected infections due to multi-drug-resistant organisms. Reserve group antibiotics should be treated as β€œlast resort” options. <% elif aware == "O" %> The use of the Not classified/Not recommended antibiotics is not evidence-based, nor recommended in high-quality international guidelines.  WHO does not recommend the use of these antibiotics in clinical practice. <% endif %>
        ?   ^^^^^^^^^
-       - display:
-       -   numDecimalPlaces: 0
    ~ Column did (changed metadata)
-       -   Total (#dod:defined-daily-doses) of AWaRe category: <<awarelabel>> used per 1000 inhabitants per day. <% if aware == "A" %> Access antibiotics have activity against a wide range of common pathogens and show lower resistance potential than antibiotics in the other groups. <% elif aware == "W" %> Watch antibiotic have higher resistance potential and include most of the highest priority agents among the Critically Important Antimicrobials for Human Medicine and/or antibiotics that are at relatively high risk of bacterial resistance. <% elif aware == "R" %> Reserve antibiotics  should be reserved for treatment of confirmed or suspected infections due to multi-drug-resistant organisms. Reserve group antibiotics should be treated as β€œlast resort” options. <% elif aware == "O" %> The use of the Not classified/Not recommended antibiotics is not evidence-based, nor recommended in high-quality international guidelines.  WHO does not recommend the use of these antibiotics in clinical practice. <% endif %>
        ?   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+       +   Volume of AWaRe category: <<awarelabel>> used per 1000 inhabitants per day. <% if aware == "A" %> Access antibiotics have activity against a wide range of common pathogens and show lower resistance potential than antibiotics in the other groups. <% elif aware == "W" %> Watch antibiotic have higher resistance potential and include most of the highest priority agents among the Critically Important Antimicrobials for Human Medicine and/or antibiotics that are at relatively high risk of bacterial resistance. <% elif aware == "R" %> Reserve antibiotics  should be reserved for treatment of confirmed or suspected infections due to multi-drug-resistant organisms. Reserve group antibiotics should be treated as β€œlast resort” options. <% elif aware == "O" %> The use of the Not classified/Not recommended antibiotics is not evidence-based, nor recommended in high-quality international guidelines.  WHO does not recommend the use of these antibiotics in clinical practice. <% endif %>
        ?   ^^^^^^^^^
-       - display:
-       -   numDecimalPlaces: 1
= Dataset garden/demography/2023-06-27/world_population_comparison
  = Table world_population_comparison
+ Dataset garden/demography/2024-12-02/survivor_percentiles
+ + Table survivor_percentiles
+   + Column age
~ Dataset garden/hmd/2024-11-27/hmd (new version)
-   - version: '2023-09-19'
    ?              ^^^^  ^
+   + version: '2024-11-27'
    ?              ^  ^^^^
- - Table hmd
-   - Column central_death_rate
-   - Column probability_of_death
-   - Column average_survival_length
-   - Column number_survivors
-   - Column number_deaths
-   - Column number_person_years_lived
-   - Column number_person_years_remaining
-   - Column life_expectancy
-   - Column exposure
-   - Column deaths
-   - Column life_expectancy_fm_diff
-   - Column life_expectancy_fm_ratio
-   - Column central_death_rate_mf_ratio
+ + Table exposures
+   + Column exposure
+ + Table diff_ratios
+   + Column life_expectancy_fm_diff
+   + Column life_expectancy_mf_ratio
+   + Column central_death_rate_mf_ratio
+ + Table life_tables
+   + Column central_death_rate
+   + Column probability_of_death
+   + Column average_survival_length
+   + Column number_survivors
+   + Column number_deaths
+   + Column number_person_years_lived
+   + Column number_person_years_remaining
+   + Column life_expectancy
+ + Table births
+   + Column births
+   + Column birth_rate
+ + Table population
+   + Column population
+ + Table deaths
+   + Column deaths
= Dataset garden/missing_data/2024-03-26/children_out_of_school
  = Table children_out_of_school
= Dataset garden/wb/2024-06-10/gender_statistics
  = Table gender_statistics
    ~ Column sg_law_eqrm_wk (changed data)
        ~ Changed values: 24 / 15124 (0.16%)
           country  year  sg_law_eqrm_wk -  sg_law_eqrm_wk +
          Slovakia  1980               1.0               0.0
          Slovakia  1981               1.0               0.0
          Slovakia  1984               1.0               0.0
          Slovakia  1990               1.0               0.0
          Slovakia  1992               1.0               0.0
= Dataset garden/wb/2024-06-10/gender_statistics_country_counts
  = Table gender_statistics
    ~ Column sg_law_eqrm_wk_no_count (changed data)
        ~ Changed values: 48 / 378 (12.70%)
          country  year  sg_law_eqrm_wk_no_count -  sg_law_eqrm_wk_no_count +
           Europe  1974                         38                         39
            World  1972                        186                        187
            World  1975                        181                        182
            World  1976                        180                        181
            World  1978                        178                        179
    ~ Column sg_law_eqrm_wk_no_pop (changed data)
        ~ Changed values: 48 / 378 (12.70%)
          country  year  sg_law_eqrm_wk_no_pop -  sg_law_eqrm_wk_no_pop +
           Europe  1974                550689102                555379750
            World  1972               3728735983               3733335785
            World  1975               3877380047               3882119222
            World  1976               3923263849               3928052362
            World  1978               4011155699               4016041432
    ~ Column sg_law_eqrm_wk_yes_count (changed data)
        ~ Changed values: 48 / 378 (12.70%)
          country  year  sg_law_eqrm_wk_yes_count -  sg_law_eqrm_wk_yes_count +
           Europe  1974                           4                           3
            World  1972                           3                           2
            World  1975                           8                           7
            World  1976                           9                           8
            World  1978                          11                          10
    ~ Column sg_law_eqrm_wk_yes_pop (changed data)
        ~ Changed values: 48 / 378 (12.70%)
          country  year  sg_law_eqrm_wk_yes_pop -  sg_law_eqrm_wk_yes_pop +
           Europe  1974                 123385892                 118695244
            World  1972                  70537112                  65937310
            World  1975                 145207798                 140468623
            World  1976                 172060729                 167272216
            World  1978                 230445466                 225559733
= Dataset garden/who/2024-09-09/flu_test
  = Table flu_test
    ~ Dim country
+       + New values: 7 / 72382 (0.01%)
                date country
          2024-08-26  Brazil
          2024-09-02  Brazil
          2024-09-09  Brazil
          2024-09-16  Brazil
          2024-11-11  Mexico
-       - Removed values: 36 / 72382 (0.05%)
                date  country
          2024-11-18 Ethiopia
          2024-11-11     Iraq
          2024-11-18  Jamaica
          2024-10-21   Uganda
          2024-10-28   Uganda
    ~ Dim date
+       + New values: 7 / 72382 (0.01%)
          country       date
           Brazil 2024-08-26
           Brazil 2024-09-02
           Brazil 2024-09-09
           Brazil 2024-09-16
           Mexico 2024-11-11
-       - Removed values: 36 / 72382 (0.05%)
           country       date
          Ethiopia 2024-11-18
              Iraq 2024-11-11
           Jamaica 2024-11-18
            Uganda 2024-10-21
            Uganda 2024-10-28
    ~ Column denomcombined (new data, changed data)
+       + New values: 7 / 72382 (0.01%)
          country       date  denomcombined
           Brazil 2024-08-26           6810
           Brazil 2024-09-02           7056
           Brazil 2024-09-09           7321
           Brazil 2024-09-16           6452
           Mexico 2024-11-11            495
-       - Removed values: 36 / 72382 (0.05%)
           country       date  denomcombined
          Ethiopia 2024-11-18            152
              Iraq 2024-11-11             35
           Jamaica 2024-11-18             19
            Uganda 2024-10-21             49
            Uganda 2024-10-28             68
        ~ Changed values: 87 / 72382 (0.12%)
            country       date  denomcombined -  denomcombined +
          Argentina 2024-09-30              166              164
             Brazil 2024-11-04             6224             5906
          Indonesia 2024-08-12               48               47
           Paraguay 2024-05-13              200              201
           Paraguay 2024-08-19              328              327
    ~ Column pcnt_poscombined (new data, changed data)
+       + New values: 7 / 72382 (0.01%)
          country       date  pcnt_poscombined
           Brazil 2024-08-26          5.800294
           Brazil 2024-09-02          5.782313
           Brazil 2024-09-09          6.979921
           Brazil 2024-09-16          7.377557
           Mexico 2024-11-11          8.686869
-       - Removed values: 36 / 72382 (0.05%)
           country       date  pcnt_poscombined
          Ethiopia 2024-11-18         18.421053
              Iraq 2024-11-11         17.142857
           Jamaica 2024-11-18         10.526316
            Uganda 2024-10-21         14.285714
            Uganda 2024-10-28          8.823529
        ~ Changed values: 94 / 72382 (0.13%)
            country       date  pcnt_poscombined -  pcnt_poscombined +
          Argentina 2024-09-30           22.289156            21.95122
             Brazil 2024-11-04            7.583548            7.162208
              Chile 2024-04-29           45.319149           45.689655
              Egypt 2024-09-30            4.118993            4.147465
            Jamaica 2024-09-30            1.041667            1.428571


Legend: +New  ~Modified  -Removed  =Identical  Details
Hint: Run this locally with etl diff REMOTE data/ --include yourdataset --verbose --snippet

Automatically updated datasets matching weekly_wildfires|excess_mortality|covid|fluid|flunet|country_profile|garden/ihme_gbd/2019/gbd_risk are not included

Edited: 2024-12-02 22:07:47 UTC
Execution time: 342.01 seconds

@lucasrodes lucasrodes marked this pull request as ready for review December 2, 2024 22:11
@lucasrodes lucasrodes merged commit bdf71c9 into master Dec 2, 2024
10 checks passed
@lucasrodes lucasrodes deleted the lucasrodes-data-hmd-update-beabe3-private branch December 2, 2024 22:23
@lucasrodes lucasrodes restored the lucasrodes-data-hmd-update-beabe3-private branch December 2, 2024 23:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants