Skip to content

Commit

Permalink
🔨 Converting UN IGME per 1,000 rates to percentages (#1918)
Browse files Browse the repository at this point in the history 
* changing units to %

* fixing short_unit

* fixing conversion
  • Loading branch information
@spoonerf
spoonerf authored Nov 8, 2023
1 parent 5af1f3c commit 6cfcf4b
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 33 deletions.
64 changes: 33 additions & 31 deletions etl/steps/data/garden/un/2023-08-16/igme.meta.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,59 +9,57 @@ definitions:
<% if wealth_quintile == "All wealth quintiles" %>all wealth quintiles<% elif wealth_quintile == "Poorest quintile" %>the poorest 20% of the population<% elif wealth_quintile == "Second poorest quintile" %>the second poorest 20% of the population<% elif wealth_quintile == "Middle wealth quintile" %>the middle 20% of the population in terms of wealth<% elif wealth_quintile == "Fourth poorest quintile" %>the second richest 20% of the population<% elif wealth_quintile == "Richest quintile" %>the richest 20% of the population<%- endif -%>
unit_of_measure:
unit: |-
<% if unit_of_measure == "Deaths per 1,000 live births" %>
deaths per 1,000 live births
<% elif unit_of_measure == "Deaths per 1000 children aged 10" %>
deaths per 1,000 children aged 10
<% elif unit_of_measure == "Deaths per 1000 children aged 15" %>
deaths per 1,000 children aged 15
<% elif unit_of_measure == "Deaths per 1000 children aged 1" %>
deaths per 1,000 children aged 1
<% elif unit_of_measure == "Deaths per 1000 children aged 20" %>
deaths per 1,000 children aged 20
<% elif unit_of_measure == "Deaths per 1000 children aged 5" %>
deaths per 1,000 children aged 5
<% if unit_of_measure == "Deaths per 100 live births" %>
deaths per 100 live births
<% elif unit_of_measure == "Deaths per 100 children aged 10" %>
deaths per 100 children aged 10
<% elif unit_of_measure == "Deaths per 100 children aged 15" %>
deaths per 100 children aged 15
<% elif unit_of_measure == "Deaths per 100 children aged 1" %>
deaths per 100 children aged 1
<% elif unit_of_measure == "Deaths per 100 children aged 20" %>
deaths per 100 children aged 20
<% elif unit_of_measure == "Deaths per 100 children aged 5" %>
deaths per 100 children aged 5
<% elif unit_of_measure == "Number of stillbirths" %>
stillbirths
<% elif unit_of_measure == "Stillbirths per 1,000 total births" %>
stillbirths per 1,000 total births
<% elif unit_of_measure == "Stillbirths per 100 total births" %>
stillbirths per 100 total births
<% elif unit_of_measure == "Number of deaths" %>
deaths
<%- endif -%>
decimal_places: |-
<% if 'rate' in indicator %>1<% else %>0<%- endif -%>
indicator:
display_name: |-
<% if indicator == "Neonatal mortality rate" %>
The estimated number of deaths of {definitions.sex} children aged less than 28 days, per 1,000 live births, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged less than 28 days, per 100 live births, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate 1-59 months" %>
The estimated number of deaths of {definitions.sex} children aged between 1 and 59 months, per 1,000 live births, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 1 and 59 months, per 100 live births, in {definitions.wealth_quintile}
<% elif indicator == "Infant mortality rate" %>
The estimated number of deaths of {definitions.sex} children aged under one year, per 1,000 live births, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged under one year, per 100 live births, in {definitions.wealth_quintile}
<% elif indicator == "Under-five mortality rate" %>
The estimated number of deaths of {definitions.sex} children aged under five years, per 1,000 live births, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged under five years, per 100 live births, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 10-14" %>
The estimated number of deaths of {definitions.sex} children aged between 10 and 14, per 1,000 children aged 10, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 10 and 14, per 100 children aged 10, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 10-19" %>
The estimated number of deaths of {definitions.sex} children aged between 10 and 19, per 1,000 children aged 10, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 10 and 19, per 100 children aged 10, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 15-19" %>
The estimated number of deaths of {definitions.sex} children aged between 15 and 19, per 1,000 children aged 15, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 15 and 19, per 100 children aged 15, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 15-24" %>
The estimated number of deaths of {definitions.sex} children aged between 15 and 24, per 1,000 children aged 15, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 15 and 24, per 100 children aged 15, in {definitions.wealth_quintile}
<% elif indicator == "Child Mortality rate age 1-4" %>
The estimated number of deaths of {definitions.sex} children aged between 1 and 4, per 1,000 children aged 1, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 1 and 4, per 100 children aged 1, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 20-24" %>
The estimated number of deaths of {definitions.sex} children aged between 20 and 24, per 1,000 children aged 20, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 20 and 24, per 100 children aged 20, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 5-14" %>
The estimated number of deaths of {definitions.sex} children aged between 5 and 14, per 1,000 children aged 5, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 5 and 14, per 100 children aged 5, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 5-24" %>
The estimated number of deaths of {definitions.sex} children aged between 5 and 24, per 1,000 children aged 5, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 5 and 24, per 100 children aged 5, in {definitions.wealth_quintile}
<% elif indicator == "Mortality rate age 5-9" %>
The estimated number of deaths of {definitions.sex} children aged between 5 and 9, per 1,000 children aged 5, in {definitions.wealth_quintile}
The estimated number of deaths of {definitions.sex} children aged between 5 and 9, per 100 children aged 5, in {definitions.wealth_quintile}
<% elif indicator == "Stillbirths" %>
The estimated number of stillbirths, in {definitions.wealth_quintile}
<% elif indicator == "Stillbirth rate" %>
The estimated number of stillbirths, per 1,000 total births, in {definitions.wealth_quintile}
The estimated number of stillbirths, per 100 total births, in {definitions.wealth_quintile}
<% elif indicator == "Neonatal deaths" %>
The estimated number of deaths of {definitions.sex} children aged less than 28 days, in {definitions.wealth_quintile}
<% elif indicator == "Deaths 1-59 months" %>
Expand Down Expand Up @@ -102,6 +100,7 @@ tables:
title: Observation value
unit: |-
{definitions.unit_of_measure.unit}
short_unit: <% if 'rate' in indicator %>%<% else %><%- endif -%>
display:
name: |-
{definitions.indicator.display_name}
Expand All @@ -110,6 +109,7 @@ tables:
title: Lower bound
unit: |-
{definitions.unit_of_measure.unit}
short_unit: <% if 'rate' in indicator %>%<% else %><%- endif -%>
display:
name: |-
{definitions.indicator.display_name}
Expand All @@ -118,6 +118,7 @@ tables:
title: Upper bound
unit: |-
{definitions.unit_of_measure.unit}
short_unit: <% if 'rate' in indicator %>%<% else %><%- endif -%>
display:
name: |-
{definitions.indicator.display_name}
Expand All @@ -127,7 +128,8 @@ tables:
obs_value:
title: Under fifteen mortality
description_processing: This indicator is processed by OWID based on the original data source. It is a combination of the under-five mortality rate and the 5-14 mortality rate.
unit: deaths per 1,000 live births
unit: share of live births
short_unit: "%"
display:
name: |-
{definitions.indicator.display_name}
Expand Down
29 changes: 27 additions & 2 deletions etl/steps/data/garden/un/2023-08-16/igme.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ def run(dest_dir: str) -> None:
tb = filter_data(tb)
tb = round_down_year(tb)
tb = clean_values(tb)

tb = convert_to_percentage(tb)
tb["source"] = "igme (current)"
# Separate out the variables needed to calculate the under-fifteen mortality rate.
tb_under_fifteen = tb[
Expand Down Expand Up @@ -78,6 +78,31 @@ def run(dest_dir: str) -> None:
ds_garden.save()


def convert_to_percentage(tb: Table) -> Table:
"""
Convert the units which are given as 'per 1,000...' into percentages.
"""
rate_conversions = {
"Deaths per 1,000 live births": "Deaths per 100 live births",
"Deaths per 1000 children aged 1": "Deaths per 100 children aged 1",
"Deaths per 1000 children aged 5": "Deaths per 100 children aged 5",
"Deaths per 1000 children aged 10": "Deaths per 100 children aged 10",
"Deaths per 1000 children aged 15": "Deaths per 100 children aged 15",
"Deaths per 1000 children aged 20": "Deaths per 100 children aged 20",
"Stillbirths per 1000 births": "Stillbirths per 100 births",
}
# Dividing values of selected rows by 10

selected_rows = tb["unit_of_measure"].isin(rate_conversions.keys())
tb.loc[selected_rows, ["obs_value", "lower_bound", "upper_bound"]] = tb.loc[
selected_rows, ["obs_value", "lower_bound", "upper_bound"]
].div(10)

tb = tb.replace({"unit_of_measure": rate_conversions})

return tb


def add_post_neonatal_deaths(tb: Table) -> Table:
"""
Calculate the deaths for the post-neonatal age-group, 28 days - 1 year
Expand Down Expand Up @@ -170,7 +195,7 @@ def calculate_under_fifteen_mortality_rates(tb: Table) -> Table:
on=["country", "year", "wealth_quintile", "sex", "source"],
suffixes=("_u5", "_5_14"),
)
tb_merge["adjusted_5_14_mortality_rate"] = (1000 - tb_merge["obs_value_u5"]) / 1000 * tb_merge["obs_value_5_14"]
tb_merge["adjusted_5_14_mortality_rate"] = (100 - tb_merge["obs_value_u5"]) / 100 * tb_merge["obs_value_5_14"]
tb_merge["obs_value"] = tb_merge["obs_value_u5"] + tb_merge["adjusted_5_14_mortality_rate"]
tb_merge["indicator"] = "Under-fifteen mortality rate"
tb_merge["unit_of_measure"] = "Deaths per 1,000 live births"
Expand Down

0 comments on commit 6cfcf4b

Please sign in to comment.