Skip to content

Commit

Permalink
18862: Updates get_prediction_stats docstrings to describe the new …
Browse files Browse the repository at this point in the history
…available stat "missing_value_accuracy", MINOR (#91)

"missing_value_accuracy" is a newly available stat to request via
`get_prediction_stats`. This stat gives the proportion of cases with
missing values that are correctly predicted as having a missing value.

This can help users identify what features have missing values that
predictable, which can help point towards correlations and meaning
behind the missing value.
  • Loading branch information
cademack authored Jan 8, 2024
1 parent f5875de commit 6bf92f6
Show file tree
Hide file tree
Showing 2 changed files with 10 additions and 2 deletions.
8 changes: 6 additions & 2 deletions howso/direct/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -4242,6 +4242,10 @@ def get_prediction_stats(
- mda_permutation : Mean decrease in accuracy that used
scrambling of feature values instead of dropping each
feature, applies to all features.
- missing_value_accuracy : The number of cases with missing
values predicted to have missing values divided by the number
of cases with missing values, applies to all features that
contain missing values.
- precision : Precision (positive predictive) value for nominal
features only.
- r2 : The r-squared coefficient of determination, for
Expand Down Expand Up @@ -4273,8 +4277,8 @@ def get_prediction_stats(
validate_list_shape(stats, 1, "stats", "str")
valid_stats = {
"accuracy", "contribution", "confusion_matrix", "mae", "mda",
"mda_permutation", "precision", "r2", "recall", "rmse",
"spearman_coeff",
"mda_permutation", "missing_value_accuracy", "precision", "r2",
"recall", "rmse", "spearman_coeff",
}

if stats is not None and not set(stats).issubset(valid_stats):
Expand Down
4 changes: 4 additions & 0 deletions howso/engine/trainee.py
Original file line number Diff line number Diff line change
Expand Up @@ -2900,6 +2900,10 @@ def get_prediction_stats(
- mda_permutation : Mean decrease in accuracy that used
scrambling of feature values instead of dropping each
feature, applies to all features.
- missing_value_accuracy : The number of cases with missing
values predicted to have missing values divided by the number
of cases with missing values, applies to all features that
contain missing values.
- precision : Precision (positive predictive) value for nominal
features only.
- r2 : The r-squared coefficient of determination, for
Expand Down

0 comments on commit 6bf92f6

Please sign in to comment.