From af5e625c264f532ad3cd64a51f63816b1673c41c Mon Sep 17 00:00:00 2001 From: Vijay Janapa Reddi Date: Sat, 16 Nov 2024 21:42:42 -0500 Subject: [PATCH] Figure placement --- contents/core/responsible_ai/responsible_ai.qmd | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/contents/core/responsible_ai/responsible_ai.qmd b/contents/core/responsible_ai/responsible_ai.qmd index 3f40fbf6..b85e5ee6 100644 --- a/contents/core/responsible_ai/responsible_ai.qmd +++ b/contents/core/responsible_ai/responsible_ai.qmd @@ -197,10 +197,10 @@ To evaluate and address these issues, fairness in machine learning is typically This concept is illustrated in @fig-fairness-example, which visualizes the performance of a machine learning model predicting loan repayment for two subgroups, Subgroup A (blue) and Subgroup B (red). Each individual in the dataset is represented by a symbol: plusses (+) indicate individuals who will repay their loans (true positives), while circles (O) indicate individuals who will default on their loans (true negatives). The model's objective is to correctly classify these individuals into repayers and defaulters. -To evaluate performance, two dotted lines are shown, representing the thresholds at which the model achieves acceptable accuracy for each subgroup. For Subgroup A, the threshold needs to be set at 81.25% accuracy (the second dotted line) to correctly classify all repayers (plusses). However, using this same threshold for Subgroup B would result in misclassifications, as some repayers in Subgroup B would incorrectly fall below this threshold and be classified as defaulters. For Subgroup B, a lower threshold of 75% accuracy (the first dotted line) is necessary to correctly classify its repayers. However, applying this lower threshold to Subgroup A would result in misclassifications for that group. This illustrates how the model performs unequally across the two subgroups, with each requiring a different threshold to maximize their true positive rates. - ![Illustrates the trade-off in setting classification thresholds for two subgroups (A and B) in a loan repayment model. Plusses (+) represent true positives (repayers), and circles (O) represent true negatives (defaulters). Different thresholds (75% for B and 81.25% for A) maximize subgroup accuracy but reveal fairness challenges.](images/png/fairness_cartoon.png){#fig-fairness-example} +To evaluate performance, two dotted lines are shown, representing the thresholds at which the model achieves acceptable accuracy for each subgroup. For Subgroup A, the threshold needs to be set at 81.25% accuracy (the second dotted line) to correctly classify all repayers (plusses). However, using this same threshold for Subgroup B would result in misclassifications, as some repayers in Subgroup B would incorrectly fall below this threshold and be classified as defaulters. For Subgroup B, a lower threshold of 75% accuracy (the first dotted line) is necessary to correctly classify its repayers. However, applying this lower threshold to Subgroup A would result in misclassifications for that group. This illustrates how the model performs unequally across the two subgroups, with each requiring a different threshold to maximize their true positive rates. + The disparity in required thresholds highlights the challenge of achieving fairness in model predictions. If positive classifications lead to loan approvals, individuals in Subgroup B would be disadvantaged unless the threshold is adjusted specifically for their subgroup. However, adjusting thresholds introduces trade-offs between group-level accuracy and fairness, demonstrating the inherent tension in optimizing for these objectives in machine learning systems. Thus, the fairness literature has proposed three main _fairness metrics_ for quantifying how fair a model performs over a dataset [@hardt2016equality]. Given a model $h$ and a dataset $D$ consisting of $(x, y, s)$ samples, where $x$ is the data features, $y$ is the label, and $s$ is the subgroup attribute, and we assume there are simply two subgroups $a$ and $b$, we can define the following: