sagemaker_processing/fairness_and_explainability/fairness_and_explainability_spark.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Fairness and Explainability with SageMaker Clarify - Spark Distributed Processing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. [Overview](#Overview)\n",
    "1. [Prerequisites and Data](#Prerequisites-and-Data)\n",
    "    1. [Initialize SageMaker](#Initialize-SageMaker)\n",
    "    1. [Download data](#Download-data)\n",
    "    1. [Loading the data: Adult Dataset](#Loading-the-data:-Adult-Dataset) \n",
    "    1. [Data inspection](#Data-inspection) \n",
    "    1. [Data encoding and upload to S3](#Encode-and-Upload-the-Data) \n",
    "1. [Train and Deploy XGBoost Model](#Train-XGBoost-Model)\n",
    "    1. [Train Model](#Train-Model)\n",
    "    1. [Deploy Model to Endpoint](#Deploy-Model)\n",
    "1. [Amazon SageMaker Clarify](#Amazon-SageMaker-Clarify)\n",
    "    1. [Detecting Bias](#Detecting-Bias)\n",
    "        1. [Writing BiasConfig](#Writing-BiasConfig)\n",
    "        1. [Pre-training Bias](#Pre-training-Bias)\n",
    "        1. [Post-training Bias](#Post-training-Bias)\n",
    "        1. [Viewing the Bias Report](#Viewing-the-Bias-Report)\n",
    "    1. [Explaining Predictions](#Explaining-Predictions)\n",
    "        1. [Viewing the Explainability Report](#Viewing-the-Explainability-Report)\n",
    "1. [Clean Up](#Clean-Up)\n",
    "\n",
    "## Overview\n",
    "Amazon SageMaker Clarify helps improve your machine learning models by detecting potential bias and helping explain how these models make predictions. The fairness and explainability functionality provided by SageMaker Clarify takes a step towards enabling AWS customers to build trustworthy and understandable machine learning models. The product comes with the tools to help you with the following tasks.\n",
    "\n",
    "* Measure biases that can occur during each stage of the ML lifecycle (data collection, model training and tuning, and monitoring of ML models deployed for inference).\n",
    "* Generate model governance reports targeting risk and compliance teams and external regulators.\n",
    "* Provide explanations of the data, models, and monitoring used to assess predictions.\n",
    "\n",
    "This sample notebook walks you through:  \n",
    "1. Key terms and concepts needed to understand SageMaker Clarify\n",
    "1. Measuring the pre-training bias of a dataset and post-training bias of a model\n",
    "1. Explaining the importance of the various input features on the model's decision\n",
    "1. Accessing the reports through SageMaker Studio if you have an instance set up.\n",
    "\n",
    "In doing so, the notebook will first train a [SageMaker XGBoost](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost.html) model using training dataset, then use SageMaker Clarify to analyze a testing dataset in CSV format. SageMaker Clarify also supports analyzing dataset in [SageMaker JSONLines dense format](https://docs.aws.amazon.com/sagemaker/latest/dg/cdf-inference.html#common-in-formats), which is illustrated in [another notebook](https://github.com/aws/amazon-sagemaker-examples/blob/master/sagemaker_processing/fairness_and_explainability/fairness_and_explainability_jsonlines_format.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites and Data\n",
    "### Initialize SageMaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import Session\n",
    "\n",
    "session = Session()\n",
    "bucket = session.default_bucket()\n",
    "prefix = \"sagemaker/DEMO-sagemaker-clarify\"\n",
    "region = session.boto_region_name\n",
    "# Define IAM role\n",
    "from sagemaker import get_execution_role\n",
    "from io import StringIO\n",
    "from s3fs import S3FileSystem\n",
    "import sagemaker\n",
    "import json\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import boto3\n",
    "\n",
    "role = get_execution_role()\n",
    "s3_client = boto3.client(\"s3\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download data\n",
    "Data Source: [https://archive.ics.uci.edu/ml/machine-learning-databases/adult/](https://archive.ics.uci.edu/ml/machine-learning-databases/adult/)\n",
    "\n",
    "Let's __download__ the data and save it in the local folder with the name adult.data and adult.test from UCI repository$^{[2]}$.\n",
    "\n",
    "$^{[2]}$Dua Dheeru, and Efi Karra Taniskidou. \"[UCI Machine Learning Repository](http://archive.ics.uci.edu/ml)\". Irvine, CA: University of California, School of Information and Computer Science (2017)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "adult_columns = [\n",
    "    \"Age\",\n",
    "    \"Workclass\",\n",
    "    \"fnlwgt\",\n",
    "    \"Education\",\n",
    "    \"Education-Num\",\n",
    "    \"Marital Status\",\n",
    "    \"Occupation\",\n",
    "    \"Relationship\",\n",
    "    \"Ethnic group\",\n",
    "    \"Sex\",\n",
    "    \"Capital Gain\",\n",
    "    \"Capital Loss\",\n",
    "    \"Hours per week\",\n",
    "    \"Country\",\n",
    "    \"Target\",\n",
    "]\n",
    "if not os.path.isfile(\"adult.data\"):\n",
    "    s3_client.download_file(\n",
    "        \"sagemaker-sample-files\", \"datasets/tabular/uci_adult/adult.data\", \"adult.data\"\n",
    "    )\n",
    "    print(\"adult.data saved!\")\n",
    "else:\n",
    "    print(\"adult.data already on disk.\")\n",
    "\n",
    "if not os.path.isfile(\"adult.test\"):\n",
    "    s3_client.download_file(\n",
    "        \"sagemaker-sample-files\", \"datasets/tabular/uci_adult/adult.test\", \"adult.test\"\n",
    "    )\n",
    "    print(\"adult.test saved!\")\n",
    "else:\n",
    "    print(\"adult.test already on disk.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the data: Adult Dataset\n",
    "From the UCI repository of machine learning datasets, this database contains 14 features concerning demographic characteristics of 45,222 rows (32,561 for training and 12,661 for testing). The task is to predict whether a person has a yearly income that is more or less than $50,000.\n",
    "\n",
    "Here are the features and their possible values:\n",
    "1. **Age**: continuous.\n",
    "1. **Workclass**: Private, Self-emp-not-inc, Self-emp-inc, Federal-gov, Local-gov, State-gov, Without-pay, Never-worked.\n",
    "1. **Fnlwgt**: continuous (the number of people the census takers believe that observation represents).\n",
    "1. **Education**: Bachelors, Some-college, 11th, HS-grad, Prof-school, Assoc-acdm, Assoc-voc, 9th, 7th-8th, 12th, Masters, 1st-4th, 10th, Doctorate, 5th-6th, Preschool.\n",
    "1. **Education-num**: continuous.\n",
    "1. **Marital-status**: Married-civ-spouse, Divorced, Never-married, Separated, Widowed, Married-spouse-absent, Married-AF-spouse.\n",
    "1. **Occupation**: Tech-support, Craft-repair, Other-service, Sales, Exec-managerial, Prof-specialty, Handlers-cleaners, Machine-op-inspct, Adm-clerical, Farming-fishing, Transport-moving, Priv-house-serv, Protective-serv, Armed-Forces.\n",
    "1. **Relationship**: Wife, Own-child, Husband, Not-in-family, Other-relative, Unmarried.\n",
    "1. **Ethnic group**: White, Asian-Pac-Islander, Amer-Indian-Eskimo, Other, Black.\n",
    "1. **Sex**: Female, Male.\n",
    "    * **Note**: this data is extracted from the 1994 Census and enforces a binary option on Sex\n",
    "1. **Capital-gain**: continuous.\n",
    "1. **Capital-loss**: continuous.\n",
    "1. **Hours-per-week**: continuous.\n",
    "1. **Native-country**: United-States, Cambodia, England, Puerto-Rico, Canada, Germany, Outlying-US(Guam-USVI-etc), India, Japan, Greece, South, China, Cuba, Iran, Honduras, Philippines, Italy, Poland, Jamaica, Vietnam, Mexico, Portugal, Ireland, France, Dominican-Republic, Laos, Ecuador, Taiwan, Haiti, Columbia, Hungary, Guatemala, Nicaragua, Scotland, Thailand, Yugoslavia, El-Salvador, Trinadad&Tobago, Peru, Hong, Holand-Netherlands.\n",
    "\n",
    "Next, we specify our binary prediction task:  \n",
    "15. **Target**: <=50,000, >$50,000."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data = pd.read_csv(\n",
    "    \"adult.data\", names=adult_columns, sep=r\"\\s*,\\s*\", engine=\"python\", na_values=\"?\"\n",
    ").dropna()\n",
    "\n",
    "testing_data = pd.read_csv(\n",
    "    \"adult.test\", names=adult_columns, sep=r\"\\s*,\\s*\", engine=\"python\", na_values=\"?\", skiprows=1\n",
    ").dropna()\n",
    "\n",
    "training_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data inspection\n",
    "Plotting histograms for the distribution of the different features is a good way to visualize the data. Let's plot a few of the features that can be considered _sensitive_.  \n",
    "Let's take a look specifically at the Sex feature of a census respondent. In the first plot we see that there are fewer Female respondents as a whole but especially in the positive outcomes, where they form ~$\\frac{1}{7}$th of respondents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data[\"Sex\"].value_counts().sort_values().plot(kind=\"bar\", title=\"Counts of Sex\", rot=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data[\"Sex\"].where(training_data[\"Target\"] == \">50K\").value_counts().sort_values().plot(\n",
    "    kind=\"bar\", title=\"Counts of Sex earning >$50K\", rot=0\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encode and Upload the Dataset\n",
    "Here we encode the training and test data. Encoding input data is not necessary for SageMaker Clarify, but is necessary for the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing\n",
    "\n",
    "\n",
    "def number_encode_features(df):\n",
    "    result = df.copy()\n",
    "    encoders = {}\n",
    "    for column in result.columns:\n",
    "        if result.dtypes[column] == np.object:\n",
    "            encoders[column] = preprocessing.LabelEncoder()\n",
    "            #  print('Column:', column, result[column])\n",
    "            result[column] = encoders[column].fit_transform(result[column].fillna(\"None\"))\n",
    "    return result, encoders\n",
    "\n",
    "\n",
    "training_data = pd.concat([training_data[\"Target\"], training_data.drop([\"Target\"], axis=1)], axis=1)\n",
    "training_data, _ = number_encode_features(training_data)\n",
    "training_data.to_csv(\"train_data.csv\", index=False, header=False)\n",
    "\n",
    "testing_data, _ = number_encode_features(testing_data)\n",
    "test_features = testing_data.drop([\"Target\"], axis=1)\n",
    "test_target = testing_data[\"Target\"]\n",
    "test_features.to_csv(\"test_features.csv\", index=False, header=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A quick note about our encoding: the \"Female\" Sex value has been encoded as 0 and \"Male\" as 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lastly, let's upload the data to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Uploader\n",
    "from sagemaker.inputs import TrainingInput\n",
    "\n",
    "train_uri = S3Uploader.upload(\"train_data.csv\", \"s3://{}/{}\".format(bucket, prefix))\n",
    "train_input = TrainingInput(train_uri, content_type=\"csv\")\n",
    "test_uri = S3Uploader.upload(\"test_features.csv\", \"s3://{}/{}\".format(bucket, prefix))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train XGBoost Model\n",
    "#### Train Model\n",
    "Since our focus is on understanding how to use SageMaker Clarify, we keep it simple by using a standard XGBoost model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.image_uris import retrieve\n",
    "from sagemaker.estimator import Estimator\n",
    "\n",
    "container = retrieve(\"xgboost\", region, version=\"1.2-1\")\n",
    "xgb = Estimator(\n",
    "    container,\n",
    "    role,\n",
    "    instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    disable_profiler=True,\n",
    "    sagemaker_session=session,\n",
    ")\n",
    "\n",
    "xgb.set_hyperparameters(\n",
    "    max_depth=5,\n",
    "    eta=0.2,\n",
    "    gamma=4,\n",
    "    min_child_weight=6,\n",
    "    subsample=0.8,\n",
    "    objective=\"binary:logistic\",\n",
    "    num_round=800,\n",
    ")\n",
    "\n",
    "xgb.fit({\"train\": train_input}, logs=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Deploy Model\n",
    "Here we create the SageMaker model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"DEMO-clarify-model\"\n",
    "model = xgb.create_model(name=model_name)\n",
    "container_def = model.prepare_container_def()\n",
    "session.create_model(model_name, role, container_def)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Amazon SageMaker Clarify\n",
    "Now that you have your model set up. Let's say hello to SageMaker Clarify!\n",
    "\n",
    "When working with large datasets, you can use the Spark processing capabilities of SageMaker Clarify to enable your Clarify processing jobs to run faster. To use Spark processing for Clarify jobs, set the instance count to a number greater than one. Clarify uses Spark distributed computing when there is more than once instance per Clarify processor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker import clarify\n",
    "\n",
    "clarify_processor = clarify.SageMakerClarifyProcessor(\n",
    "    role=role, instance_count=2, instance_type=\"ml.m5.xlarge\", sagemaker_session=session\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Detecting Bias\n",
    "SageMaker Clarify helps you detect possible pre- and post-training biases using a variety of metrics.\n",
    "#### Writing DataConfig and ModelConfig\n",
    "A `DataConfig` object communicates some basic information about data I/O to SageMaker Clarify. We specify where to find the input dataset, where to store the output, the target column (`label`), the header names, and the dataset type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path = \"s3://{}/{}/clarify-bias\".format(bucket, prefix)\n",
    "bias_data_config = clarify.DataConfig(\n",
    "    s3_data_input_path=train_uri,\n",
    "    s3_output_path=bias_report_output_path,\n",
    "    label=\"Target\",\n",
    "    headers=training_data.columns.to_list(),\n",
    "    dataset_type=\"text/csv\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `ModelConfig` object communicates information about your trained model. To avoid additional traffic to your production models, SageMaker Clarify sets up and tears down a dedicated endpoint when processing.\n",
    "* `instance_type` and `instance_count` specify your preferred instance type and instance count used to run your model on during SageMaker Clarify's processing. If you add more instances, we recommend that you also increase the number of instances in the model configuration. This is to prevent the processing instances from being bottlenecked by the shadow endpoint. \n",
    "* `accept_type` denotes the endpoint response payload format, and `content_type` denotes the payload format of request to the endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_config = clarify.ModelConfig(\n",
    "    model_name=model_name,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    instance_count=2,\n",
    "    accept_type=\"text/csv\",\n",
    "    content_type=\"text/csv\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A `ModelPredictedLabelConfig` provides information on the format of your predictions. XGBoost model outputs probabilities of samples, so SageMaker Clarify invokes the endpoint then uses `probability_threshold` to convert the probability to binary labels for bias analysis. Prediction above the threshold is interpreted as label value `1` and below or equal as label value `0`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_config = clarify.ModelPredictedLabelConfig(probability_threshold=0.8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Writing BiasConfig\n",
    "SageMaker Clarify also needs information on what the sensitive columns (`facets`) are, what the sensitive features (`facet_values_or_threshold`) may be, and what the desirable outcomes are (`label_values_or_threshold`).\n",
    "SageMaker Clarify can handle both categorical and continuous data for `facet_values_or_threshold` and for `label_values_or_threshold`. In this case we are using categorical data.\n",
    "\n",
    "We specify this information in the `BiasConfig` API. Here that the positive outcome is earning >$50,000, Sex is a sensitive category, and Female respondents are the sensitive group. `group_name` is used to form subgroups for the measurement of Conditional Demographic Disparity in Labels (CDDL) and Conditional Demographic Disparity in Predicted Labels (CDDPL) with regards to Simpson’s paradox."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_config = clarify.BiasConfig(\n",
    "    label_values_or_threshold=[1], facet_name=\"Sex\", facet_values_or_threshold=[0], group_name=\"Age\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Pre-training Bias\n",
    "Bias can be present in your data before any model training occurs. Inspecting your data for bias before training begins can help detect any data collection gaps, inform your feature engineering, and hep you understand what societal biases the data may reflect.\n",
    "\n",
    "Computing pre-training bias metrics does not require a trained model.\n",
    "\n",
    "#### Post-training Bias\n",
    "Computing post-training bias metrics does require a trained model.\n",
    "\n",
    "Unbiased training data (as determined by concepts of fairness measured by bias metric) may still result in biased model predictions after training. Whether this occurs depends on several factors including hyperparameter choices.\n",
    "\n",
    "\n",
    "You can run these options separately with `run_pre_training_bias()` and `run_post_training_bias()` or at the same time with `run_bias()` as shown below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clarify_processor.run_bias(\n",
    "    data_config=bias_data_config,\n",
    "    bias_config=bias_config,\n",
    "    model_config=model_config,\n",
    "    model_predicted_label_config=predictions_config,\n",
    "    pre_training_methods=\"all\",\n",
    "    post_training_methods=\"all\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Viewing the Bias Report\n",
    "In Studio, you can view the results under the experiments tab.\n",
    "\n",
    "<img src=\"./recordings/bias_report.gif\">\n",
    "\n",
    "Each bias metric has detailed explanations with examples that you can explore.\n",
    "\n",
    "<img src=\"./recordings/bias_detail.gif\">\n",
    "\n",
    "You could also summarize the results in a handy table!\n",
    "\n",
    "<img src=\"./recordings/bias_report_chart.gif\">\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're not a Studio user yet, you can access the bias report in pdf, html and ipynb formats in the following S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bias_report_output_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explaining Predictions\n",
    "There are expanding business needs and legislative regulations that require explanations of _why_ a model made the decision it did. SageMaker Clarify uses SHAP to explain the contribution that each input feature makes to the final decision."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Kernel SHAP algorithm requires a baseline (also known as background dataset). Baseline dataset type shall be the same as `dataset_type` of `DataConfig`, and baseline samples shall only include features. By definition, `baseline` should either be a S3 URI to the baseline dataset file, or an in-place list of samples. In this case we chose the latter, and put the first sample of the test dataset to the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "shap_config = clarify.SHAPConfig(\n",
    "    baseline=[test_features.iloc[0].values.tolist()],\n",
    "    num_samples=15,\n",
    "    agg_method=\"mean_abs\",\n",
    "    save_local_shap_values=True,\n",
    ")\n",
    "\n",
    "explainability_output_path = \"s3://{}/{}/clarify-explainability\".format(bucket, prefix)\n",
    "explainability_data_config = clarify.DataConfig(\n",
    "    s3_data_input_path=train_uri,\n",
    "    s3_output_path=explainability_output_path,\n",
    "    label=\"Target\",\n",
    "    headers=training_data.columns.to_list(),\n",
    "    dataset_type=\"text/csv\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clarify_processor.run_explainability(\n",
    "    data_config=explainability_data_config,\n",
    "    model_config=model_config,\n",
    "    explainability_config=shap_config,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Viewing the Explainability Report\n",
    "As with the bias report, you can view the explainability report in Studio under the experiments tab\n",
    "\n",
    "\n",
    "<img src=\"./recordings/explainability_detail.gif\">\n",
    "\n",
    "The Model Insights tab contains direct links to the report and model insights.\n",
    "\n",
    "If you're not a Studio user yet, as with the Bias Report, you can access this report at the following S3 bucket."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "explainability_output_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Analysis of local explanations\n",
    "It is possible to visualize the the local explanations for single examples in your dataset. You can use the obtained results from running Kernel SHAP algorithm for global explanations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "analysis_result_json = sagemaker.s3.S3Downloader.read_file(\n",
    "    explainability_output_path + \"/analysis.json\"\n",
    ")\n",
    "analysis_result = json.loads(analysis_result_json)\n",
    "shap_values = analysis_result[\"explanations\"][\"kernel_shap\"][\"label0\"][\"global_shap_values\"]\n",
    "features = pd.Series(shap_values)\n",
    "feature_names = features.index\n",
    "feature_names"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With Clarify Spark jobs, the output files that contain local SHAP values will be split into multiple files. You will need to collate them before you can visualize them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_s3 = S3FileSystem()\n",
    "s3_files = _s3.glob(\n",
    "    \"{}/**.{}\".format(explainability_output_path + \"/explanations_shap/out.csv\", \"csv\")\n",
    ")\n",
    "s3_files = [\"s3://{}\".format(s3_file) for s3_file in s3_files]\n",
    "print(f\"Found {len(s3_files)} files in S3\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "local_shap_values = pd.DataFrame()\n",
    "for file in s3_files:\n",
    "    output = sagemaker.s3.S3Downloader.read_file(file)\n",
    "    df = pd.read_csv(StringIO(output), sep=\",\")\n",
    "    local_shap_values = local_shap_values.append(df, ignore_index=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Visualize local SHAP values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_features_to_display = 15\n",
    "feature_names = local_shap_values.columns\n",
    "\n",
    "fig = plt.figure(figsize=(max_features_to_display, max_features_to_display))\n",
    "low = local_shap_values.min().min()\n",
    "high = local_shap_values.max().max()\n",
    "\n",
    "i = 1\n",
    "for feature_name in feature_names:\n",
    "    plt.subplot(max_features_to_display, 1, i)\n",
    "    shap_value = local_shap_values[f\"{feature_name}\"].to_frame()\n",
    "    feature = pd.Series([feature_name] * shap_value.shape[0]).to_frame()\n",
    "    df = pd.concat([shap_value, feature], axis=1, join=\"inner\", ignore_index=True)\n",
    "    df.columns = [\"shap_value\", \"feature\"]\n",
    "    num_rows_to_display = min(df.shape[0], 500)\n",
    "    df = df.sample(num_rows_to_display)\n",
    "    ax = sns.violinplot(\n",
    "        y=\"feature\",\n",
    "        x=\"shap_value\",\n",
    "        data=df,\n",
    "        size=6,\n",
    "        color=\"#f5f5f5\",\n",
    "        inner=\"quartile\",\n",
    "        bw=0.2,\n",
    "        cut=0,\n",
    "        orient=\"h\",\n",
    "    )\n",
    "    ax.set_xlim(low, high)\n",
    "    sns.stripplot(\n",
    "        y=\"feature\",\n",
    "        x=\"shap_value\",\n",
    "        data=df,\n",
    "        size=4,\n",
    "        orient=\"h\",\n",
    "    )\n",
    "    ax.vlines(0, -1, 1, color=\"#ff0000\", linewidth=2)\n",
    "    ax.set_ylabel(\"\")\n",
    "    ax.legend([], [], frameon=False)\n",
    "    i += 1\n",
    "\n",
    "\n",
    "plt.xlabel(\"Local SHAP Values\", fontsize=14)\n",
    "plt.tight_layout()\n",
    "plt.subplots_adjust(hspace=0, wspace=0.1)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean Up\n",
    "Finally, don't forget to clean up the resources we set up and used for this demo!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session.delete_model(model_name)"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/datascience-1.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}