notebooks/models/latent_class_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of use of Latent Class MNL\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/artefactory/choice-learn/blob/main/notebooks/models/latent_class_model.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install necessary requirements\n",
    "\n",
    "# If you run this notebook on Google Colab, or in standalone mode, you need to install the required packages.\n",
    "# Uncomment the following lines:\n",
    "\n",
    "# !pip install choice-learn\n",
    "\n",
    "# If you run the notebook within the GitHub repository, you need to run the following lines, that can skipped otherwise:\n",
    "import os\n",
    "import sys\n",
    "\n",
    "sys.path.append(\"../../\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = \"\"\n",
    "\n",
    "import matplotlib as mpl\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use the Electricity Dataset used in this [tutorial](https://rpubs.com/msarrias1986/335556)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.datasets import load_electricity\n",
    "\n",
    "elec_dataset = load_electricity(as_frame=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.models.simple_mnl import SimpleMNL\n",
    "from choice_learn.models.latent_class_mnl import LatentClassSimpleMNL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model = LatentClassSimpleMNL(n_latent_classes=3, fit_method=\"mle\", optimizer=\"lbfgs\", epochs=1000, lbfgs_tolerance=1e-20)\n",
    "hist, results = lc_model.fit(elec_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Latent Class Model weights:\")\n",
    "print(\"Classes Logits:\", lc_model.latent_logits)\n",
    "for i in range(3):\n",
    "    print(\"\\n\")\n",
    "    print(f\"Model Nb {i}, weights:\", lc_model.models[i].weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nll = (lc_model.evaluate(elec_dataset) * len(elec_dataset)).numpy()\n",
    "print(f\"Negative Log-Likelihood: {nll}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "keep_output": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using L-BFGS optimizer, setting up .fit() function\n",
      "Using L-BFGS optimizer, setting up .fit() function\n",
      "Using L-BFGS optimizer, setting up .fit() function\n",
      "WARNING:tensorflow:5 out of the last 5 calls to <function pfor.<locals>.f at 0x7fa18fb41af0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:5 out of the last 5 calls to <function pfor.<locals>.f at 0x7fa18fb41af0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:6 out of the last 6 calls to <function pfor.<locals>.f at 0x7fa18fb92280> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:6 out of the last 6 calls to <function pfor.<locals>.f at 0x7fa18fb92280> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for  more details.\n",
      "/var/folders/zz/r1py7zhj35q75v09h8_42nzh0000gp/T/ipykernel_27459/1263996749.py:4: MatplotlibDeprecationWarning: The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use ``matplotlib.colormaps[name]`` or ``matplotlib.colormaps.get_cmap(obj)`` instead.\n",
      "  cmap = mpl.cm.get_cmap(\"Set1\")\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<style type=\"text/css\">\n",
       "#T_ea03d_row0_col0, #T_ea03d_row0_col1, #T_ea03d_row0_col2, #T_ea03d_row0_col3, #T_ea03d_row0_col4, #T_ea03d_row0_col5, #T_ea03d_row1_col0, #T_ea03d_row1_col1, #T_ea03d_row1_col2, #T_ea03d_row1_col3, #T_ea03d_row1_col4, #T_ea03d_row1_col5, #T_ea03d_row2_col0, #T_ea03d_row2_col1, #T_ea03d_row2_col2, #T_ea03d_row2_col3, #T_ea03d_row2_col4, #T_ea03d_row2_col5, #T_ea03d_row3_col0, #T_ea03d_row3_col1, #T_ea03d_row3_col2, #T_ea03d_row3_col3, #T_ea03d_row3_col4, #T_ea03d_row3_col5, #T_ea03d_row4_col0, #T_ea03d_row4_col1, #T_ea03d_row4_col2, #T_ea03d_row4_col3, #T_ea03d_row4_col4, #T_ea03d_row4_col5, #T_ea03d_row5_col0, #T_ea03d_row5_col1, #T_ea03d_row5_col2, #T_ea03d_row5_col3, #T_ea03d_row5_col4, #T_ea03d_row5_col5 {\n",
       "  background-color: #e41a1c;\n",
       "}\n",
       "#T_ea03d_row6_col0, #T_ea03d_row6_col1, #T_ea03d_row6_col2, #T_ea03d_row6_col3, #T_ea03d_row6_col4, #T_ea03d_row6_col5, #T_ea03d_row7_col0, #T_ea03d_row7_col1, #T_ea03d_row7_col2, #T_ea03d_row7_col3, #T_ea03d_row7_col4, #T_ea03d_row7_col5, #T_ea03d_row8_col0, #T_ea03d_row8_col1, #T_ea03d_row8_col2, #T_ea03d_row8_col3, #T_ea03d_row8_col4, #T_ea03d_row8_col5, #T_ea03d_row9_col0, #T_ea03d_row9_col1, #T_ea03d_row9_col2, #T_ea03d_row9_col3, #T_ea03d_row9_col4, #T_ea03d_row9_col5, #T_ea03d_row10_col0, #T_ea03d_row10_col1, #T_ea03d_row10_col2, #T_ea03d_row10_col3, #T_ea03d_row10_col4, #T_ea03d_row10_col5, #T_ea03d_row11_col0, #T_ea03d_row11_col1, #T_ea03d_row11_col2, #T_ea03d_row11_col3, #T_ea03d_row11_col4, #T_ea03d_row11_col5 {\n",
       "  background-color: #377eb8;\n",
       "}\n",
       "#T_ea03d_row12_col0, #T_ea03d_row12_col1, #T_ea03d_row12_col2, #T_ea03d_row12_col3, #T_ea03d_row12_col4, #T_ea03d_row12_col5, #T_ea03d_row13_col0, #T_ea03d_row13_col1, #T_ea03d_row13_col2, #T_ea03d_row13_col3, #T_ea03d_row13_col4, #T_ea03d_row13_col5, #T_ea03d_row14_col0, #T_ea03d_row14_col1, #T_ea03d_row14_col2, #T_ea03d_row14_col3, #T_ea03d_row14_col4, #T_ea03d_row14_col5, #T_ea03d_row15_col0, #T_ea03d_row15_col1, #T_ea03d_row15_col2, #T_ea03d_row15_col3, #T_ea03d_row15_col4, #T_ea03d_row15_col5, #T_ea03d_row16_col0, #T_ea03d_row16_col1, #T_ea03d_row16_col2, #T_ea03d_row16_col3, #T_ea03d_row16_col4, #T_ea03d_row16_col5, #T_ea03d_row17_col0, #T_ea03d_row17_col1, #T_ea03d_row17_col2, #T_ea03d_row17_col3, #T_ea03d_row17_col4, #T_ea03d_row17_col5 {\n",
       "  background-color: #4daf4a;\n",
       "}\n",
       "</style>\n",
       "<table id=\"T_ea03d\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th class=\"blank level0\" >&nbsp;</th>\n",
       "      <th id=\"T_ea03d_level0_col0\" class=\"col_heading level0 col0\" >Latent Class</th>\n",
       "      <th id=\"T_ea03d_level0_col1\" class=\"col_heading level0 col1\" >Coefficient Name</th>\n",
       "      <th id=\"T_ea03d_level0_col2\" class=\"col_heading level0 col2\" >Coefficient Estimation</th>\n",
       "      <th id=\"T_ea03d_level0_col3\" class=\"col_heading level0 col3\" >Std. Err</th>\n",
       "      <th id=\"T_ea03d_level0_col4\" class=\"col_heading level0 col4\" >z_value</th>\n",
       "      <th id=\"T_ea03d_level0_col5\" class=\"col_heading level0 col5\" >P(.>z)</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "      <td id=\"T_ea03d_row0_col0\" class=\"data row0 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row0_col1\" class=\"data row0 col1\" >Weights_items_features_0</td>\n",
       "      <td id=\"T_ea03d_row0_col2\" class=\"data row0 col2\" >-0.675645</td>\n",
       "      <td id=\"T_ea03d_row0_col3\" class=\"data row0 col3\" >0.023987</td>\n",
       "      <td id=\"T_ea03d_row0_col4\" class=\"data row0 col4\" >-28.167109</td>\n",
       "      <td id=\"T_ea03d_row0_col5\" class=\"data row0 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
       "      <td id=\"T_ea03d_row1_col0\" class=\"data row1 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row1_col1\" class=\"data row1 col1\" >Weights_items_features_1</td>\n",
       "      <td id=\"T_ea03d_row1_col2\" class=\"data row1 col2\" >-0.060604</td>\n",
       "      <td id=\"T_ea03d_row1_col3\" class=\"data row1 col3\" >0.008162</td>\n",
       "      <td id=\"T_ea03d_row1_col4\" class=\"data row1 col4\" >-7.424849</td>\n",
       "      <td id=\"T_ea03d_row1_col5\" class=\"data row1 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
       "      <td id=\"T_ea03d_row2_col0\" class=\"data row2 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row2_col1\" class=\"data row2 col1\" >Weights_items_features_2</td>\n",
       "      <td id=\"T_ea03d_row2_col2\" class=\"data row2 col2\" >1.851951</td>\n",
       "      <td id=\"T_ea03d_row2_col3\" class=\"data row2 col3\" >0.054914</td>\n",
       "      <td id=\"T_ea03d_row2_col4\" class=\"data row2 col4\" >33.724579</td>\n",
       "      <td id=\"T_ea03d_row2_col5\" class=\"data row2 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
       "      <td id=\"T_ea03d_row3_col0\" class=\"data row3 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row3_col1\" class=\"data row3 col1\" >Weights_items_features_3</td>\n",
       "      <td id=\"T_ea03d_row3_col2\" class=\"data row3 col2\" >1.322549</td>\n",
       "      <td id=\"T_ea03d_row3_col3\" class=\"data row3 col3\" >0.048159</td>\n",
       "      <td id=\"T_ea03d_row3_col4\" class=\"data row3 col4\" >27.462420</td>\n",
       "      <td id=\"T_ea03d_row3_col5\" class=\"data row3 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
       "      <td id=\"T_ea03d_row4_col0\" class=\"data row4 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row4_col1\" class=\"data row4 col1\" >Weights_items_features_4</td>\n",
       "      <td id=\"T_ea03d_row4_col2\" class=\"data row4 col2\" >-5.857089</td>\n",
       "      <td id=\"T_ea03d_row4_col3\" class=\"data row4 col3\" >0.191162</td>\n",
       "      <td id=\"T_ea03d_row4_col4\" class=\"data row4 col4\" >-30.639460</td>\n",
       "      <td id=\"T_ea03d_row4_col5\" class=\"data row4 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
       "      <td id=\"T_ea03d_row5_col0\" class=\"data row5 col0\" >0</td>\n",
       "      <td id=\"T_ea03d_row5_col1\" class=\"data row5 col1\" >Weights_items_features_5</td>\n",
       "      <td id=\"T_ea03d_row5_col2\" class=\"data row5 col2\" >-6.513206</td>\n",
       "      <td id=\"T_ea03d_row5_col3\" class=\"data row5 col3\" >0.195680</td>\n",
       "      <td id=\"T_ea03d_row5_col4\" class=\"data row5 col4\" >-33.285046</td>\n",
       "      <td id=\"T_ea03d_row5_col5\" class=\"data row5 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
       "      <td id=\"T_ea03d_row6_col0\" class=\"data row6 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row6_col1\" class=\"data row6 col1\" >Weights_items_features_0</td>\n",
       "      <td id=\"T_ea03d_row6_col2\" class=\"data row6 col2\" >-1.817566</td>\n",
       "      <td id=\"T_ea03d_row6_col3\" class=\"data row6 col3\" >0.077771</td>\n",
       "      <td id=\"T_ea03d_row6_col4\" class=\"data row6 col4\" >-23.370796</td>\n",
       "      <td id=\"T_ea03d_row6_col5\" class=\"data row6 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
       "      <td id=\"T_ea03d_row7_col0\" class=\"data row7 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row7_col1\" class=\"data row7 col1\" >Weights_items_features_1</td>\n",
       "      <td id=\"T_ea03d_row7_col2\" class=\"data row7 col2\" >-1.726365</td>\n",
       "      <td id=\"T_ea03d_row7_col3\" class=\"data row7 col3\" >0.058838</td>\n",
       "      <td id=\"T_ea03d_row7_col4\" class=\"data row7 col4\" >-29.340986</td>\n",
       "      <td id=\"T_ea03d_row7_col5\" class=\"data row7 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
       "      <td id=\"T_ea03d_row8_col0\" class=\"data row8 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row8_col1\" class=\"data row8 col1\" >Weights_items_features_2</td>\n",
       "      <td id=\"T_ea03d_row8_col2\" class=\"data row8 col2\" >3.696567</td>\n",
       "      <td id=\"T_ea03d_row8_col3\" class=\"data row8 col3\" >0.160258</td>\n",
       "      <td id=\"T_ea03d_row8_col4\" class=\"data row8 col4\" >23.066404</td>\n",
       "      <td id=\"T_ea03d_row8_col5\" class=\"data row8 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
       "      <td id=\"T_ea03d_row9_col0\" class=\"data row9 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row9_col1\" class=\"data row9 col1\" >Weights_items_features_3</td>\n",
       "      <td id=\"T_ea03d_row9_col2\" class=\"data row9 col2\" >4.111840</td>\n",
       "      <td id=\"T_ea03d_row9_col3\" class=\"data row9 col3\" >0.157179</td>\n",
       "      <td id=\"T_ea03d_row9_col4\" class=\"data row9 col4\" >26.160225</td>\n",
       "      <td id=\"T_ea03d_row9_col5\" class=\"data row9 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
       "      <td id=\"T_ea03d_row10_col0\" class=\"data row10 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row10_col1\" class=\"data row10 col1\" >Weights_items_features_4</td>\n",
       "      <td id=\"T_ea03d_row10_col2\" class=\"data row10 col2\" >-26.693516</td>\n",
       "      <td id=\"T_ea03d_row10_col3\" class=\"data row10 col3\" >3.274723</td>\n",
       "      <td id=\"T_ea03d_row10_col4\" class=\"data row10 col4\" >-8.151381</td>\n",
       "      <td id=\"T_ea03d_row10_col5\" class=\"data row10 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row11\" class=\"row_heading level0 row11\" >11</th>\n",
       "      <td id=\"T_ea03d_row11_col0\" class=\"data row11 col0\" >1</td>\n",
       "      <td id=\"T_ea03d_row11_col1\" class=\"data row11 col1\" >Weights_items_features_5</td>\n",
       "      <td id=\"T_ea03d_row11_col2\" class=\"data row11 col2\" >-14.925840</td>\n",
       "      <td id=\"T_ea03d_row11_col3\" class=\"data row11 col3\" >0.634699</td>\n",
       "      <td id=\"T_ea03d_row11_col4\" class=\"data row11 col4\" >-23.516403</td>\n",
       "      <td id=\"T_ea03d_row11_col5\" class=\"data row11 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row12\" class=\"row_heading level0 row12\" >12</th>\n",
       "      <td id=\"T_ea03d_row12_col0\" class=\"data row12 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row12_col1\" class=\"data row12 col1\" >Weights_items_features_0</td>\n",
       "      <td id=\"T_ea03d_row12_col2\" class=\"data row12 col2\" >-2.104791</td>\n",
       "      <td id=\"T_ea03d_row12_col3\" class=\"data row12 col3\" >0.104296</td>\n",
       "      <td id=\"T_ea03d_row12_col4\" class=\"data row12 col4\" >-20.181009</td>\n",
       "      <td id=\"T_ea03d_row12_col5\" class=\"data row12 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row13\" class=\"row_heading level0 row13\" >13</th>\n",
       "      <td id=\"T_ea03d_row13_col0\" class=\"data row13 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row13_col1\" class=\"data row13 col1\" >Weights_items_features_1</td>\n",
       "      <td id=\"T_ea03d_row13_col2\" class=\"data row13 col2\" >-1.652622</td>\n",
       "      <td id=\"T_ea03d_row13_col3\" class=\"data row13 col3\" >0.073820</td>\n",
       "      <td id=\"T_ea03d_row13_col4\" class=\"data row13 col4\" >-22.387188</td>\n",
       "      <td id=\"T_ea03d_row13_col5\" class=\"data row13 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row14\" class=\"row_heading level0 row14\" >14</th>\n",
       "      <td id=\"T_ea03d_row14_col0\" class=\"data row14 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row14_col1\" class=\"data row14 col1\" >Weights_items_features_2</td>\n",
       "      <td id=\"T_ea03d_row14_col2\" class=\"data row14 col2\" >-5.554287</td>\n",
       "      <td id=\"T_ea03d_row14_col3\" class=\"data row14 col3\" >0.245318</td>\n",
       "      <td id=\"T_ea03d_row14_col4\" class=\"data row14 col4\" >-22.641151</td>\n",
       "      <td id=\"T_ea03d_row14_col5\" class=\"data row14 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row15\" class=\"row_heading level0 row15\" >15</th>\n",
       "      <td id=\"T_ea03d_row15_col0\" class=\"data row15 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row15_col1\" class=\"data row15 col1\" >Weights_items_features_3</td>\n",
       "      <td id=\"T_ea03d_row15_col2\" class=\"data row15 col2\" >-13.565555</td>\n",
       "      <td id=\"T_ea03d_row15_col3\" class=\"data row15 col3\" >0.544168</td>\n",
       "      <td id=\"T_ea03d_row15_col4\" class=\"data row15 col4\" >-24.928965</td>\n",
       "      <td id=\"T_ea03d_row15_col5\" class=\"data row15 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row16\" class=\"row_heading level0 row16\" >16</th>\n",
       "      <td id=\"T_ea03d_row16_col0\" class=\"data row16 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row16_col1\" class=\"data row16 col1\" >Weights_items_features_4</td>\n",
       "      <td id=\"T_ea03d_row16_col2\" class=\"data row16 col2\" >-9.794930</td>\n",
       "      <td id=\"T_ea03d_row16_col3\" class=\"data row16 col3\" >0.631004</td>\n",
       "      <td id=\"T_ea03d_row16_col4\" class=\"data row16 col4\" >-15.522781</td>\n",
       "      <td id=\"T_ea03d_row16_col5\" class=\"data row16 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th id=\"T_ea03d_level0_row17\" class=\"row_heading level0 row17\" >17</th>\n",
       "      <td id=\"T_ea03d_row17_col0\" class=\"data row17 col0\" >2</td>\n",
       "      <td id=\"T_ea03d_row17_col1\" class=\"data row17 col1\" >Weights_items_features_5</td>\n",
       "      <td id=\"T_ea03d_row17_col2\" class=\"data row17 col2\" >-12.126673</td>\n",
       "      <td id=\"T_ea03d_row17_col3\" class=\"data row17 col3\" >0.681118</td>\n",
       "      <td id=\"T_ea03d_row17_col4\" class=\"data row17 col4\" >-17.804060</td>\n",
       "      <td id=\"T_ea03d_row17_col5\" class=\"data row17 col5\" >0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n"
      ],
      "text/plain": [
       "<pandas.io.formats.style.Styler at 0x7fa1a84488b0>"
      ]
     },
     "execution_count": null,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "report = lc_model.compute_report(elec_dataset)\n",
    "\n",
    "def format_color_groups(df):\n",
    "    cmap = mpl.cm.get_cmap(\"Set1\")\n",
    "    colors = [mpl.colors.rgb2hex(cmap(i)) for i in range(cmap.N)]\n",
    "    x = df.copy()\n",
    "    factors = list(x['Latent Class'].unique())\n",
    "    i = 0\n",
    "    for factor in factors:\n",
    "        style = f'background-color: {colors[i]}'\n",
    "        x.loc[x['Latent Class'] == factor, :] = style\n",
    "        i += 1\n",
    "    return x\n",
    "\n",
    "report.style.apply(format_color_groups, axis=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2 = LatentClassSimpleMNL(n_latent_classes=3, fit_method=\"EM\", optimizer=\"lbfgs\", epochs=2000, lbfgs_tolerance=1e-6)\n",
    "hist, results = lc_model_2.fit(elec_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.latent_logits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.plot(results)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.instantiate(\n",
    "                n_items=elec_dataset.get_n_items(),\n",
    "                n_shared_features=elec_dataset.get_n_shared_features(),\n",
    "                n_items_features=elec_dataset.get_n_items_features(),\n",
    "            )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hist_logits = []\n",
    "hist_loss = []\n",
    "\n",
    "# Initialization\n",
    "init_sample_weight = np.random.rand(3, len(elec_dataset))\n",
    "init_sample_weight = init_sample_weight / np.sum(init_sample_weight, axis=0, keepdims=True)\n",
    "for i, model in enumerate(lc_model_2.models):\n",
    "    # model.instantiate()\n",
    "    model.fit(elec_dataset, sample_weight=np.clip(init_sample_weight[i], 1e-4, 1), verbose=2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.models[2].exact_nll.epsilon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_sample_weight[2].min()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.models[2].predict_probas(elec_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.math.log(1.0 * 1e-40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.models[2]._trainable_weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "myw, myl, myloss = [], [], []\n",
    "\n",
    "for i in range(10):\n",
    "    lc_model_2.weights, loss = lc_model_2._expectation(elec_dataset)\n",
    "    lc_model_2.latent_logits = lc_model_2._maximization(elec_dataset, verbose=2)\n",
    "\n",
    "    myw.append(lc_model_2.weights)\n",
    "    myl.append(lc_model_2.latent_logits)\n",
    "    myloss.append(loss)\n",
    "    if np.sum(np.isnan(self.latent_logits)) > 0:\n",
    "            print(\"Nan in logits\")\n",
    "            break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in tqdm.trange(self.epochs):\n",
    "    self.weights, loss = self._expectation(choice_dataset)\n",
    "    self.latent_logits = self._maximization(choice_dataset, verbose=verbose)\n",
    "    hist_logits.append(self.latent_logits)\n",
    "    hist_loss.append(loss)\n",
    "    if np.sum(np.isnan(self.latent_logits)) > 0:\n",
    "        print(\"Nan in logits\")\n",
    "        break\n",
    "return hist_logits, hist_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "keep_output": true
   },
   "source": [
    "## Latent Conditional Logit\n",
    "We used a very simple MNL. Here we simulate the same MNL, by using the Conditional-Logit formulation.\\\n",
    "Don't hesitate to read the conditional-MNL tutorial to better understand how to use this formulation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.models.latent_class_mnl import LatentClassConditionalLogit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2 = LatentClassConditionalLogit(n_latent_classes=3,\n",
    "                                       fit_method=\"mle\",\n",
    "                                       optimizer=\"lbfgs\",\n",
    "                                       epochs=1000,\n",
    "                                       lbfgs_tolerance=1e-12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For each feature, let's add a coefficient that is shared by all items:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model_2.add_shared_coefficient(coefficient_name=\"pf\",\n",
    "                                  feature_name=\"pf\",\n",
    "                                  items_indexes=[0, 1, 2, 3])\n",
    "lc_model_2.add_shared_coefficient(coefficient_name=\"cl\",\n",
    "                                  feature_name=\"cl\",\n",
    "                                    items_indexes=[0, 1, 2, 3])\n",
    "lc_model_2.add_shared_coefficient(coefficient_name=\"loc\",\n",
    "                                  feature_name=\"loc\",\n",
    "                                  items_indexes=[0, 1, 2, 3])\n",
    "lc_model_2.add_shared_coefficient(coefficient_name=\"wk\",\n",
    "                                feature_name=\"wk\",\n",
    "                                items_indexes=[0, 1, 2, 3])\n",
    "lc_model_2.add_shared_coefficient(coefficient_name=\"tod\",\n",
    "                                  feature_name=\"tod\",\n",
    "                                  items_indexes=[0, 1, 2, 3])\n",
    "lc_model_2.add_shared_coefficient(coefficient_name=\"seas\",\n",
    "                                  feature_name=\"seas\",\n",
    "                                  items_indexes=[0, 1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit\n",
    "hist2 = lc_model_2.fit(elec_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Negative Log-Likelihood:\", lc_model_2.evaluate(elec_dataset)*len(elec_dataset))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Latent Class Model weights:\")\n",
    "print(\"Classes Logits:\", lc_model_2.latent_logits)\n",
    "for i in range(3):\n",
    "    print(\"\\n\")\n",
    "    print(f\"Model Nb {i}, weights:\", lc_model_2.models[i].trainable_weights)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just like any ChoiceModel you can get the probabilities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lc_model.predict_probas(elec_dataset[:4])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to use more complex formulations of Latent Class models, you can directly use the *BaseLatentClassModel* from *choice_learn.models.base_model*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from choice_learn.models.latent_class_base_model import BaseLatentClassModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "manual_lc = BaseLatentClassModel(\n",
    "                                 model_class=SimpleMNL,\n",
    "                                 model_parameters={\"add_exit_choice\": False},\n",
    "                                 n_latent_classes=3,\n",
    "                                 fit_method=\"mle\",\n",
    "                                 epochs=1000,\n",
    "                                 optimizer=\"lbfgs\",\n",
    "                                 lbfgs_tolerance=1e-12\n",
    "                                 )\n",
    "manual_lc.instantiate(n_items=4,\n",
    "                      n_shared_features=0,\n",
    "                      n_items_features=6)\n",
    "manual_hist = manual_lc.fit(elec_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(manual_lc.evaluate(elec_dataset) * len(elec_dataset))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you need to go deeper, you can look [here](../choice_learn/models/latent_class_mnl.py) to see different implementations that could help you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "choice_learn",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}