CategoricalVariablesEncoding.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "CategoricalVariablesEncoding.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/pea-sys/Til/blob/master/CategoricalVariablesEncoding.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CNBEJwFvW4Lk",
        "colab_type": "text"
      },
      "source": [
        "# Overview\n",
        "カテゴリ変数の取り扱いについての勉強ノート\n",
        "\n",
        "\n",
        "## Motivation\n",
        "Kaggleに取り組むにあたり学んだあれこれをアウトプットして、知識を整理する。\n",
        "\n",
        "[参考URL](https://github.com/h2oai/h2o-meetups/blob/master/2017_11_29_Feature_Engineering/Feature%20Engineering.pdf)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kwk-_IS-CFBG",
        "colab_type": "text"
      },
      "source": [
        "まずは検証データを作成"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tfuuFE-xacqx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hVBWS9WBZV_v",
        "colab_type": "code",
        "outputId": "8ab2cba2-3902-4162-b5fd-f69205fcfcba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        }
      },
      "source": [
        "#df = pd.DataFrame([\"A\",\"A\",\"A\",\"A\",\"B\",\"B\",\"B\",\"C\",\"C\"],dtype='category')\n",
        "df = pd.DataFrame({'Feature':[\"A\",\"A\",\"A\",\"A\",\"B\",\"B\",\"B\",\"C\",\"C\"],'Target':[1,1,1,0,1,0,1,0,1]})\n",
        "df"
      ],
      "execution_count": 98,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target\n",
              "0       A       1\n",
              "1       A       1\n",
              "2       A       1\n",
              "3       A       0\n",
              "4       B       1\n",
              "5       B       0\n",
              "6       B       1\n",
              "7       C       0\n",
              "8       C       1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JWYA9EdGY4IH",
        "colab_type": "text"
      },
      "source": [
        "### Label Encoding\n",
        "単純に各カテゴリ値にユニークな正数を割り当てる。\n",
        "エンコード結果は１列となるため、メモリ効率は良い。    \n",
        "大小関係のない名義尺度の変換に使用するのはＮＧ。\n",
        "決定木ベースのモデル以外では使用しない方が良いとのこと。  "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EMpuikB_WhBR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "37577e89-1f42-48df-84c1-ce200b009b4c"
      },
      "source": [
        "from sklearn.preprocessing import LabelEncoder\n",
        "le = LabelEncoder()\n",
        "le = le.fit(df['Feature'])\n",
        "df['LabelEncoder'] = le.transform(df['Feature'])\n",
        "df"
      ],
      "execution_count": 99,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  LabelEncoder\n",
              "0       A       1             0\n",
              "1       A       1             0\n",
              "2       A       1             0\n",
              "3       A       0             0\n",
              "4       B       1             1\n",
              "5       B       0             1\n",
              "6       B       1             1\n",
              "7       C       0             2\n",
              "8       C       1             2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "S0Tti11ubupu",
        "colab_type": "text"
      },
      "source": [
        "### One Hot Encoder\n",
        "カテゴリ数分の要素を確保するベクトル表現。  \n",
        "該当する要素が1となり、それ以外は0になる(One Hot)。  \n",
        "カテゴリ変数間に数的関係が見いだせないようなデータに使用する（ほとんどがそう）。\n",
        "決定木ベースモデル以外で使用する。  \n",
        "回帰分析等で使用する場合は、多重共線性を避けるため、１つカラムをドロップするらしい  (全て0=ドロップしたカラム1)  \n",
        "デメリットはカテゴリ数 * 行だけデータが増加するので、カテゴリ数が多いとメモリ消費量がネックとなる。  \n",
        "メモリは開発環境にもよるがkaggleのkernelだと700個程度のカラムデータを扱ってメモリエラーになったりするので注意が必要。  \n",
        "個人的に気になるのが、OneHotEncoderが適用されたカラムが複数ある場合に、\n",
        "元のカテゴリグループをモデルが見分けることが出来ないんじゃないか？という心配もある。  \n",
        "見分ける必要がない場合も多いとは思うけど、学習データが大量ではない場合に怖い気もする。  \n",
        "時々、OHEと略記表記されているのを見かける。  \n",
        "pandasのget_dummiesもOHEみたいな動きだが、これはデータフレームに対して行うため、  \n",
        "訓練データまたは評価データにしか現れないカテゴリ値がある場合に一旦結合しないといけないため、 \n",
        "はっきりいってダサイし、リスクがある。    \n",
        "\n",
        "Label Binarizerというエンコーダーも似たような動きをする。ここでは省略。\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-X3qcDJ4sZz_",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 212
        },
        "outputId": "d1ff6034-0bd6-4f16-815a-260a92dd9324"
      },
      "source": [
        "! pip install category_encoders"
      ],
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: category_encoders in /usr/local/lib/python3.6/dist-packages (2.0.0)\n",
            "Requirement already satisfied: patsy>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (0.5.1)\n",
            "Requirement already satisfied: scipy>=0.19.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (1.3.1)\n",
            "Requirement already satisfied: scikit-learn>=0.20.0 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (0.21.3)\n",
            "Requirement already satisfied: pandas>=0.21.1 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (0.24.2)\n",
            "Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (1.16.4)\n",
            "Requirement already satisfied: statsmodels>=0.6.1 in /usr/local/lib/python3.6/dist-packages (from category_encoders) (0.10.1)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from patsy>=0.4.1->category_encoders) (1.12.0)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn>=0.20.0->category_encoders) (0.13.2)\n",
            "Requirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/dist-packages (from pandas>=0.21.1->category_encoders) (2.5.3)\n",
            "Requirement already satisfied: pytz>=2011k in /usr/local/lib/python3.6/dist-packages (from pandas>=0.21.1->category_encoders) (2018.9)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "g88RQfLGbLDv",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "c80e2719-e351-41d8-a3db-49af86b0270a"
      },
      "source": [
        "#sikit-learnのOHEは一旦カテゴリ値を数値にしないといけないので面倒\n",
        "#社会的には新しいライブラリをインストールする方がリスキーかもしれないが\n",
        "import category_encoders as ce\n",
        "list_cols = ['Feature']\n",
        "ce_ohe = ce.OneHotEncoder(cols=list_cols)\n",
        "df = pd.concat([df,ce_ohe.fit_transform(df['Feature'])],axis=1)\n",
        "df\n"
      ],
      "execution_count": 101,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "      <th>Feature_1</th>\n",
              "      <th>Feature_2</th>\n",
              "      <th>Feature_3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  LabelEncoder  Feature_1  Feature_2  Feature_3\n",
              "0       A       1             0          1          0          0\n",
              "1       A       1             0          1          0          0\n",
              "2       A       1             0          1          0          0\n",
              "3       A       0             0          1          0          0\n",
              "4       B       1             1          0          1          0\n",
              "5       B       0             1          0          1          0\n",
              "6       B       1             1          0          1          0\n",
              "7       C       0             2          0          0          1\n",
              "8       C       1             2          0          0          1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 101
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7RgN4RdnzI8c",
        "colab_type": "text"
      },
      "source": [
        "### Count Encoding\n",
        "データに含まれるカテゴリ変数の出現回数を数える。  \n",
        "系列的な特徴を作りたい時に使うかも  \n",
        "予兆や再発予知、リピータ検出などに良いかもしれない。  \n",
        "モデルに学習させるための積極的なエンコーディングだと思う。  \n",
        "テストデータのLeakがあるので、ホールドアウトしてから行う。\n",
        "KaggleのKernelだとバリデーション切る前にやってるのが結構ある。  \n",
        "多分ダメだと思う。  \n",
        "が、私もまだ機械学習を学んで半年なので自信はない。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ToIfXE-Kz_vq",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "22e748e1-c591-4dff-d25a-3869d10b037e"
      },
      "source": [
        "df['Feature_Count'] = df.groupby('Feature')['Target'].transform('count')\n",
        "df"
      ],
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "      <th>Feature_1</th>\n",
              "      <th>Feature_2</th>\n",
              "      <th>Feature_3</th>\n",
              "      <th>Feature_Count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  LabelEncoder  Feature_1  Feature_2  Feature_3  Feature_Count\n",
              "0       A       1             0          1          0          0              4\n",
              "1       A       1             0          1          0          0              4\n",
              "2       A       1             0          1          0          0              4\n",
              "3       A       0             0          1          0          0              4\n",
              "4       B       1             1          0          1          0              3\n",
              "5       B       0             1          0          1          0              3\n",
              "6       B       1             1          0          1          0              3\n",
              "7       C       0             2          0          0          1              2\n",
              "8       C       1             2          0          0          1              2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 102
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tyoQu7m15kXJ",
        "colab_type": "text"
      },
      "source": [
        "### LabelCount Encoding\n",
        "カテゴリの出現回数を調べ、順位を特徴量とする。  \n",
        "どういう時に使うか直感的理解は得ていない。  \n",
        "頻出データの重み係数を小さくする時に役立ったりするのかもしれない？\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0xVoVAXL7sTY",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "90becd50-ddfc-4865-9017-23a44fbbfdff"
      },
      "source": [
        "count_rank = df.groupby('Feature')['Target'].count().rank(ascending=False)\n",
        "df['Feature_LabelCount'] = df['Feature'].map(count_rank)\n",
        "df"
      ],
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "      <th>Feature_1</th>\n",
              "      <th>Feature_2</th>\n",
              "      <th>Feature_3</th>\n",
              "      <th>Feature_Count</th>\n",
              "      <th>Feature_LabelCount</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  LabelEncoder  ...  Feature_3  Feature_Count  Feature_LabelCount\n",
              "0       A       1             0  ...          0              4                 1.0\n",
              "1       A       1             0  ...          0              4                 1.0\n",
              "2       A       1             0  ...          0              4                 1.0\n",
              "3       A       0             0  ...          0              4                 1.0\n",
              "4       B       1             1  ...          0              3                 2.0\n",
              "5       B       0             1  ...          0              3                 2.0\n",
              "6       B       1             1  ...          0              3                 2.0\n",
              "7       C       0             2  ...          1              2                 3.0\n",
              "8       C       1             2  ...          1              2                 3.0\n",
              "\n",
              "[9 rows x 8 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7IdSjE-ZFQDA",
        "colab_type": "text"
      },
      "source": [
        "### Target Encoding / Target Mean Encoding / Likelihood Encoding\n",
        "目的変数の値に応じて特徴量を数値に変換する。\n",
        "目的変数がBoolean表現の時\n",
        "カテゴリごとに目的変数=Trueの確率を特徴量とする 。 \n",
        "Kaggleでよく見るし、実際自分も使用して精度は上がった。  \n",
        "テストデータのLeakに注意。  \n",
        "エンコーディングデータの寄与度みたいなイメージ。  \n",
        "モデルに対して、各カテゴリ変数の重要度をダイレクトに伝えているイメージ。\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EkRWOwQSHIWH",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "7a18e104-9a2e-4b0c-8d24-2ae772e3aefd"
      },
      "source": [
        "df['Feature_Target'] = df.groupby('Feature')['Target'].transform('mean')\n",
        "df"
      ],
      "execution_count": 104,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "      <th>Feature_1</th>\n",
              "      <th>Feature_2</th>\n",
              "      <th>Feature_3</th>\n",
              "      <th>Feature_Count</th>\n",
              "      <th>Feature_LabelCount</th>\n",
              "      <th>Feature_Target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.500000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.500000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  ...  Feature_LabelCount  Feature_Target\n",
              "0       A       1  ...                 1.0        0.750000\n",
              "1       A       1  ...                 1.0        0.750000\n",
              "2       A       1  ...                 1.0        0.750000\n",
              "3       A       0  ...                 1.0        0.750000\n",
              "4       B       1  ...                 2.0        0.666667\n",
              "5       B       0  ...                 2.0        0.666667\n",
              "6       B       1  ...                 2.0        0.666667\n",
              "7       C       0  ...                 3.0        0.500000\n",
              "8       C       1  ...                 3.0        0.500000\n",
              "\n",
              "[9 rows x 9 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 104
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eUoAILUlOg32",
        "colab_type": "text"
      },
      "source": [
        "### Feature Hashing\n",
        "ジャンル的にエンコーディングと言って良いのか不明  \n",
        "ハッシュを使って次元削減を行う 。高速かつメモリ消費量が少ない 。  \n",
        "次元削減はできればやりたくない。  \n",
        "NLPなら使えそう。  \n",
        "データが良くなかったけど３つのカテゴリ変数が２つのカラムに収まっている。"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bozunDZ5PbDC",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "outputId": "e653cfcf-c27a-4457-e3b4-9c134800422b"
      },
      "source": [
        "from sklearn.feature_extraction import FeatureHasher\n",
        "h = FeatureHasher(n_features=2,input_type=\"string\")\n",
        "f = h.transform(df['Feature'])\n",
        "df = pd.concat([df,pd.DataFrame(f.toarray())],axis = 1)\n",
        "df"
      ],
      "execution_count": 105,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature</th>\n",
              "      <th>Target</th>\n",
              "      <th>LabelEncoder</th>\n",
              "      <th>Feature_1</th>\n",
              "      <th>Feature_2</th>\n",
              "      <th>Feature_3</th>\n",
              "      <th>Feature_Count</th>\n",
              "      <th>Feature_LabelCount</th>\n",
              "      <th>Feature_Target</th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>A</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>A</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.750000</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>B</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>B</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>-1.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>C</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-1.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>C</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0.0</td>\n",
              "      <td>-1.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  Feature  Target  LabelEncoder  ...  Feature_Target    0    1\n",
              "0       A       1             0  ...        0.750000  1.0  0.0\n",
              "1       A       1             0  ...        0.750000  1.0  0.0\n",
              "2       A       1             0  ...        0.750000  1.0  0.0\n",
              "3       A       0             0  ...        0.750000  1.0  0.0\n",
              "4       B       1             1  ...        0.666667 -1.0  0.0\n",
              "5       B       0             1  ...        0.666667 -1.0  0.0\n",
              "6       B       1             1  ...        0.666667 -1.0  0.0\n",
              "7       C       0             2  ...        0.500000  0.0 -1.0\n",
              "8       C       1             2  ...        0.500000  0.0 -1.0\n",
              "\n",
              "[9 rows x 11 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 105
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_XsskEoymBAH",
        "colab_type": "text"
      },
      "source": [
        "## 感想\n",
        "エンコーディングはモデルが学習しやすいように行うことが肝要。\n",
        "モデルによってエンコーディング方法を変えるといったことは、  \n",
        "書籍では見受けられなかったが  Kaggleでは当たり前に行われている。  \n",
        "直感的理解とは裏腹に意外な特徴が効いたりするので、結局やれることはなるべく全部やりましょうってことになりそう。  \n",
        "今回はカテゴリ変数の取り扱いということで、これ位にしておく。  \n",
        "ノートブック書いてて、[ここ](https://contrib.scikit-learn.org/categorical-encoding/)が気になったので  \n",
        "時間を見つけてチュートリアルやろうと思う。\n",
        "\n",
        "\n",
        "\n",
        "#### 疑問点\n",
        "* オンライン学習などでカテゴリ変数が増える可能性がある場合、OHEは使えないのではないか(学習しなおし）？\n",
        "* カテゴリ変数をサポートしているモデルは内部で、どのエンコーディングを使用しているのか\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PjCU0v8Oy0DT",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}