submit2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.10"
    },
    "papermill": {
      "default_parameters": {},
      "duration": 346.121966,
      "end_time": "2021-06-25T22:24:11.045344",
      "environment_variables": {},
      "exception": null,
      "input_path": "__notebook__.ipynb",
      "output_path": "__notebook__.ipynb",
      "parameters": {},
      "start_time": "2021-06-25T22:18:24.923378",
      "version": "2.3.3"
    },
    "colab": {
      "name": "submit2.ipynb ",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm",
      "include_colab_link": true
    },
    "accelerator": "TPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gdollp/mlb/blob/main/submit2.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
        "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
        "papermill": {
          "duration": 0.024709,
          "end_time": "2021-06-25T22:18:33.032989",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.008280",
          "status": "completed"
        },
        "tags": [],
        "id": "resident-birthday"
      },
      "source": [
        "This notebook uses lightGBM to make predictions.\n",
        "\n",
        "We use the following features\n",
        "* playerId\n",
        "* position\n",
        "* teamId(rosters)\n",
        "* status(rosters)\n",
        "* playerBoxScores\n",
        "\n",
        "and the date 20200401~20200431 as the validation data.\n",
        "\n",
        "But I think there is room for improvement.  \n",
        "If you have better ways, I would appreciate it if you could comment on it.\n",
        "\n",
        "このnotebookではlightGBMを使って予測します。\n",
        "\n",
        "特徴量は以下のものを使用しています。\n",
        "* playerId\n",
        "* position\n",
        "* teamId(rosters)\n",
        "* status(rosters)\n",
        "* playerBoxScores\n",
        "\n",
        "20200401~20200431を日時をvalidation dataとしていますが、一考の余地がありそうです。  \n",
        "もし良さそうな方法があればコメントしていただけると幸いです。"
      ],
      "id": "resident-birthday"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "papermill": {
          "duration": 0.027024,
          "end_time": "2021-06-25T22:18:33.083611",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.056587",
          "status": "completed"
        },
        "tags": [],
        "id": "vital-water"
      },
      "source": [
        ""
      ],
      "id": "vital-water"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "papermill": {
          "duration": 0.025051,
          "end_time": "2021-06-25T22:18:33.139292",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.114241",
          "status": "completed"
        },
        "tags": [],
        "id": "operating-thunder"
      },
      "source": [
        "https://www.kaggle.com/columbia2131/mlb-lightgbm-starter-dataset-code-en-ja"
      ],
      "id": "operating-thunder"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "papermill": {
          "duration": 0.026534,
          "end_time": "2021-06-25T22:18:33.190839",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.164305",
          "status": "completed"
        },
        "tags": [],
        "id": "theoretical-damages"
      },
      "source": [
        "## About Dataset"
      ],
      "id": "theoretical-damages"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "papermill": {
          "duration": 0.02311,
          "end_time": "2021-06-25T22:18:33.237635",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.214525",
          "status": "completed"
        },
        "tags": [],
        "id": "clinical-cheat"
      },
      "source": [
        "Train.csv is stored as a csv file with each column as follows.  \n",
        "\n",
        "train.csvを以下のようにして各カラムをcsvファイルとして保管しています。"
      ],
      "id": "clinical-cheat"
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:33.300733Z",
          "iopub.status.busy": "2021-06-25T22:18:33.299967Z",
          "iopub.status.idle": "2021-06-25T22:18:33.302980Z",
          "shell.execute_reply": "2021-06-25T22:18:33.302209Z",
          "shell.execute_reply.started": "2021-06-25T21:58:13.679983Z"
        },
        "papermill": {
          "duration": 0.042271,
          "end_time": "2021-06-25T22:18:33.303166",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.260895",
          "status": "completed"
        },
        "tags": [],
        "id": "reasonable-assistant"
      },
      "source": [
        "%%capture\n",
        "\"\"\"\n",
        "!pip install pandarallel \n",
        "\n",
        "import gc\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "from pathlib import Path\n",
        "\n",
        "from pandarallel import pandarallel\n",
        "pandarallel.initialize()\n",
        "\n",
        "BASE_DIR = Path('../input/mlb-player-digital-engagement-forecasting')\n",
        "train = pd.read_csv(BASE_DIR / 'train.csv')\n",
        "\n",
        "null = np.nan\n",
        "true = True\n",
        "false = False\n",
        "\n",
        "for col in train.columns:\n",
        "\n",
        "    if col == 'date': continue\n",
        "\n",
        "    _index = train[col].notnull()\n",
        "    train.loc[_index, col] = train.loc[_index, col].parallel_apply(lambda x: eval(x))\n",
        "\n",
        "    outputs = []\n",
        "    for index, date, record in train.loc[_index, ['date', col]].itertuples():\n",
        "        _df = pd.DataFrame(record)\n",
        "        _df['index'] = index\n",
        "        _df['date'] = date\n",
        "        outputs.append(_df)\n",
        "\n",
        "    outputs = pd.concat(outputs).reset_index(drop=True)\n",
        "\n",
        "    outputs.to_csv(f'{col}_train.csv', index=False)\n",
        "    outputs.to_pickle(f'{col}_train.pkl')\n",
        "\n",
        "    del outputs\n",
        "    del train[col]\n",
        "    gc.collect()\n",
        "\"\"\""
      ],
      "id": "reasonable-assistant",
      "execution_count": 231,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "APDXL7SxlCT1",
        "outputId": "0809814a-e4b9-4e6f-c4d7-0f5dc5299a23"
      },
      "source": [
        "!pip install optuna"
      ],
      "id": "APDXL7SxlCT1",
      "execution_count": 232,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: optuna in /usr/local/lib/python3.7/dist-packages (2.8.0)\n",
            "Requirement already satisfied: alembic in /usr/local/lib/python3.7/dist-packages (from optuna) (1.6.5)\n",
            "Requirement already satisfied: scipy!=1.4.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (1.4.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (20.9)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from optuna) (4.41.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from optuna) (1.19.5)\n",
            "Requirement already satisfied: cmaes>=0.8.2 in /usr/local/lib/python3.7/dist-packages (from optuna) (0.8.2)\n",
            "Requirement already satisfied: sqlalchemy>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from optuna) (1.4.20)\n",
            "Requirement already satisfied: colorlog in /usr/local/lib/python3.7/dist-packages (from optuna) (5.0.1)\n",
            "Requirement already satisfied: cliff in /usr/local/lib/python3.7/dist-packages (from optuna) (3.8.0)\n",
            "Requirement already satisfied: python-editor>=0.3 in /usr/local/lib/python3.7/dist-packages (from alembic->optuna) (1.0.4)\n",
            "Requirement already satisfied: Mako in /usr/local/lib/python3.7/dist-packages (from alembic->optuna) (1.1.4)\n",
            "Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from alembic->optuna) (2.8.1)\n",
            "Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->optuna) (2.4.7)\n",
            "Requirement already satisfied: importlib-metadata; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from sqlalchemy>=1.1.0->optuna) (4.6.0)\n",
            "Requirement already satisfied: greenlet!=0.4.17; python_version >= \"3\" in /usr/local/lib/python3.7/dist-packages (from sqlalchemy>=1.1.0->optuna) (1.1.0)\n",
            "Requirement already satisfied: stevedore>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (3.3.0)\n",
            "Requirement already satisfied: PyYAML>=3.12 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (3.13)\n",
            "Requirement already satisfied: pbr!=2.1.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (5.6.0)\n",
            "Requirement already satisfied: PrettyTable>=0.7.2 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (2.1.0)\n",
            "Requirement already satisfied: cmd2>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from cliff->optuna) (2.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.7/dist-packages (from Mako->alembic->optuna) (2.0.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil->alembic->optuna) (1.15.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->sqlalchemy>=1.1.0->optuna) (3.4.1)\n",
            "Requirement already satisfied: typing-extensions>=3.6.4; python_version < \"3.8\" in /usr/local/lib/python3.7/dist-packages (from importlib-metadata; python_version < \"3.8\"->sqlalchemy>=1.1.0->optuna) (3.7.4.3)\n",
            "Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from PrettyTable>=0.7.2->cliff->optuna) (0.2.5)\n",
            "Requirement already satisfied: pyperclip>=1.6 in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (1.8.2)\n",
            "Requirement already satisfied: attrs>=16.3.0 in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (21.2.0)\n",
            "Requirement already satisfied: colorama>=0.3.7 in /usr/local/lib/python3.7/dist-packages (from cmd2>=1.0.0->cliff->optuna) (0.4.4)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-16T09:14:33.869905Z",
          "iopub.status.busy": "2021-06-16T09:14:33.869464Z",
          "iopub.status.idle": "2021-06-16T09:14:33.874766Z",
          "shell.execute_reply": "2021-06-16T09:14:33.873097Z",
          "shell.execute_reply.started": "2021-06-16T09:14:33.869879Z"
        },
        "papermill": {
          "duration": 0.025072,
          "end_time": "2021-06-25T22:18:33.352190",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.327118",
          "status": "completed"
        },
        "tags": [],
        "id": "frozen-alfred"
      },
      "source": [
        "## Training"
      ],
      "id": "frozen-alfred"
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:33.415350Z",
          "iopub.status.busy": "2021-06-25T22:18:33.414585Z",
          "iopub.status.idle": "2021-06-25T22:18:36.343351Z",
          "shell.execute_reply": "2021-06-25T22:18:36.343900Z",
          "shell.execute_reply.started": "2021-06-25T21:58:13.707952Z"
        },
        "papermill": {
          "duration": 2.967463,
          "end_time": "2021-06-25T22:18:36.344129",
          "exception": false,
          "start_time": "2021-06-25T22:18:33.376666",
          "status": "completed"
        },
        "tags": [],
        "id": "concrete-cruise"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "from pathlib import Path\n",
        "from sklearn.metrics import mean_absolute_error\n",
        "from datetime import timedelta\n",
        "from functools import reduce\n",
        "from tqdm import tqdm\n",
        "#import optuna.integration.lightgbm as lgbm\n",
        "import lightgbm as lgbm\n",
        "#import mlb\n",
        "import pickle\n",
        "from datetime import datetime as dt\n",
        "import copy\n",
        "import gc\n",
        "from sklearn.decomposition import PCA\n",
        "import math\n",
        "from sklearn.model_selection import KFold"
      ],
      "id": "concrete-cruise",
      "execution_count": 233,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ykU9DkvWXkRB",
        "outputId": "54862be3-321a-4755-fae9-d4d24cd34ddf"
      },
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "id": "ykU9DkvWXkRB",
      "execution_count": 234,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:36.397396Z",
          "iopub.status.busy": "2021-06-25T22:18:36.396731Z",
          "iopub.status.idle": "2021-06-25T22:18:36.401367Z",
          "shell.execute_reply": "2021-06-25T22:18:36.400852Z",
          "shell.execute_reply.started": "2021-06-25T21:58:16.647196Z"
        },
        "papermill": {
          "duration": 0.032131,
          "end_time": "2021-06-25T22:18:36.401524",
          "exception": false,
          "start_time": "2021-06-25T22:18:36.369393",
          "status": "completed"
        },
        "tags": [],
        "id": "declared-principal"
      },
      "source": [
        "BASE_DIR = Path('/content/drive/MyDrive/mlb/input')\n",
        "TRAIN_DIR = Path('/content/drive/MyDrive/mlb/input/archive')"
      ],
      "id": "declared-principal",
      "execution_count": 235,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:36.461288Z",
          "iopub.status.busy": "2021-06-25T22:18:36.460608Z",
          "iopub.status.idle": "2021-06-25T22:18:55.337941Z",
          "shell.execute_reply": "2021-06-25T22:18:55.336912Z",
          "shell.execute_reply.started": "2021-06-25T21:58:16.654219Z"
        },
        "papermill": {
          "duration": 18.911828,
          "end_time": "2021-06-25T22:18:55.338134",
          "exception": false,
          "start_time": "2021-06-25T22:18:36.426306",
          "status": "completed"
        },
        "tags": [],
        "id": "tough-launch"
      },
      "source": [
        "players = pd.read_csv(BASE_DIR / 'players.csv')\n",
        "\n",
        "rosters = pd.read_pickle(TRAIN_DIR / 'rosters_train.pkl')\n",
        "targets = pd.read_pickle(TRAIN_DIR / 'nextDayPlayerEngagement_train.pkl')\n",
        "scores1 = pd.read_pickle(TRAIN_DIR / 'playerBoxScores_train.pkl')\n",
        "scores = scores1.groupby(['playerId', 'date']).sum().reset_index()\n",
        "twitter = pd.read_pickle(\"/content/drive/MyDrive/mlb/input/archive/playerTwitterFollowers_train.pkl\")\n",
        "games = pd.read_pickle(TRAIN_DIR / 'games_train.pkl')\n",
        "events = pd.read_pickle(TRAIN_DIR / 'events_train.pkl')\n",
        "standings = pd.read_pickle(TRAIN_DIR / 'standings_train.pkl')\n",
        "teamtwitter = pd.read_pickle(TRAIN_DIR / 'teamTwitterFollowers_train.pkl')\n",
        "transaction = pd.read_pickle(TRAIN_DIR / 'teamTwitterFollowers_train.pkl')\n",
        "awards = pd.read_csv(BASE_DIR / 'awards.csv')\n",
        "seasons = pd.read_csv(BASE_DIR / 'seasons.csv')\n",
        "teams = pd.read_csv(BASE_DIR / 'teams.csv')\n",
        "player_target_stats = pd.read_csv(\"/content/drive/MyDrive/mlb/input/player_target_stats.csv\")\n",
        "\n",
        "example_test = pd.read_csv(\"/content/drive/MyDrive/mlb/input/example_test.csv\")\n",
        "salary = pd.read_csv(\"/content/drive/MyDrive/mlb/input/mlbSalaries.csv\")"
      ],
      "id": "tough-launch",
      "execution_count": 236,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4BRMq9l8YfYI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "f3f00d0a-f20b-436f-99eb-420a72d57167"
      },
      "source": [
        "def unpack_json(json_str):\n",
        "    return np.nan if pd.isna(json_str) else pd.read_json(json_str)\n",
        "\n",
        "test = unpack_json(example_test[\"rosters\"].iloc[0])\n",
        "test[\"playerId\"]\n",
        "del example_test\n",
        "gc.collect()"
      ],
      "id": "4BRMq9l8YfYI",
      "execution_count": 237,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "921"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 237
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:55.398266Z",
          "iopub.status.busy": "2021-06-25T22:18:55.397258Z",
          "iopub.status.idle": "2021-06-25T22:18:55.429108Z",
          "shell.execute_reply": "2021-06-25T22:18:55.428347Z",
          "shell.execute_reply.started": "2021-06-25T21:58:36.71955Z"
        },
        "papermill": {
          "duration": 0.065838,
          "end_time": "2021-06-25T22:18:55.429265",
          "exception": false,
          "start_time": "2021-06-25T22:18:55.363427",
          "status": "completed"
        },
        "tags": [],
        "id": "joined-traffic"
      },
      "source": [
        "awards2 = awards.groupby(\"playerId\").count()\n",
        "awards2 = awards2.reset_index()"
      ],
      "id": "joined-traffic",
      "execution_count": 238,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dRXO92-ROJ2_"
      },
      "source": [
        "teamtwitter[\"teamnumberOfFollowers\"] = teamtwitter[\"numberOfFollowers\"]\n",
        "teamtwi = teamtwitter.groupby(\"teamId\").mean()[\"teamnumberOfFollowers\"].reset_index()"
      ],
      "id": "dRXO92-ROJ2_",
      "execution_count": 239,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 414
        },
        "id": "M8Mvi9up7ypT",
        "outputId": "f1e7f991-40d6-4abc-f2e8-6bce4ad88f45"
      },
      "source": [
        "salary[\"playerName\"] = salary[\"name\"]"
      ],
      "id": "M8Mvi9up7ypT",
      "execution_count": 240,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Unnamed: 0</th>\n",
              "      <th>year</th>\n",
              "      <th>team</th>\n",
              "      <th>name</th>\n",
              "      <th>salary</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>2019</td>\n",
              "      <td>arizona-diamondbacks</td>\n",
              "      <td>Eduardo Escobar</td>\n",
              "      <td>6166666</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>2019</td>\n",
              "      <td>arizona-diamondbacks</td>\n",
              "      <td>Robbie Ray</td>\n",
              "      <td>6050000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>2019</td>\n",
              "      <td>arizona-diamondbacks</td>\n",
              "      <td>Taijuan Walker</td>\n",
              "      <td>5025000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>2019</td>\n",
              "      <td>arizona-diamondbacks</td>\n",
              "      <td>Jake Lamb</td>\n",
              "      <td>4825000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>2019</td>\n",
              "      <td>arizona-diamondbacks</td>\n",
              "      <td>Adam Jones</td>\n",
              "      <td>4500000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2335</th>\n",
              "      <td>2336</td>\n",
              "      <td>2021</td>\n",
              "      <td>washington-nationals</td>\n",
              "      <td>Paolo Espino</td>\n",
              "      <td>506055</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2336</th>\n",
              "      <td>2337</td>\n",
              "      <td>2021</td>\n",
              "      <td>washington-nationals</td>\n",
              "      <td>Kyle McGowin</td>\n",
              "      <td>460050</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2337</th>\n",
              "      <td>2338</td>\n",
              "      <td>2021</td>\n",
              "      <td>washington-nationals</td>\n",
              "      <td>Andres Machado</td>\n",
              "      <td>346571</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2338</th>\n",
              "      <td>2339</td>\n",
              "      <td>2021</td>\n",
              "      <td>washington-nationals</td>\n",
              "      <td>Jefry Rodriguez</td>\n",
              "      <td>346571</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2339</th>\n",
              "      <td>2340</td>\n",
              "      <td>2021</td>\n",
              "      <td>washington-nationals</td>\n",
              "      <td>Justin Miller</td>\n",
              "      <td>337370</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>2340 rows × 5 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "      Unnamed: 0  year                  team             name   salary\n",
              "0              1  2019  arizona-diamondbacks  Eduardo Escobar  6166666\n",
              "1              2  2019  arizona-diamondbacks       Robbie Ray  6050000\n",
              "2              3  2019  arizona-diamondbacks   Taijuan Walker  5025000\n",
              "3              4  2019  arizona-diamondbacks        Jake Lamb  4825000\n",
              "4              5  2019  arizona-diamondbacks       Adam Jones  4500000\n",
              "...          ...   ...                   ...              ...      ...\n",
              "2335        2336  2021  washington-nationals     Paolo Espino   506055\n",
              "2336        2337  2021  washington-nationals     Kyle McGowin   460050\n",
              "2337        2338  2021  washington-nationals   Andres Machado   346571\n",
              "2338        2339  2021  washington-nationals  Jefry Rodriguez   346571\n",
              "2339        2340  2021  washington-nationals    Justin Miller   337370\n",
              "\n",
              "[2340 rows x 5 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 240
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:55.485487Z",
          "iopub.status.busy": "2021-06-25T22:18:55.484858Z",
          "iopub.status.idle": "2021-06-25T22:18:55.559589Z",
          "shell.execute_reply": "2021-06-25T22:18:55.560093Z",
          "shell.execute_reply.started": "2021-06-25T21:58:36.751087Z"
        },
        "papermill": {
          "duration": 0.105303,
          "end_time": "2021-06-25T22:18:55.560260",
          "exception": false,
          "start_time": "2021-06-25T22:18:55.454957",
          "status": "completed"
        },
        "tags": [],
        "id": "saved-assignment"
      },
      "source": [
        "games = pd.read_pickle(TRAIN_DIR / 'games_train.pkl')\n",
        "#games.index[games[\"detailedGameState\"] == \"Postponed\"].shape\n",
        "#games.drop(games.loc[games['detailedGameState']=='Postponed'].index, inplace=True)\n",
        "#games[\"detailedGameState\"].unique()\n",
        "#games[games.duplicated(subset=[\"gamePk\"], keep=False)]\n",
        "#games['detailedGameState']=='Postponed']だとスコアはキロクされていない"
      ],
      "id": "saved-assignment",
      "execution_count": 201,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s5mtlhAlBjJw",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "16f7f9bc-a079-4887-8915-8b65e936f7b2"
      },
      "source": [
        "targets[\"engagementMetricsDate\"] = targets[\"engagementMetricsDate\"].str.replace('-', '')\n",
        "yesterday_targets = targets.drop('date', axis=1)\n",
        "yesterday_targets = yesterday_targets.rename(columns={'engagementMetricsDate':'date', 'target1': 'yest_target1','target2': 'yest_target2','target3': 'yest_target3','target4': 'yest_target4'})\n",
        "yesterday_targets[\"date\"] = yesterday_targets[\"date\"].astype(int)\n",
        "yesterday_targets[\"date\"]"
      ],
      "id": "s5mtlhAlBjJw",
      "execution_count": 202,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0          20180102\n",
              "1          20180102\n",
              "2          20180102\n",
              "3          20180102\n",
              "4          20180102\n",
              "             ...   \n",
              "2506171    20210501\n",
              "2506172    20210501\n",
              "2506173    20210501\n",
              "2506174    20210501\n",
              "2506175    20210501\n",
              "Name: date, Length: 2506176, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 202
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oSBa69W9UBGk",
        "outputId": "9769607b-e433-4d9c-99cc-6a4ccb41bfe6"
      },
      "source": [
        "targets_cols = ['playerId', 'target1', 'target2', 'target3', 'target4', 'date']\n",
        "players_cols = ['playerId', 'primaryPositionName',\"playerName\"]\n",
        "rosters_cols = ['playerId', 'teamId', 'status', 'date']\n",
        "yesterday_targets_cols = [\"date\",\"playerId\",\"yest_target1\",\"yest_target2\",\"yest_target3\",\"yest_target4\"]\n",
        "scores_cols = ['playerId', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored', 'doubles', 'triples', 'homeRuns',\n",
        "       'strikeOuts', 'baseOnBalls', 'intentionalWalks', 'hits', 'hitByPitch',\n",
        "       'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "       'groundIntoTriplePlay', 'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "       'pickoffs', 'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'completeGamesPitching', 'shutoutsPitching', 'winsPitching',\n",
        "       'lossesPitching', 'flyOutsPitching', 'airOutsPitching',\n",
        "       'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       'baseOnBallsPitching', 'intentionalWalksPitching', 'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched', 'saveOpportunities',\n",
        "       'earnedRuns', 'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored', 'catchersInterferencePitching',\n",
        "       'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "       'assists', 'putOuts', 'errors', 'chances', 'date',\"gamePk\"]\n",
        "games_cols = [\"gamePk\",\"homeId\",\"dayNight\",\"seriesDescription\",\"gamesInSeries\",\"homeWinner\",\"awayWinner\",\"homeScore\",\"awayScore\",\"gameType\",\n",
        "              \"gameDate\"]\n",
        "playertwitter_cols = [\"playerId\",\"numberOfFollowers\"]\n",
        "awards_cols = [\"playerId\",\"awardName\"]\n",
        "standings_cols = [\"date\",\"teamId\",\"divisionRank\",\"divisionLeader\",\"wildCardLeader\",\"leagueRank\",\"divisionId\",\"gameDate\"]\n",
        "teamtwitter_cols = [\"teamId\",\"teamnumberOfFollowers\"]\n",
        "targets[\"engagementMetricsDate\"] = targets[\"engagementMetricsDate\"].str.replace('-', '')\n",
        "yesterday_targets = targets.drop('date', axis=1)\n",
        "yesterday_targets = yesterday_targets.rename(columns={'engagementMetricsDate':'date', 'target1': 'yest_target1','target2': 'yest_target2','target3': 'yest_target3','target4': 'yest_target4'})\n",
        "yesterday_targets[\"date\"] = yesterday_targets[\"date\"].astype(int)\n",
        "yesterday_targets[\"date\"]\n",
        "salary_cols = [\"playerName\",\"year\",\"salary\"]"
      ],
      "id": "oSBa69W9UBGk",
      "execution_count": 203,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0          20180102\n",
              "1          20180102\n",
              "2          20180102\n",
              "3          20180102\n",
              "4          20180102\n",
              "             ...   \n",
              "2506171    20210501\n",
              "2506172    20210501\n",
              "2506173    20210501\n",
              "2506174    20210501\n",
              "2506175    20210501\n",
              "Name: date, Length: 2506176, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 203
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZBRcuG-4ty69"
      },
      "source": [
        "feature_cols = ['label_playerId', 'label_primaryPositionName', 'label_teamId',\n",
        "       'label_status', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored','homeRuns',\n",
        "       'strikeOuts', 'baseOnBalls',  'hits', 'hitByPitch',\n",
        "       'atBats', 'stolenBases', \n",
        "       'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacFlies',\n",
        "       'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'completeGamesPitching','winsPitching',\n",
        "       'lossesPitching', 'flyOutsPitching', 'airOutsPitching',\n",
        "       'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       'baseOnBallsPitching',  'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched', 'saveOpportunities',\n",
        "       'earnedRuns', 'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored',\n",
        "       'sacFliesPitching', 'saves', 'holds',\n",
        "       'assists', 'putOuts', 'errors', 'chances','target1_mean',\n",
        " 'target1_median',\n",
        " 'target1_std',\n",
        " 'target1_min',\n",
        " 'target1_max',\n",
        " 'target1_prob',\n",
        " 'target2_mean',\n",
        " 'target2_median',\n",
        " 'target2_std',\n",
        " 'target2_min',\n",
        " 'target2_max',\n",
        " 'target2_prob',\n",
        " 'target3_mean',\n",
        " 'target3_median',\n",
        " 'target3_std',\n",
        " 'target3_min',\n",
        " 'target3_max',\n",
        " 'target3_prob',\n",
        " 'target4_mean',\n",
        " 'target4_median',\n",
        " 'target4_std',\n",
        " 'target4_min',\n",
        " 'target4_max',\n",
        " 'target4_prob',\"divisionId\",\"teamnumberOfFollowers\",\"preseasonhits\",\"preseasonhomerun\",\"salary\"]\n",
        "feature_cols2 = ['label_playerId', 'label_primaryPositionName', 'label_teamId',\n",
        "       'label_status', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored',# 'doubles', 'triples', 'homeRuns',\n",
        "       'baseOnBalls', 'hits',\n",
        "      # 'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "       'plateAppearances', 'totalBases', 'rbi',\n",
        "      # 'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "       'pickoffs', 'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'winsPitching',\n",
        "       'lossesPitching',# 'flyOutsPitching', 'airOutsPitching',\n",
        "       'runsPitching', \n",
        "       'strikeOutsPitching',\n",
        "       #'hitsPitching',\n",
        "       'hitByPitchPitching',  'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched',\n",
        "        'battersFaced', \n",
        "        'balks', 'pickoffsPitching',\n",
        "        'inheritedRunners',\n",
        "        \n",
        "       #'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "       'putOuts','chances','target1_mean',\n",
        " 'target1_median',\n",
        " 'target1_std',\n",
        " 'target1_min',\n",
        " 'target1_max',\n",
        " 'target1_prob',\n",
        " 'target2_mean',\n",
        " 'target2_median',\n",
        " 'target2_std',\n",
        " 'target2_min',\n",
        " 'target2_max',\n",
        " 'target2_prob',\n",
        " 'target3_mean',\n",
        " 'target3_median',\n",
        " 'target3_std',\n",
        " 'target3_min',\n",
        " 'target3_max',\n",
        " 'target3_prob',\n",
        " 'target4_mean',\n",
        " 'target4_median',\n",
        " 'target4_std',\n",
        " 'target4_min',\n",
        " 'target4_max',\n",
        " 'target4_prob',\n",
        "    'target1',\"divisionId\",\"teamnumberOfFollowers\",\"preseasonhits\",\"preseasonhomerun\",\"salary\"]\n",
        "feature_cols3 = ['label_playerId', 'label_primaryPositionName', 'label_teamId',\n",
        "       'label_status', #'gamesPlayedBatting', 'flyOuts',\n",
        "        'homeRuns',\n",
        "      # 'strikeOuts', 'baseOnBalls', 'intentionalWalks', 'hits', 'hitByPitch',\n",
        "     # 'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "        'totalBases', 'rbi',\n",
        "      # 'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "        'gamesStartedPitching',\n",
        "      # 'completeGamesPitching', 'shutoutsPitching', 'winsPitching',\n",
        "       'lossesPitching', \n",
        "      # 'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "      # 'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       #'baseOnBallsPitching', 'intentionalWalksPitching', 'hitsPitching',\n",
        "       #'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'inningsPitched', \n",
        "        'battersFaced', 'pitchesThrown', \n",
        "      # 'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "      # 'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       #'inheritedRunnersScored', 'catchersInterferencePitching',\n",
        "       #'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "      # 'assists', 'putOuts', 'errors', 'chances',\n",
        "      'target1_mean',\n",
        " 'target1_median',\n",
        " 'target1_std',\n",
        " 'target1_min',\n",
        " 'target1_max',\n",
        " 'target1_prob',\n",
        " 'target2_mean',\n",
        " 'target2_median',\n",
        " 'target2_std',\n",
        " 'target2_min',\n",
        " 'target2_max',\n",
        " 'target2_prob',\n",
        " 'target3_mean',\n",
        " 'target3_median',\n",
        " 'target3_std',\n",
        " 'target3_min',\n",
        " 'target3_max',\n",
        " 'target3_prob',\n",
        " 'target4_mean',\n",
        " 'target4_median',\n",
        " 'target4_std',\n",
        " 'target4_min',\n",
        " 'target4_max',\n",
        " 'target4_prob',\n",
        "    \"divisionId\",\"teamnumberOfFollowers\",\"target2\",\"preseasonhits\",\"preseasonhomerun\",\"salary\"]\n",
        "feature_cols4 = ['label_playerId', 'label_primaryPositionName', 'label_teamId',\n",
        "       'label_status', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "        'runsScored','triples','groundIntoDoublePlay',\n",
        "       'strikeOuts', 'baseOnBalls','hits', \n",
        "       'atBats', 'caughtStealing',\n",
        "       'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacBunts', 'sacFlies',\n",
        "        'gamesPlayedPitching',\n",
        "        'winsPitching',\n",
        "       'airOutsPitching',\n",
        "       'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "        'intentionalWalksPitching', 'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', \n",
        "        'inningsPitched', 'saveOpportunities',\n",
        "       'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes','wildPitches', \n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored', \n",
        "       'sacBuntsPitching', 'holds', 'blownSaves',\n",
        "       'assists', 'putOuts', 'errors', 'chances','target1_mean',\n",
        " 'target1_median',\n",
        " 'target1_std',\n",
        " 'target1_min',\n",
        " 'target1_max',\n",
        " 'target1_prob',\n",
        " 'target2_mean',\n",
        " 'target2_median',\n",
        " 'target2_std',\n",
        " 'target2_min',\n",
        " 'target2_max',\n",
        " 'target2_prob',\n",
        " 'target3_mean',\n",
        " 'target3_median',\n",
        " 'target3_std',\n",
        " 'target3_min',\n",
        " 'target3_max',\n",
        " 'target3_prob',\n",
        " 'target4_mean',\n",
        " 'target4_median',\n",
        " 'target4_std',\n",
        " 'target4_min',\n",
        " 'target4_max',\n",
        " 'target4_prob',\n",
        " 'target1', \"divisionId\",\"target2\",\"target3\",\"teamnumberOfFollowers\",\"preseasonhits\",\"preseasonhomerun\",\"salary\"]"
      ],
      "id": "ZBRcuG-4ty69",
      "execution_count": 204,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WrdnRRfZtzBL"
      },
      "source": [
        ""
      ],
      "id": "WrdnRRfZtzBL",
      "execution_count": 204,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e-haSgyI78O6"
      },
      "source": [
        "twitter[\"strdate\"] = twitter[\"date\"].astype(str)\n",
        "twitter[\"year_months\"] = twitter[\"strdate\"].str[0:6].astype(int)\n"
      ],
      "id": "e-haSgyI78O6",
      "execution_count": 205,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HyLf8gpubQJA"
      },
      "source": [
        "targets['hasTwitterAccount'] = targets.playerId.isin(twitter.playerId)"
      ],
      "id": "HyLf8gpubQJA",
      "execution_count": 206,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:18:55.703085Z",
          "iopub.status.busy": "2021-06-25T22:18:55.702455Z",
          "iopub.status.idle": "2021-06-25T22:19:05.117538Z",
          "shell.execute_reply": "2021-06-25T22:19:05.116598Z",
          "shell.execute_reply.started": "2021-06-25T21:58:36.852876Z"
        },
        "papermill": {
          "duration": 9.448263,
          "end_time": "2021-06-25T22:19:05.117741",
          "exception": false,
          "start_time": "2021-06-25T22:18:55.669478",
          "status": "completed"
        },
        "tags": [],
        "id": "unauthorized-stretch"
      },
      "source": [
        "train = targets[targets_cols].merge(players[players_cols], on=['playerId'], how='left')\n",
        "train = train.merge(rosters[rosters_cols], on=['playerId', 'date'], how='left')\n",
        "train = train.merge(scores[scores_cols], on=['playerId', 'date'], how='left')\n",
        "train = train.merge(games[games_cols], on=[\"gamePk\"], how=\"left\")\n",
        "train = train.merge(standings[standings_cols], on=['date',\"teamId\"], how='left')\n",
        "train = train.merge(awards2[awards_cols], on=['playerId'], how='left')\n",
        "train = train.merge(player_target_stats, how='inner', left_on=[\"playerId\"],right_on=[\"playerId\"])\n",
        "train[\"strdate\"] = train[\"date\"].astype(str)\n",
        "train[\"year_months\"] = train[\"strdate\"].str[0:6].astype(int)\n",
        "#train = train.merge(twitter[playertwitter_cols], on=['playerId'], how='left')\n",
        "train = train.merge(teamtwi[teamtwitter_cols], on=['teamId'], how='left')\n",
        "train[\"year\"] = train[\"strdate\"].str[0:4].astype(int)\n",
        "train = train.merge(salary[salary_cols], on=[\"playerName\",\"year\"],how=\"left\")"
      ],
      "id": "unauthorized-stretch",
      "execution_count": 207,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mVHPPnE8GSqG"
      },
      "source": [
        "targets['hasTwitterAccount'] = targets.playerId.isin(twitter.playerId)"
      ],
      "id": "mVHPPnE8GSqG",
      "execution_count": 208,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9EexEYL53a9z",
        "outputId": "27964a1d-5653-4cbc-83d5-717a4b4d6b45"
      },
      "source": [
        "test[\"playerId\"]"
      ],
      "id": "9EexEYL53a9z",
      "execution_count": 209,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0       405395\n",
              "1       408234\n",
              "2       444482\n",
              "3       445276\n",
              "4       446334\n",
              "         ...  \n",
              "1264    676477\n",
              "1265    676755\n",
              "1266    676831\n",
              "1267    676845\n",
              "1268    676886\n",
              "Name: playerId, Length: 1269, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 209
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iCSDWeWFAPc9"
      },
      "source": [
        ""
      ],
      "id": "iCSDWeWFAPc9",
      "execution_count": 209,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4nTLjBWC6hki",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 317
        },
        "outputId": "eec698c0-ca5a-4b1e-9e92-f574591cb4fd"
      },
      "source": [
        "train[['target1_mean',\n",
        " 'target1_median',\n",
        " 'target1_std',\n",
        " 'target1_min',\n",
        " 'target1_max',\n",
        " 'target1_prob',\n",
        " 'target2_mean',\n",
        " 'target2_median',\n",
        " 'target2_std',\n",
        " 'target2_min',\n",
        " 'target2_max',\n",
        " 'target2_prob',\n",
        " 'target3_mean',\n",
        " 'target3_median',\n",
        " 'target3_std',\n",
        " 'target3_min',\n",
        " 'target3_max',\n",
        " 'target3_prob',\n",
        " 'target4_mean',\n",
        " 'target4_median',\n",
        " 'target4_std',\n",
        " 'target4_min',\n",
        " 'target4_max',\n",
        " 'target4_prob',\n",
        "    'target1',\"divisionId\",\"target2\",\"target3\"]].describe()"
      ],
      "id": "4nTLjBWC6hki",
      "execution_count": 210,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>target1_mean</th>\n",
              "      <th>target1_median</th>\n",
              "      <th>target1_std</th>\n",
              "      <th>target1_min</th>\n",
              "      <th>target1_max</th>\n",
              "      <th>target1_prob</th>\n",
              "      <th>target2_mean</th>\n",
              "      <th>target2_median</th>\n",
              "      <th>target2_std</th>\n",
              "      <th>target2_min</th>\n",
              "      <th>target2_max</th>\n",
              "      <th>target2_prob</th>\n",
              "      <th>target3_mean</th>\n",
              "      <th>target3_median</th>\n",
              "      <th>target3_std</th>\n",
              "      <th>target3_min</th>\n",
              "      <th>target3_max</th>\n",
              "      <th>target3_prob</th>\n",
              "      <th>target4_mean</th>\n",
              "      <th>target4_median</th>\n",
              "      <th>target4_std</th>\n",
              "      <th>target4_min</th>\n",
              "      <th>target4_max</th>\n",
              "      <th>target4_prob</th>\n",
              "      <th>target1</th>\n",
              "      <th>divisionId</th>\n",
              "      <th>target2</th>\n",
              "      <th>target3</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>664141.000000</td>\n",
              "      <td>2.508784e+06</td>\n",
              "      <td>2.508784e+06</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>8.076858e-01</td>\n",
              "      <td>2.086643e-01</td>\n",
              "      <td>1.491682e+00</td>\n",
              "      <td>5.020743e-03</td>\n",
              "      <td>6.708475e+00</td>\n",
              "      <td>1.127729e+00</td>\n",
              "      <td>2.099070e+00</td>\n",
              "      <td>1.240186e+00</td>\n",
              "      <td>2.463639e+00</td>\n",
              "      <td>3.085880e-01</td>\n",
              "      <td>1.121529e+01</td>\n",
              "      <td>2.467225e+00</td>\n",
              "      <td>5.694316e-01</td>\n",
              "      <td>7.745038e-02</td>\n",
              "      <td>1.400443e+00</td>\n",
              "      <td>7.896621e-03</td>\n",
              "      <td>6.701864e+00</td>\n",
              "      <td>3.599559e-01</td>\n",
              "      <td>1.637258e+00</td>\n",
              "      <td>1.148316e+00</td>\n",
              "      <td>1.656985e+00</td>\n",
              "      <td>2.317022e-01</td>\n",
              "      <td>7.629292e+00</td>\n",
              "      <td>1.803057e+00</td>\n",
              "      <td>5.705360e-01</td>\n",
              "      <td>202.500016</td>\n",
              "      <td>2.459199e+00</td>\n",
              "      <td>6.899692e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>2.345066e+00</td>\n",
              "      <td>1.035032e+00</td>\n",
              "      <td>3.735682e+00</td>\n",
              "      <td>2.681383e-02</td>\n",
              "      <td>1.673603e+01</td>\n",
              "      <td>5.960057e+00</td>\n",
              "      <td>3.812537e+00</td>\n",
              "      <td>2.774199e+00</td>\n",
              "      <td>4.353143e+00</td>\n",
              "      <td>7.452425e-01</td>\n",
              "      <td>1.939111e+01</td>\n",
              "      <td>7.834035e+00</td>\n",
              "      <td>1.812547e+00</td>\n",
              "      <td>5.275353e-01</td>\n",
              "      <td>3.645426e+00</td>\n",
              "      <td>2.884410e-02</td>\n",
              "      <td>1.675997e+01</td>\n",
              "      <td>2.807083e+00</td>\n",
              "      <td>3.375947e+00</td>\n",
              "      <td>2.673816e+00</td>\n",
              "      <td>3.324605e+00</td>\n",
              "      <td>5.438217e-01</td>\n",
              "      <td>1.490432e+01</td>\n",
              "      <td>5.750086e+00</td>\n",
              "      <td>4.179290e+00</td>\n",
              "      <td>1.708287</td>\n",
              "      <td>6.237921e+00</td>\n",
              "      <td>5.073026e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>2.006257e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>2.742816e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>8.734387e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>200.000000</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>0.000000e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>1.469648e-04</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>2.039305e-04</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>8.597541e-04</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>4.304230e-02</td>\n",
              "      <td>3.550086e-02</td>\n",
              "      <td>3.021539e-02</td>\n",
              "      <td>7.722604e-03</td>\n",
              "      <td>1.509848e-01</td>\n",
              "      <td>3.744477e-02</td>\n",
              "      <td>5.350206e-04</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>8.661010e-04</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>3.466685e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>1.682062e-01</td>\n",
              "      <td>1.245330e-01</td>\n",
              "      <td>1.259744e-01</td>\n",
              "      <td>2.410859e-02</td>\n",
              "      <td>5.491057e-01</td>\n",
              "      <td>9.594096e-02</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>201.000000</td>\n",
              "      <td>7.727312e-02</td>\n",
              "      <td>0.000000e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>2.298143e-03</td>\n",
              "      <td>7.948978e-04</td>\n",
              "      <td>2.027149e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>1.107237e-02</td>\n",
              "      <td>8.083251e-04</td>\n",
              "      <td>3.892553e-01</td>\n",
              "      <td>1.959924e-01</td>\n",
              "      <td>3.804404e-01</td>\n",
              "      <td>6.435571e-02</td>\n",
              "      <td>2.036706e+00</td>\n",
              "      <td>1.908192e-01</td>\n",
              "      <td>6.800752e-03</td>\n",
              "      <td>3.120125e-03</td>\n",
              "      <td>6.719478e-03</td>\n",
              "      <td>0.000000e+00</td>\n",
              "      <td>3.280409e-02</td>\n",
              "      <td>3.005112e-03</td>\n",
              "      <td>5.513352e-01</td>\n",
              "      <td>3.548320e-01</td>\n",
              "      <td>5.017860e-01</td>\n",
              "      <td>6.982174e-02</td>\n",
              "      <td>2.263134e+00</td>\n",
              "      <td>3.502988e-01</td>\n",
              "      <td>1.070771e-03</td>\n",
              "      <td>203.000000</td>\n",
              "      <td>5.575155e-01</td>\n",
              "      <td>1.698937e-03</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>3.930881e-01</td>\n",
              "      <td>1.362733e-02</td>\n",
              "      <td>9.672931e-01</td>\n",
              "      <td>1.668067e-03</td>\n",
              "      <td>4.532737e+00</td>\n",
              "      <td>1.688280e-02</td>\n",
              "      <td>2.718152e+00</td>\n",
              "      <td>1.341537e+00</td>\n",
              "      <td>2.816320e+00</td>\n",
              "      <td>3.178461e-01</td>\n",
              "      <td>1.301789e+01</td>\n",
              "      <td>1.808318e+00</td>\n",
              "      <td>3.037478e-01</td>\n",
              "      <td>2.156877e-02</td>\n",
              "      <td>9.168443e-01</td>\n",
              "      <td>3.192542e-03</td>\n",
              "      <td>4.347933e+00</td>\n",
              "      <td>2.131486e-02</td>\n",
              "      <td>1.662670e+00</td>\n",
              "      <td>1.049403e+00</td>\n",
              "      <td>1.692484e+00</td>\n",
              "      <td>2.072833e-01</td>\n",
              "      <td>7.636422e+00</td>\n",
              "      <td>1.215743e+00</td>\n",
              "      <td>1.833517e-02</td>\n",
              "      <td>204.000000</td>\n",
              "      <td>2.239762e+00</td>\n",
              "      <td>2.089282e-02</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>2.861945e+01</td>\n",
              "      <td>2.251564e+01</td>\n",
              "      <td>3.398812e+01</td>\n",
              "      <td>7.040277e-01</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>4.969193e+01</td>\n",
              "      <td>4.431772e+01</td>\n",
              "      <td>3.288215e+01</td>\n",
              "      <td>1.550033e+01</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>3.218307e+01</td>\n",
              "      <td>1.590317e+01</td>\n",
              "      <td>3.533198e+01</td>\n",
              "      <td>6.190328e-01</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>6.304345e+01</td>\n",
              "      <td>3.933404e+01</td>\n",
              "      <td>3.842898e+01</td>\n",
              "      <td>3.410180e+01</td>\n",
              "      <td>8.647749e+00</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>205.000000</td>\n",
              "      <td>1.000000e+02</td>\n",
              "      <td>1.000000e+02</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       target1_mean  target1_median  ...       target2       target3\n",
              "count  2.508784e+06    2.508784e+06  ...  2.508784e+06  2.508784e+06\n",
              "mean   8.076858e-01    2.086643e-01  ...  2.459199e+00  6.899692e-01\n",
              "std    2.345066e+00    1.035032e+00  ...  6.237921e+00  5.073026e+00\n",
              "min    0.000000e+00    0.000000e+00  ...  0.000000e+00  0.000000e+00\n",
              "25%    1.469648e-04    0.000000e+00  ...  7.727312e-02  0.000000e+00\n",
              "50%    2.298143e-03    7.948978e-04  ...  5.575155e-01  1.698937e-03\n",
              "75%    3.930881e-01    1.362733e-02  ...  2.239762e+00  2.089282e-02\n",
              "max    2.861945e+01    2.251564e+01  ...  1.000000e+02  1.000000e+02\n",
              "\n",
              "[8 rows x 28 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 210
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "70LUMHqddn6f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 609
        },
        "outputId": "cfe7dc19-2d97-4d41-a722-98a4336986d2"
      },
      "source": [
        "train[[\"target1\",\"target2\",\"target3\",\"target4\"]].describe()\n",
        "train[train[\"target1\"]==train[\"target1\"].max()]\n",
        "train[train[\"playerId\"]==519317]"
      ],
      "id": "70LUMHqddn6f",
      "execution_count": 211,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerId</th>\n",
              "      <th>target1</th>\n",
              "      <th>target2</th>\n",
              "      <th>target3</th>\n",
              "      <th>target4</th>\n",
              "      <th>date</th>\n",
              "      <th>primaryPositionName</th>\n",
              "      <th>playerName</th>\n",
              "      <th>teamId</th>\n",
              "      <th>status</th>\n",
              "      <th>battingOrder</th>\n",
              "      <th>gamesPlayedBatting</th>\n",
              "      <th>flyOuts</th>\n",
              "      <th>groundOuts</th>\n",
              "      <th>runsScored</th>\n",
              "      <th>doubles</th>\n",
              "      <th>triples</th>\n",
              "      <th>homeRuns</th>\n",
              "      <th>strikeOuts</th>\n",
              "      <th>baseOnBalls</th>\n",
              "      <th>intentionalWalks</th>\n",
              "      <th>hits</th>\n",
              "      <th>hitByPitch</th>\n",
              "      <th>atBats</th>\n",
              "      <th>caughtStealing</th>\n",
              "      <th>stolenBases</th>\n",
              "      <th>groundIntoDoublePlay</th>\n",
              "      <th>groundIntoTriplePlay</th>\n",
              "      <th>plateAppearances</th>\n",
              "      <th>totalBases</th>\n",
              "      <th>rbi</th>\n",
              "      <th>leftOnBase</th>\n",
              "      <th>sacBunts</th>\n",
              "      <th>sacFlies</th>\n",
              "      <th>catchersInterference</th>\n",
              "      <th>pickoffs</th>\n",
              "      <th>gamesPlayedPitching</th>\n",
              "      <th>gamesStartedPitching</th>\n",
              "      <th>completeGamesPitching</th>\n",
              "      <th>shutoutsPitching</th>\n",
              "      <th>...</th>\n",
              "      <th>homeWinner</th>\n",
              "      <th>awayWinner</th>\n",
              "      <th>homeScore</th>\n",
              "      <th>awayScore</th>\n",
              "      <th>gameType</th>\n",
              "      <th>gameDate_x</th>\n",
              "      <th>divisionRank</th>\n",
              "      <th>divisionLeader</th>\n",
              "      <th>wildCardLeader</th>\n",
              "      <th>leagueRank</th>\n",
              "      <th>divisionId</th>\n",
              "      <th>gameDate_y</th>\n",
              "      <th>awardName</th>\n",
              "      <th>target1_mean</th>\n",
              "      <th>target1_median</th>\n",
              "      <th>target1_std</th>\n",
              "      <th>target1_min</th>\n",
              "      <th>target1_max</th>\n",
              "      <th>target1_prob</th>\n",
              "      <th>target2_mean</th>\n",
              "      <th>target2_median</th>\n",
              "      <th>target2_std</th>\n",
              "      <th>target2_min</th>\n",
              "      <th>target2_max</th>\n",
              "      <th>target2_prob</th>\n",
              "      <th>target3_mean</th>\n",
              "      <th>target3_median</th>\n",
              "      <th>target3_std</th>\n",
              "      <th>target3_min</th>\n",
              "      <th>target3_max</th>\n",
              "      <th>target3_prob</th>\n",
              "      <th>target4_mean</th>\n",
              "      <th>target4_median</th>\n",
              "      <th>target4_std</th>\n",
              "      <th>target4_min</th>\n",
              "      <th>target4_max</th>\n",
              "      <th>target4_prob</th>\n",
              "      <th>strdate</th>\n",
              "      <th>year_months</th>\n",
              "      <th>teamnumberOfFollowers</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2435</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.974327</td>\n",
              "      <td>56.177043</td>\n",
              "      <td>13.693746</td>\n",
              "      <td>64.166667</td>\n",
              "      <td>20180101</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20180101</td>\n",
              "      <td>201801</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2436</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.743145</td>\n",
              "      <td>31.715912</td>\n",
              "      <td>11.397756</td>\n",
              "      <td>47.248120</td>\n",
              "      <td>20180102</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20180102</td>\n",
              "      <td>201801</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2437</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.741518</td>\n",
              "      <td>30.998168</td>\n",
              "      <td>25.975685</td>\n",
              "      <td>79.879622</td>\n",
              "      <td>20180103</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20180103</td>\n",
              "      <td>201801</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2438</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.434217</td>\n",
              "      <td>40.828745</td>\n",
              "      <td>6.552563</td>\n",
              "      <td>86.740331</td>\n",
              "      <td>20180104</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20180104</td>\n",
              "      <td>201801</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2439</th>\n",
              "      <td>519317</td>\n",
              "      <td>1.700575</td>\n",
              "      <td>20.981956</td>\n",
              "      <td>26.427778</td>\n",
              "      <td>34.202094</td>\n",
              "      <td>20180105</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20180105</td>\n",
              "      <td>201801</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3652</th>\n",
              "      <td>519317</td>\n",
              "      <td>14.208138</td>\n",
              "      <td>16.863095</td>\n",
              "      <td>6.386838</td>\n",
              "      <td>12.489944</td>\n",
              "      <td>20210426</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>200.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>4.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>R</td>\n",
              "      <td>2021-04-26</td>\n",
              "      <td>5</td>\n",
              "      <td>False</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12</td>\n",
              "      <td>201.0</td>\n",
              "      <td>2021-04-26</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20210426</td>\n",
              "      <td>202104</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3653</th>\n",
              "      <td>519317</td>\n",
              "      <td>15.521546</td>\n",
              "      <td>17.074740</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>14.683431</td>\n",
              "      <td>20210427</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>200.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>False</td>\n",
              "      <td>True</td>\n",
              "      <td>1.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>R</td>\n",
              "      <td>2021-04-27</td>\n",
              "      <td>5</td>\n",
              "      <td>False</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12</td>\n",
              "      <td>201.0</td>\n",
              "      <td>2021-04-27</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20210427</td>\n",
              "      <td>202104</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3654</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.947361</td>\n",
              "      <td>32.279561</td>\n",
              "      <td>2.343270</td>\n",
              "      <td>17.021057</td>\n",
              "      <td>20210428</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>200.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>False</td>\n",
              "      <td>True</td>\n",
              "      <td>0.0</td>\n",
              "      <td>7.0</td>\n",
              "      <td>R</td>\n",
              "      <td>2021-04-28</td>\n",
              "      <td>4</td>\n",
              "      <td>False</td>\n",
              "      <td>NaN</td>\n",
              "      <td>11</td>\n",
              "      <td>201.0</td>\n",
              "      <td>2021-04-28</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20210428</td>\n",
              "      <td>202104</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3655</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.276774</td>\n",
              "      <td>29.430518</td>\n",
              "      <td>0.485191</td>\n",
              "      <td>16.408387</td>\n",
              "      <td>20210429</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>200.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>4.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>R</td>\n",
              "      <td>2021-04-29</td>\n",
              "      <td>5</td>\n",
              "      <td>False</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12</td>\n",
              "      <td>201.0</td>\n",
              "      <td>2021-04-29</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20210429</td>\n",
              "      <td>202104</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3656</th>\n",
              "      <td>519317</td>\n",
              "      <td>0.243648</td>\n",
              "      <td>16.343241</td>\n",
              "      <td>0.239870</td>\n",
              "      <td>11.835666</td>\n",
              "      <td>20210430</td>\n",
              "      <td>Designated Hitter</td>\n",
              "      <td>Giancarlo Stanton</td>\n",
              "      <td>147.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>200.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>True</td>\n",
              "      <td>False</td>\n",
              "      <td>10.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>R</td>\n",
              "      <td>2021-04-30</td>\n",
              "      <td>5</td>\n",
              "      <td>False</td>\n",
              "      <td>NaN</td>\n",
              "      <td>12</td>\n",
              "      <td>201.0</td>\n",
              "      <td>2021-04-30</td>\n",
              "      <td>42.0</td>\n",
              "      <td>16.57155</td>\n",
              "      <td>6.296399</td>\n",
              "      <td>24.090019</td>\n",
              "      <td>0.087622</td>\n",
              "      <td>100.0</td>\n",
              "      <td>15.112738</td>\n",
              "      <td>17.019935</td>\n",
              "      <td>15.442759</td>\n",
              "      <td>8.480908</td>\n",
              "      <td>4.489628</td>\n",
              "      <td>33.868643</td>\n",
              "      <td>12.036823</td>\n",
              "      <td>5.245218</td>\n",
              "      <td>0.40027</td>\n",
              "      <td>11.07997</td>\n",
              "      <td>0.077679</td>\n",
              "      <td>39.703651</td>\n",
              "      <td>2.993143</td>\n",
              "      <td>13.301593</td>\n",
              "      <td>12.0</td>\n",
              "      <td>8.016774</td>\n",
              "      <td>2.059741</td>\n",
              "      <td>34.008584</td>\n",
              "      <td>6.248675</td>\n",
              "      <td>20210430</td>\n",
              "      <td>202104</td>\n",
              "      <td>3399726.125</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1222 rows × 128 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "      playerId    target1  ...  year_months  teamnumberOfFollowers\n",
              "2435    519317   0.974327  ...       201801            3399726.125\n",
              "2436    519317   0.743145  ...       201801            3399726.125\n",
              "2437    519317   0.741518  ...       201801            3399726.125\n",
              "2438    519317   0.434217  ...       201801            3399726.125\n",
              "2439    519317   1.700575  ...       201801            3399726.125\n",
              "...        ...        ...  ...          ...                    ...\n",
              "3652    519317  14.208138  ...       202104            3399726.125\n",
              "3653    519317  15.521546  ...       202104            3399726.125\n",
              "3654    519317   0.947361  ...       202104            3399726.125\n",
              "3655    519317   0.276774  ...       202104            3399726.125\n",
              "3656    519317   0.243648  ...       202104            3399726.125\n",
              "\n",
              "[1222 rows x 128 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 211
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vrevhu0a0MP0"
      },
      "source": [
        "train[\"stryear\"] = train[\"date\"].astype(str)\n",
        "train[\"year\"] = train[\"stryear\"].str[0:4].astype(int)"
      ],
      "id": "vrevhu0a0MP0",
      "execution_count": 212,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ViJ9Gl9aezst"
      },
      "source": [
        ""
      ],
      "id": "ViJ9Gl9aezst",
      "execution_count": 212,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2CkB9FJX3_Te",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fed00e45-9db0-4c61-b61c-f24ddc5fccd1"
      },
      "source": [
        "#make feature\n",
        "train[\"strdate\"] = train[\"date\"].astype(str)\n",
        "train[\"month\"] = train[\"strdate\"].str[5].astype(int)\n",
        "train[\"month\"]\n",
        "train[\"stryear\"] = train[\"date\"].astype(str)\n",
        "train[\"year\"] = train[\"stryear\"].str[0:4].astype(int)\n",
        "train[\"nullc\"] = train.isnull().sum(axis=1)\n",
        "train[\"ongame\"] = np.where(train[\"gamePk\"].isnull() == 1,0,1)\n",
        "train[\"ongame\"].unique()"
      ],
      "id": "2CkB9FJX3_Te",
      "execution_count": 213,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 1])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 213
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "adY9aY39Y2ez"
      },
      "source": [
        "del awards\n",
        "del scores\n",
        "del rosters\n",
        "del standings"
      ],
      "id": "adY9aY39Y2ez",
      "execution_count": 214,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:05.280987Z",
          "iopub.status.busy": "2021-06-25T22:19:05.280074Z",
          "iopub.status.idle": "2021-06-25T22:19:06.308927Z",
          "shell.execute_reply": "2021-06-25T22:19:06.308036Z",
          "shell.execute_reply.started": "2021-06-25T21:58:44.298952Z"
        },
        "papermill": {
          "duration": 1.157038,
          "end_time": "2021-06-25T22:19:06.309107",
          "exception": false,
          "start_time": "2021-06-25T22:19:05.152069",
          "status": "completed"
        },
        "tags": [],
        "id": "identified-resource"
      },
      "source": [
        ""
      ],
      "id": "identified-resource",
      "execution_count": 214,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:06.379223Z",
          "iopub.status.busy": "2021-06-25T22:19:06.378433Z",
          "iopub.status.idle": "2021-06-25T22:19:09.364865Z",
          "shell.execute_reply": "2021-06-25T22:19:09.364128Z",
          "shell.execute_reply.started": "2021-06-25T21:58:45.283486Z"
        },
        "papermill": {
          "duration": 3.0266,
          "end_time": "2021-06-25T22:19:09.365076",
          "exception": false,
          "start_time": "2021-06-25T22:19:06.338476",
          "status": "completed"
        },
        "tags": [],
        "id": "outstanding-spoke"
      },
      "source": [
        "# label encoding\n",
        "player2num = {c: i for i, c in enumerate(train['playerId'].unique())}\n",
        "position2num = {c: i for i, c in enumerate(train['primaryPositionName'].unique())}\n",
        "teamid2num = {c: i for i, c in enumerate(train['teamId'].unique())}\n",
        "status2num = {c: i for i, c in enumerate(train['status'].unique())}\n",
        "daynight2num = {c: i for i, c in enumerate(train['dayNight'].unique())}\n",
        "seriesDescription2num = {c: i for i, c in enumerate(train['seriesDescription'].unique())}\n",
        "gameType2num = {c: i for i, c in enumerate(train['gameType'].unique())}\n",
        "#bitrhCountry2num = {c: i for i, c in enumerate(train[\"birthCountry\"].unique())}\n",
        "train['label_playerId'] = train['playerId'].map(player2num)\n",
        "train['label_primaryPositionName'] = train['primaryPositionName'].map(position2num)\n",
        "train['label_teamId'] = train['teamId'].map(teamid2num)\n",
        "train['label_status'] = train['status'].map(status2num)\n",
        "train[\"label_daynight\"] = train['dayNight'].map(daynight2num)\n",
        "train[\"label_seriesDescription\"] = train[\"seriesDescription\"].map(seriesDescription2num)\n",
        "#train[\"birthCountry\"] = train[\"birthCountry\"].map(bitrhCountry2num)\n",
        "\n",
        "from sklearn.compose import TransformedTargetRegressor\n"
      ],
      "id": "outstanding-spoke",
      "execution_count": 215,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "G3RKg8r9QpDf"
      },
      "source": [
        "#train.to_csv(\"/content/drive/MyDrive/mlb/script/output/train.csv\")"
      ],
      "id": "G3RKg8r9QpDf",
      "execution_count": 216,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tnDAqSAkc7pP"
      },
      "source": [
        "playerlist = train[\"playerId\"].unique()"
      ],
      "id": "tnDAqSAkc7pP",
      "execution_count": 217,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cIq6Ji3fewUr",
        "outputId": "8d7b5737-15a4-4710-bd32-37532e48048d"
      },
      "source": [
        "train[\"playerId\"]"
      ],
      "id": "cIq6Ji3fewUr",
      "execution_count": 218,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0          628317\n",
              "1          628317\n",
              "2          628317\n",
              "3          628317\n",
              "4          628317\n",
              "            ...  \n",
              "2508779    656775\n",
              "2508780    656775\n",
              "2508781    656775\n",
              "2508782    656775\n",
              "2508783    656775\n",
              "Name: playerId, Length: 2508784, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 218
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SnLF5qfiaTAm"
      },
      "source": [
        "\n",
        "train_2017post = train[train[\"date\"]<=20180328]\n",
        "train_2018reg = train[(train[\"date\"]>=20180329)&(train[\"date\"]<=20181001)]\n",
        "train_2018post = train[(train[\"date\"]>=20181002)&(train[\"date\"]<=20190319)]\n",
        "train_2019reg = train[(train[\"date\"]>=20190320)&(train[\"date\"]<=20190929)]\n",
        "train_2019post = train[(train[\"date\"]>=20190930)&(train[\"date\"]<=20200722)]\n",
        "train_2020reg = train[(train[\"date\"]>=20200723)&(train[\"date\"]<=20200927)]\n",
        "train_2020post = train[(train[\"date\"]>=20200928)&(train[\"date\"]<=20210331)]\n",
        "train_2021reg = train[(train[\"date\"]>=20210401)&(train[\"date\"]<=20211003)]\n"
      ],
      "id": "SnLF5qfiaTAm",
      "execution_count": 219,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "id": "5wU1G9E899HT",
        "outputId": "35c14a1b-7179-4919-cc53-edc5565c69d6"
      },
      "source": [
        "seasons"
      ],
      "id": "5wU1G9E899HT",
      "execution_count": 220,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>seasonId</th>\n",
              "      <th>seasonStartDate</th>\n",
              "      <th>seasonEndDate</th>\n",
              "      <th>preSeasonStartDate</th>\n",
              "      <th>preSeasonEndDate</th>\n",
              "      <th>regularSeasonStartDate</th>\n",
              "      <th>regularSeasonEndDate</th>\n",
              "      <th>lastDate1stHalf</th>\n",
              "      <th>allStarDate</th>\n",
              "      <th>firstDate2ndHalf</th>\n",
              "      <th>postSeasonStartDate</th>\n",
              "      <th>postSeasonEndDate</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>2017</td>\n",
              "      <td>2017-04-02</td>\n",
              "      <td>2017-11-01</td>\n",
              "      <td>2017-02-22</td>\n",
              "      <td>2017-04-01</td>\n",
              "      <td>2017-04-02</td>\n",
              "      <td>2017-10-01</td>\n",
              "      <td>2017-07-09</td>\n",
              "      <td>2017-07-11</td>\n",
              "      <td>2017-07-14</td>\n",
              "      <td>2017-10-03</td>\n",
              "      <td>2017-11-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2018</td>\n",
              "      <td>2018-03-29</td>\n",
              "      <td>2018-10-28</td>\n",
              "      <td>2018-02-21</td>\n",
              "      <td>2018-03-27</td>\n",
              "      <td>2018-03-29</td>\n",
              "      <td>2018-10-01</td>\n",
              "      <td>2018-07-15</td>\n",
              "      <td>2018-07-17</td>\n",
              "      <td>2018-07-19</td>\n",
              "      <td>2018-10-02</td>\n",
              "      <td>2018-10-28</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>2019</td>\n",
              "      <td>2019-03-20</td>\n",
              "      <td>2019-10-30</td>\n",
              "      <td>2019-02-21</td>\n",
              "      <td>2019-03-26</td>\n",
              "      <td>2019-03-20</td>\n",
              "      <td>2019-09-29</td>\n",
              "      <td>2019-07-07</td>\n",
              "      <td>2019-07-09</td>\n",
              "      <td>2019-07-11</td>\n",
              "      <td>2019-10-01</td>\n",
              "      <td>2019-10-30</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>2020</td>\n",
              "      <td>2020-07-23</td>\n",
              "      <td>2020-10-28</td>\n",
              "      <td>2020-02-21</td>\n",
              "      <td>2020-07-22</td>\n",
              "      <td>2020-07-23</td>\n",
              "      <td>2020-09-27</td>\n",
              "      <td>2020-08-25</td>\n",
              "      <td>NaN</td>\n",
              "      <td>2020-08-26</td>\n",
              "      <td>2020-09-29</td>\n",
              "      <td>2020-10-28</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>2021</td>\n",
              "      <td>2021-02-28</td>\n",
              "      <td>2021-10-31</td>\n",
              "      <td>2021-02-28</td>\n",
              "      <td>2021-03-30</td>\n",
              "      <td>2021-04-01</td>\n",
              "      <td>2021-10-03</td>\n",
              "      <td>2021-07-11</td>\n",
              "      <td>2021-07-13</td>\n",
              "      <td>2021-07-15</td>\n",
              "      <td>2021-10-04</td>\n",
              "      <td>2021-10-31</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   seasonId seasonStartDate  ... postSeasonStartDate postSeasonEndDate\n",
              "0      2017      2017-04-02  ...          2017-10-03        2017-11-01\n",
              "1      2018      2018-03-29  ...          2018-10-02        2018-10-28\n",
              "2      2019      2019-03-20  ...          2019-10-01        2019-10-30\n",
              "3      2020      2020-07-23  ...          2020-09-29        2020-10-28\n",
              "4      2021      2021-02-28  ...          2021-10-04        2021-10-31\n",
              "\n",
              "[5 rows x 12 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 220
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "49AjMLJDhv97",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "82cf967d-cd28-44e8-8e5a-8a8ca6ea8ff9"
      },
      "source": [
        "#make feature\n",
        "hitscol = [\"playerId\",\"preseasonhits\"]\n",
        "hits_2018 = train_2018reg.groupby(\"playerId\").sum()[\"hits\"].reset_index()\n",
        "hits_2018[\"preseasonhits\"] = hits_2018[\"hits\"]\n",
        "train_2018post = train_2018post.merge(hits_2018[hitscol], on=['playerId'], how='left')\n",
        "train_2019reg = train_2019reg.merge(hits_2018[hitscol], on=['playerId'], how='left')\n",
        "\n",
        "hits2019col = [\"playerId\",\"hits2019\"]\n",
        "hits_2019 = train_2019reg.groupby(\"playerId\").sum()[\"hits\"].reset_index()\n",
        "hits_2019[\"preseasonhits\"] = hits_2019[\"hits\"]\n",
        "train_2019post = train_2019post.merge(hits_2019[hitscol], on=['playerId'], how='left')\n",
        "train_2020reg = train_2020reg.merge(hits_2019[hitscol], on=['playerId'], how='left')\n",
        "\n",
        "hits2020col = [\"playerId\",\"hits2020\"]\n",
        "hits_2020 = train_2020reg.groupby(\"playerId\").sum()[\"hits\"].reset_index()\n",
        "hits_2020[\"preseasonhits\"] = hits_2020[\"hits\"]\n",
        "train_2020post = train_2020post.merge(hits_2020[hitscol], on=['playerId'], how='left')\n",
        "train_2021reg = train_2021reg.merge(hits_2020[hitscol], on=['playerId'], how='left')\n",
        "del hits_2018,hits_2019,hits_2020\n",
        "gc.collect()\n",
        "\n",
        "gamescol = [\"playerId\",\"preseasonpt\"]\n",
        "game2018 = train_2018reg.groupby(\"playerId\").sum()[\"ongame\"].reset_index()\n",
        "game2018[\"preseasonpt\"] = game2018[\"ongame\"]\n",
        "train_2018post = train_2018post.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "train_2019reg = train_2019reg.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2019 = train_2019reg.groupby(\"playerId\").sum()[\"ongame\"].reset_index()\n",
        "game2019[\"preseasonpt\"] = game2019[\"ongame\"]\n",
        "train_2019post = train_2019post.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "train_2020reg = train_2020reg.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2020 = train_2020reg.groupby(\"playerId\").sum()[\"ongame\"].reset_index()\n",
        "game2020[\"preseasonpt\"] = game2020[\"ongame\"]\n",
        "train_2020post = train_2020post.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "train_2021reg = train_2021reg.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "del game2018,game2019,game2020\n",
        "gc.collect()\n",
        "\n",
        "gamescol = [\"playerId\",\"preseasonhomerun\"]\n",
        "game2018 = train_2018reg.groupby(\"playerId\").sum()[\"homeRuns\"].reset_index()\n",
        "game2018[\"preseasonhomerun\"] = game2018[\"homeRuns\"]\n",
        "train_2018post = train_2018post.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "train_2019reg = train_2019reg.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2019 = train_2019reg.groupby(\"playerId\").sum()[\"homeRuns\"].reset_index()\n",
        "game2019[\"preseasonhomerun\"] = game2019[\"homeRuns\"]\n",
        "train_2019post = train_2019post.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "train_2020reg = train_2020reg.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2020 = train_2020reg.groupby(\"playerId\").sum()[\"homeRuns\"].reset_index()\n",
        "game2020[\"preseasonhomerun\"] = game2020[\"homeRuns\"]\n",
        "train_2020post = train_2020post.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "train_2021reg = train_2021reg.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "del game2018,game2019,game2020\n",
        "gc.collect()\n",
        "\n",
        "\n",
        "gamescol = [\"playerId\",\"preseasont1mean\"]\n",
        "game2018 = train_2018reg.groupby(\"playerId\").mean()[\"target3\"].reset_index()\n",
        "game2018[\"preseasont1mean\"] = game2018[\"target3\"]\n",
        "train_2018post = train_2018post.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "train_2019reg = train_2019reg.merge(game2018[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2019 = train_2019reg.groupby(\"playerId\").mean()[\"target3\"].reset_index()\n",
        "game2019[\"preseasont1mean\"] = game2019[\"target3\"]\n",
        "train_2019post = train_2019post.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "train_2020reg = train_2020reg.merge(game2019[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "game2020 = train_2020reg.groupby(\"playerId\").mean()[\"target3\"].reset_index()\n",
        "game2020[\"preseasont1mean\"] = game2020[\"target3\"]\n",
        "train_2020post = train_2020post.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "train_2021reg = train_2021reg.merge(game2020[gamescol], on=['playerId'], how='left')\n",
        "\n",
        "del game2018,game2019,game2020\n",
        "gc.collect()"
      ],
      "id": "49AjMLJDhv97",
      "execution_count": 221,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 221
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xT9yVB1Qzz44"
      },
      "source": [
        "train = pd.concat([train_2017post,train_2018reg,train_2018post,train_2019reg,train_2019post,train_2020reg,train_2020post,train_2021reg])\n",
        "train[\"preseasonhits\"]\n",
        "del train_2017post,train_2018post,train_2019post, train_2020post, train_2018reg, train_2019reg, train_2020reg"
      ],
      "id": "xT9yVB1Qzz44",
      "execution_count": 222,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "P7S_fxrw4NCB"
      },
      "source": [
        "train[\"hitpergame\"] = train[\"preseasonhits\"] / train[\"preseasonpt\"]"
      ],
      "id": "P7S_fxrw4NCB",
      "execution_count": 223,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SwHCG2X0rmhR"
      },
      "source": [
        ""
      ],
      "id": "SwHCG2X0rmhR",
      "execution_count": 223,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "krSBoczTuaVd"
      },
      "source": [
        ""
      ],
      "id": "krSBoczTuaVd",
      "execution_count": 223,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:09.441422Z",
          "iopub.status.busy": "2021-06-25T22:19:09.438428Z",
          "iopub.status.idle": "2021-06-25T22:19:17.553819Z",
          "shell.execute_reply": "2021-06-25T22:19:17.554566Z",
          "shell.execute_reply.started": "2021-06-25T21:58:48.179255Z"
        },
        "papermill": {
          "duration": 8.158299,
          "end_time": "2021-06-25T22:19:17.555101",
          "exception": false,
          "start_time": "2021-06-25T22:19:09.396802",
          "status": "completed"
        },
        "tags": [],
        "id": "negative-offset",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bdb57d29-dfb2-4d06-84a9-6dfd4b7d953c"
      },
      "source": [
        "\n",
        "train_y = train[['target1', 'target2', 'target3', 'target4']]\n",
        "\n",
        "\n",
        "_index = (train['date'] < 20210401)\n",
        "gc.collect()\n",
        "\n",
        "\n",
        "x_train = train.loc[_index].reset_index(drop=True)\n",
        "y_train = train_y.loc[_index].reset_index(drop=True)\n",
        "x_valid = train.loc[~_index].reset_index(drop=True)\n",
        "y_valid = train_y.loc[~_index].reset_index(drop=True)\n",
        "\n",
        "\n",
        "\n",
        "gc.collect()"
      ],
      "id": "negative-offset",
      "execution_count": 224,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 224
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 609
        },
        "id": "1bOsn97CzD8q",
        "outputId": "4e47e7ac-c8a7-44db-f51b-f8796bab32a9"
      },
      "source": [
        "train"
      ],
      "id": "1bOsn97CzD8q",
      "execution_count": 225,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>playerId</th>\n",
              "      <th>target1</th>\n",
              "      <th>target2</th>\n",
              "      <th>target3</th>\n",
              "      <th>target4</th>\n",
              "      <th>date</th>\n",
              "      <th>primaryPositionName</th>\n",
              "      <th>playerName</th>\n",
              "      <th>teamId</th>\n",
              "      <th>status</th>\n",
              "      <th>battingOrder</th>\n",
              "      <th>gamesPlayedBatting</th>\n",
              "      <th>flyOuts</th>\n",
              "      <th>groundOuts</th>\n",
              "      <th>runsScored</th>\n",
              "      <th>doubles</th>\n",
              "      <th>triples</th>\n",
              "      <th>homeRuns</th>\n",
              "      <th>strikeOuts</th>\n",
              "      <th>baseOnBalls</th>\n",
              "      <th>intentionalWalks</th>\n",
              "      <th>hits</th>\n",
              "      <th>hitByPitch</th>\n",
              "      <th>atBats</th>\n",
              "      <th>caughtStealing</th>\n",
              "      <th>stolenBases</th>\n",
              "      <th>groundIntoDoublePlay</th>\n",
              "      <th>groundIntoTriplePlay</th>\n",
              "      <th>plateAppearances</th>\n",
              "      <th>totalBases</th>\n",
              "      <th>rbi</th>\n",
              "      <th>leftOnBase</th>\n",
              "      <th>sacBunts</th>\n",
              "      <th>sacFlies</th>\n",
              "      <th>catchersInterference</th>\n",
              "      <th>pickoffs</th>\n",
              "      <th>gamesPlayedPitching</th>\n",
              "      <th>gamesStartedPitching</th>\n",
              "      <th>completeGamesPitching</th>\n",
              "      <th>shutoutsPitching</th>\n",
              "      <th>...</th>\n",
              "      <th>target1_min</th>\n",
              "      <th>target1_max</th>\n",
              "      <th>target1_prob</th>\n",
              "      <th>target2_mean</th>\n",
              "      <th>target2_median</th>\n",
              "      <th>target2_std</th>\n",
              "      <th>target2_min</th>\n",
              "      <th>target2_max</th>\n",
              "      <th>target2_prob</th>\n",
              "      <th>target3_mean</th>\n",
              "      <th>target3_median</th>\n",
              "      <th>target3_std</th>\n",
              "      <th>target3_min</th>\n",
              "      <th>target3_max</th>\n",
              "      <th>target3_prob</th>\n",
              "      <th>target4_mean</th>\n",
              "      <th>target4_median</th>\n",
              "      <th>target4_std</th>\n",
              "      <th>target4_min</th>\n",
              "      <th>target4_max</th>\n",
              "      <th>target4_prob</th>\n",
              "      <th>strdate</th>\n",
              "      <th>year_months</th>\n",
              "      <th>teamnumberOfFollowers</th>\n",
              "      <th>stryear</th>\n",
              "      <th>year</th>\n",
              "      <th>month</th>\n",
              "      <th>nullc</th>\n",
              "      <th>ongame</th>\n",
              "      <th>label_playerId</th>\n",
              "      <th>label_primaryPositionName</th>\n",
              "      <th>label_teamId</th>\n",
              "      <th>label_status</th>\n",
              "      <th>label_daynight</th>\n",
              "      <th>label_seriesDescription</th>\n",
              "      <th>preseasonhits</th>\n",
              "      <th>preseasonpt</th>\n",
              "      <th>preseasonhomerun</th>\n",
              "      <th>preseasont1mean</th>\n",
              "      <th>hitpergame</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>628317</td>\n",
              "      <td>0.011167</td>\n",
              "      <td>4.474708</td>\n",
              "      <td>0.005168</td>\n",
              "      <td>5.735294</td>\n",
              "      <td>20180101</td>\n",
              "      <td>Pitcher</td>\n",
              "      <td>Kenta Maeda</td>\n",
              "      <td>119.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003102</td>\n",
              "      <td>7.193825</td>\n",
              "      <td>0.038621</td>\n",
              "      <td>7.595162</td>\n",
              "      <td>3.207558</td>\n",
              "      <td>11.575222</td>\n",
              "      <td>0.793096</td>\n",
              "      <td>49.982843</td>\n",
              "      <td>5.240148</td>\n",
              "      <td>0.369429</td>\n",
              "      <td>0.130267</td>\n",
              "      <td>0.733202</td>\n",
              "      <td>0.018696</td>\n",
              "      <td>3.236853</td>\n",
              "      <td>0.244629</td>\n",
              "      <td>2.997316</td>\n",
              "      <td>2.331033</td>\n",
              "      <td>2.187388</td>\n",
              "      <td>0.597849</td>\n",
              "      <td>9.648069</td>\n",
              "      <td>3.094059</td>\n",
              "      <td>20180101</td>\n",
              "      <td>201801</td>\n",
              "      <td>2137878.1</td>\n",
              "      <td>20180101</td>\n",
              "      <td>2018</td>\n",
              "      <td>1</td>\n",
              "      <td>90</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>628317</td>\n",
              "      <td>0.006356</td>\n",
              "      <td>2.521008</td>\n",
              "      <td>0.001648</td>\n",
              "      <td>2.766917</td>\n",
              "      <td>20180102</td>\n",
              "      <td>Pitcher</td>\n",
              "      <td>Kenta Maeda</td>\n",
              "      <td>119.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003102</td>\n",
              "      <td>7.193825</td>\n",
              "      <td>0.038621</td>\n",
              "      <td>7.595162</td>\n",
              "      <td>3.207558</td>\n",
              "      <td>11.575222</td>\n",
              "      <td>0.793096</td>\n",
              "      <td>49.982843</td>\n",
              "      <td>5.240148</td>\n",
              "      <td>0.369429</td>\n",
              "      <td>0.130267</td>\n",
              "      <td>0.733202</td>\n",
              "      <td>0.018696</td>\n",
              "      <td>3.236853</td>\n",
              "      <td>0.244629</td>\n",
              "      <td>2.997316</td>\n",
              "      <td>2.331033</td>\n",
              "      <td>2.187388</td>\n",
              "      <td>0.597849</td>\n",
              "      <td>9.648069</td>\n",
              "      <td>3.094059</td>\n",
              "      <td>20180102</td>\n",
              "      <td>201801</td>\n",
              "      <td>2137878.1</td>\n",
              "      <td>20180102</td>\n",
              "      <td>2018</td>\n",
              "      <td>1</td>\n",
              "      <td>90</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>628317</td>\n",
              "      <td>0.003556</td>\n",
              "      <td>2.380952</td>\n",
              "      <td>0.003624</td>\n",
              "      <td>3.439381</td>\n",
              "      <td>20180103</td>\n",
              "      <td>Pitcher</td>\n",
              "      <td>Kenta Maeda</td>\n",
              "      <td>119.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003102</td>\n",
              "      <td>7.193825</td>\n",
              "      <td>0.038621</td>\n",
              "      <td>7.595162</td>\n",
              "      <td>3.207558</td>\n",
              "      <td>11.575222</td>\n",
              "      <td>0.793096</td>\n",
              "      <td>49.982843</td>\n",
              "      <td>5.240148</td>\n",
              "      <td>0.369429</td>\n",
              "      <td>0.130267</td>\n",
              "      <td>0.733202</td>\n",
              "      <td>0.018696</td>\n",
              "      <td>3.236853</td>\n",
              "      <td>0.244629</td>\n",
              "      <td>2.997316</td>\n",
              "      <td>2.331033</td>\n",
              "      <td>2.187388</td>\n",
              "      <td>0.597849</td>\n",
              "      <td>9.648069</td>\n",
              "      <td>3.094059</td>\n",
              "      <td>20180103</td>\n",
              "      <td>201801</td>\n",
              "      <td>2137878.1</td>\n",
              "      <td>20180103</td>\n",
              "      <td>2018</td>\n",
              "      <td>1</td>\n",
              "      <td>90</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>628317</td>\n",
              "      <td>0.003363</td>\n",
              "      <td>3.332367</td>\n",
              "      <td>0.012294</td>\n",
              "      <td>4.849601</td>\n",
              "      <td>20180104</td>\n",
              "      <td>Pitcher</td>\n",
              "      <td>Kenta Maeda</td>\n",
              "      <td>119.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003102</td>\n",
              "      <td>7.193825</td>\n",
              "      <td>0.038621</td>\n",
              "      <td>7.595162</td>\n",
              "      <td>3.207558</td>\n",
              "      <td>11.575222</td>\n",
              "      <td>0.793096</td>\n",
              "      <td>49.982843</td>\n",
              "      <td>5.240148</td>\n",
              "      <td>0.369429</td>\n",
              "      <td>0.130267</td>\n",
              "      <td>0.733202</td>\n",
              "      <td>0.018696</td>\n",
              "      <td>3.236853</td>\n",
              "      <td>0.244629</td>\n",
              "      <td>2.997316</td>\n",
              "      <td>2.331033</td>\n",
              "      <td>2.187388</td>\n",
              "      <td>0.597849</td>\n",
              "      <td>9.648069</td>\n",
              "      <td>3.094059</td>\n",
              "      <td>20180104</td>\n",
              "      <td>201801</td>\n",
              "      <td>2137878.1</td>\n",
              "      <td>20180104</td>\n",
              "      <td>2018</td>\n",
              "      <td>1</td>\n",
              "      <td>90</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>628317</td>\n",
              "      <td>0.023976</td>\n",
              "      <td>1.440761</td>\n",
              "      <td>0.001501</td>\n",
              "      <td>2.371288</td>\n",
              "      <td>20180105</td>\n",
              "      <td>Pitcher</td>\n",
              "      <td>Kenta Maeda</td>\n",
              "      <td>119.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>...</td>\n",
              "      <td>0.003102</td>\n",
              "      <td>7.193825</td>\n",
              "      <td>0.038621</td>\n",
              "      <td>7.595162</td>\n",
              "      <td>3.207558</td>\n",
              "      <td>11.575222</td>\n",
              "      <td>0.793096</td>\n",
              "      <td>49.982843</td>\n",
              "      <td>5.240148</td>\n",
              "      <td>0.369429</td>\n",
              "      <td>0.130267</td>\n",
              "      <td>0.733202</td>\n",
              "      <td>0.018696</td>\n",
              "      <td>3.236853</td>\n",
              "      <td>0.244629</td>\n",
              "      <td>2.997316</td>\n",
              "      <td>2.331033</td>\n",
              "      <td>2.187388</td>\n",
              "      <td>0.597849</td>\n",
              "      <td>9.648069</td>\n",
              "      <td>3.094059</td>\n",
              "      <td>20180105</td>\n",
              "      <td>201801</td>\n",
              "      <td>2137878.1</td>\n",
              "      <td>20180105</td>\n",
              "      <td>2018</td>\n",
              "      <td>1</td>\n",
              "      <td>90</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61835</th>\n",
              "      <td>656775</td>\n",
              "      <td>16.920553</td>\n",
              "      <td>13.339693</td>\n",
              "      <td>5.035756</td>\n",
              "      <td>4.136498</td>\n",
              "      <td>20210426</td>\n",
              "      <td>Outfielder</td>\n",
              "      <td>Cedric Mullins</td>\n",
              "      <td>110.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>100.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>10.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>43.288911</td>\n",
              "      <td>1.447754</td>\n",
              "      <td>9.850177</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>6.465414</td>\n",
              "      <td>1.317686</td>\n",
              "      <td>26.322296</td>\n",
              "      <td>8.339773</td>\n",
              "      <td>1.634240</td>\n",
              "      <td>0.093914</td>\n",
              "      <td>3.148632</td>\n",
              "      <td>0.002389</td>\n",
              "      <td>11.281962</td>\n",
              "      <td>0.035505</td>\n",
              "      <td>2.294898</td>\n",
              "      <td>1.479832</td>\n",
              "      <td>2.269158</td>\n",
              "      <td>0.244376</td>\n",
              "      <td>10.180258</td>\n",
              "      <td>1.069359</td>\n",
              "      <td>20210426</td>\n",
              "      <td>202104</td>\n",
              "      <td>785074.3</td>\n",
              "      <td>20210426</td>\n",
              "      <td>2021</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2060</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.236877</td>\n",
              "      <td>0.826087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61836</th>\n",
              "      <td>656775</td>\n",
              "      <td>1.779862</td>\n",
              "      <td>8.035688</td>\n",
              "      <td>0.287214</td>\n",
              "      <td>2.361922</td>\n",
              "      <td>20210427</td>\n",
              "      <td>Outfielder</td>\n",
              "      <td>Cedric Mullins</td>\n",
              "      <td>110.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>100.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>43.288911</td>\n",
              "      <td>1.447754</td>\n",
              "      <td>9.850177</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>6.465414</td>\n",
              "      <td>1.317686</td>\n",
              "      <td>26.322296</td>\n",
              "      <td>8.339773</td>\n",
              "      <td>1.634240</td>\n",
              "      <td>0.093914</td>\n",
              "      <td>3.148632</td>\n",
              "      <td>0.002389</td>\n",
              "      <td>11.281962</td>\n",
              "      <td>0.035505</td>\n",
              "      <td>2.294898</td>\n",
              "      <td>1.479832</td>\n",
              "      <td>2.269158</td>\n",
              "      <td>0.244376</td>\n",
              "      <td>10.180258</td>\n",
              "      <td>1.069359</td>\n",
              "      <td>20210427</td>\n",
              "      <td>202104</td>\n",
              "      <td>785074.3</td>\n",
              "      <td>20210427</td>\n",
              "      <td>2021</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2060</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.236877</td>\n",
              "      <td>0.826087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61837</th>\n",
              "      <td>656775</td>\n",
              "      <td>12.440696</td>\n",
              "      <td>17.293955</td>\n",
              "      <td>11.027232</td>\n",
              "      <td>4.128819</td>\n",
              "      <td>20210428</td>\n",
              "      <td>Outfielder</td>\n",
              "      <td>Cedric Mullins</td>\n",
              "      <td>110.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>100.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>43.288911</td>\n",
              "      <td>1.447754</td>\n",
              "      <td>9.850177</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>6.465414</td>\n",
              "      <td>1.317686</td>\n",
              "      <td>26.322296</td>\n",
              "      <td>8.339773</td>\n",
              "      <td>1.634240</td>\n",
              "      <td>0.093914</td>\n",
              "      <td>3.148632</td>\n",
              "      <td>0.002389</td>\n",
              "      <td>11.281962</td>\n",
              "      <td>0.035505</td>\n",
              "      <td>2.294898</td>\n",
              "      <td>1.479832</td>\n",
              "      <td>2.269158</td>\n",
              "      <td>0.244376</td>\n",
              "      <td>10.180258</td>\n",
              "      <td>1.069359</td>\n",
              "      <td>20210428</td>\n",
              "      <td>202104</td>\n",
              "      <td>785074.3</td>\n",
              "      <td>20210428</td>\n",
              "      <td>2021</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2060</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.236877</td>\n",
              "      <td>0.826087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61838</th>\n",
              "      <td>656775</td>\n",
              "      <td>10.765473</td>\n",
              "      <td>9.599092</td>\n",
              "      <td>7.968079</td>\n",
              "      <td>2.995182</td>\n",
              "      <td>20210429</td>\n",
              "      <td>Outfielder</td>\n",
              "      <td>Cedric Mullins</td>\n",
              "      <td>110.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>100.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>5.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>43.288911</td>\n",
              "      <td>1.447754</td>\n",
              "      <td>9.850177</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>6.465414</td>\n",
              "      <td>1.317686</td>\n",
              "      <td>26.322296</td>\n",
              "      <td>8.339773</td>\n",
              "      <td>1.634240</td>\n",
              "      <td>0.093914</td>\n",
              "      <td>3.148632</td>\n",
              "      <td>0.002389</td>\n",
              "      <td>11.281962</td>\n",
              "      <td>0.035505</td>\n",
              "      <td>2.294898</td>\n",
              "      <td>1.479832</td>\n",
              "      <td>2.269158</td>\n",
              "      <td>0.244376</td>\n",
              "      <td>10.180258</td>\n",
              "      <td>1.069359</td>\n",
              "      <td>20210429</td>\n",
              "      <td>202104</td>\n",
              "      <td>785074.3</td>\n",
              "      <td>20210429</td>\n",
              "      <td>2021</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2060</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.236877</td>\n",
              "      <td>0.826087</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>61839</th>\n",
              "      <td>656775</td>\n",
              "      <td>7.448023</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>0.855258</td>\n",
              "      <td>2.674405</td>\n",
              "      <td>20210430</td>\n",
              "      <td>Outfielder</td>\n",
              "      <td>Cedric Mullins</td>\n",
              "      <td>110.0</td>\n",
              "      <td>Active</td>\n",
              "      <td>100.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>2.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>...</td>\n",
              "      <td>0.001855</td>\n",
              "      <td>43.288911</td>\n",
              "      <td>1.447754</td>\n",
              "      <td>9.850177</td>\n",
              "      <td>8.171621</td>\n",
              "      <td>6.465414</td>\n",
              "      <td>1.317686</td>\n",
              "      <td>26.322296</td>\n",
              "      <td>8.339773</td>\n",
              "      <td>1.634240</td>\n",
              "      <td>0.093914</td>\n",
              "      <td>3.148632</td>\n",
              "      <td>0.002389</td>\n",
              "      <td>11.281962</td>\n",
              "      <td>0.035505</td>\n",
              "      <td>2.294898</td>\n",
              "      <td>1.479832</td>\n",
              "      <td>2.269158</td>\n",
              "      <td>0.244376</td>\n",
              "      <td>10.180258</td>\n",
              "      <td>1.069359</td>\n",
              "      <td>20210430</td>\n",
              "      <td>202104</td>\n",
              "      <td>785074.3</td>\n",
              "      <td>20210430</td>\n",
              "      <td>2021</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2060</td>\n",
              "      <td>3</td>\n",
              "      <td>20</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>46.0</td>\n",
              "      <td>3.0</td>\n",
              "      <td>0.236877</td>\n",
              "      <td>0.826087</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>2508784 rows × 144 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "       playerId    target1  ...  preseasont1mean  hitpergame\n",
              "0        628317   0.011167  ...              NaN         NaN\n",
              "1        628317   0.006356  ...              NaN         NaN\n",
              "2        628317   0.003556  ...              NaN         NaN\n",
              "3        628317   0.003363  ...              NaN         NaN\n",
              "4        628317   0.023976  ...              NaN         NaN\n",
              "...         ...        ...  ...              ...         ...\n",
              "61835    656775  16.920553  ...         0.236877    0.826087\n",
              "61836    656775   1.779862  ...         0.236877    0.826087\n",
              "61837    656775  12.440696  ...         0.236877    0.826087\n",
              "61838    656775  10.765473  ...         0.236877    0.826087\n",
              "61839    656775   7.448023  ...         0.236877    0.826087\n",
              "\n",
              "[2508784 rows x 144 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 225
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:17.614762Z",
          "iopub.status.busy": "2021-06-25T22:19:17.614054Z",
          "iopub.status.idle": "2021-06-25T22:19:17.624743Z",
          "shell.execute_reply": "2021-06-25T22:19:17.625239Z",
          "shell.execute_reply.started": "2021-06-25T21:58:57.444888Z"
        },
        "papermill": {
          "duration": 0.04094,
          "end_time": "2021-06-25T22:19:17.625445",
          "exception": false,
          "start_time": "2021-06-25T22:19:17.584505",
          "status": "completed"
        },
        "tags": [],
        "id": "sticky-stand",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "bfd86e79-a98c-4de4-d737-828284d2ec4c"
      },
      "source": [
        "def fit_lgbm(x_train, y_train, x_valid, y_valid, params: dict=None, verbose=100):\n",
        "    oof_pred = np.zeros(len(y_valid), dtype=np.float32)\n",
        "    #model = lgbm.LGBMRegressor(**params)\n",
        "    best_params, tuning_history = {}, []\n",
        "    \n",
        "    lgb_train = lgbm.Dataset(x_train, y_train)\n",
        "    lgb_eval = lgbm.Dataset(x_valid, y_valid, reference=lgb_train)\n",
        "    model = lgbm.train(params, \n",
        "        lgb_train, valid_sets=lgb_eval,  \n",
        "        early_stopping_rounds=100,\n",
        "        num_boost_round=10000,\n",
        "        verbose_eval=100,\n",
        "        #verbose=verbose,               \n",
        "        )\n",
        "    oof_pred = model.predict(x_valid)\n",
        "    score = mean_absolute_error(oof_pred, y_valid)\n",
        "    print('mae:', score)\n",
        "    return oof_pred, model, score\n",
        "\n",
        "\n",
        "\n",
        "import pickle\n",
        "model1p = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model1.pkl', 'rb'))\n",
        "model2p = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model2.pkl', 'rb'))\n",
        "model3p = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model3.pkl', 'rb'))\n",
        "model4p = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model4.pkl', 'rb'))\n",
        "\n",
        "\n",
        "params1 = {'objective':'mae','reg_alpha': 0.14947461820098767, 'reg_lambda': 0.10185644384043743, 'n_estimators': 3633, 'learning_rate': 0.08046301304430488, 'num_leaves': 674, 'feature_fraction': 0.9101240539122566, 'bagging_fraction': 0.9884451442950513, 'bagging_freq': 8, 'min_child_samples': 51}\n",
        "\n",
        "params2 = {\n",
        " 'objective':'mae',\n",
        " 'reg_alpha': 0.1,\n",
        " 'reg_lambda': 0.1, \n",
        " 'n_estimators': 5000,\n",
        " 'learning_rate': 0.1,\n",
        " 'random_state': 42,\n",
        " \"num_leaves\": 22\n",
        "}\n",
        "\n",
        "params4 = {'objective':'mae','reg_alpha': 0.016468100279441976, 'reg_lambda': 0.09128335764019105, 'n_estimators': 9868, 'learning_rate': 0.10528150510326864, 'num_leaves': 157, 'feature_fraction': 0.5419185713426886, 'bagging_fraction': 0.2637405128936662, 'bagging_freq': 19, 'min_child_samples': 71}\n",
        "\n",
        "\n",
        "params3 = {\n",
        " 'objective':'mae',\n",
        " 'reg_alpha': 0.1,\n",
        " 'reg_lambda': 0.1, \n",
        " 'n_estimators': 10000,\n",
        " 'learning_rate': 0.1,\n",
        " 'random_state': 42,\n",
        " \"num_leaves\": 100\n",
        "}\n",
        "\n",
        "oof1, model1, score1 = fit_lgbm(\n",
        "    x_train[feature_cols], y_train['target1'],\n",
        "    x_valid[feature_cols], y_valid['target1'],\n",
        "    params1\n",
        ")\n",
        "\n",
        "file = '/content/drive/MyDrive/mlb/script/output/model1.pkl'\n",
        "pickle.dump(model1, open(file, 'wb'))\n",
        "\n",
        "oof2, model2, score2 = fit_lgbm(\n",
        "    x_train[feature_cols2], y_train['target2'],\n",
        "    x_valid[feature_cols2], y_valid['target2'],\n",
        "    params2\n",
        ")\n",
        "\n",
        "file = '/content/drive/MyDrive/mlb/script/output/model2.pkl'\n",
        "pickle.dump(model2, open(file, 'wb'))\n",
        "\n",
        "oof3, model3, score3 = fit_lgbm(\n",
        "    x_train[feature_cols3], y_train['target3'],\n",
        "    x_valid[feature_cols3], y_valid['target3'],\n",
        "    params3\n",
        ")\n",
        "\n",
        "file = '/content/drive/MyDrive/mlb/script/output/model3.pkl'\n",
        "pickle.dump(model3, open(file, 'wb'))\n",
        "\n",
        "\n",
        "oof4, model4, score4 = fit_lgbm(\n",
        "    x_train[feature_cols4], y_train['target4'],\n",
        "    x_valid[feature_cols4], y_valid['target4'],\n",
        "    params4\n",
        ")\n",
        "\n",
        "\n",
        "file = '/content/drive/MyDrive/mlb/script/output/model4.pkl'\n",
        "pickle.dump(model4, open(file, 'wb'))\n",
        "score = (score1+score2+score3+score4) / 4\n",
        "print(f'score: {score}')\n"
      ],
      "id": "sticky-stand",
      "execution_count": 226,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/lightgbm/engine.py:118: UserWarning: Found `n_estimators` in params. Will use it instead of argument\n",
            "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Training until validation scores don't improve for 100 rounds.\n",
            "[100]\tvalid_0's l1: 0.677665\n",
            "[200]\tvalid_0's l1: 0.67696\n",
            "[300]\tvalid_0's l1: 0.676784\n",
            "Early stopping, best iteration is:\n",
            "[226]\tvalid_0's l1: 0.676771\n",
            "mae: 0.6767709627997939\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/lightgbm/engine.py:118: UserWarning: Found `n_estimators` in params. Will use it instead of argument\n",
            "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Training until validation scores don't improve for 100 rounds.\n",
            "[100]\tvalid_0's l1: 1.33172\n",
            "[200]\tvalid_0's l1: 1.32749\n",
            "[300]\tvalid_0's l1: 1.32249\n",
            "[400]\tvalid_0's l1: 1.32262\n",
            "Early stopping, best iteration is:\n",
            "[323]\tvalid_0's l1: 1.32193\n",
            "mae: 1.3219301523752427\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/lightgbm/engine.py:118: UserWarning: Found `n_estimators` in params. Will use it instead of argument\n",
            "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Training until validation scores don't improve for 100 rounds.\n",
            "[100]\tvalid_0's l1: 0.53873\n",
            "[200]\tvalid_0's l1: 0.53818\n",
            "Early stopping, best iteration is:\n",
            "[129]\tvalid_0's l1: 0.53818\n",
            "mae: 0.5381795835267164\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/lightgbm/engine.py:118: UserWarning: Found `n_estimators` in params. Will use it instead of argument\n",
            "  warnings.warn(\"Found `{}` in params. Will use it instead of argument\".format(alias))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Training until validation scores don't improve for 100 rounds.\n",
            "[100]\tvalid_0's l1: 1.00492\n",
            "[200]\tvalid_0's l1: 0.996887\n",
            "[300]\tvalid_0's l1: 0.992907\n",
            "[400]\tvalid_0's l1: 0.992189\n",
            "[500]\tvalid_0's l1: 0.992048\n",
            "[600]\tvalid_0's l1: 0.990627\n",
            "[700]\tvalid_0's l1: 0.989141\n",
            "[800]\tvalid_0's l1: 0.987687\n",
            "[900]\tvalid_0's l1: 0.987769\n",
            "[1000]\tvalid_0's l1: 0.987134\n",
            "[1100]\tvalid_0's l1: 0.986923\n",
            "[1200]\tvalid_0's l1: 0.986515\n",
            "Early stopping, best iteration is:\n",
            "[1129]\tvalid_0's l1: 0.985998\n",
            "mae: 0.9859983876733532\n",
            "score: 0.8807197715937766\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:17.682673Z",
          "iopub.status.busy": "2021-06-25T22:19:17.681978Z",
          "iopub.status.idle": "2021-06-25T22:19:19.467296Z",
          "shell.execute_reply": "2021-06-25T22:19:19.467916Z",
          "shell.execute_reply.started": "2021-06-25T21:58:57.456212Z"
        },
        "papermill": {
          "duration": 1.81619,
          "end_time": "2021-06-25T22:19:19.468136",
          "exception": false,
          "start_time": "2021-06-25T22:19:17.651946",
          "status": "completed"
        },
        "tags": [],
        "id": "large-orlando"
      },
      "source": [
        "import pickle\n",
        "model1 = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model1.pkl', 'rb'))\n",
        "model2 = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model2.pkl', 'rb'))\n",
        "model3 = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model3.pkl', 'rb'))\n",
        "model4 = pickle.load(open('/content/drive/MyDrive/mlb/script/params/model4.pkl', 'rb'))"
      ],
      "id": "large-orlando",
      "execution_count": 227,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xkqkayONf3SC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1fbad0ea-62be-4e56-fef4-c065dd083226"
      },
      "source": [
        "model1.params"
      ],
      "id": "xkqkayONf3SC",
      "execution_count": 228,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'bagging_fraction': 0.5084433811583908,\n",
              " 'bagging_freq': 7,\n",
              " 'feature_fraction': 0.5,\n",
              " 'feature_pre_filter': False,\n",
              " 'lambda_l1': 4.3588274813177543e-07,\n",
              " 'lambda_l2': 0.0004358428728553654,\n",
              " 'learning_rate': 0.1,\n",
              " 'metric': 'l1',\n",
              " 'min_child_samples': 10,\n",
              " 'min_data_in_leaf': 20,\n",
              " 'num_leaves': 253,\n",
              " 'objective': 'mae',\n",
              " 'random_state': 42}"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 228
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:19.523599Z",
          "iopub.status.busy": "2021-06-25T22:19:19.522922Z",
          "iopub.status.idle": "2021-06-25T22:19:19.531732Z",
          "shell.execute_reply": "2021-06-25T22:19:19.532260Z",
          "shell.execute_reply.started": "2021-06-25T21:58:59.002054Z"
        },
        "papermill": {
          "duration": 0.038158,
          "end_time": "2021-06-25T22:19:19.532427",
          "exception": false,
          "start_time": "2021-06-25T22:19:19.494269",
          "status": "completed"
        },
        "tags": [],
        "id": "sensitive-establishment"
      },
      "source": [
        "players_cols = ['playerId', 'primaryPositionName']\n",
        "rosters_cols = ['playerId', 'teamId', 'status']\n",
        "scores_cols = ['playerId', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored', 'doubles', 'triples', 'homeRuns',\n",
        "       'strikeOuts', 'baseOnBalls', 'intentionalWalks', 'hits', 'hitByPitch',\n",
        "       'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "       'groundIntoTriplePlay', 'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "       'pickoffs', 'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'completeGamesPitching', 'shutoutsPitching', 'winsPitching',\n",
        "       'lossesPitching', 'flyOutsPitching', 'airOutsPitching',\n",
        "       'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       'baseOnBallsPitching', 'intentionalWalksPitching', 'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched', 'saveOpportunities',\n",
        "       'earnedRuns', 'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored', 'catchersInterferencePitching',\n",
        "       'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "       'assists', 'putOuts', 'errors', 'chances']\n",
        "standings_cols = [\"teamId\",\"divisionRank\",\"divisionLeader\",\"wildCardLeader\",\"leagueRank\",\"divisionId\",\"gameDate\"]\n",
        "null = np.nan\n",
        "true = True\n",
        "false = False"
      ],
      "id": "sensitive-establishment",
      "execution_count": 229,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:19:19.604016Z",
          "iopub.status.busy": "2021-06-25T22:19:19.598864Z",
          "iopub.status.idle": "2021-06-25T22:23:45.296062Z",
          "shell.execute_reply": "2021-06-25T22:23:45.295267Z",
          "shell.execute_reply.started": "2021-06-25T21:58:59.015564Z"
        },
        "papermill": {
          "duration": 265.737683,
          "end_time": "2021-06-25T22:23:45.296307",
          "exception": false,
          "start_time": "2021-06-25T22:19:19.558624",
          "status": "completed"
        },
        "tags": [],
        "id": "hundred-thunder",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 447
        },
        "outputId": "37fb8b45-ea38-4fd6-e691-38ac67131273"
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from datetime import timedelta\n",
        "from tqdm import tqdm\n",
        "import gc\n",
        "from functools import reduce\n",
        "from sklearn.model_selection import StratifiedKFold\n",
        "\n",
        "ROOT_DIR = \"../input/mlb-player-digital-engagement-forecasting\"\n",
        "\n",
        "#=======================#\n",
        "def flatten(df, col):\n",
        "    du = (df.pivot(index=\"playerId\", columns=\"EvalDate\", \n",
        "               values=col).add_prefix(f\"{col}_\").\n",
        "      rename_axis(None, axis=1).reset_index())\n",
        "    return du\n",
        "#============================#\n",
        "def reducer(left, right):\n",
        "    return left.merge(right, on=\"playerId\")\n",
        "#========================\n",
        "\n",
        "TGTCOLS = [\"target1\",\"target2\",\"target3\",\"target4\"]\n",
        "def train_lag(df, lag=1):\n",
        "    dp = df[[\"playerId\",\"EvalDate\"]+TGTCOLS].copy()\n",
        "    dp[\"EvalDate\"]  =dp[\"EvalDate\"] + timedelta(days=lag) \n",
        "    df = df.merge(dp, on=[\"playerId\", \"EvalDate\"], suffixes=[\"\",f\"_{lag}\"], how=\"left\")\n",
        "    return df\n",
        "#=================================\n",
        "def test_lag(sub):\n",
        "    sub[\"playerId\"] = sub[\"date_playerId\"].apply(lambda s: int(  s.split(\"_\")[1]  ) )\n",
        "    assert sub.date.nunique() == 1\n",
        "    dte = sub[\"date\"].unique()[0]\n",
        "    \n",
        "    eval_dt = pd.to_datetime(dte, format=\"%Y%m%d\")\n",
        "    dtes = [eval_dt + timedelta(days=-k) for k in LAGS]\n",
        "    mp_dtes = {eval_dt + timedelta(days=-k):k for k in LAGS}\n",
        "    \n",
        "    sl = LAST.loc[LAST.EvalDate.between(dtes[-1], dtes[0]), [\"EvalDate\",\"playerId\"]+TGTCOLS].copy()\n",
        "    sl[\"EvalDate\"] = sl[\"EvalDate\"].map(mp_dtes)\n",
        "    du = [flatten(sl, col) for col in TGTCOLS]\n",
        "    du = reduce(reducer, du)\n",
        "    return du, eval_dt\n",
        "    #\n",
        "#===============\n",
        "\n",
        "tr = pd.read_csv(\"../input/mlb-data/target.csv\")\n",
        "print(tr.shape)\n",
        "gc.collect()\n",
        "\n",
        "tr[\"EvalDate\"] = pd.to_datetime(tr[\"EvalDate\"])\n",
        "tr[\"EvalDate\"] = tr[\"EvalDate\"] + timedelta(days=-1)\n",
        "tr[\"EvalYear\"] = tr[\"EvalDate\"].dt.year\n",
        "\n",
        "MED_DF = tr.groupby([\"playerId\",\"EvalYear\"])[TGTCOLS].median().reset_index()\n",
        "MEDCOLS = [\"tgt1_med\",\"tgt2_med\", \"tgt3_med\", \"tgt4_med\"]\n",
        "MED_DF.columns = [\"playerId\",\"EvalYear\"] + MEDCOLS\n",
        "\n",
        "LAGS = list(range(1,21))\n",
        "FECOLS = [f\"{col}_{lag}\" for lag in reversed(LAGS) for col in TGTCOLS]\n",
        "\n",
        "for lag in tqdm(LAGS):\n",
        "    tr = train_lag(tr, lag=lag)\n",
        "    gc.collect()\n",
        "#===========\n",
        "tr = tr.sort_values(by=[\"playerId\", \"EvalDate\"])\n",
        "print(tr.shape)\n",
        "tr = tr.dropna()\n",
        "print(tr.shape)\n",
        "tr = tr.merge(MED_DF, on=[\"playerId\",\"EvalYear\"])\n",
        "gc.collect()\n",
        "\n",
        "X = tr[FECOLS+MEDCOLS].values\n",
        "y = tr[TGTCOLS].values\n",
        "cl = tr[\"playerId\"].values\n",
        "\n",
        "NFOLDS = 6\n",
        "skf = StratifiedKFold(n_splits=NFOLDS)\n",
        "folds = skf.split(X, cl)\n",
        "folds = list(folds)\n",
        "\n",
        "import tensorflow as tf\n",
        "import tensorflow.keras.layers as L\n",
        "import tensorflow.keras.models as M\n",
        "from sklearn.metrics import mean_absolute_error, mean_squared_error\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping\n",
        "\n",
        "tf.random.set_seed(777)\n",
        "\n",
        "def make_model(n_in):\n",
        "    inp = L.Input(name=\"inputs\", shape=(n_in,))\n",
        "    x = L.Dense(50, activation=\"relu\", name=\"d1\")(inp)\n",
        "    x = L.Dense(50, activation=\"relu\", name=\"d2\")(x)\n",
        "    preds = L.Dense(4, activation=\"linear\", name=\"preds\")(x)\n",
        "    \n",
        "    model = M.Model(inp, preds, name=\"ANN\")\n",
        "    model.compile(loss=\"mean_absolute_error\", optimizer=\"adam\")\n",
        "    return model\n",
        "\n",
        "net = make_model(X.shape[1])\n",
        "print(net.summary())\n",
        "\n",
        "oof = np.zeros(y.shape)\n",
        "nets = []\n",
        "for idx in range(NFOLDS):\n",
        "    print(\"FOLD:\", idx)\n",
        "    tr_idx, val_idx = folds[idx]\n",
        "    ckpt = ModelCheckpoint(f\"w{idx}.h5\", monitor='val_loss', verbose=1, save_best_only=True,mode='min')\n",
        "    reduce_lr = ReduceLROnPlateau(monitor='val_loss', factor=0.2,patience=3, min_lr=0.0005)\n",
        "    es = EarlyStopping(monitor='val_loss', patience=6)\n",
        "    reg = make_model(X.shape[1])\n",
        "    reg.fit(X[tr_idx], y[tr_idx], epochs=10, batch_size=35_000, validation_data=(X[val_idx], y[val_idx]),\n",
        "            verbose=1, callbacks=[ckpt, reduce_lr, es])\n",
        "    reg.load_weights(f\"w{idx}.h5\")\n",
        "    oof[val_idx] = reg.predict(X[val_idx], batch_size=50_000, verbose=1)\n",
        "    nets.append(reg)\n",
        "    gc.collect()\n",
        "    #\n",
        "#\n",
        "\n",
        "mae = mean_absolute_error(y, oof)\n",
        "mse = mean_squared_error(y, oof, squared=False)\n",
        "print(\"mae:\", mae)\n",
        "print(\"mse:\", mse)\n",
        "\n",
        "# Historical information to use in prediction time\n",
        "bound_dt = pd.to_datetime(\"2021-01-01\")\n",
        "LAST = tr.loc[tr.EvalDate>bound_dt].copy()\n",
        "\n",
        "LAST_MED_DF = MED_DF.loc[MED_DF.EvalYear==2021].copy()\n",
        "LAST_MED_DF.drop(\"EvalYear\", axis=1, inplace=True)\n",
        "del tr\n",
        "\n",
        "#\"\"\"\n",
        "import mlb\n",
        "FE = []; SUB = [];"
      ],
      "id": "hundred-thunder",
      "execution_count": 230,
      "outputs": [
        {
          "output_type": "error",
          "ename": "FileNotFoundError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-230-248adacbd808>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     44\u001b[0m \u001b[0;31m#===============\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     45\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 46\u001b[0;31m \u001b[0mtr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_csv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"../input/mlb-data/target.csv\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     47\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     48\u001b[0m \u001b[0mgc\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcollect\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36mread_csv\u001b[0;34m(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)\u001b[0m\n\u001b[1;32m    686\u001b[0m     )\n\u001b[1;32m    687\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 688\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0m_read\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfilepath_or_buffer\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    689\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    690\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_read\u001b[0;34m(filepath_or_buffer, kwds)\u001b[0m\n\u001b[1;32m    452\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    453\u001b[0m     \u001b[0;31m# Create the parser.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 454\u001b[0;31m     \u001b[0mparser\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTextFileReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfp_or_buf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    455\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    456\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mchunksize\u001b[0m \u001b[0;32mor\u001b[0m \u001b[0miterator\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, f, engine, **kwds)\u001b[0m\n\u001b[1;32m    946\u001b[0m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"has_index_names\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    947\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 948\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mengine\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    949\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    950\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mclose\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m_make_engine\u001b[0;34m(self, engine)\u001b[0m\n\u001b[1;32m   1178\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_make_engine\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mengine\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m\"c\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1179\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"c\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1180\u001b[0;31m             \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_engine\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mCParserWrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moptions\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1181\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1182\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0mengine\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m\"python\"\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/pandas/io/parsers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, src, **kwds)\u001b[0m\n\u001b[1;32m   2008\u001b[0m         \u001b[0mkwds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"usecols\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0musecols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2009\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2010\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mparsers\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTextReader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0msrc\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2011\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munnamed_cols\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0munnamed_cols\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2012\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader.__cinit__\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;32mpandas/_libs/parsers.pyx\u001b[0m in \u001b[0;36mpandas._libs.parsers.TextReader._setup_parser_source\u001b[0;34m()\u001b[0m\n",
            "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: '../input/mlb-data/target.csv'"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K1VLUowOyspF"
      },
      "source": [
        ""
      ],
      "id": "K1VLUowOyspF",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "papermill": {
          "duration": 0.643124,
          "end_time": "2021-06-25T22:23:46.587799",
          "exception": false,
          "start_time": "2021-06-25T22:23:45.944675",
          "status": "completed"
        },
        "tags": [],
        "id": "defined-colors"
      },
      "source": [
        "## Inference"
      ],
      "id": "defined-colors"
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:23:47.872922Z",
          "iopub.status.busy": "2021-06-25T22:23:47.871916Z",
          "iopub.status.idle": "2021-06-25T22:23:47.875718Z",
          "shell.execute_reply": "2021-06-25T22:23:47.875029Z",
          "shell.execute_reply.started": "2021-06-25T22:03:06.318592Z"
        },
        "papermill": {
          "duration": 0.649231,
          "end_time": "2021-06-25T22:23:47.875889",
          "exception": false,
          "start_time": "2021-06-25T22:23:47.226658",
          "status": "completed"
        },
        "tags": [],
        "id": "classified-cooper"
      },
      "source": [
        "def unpack_json(json_str):\n",
        "    return np.nan if pd.isna(json_str) else pd.read_json(json_str)"
      ],
      "id": "classified-cooper",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:23:49.244729Z",
          "iopub.status.busy": "2021-06-25T22:23:49.243737Z",
          "iopub.status.idle": "2021-06-25T22:23:49.248381Z",
          "shell.execute_reply": "2021-06-25T22:23:49.247718Z",
          "shell.execute_reply.started": "2021-06-25T22:03:06.324648Z"
        },
        "papermill": {
          "duration": 0.652999,
          "end_time": "2021-06-25T22:23:49.248559",
          "exception": false,
          "start_time": "2021-06-25T22:23:48.595560",
          "status": "completed"
        },
        "tags": [],
        "id": "electoral-tribute"
      },
      "source": [
        "\"\"\"players_cols = ['playerId', 'primaryPositionName']\n",
        "rosters_cols = ['playerId', 'teamId', 'status']\n",
        "scores_cols = ['playerId', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored', 'doubles', 'triples', 'homeRuns',\n",
        "       'strikeOuts', 'baseOnBalls', 'intentionalWalks', 'hits', 'hitByPitch',\n",
        "       'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "       'groundIntoTriplePlay', 'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "       'pickoffs', 'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'completeGamesPitching', 'shutoutsPitching', 'winsPitching',\n",
        "       'lossesPitching', 'flyOutsPitching', 'airOutsPitching',\n",
        "       'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       'baseOnBallsPitching', 'intentionalWalksPitching', 'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched', 'saveOpportunities',\n",
        "       'earnedRuns', 'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored', 'catchersInterferencePitching',\n",
        "       'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "       'assists', 'putOuts', 'errors', 'chances']\n",
        "\n",
        "null = np.nan\n",
        "true = True\n",
        "false = False\"\"\""
      ],
      "id": "electoral-tribute",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:23:50.567106Z",
          "iopub.status.busy": "2021-06-25T22:23:50.561703Z",
          "iopub.status.idle": "2021-06-25T22:24:01.199513Z",
          "shell.execute_reply": "2021-06-25T22:24:01.198747Z",
          "shell.execute_reply.started": "2021-06-25T22:03:06.3406Z"
        },
        "papermill": {
          "duration": 11.293137,
          "end_time": "2021-06-25T22:24:01.199706",
          "exception": false,
          "start_time": "2021-06-25T22:23:49.906569",
          "status": "completed"
        },
        "tags": [],
        "id": "thousand-comfort"
      },
      "source": [
        "\"\"\"players_cols = ['playerId', 'primaryPositionName']\n",
        "rosters_cols = ['playerId', 'teamId', 'status']\n",
        "scores_cols = ['playerId', 'battingOrder', 'gamesPlayedBatting', 'flyOuts',\n",
        "       'groundOuts', 'runsScored', 'doubles', 'triples', 'homeRuns',\n",
        "       'strikeOuts', 'baseOnBalls', 'intentionalWalks', 'hits', 'hitByPitch',\n",
        "       'atBats', 'caughtStealing', 'stolenBases', 'groundIntoDoublePlay',\n",
        "       'groundIntoTriplePlay', 'plateAppearances', 'totalBases', 'rbi',\n",
        "       'leftOnBase', 'sacBunts', 'sacFlies', 'catchersInterference',\n",
        "       'pickoffs', 'gamesPlayedPitching', 'gamesStartedPitching',\n",
        "       'completeGamesPitching', 'shutoutsPitching', 'winsPitching',\n",
        "       'lossesPitching', 'flyOutsPitching', 'airOutsPitching',\n",
        "       'groundOutsPitching', 'runsPitching', 'doublesPitching',\n",
        "       'triplesPitching', 'homeRunsPitching', 'strikeOutsPitching',\n",
        "       'baseOnBallsPitching', 'intentionalWalksPitching', 'hitsPitching',\n",
        "       'hitByPitchPitching', 'atBatsPitching', 'caughtStealingPitching',\n",
        "       'stolenBasesPitching', 'inningsPitched', 'saveOpportunities',\n",
        "       'earnedRuns', 'battersFaced', 'outsPitching', 'pitchesThrown', 'balls',\n",
        "       'strikes', 'hitBatsmen', 'balks', 'wildPitches', 'pickoffsPitching',\n",
        "       'rbiPitching', 'gamesFinishedPitching', 'inheritedRunners',\n",
        "       'inheritedRunnersScored', 'catchersInterferencePitching',\n",
        "       'sacBuntsPitching', 'sacFliesPitching', 'saves', 'holds', 'blownSaves',\n",
        "       'assists', 'putOuts', 'errors', 'chances',\"gamePk\"]\n",
        "\n",
        "null = np.nan\n",
        "true = True\n",
        "false = False\n",
        "\n",
        "env = mlb.make_env() # initialize the environment\n",
        "iter_test = env.iter_test() # iterator which loops over each date in test set\n",
        "\n",
        "for (test_df, sample_prediction_df) in iter_test: # make predictions here\n",
        "    \n",
        "    sub = copy.deepcopy(sample_prediction_df.reset_index())\n",
        "    sample_prediction_df = copy.deepcopy(sample_prediction_df.reset_index(drop=True))\n",
        "    \n",
        "    \n",
        "    # creat dataset\n",
        "    sample_prediction_df['playerId'] = sample_prediction_df['date_playerId']\\\n",
        "                                        .map(lambda x: int(x.split('_')[1]))\n",
        "    # Dealing with missing values\n",
        "    if test_df['rosters'].iloc[0] == test_df['rosters'].iloc[0]:\n",
        "        test_rosters = pd.DataFrame(eval(test_df['rosters'].iloc[0]))\n",
        "    else:\n",
        "        test_rosters = pd.DataFrame({'playerId': sample_prediction_df['playerId']})\n",
        "        for col in rosters.columns:\n",
        "            if col == 'playerId': continue\n",
        "            test_rosters[col] = np.nan\n",
        "            \n",
        "    if test_df['playerBoxScores'].iloc[0] == test_df['playerBoxScores'].iloc[0]:\n",
        "        test_scores = pd.DataFrame(eval(test_df['playerBoxScores'].iloc[0]))\n",
        "    else:\n",
        "        test_scores = pd.DataFrame({'playerId': sample_prediction_df['playerId']})\n",
        "        for col in scores.columns:\n",
        "            if col == 'playerId': continue\n",
        "            test_scores[col] = np.nan\n",
        "    \n",
        "    test_games = unpack_json(test_df[\"games\"].iloc[0])\n",
        "    test_standings = unpack_json(test_df[\"standings\"].iloc[0])\n",
        "    \n",
        "            \n",
        "            \n",
        "    \n",
        "\n",
        "    \n",
        "\n",
        "\n",
        "\n",
        "        \n",
        "    test_scores = test_scores.groupby('playerId').sum().reset_index()\n",
        "    test = sample_prediction_df[['playerId']].copy()\n",
        "    test = test.merge(players[players_cols], on='playerId', how='left')\n",
        "    test = test.merge(test_rosters[rosters_cols], on='playerId', how='left')\n",
        "    test = test.merge(test_scores[scores_cols], on='playerId', how='left')\n",
        "    test = test.merge(test_games[games_cols], on=\"gamePk\", how=\"left\")\n",
        "    test = test.merge(awards2[awards_cols], on=['playerId'], how='left')\n",
        "    test = test.merge(player_target_stats, how='inner', left_on=[\"playerId\"],right_on=[\"playerId\"])\n",
        "    test = test.merge(test_standings[standings_cols], on=[\"teamId\"], how='left')\n",
        "    \n",
        "    #add feature\n",
        "    test[\"ongame\"] = np.where(test[\"gamePk\"].isnull() == 1,0,1)\n",
        "    test[\"ishome\"] = np.where(test[\"teamId\"]==test[\"homeId\"],2,test[\"ongame\"])\n",
        "    test[\"winorlose\"] = np.where(test[\"teamId\"]==test[\"homeId\"],test[\"homeWinner\"],test[\"awayWinner\"])\n",
        "    test[\"winorlose\"] = test[\"winorlose\"].fillna(2.0).astype(int)\n",
        "    test[\"score\"] = np.where(test[\"teamId\"]==test[\"homeId\"],test[\"homeScore\"],test[\"awayScore\"])\n",
        "    test[\"divisionRank\"] = test[\"divisionRank\"].fillna(7.0).astype(int)\n",
        "    test[\"divisionLeader\"] = test[\"divisionLeader\"].fillna(-1.0).astype(int)\n",
        "    test[\"wildCardLeader\"] = np.where(test[\"wildCardLeader\"]==\"True\",1,0)\n",
        "    \n",
        "    \n",
        "    \n",
        "    \n",
        "    #label encoding\n",
        "    test['label_playerId'] = test['playerId'].map(player2num)\n",
        "    test['label_primaryPositionName'] = test['primaryPositionName'].map(position2num)\n",
        "    test['label_teamId'] = test['teamId'].map(teamid2num)\n",
        "    test['label_status'] = test['status'].map(status2num)\n",
        "    test[\"label_daynight\"] = test['dayNight'].map(daynight2num)\n",
        "    test[\"label_seriesDescription\"] = test[\"seriesDescription\"].map(seriesDescription2num)\n",
        "    test[\"gameType\"] = test[\"gameType\"].map(gameType2num)\n",
        "    display(test)\n",
        "    \n",
        "    \n",
        "    test_X1 = test[feature_cols1]\n",
        "    test_X2 = test[feature_cols2]\n",
        "    test_X3 = test[feature_cols3]\n",
        "    test_X4 = test[feature_cols4]\n",
        "    \n",
        "    # predict\n",
        "    pred1 = model1.predict(test_X1)\n",
        "    pred2 = model2.predict(test_X2)\n",
        "    pred3 = model3.predict(test_X3)\n",
        "    pred4 = model4.predict(test_X4)\n",
        "    \n",
        "    # merge submission\n",
        "    sample_prediction_df['target1'] = np.clip(pred1, 0, 100)\n",
        "    sample_prediction_df['target2'] = np.clip(pred2, 0, 100)\n",
        "    sample_prediction_df['target3'] = np.clip(pred3, 0, 100)\n",
        "    sample_prediction_df['target4'] = np.clip(pred4, 0, 100)\n",
        "    sample_prediction_df = sample_prediction_df.fillna(0.)\n",
        "    del sample_prediction_df['playerId']\n",
        "    \n",
        "    # TF summit\n",
        "    # Features computation at Evaluation Date\n",
        "    sub_fe, eval_dt = test_lag(sub)\n",
        "    sub_fe = sub_fe.merge(LAST_MED_DF, on=\"playerId\", how=\"left\")\n",
        "    sub_fe = sub_fe.fillna(0.)\n",
        "    \n",
        "    _preds = 0.\n",
        "    for reg in nets:\n",
        "        _preds += reg.predict(sub_fe[FECOLS + MEDCOLS]) / NFOLDS\n",
        "    sub_fe[TGTCOLS] = np.clip(_preds, 0, 100)\n",
        "    sub.drop([\"date\"]+TGTCOLS, axis=1, inplace=True)\n",
        "    sub = sub.merge(sub_fe[[\"playerId\"]+TGTCOLS], on=\"playerId\", how=\"left\")\n",
        "    sub.drop(\"playerId\", axis=1, inplace=True)\n",
        "    sub = sub.fillna(0.)\n",
        "    # Blending\n",
        "    blend = pd.concat(\n",
        "        [sub[['date_playerId']],\n",
        "        (0.1*sub.drop('date_playerId', axis=1) + 0.9*sample_prediction_df.drop('date_playerId', axis=1))],\n",
        "        axis=1\n",
        "    )\n",
        "    env.predict(blend)\n",
        "    # Update Available information\n",
        "    sub_fe[\"EvalDate\"] = eval_dt\n",
        "    #sub_fe.drop(MEDCOLS, axis=1, inplace=True)\n",
        "    LAST = LAST.append(sub_fe)\n",
        "    LAST = LAST.drop_duplicates(subset=[\"EvalDate\",\"playerId\"], keep=\"last\")\"\"\""
      ],
      "id": "thousand-comfort",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:24:02.554240Z",
          "iopub.status.busy": "2021-06-25T22:24:02.553286Z",
          "iopub.status.idle": "2021-06-25T22:24:02.557808Z",
          "shell.execute_reply": "2021-06-25T22:24:02.557241Z"
        },
        "papermill": {
          "duration": 0.688226,
          "end_time": "2021-06-25T22:24:02.557998",
          "exception": false,
          "start_time": "2021-06-25T22:24:01.869772",
          "status": "completed"
        },
        "tags": [],
        "id": "brief-palmer"
      },
      "source": [
        "display(test_df['games'])"
      ],
      "id": "brief-palmer",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:24:03.891241Z",
          "iopub.status.busy": "2021-06-25T22:24:03.890528Z",
          "iopub.status.idle": "2021-06-25T22:24:03.893839Z",
          "shell.execute_reply": "2021-06-25T22:24:03.893065Z"
        },
        "papermill": {
          "duration": 0.657564,
          "end_time": "2021-06-25T22:24:03.894025",
          "exception": false,
          "start_time": "2021-06-25T22:24:03.236461",
          "status": "completed"
        },
        "tags": [],
        "id": "partial-morning"
      },
      "source": [
        "def unpack_json(json_str):\n",
        "    return np.nan if pd.isna(json_str) else pd.read_json(json_str)"
      ],
      "id": "partial-morning",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:24:05.315595Z",
          "iopub.status.busy": "2021-06-25T22:24:05.314919Z",
          "iopub.status.idle": "2021-06-25T22:24:05.325682Z",
          "shell.execute_reply": "2021-06-25T22:24:05.325144Z"
        },
        "papermill": {
          "duration": 0.761079,
          "end_time": "2021-06-25T22:24:05.325843",
          "exception": false,
          "start_time": "2021-06-25T22:24:04.564764",
          "status": "completed"
        },
        "tags": [],
        "id": "smoking-discretion"
      },
      "source": [
        "unpack_json(test_df[\"games\"].iloc[0])"
      ],
      "id": "smoking-discretion",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:24:06.665554Z",
          "iopub.status.busy": "2021-06-25T22:24:06.663884Z",
          "iopub.status.idle": "2021-06-25T22:24:06.679898Z",
          "shell.execute_reply": "2021-06-25T22:24:06.679341Z"
        },
        "papermill": {
          "duration": 0.692076,
          "end_time": "2021-06-25T22:24:06.680048",
          "exception": false,
          "start_time": "2021-06-25T22:24:05.987972",
          "status": "completed"
        },
        "tags": [],
        "id": "talented-generic"
      },
      "source": [
        "pd.concat(\n",
        "    [sub[['date_playerId']],\n",
        "    (sub.drop('date_playerId', axis=1) + sample_prediction_df.drop('date_playerId', axis=1)) / 2],\n",
        "    axis=1\n",
        ")"
      ],
      "id": "talented-generic",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "execution": {
          "iopub.execute_input": "2021-06-25T22:24:08.007421Z",
          "iopub.status.busy": "2021-06-25T22:24:08.006746Z",
          "iopub.status.idle": "2021-06-25T22:24:08.009957Z",
          "shell.execute_reply": "2021-06-25T22:24:08.010404Z"
        },
        "papermill": {
          "duration": 0.679239,
          "end_time": "2021-06-25T22:24:08.010626",
          "exception": false,
          "start_time": "2021-06-25T22:24:07.331387",
          "status": "completed"
        },
        "tags": [],
        "id": "vertical-ethernet"
      },
      "source": [
        "sample_prediction_df"
      ],
      "id": "vertical-ethernet",
      "execution_count": null,
      "outputs": []
    }
  ]
}