sklearn_Train_Test_Split.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train Test Split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split # train_test_split module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate some data we are going to split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's generate a new data frame 'a' which will contain all integers from 1 to 100\n",
    "# The method np.arange works like the built-in method 'range' with the difference it creates an array\n",
    "a = np.arange(1,101)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,\n",
       "        14,  15,  16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,\n",
       "        27,  28,  29,  30,  31,  32,  33,  34,  35,  36,  37,  38,  39,\n",
       "        40,  41,  42,  43,  44,  45,  46,  47,  48,  49,  50,  51,  52,\n",
       "        53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,  64,  65,\n",
       "        66,  67,  68,  69,  70,  71,  72,  73,  74,  75,  76,  77,  78,\n",
       "        79,  80,  81,  82,  83,  84,  85,  86,  87,  88,  89,  90,  91,\n",
       "        92,  93,  94,  95,  96,  97,  98,  99, 100])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check it out\n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513,\n",
       "       514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526,\n",
       "       527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539,\n",
       "       540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552,\n",
       "       553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565,\n",
       "       566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578,\n",
       "       579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591,\n",
       "       592, 593, 594, 595, 596, 597, 598, 599, 600])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Similarly, let's create another ndarray 'b', which will contain integers from 501 to 600\n",
    "# We have intentionally picked these numbers so we can easily compare the two\n",
    "# Obviously, the difference between the elements of the two arrays is 500 for any two corresponding elements\n",
    "b = np.arange(501,601)\n",
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the data\n",
    "Documentation: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([ 82,  58,  64,  20,  42,  68,  34,  47,  94,   3,  72,  71,  31,\n",
       "         53,  81,   6,  67,  99,  21,   2,  18,  87,  40,   5,  92,   4,\n",
       "         24,  93,  29,  95,  62,  98,  97,  15,  89,  56,  63,  80,  60,\n",
       "         26,  45,  23,  57,  51,  86,   1,  33,  19,  54,   8,  27,  96,\n",
       "         25,  85,  77,  43,   9,  88, 100,  44,  35,  17,  65,  84,  76,\n",
       "         91,  55,  28,  38,  49,  36,  39,  75,  50,   7]),\n",
       " array([30, 70, 78, 69, 11, 16, 37, 74, 73, 59, 66, 46, 13, 12, 83, 14, 41,\n",
       "        90, 48, 79, 52, 32, 22, 10, 61])]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check out how this works\n",
    "train_test_split(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to split the data??\n",
    "\n",
    "There are several different arguments we can set when we employ this method\n",
    "Most often, we have inputs and targets, so we have to split 2 different arrays\n",
    "we are simulating this situation by splitting 'a' and 'b'\n",
    "\n",
    "You can specify the 'test_size' or the 'train_size' (but the latter is deprecated and will be removed)\n",
    "essentially the two have the same meaning \n",
    "Common splits are 75-25, 80-20, 85-15, 90-10\n",
    "\n",
    "Finally, you should always employ a 'random_state'\n",
    "In this way you ensure that when you are splitting the data you will always get the SAME random shuffle\n",
    "\n",
    "Note 2 arrays will be split into 4\n",
    "The order is train1, test1, train2, test2 \n",
    "It is very useful to store them in 4 variables, so we can later use them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "a_train, a_test, b_train, b_test = train_test_split(a, b, test_size=0.2, random_state=365)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore the result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((80,), (20,))"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check the shapes\n",
    "# Basically, we are checking how does the 'test_size' work\n",
    "a_train.shape, a_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 25,  32,  99,  73,  91,  66,   3,  59,  94,   1,   8,  15,  90,\n",
       "        54,  31,  20,  77,  82,  30,  35,  95,  42,  38,   7,  11,  50,\n",
       "        21,  48,   2,  17,  10,  58,  68,  43,  41,  16,  88,  72,  79,\n",
       "       100,  80,  39,  24,  86,  22,  23,  62,  76,  18,  47,  55,  26,\n",
       "        60,  19,  71,  64,  51,  63,  65,  28,  12,  78,  13,  44,  75,\n",
       "        87,  40,   4,  29,  49,  37,  57,  27,  74,   6,  45,  92,  34,\n",
       "        53,  83])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Explore manually\n",
    "a_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 9, 69, 81, 56, 33, 93, 84, 61, 46, 89, 85, 67, 97,  5, 70, 36, 98,\n",
       "       96, 14, 52])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Explore manually\n",
    "a_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((80,), (20,))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b_train.shape, b_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([525, 532, 599, 573, 591, 566, 503, 559, 594, 501, 508, 515, 590,\n",
       "       554, 531, 520, 577, 582, 530, 535, 595, 542, 538, 507, 511, 550,\n",
       "       521, 548, 502, 517, 510, 558, 568, 543, 541, 516, 588, 572, 579,\n",
       "       600, 580, 539, 524, 586, 522, 523, 562, 576, 518, 547, 555, 526,\n",
       "       560, 519, 571, 564, 551, 563, 565, 528, 512, 578, 513, 544, 575,\n",
       "       587, 540, 504, 529, 549, 537, 557, 527, 574, 506, 545, 592, 534,\n",
       "       553, 583])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([509, 569, 581, 556, 533, 593, 584, 561, 546, 589, 585, 567, 597,\n",
       "       505, 570, 536, 598, 596, 514, 552])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "b_test"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}