逻辑回归/01. 从回归到分类.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们前面学习了线性回归，可以知道，线性回归是通过对一类呈线性分布的数据进行拟合，然后训练一个线性模型对数据进行预测。在回归问题中，这些数据是连续值，但如果我们处理的数据是离散的，那么就是分类问题，线性回归将不再适用。\n",
    "\n",
    "和线性回归一样，我们从一个最简单的例子开始探索分类问题。假设我们有下面一组数据，表示某个班级中所有学生某一次考试的成绩以及通过情况，但是我们不知道这次考试老师定的及格线是多少。\n",
    "\n",
    "|考试成绩|是否通过|\n",
    "|-------|-------|\n",
    "|80  |通过     |\n",
    "|85  |通过     |\n",
    "|74  |未通过     |\n",
    "|63  |未通过     |\n",
    "|90  |通过     |\n",
    "|70  |未通过     |\n",
    "\n",
    "现在假如张三的考试成绩是 82，那么能推测出张三有没有通过考试吗？又或者李四考试成绩是 76，他有没有通过考试？\n",
    "\n",
    "很显然，输入变量只有一个，所以这可以称为**一元分类问题**，而输出只有两种情况，通过和未通过，所以也可以称为**二分类问题**。我们把通过记作 1，未通过记作 0，在坐标系中画出相应的散点图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAD8CAYAAABkbJM/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAFKpJREFUeJzt3X+QXeV93/H3pxI/ZJpUgGQbBLJxqmI3/gH2HWyXjoMbA7LHgxjqFNGmxg6uZjKhruMJDdRpmJJ4TEpnbHfs2laIbMg0kJoCUX/YCsbxuNOElJXBCOTIKMI1iwhSjMHjogGEv/3jno0uy670rO7V7mr3/Zq5s+c853nu/d5nrvaj8+OeTVUhSdKh/K25LkCSdHQwMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU1GEhhJNiXZk+TBabafl+TpJPd3j98c2LY2yY4kO5NcPYp6JEmjl1F8DyPJO4AfAzdX1eun2H4e8GtV9d5J7UuA7wLnA+PAvcBlVbV96KIkSSM1kj2Mqvom8ORhDD0H2FlVu6rqOeBWYN0oapIkjdbSWXyttyf5NrCb/t7GQ8Aq4NGBPuPAW6canGQDsAHghBNOeMtrX/vaI1yuJC0sW7du/euqWnm442crML4FvKqqfpzkPcCdwBogU/Sd8hhZVW0ENgL0er0aGxs7UrVK0oKU5P8OM35WrpKqqh9V1Y+75f8JHJNkBf09itMHup5Gfw9EkjTPzEpgJHllknTL53Sv+wP6J7nXJDkjybHAemDzbNQkSZqZkRySSnILcB6wIsk4cC1wDEBVfR54H/DLSfYD+4D11b88a3+SK4EtwBJgU3duQ5I0z4zkstrZ5jkMSZq5JFurqne44/2mtySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqMpLASLIpyZ4kD06z/Z8leaB7/GmSNw1s+16SbUnuT+LfXZWkeWpUexhfAtYeZPsjwM9V1RuB3wI2Ttr+zqo6a5i/NStJOrKWjuJJquqbSV59kO1/OrB6D3DaKF5XkjR75uIcxhXAVwbWC/jjJFuTbJiDeiRJDUayh9EqyTvpB8Y/HGg+t6p2J3k5cFeSv6iqb04xdgOwAWD16tWzUq8k6YBZ28NI8kbgRmBdVf1gor2qdnc/9wB3AOdMNb6qNlZVr6p6K1eunI2SJUkDZiUwkqwGbgf+eVV9d6D9hCQ/NbEMXABMeaWVJGlujeSQVJJbgPOAFUnGgWuBYwCq6vPAbwInA/8pCcD+7oqoVwB3dG1LgT+oqq+OoiZJ0miN6iqpyw6x/UPAh6Zo3wW86aUjJEnzjd/0liQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUpOR/E3vJJuA9wJ7qur1U2wP8GngPcAzwAeq6lvdtsuB3+i6/nZV3TSKmqTF6M77HuOGLTvY/dQ+Tl2+jKsuPJOLz1616GqYT3UsJCMJDOBLwGeAm6fZ/m5gTfd4K/A54K1JTgKuBXpAAVuTbK6qH46oLmnRuPO+x7jm9m3se/4FAB57ah/X3L4NYNZ+Uc6HGuZTHQvNSA5JVdU3gScP0mUdcHP13QMsT3IKcCFwV1U92YXEXcDaUdQkLTY3bNnxN78gJ+x7/gVu2LJjUdUwn+pYaGbrHMYq4NGB9fGubbr2l0iyIclYkrG9e/cesUKlo9Xup/bNqH2h1jCf6lhoZiswMkVbHaT9pY1VG6uqV1W9lStXjrQ4aSE4dfmyGbUv1BrmUx0LzWwFxjhw+sD6acDug7RLmqGrLjyTZccseVHbsmOWcNWFZy6qGuZTHQvNbAXGZuD96Xsb8HRVPQ5sAS5IcmKSE4ELujZJM3Tx2av4xCVvYNXyZQRYtXwZn7jkDbN6knc+1DCf6lhoUjXlEaCZPUlyC3AesAJ4gv6VT8cAVNXnu8tqP0P/hPYzwAeraqwb+0vAv+me6uNV9cVDvV6v16uxsbGh65akxSTJ1qrqHe74kVxWW1WXHWJ7Ab8yzbZNwKZR1CFJOnL8prckqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJajKSwEiyNsmOJDuTXD3F9k8mub97fDfJUwPbXhjYtnkU9UiSRm/ov+mdZAnwWeB8YBy4N8nmqto+0aeqfnWg/78Ezh54in1VddawdUiSjqxR7GGcA+ysql1V9RxwK7DuIP0vA24ZwetKkmbRKAJjFfDowPp41/YSSV4FnAF8faD5+CRjSe5JcvF0L5JkQ9dvbO/evSMoW5I0E6MIjEzRVtP0XQ/cVlUvDLStrqoe8E+BTyX5makGVtXGqupVVW/lypXDVSxJmrFRBMY4cPrA+mnA7mn6rmfS4aiq2t393AV8gxef35AkzROjCIx7gTVJzkhyLP1QeMnVTknOBE4E/myg7cQkx3XLK4Bzge2Tx0qS5t7QV0lV1f4kVwJbgCXApqp6KMl1wFhVTYTHZcCtVTV4uOp1wBeS/IR+eF0/eHWVJGn+yIt/fx8der1ejY2NzXUZknRUSbK1O2d8WPymtySpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqMpLASLI2yY4kO5NcPcX2DyTZm+T+7vGhgW2XJ3m4e1w+inokSaO3dNgnSLIE+CxwPjAO3Jtkc1Vtn9T1D6vqykljTwKuBXpAAVu7sT8cti5J0miNYg/jHGBnVe2qqueAW4F1jWMvBO6qqie7kLgLWDuCmiRJIzaKwFgFPDqwPt61TfaPkzyQ5LYkp89wLEk2JBlLMrZ3794RlC1JmolRBEamaKtJ6/8NeHVVvRH4GnDTDMb2G6s2VlWvqnorV6487GIlSYdnFIExDpw+sH4asHuwQ1X9oKqe7VZ/F3hL61hJ0vwwisC4F1iT5IwkxwLrgc2DHZKcMrB6EfCdbnkLcEGSE5OcCFzQtUmS5pmhr5Kqqv1JrqT/i34JsKmqHkpyHTBWVZuBDye5CNgPPAl8oBv7ZJLfoh86ANdV1ZPD1iRJGr1UTXnKYF7r9Xo1NjY212VI0lElydaq6h3ueL/pLUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKajCQwkqxNsiPJziRXT7H9o0m2J3kgyd1JXjWw7YUk93ePzaOoR5I0ekuHfYIkS4DPAucD48C9STZX1faBbvcBvap6JskvA/8euLTbtq+qzhq2DknSkTWKPYxzgJ1VtauqngNuBdYNdqiqP6mqZ7rVe4DTRvC6kqRZNIrAWAU8OrA+3rVN5wrgKwPrxycZS3JPkounG5RkQ9dvbO/evcNVLEmasaEPSQGZoq2m7Jj8ItADfm6geXVV7U7yGuDrSbZV1V++5AmrNgIbAXq93pTPL0k6ckaxhzEOnD6wfhqwe3KnJO8CPgZcVFXPTrRX1e7u5y7gG8DZI6hJkjRiowiMe4E1Sc5IciywHnjR1U5Jzga+QD8s9gy0n5jkuG55BXAuMHiyXJI0Twx9SKqq9ie5EtgCLAE2VdVDSa4DxqpqM3AD8LeBLycB+H5VXQS8DvhCkp/QD6/rJ11dJUmaJ1J19J0O6PV6NTY2NtdlSNJRJcnWquod7ni/6S1JamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmgz9N70BkqwFPk3/b3rfWFXXT9p+HHAz8BbgB8ClVfW9bts1wBXAC8CHq2rLKGrSzNx532PcsGUHu5/ax6nLl3HVhWdy8dmr5rqsJrNZ+9E8T9Kwhg6MJEuAzwLnA+PAvUk2V9X2gW5XAD+sqr+bZD3wO8ClSf4+sB74WeBU4GtJ/l5VvTBsXWp3532Pcc3t29j3fH/aH3tqH9fcvg1g3v8ynM3aj+Z5kkZhFIekzgF2VtWuqnoOuBVYN6nPOuCmbvk24OeTpGu/taqerapHgJ3d82kW3bBlx9/8Epyw7/kXuGHLjjmqqN1s1n40z5M0CqMIjFXAowPr413blH2qaj/wNHBy41gAkmxIMpZkbO/evSMoWxN2P7VvRu3zyWzWfjTPkzQKowiMTNFWjX1axvYbqzZWVa+qeitXrpxhiTqYU5cvm1H7fDKbtR/N8ySNwigCYxw4fWD9NGD3dH2SLAX+DvBk41gdYVddeCbLjlnyorZlxyzhqgvPnKOK2s1m7UfzPEmjMIrAuBdYk+SMJMfSP4m9eVKfzcDl3fL7gK9XVXXt65Mcl+QMYA3wf0ZQk2bg4rNX8YlL3sCq5csIsGr5Mj5xyRuOihO5s1n70TxP0iik/3t7yCdJ3gN8iv5ltZuq6uNJrgPGqmpzkuOB3wfOpr9nsb6qdnVjPwb8ErAf+EhVfeVQr9fr9WpsbGzouiVpMUmytap6hz1+FIEx2wwMSZq5YQPDb3pLkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCZDBUaSk5LcleTh7ueJU/Q5K8mfJXkoyQNJLh3Y9qUkjyS5v3ucNUw9kqQjZ9g9jKuBu6tqDXB3tz7ZM8D7q+pngbXAp5IsH9h+VVWd1T3uH7IeSdIRMmxgrANu6pZvAi6e3KGqvltVD3fLu4E9wMohX1eSNMuGDYxXVNXjAN3Plx+sc5JzgGOBvxxo/nh3qOqTSY4bsh5J0hGy9FAdknwNeOUUmz42kxdKcgrw+8DlVfWTrvka4K/oh8hG4NeB66YZvwHYALB69eqZvLQkaQQOGRhV9a7ptiV5IskpVfV4Fwh7pun308D/AH6jqu4ZeO7Hu8Vnk3wR+LWD1LGRfqjQ6/XqUHVLkkZr2ENSm4HLu+XLgT+a3CHJscAdwM1V9eVJ207pfob++Y8Hh6xHknSEDBsY1wPnJ3kYOL9bJ0kvyY1dn38CvAP4wBSXz/7nJNuAbcAK4LeHrEeSdISk6ug7utPr9WpsbGyuy5Cko0qSrVXVO9zxftNbktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTYYKjCQnJbkrycPdzxOn6fdCkvu7x+aB9jOS/Hk3/g+THDtMPZKkI2fYPYyrgburag1wd7c+lX1VdVb3uGig/XeAT3bjfwhcMWQ9kqQjZNjAWAfc1C3fBFzcOjBJgH8E3HY44yVJs2vpkONfUVWPA1TV40lePk2/45OMAfuB66vqTuBk4Kmq2t/1GQdWTfdCSTYAG7rVZ5M8OGTtC8UK4K/nuoh5wrk4wLk4wLk44MxhBh8yMJJ8DXjlFJs+NoPXWV1Vu5O8Bvh6km3Aj6boV9M9QVVtBDZ2NY1VVW8Gr79gORcHOBcHOBcHOBcHdP9xP2yHDIyqetdBXvyJJKd0exenAHumeY7d3c9dSb4BnA38V2B5kqXdXsZpwO7DeA+SpFkw7DmMzcDl3fLlwB9N7pDkxCTHdcsrgHOB7VVVwJ8A7zvYeEnS/DBsYFwPnJ/kYeD8bp0kvSQ3dn1eB4wl+Tb9gLi+qrZ3234d+GiSnfTPafxe4+tuHLLuhcS5OMC5OMC5OMC5OGCouUj/P/qSJB2c3/SWJDUxMCRJTeZ9YCT5XpJt3W1Fxrq2pluSLDRJlie5LclfJPlOkrcvxrlIcubArWbuT/KjJB9ZjHMBkORXkzyU5MEktyQ5frHedifJv+rm4aEkH+naFsXnIsmmJHsGv6M23XtP339MsjPJA0ne3PIa8z4wOu/sbisycS116y1JFppPA1+tqtcCbwK+wyKci6raMXGrGeAtwDPAHSzCuUiyCvgw0Kuq1wNLgPUswtvuJHk98C+Ac+j/+3hvkjUsns/Fl4C1k9qme+/vBtZ0jw3A55peoarm9QP4HrBiUtsO4JRu+RRgx1zXOQvz8NPAI3QXKizmuZj0/i8A/vdinQv6d0d4FDiJ/veq/jtwIf1vNi/t+rwd2DLXtc7CXPwCcOPA+r8F/vVi+lwArwYeHFif8r0DXwAum6rfwR5Hwx5GAX+cZGt3exCYdEsSYLpbkiwkrwH2Al9Mcl+SG5OcwOKci0HrgVu65UU3F1X1GPAfgO8DjwNPA1uZwW13FpAHgXckOTnJy4D3AKezCD8XA6Z77xP/0ZjQ9Bk5GgLj3Kp6M/1dqF9J8o65LmiOLAXeDHyuqs4G/h8Ld9e6SXdc/iLgy3Ndy1zpjkmvA84ATgVOoP9vZbIFf/18VX2H/qG4u4CvAt+mf/86vVSmaDvkZ2TeB0YduK3IHvrHqc8BnuhuRcLBbkmywIwD41X15936bfQDZDHOxYR3A9+qqie69cU4F+8CHqmqvVX1PHA78A/obrvT9Vk0t92pqt+rqjdX1TuAJ4GHWZyfiwnTvfdx+ntfE5o+I/M6MJKckOSnJpbpH69+kIZbkiw0VfVXwKNJJu42+fPAdhbhXAy4jAOHo2BxzsX3gbcleVn3JwMmPheL8rY7E3fMTrIauIT+52Mxfi4mTPfeNwPv766Wehvw9MShq4OZ19/07u5ue0e3uhT4g6r6eJKTgf8CrKb/D+YXqurJOSpz1iQ5C7gROBbYBXyQfugvxrl4Gf1jsK+pqqe7tsX6ufh3wKX0D7/cB3yI/vHoW+mfDL8P+MWqenbOipwlSf4X/dsMPQ98tKruXiyfiyS3AOfRv537E8C1wJ1M8d67/1x8hv5VVc8AH6yqQ97Jdl4HhiRp/pjXh6QkSfOHgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmvx/hYgEHyerDbYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.xlim(50, 100)\n",
    "plt.ylim(-0.5, 1.5)\n",
    "\n",
    "X = np.array([[80,1],[85,1],[74,0],[63,0],[90,1],[70,0]])\n",
    "plt.scatter(X[:,0], X[:,1])\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "直观上去看，上面有一个人考了 80 分通过了，那么张三考 82 分肯定也通过了，而另外有一个人考 74 分没过，那么李四考 76 分就很悬，到底通过没有，估计要看他运气了。回到我们的问题，如何让机器来预测他们是否通过呢？这个散点明显不是一条直线，如果用线性回归肯定得不到满意的结果，实际上，我们希望找到下面这样的一个函数：\n",
    "\n",
    "$$\n",
    "y = \n",
    "\\begin{cases}\n",
    "1,\\quad x \\geq k \\\\ \n",
    "0,\\quad x < k \n",
    "\\end{cases}\n",
    "$$\n",
    "\n",
    "当分数大于某个值时，考试通过，当分数低于某个值时，考试未通过。分类问题的关键就是确定这个中间值，这个中间值被称为**决策边界**（decision boundary），一元分类问题的决策边界就是一个点。\n",
    "\n",
    "实际上，上面这个函数又叫做**单位阶跃函数**（unit step function），它是数学、物理以及信号处理等其他学科中的一个重要函数。它最初由英国的物理学家奥利弗·亥维赛（Oliver Heaviside）提出，所以又叫 **Heaviside 函数**。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}