【动手学Paddle2系列】模型训练的N种姿势.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# 【动手学Paddle2.0系列】模型训练的N种姿势\n",
    "\n",
    "\n",
    "最开始接触深度学习的时候，我的码力几乎等于0，所以在最初的时候，几乎都是使用封装好的高层API进行训练，自由度很低。\n",
    "\n",
    "\n",
    "想通过这个教程，对paddle2.0中的各种开启训练的方式进行一个总结。让我们开始愉快的学（ban）习（zhuan）吧！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 1 Paddle2.0 主要思想\n",
    "\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/785d759b1761429c8d433c1ed6219a4e5e61f800846340069235bf793a4a8dcd)\n",
    "\n",
    "\n",
    "那么，用框架来类比，飞桨框架基础API对应方法一，飞桨框架高层API对应方法二。使用基础API，我们可以随心所欲的搭建自己的深度学习模型，不会受到任何限制；而使用方法二，我们可以很快的实现模型，达到自己想要的效果，缺点是少了一些自主性。但是，与制作披萨不同的是，飞桨框架可以做到真正的”鱼与熊掌”可以兼得，我们在飞桨框架中实现了API的高低融合，使我们的开发者既可以享受到基础API的强大，又可以兼顾高层API的快捷。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 1.1 数据加载"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/train-images-idx3-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/train-images-idx3-ubyte.gz \n",
      "Begin to download\n",
      "\n",
      "Download finished\n",
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/train-labels-idx1-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/train-labels-idx1-ubyte.gz \n",
      "Begin to download\n",
      "........\n",
      "Download finished\n",
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/t10k-images-idx3-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/t10k-images-idx3-ubyte.gz \n",
      "Begin to download\n",
      "\n",
      "Download finished\n",
      "Cache file /home/aistudio/.cache/paddle/dataset/mnist/t10k-labels-idx1-ubyte.gz not found, downloading https://dataset.bj.bcebos.com/mnist/t10k-labels-idx1-ubyte.gz \n",
      "Begin to download\n",
      "..\n",
      "Download finished\n"
     ]
    }
   ],
   "source": [
    "from paddle.vision.transforms import Compose, Normalize\r\n",
    "from paddle.vision.datasets import MNIST\r\n",
    "\r\n",
    "\r\n",
    "# 数据预处理，这里用到了随机调整亮度、对比度和饱和度\r\n",
    "transform = Compose([Normalize(mean=[127.5],\r\n",
    "                               std=[127.5],\r\n",
    "                               data_format='CHW')])\r\n",
    "\r\n",
    "\r\n",
    "# 数据加载，在训练集上应用数据预处理的操作\r\n",
    "train_dataset = MNIST(mode='train', transform=transform)\r\n",
    "test_dataset = MNIST(mode='test')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 1.2 加载模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import paddle\r\n",
    "from paddle.vision.models import LeNet\r\n",
    "import paddle.nn as nn\r\n",
    "\r\n",
    "mnist = LeNet()\r\n",
    "# # 模型封装，用Model类封装\r\n",
    "# model = paddle.Model(mnist)\r\n",
    "\r\n",
    "# # 模型配置：为模型训练做准备，设置优化器，损失函数和精度计算方式\r\n",
    "# model.prepare(optimizer=paddle.optimizer.Adam(parameters=model.parameters()),\r\n",
    "#               loss=nn.CrossEntropyLoss(),\r\n",
    "#               metrics=paddle.metric.Accuracy())\r\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 2 模型训练的N种姿势"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 2.1 高层API训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 2.1.1 方式一"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The loss value printed in the log is the current step, and the metric is the average value of previous step.\n",
      "Epoch 1/5\n",
      "step 938/938 [==============================] - loss: 0.0308 - acc: 0.9332 - 10ms/step         \n",
      "Epoch 2/5\n",
      "step 938/938 [==============================] - loss: 0.0267 - acc: 0.9752 - 9ms/step         \n",
      "Epoch 3/5\n",
      "step 938/938 [==============================] - loss: 0.0065 - acc: 0.9812 - 9ms/step         \n",
      "Epoch 4/5\n",
      "step 938/938 [==============================] - loss: 0.0057 - acc: 0.9841 - 10ms/step         \n",
      "Epoch 5/5\n",
      "step 938/938 [==============================] - loss: 0.0144 - acc: 0.9854 - 9ms/step         \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:DataLoader reader thread raised an exception.\n",
      "Exception in thread Thread-15:\n",
      "Traceback (most recent call last):\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 926, in _bootstrap_inner\n",
      "    self.run()\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 870, in run\n",
      "    self._target(*self._args, **self._kwargs)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 338, in _thread_loop\n",
      "    six.reraise(*sys.exc_info())\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py\", line 703, in reraise\n",
      "    raise value\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 308, in _thread_loop\n",
      "    batch = self._dataset_fetcher.fetch(indices)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/fetcher.py\", line 65, in fetch\n",
      "    data = self.collate_fn(data)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 95, in default_collate_fn\n",
      "    raise RuntimeError(\"Unknown data type {}\".format(type(slot[0])))\n",
      "RuntimeError: Unknown data type <class 'PIL.Image.Image'>\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eval begin...\n",
      "The loss value printed in the log is the current batch, and the metric is the average value of previous step.\n"
     ]
    },
    {
     "ename": "SystemError",
     "evalue": "(Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mSystemError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-fc2828e0be70>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      7\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;31m# 模型评估，\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 9\u001b[0;31m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mevaluate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_dataset\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/hapi/model.py\u001b[0m in \u001b[0;36mevaluate\u001b[0;34m(self, eval_data, batch_size, log_freq, verbose, num_workers, callbacks)\u001b[0m\n\u001b[1;32m   1592\u001b[0m                        'metrics': self._metrics_name()})\n\u001b[1;32m   1593\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1594\u001b[0;31m         \u001b[0mlogs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_run_one_epoch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0meval_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcbks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'eval'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1595\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1596\u001b[0m         \u001b[0mcbks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_end\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'eval'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlogs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/hapi/model.py\u001b[0m in \u001b[0;36m_run_one_epoch\u001b[0;34m(self, data_loader, callbacks, mode, logs)\u001b[0m\n\u001b[1;32m   1774\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_run_one_epoch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlogs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1775\u001b[0m         \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1776\u001b[0;31m         \u001b[0;32mfor\u001b[0m \u001b[0mstep\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_loader\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1777\u001b[0m             \u001b[0;31m# data might come from different types of data_loader and have\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1778\u001b[0m             \u001b[0;31m# different format, as following:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\u001b[0m in \u001b[0;36m__next__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    354\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    355\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0min_dygraph_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 356\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_next_var_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    357\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    358\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_return_list\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mSystemError\u001b[0m: (Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n"
     ]
    }
   ],
   "source": [
    "\r\n",
    "# 模型训练，\r\n",
    "model.fit(train_dataset,\r\n",
    "          epochs=5,\r\n",
    "          batch_size=64,\r\n",
    "          verbose=1)\r\n",
    "\r\n",
    "# 模型评估，\r\n",
    "# model.evaluate(test_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 2.1.2 方式二"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# 构建训练集数据加载器\r\n",
    "train_loader = paddle.io.DataLoader(train_dataset, batch_size=64, shuffle=True)\r\n",
    "\r\n",
    "# 构建测试集数据加载器\r\n",
    "test_loader = paddle.io.DataLoader(test_dataset, batch_size=64, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The loss value printed in the log is the current step, and the metric is the average value of previous step.\n",
      "Epoch 1/5\n",
      "step 938/938 [==============================] - loss: 0.3847 - acc: 0.9869 - 9ms/step         \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:DataLoader reader thread raised an exception.\n",
      "Exception in thread Thread-17:\n",
      "Traceback (most recent call last):\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 926, in _bootstrap_inner\n",
      "    self.run()\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 870, in run\n",
      "    self._target(*self._args, **self._kwargs)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 338, in _thread_loop\n",
      "    six.reraise(*sys.exc_info())\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py\", line 703, in reraise\n",
      "    raise value\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 308, in _thread_loop\n",
      "    batch = self._dataset_fetcher.fetch(indices)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/fetcher.py\", line 65, in fetch\n",
      "    data = self.collate_fn(data)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 95, in default_collate_fn\n",
      "    raise RuntimeError(\"Unknown data type {}\".format(type(slot[0])))\n",
      "RuntimeError: Unknown data type <class 'PIL.Image.Image'>\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eval begin...\n",
      "The loss value printed in the log is the current batch, and the metric is the average value of previous step.\n"
     ]
    },
    {
     "ename": "SystemError",
     "evalue": "(Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mSystemError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-39653fd0f7aa>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      2\u001b[0m         \u001b[0meval_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtest_loader\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      3\u001b[0m         \u001b[0mepochs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m5\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m         \u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      5\u001b[0m         )\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/hapi/model.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, train_data, eval_data, batch_size, epochs, eval_freq, log_freq, save_dir, save_freq, verbose, drop_last, shuffle, num_workers, callbacks)\u001b[0m\n\u001b[1;32m   1501\u001b[0m                 })\n\u001b[1;32m   1502\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1503\u001b[0;31m                 \u001b[0meval_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_run_one_epoch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0meval_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcbks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'eval'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1504\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1505\u001b[0m                 \u001b[0mcbks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_end\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'eval'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0meval_logs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/hapi/model.py\u001b[0m in \u001b[0;36m_run_one_epoch\u001b[0;34m(self, data_loader, callbacks, mode, logs)\u001b[0m\n\u001b[1;32m   1774\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m_run_one_epoch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata_loader\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcallbacks\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlogs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1775\u001b[0m         \u001b[0moutputs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1776\u001b[0;31m         \u001b[0;32mfor\u001b[0m \u001b[0mstep\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata_loader\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1777\u001b[0m             \u001b[0;31m# data might come from different types of data_loader and have\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1778\u001b[0m             \u001b[0;31m# different format, as following:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\u001b[0m in \u001b[0;36m__next__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    354\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    355\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0min_dygraph_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 356\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_next_var_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    357\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    358\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_return_list\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mSystemError\u001b[0m: (Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n"
     ]
    }
   ],
   "source": [
    "model.fit(train_loader,\r\n",
    "        eval_data = test_loader,\r\n",
    "        epochs=5,\r\n",
    "        verbose=1,\r\n",
    "        )\r\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model.evaluate(test_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 2.1.3 方式三"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "([array([0.00321424], dtype=float32)], [1.0])\n",
      "([array([0.02877501], dtype=float32)], [0.984375])\n",
      "([array([0.03653406], dtype=float32)], [0.984375])\n",
      "([array([0.00271822], dtype=float32)], [1.0])\n",
      "([array([0.02068332], dtype=float32)], [1.0])\n",
      "([array([0.01921546], dtype=float32)], [1.0])\n",
      "([array([0.08911473], dtype=float32)], [0.96875])\n",
      "([array([0.00550561], dtype=float32)], [1.0])\n",
      "([array([0.03885828], dtype=float32)], [0.984375])\n",
      "([array([0.0128423], dtype=float32)], [1.0])\n",
      "([array([0.00481672], dtype=float32)], [1.0])\n",
      "([array([0.00461083], dtype=float32)], [1.0])\n",
      "([array([0.04832631], dtype=float32)], [0.96875])\n",
      "([array([0.00081611], dtype=float32)], [1.0])\n",
      "([array([0.0527472], dtype=float32)], [0.984375])\n",
      "([array([0.02105747], dtype=float32)], [0.984375])\n",
      "([array([0.03141265], dtype=float32)], [0.984375])\n",
      "([array([0.06591309], dtype=float32)], [0.953125])\n",
      "([array([0.00373719], dtype=float32)], [1.0])\n",
      "([array([0.01238431], dtype=float32)], [1.0])\n",
      "([array([0.069947], dtype=float32)], [0.96875])\n",
      "([array([0.03446893], dtype=float32)], [0.984375])\n",
      "([array([0.00524848], dtype=float32)], [1.0])\n",
      "([array([0.02650649], dtype=float32)], [0.984375])\n",
      "([array([0.02402489], dtype=float32)], [0.984375])\n",
      "([array([0.03221917], dtype=float32)], [0.984375])\n",
      "([array([0.03255298], dtype=float32)], [0.984375])\n",
      "([array([0.00028397], dtype=float32)], [1.0])\n",
      "([array([0.0117424], dtype=float32)], [0.984375])\n",
      "([array([0.02055109], dtype=float32)], [0.984375])\n",
      "([array([0.01991917], dtype=float32)], [0.984375])\n",
      "([array([0.09122707], dtype=float32)], [0.953125])\n",
      "([array([0.02635681], dtype=float32)], [0.984375])\n",
      "([array([0.04663792], dtype=float32)], [0.96875])\n",
      "([array([0.00369589], dtype=float32)], [1.0])\n",
      "([array([0.07202551], dtype=float32)], [0.953125])\n",
      "([array([0.02525014], dtype=float32)], [0.984375])\n",
      "([array([0.02041636], dtype=float32)], [1.0])\n",
      "([array([0.00087871], dtype=float32)], [1.0])\n",
      "([array([0.01023444], dtype=float32)], [1.0])\n",
      "([array([0.01818169], dtype=float32)], [0.984375])\n",
      "([array([0.00543391], dtype=float32)], [1.0])\n",
      "([array([0.00069047], dtype=float32)], [1.0])\n",
      "([array([0.11813676], dtype=float32)], [0.96875])\n",
      "([array([0.00141953], dtype=float32)], [1.0])\n",
      "([array([0.000812], dtype=float32)], [1.0])\n",
      "([array([0.00220475], dtype=float32)], [1.0])\n",
      "([array([0.01065434], dtype=float32)], [1.0])\n",
      "([array([0.01594276], dtype=float32)], [1.0])\n",
      "([array([0.01709219], dtype=float32)], [0.984375])\n",
      "([array([0.06714898], dtype=float32)], [0.96875])\n",
      "([array([0.02291741], dtype=float32)], [0.984375])\n",
      "([array([0.00228298], dtype=float32)], [1.0])\n",
      "([array([0.05858309], dtype=float32)], [0.984375])\n",
      "([array([0.00179594], dtype=float32)], [1.0])\n",
      "([array([0.06001876], dtype=float32)], [0.984375])\n",
      "([array([0.02336162], dtype=float32)], [0.984375])\n",
      "([array([0.01184798], dtype=float32)], [1.0])\n",
      "([array([0.02464829], dtype=float32)], [0.984375])\n",
      "([array([0.01763016], dtype=float32)], [1.0])\n",
      "([array([0.05141861], dtype=float32)], [0.984375])\n",
      "([array([0.00206836], dtype=float32)], [1.0])\n",
      "([array([0.11337481], dtype=float32)], [0.984375])\n",
      "([array([0.02301983], dtype=float32)], [0.984375])\n",
      "([array([0.00245143], dtype=float32)], [1.0])\n",
      "([array([0.13529927], dtype=float32)], [0.96875])\n",
      "([array([0.00259266], dtype=float32)], [1.0])\n",
      "([array([0.01659089], dtype=float32)], [1.0])\n",
      "([array([0.05693113], dtype=float32)], [0.96875])\n",
      "([array([0.0150041], dtype=float32)], [1.0])\n",
      "([array([0.01858272], dtype=float32)], [0.984375])\n",
      "([array([0.02312807], dtype=float32)], [1.0])\n",
      "([array([0.04985583], dtype=float32)], [0.984375])\n",
      "([array([0.00987223], dtype=float32)], [1.0])\n",
      "([array([0.00403416], dtype=float32)], [1.0])\n",
      "([array([0.00959015], dtype=float32)], [1.0])\n",
      "([array([0.07591782], dtype=float32)], [0.96875])\n",
      "([array([0.07786051], dtype=float32)], [0.96875])\n",
      "([array([0.00717475], dtype=float32)], [1.0])\n",
      "([array([0.02681538], dtype=float32)], [0.984375])\n",
      "([array([0.02761941], dtype=float32)], [0.984375])\n",
      "([array([0.01321609], dtype=float32)], [1.0])\n",
      "([array([0.00646588], dtype=float32)], [1.0])\n",
      "([array([0.00098096], dtype=float32)], [1.0])\n",
      "([array([0.00439962], dtype=float32)], [1.0])\n",
      "([array([0.01721995], dtype=float32)], [0.984375])\n",
      "([array([0.11859447], dtype=float32)], [0.984375])\n",
      "([array([0.02891131], dtype=float32)], [0.984375])\n",
      "([array([0.01946582], dtype=float32)], [0.984375])\n",
      "([array([0.00335281], dtype=float32)], [1.0])\n",
      "([array([0.0279369], dtype=float32)], [0.984375])\n",
      "([array([0.00933385], dtype=float32)], [1.0])\n",
      "([array([0.02781609], dtype=float32)], [0.984375])\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:DataLoader reader thread raised an exception.\n",
      "Exception in thread Thread-21:\n",
      "Traceback (most recent call last):\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 926, in _bootstrap_inner\n",
      "    self.run()\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/threading.py\", line 870, in run\n",
      "    self._target(*self._args, **self._kwargs)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 338, in _thread_loop\n",
      "    six.reraise(*sys.exc_info())\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/six.py\", line 703, in reraise\n",
      "    raise value\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 308, in _thread_loop\n",
      "    batch = self._dataset_fetcher.fetch(indices)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/fetcher.py\", line 65, in fetch\n",
      "    data = self.collate_fn(data)\n",
      "  File \"/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\", line 95, in default_collate_fn\n",
      "    raise RuntimeError(\"Unknown data type {}\".format(type(slot[0])))\n",
      "RuntimeError: Unknown data type <class 'PIL.Image.Image'>\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "([array([0.00776773], dtype=float32)], [1.0])\n"
     ]
    },
    {
     "ename": "SystemError",
     "evalue": "(Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mSystemError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-10-590afa885d01>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      9\u001b[0m             \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minfo\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m     \u001b[0;32mfor\u001b[0m \u001b[0mebatch_id\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0medata\u001b[0m \u001b[0;32min\u001b[0m \u001b[0menumerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtest_loader\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     12\u001b[0m         \u001b[0mx_data_v\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0medata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m         \u001b[0my_data_v\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0medata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dataloader/dataloader_iter.py\u001b[0m in \u001b[0;36m__next__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    354\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    355\u001b[0m             \u001b[0;32mif\u001b[0m \u001b[0min_dygraph_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 356\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reader\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread_next_var_list\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    357\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    358\u001b[0m                 \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_return_list\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mSystemError\u001b[0m: (Fatal) Blocking queue is killed because the data reader raises an exception.\n  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:154)\n  [Hint: If you need C++ stacktraces for debugging, please set `FLAGS_call_stack_level=2`.]\n"
     ]
    }
   ],
   "source": [
    "\r\n",
    "for epoch in range(5):\r\n",
    "    for batch_id, data in enumerate(train_loader()):\r\n",
    "        x_data = data[0]\r\n",
    "        y_data = data[1]\r\n",
    "        info = model.train_batch([x_data], [y_data])\r\n",
    "\r\n",
    "        if batch_id % 10 == 0:\r\n",
    "            print(info)\r\n",
    "             \r\n",
    "    for ebatch_id, edata in enumerate(test_loader()):\r\n",
    "        x_data_v = edata[0]\r\n",
    "        y_data_v = edata[1]\r\n",
    "        info = model.eval_batch([x_data_v], [y_data_v])\r\n",
    "\r\n",
    "        if ebatch_id % 10 == 0:\r\n",
    "            print(info)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "model.evaluate(test_dataset, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#### 2.1.4 方式四"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        [1]])\r"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-2484f1b206b2>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     10\u001b[0m         \u001b[0mx_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     11\u001b[0m         \u001b[0my_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 12\u001b[0;31m         \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my_data\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     13\u001b[0m     \u001b[0;31m#     predicts = mnist(x_data)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     14\u001b[0m     \u001b[0;31m#     loss = F.cross_entropy(predicts, y_data)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/varbase_patch_methods.py\u001b[0m in \u001b[0;36m__str__\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    263\u001b[0m         \"\"\"\n\u001b[1;32m    264\u001b[0m         \u001b[0;32mfrom\u001b[0m \u001b[0mpaddle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtensor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mto_string\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mto_string\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 265\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mto_string\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    266\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    267\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__deepcopy__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmemo\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/to_string.py\u001b[0m in \u001b[0;36mto_string\u001b[0;34m(var, prefix)\u001b[0m\n\u001b[1;32m    215\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    216\u001b[0m     data = _format_tensor(\n\u001b[0;32m--> 217\u001b[0;31m         np_var, sumary, indent=indent, max_width=max_width, signed=signed)\n\u001b[0m\u001b[1;32m    218\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    219\u001b[0m     return _template.format(\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/to_string.py\u001b[0m in \u001b[0;36m_format_tensor\u001b[0;34m(var, sumary, indent, max_width, signed)\u001b[0m\n\u001b[1;32m    183\u001b[0m             vars = [\n\u001b[1;32m    184\u001b[0m                 \u001b[0m_format_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msumary\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindent\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_width\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msigned\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 185\u001b[0;31m                 \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    186\u001b[0m             ]\n\u001b[1;32m    187\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/tensor/to_string.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    183\u001b[0m             vars = [\n\u001b[1;32m    184\u001b[0m                 \u001b[0m_format_tensor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msumary\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mindent\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmax_width\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msigned\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 185\u001b[0;31m                 \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mvar\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    186\u001b[0m             ]\n\u001b[1;32m    187\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "import paddle.nn.functional as F\r\n",
    "\r\n",
    "\r\n",
    "mnist.train()\r\n",
    "epochs = 5\r\n",
    "optim = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())\r\n",
    "# 用Adam作为优化函数\r\n",
    "for epoch in range(epochs):\r\n",
    "    for batch_id, data in enumerate(train_loader()):\r\n",
    "        x_data = data[0]\r\n",
    "        y_data = data[1]\r\n",
    "        print(y_data)\r\n",
    "    #     predicts = mnist(x_data)\r\n",
    "    #     loss = F.cross_entropy(predicts, y_data)\r\n",
    "    #     # 计算损失\r\n",
    "    #     acc = paddle.metric.accuracy(predicts, y_data, k=2)\r\n",
    "    #     loss.backward()\r\n",
    "    #     if batch_id % 5 == 0:\r\n",
    "    #         print(\"epoch: {}, batch_id: {}, loss is: {}, acc is: {}\".format(epoch, batch_id, loss.numpy(), acc.numpy()))\r\n",
    "    #     optim.step()\r\n",
    "    #     optim.clear_grad()\r\n",
    "    # for batch_id, data in enumerate(test_loader()):\r\n",
    "    #     x_data = data[0]\r\n",
    "    #     y_data = data[1]\r\n",
    "        \r\n",
    "    #     predicts = mnist(x_data)\r\n",
    "    #     loss = F.cross_entropy(predicts, y_data)\r\n",
    "    #     # 计算损失\r\n",
    "    #     acc = paddle.metric.accuracy(predicts, y_data, k=2)\r\n",
    "    #     if batch_id % 5 == 0:\r\n",
    "    #         print(\"epoch: {}, batch_id: {}, loss is: {}, acc is: {}\".format(epoch, batch_id, loss.numpy(), acc.numpy()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## 总结\n",
    "\n",
    "\n",
    "在上面的四种训练方式中，实际上每种方式都有其优缺点，完全使用高层API能够方便的开启训练。对于完全的新手用户，推荐使用方式一、方式二。对于已经熟悉Paddle2.0的用户推荐使用方式四。方式四可以更加灵活，并且能够灵活地加入混合精度训练，训练方式更加自由。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### 一点小小的宣传&下期预告\n",
    "\n",
    "\n",
    "我会在下一个项目中介绍在Paddle2.0中使用混合精度训练，欢迎大家关注哦。\n",
    "\n",
    "\n",
    "我目前在上海，感兴趣的领域包括模型压缩、小目标检测、嵌入式，欢迎交流关注。[来AI Studio互粉吧~等你哦~ ](https://aistudio.baidu.com/aistudio/personalcenter/thirdview/228777)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PaddlePaddle 2.0.0b0 (Python 3.5)",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}