main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A 股预测\n",
    "<br>\n",
    "<hr>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1.实验介绍"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 1.1 实验背景\n",
    "时间序列分析在金融、证券领域的应用非常广泛，尤其是对股票价格的预测。我们对数据进行预处理，接着使用数据分析方法，建立基础特征，进一步构建线性回归模型，且基于新数据验证模型效果。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 实验要求\n",
    "输入某股票前 14 个交易日的收盘价，预测下一个交易日的收盘价。\n",
    "\n",
    "实验指标为平均绝对百分比误差（ `MAPE` ）和平均绝对误差（ `MAE` ）。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 实验环境 \n",
    "可以使用基于 `Python` 的 `Pandas` 、 `Numpy` 、`Scikit-learn` 等库进行相关特征处理，使用 `Keras`、`Tensorflow`、`Pytorch` 等框架建立深度学习模型，使用过程中请注意 `Python` 包（库）的版本。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 注意事项\n",
    "- 使用平台的注意事项\n",
    "\n",
    "该平台的 `Notebook` 在 `CPU` 上运行，故尽量不要尝试在 `Notebook` 上做希望让 `GPU` 做的工作。\n",
    "\n",
    "- 训练模型的注意事项\n",
    "\n",
    "如果想要线下训练模型，请保证线下的环境与该平台一致，否则可能无法在该平台运行，可以在该平台的 `terminal` 输入```pip list```查看对应包版本。\n",
    "\n",
    "- 该作业的注意事项\n",
    "\n",
    "该作业目的在于加深对空间和时序模型的理解和运用，理论上作品的预测相关指标不应低于基本模型。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 参考资料\n",
    "- 相关框架的文档\n",
    "\n",
    "scikit-learn: https://scikit-learn.org/stable/\n",
    "\n",
    "tensorflow: https://tensorflow.google.cn/tutorials?hl=zh_cn\n",
    "\n",
    "pytorch: https://pytorch.org/tutorials/\n",
    "\n",
    "- 框架的学习教程\n",
    "\n",
    "《动手学深度学习》(Pytorch版): https://tangshusen.me/Dive-into-DL-PyTorch/\n",
    "\n",
    "《深度学习框架PyTorch：入门与实战》: https://github.com/chenyuntc/pytorch-book"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.实验内容"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.1 数据集\n",
    "\n",
    "数据集由网上的相关平台获取，训练集给出了五十几支股票的情况。数据以 `npy` 格式给出，名称为`train_data.npy` 。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 首先 import 一些主要的包\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "\n",
    "# 画图使用\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 简单读出一个股票\n",
    "\n",
    "# 获取文件名\n",
    "file_name = 'train_data.npy'\n",
    "\n",
    "# 读取数组\n",
    "data = np.load(file_name)\n",
    "\n",
    "# 简单展示信息\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来可以对其进行绘制，这样可以具体感受到股价的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 新建一个图像\n",
    "plt.figure(figsize=(20,10))\n",
    "\n",
    "# 绘画该股票不同的时间段的图像\n",
    "plt.plot(data,c='blue')\n",
    "\n",
    "# 展示图像\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到波动还是比较大的，且数值较大处和较小处相比差距存在，为了深度模型更好的工作，我们使用 `MinMaxScaler` 进行归一化。当然，用户也可以自行选择其他的预处理方式。\n",
    "\n",
    "这里我们选用 `sklearn` 的 `Scaler` ，如果有兴趣，也可以使用 `torchvision` 或者自己实现相关内容。\n",
    "\n",
    "当然， `MinMaxScaler` 并不是唯一的选择，甚至可能并不是正确的选择，这里只是用作示范，请自行学习相关内容并使用。\n",
    "\n",
    "但注意，如果 `scaler` 运算后的结果小于或等于 $0$ 可能带来严重后果，因为指标之一是 `MAPE` 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# 这个 [0, 300] 是手动的预设值，可以自己更改\n",
    "scaler = MinMaxScaler().fit(np.array([0, 300]).reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 数据处理\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先需要生成题目所需的训练集合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成题目所需的训练集合\n",
    "def generate_data(data):\n",
    "    \n",
    "    # 记录 data 的长度\n",
    "    n = data.shape[0]\n",
    "    \n",
    "    # 目标是生成可直接用于训练和测试的 x 和 y\n",
    "    x = []\n",
    "    y = []\n",
    "        \n",
    "    # 建立 (14 -> 1) 的 x 和 y\n",
    "    for i in range(15, n):\n",
    "        x.append(data[i-15:i-1])\n",
    "        y.append(data[i])\n",
    "            \n",
    "    # 转换为 numpy 数组\n",
    "    x = np.array(x)\n",
    "    y = np.array(y)\n",
    "        \n",
    "    return x,y\n",
    "\n",
    "x,y = generate_data(data)\n",
    "print('x.shape : ', x.shape)\n",
    "print('y.shape : ', y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后对数据集合进行分割，其中训练集用于训练，校验集用于检验模型训练情况，测试集合用于测试模型效果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 生成 train valid test 集合，以供训练所需\n",
    "def generate_training_data(x, y):\n",
    "    # 样本总数\n",
    "    num_samples = x.shape[0]\n",
    "    # 测试集大小\n",
    "    num_test = round(num_samples * 0.2)\n",
    "    # 训练集大小\n",
    "    num_train = round(num_samples * 0.7)\n",
    "    # 校验集大小\n",
    "    num_val = num_samples - num_test - num_train\n",
    "    \n",
    "    # 训练集拥有从 0 起长度为 num_train 的样本\n",
    "    x_train, y_train = x[:num_train], y[:num_train]\n",
    "    # 校验集拥有从 num_train 起长度为 num_val 的样本\n",
    "    x_val, y_val = (\n",
    "        x[num_train: num_train + num_val],\n",
    "        y[num_train: num_train + num_val],\n",
    "    )\n",
    "    # 测试集拥有尾部 num_test 个样本\n",
    "    x_test, y_test = x[-num_test:], y[-num_test:]\n",
    "    \n",
    "    # 返回这些集合\n",
    "    return x_train, y_train, x_val, y_val, x_test, y_test\n",
    "\n",
    "x_train, y_train, x_val, y_val, x_test, y_test = generate_training_data(x, y)\n",
    "print('x_train.shape : ', x_train.shape)\n",
    "print('y_train.shape : ', y_train.shape)\n",
    "print('x_val.shape : ', x_val.shape)\n",
    "print('y_val.shape : ', y_val.shape)\n",
    "print('x_test.shape : ', x_test.shape)\n",
    "print('y_test.shape : ', y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 建立一个简单的模型\n",
    "\n",
    "- 选用一种框架，告诉其创建模型的常用方式以及常用的接口\n",
    "- 建立一个简单模型并进行训练保存\n",
    "- 分析模型训练过程以及模型概况\n",
    "- 加载模型并对模型进行评估\n",
    "- **加载模型并预测输入数据的结果**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.1 处理数据\n",
    "\n",
    "该实验示范使用 `Pytorch` 完成。也可以选用其他框架进行训练并预测结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载 pytorch\n",
    "import torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "处理数据，并将其转化为 `Pytorch` 的形式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取数据中的 x, y\n",
    "x,y = generate_data(data)\n",
    "\n",
    "# 将 x,y 转换乘 tensor ， Pytorch 模型默认的类型是 float32\n",
    "x = torch.tensor(x)\n",
    "y = torch.tensor(y)\n",
    "\n",
    "print(x.shape,y.shape)\n",
    "\n",
    "# 将 y 转化形状\n",
    "y = y.view(y.shape[0],1)\n",
    "\n",
    "print(x.shape,y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对 x, y 进行 minmaxscale\n",
    "x_scaled = scaler.transform(x.reshape(-1,1)).reshape(-1,14)\n",
    "y_scaled = scaler.transform(y)\n",
    "\n",
    "x_scaled = torch.tensor(x_scaled, dtype=torch.float32)\n",
    "y_scaled = torch.tensor(y_scaled, dtype=torch.float32)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 处理出训练集，校验集和测试集\n",
    "x_train, y_train, x_val, y_val, x_test, y_test = generate_training_data(x_scaled, y_scaled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了方便使用 `DataLoader` ，我们需要自定义一个 `Dataset` ，自定义的 `Dataset` 只需要继承后实现下面三个函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 建立一个自定 Dataset\n",
    "class MyDataset(torch.utils.data.Dataset):\n",
    "    def __init__(self, x, y):\n",
    "        self.x = x\n",
    "        self.y = y\n",
    " \n",
    "    def __getitem__(self, item):\n",
    "        return self.x[item], self.y[item]\n",
    " \n",
    "    def __len__(self):\n",
    "        return len(self.x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 建立训练数据集、校验数据集和测试数据集\n",
    "train_data = MyDataset(x_train,y_train)\n",
    "valid_data = MyDataset(x_val,y_val)\n",
    "test_data = MyDataset(x_test,y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 规定批次的大小\n",
    "batch_size = 512\n",
    "\n",
    "# 创建对应的 DataLoader\n",
    "train_iter = torch.utils.data.DataLoader(train_data, batch_size=batch_size, shuffle=True)\n",
    "\n",
    "# 校验集和测试集的 shuffle 是没有必要的，因为每次都会全部跑一遍\n",
    "valid_iter = torch.utils.data.DataLoader(valid_data, batch_size=batch_size, shuffle=False)\n",
    "test_iter = torch.utils.data.DataLoader(test_data, batch_size=batch_size, shuffle=False)\n",
    "for i, read_data in enumerate(test_iter):\n",
    "    # i表示第几个batch， data表示该batch对应的数据，包含data和对应的labels\n",
    "    print(\"第 {} 个Batch \\n{}\".format(i, read_data))\n",
    "    break\n",
    "# 表示输出数据\n",
    "print(read_data[0].shape, read_data[0])\n",
    "# 表示输出标签\n",
    "print(read_data[1].shape, read_data[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.2 建立模型\n",
    "\n",
    "下面展示如何建立模型， `Pytorch` 的建立模型较为简单，只需要完成 `forward` ，即前向传播函数即可进行训练。这里展示建立一个简单的线性模型。参数 `Pytorch` 会自动初始化，具体请查看官方文档。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 输入的数量是前 14 个交易日的收盘价\n",
    "num_inputs = 14\n",
    "# 输出是下一个交易日的收盘价\n",
    "num_outputs = 1\n",
    "\n",
    "# 建立一个简单的线性模型\n",
    "class LinearNet(torch.nn.Module):\n",
    "    def __init__(self, num_inputs, num_outputs):\n",
    "        super(LinearNet, self).__init__()\n",
    "        # 一个线性层\n",
    "        self.linear = torch.nn.Linear(num_inputs, num_outputs)\n",
    "        \n",
    "    # 前向传播函数\n",
    "    def forward(self, x): # x shape: (batch, 14)\n",
    "        y = self.linear(x)\n",
    "        return y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面建立一个复杂但不太有效的 `LSTM` 模型，仅供理解 `Pytorch` 的运行方式而使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 隐藏层的个数\n",
    "num_hiddens = 128 \n",
    "# 建立一个稍微复杂的 LSTM 模型\n",
    "class LSTMNet(torch.nn.Module):\n",
    "    def __init__(self, num_hiddens, num_outputs):\n",
    "        super(LSTMNet, self).__init__()\n",
    "        self.hidden_size = num_hiddens\n",
    "        # RNN 层，这里的 batch_first 指定传入的是 (批大小，序列长度，序列每个位置的大小)\n",
    "        # 如果不指定其为 True，传入顺序应当是 (序列长度，批大小，序列每个位置的大小)\n",
    "        self.rnn = torch.nn.LSTM(input_size=num_inputs//24, hidden_size=num_hiddens,batch_first=True)\n",
    "        # 线性层\n",
    "        self.dense = torch.nn.Linear(self.hidden_size*24, 256)\n",
    "        self.dense2 = torch.nn.Linear(256,num_outputs)\n",
    "        # dropout 层，这里的参数指 dropout 的概率\n",
    "        self.dropout = torch.nn.Dropout(0.3)\n",
    "        self.dropout2 = torch.nn.Dropout(0.5)\n",
    "        # ReLU 层\n",
    "        self.relu = torch.nn.ReLU()\n",
    "    \n",
    "    # 前向传播函数，这是一个拼接的过程，使用大量变量是为了避免混淆，不做过多讲解\n",
    "    def forward(self, x): # x shape: (batch_size, 24, 307)\n",
    "        # LSTM 层会传出其参数，这里用 _ 将其舍弃\n",
    "        h, _ = self.rnn(x)\n",
    "        # LSTM 层会传出 (batch_size, 24, num_hiddens) 个参数，故需要 reshape 后丢入全连接层\n",
    "        h_r = h.reshape(-1,self.hidden_size*24)\n",
    "        h_d = self.dropout(h_r)\n",
    "        y = self.dense(h_d)\n",
    "        drop_y = self.dropout2(y)\n",
    "        a = self.relu(drop_y)\n",
    "        y2 = self.dense2(a)\n",
    "        return y2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，`Pytorch`建立一个模型较为清楚简单，具体使用可以参考文档。\n",
    "\n",
    "`Pytorch` 在使用 `GPU` 和 `CPU` 上的写法有所不同。在需要将保存在内存中的数据在 `GPU` 上运行时，需要主动将数据和模型拷贝到显存。\n",
    "\n",
    "为了简化差异，我们使用一个布尔值：`use_gpu` 来判断是否可用 `GPU` ，从而淡化差异。这样就不需要写两份代码。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 判断 gpu 是否可用\n",
    "use_gpu = torch.cuda.is_available()\n",
    "\n",
    "# 另一种写法是固定 device，每次调用数据都 to(device)即可\n",
    "# device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.3 评估函数建立\n",
    "\n",
    "这里给出了评估使用的函数，可以自测以获得信息。\n",
    "\n",
    "实验指标为均方根误差（ `RMSE` ）和平均绝对误差（ `MAE` ）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_mae(y_hat, y):\n",
    "    '''\n",
    "    :param y: 标准值\n",
    "    :param y_hat: 用户的预测值\n",
    "    :return: MAE 平均绝对误差 mean(|y*-y|)\n",
    "    '''\n",
    "    return torch.mean(torch.abs(y_hat - y))\n",
    "\n",
    "def compute_mape(y_hat, y):\n",
    "    '''\n",
    "    :param y: 标准值\n",
    "    :param y_hat: 用户的预测值\n",
    "    :return: MAPE 平均百分比误差 mean(|y*-y|/y)\n",
    "    '''\n",
    "    return torch.mean(torch.abs(y_hat - y)/y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面描绘评估函数，输入 `DataLoader` 和用户的模型，返回对应的 `MAE` 和 `RMSE` 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_accuracy(data_iter, model):\n",
    "    '''\n",
    "    :param data_iter: 输入的 DataLoader\n",
    "    :param model: 用户的模型\n",
    "    :return: 对应的 MAE 和 MAPE\n",
    "    '''\n",
    "    # 初始化参数\n",
    "    mae_sum, mape_sum, n = 0.0, 0.0, 0\n",
    "    \n",
    "    # 对每一个 data_iter 的每一个 x,y 进行计算\n",
    "    for x, y in data_iter:\n",
    "        \n",
    "        # 如果运行在 GPU 上，需要将内存中的 x 拷贝到显存中\n",
    "        if (use_gpu):\n",
    "            x=x.cuda()\n",
    "            \n",
    "        # 计算模型得出的 y_hat\n",
    "        y_hat = model(x)\n",
    "        \n",
    "        # 将 y_hat 逆归一化，这里逆归一化需要将数据转移到 CPU 才可以进行\n",
    "        y_hat_real = torch.from_numpy(scaler.inverse_transform(np.array(y_hat.detach().cpu()).reshape(-1,1)).reshape(y_hat.shape))\n",
    "        y_real = torch.from_numpy(scaler.inverse_transform(np.array(y.reshape(-1,1))).reshape(y.shape))\n",
    "        \n",
    "        # 计算对应的 MAE 和 RMSE 对应的和，并乘以 batch 大小\n",
    "        mae_sum += compute_mae(y_hat_real,y_real) * y.shape[0]\n",
    "        mape_sum += compute_mape(y_hat_real,y_real) * y.shape[0]\n",
    "        \n",
    "        # n 用于统计 DataLoader 中一共有多少数量\n",
    "        n += y.shape[0]\n",
    "        \n",
    "    # 返回时需要除以 batch 大小，得到平均值\n",
    "    return mae_sum / n, mape_sum / n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.4 模型训练\n",
    "\n",
    "首先我们需要选取优化器和损失函数。\n",
    "\n",
    "`Pytorch` 使用的优化器和损失函数可以选用其提供的，也可以自己写。一般来说， `Pytorch` 自带的具有更好的数值稳定性，这里给出参考。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用均方根误差\n",
    "loss = torch.nn.MSELoss()\n",
    "\n",
    "# 自定义的损失函数，可以直接调用\n",
    "def my_loss_func(y_hat, y):\n",
    "    return compute_mae(y_hat, y)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Pytorch` 的优化器需要提供 `model` 的 `parameters` ，故需要先定义网络。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用上面描述的线性网络\n",
    "model = LinearNet(num_inputs,num_outputs)\n",
    "\n",
    "# 使用 Adam 优化器， learning rate 调至 0.0001\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=1e-3)\n",
    "\n",
    "# 也可选用 SGD 或其他优化器\n",
    "# optimizer = torch.optim.SGD(model.parameters(), lr=1e-3, momentum=0.9, weight_decay=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面是训练函数。用于模型的直接训练。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_model(model, train_iter, valid_iter, loss, num_epochs,\n",
    "          params=None, optimizer=None):\n",
    "    \n",
    "    # 用于绘图用的信息\n",
    "    train_losses, valid_losses, train_maes, train_mapes, valid_maes, valid_mapes = [], [], [], [], [], []\n",
    "    \n",
    "    # 循环 num_epochs 次\n",
    "    for epoch in range(num_epochs):\n",
    "        # 初始化参数\n",
    "        train_l_sum, n = 0.0, 0\n",
    "        # 初始化时间\n",
    "        start = time.time()\n",
    "        # 模型改为训练状态，如果使用了 dropout, batchnorm 之类的层时，训练状态和评估状态的表现会有巨大差别\n",
    "        model.train()\n",
    "        \n",
    "        # 对训练数据集的每个 batch 执行\n",
    "        for x, y in train_iter:\n",
    "            \n",
    "            # 如果使用了 GPU 则拷贝进显存\n",
    "            if (use_gpu):\n",
    "                x,y = x.cuda(),y.cuda()\n",
    "            \n",
    "            # 计算 y_hat\n",
    "            y_hat = model(x)\n",
    "            \n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y).mean()\n",
    "\n",
    "            # 梯度清零\n",
    "            optimizer.zero_grad()\n",
    "            \n",
    "            # L1 正则化\n",
    "            # for param in params:\n",
    "            #     l += torch.sum(torch.abs(param))\n",
    "            \n",
    "            # L2 正则化可以在 optimizer 上加入 weight_decay 的方式加入\n",
    "\n",
    "            # 求好对应的梯度\n",
    "            l.backward()\n",
    "\n",
    "            # 执行一次反向传播\n",
    "            optimizer.step()\n",
    "\n",
    "            # 对 loss 求和（在下面打印出来）\n",
    "            train_l_sum += l.item() * y.shape[0]\n",
    "            \n",
    "            # 计数一共有多少个元素\n",
    "            n += y.shape[0]\n",
    "            \n",
    "        # 模型开启预测状态\n",
    "        model.eval()\n",
    "        \n",
    "        # 同样的，我们可以计算验证集上的 loss\n",
    "        valid_l_sum, valid_n = 0, 0\n",
    "        for x, y in valid_iter:\n",
    "            # 如果使用了 GPU 则拷贝进显存\n",
    "            if (use_gpu):\n",
    "                x,y = x.cuda(),y.cuda()\n",
    "            \n",
    "            # 计算 y_hat\n",
    "            y_hat = model(x)\n",
    "            \n",
    "            # 计算损失\n",
    "            l = loss(y_hat, y).mean()\n",
    "\n",
    "            # 对 loss 求和（在下面打印出来）\n",
    "            valid_l_sum += l.item() * y.shape[0]\n",
    "            \n",
    "            # 计数一共有多少个元素\n",
    "            valid_n += y.shape[0]\n",
    "        \n",
    "        # 对验证集合求指标\n",
    "        # 这里训练集其实可以在循环内高效地直接算出，这里为了代码的可读性牺牲了效率\n",
    "        train_mae, train_mape = evaluate_accuracy(train_iter, model)\n",
    "        valid_mae, valid_mape = evaluate_accuracy(valid_iter, model)\n",
    "        if (epoch+1) % 10 == 0:\n",
    "            print('epoch %d, train loss %.6f, valid loss %.6f, train mae %.6f, mape %.6f, valid mae %.6f,mape %.6f, time %.2f sec'\n",
    "              % (epoch + 1, train_l_sum / n, valid_l_sum / valid_n, train_mae, train_mape, valid_mae, valid_mape, time.time() - start))\n",
    "        \n",
    "        # 记录绘图有关的信息\n",
    "        train_losses.append(train_l_sum / n)\n",
    "        valid_losses.append(valid_l_sum / valid_n)\n",
    "        train_maes.append(train_mae)\n",
    "        train_mapes.append(train_mape)\n",
    "        valid_maes.append(valid_mae)\n",
    "        valid_mapes.append(valid_mape)\n",
    "        \n",
    "    # 返回一个训练好的模型和用于绘图的集合\n",
    "    return model, (train_losses, valid_losses, train_maes, train_mapes, valid_maes, valid_mapes)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面进行正式的模型训练，但是这里的模型训练在这里的 `Notebook(CPU)` 上要耗费较长的时间（单 `epoch` 约 $20$ 秒），建议使用离线任务中的 `GPU` 完成该步骤。将对应的数据保存到 `results` 文件夹中，在 `Notebook` 中读取并绘图。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练模型\n",
    "model, (train_losses, valid_losses, train_maes, train_mapes, valid_maes, valid_mapes) = train_model(model, train_iter, test_iter, loss, 200, model.parameters(), optimizer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以直接使用 `numpy` 保存并读取。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 为了方便储存与读取，建立成一个元组\n",
    "draw_data = (train_losses, valid_losses, train_maes, train_mapes, valid_maes, valid_mapes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 记录保存路径\n",
    "save_path = 'results/datas.npz'\n",
    "# 保存到硬盘\n",
    "np.savez(save_path, draw_data=draw_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 读取数据\n",
    "draw_data = np.load(save_path)['draw_data']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 提取其中的数据\n",
    "(train_losses, valid_losses, train_maes, train_mapes, valid_maes, valid_mapes) = draw_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.5 模型的评估\n",
    "\n",
    "首先绘制训练图像，以供观测，下面绘制 `loss` 图像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 新建一个图像\n",
    "plt.figure(figsize=(16,8))\n",
    "\n",
    "# 绘制 train_loss 曲线\n",
    "plt.plot(train_losses, label='train_loss')\n",
    "\n",
    "# 绘制 valid_loss 曲线\n",
    "plt.plot(valid_losses, label='valid_loss')\n",
    "\n",
    "# 展示带标签的图像\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面绘制 `MAE` 与 `RMSE` 在 `epoch` 中的变化。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 新建一个图像\n",
    "plt.figure(figsize=(16,8))\n",
    "\n",
    "# 绘画结点\n",
    "plt.plot(train_maes, c='blue', label='train_mae')\n",
    "\n",
    "plt.plot(train_mapes, c='red', label='train_rmse')\n",
    "\n",
    "plt.plot(valid_maes, c='green', label='valid_mae')\n",
    "\n",
    "plt.plot(valid_mapes, c='orange', label='valid_rmse')\n",
    "\n",
    "# 展示图像\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面绘制结点 $5$ 在校验集中与真实值的差距。这里仅考虑 `Notebook(CPU)` ， `GPU` 版本的需要稍加修改。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 新建一个图像\n",
    "plt.figure(figsize=(16,8))\n",
    "\n",
    "# 预测结果\n",
    "y_hat = model(x_test).detach()\n",
    "\n",
    "# 取前 300 个测试集\n",
    "num_for_draw = 300\n",
    "\n",
    "# 绘画某些结点第一天的情况\n",
    "plt.plot(scaler.inverse_transform(y_test[:num_for_draw].reshape(-1,1)).reshape(-1), c='blue', label='y_test')\n",
    "\n",
    "plt.plot(scaler.inverse_transform(y_hat[:num_for_draw].reshape(-1,1)).reshape(-1), c='red', label='y_hat')\n",
    "\n",
    "# 展示图像\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当在校验集上取得较为满意的结果的时候，可以来到测试集一试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获得测试集的数据\n",
    "test_mae, test_mape = evaluate_accuracy(test_iter, model)\n",
    "\n",
    "print('test mae, rmse: %.3f,%.3f' % (test_mae,test_mape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在测试集也能取得满意结果的时候，可以在平台上测试并提交。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3.6 保存和读取模型\n",
    "\n",
    "下面介绍保存和读取模型。模型应当保存在`results`文件夹下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设计目录\n",
    "model_path = 'results/mymodel.pt'\n",
    "# 保存模型\n",
    "torch.save(model.state_dict(), model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "读取模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 指定目录\n",
    "model_path = 'results/mymodel.pt'\n",
    "# 选用使用的模型类\n",
    "model = LinearNet(num_inputs,num_outputs)\n",
    "# 读入对应的参数\n",
    "model.load_state_dict(torch.load(model_path))\n",
    "# \n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ### 2.3.7  torch 张量 和 numpy.ndarray 数据类型相互转换  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# torch.Tensor 转化为 numpy.ndarray\n",
    "x_torch = torch.empty(3,5)\n",
    "print(type(x_torch))\n",
    "\n",
    "# torch to numpy\n",
    "x_numpy = x_torch.numpy()\n",
    "x_numpy_v2 = np.array(x_torch)\n",
    "print(type(x_numpy))\n",
    "print(type(x_numpy_v2))\n",
    "\n",
    "# numpy to torch\n",
    "x_torch_v2 = torch.from_numpy(x_numpy)\n",
    "print(type(x_torch_v2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3.作业\n",
    "\n",
    "## 3.1 训练模型\n",
    "- 模型训练时请主要在 `GPU` 上训练，在平台上可以使用离线任务 `GPU` 完成，并将模型保存到 `results` 文件夹中，并在模型预测时读取。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train():\n",
    "    '''训练模型\n",
    "    :return: model 一个训练好的模型\n",
    "    '''\n",
    "    \n",
    "    model = None\n",
    "    # --------------------------- 此处下方加入训练模型相关代码 -------------------------------\n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    \n",
    "    # 如果使用的不是 pytorch 框架，还需要改动下面的代码\n",
    "    # 模型保存的位置\n",
    "    model_path = 'results/mymodel.pt'\n",
    "    # 保存模型\n",
    "    torch.save(model.state_dict(), model_path)\n",
    "    # --------------------------- 此处上方加入训练模型相关代码 -------------------------------\n",
    "    \n",
    "    \n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 3.2 模型预测\n",
    "\n",
    "注意事项：\n",
    "1. 本实验并不严格限定使用的框架，可以使用 `Pytorch` , `Tensorflow` 或其他框架。只需训练好模型并保存，并在下文中写入合适的读取模型并实现预测即可。\n",
    "2. 点击左侧栏`提交作业`后点击`生成文件`则只需勾选 `predict()` 函数的cell，即【**模型预测代码答题区域**】的 cell。\n",
    "3. 请导入必要的包和第三方库 (包括此文件中曾经导入过的)。\n",
    "4. 请加载你认为训练最佳的模型，即请按要求填写模型路径。\n",
    "5. `predict()`函数的输入和输出请**不要改动**。\n",
    "6. 注意，模型预测 `x.shape[0] < 20000` 的数据不能超过 $5$ 分钟，否则将被记为超时。\n",
    "7. `predict()`函数 返回的类型必须是 `numpy` 数组类型。\n",
    "8. 实验指标为平均绝对误差（ `MAPE` ）和平均绝对误差（ `MAE` ）。\n",
    "9. 作业测试时记得填写你的模型路径及名称, 如果采用 [离线任务](https://momodel.cn/docs/#/zh-cn/%E5%9C%A8GPU%E6%88%96CPU%E8%B5%84%E6%BA%90%E4%B8%8A%E8%AE%AD%E7%BB%83%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E6%A8%A1%E5%9E%8B) 请将模型保存在 **results** 文件夹下。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在下方规定区域内写入**加载模型**的方式，该函数是在被测试和评估时调用的**预测函数**。注意，为了便于用户使用各种框架，输出的数组必须为 `numpy` 数组。\n",
    "\n",
    " ==================  **提交 Notebook 训练模型结果数据处理参考示范**  =================="
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "# 1.导入相关第三方库或者包（根据自己需求，可以增加、删除等改动）\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import torch\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# 2.导入 Notebook 使用的模型\n",
    "# 建立一个简单的线性模型\n",
    "class LinearNet(torch.nn.Module):\n",
    "    def __init__(self, num_inputs, num_outputs):\n",
    "        super(LinearNet, self).__init__()\n",
    "        # 一个线性层\n",
    "        self.linear = torch.nn.Linear(num_inputs, num_outputs)\n",
    "        \n",
    "    # 前向传播函数\n",
    "    def forward(self, x): # x shape: (batch, 14)\n",
    "        y = self.linear(x)\n",
    "        return y\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "============================  **模型预测代码答题区域**  ============================\n",
    "<br>\n",
    "在下方的代码块中编写 **模型预测** 部分的代码，请勿在别的位置作答"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "deletable": false,
    "select": true
   },
   "outputs": [],
   "source": [
    "# 加载 Notebook 模型流程\n",
    "\n",
    "# 输入的数量是前 14 个交易日的收盘价\n",
    "num_inputs = 14\n",
    "# 输出是下一个交易日的收盘价\n",
    "num_outputs = 1\n",
    "\n",
    "# ------------------------- 请加载您最满意的模型网络结构 -----------------------------\n",
    "# 读取模型\n",
    "model = LinearNet(num_inputs,num_outputs)\n",
    "    \n",
    "# ----------------------------- 请加载您最满意的模型 -------------------------------\n",
    "# 加载模型(请加载你认为的最佳模型)\n",
    "# 加载模型,加载请注意 model_path 是相对路径, 与当前文件同级。\n",
    "# 如果你的模型是在 results 文件夹下的 temp.pth 模型，则 model_path = 'results/mymodel.pt'\n",
    "# 模型保存的位置，如果模型路径不同，请修改！！！\n",
    "model_path = 'results/mymodel.pt'\n",
    "model.load_state_dict(torch.load(model_path))\n",
    "model.eval()\n",
    "\n",
    "def predict(test_x):\n",
    "    '''\n",
    "    对于给定的 x 预测未来的 y 。\n",
    "    :param test_x: 给定的数据集合 x ，对于其中的每一个元素需要预测对应的 y 。e.g.:np.array([[6.69,6.72,6.52,6.66,6.74,6.55,6.35,6.14,6.18,6.17,5.72,5.78,5.69,5.67]]\n",
    "    :return: test_y 对于每一个 test_x 中的元素，给出一个对应的预测值。e.g.:np.array([[0.0063614]])\n",
    "    '''\n",
    "    # test 的数目\n",
    "    n_test = test_x.shape[0]\n",
    "    \n",
    "    test_y = None\n",
    "    # --------------------------- 此处下方加入读入模型和预测相关代码 -------------------------------\n",
    "    # 此处为 Notebook 模型示范，你可以根据自己数据处理方式进行改动\n",
    "    # scaler = MinMaxScaler().fit(np.array([0, 300]).reshape(-1, 1))\n",
    "    # test_x = scaler.transform(test_x.reshape(-1, 1)).reshape(-1, 14)\n",
    "    test_x = torch.tensor(test_x, dtype=torch.float32)\n",
    "    \n",
    "    test_y = model(test_x)\n",
    "    \n",
    "    # 如果使用 MinMaxScaler 进行数据处理，预测后应使用下一句将预测值放缩到原范围内\n",
    "    # test_y = scaler.inverse_transform(test_y.detach().cpu())\n",
    "    test_y = test_y.detach().cpu().numpy()\n",
    "    # --------------------------- 此处上方加入读入模型和预测相关代码 -------------------------------\n",
    "    \n",
    "    # 保证输出的是一个 numpy 数组\n",
    "    assert(type(test_y) == np.ndarray)\n",
    "    \n",
    "    # 保证 test_y 的 shape 正确\n",
    "    assert(test_y.shape == (n_test, 1))\n",
    "    \n",
    "    return test_y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 测试用例\n",
    "model_test_x = np.array([[6.69,6.72,6.52,6.66,6.74,6.55,6.35,6.14,6.18,6.17,5.72,5.78,5.69,5.67]])\n",
    "print(predict(test_x = model_test_x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 提交程序报告\n",
    "\n",
    "为了检查作业的详实程度及具体方法，本实验需要提交程序报告。\n",
    "\n",
    "提交作业时请记得左侧文件列表中上传『程序报告.docx』或者 『程序报告.pdf』。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}