main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CVPR2022 NAS竞赛Track 2第18名方案\n",
    "\n",
    "方案说明：\n",
    "1. 本方案基于官方的[baseline](https://aistudio.baidu.com/aistudio/projectdetail/3720834)修改而来。\n",
    "2. 基于上述方案主要有两个方面的修改：\n",
    "    * 修改了模型的表达方式\n",
    "    * 修改了源GPNAS中的kernel function\n",
    "\n",
    "\n",
    "**模型表达方式**\n",
    "\n",
    "原始的模型表达方式（编码）太\"直观\"了，比如\"j111231321311311221231121111231000000\"这样的一串字符串，而模型的表达远比这来的要复杂，所以我的第一个想法就是如何表达模型。曾经尝试过用embdding的方式(可以参考NLP)，确实能够看到不同的embedding对模型的表示结果不同，但是反映到赛题上结果都很差。\n",
    "\n",
    "而我这里尝试直接修改原编码字符串，参考代码中的convert_X函数，分数直接由baseline的**0.66提升到0.76**。再简单微调一下就到了**0.78**，ab榜都在18名，而与第一名也就相差1个百分点。\n",
    "\n",
    "实话讲，不清楚为什么这样做会有这么大的提升，这样刷榜也实在不妥，所以比赛后半端就不再提交了...\n",
    "\n",
    "这里只能分享一下我修改编码字符串的一些经验：\n",
    "1. 编码字符串的长短与结果没有必然联系\n",
    "2. 原始字符串与拓展字符串拼接结果最好\n",
    "3. 拓展字符串需要能够最大程度区分原始串中\"1,2,3\"等表示方式，比如用5次方\n",
    "\n",
    "**GPNAS中的kernel function**\n",
    "\n",
    "GPNAS源代码中_get_corelation中提供了两个kernel function，而我这里直接使用np.abs，提升在1-2个百分点。\n",
    "```python\n",
    "    def _get_corelation(self, mat1, mat2):\n",
    "        \"\"\"\n",
    "        give two typical kernel function\n",
    "        \n",
    "        Auto kernel hyperparameters estimation to be updated\n",
    "        \"\"\"\n",
    "\n",
    "        mat_diff = abs(mat1 - mat2)\n",
    "\n",
    "        if self.c_flag == 1:\n",
    "\n",
    "            return 0.5 * np.exp(-np.dot(mat_diff, mat_diff) / 16)\n",
    "\n",
    "        elif self.c_flag == 2:\n",
    "\n",
    "            # return 1 * np.exp(-np.sqrt(np.dot(mat_diff, mat_diff)) / 12)\n",
    "            _data = np.dot(mat1, mat2)\n",
    "            return np.abs(_data)\n",
    "```\n",
    "\n",
    "**最后**\n",
    "\n",
    "这次成绩偶然成分较大，欢迎大家多多交流 ：）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:46.611188Z",
     "iopub.status.busy": "2022-05-20T03:45:46.610919Z",
     "iopub.status.idle": "2022-05-20T03:45:47.031553Z",
     "shell.execute_reply": "2022-05-20T03:45:47.030734Z",
     "shell.execute_reply.started": "2022-05-20T03:45:46.611155Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "data134077\n",
      "CVPR_2022_NAS_Track1_test.json\tCVPR_2022_NAS_Track2_train.json\n",
      "CVPR_2022_NAS_Track2_test.json\tresnet48.pdparams\n"
     ]
    }
   ],
   "source": [
    "!ls /home/aistudio/data\n",
    "!ls /home/aistudio/data/data134077"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:47.034240Z",
     "iopub.status.busy": "2022-05-20T03:45:47.033530Z",
     "iopub.status.idle": "2022-05-20T03:45:48.513360Z",
     "shell.execute_reply": "2022-05-20T03:45:48.512527Z",
     "shell.execute_reply.started": "2022-05-20T03:45:47.034178Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import itertools\n",
    "import sklearn\n",
    "import numpy as np\n",
    "import lightgbm as lgb\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.515334Z",
     "iopub.status.busy": "2022-05-20T03:45:48.514713Z",
     "iopub.status.idle": "2022-05-20T03:45:48.522099Z",
     "shell.execute_reply": "2022-05-20T03:45:48.521485Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.515302Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'cplfw_rank': 9, 'market1501_rank': 361, 'dukemtmc_rank': 426, 'msmt17_rank': 433, 'veri_rank': 327, 'vehicleid_rank': 480, 'veriwild_rank': 425, 'sop_rank': 367, 'arch': 'l231131331121121331111211121331321321'}\n",
      "train_num: 500\n"
     ]
    }
   ],
   "source": [
    "# 读取训练数据, 训练集包含500个模型结构，以及这些结构在cplfw，market1501，dukemtmc等8个任务上的性能排序\n",
    "import json\n",
    "with open('/home/aistudio/data/data134077/CVPR_2022_NAS_Track2_train.json', 'r') as f:\n",
    "    train_data = json.load(f)\n",
    "print(train_data['arch1'])\n",
    "print('train_num:',len(train_data.keys()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.523147Z",
     "iopub.status.busy": "2022-05-20T03:45:48.522967Z",
     "iopub.status.idle": "2022-05-20T03:45:48.591456Z",
     "shell.execute_reply": "2022-05-20T03:45:48.590772Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.523126Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3, 2, 3, 1, 1, 3, 1, 3, 3, 1, 1, 2, 1, 1, 2, 1, 3, 3, 1, 1, 1, 1, 2, 1, 1, 1, 2, 1, 3, 3, 1, 3, 2, 1, 3, 2, 1, 0.1411200080598672, 0.9092974268256817, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 20.085536923187668, 7.38905609893065, 20.085536923187668, 2.718281828459045, 2.718281828459045, 20.085536923187668, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 2.718281828459045, 7.38905609893065, 2.718281828459045, 2.718281828459045, 7.38905609893065, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 2.718281828459045, 2.718281828459045, 2.718281828459045, 7.38905609893065, 2.718281828459045, 2.718281828459045, 2.718281828459045, 7.38905609893065, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 243, 32, 243, 1, 1, 243, 1, 243, 243, 1, 1, 32, 1, 1, 32, 1, 243, 243, 1, 1, 1, 1, 32, 1, 1, 1, 32, 1, 243, 243, 1, 243, 32, 1, 243, 32, 1] [2, 1, 1, 1, 2, 2, 1, 3, 2, 1, 1, 3, 1, 3, 3, 1, 3, 2, 1, 3, 1, 1, 3, 2, 1, 3, 1, 1, 3, 1, 1, 3, 2, 1, 0, 0, 0, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.0, 0.0, 0.0, 7.38905609893065, 2.718281828459045, 2.718281828459045, 2.718281828459045, 7.38905609893065, 7.38905609893065, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 2.718281828459045, 20.085536923187668, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 20.085536923187668, 2.718281828459045, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 20.085536923187668, 2.718281828459045, 2.718281828459045, 20.085536923187668, 2.718281828459045, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 0, 0, 0, 32, 1, 1, 1, 32, 32, 1, 243, 32, 1, 1, 243, 1, 243, 243, 1, 243, 32, 1, 243, 1, 1, 243, 32, 1, 243, 1, 1, 243, 1, 1, 243, 32, 1, 0, 0, 0]\n"
     ]
    }
   ],
   "source": [
    "def convert_X_raw(arch_str):\n",
    "    temp_arch = []\n",
    "    for elm in arch_str:\n",
    "        if elm == 'l':\n",
    "            temp_arch.append(1)\n",
    "        elif elm == 'j':\n",
    "            temp_arch.append(2)\n",
    "        elif elm == 'k':\n",
    "            temp_arch.append(3)\n",
    "        else: \n",
    "            temp_arch.append(int(elm))\n",
    "    return(temp_arch)\n",
    "\n",
    "def convert_X(arch_str):\n",
    "    temp_arch = []\n",
    "    # ‘j’，'k’和’l’分别表示10，11和12\n",
    "    _length = 0\n",
    "    _head = 0\n",
    "    if arch_str[0] == 'j':\n",
    "        _length = 10\n",
    "        _head = 1\n",
    "    elif arch_str[0] == 'k':\n",
    "        _length = 11\n",
    "        _head = 2\n",
    "    elif arch_str[0] == 'l':\n",
    "        _length = 12\n",
    "        _head = 3\n",
    "\n",
    "    temp_arch = temp_arch + [_head] + [int(s) for s in arch_str[1:]]\n",
    "    temp_arch = temp_arch + [np.sin(_head)] + [np.sin(int(s)) for s in arch_str[1:]]\n",
    "    temp_arch = temp_arch + [np.exp(_head)] + [np.exp(int(s)) for i in range(3)\n",
    "    for s in (arch_str[1+i*_length:1+(i+1)*_length])] + [0]*(12-_length)*3\n",
    "    temp_arch = temp_arch + [_head**5] + [int(s)**5 for i in range(3)\n",
    "    for s in (arch_str[1+i*_length:1+(i+1)*_length])] + [0]*(12-_length)*3\n",
    "\n",
    "    return temp_arch\n",
    "\n",
    "train_list = [[],[],[],[],[],[],[],[]]\n",
    "arch_list_train = []\n",
    "name_list = ['cplfw_rank', 'market1501_rank', 'dukemtmc_rank', 'msmt17_rank', 'veri_rank', 'vehicleid_rank', 'veriwild_rank', 'sop_rank']\n",
    "for key in train_data.keys():\n",
    "    for idx, name in enumerate(name_list):\n",
    "        train_list[idx].append(train_data[key][name])\n",
    "    arch_list_train.append(convert_X(train_data[key]['arch']))\n",
    "print(arch_list_train[0], arch_list_train[1])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.592969Z",
     "iopub.status.busy": "2022-05-20T03:45:48.592460Z",
     "iopub.status.idle": "2022-05-20T03:45:48.604222Z",
     "shell.execute_reply": "2022-05-20T03:45:48.603669Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.592942Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(500, 148)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(arch_list_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.605511Z",
     "iopub.status.busy": "2022-05-20T03:45:48.605320Z",
     "iopub.status.idle": "2022-05-20T03:45:48.627680Z",
     "shell.execute_reply": "2022-05-20T03:45:48.627046Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.605491Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "\n",
    "#\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "\n",
    "import numpy as np\n",
    "import copy\n",
    "\n",
    "# __all__ = [\"GPNAS\"]\n",
    "\n",
    "\n",
    "class GPNAS(object):\n",
    "    \"\"\"\n",
    "    GPNAS(Gaussian Process based Neural Architecture Search) is a neural architecture search algorithm. \n",
    "    We model the correlation between architectue and performance from a Bayesian perspective. Specifically, by introducing a novel Gaussian Process based\n",
    "    NAS (GP-NAS) method, the correlations are modeled by the kernel function and mean function. The kernel function is also learnable to enable adaptive modeling for complex \n",
    "    correlations in different search spaces. Furthermore, by in-corporating a mutual information based sampling method, we can theoretically ensure the high-performance \n",
    "    architecture with only a small set of samples. After addressing these problems, training GP-NAS once enables direct performance prediction of any architecture in different \n",
    "    scenarios and may obtain efficient networks for different deployment platforms.\n",
    "    \"\"\"\n",
    "\n",
    "    def __init__(self, c_flag=2, m_flag=2):\n",
    "\n",
    "        self.hp_mat = 0.0000001\n",
    "        self.hp_cov = 0.01\n",
    "        self.cov_w = None\n",
    "        self.w = None\n",
    "        self.c_flag = c_flag\n",
    "        self.m_flag = m_flag\n",
    "\n",
    "    def _get_corelation(self, mat1, mat2):\n",
    "        \"\"\"\n",
    "        give two typical kernel function\n",
    "        \n",
    "        Auto kernel hyperparameters estimation to be updated\n",
    "        \"\"\"\n",
    "\n",
    "        mat_diff = abs(mat1 - mat2)\n",
    "\n",
    "        if self.c_flag == 1:\n",
    "\n",
    "            return 0.5 * np.exp(-np.dot(mat_diff, mat_diff) / 16)\n",
    "\n",
    "        elif self.c_flag == 2:\n",
    "\n",
    "            # return 1 * np.exp(-np.sqrt(np.dot(mat_diff, mat_diff)) / 12)\n",
    "            _data = np.dot(mat1, mat2)\n",
    "            return np.abs(_data)\n",
    "\n",
    "    def _preprocess_X(self, X):\n",
    "        \"\"\"\n",
    "        preprocess of input feature/ tokens of architecture\n",
    "        more complicated preprocess can be added such as nonlineaer transformation\n",
    "        \"\"\"\n",
    "\n",
    "        X = X.tolist()\n",
    "        p_X = copy.deepcopy(X)\n",
    "\n",
    "        # for feature in p_X:\n",
    "        #     feature.append(1)\n",
    "        \n",
    "        return p_X\n",
    "\n",
    "    def _get_cor_mat(self, X):\n",
    "        \"\"\"\n",
    "        get kernel matrix\n",
    "        \"\"\"\n",
    "        X = np.array(X)\n",
    "        l = X.shape[0]\n",
    "        cor_mat = []\n",
    "\n",
    "        for c_idx in range(l):\n",
    "            col = []\n",
    "            c_mat = X[c_idx].copy()\n",
    "\n",
    "            for r_idx in range(l):\n",
    "                r_mat = X[r_idx].copy()\n",
    "                temp_cor = self._get_corelation(c_mat, r_mat)\n",
    "                col.append(temp_cor)\n",
    "            cor_mat.append(col)\n",
    "\n",
    "        return np.mat(cor_mat)\n",
    "\n",
    "    def _get_cor_mat_joint(self, X, X_train):\n",
    "        \"\"\"\n",
    "        get kernel matrix\n",
    "        \"\"\"\n",
    "        X = np.array(X)\n",
    "        X_train = np.array(X_train)\n",
    "        l_c = X.shape[0]\n",
    "        l_r = X_train.shape[0]\n",
    "        cor_mat = []\n",
    "\n",
    "        for c_idx in range(l_c):\n",
    "            col = []\n",
    "            c_mat = X[c_idx].copy()\n",
    "\n",
    "            for r_idx in range(l_r):\n",
    "                r_mat = X_train[r_idx].copy()\n",
    "                temp_cor = self._get_corelation(c_mat, r_mat)\n",
    "                col.append(temp_cor)\n",
    "            cor_mat.append(col)\n",
    "\n",
    "        return np.mat(cor_mat)\n",
    "\n",
    "    def get_predict(self, X):\n",
    "        \"\"\"\n",
    "        get the prediction of network architecture X\n",
    "        \"\"\"\n",
    "        X = self._preprocess_X(X)\n",
    "        X = np.mat(X)\n",
    "\n",
    "        return X * self.w\n",
    "\n",
    "    def get_predict_jiont(self, X, X_train, Y_train):\n",
    "        \"\"\"\n",
    "        get the prediction of network architecture X based on X_train and Y_train\n",
    "        \"\"\"\n",
    "        X = np.mat(X)\n",
    "        X_train = np.mat(X_train)\n",
    "        Y_train = np.mat(Y_train)\n",
    "        m_X = self.get_predict(X)\n",
    "        m_X_train = self.get_predict(X_train)\n",
    "        mat_train = self._get_cor_mat(X_train)\n",
    "        mat_joint = self._get_cor_mat_joint(X, X_train)\n",
    "\n",
    "        return m_X + mat_joint * np.linalg.inv(mat_train + self.hp_mat * np.eye(\n",
    "            X_train.shape[0])) * (Y_train.T - m_X_train)\n",
    "\n",
    "    def get_initial_mean(self, X, Y):\n",
    "        \"\"\"\n",
    "        get initial mean of w\n",
    "        \"\"\"\n",
    "\n",
    "        X = self._preprocess_X(X)\n",
    "        X = np.mat(X)\n",
    "        Y = np.mat(Y)\n",
    "        self.w = np.linalg.inv(X.T * X + self.hp_mat * np.eye(X.shape[\n",
    "            1])) * X.T * Y.T\n",
    "\n",
    "        return self.w\n",
    "\n",
    "    def get_initial_cov(self, X):\n",
    "        \"\"\"\n",
    "        get initial coviarnce matrix of w\n",
    "        \"\"\"\n",
    "\n",
    "        X = self._preprocess_X(X)\n",
    "        X = np.mat(X)\n",
    "        self.cov_w = self.hp_cov * np.eye(X.shape[1])\n",
    "\n",
    "        return self.cov_w\n",
    "\n",
    "    def get_posterior_mean(self, X, Y):\n",
    "        \"\"\"\n",
    "        get posterior mean of w\n",
    "        \"\"\"\n",
    "\n",
    "        X = self._preprocess_X(X)\n",
    "        X = np.mat(X)\n",
    "        Y = np.mat(Y)\n",
    "        cov_mat = self._get_cor_mat(X)\n",
    "        if self.m_flag == 1:\n",
    "            self.w = self.w + self.cov_w * X.T * np.linalg.inv(\n",
    "                np.linalg.inv(cov_mat + self.hp_mat * np.eye(X.shape[0])) + X *\n",
    "                self.cov_w * X.T + self.hp_mat * np.eye(X.shape[0])) * (\n",
    "                    Y.T - X * self.w)\n",
    "        else:\n",
    "            self.w = np.linalg.inv(X.T * np.linalg.inv(\n",
    "                cov_mat + self.hp_mat * np.eye(X.shape[0])) * X + np.linalg.inv(\n",
    "                    self.cov_w + self.hp_mat * np.eye(X.shape[\n",
    "                        1])) + self.hp_mat * np.eye(X.shape[1])) * (\n",
    "                            X.T * np.linalg.inv(cov_mat + self.hp_mat * np.eye(\n",
    "                                X.shape[0])) * Y.T +\n",
    "                            np.linalg.inv(self.cov_w + self.hp_mat * np.eye(\n",
    "                                X.shape[1])) * self.w)\n",
    "\n",
    "        return self.w\n",
    "\n",
    "    def get_posterior_cov(self, X, Y):\n",
    "        \"\"\"\n",
    "        get posterior coviarnce matrix of w\n",
    "        \"\"\"\n",
    "\n",
    "        X = self._preprocess_X(X)\n",
    "        X = np.mat(X)\n",
    "        Y = np.mat(Y)\n",
    "        cov_mat = self._get_cor_mat(X)\n",
    "        self.cov_mat = np.linalg.inv(\n",
    "            np.linalg.inv(X.T * cov_mat * X + self.hp_mat * np.eye(X.shape[1]))\n",
    "            + np.linalg.inv(self.cov_w + self.hp_mat * np.eye(X.shape[\n",
    "                1])) + self.hp_mat * np.eye(X.shape[1]))\n",
    "\n",
    "        return self.cov_mat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.628832Z",
     "iopub.status.busy": "2022-05-20T03:45:48.628640Z",
     "iopub.status.idle": "2022-05-20T03:45:48.639377Z",
     "shell.execute_reply": "2022-05-20T03:45:48.638766Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.628811Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(500, 148) (8, 500)\n"
     ]
    }
   ],
   "source": [
    "data_x = np.array(arch_list_train)\n",
    "data_y = np.array(train_list)\n",
    "print(data_x.shape, data_y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T03:45:48.640824Z",
     "iopub.status.busy": "2022-05-20T03:45:48.640478Z",
     "iopub.status.idle": "2022-05-20T04:11:39.442336Z",
     "shell.execute_reply": "2022-05-20T04:11:39.441596Z",
     "shell.execute_reply.started": "2022-05-20T03:45:48.640801Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "AVE mean gp : 1.299788574090299\n",
      "AVE mean gp jonit : 1.0723321789788158\n",
      "Kendalltau: KendalltauResult(correlation=0.16842105263157894, pvalue=0.3188964804719496)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.23157894736842105, pvalue=0.1649764063636054)\n",
      "AVE mean gp : 0.26316373137516885\n",
      "AVE mean gp jonit : 0.22960139533501134\n",
      "Kendalltau: KendalltauResult(correlation=0.6947368421052632, pvalue=3.318592792384182e-06)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.6947368421052632, pvalue=3.318592792384182e-06)\n",
      "AVE mean gp : 0.2908729038847623\n",
      "AVE mean gp jonit : 0.3587614045704137\n",
      "Kendalltau: KendalltauResult(correlation=0.9263157894736842, pvalue=4.97267870195355e-13)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.9263157894736842, pvalue=4.97267870195355e-13)\n",
      "AVE mean gp : 0.9686897408607986\n",
      "AVE mean gp jonit : 0.8820272111648467\n",
      "Kendalltau: KendalltauResult(correlation=0.9473684210526316, pvalue=3.348593561359954e-14)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.9157894736842106, pvalue=1.637252132068114e-12)\n",
      "AVE mean gp : 0.3936245239052972\n",
      "AVE mean gp jonit : 0.32119456029801186\n",
      "Kendalltau: KendalltauResult(correlation=0.9263157894736842, pvalue=4.97267870195355e-13)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.9263157894736842, pvalue=4.97267870195355e-13)\n",
      "AVE mean gp : 0.37736023476766356\n",
      "AVE mean gp jonit : 0.3311073952131503\n",
      "Kendalltau: KendalltauResult(correlation=0.7263157894736842, pvalue=8.334889271753901e-07)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.7368421052631579, pvalue=5.089279304093134e-07)\n",
      "AVE mean gp : 0.8331269635708216\n",
      "AVE mean gp jonit : 0.7163080760790616\n",
      "Kendalltau: KendalltauResult(correlation=0.9263157894736842, pvalue=4.97267870195355e-13)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.9368421052631579, pvalue=1.3686946653867177e-13)\n",
      "AVE mean gp : 3.246776509285374\n",
      "AVE mean gp jonit : 2.5000252682944244\n",
      "Kendalltau: KendalltauResult(correlation=0.8105263157894737, pvalue=9.06108549128192e-09)\n",
      "Kendalltau joint: KendalltauResult(correlation=0.8631578947368421, pvalue=2.1870145045372028e-10)\n",
      "amg DescribeResult(nobs=8, minmax=(0.26316373137516885, 3.246776509285374), mean=0.9591753977175231, variance=0.9935045785281993, skewness=1.6957062789398027, kurtosis=1.6397168658929804)\n",
      "amgl DescribeResult(nobs=8, minmax=(0.22960139533501134, 2.5000252682944244), mean=0.8014196862417169, variance=0.5638754851697432, skewness=1.5986186967458167, kurtosis=1.404481859417137)\n",
      "kt DescribeResult(nobs=8, minmax=(array([1.68421053e-01, 3.34859356e-14]), array([0.94736842, 0.31889648])), mean=array([0.76578947, 0.03986258]), variance=array([0.06796201, 0.01271182]), skewness=array([-1.68176756,  2.26778684]), kurtosis=array([1.61119243, 3.14285714]))\n",
      "ktj DescribeResult(nobs=8, minmax=(array([2.31578947e-01, 1.36869467e-13]), array([0.93684211, 0.16497641])), mean=array([0.77894737, 0.02062253]), variance=array([0.05752275, 0.00340213]), skewness=array([-1.67345031,  2.26778684]), kurtosis=array([1.52891529, 3.14285714]))\n"
     ]
    }
   ],
   "source": [
    "# from paddleslim.nas import GPNAS\n",
    "import numpy as np\n",
    "import scipy\n",
    "import scipy.stats\n",
    "\n",
    "gp_list = []\n",
    "\n",
    "\n",
    "for i in range(len(train_list[:])):\n",
    "    # 每个任务有该任务专属的gpnas预测器\n",
    "    gp_list.append(GPNAS(2,2)) # baseline\n",
    "    # gp_list.append(GPNAS(1,2)) # v1\n",
    "    # gp_list.append(GPNAS(1,1)) # v2\n",
    "    # gp_list.append(GPNAS(2,1)) # v3\n",
    "\n",
    "train_num = 480\n",
    "\n",
    "amg_list = []\n",
    "amgj_list = []\n",
    "kt_list = []\n",
    "ktj_list = []\n",
    "\n",
    "for i in range(len(train_list[:])):\n",
    "    # 划分训练及测试集\n",
    "    X_all_k, Y_all_k  = data_x, data_y[i]\n",
    "    X_train_k, Y_train_k, X_test_k, Y_test_k = X_all_k[0:train_num:1], Y_all_k[0:train_num:1], X_all_k[train_num::1], Y_all_k[train_num::1]\n",
    "    # 初始该任务的gpnas预测器参数\n",
    "    gp_list[i].get_initial_mean(X_train_k[0::2],Y_train_k[0::2])\n",
    "    init_cov = gp_list[i].get_initial_cov(X_train_k)\n",
    "    # 更新（训练）gpnas预测器超参数\n",
    "    gp_list[i].get_posterior_mean(X_train_k[1::2],Y_train_k[1::2])  \n",
    "   \n",
    "    # 基于测试评估预测误差   \n",
    "    y_pred = gp_list[i].get_predict(X_test_k)\n",
    "    y_pred_joint = gp_list[i].get_predict_jiont(X_test_k, X_train_k[::1], Y_train_k[::1])\n",
    "\n",
    "    error_list_gp = np.array(Y_test_k.reshape(len(Y_test_k),1)-y_pred)\n",
    "    error_list_gp_j = np.array(Y_test_k.reshape(len(Y_test_k),1)-y_pred_joint)\n",
    "    \n",
    "    ave_mean_gp = np.mean(abs(np.divide(error_list_gp,Y_test_k.reshape(len(Y_test_k),1) )))\n",
    "    ave_mean_gp_joint = np.mean(abs(np.divide(error_list_gp_j,Y_test_k.reshape(len(Y_test_k),1) )))\n",
    "    print('AVE mean gp :', ave_mean_gp)\n",
    "    print('AVE mean gp jonit :', ave_mean_gp_joint)\n",
    "    # y_predict = gp_list[i].get_predict_jiont(X_test_k, X_train_k[::1], Y_train_k[::1])\n",
    "\n",
    "    #基于测试集评估预测的Kendalltau\n",
    "    kt = scipy.stats.stats.kendalltau(y_pred,Y_test_k)\n",
    "    print('Kendalltau:', kt)\n",
    "    ktj = scipy.stats.stats.kendalltau(y_pred_joint,Y_test_k)\n",
    "    print('Kendalltau joint:', ktj)\n",
    "\n",
    "    amg_list.append(ave_mean_gp)\n",
    "    amgj_list.append(ave_mean_gp_joint)\n",
    "    kt_list.append(kt)\n",
    "    ktj_list.append(ktj)\n",
    "\n",
    "import scipy.stats\n",
    "print('amg', scipy.stats.describe(amg_list))\n",
    "print('amgl', scipy.stats.describe(amgj_list))\n",
    "print('kt', scipy.stats.describe(kt_list))\n",
    "print('ktj', scipy.stats.describe(ktj_list))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T04:11:39.444176Z",
     "iopub.status.busy": "2022-05-20T04:11:39.443428Z",
     "iopub.status.idle": "2022-05-20T04:11:39.476599Z",
     "shell.execute_reply": "2022-05-20T04:11:39.475926Z",
     "shell.execute_reply.started": "2022-05-20T04:11:39.444143Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_predict = y_pred"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T04:11:39.478104Z",
     "iopub.status.busy": "2022-05-20T04:11:39.477633Z",
     "iopub.status.idle": "2022-05-20T04:11:39.801079Z",
     "shell.execute_reply": "2022-05-20T04:11:39.800421Z",
     "shell.execute_reply.started": "2022-05-20T04:11:39.478079Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fad212ee050>]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXd4XNW1t989M9KojfpIVnXvHRuMMb33jmkhhOQGEiD3Em4oSe5303NJJSQhJCEFkgCh2JQQmimm22CDcS+SLKt3aUa9zOzvj32ONJJlq03Xfp/Hz2jOnJmzJUtr1qz9W78lpJRoNBqNJnqxhHoBGo1GowksOtBrNBpNlKMDvUaj0UQ5OtBrNBpNlKMDvUaj0UQ5OtBrNBpNlKMDvUaj0UQ5OtBrNBpNlDOqQC+EKBVC7BBCbBNCbDGOpQshNgghDhi3acZxIYT4tRCiSAixXQhxTCC/AY1Go9EcHTGazlghRCmwUkrZ4HPsp0CTlPI+IcS9QJqU8h4hxPnA14DzgVXAA1LKVUd7/czMTDlt2rTxfxcajUYzCdm6dWuDlNI50nm2CVzjEuBU4+tHgY3APcbxv0n1DrJJCJEqhMiRUlYf6YWmTZvGli1bJrAUjUajmXwIIQ6N5rzR1ugl8JoQYqsQ4mbjWLZP8K4Bso2v84Byn+dWGMc0Go1GEwJGm9GfKKWsFEJkARuEEHt9H5RSSiHEmNzRjDeMmwEKCwvH8lSNRqPRjIFRZfRSykrjtg54FjgOqBVC5AAYt3XG6ZVAgc/T841jQ1/zj1LKlVLKlU7niCUmjUaj0YyTEQO9ECJRCOEwvwbOBnYCLwA3GqfdCDxvfP0C8HlDfXM84DpafV6j0Wg0gWU0pZts4FkhhHn+41LKV4QQHwNPCSG+BBwC1hrnv4RS3BQBHcBNfl+1RqPRaEbNiIFeSlkCLB3meCNwxjDHJXCbX1an0Wg0mgmjO2M1Go0mytGBXqPRaEKB1wuvfhuqtgX8UjrQazQaTSio+Bg+/C3U7x353AmiA71Go9GEgl3rwWqHuecH/FI60Gs0mvHR3gi/PwkaDoR6JZGH1wO7noPZZ0FccsAvpwN9BNPV6+Gt3bpFQRMiqrdBzXYo/yjUK4k8yj6EthpYeFlQLqcDfaTiqqDioUtZ8eRySkp0RqUJAa4KddtWG9p1RCI714MtHuacG5TL6UAfaXj64IPfIn97HAVNH5IsOmne+lyoV6WZjLgNZ5O2uqOfpxmMpw92P493zjl8+6WDfFLWHPBL6kAfSVRsgT+eCq99m4bMYzmj++cUe3NwHHot1CvTTEbMjL5dB/oxUfoudDRwMPscHttcRnlTR8AvqQN9JNDZAi/eCX86EzoaYe3fuTvmW/Qk5bPLsYbpbZ9AlyvUq9RMNlyGG7nO6MfGrvUQm8SzbQuwWQSnzs0K+CV1oA9npIQdz8CDx8HWv8Kqr8DtH1GZexYbDzRw9bEFtE07hxj66Nmrs3pNkHGZpRtdox81nl7Y8y+Yez4v721h1Yx0UuJjAn5ZHejDlaYS+MflsO5LkJwLX34TzrsP7A6e/FhlUlcfW4Bz/ok0yGRaP3shxAvWTCqk1DX68VCyETqbqSk4j+L6ds6anz3iU/zBREYJagJBXze8/2t49+dgiYHzfgrH/gdYrOphj5cnPy7jlDlO8tMSiLVZeMNzDJeWv6myBWvgswONho5G6OuChEzoaFC/tzZ7qFcV/uxcB/YUXuyYD5Rw5oLgBHqd0YcTpe/B70+Et36oZFe3fwSrbukP8gBv7aun1t3NtcepqVxZjjg+iV+Nva9NPV+jCQZmfT7vGHWrs/qR6e2Cvf+G+Rfy6t5m5uckk5+WEJRL60AfDrQ3wnO3wiMXqCzpuqdh7aOqZDOEJz4qI8th5/R5Axs43YUn00Us7HspmKvWTGbM+nzeCnWrlTcjU/wGdLtxz7yIrYeaOStI2TzoQB9apIRP/wG/XQnbn4QTvw63boY5Zw97emVLJxv31bF2ZQEx1oH/ugVTs3nHsxjPnn+r19RoAo0prczVGf2o2bke4tN5rXMuXgln60A/SdjzAjx/G2TOgVvehTO/C7FH/ij35MflSNQmrC/LCtLY4F2BtbVStaRrNIHGXQG2OMiap+5r5c3R6emAfS/D/It4bW8TOSlxLMwNvMeNiQ70oaR2FyDgCy9C9oKjntrn8fLUx+WcPNtJQfrgN4NFeclslCvwYoG9unyjCQKuCkjJhyQjK22rD+16wp0Dr0FvOz3zLuXdAw2cOT8bYzxrUNCBPpS4K9UfyiiUMhv31VPj7urfhPUlIdZGZnYeB2IXwL5/B2KlGs1gXBWQnKeUNnEpOqMfiV3rIdHJe71z6ez1BLU+DzrQhxZ3NSTnjOrUxz8qw+mwc8b84bvolhWk8GLPMqjZAS1l/lylRnM4rkpIMUqISdk60B+N7jbY/xosuITX9jaSZLdx/IyMoC5BB/pQ4q5SWdEImJuwVw/ZhPVlWUEq/+peru7se9mfq9RoBuPphdZqSDF+d5OyoV2Xbo7I/legrxPvgst4fU8dp8x1EmsLbujVgT6UuKuGlVAO5akjbML6srQglVKZgztphtLqajSBorUakKpGD5Do1Bn90di5Hhw5bLPMp6GtO6hqGxMd6ENFdxt0u0YM9KoTtpyThtmE9WV2loOEWCufJZ4Ah95XRmgaTSAwpZXJPhm9llcOT5cLijbAgkvZsKc+aCZmQ9GBPlS0GpOhHEcP9OYm7HXDbML6YrUIFuel8EL3MeDtgwMb/LVSjWYwZrNUf40+C3raoKc9dGsKV/a+BJ4eWHQ5G3bXBs3EbCg60IcK0xBqhIz+iRE2YX1ZVpjKC/VTkEnZWn2jCRym/UF/jd743dRZ/eHsWg8pBRyMW0BRXRtnBsnEbCg60IcKd5W6PUqgr2rp5K19daxdmX/ETVhfluWn0u2BxrzT4cDrymhKo/E37kqIT4PYRHW/X0uvA/0gOpqg+E1YeCmv71E/m2DLKk10oA8Vowj0ZifsNccevWxjsqwwFYBt8auhp1VNstFo/I2rApLzB+4nOtWt9rsZzN4XVRl1oSrbBNPEbCg60IcKd5XKimLih324z+PlqS0jb8L6MiU5jiyHnVc750FMou6S1QQGsysW2FTSiCfRLN1o5c0gdq6HtGk0pSxky6GmkGXzMIZAL4SwCiE+FUK8aNyfLoTYLIQoEkI8KYSINY7bjftFxuPTArP0CGcEDf3b++updnVx3XFHllQORQjB0oJUtlR2wqzTlZ5em5xp/I0R6HdWurjmj5vYWO4FhLZB8KW9AQ6+Awsv58199XglQRsyMhxjyej/C9jjc/8nwP1SyllAM/Al4/iXgGbj+P3GeZqhtB5dQ//4ZnMTdmy/HMsKUjnY0E779HPUNao+nehKNZoButugqwVS8thb0wpAZWsfJKTrjN6X3c+D9BhqmxpyUuJYlBc8E7OhjCrQCyHygQuAPxn3BXA68IxxyqPApcbXlxj3MR4/QwTTvSdScFeBY3j7g7FuwvqyrMCs068CYdUe9Rr/4h6QVpbUtwFQ39qttfRD2fUsZMymK30+7+wPvonZUEYbRX4F3A14jfsZQIuUss+4XwGYdYg8oBzAeNxlnD8IIcTNQogtQogt9fWT7CNfX7dqGT9C6eapLeV45eg3YX1ZnJ+CELC1TkDhal2n1/gXU1qZnEdJvdLNq0CfpTdjTVpr1LS3RZfzQUkjnb2eoI0MPBIjBnohxIVAnZRyqz8vLKX8o5RypZRypdPp9OdLhz9ms9QwpRuPVxqdsJmj3oT1JTkuhpnOJLaVt8C886FuFzSXTnDBGo1Bf7NUPiUNPhl9YpYu3Zjsfh6Q/WobZWKWHtIljSajXwNcLIQoBf6JKtk8AKQKIczh4vmA8RtAJVAAYDyeAjT6cc2Rj9sM9IeXbjbuqzM2YceezZssK0jls/IW5Nzz1QGd1Wv8hasChAVP0hRKGzoAqG8zMvq2Or35D2oAeNYCvJlz+03M7DbryM8LICMGeinlN6WU+VLKacA1wJtSyuuBt4ArjdNuBJ43vn7BuI/x+JtS6v/9QfR3xR5eunniozIyk+wT+qi3tCCVxvYeKsiGrAW6Tq/xH+5KcORQ4eqhx+Ml1mqhwazR93VBd2uoVxhaXBVQvhkWXs62ihbqW0NjYjaUiejo7wHuFEIUoWrwfzaO/xnIMI7fCdw7sSVGIUdolqp2dfLm3vFtwvqy3NyQLW+BuefDoQ9Ul55GM1Fc5YPq80sLUqhv60aaTVOTfUN217PqdtHlvL67FqtFcOqc4JuYDWVM0URKuVFKeaHxdYmU8jgp5Swp5VVSym7jeJdxf5bxeEkgFh7RtFZDbBLYB8utnvx4/Juwvsyd4sBuswzU6aVHjTLTaCaKoaEvNhQ3x8/IoNcjaY819BaTvU6/cz1MWQIZM5WJ2fR0UhKCb2I2FN0ZGwqMj7/4yK18N2ELMybWJh1jtbAoL4XPylsgZ7m6lvao10wUKY3JUvmUNLSTmhDDnGwHAI2oT5GTWnnTdBCqPoFFV1Da0M6BuraQdsP6ogN9KBhm4Mjb+ye+CevL0vxUdla56JXA3POg6A3o7fLLa2smKe0N4OlWgb6+jRmZiTgddgDqvMan08lcujHLNgsv4/U96pNNqNwqh6IDfSgYxv7g8c0T34T1ZVlhKl29XvbVtMLcC6C3XbVkazTjxW0MHEnJp6S+nRnOJDKTVKCv6k1QDXqTuXSzaz3krYS0qby2u5Z5UxzjkkgHAh3og43XoxoqfKSV5ibsVRPchPVlWb76KP1ZRQtMPwliHdqjXjMxjMlS7XHZ1LV2M8M5kNHXt/UZIwUnaUbfUAQ1O2DR5TS197CltCks1DYmOtAHm7Y6tTnqU7p56uMKvBKuneAmrC8F6fGkJ8ayrawFbHaYdYYyOfN6R36yRjMcRrPUwV7V/DMjM4nkOBuxNovRHTuJA/2u9ep2waW8tbdOmZgtmBLaNfmgA32w6ZdWqtKN2oQt88smrC9CCJbmp6iMHmDeBepjddUnfruGZpLhKgdbHAdalYpkpjMRIQTOJLuP380kLd3sXK8sR1Ly2LC7linJoTUxG4oO9MGmdbCG/u39dVS5urjWT5uwviwrSONAXRutXb0w+yxVQ9XqG814cZuKmw4sgv7ExOmwG92x2crDabJRtwfq98DCy+nq9fDOgXrOXJAVUhOzoehAH2zMjN4YCv745nIyk2IDsju/tCAFKWFHpUsNOZm2RnfJasaPq6K/WaowPaG/rd/pMDJ6s0Y/2cqDO9eDsMCCS/iguIGOHk9YlW1AB/rg464EaywkZFDj6uLNvbVctbKAWJv//yuW+XbIglLf1O+FxmK/X0szCXBVQEoBxfVtzHAm9R92Ouw0mBm9t1f51U8WpFT1+alrwJHNht11YWFiNhQd6ION6UNvsbBhTy1eCVetyB/5eeMgNSGWaRkJqnEKVJcs6KxeM3Y8vdBag0zOo7SxnRmZif0POZPsNLb34EmYhDYINTugsQgWXY7XK3l9Ty2nzAm9idlQdKAPNu7q/vr8wfp24mOsTPf5o/E3ywpSBzL61ELIXqzdLDVjx10FSJpjsujq9R6W0UsJbmuaOjCZNmR3rVd7X/Mv4TPDxCxcumF90YE+2Lgr+wP9ocZ2pmYkBHTTZmlBKrXubmpcRlfsvPOhfBO0a+dozRgwHFcrvMrTZobTJ6M3tPQNpg3CZMnopVT1+RmnQGIGr+9RJmanzQ29idlQdKAPJlIOsj8obWxnWkbgsnlQgR5gW3mzOjD3fJBe2P9KQK+riTKMZqmi7hRg+EBfI9Vjk8bvpqkEWg7BvAsB2LC7luOmhYeJ2VB0oA8mnc3KKyQ5D49XUt7UydTMwLZIL8hJJsYq2FbuUgdylkJyvq7Ta8aGEeh3tjpw2G04DesDoP/r6i67EhpMltKNKWrIXsihxnb214aPidlQdKAPJubAEUcO1a5OejzegGf0cTFW5uckD2T0QgyYnPV0BPTamijCVQHxaexr9jAjK2lQudH0u6lv6zFGCk6SjL6xSN2mK0tiQAd6DYO6Yg81qiA71Y/dsEdiWUEqOypceLzGoK9550NfJ5RsDPi1NVGC2SxV387MIeKB+FgrDrttYEj4ZAn0TcVgT4HETDaEmYnZUHSgDyb9IwRzKW1UE3oCndGDsixu7/FQVKeGRTD1RDX0RJucaUaLq4I+Rx7Vrq5B9XmTge7YSRToG4shYwbNHb18XNoUttk86EAfXNzVqoMuKZtDjR3E2ixMSY4L+GWXFRpOlqbM0harLBH2vaLcNDWakXCV44pRgcxXWmmSaXbHJmVNnhp9UzGkz+TNfhMzHeg1oEo3SdlgtVHa0M7U9AQslsD7YUzPSMQRZ+PTcp+OxbnnQ0cDVHwc8OtrIpzuVuhyUSsOl1aa9HfHJmap36toTyB6u6ClHDJm8vqeWrKT7SzOSwn1qo6IDvTBZJCGvoOpQSjbAFgsgmUFqQMZPaiM3hIT2SZnZZvgJ9O1pUOgMeyJS3vTEWL4cuMgB0vphY4o79NoLgUkPakzeHt/PWfOzw4rE7Oh6EAfTFpVV6zXKznU1M60IGzEmizNT2VfbSudPUamFZcC006MbJnl1kegswm2PRbqlUQ3xmSpvZ0p5KXGExdzeHu/02GntauPnvhMdSDa6/RNKrnY3pFhmJiFb9kGdKAPLu4qcORS19pNV6+XqQG0PhjKsoJUPF7JzirXwMF5FyiJWFNJ0NbhN3o6YM+/1Nfbn5p8jonBxNDQb3cnDlufh4GmqWZhdsdGeZ3ekFa+VJVAYqyV1TMzQrygo6MDfbDockO3e4jiJogZfcGQDVlQgxIAKrYGbR1+Y/8r0NMGy29QAzHKPgz1iqIXVyVSWNjaHMfMYerzMAltEBqLkfHpvLi/k1Pmhp+J2VB0oA8WrdXqNjmPQ0GUVpo4HXbyUuMHb8g650FMAlRGYKDf8TQkTYFz/w9iEmH7k6FeUfTiqsCbOIXWnuEVN+DTHesxpipFuw1CUwndKdOpa+3mlDnOUK9mRHSgDxb9zVI5lDZ2EGMV5KQEXlrpy2EbslYb5C6Hyi1BXceE6WiCAxtg8ZVgd8D8i2DXc0oJofE/7gra49UgjaHNUib9fjedNrDFT4qMviVOTYWbeYQ3v3BCB/pg4R4YIXiosZ2CtARs1uD++JcVpFLR3KlkcCZ5x0D1dujrCepaJsTu59WAi8VXqftL1kK3Cw68Ftp1RSuuCpqsKms9UkafnhiLEIYNQrRr6XvaobWKGptS0BWGaTesLzrQBwufEYKlDR1BsT4YyrB1+rwVymitdmfQ1zNudjwNGbOVQRvA9FOUrE+Xb/yPlOCqpNKbSWKslexk+7CnxVgtpCfEDsyOjeaM3hAvlMgp2G2W/k8z4cyIgV4IESeE+EgI8ZkQYpcQ4nvG8elCiM1CiCIhxJNCiFjjuN24X2Q8Pi2w30KE0FoFCRlIm93woQ9efd5kUV4yVosYGEQCkLdS3UZKnd5VAYfeV1m8qVu22mDRlSqj72gK7fqijfYG8HRT3JPCdGfiUbXizkHdsVEc6I2+jd3dTgrTAztPwl+MJqPvBk6XUi4FlgHnCiGOB34C3C+lnAU0A18yzv8S0Gwcv984T2NIKxvaemjv8QRVcWOSEGtjTrZjcKBPyVfdjJWfBH0942LHM+p20RWDjy9ZC54eVdbR+A9XOQC72pOZkXn0WrRzstggGBr6T9rSI6JsA6MI9FJhuGERY/yTwOmA8VfHo8ClxteXGPcxHj9DRMJbXqAxumJNxU0wNfS+mBuyXtPJUghVvomUDdkdz6hPIRkzBx/PWQqZc5WmXuM/DCO+ne3JI246OpN8hoR3Nqk5s9FIYzEyKZv9zYStW+VQRlWjF0JYhRDbgDpgA1AMtEgp+4xTKoA84+s8oBzAeNwFhHc3QTAwJkuVGvbEwZRW+rKsIAV3V1+/lh+A/BXQsB+6XEd+YjhQtwdqdwxswvoihMrqyz6A5kPBX1u0YjRLVXnTh/W48cXM6GWiITdsrw/06kJDYzF9qTNo6+6LnoweQErpkVIuA/KB44B5E72wEOJmIcQWIcSW+voo/YUw6e1S3h9GRm+1CPJS40OylGUFaoDztqEbsgBVn4ZgRWNgx9NqEPOiy4d/3HwD2PF08NYU7bgq8FjjaMYxqkDf3eel027aIERp+aapGHeCklZGVaA3kVK2AG8Bq4FUIYTNeCgfMMzWqQQKAIzHU4DDHI6klH+UUq6UUq50OsO/4WBC9DdLqYw+LzWeWFtoBE+zspJIiLUOVt7kHqNuK8K4fCOlCuAzTlU14OFImwqFJyj1jZTBXF304qrAHZsFCKaPUG401SdN/TYIUZjAdbmgvZ7aGFXAKAzBXtt4GI3qximE+p8TQsQDZwF7UAH/SuO0GwFzF+wF4z7G429KOcn/6oZo6EMhrTSxWgSL81IGZ/TxqUquGM4bsuWboaVs+LKNL0vWqjJU9bbgrCvacVdSZ3GSmxJHQqztqKf2jxT0Gna90ZjRG4qbQzIHgIK0KAn0QA7wlhBiO/AxsEFK+SJwD3CnEKIIVYP/s3H+n4EM4/idwL3+X3aEYQR66cjlYEN7yOrzJssKU9ld7aa7z8cz3NyQDdf35B1Pgy0O5l949PMWXqoGVOtNWf/gqqC8L/2IjVK+mBl9dZ9DHYjGQG9o6Pf0ZOF02ImPDW+PG5Ojv0UDUsrtwPJhjpeg6vVDj3cBI6Rdk4xWFehbbE5au4pDmtEDLMtPpdcj2V3lZnmhqtmTvxK2/7N/NmhY4emFXc+qoeZ2x9HPjU+DOecodc5ZP1Aae8346OtBttawX64ZsT4PA343NZ0WNaoyGjdjjYx+W3sahekxIV7M6NGdscHAXQWxDkrb1I97pFpnoDlstCAoKwQIz8ap4rfUZvbitaM7f8nVylTr4MaALivqaa1GICntS2PGKH5nU+JjiLEK1R2b6IzSjL4YkvMpbvZEzEYs6EAfHPo19EpaGYquWF+mJMeR5bAPrtNnL1Ilj3AM9DuegrhUmHXm6M6ffbYarKLLNxPDlFbKTGZmjVy6sVgEmb6TpqKxO7axCG/6TKpdnRGjoQcd6IODu9pwrWxHCChID4200kQIY7RghY9u3maHKUvCz5u+p12NO1x4qRpqPhpsdlh4mRpM0t028vma4TGapapkxqhq9DAJbBAai2lLmopXRo60EnSgDw7uKsOHvoPclPiwGFKwtCCVgw3ttHT4uFbmrVBa+nAa7Lz3JejtGLZsI6Vkm2+Xry9LrlbPi+RRiaHGsD9osjnJSR6dpfZAd2wUBvqOJuhqod6UVupAr+nH0wdtNf2TpaZlhscvx3LTydI3q89bAb3tUL8vRKsahh1PQXL+wDQsH574qJxLH3yfu9dtp88zZJRgwfGQUqgdLSeCq5JWSzI5mRlYLKNzMRmU0Xe7omtGgDE+sEwoaaUO9JoB2mpBevtr9KGuz5sszk9BCNhW5lOnzzedLMOkcaq9AYregMVXgGXwr6qUkr+8f5DkOBvPbK3g1sc+oavX55OIxQJLroLiN6MvswwWrgqqZcaoFDcmToedxvYevIlGU1s0TZoyFDf7e7OJtVnIigB7YhMd6AON0RXbbs+iqb0nJK6Vw+GIi2GWM4nPKnwCffoMtYkZLhuyu54F6Rm2bPNeUQNFdW1856KFfOeiBby2u5YvPvIxbd19AyctXqveZHeuC+Kiowevq4JDfelHnCo1HE6HHY9X0mpLVwei6U22qRiEhR0dqRSkxY/6U044oAN9oDE3tLzqFz9cMnpQdfrPylvob1zud7IMk0C/4xlwzofshYc99Mj7pWQmxXLh0hxuWjOdX65dyuaDTVz/8Caa2o19h6x5ytVSl2/GhWypoEqOrlnKxOyOHbBBiCKJZWMxpBZysLk3oso2oAN94DG6Yku6VVt4qLtifVlemEpjew+7qtwDB/NWQu1u6OkI3cJAOVCWb1LllyEu16UN7by5r47rVk3t39i+/Jh8/vC5FeypaWXtHz6k2tWpTl5ytdpgrt8f7O8gsuluxdrjokpmjrl0A1Dbb4MQRRl9YxEyfSZljR060GuG4K4Cq50DraqLLpx+QS5cnEtynI2fv+az+Zq3QpVLqj8L3cJgwIFy0ZWHPfToh6XYLILPrSocdPzMBdn87YvHUePq4sqHPuRgQ7saUCIsalNXM3pc6pNo9RiklTDQHVvVazwnWgK9lNBUQk/KdFq7+yJKQw860Aced5XS0Dd1MiU5Lqy8MVISYrj1tFls3FfPh8WGwahpWRzK8o3pVFlwvHKk9KG1q5ent1RwweIcsoaR/B0/I4Mnvnw8nb0ervr9B+xqjVeOl9ufBK/3sPM1R8BolupMyCHJPnobCTOjr+uQyo4iWko3bXXQ00aDXdmDhFPCNhp0oA80/Rr60LpWHokvnDCNKclx3PfKXlWrT3JCamFolTe1O6F+ryrbDGHd1grauvv4wprpR3z64vwUnrplNbFWC9f8YRPFORco58vyzYFcdXThVoE+NqNwhBMHk2i3kRBrVRLLxKzoUd0Y4wMrRC4QOfbEJjrQBxrD/qC0sSOs6vMmcTFWvn7WbD4rb+HVXTXqYKg3ZHc8DRYbLLhs0GGvV/Loh4dYXpjKMqMP4EjMykri6a+egDPZzpUbM/BY4/Wm7BiQLeV4EKRnF4z5uVHZHWto6A94lGw0UuyJTXSgDyRSQms1PQlTqG/tZmqYNEsN5Ypj8pnpTOSnr+5TjUd5K1QGHIrBEV4v7FgHM8+AxMETKN/eX8/Bhna+cMK0Ub1UXmo8T9+ymrzsTF7sOYae7eugrzsAi44+upvKqZHpTMs6+hvqcDij0e+msRgsMexuTyEzKZbEMZSzwgEd6ANJRyN4emiyqtFq4ZjRA9isFu4+dx4l9e08vbVCKW8gNFl92YeqbDDMgJG/flBKlsPOeYtyRv1yGUl2nvjy8ezKPI/YXjcbX3zMn6uNWnoay8bcLGXidEShDUJTMaRNo7S5J+Lq86ADfWDp19Arz/dwrNGbnL0gm2MKU7l/w346Mxaq2ayhCPQ7noKtrAIXAAAgAElEQVSYRJh3/qDDRXVtvLO/nhuOnzrmMYyOuBju/MrNuC2pdGx9gt+8cYDJPvRsJIS7kiqZwczM0StuTJwOu7IqTspSlhrRYCzXWAIZMylrijxpJehAH1jcqiu2pEdpisOpWWooQgjuPW8+da3d/HVLHWQtCP6GbF8P7HoO5l0AsYN/Vo9+UEqs1cK1q8a2OWgSZ7eTuPIazrZ9ysMbPuWH/94zvBmaBqQkvrOGWpFJXtrYnVadSXZaOnrpjTdmQUf6hqzXC03FeNJnUtXSqQO9ZghGRr+v3UFmkn1MMrVQcNz0dM6Yl8VDG4vpzl6uMvpgZr5Fr0NXy2FlG1dnL+s+qeDiZbn9nZfjwbr0amyylx/OKebP7x0c3gxNA+0N2GQPPQm5WMfR5m9KLN1WY3pZpJdvWqugr4uWuAK8kojT0IMO9IHFXQXCyg6XPWw8bkbirnPn0tbdxwZ3npp4b8zIDAo7noKEDJh52qDDT28pp6PHM+pN2COSuxwyZnOReJc7zpzdb4bW06eD/SAMe2JL6vhGSppvxg1EiQ2CYWZWZTWklTrQawbhrgLHFEqbusO6bOPLvCnJXL48n98XGdlYRZDKN92tsO9lNTDEOjCL0+OVPPphKcdOS2NRXsrEriEELLkaceh97lgZ32+G9ts3D0zsdaOMvmYV6BOypo5w5vD02yB4osQGwdDQF3mygcjT0IMO9IGltQpv0hRq3F0Rk9EDfP2s2RTJfLot8cHbkN3zIvR1HeZU+caeWsqbOrnpKA1SY2KxYamw42luWjOdy4/J48GNxYPn505yWmoOApA+Zca4nm8G+qreBEBEfqBvLAZbHHs7HMRaLWQ7RjeEJZzQgT6QuKtoj1NZwNQQDwQfC/lpCXxu9XQ+7ZtGZ+lHwbnojqdVR27BcYMOP/JBKbkpcZy9INs/10mfrqwVtj8JUvKdixbiTLLz309/NtjPfhLTVldKp4wlPy9vXM/PSFIjH+vaPZCYGR2lm/QZlDd3kZ8eWfbEJjrQBxJ3lY+GPnIyeoDbTpvFbjEbW93OwDcZtdVByVtqE9bHqXJvjZsPihu5YfU0bFY//qouWassFmq2kxIfw0+uXEJRXRv3b9AOlwCe5go1JzbLMa7n221WUhNiBpqm2kPQeOdPmlSgj1RpJehAHzi63NDTRrU0fOjTIyejB0hLjCVn4YnE0MvubR8E9mI716sBIUPKNo9+UIrdZuGaY8fehn9UFl4GlhjYrhwtT5nj5NrjCvnjuyVsKW3y77UiEFtbJQ1WJynxMSOffAT6u2MTnZGd0Xs90HRQaegj0J7YRAf6QGH40Jf2pJCWEENKwvj/aELFqaefC8D7b78W2AajHU9D9mI1KMSgub2HZz+t5LLleaQlxvr3egnpMPtsdV1jEPq3L5hPXmo833j6Mzp6+kZ4gegmqbuW9rgpE3qNgaapCLdBaCkDby8djum4u/p0oNcMwdDQ7+90RIziZijxGYV02J2kt2zn9T0B+mNtLFaNWUOcKv/5cTldvV6+sGZaYK67ZK3KNA++DUCS3cbPrlxKaWMHP30ljIajB5u+HtK8TXgd46vPmxxmgxCpnciG4qbaqn4ekaihBx3oA4eR0W93JUVcfb4fIYibeizH2g7y01f2Bqa5aOc6QKgBIQZ9Hi9//7CU1TMymDcl2f/XBJhzLtiT+8s3AKtnZvCFE6bxyAelfFDUEJjrhjnu+jIsSGLSx9eBbDJgbJYFnm7VkxGJNKo+koNeQ1qpA71mEMZQ8B2t8RGb0QNY8ldQKCupratl/SeV/n1xKVWgnboGUgaac17bXUuVq4ubApXNA8TEwYJLYPcL0NPef/iec+cxPTORu57ZTmtXb+CuH6ZUlyk73uTsaRN6HafDTkePhy67EiNEbPmmsQhikzjQof6GozajF0IUCCHeEkLsFkLsEkL8l3E8XQixQQhxwLhNM44LIcSvhRBFQojtQohjAv1NhCXuSvriM+iWMUwLU3viUZGvnCwvza7l/tf3+1eCWPUpNB44rGzzyPul5KfFc8Z8P0kqj8TyG5Tp1qaH+g/Fx1r5+VVLqXZ18uOX9gT2+mFIc7XKYJ35Myf0OmZ3bIvFaLyLVL8bU3HT3ElGYmzY25gcidFk9H3Af0spFwDHA7cJIRYA9wJvSClnA28Y9wHOA2Yb/24GHjr8JScB7io67IaGPoIzenKXA/Cl6U1Uu7p49INS/732+7+CWIfKrA12Vrr4qLSJG1dPG5fPypgoXAXzL4J3f9E/IxVgxdQ0vnzyDJ74qJy39kVogBonXQ1lAEwpmFigN5umGjC7YyNUedNYDBkzKW/qiNhsHkYR6KWU1VLKT4yvW4E9QB5wCfCocdqjwKXG15cAf5OKTUCqEGL0BuLRgruaZpty7wtXH/pREZcCmXOY2rmXU+Y4+d3GYlwdfihp1O6C3c/D8V9Rs0UNHvmglPgYK2v9Lak8Emf/SEk7N/y/QYe/fuYc5mQnce+67f75fiMFVwUu4cAWN3Z7Yl/MQF8dyTYIfT1KdZMeufbEJmOq0QshpgHLgc1AtpSy2nioBjA/Z+cB5T5PqzCOTS7cldSShiPORloESisHkbcSKrZwzzlzcXf18tDbxRN/zbd/orL542/tP9TQ1s0L26q4YkXehDTcYyJtKqy5Q20Kl77XfzguxsovrlpGQ1sP3/vXruCsJQyI66zGFTPxklm/DUJ3nBoLGYmBvuUQSA+etJlURqg9scmoA70QIglYB9whpXT7PiaVyHpM+ikhxM1CiC1CiC319RHeOTeU3k7obOJQbyrTMhIRIvJapgeRdwy017Eg0c0lS3P56/sHqXF1jf/1fLP5hPT+w09sLqPH4524S+VYWfNfkFIAL98DngEN/eL8FG4/bRbrP60cmKcbxfR5vKT21tKdMPEP4GkJsVgtgvq2XqNpKgIDveFa2WDPx+OV0R/ohRAxqCD/mJRyvXG41izJGLfm/2Ql4Pu5O984Nggp5R+llCullCudTud41x+eGNLKA53JYT1VatTkrVC3lVv477Pn4pWSX70+AbuAYbL5Xo+Xv286xEmzM5k1ztb7cRObAOf8CGp3wta/Dnro9tNnsTA3mW8/u4Om9p7grivIVDR3kkMjpE68bGa1CDISY32GhEdgjd7Q0JdK1TwW1TV6odLRPwN7pJS/9HnoBeBG4+sbged9jn/eUN8cD7h8SjyTA0Nauac9KbLr8ybZi8Bqh8qtFKQncP2qqTy1pZyiunGMiDtCNv/SjmrqWrsDK6k8GvMvhuknw5s/hPbG/sMxVgu/WLsUV2cv//PcjrAbQejP9RyqriFFdBCf6Z/9kUHdsZGoumkshrgUStpVGSoS7YlNRpPRrwFuAE4XQmwz/p0P3AecJYQ4AJxp3Ad4CSgBioCHgVuHec3oxsjoq7xp0ZHR22IhZwlUfgLA106fRUKsjZ+9unfsrzVMNg9qE3ZaRgKnzsnyx4rHjhBw3k+VL/6bPxj00LwpyXz9rDm8tKOGf20Pn5zlnx+VsfKHr/PeAf80d9VXKGll6pSJKW5M+rtjEyN0SHhjEWTMoqy5kxirYEpy5NkTm4xGdfOelFJIKZdIKZcZ/16SUjZKKc+QUs6WUp4ppWwyzpdSytuklDOllIullEEePBoGGPYHNTKdaRFkT3xU8lYo3bunj4wkO18+aQav7qpl66Hm0b/GEbL5beUtfFrWwo0nTAutBWzWfFh1C2x9BKo/G/TQzSfNYFlBKv/7/E7qWiewP+EHvF7JfS/v5d71O3B19nLPuu20dU/cn6e1rhSApHEOHBnKoO7Ytjo1ezWSaCrpV9zkpyUEXu4bQHRnbCBwV9Njc9BOfHRk9KCUN70dyt4X+I+TppOZFMv/vbSHqpbO0ZUQ3v6Jsh0Yms2/f5Aku40rV4xvdJ1fOeUeNc7wpbsH+bPYjBJOZ4+Hb64LXQmnq9fD7U98wu/fLua6VYU89h+rqHJ18pOXx/Hpagg9TUpD79ulPBHMjN6bmAXSA51jSApCTW8XuCqiQkMPOtAHBnclLbZMEmKtOCcwzDqsyDManI2JU4l2G3eeNZcth5o54b43WfHD17nhz5v5ySt7+ff2ag41tg8OhmY2v2pwNl/n7uLfO6q5ckU+jrgwkKHGp8KZ34XyTYN8cABmOpO4+9x5vLG3jme2VgR9aQ1t3Vzzx028vLOGb58/nx9duohVMzK46YTp/H3TITaXNI78IkfB6q7CiwWSJuZcaZKZZKfXI+mINf6/I2lDtvkgIH009PGhXtGEiMx+3nDHXUUd6UyNBmmlSfoM1dhUuQVWqD3461YVsjA3mc8qWthZ6WJnpZuH3ymhz6sCvCPOxsLcZBblpnBT1XeZEuuAVV/F6vOy/9hcRp9XcmOwJZVHY9n1sOUvsOF/Yd75YB9QAd10wjRe3VXD9/+1mzWzMslNDU4AOFDbyk2PfExDWzcPXb+CcxcNBONvnDOH1/fUcs+67bz8XycTH2s9yisNj7url7S+WtoTsnBY/RMWTC19syWNJFCBPnuBX1474DQqz5+2pKm0dNRHtLQSdKAPDO4qyjyLIte1cjiEUHV6Y0PWZGlBKksLUvvvd/d52F/Txs4qlwr+VW42bXqX/7G9xgN9l/H7+z5iQW4yi3KTWZibwuObD3Ha3Cymh9NehsUC5/8M/nQGvPMzOOv7Pg8Jfn7lUs594B3uWbedv33xuIC/mb9f1MBX/rGVuBgrT968etDPGyAh1sZ9Vyzmuoc388sN+/j2BWMPpiX17eTQSF9Srr+W3R/o670pSm8dSZOmDA19mcgBIj/Q69KNv/H0IttqKelKjmyPm+HIWwF1uwe5PQ7FbrOyOD+Fa48r5EeXLeb529bwwuL38cQ6mHbBN7j62AIsAp7ZWsHd67bT0NYTOknl0chfqTL7D38HDUWDHirMSOBb58/n3QMNPLa5LKDLePLjMm78y0fkpsTz7K0nHBbkTU6Ymcl1qwr583sH+bRs7LXwkvo2ckUjtjT/WU8M2CAYVtORVLppKoaETA61qVw40mv0OqP3N221CCSV3jSWRlNGDyrQSy9UbYNpa0b3nNpdWPY8DyffzSWrF2Hal3m9ktLGdhraejhuevpRXyJknPld2PMveOVeuP7pQfNsr19VyKu7avjxS3vo6vVwxTH5fp2E5fVKfvbaPh7aWMxJszN58PpjSB5hD+Ob583jrb113P3Mdl78zxOx20Zfwimpa+UC0YTN6R/FDfjYIHTGqj6MSJJYNpZAxkwONXUAkR/odUbvbwwNfbVMj86MHvo3ZEfFxvsMpc1XBx22WAQznEnhG+RByQJPuQeKNsD+VwY9JITgZ1cuZVFuCj/89x5W/d8b3PnkNrYeapqwIqer18PXnviUhzYqZc1fvnDsiEEewBEXw48vX8yBujZ++2bRiOf7Ul9bgV30YvVDV2z/euw27DYL9e09AxLLSMHU0Dd1kJYQM6qffzijA72/MQJ9jcyIbB/64UjMhNSpow/0NTthzwuHKW0iilW3QOZcldX3DtbPT0mJ46mvrOaVO07immMLeG13LVc89CHnPfAuf990aFyDSxraurn24U28tLO6X1kTYx39n+lpc7O4/Jg8frexmF1Vo5/q1F7nX2klqDdDp8MeeTYI3W3QVgPpMyiPcNdKEx3o/Y0R6JutGWQ7IreT7ojkrRh9oO/XzX915HPDFWsMnHcfNJfCh78d9pR5U5L5/iWL2PytM7jv8sXYrIL/99xOVv34Db65fgc7K0cXcIvqWrnsd++zp9rNQ9ev4MsnzxjXRu//XriAtIRY7n5mO72jGP/o8UpwG3LRFP8azQ4E+uzI2YxtUh3CZChpZaSXbUAHev/jrqRHxJKSnhXaLs9Akb8SXOXQOkJ2Fg3ZvMnM02HehYcNKBlKot3GNccV8uLXTuL529Zw4ZIcnv20ggt/8x6XPPg+T20pp7Nn+AldHxQ1cNnvPqCzx8uTN68eJJ8cK6kJsfzw0oXsqnLzx3dKRjy/qqWTLK8RhFP8OwfAmWTaIDgjJ6M3pJV9qTOobI5se2ITHej9jbuKepHB1MyJDW4IW8w6fdUnRz8vGrJ5X8758bADSo7E0oJUfnrlUjZ/60y+e9ECOrr7uPuZ7az68et871+7KKpr7T/3qY/L+fxfPiInJY7nbjuysmYsnLsohwsW5/DA6wc4UNt61HOL69vIEY14rHGDhsD4g8EZfcMgG+iwxXCtrLHl0Rfh9sQmkR3oG4vh9e+GlYeGdFdT4UmNLg29L1OWgLBCxVEsjKIpmzdJm6p863eug9L3R/20lPgYvrBmOq99/WSeumU1p83L4rFNZZz5y3e4+g8f8s3127l73XZWz8zgma+eQH6a/35vvnvxQhLtVu56ZrsqzxyBkvp2ckUDMiV/kLLIH2Qm2Wnq6MGT6AQkdEysezcoNJaAI4cy4/1RB/pQs+8leO9++NfXwibYe1wVVHmjUHFjEpsA2QuPXqePtmzeZM0dxoCSu8ecmQohOG56Og9cs5wPv3k63zxvHjXuLp74qJxrjxu9smYsOB12vnvxQraVt/DX9w8e8byShjYKrM1YU/3vNeR02JESWq0RZIPQVNxvfQCRL62ESA/0J3wNTv0mfPoPeP428A5f/wwaXi+WthrlWhmtgR4MJ8tPhn9zjcZs3iQ2Ac7+4bADSsZCRpKdW06ZyVv/fSpvfeNUfnzZ2JQ1Y+HipbmcMS+Ln7+2j9KG4RvdSurbybc0IvyouDExtfRNwihHRYLEsrGofyPWZhHkpES+qCKyAz1Qs/wOOO3b8Nnj8NytoQ32HY1YvL2Ghj7ys4AjkrcCulz9tcxBRGs2b7LgkmEHlIwHi0UwPTOwfkhCCH502WJiLBbuWbcd7zAlnEN1zaR6m/2+EQsDgb7WawwJD/cBJJ0tqrxkBPq8tHhsAXoTDiYR/R38bmMRZ9//NuWLb4fT/x9s/yc8e0voNnwMH/oGkRE0s6uQkL9S3Q4t30RzNm9ylAEl4cqUlDj+58L5bD7YxOMfDbZsaOvuQ7TVYEFCsn+llUC/e2t1X4TYIJjJS/rMqNHQQ4QH+gsX5yKB2x7/hO4Tvq5a1nc8Deu/HJpgb2joZXJuRA8pGJHMORCbdHigN7P51VE+VCxrPhx387ADSsKVtSsLOHFWJv/30h4qWzr7jx+sbycX45NJAEs31V1WiEkM/9JN42ANvQ70YUBhRgI/u3Ip2ytc/Ojfe+DEr8NZP4Bd62HdF8Ez9s7ECWFk9HHp/v8IHFZYrJC7fLDyxszmj/+q3yV6Ycmp9xoDSu6Cur3Q0TRoUEm4IYTg/y5fjAS+uX5gcEpJgzIzAwIS6ONirDjibIMnTYUzjUWAwJ2QT3NHb9QE+og3NTt30RT+48Tp/Om9gxw7LZ2L1vynCkSvfkvpnq/4i5p5GgSkuxqPtJCW5T+r17AlbwV8+CD0dYPNHv21+aHEp8JZ31MigN+tUscsMaoxKClr4F9iltKQJznVbaJxPC7F71LGkShIT+Cec+fxnRd28czWCq5aWUBxfTv5FmPmbABKN0Bk2SA0FUNKAeVuJTTQgT6MuOe8eXxS1sy967azIDeZmatvA2FR/iRPfwGueiQowb6rqZwm0piamRzwa4WcvBXg7YWaHWCLU9n8KfdMjmzeZPnnVBmn6aDKVNvr1G1bHbTWqJ9NW50aozcUq10FvtSpyvs+SAM5bjh+Ki9ur+IHL+7mlDlOSurbODPODTHpSlUUAJxJdurbuiE1C+r3B+QafqOxGDKUxw1Eh7QSoiTQx1gt/Pa6Y7jg1+9y6z8+4bnb1hB//FdVY8/Ld8FTn4e1j6rMM4D0NldQE+2KGxPfDdnS9yZXNu9L3oqBbuHh8HrVrNS22sFvBG21yvul6A3428Vw08uQOTvgy7VYBD+5YgnnPfAu335uJ5XNnUyPaQpI2cbE6bCzu8oN+dnqdyVckVJl9Iuu7NfQF0bJ33JE1+h9yU2N5/6rl7G/rpX/fX6nOrjqZrjgF7D/ZXjyhsPcB/2Ou5rqaNfQmyTngiMHtj0+uWrzY8VigcQMlbHPOBWWrIUTboezfwCX/R6+8G913qMXDZhpBZgZziTuPGsOG3bXsrvazRQaAx7o61u7Vdmqs1mV+8KRjkYlGzbsiVOjwJ7YJGoCPcCpc7O4/bRZPL21gqe2lKuDx/4HXPgrOPAqPHl94IK9lMR11lBHOnlpUSyt9CVvBVRvm7zZvD9wzoHPPw99XfDoxdAS2IlVJl86cTpL85W2Pa23LqCBPjPJTmt3Hz3xmepAuLpYGuMDleImOszMTKIq0APcceYcTpiZwf97bid7qt3q4Mqb4OLfqI/J/7wWejuP/iLjoctFrLeTzvjsgHU5hh1myUJn8xMjeyHc8Bx0uVWwN2S6gcRmtfCzq5ayPMtKbF9rwDN6AFe/DUKYKm+GaOijpT4PURjorRbBA9csJzk+htse+2Rg+MMxn4dLHoTit+CJa6Cnw78XNv84HYFRLoQlCy+FhZfD8VGumw8Gucvgc+tUtvu3S4ISDOdkO3j2c8bowAApbmAg0DdgdMeGa6BvLAZhxZNSSEVz9GjoIQoDPahfrN9cu5zSxvZBmmGWXw+XPgQlb8Pja4865HqsSCPQ29MnUaBPnwFX/VVJDTUTp+BYNZvWVQF/u1Rp8wON6a8fAPsDE7M7ttZjBvowlVg2FkHaVGraPfR6osOe2CQqAz3A8TMy+MY5c3lxezV/33Ro4IFl18Llf4RD78Nja9XYMD/Q3qBqq44s/w1X1kxCpp4A1z6hgs7fL1XeK4HEZexl+XmylC9ZRkZf2edQB8LV78Z0rWw0FDc60EcGXzl5JqfNdfKDF3fzWbnPH8yStXD5w1D2gcrs/dBB665TbybOHB3oNRNkxqlwzWNQuxv+cYXy1QkUrgolQ04a/0SrkUhPjEUIqO0A7CnhWbqRUtkfZMzs19BPqkAvhPiLEKJOCLHT51i6EGKDEOKAcZtmHBdCiF8LIYqEENuFEMcEcvEjYbEIfrl2GVmOOG597BNcHT4BffGVqoxz6H3Y/PsJX6unqZx6mUJhlt6U1PiB2WepRr+qT9UnTz+WGQfhrlQyWWvgWmpsVgsZibHh3R3bVgu97f3SSmuU2BObjCajfwQ4d8ixe4E3pJSzgTeM+wDnAbONfzcDD/lnmeMnLTGW3163nLrWLv776W2DbVqXXA2zz4GN901Y6SDd1dTINArSJ4m0UhN45l8IVzwM5ZvgiQCpxVwVAVXcmGQm+YwUbAtDeaUxJ5b0GcqeODU67IlNRvxOpJTvAEN3hS4BHjW+fhS41Of436RiE5AqhMjx12LHy/LCNL51/nxe31PHw+/6NKUIAef9BLx98Oq3J3QNe0cNLTYndpt1gqvVaHxYdAVc8js4+I7q8O7r8e/ruyoCWp83cTqMIeFJYTokfJCGProUNzD+Gn22lLLa+LoGyDa+zgPKfc6rMI6FnC+cMI3zF0/hp6/u46ODPu9b6dPhxDuV42XJxnG/vqOnjq74wNU5NZOYZdfChffDgdfgmZv858rq9arSTRAy+kFDwsOxRt9UDNZYZWgWZRp68MNmrFTaxTH7swohbhZCbBFCbKmvD/xHOSEE912xhIK0eL72xCcquzBZ81+QNh3+/Y3xtWf3dOCQbXgdIf/woolWVt6kBp7sfVEN1/HHJLX2OvD0BFRaaWIam8lEJ/S0+r+PZaI0FkPadNp6JY3tPTqjN6g1SzLGrfkWXQn4/tbkG8cOQ0r5RynlSinlSqfTOc5ljI3kuBh+d/0KWjp6ueOf2/CY9fqYOOUg2HgAPvztmF+3tU5JK2PTAp8ZaSYxq26Bs74PO9fB87cPP7P3SHQ2w8F3YdND8Nxt8IeT4VdL1GOpgVeKOR12evq8dNpNG4Qwy+obi6NWcQPjd698AbgRuM+4fd7n+O1CiH8CqwCXT4knLFiQm8z3L1nIPet28Os3DvD1s+aoB2afBfMuhLd/BovXQuros5y6qhIcgCOrMDCL1mhM1vyX+tT51o+U9faFvxrsa+/1KnO02h1qGEztTnXrrhg4J9EJ2YuU6V/OMph5esCXbXbHtljSSAC1IZs2LeDXHRVeLzQfhFlncCgKNfQwikAvhHgCOBXIFEJUAN9BBfinhBBfAg4Ba43TXwLOB4qADuCmAKx5wqxdWcDmg038+s0DrJyWxkmzjU8U594HDx6nfOyveWzUr+eqVRr6jNwZgViuRjOYk+9SCpz3fqnuZy8aCOh1u6HXKIsIq7I+LjwepiyC7MUwZTE4so/82gHC7I5tkCnkQnhtyLorlalcxqzJm9FLKa89wkNnDHOuBG6b6KICjRCCH166iJ2VLv7ziU/5603HsawgVWXxJ98Fb3wPDmxQWf4o6GpU2VJO/vRALlujUQgBZ/yvyuw3PaiOxaWoQH7M51Xgn7IInPNVWTIMMDP6Gm8KSyC8Ar0prcyYSVlFB8lxNlISosOe2CQqBo+Mh4RYG3+4YSU3/HkzV//hQ36xdikXLsmF1bcrj/WX7oJbN43qD8XrqsRNIsmJk2CylCY8EALO+REsvRri05VyJsijCceCGegreoxZDeFkVezjWlnWVBk1w0Z8iZ6OgHEwPTOR529bw+K8FG5//FN+88YBpDUGLvi5qtm9/8CoXifW0NBrNEFFCMhZqj6JhnGQB0iJjyHGKqhr96o3prDK6EvAFg+OHMqjUEMPkzzQA2Qk2Xnsy6u4fHkev9iwn68/uY2ugpOU/e57v1TzQEcgsbuOzrisIKxWo4lMhBBKYhmOWvompbjxIKho7ow6DT3oQA+A3WblF2uXctc5c3luWxXXPbyJphO/AxYbvHyPMjw6Am3dfThlI96k3CCuWKOJPAa6Y7PCK9A3FkH6DGrdXfR4vDqjj2aEENx22ix+d/0x7K52c3V8jd0AAA+VSURBVNGjJdStuFONINz30hGfV1bXTCYubGlh0QCs0YQtA92xYWRs5umD5tJ+6wOIPsUN6EB/GOcvzuGpW1bT6/Fy9gfzaEuZDS/fe8ROvprKQ1iEJNGp7Yk1mqORaXTH9pdujvJJOWi4ypTXVboO9JOOJfmpPH/7GnLTk/lS/TXgKkO++4thz20xNPTpU3Sg12iOhtNhp7GtG29iFvR1Qo9/hv5MiEbD5NDQ0FsE5KZGnwOtDvRHICclnqe/spqUeaeyznMinvceoLdu/2HndRiTpeIyAu8XotFEMk6HHa+ENpsxsyEc6vS+GvqmDnJT44mJIntik+j7jvxIot3G7z+3gspjv0WH18beP9+Cq2OwTaynxbDySdabsRrN0TC7Y5stZqAPgzp9UzHEOiDRGZX2xCY60I+AxSL4z4vXULzoDhZ3f8Kvfv0zShsGpv3Y2mvoEXaI0wOyNZqjYTZN1UvjbyUsMvpiyJgBQkSthh50oB81yy//Bu1pC/hK15+49sHX2VzSSEdPHym99XTEZYd9w4pGE2rMQF/tMYaEh0OgbyqGjFm0d/fR0NYTlRp60IF+9FhtJF7+ANk0cYftWT7358088PoBpogm+hK1D71GMxKZRummsjsBhCX0pZu+bmgpg/SZlDdHr+IGdKAfGwXHwfIbWOv5F5flt/GHd0qYIpqwpmoNvUYzEol2G4mxVura+pRVcqg86aWEPf9SnvzSC3nHUBal9sQmOtCPlTO/i4hN4r64R7nx+HymiGYSnVpxo9GMBqfD0NInhqg7tuRt+NMZ8OTn1JSuqx6FOedGtYYeJrF75bhJzIQzv4Plxa/zvbnvAh5I04FeoxkNToedhlB0x1ZuhTe+r+ZCJ+fBxb+BpdeBVYXA8qYOHHYbqVFmT2yiA/14OOZG+OTvyrcetLRSoxklmUl2DtS1gTMb6vcF/oL1++DNH8KeFyAhA875Maz80mH242XGQHARpaIKHejHg8UKF/wCHjZGsOmh4BrNqHA67HxQ3AhJTpXRSxkYxVpLOWy8Dz57HGIS4JR7YfVtEDf8zIiypg5mZzn8v44wQQf68ZJ3DKz8Imz9K6TqWbEazWhwJtlxdfbSl+DE5u1VQ8sT0v13gfYGePcX8PGf1P1VX4WT7lQl1yPg9UrKmzs5c37wRywGCx3oJ8K596nRbUf5JdJoNAOYWnq3NYN0UJOm/BHou9zw4W/hwwfVzNxl16ksPnXk/bO61m56+rxRq6EHHegnhi0WcpeFehUaTcRgBvomkaICfVstOOeO/wV7u1T2/u4voLMJFlwCp/0POOeM+iWiXXEDOtBrNJogYgb6Om8qs2BiEsu9/4aX7gZ3Bcw8XQ1Mz10+5pfRgV6j0Wj8iBnoq/qMTdHxBPq2OnjpLtj9HGQthMseguknj3tNZVFsT2yiA71GowkaGYmGDUKXHSwxY9PSSwnbHoNXv63q8Kf/D6y5A6wT076XN3WQkxJPrC16+0d1oNdoNEEj1mYhLSGG+vYxzo5tOggv3qEangpXw0W/HlMd/mhEsz2xiQ70Go0mqKju2J7R+d14PbDpIXjrRyCM/pUVXwSL/7LvsqYOTp+b5bfXC0d0oNdoNEGl3+8mORtaq458Ys1OeOFrUPUJzDlPBfkU/xoIdvT0Ud/aTWGGzug1Go3Gb2Qm2fm0rAVys6D6s8NP6O2Cd34G7/9KDfS58i+w8PKAdNCWN3UCRLWGHnSg12g0QcaZZKe+tRuZmIVor1flGYtVPXjoQ5XFNx6Apdcqbxp/ds4OYTJIKyFAgV4IcS7wAGAF/iSlvC8Q19FoNJGH02Gns9dDT3wmdumBjiaw2ZVJ4Md/UpYin1sPs8446uvsr23luoc3M9OZyNkLp3D2guwxZ+Y60I8TIYQVeBA4C6gAPhZCvCCl3O3va2k0msjD1NK3WNPJBtj2D/joYXBXwfG3wmnfBnvSUV+jz+Plrqc/o9fjpaWjlx+8uJsfvLibhbnJnL1gCucsymZutmNEN8rypg6S7DbSotSe2CQQGf1xQJGUsgRACPFP4BJAB3qNRjNggyBTVKB//buQtQDW/g3yV47qNf703kE+q3Dx62uXc/HSXEob2nltdw2v7qrlV2/s5/7X9zM1I4GzF2RzzsIpLC9Mw2o5POhHuz2xSSACfR5Q7nO/AlgVgOtoNJoIxAz05THTmD9lCcy/SDU+2WJH9fyiulZ+uWE/5yzM5qIlyiJ8WmYiN588k5tPnkldaxev767j1V01PPJBKQ+/e5DMJDtnLcji7IVTOGFmBnab2hMoa+pgpjMxMN9oGBGyzVghxM3AzQCFhdrmV6OZLDiNIeFV3XHwlXfH9FyPV3LXM9tJiLXyg0sXDZuJZzniuG5VIdetKsTd1cvGffW8uquGF7ZV8cRH5STZbZw618n/b+/eY6Q6yziOf59llyWWLYXucicFlHCLheKWUKUN2IZSQkEqQWqjKCamsST2D4PEGtKY9Ia1MRqjqYqitpZIrZIKAl4SUwOUFrkWKAsst3BfCgWsXPbxj3OGDLMzs7O7M+fMHn6fZDJnzvsezsM773n2zHvemfPg6L4cbrrE5OF1Rfl/lbNSJPqjQPpvgw4M193A3V8GXgaor6/3EsQhImWo58e60qXCgrn0bbT0rQP859AH/PALY+ld063V+rd2q2LGmP7MGNOfj65cY/2+M6zZeZx1753gzW3HgORfiIXSJPpNwDAzG0KQ4OcCXyzBfkSkE6qoMGq7d+XUh21L9PtOXeDFtXt4YGQfZo5t++07u1V1YfKI3kwe0ZtnZjmbD51lU2MTD49J/q1Ai57o3f2qmS0A1hBMr1zq7juLvR8R6bzqaqo5feFywfWvNTsLV2yjurKCZ2dlH7Jpiy4Vxt2De3H34NLN0S8nJRmjd/dVwKpS/Nsi0vnVhl+aKtSv/n2Adw+e5aU5Y+h9a+tDNnKj5P4up4iUrbo2JPoDpy/y4to9fHZEb2bdVdzfurlZKNGLSOSCoZv/0dycfx5Gc7OzcMVWqrpU8OysTyZ+vnupKNGLSOTqaqq52ux88N8reestW9/IpsazLJ4+ir49NGTTXkr0IhK51Jem8g3fHDxzkSV/3cOk4XXM/tTAqEJLJCV6EYlc6ktTuRJ9czjLprLCNGRTBEr0IhK562f0Fz7KWv67jQfZeKCJ704fmeibdkdFiV5EIpdv6OZw0yWeX72be4fVMqd+UItyaTslehGJXPfqSrpVVbRI9Kkhmwoznv/8nRqyKRIlehGJnJkF947NSPSvvn2I9fvP8J1pIxmgIZuiUaIXkVjUdr/xZxCOnL3Ec6t2MfETtTw6XkM2xaRELyKxSP92rLuz6PXtADz3iGbZFJsSvYjEoq6m+vpPFb+26TBvNZxm0bSRbb7vq7ROiV5EYlFXU03TxcscPHORZ/6yi3uG3s5j43UTolJQoheRWKSmWD7x6maa3Vky+04qstzXVTpOiV5EYpH6duyOo+f59tQRGrIpISV6EYlF6ox+/JBefGnCHTFHk2yx3RxcRG5uo/v3YP5nhjB/4mAN2ZSYEr2IxKJrZQWLHx4Vdxg3BQ3diIgknBK9iEjCKdGLiCScEr2ISMIp0YuIJJwSvYhIwinRi4gknBK9iEjCmbvHHQNmdgo42M7Na4HTRQyn2BRfxyi+jiv3GBVf+93h7nWtVSqLRN8RZvaOu9fHHUcuiq9jFF/HlXuMiq/0NHQjIpJwSvQiIgmXhET/ctwBtELxdYzi67hyj1HxlVinH6MXEZH8knBGLyIieXSaRG9mU81sj5k1mNmiLOXVZrY8LN9oZoMjjG2Qmf3TzN4zs51m9s0sdSaZ2Tkz2xI+FkcVX7j/RjPbHu77nSzlZmY/Cttvm5mNizC24WntssXMzpvZkxl1Im8/M1tqZifNbEfaul5mts7M9obPPXNsOy+ss9fM5kUU2/fNbHf4/r1hZrfl2DZvXyhxjE+b2dG093Fajm3zHu8ljG95WmyNZrYlx7aRtGHRuHvZP4AuwD5gKNAV2AqMyqjzDeBn4fJcYHmE8fUDxoXLNcD7WeKbBLwZYxs2ArV5yqcBqwEDJgAbY3yvjxPMD461/YD7gHHAjrR1S4BF4fIi4IUs2/UC9ofPPcPlnhHENgWoDJdfyBZbIX2hxDE+DXyrgD6Q93gvVXwZ5T8AFsfZhsV6dJYz+vFAg7vvd/fLwGvAzIw6M4Fl4fIK4H4zi+T+ZO5+zN03h8sfAruAAVHsu4hmAr/xwAbgNjPrF0Mc9wP73L29X6ArGnf/F9CUsTq9ny0DPpdl0weBde7e5O5ngXXA1FLH5u5r3f1q+HIDMLCY+2yrHO1XiEKO9w7LF1+YO+YAvy/2fuPQWRL9AOBw2usjtEyk1+uEnf0ccHsk0aUJh4zuAjZmKb7HzLaa2WozGx1pYODAWjN718y+nqW8kDaOwlxyH1xxtl9KH3c/Fi4fB/pkqVMObTmf4BNaNq31hVJbEA4vLc0x9FUO7XcvcMLd9+Yoj7sN26SzJPpOwcy6A68DT7r7+YzizQTDEWOAHwN/iji8ie4+DngIeMLM7ot4/60ys67ADOAPWYrjbr8WPPgMX3bT1szsKeAq8EqOKnH2hZ8CHwfGAscIhkfK0aPkP5sv++MpXWdJ9EeBQWmvB4brstYxs0qgB3AmkuiCfVYRJPlX3P2PmeXuft7dL4TLq4AqM6uNKj53Pxo+nwTeIPh4nK6QNi61h4DN7n4isyDu9ktzIjWkFT6fzFIntrY0s68A04HHwj9ELRTQF0rG3U+4+zV3bwZ+nmPfsfbFMH88AizPVSfONmyPzpLoNwHDzGxIeNY3F1iZUWclkJrdMBv4R66OXmzheN4vgV3u/lKOOn1T1wzMbDxB20fyh8jMbjGzmtQywUW7HRnVVgJfDmffTADOpQ1RRCXnWVSc7ZchvZ/NA/6cpc4aYIqZ9QyHJqaE60rKzKYCC4EZ7n4pR51C+kIpY0y/7jMrx74LOd5L6QFgt7sfyVYYdxu2S9xXgwt9EMwKeZ/gavxT4brvEXRqgG4EH/kbgLeBoRHGNpHgI/w2YEv4mAY8Djwe1lkA7CSYQbAB+HSE8Q0N97s1jCHVfunxGfCTsH23A/URv7+3ECTuHmnrYm0/gj86x4ArBOPEXyO47vN3YC/wN6BXWLce+EXatvPDvtgAfDWi2BoIxrZTfTA1C60/sCpfX4iw/X4b9q9tBMm7X2aM4esWx3sU8YXrf53qd2l1Y2nDYj30zVgRkYTrLEM3IiLSTkr0IiIJp0QvIpJwSvQiIgmnRC8iknBK9CIiCadELyKScEr0IiIJ939iAAFectt/SwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(y_predict)\n",
    "plt.plot(Y_test_k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T04:11:39.804039Z",
     "iopub.status.busy": "2022-05-20T04:11:39.803542Z",
     "iopub.status.idle": "2022-05-20T04:11:40.188527Z",
     "shell.execute_reply": "2022-05-20T04:11:40.187982Z",
     "shell.execute_reply.started": "2022-05-20T04:11:39.804009Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'cplfw_rank': 0,\n",
       " 'market1501_rank': 0,\n",
       " 'dukemtmc_rank': 0,\n",
       " 'msmt17_rank': 0,\n",
       " 'veri_rank': 0,\n",
       " 'vehicleid_rank': 0,\n",
       " 'veriwild_rank': 0,\n",
       " 'sop_rank': 0,\n",
       " 'arch': 'j121221121221221311331321121221000000'}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with open('/home/aistudio/data/data134077/CVPR_2022_NAS_Track2_test.json', 'r') as f:\n",
    "    test_data = json.load(f)\n",
    "test_data['arch99997']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T04:11:40.189989Z",
     "iopub.status.busy": "2022-05-20T04:11:40.189529Z",
     "iopub.status.idle": "2022-05-20T04:11:51.893288Z",
     "shell.execute_reply": "2022-05-20T04:11:51.892501Z",
     "shell.execute_reply.started": "2022-05-20T04:11:40.189963Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3, 2, 2, 1, 3, 3, 1, 1, 1, 1, 2, 2, 1, 1, 3, 1, 2, 1, 1, 1, 1, 1, 1, 2, 1, 2, 3, 1, 3, 1, 1, 3, 2, 1, 3, 3, 1, 0.1411200080598672, 0.9092974268256817, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.8414709848078965, 0.9092974268256817, 0.8414709848078965, 0.9092974268256817, 0.1411200080598672, 0.8414709848078965, 0.1411200080598672, 0.8414709848078965, 0.8414709848078965, 0.1411200080598672, 0.9092974268256817, 0.8414709848078965, 0.1411200080598672, 0.1411200080598672, 0.8414709848078965, 20.085536923187668, 7.38905609893065, 7.38905609893065, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 2.718281828459045, 2.718281828459045, 2.718281828459045, 7.38905609893065, 7.38905609893065, 2.718281828459045, 2.718281828459045, 20.085536923187668, 2.718281828459045, 7.38905609893065, 2.718281828459045, 2.718281828459045, 2.718281828459045, 2.718281828459045, 2.718281828459045, 2.718281828459045, 7.38905609893065, 2.718281828459045, 7.38905609893065, 20.085536923187668, 2.718281828459045, 20.085536923187668, 2.718281828459045, 2.718281828459045, 20.085536923187668, 7.38905609893065, 2.718281828459045, 20.085536923187668, 20.085536923187668, 2.718281828459045, 243, 32, 32, 1, 243, 243, 1, 1, 1, 1, 32, 32, 1, 1, 243, 1, 32, 1, 1, 1, 1, 1, 1, 32, 1, 32, 243, 1, 243, 1, 1, 243, 32, 1, 243, 243, 1]\n"
     ]
    }
   ],
   "source": [
    "test_arch_list = []\n",
    "for key in test_data.keys():\n",
    "    test_arch =  convert_X(test_data[key]['arch'])\n",
    "    test_arch_list.append(test_arch)\n",
    "print(test_arch_list[99499])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-05-20T04:11:51.894916Z",
     "iopub.status.busy": "2022-05-20T04:11:51.894356Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predict the rank of: cplfw_rank\n",
      "Predict the rank of: market1501_rank\n",
      "Predict the rank of: dukemtmc_rank\n",
      "Predict the rank of: msmt17_rank\n",
      "Predict the rank of: veri_rank\n",
      "Predict the rank of: vehicleid_rank\n",
      "Predict the rank of: veriwild_rank\n",
      "Predict the rank of: sop_rank\n"
     ]
    }
   ],
   "source": [
    "rank_all_slow = []\n",
    "rank_all_fast = []\n",
    "for task in range(len(name_list)):\n",
    "    print('Predict the rank of:', name_list[task])\n",
    "    # slow mode\n",
    "    rank_all_slow.append(gp_list[task].get_predict_jiont(np.array(test_arch_list), np.array(arch_list_train), np.array(train_list[task])))\n",
    "    # fast mode\n",
    "    rank_all_fast.append(gp_list[task].get_predict(np.array(test_arch_list)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "base_i = 1\n",
    "for idx,key in enumerate(test_data.keys()):\n",
    "    test_data[key]['cplfw_rank'] = int(rank_all_fast[0][idx][0]*base_i)\n",
    "    test_data[key]['market1501_rank'] = int(rank_all_fast[1][idx][0]*base_i)\n",
    "    test_data[key]['dukemtmc_rank'] = int(rank_all_fast[2][idx][0]*base_i)\n",
    "    test_data[key]['msmt17_rank'] = int(rank_all_fast[3][idx][0]*base_i)\n",
    "    test_data[key]['veri_rank'] = int(rank_all_fast[4][idx][0]*base_i)\n",
    "    test_data[key]['vehicleid_rank'] = int(rank_all_fast[5][idx][0]*base_i)\n",
    "    test_data[key]['veriwild_rank'] = int(rank_all_fast[6][idx][0]*base_i)\n",
    "    test_data[key]['sop_rank'] = int(rank_all_fast[7][idx][0]*base_i)\n",
    "print('Ready to save results!')\n",
    "with open('./result/submit_gpnas_v6_6_fast.json', 'w') as f:\n",
    "    json.dump(test_data, f)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "base_i = 1\n",
    "for idx,key in enumerate(test_data.keys()):\n",
    "    test_data[key]['cplfw_rank'] = int(rank_all_slow[0][idx][0]*base_i)\n",
    "    test_data[key]['market1501_rank'] = int(rank_all_slow[1][idx][0]*base_i)\n",
    "    test_data[key]['dukemtmc_rank'] = int(rank_all_slow[2][idx][0]*base_i)\n",
    "    test_data[key]['msmt17_rank'] = int(rank_all_slow[3][idx][0]*base_i)\n",
    "    test_data[key]['veri_rank'] = int(rank_all_slow[4][idx][0]*base_i)\n",
    "    test_data[key]['vehicleid_rank'] = int(rank_all_slow[5][idx][0]*base_i)\n",
    "    test_data[key]['veriwild_rank'] = int(rank_all_slow[6][idx][0]*base_i)\n",
    "    test_data[key]['sop_rank'] = int(rank_all_slow[7][idx][0]*base_i)\n",
    "print('Ready to save results!')\n",
    "with open('./result/submit_gpnas_v6_6_slow.json', 'w') as f:\n",
    "    json.dump(test_data, f)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}