flair_cadec_v2_no_rnn.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "xONJpNLKtGie"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import os\n",
        "import pickle"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nw6RM8uotGih",
        "outputId": "67f62abb-6dbb-451f-8e2c-ea992d0226fb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount(\"/content/drive\")\n",
        "\n",
        "os.chdir('/content/drive/MyDrive/SI 630 project')\n",
        "# os.listdir('.')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P9LwpBadtGih",
        "outputId": "f0991d5a-c723-4451-8320-f821fd8a9454"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['sct',\n",
              " 'text',\n",
              " 'original - unedited version of edited files',\n",
              " 'meddra - unedited version of edited files',\n",
              " 'sct - unedited version of edited files',\n",
              " 'original',\n",
              " 'meddra']"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "os.listdir('datasets/CADEC/CADEC.v2/cadec')\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "VNZtmxiPb5uV"
      },
      "outputs": [],
      "source": [
        "suf = '_v2'\n",
        "\n",
        "data_save_dir = 'data'+suf\n",
        "\n",
        "suf += '_no_rnn'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "BIUl2-W9tGii",
        "outputId": "b5e662a6-fcdf-4a00-fcf3-138d3cbed042"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-963bb5fa-8d24-4628-8cd6-22d1d5b89888\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>ner</th>\n",
              "      <th>meddra</th>\n",
              "      <th>counts_exploded</th>\n",
              "      <th>counts</th>\n",
              "      <th>counts_exploded_rank</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Drug</td>\n",
              "      <td>Lipitor</td>\n",
              "      <td>1081</td>\n",
              "      <td>1073</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ADR</td>\n",
              "      <td>CONCEPT_LESS</td>\n",
              "      <td>1016</td>\n",
              "      <td>266</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Severe_pain</td>\n",
              "      <td>722</td>\n",
              "      <td>247</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Myalgia</td>\n",
              "      <td>619</td>\n",
              "      <td>305</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Pain</td>\n",
              "      <td>292</td>\n",
              "      <td>260</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1294</th>\n",
              "      <td>Finding</td>\n",
              "      <td>Senility</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1295</th>\n",
              "      <td>Drug</td>\n",
              "      <td>Seroquel</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1296</th>\n",
              "      <td>Drug</td>\n",
              "      <td>Solaraze</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1297</th>\n",
              "      <td>Drug</td>\n",
              "      <td>Tagamet</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1298</th>\n",
              "      <td>Drug</td>\n",
              "      <td>testosterone</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>1299 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-963bb5fa-8d24-4628-8cd6-22d1d5b89888')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-963bb5fa-8d24-4628-8cd6-22d1d5b89888 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-963bb5fa-8d24-4628-8cd6-22d1d5b89888');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "          ner        meddra  counts_exploded  counts  counts_exploded_rank\n",
              "0        Drug       Lipitor             1081    1073                     1\n",
              "1         ADR  CONCEPT_LESS             1016     266                     2\n",
              "2         ADR   Severe_pain              722     247                     3\n",
              "3         ADR       Myalgia              619     305                     4\n",
              "4         ADR          Pain              292     260                     5\n",
              "...       ...           ...              ...     ...                   ...\n",
              "1294  Finding      Senility                1       1                  1233\n",
              "1295     Drug      Seroquel                1       1                  1233\n",
              "1296     Drug      Solaraze                1       1                  1233\n",
              "1297     Drug       Tagamet                1       1                  1233\n",
              "1298     Drug  testosterone                1       1                  1233\n",
              "\n",
              "[1299 rows x 5 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "meddra_counts = pd.read_csv(data_save_dir+'/meddra_counts.csv')\n",
        "meddra_counts"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "xzc4Zo0PtGii",
        "outputId": "b1dd074a-ef87-4144-d14f-131943aefdc5"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-e429d28f-b891-4418-89b6-e3bf4323f520\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>ner</th>\n",
              "      <th>meddra</th>\n",
              "      <th>counts_exploded</th>\n",
              "      <th>counts</th>\n",
              "      <th>counts_exploded_rank</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ADR</td>\n",
              "      <td>CONCEPT_LESS</td>\n",
              "      <td>1016</td>\n",
              "      <td>266</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Severe_pain</td>\n",
              "      <td>722</td>\n",
              "      <td>247</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Myalgia</td>\n",
              "      <td>619</td>\n",
              "      <td>305</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Pain</td>\n",
              "      <td>292</td>\n",
              "      <td>260</td>\n",
              "      <td>5</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Arthralgia</td>\n",
              "      <td>280</td>\n",
              "      <td>163</td>\n",
              "      <td>6</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1268</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Labyrinthitis</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1269</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Dysphagia</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1275</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Lupus_erythematosus</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1278</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Cataract</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1283</th>\n",
              "      <td>ADR</td>\n",
              "      <td>Crying_associated_with_mood</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1233</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>800 rows × 5 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e429d28f-b891-4418-89b6-e3bf4323f520')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e429d28f-b891-4418-89b6-e3bf4323f520 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e429d28f-b891-4418-89b6-e3bf4323f520');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "      ner                       meddra  counts_exploded  counts  \\\n",
              "1     ADR                 CONCEPT_LESS             1016     266   \n",
              "2     ADR                  Severe_pain              722     247   \n",
              "3     ADR                      Myalgia              619     305   \n",
              "4     ADR                         Pain              292     260   \n",
              "5     ADR                   Arthralgia              280     163   \n",
              "...   ...                          ...              ...     ...   \n",
              "1268  ADR                Labyrinthitis                1       1   \n",
              "1269  ADR                    Dysphagia                1       1   \n",
              "1275  ADR          Lupus_erythematosus                1       1   \n",
              "1278  ADR                     Cataract                1       1   \n",
              "1283  ADR  Crying_associated_with_mood                1       1   \n",
              "\n",
              "      counts_exploded_rank  \n",
              "1                        2  \n",
              "2                        3  \n",
              "3                        4  \n",
              "4                        5  \n",
              "5                        6  \n",
              "...                    ...  \n",
              "1268                  1233  \n",
              "1269                  1233  \n",
              "1275                  1233  \n",
              "1278                  1233  \n",
              "1283                  1233  \n",
              "\n",
              "[800 rows x 5 columns]"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "meddra_counts.loc[meddra_counts.ner == 'ADR',:]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hAGPini7tGii",
        "outputId": "94c86e14-81cb-4412-ccf3-00e801f742be"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[Sentence[117]: \"i feel a bit drowsy & have a little blurred vision , so far no gastric problems . ive been on arthrotec 50 for over 10 years on and off , only taking it when i needed it . due to my arthritis getting progressively worse , to the point where i am in tears with the agony , gp ' s started me on 75 twice a day and i have to take it . every day for the next month to see how i get on , here goes . so far its been very good , pains almost gone , but i feel a bit weird , didnt have that when on 50 .\" → [\"i\"/O/O, \"feel\"/O/O, \"a\"/O/O, \"bit\"/B-ADR/B-Drowsy, \"drowsy\"/E-ADR/E-Drowsy, \"&\"/O/O, \"have\"/O/O, \"a\"/O/O, \"little\"/B-ADR/B-Blurred_vision_-_hazy, \"blurred\"/I-ADR/I-Blurred_vision_-_hazy, \"vision\"/E-ADR/E-Blurred_vision_-_hazy, \",\"/O/O, \"so\"/O/O, \"far\"/O/O, \"no\"/O/O, \"gastric\"/B-ADR/B-Excessive_upper_gastrointestinal_gas, \"problems\"/E-ADR/E-Excessive_upper_gastrointestinal_gas, \".\"/O/O, \"ive\"/O/O, \"been\"/O/O, \"on\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \"50\"/O/O, \"for\"/O/O, \"over\"/O/O, \"10\"/O/O, \"years\"/O/O, \"on\"/O/O, \"and\"/O/O, \"off\"/O/O, \",\"/O/O, \"only\"/O/O, \"taking\"/O/O, \"it\"/O/O, \"when\"/O/O, \"i\"/O/O, \"needed\"/O/O, \"it\"/O/O, \".\"/O/O, \"due\"/O/O, \"to\"/O/O, \"my\"/O/O, \"arthritis\"/S-Disease/S-Arthritis, \"getting\"/O/O, \"progressively\"/O/O, \"worse\"/O/O, \",\"/O/O, \"to\"/O/O, \"the\"/O/O, \"point\"/O/O, \"where\"/O/O, \"i\"/O/O, \"am\"/O/O, \"in\"/O/O, \"tears\"/O/O, \"with\"/O/O, \"the\"/O/O, \"agony\"/S-Symptom/S-Agony, \",\"/O/O, \"gp\"/O/O, \"'\"/O/O, \"s\"/O/O, \"started\"/O/O, \"me\"/O/O, \"on\"/O/O, \"75\"/O/O, \"twice\"/O/O, \"a\"/O/O, \"day\"/O/O, \"and\"/O/O, \"i\"/O/O, \"have\"/O/O, \"to\"/O/O, \"take\"/O/O, \"it\"/O/O, \".\"/O/O, \"every\"/O/O, \"day\"/O/O, \"for\"/O/O, \"the\"/O/O, \"next\"/O/O, \"month\"/O/O, \"to\"/O/O, \"see\"/O/O, \"how\"/O/O, \"i\"/O/O, \"get\"/O/O, \"on\"/O/O, \",\"/O/O, \"here\"/O/O, \"goes\"/O/O, \".\"/O/O, \"so\"/O/O, \"far\"/O/O, \"its\"/O/O, \"been\"/O/O, \"very\"/O/O, \"good\"/O/O, \",\"/O/O, \"pains\"/S-Symptom/S-Pain, \"almost\"/O/O, \"gone\"/O/O, \",\"/O/O, \"but\"/O/O, \"i\"/O/O, \"feel\"/B-ADR/B-Malaise, \"a\"/I-ADR/I-Malaise, \"bit\"/I-ADR/I-Malaise, \"weird\"/E-ADR/E-Malaise, \",\"/O/O, \"didnt\"/O/O, \"have\"/O/O, \"that\"/O/O, \"when\"/O/O, \"on\"/O/O, \"50\"/O/O, \".\"/O/O],\n",
              " Sentence[39]: \"hunger pangs . brilliant , i have a new lease of life , i walk up & down steps properly , no longer sideways like a toddler , hip pain as gone other than if i jar it .\" → [\"hunger\"/B-ADR/B-Hunger_pain, \"pangs\"/E-ADR/E-Hunger_pain, \".\"/O/O, \"brilliant\"/O/O, \",\"/O/O, \"i\"/O/O, \"have\"/O/O, \"a\"/O/O, \"new\"/O/O, \"lease\"/O/O, \"of\"/O/O, \"life\"/O/O, \",\"/O/O, \"i\"/O/O, \"walk\"/B-Symptom/B-CONCEPT_LESS, \"up\"/I-Symptom/I-CONCEPT_LESS, \"&\"/I-Symptom/I-CONCEPT_LESS, \"down\"/I-Symptom/I-CONCEPT_LESS, \"steps\"/I-Symptom/I-CONCEPT_LESS, \"properly\"/O/O, \",\"/O/O, \"no\"/O/O, \"longer\"/O/O, \"sideways\"/E-Symptom/E-CONCEPT_LESS, \"like\"/O/O, \"a\"/O/O, \"toddler\"/O/O, \",\"/O/O, \"hip\"/B-Symptom/B-Hip_pain, \"pain\"/E-Symptom/E-Hip_pain, \"as\"/O/O, \"gone\"/O/O, \"other\"/O/O, \"than\"/O/O, \"if\"/O/O, \"i\"/O/O, \"jar\"/O/O, \"it\"/O/O, \".\"/O/O],\n",
              " Sentence[51]: \"no side effects for the first two months . then vaginal bleeding 2 wks after menstral cycle . stomach pain . canker sores in my mouth . headache . been off for 1 week still have bleeding . helped my pain alot . too scared to take this drug again .\" → [\"no\"/O/O, \"side\"/O/O, \"effects\"/O/O, \"for\"/O/O, \"the\"/O/O, \"first\"/O/O, \"two\"/O/O, \"months\"/O/O, \".\"/O/O, \"then\"/O/O, \"vaginal\"/B-ADR/B-Vaginal_bleeding, \"bleeding\"/E-ADR/E-Vaginal_bleeding, \"2\"/O/O, \"wks\"/O/O, \"after\"/O/O, \"menstral\"/O/O, \"cycle\"/O/O, \".\"/O/O, \"stomach\"/B-ADR/B-Stomach_ache, \"pain\"/E-ADR/E-Stomach_ache, \".\"/O/O, \"canker\"/B-ADR/B-Aphthous_ulcer, \"sores\"/E-ADR/E-Aphthous_ulcer, \"in\"/O/O, \"my\"/O/O, \"mouth\"/O/O, \".\"/O/O, \"headache\"/S-ADR/S-Headache, \".\"/O/O, \"been\"/O/O, \"off\"/O/O, \"for\"/O/O, \"1\"/O/O, \"week\"/O/O, \"still\"/O/O, \"have\"/O/O, \"bleeding\"/O/O, \".\"/O/O, \"helped\"/O/O, \"my\"/O/O, \"pain\"/S-Symptom/S-Pain, \"alot\"/O/O, \".\"/O/O, \"too\"/O/O, \"scared\"/O/O, \"to\"/O/O, \"take\"/O/O, \"this\"/O/O, \"drug\"/O/O, \"again\"/O/O, \".\"/O/O],\n",
              " Sentence[68]: \"1st pill taken with food , a few hours after i experienced shortness of breath , a sense of depression , cramping , upset stomache will stop taking immediately . honestly can not recommend this drug , i am still experiencing side effects just after 1 pill and im supposed to take twice / day 75mg . i just cant keep going have to try something else .\" → [\"1st\"/O/O, \"pill\"/O/O, \"taken\"/O/O, \"with\"/O/O, \"food\"/O/O, \",\"/O/O, \"a\"/O/O, \"few\"/O/O, \"hours\"/O/O, \"after\"/O/O, \"i\"/O/O, \"experienced\"/O/O, \"shortness\"/B-ADR/B-Dyspnoea, \"of\"/I-ADR/I-Dyspnoea, \"breath\"/E-ADR/E-Dyspnoea, \",\"/O/O, \"a\"/O/O, \"sense\"/O/O, \"of\"/O/O, \"depression\"/S-ADR/S-Depression, \",\"/O/O, \"cramping\"/S-ADR/S-Muscle_cramp, \",\"/O/O, \"upset\"/B-ADR/B-Upset_stomach, \"stomache\"/E-ADR/E-Upset_stomach, \"will\"/O/O, \"stop\"/O/O, \"taking\"/O/O, \"immediately\"/O/O, \".\"/O/O, \"honestly\"/O/O, \"can\"/O/O, \"not\"/O/O, \"recommend\"/O/O, \"this\"/O/O, \"drug\"/O/O, \",\"/O/O, \"i\"/O/O, \"am\"/O/O, \"still\"/O/O, \"experiencing\"/O/O, \"side\"/O/O, \"effects\"/O/O, \"just\"/O/O, \"after\"/O/O, \"1\"/O/O, \"pill\"/O/O, \"and\"/O/O, \"im\"/O/O, \"supposed\"/O/O, \"to\"/O/O, \"take\"/O/O, \"twice\"/O/O, \"/\"/O/O, \"day\"/O/O, \"75mg\"/O/O, \".\"/O/O, \"i\"/O/O, \"just\"/O/O, \"cant\"/O/O, \"keep\"/O/O, \"going\"/O/O, \"have\"/O/O, \"to\"/O/O, \"try\"/O/O, \"something\"/O/O, \"else\"/O/O, \".\"/O/O],\n",
              " Sentence[50]: \"i have had no side effects been taking arthrotec a little over a year , have not noticed any side effects . it does help alot i noticed that when there are times when i forget to take it i cant stand or walk for any lengths of time .\" → [\"i\"/O/O, \"have\"/O/O, \"had\"/O/O, \"no\"/O/O, \"side\"/O/O, \"effects\"/O/O, \"been\"/O/O, \"taking\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \"a\"/O/O, \"little\"/O/O, \"over\"/O/O, \"a\"/O/O, \"year\"/O/O, \",\"/O/O, \"have\"/O/O, \"not\"/O/O, \"noticed\"/O/O, \"any\"/O/O, \"side\"/O/O, \"effects\"/O/O, \".\"/O/O, \"it\"/O/O, \"does\"/O/O, \"help\"/O/O, \"alot\"/O/O, \"i\"/O/O, \"noticed\"/O/O, \"that\"/O/O, \"when\"/O/O, \"there\"/O/O, \"are\"/O/O, \"times\"/O/O, \"when\"/O/O, \"i\"/O/O, \"forget\"/O/O, \"to\"/O/O, \"take\"/O/O, \"it\"/O/O, \"i\"/B-Symptom/B-Reduced_mobility, \"cant\"/I-Symptom/I-Reduced_mobility, \"stand\"/I-Symptom/I-Reduced_mobility, \"or\"/I-Symptom/I-Reduced_mobility, \"walk\"/I-Symptom/I-Reduced_mobility, \"for\"/I-Symptom/I-Reduced_mobility, \"any\"/I-Symptom/I-Reduced_mobility, \"lengths\"/I-Symptom/I-Reduced_mobility, \"of\"/I-Symptom/I-Reduced_mobility, \"time\"/E-Symptom/E-Reduced_mobility, \".\"/O/O],\n",
              " Sentence[89]: \"it seems as if bleeding forever is a side effect but doctors tell you that its not the drug . i have been on arthotec for three months i still have pain and stiffness . at first week into this drug i was in so much pain i sat on the floor at work and thought i was gaving a baby . i always feel weak and almost fainted a bunch of times . consider somthing that works i am switching today as this isnt in my head .\" → [\"it\"/O/O, \"seems\"/O/O, \"as\"/O/O, \"if\"/O/O, \"bleeding\"/S-ADR/S-Bleeding, \"forever\"/O/O, \"is\"/O/O, \"a\"/O/O, \"side\"/O/O, \"effect\"/O/O, \"but\"/O/O, \"doctors\"/O/O, \"tell\"/O/O, \"you\"/O/O, \"that\"/O/O, \"its\"/O/O, \"not\"/O/O, \"the\"/O/O, \"drug\"/O/O, \".\"/O/O, \"i\"/O/O, \"have\"/O/O, \"been\"/O/O, \"on\"/O/O, \"arthotec\"/S-Drug/S-Arthrotec, \"for\"/O/O, \"three\"/O/O, \"months\"/O/O, \"i\"/O/O, \"still\"/O/O, \"have\"/O/O, \"pain\"/B-Symptom/B-Pain, \"and\"/I-Symptom/I-Pain, \"stiffness\"/E-Symptom/E-Pain, \".\"/O/O, \"at\"/O/O, \"first\"/O/O, \"week\"/O/O, \"into\"/O/O, \"this\"/O/O, \"drug\"/O/O, \"i\"/O/O, \"was\"/O/O, \"in\"/O/O, \"so\"/B-ADR/B-Pain, \"much\"/I-ADR/I-Pain, \"pain\"/E-ADR/E-Pain, \"i\"/O/O, \"sat\"/O/O, \"on\"/O/O, \"the\"/O/O, \"floor\"/O/O, \"at\"/O/O, \"work\"/O/O, \"and\"/O/O, \"thought\"/O/O, \"i\"/O/O, \"was\"/O/O, \"gaving\"/O/O, \"a\"/O/O, \"baby\"/O/O, \".\"/O/O, \"i\"/O/O, \"always\"/O/O, \"feel\"/B-ADR/B-Malaise_and_fatigue, \"weak\"/I-ADR/I-Malaise_and_fatigue, \"and\"/I-ADR/I-Malaise_and_fatigue, \"almost\"/I-ADR/I-Malaise_and_fatigue, \"fainted\"/E-ADR/E-Malaise_and_fatigue, \"a\"/O/O, \"bunch\"/O/O, \"of\"/O/O, \"times\"/O/O, \".\"/O/O, \"consider\"/O/O, \"somthing\"/O/O, \"that\"/O/O, \"works\"/O/O, \"i\"/O/O, \"am\"/O/O, \"switching\"/O/O, \"today\"/O/O, \"as\"/O/O, \"this\"/O/O, \"isnt\"/O/O, \"in\"/O/O, \"my\"/O/O, \"head\"/O/O, \".\"/O/O],\n",
              " Sentence[56]: \"some diarrhea , some constipation , fatigue . it works very well for the heel ; i couldnt walk on it very well before taking it . i take 75mg . twice a day . also have severe osteoarthritis in the knees and hands , but it doesnt seem to relieve that pain at all .\" → [\"some\"/O/O, \"diarrhea\"/S-ADR/S-Diarrhoea, \",\"/O/O, \"some\"/O/O, \"constipation\"/S-ADR/S-Constipation, \",\"/O/O, \"fatigue\"/S-ADR/S-Fatigue, \".\"/O/O, \"it\"/O/O, \"works\"/O/O, \"very\"/O/O, \"well\"/O/O, \"for\"/O/O, \"the\"/B-Symptom/B-Walking_disability, \"heel\"/I-Symptom/I-Walking_disability, \";\"/O/O, \"i\"/I-Symptom/I-Walking_disability, \"couldnt\"/I-Symptom/I-Walking_disability, \"walk\"/I-Symptom/I-Walking_disability, \"on\"/I-Symptom/I-Walking_disability, \"it\"/O/O, \"very\"/I-Symptom/I-Walking_disability, \"well\"/E-Symptom/E-Walking_disability, \"before\"/O/O, \"taking\"/O/O, \"it\"/O/O, \".\"/O/O, \"i\"/O/O, \"take\"/O/O, \"75mg\"/O/O, \".\"/O/O, \"twice\"/O/O, \"a\"/O/O, \"day\"/O/O, \".\"/O/O, \"also\"/O/O, \"have\"/O/O, \"severe\"/B-Disease/B-Osteoarthritis_of_knee/B-Osteoarthrosis_of_hand, \"osteoarthritis\"/I-Disease/I-Osteoarthritis_of_knee/I-Osteoarthrosis_of_hand, \"in\"/I-Disease/I-Osteoarthritis_of_knee/I-Osteoarthrosis_of_hand, \"the\"/I-Disease/I-Osteoarthritis_of_knee/I-Osteoarthrosis_of_hand, \"knees\"/E-Disease/E-Osteoarthritis_of_knee, \"and\"/O/O, \"hands\"/E-Disease/E-Osteoarthrosis_of_hand, \",\"/O/O, \"but\"/O/O, \"it\"/O/O, \"doesnt\"/O/O, \"seem\"/O/O, \"to\"/O/O, \"relieve\"/O/O, \"that\"/O/O, \"pain\"/S-Symptom/S-Pain, \"at\"/O/O, \"all\"/O/O, \".\"/O/O],\n",
              " Sentence[84]: \"ive been on 75mg twice daily for 13 days , with recent body swelling , face , wrists , abdomen , thighs - clothes fit tight . insomina is much worse . i have forgetfulnes and confussion - really bad - almost dangerous . have felt \" empty stomach \" feeling , but not bothersome . have had more loose stools , but tolerable . im looking for a more natural therapy , checking into ha ( hyaluronic acid ) to replace arthrotec .\" → [\"ive\"/O/O, \"been\"/O/O, \"on\"/O/O, \"75mg\"/O/O, \"twice\"/O/O, \"daily\"/O/O, \"for\"/O/O, \"13\"/O/O, \"days\"/O/O, \",\"/O/O, \"with\"/O/O, \"recent\"/O/O, \"body\"/B-ADR/B-Swelling_of_body_region, \"swelling\"/B-ADR/E-ADR/B-Abdominal_swelling/B-Facial_swelling/E-Swelling_of_body_region/B-Swelling_of_lower_limb/B-Swelling_of_wrist_joint, \",\"/O/O, \"face\"/E-ADR/E-Facial_swelling, \",\"/O/O, \"wrists\"/E-ADR/E-Swelling_of_wrist_joint, \",\"/O/O, \"abdomen\"/E-ADR/E-Abdominal_swelling, \",\"/O/O, \"thighs\"/E-ADR/E-Swelling_of_lower_limb, \"-\"/O/O, \"clothes\"/O/O, \"fit\"/O/O, \"tight\"/O/O, \".\"/O/O, \"insomina\"/S-ADR/S-Insomnia, \"is\"/O/O, \"much\"/O/O, \"worse\"/O/O, \".\"/O/O, \"i\"/O/O, \"have\"/O/O, \"forgetfulnes\"/B-ADR/B-Forgetful, \"and\"/I-ADR/I-Forgetful, \"confussion\"/E-ADR/E-Forgetful, \"-\"/O/O, \"really\"/O/O, \"bad\"/O/O, \"-\"/O/O, \"almost\"/O/O, \"dangerous\"/O/O, \".\"/O/O, \"have\"/O/O, \"felt\"/O/O, \"\"\"/B-ADR/B-Feeling_empty, \"empty\"/I-ADR/I-Feeling_empty, \"stomach\"/I-ADR/I-Feeling_empty, \"\"\"/I-ADR/I-Feeling_empty, \"feeling\"/E-ADR/E-Feeling_empty, \",\"/O/O, \"but\"/O/O, \"not\"/O/O, \"bothersome\"/O/O, \".\"/O/O, \"have\"/O/O, \"had\"/O/O, \"more\"/O/O, \"loose\"/B-ADR/B-Loose_stool, \"stools\"/E-ADR/E-Loose_stool, \",\"/O/O, \"but\"/O/O, \"tolerable\"/O/O, \".\"/O/O, \"im\"/O/O, \"looking\"/O/O, \"for\"/O/O, \"a\"/O/O, \"more\"/O/O, \"natural\"/O/O, \"therapy\"/O/O, \",\"/O/O, \"checking\"/O/O, \"into\"/O/O, \"ha\"/B-Drug/B-CONCEPT_LESS, \"(\"/I-Drug/I-CONCEPT_LESS, \"hyaluronic\"/I-Drug/I-CONCEPT_LESS, \"acid\"/I-Drug/I-CONCEPT_LESS, \")\"/E-Drug/E-CONCEPT_LESS, \"to\"/O/O, \"replace\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \".\"/O/O],\n",
              " Sentence[68]: \"none that i can think of . i recently had bilateral knee replacement . prior to that , arthrotec was the only drug that helped . i tried every drug on the market , but , to no avail . i took 50 mg . 3 times a day of arthrotec and was almost pain free . i highly recommend it to anyone that suffers from arthritis .\" → [\"none\"/O/O, \"that\"/O/O, \"i\"/O/O, \"can\"/O/O, \"think\"/O/O, \"of\"/O/O, \".\"/O/O, \"i\"/O/O, \"recently\"/O/O, \"had\"/O/O, \"bilateral\"/O/O, \"knee\"/O/O, \"replacement\"/O/O, \".\"/O/O, \"prior\"/O/O, \"to\"/O/O, \"that\"/O/O, \",\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \"was\"/O/O, \"the\"/O/O, \"only\"/O/O, \"drug\"/O/O, \"that\"/O/O, \"helped\"/O/O, \".\"/O/O, \"i\"/O/O, \"tried\"/O/O, \"every\"/O/O, \"drug\"/O/O, \"on\"/O/O, \"the\"/O/O, \"market\"/O/O, \",\"/O/O, \"but\"/O/O, \",\"/O/O, \"to\"/O/O, \"no\"/O/O, \"avail\"/O/O, \".\"/O/O, \"i\"/O/O, \"took\"/O/O, \"50\"/O/O, \"mg\"/O/O, \".\"/O/O, \"3\"/O/O, \"times\"/O/O, \"a\"/O/O, \"day\"/O/O, \"of\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \"and\"/O/O, \"was\"/O/O, \"almost\"/O/O, \"pain\"/S-Symptom/S-Pain, \"free\"/O/O, \".\"/O/O, \"i\"/O/O, \"highly\"/O/O, \"recommend\"/O/O, \"it\"/O/O, \"to\"/O/O, \"anyone\"/O/O, \"that\"/O/O, \"suffers\"/O/O, \"from\"/O/O, \"arthritis\"/S-Finding/S-Arthritis, \".\"/O/O],\n",
              " Sentence[28]: \"hospitalization due extreme rectal bleed that required blood transfusion . i was extremely sick and initially felt i might not survive . should be removed from market .\" → [\"hospitalization\"/O/O, \"due\"/O/O, \"extreme\"/O/O, \"rectal\"/B-ADR/B-Rectal_haemorrhage, \"bleed\"/E-ADR/E-Rectal_haemorrhage, \"that\"/O/O, \"required\"/O/O, \"blood\"/O/O, \"transfusion\"/O/O, \".\"/O/O, \"i\"/O/O, \"was\"/O/O, \"extremely\"/B-ADR/B-Generally_unwell, \"sick\"/E-ADR/E-Generally_unwell, \"and\"/O/O, \"initially\"/O/O, \"felt\"/B-ADR/B-CONCEPT_LESS, \"i\"/I-ADR/I-CONCEPT_LESS, \"might\"/I-ADR/I-CONCEPT_LESS, \"not\"/I-ADR/I-CONCEPT_LESS, \"survive\"/E-ADR/E-CONCEPT_LESS, \".\"/O/O, \"should\"/O/O, \"be\"/O/O, \"removed\"/O/O, \"from\"/O/O, \"market\"/O/O, \".\"/O/O]]"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "with open(data_save_dir+'/cadec_flair_objects.pkl', 'rb') as f:\n",
        "    docs = pickle.load(f)\n",
        "docs[:10]"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sW10pPAPtGij",
        "outputId": "9dfd2971-c5b9-434d-e9dc-41851d2bc41c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "1186"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "len(docs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "EMT5TsNStGij",
        "outputId": "0ab84cad-8ff3-4b40-b5be-75c5eebf8f35"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Sentence[117]: \"i feel a bit drowsy & have a little blurred vision , so far no gastric problems . ive been on arthrotec 50 for over 10 years on and off , only taking it when i needed it . due to my arthritis getting progressively worse , to the point where i am in tears with the agony , gp ' s started me on 75 twice a day and i have to take it . every day for the next month to see how i get on , here goes . so far its been very good , pains almost gone , but i feel a bit weird , didnt have that when on 50 .\" → [\"i\"/O/O, \"feel\"/O/O, \"a\"/O/O, \"bit\"/B-ADR/B-Drowsy, \"drowsy\"/E-ADR/E-Drowsy, \"&\"/O/O, \"have\"/O/O, \"a\"/O/O, \"little\"/B-ADR/B-Blurred_vision_-_hazy, \"blurred\"/I-ADR/I-Blurred_vision_-_hazy, \"vision\"/E-ADR/E-Blurred_vision_-_hazy, \",\"/O/O, \"so\"/O/O, \"far\"/O/O, \"no\"/O/O, \"gastric\"/B-ADR/B-Excessive_upper_gastrointestinal_gas, \"problems\"/E-ADR/E-Excessive_upper_gastrointestinal_gas, \".\"/O/O, \"ive\"/O/O, \"been\"/O/O, \"on\"/O/O, \"arthrotec\"/S-Drug/S-Arthrotec, \"50\"/O/O, \"for\"/O/O, \"over\"/O/O, \"10\"/O/O, \"years\"/O/O, \"on\"/O/O, \"and\"/O/O, \"off\"/O/O, \",\"/O/O, \"only\"/O/O, \"taking\"/O/O, \"it\"/O/O, \"when\"/O/O, \"i\"/O/O, \"needed\"/O/O, \"it\"/O/O, \".\"/O/O, \"due\"/O/O, \"to\"/O/O, \"my\"/O/O, \"arthritis\"/S-Disease/S-Arthritis, \"getting\"/O/O, \"progressively\"/O/O, \"worse\"/O/O, \",\"/O/O, \"to\"/O/O, \"the\"/O/O, \"point\"/O/O, \"where\"/O/O, \"i\"/O/O, \"am\"/O/O, \"in\"/O/O, \"tears\"/O/O, \"with\"/O/O, \"the\"/O/O, \"agony\"/S-Symptom/S-Agony, \",\"/O/O, \"gp\"/O/O, \"'\"/O/O, \"s\"/O/O, \"started\"/O/O, \"me\"/O/O, \"on\"/O/O, \"75\"/O/O, \"twice\"/O/O, \"a\"/O/O, \"day\"/O/O, \"and\"/O/O, \"i\"/O/O, \"have\"/O/O, \"to\"/O/O, \"take\"/O/O, \"it\"/O/O, \".\"/O/O, \"every\"/O/O, \"day\"/O/O, \"for\"/O/O, \"the\"/O/O, \"next\"/O/O, \"month\"/O/O, \"to\"/O/O, \"see\"/O/O, \"how\"/O/O, \"i\"/O/O, \"get\"/O/O, \"on\"/O/O, \",\"/O/O, \"here\"/O/O, \"goes\"/O/O, \".\"/O/O, \"so\"/O/O, \"far\"/O/O, \"its\"/O/O, \"been\"/O/O, \"very\"/O/O, \"good\"/O/O, \",\"/O/O, \"pains\"/S-Symptom/S-Pain, \"almost\"/O/O, \"gone\"/O/O, \",\"/O/O, \"but\"/O/O, \"i\"/O/O, \"feel\"/B-ADR/B-Malaise, \"a\"/I-ADR/I-Malaise, \"bit\"/I-ADR/I-Malaise, \"weird\"/E-ADR/E-Malaise, \",\"/O/O, \"didnt\"/O/O, \"have\"/O/O, \"that\"/O/O, \"when\"/O/O, \"on\"/O/O, \"50\"/O/O, \".\"/O/O]"
            ]
          },
          "execution_count": 10,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "temp = docs[0]\n",
        "temp"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2EeTdfHstGij",
        "outputId": "2ee78332-2ce1-4ca3-f202-1a32abf4dd8e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'ner': ['Token[0]: \"i\"'/'O' (1.0),\n",
              "  'Token[1]: \"feel\"'/'O' (1.0),\n",
              "  'Token[2]: \"a\"'/'O' (1.0),\n",
              "  'Token[3]: \"bit\"'/'B-ADR' (1.0),\n",
              "  'Token[4]: \"drowsy\"'/'E-ADR' (1.0),\n",
              "  'Token[5]: \"&\"'/'O' (1.0),\n",
              "  'Token[6]: \"have\"'/'O' (1.0),\n",
              "  'Token[7]: \"a\"'/'O' (1.0),\n",
              "  'Token[8]: \"little\"'/'B-ADR' (1.0),\n",
              "  'Token[9]: \"blurred\"'/'I-ADR' (1.0),\n",
              "  'Token[10]: \"vision\"'/'E-ADR' (1.0),\n",
              "  'Token[11]: \",\"'/'O' (1.0),\n",
              "  'Token[12]: \"so\"'/'O' (1.0),\n",
              "  'Token[13]: \"far\"'/'O' (1.0),\n",
              "  'Token[14]: \"no\"'/'O' (1.0),\n",
              "  'Token[15]: \"gastric\"'/'B-ADR' (1.0),\n",
              "  'Token[16]: \"problems\"'/'E-ADR' (1.0),\n",
              "  'Token[17]: \".\"'/'O' (1.0),\n",
              "  'Token[18]: \"ive\"'/'O' (1.0),\n",
              "  'Token[19]: \"been\"'/'O' (1.0),\n",
              "  'Token[20]: \"on\"'/'O' (1.0),\n",
              "  'Token[21]: \"arthrotec\"'/'S-Drug' (1.0),\n",
              "  'Token[22]: \"50\"'/'O' (1.0),\n",
              "  'Token[23]: \"for\"'/'O' (1.0),\n",
              "  'Token[24]: \"over\"'/'O' (1.0),\n",
              "  'Token[25]: \"10\"'/'O' (1.0),\n",
              "  'Token[26]: \"years\"'/'O' (1.0),\n",
              "  'Token[27]: \"on\"'/'O' (1.0),\n",
              "  'Token[28]: \"and\"'/'O' (1.0),\n",
              "  'Token[29]: \"off\"'/'O' (1.0),\n",
              "  'Token[30]: \",\"'/'O' (1.0),\n",
              "  'Token[31]: \"only\"'/'O' (1.0),\n",
              "  'Token[32]: \"taking\"'/'O' (1.0),\n",
              "  'Token[33]: \"it\"'/'O' (1.0),\n",
              "  'Token[34]: \"when\"'/'O' (1.0),\n",
              "  'Token[35]: \"i\"'/'O' (1.0),\n",
              "  'Token[36]: \"needed\"'/'O' (1.0),\n",
              "  'Token[37]: \"it\"'/'O' (1.0),\n",
              "  'Token[38]: \".\"'/'O' (1.0),\n",
              "  'Token[39]: \"due\"'/'O' (1.0),\n",
              "  'Token[40]: \"to\"'/'O' (1.0),\n",
              "  'Token[41]: \"my\"'/'O' (1.0),\n",
              "  'Token[42]: \"arthritis\"'/'S-Disease' (1.0),\n",
              "  'Token[43]: \"getting\"'/'O' (1.0),\n",
              "  'Token[44]: \"progressively\"'/'O' (1.0),\n",
              "  'Token[45]: \"worse\"'/'O' (1.0),\n",
              "  'Token[46]: \",\"'/'O' (1.0),\n",
              "  'Token[47]: \"to\"'/'O' (1.0),\n",
              "  'Token[48]: \"the\"'/'O' (1.0),\n",
              "  'Token[49]: \"point\"'/'O' (1.0),\n",
              "  'Token[50]: \"where\"'/'O' (1.0),\n",
              "  'Token[51]: \"i\"'/'O' (1.0),\n",
              "  'Token[52]: \"am\"'/'O' (1.0),\n",
              "  'Token[53]: \"in\"'/'O' (1.0),\n",
              "  'Token[54]: \"tears\"'/'O' (1.0),\n",
              "  'Token[55]: \"with\"'/'O' (1.0),\n",
              "  'Token[56]: \"the\"'/'O' (1.0),\n",
              "  'Token[57]: \"agony\"'/'S-Symptom' (1.0),\n",
              "  'Token[58]: \",\"'/'O' (1.0),\n",
              "  'Token[59]: \"gp\"'/'O' (1.0),\n",
              "  'Token[60]: \"'\"'/'O' (1.0),\n",
              "  'Token[61]: \"s\"'/'O' (1.0),\n",
              "  'Token[62]: \"started\"'/'O' (1.0),\n",
              "  'Token[63]: \"me\"'/'O' (1.0),\n",
              "  'Token[64]: \"on\"'/'O' (1.0),\n",
              "  'Token[65]: \"75\"'/'O' (1.0),\n",
              "  'Token[66]: \"twice\"'/'O' (1.0),\n",
              "  'Token[67]: \"a\"'/'O' (1.0),\n",
              "  'Token[68]: \"day\"'/'O' (1.0),\n",
              "  'Token[69]: \"and\"'/'O' (1.0),\n",
              "  'Token[70]: \"i\"'/'O' (1.0),\n",
              "  'Token[71]: \"have\"'/'O' (1.0),\n",
              "  'Token[72]: \"to\"'/'O' (1.0),\n",
              "  'Token[73]: \"take\"'/'O' (1.0),\n",
              "  'Token[74]: \"it\"'/'O' (1.0),\n",
              "  'Token[75]: \".\"'/'O' (1.0),\n",
              "  'Token[76]: \"every\"'/'O' (1.0),\n",
              "  'Token[77]: \"day\"'/'O' (1.0),\n",
              "  'Token[78]: \"for\"'/'O' (1.0),\n",
              "  'Token[79]: \"the\"'/'O' (1.0),\n",
              "  'Token[80]: \"next\"'/'O' (1.0),\n",
              "  'Token[81]: \"month\"'/'O' (1.0),\n",
              "  'Token[82]: \"to\"'/'O' (1.0),\n",
              "  'Token[83]: \"see\"'/'O' (1.0),\n",
              "  'Token[84]: \"how\"'/'O' (1.0),\n",
              "  'Token[85]: \"i\"'/'O' (1.0),\n",
              "  'Token[86]: \"get\"'/'O' (1.0),\n",
              "  'Token[87]: \"on\"'/'O' (1.0),\n",
              "  'Token[88]: \",\"'/'O' (1.0),\n",
              "  'Token[89]: \"here\"'/'O' (1.0),\n",
              "  'Token[90]: \"goes\"'/'O' (1.0),\n",
              "  'Token[91]: \".\"'/'O' (1.0),\n",
              "  'Token[92]: \"so\"'/'O' (1.0),\n",
              "  'Token[93]: \"far\"'/'O' (1.0),\n",
              "  'Token[94]: \"its\"'/'O' (1.0),\n",
              "  'Token[95]: \"been\"'/'O' (1.0),\n",
              "  'Token[96]: \"very\"'/'O' (1.0),\n",
              "  'Token[97]: \"good\"'/'O' (1.0),\n",
              "  'Token[98]: \",\"'/'O' (1.0),\n",
              "  'Token[99]: \"pains\"'/'S-Symptom' (1.0),\n",
              "  'Token[100]: \"almost\"'/'O' (1.0),\n",
              "  'Token[101]: \"gone\"'/'O' (1.0),\n",
              "  'Token[102]: \",\"'/'O' (1.0),\n",
              "  'Token[103]: \"but\"'/'O' (1.0),\n",
              "  'Token[104]: \"i\"'/'O' (1.0),\n",
              "  'Token[105]: \"feel\"'/'B-ADR' (1.0),\n",
              "  'Token[106]: \"a\"'/'I-ADR' (1.0),\n",
              "  'Token[107]: \"bit\"'/'I-ADR' (1.0),\n",
              "  'Token[108]: \"weird\"'/'E-ADR' (1.0),\n",
              "  'Token[109]: \",\"'/'O' (1.0),\n",
              "  'Token[110]: \"didnt\"'/'O' (1.0),\n",
              "  'Token[111]: \"have\"'/'O' (1.0),\n",
              "  'Token[112]: \"that\"'/'O' (1.0),\n",
              "  'Token[113]: \"when\"'/'O' (1.0),\n",
              "  'Token[114]: \"on\"'/'O' (1.0),\n",
              "  'Token[115]: \"50\"'/'O' (1.0),\n",
              "  'Token[116]: \".\"'/'O' (1.0)],\n",
              " 'meddra': ['Token[0]: \"i\"'/'O' (1.0),\n",
              "  'Token[1]: \"feel\"'/'O' (1.0),\n",
              "  'Token[2]: \"a\"'/'O' (1.0),\n",
              "  'Token[3]: \"bit\"'/'B-Drowsy' (1.0),\n",
              "  'Token[4]: \"drowsy\"'/'E-Drowsy' (1.0),\n",
              "  'Token[5]: \"&\"'/'O' (1.0),\n",
              "  'Token[6]: \"have\"'/'O' (1.0),\n",
              "  'Token[7]: \"a\"'/'O' (1.0),\n",
              "  'Token[8]: \"little\"'/'B-Blurred_vision_-_hazy' (1.0),\n",
              "  'Token[9]: \"blurred\"'/'I-Blurred_vision_-_hazy' (1.0),\n",
              "  'Token[10]: \"vision\"'/'E-Blurred_vision_-_hazy' (1.0),\n",
              "  'Token[11]: \",\"'/'O' (1.0),\n",
              "  'Token[12]: \"so\"'/'O' (1.0),\n",
              "  'Token[13]: \"far\"'/'O' (1.0),\n",
              "  'Token[14]: \"no\"'/'O' (1.0),\n",
              "  'Token[15]: \"gastric\"'/'B-Excessive_upper_gastrointestinal_gas' (1.0),\n",
              "  'Token[16]: \"problems\"'/'E-Excessive_upper_gastrointestinal_gas' (1.0),\n",
              "  'Token[17]: \".\"'/'O' (1.0),\n",
              "  'Token[18]: \"ive\"'/'O' (1.0),\n",
              "  'Token[19]: \"been\"'/'O' (1.0),\n",
              "  'Token[20]: \"on\"'/'O' (1.0),\n",
              "  'Token[21]: \"arthrotec\"'/'S-Arthrotec' (1.0),\n",
              "  'Token[22]: \"50\"'/'O' (1.0),\n",
              "  'Token[23]: \"for\"'/'O' (1.0),\n",
              "  'Token[24]: \"over\"'/'O' (1.0),\n",
              "  'Token[25]: \"10\"'/'O' (1.0),\n",
              "  'Token[26]: \"years\"'/'O' (1.0),\n",
              "  'Token[27]: \"on\"'/'O' (1.0),\n",
              "  'Token[28]: \"and\"'/'O' (1.0),\n",
              "  'Token[29]: \"off\"'/'O' (1.0),\n",
              "  'Token[30]: \",\"'/'O' (1.0),\n",
              "  'Token[31]: \"only\"'/'O' (1.0),\n",
              "  'Token[32]: \"taking\"'/'O' (1.0),\n",
              "  'Token[33]: \"it\"'/'O' (1.0),\n",
              "  'Token[34]: \"when\"'/'O' (1.0),\n",
              "  'Token[35]: \"i\"'/'O' (1.0),\n",
              "  'Token[36]: \"needed\"'/'O' (1.0),\n",
              "  'Token[37]: \"it\"'/'O' (1.0),\n",
              "  'Token[38]: \".\"'/'O' (1.0),\n",
              "  'Token[39]: \"due\"'/'O' (1.0),\n",
              "  'Token[40]: \"to\"'/'O' (1.0),\n",
              "  'Token[41]: \"my\"'/'O' (1.0),\n",
              "  'Token[42]: \"arthritis\"'/'S-Arthritis' (1.0),\n",
              "  'Token[43]: \"getting\"'/'O' (1.0),\n",
              "  'Token[44]: \"progressively\"'/'O' (1.0),\n",
              "  'Token[45]: \"worse\"'/'O' (1.0),\n",
              "  'Token[46]: \",\"'/'O' (1.0),\n",
              "  'Token[47]: \"to\"'/'O' (1.0),\n",
              "  'Token[48]: \"the\"'/'O' (1.0),\n",
              "  'Token[49]: \"point\"'/'O' (1.0),\n",
              "  'Token[50]: \"where\"'/'O' (1.0),\n",
              "  'Token[51]: \"i\"'/'O' (1.0),\n",
              "  'Token[52]: \"am\"'/'O' (1.0),\n",
              "  'Token[53]: \"in\"'/'O' (1.0),\n",
              "  'Token[54]: \"tears\"'/'O' (1.0),\n",
              "  'Token[55]: \"with\"'/'O' (1.0),\n",
              "  'Token[56]: \"the\"'/'O' (1.0),\n",
              "  'Token[57]: \"agony\"'/'S-Agony' (1.0),\n",
              "  'Token[58]: \",\"'/'O' (1.0),\n",
              "  'Token[59]: \"gp\"'/'O' (1.0),\n",
              "  'Token[60]: \"'\"'/'O' (1.0),\n",
              "  'Token[61]: \"s\"'/'O' (1.0),\n",
              "  'Token[62]: \"started\"'/'O' (1.0),\n",
              "  'Token[63]: \"me\"'/'O' (1.0),\n",
              "  'Token[64]: \"on\"'/'O' (1.0),\n",
              "  'Token[65]: \"75\"'/'O' (1.0),\n",
              "  'Token[66]: \"twice\"'/'O' (1.0),\n",
              "  'Token[67]: \"a\"'/'O' (1.0),\n",
              "  'Token[68]: \"day\"'/'O' (1.0),\n",
              "  'Token[69]: \"and\"'/'O' (1.0),\n",
              "  'Token[70]: \"i\"'/'O' (1.0),\n",
              "  'Token[71]: \"have\"'/'O' (1.0),\n",
              "  'Token[72]: \"to\"'/'O' (1.0),\n",
              "  'Token[73]: \"take\"'/'O' (1.0),\n",
              "  'Token[74]: \"it\"'/'O' (1.0),\n",
              "  'Token[75]: \".\"'/'O' (1.0),\n",
              "  'Token[76]: \"every\"'/'O' (1.0),\n",
              "  'Token[77]: \"day\"'/'O' (1.0),\n",
              "  'Token[78]: \"for\"'/'O' (1.0),\n",
              "  'Token[79]: \"the\"'/'O' (1.0),\n",
              "  'Token[80]: \"next\"'/'O' (1.0),\n",
              "  'Token[81]: \"month\"'/'O' (1.0),\n",
              "  'Token[82]: \"to\"'/'O' (1.0),\n",
              "  'Token[83]: \"see\"'/'O' (1.0),\n",
              "  'Token[84]: \"how\"'/'O' (1.0),\n",
              "  'Token[85]: \"i\"'/'O' (1.0),\n",
              "  'Token[86]: \"get\"'/'O' (1.0),\n",
              "  'Token[87]: \"on\"'/'O' (1.0),\n",
              "  'Token[88]: \",\"'/'O' (1.0),\n",
              "  'Token[89]: \"here\"'/'O' (1.0),\n",
              "  'Token[90]: \"goes\"'/'O' (1.0),\n",
              "  'Token[91]: \".\"'/'O' (1.0),\n",
              "  'Token[92]: \"so\"'/'O' (1.0),\n",
              "  'Token[93]: \"far\"'/'O' (1.0),\n",
              "  'Token[94]: \"its\"'/'O' (1.0),\n",
              "  'Token[95]: \"been\"'/'O' (1.0),\n",
              "  'Token[96]: \"very\"'/'O' (1.0),\n",
              "  'Token[97]: \"good\"'/'O' (1.0),\n",
              "  'Token[98]: \",\"'/'O' (1.0),\n",
              "  'Token[99]: \"pains\"'/'S-Pain' (1.0),\n",
              "  'Token[100]: \"almost\"'/'O' (1.0),\n",
              "  'Token[101]: \"gone\"'/'O' (1.0),\n",
              "  'Token[102]: \",\"'/'O' (1.0),\n",
              "  'Token[103]: \"but\"'/'O' (1.0),\n",
              "  'Token[104]: \"i\"'/'O' (1.0),\n",
              "  'Token[105]: \"feel\"'/'B-Malaise' (1.0),\n",
              "  'Token[106]: \"a\"'/'I-Malaise' (1.0),\n",
              "  'Token[107]: \"bit\"'/'I-Malaise' (1.0),\n",
              "  'Token[108]: \"weird\"'/'E-Malaise' (1.0),\n",
              "  'Token[109]: \",\"'/'O' (1.0),\n",
              "  'Token[110]: \"didnt\"'/'O' (1.0),\n",
              "  'Token[111]: \"have\"'/'O' (1.0),\n",
              "  'Token[112]: \"that\"'/'O' (1.0),\n",
              "  'Token[113]: \"when\"'/'O' (1.0),\n",
              "  'Token[114]: \"on\"'/'O' (1.0),\n",
              "  'Token[115]: \"50\"'/'O' (1.0),\n",
              "  'Token[116]: \".\"'/'O' (1.0)]}"
            ]
          },
          "execution_count": 11,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "temp.annotation_layers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "4CsGKKB6tGik"
      },
      "outputs": [],
      "source": [
        "\n",
        "# define your NER tags\n",
        "label_dict = {\n",
        "    'ADR': 'ADR',\n",
        "    'Drug': 'Drug',\n",
        "    'Finding': 'Finding',\n",
        "    'Symptom': 'Symptom',\n",
        "    'Disease': 'Disease'\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QmNsy8AltGik",
        "outputId": "bb66972d-496a-424b-b071-8e08e1adda49"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:31:49,779 Filtering empty sentences\n",
            "2023-04-21 06:31:49,842 Corpus: 948 train + 119 dev + 119 test sentences\n",
            "2023-04-21 06:31:49,844 Computing label dictionary. Progress:\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "948it [00:00, 6626.39it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:31:49,992 Dictionary created for label 'ner' with 22 values: O (seen 79057 times), I-ADR (seen 4449 times), E-ADR (seen 3506 times), B-ADR (seen 3349 times), S-ADR (seen 1363 times), S-Drug (seen 1286 times), B-Finding (seen 200 times), E-Finding (seen 200 times), I-Finding (seen 149 times), B-Disease (seen 119 times), E-Disease (seen 119 times), E-Symptom (seen 114 times), S-Disease (seen 112 times), S-Symptom (seen 112 times), E-Drug (seen 111 times), B-Symptom (seen 110 times), B-Drug (seen 110 times), S-Finding (seen 109 times), I-Symptom (seen 104 times), I-Drug (seen 50 times)\n",
            "False\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:31:51,315 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim.vectors.npy not found in cache, downloading to /tmp/tmpkhzimo9h\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 153M/153M [00:13<00:00, 12.2MB/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:05,041 copying /tmp/tmpkhzimo9h to cache at /root/.flair/embeddings/glove.gensim.vectors.npy\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:05,154 removing temp file /tmp/tmpkhzimo9h\n",
            "2023-04-21 06:32:05,833 https://flair.informatik.hu-berlin.de/resources/embeddings/token/glove.gensim not found in cache, downloading to /tmp/tmpzp5yvot8\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 20.5M/20.5M [00:02<00:00, 7.33MB/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:09,419 copying /tmp/tmpzp5yvot8 to cache at /root/.flair/embeddings/glove.gensim\n",
            "2023-04-21 06:32:09,438 removing temp file /tmp/tmpzp5yvot8\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:22,303 https://flair.informatik.hu-berlin.de/resources/embeddings/flair/news-forward-0.4.1.pt not found in cache, downloading to /tmp/tmpkw777zev\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 69.7M/69.7M [00:06<00:00, 10.8MB/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:29,743 copying /tmp/tmpkw777zev to cache at /root/.flair/embeddings/news-forward-0.4.1.pt\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:29,799 removing temp file /tmp/tmpkw777zev\n",
            "2023-04-21 06:32:35,487 https://flair.informatik.hu-berlin.de/resources/embeddings/flair/news-backward-0.4.1.pt not found in cache, downloading to /tmp/tmppwg7458x\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 69.7M/69.7M [00:06<00:00, 10.7MB/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:42,951 copying /tmp/tmppwg7458x to cache at /root/.flair/embeddings/news-backward-0.4.1.pt\n",
            "2023-04-21 06:32:43,008 removing temp file /tmp/tmppwg7458x\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:43,235 SequenceTagger predicts: Dictionary with 22 tags: <unk>, O, I-ADR, E-ADR, B-ADR, S-ADR, S-Drug, B-Finding, E-Finding, I-Finding, B-Disease, E-Disease, E-Symptom, S-Disease, S-Symptom, E-Drug, B-Symptom, B-Drug, S-Finding, I-Symptom, I-Drug, I-Disease\n",
            "True\n",
            "False\n"
          ]
        }
      ],
      "source": [
        "from flair.data import Corpus\n",
        "\n",
        "from flair.embeddings import WordEmbeddings, FlairEmbeddings, StackedEmbeddings, TransformerWordEmbeddings\n",
        "from flair.models import SequenceTagger\n",
        "from flair.trainers import ModelTrainer\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "from torch.optim import AdamW, SGD\n",
        "\n",
        "from flair.visual.training_curves import Plotter\n",
        "plotter = Plotter()\n",
        "\n",
        "\n",
        "\n",
        "train_docs, test_docs = train_test_split(docs, test_size=0.2, random_state=321)\n",
        "dev_docs, test_docs = train_test_split(test_docs, test_size=0.5, random_state=123)\n",
        "\n",
        "corpus = Corpus(train=train_docs, dev=dev_docs, test=test_docs) # Corpus: 948 train + 119 dev + 119 test sentences\n",
        "corpus.filter_empty_sentences()\n",
        "# # create a tag dictionary\n",
        "labels = ['ADR', 'Drug', 'Finding', 'Symptom', 'Disease']\n",
        "# tag_dictionary = Dictionary(add_unk=True)\n",
        "# for label in labels:\n",
        "#     tag_dictionary.add_item(label)\n",
        "tag_dictionary = corpus.make_label_dictionary(label_type='ner')\n",
        "print(tag_dictionary.span_labels)\n",
        "\n",
        "# create a stacked embedding using GloVe and Flair embeddings\n",
        "stacked_embeddings = StackedEmbeddings([\n",
        "    WordEmbeddings('glove',\n",
        "                   force_cpu=False,\n",
        "                #    embedding_length=1024,\n",
        "                  #  fine_tune=True,\n",
        "                   ),\n",
        "    # TransformerWordEmbeddings('emilyalsentzer/Bio_ClinicalBERT', layers=\"-1\",\n",
        "    #                                    subtoken_pooling=\"first\",\n",
        "    #                                    is_document_embedding=True,\n",
        "    #                                    fine_tune=True,\n",
        "    #                                    use_context=True,\n",
        "    #                                    max_seq_length=1024,\n",
        "    #                                    ),\n",
        "    FlairEmbeddings('news-forward',\n",
        "                    # fine_tune=True\n",
        "                    ),\n",
        "    FlairEmbeddings('news-backward',\n",
        "                    # fine_tune=True\n",
        "                    ),\n",
        "])\n",
        "# https://github.com/flairNLP/flair/blob/master/resources/docs/embeddings/FLAIR_EMBEDDINGS.md\n",
        "\n",
        "# create a SequenceTagger object and specify the tag dictionary and embeddings to use\n",
        "tagger = SequenceTagger(hidden_size=256,\n",
        "                        embeddings=stacked_embeddings,\n",
        "                        tag_dictionary=tag_dictionary,\n",
        "                        tag_type='ner',\n",
        "                        # use_crf=False,\n",
        "                        use_rnn=False,\n",
        "                        # reproject_embeddings=False,\n",
        "                        )\n",
        "tagger.predict_spans = False\n",
        "\n",
        "print(tagger._determine_if_span_prediction_problem(tagger.label_dictionary))\n",
        "print(tagger.predict_spans)\n",
        "\n",
        "# https://github.com/flairNLP/flair/blob/master/flair/trainers/trainer.py\n",
        "# create a ModelTrainer object and train the tagger\n",
        "trainer = ModelTrainer(tagger, corpus)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "fh0zU2jHAA4J"
      },
      "outputs": [],
      "source": [
        "# !tensorboard --logdir=runs\n",
        "# http://localhost:6006/"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UKTMzxV8zF9o",
        "outputId": "d9005b82-fdb9-4894-c63a-687dbd46a05d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:32:43,439 tensorboard logging path is runs/tb_logs_v2_no_rnn\n",
            "2023-04-21 06:32:43,449 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,450 Model: \"SequenceTagger(\n",
            "  (embeddings): StackedEmbeddings(\n",
            "    (list_embedding_0): WordEmbeddings(\n",
            "      'glove'\n",
            "      (embedding): Embedding(400001, 100)\n",
            "    )\n",
            "    (list_embedding_1): FlairEmbeddings(\n",
            "      (lm): LanguageModel(\n",
            "        (drop): Dropout(p=0.05, inplace=False)\n",
            "        (encoder): Embedding(300, 100)\n",
            "        (rnn): LSTM(100, 2048)\n",
            "      )\n",
            "    )\n",
            "    (list_embedding_2): FlairEmbeddings(\n",
            "      (lm): LanguageModel(\n",
            "        (drop): Dropout(p=0.05, inplace=False)\n",
            "        (encoder): Embedding(300, 100)\n",
            "        (rnn): LSTM(100, 2048)\n",
            "      )\n",
            "    )\n",
            "  )\n",
            "  (word_dropout): WordDropout(p=0.05)\n",
            "  (locked_dropout): LockedDropout(p=0.5)\n",
            "  (embedding2nn): Linear(in_features=4196, out_features=4196, bias=True)\n",
            "  (linear): Linear(in_features=4196, out_features=24, bias=True)\n",
            "  (loss_function): ViterbiLoss()\n",
            "  (crf): CRF()\n",
            ")\"\n",
            "2023-04-21 06:32:43,452 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,453 Corpus: \"Corpus: 948 train + 119 dev + 119 test sentences\"\n",
            "2023-04-21 06:32:43,454 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,456 Parameters:\n",
            "2023-04-21 06:32:43,457  - learning_rate: \"0.100000\"\n",
            "2023-04-21 06:32:43,458  - mini_batch_size: \"12\"\n",
            "2023-04-21 06:32:43,459  - patience: \"1\"\n",
            "2023-04-21 06:32:43,460  - anneal_factor: \"0.5\"\n",
            "2023-04-21 06:32:43,462  - max_epochs: \"100\"\n",
            "2023-04-21 06:32:43,463  - shuffle: \"True\"\n",
            "2023-04-21 06:32:43,465  - train_with_dev: \"False\"\n",
            "2023-04-21 06:32:43,466  - batch_growth_annealing: \"False\"\n",
            "2023-04-21 06:32:43,467 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,469 Model training base path: \"flair_ner_model_v2_no_rnn\"\n",
            "2023-04-21 06:32:43,470 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,471 Device: cuda:0\n",
            "2023-04-21 06:32:43,473 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:43,474 Embeddings storage mode: gpu\n",
            "2023-04-21 06:32:43,476 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:32:46,343 epoch 1 - iter 7/79 - loss 4950.79683275 - time (sec): 2.87 - samples/sec: 3189.63 - lr: 0.100000\n",
            "2023-04-21 06:32:48,509 epoch 1 - iter 14/79 - loss 3389.20917002 - time (sec): 5.03 - samples/sec: 3249.73 - lr: 0.100000\n",
            "2023-04-21 06:32:51,100 epoch 1 - iter 21/79 - loss 2507.34957014 - time (sec): 7.62 - samples/sec: 3205.10 - lr: 0.100000\n",
            "2023-04-21 06:32:53,584 epoch 1 - iter 28/79 - loss 2039.11306776 - time (sec): 10.11 - samples/sec: 3225.79 - lr: 0.100000\n",
            "2023-04-21 06:32:56,139 epoch 1 - iter 35/79 - loss 1720.47270840 - time (sec): 12.66 - samples/sec: 3260.40 - lr: 0.100000\n",
            "2023-04-21 06:32:58,644 epoch 1 - iter 42/79 - loss 1490.41153158 - time (sec): 15.17 - samples/sec: 3262.22 - lr: 0.100000\n",
            "2023-04-21 06:33:01,185 epoch 1 - iter 49/79 - loss 1320.98058686 - time (sec): 17.71 - samples/sec: 3268.60 - lr: 0.100000\n",
            "2023-04-21 06:33:04,774 epoch 1 - iter 56/79 - loss 1185.97696888 - time (sec): 21.30 - samples/sec: 3124.81 - lr: 0.100000\n",
            "2023-04-21 06:33:06,965 epoch 1 - iter 63/79 - loss 1091.77088861 - time (sec): 23.49 - samples/sec: 3188.47 - lr: 0.100000\n",
            "2023-04-21 06:33:09,757 epoch 1 - iter 70/79 - loss 1039.80573307 - time (sec): 26.28 - samples/sec: 3164.93 - lr: 0.100000\n",
            "2023-04-21 06:33:13,421 epoch 1 - iter 77/79 - loss 976.94473350 - time (sec): 29.94 - samples/sec: 3069.75 - lr: 0.100000\n",
            "2023-04-21 06:33:14,917 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:33:14,920 EPOCH 1 done: loss 955.5350 - lr 0.100000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.08it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:33:34,409 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:33:37,104 TRAIN : loss 250.87393188476562 - f1-score (weighted avg)  0.4569\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:04<00:00,  2.42it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:33:41,313 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:33:41,384 DEV : loss 284.189208984375 - f1-score (weighted avg)  0.4268\n",
            "2023-04-21 06:33:41,394 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:33:41,397 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:33:43,143 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:33:44,754 epoch 2 - iter 7/79 - loss 285.03493586 - time (sec): 1.61 - samples/sec: 5959.19 - lr: 0.100000\n",
            "2023-04-21 06:33:45,931 epoch 2 - iter 14/79 - loss 264.87572338 - time (sec): 2.79 - samples/sec: 6200.99 - lr: 0.100000\n",
            "2023-04-21 06:33:47,337 epoch 2 - iter 21/79 - loss 262.73748806 - time (sec): 4.19 - samples/sec: 5995.53 - lr: 0.100000\n",
            "2023-04-21 06:33:49,004 epoch 2 - iter 28/79 - loss 281.23855534 - time (sec): 5.86 - samples/sec: 5696.54 - lr: 0.100000\n",
            "2023-04-21 06:33:50,395 epoch 2 - iter 35/79 - loss 308.37935284 - time (sec): 7.25 - samples/sec: 5651.64 - lr: 0.100000\n",
            "2023-04-21 06:33:52,187 epoch 2 - iter 42/79 - loss 322.77862595 - time (sec): 9.04 - samples/sec: 5562.99 - lr: 0.100000\n",
            "2023-04-21 06:33:54,507 epoch 2 - iter 49/79 - loss 325.31286140 - time (sec): 11.36 - samples/sec: 5342.26 - lr: 0.100000\n",
            "2023-04-21 06:33:56,023 epoch 2 - iter 56/79 - loss 328.44088631 - time (sec): 12.88 - samples/sec: 5344.39 - lr: 0.100000\n",
            "2023-04-21 06:33:57,813 epoch 2 - iter 63/79 - loss 328.74083367 - time (sec): 14.67 - samples/sec: 5325.06 - lr: 0.100000\n",
            "2023-04-21 06:33:59,159 epoch 2 - iter 70/79 - loss 323.58756729 - time (sec): 16.01 - samples/sec: 5300.63 - lr: 0.100000\n",
            "2023-04-21 06:34:00,593 epoch 2 - iter 77/79 - loss 328.95446815 - time (sec): 17.45 - samples/sec: 5290.45 - lr: 0.100000\n",
            "2023-04-21 06:34:01,063 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:34:01,065 EPOCH 2 done: loss 329.5385 - lr 0.100000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.80it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:34:22,003 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:34:24,753 TRAIN : loss 255.85723876953125 - f1-score (weighted avg)  0.4699\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.90it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:34:27,397 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:34:27,467 DEV : loss 297.12298583984375 - f1-score (weighted avg)  0.444\n",
            "2023-04-21 06:34:27,476 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:34:27,480 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:34:28,967 epoch 3 - iter 7/79 - loss 376.65685535 - time (sec): 1.49 - samples/sec: 5352.71 - lr: 0.100000\n",
            "2023-04-21 06:34:30,300 epoch 3 - iter 14/79 - loss 360.99435949 - time (sec): 2.82 - samples/sec: 5661.08 - lr: 0.100000\n",
            "2023-04-21 06:34:31,738 epoch 3 - iter 21/79 - loss 348.52933037 - time (sec): 4.26 - samples/sec: 5603.66 - lr: 0.100000\n",
            "2023-04-21 06:34:33,207 epoch 3 - iter 28/79 - loss 352.27167132 - time (sec): 5.73 - samples/sec: 5571.50 - lr: 0.100000\n",
            "2023-04-21 06:34:34,630 epoch 3 - iter 35/79 - loss 377.44742615 - time (sec): 7.15 - samples/sec: 5611.59 - lr: 0.100000\n",
            "2023-04-21 06:34:36,118 epoch 3 - iter 42/79 - loss 370.10763032 - time (sec): 8.64 - samples/sec: 5675.71 - lr: 0.100000\n",
            "2023-04-21 06:34:37,736 epoch 3 - iter 49/79 - loss 366.85351949 - time (sec): 10.25 - samples/sec: 5619.35 - lr: 0.100000\n",
            "2023-04-21 06:34:39,575 epoch 3 - iter 56/79 - loss 365.56334518 - time (sec): 12.09 - samples/sec: 5607.39 - lr: 0.100000\n",
            "2023-04-21 06:34:40,752 epoch 3 - iter 63/79 - loss 359.60952100 - time (sec): 13.27 - samples/sec: 5628.97 - lr: 0.100000\n",
            "2023-04-21 06:34:42,686 epoch 3 - iter 70/79 - loss 357.31875641 - time (sec): 15.20 - samples/sec: 5515.04 - lr: 0.100000\n",
            "2023-04-21 06:34:44,471 epoch 3 - iter 77/79 - loss 355.53055230 - time (sec): 16.99 - samples/sec: 5470.38 - lr: 0.100000\n",
            "2023-04-21 06:34:44,800 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:34:44,801 EPOCH 3 done: loss 357.2746 - lr 0.100000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.80it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:05,727 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:08,440 TRAIN : loss 347.0096435546875 - f1-score (weighted avg)  0.4372\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.90it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:11,080 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:35:11,154 DEV : loss 394.6324462890625 - f1-score (weighted avg)  0.3935\n",
            "2023-04-21 06:35:11,163 Epoch     3: reducing learning rate of group 0 to 5.0000e-02.\n",
            "2023-04-21 06:35:11,165 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:35:11,169 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:13,150 epoch 4 - iter 7/79 - loss 402.65163847 - time (sec): 1.98 - samples/sec: 4718.05 - lr: 0.050000\n",
            "2023-04-21 06:35:14,439 epoch 4 - iter 14/79 - loss 362.07837313 - time (sec): 3.27 - samples/sec: 5201.73 - lr: 0.050000\n",
            "2023-04-21 06:35:15,709 epoch 4 - iter 21/79 - loss 324.73046463 - time (sec): 4.54 - samples/sec: 5425.11 - lr: 0.050000\n",
            "2023-04-21 06:35:17,410 epoch 4 - iter 28/79 - loss 344.75052743 - time (sec): 6.24 - samples/sec: 5317.45 - lr: 0.050000\n",
            "2023-04-21 06:35:18,831 epoch 4 - iter 35/79 - loss 325.03474198 - time (sec): 7.66 - samples/sec: 5504.44 - lr: 0.050000\n",
            "2023-04-21 06:35:20,196 epoch 4 - iter 42/79 - loss 314.14743744 - time (sec): 9.03 - samples/sec: 5525.78 - lr: 0.050000\n",
            "2023-04-21 06:35:22,024 epoch 4 - iter 49/79 - loss 303.96060466 - time (sec): 10.85 - samples/sec: 5400.17 - lr: 0.050000\n",
            "2023-04-21 06:35:23,457 epoch 4 - iter 56/79 - loss 290.27039646 - time (sec): 12.29 - samples/sec: 5465.03 - lr: 0.050000\n",
            "2023-04-21 06:35:24,597 epoch 4 - iter 63/79 - loss 286.53953526 - time (sec): 13.43 - samples/sec: 5522.60 - lr: 0.050000\n",
            "2023-04-21 06:35:26,352 epoch 4 - iter 70/79 - loss 278.46813855 - time (sec): 15.18 - samples/sec: 5467.26 - lr: 0.050000\n",
            "2023-04-21 06:35:27,773 epoch 4 - iter 77/79 - loss 270.22339075 - time (sec): 16.60 - samples/sec: 5549.68 - lr: 0.050000\n",
            "2023-04-21 06:35:28,396 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:35:28,397 EPOCH 4 done: loss 269.1341 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:48,078 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:52,063 TRAIN : loss 169.89540100097656 - f1-score (weighted avg)  0.4667\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:54,677 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:35:54,745 DEV : loss 211.85256958007812 - f1-score (weighted avg)  0.4282\n",
            "2023-04-21 06:35:54,755 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:35:54,759 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:35:56,548 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:35:57,808 epoch 5 - iter 7/79 - loss 200.10351302 - time (sec): 1.26 - samples/sec: 6558.01 - lr: 0.050000\n",
            "2023-04-21 06:35:59,653 epoch 5 - iter 14/79 - loss 200.80843547 - time (sec): 3.10 - samples/sec: 5152.22 - lr: 0.050000\n",
            "2023-04-21 06:36:01,313 epoch 5 - iter 21/79 - loss 185.78865896 - time (sec): 4.76 - samples/sec: 5173.59 - lr: 0.050000\n",
            "2023-04-21 06:36:03,139 epoch 5 - iter 28/79 - loss 185.52788163 - time (sec): 6.59 - samples/sec: 5235.73 - lr: 0.050000\n",
            "2023-04-21 06:36:04,769 epoch 5 - iter 35/79 - loss 191.47129698 - time (sec): 8.22 - samples/sec: 5295.98 - lr: 0.050000\n",
            "2023-04-21 06:36:06,092 epoch 5 - iter 42/79 - loss 193.82940717 - time (sec): 9.54 - samples/sec: 5364.58 - lr: 0.050000\n",
            "2023-04-21 06:36:07,583 epoch 5 - iter 49/79 - loss 190.71521798 - time (sec): 11.03 - samples/sec: 5367.89 - lr: 0.050000\n",
            "2023-04-21 06:36:08,981 epoch 5 - iter 56/79 - loss 193.04835058 - time (sec): 12.43 - samples/sec: 5401.14 - lr: 0.050000\n",
            "2023-04-21 06:36:10,711 epoch 5 - iter 63/79 - loss 190.11113982 - time (sec): 14.16 - samples/sec: 5328.30 - lr: 0.050000\n",
            "2023-04-21 06:36:12,151 epoch 5 - iter 70/79 - loss 189.36874724 - time (sec): 15.60 - samples/sec: 5362.53 - lr: 0.050000\n",
            "2023-04-21 06:36:13,579 epoch 5 - iter 77/79 - loss 188.64535138 - time (sec): 17.03 - samples/sec: 5376.26 - lr: 0.050000\n",
            "2023-04-21 06:36:14,128 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:36:14,130 EPOCH 5 done: loss 187.8055 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:36:33,834 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:36:36,538 TRAIN : loss 120.67204284667969 - f1-score (weighted avg)  0.5414\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:36:39,130 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:36:39,200 DEV : loss 158.92457580566406 - f1-score (weighted avg)  0.488\n",
            "2023-04-21 06:36:39,210 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:36:39,214 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:36:41,020 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:36:42,368 epoch 6 - iter 7/79 - loss 180.70372654 - time (sec): 1.35 - samples/sec: 5422.41 - lr: 0.050000\n",
            "2023-04-21 06:36:44,013 epoch 6 - iter 14/79 - loss 157.44287005 - time (sec): 2.99 - samples/sec: 5180.41 - lr: 0.050000\n",
            "2023-04-21 06:36:45,490 epoch 6 - iter 21/79 - loss 159.03819217 - time (sec): 4.47 - samples/sec: 5464.56 - lr: 0.050000\n",
            "2023-04-21 06:36:46,943 epoch 6 - iter 28/79 - loss 161.07211524 - time (sec): 5.92 - samples/sec: 5428.68 - lr: 0.050000\n",
            "2023-04-21 06:36:48,416 epoch 6 - iter 35/79 - loss 166.25833062 - time (sec): 7.39 - samples/sec: 5406.02 - lr: 0.050000\n",
            "2023-04-21 06:36:49,890 epoch 6 - iter 42/79 - loss 164.94545371 - time (sec): 8.87 - samples/sec: 5490.45 - lr: 0.050000\n",
            "2023-04-21 06:36:51,596 epoch 6 - iter 49/79 - loss 174.66523544 - time (sec): 10.57 - samples/sec: 5435.07 - lr: 0.050000\n",
            "2023-04-21 06:36:53,016 epoch 6 - iter 56/79 - loss 174.23391771 - time (sec): 11.99 - samples/sec: 5481.02 - lr: 0.050000\n",
            "2023-04-21 06:36:54,545 epoch 6 - iter 63/79 - loss 174.86125625 - time (sec): 13.52 - samples/sec: 5505.66 - lr: 0.050000\n",
            "2023-04-21 06:36:56,415 epoch 6 - iter 70/79 - loss 173.53364934 - time (sec): 15.39 - samples/sec: 5410.22 - lr: 0.050000\n",
            "2023-04-21 06:36:58,553 epoch 6 - iter 77/79 - loss 172.04298356 - time (sec): 17.53 - samples/sec: 5250.11 - lr: 0.050000\n",
            "2023-04-21 06:36:59,061 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:36:59,063 EPOCH 6 done: loss 171.7637 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.84it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:37:19,792 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:37:22,526 TRAIN : loss 113.7910385131836 - f1-score (weighted avg)  0.5091\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:37:25,132 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:37:25,201 DEV : loss 156.44677734375 - f1-score (weighted avg)  0.4495\n",
            "2023-04-21 06:37:25,211 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:37:25,215 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:37:27,000 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:37:28,480 epoch 7 - iter 7/79 - loss 152.08009650 - time (sec): 1.48 - samples/sec: 5610.09 - lr: 0.050000\n",
            "2023-04-21 06:37:30,367 epoch 7 - iter 14/79 - loss 150.92055605 - time (sec): 3.37 - samples/sec: 5322.76 - lr: 0.050000\n",
            "2023-04-21 06:37:32,210 epoch 7 - iter 21/79 - loss 154.41213257 - time (sec): 5.21 - samples/sec: 5173.75 - lr: 0.050000\n",
            "2023-04-21 06:37:33,642 epoch 7 - iter 28/79 - loss 161.69889232 - time (sec): 6.64 - samples/sec: 5220.72 - lr: 0.050000\n",
            "2023-04-21 06:37:35,054 epoch 7 - iter 35/79 - loss 164.60552146 - time (sec): 8.05 - samples/sec: 5301.41 - lr: 0.050000\n",
            "2023-04-21 06:37:36,651 epoch 7 - iter 42/79 - loss 165.55491901 - time (sec): 9.65 - samples/sec: 5214.80 - lr: 0.050000\n",
            "2023-04-21 06:37:37,903 epoch 7 - iter 49/79 - loss 166.49621536 - time (sec): 10.90 - samples/sec: 5356.79 - lr: 0.050000\n",
            "2023-04-21 06:37:39,411 epoch 7 - iter 56/79 - loss 170.74447561 - time (sec): 12.41 - samples/sec: 5372.05 - lr: 0.050000\n",
            "2023-04-21 06:37:41,002 epoch 7 - iter 63/79 - loss 174.77209170 - time (sec): 14.00 - samples/sec: 5287.31 - lr: 0.050000\n",
            "2023-04-21 06:37:42,739 epoch 7 - iter 70/79 - loss 174.21247002 - time (sec): 15.74 - samples/sec: 5299.59 - lr: 0.050000\n",
            "2023-04-21 06:37:44,288 epoch 7 - iter 77/79 - loss 174.75095686 - time (sec): 17.29 - samples/sec: 5350.60 - lr: 0.050000\n",
            "2023-04-21 06:37:44,740 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:37:44,742 EPOCH 7 done: loss 174.6586 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.07it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:04,293 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:08,278 TRAIN : loss 169.2458038330078 - f1-score (weighted avg)  0.4564\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:10,868 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:38:10,934 DEV : loss 214.14662170410156 - f1-score (weighted avg)  0.4101\n",
            "2023-04-21 06:38:10,943 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:38:10,947 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:12,426 epoch 8 - iter 7/79 - loss 156.43405398 - time (sec): 1.48 - samples/sec: 5291.76 - lr: 0.050000\n",
            "2023-04-21 06:38:13,835 epoch 8 - iter 14/79 - loss 157.91840927 - time (sec): 2.89 - samples/sec: 5650.00 - lr: 0.050000\n",
            "2023-04-21 06:38:15,887 epoch 8 - iter 21/79 - loss 149.48694331 - time (sec): 4.94 - samples/sec: 5397.07 - lr: 0.050000\n",
            "2023-04-21 06:38:17,505 epoch 8 - iter 28/79 - loss 147.97869018 - time (sec): 6.56 - samples/sec: 5421.69 - lr: 0.050000\n",
            "2023-04-21 06:38:19,246 epoch 8 - iter 35/79 - loss 146.33907761 - time (sec): 8.30 - samples/sec: 5346.57 - lr: 0.050000\n",
            "2023-04-21 06:38:20,604 epoch 8 - iter 42/79 - loss 145.85657073 - time (sec): 9.66 - samples/sec: 5404.72 - lr: 0.050000\n",
            "2023-04-21 06:38:21,826 epoch 8 - iter 49/79 - loss 148.59600712 - time (sec): 10.88 - samples/sec: 5446.52 - lr: 0.050000\n",
            "2023-04-21 06:38:23,565 epoch 8 - iter 56/79 - loss 148.04089736 - time (sec): 12.62 - samples/sec: 5374.90 - lr: 0.050000\n",
            "2023-04-21 06:38:24,992 epoch 8 - iter 63/79 - loss 155.73520418 - time (sec): 14.04 - samples/sec: 5414.29 - lr: 0.050000\n",
            "2023-04-21 06:38:26,504 epoch 8 - iter 70/79 - loss 157.22107783 - time (sec): 15.56 - samples/sec: 5391.31 - lr: 0.050000\n",
            "2023-04-21 06:38:28,068 epoch 8 - iter 77/79 - loss 157.96063919 - time (sec): 17.12 - samples/sec: 5420.07 - lr: 0.050000\n",
            "2023-04-21 06:38:28,416 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:38:28,417 EPOCH 8 done: loss 158.1735 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:48,124 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:50,867 TRAIN : loss 101.64462280273438 - f1-score (weighted avg)  0.5729\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:53,464 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:38:53,533 DEV : loss 148.8850555419922 - f1-score (weighted avg)  0.5232\n",
            "2023-04-21 06:38:53,542 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:38:53,547 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:38:55,320 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:38:57,065 epoch 9 - iter 7/79 - loss 163.64700042 - time (sec): 1.74 - samples/sec: 5490.71 - lr: 0.050000\n",
            "2023-04-21 06:38:58,505 epoch 9 - iter 14/79 - loss 156.25859281 - time (sec): 3.18 - samples/sec: 5731.90 - lr: 0.050000\n",
            "2023-04-21 06:39:00,083 epoch 9 - iter 21/79 - loss 164.69696433 - time (sec): 4.76 - samples/sec: 5629.23 - lr: 0.050000\n",
            "2023-04-21 06:39:02,030 epoch 9 - iter 28/79 - loss 163.19165457 - time (sec): 6.71 - samples/sec: 5236.95 - lr: 0.050000\n",
            "2023-04-21 06:39:03,366 epoch 9 - iter 35/79 - loss 164.24483717 - time (sec): 8.04 - samples/sec: 5279.78 - lr: 0.050000\n",
            "2023-04-21 06:39:04,842 epoch 9 - iter 42/79 - loss 166.73932459 - time (sec): 9.52 - samples/sec: 5240.02 - lr: 0.050000\n",
            "2023-04-21 06:39:06,825 epoch 9 - iter 49/79 - loss 165.27160968 - time (sec): 11.50 - samples/sec: 5153.51 - lr: 0.050000\n",
            "2023-04-21 06:39:08,267 epoch 9 - iter 56/79 - loss 166.78207857 - time (sec): 12.94 - samples/sec: 5244.58 - lr: 0.050000\n",
            "2023-04-21 06:39:09,798 epoch 9 - iter 63/79 - loss 169.19971204 - time (sec): 14.48 - samples/sec: 5289.80 - lr: 0.050000\n",
            "2023-04-21 06:39:11,235 epoch 9 - iter 70/79 - loss 169.69503486 - time (sec): 15.91 - samples/sec: 5354.41 - lr: 0.050000\n",
            "2023-04-21 06:39:12,603 epoch 9 - iter 77/79 - loss 169.61894445 - time (sec): 17.28 - samples/sec: 5355.16 - lr: 0.050000\n",
            "2023-04-21 06:39:13,045 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:39:13,047 EPOCH 9 done: loss 169.1316 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:39:33,810 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:39:36,525 TRAIN : loss 161.5712890625 - f1-score (weighted avg)  0.4473\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:39:39,106 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:39:39,174 DEV : loss 205.24176025390625 - f1-score (weighted avg)  0.4216\n",
            "2023-04-21 06:39:39,183 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:39:39,187 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:39:40,692 epoch 10 - iter 7/79 - loss 177.59155018 - time (sec): 1.50 - samples/sec: 5279.90 - lr: 0.050000\n",
            "2023-04-21 06:39:42,166 epoch 10 - iter 14/79 - loss 171.47624582 - time (sec): 2.98 - samples/sec: 5417.78 - lr: 0.050000\n",
            "2023-04-21 06:39:44,146 epoch 10 - iter 21/79 - loss 163.52083415 - time (sec): 4.96 - samples/sec: 5123.70 - lr: 0.050000\n",
            "2023-04-21 06:39:45,608 epoch 10 - iter 28/79 - loss 166.25523139 - time (sec): 6.42 - samples/sec: 5277.44 - lr: 0.050000\n",
            "2023-04-21 06:39:46,875 epoch 10 - iter 35/79 - loss 161.76252914 - time (sec): 7.69 - samples/sec: 5523.66 - lr: 0.050000\n",
            "2023-04-21 06:39:48,535 epoch 10 - iter 42/79 - loss 160.76380239 - time (sec): 9.35 - samples/sec: 5480.53 - lr: 0.050000\n",
            "2023-04-21 06:39:50,022 epoch 10 - iter 49/79 - loss 155.93009398 - time (sec): 10.83 - samples/sec: 5421.80 - lr: 0.050000\n",
            "2023-04-21 06:39:51,330 epoch 10 - iter 56/79 - loss 158.62080395 - time (sec): 12.14 - samples/sec: 5453.97 - lr: 0.050000\n",
            "2023-04-21 06:39:53,220 epoch 10 - iter 63/79 - loss 160.72321173 - time (sec): 14.03 - samples/sec: 5473.80 - lr: 0.050000\n",
            "2023-04-21 06:39:54,571 epoch 10 - iter 70/79 - loss 162.98672076 - time (sec): 15.38 - samples/sec: 5512.33 - lr: 0.050000\n",
            "2023-04-21 06:39:56,094 epoch 10 - iter 77/79 - loss 162.17169344 - time (sec): 16.91 - samples/sec: 5487.59 - lr: 0.050000\n",
            "2023-04-21 06:39:56,497 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:39:56,498 EPOCH 10 done: loss 161.5928 - lr 0.050000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:40:16,180 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:40:18,908 TRAIN : loss 104.14134216308594 - f1-score (weighted avg)  0.5495\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.01it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:40:21,482 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:40:21,551 DEV : loss 163.7177734375 - f1-score (weighted avg)  0.4818\n",
            "2023-04-21 06:40:21,560 Epoch    10: reducing learning rate of group 0 to 2.5000e-02.\n",
            "2023-04-21 06:40:21,562 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:40:21,566 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:40:23,028 epoch 11 - iter 7/79 - loss 150.29180805 - time (sec): 1.46 - samples/sec: 5648.06 - lr: 0.025000\n",
            "2023-04-21 06:40:24,517 epoch 11 - iter 14/79 - loss 144.43077730 - time (sec): 2.95 - samples/sec: 5818.46 - lr: 0.025000\n",
            "2023-04-21 06:40:26,002 epoch 11 - iter 21/79 - loss 137.39283481 - time (sec): 4.44 - samples/sec: 5769.21 - lr: 0.025000\n",
            "2023-04-21 06:40:27,541 epoch 11 - iter 28/79 - loss 134.29979042 - time (sec): 5.97 - samples/sec: 5750.67 - lr: 0.025000\n",
            "2023-04-21 06:40:29,512 epoch 11 - iter 35/79 - loss 127.10896116 - time (sec): 7.95 - samples/sec: 5441.01 - lr: 0.025000\n",
            "2023-04-21 06:40:31,170 epoch 11 - iter 42/79 - loss 121.97383228 - time (sec): 9.60 - samples/sec: 5339.49 - lr: 0.025000\n",
            "2023-04-21 06:40:32,405 epoch 11 - iter 49/79 - loss 120.37203299 - time (sec): 10.84 - samples/sec: 5415.05 - lr: 0.025000\n",
            "2023-04-21 06:40:33,903 epoch 11 - iter 56/79 - loss 119.00384356 - time (sec): 12.34 - samples/sec: 5415.02 - lr: 0.025000\n",
            "2023-04-21 06:40:35,316 epoch 11 - iter 63/79 - loss 117.77190732 - time (sec): 13.75 - samples/sec: 5400.66 - lr: 0.025000\n",
            "2023-04-21 06:40:36,740 epoch 11 - iter 70/79 - loss 115.07711900 - time (sec): 15.17 - samples/sec: 5485.70 - lr: 0.025000\n",
            "2023-04-21 06:40:38,509 epoch 11 - iter 77/79 - loss 112.74883121 - time (sec): 16.94 - samples/sec: 5453.98 - lr: 0.025000\n",
            "2023-04-21 06:40:38,898 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:40:38,900 EPOCH 11 done: loss 111.9479 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.83it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:40:59,639 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:02,361 TRAIN : loss 61.58018493652344 - f1-score (weighted avg)  0.586\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:04,945 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:41:05,016 DEV : loss 111.6823501586914 - f1-score (weighted avg)  0.5283\n",
            "2023-04-21 06:41:05,025 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:41:05,030 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:06,844 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:41:08,216 epoch 12 - iter 7/79 - loss 88.91864986 - time (sec): 1.37 - samples/sec: 6193.70 - lr: 0.025000\n",
            "2023-04-21 06:41:10,061 epoch 12 - iter 14/79 - loss 83.55527596 - time (sec): 3.21 - samples/sec: 5562.95 - lr: 0.025000\n",
            "2023-04-21 06:41:11,589 epoch 12 - iter 21/79 - loss 84.57307167 - time (sec): 4.74 - samples/sec: 5553.78 - lr: 0.025000\n",
            "2023-04-21 06:41:13,305 epoch 12 - iter 28/79 - loss 84.46211449 - time (sec): 6.46 - samples/sec: 5406.08 - lr: 0.025000\n",
            "2023-04-21 06:41:14,920 epoch 12 - iter 35/79 - loss 89.59792483 - time (sec): 8.07 - samples/sec: 5344.56 - lr: 0.025000\n",
            "2023-04-21 06:41:16,727 epoch 12 - iter 42/79 - loss 91.30128242 - time (sec): 9.88 - samples/sec: 5287.41 - lr: 0.025000\n",
            "2023-04-21 06:41:18,000 epoch 12 - iter 49/79 - loss 91.69350271 - time (sec): 11.15 - samples/sec: 5413.23 - lr: 0.025000\n",
            "2023-04-21 06:41:19,628 epoch 12 - iter 56/79 - loss 92.73177376 - time (sec): 12.78 - samples/sec: 5355.95 - lr: 0.025000\n",
            "2023-04-21 06:41:21,312 epoch 12 - iter 63/79 - loss 91.69903452 - time (sec): 14.47 - samples/sec: 5291.22 - lr: 0.025000\n",
            "2023-04-21 06:41:22,939 epoch 12 - iter 70/79 - loss 91.25175265 - time (sec): 16.09 - samples/sec: 5282.72 - lr: 0.025000\n",
            "2023-04-21 06:41:24,315 epoch 12 - iter 77/79 - loss 91.60801342 - time (sec): 17.47 - samples/sec: 5297.86 - lr: 0.025000\n",
            "2023-04-21 06:41:24,809 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:41:24,810 EPOCH 12 done: loss 91.2877 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:45,548 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:41:48,295 TRAIN : loss 54.17228317260742 - f1-score (weighted avg)  0.6008\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.90it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:50,935 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:41:51,005 DEV : loss 98.98755645751953 - f1-score (weighted avg)  0.524\n",
            "2023-04-21 06:41:51,014 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:41:51,018 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:41:52,811 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:41:54,178 epoch 13 - iter 7/79 - loss 88.13513104 - time (sec): 1.37 - samples/sec: 6037.40 - lr: 0.025000\n",
            "2023-04-21 06:41:55,772 epoch 13 - iter 14/79 - loss 88.34911705 - time (sec): 2.96 - samples/sec: 5512.42 - lr: 0.025000\n",
            "2023-04-21 06:41:57,282 epoch 13 - iter 21/79 - loss 88.80637597 - time (sec): 4.47 - samples/sec: 5606.71 - lr: 0.025000\n",
            "2023-04-21 06:41:58,797 epoch 13 - iter 28/79 - loss 87.81762343 - time (sec): 5.98 - samples/sec: 5740.44 - lr: 0.025000\n",
            "2023-04-21 06:42:00,070 epoch 13 - iter 35/79 - loss 83.67438596 - time (sec): 7.26 - samples/sec: 5694.43 - lr: 0.025000\n",
            "2023-04-21 06:42:01,625 epoch 13 - iter 42/79 - loss 83.59050433 - time (sec): 8.81 - samples/sec: 5607.41 - lr: 0.025000\n",
            "2023-04-21 06:42:03,411 epoch 13 - iter 49/79 - loss 81.58611779 - time (sec): 10.60 - samples/sec: 5376.89 - lr: 0.025000\n",
            "2023-04-21 06:42:05,041 epoch 13 - iter 56/79 - loss 83.84897198 - time (sec): 12.23 - samples/sec: 5337.73 - lr: 0.025000\n",
            "2023-04-21 06:42:06,898 epoch 13 - iter 63/79 - loss 81.91519783 - time (sec): 14.08 - samples/sec: 5349.40 - lr: 0.025000\n",
            "2023-04-21 06:42:08,257 epoch 13 - iter 70/79 - loss 81.67801291 - time (sec): 15.44 - samples/sec: 5457.16 - lr: 0.025000\n",
            "2023-04-21 06:42:09,813 epoch 13 - iter 77/79 - loss 81.24308898 - time (sec): 17.00 - samples/sec: 5438.91 - lr: 0.025000\n",
            "2023-04-21 06:42:10,195 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:42:10,197 EPOCH 13 done: loss 81.2383 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:42:29,926 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:42:32,697 TRAIN : loss 43.11811447143555 - f1-score (weighted avg)  0.583\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:03<00:00,  2.77it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:42:36,380 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:42:36,450 DEV : loss 84.73545837402344 - f1-score (weighted avg)  0.5111\n",
            "2023-04-21 06:42:36,459 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:42:36,463 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:42:38,261 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:42:39,972 epoch 14 - iter 7/79 - loss 73.83796913 - time (sec): 1.71 - samples/sec: 5271.89 - lr: 0.025000\n",
            "2023-04-21 06:42:41,514 epoch 14 - iter 14/79 - loss 73.13464428 - time (sec): 3.25 - samples/sec: 5324.20 - lr: 0.025000\n",
            "2023-04-21 06:42:42,941 epoch 14 - iter 21/79 - loss 71.56508337 - time (sec): 4.68 - samples/sec: 5462.43 - lr: 0.025000\n",
            "2023-04-21 06:42:44,595 epoch 14 - iter 28/79 - loss 69.33781675 - time (sec): 6.33 - samples/sec: 5516.61 - lr: 0.025000\n",
            "2023-04-21 06:42:46,019 epoch 14 - iter 35/79 - loss 72.19016148 - time (sec): 7.76 - samples/sec: 5501.67 - lr: 0.025000\n",
            "2023-04-21 06:42:47,498 epoch 14 - iter 42/79 - loss 76.61369771 - time (sec): 9.23 - samples/sec: 5537.12 - lr: 0.025000\n",
            "2023-04-21 06:42:49,344 epoch 14 - iter 49/79 - loss 76.98334744 - time (sec): 11.08 - samples/sec: 5437.46 - lr: 0.025000\n",
            "2023-04-21 06:42:51,166 epoch 14 - iter 56/79 - loss 77.14701905 - time (sec): 12.90 - samples/sec: 5301.45 - lr: 0.025000\n",
            "2023-04-21 06:42:52,539 epoch 14 - iter 63/79 - loss 76.74406685 - time (sec): 14.28 - samples/sec: 5289.53 - lr: 0.025000\n",
            "2023-04-21 06:42:53,952 epoch 14 - iter 70/79 - loss 76.60532462 - time (sec): 15.69 - samples/sec: 5376.39 - lr: 0.025000\n",
            "2023-04-21 06:42:55,382 epoch 14 - iter 77/79 - loss 77.40168512 - time (sec): 17.12 - samples/sec: 5356.06 - lr: 0.025000\n",
            "2023-04-21 06:42:55,998 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:42:55,999 EPOCH 14 done: loss 77.2644 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:43:15,757 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:43:18,469 TRAIN : loss 47.37251663208008 - f1-score (weighted avg)  0.5901\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:43:21,065 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:43:21,133 DEV : loss 92.46317291259766 - f1-score (weighted avg)  0.5041\n",
            "2023-04-21 06:43:21,142 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:43:21,146 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:43:22,831 epoch 15 - iter 7/79 - loss 67.51546477 - time (sec): 1.68 - samples/sec: 5387.24 - lr: 0.025000\n",
            "2023-04-21 06:43:24,149 epoch 15 - iter 14/79 - loss 69.25066744 - time (sec): 3.00 - samples/sec: 5741.43 - lr: 0.025000\n",
            "2023-04-21 06:43:25,374 epoch 15 - iter 21/79 - loss 69.80302200 - time (sec): 4.23 - samples/sec: 5829.26 - lr: 0.025000\n",
            "2023-04-21 06:43:26,832 epoch 15 - iter 28/79 - loss 68.14311660 - time (sec): 5.68 - samples/sec: 5787.89 - lr: 0.025000\n",
            "2023-04-21 06:43:28,305 epoch 15 - iter 35/79 - loss 71.01505438 - time (sec): 7.16 - samples/sec: 5742.38 - lr: 0.025000\n",
            "2023-04-21 06:43:30,131 epoch 15 - iter 42/79 - loss 70.30446158 - time (sec): 8.98 - samples/sec: 5616.12 - lr: 0.025000\n",
            "2023-04-21 06:43:31,634 epoch 15 - iter 49/79 - loss 71.58550787 - time (sec): 10.49 - samples/sec: 5576.94 - lr: 0.025000\n",
            "2023-04-21 06:43:33,830 epoch 15 - iter 56/79 - loss 71.94614623 - time (sec): 12.68 - samples/sec: 5376.15 - lr: 0.025000\n",
            "2023-04-21 06:43:35,196 epoch 15 - iter 63/79 - loss 72.15820320 - time (sec): 14.05 - samples/sec: 5428.97 - lr: 0.025000\n",
            "2023-04-21 06:43:36,694 epoch 15 - iter 70/79 - loss 72.17243882 - time (sec): 15.55 - samples/sec: 5406.62 - lr: 0.025000\n",
            "2023-04-21 06:43:38,206 epoch 15 - iter 77/79 - loss 72.21862095 - time (sec): 17.06 - samples/sec: 5408.31 - lr: 0.025000\n",
            "2023-04-21 06:43:38,726 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:43:38,728 EPOCH 15 done: loss 72.2440 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.82it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:43:59,530 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:02,261 TRAIN : loss 36.1077995300293 - f1-score (weighted avg)  0.6431\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:04,838 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:44:04,909 DEV : loss 76.43682861328125 - f1-score (weighted avg)  0.5595\n",
            "2023-04-21 06:44:04,918 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:44:04,922 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:06,724 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:44:08,148 epoch 16 - iter 7/79 - loss 68.06825839 - time (sec): 1.42 - samples/sec: 6051.74 - lr: 0.025000\n",
            "2023-04-21 06:44:09,873 epoch 16 - iter 14/79 - loss 62.42020671 - time (sec): 3.15 - samples/sec: 5580.94 - lr: 0.025000\n",
            "2023-04-21 06:44:11,579 epoch 16 - iter 21/79 - loss 63.89735709 - time (sec): 4.85 - samples/sec: 5262.80 - lr: 0.025000\n",
            "2023-04-21 06:44:12,954 epoch 16 - iter 28/79 - loss 65.27872637 - time (sec): 6.23 - samples/sec: 5580.75 - lr: 0.025000\n",
            "2023-04-21 06:44:14,443 epoch 16 - iter 35/79 - loss 66.44611326 - time (sec): 7.72 - samples/sec: 5599.61 - lr: 0.025000\n",
            "2023-04-21 06:44:16,288 epoch 16 - iter 42/79 - loss 67.08694714 - time (sec): 9.56 - samples/sec: 5452.42 - lr: 0.025000\n",
            "2023-04-21 06:44:17,791 epoch 16 - iter 49/79 - loss 67.89655357 - time (sec): 11.06 - samples/sec: 5484.66 - lr: 0.025000\n",
            "2023-04-21 06:44:19,168 epoch 16 - iter 56/79 - loss 67.11612960 - time (sec): 12.44 - samples/sec: 5452.53 - lr: 0.025000\n",
            "2023-04-21 06:44:20,624 epoch 16 - iter 63/79 - loss 68.16367530 - time (sec): 13.90 - samples/sec: 5417.90 - lr: 0.025000\n",
            "2023-04-21 06:44:22,478 epoch 16 - iter 70/79 - loss 68.87802240 - time (sec): 15.75 - samples/sec: 5327.41 - lr: 0.025000\n",
            "2023-04-21 06:44:23,987 epoch 16 - iter 77/79 - loss 69.91936216 - time (sec): 17.26 - samples/sec: 5362.33 - lr: 0.025000\n",
            "2023-04-21 06:44:24,410 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:44:24,412 EPOCH 16 done: loss 70.4578 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:44,022 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:48,007 TRAIN : loss 40.07545852661133 - f1-score (weighted avg)  0.6351\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.89it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:50,657 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:44:50,730 DEV : loss 78.54093933105469 - f1-score (weighted avg)  0.5694\n",
            "2023-04-21 06:44:50,739 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:44:50,744 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:44:52,095 epoch 17 - iter 7/79 - loss 66.33061624 - time (sec): 1.35 - samples/sec: 6815.72 - lr: 0.025000\n",
            "2023-04-21 06:44:53,507 epoch 17 - iter 14/79 - loss 67.21633107 - time (sec): 2.76 - samples/sec: 6348.41 - lr: 0.025000\n",
            "2023-04-21 06:44:55,395 epoch 17 - iter 21/79 - loss 64.85576960 - time (sec): 4.65 - samples/sec: 5677.74 - lr: 0.025000\n",
            "2023-04-21 06:44:56,838 epoch 17 - iter 28/79 - loss 65.37835077 - time (sec): 6.09 - samples/sec: 5583.95 - lr: 0.025000\n",
            "2023-04-21 06:44:58,743 epoch 17 - iter 35/79 - loss 68.20609773 - time (sec): 8.00 - samples/sec: 5404.49 - lr: 0.025000\n",
            "2023-04-21 06:45:00,182 epoch 17 - iter 42/79 - loss 67.86223436 - time (sec): 9.44 - samples/sec: 5440.47 - lr: 0.025000\n",
            "2023-04-21 06:45:01,544 epoch 17 - iter 49/79 - loss 67.84778091 - time (sec): 10.80 - samples/sec: 5481.53 - lr: 0.025000\n",
            "2023-04-21 06:45:02,981 epoch 17 - iter 56/79 - loss 68.20396581 - time (sec): 12.24 - samples/sec: 5540.37 - lr: 0.025000\n",
            "2023-04-21 06:45:04,541 epoch 17 - iter 63/79 - loss 67.89656456 - time (sec): 13.80 - samples/sec: 5454.41 - lr: 0.025000\n",
            "2023-04-21 06:45:06,116 epoch 17 - iter 70/79 - loss 69.68869799 - time (sec): 15.37 - samples/sec: 5451.08 - lr: 0.025000\n",
            "2023-04-21 06:45:08,023 epoch 17 - iter 77/79 - loss 70.35959768 - time (sec): 17.28 - samples/sec: 5368.96 - lr: 0.025000\n",
            "2023-04-21 06:45:08,390 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:45:08,392 EPOCH 17 done: loss 70.6549 - lr 0.025000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:45:28,022 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:45:30,756 TRAIN : loss 44.49900817871094 - f1-score (weighted avg)  0.533\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.89it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:45:33,404 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:45:33,473 DEV : loss 81.85240936279297 - f1-score (weighted avg)  0.4674\n",
            "2023-04-21 06:45:33,482 Epoch    17: reducing learning rate of group 0 to 1.2500e-02.\n",
            "2023-04-21 06:45:33,484 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:45:33,488 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:45:34,853 epoch 18 - iter 7/79 - loss 54.36865342 - time (sec): 1.36 - samples/sec: 5648.30 - lr: 0.012500\n",
            "2023-04-21 06:45:36,315 epoch 18 - iter 14/79 - loss 60.85869760 - time (sec): 2.83 - samples/sec: 5926.35 - lr: 0.012500\n",
            "2023-04-21 06:45:37,784 epoch 18 - iter 21/79 - loss 59.43976598 - time (sec): 4.29 - samples/sec: 5760.92 - lr: 0.012500\n",
            "2023-04-21 06:45:39,040 epoch 18 - iter 28/79 - loss 58.06924046 - time (sec): 5.55 - samples/sec: 5937.17 - lr: 0.012500\n",
            "2023-04-21 06:45:40,848 epoch 18 - iter 35/79 - loss 56.81764850 - time (sec): 7.36 - samples/sec: 5559.51 - lr: 0.012500\n",
            "2023-04-21 06:45:42,255 epoch 18 - iter 42/79 - loss 55.78407296 - time (sec): 8.77 - samples/sec: 5632.71 - lr: 0.012500\n",
            "2023-04-21 06:45:43,643 epoch 18 - iter 49/79 - loss 56.96832863 - time (sec): 10.15 - samples/sec: 5653.09 - lr: 0.012500\n",
            "2023-04-21 06:45:44,992 epoch 18 - iter 56/79 - loss 56.41587452 - time (sec): 11.50 - samples/sec: 5715.99 - lr: 0.012500\n",
            "2023-04-21 06:45:46,403 epoch 18 - iter 63/79 - loss 56.04024228 - time (sec): 12.91 - samples/sec: 5708.19 - lr: 0.012500\n",
            "2023-04-21 06:45:47,921 epoch 18 - iter 70/79 - loss 55.15745165 - time (sec): 14.43 - samples/sec: 5699.87 - lr: 0.012500\n",
            "2023-04-21 06:45:49,926 epoch 18 - iter 77/79 - loss 55.05528435 - time (sec): 16.44 - samples/sec: 5589.51 - lr: 0.012500\n",
            "2023-04-21 06:45:50,470 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:45:50,471 EPOCH 18 done: loss 55.0752 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:46:11,233 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:46:13,968 TRAIN : loss 26.89131736755371 - f1-score (weighted avg)  0.6319\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:46:16,581 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:46:16,651 DEV : loss 62.74849319458008 - f1-score (weighted avg)  0.5285\n",
            "2023-04-21 06:46:16,660 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:46:16,664 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:46:18,448 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:46:20,247 epoch 19 - iter 7/79 - loss 41.34369842 - time (sec): 1.80 - samples/sec: 4719.76 - lr: 0.012500\n",
            "2023-04-21 06:46:21,735 epoch 19 - iter 14/79 - loss 42.99800308 - time (sec): 3.28 - samples/sec: 5145.41 - lr: 0.012500\n",
            "2023-04-21 06:46:23,361 epoch 19 - iter 21/79 - loss 41.85633903 - time (sec): 4.91 - samples/sec: 5274.63 - lr: 0.012500\n",
            "2023-04-21 06:46:25,279 epoch 19 - iter 28/79 - loss 40.86260046 - time (sec): 6.83 - samples/sec: 5101.52 - lr: 0.012500\n",
            "2023-04-21 06:46:27,040 epoch 19 - iter 35/79 - loss 41.59581155 - time (sec): 8.59 - samples/sec: 5055.44 - lr: 0.012500\n",
            "2023-04-21 06:46:28,544 epoch 19 - iter 42/79 - loss 41.61109959 - time (sec): 10.09 - samples/sec: 5016.47 - lr: 0.012500\n",
            "2023-04-21 06:46:30,172 epoch 19 - iter 49/79 - loss 42.21734162 - time (sec): 11.72 - samples/sec: 5057.71 - lr: 0.012500\n",
            "2023-04-21 06:46:31,827 epoch 19 - iter 56/79 - loss 43.71503031 - time (sec): 13.38 - samples/sec: 5055.68 - lr: 0.012500\n",
            "2023-04-21 06:46:33,148 epoch 19 - iter 63/79 - loss 43.93024604 - time (sec): 14.70 - samples/sec: 5165.64 - lr: 0.012500\n",
            "2023-04-21 06:46:34,578 epoch 19 - iter 70/79 - loss 43.22397829 - time (sec): 16.13 - samples/sec: 5209.86 - lr: 0.012500\n",
            "2023-04-21 06:46:36,078 epoch 19 - iter 77/79 - loss 43.25946771 - time (sec): 17.63 - samples/sec: 5241.10 - lr: 0.012500\n",
            "2023-04-21 06:46:36,605 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:46:36,606 EPOCH 19 done: loss 43.2972 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:46:56,343 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:46:59,068 TRAIN : loss 33.19255065917969 - f1-score (weighted avg)  0.5305\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:47:02,758 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:47:02,830 DEV : loss 66.67395782470703 - f1-score (weighted avg)  0.4747\n",
            "2023-04-21 06:47:02,839 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:47:02,843 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:47:04,307 epoch 20 - iter 7/79 - loss 40.78508214 - time (sec): 1.46 - samples/sec: 5913.55 - lr: 0.012500\n",
            "2023-04-21 06:47:05,914 epoch 20 - iter 14/79 - loss 42.22411444 - time (sec): 3.07 - samples/sec: 5381.94 - lr: 0.012500\n",
            "2023-04-21 06:47:07,373 epoch 20 - iter 21/79 - loss 40.91729867 - time (sec): 4.53 - samples/sec: 5648.90 - lr: 0.012500\n",
            "2023-04-21 06:47:08,911 epoch 20 - iter 28/79 - loss 40.95349407 - time (sec): 6.07 - samples/sec: 5715.51 - lr: 0.012500\n",
            "2023-04-21 06:47:10,469 epoch 20 - iter 35/79 - loss 39.22507811 - time (sec): 7.62 - samples/sec: 5751.12 - lr: 0.012500\n",
            "2023-04-21 06:47:12,124 epoch 20 - iter 42/79 - loss 38.81099442 - time (sec): 9.28 - samples/sec: 5694.81 - lr: 0.012500\n",
            "2023-04-21 06:47:13,398 epoch 20 - iter 49/79 - loss 38.98730074 - time (sec): 10.55 - samples/sec: 5736.32 - lr: 0.012500\n",
            "2023-04-21 06:47:14,706 epoch 20 - iter 56/79 - loss 40.18668523 - time (sec): 11.86 - samples/sec: 5723.33 - lr: 0.012500\n",
            "2023-04-21 06:47:16,049 epoch 20 - iter 63/79 - loss 39.91928663 - time (sec): 13.20 - samples/sec: 5694.78 - lr: 0.012500\n",
            "2023-04-21 06:47:18,078 epoch 20 - iter 70/79 - loss 39.56553398 - time (sec): 15.23 - samples/sec: 5497.06 - lr: 0.012500\n",
            "2023-04-21 06:47:19,704 epoch 20 - iter 77/79 - loss 40.72860213 - time (sec): 16.86 - samples/sec: 5492.19 - lr: 0.012500\n",
            "2023-04-21 06:47:20,059 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:47:20,061 EPOCH 20 done: loss 40.4544 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:47:39,703 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:47:42,430 TRAIN : loss 24.0615291595459 - f1-score (weighted avg)  0.5525\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:47:45,037 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:47:45,104 DEV : loss 56.29933547973633 - f1-score (weighted avg)  0.4861\n",
            "2023-04-21 06:47:45,113 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:47:45,117 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:47:46,905 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:47:48,422 epoch 21 - iter 7/79 - loss 38.62589189 - time (sec): 1.51 - samples/sec: 5460.51 - lr: 0.012500\n",
            "2023-04-21 06:47:50,155 epoch 21 - iter 14/79 - loss 43.11736696 - time (sec): 3.25 - samples/sec: 5335.55 - lr: 0.012500\n",
            "2023-04-21 06:47:51,662 epoch 21 - iter 21/79 - loss 42.03060980 - time (sec): 4.75 - samples/sec: 5266.76 - lr: 0.012500\n",
            "2023-04-21 06:47:53,114 epoch 21 - iter 28/79 - loss 41.08282222 - time (sec): 6.21 - samples/sec: 5332.50 - lr: 0.012500\n",
            "2023-04-21 06:47:54,722 epoch 21 - iter 35/79 - loss 40.27950775 - time (sec): 7.81 - samples/sec: 5391.44 - lr: 0.012500\n",
            "2023-04-21 06:47:55,960 epoch 21 - iter 42/79 - loss 39.18415028 - time (sec): 9.05 - samples/sec: 5476.52 - lr: 0.012500\n",
            "2023-04-21 06:47:57,384 epoch 21 - iter 49/79 - loss 39.93400413 - time (sec): 10.48 - samples/sec: 5452.02 - lr: 0.012500\n",
            "2023-04-21 06:47:59,038 epoch 21 - iter 56/79 - loss 39.11213466 - time (sec): 12.13 - samples/sec: 5463.00 - lr: 0.012500\n",
            "2023-04-21 06:48:01,068 epoch 21 - iter 63/79 - loss 40.09988857 - time (sec): 14.16 - samples/sec: 5323.41 - lr: 0.012500\n",
            "2023-04-21 06:48:02,588 epoch 21 - iter 70/79 - loss 40.12400789 - time (sec): 15.68 - samples/sec: 5383.58 - lr: 0.012500\n",
            "2023-04-21 06:48:04,195 epoch 21 - iter 77/79 - loss 39.89105238 - time (sec): 17.29 - samples/sec: 5352.37 - lr: 0.012500\n",
            "2023-04-21 06:48:04,608 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:48:04,610 EPOCH 21 done: loss 39.9035 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  3.99it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:48:25,684 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:48:28,417 TRAIN : loss 20.15817642211914 - f1-score (weighted avg)  0.6347\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:48:31,029 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:48:31,097 DEV : loss 52.01753616333008 - f1-score (weighted avg)  0.5595\n",
            "2023-04-21 06:48:31,106 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:48:31,110 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:48:32,915 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:48:34,428 epoch 22 - iter 7/79 - loss 31.79551459 - time (sec): 1.51 - samples/sec: 5514.53 - lr: 0.012500\n",
            "2023-04-21 06:48:35,796 epoch 22 - iter 14/79 - loss 32.12396469 - time (sec): 2.88 - samples/sec: 5746.69 - lr: 0.012500\n",
            "2023-04-21 06:48:37,253 epoch 22 - iter 21/79 - loss 34.90525355 - time (sec): 4.34 - samples/sec: 5686.12 - lr: 0.012500\n",
            "2023-04-21 06:48:38,661 epoch 22 - iter 28/79 - loss 35.34327438 - time (sec): 5.74 - samples/sec: 5719.88 - lr: 0.012500\n",
            "2023-04-21 06:48:40,243 epoch 22 - iter 35/79 - loss 35.61042934 - time (sec): 7.32 - samples/sec: 5679.39 - lr: 0.012500\n",
            "2023-04-21 06:48:42,181 epoch 22 - iter 42/79 - loss 36.06146713 - time (sec): 9.26 - samples/sec: 5559.36 - lr: 0.012500\n",
            "2023-04-21 06:48:43,720 epoch 22 - iter 49/79 - loss 35.71982359 - time (sec): 10.80 - samples/sec: 5494.37 - lr: 0.012500\n",
            "2023-04-21 06:48:45,498 epoch 22 - iter 56/79 - loss 36.53891182 - time (sec): 12.58 - samples/sec: 5411.42 - lr: 0.012500\n",
            "2023-04-21 06:48:47,573 epoch 22 - iter 63/79 - loss 35.64837885 - time (sec): 14.66 - samples/sec: 5271.78 - lr: 0.012500\n",
            "2023-04-21 06:48:49,008 epoch 22 - iter 70/79 - loss 36.01168067 - time (sec): 16.09 - samples/sec: 5304.21 - lr: 0.012500\n",
            "2023-04-21 06:48:50,509 epoch 22 - iter 77/79 - loss 35.81430290 - time (sec): 17.59 - samples/sec: 5267.53 - lr: 0.012500\n",
            "2023-04-21 06:48:50,826 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:48:50,828 EPOCH 22 done: loss 35.7062 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:10,456 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:13,189 TRAIN : loss 16.528093338012695 - f1-score (weighted avg)  0.6704\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:03<00:00,  2.74it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:16,918 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:49:16,988 DEV : loss 48.495853424072266 - f1-score (weighted avg)  0.5749\n",
            "2023-04-21 06:49:16,997 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:49:17,001 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:18,786 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:49:20,341 epoch 23 - iter 7/79 - loss 38.44489231 - time (sec): 1.55 - samples/sec: 5473.81 - lr: 0.012500\n",
            "2023-04-21 06:49:22,280 epoch 23 - iter 14/79 - loss 36.93903537 - time (sec): 3.49 - samples/sec: 4932.11 - lr: 0.012500\n",
            "2023-04-21 06:49:23,633 epoch 23 - iter 21/79 - loss 38.11800895 - time (sec): 4.85 - samples/sec: 5074.16 - lr: 0.012500\n",
            "2023-04-21 06:49:25,411 epoch 23 - iter 28/79 - loss 37.91367485 - time (sec): 6.62 - samples/sec: 5118.23 - lr: 0.012500\n",
            "2023-04-21 06:49:27,293 epoch 23 - iter 35/79 - loss 37.39749102 - time (sec): 8.51 - samples/sec: 4908.82 - lr: 0.012500\n",
            "2023-04-21 06:49:28,934 epoch 23 - iter 42/79 - loss 37.14809709 - time (sec): 10.15 - samples/sec: 4879.02 - lr: 0.012500\n",
            "2023-04-21 06:49:30,653 epoch 23 - iter 49/79 - loss 37.14481633 - time (sec): 11.86 - samples/sec: 4855.18 - lr: 0.012500\n",
            "2023-04-21 06:49:32,062 epoch 23 - iter 56/79 - loss 36.78470820 - time (sec): 13.27 - samples/sec: 4923.45 - lr: 0.012500\n",
            "2023-04-21 06:49:33,590 epoch 23 - iter 63/79 - loss 36.67781735 - time (sec): 14.80 - samples/sec: 4997.89 - lr: 0.012500\n",
            "2023-04-21 06:49:35,053 epoch 23 - iter 70/79 - loss 36.84539268 - time (sec): 16.26 - samples/sec: 5118.47 - lr: 0.012500\n",
            "2023-04-21 06:49:36,603 epoch 23 - iter 77/79 - loss 36.48827346 - time (sec): 17.81 - samples/sec: 5151.95 - lr: 0.012500\n",
            "2023-04-21 06:49:37,021 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:49:37,023 EPOCH 23 done: loss 36.9992 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:56,710 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:49:59,448 TRAIN : loss 19.106950759887695 - f1-score (weighted avg)  0.67\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:50:02,065 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:50:02,133 DEV : loss 51.364601135253906 - f1-score (weighted avg)  0.5779\n",
            "2023-04-21 06:50:02,143 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:50:02,146 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:50:03,718 epoch 24 - iter 7/79 - loss 37.72708740 - time (sec): 1.57 - samples/sec: 5544.64 - lr: 0.012500\n",
            "2023-04-21 06:50:05,321 epoch 24 - iter 14/79 - loss 35.89374893 - time (sec): 3.17 - samples/sec: 5514.89 - lr: 0.012500\n",
            "2023-04-21 06:50:06,679 epoch 24 - iter 21/79 - loss 34.58389431 - time (sec): 4.53 - samples/sec: 5671.27 - lr: 0.012500\n",
            "2023-04-21 06:50:08,673 epoch 24 - iter 28/79 - loss 35.60477199 - time (sec): 6.53 - samples/sec: 5427.18 - lr: 0.012500\n",
            "2023-04-21 06:50:10,189 epoch 24 - iter 35/79 - loss 34.10592042 - time (sec): 8.04 - samples/sec: 5457.05 - lr: 0.012500\n",
            "2023-04-21 06:50:11,909 epoch 24 - iter 42/79 - loss 34.78778175 - time (sec): 9.76 - samples/sec: 5358.33 - lr: 0.012500\n",
            "2023-04-21 06:50:13,609 epoch 24 - iter 49/79 - loss 34.30151427 - time (sec): 11.46 - samples/sec: 5271.98 - lr: 0.012500\n",
            "2023-04-21 06:50:15,234 epoch 24 - iter 56/79 - loss 34.61569007 - time (sec): 13.09 - samples/sec: 5229.61 - lr: 0.012500\n",
            "2023-04-21 06:50:16,628 epoch 24 - iter 63/79 - loss 34.98340158 - time (sec): 14.48 - samples/sec: 5329.60 - lr: 0.012500\n",
            "2023-04-21 06:50:18,041 epoch 24 - iter 70/79 - loss 35.08083569 - time (sec): 15.89 - samples/sec: 5363.55 - lr: 0.012500\n",
            "2023-04-21 06:50:19,479 epoch 24 - iter 77/79 - loss 35.59469530 - time (sec): 17.33 - samples/sec: 5369.49 - lr: 0.012500\n",
            "2023-04-21 06:50:19,832 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:50:19,834 EPOCH 24 done: loss 35.6841 - lr 0.012500\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:50:40,587 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:50:43,312 TRAIN : loss 21.047624588012695 - f1-score (weighted avg)  0.6091\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:50:45,925 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:50:45,994 DEV : loss 52.17866897583008 - f1-score (weighted avg)  0.5128\n",
            "2023-04-21 06:50:46,003 Epoch    24: reducing learning rate of group 0 to 6.2500e-03.\n",
            "2023-04-21 06:50:46,005 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:50:46,008 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:50:47,429 epoch 25 - iter 7/79 - loss 32.38326982 - time (sec): 1.42 - samples/sec: 6655.95 - lr: 0.006250\n",
            "2023-04-21 06:50:48,924 epoch 25 - iter 14/79 - loss 32.46461342 - time (sec): 2.91 - samples/sec: 6058.17 - lr: 0.006250\n",
            "2023-04-21 06:50:50,133 epoch 25 - iter 21/79 - loss 32.65960537 - time (sec): 4.12 - samples/sec: 6011.12 - lr: 0.006250\n",
            "2023-04-21 06:50:51,976 epoch 25 - iter 28/79 - loss 31.58815746 - time (sec): 5.97 - samples/sec: 5605.02 - lr: 0.006250\n",
            "2023-04-21 06:50:53,899 epoch 25 - iter 35/79 - loss 30.99441363 - time (sec): 7.89 - samples/sec: 5462.81 - lr: 0.006250\n",
            "2023-04-21 06:50:55,418 epoch 25 - iter 42/79 - loss 30.65838910 - time (sec): 9.41 - samples/sec: 5558.91 - lr: 0.006250\n",
            "2023-04-21 06:50:56,809 epoch 25 - iter 49/79 - loss 30.11732190 - time (sec): 10.80 - samples/sec: 5565.72 - lr: 0.006250\n",
            "2023-04-21 06:50:58,249 epoch 25 - iter 56/79 - loss 29.34685478 - time (sec): 12.24 - samples/sec: 5592.77 - lr: 0.006250\n",
            "2023-04-21 06:50:59,753 epoch 25 - iter 63/79 - loss 29.61579375 - time (sec): 13.74 - samples/sec: 5598.75 - lr: 0.006250\n",
            "2023-04-21 06:51:01,138 epoch 25 - iter 70/79 - loss 29.39860620 - time (sec): 15.13 - samples/sec: 5565.26 - lr: 0.006250\n",
            "2023-04-21 06:51:03,015 epoch 25 - iter 77/79 - loss 29.30473889 - time (sec): 17.00 - samples/sec: 5444.63 - lr: 0.006250\n",
            "2023-04-21 06:51:03,381 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:51:03,383 EPOCH 25 done: loss 29.2389 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:51:23,134 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:51:27,132 TRAIN : loss 13.492828369140625 - f1-score (weighted avg)  0.6998\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:51:29,748 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:51:29,819 DEV : loss 44.305694580078125 - f1-score (weighted avg)  0.6002\n",
            "2023-04-21 06:51:29,829 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:51:29,833 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:51:31,628 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:51:32,859 epoch 26 - iter 7/79 - loss 22.18731118 - time (sec): 1.23 - samples/sec: 5926.47 - lr: 0.006250\n",
            "2023-04-21 06:51:34,419 epoch 26 - iter 14/79 - loss 23.76165684 - time (sec): 2.79 - samples/sec: 5656.85 - lr: 0.006250\n",
            "2023-04-21 06:51:36,513 epoch 26 - iter 21/79 - loss 23.04034365 - time (sec): 4.88 - samples/sec: 5134.42 - lr: 0.006250\n",
            "2023-04-21 06:51:37,973 epoch 26 - iter 28/79 - loss 23.48529571 - time (sec): 6.34 - samples/sec: 5102.83 - lr: 0.006250\n",
            "2023-04-21 06:51:40,018 epoch 26 - iter 35/79 - loss 23.83104737 - time (sec): 8.39 - samples/sec: 5030.87 - lr: 0.006250\n",
            "2023-04-21 06:51:41,370 epoch 26 - iter 42/79 - loss 24.23557605 - time (sec): 9.74 - samples/sec: 5117.31 - lr: 0.006250\n",
            "2023-04-21 06:51:42,849 epoch 26 - iter 49/79 - loss 24.35890667 - time (sec): 11.22 - samples/sec: 5195.42 - lr: 0.006250\n",
            "2023-04-21 06:51:44,275 epoch 26 - iter 56/79 - loss 24.70612426 - time (sec): 12.65 - samples/sec: 5228.13 - lr: 0.006250\n",
            "2023-04-21 06:51:46,128 epoch 26 - iter 63/79 - loss 25.13787429 - time (sec): 14.50 - samples/sec: 5233.76 - lr: 0.006250\n",
            "2023-04-21 06:51:47,578 epoch 26 - iter 70/79 - loss 25.25523924 - time (sec): 15.95 - samples/sec: 5273.41 - lr: 0.006250\n",
            "2023-04-21 06:51:49,160 epoch 26 - iter 77/79 - loss 25.32135876 - time (sec): 17.53 - samples/sec: 5296.83 - lr: 0.006250\n",
            "2023-04-21 06:51:49,441 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:51:49,442 EPOCH 26 done: loss 25.2567 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:52:09,142 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:52:11,880 TRAIN : loss 11.34473991394043 - f1-score (weighted avg)  0.712\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:52:14,499 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:52:14,571 DEV : loss 40.059295654296875 - f1-score (weighted avg)  0.615\n",
            "2023-04-21 06:52:14,581 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:52:14,585 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:52:16,407 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:52:18,471 epoch 27 - iter 7/79 - loss 20.66338996 - time (sec): 2.06 - samples/sec: 5081.67 - lr: 0.006250\n",
            "2023-04-21 06:52:20,559 epoch 27 - iter 14/79 - loss 22.01924031 - time (sec): 4.15 - samples/sec: 4859.56 - lr: 0.006250\n",
            "2023-04-21 06:52:22,046 epoch 27 - iter 21/79 - loss 23.37034076 - time (sec): 5.64 - samples/sec: 5158.82 - lr: 0.006250\n",
            "2023-04-21 06:52:23,384 epoch 27 - iter 28/79 - loss 22.36078921 - time (sec): 6.98 - samples/sec: 5220.77 - lr: 0.006250\n",
            "2023-04-21 06:52:24,848 epoch 27 - iter 35/79 - loss 23.27155258 - time (sec): 8.44 - samples/sec: 5340.53 - lr: 0.006250\n",
            "2023-04-21 06:52:26,399 epoch 27 - iter 42/79 - loss 23.53294782 - time (sec): 9.99 - samples/sec: 5346.53 - lr: 0.006250\n",
            "2023-04-21 06:52:28,053 epoch 27 - iter 49/79 - loss 23.11400484 - time (sec): 11.64 - samples/sec: 5310.37 - lr: 0.006250\n",
            "2023-04-21 06:52:29,479 epoch 27 - iter 56/79 - loss 23.47996070 - time (sec): 13.07 - samples/sec: 5343.76 - lr: 0.006250\n",
            "2023-04-21 06:52:30,893 epoch 27 - iter 63/79 - loss 23.24971225 - time (sec): 14.48 - samples/sec: 5368.45 - lr: 0.006250\n",
            "2023-04-21 06:52:32,139 epoch 27 - iter 70/79 - loss 23.23336455 - time (sec): 15.73 - samples/sec: 5379.31 - lr: 0.006250\n",
            "2023-04-21 06:52:33,404 epoch 27 - iter 77/79 - loss 23.36051915 - time (sec): 17.00 - samples/sec: 5464.19 - lr: 0.006250\n",
            "2023-04-21 06:52:33,695 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:52:33,696 EPOCH 27 done: loss 23.5351 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:52:54,615 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:52:57,370 TRAIN : loss 10.853235244750977 - f1-score (weighted avg)  0.6944\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.91it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:00,005 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:53:00,073 DEV : loss 39.7330322265625 - f1-score (weighted avg)  0.5877\n",
            "2023-04-21 06:53:00,082 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:53:00,086 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:01,910 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:53:03,436 epoch 28 - iter 7/79 - loss 26.20876579 - time (sec): 1.51 - samples/sec: 5860.14 - lr: 0.006250\n",
            "2023-04-21 06:53:04,828 epoch 28 - iter 14/79 - loss 23.61359401 - time (sec): 2.90 - samples/sec: 6096.65 - lr: 0.006250\n",
            "2023-04-21 06:53:06,023 epoch 28 - iter 21/79 - loss 23.07208669 - time (sec): 4.10 - samples/sec: 6049.84 - lr: 0.006250\n",
            "2023-04-21 06:53:07,508 epoch 28 - iter 28/79 - loss 22.01372360 - time (sec): 5.58 - samples/sec: 5883.06 - lr: 0.006250\n",
            "2023-04-21 06:53:08,763 epoch 28 - iter 35/79 - loss 21.84530099 - time (sec): 6.84 - samples/sec: 5894.49 - lr: 0.006250\n",
            "2023-04-21 06:53:10,198 epoch 28 - iter 42/79 - loss 21.66470380 - time (sec): 8.27 - samples/sec: 5808.15 - lr: 0.006250\n",
            "2023-04-21 06:53:11,702 epoch 28 - iter 49/79 - loss 22.36087870 - time (sec): 9.78 - samples/sec: 5842.62 - lr: 0.006250\n",
            "2023-04-21 06:53:13,936 epoch 28 - iter 56/79 - loss 21.91237815 - time (sec): 12.01 - samples/sec: 5507.82 - lr: 0.006250\n",
            "2023-04-21 06:53:15,531 epoch 28 - iter 63/79 - loss 21.59081996 - time (sec): 13.61 - samples/sec: 5434.10 - lr: 0.006250\n",
            "2023-04-21 06:53:17,427 epoch 28 - iter 70/79 - loss 22.02240384 - time (sec): 15.50 - samples/sec: 5339.09 - lr: 0.006250\n",
            "2023-04-21 06:53:19,015 epoch 28 - iter 77/79 - loss 22.05882979 - time (sec): 17.09 - samples/sec: 5362.20 - lr: 0.006250\n",
            "2023-04-21 06:53:19,662 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:53:19,664 EPOCH 28 done: loss 22.3753 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:39,401 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:43,407 TRAIN : loss 9.414450645446777 - f1-score (weighted avg)  0.6936\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:46,005 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:53:46,074 DEV : loss 37.58403015136719 - f1-score (weighted avg)  0.5795\n",
            "2023-04-21 06:53:46,083 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:53:46,087 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:53:47,880 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:53:49,986 epoch 29 - iter 7/79 - loss 22.74633925 - time (sec): 2.10 - samples/sec: 4446.97 - lr: 0.006250\n",
            "2023-04-21 06:53:51,596 epoch 29 - iter 14/79 - loss 21.33618768 - time (sec): 3.71 - samples/sec: 4952.45 - lr: 0.006250\n",
            "2023-04-21 06:53:53,317 epoch 29 - iter 21/79 - loss 20.56423111 - time (sec): 5.43 - samples/sec: 4982.59 - lr: 0.006250\n",
            "2023-04-21 06:53:54,598 epoch 29 - iter 28/79 - loss 21.48456015 - time (sec): 6.72 - samples/sec: 5278.06 - lr: 0.006250\n",
            "2023-04-21 06:53:56,294 epoch 29 - iter 35/79 - loss 22.28173209 - time (sec): 8.41 - samples/sec: 5278.68 - lr: 0.006250\n",
            "2023-04-21 06:53:57,790 epoch 29 - iter 42/79 - loss 22.68553773 - time (sec): 9.91 - samples/sec: 5304.77 - lr: 0.006250\n",
            "2023-04-21 06:53:58,963 epoch 29 - iter 49/79 - loss 22.71053114 - time (sec): 11.08 - samples/sec: 5416.84 - lr: 0.006250\n",
            "2023-04-21 06:54:00,703 epoch 29 - iter 56/79 - loss 23.03394695 - time (sec): 12.82 - samples/sec: 5387.59 - lr: 0.006250\n",
            "2023-04-21 06:54:02,078 epoch 29 - iter 63/79 - loss 23.03016435 - time (sec): 14.20 - samples/sec: 5367.26 - lr: 0.006250\n",
            "2023-04-21 06:54:03,447 epoch 29 - iter 70/79 - loss 23.25847229 - time (sec): 15.56 - samples/sec: 5396.12 - lr: 0.006250\n",
            "2023-04-21 06:54:05,053 epoch 29 - iter 77/79 - loss 23.07680425 - time (sec): 17.17 - samples/sec: 5374.89 - lr: 0.006250\n",
            "2023-04-21 06:54:05,482 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:54:05,483 EPOCH 29 done: loss 23.2402 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:54:25,451 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:54:28,176 TRAIN : loss 9.3574800491333 - f1-score (weighted avg)  0.6968\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:54:30,753 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:54:30,823 DEV : loss 36.76661682128906 - f1-score (weighted avg)  0.5879\n",
            "2023-04-21 06:54:30,832 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:54:30,845 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:54:32,629 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:54:34,274 epoch 30 - iter 7/79 - loss 24.73620668 - time (sec): 1.63 - samples/sec: 4987.88 - lr: 0.006250\n",
            "2023-04-21 06:54:35,691 epoch 30 - iter 14/79 - loss 22.73372734 - time (sec): 3.05 - samples/sec: 5176.28 - lr: 0.006250\n",
            "2023-04-21 06:54:37,190 epoch 30 - iter 21/79 - loss 22.29884335 - time (sec): 4.55 - samples/sec: 5332.41 - lr: 0.006250\n",
            "2023-04-21 06:54:38,582 epoch 30 - iter 28/79 - loss 21.96258896 - time (sec): 5.94 - samples/sec: 5464.68 - lr: 0.006250\n",
            "2023-04-21 06:54:40,313 epoch 30 - iter 35/79 - loss 21.52916184 - time (sec): 7.67 - samples/sec: 5400.42 - lr: 0.006250\n",
            "2023-04-21 06:54:41,846 epoch 30 - iter 42/79 - loss 21.75332346 - time (sec): 9.20 - samples/sec: 5500.84 - lr: 0.006250\n",
            "2023-04-21 06:54:43,313 epoch 30 - iter 49/79 - loss 21.72445299 - time (sec): 10.67 - samples/sec: 5482.16 - lr: 0.006250\n",
            "2023-04-21 06:54:45,010 epoch 30 - iter 56/79 - loss 21.51111582 - time (sec): 12.37 - samples/sec: 5483.71 - lr: 0.006250\n",
            "2023-04-21 06:54:46,693 epoch 30 - iter 63/79 - loss 21.25176791 - time (sec): 14.05 - samples/sec: 5490.07 - lr: 0.006250\n",
            "2023-04-21 06:54:48,361 epoch 30 - iter 70/79 - loss 21.12064279 - time (sec): 15.72 - samples/sec: 5429.74 - lr: 0.006250\n",
            "2023-04-21 06:54:49,865 epoch 30 - iter 77/79 - loss 21.05167341 - time (sec): 17.22 - samples/sec: 5378.10 - lr: 0.006250\n",
            "2023-04-21 06:54:50,280 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:54:50,282 EPOCH 30 done: loss 21.0190 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:11,072 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:55:13,811 TRAIN : loss 10.716641426086426 - f1-score (weighted avg)  0.6383\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:16,409 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:55:16,477 DEV : loss 37.694305419921875 - f1-score (weighted avg)  0.529\n",
            "2023-04-21 06:55:16,488 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:55:16,492 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:17,751 epoch 31 - iter 7/79 - loss 19.53654683 - time (sec): 1.26 - samples/sec: 5662.15 - lr: 0.006250\n",
            "2023-04-21 06:55:19,267 epoch 31 - iter 14/79 - loss 18.74983963 - time (sec): 2.77 - samples/sec: 5621.98 - lr: 0.006250\n",
            "2023-04-21 06:55:20,613 epoch 31 - iter 21/79 - loss 17.62377961 - time (sec): 4.12 - samples/sec: 5892.95 - lr: 0.006250\n",
            "2023-04-21 06:55:22,224 epoch 31 - iter 28/79 - loss 18.42963392 - time (sec): 5.73 - samples/sec: 5725.81 - lr: 0.006250\n",
            "2023-04-21 06:55:23,879 epoch 31 - iter 35/79 - loss 18.50481778 - time (sec): 7.38 - samples/sec: 5677.65 - lr: 0.006250\n",
            "2023-04-21 06:55:25,206 epoch 31 - iter 42/79 - loss 19.45147272 - time (sec): 8.71 - samples/sec: 5790.75 - lr: 0.006250\n",
            "2023-04-21 06:55:26,755 epoch 31 - iter 49/79 - loss 19.49599189 - time (sec): 10.26 - samples/sec: 5744.52 - lr: 0.006250\n",
            "2023-04-21 06:55:28,545 epoch 31 - iter 56/79 - loss 19.39449083 - time (sec): 12.05 - samples/sec: 5525.59 - lr: 0.006250\n",
            "2023-04-21 06:55:30,157 epoch 31 - iter 63/79 - loss 19.82327974 - time (sec): 13.66 - samples/sec: 5526.56 - lr: 0.006250\n",
            "2023-04-21 06:55:31,959 epoch 31 - iter 70/79 - loss 19.60422060 - time (sec): 15.47 - samples/sec: 5419.61 - lr: 0.006250\n",
            "2023-04-21 06:55:33,639 epoch 31 - iter 77/79 - loss 19.81410845 - time (sec): 17.14 - samples/sec: 5400.82 - lr: 0.006250\n",
            "2023-04-21 06:55:33,975 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:55:33,976 EPOCH 31 done: loss 19.9303 - lr 0.006250\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:53,601 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:56,316 TRAIN : loss 10.767945289611816 - f1-score (weighted avg)  0.6092\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.88it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:55:58,973 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:55:59,042 DEV : loss 37.37782669067383 - f1-score (weighted avg)  0.5305\n",
            "2023-04-21 06:55:59,051 Epoch    31: reducing learning rate of group 0 to 3.1250e-03.\n",
            "2023-04-21 06:55:59,053 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:55:59,057 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:56:00,433 epoch 32 - iter 7/79 - loss 19.17622627 - time (sec): 1.37 - samples/sec: 5982.23 - lr: 0.003125\n",
            "2023-04-21 06:56:02,053 epoch 32 - iter 14/79 - loss 18.19795798 - time (sec): 2.99 - samples/sec: 5644.00 - lr: 0.003125\n",
            "2023-04-21 06:56:03,401 epoch 32 - iter 21/79 - loss 16.96024703 - time (sec): 4.34 - samples/sec: 5761.75 - lr: 0.003125\n",
            "2023-04-21 06:56:04,805 epoch 32 - iter 28/79 - loss 16.62626396 - time (sec): 5.75 - samples/sec: 5766.23 - lr: 0.003125\n",
            "2023-04-21 06:56:06,550 epoch 32 - iter 35/79 - loss 17.14271583 - time (sec): 7.49 - samples/sec: 5549.99 - lr: 0.003125\n",
            "2023-04-21 06:56:08,077 epoch 32 - iter 42/79 - loss 17.07032573 - time (sec): 9.02 - samples/sec: 5564.24 - lr: 0.003125\n",
            "2023-04-21 06:56:09,349 epoch 32 - iter 49/79 - loss 17.04313511 - time (sec): 10.29 - samples/sec: 5584.08 - lr: 0.003125\n",
            "2023-04-21 06:56:10,548 epoch 32 - iter 56/79 - loss 16.92036536 - time (sec): 11.49 - samples/sec: 5634.06 - lr: 0.003125\n",
            "2023-04-21 06:56:12,309 epoch 32 - iter 63/79 - loss 17.10953492 - time (sec): 13.25 - samples/sec: 5545.05 - lr: 0.003125\n",
            "2023-04-21 06:56:13,898 epoch 32 - iter 70/79 - loss 16.93438566 - time (sec): 14.84 - samples/sec: 5569.33 - lr: 0.003125\n",
            "2023-04-21 06:56:15,999 epoch 32 - iter 77/79 - loss 17.15929198 - time (sec): 16.94 - samples/sec: 5455.03 - lr: 0.003125\n",
            "2023-04-21 06:56:16,482 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:56:16,484 EPOCH 32 done: loss 17.1413 - lr 0.003125\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.85it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:56:37,155 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:56:39,887 TRAIN : loss 6.99954080581665 - f1-score (weighted avg)  0.7368\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:56:42,499 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:56:42,570 DEV : loss 33.24990463256836 - f1-score (weighted avg)  0.6159\n",
            "2023-04-21 06:56:42,579 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:56:42,583 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:56:44,364 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:56:46,004 epoch 33 - iter 7/79 - loss 16.18660025 - time (sec): 1.64 - samples/sec: 5302.55 - lr: 0.003125\n",
            "2023-04-21 06:56:47,914 epoch 33 - iter 14/79 - loss 17.25576741 - time (sec): 3.55 - samples/sec: 4752.30 - lr: 0.003125\n",
            "2023-04-21 06:56:49,361 epoch 33 - iter 21/79 - loss 18.30581283 - time (sec): 4.99 - samples/sec: 4777.22 - lr: 0.003125\n",
            "2023-04-21 06:56:51,215 epoch 33 - iter 28/79 - loss 17.35929705 - time (sec): 6.85 - samples/sec: 4887.65 - lr: 0.003125\n",
            "2023-04-21 06:56:52,745 epoch 33 - iter 35/79 - loss 16.61808354 - time (sec): 8.38 - samples/sec: 4915.47 - lr: 0.003125\n",
            "2023-04-21 06:56:54,159 epoch 33 - iter 42/79 - loss 16.53294838 - time (sec): 9.79 - samples/sec: 5022.04 - lr: 0.003125\n",
            "2023-04-21 06:56:55,986 epoch 33 - iter 49/79 - loss 16.73237973 - time (sec): 11.62 - samples/sec: 4989.02 - lr: 0.003125\n",
            "2023-04-21 06:56:57,509 epoch 33 - iter 56/79 - loss 16.50320479 - time (sec): 13.14 - samples/sec: 5006.57 - lr: 0.003125\n",
            "2023-04-21 06:56:59,156 epoch 33 - iter 63/79 - loss 16.78599930 - time (sec): 14.79 - samples/sec: 5087.86 - lr: 0.003125\n",
            "2023-04-21 06:57:00,712 epoch 33 - iter 70/79 - loss 17.01984271 - time (sec): 16.35 - samples/sec: 5139.92 - lr: 0.003125\n",
            "2023-04-21 06:57:02,107 epoch 33 - iter 77/79 - loss 16.85634598 - time (sec): 17.74 - samples/sec: 5222.95 - lr: 0.003125\n",
            "2023-04-21 06:57:02,445 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:57:02,447 EPOCH 33 done: loss 16.8070 - lr 0.003125\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  3.97it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:57:23,635 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:57:26,370 TRAIN : loss 6.522021293640137 - f1-score (weighted avg)  0.7179\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:57:28,956 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:57:29,026 DEV : loss 32.86244583129883 - f1-score (weighted avg)  0.5809\n",
            "2023-04-21 06:57:29,037 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:57:29,040 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:57:30,848 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:57:32,230 epoch 34 - iter 7/79 - loss 13.77644565 - time (sec): 1.38 - samples/sec: 5728.29 - lr: 0.003125\n",
            "2023-04-21 06:57:34,089 epoch 34 - iter 14/79 - loss 13.05748939 - time (sec): 3.24 - samples/sec: 5238.60 - lr: 0.003125\n",
            "2023-04-21 06:57:35,576 epoch 34 - iter 21/79 - loss 13.60067034 - time (sec): 4.73 - samples/sec: 5303.51 - lr: 0.003125\n",
            "2023-04-21 06:57:36,837 epoch 34 - iter 28/79 - loss 13.71712258 - time (sec): 5.99 - samples/sec: 5520.40 - lr: 0.003125\n",
            "2023-04-21 06:57:38,373 epoch 34 - iter 35/79 - loss 14.48531762 - time (sec): 7.52 - samples/sec: 5573.07 - lr: 0.003125\n",
            "2023-04-21 06:57:39,928 epoch 34 - iter 42/79 - loss 14.67833284 - time (sec): 9.08 - samples/sec: 5565.67 - lr: 0.003125\n",
            "2023-04-21 06:57:41,622 epoch 34 - iter 49/79 - loss 15.01412894 - time (sec): 10.77 - samples/sec: 5484.24 - lr: 0.003125\n",
            "2023-04-21 06:57:42,815 epoch 34 - iter 56/79 - loss 15.11209541 - time (sec): 11.96 - samples/sec: 5544.10 - lr: 0.003125\n",
            "2023-04-21 06:57:44,273 epoch 34 - iter 63/79 - loss 15.60375637 - time (sec): 13.42 - samples/sec: 5511.74 - lr: 0.003125\n",
            "2023-04-21 06:57:46,269 epoch 34 - iter 70/79 - loss 15.57021058 - time (sec): 15.42 - samples/sec: 5374.32 - lr: 0.003125\n",
            "2023-04-21 06:57:48,221 epoch 34 - iter 77/79 - loss 15.51505600 - time (sec): 17.37 - samples/sec: 5319.57 - lr: 0.003125\n",
            "2023-04-21 06:57:48,586 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:57:48,588 EPOCH 34 done: loss 15.6678 - lr 0.003125\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:08,302 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:12,300 TRAIN : loss 6.121437072753906 - f1-score (weighted avg)  0.7292\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:14,899 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:58:14,967 DEV : loss 31.64256477355957 - f1-score (weighted avg)  0.6162\n",
            "2023-04-21 06:58:14,976 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 06:58:14,980 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:16,793 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:58:18,236 epoch 35 - iter 7/79 - loss 14.78990238 - time (sec): 1.44 - samples/sec: 5470.81 - lr: 0.003125\n",
            "2023-04-21 06:58:19,698 epoch 35 - iter 14/79 - loss 15.10347006 - time (sec): 2.90 - samples/sec: 5862.05 - lr: 0.003125\n",
            "2023-04-21 06:58:21,973 epoch 35 - iter 21/79 - loss 14.50470440 - time (sec): 5.18 - samples/sec: 4997.79 - lr: 0.003125\n",
            "2023-04-21 06:58:23,596 epoch 35 - iter 28/79 - loss 14.68638225 - time (sec): 6.80 - samples/sec: 4974.07 - lr: 0.003125\n",
            "2023-04-21 06:58:25,030 epoch 35 - iter 35/79 - loss 14.51676311 - time (sec): 8.24 - samples/sec: 5068.01 - lr: 0.003125\n",
            "2023-04-21 06:58:26,571 epoch 35 - iter 42/79 - loss 14.30281307 - time (sec): 9.78 - samples/sec: 5127.79 - lr: 0.003125\n",
            "2023-04-21 06:58:27,942 epoch 35 - iter 49/79 - loss 14.08796376 - time (sec): 11.15 - samples/sec: 5200.76 - lr: 0.003125\n",
            "2023-04-21 06:58:29,483 epoch 35 - iter 56/79 - loss 14.03435423 - time (sec): 12.69 - samples/sec: 5193.69 - lr: 0.003125\n",
            "2023-04-21 06:58:31,271 epoch 35 - iter 63/79 - loss 14.23323668 - time (sec): 14.48 - samples/sec: 5211.31 - lr: 0.003125\n",
            "2023-04-21 06:58:32,844 epoch 35 - iter 70/79 - loss 14.56231973 - time (sec): 16.05 - samples/sec: 5227.78 - lr: 0.003125\n",
            "2023-04-21 06:58:34,065 epoch 35 - iter 77/79 - loss 14.82609443 - time (sec): 17.27 - samples/sec: 5303.86 - lr: 0.003125\n",
            "2023-04-21 06:58:34,502 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:58:34,504 EPOCH 35 done: loss 15.1141 - lr 0.003125\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:54,248 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:56,996 TRAIN : loss 6.181039333343506 - f1-score (weighted avg)  0.7435\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:58:59,579 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:58:59,650 DEV : loss 31.694461822509766 - f1-score (weighted avg)  0.6369\n",
            "2023-04-21 06:58:59,660 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 06:58:59,664 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:59:01,041 epoch 36 - iter 7/79 - loss 13.92884691 - time (sec): 1.38 - samples/sec: 5562.29 - lr: 0.003125\n",
            "2023-04-21 06:59:02,391 epoch 36 - iter 14/79 - loss 13.78796280 - time (sec): 2.72 - samples/sec: 6155.23 - lr: 0.003125\n",
            "2023-04-21 06:59:03,949 epoch 36 - iter 21/79 - loss 14.24425032 - time (sec): 4.28 - samples/sec: 5903.91 - lr: 0.003125\n",
            "2023-04-21 06:59:05,359 epoch 36 - iter 28/79 - loss 14.79526264 - time (sec): 5.69 - samples/sec: 5939.08 - lr: 0.003125\n",
            "2023-04-21 06:59:06,834 epoch 36 - iter 35/79 - loss 15.14999941 - time (sec): 7.17 - samples/sec: 5896.14 - lr: 0.003125\n",
            "2023-04-21 06:59:08,997 epoch 36 - iter 42/79 - loss 14.83021773 - time (sec): 9.33 - samples/sec: 5463.50 - lr: 0.003125\n",
            "2023-04-21 06:59:10,276 epoch 36 - iter 49/79 - loss 15.07805743 - time (sec): 10.61 - samples/sec: 5492.58 - lr: 0.003125\n",
            "2023-04-21 06:59:11,661 epoch 36 - iter 56/79 - loss 14.90629137 - time (sec): 12.00 - samples/sec: 5541.25 - lr: 0.003125\n",
            "2023-04-21 06:59:13,238 epoch 36 - iter 63/79 - loss 14.72569256 - time (sec): 13.57 - samples/sec: 5580.05 - lr: 0.003125\n",
            "2023-04-21 06:59:14,670 epoch 36 - iter 70/79 - loss 14.41303165 - time (sec): 15.00 - samples/sec: 5572.51 - lr: 0.003125\n",
            "2023-04-21 06:59:16,072 epoch 36 - iter 77/79 - loss 14.49143644 - time (sec): 16.41 - samples/sec: 5568.76 - lr: 0.003125\n",
            "2023-04-21 06:59:16,807 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 06:59:16,809 EPOCH 36 done: loss 14.4008 - lr 0.003125\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:59:37,557 Evaluating as a multi-label problem: True\n",
            "2023-04-21 06:59:40,307 TRAIN : loss 6.831955909729004 - f1-score (weighted avg)  0.6862\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:59:42,924 Evaluating as a multi-label problem: False\n",
            "2023-04-21 06:59:42,992 DEV : loss 32.12389373779297 - f1-score (weighted avg)  0.586\n",
            "2023-04-21 06:59:43,001 Epoch    36: reducing learning rate of group 0 to 1.5625e-03.\n",
            "2023-04-21 06:59:43,003 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 06:59:43,006 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 06:59:44,470 epoch 37 - iter 7/79 - loss 13.23493697 - time (sec): 1.46 - samples/sec: 5532.76 - lr: 0.001563\n",
            "2023-04-21 06:59:45,795 epoch 37 - iter 14/79 - loss 11.89355531 - time (sec): 2.79 - samples/sec: 5598.31 - lr: 0.001563\n",
            "2023-04-21 06:59:47,249 epoch 37 - iter 21/79 - loss 11.82298983 - time (sec): 4.24 - samples/sec: 5539.15 - lr: 0.001563\n",
            "2023-04-21 06:59:48,867 epoch 37 - iter 28/79 - loss 12.56203958 - time (sec): 5.86 - samples/sec: 5467.19 - lr: 0.001563\n",
            "2023-04-21 06:59:50,569 epoch 37 - iter 35/79 - loss 12.68561456 - time (sec): 7.56 - samples/sec: 5361.54 - lr: 0.001563\n",
            "2023-04-21 06:59:52,104 epoch 37 - iter 42/79 - loss 12.84572421 - time (sec): 9.10 - samples/sec: 5427.48 - lr: 0.001563\n",
            "2023-04-21 06:59:53,466 epoch 37 - iter 49/79 - loss 13.28359726 - time (sec): 10.46 - samples/sec: 5443.25 - lr: 0.001563\n",
            "2023-04-21 06:59:54,775 epoch 37 - iter 56/79 - loss 13.58898866 - time (sec): 11.77 - samples/sec: 5460.94 - lr: 0.001563\n",
            "2023-04-21 06:59:56,185 epoch 37 - iter 63/79 - loss 13.59565486 - time (sec): 13.18 - samples/sec: 5536.58 - lr: 0.001563\n",
            "2023-04-21 06:59:58,347 epoch 37 - iter 70/79 - loss 13.56226795 - time (sec): 15.34 - samples/sec: 5407.91 - lr: 0.001563\n",
            "2023-04-21 07:00:00,131 epoch 37 - iter 77/79 - loss 13.37905049 - time (sec): 17.12 - samples/sec: 5372.47 - lr: 0.001563\n",
            "2023-04-21 07:00:00,533 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:00:00,535 EPOCH 37 done: loss 13.3709 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:00:20,178 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:00:22,960 TRAIN : loss 5.545739650726318 - f1-score (weighted avg)  0.7324\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.91it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:00:26,690 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:00:26,759 DEV : loss 30.76970672607422 - f1-score (weighted avg)  0.6097\n",
            "2023-04-21 07:00:26,768 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:00:26,772 saving best model\n",
            "2023-04-21 07:00:28,573 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:00:30,479 epoch 38 - iter 7/79 - loss 10.76299293 - time (sec): 1.89 - samples/sec: 4638.04 - lr: 0.001563\n",
            "2023-04-21 07:00:31,884 epoch 38 - iter 14/79 - loss 11.39663862 - time (sec): 3.30 - samples/sec: 5280.11 - lr: 0.001563\n",
            "2023-04-21 07:00:33,218 epoch 38 - iter 21/79 - loss 12.46366782 - time (sec): 4.63 - samples/sec: 5366.57 - lr: 0.001563\n",
            "2023-04-21 07:00:34,794 epoch 38 - iter 28/79 - loss 12.35318044 - time (sec): 6.21 - samples/sec: 5312.12 - lr: 0.001563\n",
            "2023-04-21 07:00:36,442 epoch 38 - iter 35/79 - loss 12.81437782 - time (sec): 7.86 - samples/sec: 5414.91 - lr: 0.001563\n",
            "2023-04-21 07:00:38,077 epoch 38 - iter 42/79 - loss 12.86576319 - time (sec): 9.49 - samples/sec: 5323.24 - lr: 0.001563\n",
            "2023-04-21 07:00:39,654 epoch 38 - iter 49/79 - loss 12.79309577 - time (sec): 11.07 - samples/sec: 5274.15 - lr: 0.001563\n",
            "2023-04-21 07:00:41,181 epoch 38 - iter 56/79 - loss 12.93551599 - time (sec): 12.59 - samples/sec: 5362.66 - lr: 0.001563\n",
            "2023-04-21 07:00:42,624 epoch 38 - iter 63/79 - loss 12.92069965 - time (sec): 14.04 - samples/sec: 5393.42 - lr: 0.001563\n",
            "2023-04-21 07:00:44,161 epoch 38 - iter 70/79 - loss 12.64290474 - time (sec): 15.57 - samples/sec: 5395.87 - lr: 0.001563\n",
            "2023-04-21 07:00:45,872 epoch 38 - iter 77/79 - loss 12.96322958 - time (sec): 17.28 - samples/sec: 5355.68 - lr: 0.001563\n",
            "2023-04-21 07:00:46,249 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:00:46,251 EPOCH 38 done: loss 12.9956 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:05,999 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:08,736 TRAIN : loss 5.213384628295898 - f1-score (weighted avg)  0.7502\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:11,326 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:01:11,397 DEV : loss 30.146928787231445 - f1-score (weighted avg)  0.6266\n",
            "2023-04-21 07:01:11,407 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:01:11,411 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:13,211 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:01:14,792 epoch 39 - iter 7/79 - loss 11.63111972 - time (sec): 1.58 - samples/sec: 4619.18 - lr: 0.001563\n",
            "2023-04-21 07:01:16,348 epoch 39 - iter 14/79 - loss 11.32594502 - time (sec): 3.13 - samples/sec: 5106.14 - lr: 0.001563\n",
            "2023-04-21 07:01:17,879 epoch 39 - iter 21/79 - loss 11.39921801 - time (sec): 4.67 - samples/sec: 5193.59 - lr: 0.001563\n",
            "2023-04-21 07:01:19,491 epoch 39 - iter 28/79 - loss 11.35944903 - time (sec): 6.28 - samples/sec: 5244.65 - lr: 0.001563\n",
            "2023-04-21 07:01:20,849 epoch 39 - iter 35/79 - loss 11.72959941 - time (sec): 7.64 - samples/sec: 5296.43 - lr: 0.001563\n",
            "2023-04-21 07:01:22,207 epoch 39 - iter 42/79 - loss 11.75136267 - time (sec): 8.99 - samples/sec: 5507.65 - lr: 0.001563\n",
            "2023-04-21 07:01:23,553 epoch 39 - iter 49/79 - loss 11.85045351 - time (sec): 10.34 - samples/sec: 5550.27 - lr: 0.001563\n",
            "2023-04-21 07:01:24,801 epoch 39 - iter 56/79 - loss 12.15017675 - time (sec): 11.59 - samples/sec: 5541.98 - lr: 0.001563\n",
            "2023-04-21 07:01:26,385 epoch 39 - iter 63/79 - loss 12.12470674 - time (sec): 13.17 - samples/sec: 5533.53 - lr: 0.001563\n",
            "2023-04-21 07:01:28,196 epoch 39 - iter 70/79 - loss 12.51138539 - time (sec): 14.98 - samples/sec: 5405.07 - lr: 0.001563\n",
            "2023-04-21 07:01:30,360 epoch 39 - iter 77/79 - loss 12.03359289 - time (sec): 17.15 - samples/sec: 5299.15 - lr: 0.001563\n",
            "2023-04-21 07:01:30,926 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:01:30,928 EPOCH 39 done: loss 12.1331 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:51,694 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:01:54,430 TRAIN : loss 5.184724807739258 - f1-score (weighted avg)  0.7188\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:57,016 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:01:57,085 DEV : loss 29.77825927734375 - f1-score (weighted avg)  0.6009\n",
            "2023-04-21 07:01:57,094 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:01:57,098 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:01:58,893 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:02:00,337 epoch 40 - iter 7/79 - loss 9.61143487 - time (sec): 1.43 - samples/sec: 5311.59 - lr: 0.001563\n",
            "2023-04-21 07:02:01,737 epoch 40 - iter 14/79 - loss 9.40430830 - time (sec): 2.83 - samples/sec: 5073.88 - lr: 0.001563\n",
            "2023-04-21 07:02:03,532 epoch 40 - iter 21/79 - loss 10.62074404 - time (sec): 4.63 - samples/sec: 5003.92 - lr: 0.001563\n",
            "2023-04-21 07:02:05,592 epoch 40 - iter 28/79 - loss 10.55999092 - time (sec): 6.68 - samples/sec: 4944.78 - lr: 0.001563\n",
            "2023-04-21 07:02:07,180 epoch 40 - iter 35/79 - loss 10.21826664 - time (sec): 8.27 - samples/sec: 5072.01 - lr: 0.001563\n",
            "2023-04-21 07:02:08,625 epoch 40 - iter 42/79 - loss 10.58861707 - time (sec): 9.72 - samples/sec: 5139.09 - lr: 0.001563\n",
            "2023-04-21 07:02:10,339 epoch 40 - iter 49/79 - loss 10.98116433 - time (sec): 11.43 - samples/sec: 5188.02 - lr: 0.001563\n",
            "2023-04-21 07:02:11,754 epoch 40 - iter 56/79 - loss 11.25031775 - time (sec): 12.85 - samples/sec: 5266.92 - lr: 0.001563\n",
            "2023-04-21 07:02:13,340 epoch 40 - iter 63/79 - loss 11.33959718 - time (sec): 14.43 - samples/sec: 5307.12 - lr: 0.001563\n",
            "2023-04-21 07:02:14,749 epoch 40 - iter 70/79 - loss 11.41785359 - time (sec): 15.84 - samples/sec: 5327.32 - lr: 0.001563\n",
            "2023-04-21 07:02:16,164 epoch 40 - iter 77/79 - loss 11.49093960 - time (sec): 17.26 - samples/sec: 5353.25 - lr: 0.001563\n",
            "2023-04-21 07:02:16,570 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:02:16,571 EPOCH 40 done: loss 11.6062 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:02:36,249 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:02:40,255 TRAIN : loss 4.83873176574707 - f1-score (weighted avg)  0.7485\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.84it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:02:42,948 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:02:43,018 DEV : loss 29.18534278869629 - f1-score (weighted avg)  0.6332\n",
            "2023-04-21 07:02:43,028 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:02:43,032 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:02:44,821 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:02:46,374 epoch 41 - iter 7/79 - loss 9.53879163 - time (sec): 1.55 - samples/sec: 5676.09 - lr: 0.001563\n",
            "2023-04-21 07:02:47,838 epoch 41 - iter 14/79 - loss 10.10620255 - time (sec): 3.01 - samples/sec: 5463.29 - lr: 0.001563\n",
            "2023-04-21 07:02:49,397 epoch 41 - iter 21/79 - loss 9.89929444 - time (sec): 4.57 - samples/sec: 5601.80 - lr: 0.001563\n",
            "2023-04-21 07:02:51,114 epoch 41 - iter 28/79 - loss 9.92529879 - time (sec): 6.29 - samples/sec: 5454.21 - lr: 0.001563\n",
            "2023-04-21 07:02:52,766 epoch 41 - iter 35/79 - loss 10.31774796 - time (sec): 7.94 - samples/sec: 5445.20 - lr: 0.001563\n",
            "2023-04-21 07:02:54,164 epoch 41 - iter 42/79 - loss 10.74669380 - time (sec): 9.34 - samples/sec: 5452.66 - lr: 0.001563\n",
            "2023-04-21 07:02:55,400 epoch 41 - iter 49/79 - loss 10.95644467 - time (sec): 10.58 - samples/sec: 5500.64 - lr: 0.001563\n",
            "2023-04-21 07:02:56,903 epoch 41 - iter 56/79 - loss 11.20195346 - time (sec): 12.08 - samples/sec: 5528.47 - lr: 0.001563\n",
            "2023-04-21 07:02:58,990 epoch 41 - iter 63/79 - loss 11.25309029 - time (sec): 14.17 - samples/sec: 5382.50 - lr: 0.001563\n",
            "2023-04-21 07:03:00,777 epoch 41 - iter 70/79 - loss 11.31868619 - time (sec): 15.95 - samples/sec: 5286.20 - lr: 0.001563\n",
            "2023-04-21 07:03:02,205 epoch 41 - iter 77/79 - loss 11.60072695 - time (sec): 17.38 - samples/sec: 5316.47 - lr: 0.001563\n",
            "2023-04-21 07:03:02,585 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:03:02,587 EPOCH 41 done: loss 11.6548 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.02it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:03:22,399 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:03:25,165 TRAIN : loss 5.359823703765869 - f1-score (weighted avg)  0.7033\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:03:27,747 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:03:27,817 DEV : loss 29.90131187438965 - f1-score (weighted avg)  0.586\n",
            "2023-04-21 07:03:27,827 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:03:27,831 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:03:29,310 epoch 42 - iter 7/79 - loss 12.61840138 - time (sec): 1.48 - samples/sec: 5206.16 - lr: 0.001563\n",
            "2023-04-21 07:03:30,949 epoch 42 - iter 14/79 - loss 10.72858199 - time (sec): 3.12 - samples/sec: 5315.68 - lr: 0.001563\n",
            "2023-04-21 07:03:32,214 epoch 42 - iter 21/79 - loss 10.73944865 - time (sec): 4.38 - samples/sec: 5613.29 - lr: 0.001563\n",
            "2023-04-21 07:03:33,711 epoch 42 - iter 28/79 - loss 11.84216687 - time (sec): 5.88 - samples/sec: 5630.85 - lr: 0.001563\n",
            "2023-04-21 07:03:35,074 epoch 42 - iter 35/79 - loss 11.72701451 - time (sec): 7.24 - samples/sec: 5718.84 - lr: 0.001563\n",
            "2023-04-21 07:03:36,902 epoch 42 - iter 42/79 - loss 11.78721033 - time (sec): 9.07 - samples/sec: 5552.47 - lr: 0.001563\n",
            "2023-04-21 07:03:38,838 epoch 42 - iter 49/79 - loss 11.56189883 - time (sec): 11.01 - samples/sec: 5450.38 - lr: 0.001563\n",
            "2023-04-21 07:03:40,226 epoch 42 - iter 56/79 - loss 11.52234647 - time (sec): 12.39 - samples/sec: 5494.81 - lr: 0.001563\n",
            "2023-04-21 07:03:41,639 epoch 42 - iter 63/79 - loss 11.46060586 - time (sec): 13.81 - samples/sec: 5499.04 - lr: 0.001563\n",
            "2023-04-21 07:03:43,372 epoch 42 - iter 70/79 - loss 11.50198903 - time (sec): 15.54 - samples/sec: 5439.47 - lr: 0.001563\n",
            "2023-04-21 07:03:44,856 epoch 42 - iter 77/79 - loss 11.86270432 - time (sec): 17.02 - samples/sec: 5426.70 - lr: 0.001563\n",
            "2023-04-21 07:03:45,207 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:03:45,209 EPOCH 42 done: loss 11.9315 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:06,015 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:04:08,748 TRAIN : loss 4.422117710113525 - f1-score (weighted avg)  0.7552\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:11,345 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:04:11,414 DEV : loss 29.04127311706543 - f1-score (weighted avg)  0.6315\n",
            "2023-04-21 07:04:11,423 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:04:11,427 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:13,211 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:04:14,596 epoch 43 - iter 7/79 - loss 11.49475081 - time (sec): 1.38 - samples/sec: 5996.18 - lr: 0.001563\n",
            "2023-04-21 07:04:16,118 epoch 43 - iter 14/79 - loss 13.31605646 - time (sec): 2.90 - samples/sec: 6160.59 - lr: 0.001563\n",
            "2023-04-21 07:04:17,566 epoch 43 - iter 21/79 - loss 13.48123801 - time (sec): 4.35 - samples/sec: 6107.63 - lr: 0.001563\n",
            "2023-04-21 07:04:19,642 epoch 43 - iter 28/79 - loss 12.65823236 - time (sec): 6.43 - samples/sec: 5558.92 - lr: 0.001563\n",
            "2023-04-21 07:04:21,140 epoch 43 - iter 35/79 - loss 11.75265225 - time (sec): 7.93 - samples/sec: 5541.82 - lr: 0.001563\n",
            "2023-04-21 07:04:22,631 epoch 43 - iter 42/79 - loss 11.86318770 - time (sec): 9.42 - samples/sec: 5475.61 - lr: 0.001563\n",
            "2023-04-21 07:04:24,058 epoch 43 - iter 49/79 - loss 12.13099578 - time (sec): 10.84 - samples/sec: 5488.64 - lr: 0.001563\n",
            "2023-04-21 07:04:25,691 epoch 43 - iter 56/79 - loss 11.95247045 - time (sec): 12.48 - samples/sec: 5458.13 - lr: 0.001563\n",
            "2023-04-21 07:04:27,292 epoch 43 - iter 63/79 - loss 11.76991696 - time (sec): 14.08 - samples/sec: 5401.76 - lr: 0.001563\n",
            "2023-04-21 07:04:28,567 epoch 43 - iter 70/79 - loss 11.77321751 - time (sec): 15.35 - samples/sec: 5417.14 - lr: 0.001563\n",
            "2023-04-21 07:04:30,623 epoch 43 - iter 77/79 - loss 12.00848894 - time (sec): 17.41 - samples/sec: 5321.94 - lr: 0.001563\n",
            "2023-04-21 07:04:31,090 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:04:31,091 EPOCH 43 done: loss 11.9852 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:50,838 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:53,586 TRAIN : loss 4.969204425811768 - f1-score (weighted avg)  0.7511\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:03<00:00,  2.77it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:57,277 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:04:57,347 DEV : loss 28.922563552856445 - f1-score (weighted avg)  0.6313\n",
            "2023-04-21 07:04:57,356 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:04:57,360 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:04:59,154 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:05:00,437 epoch 44 - iter 7/79 - loss 11.91193068 - time (sec): 1.28 - samples/sec: 5841.84 - lr: 0.001563\n",
            "2023-04-21 07:05:02,329 epoch 44 - iter 14/79 - loss 10.59088356 - time (sec): 3.17 - samples/sec: 4908.18 - lr: 0.001563\n",
            "2023-04-21 07:05:03,896 epoch 44 - iter 21/79 - loss 10.48878346 - time (sec): 4.74 - samples/sec: 5296.46 - lr: 0.001563\n",
            "2023-04-21 07:05:05,539 epoch 44 - iter 28/79 - loss 10.06273697 - time (sec): 6.38 - samples/sec: 5288.66 - lr: 0.001563\n",
            "2023-04-21 07:05:07,316 epoch 44 - iter 35/79 - loss 10.35104916 - time (sec): 8.16 - samples/sec: 5176.96 - lr: 0.001563\n",
            "2023-04-21 07:05:08,973 epoch 44 - iter 42/79 - loss 10.43522432 - time (sec): 9.82 - samples/sec: 5180.89 - lr: 0.001563\n",
            "2023-04-21 07:05:10,620 epoch 44 - iter 49/79 - loss 10.61808828 - time (sec): 11.46 - samples/sec: 5109.46 - lr: 0.001563\n",
            "2023-04-21 07:05:12,290 epoch 44 - iter 56/79 - loss 10.76783998 - time (sec): 13.14 - samples/sec: 5148.18 - lr: 0.001563\n",
            "2023-04-21 07:05:13,816 epoch 44 - iter 63/79 - loss 10.98582378 - time (sec): 14.66 - samples/sec: 5212.67 - lr: 0.001563\n",
            "2023-04-21 07:05:15,510 epoch 44 - iter 70/79 - loss 10.83602103 - time (sec): 16.35 - samples/sec: 5147.25 - lr: 0.001563\n",
            "2023-04-21 07:05:16,898 epoch 44 - iter 77/79 - loss 11.03804591 - time (sec): 17.74 - samples/sec: 5167.38 - lr: 0.001563\n",
            "2023-04-21 07:05:17,478 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:05:17,480 EPOCH 44 done: loss 10.9284 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.08it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:05:36,995 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:05:39,723 TRAIN : loss 4.851457118988037 - f1-score (weighted avg)  0.6623\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.01it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:05:42,295 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:05:42,366 DEV : loss 28.422212600708008 - f1-score (weighted avg)  0.5629\n",
            "2023-04-21 07:05:42,375 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:05:42,379 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:05:44,185 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:05:45,769 epoch 45 - iter 7/79 - loss 11.95397246 - time (sec): 1.57 - samples/sec: 5342.26 - lr: 0.001563\n",
            "2023-04-21 07:05:47,678 epoch 45 - iter 14/79 - loss 10.99515172 - time (sec): 3.48 - samples/sec: 5209.65 - lr: 0.001563\n",
            "2023-04-21 07:05:49,007 epoch 45 - iter 21/79 - loss 11.66528239 - time (sec): 4.81 - samples/sec: 5446.13 - lr: 0.001563\n",
            "2023-04-21 07:05:50,489 epoch 45 - iter 28/79 - loss 11.58869075 - time (sec): 6.29 - samples/sec: 5587.94 - lr: 0.001563\n",
            "2023-04-21 07:05:52,387 epoch 45 - iter 35/79 - loss 11.63934408 - time (sec): 8.19 - samples/sec: 5358.63 - lr: 0.001563\n",
            "2023-04-21 07:05:53,812 epoch 45 - iter 42/79 - loss 11.36328035 - time (sec): 9.61 - samples/sec: 5388.89 - lr: 0.001563\n",
            "2023-04-21 07:05:55,394 epoch 45 - iter 49/79 - loss 11.47354080 - time (sec): 11.20 - samples/sec: 5327.22 - lr: 0.001563\n",
            "2023-04-21 07:05:56,785 epoch 45 - iter 56/79 - loss 11.36889777 - time (sec): 12.59 - samples/sec: 5397.00 - lr: 0.001563\n",
            "2023-04-21 07:05:58,370 epoch 45 - iter 63/79 - loss 11.54768935 - time (sec): 14.17 - samples/sec: 5329.28 - lr: 0.001563\n",
            "2023-04-21 07:05:59,953 epoch 45 - iter 70/79 - loss 11.58213941 - time (sec): 15.75 - samples/sec: 5325.57 - lr: 0.001563\n",
            "2023-04-21 07:06:01,361 epoch 45 - iter 77/79 - loss 11.42774573 - time (sec): 17.16 - samples/sec: 5373.12 - lr: 0.001563\n",
            "2023-04-21 07:06:01,942 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:06:01,943 EPOCH 45 done: loss 11.3603 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:06:22,827 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:06:25,566 TRAIN : loss 4.561158657073975 - f1-score (weighted avg)  0.725\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:06:28,179 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:06:28,247 DEV : loss 28.38641929626465 - f1-score (weighted avg)  0.6008\n",
            "2023-04-21 07:06:28,256 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:06:28,260 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:06:30,037 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:06:31,762 epoch 46 - iter 7/79 - loss 11.48739078 - time (sec): 1.72 - samples/sec: 5248.60 - lr: 0.001563\n",
            "2023-04-21 07:06:33,607 epoch 46 - iter 14/79 - loss 10.68897627 - time (sec): 3.57 - samples/sec: 5293.16 - lr: 0.001563\n",
            "2023-04-21 07:06:35,366 epoch 46 - iter 21/79 - loss 10.65027582 - time (sec): 5.33 - samples/sec: 5241.81 - lr: 0.001563\n",
            "2023-04-21 07:06:36,664 epoch 46 - iter 28/79 - loss 10.67624408 - time (sec): 6.62 - samples/sec: 5351.32 - lr: 0.001563\n",
            "2023-04-21 07:06:37,920 epoch 46 - iter 35/79 - loss 10.76971849 - time (sec): 7.88 - samples/sec: 5413.54 - lr: 0.001563\n",
            "2023-04-21 07:06:39,431 epoch 46 - iter 42/79 - loss 10.94774920 - time (sec): 9.39 - samples/sec: 5351.87 - lr: 0.001563\n",
            "2023-04-21 07:06:41,376 epoch 46 - iter 49/79 - loss 10.90265556 - time (sec): 11.34 - samples/sec: 5202.84 - lr: 0.001563\n",
            "2023-04-21 07:06:42,945 epoch 46 - iter 56/79 - loss 11.17984146 - time (sec): 12.91 - samples/sec: 5280.85 - lr: 0.001563\n",
            "2023-04-21 07:06:44,414 epoch 46 - iter 63/79 - loss 11.18507467 - time (sec): 14.37 - samples/sec: 5292.14 - lr: 0.001563\n",
            "2023-04-21 07:06:46,117 epoch 46 - iter 70/79 - loss 11.16849164 - time (sec): 16.08 - samples/sec: 5230.34 - lr: 0.001563\n",
            "2023-04-21 07:06:47,683 epoch 46 - iter 77/79 - loss 11.13713230 - time (sec): 17.64 - samples/sec: 5262.25 - lr: 0.001563\n",
            "2023-04-21 07:06:48,014 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:06:48,016 EPOCH 46 done: loss 11.1370 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:07,969 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:11,969 TRAIN : loss 4.873581886291504 - f1-score (weighted avg)  0.705\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.02it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:14,541 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:07:14,611 DEV : loss 28.59395408630371 - f1-score (weighted avg)  0.59\n",
            "2023-04-21 07:07:14,621 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:07:14,625 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:16,161 epoch 47 - iter 7/79 - loss 10.34270944 - time (sec): 1.54 - samples/sec: 5497.66 - lr: 0.001563\n",
            "2023-04-21 07:07:17,680 epoch 47 - iter 14/79 - loss 9.69019241 - time (sec): 3.05 - samples/sec: 5344.36 - lr: 0.001563\n",
            "2023-04-21 07:07:18,929 epoch 47 - iter 21/79 - loss 10.41460140 - time (sec): 4.30 - samples/sec: 5627.04 - lr: 0.001563\n",
            "2023-04-21 07:07:20,711 epoch 47 - iter 28/79 - loss 9.95013097 - time (sec): 6.08 - samples/sec: 5395.99 - lr: 0.001563\n",
            "2023-04-21 07:07:22,298 epoch 47 - iter 35/79 - loss 10.21617461 - time (sec): 7.67 - samples/sec: 5366.88 - lr: 0.001563\n",
            "2023-04-21 07:07:24,070 epoch 47 - iter 42/79 - loss 10.40407056 - time (sec): 9.44 - samples/sec: 5412.99 - lr: 0.001563\n",
            "2023-04-21 07:07:25,485 epoch 47 - iter 49/79 - loss 10.28428745 - time (sec): 10.86 - samples/sec: 5505.66 - lr: 0.001563\n",
            "2023-04-21 07:07:26,885 epoch 47 - iter 56/79 - loss 10.54042360 - time (sec): 12.26 - samples/sec: 5458.90 - lr: 0.001563\n",
            "2023-04-21 07:07:28,140 epoch 47 - iter 63/79 - loss 10.72658794 - time (sec): 13.51 - samples/sec: 5544.96 - lr: 0.001563\n",
            "2023-04-21 07:07:29,910 epoch 47 - iter 70/79 - loss 10.63699049 - time (sec): 15.28 - samples/sec: 5521.81 - lr: 0.001563\n",
            "2023-04-21 07:07:31,247 epoch 47 - iter 77/79 - loss 10.76161011 - time (sec): 16.62 - samples/sec: 5568.61 - lr: 0.001563\n",
            "2023-04-21 07:07:31,617 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:07:31,619 EPOCH 47 done: loss 10.7275 - lr 0.001563\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:51,342 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:54,065 TRAIN : loss 4.580204486846924 - f1-score (weighted avg)  0.7027\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:56,666 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:07:56,738 DEV : loss 28.492897033691406 - f1-score (weighted avg)  0.5702\n",
            "2023-04-21 07:07:56,748 Epoch    47: reducing learning rate of group 0 to 7.8125e-04.\n",
            "2023-04-21 07:07:56,749 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:07:56,752 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:07:58,262 epoch 48 - iter 7/79 - loss 9.23495690 - time (sec): 1.51 - samples/sec: 5771.17 - lr: 0.000781\n",
            "2023-04-21 07:07:59,654 epoch 48 - iter 14/79 - loss 9.83544237 - time (sec): 2.90 - samples/sec: 5757.64 - lr: 0.000781\n",
            "2023-04-21 07:08:01,148 epoch 48 - iter 21/79 - loss 10.72215047 - time (sec): 4.39 - samples/sec: 5594.82 - lr: 0.000781\n",
            "2023-04-21 07:08:02,428 epoch 48 - iter 28/79 - loss 10.87057477 - time (sec): 5.67 - samples/sec: 5560.86 - lr: 0.000781\n",
            "2023-04-21 07:08:03,775 epoch 48 - iter 35/79 - loss 10.50561218 - time (sec): 7.02 - samples/sec: 5520.22 - lr: 0.000781\n",
            "2023-04-21 07:08:05,204 epoch 48 - iter 42/79 - loss 10.34105251 - time (sec): 8.45 - samples/sec: 5551.39 - lr: 0.000781\n",
            "2023-04-21 07:08:06,739 epoch 48 - iter 49/79 - loss 10.24743137 - time (sec): 9.98 - samples/sec: 5527.19 - lr: 0.000781\n",
            "2023-04-21 07:08:08,085 epoch 48 - iter 56/79 - loss 10.34698349 - time (sec): 11.33 - samples/sec: 5683.89 - lr: 0.000781\n",
            "2023-04-21 07:08:09,958 epoch 48 - iter 63/79 - loss 10.33362065 - time (sec): 13.20 - samples/sec: 5612.41 - lr: 0.000781\n",
            "2023-04-21 07:08:11,600 epoch 48 - iter 70/79 - loss 10.20254995 - time (sec): 14.85 - samples/sec: 5606.12 - lr: 0.000781\n",
            "2023-04-21 07:08:13,667 epoch 48 - iter 77/79 - loss 10.18360167 - time (sec): 16.91 - samples/sec: 5484.77 - lr: 0.000781\n",
            "2023-04-21 07:08:14,114 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:08:14,115 EPOCH 48 done: loss 10.2909 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:08:34,991 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:08:37,730 TRAIN : loss 4.508190631866455 - f1-score (weighted avg)  0.7127\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:08:40,314 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:08:40,382 DEV : loss 28.292133331298828 - f1-score (weighted avg)  0.5941\n",
            "2023-04-21 07:08:40,393 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:08:40,396 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:08:42,201 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:08:43,529 epoch 49 - iter 7/79 - loss 11.13507732 - time (sec): 1.33 - samples/sec: 5951.58 - lr: 0.000781\n",
            "2023-04-21 07:08:44,982 epoch 49 - iter 14/79 - loss 10.28253041 - time (sec): 2.78 - samples/sec: 5622.08 - lr: 0.000781\n",
            "2023-04-21 07:08:46,353 epoch 49 - iter 21/79 - loss 10.70974135 - time (sec): 4.15 - samples/sec: 5791.17 - lr: 0.000781\n",
            "2023-04-21 07:08:47,999 epoch 49 - iter 28/79 - loss 10.48503666 - time (sec): 5.80 - samples/sec: 5671.89 - lr: 0.000781\n",
            "2023-04-21 07:08:49,410 epoch 49 - iter 35/79 - loss 10.57362699 - time (sec): 7.21 - samples/sec: 5730.32 - lr: 0.000781\n",
            "2023-04-21 07:08:50,955 epoch 49 - iter 42/79 - loss 10.30611248 - time (sec): 8.75 - samples/sec: 5670.43 - lr: 0.000781\n",
            "2023-04-21 07:08:52,324 epoch 49 - iter 49/79 - loss 10.12108841 - time (sec): 10.12 - samples/sec: 5632.94 - lr: 0.000781\n",
            "2023-04-21 07:08:54,289 epoch 49 - iter 56/79 - loss 9.76849638 - time (sec): 12.09 - samples/sec: 5535.96 - lr: 0.000781\n",
            "2023-04-21 07:08:55,822 epoch 49 - iter 63/79 - loss 9.44518231 - time (sec): 13.62 - samples/sec: 5485.82 - lr: 0.000781\n",
            "2023-04-21 07:08:57,382 epoch 49 - iter 70/79 - loss 9.62380206 - time (sec): 15.18 - samples/sec: 5451.45 - lr: 0.000781\n",
            "2023-04-21 07:08:59,492 epoch 49 - iter 77/79 - loss 9.64206676 - time (sec): 17.29 - samples/sec: 5306.99 - lr: 0.000781\n",
            "2023-04-21 07:09:00,095 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:09:00,097 EPOCH 49 done: loss 9.7121 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:09:20,031 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:09:22,755 TRAIN : loss 4.4568915367126465 - f1-score (weighted avg)  0.7317\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:09:25,372 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:09:25,441 DEV : loss 28.18021583557129 - f1-score (weighted avg)  0.6092\n",
            "2023-04-21 07:09:25,451 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:09:25,455 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:09:27,266 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:09:28,821 epoch 50 - iter 7/79 - loss 9.84920845 - time (sec): 1.54 - samples/sec: 5512.08 - lr: 0.000781\n",
            "2023-04-21 07:09:30,083 epoch 50 - iter 14/79 - loss 9.96247747 - time (sec): 2.80 - samples/sec: 5739.36 - lr: 0.000781\n",
            "2023-04-21 07:09:31,303 epoch 50 - iter 21/79 - loss 10.65045279 - time (sec): 4.02 - samples/sec: 6162.29 - lr: 0.000781\n",
            "2023-04-21 07:09:32,617 epoch 50 - iter 28/79 - loss 10.78525513 - time (sec): 5.34 - samples/sec: 6077.63 - lr: 0.000781\n",
            "2023-04-21 07:09:34,647 epoch 50 - iter 35/79 - loss 11.08311103 - time (sec): 7.37 - samples/sec: 5495.78 - lr: 0.000781\n",
            "2023-04-21 07:09:36,591 epoch 50 - iter 42/79 - loss 11.18002231 - time (sec): 9.31 - samples/sec: 5200.47 - lr: 0.000781\n",
            "2023-04-21 07:09:38,373 epoch 50 - iter 49/79 - loss 10.91157143 - time (sec): 11.09 - samples/sec: 5185.39 - lr: 0.000781\n",
            "2023-04-21 07:09:39,859 epoch 50 - iter 56/79 - loss 10.69309468 - time (sec): 12.58 - samples/sec: 5204.24 - lr: 0.000781\n",
            "2023-04-21 07:09:41,512 epoch 50 - iter 63/79 - loss 10.81367947 - time (sec): 14.23 - samples/sec: 5234.83 - lr: 0.000781\n",
            "2023-04-21 07:09:42,997 epoch 50 - iter 70/79 - loss 10.77142146 - time (sec): 15.72 - samples/sec: 5269.38 - lr: 0.000781\n",
            "2023-04-21 07:09:44,724 epoch 50 - iter 77/79 - loss 10.61642673 - time (sec): 17.44 - samples/sec: 5294.13 - lr: 0.000781\n",
            "2023-04-21 07:09:45,152 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:09:45,154 EPOCH 50 done: loss 10.6492 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.82it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:05,983 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:08,727 TRAIN : loss 4.151300430297852 - f1-score (weighted avg)  0.7579\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.89it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:11,377 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:10:11,447 DEV : loss 28.04401206970215 - f1-score (weighted avg)  0.6368\n",
            "2023-04-21 07:10:11,456 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:10:11,460 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:13,247 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:10:15,288 epoch 51 - iter 7/79 - loss 9.76380229 - time (sec): 2.04 - samples/sec: 4877.68 - lr: 0.000781\n",
            "2023-04-21 07:10:16,764 epoch 51 - iter 14/79 - loss 9.01581274 - time (sec): 3.51 - samples/sec: 5021.22 - lr: 0.000781\n",
            "2023-04-21 07:10:18,728 epoch 51 - iter 21/79 - loss 9.40587583 - time (sec): 5.48 - samples/sec: 4771.73 - lr: 0.000781\n",
            "2023-04-21 07:10:20,306 epoch 51 - iter 28/79 - loss 9.33969156 - time (sec): 7.06 - samples/sec: 4845.26 - lr: 0.000781\n",
            "2023-04-21 07:10:21,753 epoch 51 - iter 35/79 - loss 9.26726974 - time (sec): 8.50 - samples/sec: 4933.72 - lr: 0.000781\n",
            "2023-04-21 07:10:23,101 epoch 51 - iter 42/79 - loss 9.80539458 - time (sec): 9.85 - samples/sec: 5088.34 - lr: 0.000781\n",
            "2023-04-21 07:10:24,515 epoch 51 - iter 49/79 - loss 10.04649153 - time (sec): 11.27 - samples/sec: 5237.36 - lr: 0.000781\n",
            "2023-04-21 07:10:25,986 epoch 51 - iter 56/79 - loss 9.80342418 - time (sec): 12.74 - samples/sec: 5274.87 - lr: 0.000781\n",
            "2023-04-21 07:10:27,431 epoch 51 - iter 63/79 - loss 9.71255235 - time (sec): 14.18 - samples/sec: 5337.39 - lr: 0.000781\n",
            "2023-04-21 07:10:28,910 epoch 51 - iter 70/79 - loss 9.55239726 - time (sec): 15.66 - samples/sec: 5384.00 - lr: 0.000781\n",
            "2023-04-21 07:10:30,749 epoch 51 - iter 77/79 - loss 9.56875054 - time (sec): 17.50 - samples/sec: 5308.19 - lr: 0.000781\n",
            "2023-04-21 07:10:31,080 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:10:31,082 EPOCH 51 done: loss 9.6594 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.01it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:52,088 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:10:54,821 TRAIN : loss 4.2927165031433105 - f1-score (weighted avg)  0.7532\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:57,428 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:10:57,498 DEV : loss 28.028043746948242 - f1-score (weighted avg)  0.625\n",
            "2023-04-21 07:10:57,507 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:10:57,511 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:10:59,288 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:11:00,897 epoch 52 - iter 7/79 - loss 9.35869696 - time (sec): 1.61 - samples/sec: 5196.14 - lr: 0.000781\n",
            "2023-04-21 07:11:02,417 epoch 52 - iter 14/79 - loss 10.00974176 - time (sec): 3.13 - samples/sec: 5524.72 - lr: 0.000781\n",
            "2023-04-21 07:11:03,869 epoch 52 - iter 21/79 - loss 10.71763276 - time (sec): 4.58 - samples/sec: 5614.89 - lr: 0.000781\n",
            "2023-04-21 07:11:05,428 epoch 52 - iter 28/79 - loss 10.56548223 - time (sec): 6.14 - samples/sec: 5457.73 - lr: 0.000781\n",
            "2023-04-21 07:11:06,866 epoch 52 - iter 35/79 - loss 10.18136956 - time (sec): 7.57 - samples/sec: 5631.45 - lr: 0.000781\n",
            "2023-04-21 07:11:08,383 epoch 52 - iter 42/79 - loss 10.45444186 - time (sec): 9.09 - samples/sec: 5562.14 - lr: 0.000781\n",
            "2023-04-21 07:11:09,851 epoch 52 - iter 49/79 - loss 10.39874942 - time (sec): 10.56 - samples/sec: 5505.02 - lr: 0.000781\n",
            "2023-04-21 07:11:11,477 epoch 52 - iter 56/79 - loss 10.47018195 - time (sec): 12.19 - samples/sec: 5515.85 - lr: 0.000781\n",
            "2023-04-21 07:11:13,238 epoch 52 - iter 63/79 - loss 10.30515461 - time (sec): 13.95 - samples/sec: 5444.47 - lr: 0.000781\n",
            "2023-04-21 07:11:14,887 epoch 52 - iter 70/79 - loss 10.03604398 - time (sec): 15.60 - samples/sec: 5417.85 - lr: 0.000781\n",
            "2023-04-21 07:11:16,339 epoch 52 - iter 77/79 - loss 9.95923526 - time (sec): 17.05 - samples/sec: 5441.49 - lr: 0.000781\n",
            "2023-04-21 07:11:16,695 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:11:16,696 EPOCH 52 done: loss 9.9071 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:11:36,402 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:11:40,408 TRAIN : loss 3.9862284660339355 - f1-score (weighted avg)  0.7389\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:11:43,028 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:11:43,096 DEV : loss 27.35747528076172 - f1-score (weighted avg)  0.62\n",
            "2023-04-21 07:11:43,105 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:11:43,109 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:11:44,917 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:11:46,583 epoch 53 - iter 7/79 - loss 9.58343923 - time (sec): 1.66 - samples/sec: 5442.44 - lr: 0.000781\n",
            "2023-04-21 07:11:47,900 epoch 53 - iter 14/79 - loss 9.82389549 - time (sec): 2.98 - samples/sec: 5566.97 - lr: 0.000781\n",
            "2023-04-21 07:11:49,406 epoch 53 - iter 21/79 - loss 9.58425030 - time (sec): 4.49 - samples/sec: 5692.54 - lr: 0.000781\n",
            "2023-04-21 07:11:51,439 epoch 53 - iter 28/79 - loss 9.19209732 - time (sec): 6.52 - samples/sec: 5208.97 - lr: 0.000781\n",
            "2023-04-21 07:11:52,912 epoch 53 - iter 35/79 - loss 9.54665978 - time (sec): 7.99 - samples/sec: 5359.43 - lr: 0.000781\n",
            "2023-04-21 07:11:54,754 epoch 53 - iter 42/79 - loss 9.27349847 - time (sec): 9.83 - samples/sec: 5224.93 - lr: 0.000781\n",
            "2023-04-21 07:11:56,223 epoch 53 - iter 49/79 - loss 9.51284152 - time (sec): 11.30 - samples/sec: 5278.25 - lr: 0.000781\n",
            "2023-04-21 07:11:57,778 epoch 53 - iter 56/79 - loss 9.78484247 - time (sec): 12.86 - samples/sec: 5322.14 - lr: 0.000781\n",
            "2023-04-21 07:11:59,333 epoch 53 - iter 63/79 - loss 9.49280299 - time (sec): 14.41 - samples/sec: 5303.30 - lr: 0.000781\n",
            "2023-04-21 07:12:01,143 epoch 53 - iter 70/79 - loss 9.70264093 - time (sec): 16.22 - samples/sec: 5194.92 - lr: 0.000781\n",
            "2023-04-21 07:12:02,470 epoch 53 - iter 77/79 - loss 9.61567495 - time (sec): 17.55 - samples/sec: 5276.39 - lr: 0.000781\n",
            "2023-04-21 07:12:02,874 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:12:02,875 EPOCH 53 done: loss 9.6060 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:12:22,741 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:12:25,478 TRAIN : loss 4.468001365661621 - f1-score (weighted avg)  0.7204\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:12:28,102 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:12:28,171 DEV : loss 27.950645446777344 - f1-score (weighted avg)  0.605\n",
            "2023-04-21 07:12:28,181 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:12:28,185 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:12:29,558 epoch 54 - iter 7/79 - loss 10.14712885 - time (sec): 1.37 - samples/sec: 5207.15 - lr: 0.000781\n",
            "2023-04-21 07:12:31,036 epoch 54 - iter 14/79 - loss 10.14773416 - time (sec): 2.85 - samples/sec: 5212.81 - lr: 0.000781\n",
            "2023-04-21 07:12:33,030 epoch 54 - iter 21/79 - loss 9.48652739 - time (sec): 4.84 - samples/sec: 4911.48 - lr: 0.000781\n",
            "2023-04-21 07:12:34,563 epoch 54 - iter 28/79 - loss 9.79371042 - time (sec): 6.38 - samples/sec: 4978.62 - lr: 0.000781\n",
            "2023-04-21 07:12:35,988 epoch 54 - iter 35/79 - loss 9.91819294 - time (sec): 7.80 - samples/sec: 5161.78 - lr: 0.000781\n",
            "2023-04-21 07:12:37,455 epoch 54 - iter 42/79 - loss 10.06331847 - time (sec): 9.27 - samples/sec: 5190.61 - lr: 0.000781\n",
            "2023-04-21 07:12:38,904 epoch 54 - iter 49/79 - loss 10.29050465 - time (sec): 10.72 - samples/sec: 5252.76 - lr: 0.000781\n",
            "2023-04-21 07:12:40,284 epoch 54 - iter 56/79 - loss 10.18799835 - time (sec): 12.10 - samples/sec: 5307.80 - lr: 0.000781\n",
            "2023-04-21 07:12:41,757 epoch 54 - iter 63/79 - loss 9.97555874 - time (sec): 13.57 - samples/sec: 5420.50 - lr: 0.000781\n",
            "2023-04-21 07:12:43,351 epoch 54 - iter 70/79 - loss 10.05932145 - time (sec): 15.16 - samples/sec: 5475.91 - lr: 0.000781\n",
            "2023-04-21 07:12:45,445 epoch 54 - iter 77/79 - loss 9.91487823 - time (sec): 17.26 - samples/sec: 5339.19 - lr: 0.000781\n",
            "2023-04-21 07:12:45,964 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:12:45,966 EPOCH 54 done: loss 9.8863 - lr 0.000781\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:06,736 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:13:09,502 TRAIN : loss 4.164370059967041 - f1-score (weighted avg)  0.7377\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:12,099 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:13:12,168 DEV : loss 27.690391540527344 - f1-score (weighted avg)  0.6134\n",
            "2023-04-21 07:13:12,177 Epoch    54: reducing learning rate of group 0 to 3.9063e-04.\n",
            "2023-04-21 07:13:12,179 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:13:12,182 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:13,283 epoch 55 - iter 7/79 - loss 10.08434673 - time (sec): 1.10 - samples/sec: 6134.45 - lr: 0.000391\n",
            "2023-04-21 07:13:14,914 epoch 55 - iter 14/79 - loss 9.46035671 - time (sec): 2.73 - samples/sec: 5498.99 - lr: 0.000391\n",
            "2023-04-21 07:13:16,390 epoch 55 - iter 21/79 - loss 9.37356932 - time (sec): 4.21 - samples/sec: 5545.68 - lr: 0.000391\n",
            "2023-04-21 07:13:17,994 epoch 55 - iter 28/79 - loss 8.82629784 - time (sec): 5.81 - samples/sec: 5526.72 - lr: 0.000391\n",
            "2023-04-21 07:13:19,411 epoch 55 - iter 35/79 - loss 8.86024989 - time (sec): 7.23 - samples/sec: 5589.64 - lr: 0.000391\n",
            "2023-04-21 07:13:20,909 epoch 55 - iter 42/79 - loss 8.87578026 - time (sec): 8.73 - samples/sec: 5545.00 - lr: 0.000391\n",
            "2023-04-21 07:13:22,310 epoch 55 - iter 49/79 - loss 8.59278237 - time (sec): 10.13 - samples/sec: 5603.60 - lr: 0.000391\n",
            "2023-04-21 07:13:24,434 epoch 55 - iter 56/79 - loss 8.84047924 - time (sec): 12.25 - samples/sec: 5416.58 - lr: 0.000391\n",
            "2023-04-21 07:13:26,385 epoch 55 - iter 63/79 - loss 9.02106993 - time (sec): 14.20 - samples/sec: 5270.50 - lr: 0.000391\n",
            "2023-04-21 07:13:27,937 epoch 55 - iter 70/79 - loss 9.06757621 - time (sec): 15.75 - samples/sec: 5357.58 - lr: 0.000391\n",
            "2023-04-21 07:13:29,247 epoch 55 - iter 77/79 - loss 9.25305545 - time (sec): 17.06 - samples/sec: 5437.72 - lr: 0.000391\n",
            "2023-04-21 07:13:29,591 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:13:29,593 EPOCH 55 done: loss 9.2376 - lr 0.000391\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:49,169 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:51,904 TRAIN : loss 4.011787414550781 - f1-score (weighted avg)  0.7297\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.02it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:54,472 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:13:54,540 DEV : loss 27.355224609375 - f1-score (weighted avg)  0.5926\n",
            "2023-04-21 07:13:54,549 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:13:54,553 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:13:56,336 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:13:57,795 epoch 56 - iter 7/79 - loss 7.86060810 - time (sec): 1.44 - samples/sec: 5659.38 - lr: 0.000391\n",
            "2023-04-21 07:13:59,447 epoch 56 - iter 14/79 - loss 8.36716557 - time (sec): 3.10 - samples/sec: 5291.14 - lr: 0.000391\n",
            "2023-04-21 07:14:00,827 epoch 56 - iter 21/79 - loss 8.77336468 - time (sec): 4.48 - samples/sec: 5511.27 - lr: 0.000391\n",
            "2023-04-21 07:14:02,638 epoch 56 - iter 28/79 - loss 8.63174515 - time (sec): 6.29 - samples/sec: 5464.05 - lr: 0.000391\n",
            "2023-04-21 07:14:03,841 epoch 56 - iter 35/79 - loss 9.05965821 - time (sec): 7.49 - samples/sec: 5598.94 - lr: 0.000391\n",
            "2023-04-21 07:14:05,540 epoch 56 - iter 42/79 - loss 9.22222361 - time (sec): 9.19 - samples/sec: 5450.10 - lr: 0.000391\n",
            "2023-04-21 07:14:06,930 epoch 56 - iter 49/79 - loss 9.29793207 - time (sec): 10.58 - samples/sec: 5446.18 - lr: 0.000391\n",
            "2023-04-21 07:14:08,350 epoch 56 - iter 56/79 - loss 9.34520716 - time (sec): 12.00 - samples/sec: 5466.96 - lr: 0.000391\n",
            "2023-04-21 07:14:09,845 epoch 56 - iter 63/79 - loss 9.22269317 - time (sec): 13.49 - samples/sec: 5402.23 - lr: 0.000391\n",
            "2023-04-21 07:14:11,623 epoch 56 - iter 70/79 - loss 9.25774739 - time (sec): 15.27 - samples/sec: 5358.27 - lr: 0.000391\n",
            "2023-04-21 07:14:13,682 epoch 56 - iter 77/79 - loss 9.02934129 - time (sec): 17.33 - samples/sec: 5333.43 - lr: 0.000391\n",
            "2023-04-21 07:14:14,076 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:14:14,078 EPOCH 56 done: loss 9.0796 - lr 0.000391\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.82it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:14:34,898 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:14:37,620 TRAIN : loss 3.695814847946167 - f1-score (weighted avg)  0.768\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.00it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:14:40,203 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:14:40,271 DEV : loss 26.905675888061523 - f1-score (weighted avg)  0.6404\n",
            "2023-04-21 07:14:40,280 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:14:40,284 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:14:42,076 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:14:43,735 epoch 57 - iter 7/79 - loss 6.57334349 - time (sec): 1.64 - samples/sec: 5385.74 - lr: 0.000391\n",
            "2023-04-21 07:14:45,556 epoch 57 - iter 14/79 - loss 9.12327131 - time (sec): 3.47 - samples/sec: 5194.83 - lr: 0.000391\n",
            "2023-04-21 07:14:47,031 epoch 57 - iter 21/79 - loss 9.66611393 - time (sec): 4.94 - samples/sec: 5263.90 - lr: 0.000391\n",
            "2023-04-21 07:14:48,757 epoch 57 - iter 28/79 - loss 9.79245824 - time (sec): 6.67 - samples/sec: 5227.28 - lr: 0.000391\n",
            "2023-04-21 07:14:50,302 epoch 57 - iter 35/79 - loss 9.89809317 - time (sec): 8.21 - samples/sec: 5147.50 - lr: 0.000391\n",
            "2023-04-21 07:14:51,930 epoch 57 - iter 42/79 - loss 10.02870701 - time (sec): 9.84 - samples/sec: 5091.77 - lr: 0.000391\n",
            "2023-04-21 07:14:53,301 epoch 57 - iter 49/79 - loss 9.74631984 - time (sec): 11.21 - samples/sec: 5153.65 - lr: 0.000391\n",
            "2023-04-21 07:14:54,614 epoch 57 - iter 56/79 - loss 9.59290012 - time (sec): 12.52 - samples/sec: 5228.90 - lr: 0.000391\n",
            "2023-04-21 07:14:55,989 epoch 57 - iter 63/79 - loss 9.49806639 - time (sec): 13.90 - samples/sec: 5269.93 - lr: 0.000391\n",
            "2023-04-21 07:14:57,745 epoch 57 - iter 70/79 - loss 9.39600061 - time (sec): 15.65 - samples/sec: 5298.27 - lr: 0.000391\n",
            "2023-04-21 07:14:59,102 epoch 57 - iter 77/79 - loss 9.19872058 - time (sec): 17.01 - samples/sec: 5360.62 - lr: 0.000391\n",
            "2023-04-21 07:14:59,676 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:14:59,678 EPOCH 57 done: loss 9.2030 - lr 0.000391\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.02it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:15:20,623 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:15:23,357 TRAIN : loss 3.714247465133667 - f1-score (weighted avg)  0.7587\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:15:25,945 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:15:26,014 DEV : loss 26.884872436523438 - f1-score (weighted avg)  0.6408\n",
            "2023-04-21 07:15:26,024 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:15:26,028 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:15:27,822 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:15:29,797 epoch 58 - iter 7/79 - loss 7.06767406 - time (sec): 1.97 - samples/sec: 4389.19 - lr: 0.000391\n",
            "2023-04-21 07:15:31,540 epoch 58 - iter 14/79 - loss 8.70225390 - time (sec): 3.72 - samples/sec: 4904.22 - lr: 0.000391\n",
            "2023-04-21 07:15:32,790 epoch 58 - iter 21/79 - loss 9.03514863 - time (sec): 4.97 - samples/sec: 5365.09 - lr: 0.000391\n",
            "2023-04-21 07:15:34,143 epoch 58 - iter 28/79 - loss 9.49706180 - time (sec): 6.32 - samples/sec: 5554.30 - lr: 0.000391\n",
            "2023-04-21 07:15:35,886 epoch 58 - iter 35/79 - loss 9.02305272 - time (sec): 8.06 - samples/sec: 5331.57 - lr: 0.000391\n",
            "2023-04-21 07:15:37,325 epoch 58 - iter 42/79 - loss 9.07639542 - time (sec): 9.50 - samples/sec: 5374.18 - lr: 0.000391\n",
            "2023-04-21 07:15:38,508 epoch 58 - iter 49/79 - loss 8.93182839 - time (sec): 10.68 - samples/sec: 5418.15 - lr: 0.000391\n",
            "2023-04-21 07:15:40,086 epoch 58 - iter 56/79 - loss 8.82576700 - time (sec): 12.26 - samples/sec: 5475.99 - lr: 0.000391\n",
            "2023-04-21 07:15:42,136 epoch 58 - iter 63/79 - loss 9.04834944 - time (sec): 14.31 - samples/sec: 5364.42 - lr: 0.000391\n",
            "2023-04-21 07:15:43,594 epoch 58 - iter 70/79 - loss 9.05848575 - time (sec): 15.77 - samples/sec: 5380.89 - lr: 0.000391\n",
            "2023-04-21 07:15:45,140 epoch 58 - iter 77/79 - loss 9.01066872 - time (sec): 17.32 - samples/sec: 5372.40 - lr: 0.000391\n",
            "2023-04-21 07:15:45,495 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:15:45,497 EPOCH 58 done: loss 9.0927 - lr 0.000391\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:05,235 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:09,250 TRAIN : loss 3.8045568466186523 - f1-score (weighted avg)  0.7587\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.92it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:11,896 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:16:11,964 DEV : loss 27.043445587158203 - f1-score (weighted avg)  0.6286\n",
            "2023-04-21 07:16:11,973 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:16:11,977 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:13,707 epoch 59 - iter 7/79 - loss 9.30613664 - time (sec): 1.73 - samples/sec: 4784.08 - lr: 0.000391\n",
            "2023-04-21 07:16:15,059 epoch 59 - iter 14/79 - loss 9.95496967 - time (sec): 3.08 - samples/sec: 5243.64 - lr: 0.000391\n",
            "2023-04-21 07:16:16,564 epoch 59 - iter 21/79 - loss 9.61933735 - time (sec): 4.58 - samples/sec: 5430.82 - lr: 0.000391\n",
            "2023-04-21 07:16:18,331 epoch 59 - iter 28/79 - loss 10.14035648 - time (sec): 6.35 - samples/sec: 5294.63 - lr: 0.000391\n",
            "2023-04-21 07:16:19,797 epoch 59 - iter 35/79 - loss 10.09266211 - time (sec): 7.82 - samples/sec: 5298.12 - lr: 0.000391\n",
            "2023-04-21 07:16:21,161 epoch 59 - iter 42/79 - loss 10.08195874 - time (sec): 9.18 - samples/sec: 5374.27 - lr: 0.000391\n",
            "2023-04-21 07:16:23,155 epoch 59 - iter 49/79 - loss 9.74683948 - time (sec): 11.18 - samples/sec: 5258.45 - lr: 0.000391\n",
            "2023-04-21 07:16:24,806 epoch 59 - iter 56/79 - loss 9.47259595 - time (sec): 12.83 - samples/sec: 5307.50 - lr: 0.000391\n",
            "2023-04-21 07:16:26,652 epoch 59 - iter 63/79 - loss 9.26930129 - time (sec): 14.67 - samples/sec: 5213.01 - lr: 0.000391\n",
            "2023-04-21 07:16:28,018 epoch 59 - iter 70/79 - loss 9.44239061 - time (sec): 16.04 - samples/sec: 5277.89 - lr: 0.000391\n",
            "2023-04-21 07:16:29,492 epoch 59 - iter 77/79 - loss 9.39148108 - time (sec): 17.51 - samples/sec: 5253.69 - lr: 0.000391\n",
            "2023-04-21 07:16:29,891 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:16:29,893 EPOCH 59 done: loss 9.4303 - lr 0.000391\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:49,556 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:52,292 TRAIN : loss 3.94146466255188 - f1-score (weighted avg)  0.756\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:54,899 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:16:54,969 DEV : loss 27.261234283447266 - f1-score (weighted avg)  0.6408\n",
            "2023-04-21 07:16:54,978 Epoch    59: reducing learning rate of group 0 to 1.9531e-04.\n",
            "2023-04-21 07:16:54,980 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:16:54,983 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:16:56,742 epoch 60 - iter 7/79 - loss 8.35173927 - time (sec): 1.76 - samples/sec: 5382.38 - lr: 0.000195\n",
            "2023-04-21 07:16:58,194 epoch 60 - iter 14/79 - loss 8.05104589 - time (sec): 3.21 - samples/sec: 5147.05 - lr: 0.000195\n",
            "2023-04-21 07:16:59,727 epoch 60 - iter 21/79 - loss 8.93892483 - time (sec): 4.74 - samples/sec: 5197.82 - lr: 0.000195\n",
            "2023-04-21 07:17:01,155 epoch 60 - iter 28/79 - loss 9.69527925 - time (sec): 6.17 - samples/sec: 5330.23 - lr: 0.000195\n",
            "2023-04-21 07:17:02,537 epoch 60 - iter 35/79 - loss 8.90216252 - time (sec): 7.55 - samples/sec: 5496.78 - lr: 0.000195\n",
            "2023-04-21 07:17:04,436 epoch 60 - iter 42/79 - loss 8.98677454 - time (sec): 9.45 - samples/sec: 5408.96 - lr: 0.000195\n",
            "2023-04-21 07:17:06,018 epoch 60 - iter 49/79 - loss 8.78611612 - time (sec): 11.03 - samples/sec: 5433.48 - lr: 0.000195\n",
            "2023-04-21 07:17:07,669 epoch 60 - iter 56/79 - loss 8.82180409 - time (sec): 12.68 - samples/sec: 5402.40 - lr: 0.000195\n",
            "2023-04-21 07:17:08,862 epoch 60 - iter 63/79 - loss 8.79850518 - time (sec): 13.88 - samples/sec: 5413.42 - lr: 0.000195\n",
            "2023-04-21 07:17:10,417 epoch 60 - iter 70/79 - loss 8.68804235 - time (sec): 15.43 - samples/sec: 5443.82 - lr: 0.000195\n",
            "2023-04-21 07:17:11,867 epoch 60 - iter 77/79 - loss 8.80075409 - time (sec): 16.88 - samples/sec: 5486.69 - lr: 0.000195\n",
            "2023-04-21 07:17:12,279 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:17:12,280 EPOCH 60 done: loss 8.7573 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:17:33,081 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:17:35,823 TRAIN : loss 3.6518635749816895 - f1-score (weighted avg)  0.7605\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:17:38,405 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:17:38,478 DEV : loss 26.728435516357422 - f1-score (weighted avg)  0.6306\n",
            "2023-04-21 07:17:38,488 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:17:38,492 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:17:40,257 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:17:41,740 epoch 61 - iter 7/79 - loss 9.53934058 - time (sec): 1.48 - samples/sec: 5408.81 - lr: 0.000195\n",
            "2023-04-21 07:17:42,994 epoch 61 - iter 14/79 - loss 8.33270113 - time (sec): 2.73 - samples/sec: 5737.98 - lr: 0.000195\n",
            "2023-04-21 07:17:44,769 epoch 61 - iter 21/79 - loss 8.54480393 - time (sec): 4.51 - samples/sec: 5519.88 - lr: 0.000195\n",
            "2023-04-21 07:17:46,146 epoch 61 - iter 28/79 - loss 8.30219454 - time (sec): 5.89 - samples/sec: 5506.38 - lr: 0.000195\n",
            "2023-04-21 07:17:47,983 epoch 61 - iter 35/79 - loss 8.10391777 - time (sec): 7.72 - samples/sec: 5277.78 - lr: 0.000195\n",
            "2023-04-21 07:17:49,607 epoch 61 - iter 42/79 - loss 8.20451983 - time (sec): 9.35 - samples/sec: 5408.92 - lr: 0.000195\n",
            "2023-04-21 07:17:51,158 epoch 61 - iter 49/79 - loss 8.02635804 - time (sec): 10.90 - samples/sec: 5386.98 - lr: 0.000195\n",
            "2023-04-21 07:17:52,577 epoch 61 - iter 56/79 - loss 8.14320685 - time (sec): 12.32 - samples/sec: 5389.58 - lr: 0.000195\n",
            "2023-04-21 07:17:53,980 epoch 61 - iter 63/79 - loss 8.18457820 - time (sec): 13.72 - samples/sec: 5432.09 - lr: 0.000195\n",
            "2023-04-21 07:17:55,278 epoch 61 - iter 70/79 - loss 8.44926014 - time (sec): 15.02 - samples/sec: 5460.64 - lr: 0.000195\n",
            "2023-04-21 07:17:57,191 epoch 61 - iter 77/79 - loss 8.73418232 - time (sec): 16.93 - samples/sec: 5440.87 - lr: 0.000195\n",
            "2023-04-21 07:17:57,636 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:17:57,637 EPOCH 61 done: loss 8.8268 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:18:17,282 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:18:21,276 TRAIN : loss 3.6387858390808105 - f1-score (weighted avg)  0.7594\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:18:23,879 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:18:23,948 DEV : loss 26.708513259887695 - f1-score (weighted avg)  0.6283\n",
            "2023-04-21 07:18:23,958 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:18:23,961 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:18:25,762 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:18:27,736 epoch 62 - iter 7/79 - loss 10.11965248 - time (sec): 1.97 - samples/sec: 4705.61 - lr: 0.000195\n",
            "2023-04-21 07:18:29,251 epoch 62 - iter 14/79 - loss 9.12166260 - time (sec): 3.49 - samples/sec: 5178.18 - lr: 0.000195\n",
            "2023-04-21 07:18:30,556 epoch 62 - iter 21/79 - loss 9.07347790 - time (sec): 4.79 - samples/sec: 5302.17 - lr: 0.000195\n",
            "2023-04-21 07:18:32,096 epoch 62 - iter 28/79 - loss 9.18142313 - time (sec): 6.33 - samples/sec: 5410.42 - lr: 0.000195\n",
            "2023-04-21 07:18:33,610 epoch 62 - iter 35/79 - loss 9.74319113 - time (sec): 7.85 - samples/sec: 5478.05 - lr: 0.000195\n",
            "2023-04-21 07:18:35,291 epoch 62 - iter 42/79 - loss 9.74997796 - time (sec): 9.53 - samples/sec: 5359.09 - lr: 0.000195\n",
            "2023-04-21 07:18:37,256 epoch 62 - iter 49/79 - loss 9.51392172 - time (sec): 11.49 - samples/sec: 5264.75 - lr: 0.000195\n",
            "2023-04-21 07:18:38,830 epoch 62 - iter 56/79 - loss 9.44774336 - time (sec): 13.07 - samples/sec: 5331.66 - lr: 0.000195\n",
            "2023-04-21 07:18:40,335 epoch 62 - iter 63/79 - loss 9.37298648 - time (sec): 14.57 - samples/sec: 5312.88 - lr: 0.000195\n",
            "2023-04-21 07:18:41,723 epoch 62 - iter 70/79 - loss 9.62134640 - time (sec): 15.96 - samples/sec: 5348.82 - lr: 0.000195\n",
            "2023-04-21 07:18:43,184 epoch 62 - iter 77/79 - loss 9.49796405 - time (sec): 17.42 - samples/sec: 5325.55 - lr: 0.000195\n",
            "2023-04-21 07:18:43,674 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:18:43,676 EPOCH 62 done: loss 9.4788 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:03,301 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:06,038 TRAIN : loss 3.661410331726074 - f1-score (weighted avg)  0.7493\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:08,622 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:19:08,694 DEV : loss 26.706087112426758 - f1-score (weighted avg)  0.6245\n",
            "2023-04-21 07:19:08,705 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:19:08,709 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:10,507 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:19:11,902 epoch 63 - iter 7/79 - loss 8.57583582 - time (sec): 1.39 - samples/sec: 5284.34 - lr: 0.000195\n",
            "2023-04-21 07:19:13,370 epoch 63 - iter 14/79 - loss 8.30893937 - time (sec): 2.86 - samples/sec: 5545.84 - lr: 0.000195\n",
            "2023-04-21 07:19:14,756 epoch 63 - iter 21/79 - loss 8.44633869 - time (sec): 4.25 - samples/sec: 5893.77 - lr: 0.000195\n",
            "2023-04-21 07:19:16,386 epoch 63 - iter 28/79 - loss 8.75290290 - time (sec): 5.88 - samples/sec: 5817.88 - lr: 0.000195\n",
            "2023-04-21 07:19:18,213 epoch 63 - iter 35/79 - loss 8.49629669 - time (sec): 7.70 - samples/sec: 5511.68 - lr: 0.000195\n",
            "2023-04-21 07:19:19,721 epoch 63 - iter 42/79 - loss 8.43051049 - time (sec): 9.21 - samples/sec: 5587.12 - lr: 0.000195\n",
            "2023-04-21 07:19:21,420 epoch 63 - iter 49/79 - loss 8.30070582 - time (sec): 10.91 - samples/sec: 5495.81 - lr: 0.000195\n",
            "2023-04-21 07:19:22,961 epoch 63 - iter 56/79 - loss 8.19625143 - time (sec): 12.45 - samples/sec: 5480.84 - lr: 0.000195\n",
            "2023-04-21 07:19:24,395 epoch 63 - iter 63/79 - loss 8.24579513 - time (sec): 13.89 - samples/sec: 5497.58 - lr: 0.000195\n",
            "2023-04-21 07:19:26,126 epoch 63 - iter 70/79 - loss 8.44101400 - time (sec): 15.62 - samples/sec: 5427.96 - lr: 0.000195\n",
            "2023-04-21 07:19:27,617 epoch 63 - iter 77/79 - loss 8.45261565 - time (sec): 17.11 - samples/sec: 5416.98 - lr: 0.000195\n",
            "2023-04-21 07:19:28,000 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:19:28,002 EPOCH 63 done: loss 8.4475 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:48,835 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:19:51,590 TRAIN : loss 3.564180374145508 - f1-score (weighted avg)  0.744\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:54,186 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:19:54,254 DEV : loss 26.501018524169922 - f1-score (weighted avg)  0.6077\n",
            "2023-04-21 07:19:54,264 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:19:54,268 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:19:56,056 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:19:57,402 epoch 64 - iter 7/79 - loss 7.41004805 - time (sec): 1.34 - samples/sec: 6583.96 - lr: 0.000195\n",
            "2023-04-21 07:19:58,671 epoch 64 - iter 14/79 - loss 8.25290374 - time (sec): 2.61 - samples/sec: 5847.74 - lr: 0.000195\n",
            "2023-04-21 07:19:59,968 epoch 64 - iter 21/79 - loss 8.45271096 - time (sec): 3.91 - samples/sec: 5858.22 - lr: 0.000195\n",
            "2023-04-21 07:20:01,529 epoch 64 - iter 28/79 - loss 9.08097631 - time (sec): 5.47 - samples/sec: 5865.16 - lr: 0.000195\n",
            "2023-04-21 07:20:03,082 epoch 64 - iter 35/79 - loss 9.17118963 - time (sec): 7.02 - samples/sec: 5777.34 - lr: 0.000195\n",
            "2023-04-21 07:20:04,597 epoch 64 - iter 42/79 - loss 8.91260592 - time (sec): 8.54 - samples/sec: 5790.97 - lr: 0.000195\n",
            "2023-04-21 07:20:06,437 epoch 64 - iter 49/79 - loss 8.50103947 - time (sec): 10.38 - samples/sec: 5538.24 - lr: 0.000195\n",
            "2023-04-21 07:20:08,058 epoch 64 - iter 56/79 - loss 8.54017433 - time (sec): 12.00 - samples/sec: 5525.96 - lr: 0.000195\n",
            "2023-04-21 07:20:09,885 epoch 64 - iter 63/79 - loss 8.69619807 - time (sec): 13.83 - samples/sec: 5412.46 - lr: 0.000195\n",
            "2023-04-21 07:20:11,206 epoch 64 - iter 70/79 - loss 8.95159105 - time (sec): 15.15 - samples/sec: 5458.18 - lr: 0.000195\n",
            "2023-04-21 07:20:12,962 epoch 64 - iter 77/79 - loss 9.03717868 - time (sec): 16.90 - samples/sec: 5422.07 - lr: 0.000195\n",
            "2023-04-21 07:20:13,450 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:20:13,451 EPOCH 64 done: loss 9.0131 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:20:33,185 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:20:37,243 TRAIN : loss 3.504018783569336 - f1-score (weighted avg)  0.7643\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:20:39,835 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:20:39,903 DEV : loss 26.4100399017334 - f1-score (weighted avg)  0.638\n",
            "2023-04-21 07:20:39,913 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:20:39,916 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:20:41,694 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:20:43,083 epoch 65 - iter 7/79 - loss 9.41460112 - time (sec): 1.39 - samples/sec: 5662.91 - lr: 0.000195\n",
            "2023-04-21 07:20:44,526 epoch 65 - iter 14/79 - loss 8.92937959 - time (sec): 2.83 - samples/sec: 5444.92 - lr: 0.000195\n",
            "2023-04-21 07:20:46,446 epoch 65 - iter 21/79 - loss 8.91377322 - time (sec): 4.75 - samples/sec: 5340.71 - lr: 0.000195\n",
            "2023-04-21 07:20:47,937 epoch 65 - iter 28/79 - loss 9.22907854 - time (sec): 6.24 - samples/sec: 5418.00 - lr: 0.000195\n",
            "2023-04-21 07:20:49,407 epoch 65 - iter 35/79 - loss 8.79915288 - time (sec): 7.71 - samples/sec: 5520.16 - lr: 0.000195\n",
            "2023-04-21 07:20:50,912 epoch 65 - iter 42/79 - loss 8.72985477 - time (sec): 9.21 - samples/sec: 5611.82 - lr: 0.000195\n",
            "2023-04-21 07:20:52,828 epoch 65 - iter 49/79 - loss 8.95631409 - time (sec): 11.13 - samples/sec: 5493.04 - lr: 0.000195\n",
            "2023-04-21 07:20:54,161 epoch 65 - iter 56/79 - loss 8.96089730 - time (sec): 12.46 - samples/sec: 5489.51 - lr: 0.000195\n",
            "2023-04-21 07:20:56,006 epoch 65 - iter 63/79 - loss 8.68052598 - time (sec): 14.31 - samples/sec: 5382.11 - lr: 0.000195\n",
            "2023-04-21 07:20:57,473 epoch 65 - iter 70/79 - loss 8.89253531 - time (sec): 15.78 - samples/sec: 5381.43 - lr: 0.000195\n",
            "2023-04-21 07:20:58,885 epoch 65 - iter 77/79 - loss 8.72080127 - time (sec): 17.19 - samples/sec: 5383.81 - lr: 0.000195\n",
            "2023-04-21 07:20:59,351 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:20:59,352 EPOCH 65 done: loss 8.6762 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.02it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:21:19,120 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:21:21,855 TRAIN : loss 3.428443670272827 - f1-score (weighted avg)  0.7636\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:21:24,447 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:21:24,517 DEV : loss 26.208669662475586 - f1-score (weighted avg)  0.6374\n",
            "2023-04-21 07:21:24,526 BAD EPOCHS (no improvement): 0\n",
            "2023-04-21 07:21:24,531 saving best model\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:21:26,305 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:21:28,319 epoch 66 - iter 7/79 - loss 8.70298052 - time (sec): 2.00 - samples/sec: 4542.16 - lr: 0.000195\n",
            "2023-04-21 07:21:29,689 epoch 66 - iter 14/79 - loss 9.63616656 - time (sec): 3.37 - samples/sec: 4876.54 - lr: 0.000195\n",
            "2023-04-21 07:21:31,033 epoch 66 - iter 21/79 - loss 9.90015971 - time (sec): 4.71 - samples/sec: 5179.99 - lr: 0.000195\n",
            "2023-04-21 07:21:32,475 epoch 66 - iter 28/79 - loss 9.16183747 - time (sec): 6.16 - samples/sec: 5351.03 - lr: 0.000195\n",
            "2023-04-21 07:21:34,106 epoch 66 - iter 35/79 - loss 8.81930622 - time (sec): 7.79 - samples/sec: 5381.27 - lr: 0.000195\n",
            "2023-04-21 07:21:35,665 epoch 66 - iter 42/79 - loss 8.68312056 - time (sec): 9.35 - samples/sec: 5376.45 - lr: 0.000195\n",
            "2023-04-21 07:21:37,474 epoch 66 - iter 49/79 - loss 8.56159559 - time (sec): 11.15 - samples/sec: 5253.61 - lr: 0.000195\n",
            "2023-04-21 07:21:39,249 epoch 66 - iter 56/79 - loss 8.74805387 - time (sec): 12.93 - samples/sec: 5201.09 - lr: 0.000195\n",
            "2023-04-21 07:21:40,643 epoch 66 - iter 63/79 - loss 8.66054242 - time (sec): 14.32 - samples/sec: 5229.86 - lr: 0.000195\n",
            "2023-04-21 07:21:42,343 epoch 66 - iter 70/79 - loss 8.62648763 - time (sec): 16.02 - samples/sec: 5196.73 - lr: 0.000195\n",
            "2023-04-21 07:21:43,920 epoch 66 - iter 77/79 - loss 8.75406486 - time (sec): 17.60 - samples/sec: 5241.44 - lr: 0.000195\n",
            "2023-04-21 07:21:44,340 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:21:44,342 EPOCH 66 done: loss 8.6927 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:05,196 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:22:07,941 TRAIN : loss 3.5574731826782227 - f1-score (weighted avg)  0.7471\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:10,529 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:22:10,599 DEV : loss 26.388629913330078 - f1-score (weighted avg)  0.6203\n",
            "2023-04-21 07:22:10,609 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:22:10,613 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:12,242 epoch 67 - iter 7/79 - loss 9.41329705 - time (sec): 1.63 - samples/sec: 5528.96 - lr: 0.000195\n",
            "2023-04-21 07:22:14,047 epoch 67 - iter 14/79 - loss 9.92389228 - time (sec): 3.43 - samples/sec: 5339.05 - lr: 0.000195\n",
            "2023-04-21 07:22:15,415 epoch 67 - iter 21/79 - loss 10.18052977 - time (sec): 4.80 - samples/sec: 5720.39 - lr: 0.000195\n",
            "2023-04-21 07:22:16,894 epoch 67 - iter 28/79 - loss 10.18715882 - time (sec): 6.28 - samples/sec: 5600.84 - lr: 0.000195\n",
            "2023-04-21 07:22:18,270 epoch 67 - iter 35/79 - loss 9.98129799 - time (sec): 7.66 - samples/sec: 5681.37 - lr: 0.000195\n",
            "2023-04-21 07:22:19,683 epoch 67 - iter 42/79 - loss 9.90869656 - time (sec): 9.07 - samples/sec: 5658.93 - lr: 0.000195\n",
            "2023-04-21 07:22:21,593 epoch 67 - iter 49/79 - loss 9.63616757 - time (sec): 10.98 - samples/sec: 5516.61 - lr: 0.000195\n",
            "2023-04-21 07:22:23,235 epoch 67 - iter 56/79 - loss 9.42463049 - time (sec): 12.62 - samples/sec: 5462.48 - lr: 0.000195\n",
            "2023-04-21 07:22:24,515 epoch 67 - iter 63/79 - loss 9.49216468 - time (sec): 13.90 - samples/sec: 5517.00 - lr: 0.000195\n",
            "2023-04-21 07:22:25,970 epoch 67 - iter 70/79 - loss 9.39192368 - time (sec): 15.36 - samples/sec: 5529.90 - lr: 0.000195\n",
            "2023-04-21 07:22:27,306 epoch 67 - iter 77/79 - loss 9.14834002 - time (sec): 16.69 - samples/sec: 5536.28 - lr: 0.000195\n",
            "2023-04-21 07:22:27,757 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:22:27,759 EPOCH 67 done: loss 9.1157 - lr 0.000195\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:47,354 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:50,085 TRAIN : loss 3.512056589126587 - f1-score (weighted avg)  0.7576\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.90it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:52,729 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:22:52,800 DEV : loss 26.325578689575195 - f1-score (weighted avg)  0.6325\n",
            "2023-04-21 07:22:52,809 Epoch    67: reducing learning rate of group 0 to 9.7656e-05.\n",
            "2023-04-21 07:22:52,811 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:22:52,815 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:22:54,231 epoch 68 - iter 7/79 - loss 6.61882391 - time (sec): 1.42 - samples/sec: 6033.71 - lr: 0.000098\n",
            "2023-04-21 07:22:55,768 epoch 68 - iter 14/79 - loss 7.17684682 - time (sec): 2.95 - samples/sec: 5354.72 - lr: 0.000098\n",
            "2023-04-21 07:22:57,446 epoch 68 - iter 21/79 - loss 7.20728664 - time (sec): 4.63 - samples/sec: 5278.70 - lr: 0.000098\n",
            "2023-04-21 07:22:58,872 epoch 68 - iter 28/79 - loss 7.47949631 - time (sec): 6.06 - samples/sec: 5353.29 - lr: 0.000098\n",
            "2023-04-21 07:23:00,187 epoch 68 - iter 35/79 - loss 7.40419947 - time (sec): 7.37 - samples/sec: 5445.88 - lr: 0.000098\n",
            "2023-04-21 07:23:01,599 epoch 68 - iter 42/79 - loss 7.53115433 - time (sec): 8.78 - samples/sec: 5504.07 - lr: 0.000098\n",
            "2023-04-21 07:23:02,954 epoch 68 - iter 49/79 - loss 7.68684243 - time (sec): 10.14 - samples/sec: 5587.37 - lr: 0.000098\n",
            "2023-04-21 07:23:04,892 epoch 68 - iter 56/79 - loss 7.97351868 - time (sec): 12.08 - samples/sec: 5423.70 - lr: 0.000098\n",
            "2023-04-21 07:23:06,590 epoch 68 - iter 63/79 - loss 8.46921228 - time (sec): 13.77 - samples/sec: 5428.01 - lr: 0.000098\n",
            "2023-04-21 07:23:08,035 epoch 68 - iter 70/79 - loss 8.81066333 - time (sec): 15.22 - samples/sec: 5484.83 - lr: 0.000098\n",
            "2023-04-21 07:23:09,677 epoch 68 - iter 77/79 - loss 8.64451635 - time (sec): 16.86 - samples/sec: 5503.43 - lr: 0.000098\n",
            "2023-04-21 07:23:10,021 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:23:10,022 EPOCH 68 done: loss 8.6236 - lr 0.000098\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.84it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:23:30,744 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:23:33,480 TRAIN : loss 3.5724213123321533 - f1-score (weighted avg)  0.7535\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:23:36,088 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:23:36,158 DEV : loss 26.439882278442383 - f1-score (weighted avg)  0.6226\n",
            "2023-04-21 07:23:36,168 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:23:36,172 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:23:37,640 epoch 69 - iter 7/79 - loss 9.72125447 - time (sec): 1.47 - samples/sec: 5345.51 - lr: 0.000098\n",
            "2023-04-21 07:23:39,948 epoch 69 - iter 14/79 - loss 8.66066779 - time (sec): 3.77 - samples/sec: 4883.81 - lr: 0.000098\n",
            "2023-04-21 07:23:41,360 epoch 69 - iter 21/79 - loss 8.83701388 - time (sec): 5.19 - samples/sec: 4988.14 - lr: 0.000098\n",
            "2023-04-21 07:23:42,726 epoch 69 - iter 28/79 - loss 9.31338688 - time (sec): 6.55 - samples/sec: 5173.35 - lr: 0.000098\n",
            "2023-04-21 07:23:44,072 epoch 69 - iter 35/79 - loss 9.55911624 - time (sec): 7.90 - samples/sec: 5282.97 - lr: 0.000098\n",
            "2023-04-21 07:23:45,710 epoch 69 - iter 42/79 - loss 9.34420405 - time (sec): 9.54 - samples/sec: 5261.20 - lr: 0.000098\n",
            "2023-04-21 07:23:46,989 epoch 69 - iter 49/79 - loss 9.37360451 - time (sec): 10.82 - samples/sec: 5350.35 - lr: 0.000098\n",
            "2023-04-21 07:23:48,378 epoch 69 - iter 56/79 - loss 9.00578553 - time (sec): 12.20 - samples/sec: 5422.57 - lr: 0.000098\n",
            "2023-04-21 07:23:50,043 epoch 69 - iter 63/79 - loss 8.93022105 - time (sec): 13.87 - samples/sec: 5409.75 - lr: 0.000098\n",
            "2023-04-21 07:23:51,481 epoch 69 - iter 70/79 - loss 8.79032088 - time (sec): 15.31 - samples/sec: 5430.88 - lr: 0.000098\n",
            "2023-04-21 07:23:53,058 epoch 69 - iter 77/79 - loss 8.88790371 - time (sec): 16.88 - samples/sec: 5464.87 - lr: 0.000098\n",
            "2023-04-21 07:23:53,500 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:23:53,502 EPOCH 69 done: loss 8.9025 - lr 0.000098\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:24:14,280 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:24:17,021 TRAIN : loss 3.500731945037842 - f1-score (weighted avg)  0.7583\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.90it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:24:19,665 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:24:19,734 DEV : loss 26.32992172241211 - f1-score (weighted avg)  0.6241\n",
            "2023-04-21 07:24:19,743 Epoch    69: reducing learning rate of group 0 to 4.8828e-05.\n",
            "2023-04-21 07:24:19,745 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:24:19,748 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:24:21,538 epoch 70 - iter 7/79 - loss 8.76418272 - time (sec): 1.79 - samples/sec: 5314.83 - lr: 0.000049\n",
            "2023-04-21 07:24:23,412 epoch 70 - iter 14/79 - loss 8.08699253 - time (sec): 3.66 - samples/sec: 5153.20 - lr: 0.000049\n",
            "2023-04-21 07:24:24,527 epoch 70 - iter 21/79 - loss 8.07019182 - time (sec): 4.78 - samples/sec: 5353.34 - lr: 0.000049\n",
            "2023-04-21 07:24:26,182 epoch 70 - iter 28/79 - loss 7.90501930 - time (sec): 6.43 - samples/sec: 5155.93 - lr: 0.000049\n",
            "2023-04-21 07:24:27,736 epoch 70 - iter 35/79 - loss 8.12956271 - time (sec): 7.99 - samples/sec: 5180.33 - lr: 0.000049\n",
            "2023-04-21 07:24:29,308 epoch 70 - iter 42/79 - loss 8.44744049 - time (sec): 9.56 - samples/sec: 5195.76 - lr: 0.000049\n",
            "2023-04-21 07:24:30,548 epoch 70 - iter 49/79 - loss 8.45994314 - time (sec): 10.80 - samples/sec: 5350.43 - lr: 0.000049\n",
            "2023-04-21 07:24:32,352 epoch 70 - iter 56/79 - loss 8.64379910 - time (sec): 12.60 - samples/sec: 5212.76 - lr: 0.000049\n",
            "2023-04-21 07:24:33,849 epoch 70 - iter 63/79 - loss 8.84310383 - time (sec): 14.10 - samples/sec: 5330.43 - lr: 0.000049\n",
            "2023-04-21 07:24:35,588 epoch 70 - iter 70/79 - loss 8.85271163 - time (sec): 15.84 - samples/sec: 5286.76 - lr: 0.000049\n",
            "2023-04-21 07:24:36,994 epoch 70 - iter 77/79 - loss 8.73369406 - time (sec): 17.24 - samples/sec: 5361.47 - lr: 0.000049\n",
            "2023-04-21 07:24:37,376 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:24:37,378 EPOCH 70 done: loss 8.7002 - lr 0.000049\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:24:57,004 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:01,035 TRAIN : loss 3.498046398162842 - f1-score (weighted avg)  0.7614\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:03,618 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:25:03,687 DEV : loss 26.3389835357666 - f1-score (weighted avg)  0.6279\n",
            "2023-04-21 07:25:03,696 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:25:03,700 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:05,339 epoch 71 - iter 7/79 - loss 6.24757134 - time (sec): 1.64 - samples/sec: 5029.74 - lr: 0.000049\n",
            "2023-04-21 07:25:06,632 epoch 71 - iter 14/79 - loss 6.65774382 - time (sec): 2.93 - samples/sec: 5326.62 - lr: 0.000049\n",
            "2023-04-21 07:25:08,142 epoch 71 - iter 21/79 - loss 7.33765936 - time (sec): 4.44 - samples/sec: 5405.78 - lr: 0.000049\n",
            "2023-04-21 07:25:09,825 epoch 71 - iter 28/79 - loss 7.92478067 - time (sec): 6.12 - samples/sec: 5398.91 - lr: 0.000049\n",
            "2023-04-21 07:25:11,689 epoch 71 - iter 35/79 - loss 7.94405626 - time (sec): 7.99 - samples/sec: 5263.42 - lr: 0.000049\n",
            "2023-04-21 07:25:12,974 epoch 71 - iter 42/79 - loss 7.78179169 - time (sec): 9.27 - samples/sec: 5412.85 - lr: 0.000049\n",
            "2023-04-21 07:25:14,465 epoch 71 - iter 49/79 - loss 8.10345583 - time (sec): 10.76 - samples/sec: 5457.09 - lr: 0.000049\n",
            "2023-04-21 07:25:15,904 epoch 71 - iter 56/79 - loss 8.15764809 - time (sec): 12.20 - samples/sec: 5449.62 - lr: 0.000049\n",
            "2023-04-21 07:25:17,605 epoch 71 - iter 63/79 - loss 8.62479117 - time (sec): 13.90 - samples/sec: 5381.72 - lr: 0.000049\n",
            "2023-04-21 07:25:19,185 epoch 71 - iter 70/79 - loss 8.40641222 - time (sec): 15.48 - samples/sec: 5412.25 - lr: 0.000049\n",
            "2023-04-21 07:25:20,513 epoch 71 - iter 77/79 - loss 8.31346526 - time (sec): 16.81 - samples/sec: 5469.29 - lr: 0.000049\n",
            "2023-04-21 07:25:20,894 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:25:20,896 EPOCH 71 done: loss 8.2449 - lr 0.000049\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.02it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:40,680 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:43,461 TRAIN : loss 3.5191988945007324 - f1-score (weighted avg)  0.7648\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:46,067 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:25:46,137 DEV : loss 26.41510772705078 - f1-score (weighted avg)  0.6381\n",
            "2023-04-21 07:25:46,147 Epoch    71: reducing learning rate of group 0 to 2.4414e-05.\n",
            "2023-04-21 07:25:46,149 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:25:46,153 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:25:47,823 epoch 72 - iter 7/79 - loss 8.14917314 - time (sec): 1.67 - samples/sec: 5327.62 - lr: 0.000024\n",
            "2023-04-21 07:25:49,247 epoch 72 - iter 14/79 - loss 8.22436148 - time (sec): 3.09 - samples/sec: 5381.49 - lr: 0.000024\n",
            "2023-04-21 07:25:51,035 epoch 72 - iter 21/79 - loss 8.73843422 - time (sec): 4.88 - samples/sec: 5321.84 - lr: 0.000024\n",
            "2023-04-21 07:25:52,935 epoch 72 - iter 28/79 - loss 9.75554040 - time (sec): 6.78 - samples/sec: 5137.92 - lr: 0.000024\n",
            "2023-04-21 07:25:54,412 epoch 72 - iter 35/79 - loss 9.63634803 - time (sec): 8.26 - samples/sec: 5204.20 - lr: 0.000024\n",
            "2023-04-21 07:25:55,805 epoch 72 - iter 42/79 - loss 9.34743721 - time (sec): 9.65 - samples/sec: 5268.73 - lr: 0.000024\n",
            "2023-04-21 07:25:57,016 epoch 72 - iter 49/79 - loss 9.25040377 - time (sec): 10.86 - samples/sec: 5387.16 - lr: 0.000024\n",
            "2023-04-21 07:25:58,456 epoch 72 - iter 56/79 - loss 9.24554956 - time (sec): 12.30 - samples/sec: 5399.74 - lr: 0.000024\n",
            "2023-04-21 07:26:00,132 epoch 72 - iter 63/79 - loss 9.10589318 - time (sec): 13.98 - samples/sec: 5433.82 - lr: 0.000024\n",
            "2023-04-21 07:26:01,583 epoch 72 - iter 70/79 - loss 8.92348041 - time (sec): 15.43 - samples/sec: 5492.61 - lr: 0.000024\n",
            "2023-04-21 07:26:02,987 epoch 72 - iter 77/79 - loss 8.82531026 - time (sec): 16.83 - samples/sec: 5495.48 - lr: 0.000024\n",
            "2023-04-21 07:26:03,440 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:26:03,441 EPOCH 72 done: loss 8.7289 - lr 0.000024\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:26:24,309 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:26:27,032 TRAIN : loss 3.5502848625183105 - f1-score (weighted avg)  0.7586\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:26:29,615 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:26:29,684 DEV : loss 26.429452896118164 - f1-score (weighted avg)  0.6263\n",
            "2023-04-21 07:26:29,695 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:26:29,699 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:26:31,020 epoch 73 - iter 7/79 - loss 7.38952719 - time (sec): 1.32 - samples/sec: 6299.03 - lr: 0.000024\n",
            "2023-04-21 07:26:32,522 epoch 73 - iter 14/79 - loss 8.62343537 - time (sec): 2.82 - samples/sec: 6229.32 - lr: 0.000024\n",
            "2023-04-21 07:26:34,033 epoch 73 - iter 21/79 - loss 8.26517180 - time (sec): 4.33 - samples/sec: 5602.16 - lr: 0.000024\n",
            "2023-04-21 07:26:36,038 epoch 73 - iter 28/79 - loss 8.40105161 - time (sec): 6.34 - samples/sec: 5371.79 - lr: 0.000024\n",
            "2023-04-21 07:26:37,606 epoch 73 - iter 35/79 - loss 8.50260998 - time (sec): 7.91 - samples/sec: 5325.32 - lr: 0.000024\n",
            "2023-04-21 07:26:39,020 epoch 73 - iter 42/79 - loss 8.35402605 - time (sec): 9.32 - samples/sec: 5516.23 - lr: 0.000024\n",
            "2023-04-21 07:26:40,461 epoch 73 - iter 49/79 - loss 8.54363961 - time (sec): 10.76 - samples/sec: 5540.61 - lr: 0.000024\n",
            "2023-04-21 07:26:41,959 epoch 73 - iter 56/79 - loss 8.49830665 - time (sec): 12.26 - samples/sec: 5558.09 - lr: 0.000024\n",
            "2023-04-21 07:26:43,675 epoch 73 - iter 63/79 - loss 8.40564646 - time (sec): 13.97 - samples/sec: 5457.62 - lr: 0.000024\n",
            "2023-04-21 07:26:45,160 epoch 73 - iter 70/79 - loss 8.40171956 - time (sec): 15.46 - samples/sec: 5429.91 - lr: 0.000024\n",
            "2023-04-21 07:26:46,957 epoch 73 - iter 77/79 - loss 8.64656182 - time (sec): 17.26 - samples/sec: 5392.47 - lr: 0.000024\n",
            "2023-04-21 07:26:47,344 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:26:47,346 EPOCH 73 done: loss 8.6298 - lr 0.000024\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:07,041 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:09,784 TRAIN : loss 3.54514741897583 - f1-score (weighted avg)  0.7592\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:13,465 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:27:13,534 DEV : loss 26.41497039794922 - f1-score (weighted avg)  0.6255\n",
            "2023-04-21 07:27:13,544 Epoch    73: reducing learning rate of group 0 to 1.2207e-05.\n",
            "2023-04-21 07:27:13,546 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:27:13,549 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:27:14,890 epoch 74 - iter 7/79 - loss 6.35404606 - time (sec): 1.34 - samples/sec: 5807.04 - lr: 0.000012\n",
            "2023-04-21 07:27:16,616 epoch 74 - iter 14/79 - loss 7.49959800 - time (sec): 3.06 - samples/sec: 5580.09 - lr: 0.000012\n",
            "2023-04-21 07:27:18,040 epoch 74 - iter 21/79 - loss 7.68308571 - time (sec): 4.49 - samples/sec: 5569.68 - lr: 0.000012\n",
            "2023-04-21 07:27:19,634 epoch 74 - iter 28/79 - loss 8.26785878 - time (sec): 6.08 - samples/sec: 5367.59 - lr: 0.000012\n",
            "2023-04-21 07:27:21,237 epoch 74 - iter 35/79 - loss 8.34447598 - time (sec): 7.69 - samples/sec: 5443.37 - lr: 0.000012\n",
            "2023-04-21 07:27:22,869 epoch 74 - iter 42/79 - loss 8.48631553 - time (sec): 9.32 - samples/sec: 5425.01 - lr: 0.000012\n",
            "2023-04-21 07:27:24,571 epoch 74 - iter 49/79 - loss 9.00165161 - time (sec): 11.02 - samples/sec: 5384.18 - lr: 0.000012\n",
            "2023-04-21 07:27:25,803 epoch 74 - iter 56/79 - loss 8.92895843 - time (sec): 12.25 - samples/sec: 5487.41 - lr: 0.000012\n",
            "2023-04-21 07:27:27,159 epoch 74 - iter 63/79 - loss 8.74201701 - time (sec): 13.61 - samples/sec: 5498.84 - lr: 0.000012\n",
            "2023-04-21 07:27:29,046 epoch 74 - iter 70/79 - loss 8.71369654 - time (sec): 15.50 - samples/sec: 5408.27 - lr: 0.000012\n",
            "2023-04-21 07:27:30,546 epoch 74 - iter 77/79 - loss 8.55549862 - time (sec): 16.99 - samples/sec: 5397.13 - lr: 0.000012\n",
            "2023-04-21 07:27:31,064 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:27:31,066 EPOCH 74 done: loss 8.5581 - lr 0.000012\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:50,676 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:53,398 TRAIN : loss 3.555840253829956 - f1-score (weighted avg)  0.7585\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:55,986 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:27:56,055 DEV : loss 26.42544174194336 - f1-score (weighted avg)  0.6242\n",
            "2023-04-21 07:27:56,064 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:27:56,068 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:27:57,549 epoch 75 - iter 7/79 - loss 8.42375417 - time (sec): 1.48 - samples/sec: 5871.37 - lr: 0.000012\n",
            "2023-04-21 07:27:59,077 epoch 75 - iter 14/79 - loss 8.46524721 - time (sec): 3.01 - samples/sec: 5814.79 - lr: 0.000012\n",
            "2023-04-21 07:28:01,188 epoch 75 - iter 21/79 - loss 7.83750481 - time (sec): 5.12 - samples/sec: 5080.46 - lr: 0.000012\n",
            "2023-04-21 07:28:02,622 epoch 75 - iter 28/79 - loss 8.11130873 - time (sec): 6.55 - samples/sec: 5179.88 - lr: 0.000012\n",
            "2023-04-21 07:28:04,287 epoch 75 - iter 35/79 - loss 7.91299184 - time (sec): 8.22 - samples/sec: 5186.08 - lr: 0.000012\n",
            "2023-04-21 07:28:05,985 epoch 75 - iter 42/79 - loss 8.46051224 - time (sec): 9.92 - samples/sec: 5233.88 - lr: 0.000012\n",
            "2023-04-21 07:28:07,314 epoch 75 - iter 49/79 - loss 8.67894140 - time (sec): 11.24 - samples/sec: 5331.05 - lr: 0.000012\n",
            "2023-04-21 07:28:08,809 epoch 75 - iter 56/79 - loss 8.61247912 - time (sec): 12.74 - samples/sec: 5404.31 - lr: 0.000012\n",
            "2023-04-21 07:28:10,305 epoch 75 - iter 63/79 - loss 8.67917959 - time (sec): 14.24 - samples/sec: 5391.43 - lr: 0.000012\n",
            "2023-04-21 07:28:11,531 epoch 75 - iter 70/79 - loss 8.87212952 - time (sec): 15.46 - samples/sec: 5460.76 - lr: 0.000012\n",
            "2023-04-21 07:28:12,999 epoch 75 - iter 77/79 - loss 8.72327414 - time (sec): 16.93 - samples/sec: 5454.40 - lr: 0.000012\n",
            "2023-04-21 07:28:13,467 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:28:13,469 EPOCH 75 done: loss 8.7573 - lr 0.000012\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:28:34,241 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:28:36,988 TRAIN : loss 3.5314102172851562 - f1-score (weighted avg)  0.7614\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:28:39,575 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:28:39,647 DEV : loss 26.391801834106445 - f1-score (weighted avg)  0.6308\n",
            "2023-04-21 07:28:39,656 Epoch    75: reducing learning rate of group 0 to 6.1035e-06.\n",
            "2023-04-21 07:28:39,658 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:28:39,661 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:28:41,307 epoch 76 - iter 7/79 - loss 11.13204604 - time (sec): 1.64 - samples/sec: 5652.41 - lr: 0.000006\n",
            "2023-04-21 07:28:42,468 epoch 76 - iter 14/79 - loss 10.64784351 - time (sec): 2.81 - samples/sec: 5715.57 - lr: 0.000006\n",
            "2023-04-21 07:28:43,756 epoch 76 - iter 21/79 - loss 9.76789469 - time (sec): 4.09 - samples/sec: 5809.20 - lr: 0.000006\n",
            "2023-04-21 07:28:45,141 epoch 76 - iter 28/79 - loss 9.34409688 - time (sec): 5.48 - samples/sec: 5739.33 - lr: 0.000006\n",
            "2023-04-21 07:28:46,832 epoch 76 - iter 35/79 - loss 8.87561026 - time (sec): 7.17 - samples/sec: 5714.30 - lr: 0.000006\n",
            "2023-04-21 07:28:48,346 epoch 76 - iter 42/79 - loss 9.10131592 - time (sec): 8.68 - samples/sec: 5819.65 - lr: 0.000006\n",
            "2023-04-21 07:28:49,817 epoch 76 - iter 49/79 - loss 8.93340509 - time (sec): 10.15 - samples/sec: 5856.18 - lr: 0.000006\n",
            "2023-04-21 07:28:51,301 epoch 76 - iter 56/79 - loss 8.88611527 - time (sec): 11.64 - samples/sec: 5908.65 - lr: 0.000006\n",
            "2023-04-21 07:28:52,376 epoch 76 - iter 63/79 - loss 8.83282092 - time (sec): 12.71 - samples/sec: 5896.90 - lr: 0.000006\n",
            "2023-04-21 07:28:54,267 epoch 76 - iter 70/79 - loss 8.82719845 - time (sec): 14.60 - samples/sec: 5690.25 - lr: 0.000006\n",
            "2023-04-21 07:28:56,406 epoch 76 - iter 77/79 - loss 8.86988697 - time (sec): 16.74 - samples/sec: 5506.10 - lr: 0.000006\n",
            "2023-04-21 07:28:56,888 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:28:56,890 EPOCH 76 done: loss 8.7834 - lr 0.000006\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.05it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:29:16,549 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:29:20,554 TRAIN : loss 3.5260775089263916 - f1-score (weighted avg)  0.7616\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.95it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:29:23,165 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:29:23,235 DEV : loss 26.384521484375 - f1-score (weighted avg)  0.6291\n",
            "2023-04-21 07:29:23,245 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:29:23,250 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:29:24,723 epoch 77 - iter 7/79 - loss 10.02191612 - time (sec): 1.47 - samples/sec: 5114.24 - lr: 0.000006\n",
            "2023-04-21 07:29:26,391 epoch 77 - iter 14/79 - loss 7.60410569 - time (sec): 3.14 - samples/sec: 5007.21 - lr: 0.000006\n",
            "2023-04-21 07:29:27,993 epoch 77 - iter 21/79 - loss 7.98075909 - time (sec): 4.74 - samples/sec: 5201.32 - lr: 0.000006\n",
            "2023-04-21 07:29:29,269 epoch 77 - iter 28/79 - loss 7.86563399 - time (sec): 6.02 - samples/sec: 5369.90 - lr: 0.000006\n",
            "2023-04-21 07:29:30,749 epoch 77 - iter 35/79 - loss 7.90707066 - time (sec): 7.50 - samples/sec: 5449.84 - lr: 0.000006\n",
            "2023-04-21 07:29:32,216 epoch 77 - iter 42/79 - loss 8.53620494 - time (sec): 8.96 - samples/sec: 5595.09 - lr: 0.000006\n",
            "2023-04-21 07:29:33,553 epoch 77 - iter 49/79 - loss 8.67317244 - time (sec): 10.30 - samples/sec: 5638.41 - lr: 0.000006\n",
            "2023-04-21 07:29:35,096 epoch 77 - iter 56/79 - loss 8.43803760 - time (sec): 11.84 - samples/sec: 5663.67 - lr: 0.000006\n",
            "2023-04-21 07:29:36,684 epoch 77 - iter 63/79 - loss 8.47244295 - time (sec): 13.43 - samples/sec: 5555.76 - lr: 0.000006\n",
            "2023-04-21 07:29:38,160 epoch 77 - iter 70/79 - loss 8.70546379 - time (sec): 14.91 - samples/sec: 5565.33 - lr: 0.000006\n",
            "2023-04-21 07:29:39,682 epoch 77 - iter 77/79 - loss 8.56662656 - time (sec): 16.43 - samples/sec: 5577.10 - lr: 0.000006\n",
            "2023-04-21 07:29:40,175 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:29:40,177 EPOCH 77 done: loss 8.5440 - lr 0.000006\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:29:59,782 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:02,519 TRAIN : loss 3.514404535293579 - f1-score (weighted avg)  0.7626\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:05,127 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:30:05,200 DEV : loss 26.367515563964844 - f1-score (weighted avg)  0.6308\n",
            "2023-04-21 07:30:05,211 Epoch    77: reducing learning rate of group 0 to 3.0518e-06.\n",
            "2023-04-21 07:30:05,213 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:30:05,217 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:06,579 epoch 78 - iter 7/79 - loss 6.99105535 - time (sec): 1.36 - samples/sec: 6118.40 - lr: 0.000003\n",
            "2023-04-21 07:30:08,355 epoch 78 - iter 14/79 - loss 7.62148967 - time (sec): 3.14 - samples/sec: 5665.20 - lr: 0.000003\n",
            "2023-04-21 07:30:09,814 epoch 78 - iter 21/79 - loss 8.02074211 - time (sec): 4.60 - samples/sec: 5764.62 - lr: 0.000003\n",
            "2023-04-21 07:30:11,236 epoch 78 - iter 28/79 - loss 8.08369263 - time (sec): 6.02 - samples/sec: 5684.81 - lr: 0.000003\n",
            "2023-04-21 07:30:12,742 epoch 78 - iter 35/79 - loss 7.95101816 - time (sec): 7.52 - samples/sec: 5648.87 - lr: 0.000003\n",
            "2023-04-21 07:30:14,613 epoch 78 - iter 42/79 - loss 8.06869998 - time (sec): 9.40 - samples/sec: 5571.92 - lr: 0.000003\n",
            "2023-04-21 07:30:16,095 epoch 78 - iter 49/79 - loss 7.88280406 - time (sec): 10.88 - samples/sec: 5445.85 - lr: 0.000003\n",
            "2023-04-21 07:30:18,048 epoch 78 - iter 56/79 - loss 8.13857138 - time (sec): 12.83 - samples/sec: 5328.96 - lr: 0.000003\n",
            "2023-04-21 07:30:19,441 epoch 78 - iter 63/79 - loss 8.29772305 - time (sec): 14.22 - samples/sec: 5339.78 - lr: 0.000003\n",
            "2023-04-21 07:30:21,023 epoch 78 - iter 70/79 - loss 8.34802372 - time (sec): 15.80 - samples/sec: 5353.78 - lr: 0.000003\n",
            "2023-04-21 07:30:22,393 epoch 78 - iter 77/79 - loss 8.34790945 - time (sec): 17.17 - samples/sec: 5375.41 - lr: 0.000003\n",
            "2023-04-21 07:30:22,722 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:30:22,724 EPOCH 78 done: loss 8.4120 - lr 0.000003\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:43,486 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:30:46,244 TRAIN : loss 3.5155229568481445 - f1-score (weighted avg)  0.7617\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.92it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:48,876 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:30:48,955 DEV : loss 26.366165161132812 - f1-score (weighted avg)  0.63\n",
            "2023-04-21 07:30:48,965 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:30:48,970 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:30:50,445 epoch 79 - iter 7/79 - loss 8.35957348 - time (sec): 1.47 - samples/sec: 5824.11 - lr: 0.000003\n",
            "2023-04-21 07:30:52,422 epoch 79 - iter 14/79 - loss 8.44501153 - time (sec): 3.45 - samples/sec: 5152.74 - lr: 0.000003\n",
            "2023-04-21 07:30:53,858 epoch 79 - iter 21/79 - loss 8.34990516 - time (sec): 4.89 - samples/sec: 5286.03 - lr: 0.000003\n",
            "2023-04-21 07:30:55,465 epoch 79 - iter 28/79 - loss 8.25048186 - time (sec): 6.49 - samples/sec: 5353.14 - lr: 0.000003\n",
            "2023-04-21 07:30:57,182 epoch 79 - iter 35/79 - loss 8.69070426 - time (sec): 8.21 - samples/sec: 5200.19 - lr: 0.000003\n",
            "2023-04-21 07:30:58,887 epoch 79 - iter 42/79 - loss 8.64401962 - time (sec): 9.92 - samples/sec: 5162.12 - lr: 0.000003\n",
            "2023-04-21 07:31:00,213 epoch 79 - iter 49/79 - loss 8.49276976 - time (sec): 11.24 - samples/sec: 5224.40 - lr: 0.000003\n",
            "2023-04-21 07:31:01,801 epoch 79 - iter 56/79 - loss 8.58139995 - time (sec): 12.83 - samples/sec: 5266.06 - lr: 0.000003\n",
            "2023-04-21 07:31:03,108 epoch 79 - iter 63/79 - loss 8.53417979 - time (sec): 14.14 - samples/sec: 5344.97 - lr: 0.000003\n",
            "2023-04-21 07:31:04,602 epoch 79 - iter 70/79 - loss 8.63706560 - time (sec): 15.63 - samples/sec: 5395.86 - lr: 0.000003\n",
            "2023-04-21 07:31:05,996 epoch 79 - iter 77/79 - loss 8.55118479 - time (sec): 17.02 - samples/sec: 5382.85 - lr: 0.000003\n",
            "2023-04-21 07:31:06,431 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:31:06,433 EPOCH 79 done: loss 8.6317 - lr 0.000003\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:31:26,019 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:31:30,047 TRAIN : loss 3.5233352184295654 - f1-score (weighted avg)  0.76\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:31:32,638 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:31:32,707 DEV : loss 26.374052047729492 - f1-score (weighted avg)  0.6243\n",
            "2023-04-21 07:31:32,717 Epoch    79: reducing learning rate of group 0 to 1.5259e-06.\n",
            "2023-04-21 07:31:32,718 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:31:32,722 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:31:34,264 epoch 80 - iter 7/79 - loss 6.38647052 - time (sec): 1.54 - samples/sec: 5661.42 - lr: 0.000002\n",
            "2023-04-21 07:31:35,709 epoch 80 - iter 14/79 - loss 7.60068915 - time (sec): 2.99 - samples/sec: 5710.81 - lr: 0.000002\n",
            "2023-04-21 07:31:37,429 epoch 80 - iter 21/79 - loss 8.00923808 - time (sec): 4.70 - samples/sec: 5481.51 - lr: 0.000002\n",
            "2023-04-21 07:31:38,966 epoch 80 - iter 28/79 - loss 8.92046251 - time (sec): 6.24 - samples/sec: 5486.51 - lr: 0.000002\n",
            "2023-04-21 07:31:40,291 epoch 80 - iter 35/79 - loss 8.73698568 - time (sec): 7.57 - samples/sec: 5601.15 - lr: 0.000002\n",
            "2023-04-21 07:31:41,872 epoch 80 - iter 42/79 - loss 8.52477376 - time (sec): 9.15 - samples/sec: 5592.42 - lr: 0.000002\n",
            "2023-04-21 07:31:43,210 epoch 80 - iter 49/79 - loss 8.41277079 - time (sec): 10.49 - samples/sec: 5555.63 - lr: 0.000002\n",
            "2023-04-21 07:31:45,078 epoch 80 - iter 56/79 - loss 8.31355280 - time (sec): 12.35 - samples/sec: 5473.18 - lr: 0.000002\n",
            "2023-04-21 07:31:46,464 epoch 80 - iter 63/79 - loss 8.31249013 - time (sec): 13.74 - samples/sec: 5529.26 - lr: 0.000002\n",
            "2023-04-21 07:31:48,384 epoch 80 - iter 70/79 - loss 8.28275352 - time (sec): 15.66 - samples/sec: 5348.66 - lr: 0.000002\n",
            "2023-04-21 07:31:49,903 epoch 80 - iter 77/79 - loss 8.41050486 - time (sec): 17.18 - samples/sec: 5375.13 - lr: 0.000002\n",
            "2023-04-21 07:31:50,223 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:31:50,225 EPOCH 80 done: loss 8.4770 - lr 0.000002\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:09,946 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:12,698 TRAIN : loss 3.5254602432250977 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:15,285 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:32:15,353 DEV : loss 26.376419067382812 - f1-score (weighted avg)  0.6243\n",
            "2023-04-21 07:32:15,364 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:32:15,367 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:16,881 epoch 81 - iter 7/79 - loss 6.61642269 - time (sec): 1.51 - samples/sec: 5784.86 - lr: 0.000002\n",
            "2023-04-21 07:32:18,439 epoch 81 - iter 14/79 - loss 8.28945598 - time (sec): 3.07 - samples/sec: 5206.65 - lr: 0.000002\n",
            "2023-04-21 07:32:20,098 epoch 81 - iter 21/79 - loss 7.84391518 - time (sec): 4.73 - samples/sec: 5000.92 - lr: 0.000002\n",
            "2023-04-21 07:32:21,807 epoch 81 - iter 28/79 - loss 8.20405507 - time (sec): 6.44 - samples/sec: 5085.96 - lr: 0.000002\n",
            "2023-04-21 07:32:23,243 epoch 81 - iter 35/79 - loss 8.48263636 - time (sec): 7.87 - samples/sec: 5224.93 - lr: 0.000002\n",
            "2023-04-21 07:32:24,587 epoch 81 - iter 42/79 - loss 8.85091715 - time (sec): 9.22 - samples/sec: 5299.10 - lr: 0.000002\n",
            "2023-04-21 07:32:26,503 epoch 81 - iter 49/79 - loss 8.77269823 - time (sec): 11.13 - samples/sec: 5201.39 - lr: 0.000002\n",
            "2023-04-21 07:32:28,014 epoch 81 - iter 56/79 - loss 8.67879600 - time (sec): 12.65 - samples/sec: 5275.94 - lr: 0.000002\n",
            "2023-04-21 07:32:29,323 epoch 81 - iter 63/79 - loss 8.61412196 - time (sec): 13.95 - samples/sec: 5329.60 - lr: 0.000002\n",
            "2023-04-21 07:32:31,004 epoch 81 - iter 70/79 - loss 8.48658680 - time (sec): 15.64 - samples/sec: 5320.22 - lr: 0.000002\n",
            "2023-04-21 07:32:32,388 epoch 81 - iter 77/79 - loss 8.51729055 - time (sec): 17.02 - samples/sec: 5392.74 - lr: 0.000002\n",
            "2023-04-21 07:32:32,934 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:32:32,936 EPOCH 81 done: loss 8.5810 - lr 0.000002\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.02it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:53,898 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:32:56,625 TRAIN : loss 3.5275614261627197 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.98it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:32:59,221 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:32:59,292 DEV : loss 26.377769470214844 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:32:59,302 Epoch    81: reducing learning rate of group 0 to 7.6294e-07.\n",
            "2023-04-21 07:32:59,303 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:32:59,307 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:33:00,683 epoch 82 - iter 7/79 - loss 6.69475395 - time (sec): 1.38 - samples/sec: 5482.89 - lr: 0.000001\n",
            "2023-04-21 07:33:02,203 epoch 82 - iter 14/79 - loss 7.25262593 - time (sec): 2.90 - samples/sec: 5787.45 - lr: 0.000001\n",
            "2023-04-21 07:33:04,318 epoch 82 - iter 21/79 - loss 6.85197661 - time (sec): 5.01 - samples/sec: 4940.79 - lr: 0.000001\n",
            "2023-04-21 07:33:05,939 epoch 82 - iter 28/79 - loss 7.28321613 - time (sec): 6.63 - samples/sec: 5072.11 - lr: 0.000001\n",
            "2023-04-21 07:33:07,634 epoch 82 - iter 35/79 - loss 7.92252189 - time (sec): 8.33 - samples/sec: 5129.26 - lr: 0.000001\n",
            "2023-04-21 07:33:09,378 epoch 82 - iter 42/79 - loss 8.11791650 - time (sec): 10.07 - samples/sec: 5108.20 - lr: 0.000001\n",
            "2023-04-21 07:33:10,681 epoch 82 - iter 49/79 - loss 8.29931714 - time (sec): 11.37 - samples/sec: 5166.75 - lr: 0.000001\n",
            "2023-04-21 07:33:12,007 epoch 82 - iter 56/79 - loss 8.26136720 - time (sec): 12.70 - samples/sec: 5275.17 - lr: 0.000001\n",
            "2023-04-21 07:33:13,262 epoch 82 - iter 63/79 - loss 8.26085033 - time (sec): 13.95 - samples/sec: 5365.44 - lr: 0.000001\n",
            "2023-04-21 07:33:14,944 epoch 82 - iter 70/79 - loss 8.21978501 - time (sec): 15.64 - samples/sec: 5406.62 - lr: 0.000001\n",
            "2023-04-21 07:33:16,385 epoch 82 - iter 77/79 - loss 8.39530782 - time (sec): 17.08 - samples/sec: 5420.78 - lr: 0.000001\n",
            "2023-04-21 07:33:16,776 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:33:16,777 EPOCH 82 done: loss 8.4209 - lr 0.000001\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:33:36,515 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:33:39,254 TRAIN : loss 3.527238130569458 - f1-score (weighted avg)  0.7596\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:03<00:00,  2.74it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:33:42,997 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:33:43,068 DEV : loss 26.377084732055664 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:33:43,077 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:33:43,082 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:33:44,594 epoch 83 - iter 7/79 - loss 8.65571836 - time (sec): 1.51 - samples/sec: 6031.67 - lr: 0.000001\n",
            "2023-04-21 07:33:46,196 epoch 83 - iter 14/79 - loss 8.14464814 - time (sec): 3.11 - samples/sec: 5647.61 - lr: 0.000001\n",
            "2023-04-21 07:33:47,752 epoch 83 - iter 21/79 - loss 8.78327844 - time (sec): 4.67 - samples/sec: 5755.46 - lr: 0.000001\n",
            "2023-04-21 07:33:49,085 epoch 83 - iter 28/79 - loss 8.59324469 - time (sec): 6.00 - samples/sec: 5739.83 - lr: 0.000001\n",
            "2023-04-21 07:33:50,649 epoch 83 - iter 35/79 - loss 8.42778897 - time (sec): 7.57 - samples/sec: 5574.92 - lr: 0.000001\n",
            "2023-04-21 07:33:52,238 epoch 83 - iter 42/79 - loss 8.71108905 - time (sec): 9.15 - samples/sec: 5569.75 - lr: 0.000001\n",
            "2023-04-21 07:33:53,614 epoch 83 - iter 49/79 - loss 8.54585687 - time (sec): 10.53 - samples/sec: 5530.79 - lr: 0.000001\n",
            "2023-04-21 07:33:55,182 epoch 83 - iter 56/79 - loss 8.40443613 - time (sec): 12.10 - samples/sec: 5512.20 - lr: 0.000001\n",
            "2023-04-21 07:33:56,693 epoch 83 - iter 63/79 - loss 8.58244095 - time (sec): 13.61 - samples/sec: 5506.71 - lr: 0.000001\n",
            "2023-04-21 07:33:58,784 epoch 83 - iter 70/79 - loss 8.66367494 - time (sec): 15.70 - samples/sec: 5355.35 - lr: 0.000001\n",
            "2023-04-21 07:34:00,635 epoch 83 - iter 77/79 - loss 8.50884383 - time (sec): 17.55 - samples/sec: 5270.60 - lr: 0.000001\n",
            "2023-04-21 07:34:01,047 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:34:01,049 EPOCH 83 done: loss 8.4283 - lr 0.000001\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.06it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:34:20,664 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:34:23,399 TRAIN : loss 3.5254218578338623 - f1-score (weighted avg)  0.76\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:34:26,002 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:34:26,071 DEV : loss 26.375362396240234 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:34:26,081 Epoch    83: reducing learning rate of group 0 to 3.8147e-07.\n",
            "2023-04-21 07:34:26,083 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:34:26,087 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:34:27,603 epoch 84 - iter 7/79 - loss 7.87967767 - time (sec): 1.51 - samples/sec: 5980.70 - lr: 0.000000\n",
            "2023-04-21 07:34:28,940 epoch 84 - iter 14/79 - loss 8.82717563 - time (sec): 2.85 - samples/sec: 6007.88 - lr: 0.000000\n",
            "2023-04-21 07:34:30,728 epoch 84 - iter 21/79 - loss 8.90357102 - time (sec): 4.64 - samples/sec: 5589.99 - lr: 0.000000\n",
            "2023-04-21 07:34:32,247 epoch 84 - iter 28/79 - loss 9.18484999 - time (sec): 6.16 - samples/sec: 5553.02 - lr: 0.000000\n",
            "2023-04-21 07:34:33,734 epoch 84 - iter 35/79 - loss 8.77024491 - time (sec): 7.65 - samples/sec: 5594.17 - lr: 0.000000\n",
            "2023-04-21 07:34:35,614 epoch 84 - iter 42/79 - loss 8.39951748 - time (sec): 9.53 - samples/sec: 5433.80 - lr: 0.000000\n",
            "2023-04-21 07:34:37,072 epoch 84 - iter 49/79 - loss 8.26374185 - time (sec): 10.98 - samples/sec: 5443.85 - lr: 0.000000\n",
            "2023-04-21 07:34:38,366 epoch 84 - iter 56/79 - loss 8.13360666 - time (sec): 12.28 - samples/sec: 5453.57 - lr: 0.000000\n",
            "2023-04-21 07:34:39,956 epoch 84 - iter 63/79 - loss 8.00497512 - time (sec): 13.87 - samples/sec: 5433.59 - lr: 0.000000\n",
            "2023-04-21 07:34:41,627 epoch 84 - iter 70/79 - loss 8.00570698 - time (sec): 15.54 - samples/sec: 5405.84 - lr: 0.000000\n",
            "2023-04-21 07:34:42,928 epoch 84 - iter 77/79 - loss 8.12535911 - time (sec): 16.84 - samples/sec: 5477.77 - lr: 0.000000\n",
            "2023-04-21 07:34:43,462 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:34:43,463 EPOCH 84 done: loss 8.1172 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.84it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:04,197 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:06,932 TRAIN : loss 3.525085687637329 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.96it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:09,540 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:35:09,610 DEV : loss 26.37506866455078 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:35:09,620 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:35:09,624 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:11,058 epoch 85 - iter 7/79 - loss 6.98143855 - time (sec): 1.43 - samples/sec: 5344.50 - lr: 0.000000\n",
            "2023-04-21 07:35:12,594 epoch 85 - iter 14/79 - loss 8.76961595 - time (sec): 2.97 - samples/sec: 5297.97 - lr: 0.000000\n",
            "2023-04-21 07:35:14,464 epoch 85 - iter 21/79 - loss 8.29540773 - time (sec): 4.84 - samples/sec: 5317.92 - lr: 0.000000\n",
            "2023-04-21 07:35:15,883 epoch 85 - iter 28/79 - loss 8.63038322 - time (sec): 6.26 - samples/sec: 5592.54 - lr: 0.000000\n",
            "2023-04-21 07:35:17,432 epoch 85 - iter 35/79 - loss 8.73126346 - time (sec): 7.81 - samples/sec: 5470.55 - lr: 0.000000\n",
            "2023-04-21 07:35:18,940 epoch 85 - iter 42/79 - loss 8.73918704 - time (sec): 9.31 - samples/sec: 5432.26 - lr: 0.000000\n",
            "2023-04-21 07:35:20,426 epoch 85 - iter 49/79 - loss 8.99234877 - time (sec): 10.80 - samples/sec: 5432.14 - lr: 0.000000\n",
            "2023-04-21 07:35:22,036 epoch 85 - iter 56/79 - loss 8.76521340 - time (sec): 12.41 - samples/sec: 5380.53 - lr: 0.000000\n",
            "2023-04-21 07:35:24,042 epoch 85 - iter 63/79 - loss 8.89901924 - time (sec): 14.42 - samples/sec: 5239.20 - lr: 0.000000\n",
            "2023-04-21 07:35:25,490 epoch 85 - iter 70/79 - loss 8.86022193 - time (sec): 15.86 - samples/sec: 5258.62 - lr: 0.000000\n",
            "2023-04-21 07:35:26,916 epoch 85 - iter 77/79 - loss 8.72672998 - time (sec): 17.29 - samples/sec: 5312.38 - lr: 0.000000\n",
            "2023-04-21 07:35:27,342 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:35:27,344 EPOCH 85 done: loss 8.7938 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:47,079 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:51,078 TRAIN : loss 3.5250961780548096 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.93it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:53,704 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:35:53,775 DEV : loss 26.3751277923584 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:35:53,785 Epoch    85: reducing learning rate of group 0 to 1.9073e-07.\n",
            "2023-04-21 07:35:53,787 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:35:53,790 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:35:55,050 epoch 86 - iter 7/79 - loss 7.57177398 - time (sec): 1.26 - samples/sec: 5878.67 - lr: 0.000000\n",
            "2023-04-21 07:35:56,373 epoch 86 - iter 14/79 - loss 7.73151761 - time (sec): 2.58 - samples/sec: 5510.43 - lr: 0.000000\n",
            "2023-04-21 07:35:58,640 epoch 86 - iter 21/79 - loss 8.10513526 - time (sec): 4.85 - samples/sec: 5040.29 - lr: 0.000000\n",
            "2023-04-21 07:36:00,070 epoch 86 - iter 28/79 - loss 7.99070194 - time (sec): 6.28 - samples/sec: 5169.75 - lr: 0.000000\n",
            "2023-04-21 07:36:01,600 epoch 86 - iter 35/79 - loss 8.35903900 - time (sec): 7.81 - samples/sec: 5270.77 - lr: 0.000000\n",
            "2023-04-21 07:36:02,979 epoch 86 - iter 42/79 - loss 8.53673323 - time (sec): 9.19 - samples/sec: 5366.03 - lr: 0.000000\n",
            "2023-04-21 07:36:04,411 epoch 86 - iter 49/79 - loss 9.00536661 - time (sec): 10.62 - samples/sec: 5472.66 - lr: 0.000000\n",
            "2023-04-21 07:36:06,231 epoch 86 - iter 56/79 - loss 8.76251710 - time (sec): 12.44 - samples/sec: 5407.03 - lr: 0.000000\n",
            "2023-04-21 07:36:07,604 epoch 86 - iter 63/79 - loss 8.71085002 - time (sec): 13.81 - samples/sec: 5463.56 - lr: 0.000000\n",
            "2023-04-21 07:36:08,984 epoch 86 - iter 70/79 - loss 8.51084227 - time (sec): 15.19 - samples/sec: 5569.29 - lr: 0.000000\n",
            "2023-04-21 07:36:10,317 epoch 86 - iter 77/79 - loss 8.72061753 - time (sec): 16.52 - samples/sec: 5544.11 - lr: 0.000000\n",
            "2023-04-21 07:36:10,797 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:36:10,799 EPOCH 86 done: loss 8.6549 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:36:30,532 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:36:33,304 TRAIN : loss 3.525413990020752 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.82it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:36:36,006 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:36:36,074 DEV : loss 26.375303268432617 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:36:36,084 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:36:36,088 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:36:37,611 epoch 87 - iter 7/79 - loss 8.71107463 - time (sec): 1.52 - samples/sec: 5655.43 - lr: 0.000000\n",
            "2023-04-21 07:36:39,005 epoch 87 - iter 14/79 - loss 7.68381370 - time (sec): 2.92 - samples/sec: 5646.12 - lr: 0.000000\n",
            "2023-04-21 07:36:40,325 epoch 87 - iter 21/79 - loss 7.80332976 - time (sec): 4.24 - samples/sec: 5845.01 - lr: 0.000000\n",
            "2023-04-21 07:36:41,725 epoch 87 - iter 28/79 - loss 8.70376656 - time (sec): 5.64 - samples/sec: 5985.40 - lr: 0.000000\n",
            "2023-04-21 07:36:43,253 epoch 87 - iter 35/79 - loss 8.63404713 - time (sec): 7.16 - samples/sec: 5740.15 - lr: 0.000000\n",
            "2023-04-21 07:36:45,057 epoch 87 - iter 42/79 - loss 8.78238773 - time (sec): 8.97 - samples/sec: 5562.01 - lr: 0.000000\n",
            "2023-04-21 07:36:46,476 epoch 87 - iter 49/79 - loss 8.60567894 - time (sec): 10.39 - samples/sec: 5588.09 - lr: 0.000000\n",
            "2023-04-21 07:36:47,850 epoch 87 - iter 56/79 - loss 8.47868740 - time (sec): 11.76 - samples/sec: 5595.91 - lr: 0.000000\n",
            "2023-04-21 07:36:49,630 epoch 87 - iter 63/79 - loss 8.56126219 - time (sec): 13.54 - samples/sec: 5552.23 - lr: 0.000000\n",
            "2023-04-21 07:36:51,243 epoch 87 - iter 70/79 - loss 8.51335219 - time (sec): 15.15 - samples/sec: 5528.15 - lr: 0.000000\n",
            "2023-04-21 07:36:53,181 epoch 87 - iter 77/79 - loss 8.47712677 - time (sec): 17.09 - samples/sec: 5438.87 - lr: 0.000000\n",
            "2023-04-21 07:36:53,452 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:36:53,454 EPOCH 87 done: loss 8.5033 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.01it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:37:14,435 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:37:17,195 TRAIN : loss 3.5252609252929688 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.94it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:37:19,814 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:37:19,883 DEV : loss 26.37526512145996 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:37:19,895 Epoch    87: reducing learning rate of group 0 to 9.5367e-08.\n",
            "2023-04-21 07:37:19,896 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:37:19,899 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:37:21,473 epoch 88 - iter 7/79 - loss 10.17601047 - time (sec): 1.57 - samples/sec: 5713.79 - lr: 0.000000\n",
            "2023-04-21 07:37:22,825 epoch 88 - iter 14/79 - loss 10.04151071 - time (sec): 2.92 - samples/sec: 5573.70 - lr: 0.000000\n",
            "2023-04-21 07:37:24,324 epoch 88 - iter 21/79 - loss 9.58183069 - time (sec): 4.42 - samples/sec: 5526.31 - lr: 0.000000\n",
            "2023-04-21 07:37:25,818 epoch 88 - iter 28/79 - loss 8.92625130 - time (sec): 5.92 - samples/sec: 5522.13 - lr: 0.000000\n",
            "2023-04-21 07:37:27,228 epoch 88 - iter 35/79 - loss 8.86722986 - time (sec): 7.33 - samples/sec: 5612.96 - lr: 0.000000\n",
            "2023-04-21 07:37:28,652 epoch 88 - iter 42/79 - loss 8.83821264 - time (sec): 8.75 - samples/sec: 5618.64 - lr: 0.000000\n",
            "2023-04-21 07:37:30,121 epoch 88 - iter 49/79 - loss 9.14715248 - time (sec): 10.22 - samples/sec: 5696.26 - lr: 0.000000\n",
            "2023-04-21 07:37:31,797 epoch 88 - iter 56/79 - loss 9.03308292 - time (sec): 11.90 - samples/sec: 5635.45 - lr: 0.000000\n",
            "2023-04-21 07:37:33,146 epoch 88 - iter 63/79 - loss 9.12001164 - time (sec): 13.25 - samples/sec: 5691.80 - lr: 0.000000\n",
            "2023-04-21 07:37:34,500 epoch 88 - iter 70/79 - loss 9.04171834 - time (sec): 14.60 - samples/sec: 5661.42 - lr: 0.000000\n",
            "2023-04-21 07:37:36,833 epoch 88 - iter 77/79 - loss 8.73181395 - time (sec): 16.93 - samples/sec: 5471.90 - lr: 0.000000\n",
            "2023-04-21 07:37:37,205 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:37:37,207 EPOCH 88 done: loss 8.7349 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:37:56,907 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:00,948 TRAIN : loss 3.525001287460327 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  4.03it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:03,511 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:38:03,581 DEV : loss 26.374853134155273 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:38:03,590 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:38:03,594 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:05,072 epoch 89 - iter 7/79 - loss 9.20263809 - time (sec): 1.48 - samples/sec: 6023.16 - lr: 0.000000\n",
            "2023-04-21 07:38:06,473 epoch 89 - iter 14/79 - loss 8.29547236 - time (sec): 2.88 - samples/sec: 5985.16 - lr: 0.000000\n",
            "2023-04-21 07:38:08,482 epoch 89 - iter 21/79 - loss 8.85567729 - time (sec): 4.89 - samples/sec: 5394.05 - lr: 0.000000\n",
            "2023-04-21 07:38:10,069 epoch 89 - iter 28/79 - loss 8.84573792 - time (sec): 6.47 - samples/sec: 5228.39 - lr: 0.000000\n",
            "2023-04-21 07:38:11,467 epoch 89 - iter 35/79 - loss 8.76076571 - time (sec): 7.87 - samples/sec: 5364.82 - lr: 0.000000\n",
            "2023-04-21 07:38:13,103 epoch 89 - iter 42/79 - loss 9.07009687 - time (sec): 9.51 - samples/sec: 5358.15 - lr: 0.000000\n",
            "2023-04-21 07:38:14,474 epoch 89 - iter 49/79 - loss 9.16679426 - time (sec): 10.88 - samples/sec: 5403.29 - lr: 0.000000\n",
            "2023-04-21 07:38:15,785 epoch 89 - iter 56/79 - loss 9.04028256 - time (sec): 12.19 - samples/sec: 5455.38 - lr: 0.000000\n",
            "2023-04-21 07:38:17,292 epoch 89 - iter 63/79 - loss 8.98015379 - time (sec): 13.70 - samples/sec: 5511.79 - lr: 0.000000\n",
            "2023-04-21 07:38:19,157 epoch 89 - iter 70/79 - loss 9.03773080 - time (sec): 15.56 - samples/sec: 5429.60 - lr: 0.000000\n",
            "2023-04-21 07:38:20,447 epoch 89 - iter 77/79 - loss 9.10638387 - time (sec): 16.85 - samples/sec: 5460.55 - lr: 0.000000\n",
            "2023-04-21 07:38:20,856 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:38:20,858 EPOCH 89 done: loss 8.9490 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:40,558 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:43,312 TRAIN : loss 3.524977445602417 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:45,908 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:38:45,982 DEV : loss 26.37471580505371 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:38:45,991 Epoch    89: reducing learning rate of group 0 to 4.7684e-08.\n",
            "2023-04-21 07:38:45,993 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:38:45,999 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:38:47,425 epoch 90 - iter 7/79 - loss 7.11120723 - time (sec): 1.42 - samples/sec: 5476.68 - lr: 0.000000\n",
            "2023-04-21 07:38:48,999 epoch 90 - iter 14/79 - loss 9.14951552 - time (sec): 3.00 - samples/sec: 5575.03 - lr: 0.000000\n",
            "2023-04-21 07:38:50,605 epoch 90 - iter 21/79 - loss 9.54897457 - time (sec): 4.60 - samples/sec: 5448.26 - lr: 0.000000\n",
            "2023-04-21 07:38:52,061 epoch 90 - iter 28/79 - loss 9.31196246 - time (sec): 6.06 - samples/sec: 5582.45 - lr: 0.000000\n",
            "2023-04-21 07:38:53,364 epoch 90 - iter 35/79 - loss 9.27827679 - time (sec): 7.36 - samples/sec: 5563.44 - lr: 0.000000\n",
            "2023-04-21 07:38:54,794 epoch 90 - iter 42/79 - loss 8.85366621 - time (sec): 8.79 - samples/sec: 5515.01 - lr: 0.000000\n",
            "2023-04-21 07:38:56,305 epoch 90 - iter 49/79 - loss 8.93628069 - time (sec): 10.30 - samples/sec: 5581.25 - lr: 0.000000\n",
            "2023-04-21 07:38:57,823 epoch 90 - iter 56/79 - loss 8.93085428 - time (sec): 11.82 - samples/sec: 5592.03 - lr: 0.000000\n",
            "2023-04-21 07:38:59,376 epoch 90 - iter 63/79 - loss 8.72719987 - time (sec): 13.38 - samples/sec: 5597.40 - lr: 0.000000\n",
            "2023-04-21 07:39:00,978 epoch 90 - iter 70/79 - loss 8.57506056 - time (sec): 14.98 - samples/sec: 5540.09 - lr: 0.000000\n",
            "2023-04-21 07:39:02,602 epoch 90 - iter 77/79 - loss 8.55649338 - time (sec): 16.60 - samples/sec: 5512.89 - lr: 0.000000\n",
            "2023-04-21 07:39:03,499 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:39:03,500 EPOCH 90 done: loss 8.5244 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:39:24,343 Evaluating as a multi-label problem: True\n",
            "2023-04-21 07:39:27,091 TRAIN : loss 3.5248889923095703 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.99it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:39:29,678 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:39:29,749 DEV : loss 26.374658584594727 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:39:29,758 BAD EPOCHS (no improvement): 1\n",
            "2023-04-21 07:39:29,762 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:39:31,304 epoch 91 - iter 7/79 - loss 6.61376865 - time (sec): 1.54 - samples/sec: 5743.49 - lr: 0.000000\n",
            "2023-04-21 07:39:32,830 epoch 91 - iter 14/79 - loss 8.05702156 - time (sec): 3.07 - samples/sec: 5522.04 - lr: 0.000000\n",
            "2023-04-21 07:39:34,737 epoch 91 - iter 21/79 - loss 8.52479990 - time (sec): 4.97 - samples/sec: 5325.80 - lr: 0.000000\n",
            "2023-04-21 07:39:36,254 epoch 91 - iter 28/79 - loss 7.93099749 - time (sec): 6.49 - samples/sec: 5404.24 - lr: 0.000000\n",
            "2023-04-21 07:39:37,722 epoch 91 - iter 35/79 - loss 8.08168065 - time (sec): 7.96 - samples/sec: 5375.96 - lr: 0.000000\n",
            "2023-04-21 07:39:39,303 epoch 91 - iter 42/79 - loss 8.33521319 - time (sec): 9.54 - samples/sec: 5371.87 - lr: 0.000000\n",
            "2023-04-21 07:39:40,897 epoch 91 - iter 49/79 - loss 8.16481665 - time (sec): 11.13 - samples/sec: 5298.04 - lr: 0.000000\n",
            "2023-04-21 07:39:42,397 epoch 91 - iter 56/79 - loss 8.20830516 - time (sec): 12.63 - samples/sec: 5385.70 - lr: 0.000000\n",
            "2023-04-21 07:39:44,022 epoch 91 - iter 63/79 - loss 8.02498016 - time (sec): 14.26 - samples/sec: 5391.82 - lr: 0.000000\n",
            "2023-04-21 07:39:45,365 epoch 91 - iter 70/79 - loss 8.35981901 - time (sec): 15.60 - samples/sec: 5469.93 - lr: 0.000000\n",
            "2023-04-21 07:39:46,501 epoch 91 - iter 77/79 - loss 8.43039607 - time (sec): 16.74 - samples/sec: 5530.90 - lr: 0.000000\n",
            "2023-04-21 07:39:46,964 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:39:46,966 EPOCH 91 done: loss 8.3850 - lr 0.000000\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:19<00:00,  4.04it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:06,663 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:10,685 TRAIN : loss 3.524850845336914 - f1-score (weighted avg)  0.7599\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:02<00:00,  3.97it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:13,285 Evaluating as a multi-label problem: False\n",
            "2023-04-21 07:40:13,355 DEV : loss 26.374540328979492 - f1-score (weighted avg)  0.6254\n",
            "2023-04-21 07:40:13,364 Epoch    91: reducing learning rate of group 0 to 2.3842e-08.\n",
            "2023-04-21 07:40:13,366 BAD EPOCHS (no improvement): 2\n",
            "2023-04-21 07:40:13,369 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:13,371 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:13,372 learning rate too small - quitting training!\n",
            "2023-04-21 07:40:13,374 ----------------------------------------------------------------------------------------------------\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:15,126 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:16,897 SequenceTagger predicts: Dictionary with 24 tags: <unk>, O, I-ADR, E-ADR, B-ADR, S-ADR, S-Drug, B-Finding, E-Finding, I-Finding, B-Disease, E-Disease, E-Symptom, S-Disease, S-Symptom, E-Drug, B-Symptom, B-Drug, S-Finding, I-Symptom, I-Drug, I-Disease, <START>, <STOP>\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 10/10 [00:03<00:00,  2.75it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:20,840 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:21,175 0.7394\t0.632\t0.6815\t0.9208\n",
            "2023-04-21 07:40:21,176 \n",
            "Results:\n",
            "- F-score (micro) 0.6815\n",
            "- F-score (macro) 0.3584\n",
            "- Accuracy 0.9208\n",
            "\n",
            "By class:\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "       I-ADR     0.6717    0.5067    0.5776       525\n",
            "       E-ADR     0.7636    0.7308    0.7468       442\n",
            "       B-ADR     0.7446    0.7392    0.7419       418\n",
            "      S-Drug     0.9158    0.9635    0.9391       192\n",
            "       S-ADR     0.7000    0.7532    0.7256       158\n",
            "      B-Drug     0.8421    0.5926    0.6957        27\n",
            "      E-Drug     0.7222    0.4815    0.5778        27\n",
            "   S-Finding     0.3333    0.1071    0.1622        28\n",
            "   B-Finding     0.4545    0.2083    0.2857        24\n",
            "   E-Finding     0.3333    0.0741    0.1212        27\n",
            "      I-Drug     0.8750    0.2800    0.4242        25\n",
            "   I-Symptom     0.0000    0.0000    0.0000        20\n",
            "   S-Disease     0.8333    0.4167    0.5556        12\n",
            "   B-Disease     0.2000    0.1250    0.1538         8\n",
            "   E-Disease     0.2500    0.1111    0.1538         9\n",
            "   B-Symptom     0.0000    0.0000    0.0000        13\n",
            "   E-Symptom     0.0000    0.0000    0.0000        13\n",
            "   S-Symptom     0.5000    0.2222    0.3077         9\n",
            "   I-Finding     0.0000    0.0000    0.0000         9\n",
            "   I-Disease     0.0000    0.0000    0.0000         3\n",
            "\n",
            "   micro avg     0.7394    0.6320    0.6815      1989\n",
            "   macro avg     0.4570    0.3156    0.3584      1989\n",
            "weighted avg     0.7036    0.6320    0.6587      1989\n",
            " samples avg     0.1062    0.1061    0.1062      1989\n",
            "\n",
            "2023-04-21 07:40:21,178 ----------------------------------------------------------------------------------------------------\n"
          ]
        }
      ],
      "source": [
        "hist = trainer.train('flair_ner_model'+suf,\n",
        "              embeddings_storage_mode='gpu',\n",
        "              gold_label_dictionary_for_eval=tag_dictionary,\n",
        "              mini_batch_size = 12,\n",
        "              eval_batch_size = 12,\n",
        "              patience=1,\n",
        "              anneal_factor = 0.5,\n",
        "              anneal_against_dev_loss=True,\n",
        "              main_evaluation_metric = (\"weighted avg\", \"f1-score\"),\n",
        "              metrics_for_tensorboard=[(\"weighted avg\", 'precision'), (\"weighted avg\", 'recall'), (\"macro avg\", 'f1-score'), (\"macro avg\", 'precision'), (\"macro avg\", 'recall')],\n",
        "              use_tensorboard=True,\n",
        "              tensorboard_comment='Flair NER',\n",
        "              tensorboard_log_dir='runs/tb_logs'+suf,\n",
        "              min_learning_rate=3e-8,\n",
        "              optimizer=AdamW,\n",
        "              learning_rate=1e-1,\n",
        "              weight_decay=0.001,\n",
        "              max_epochs=100,\n",
        "              monitor_train=True,\n",
        "              save_optimizer_state=True,)\n",
        "# hist"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "ExiLRreqXsDJ"
      },
      "outputs": [],
      "source": [
        "# print(hist.keys())\n",
        "# hist"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "P5Uu5qkN0pia",
        "outputId": "16651f7d-446e-40b0-f23f-007e6d5d4627"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "_v2_no_rnn\n",
            "2023-04-21 07:40:21,206 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:21,207 WARNING: No LOSS found for test split in this data.\n",
            "2023-04-21 07:40:21,208 Are you sure you want to plot LOSS and not another value?\n",
            "2023-04-21 07:40:21,209 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:21,234 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:21,235 WARNING: No F1 found for test split in this data.\n",
            "2023-04-21 07:40:21,236 Are you sure you want to plot F1 and not another value?\n",
            "2023-04-21 07:40:21,236 ----------------------------------------------------------------------------------------------------\n",
            "2023-04-21 07:40:22,398 Loss and F1 plots are saved in flair_ner_model_v2_no_rnn/training.png\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABdYAAAPfCAYAAADKU5s5AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOzdeZhcBZk+7Kd6TbqTdEhCNggQAdlklcUAP0fGCMoyIjjIyCgoyoyCChEUVHBBiSIqAiKjnyM6iruggKKAgophMexbkDVhSQKEpLP2Wt8f3V3pJiHpJkt1p+/7uurqqlOnqt6qrlNJP+et9xSKxWIxAAAAAABAr1SUuwAAAAAAABhIBOsAAAAAANAHgnUAAAAAAOgDwToAAAAAAPSBYB0AAAAAAPpAsA4AAAAAAH0gWAcAAAAAgD4QrAMAAAAAQB8I1gEAAAAAoA8E6wAAAAAA0AdlDdb/8pe/5IgjjsjEiRNTKBRy1VVX9bi+WCzmnHPOyYQJEzJ06NBMnTo1//znP3uss2DBghx33HEZMWJERo4cmRNPPDFLlizpsc69996b//f//l+GDBmSSZMm5fzzz9/QTw0AAAAAgE1UVTkffOnSpdl9993z/ve/P0cdddQq159//vm56KKL8oMf/CCTJ0/O2WefnUMOOSQPPvhghgwZkiQ57rjj8txzz+X6669PS0tL3ve+9+Wkk07KFVdckSRpbGzMwQcfnKlTp+ayyy7Lfffdl/e///0ZOXJkTjrppF7V2d7enmeffTbDhw9PoVBYfy8AAAAAAJRBsVjM4sWLM3HixFRUGGoBfVbsJ5IUr7zyytLl9vb24vjx44tf/epXS8sWLlxYrK2tLf7kJz8pFovF4oMPPlhMUrzjjjtK6/z+978vFgqF4jPPPFMsFovFSy+9tLjZZpsVm5qaSut88pOfLO6www69rm3OnDnFJE5OTk5OTk5OTk5OTk5OTk5OTpvUac6cOa82zoNBrawd62vyxBNPZO7cuZk6dWppWUNDQ/bbb7/MmDEjxx57bGbMmJGRI0dm7733Lq0zderUVFRU5Lbbbss73vGOzJgxI2984xtTU1NTWueQQw7JV77ylbz00kvZbLPNVnnspqamNDU1lS4Xi8UkyZw5czJixIgN8XQBAAAAYKNpbGzMpEmTMnz48HKXAgNSvw3W586dmyQZN25cj+Xjxo0rXTd37tyMHTu2x/VVVVUZNWpUj3UmT568yn10Xbe6YH369On5/Oc/v8ryESNGCNYBAAAA2GQYewyvjgFKq3HWWWdl0aJFpdOcOXPKXRIAAAAAAP1Evw3Wx48fnySZN29ej+Xz5s0rXTd+/PjMnz+/x/Wtra1ZsGBBj3VWdx/dH+PlamtrS93putQBAAAAAOiu3wbrkydPzvjx43PjjTeWljU2Nua2227LlClTkiRTpkzJwoULM3PmzNI6f/rTn9Le3p799tuvtM5f/vKXtLS0lNa5/vrrs8MOO6x2DAwAAAAAAKxJWYP1JUuW5O67787dd9+dpOOApXfffXdmz56dQqGQU089NV/84hfz29/+Nvfdd1/e+973ZuLEiTnyyCOTJDvttFPe+ta35oMf/GBuv/323HLLLTnllFNy7LHHZuLEiUmSd7/73ampqcmJJ56YBx54ID/72c/yzW9+M9OmTSvTswYAAAAAYCArFIvFYrke/KabbspBBx20yvLjjz8+l19+eYrFYj772c/mO9/5ThYuXJgDDzwwl156aV772teW1l2wYEFOOeWUXH311amoqMjRRx+diy66KMOGDSutc++99+bkk0/OHXfckTFjxuQjH/lIPvnJT/a6zsbGxjQ0NGTRokXGwgAAAAAw4Mm7YN2UNVgfKHzQAAAAALApkXfBuum3M9YBAAAAAKA/EqwDAAAAAEAfCNYBAAAAAKAPBOsAAAAAANAHgnUAAAAAAOgDwToAAAAAAPSBYB0AAAAAAPpAsA4AAAAAAH0gWAcAAAAAgD6oKncBlNfCZc1ZsLQ5I4ZWZ8yw2nKXAwAAAADQ7+lYH+S++odZ+dev3Zwf3fpUuUsBAAAAABgQBOuDXH1tx5cWljW3lbkSAAAAAICBQbA+yNXVVCZJlja1lrkSAAAAAICBQbA+yNXX6FgHAAAAAOgLwfogV1erYx0AAAAAoC8E64NcV8f60mbBOgAAAABAbwjWB7mVM9aNggEAAAAA6A3B+iBXX9s1Y13HOgAAAABAbwjWB7muYF3HOgAAAABA7wjWB7n6zlEwOtYBAAAAAHpHsD7I1XV1rDfrWAcAAAAA6A3B+iDX1bHe3Nqelrb2MlcDAAAAAND/CdYHubqaqtL5ZbrWAQAAAADWSrA+yNVUVaS6spDEnHUAAAAAgN4QrFPqWl/aJFgHAAAAAFgbwTqlOetLm4yCAQAAAABYG8E6qa/t7Fg3CgYAAAAAYK0E66SuM1hfpmMdAAAAAGCtBOusHAWjYx0AAAAAYK0E65QOXrqsWcc6AAAAAMDaCNZJfW3XwUt1rAMAAAAArI1gHR3rAAAAAAB9IFhn5Yx1HesAAAAAAGslWCd1tR0d6w5eCgAAAACwdoJ1MqxzxvqyJqNgAAAAAADWRrBOaca6jnUAAAAAgLUTrJP6ro51By8FAAAAAFgrwTorO9YdvBQAAAAAYK0E66S+M1jXsQ4AAAAAsHaCdVLXOQrGjHUAAAAAgLUTrFPqWF/apGMdAAAAAGBtBOukrqazY92MdQAAAACAtRKsk2G1HR3rTa3taW1rL3M1AAAAAAD9m2Cd0oz1JFnWYhwMAAAAAMCaCNZJTWVFqioKSZJl5qwDAAAAAKyRYJ0UCoWVc9abzVkHAAAAAFgTwTpJkvrOOes61gEAAAAA1kywTpLoWAcAAAAA6CXBOkm6dawL1gEAAAAA1kiwTpKVHetLjIIBAAAAAFgjwTpJkmGlGes61gEAAAAA1kSwTpKkrqYjWF/arGMdAAAAAGBNBOskSeprO0bB6FgHAAAAAFgzwTpJdKwDAAAAAPSWYJ0kSX3nwUuXNetYBwAAAABYE8E6SZK6zoOXLm3SsQ4AAAAAsCaCdZLoWAcAAAAA6C3BOklWzlhf4uClAAAAAABrJFgnSVJf29WxbhQMAAAAAMCaCNZJktSXZqzrWAcAAAAAWBPBOklWjoLRsQ4AAAAAsGaCdZJ0HwWjYx0AAAAAYE0E6yRJ6mu6RsHoWAcAAAAAWBPBOkmSupqOjvXlLW1pay+WuRoAAAAAgP5LsE6SlQcvTTrCdQAAAAAAVk+wTpKktqoiFYWO80ubzFkHAAAAAHglgnWSJIVCoducdcE6AAAAAMArEaxT0jUOZlmzUTAAAAAAAK9EsE5JXW3HAUx1rAMAAAAAvDLBOiVdo2B0rAMAAAAAvDLBOiV1NZ0d68061gEAAAAAXolgnZLSjPUmHesAAAAAAK+kXwfrbW1tOfvsszN58uQMHTo02267bc4999wUi8XSOsViMeecc04mTJiQoUOHZurUqfnnP//Z434WLFiQ4447LiNGjMjIkSNz4oknZsmSJRv76fR7OtYBAAAAANauXwfrX/nKV/Ltb387l1xySR566KF85Stfyfnnn5+LL764tM7555+fiy66KJdddlluu+221NfX55BDDsmKFStK6xx33HF54IEHcv311+eaa67JX/7yl5x00knleEr9mhnrAAAAAABrV1XuAtbk73//e97+9rfnsMMOS5Jss802+clPfpLbb789SUe3+oUXXpjPfOYzefvb354k+eEPf5hx48blqquuyrHHHpuHHnoo1113Xe64447svffeSZKLL744hx56aC644IJMnDhxlcdtampKU1NT6XJjY+OGfqr9Ql1tR8f6kiYd6wAAAAAAr6Rfd6zvv//+ufHGG/PII48kSe6555787W9/y9ve9rYkyRNPPJG5c+dm6tSppds0NDRkv/32y4wZM5IkM2bMyMiRI0uhepJMnTo1FRUVue2221b7uNOnT09DQ0PpNGnSpA31FPuVYaUZ64J1AAAAAIBX0q871s8888w0NjZmxx13TGVlZdra2vKlL30pxx13XJJk7ty5SZJx48b1uN24ceNK182dOzdjx47tcX1VVVVGjRpVWuflzjrrrEybNq10ubGxcVCE63Wdo2CWGgUDAAAAAPCK+nWw/vOf/zw//vGPc8UVV2SXXXbJ3XffnVNPPTUTJ07M8ccfv8Eet7a2NrW1tRvs/vur+s5RMMscvBQAAAAA4BX162D9jDPOyJlnnpljjz02SbLrrrvmqaeeyvTp03P88cdn/PjxSZJ58+ZlwoQJpdvNmzcve+yxR5Jk/PjxmT9/fo/7bW1tzYIFC0q3p0OpY71JxzoAAAAAwCvp1zPWly1bloqKniVWVlamvb09STJ58uSMHz8+N954Y+n6xsbG3HbbbZkyZUqSZMqUKVm4cGFmzpxZWudPf/pT2tvbs99++22EZzFw1NfoWAcAAAAAWJt+3bF+xBFH5Etf+lK22mqr7LLLLrnrrrvy9a9/Pe9///uTJIVCIaeeemq++MUvZvvtt8/kyZNz9tlnZ+LEiTnyyCOTJDvttFPe+ta35oMf/GAuu+yytLS05JRTTsmxxx6biRMnlvHZ9T91tTrWAQAAAADWpl8H6xdffHHOPvvsfPjDH878+fMzceLE/Nd//VfOOeec0jqf+MQnsnTp0px00klZuHBhDjzwwFx33XUZMmRIaZ0f//jHOeWUU/LmN785FRUVOfroo3PRRReV4yn1azrWAQAAAADWrlAsFovlLqK/a2xsTENDQxYtWpQRI0aUu5wN5sFnG3PoRX/NmGG1+cdnppa7HAAAAAA2kMGSd8GG0q9nrLNxDescBaNjHQAAAADglQnWKamr7RoF05b2dl9kAAAAAABYHcE6JfU1K0fuL29xAFMAAAAAgNURrFMypLoihULH+aXGwQAAAAAArJZgnZJCoVDqWl/WpGMdAAAAAGB1BOv0UFfTMWddxzoAAAAAwOoJ1umhvrazY71ZxzoAAAAAwOoI1umh1LHepGMdAAAAAGB1BOv00NWxvtSMdQAAAACA1RKs00O9GesAAAAAAGskWKeHuq4Z60bBAAAAAACslmCdHlZ2rBsFAwAAAACwOoJ1eqir6exYNwoGAAAAAGC1BOv0UF/b2bHu4KUAAAAAAKslWKcHHesAAAAAAGsmWKcHM9YBAAAAANZMsE4PdbUdHetLm3SsAwAAAACsjmCdHoZ1BuvLzFgHAAAAAFgtwTo91JVGwehYBwAAAABYHcE6PdR3daybsQ4AAAAAsFqCdXoodaybsQ4AAAAAsFqCdXqor9GxDgAAAACwJoJ1eqirXTljvVgslrkaAAAAAID+R7BOD10d68VisqKlvczVAAAAAAD0P4J1ehhaXVk6v8ScdQAAAACAVQjW6aGiopD6zgOYLmsWrAMAAAAAvJxgnVXU1XaMg1na5ACmAAAAAAAvJ1hnFTrWAQAAAABemWCdVdR1HsB0abOOdQAAAACAlxOss4r62s6OdQcvBQAAAABYhWCdVehYBwAAAAB4ZYJ1VlHqWDdjHQAAAABgFYJ1VlHqWG/SsQ4AAAAA8HKCdVYxrLYrWNexDgAAAADwcoJ1VlFX0zEKZqlRMAAAAAAAqxCss4r6zo71ZUbBAAAAAACsQrDOKnSsAwAAAAC8MsE6q6jvPHjpsmYd6wAAAAAALydYZxV1tZ0d6w5eCgAAAACwCsE6q9CxDgAAAADwygTrrMKMdQAAAACAVyZYZxX1tR0d60bBAAAAAACsSrDOKrqC9WVNRsEAAAAAALycYJ1V1HcbBVMsFstcDQAAAABA/yJYZxV1nR3r7cWkqbW9zNUAAAAAAPQvgnVWMbS6snTenHUAAAAAgJ4E66yisqJQCteXNZuzDgAAAADQnWCd1aqvXTlnHQAAAACAlQTrrFZdTcec9aVNOtYBAAAAALoTrLNa9bVdwbqOdQAAAACA7gTrrFZ9TdeMdcE6AAAAAEB3gnVWq67WKBgAAAAAgNURrLNaOtYBAAAAAFZPsM5qlQ5e2qxjHQAAAACgO8E6q1Vf29mx7uClAAAAAAA9CNZZLR3rAAAAAACrJ1hntcxYBwAAAABYPcE6q1Vf29mx3qRjHQAAAACgO8E6q9U1Y32pGesAAAAAAD0I1lmtlTPWBesAAAAAAN0J1lmtro71ZQ5eCgAAAADQg2Cd1Sp1rBsFAwAAAADQg2Cd1arvDNZ1rAMAAAAA9CRYZ7XqHLwUAAAAAGC1BOusVveO9WKxWOZqAAAAAAD6D8E6q9XVsd7aXkxzW3uZqwEAAAAA6D8E66xWV8d6kixtMmcdAAAAAKCLYJ3VqqwoZEh1x9vDnHUAAAAAgJUE67yi7nPWAQAAAADoIFjnFXXNWV/arGMdAAAAAKCLYJ1XVOpYN2MdAAAAAKCk3wfrzzzzTP7zP/8zo0ePztChQ7PrrrvmH//4R+n6YrGYc845JxMmTMjQoUMzderU/POf/+xxHwsWLMhxxx2XESNGZOTIkTnxxBOzZMmSjf1UBpy6Gh3rAAAAAAAv16+D9ZdeeikHHHBAqqur8/vf/z4PPvhgvva1r2WzzTYrrXP++efnoosuymWXXZbbbrst9fX1OeSQQ7JixYrSOscdd1weeOCBXH/99bnmmmvyl7/8JSeddFI5ntKAUl/bNWNdsA4AAAAA0KWq3AWsyVe+8pVMmjQp3//+90vLJk+eXDpfLBZz4YUX5jOf+Uze/va3J0l++MMfZty4cbnqqqty7LHH5qGHHsp1112XO+64I3vvvXeS5OKLL86hhx6aCy64IBMnTty4T2oAKXWsGwUDAAAAAFDSrzvWf/vb32bvvffOv//7v2fs2LHZc889893vfrd0/RNPPJG5c+dm6tSppWUNDQ3Zb7/9MmPGjCTJjBkzMnLkyFKoniRTp05NRUVFbrvtttU+blNTUxobG3ucBiMd6wAAAAAAq+rXwfrjjz+eb3/729l+++3zhz/8IR/60Ify0Y9+ND/4wQ+SJHPnzk2SjBs3rsftxo0bV7pu7ty5GTt2bI/rq6qqMmrUqNI6Lzd9+vQ0NDSUTpMmTVrfT21A6Dp46RId6wAAAAAAJf06WG9vb89ee+2V8847L3vuuWdOOumkfPCDH8xll122QR/3rLPOyqJFi0qnOXPmbNDH66/qajtGwSxr0rEOAAAAANClXwfrEyZMyM4779xj2U477ZTZs2cnScaPH58kmTdvXo915s2bV7pu/PjxmT9/fo/rW1tbs2DBgtI6L1dbW5sRI0b0OA1GXR3rS5t1rAMAAAAAdOnXwfoBBxyQWbNm9Vj2yCOPZOutt07ScSDT8ePH58Ybbyxd39jYmNtuuy1TpkxJkkyZMiULFy7MzJkzS+v86U9/Snt7e/bbb7+N8CwGrq6Dl5qxDgAAAACwUlW5C1iT0047Lfvvv3/OO++8HHPMMbn99tvzne98J9/5zneSJIVCIaeeemq++MUvZvvtt8/kyZNz9tlnZ+LEiTnyyCOTdHS4v/Wtby2NkGlpackpp5ySY489NhMnTizjs+v/ug5eutSMdQAAAACAkn4drO+zzz658sorc9ZZZ+ULX/hCJk+enAsvvDDHHXdcaZ1PfOITWbp0aU466aQsXLgwBx54YK677roMGTKktM6Pf/zjnHLKKXnzm9+cioqKHH300bnooovK8ZQGFB3rAAAAAACrKhSLxWK5i+jvGhsb09DQkEWLFg2qees3PDgvH/jhP7L7pJH5zckHlLscAAAAANaTwZp3wfrSr2esU15do2CWNelYBwAAAADoIljnFdXXdoyCWSpYBwAAAAAoEazziupqOg9e2uzgpQAAAAAAXQTrvKKujnUHLwUAAAAAWEmwzivq6lhvaSumubW9zNUAAAAAAPQPgnVeUV1NZem8rnUAAAAAgA6CdV5RdWVFaqo63iLmrAMAAAAAdBCss0b1nV3ry5p0rAMAAAAAJIJ11qK+tmPOuo51AAAAAIAOgnXWqL7zAKZLdawDAAAAACQRrLMWdbUdo2AE6wAAAAAAHQTrrFFXx/oyo2AAAAAAAJII1lmLus6Dly5t1rEOAAAAAJAI1lmLroOXLmvSsQ4AAAAAkAjWWQsd6wAAAAAAPQnWWaNSx7oZ6wAAAAAASQTrrEXXwUuXNulYBwAAAABIBOusRX1txygYHesAAAAAAB0E66xRXWfH+hId6wAAAAAASQTrrMXKjnXBOgAAAABAIlhnLepKM9aNggEAAAAASATrrEV9jY51AAAAAIDuBOusUV2tjnUAAAAAgO4E66yRjnUAAAAAgJ4E66xRqWO9Wcc6AAAAAEAiWGcthnUevLS5tT0tbe1lrgYAAAAAoPwE66zR0M5RMEmyzJx1AAAAAADBOmtWU1WRmsqOt8lSc9YBAAAAAATrrF1drQOYAgAAAAB0EayzVvWdc9aXGgUDAAAAACBYZ+3qOuesGwUDAAAAACBYpxfqajs61h28FAAAAABAsE4v1OtYBwAAAAAoEayzVvVdHevNOtYBAAAAAF5VsP6DH/wg1157benyJz7xiYwcOTL7779/nnrqqfVWHP1DqWO9Scc6AAAAAMCrCtbPO++8DB06NEkyY8aMfOtb38r555+fMWPG5LTTTluvBVJ+XTPWl5qxDgAAAACQqldzozlz5mS77bZLklx11VU5+uijc9JJJ+WAAw7Im970pvVZH/1AV8f6MjPWAQAAAABeXcf6sGHD8uKLLyZJ/vjHP+Ytb3lLkmTIkCFZvnz5+quOfqGuprNjXbAOAAAAAPDqOtbf8pa35AMf+ED23HPPPPLIIzn00EOTJA888EC22Wab9Vkf/UB9bWfHulEwAAAAAACvrmP9W9/6VqZMmZLnn38+v/rVrzJ69OgkycyZM/Mf//Ef67VAyk/HOgAAAADASq+qY33kyJG55JJLVln++c9/fp0Lov8pdaw361gHAAAAAHhVHevXXXdd/va3v5Uuf+tb38oee+yRd7/73XnppZfWW3H0D/VdHetNOtYBAAAAAF5VsH7GGWeksbExSXLffffl4x//eA499NA88cQTmTZt2notkPKrr+0I1nWsAwAAAAC8ylEwTzzxRHbeeeckya9+9ascfvjhOe+883LnnXeWDmTKpqOupmMUzBId6wAAAAAAr65jvaamJsuWLUuS3HDDDTn44IOTJKNGjSp1srPp0LEOAAAAALDSq+pYP/DAAzNt2rQccMABuf322/Ozn/0sSfLII49kyy23XK8FUn5dHetmrAMAAAAAvMqO9UsuuSRVVVX55S9/mW9/+9vZYostkiS///3v89a3vnW9Fkj5dR28tKm1Pa1t7WWuBgAAAACgvF5Vx/pWW22Va665ZpXl3/jGN9a5IPqfutrK0vllLW0ZUfmq9scAAAAAAGwSXlWwniRtbW256qqr8tBDDyVJdtlll/zbv/1bKisr13JLBpqayopUVRTS2l7Msqa2jBhSXe6SAAAAAADK5lUF648++mgOPfTQPPPMM9lhhx2SJNOnT8+kSZNy7bXXZtttt12vRVJehUIh9bVVWbS8JUubzVkHAAAAAAa3VzXT46Mf/Wi23XbbzJkzJ3feeWfuvPPOzJ49O5MnT85HP/rR9V0j/UB95wFMlzW1lbkSAAAAAIDyelUd6zfffHNuvfXWjBo1qrRs9OjR+fKXv5wDDjhgvRVH/1FX2/FWWdKkYx0AAAAAGNxeVcd6bW1tFi9evMryJUuWpKamZp2Lov8pdawbBQMAAAAADHKvKlg//PDDc9JJJ+W2225LsVhMsVjMrbfemv/+7//Ov/3bv63vGukH6mo6OtaXNhsFAwAAAAAMbq8qWL/ooouy7bbbZsqUKRkyZEiGDBmS/fffP9ttt10uvPDC9Vwi/UF9bdeMdR3rAAAAAMDg9qpmrI8cOTK/+c1v8uijj+ahhx5Kkuy0007Zbrvt1mtx9B861gEAAAAAOvQ6WJ82bdoar//zn/9cOv/1r3/91VdEv6RjHQAAAACgQ6+D9bvuuqtX6xUKhVddDP1XvY51AAAAAIAkfQjWu3ekM/jU1Xa8VZY161gHAAAAAAa3V3XwUgaf+pqOUTBLm3SsAwAAAACDm2CdXunqWF9qxjoAAAAAMMgJ1umVUse6UTAAAAAAwCAnWKdX6mq6ZqwbBQMAAAAADG6CdXqlvrZrxrqOdQAAAABgcBOs0ys61gEAAAAAOgjW6ZWujvVlZqwDAAAAAIOcYJ1eqe/sWF/apGMdAAAAABjcBOv0Sn1tR7C+vKUtbe3FMlcDAAAAAFA+gnV6pa6msnTeOBgAAAAAYDATrNMrtVUVqawoJHEAUwAAAABgcBOs0yuFQqHUtb60Scc6AAAAADB4Cdbpta4DmOpYBwAAAAAGswEVrH/5y19OoVDIqaeeWlq2YsWKnHzyyRk9enSGDRuWo48+OvPmzetxu9mzZ+ewww5LXV1dxo4dmzPOOCOtrbqu+6quVsc6AAAAAMCACdbvuOOO/M///E922223HstPO+20XH311fnFL36Rm2++Oc8++2yOOuqo0vVtbW057LDD0tzcnL///e/5wQ9+kMsvvzznnHPOxn4KA56OdQAAAACAARKsL1myJMcdd1y++93vZrPNNistX7RoUb73ve/l61//ev71X/81r3/96/P9738/f//733PrrbcmSf74xz/mwQcfzI9+9KPsscceedvb3pZzzz033/rWt9Lc3Lzax2tqakpjY2OPE0l9V8d6s451AAAAAGDwGhDB+sknn5zDDjssU6dO7bF85syZaWlp6bF8xx13zFZbbZUZM2YkSWbMmJFdd90148aNK61zyCGHpLGxMQ888MBqH2/69OlpaGgonSZNmrQBntXAU+pYb9KxDgAAAAAMXv0+WP/pT3+aO++8M9OnT1/lurlz56ampiYjR47ssXzcuHGZO3duaZ3uoXrX9V3Xrc5ZZ52VRYsWlU5z5sxZD89k4Kur7QjWl5ixDgAAAAAMYlXlLmBN5syZk4997GO5/vrrM2TIkI32uLW1tamtrd1ojzdQ1Nd0jIJZZhQMAAAAADCI9euO9ZkzZ2b+/PnZa6+9UlVVlaqqqtx888256KKLUlVVlXHjxqW5uTkLFy7scbt58+Zl/PjxSZLx48dn3rx5q1zfdR29V9c5Cmapg5cCAAAAAINYvw7W3/zmN+e+++7L3XffXTrtvffeOe6440rnq6urc+ONN5ZuM2vWrMyePTtTpkxJkkyZMiX33Xdf5s+fX1rn+uuvz4gRI7Lzzjtv9Oc0kHUdvHSZUTAAAAAAwCDWr0fBDB8+PK973et6LKuvr8/o0aNLy0888cRMmzYto0aNyogRI/KRj3wkU6ZMyRve8IYkycEHH5ydd94573nPe3L++edn7ty5+cxnPpOTTz7ZuJc+0rEOAAAAANDPg/Xe+MY3vpGKioocffTRaWpqyiGHHJJLL720dH1lZWWuueaafOhDH8qUKVNSX1+f448/Pl/4whfKWPXAVOpYN2MdAAAAABjECsVisVjuIvq7xsbGNDQ0ZNGiRRkxYkS5yymbX818Oh//xT35l9dunh+8f99ylwMAAADAqyTvgnXTr2es07/oWAcAAAAAEKzTB6UZ601mrAMAAAAAg5dgnV7r6lhfqmMdAAAAABjEBOv0mo51AAAAAADBOn1Q3xmsm7EOAAAAAAxmgnV6ra508NK2tLcXy1wNAAAAAEB5CNbpta6O9SRZ3mIcDAAAAAAwOAnW6bUh1RWpKHScdwBTAAAAAGCwEqzTa4VCYeWc9T4cwLSptS1tRscAAAAAAJsIwTp90jVnvbcd6/c+vTAHfPnPOeyiv5rLDgAAAABsEgTr9ElXx/rSXnSs3/r4i3n3d2/LC0ua8vDcxfnHUy9t6PIAAAAAADY4wTp90tuO9T89PC/H/+/tWdLUmprKjrfZtfc+u8HrAwAAAADY0ATr9EldL2as/+buZ3LSD2emqbU9U3cam4v+Y48kye/un2vWOgAAAAAw4FWVuwAGlvqaNXes/+jWp3L2b+5PsZgcucfEfPXfd0+xmDQMrc7zi5tyx5ML8obXjN6YJQMAAAAArFc61umTutqujvVVg/VLb3o0n7mqI1R/zxu2zteP2SPVlRWpqarIIbuMS5JcYxwMAAAAADDACdbpk2FdBy9tXjkKplgs5su/fzjnXzcrSXLyQdvmC2/fJRUVhdI6h+02MUly3f1z09rWvhErBgAAAABYvwTr9EnXwUuXdY6CaWsv5jNX3Z/Lbn4sSXLW23bMGYfsmEKh0ON2+287OiPrqvPCkubc/sSCjVs0AAAAAMB6JFinT+q7Otab2tLS1p5Tf3Z3fnzb7BQKyfSjds1//cu2q71ddWVF3rrL+CTJNfc9t9HqBQAAAABY3wTr9ElXx/qLS5vzX/83M1ff82yqKgq56Ng98x/7brXG2x5uHAwAAAAAsAmoKncBDCxdHetX39NxENLaqopc9p+vz0E7jl3rbd/wmlEZVV+TBUubc+vjC3Lg9mM2aK0AAAAAABuCjnX6pK6msnR+WG1Vfvj+fXsVqidJVWVF3vq6jnEw19737AapDwAAAABgQxOs0ycTRw5Nkoyqr8lPPviG7Pea0X26/eG7TkjSMQ6mxTgYAAAAAGAAMgqGPpnymtH5znten123bMiEhqF9vv2+k0dlzLCavLCkOTMeezFvfO3mG6BKAAAAAIANR8c6fVJRUcjBu4x/VaF68rJxMPc+tz5LAwAAAADYKATrbHSH7ToxSXLdA8bBAAAAAAADj2Cdja5jHExtFi1vyS2PvlDucgAAAAAA+kSwzkZXWVHIobt2jIO5xjgYAAAAAGCAEaxTFoftOiFJ8ocH5qa51TgYAAAAAGDgEKxTFntvMypjh9dm8YrW/O3R58tdDgAAAABArwnWKYuOcTAdXevGwQAAAAAAA4lgnbI5fLeOYP36B+alqbWtzNUAAAAAAPSOYJ2y2WurzTJ+xJAsbmrNXx95odzlAAAAAAD0imCdsqnoNg7m2vuMgwEAAAAABgbBOmV1WNc4mAfnZUWLcTAAAAAAQP8nWKes9pw0MhMbhmRJU2v+8sjz5S4HAAAAAGCtBOuUlXEwAAAAAMBAI1in7LrGwdxgHAwAAAAAMAAI1im7PSaNzBYjh2Zpc1tumjW/3OUAAAAAAKyRYJ2yKxQKpa71a+41DgYAAAAA6N8E6/QLh3XOWb/xoflZ3mwcDAAAAADQfwnW6Rd227Ihk0YNzfKWtvzZOBgAAAAAoB8TrNMvFAqFHLbrxCTJtcbBAAAAAAD9mGCdfuPwzjnrNz48L8uaW8tcDQAAAADA6gnW6Td2mTgiW4+uy4qW9vzpYeNgAAAAAID+SbBOv9ExDqaja904GAAAAACgvxKs068c1jkO5k8Pz8+SJuNgAAAAAID+R7BOv7LzhBGZPKY+Ta3tuf7BueUuBwAAAABgFYJ1+pVCoZAj99giSfLrO58pczUAAAAAAKsSrNPvvGPPjmD9lkdfyLzGFWWuBgAAAACgJ8E6/c5Wo+uy99abpb2Y/OZuXesAAAAAQP8iWKdfOmqvLZMYBwMAAAAA9D+Cdfqlw3adkJrKijw8d3EefLax3OUAAAAAAJQI1umXGuqq8+adxiZJrrzr6TJXAwAAAACwkmCdfqtrHMxVdz+b1rb2MlcDAAAAANBBsE6/9S+v3Tyb1VXn+cVNueWxF8tdDgAAAABAEsE6/VhNVUWO2H1ikuTKO42DAQAAAAD6B8E6/do79twiSfKHB+ZlaVNrmasBAAAAABCs08/tMWlkXjOmPstb2nLd/XPLXQ4AAAAAgGCd/q1QKJS61n99l3EwAAAAAED5Cdbp947sDNb//tiLeW7R8jJXAwAAAAAMdoJ1+r1Jo+qy7+RRKRaT39z9bLnLAQAAAAAGOcE6A8JRXeNg7nw6xWKxzNUAAAAAAIOZYJ0B4W27TkhNVUUembckDzzbWO5yAAAAAIBBTLDOgNAwtDpv2WlckuTKu54pczUAAAAAwGAmWGfAOGqvjnEwv7n72bS2tZe5GgAAAABgsBKsM2C88bWbZ3R9TV5Y0pS/PvpCucsBAAAAAAYpwToDRnVlRY7YfWKS5Mo7jYMBAAAAAMpDsM6A0jUO5o8Pzs3iFS1lrgYAAAAAGIwE6wwou27RkG03r8+Klvb8/v655S4HAAAAABiEBOsMKIVCIUfttWUS42AAAAAAgPIQrDPgvH2Pjjnrtz7xYp5duLzM1QAAAAAAg02/DtanT5+effbZJ8OHD8/YsWNz5JFHZtasWT3WWbFiRU4++eSMHj06w4YNy9FHH5158+b1WGf27Nk57LDDUldXl7Fjx+aMM85Ia2vrxnwqrEdbblaXN7xmVIrF5Kq7da0DAAAAABtXvw7Wb7755px88sm59dZbc/3116elpSUHH3xwli5dWlrntNNOy9VXX51f/OIXufnmm/Pss8/mqKOOKl3f1taWww47LM3Nzfn73/+eH/zgB7n88stzzjnnlOMpsZ4ctWfHOJhf3/lMisVimasBAAAAAAaTQnEApZLPP/98xo4dm5tvvjlvfOMbs2jRomy++ea54oor8s53vjNJ8vDDD2ennXbKjBkz8oY3vCG///3vc/jhh+fZZ5/NuHHjkiSXXXZZPvnJT+b5559PTU3NWh+3sbExDQ0NWbRoUUaMGLFBnyO9s3hFS/b+4g1pam3P1accmF23bCh3SQAAAAADhrwL1k2/7lh/uUWLFiVJRo0alSSZOXNmWlpaMnXq1NI6O+64Y7baaqvMmDEjSTJjxozsuuuupVA9SQ455JA0NjbmgQceWO3jNDU1pbGxsceJ/mX4kOocvMv4JMmv73q6zNUAAAAAAIPJgAnW29vbc+qpp+aAAw7I6173uiTJ3LlzU1NTk5EjR/ZYd9y4cZk7d25pne6hetf1XdetzvTp09PQ0FA6TZo0aT0/G9aHo/bcIkny27ufTUtbe5mrAQAAAAAGiwETrJ988sm5//7789Of/nSDP9ZZZ52VRYsWlU5z5szZ4I9J3/2/7cdkzLCavLi0OX/95/PlLgcAAAAAGCQGRLB+yimn5Jprrsmf//znbLnllqXl48ePT3NzcxYuXNhj/Xnz5mX8+PGldebNm7fK9V3XrU5tbW1GjBjR40T/U1VZkX/bvaNr/dd3PlPmagAAAACAwaJfB+vFYjGnnHJKrrzyyvzpT3/K5MmTe1z/+te/PtXV1bnxxhtLy2bNmpXZs2dnypQpSZIpU6bkvvvuy/z580vrXH/99RkxYkR23nnnjfNE2GCO2qsjWL/+wXlpXNFS5moAAAAAgMGgXwfrJ598cn70ox/liiuuyPDhwzN37tzMnTs3y5cvT5I0NDTkxBNPzLRp0/LnP/85M2fOzPve975MmTIlb3jDG5IkBx98cHbeeee85z3vyT333JM//OEP+cxnPpOTTz45tbW15Xx6rAe7TByR7ccOS1Nre0654q48Mm9xuUsCAAAAADZxhWKxWCx3Ea+kUCisdvn3v//9nHDCCUmSFStW5OMf/3h+8pOfpKmpKYccckguvfTSHmNennrqqXzoQx/KTTfdlPr6+hx//PH58pe/nKqqql7V0djYmIaGhixatMhYmH7omnufzcd+enfa2oupKCRH77Vlph382kxoGFru0gAAAAD6JXkXrJt+Haz3Fz5o+r/Hnl+SC/4wK7+/f26SpLaqIiccsE0+/C/bpaGuuszVAQAAAPQv8i5YN4L1XvBBM3DcOfulfPl3D+f2JxckSRqGVufkg7bNe6dskyHVlWWuDgAAAKB/kHfBuhGs94IPmoGlWCzmTw/Pz1eueziPzFuSJNli5NBMe8trc+SeW6SyYvUjhgAAAAAGC3kXrBvBei/4oBmY2tqL+dWdT+cb1z+S5xatSJLsOH54Pvm2HfOm126+ygz/tvZiXlzalPmNTXl+cVPmL17R+bMpLyxpymvHDc+JB07O8CFGywAAAAADm7wL1o1gvRd80AxsK1racvnfn8ylf340jStakyT7TR6VyWPqM78zQJ/f2JQXlzanrX3Nm8Po+pqcOnX7HLvvVqmurNgY5QMAAACsd/IuWDeC9V7wQbNpWLisOZfe9Fgu//uTaW5tX+06hUJHeL758CEZO7y24zSiNg1Dq/PTO+bk8eeXJkm23bw+Z71tp7x5p7GrdL6/GrPmLs79zyzKzhNHZIdxw1NhXA0AAACwAcm7YN0I1nvBB82m5ZmFy/PzO+akUEjGdgXoI2ozdviQjBlWk6pX6ERvaWvPT2+fnW/c8M8sWNqcJJnymtH59GE75XVbNPS5jheXNOU3dz+bX9/1dO5/prG0fFR9Td7wmlGZ8prRmbLt6Gy7+bD1Et4DAAAAdJF3wboRrPeCDxq6a1zRkm/f9Fi+97cnSp3vR+25RU4/ZIdMHDl0jbdtam3Lnx6an1/d+UxumjU/rZ2jZ6orC9l5YkMembs4y1vaetxm8+G1ecNrRpeC9m1G1wnaAQAAgHUi74J1I1jvBR80rM7TLy3LBX+YlavufjZJUltVkQ/8v8n573/ZtscBTovFYu55elF+NfPpXH3vs1m4rKV03W5bNuTovbbMEbtPzKj6mjS3tufepxfm74+9mBmPvZiZs19aZWzNhIYhpZD97XtskZoqs94BAACAvpF3wboRrPeCDxrW5N6nF+aL1z6U259YkCQZM6wmp059bf7ltZvnt/c8m1/f+XQe65zNniTjRwzJkXtukaP32iLbjxu+xvte0dKWu2YvzIzHX8ytj72Yu+a8lJa2lZvsIbuMy/+8Z+8N88QAAACATZa8C9aNYL0XfNCwNsViMdc/OC9f/v3DefyFpatcP6S6Im973YQctdcW2X/bMal8lQcnXd7clplPvZS/P/ZCvvOXx9PaXsz3T9gnB+04dl2fAgAAADCIyLtg3VSVuwDYFBQKhRy8y/gctOPYXHHb7Fx4wyN5aVlL3vCaUTlqry1z6K4TMqx23Te3oTWVOXD7MTlw+zFpbS/mO395PJ+7+oFM2XZ0hlRXrodnAgAAAACsjY71XrAHj75qam3L8ua2jKyr2WCPsaSpNf96wU2Zv7gpH3/La/ORN2+/wR4LAAAA2LTIu2DdOOohbAC1VZUbNFRPkmG1Vfn0YTslSb5106N5+qVlG/TxAAAAAIAOgnUYwP5t94nZb/KorGhpzxeveajc5QAAAADAoCBYhwGsUCjkC29/XSorCrnugbn5yyPPl7skAAAAANjkCdZhgNth/PAcP2WbJMnnfvtAmlrbylsQAAAAAGziBOuwCTj1LdtnzLDaPP7C0nzvb0+UuxwAAAAA2KQJ1mETMGJIdT516I5JkotvfDTPLlxe5ooAAAAAYNMlWIdNxDv23CL7bLNZlre05Uu/cyBTAAAAANhQBOuwiSgUCvn8v70uFYXk2nufyy2PvlDukgAAAABgkyRYh03IzhNH5D1v2DpJ8tnfPpDm1vYyVwQAAAAAmx7BOmxiph28Q0bX1+TR+Uty+d8dyBQAAAAA1jfBOmxiGoZW55Nv6ziQ6Tdv+GfmNa4oc0UAAAAAsGkRrMMm6J17bZk9txqZpc1tOc+BTAEAAABgvRKswyaooqKQc9/+uhQKyW/ufja3Pv5iuUsCAAAAgE2GYB02Ua/boiHH7bdVkuSzv3kgLW0OZAoAAAAA64NgHTZhpx+8Qzarq86seYvzwxlPlbscAAAAANgkCNZhEzayriafeGvHgUwvvP6RzF/sQKYAAAAAsK4E67CJe9fek7L7lg1Z3NSaT/36vixrbi13SQAAAAAwoAnWYRNXUVHIF97+ulRWFHLDQ/Nz2EV/y12zXyp3WQAAAAAwYAnWYRDYfdLI/N/7982EhiF54oWleedlM/KN6x9xQFMAAAAAeBUE6zBI7L/dmFz3sTfm7XtMTFt7Md+88Z9552Uz8vjzS8pdGgAAAAAMKIJ1GEQa6qrzzWP3zEX/sWdGDKnKPXMW5tCL/pr/u/WpFIvFcpcHAAAAAAOCYB0GoX/bfWL+cNobc8B2o7OipT1nX3V/3nf5HZm/eEW5SwMAAACAfk+wDoPUhIah+b/375dzDt85NVUVuWnW8znkG3/Jdfc/V+7SAAAAAKBfE6zDIFZRUcj7D5ycaz9yYHaeMCIvLWvJf//ozpz+i3uyeEVLucsDAAAAgH5JsA5k+3HDc9XJB+TDb9o2hULyy5lP523f/GtuefQFs9cBAAAA4GUKRanZWjU2NqahoSGLFi3KiBEjyl0ObFB3PLkgp/3s7jz90vIkyVaj6nLYbhNy+G4TsvOEESkUCmWuEAAAAFhX8i5YN4L1XvBBw2CzeEVLvvz7h/PrO5/J8pa20vLXjKnP4btNyGG7TcwO44eXsUIAAABgXci7YN0I1nvBBw2D1bLm1vzp4fm55p7n8udZ89PU2l66bvuxw3L4bhNz+O4Tsu3mw8pYJQAAANBX8i5YN4L1XvBBA8mSptbc8OC8XHPvc/nLI8+nuW1lyL7ThBE5fLcJOWSX8Zk8pj6VFcbFAAAAQH8m74J1I1jvBR800NOi5S25/sF5uebeZ/O3f76Q1vaVHyPVlYVsuVldJo2qy1ajhmbrUfWd5+uy1ei6DKutKmPlAAAAQCLvgnUlWO8FHzTwyhYua84fHpiba+59Lrc9vqBHJ/vqjK6vKQXtW4+uy3Zjh2X7scPzms3rM6S6ciNVDQAAAIObvAvWjWC9F3zQQO+0tRczt3FFZr+4LHMWLMtTC5Zm9oLlmf3i0sxesCwvLWt5xdtWFJJtxtTntWOH57Xjh+e144blteOGZ/KY+lRXVmzEZwEAAACbPnkXrBvBei/4oIH1o3FFSyl0n71gWZ58cWn+OW9JHpm3OI0rWld7m+rKQiaPqc/244Znh3HDM37EkIwYWpXhQ6ozYkh1hg+pyoihHT8F8AAAANA78i5YN4L1XvBBAxtWsVjMvMamPDJvcR6Ztzj/nLcks+Ytzj/nLc7S5rZe38+Q6oqXhe3VGVJVkfZix2O0F4spJqXLxWLS3rm8vZikmIwYWpX3HTA5B2w3ZoM9XwAAACg3eResG8F6L/iggfIoFot5dtGKPDK3M3CfvyQvLmlK44rWLF7RksblHT/7Er731gHbjc7pB++QPbfabL3fNwAAAJSbvAvWjWC9F3zQQP/W2taeJU2tWbyiNYuWt2RxV/C+ojVNrW2pLBRSKCSFQiEVhUIKSSoqkopCIUnHz4rOdW5/YkF+fNtTaWnr+Gg8eOdx+fjBO2SH8cPL+AwBAABg/ZJ3wboRrPeCDxoYXOYsWJZv3vjP/PrOp9NeTAqF5B17bJHT3vLaTBpVV+7yAAAAYJ1tSnlXW1tbWlpayl0GA1x1dXUqKyt7vb5gvRc2pQ8aoPf+OW9xvn79I/n9/XOTdBxI9T/23SqnHLRdxo4YUubqAAAA4NXbFPKuYrGYuXPnZuHCheUuhU3EyJEjM378+BQ6pxysiWC9FzaFDxrg1bv36YX56h9m5a//fCFJx0FS33fA5Pz3G7dNQ111masDAACAvtsU8q7nnnsuCxcuzNixY1NXV9erMBRWp1gsZtmyZZk/f35GjhyZCRMmrPU2gvVe2BQ+aIB1N+OxF3P+Hx7OXbMXJkmGD6nKf//LtvnPN2ydhqECdgAAAAaOgZ53tbW15ZFHHsnYsWMzevTocpfDJuLFF1/M/Pnz89rXvnatY2EqNlJNAAPelG1H59cf2j//33v3zo7jh2fxitZ89Q+zst95N+T0X9yTmU8tiH2VAAAAsOF1zVSvq3MsNNafrvdTb2b2V23oYgA2JYVCIVN3Hpd/3XFsrr732Vz658cya97i/HLm0/nlzKfz2nHD8h/7bpV37LlFRtbVlLtcAAAA2KQZ/8L61Jf3k2Ad4FWoqCjk7XtskX/bfWLumrMwP7ltdq6+99k8Mm9JPn/1g5n++4dz2K4T8h/7bpV9ttnMP/QAAAAAmxCjYADWQaFQyF5bbZav/vvuuf3TU3Pu23fJThNGpLm1PVfe9UyO+Z8Zmfr1m/P//fXxLFjaXO5yAQAAgE3MNttskwsvvLDX6990000pFApZuHDhBqspSS6//PKMHDlygz5GOelYB1hPRgypznumbJP/fMPWuffpRfnJ7bPz23uezWPPL80Xr30o5183K2993fjs95pRGTt8SDYfXpuxw2szZlhtaqrs5wQAAIDB4E1velP22GOPPoXha3LHHXekvr6+1+vvv//+ee6559LQ0LBeHn+wEqwDrGeFQiG7TxqZ3SeNzKcP2ylX3/NcfnL77Nz3zKL89p5n89t7nl3lNpvVVXcG7SsD9807TyOGVKeqspCqiopUVxZSVVmRqorCKsuqKwqprCikvrYqQ6rXfORqAAAAoP8qFotpa2tLVdXa49vNN9+8T/ddU1OT8ePHv9rS6CRYB9iAhg+pzrv32yrv3m+r3P/Molx51zN56sVleX7xijy/uCnPL2lKS1sxLy1ryUvLWvLIvCXr5XHHDq/NNqPrs/Xoumwzpr7H+WG1PvoBAACgHE444YTcfPPNufnmm/PNb34zSfLEE0/kySefzEEHHZTf/e53+cxnPpP77rsvf/zjHzNp0qRMmzYtt956a5YuXZqddtop06dPz9SpU0v3uc022+TUU0/NqaeemqSj4e+73/1urr322vzhD3/IFltska997Wv5t3/7tyQdo2AOOuigvPTSSxk5cmQuv/zynHrqqfnZz36WU089NXPmzMmBBx6Y73//+5kwYUKSpLW1NdOmTcsPf/jDVFZW5gMf+EDmzp2bRYsW5aqrrur18//2t7+dCy64IHPmzMnkyZPzmc98Ju95z3uSdOxM+PznP5///d//zbx58zJ69Oi8853vzEUXXZQkufTSS/ONb3wjc+bMSUNDQ/7f//t/+eUvf7muv5JXTboCsJG8bouGvG6Lnl+zam8vZtHylsxf3JTnFzdl/uIV3c43ZX7jiixtbk1rWzGt7cW0trWnpa2YtvZiWts7zre2tXdc196xPEnHbRc35fYnF6xSx5hhtdlmdF22Hl2fbUbXZYvNhqa1rZhlza1Z1tKW5c1tWdbc1nG58/zybpeXt7SlurIi9TWVGVpTmbqaqtTVVHaeVnO+tipbjBySXSY26KQHAABggykWi1ne0laWxx5aXZlCobDW9b75zW/mkUceyete97p84QtfSNLRcf7kk08mSc4888xccMEFec1rXpPNNtssc+bMyaGHHpovfelLqa2tzQ9/+MMcccQRmTVrVrbaaqtXfJzPf/7zOf/88/PVr341F198cY477rg89dRTGTVq1GrXX7ZsWS644IL83//9XyoqKvKf//mfOf300/PjH/84SfKVr3wlP/7xj/P9738/O+20U775zW/mqquuykEHHdTr1+jKK6/Mxz72sVx44YWZOnVqrrnmmrzvfe/LlltumYMOOii/+tWv8o1vfCM//elPs8suu2Tu3Lm55557kiT/+Mc/8tGPfjT/93//l/333z8LFizIX//6114/9oYgWAcoo4qKQjarr8lm9TXZYfzwdb6/9vZiFq9ozZMvLs2TLy7NUy8u6zj/Qsf5F5c254UlTXlhSVP+8dRL6+EZ9F5VRSE7TxyRPSeNzB5bjcyekzbL1qPrevUfDwAAAFib5S1t2fmcP5TlsR/8wiGpq1l71NrQ0JCamprU1dWtdhzLF77whbzlLW8pXR41alR233330uVzzz03V155ZX7729/mlFNOecXHOeGEE/If//EfSZLzzjsvF110UW6//fa89a1vXe36LS0tueyyy7LtttsmSU455ZRS8J8kF198cc4666y84x3vSJJccskl+d3vfrfW59vdBRdckBNOOCEf/vCHk6TUiX/BBRfkoIMOyuzZszN+/PhMnTo11dXV2WqrrbLvvvsmSWbPnp36+vocfvjhGT58eLbeeuvsueeefXr89U2wDrAJqagopKGuOrvXdcx4f7nGFS156oVlnaH70jzxwrLMbVyemsqK1NVUdXag9+w871o2tLoq9bWVGVpdmea29ixvbsvS5rYs79bZvqy5NUubOjvcW9qyrKk1S5tb8+j8pXlhSVPufXpR7n16UX4w46kkHbPl95g0MntutVn23Kqj5hFDqjfyqwYAAAD9w957793j8pIlS/K5z30u1157bZ577rm0trZm+fLlmT179hrvZ7fddiudr6+vz4gRIzJ//vxXXL+urq4UqifJhAkTSusvWrQo8+bNK4XcSVJZWZnXv/71aW9v7/Vze+ihh3LSSSf1WHbAAQeURuL8+7//ey688MK85jWvyVvf+tYceuihOeKII1JVVZW3vOUt2XrrrUvXvfWtb8073vGO1NXV9frx1zfBOsAgMmJIdXbdsiG7brlxj/xdLBbzzMLluWv2wtw1e2HunvNS7n+mMS8ta8mfZz2fP896vrTudmOHZdctGrL16LpsPbouW42qy6RRddl8WK3udgAAAF7R0OrKPPiFQ8r22OtDfX19j8unn356rr/++lxwwQXZbrvtMnTo0Lzzne9Mc3PzGu+nurpn01qhUFhjCL669YvFYh+rXzeTJk3KrFmzcsMNN+T666/Phz/84Xz1q1/NzTffnOHDh+fOO+/MTTfdlD/+8Y8555xz8rnPfS533HFHRo4cuVHr7CJYB2CDKxQK2XKzumy5WV2O2H1ikqSptS0PPbc4d89+KXfNWZi75yzMUy8uy6Pzl+TR+asexHVodWW2GlWXrTrD9u7nt9xsaGqrzG8HAAAYzAqFQq/GsZRbTU1N2tp6Nwv+lltuyQknnFAawbJkyZLSPPaNpaGhIePGjcsdd9yRN77xjUmStra23Hnnndljjz16fT877bRTbrnllhx//PGlZbfcckt23nnn0uWhQ4fmiCOOyBFHHJGTTz45O+64Y+67777stddeqaqqytSpUzN16tR89rOfzciRI/OnP/0pRx111Hp7rn3R/99pAGySaqsqs8ekkdlj0sic0LnsxSVNuefphXnoucWZs2BZnnpxWWYvWJbnFi3P8pa2zJq3OLPmLV7t/Q2trsywIVUZPqQqw4dUZ0TX+drqHsuHD6nKiCFVGVpTlaHVHaNthtZUZEjpfGWGVFWmomLt3fHFYjEtbcU0t7WnqaWt82d7mtvaU1FIRtbVZOTQ6lRVVqy/Fw4AAIABbZtttsltt92WJ598MsOGDXvFA4omyfbbb59f//rXOeKII1IoFHL22Wf3afzK+vKRj3wk06dPz3bbbZcdd9wxF198cV566aU+fbP8jDPOyDHHHJM999wzU6dOzdVXX51f//rXueGGG5Ikl19+edra2rLffvulrq4uP/rRjzJ06NBsvfXWueaaa/L444/njW98YzbbbLP87ne/S3t7e3bYYYcN9ZTXSrAOQL8xelht/nXHcfnXHcf1WN7c2p5nFi7P7AXLMvvFpZndLXSfvWBZljW3ZXlLx+n5xU3rpZbaqooMrakshe9J0tTa3nlqS3NrR4Dem2/GDR9Slc3qarJZXXVG1tVkVH1NRtZV91g2sq46Q6srM6R0quhxubIXQT8AAAD93+mnn57jjz8+O++8c5YvX54nnnjiFdf9+te/nve///3Zf//9M2bMmHzyk59MY2PjRqy2wyc/+cnMnTs3733ve1NZWZmTTjophxxySCore//t8SOPPDLf/OY3c8EFF+RjH/tYJk+enO9///t505velCQZOXJkvvzlL2fatGlpa2vLrrvumquvvjqjR4/OyJEj8+tf/zqf+9znsmLFimy//fb5yU9+kl122WUDPeO1KxQ39rCcAaixsTENDQ1ZtGhRRowYUe5yAOimWCxm0fKWLF7RmsYVHT87Ti09fjauaM2SppWXlze3ZUXLykB+eXNbmlrXba9/dWUhNZUVqa2uTGtbexpXtK6nZ5nO+10Ztg+trsyo+ppsPrw2mw+vzdjOn6XTsNpsVlfTq857AABg8BnoedeKFSvyxBNPZPLkyRkyZEi5yxl02tvbs9NOO+WYY47JueeeW+5y1pu+vK90rAMwoBUKhc6O75p1vq+29mKaWjtC9uUtHcH7suaOy4VCITVVFamtquj5s7IytdUVqamsWCXEbm1rz6LlLXlpWUsWLmvOS8ta8tKy5ry0tLnbso7zjctbSo+5vLktK1rb09wt6G9u6+iQX9yHsL6yopAxw2pKQXtdbVUqC4VUFJKKikIqC4VUVhR6nu+8vrKikOrKjq79riC/1EXfrZO/dF3nOJ3qiorO2xYcbBYAAGAT8dRTT+WPf/xj/uVf/iVNTU255JJL8sQTT+Td7353uUsrG8E6AHSqrOg40M36OthNVWVFRg+rzehhta/q9l1B/4qW9h6he1NrR+C/YGlz5jc25fklTXl+cbfTkqYsWNqctvZi5jU2ZV7j+hmP01eVFYVUdQb0VZUd56sqOs5XV1aUrquuqkhN57KaqoqOn5UVqa7sfn3H5dqqjoB/SHVlaqsrM6Sq83znz64xOkOqO2bl11RVpKIipR0HXaeKzstVFXYAAAAArE1FRUUuv/zynH766SkWi3nd616XG264ITvttFO5SysbwToA9FMrg/6+37alrT0vLmnuDNpX5PnFTVne3Ja2YtLeXkx7sZi2YjHt7cW0tWfl+WLHde3txTS3tpdC/a5gf0W30Tml6zo7/F+urb3YuXNg4x9Ypy8KhZXBe1VF1zcTOkL5Vb+lUNnt2wod43mqunXpV1ZUdP5cufOg63ypkz8dQX4xHdP4ug/l6zrbtayYYorFjpFHbe3FtBfT8fspdvzeun5X3Ze3F9d+v90fv5CO5zykuiJDqlad8V/bbUdF106LYjreR63txbS1t6etPWltby/9zts6r+tap5h0fFOiUEih82fXtyMKPZatXKfrde36vVR0/qzs+oZFZaHH763SThIAANhgJk2alFtuuaXcZfQrgnUA2ARVV1ZkfMOQjG8YkqRhgz9esdgRoLe2F9PWVkxLe3ta24pp7fazpa0jcG1p61ivpa1jWUtre1o6R920tHUE+h3XdS5rLaa5rfOAsZ1h/4rWrqC/vdTVv6KlY05+1w6Aps7124rFNR5ktlhMWosdAXBTkqXNbUlaNvhrxvpX8bJAvnvo/vLLXd+eqKqsSHVFocc3Kao6vyFR+oZFRUXai8WsaG1PU+d7q6m182dLt/Pdri8WOw6CXFvdsaOmtnPHxSv9rK4qpFhM2jt3pHTsJFm5I6xr50mx206U0g6Kis4RTt3Pd+5o6Nr50LXPoWtHTMd28bIdNi+7rpCVr0vpGySd30CpWc35rm+DdOwo6dhpk66dJUkqKjqWFbp2qKRjp0+x286glTuTVl7XtbxL186pzrsv3VfH/a58jC5dO3u6dkh1f+7dl3ftlFrdt2mqKleeX3nqeK+s8jzSfSdWsduOss7Pm87Pw9a29pWfg2s4X0hh5Tdxur6x021HV2l55/upurIirW0rP2vbSo/X7bO587quz+iub/F0jQYrjQzr9g2fjvfXyp1fvVV6z77CDri2Yse/G62dv4vW9o7fT9e2XNW5fZYud/sGVNflyopC5+80nb/Pjte9+87il7/Pu79fup5vRbf3Ztf21bXjb3WfKd0/Tyoru18ulN57rV3Ps/PfwtLz7vx3sPvlZGVNpfPd3tNdL3vH67/y99C1Haw8v3JbefnvqvtnSMdDrvr50v1nj9uu8rtd9R/Xl79uXe+Zrtf25TtRu2+/XZ8VedlzLnTdPoUU03Nncjo/N7sur3x+K7fpl99H6XXt9rsuvb6dG3TX9lr6TOr2nEvbdLH776Pbc+j2vur+Gdi9hj7p/ExdObKva9tc+XtfnWLn9tTaVuz8P1bnqbXjs6D7+a7fXemztPvr1O1893WSVbeZrnIqKnr+Xotr+B0Vi923z1X/revxPu3cfovFjn8Vuq4vFosr/60vdH0uVKzyjcXuO+srXvZ76v6e6/57TLfnPqxWhAb05FMBAFhnhUIhQ6p7fzT4ja0rROsKbtrau4U8nX/ItRU7wqbmroC0tet8W2lZc2tH2N/U0tb5s2c41fEHbHvpD9me13X87G7l38OFVZZ1D1C6h6RdgUTXH4UvD1a7/5HdPVx5pftuLybNre3dvpXQtbOi5xiiFS0dz3tFa1sK6RYqvbxzvMflitIf111/KBdXE+J0/THd9cd11zcoune9t3ULQl9Je7HjeARZ9QsUZdHc1p7F5ZnEBMAgUNrx020nRpJSmM76M6y2Kvd//pBylwH0M4MqWP/Wt76Vr371q5k7d2523333XHzxxdl3333LXRYAsIEVOruXqvpv9k8vdYXvre3tq3bAdu1Aaeu5I2Xl5fa0F4s9vz3R7RsVrd06e7t3Fbe2d3T0dnWgd3UG13aOBupY3vN8IR07LFa0dt8psfJn17cqVnbAt6/cUfKyzs6uHSvdr0+hkPRiRFD37uyeY3i6d+m+vMu9Y3nXTo2uzunWtvY0d+u07n6+pXOnUlfXfVd34cquwp7d5107VVbXFZl074Ds2dHao5M9PS8nq45RStKj67rjZ1bTid3xM0mpq7ul2/Ps+mZNS3u3851d5V26d62+8s6sleOhaipXfjuiexf8yzvii8VimlpWvpdWdNvZtaK1reO6rmWtHdvDqse46Lq/Qukg012PXVlRKHWCruzgL/YcEdatE7yrC3j1Vt892/FtktXveFvdt0qSld80aOnsuu++Xbe2t5e63Lu6vVe707Fb2FjZ2YFaWbFyJFepgzYrO7lX97NrO1rZed79M2VNn1irenm3+8pve3Rcv/I9nXT1Sa/2fd/tGxJdZ7p3VXe/rmu7K3Xnd+sWf8XLWX039MsXdb9c+rZJsefnUvdvC3TvVm7rfGLdu8JX1yn/SlZ+pq1ae/fu6a6XcpXfdddr+rLH7t4p372Du9uXZXp8ZnW/v+6P05fn8mp1vT/bUuzVTuXql33zpqayY7vsuq+u31/3jv324sr3Y/Flr2P316/r997zm0cdy1b9Nki38XDd/h3q3tXf9W2rNY+OW7ntdH0+dW2fqxtN19Zt23153WvTx+8ZAIPEoAnWf/azn2XatGm57LLLst9+++XCCy/MIYccklmzZmXs2LHlLg8AgF7oCn4rK7r2kthbAt0Vi0XHG9iIun8jqvvP9mKxYydGaXSNY0H01epC25fvCBgIXh7e97XqYlaGxu3FzvNdO8FeYcfXyuC8osdIr4HympXL6na0dN9RC/BygyZY//rXv54PfvCDed/73pckueyyy3Lttdfmf//3f3PmmWf2WLepqSlNTSu/t9vY2LhRa92o/nh2MvPyjvOr/GvR7fLq/iUpVKwc/JdCt8sVq14uFjvur9jeueu6/WWXV7NsQ+vLfyo21r+k66Um/+pvUjb5/8Wtz+fXlz8U+uvruj7+2Omvz61MNoVtqK9/BK/P5+wP8A6bwvuIQcNWu3EV0vFH9aD5w3oj6vwrc8Dr/jxe7a7g6vVUC2u2xt9V7fDkk09s3IKAfm9Q/Pvf3NycmTNn5qyzziotq6ioyNSpUzNjxoxV1p8+fXo+//nPb8wSy6e1KWnahHccAAAAAKyL9tZyVwD0Q4MiWH/hhRfS1taWcePG9Vg+bty4PPzww6usf9ZZZ2XatGmly42NjZk0adIGr7Ms/uUTyX7/1XPZarvTXr6suLLTPK/Ucd51vj1r7WgvVHQ+TMXKDvh+pZg11rQ+OvpeTTdcnx93Q7+ur6ajb4D9rtfL/ffVRniNNmRXatewStZsvX4GDJTXu59uD6zFhv6cfDX6Y02D1ED/vO9v/2a92m9L9Lfn0J/qAXhVfI7R/2yzzTY59dRTc+qppybpGJF15ZVX5sgjj1zt+k8++WQmT56cu+66K3vsscerftz1dT9rc8IJJ2ThwoW56qqrNthjrKtBEaz3VW1tbWpra8tdxsZRP6bjBAAAAAAMSM8991w222yz9Xqfqwu3J02alOeeey5jxsgTB0WwPmbMmFRWVmbevHk9ls+bNy/jx48vU1UAAAAAAOtuY2WclZWV8tROFeUuYGOoqanJ61//+tx4442lZe3t7bnxxhszZcqUMlYGAAAAAKwXxWLSvLQ8p16OcPvOd76TiRMnpr29vcfyt7/97Xn/+9+fJHnsscfy9re/PePGjcuwYcOyzz775IYbbljj/RYKhR6d5bfffnv23HPPDBkyJHvvvXfuuuuuHuu3tbXlxBNPzOTJkzN06NDssMMO+eY3v1m6/nOf+1x+8IMf5De/+U0KhUIKhUJuuummPPnkkykUCrn77rtL6958883Zd999U1tbmwkTJuTMM89Ma+vKYxO86U1vykc/+tF84hOfyKhRozJ+/Ph87nOf69Xr1aWpqSkf/ehHM3bs2AwZMiQHHnhg7rjjjtL1L730Uo477rhsvvnmGTp0aLbffvt8//vfT9Jx/M1TTjklEyZMyJAhQ7L11ltn+vTpfXr81RkUHetJMm3atBx//PHZe++9s+++++bCCy/M0qVL8773va/cpQEAAAAA66plWXLexPI89qeeTWrq17rav//7v+cjH/lI/vznP+fNb35zkmTBggW57rrr8rvf/S5JsmTJkhx66KH50pe+lNra2vzwhz/MEUcckVmzZmWrrbZa62MsWbIkhx9+eN7ylrfkRz/6UZ544ol87GMf67FOe3t7ttxyy/ziF7/I6NGj8/e//z0nnXRSJkyYkGOOOSann356HnrooTQ2NpYC6lGjRuXZZ5/tcT/PPPNMDj300Jxwwgn54Q9/mIcffjgf/OAHM2TIkB7h+Q9+8INMmzYtt912W2bMmJETTjghBxxwQN7ylres9fkkySc+8Yn86le/yg9+8INsvfXWOf/883PIIYfk0UcfzahRo3L22WfnwQcfzO9///uMGTMmjz76aJYvX54kueiii/Lb3/42P//5z7PVVltlzpw5mTNnTq8ed00GTbD+rne9K88//3zOOeeczJ07N3vssUeuu+66VQ5oCgAAAACwIWy22WZ529veliuuuKIUrP/yl7/MmDFjctBBByVJdt999+y+++6l25x77rm58sor89vf/jannHLKWh/jiiuuSHt7e773ve9lyJAh2WWXXfL000/nQx/6UGmd6urqfP7zny9dnjx5cmbMmJGf//znOeaYYzJs2LAMHTo0TU1Naxz9cumll2bSpEm55JJLUigUsuOOO+bZZ5/NJz/5yZxzzjmpqOgYmLLbbrvls5/9bJJk++23zyWXXJIbb7yxV8H60qVL8+1vfzuXX3553va2tyVJvvvd7+b666/P9773vZxxxhmZPXt29txzz+y9995JOg7u2mX27NnZfvvtc+CBB6ZQKGTrrbde62P2xqAJ1pPklFNO6dWbDwAAAAAYYKrrOjrHy/XYvXTcccflgx/8YC699NLU1tbmxz/+cY499thSCL1kyZJ87nOfy7XXXpvnnnsura2tWb58eWbPnt2r+3/ooYey2267ZciQIaVlqxuH/a1vfSv/+7//m9mzZ2f58uVpbm7OHnvs0evn0fVYU6ZMSaFQKC074IADsmTJkjz99NOlDvvddtutx+0mTJiQ+fPn9+oxHnvssbS0tOSAAw4oLauurs6+++6bhx56KEnyoQ99KEcffXTuvPPOHHzwwTnyyCOz//77J+k4COtb3vKW7LDDDnnrW9+aww8/PAcffHCfnufqDIoZ6wAAAADAJq5Q6BjHUo5Tt2B5bY444ogUi8Vce+21mTNnTv7617/muOOOK11/+umn58orr8x5552Xv/71r7n77ruz6667prm5eb29VD/96U9z+umn58QTT8wf//jH3H333Xnf+963Xh+ju+rq6h6XC4XCKnPm18Xb3va2PPXUUznttNPy7LPP5s1vfnNOP/30JMlee+2VJ554Iueee26WL1+eY445Ju985zvX+TEF6wAAAAAAG8mQIUNy1FFH5cc//nF+8pOfZIcddshee+1Vuv6WW27JCSeckHe84x3ZddddM378+Dz55JO9vv+ddtop9957b1asWFFaduutt/ZY55Zbbsn++++fD3/4w9lzzz2z3Xbb5bHHHuuxTk1NTdra2tb6WDNmzEix28Fbb7nllgwfPjxbbrllr2tek2233TY1NTW55ZZbSstaWlpyxx13ZOeddy4t23zzzXP88cfnRz/6US688MJ85zvfKV03YsSIvOtd78p3v/vd/OxnP8uvfvWrLFiwYJ3qEqwDAAAAAGxExx13XK699tr87//+b49u9aRjBvmvf/3r3H333bnnnnvy7ne/u0/d3e9+97tTKBTywQ9+MA8++GB+97vf5YILLljlMf7xj3/kD3/4Qx555JGcffbZueOOO3qss8022+Tee+/NrFmz8sILL6SlpWWVx/rwhz+cOXPm5CMf+Ugefvjh/OY3v8lnP/vZTJs2rTTaZl3V19fnQx/6UM4444xcd911efDBB/PBD34wy5Yty4knnpgkOeecc/Kb3/wmjz76aB544IFcc8012WmnnZIkX//61/OTn/wkDz/8cB555JH84he/yPjx4zNy5Mh1qkuwDgAAAACwEf3rv/5rRo0alVmzZuXd7353j+u+/vWvZ7PNNsv++++fI444IoccckiPjva1GTZsWK6++urcd9992XPPPfPpT386X/nKV3qs81//9V856qij8q53vSv77bdfXnzxxXz4wx/usc4HP/jB7LDDDtl7772z+eab9+gY77LFFlvkd7/7XW6//fbsvvvu+e///u+ceOKJ+cxnPtOHV2PtvvzlL+foo4/Oe97znuy111559NFH84c//CGbbbZZko7u+rPOOiu77bZb3vjGN6aysjI//elPkyTDhw/P+eefn7333jv77LNPnnzyyfzud79b5+C/UOzep89qNTY2pqGhIYsWLcqIESPKXQ4AAAAArJOBnnetWLEiTzzxRCZPntzjIJ2wLvryvtKxDgAAAAAAfSBYBwAAAACAPhCsAwAAAABAHwjWAQAAAACgDwTrAAAAAMCAVCwWy10Cm5C+vJ8E6wAAAADAgFJdXZ0kWbZsWZkrYVPS9X7qen+tSdWGLmZT0LWnorGxscyVAAAAAMC668q5BmrHd2VlZUaOHJn58+cnSerq6lIoFMpcFQNVsVjMsmXLMn/+/IwcOTKVlZVrvY1gvRcWL16cJJk0aVKZKwEAAACA9Wfx4sVpaGgodxmvyvjx45OkFK7Duho5cmTpfbU2heJA3S21EbW3t+fZZ5/N8OHDN7k9X42NjZk0aVLmzJmTESNGlLscYB3ZpmHTYpuGTYttGjYttmkGumKxmMWLF2fixImpqBjY06Lb2trS0tJS7jIY4Kqrq3vVqd5Fx3ovVFRUZMsttyx3GRvUiBEj/EcANiG2adi02KZh02Kbhk2LbZqBbKB2qr9cZWVlnwJRWB8G9u4oAAAAAADYyATrAAAAAADQB4L1Qa62tjaf/exnU1tbW+5SgPXANg2bFts0bFps07BpsU0DDG4OXgoAAAAAAH2gYx0AAAAAAPpAsA4AAAAAAH0gWAcAAAAAgD4QrAMAAAAAQB8I1gEAAAAAoA8E64Pct771rWyzzTYZMmRI9ttvv9x+++3lLglYi+nTp2efffbJ8OHDM3bs2Bx55JGZNWtWj3VWrFiRk08+OaNHj86wYcNy9NFHZ968eWWqGOiLL3/5yykUCjn11FNLy2zTMLA888wz+c///M+MHj06Q4cOza677pp//OMfpeuLxWLOOeecTJgwIUOHDs3UqVPzz3/+s4wVA6+kra0tZ599diZPnpyhQ4dm2223zbnnnptisVhaxzYNMDgJ1gexn/3sZ5k2bVo++9nP5s4778zuu++eQw45JPPnzy93acAa3HzzzTn55JNz66235vrrr09LS0sOPvjgLF26tLTOaaedlquvvjq/+MUvcvPNN+fZZ5/NUUcdVcaqgd6444478j//8z/Zbbfdeiy3TcPA8dJLL+WAAw5IdXV1fv/73+fBBx/M1772tWy22Waldc4///xcdNFFueyyy3Lbbbelvr4+hxxySFasWFHGyoHV+cpXvpJvf/vbueSSS/LQQw/lK1/5Ss4///xcfPHFpXVs0wCDU6HYfTcrg8p+++2XffbZJ5dcckmSpL29PZMmTcpHPvKRnHnmmWWuDuit559/PmPHjs3NN9+cN77xjVm0aFE233zzXHHFFXnnO9+ZJHn44Yez0047ZcaMGXnDG95Q5oqB1VmyZEn22muvXHrppfniF7+YPfbYIxdeeKFtGgaYM888M7fcckv++te/rvb6YrGYiRMn5uMf/3hOP/30JMmiRYsybty4XH755Tn22GM3ZrnAWhx++OEZN25cvve975WWHX300Rk6dGh+9KMf2aYBBjEd64NUc3NzZs6cmalTp5aWVVRUZOrUqZkxY0YZKwP6atGiRUmSUaNGJUlmzpyZlpaWHtv3jjvumK222sr2Df3YySefnMMOO6zHtpvYpmGg+e1vf5u99947//7v/56xY8dmzz33zHe/+93S9U888UTmzp3bY5tuaGjIfvvtZ5uGfmj//ffPjTfemEceeSRJcs899+Rvf/tb3va2tyWxTQMMZlXlLoDyeOGFF9LW1pZx48b1WD5u3Lg8/PDDZaoK6Kv29vaceuqpOeCAA/K6170uSTJ37tzU1NRk5MiRPdYdN25c5s6dW4YqgbX56U9/mjvvvDN33HHHKtfZpmFgefzxx/Ptb38706ZNy6c+9anccccd+ehHP5qampocf/zxpe12df8Pt01D/3PmmWemsbExO+64YyorK9PW1pYvfelLOe6445LENg0wiAnWAQawk08+Offff3/+9re/lbsU4FWaM2dOPvaxj+X666/PkCFDyl0OsI7a29uz995757zzzkuS7Lnnnrn//vtz2WWX5fjjjy9zdUBf/fznP8+Pf/zjXHHFFdlll11y991359RTT83EiRNt0wCDnFEwg9SYMWNSWVmZefPm9Vg+b968jB8/vkxVAX1xyimn5Jprrsmf//znbLnllqXl48ePT3NzcxYuXNhjfds39E8zZ87M/Pnzs9dee6WqqipVVVW5+eabc9FFF6Wqqirjxo2zTcMAMmHChOy88849lu20006ZPXt2kpS2W/8Ph4HhjDPOyJlnnpljjz02u+66a97znvfktNNOy/Tp05PYpgEGM8H6IFVTU5PXv/71ufHGG0vL2tvbc+ONN2bKlCllrAxYm2KxmFNOOSVXXnll/vSnP2Xy5Mk9rn/961+f6urqHtv3rFmzMnv2bNs39ENvfvObc9999+Xuu+8unfbee+8cd9xxpfO2aRg4DjjggMyaNavHskceeSRbb711kmTy5MkZP358j226sbExt912m20a+qFly5aloqJndFJZWZn29vYktmmAwcwomEFs2rRpOf7447P33ntn3333zYUXXpilS5fmfe97X7lLA9bg5JNPzhVXXJHf/OY3GT58eGl2Y0NDQ4YOHZqGhoaceOKJmTZtWkaNGpURI0bkIx/5SKZMmZI3vOENZa4eeLnhw4eXjpHQpb6+PqNHjy4tt03DwHHaaadl//33z3nnnZdjjjkmt99+e77zne/kO9/5TpKkUCjk1FNPzRe/+MVsv/32mTx5cs4+++xMnDgxRx55ZHmLB1ZxxBFH5Etf+lK22mqr7LLLLrnrrrvy9a9/Pe9///uT2KYBBjPB+iD2rne9K88//3zOOeeczJ07N3vssUeuu+66VQ66AvQv3/72t5Mkb3rTm3os//73v58TTjghSfKNb3wjFRUVOfroo9PU1JRDDjkkl1566UauFFhfbNMwcOyzzz658sorc9ZZZ+ULX/hCJk+enAsvvLB0oMMk+cQnPpGlS5fmpJNOysKFC3PggQfmuuuuc5wF6IcuvvjinH322fnwhz+c+fPnZ+LEifmv//qvnHPOOaV1bNMAg1OhWCwWy10EAAAAAAAMFGasAwAAAABAHwjWAQAAAACgDwTrAAAAAADQB4J1AAAAAADoA8E6AAAAAAD0gWAdAAAAAAD6QLAOAAAAAAB9IFgHAGDAuemmm1IoFLJw4cJylwIAAAxCgnUAAAAAAOgDwToAAAAAAPSBYB0AgD5rb2/P9OnTM3ny5AwdOjS77757fvnLXyZZOabl2muvzW677ZYhQ4bkDW94Q+6///4e9/GrX/0qu+yyS2pra7PNNtvka1/7Wo/rm5qa8slPfjKTJk1KbW1ttttuu3zve9/rsc7MmTOz9957p66uLvvvv39mzZpVuu6ee+7JQQcdlOHDh2fEiBF5/etfn3/84x8b6BUBAAAGE8E6AAB9Nn369Pzwhz/MZZddlgceeCCnnXZa/vM//zM333xzaZ0zzjgjX/va13LHHXdk8803zxFHHJGWlpYkHYH4Mccck2OPPTb33XdfPve5z+Xss8/O5ZdfXrr9e9/73vzkJz/JRRddlIceeij/8z//k2HDhvWo49Of/nS+9rWv5R//+Eeqqqry/ve/v3Tdcccdly233DJ33HFHZs6cmTPPPDPV1dUb9oUBAAAGhUKxWCyWuwgAAAaOpqamjBo1KjfccEOmTJlSWv6BD3wgy5Yty0knnZSDDjooP/3pT/Oud70rSbJgwYJsueWWufzyy3PMMcfkuOOOy/PPP58//vGPpdt/4hOfyLXXXpsHHnggjzzySHbYYYdcf/31mTp16io13HTTTTnooINyww035M1vfnOS5He/+10OO+ywLF++PEOGDMmIESNy8cUX5/jjj9/ArwgAADDY6FgHAKBPHn300SxbtixvectbMmzYsNLphz/8YR577LHSet1D91GjRmWHHXbIQw89lCR56KGHcsABB/S43wMOOCD//Oc/09bWlrvvvjuVlZX5l3/5lzXWsttuu5XOT5gwIUkyf/78JMm0adPygQ98IFOnTs2Xv/zlHrUBAACsC8E6AAB9smTJkiTJtddem7vvvrt0evDBB0tz1tfV0KFDe7Ve99EuhUIhScf89yT53Oc+lwceeCCHHXZY/vSnP2XnnXfOlVdeuV7qAwAABjfBOgAAfbLzzjuntrY2s2fPznbbbdfjNGnSpNJ6t956a+n8Sy+9lEceeSQ77bRTkmSnnXbKLbfc0uN+b7nllrz2ta9NZWVldt1117S3t/eY2f5qvPa1r81pp52WP/7xjznqqKPy/e9/f53uDwAAIEmqyl0AAAADy/Dhw3P66afntNNOS3t7ew488MAsWrQot9xyS0aMGJGtt946SfKFL3who0ePzrhx4/LpT386Y8aMyZFHHpkk+fjHP5599tkn5557bt71rndlxowZueSSS3LppZcmSbbZZpscf/zxef/735+LLroou+++e5566qnMnz8/xxxzzFprXL58ec4444y8853vzOTJk/P000/njjvuyNFHH73BXhcAAGDwEKwDANBn5557bjbffPNMnz49jz/+eEaOHJm99torn/rUp0qjWL785S/nYx/7WP75z39mjz32yNVXX52ampokyV577ZWf//znOeecc3LuuedmwoQJ+cIXvpATTjih9Bjf/va386lPfSof/vCH8+KLL2arrbbKpz71qV7VV1lZmRdffDHvfe97M2/evIwZMyZHHXVUPv/5z6/31wIAABh8CsVisVjuIgAA2HTcdNNNOeigg/LSSy9l5MiR5S4HAABgvTNjHQAAAAAA+kCwDgAAAAAAfWAUDAAAAAAA9IGOdQAAAAAA6APBOgAAAAAA9IFgHQAAAAAA+kCwDgAAAAAAfSBYBwAAAACAPhCsAwAAAABAHwjWAQAAAACgDwTrAAAAAADQB4J1AAAAAADoA8E6AAAAAAD0gWAdAAAAAAD6oKrcBQwE7e3tefbZZzN8+PAUCoVylwMAAAAA66RYLGbx4sWZOHFiKir03kJfCdZ74dlnn82kSZPKXQYAAAAArFdz5szJlltuWe4yYMARrPfC8OHDk3R80IwYMaLM1QAAAADAumlsbMykSZNKuRfQN4L1Xuga/zJixAjBOgAAAACbDGOP4dUxQAkAAAAAAPpAsA4AAAAAAH0gWAcAAAAAgD4QrAMAAAAAQB8I1gEAAAAAoA8E6wAAAAAA0AeCdQAAAAAA6APBOgAAAAAA9IFgHQAAAAAA+kCwDgAAAAAAfSBYBwAAAACAPhiwwfq3vvWtbLPNNhkyZEj222+/3H777Wtc/8ILL8wOO+yQoUOHZtKkSTnttNOyYsWKjVQtAAAAAACbigEZrP/sZz/LtGnT8tnPfjZ33nlndt999xxyyCGZP3/+ate/4oorcuaZZ+azn/1sHnrooXzve9/Lz372s3zqU5/ayJUDAAAAADDQFYrFYrHcRfTVfvvtl3322SeXXHJJkqS9vT2TJk3KRz7ykZx55pmrrH/KKafkoYceyo033lha9vGPfzy33XZb/va3v62yflNTU5qamkqXGxsbM2nSpCxatCgjRozYAM8IAAAAADaexsbGNDQ0yLvgVRpwHevNzc2ZOXNmpk6dWlpWUVGRqVOnZsaMGau9zf7775+ZM2eWxsU8/vjj+d3vfpdDDz10tetPnz49DQ0NpdOkSZPW/xMBAAAAAGBAGnDB+gsvvJC2traMGzeux/Jx48Zl7ty5q73Nu9/97nzhC1/IgQcemOrq6my77bZ505ve9IqjYM4666wsWrSodJozZ856fx4AAED/sbSpNY/OX1LuMgAAGCAGXLD+atx0000577zzcumll+bOO+/Mr3/961x77bU599xzV7t+bW1tRowY0eMEAABsuj7xy3vzlm/cnJlPLSh3KQAADABV5S6gr8aMGZPKysrMmzevx/J58+Zl/Pjxq73N2Wefnfe85z35wAc+kCTZdddds3Tp0px00kn59Kc/nYqKQbF/AQAAWI0VLW254aF5KRaTPz08P6/felS5SwIAoJ8bcIlyTU1NXv/61/c4EGl7e3tuvPHGTJkyZbW3WbZs2SrheWVlZZJkAB67FQAAWI/ufOqlNLW2d55fWN5iAAAYEAZcx3qSTJs2Lccff3z23nvv7LvvvrnwwguzdOnSvO9970uSvPe9780WW2yR6dOnJ0mOOOKIfP3rX8+ee+6Z/fbbL48++mjOPvvsHHHEEaWAHQAAGJz+/tiLpfP3PL0wrW3tqaoccD1IAABsRAMyWH/Xu96V559/Puecc07mzp2bPfbYI9ddd13pgKazZ8/u0aH+mc98JoVCIZ/5zGfyzDPPZPPNN88RRxyRL33pS+V6CgAAQD9xy2MvlM4va27LrHmLs8vEhjJWBABAf1comoWyVo2NjWloaMiiRYscyBQAADYhi1e0ZI8vXJ+29mK2Gzssj85fknPfvkveM2WbcpcGABuUvAvWje83AgAAg9Ztjy9IW3sx24yuy6G7TkiS3Dl7YXmLAgCg3xOsAwAAg1bXGJj9txuTvbYamSS5c/ZLZawIAICBQLAOAAAMWn9/tOPApQdsOyZ7brVZkuSpF5flhSVN5SwLAIB+TrAOAAAMSs8vbsqseYuTJFO2HZ2GodXZfuywJMldxsEAALAGgnUAAGBQ+nvnGJidJ4zIqPqaJMlenV3rM58yDgYAgFcmWAcAAAalrjEw+287urRsr61HJjFnHQCANROsAwAAg9LfH+/oWD9guzGlZV0d6/c+vTAtbe1lqQsAgP5PsA4AAAw6cxYsy5wFy1NVUci+k0eVlm+7+bCMGFKVFS3tefi5xWWsEACA/kywDgAADDq3PNrRrb7HpJGpr60qLa+oKGTPzq5142AAAHglgnUAAGDQueWxzvnq3cbAdHEAUwAA1kawDgAADCrFYjEzHuucr97twKVdHMAUAIC1EawDAACDyqx5i/PCkuYMra4sjX3pbo9JI1MoJE+/tDzzF68oQ4UAAPR3gnUAAGBQueXRjjEw+0welZqqVf8kGj6kOjuMG54kufOphRuzNAAABgjBOgCw0T30XGPe/q1b8pdHni93KcAg9PdHX3kMTJeuTva7jIMBAGA1BOsAwEZ35V3P5J45C/PrO58udynAINPa1p7bnliQJNl/21UPXNplr61GJnEAUwAAVk+wDgBsdI/OX5IkeXFpc5krAQabe55elCVNrWkYWp2dJ454xfX22rqjY/3eZxalubV9Y5UHAMAAIVgHADa6UrC+RLAObFwzHusYAzPlNaNTWVF4xfVeM6Y+I+uq09zangefa9xY5QEAMEAI1gGAjWpFS1vmvLQsSbJAxzqwkXUduPSA7V55vnqSFAqF7NU5Z/1O42AAAHgZwToAsFE9/vzSFIsd5xcsbU6x6wLABraipS0zOw9Guv92rzxfvUvXnPU7HcAUAICXEawDABvVY88vKZ1vbmvP4qbWMlYDDCb/ePKlNLe2Z/yIIXnNmPq1rq9jHQCAVyJYBwA2qq756l0WmLMObCS3dM5X33+70SkUXnm+epfdJ41MRSF5dtGKzF20YkOXBwDAACJYBwA2qkef7xmsv7i0qUyVbHra2otZtLyl3GVAv/X3RzuC9QO2XfsYmCSpr63KjuNHJDEOBgCAngTrAMBG9djLOtZf1LG+3pzxi3vyhvNuzP3PLCp3KdDvLFrekvs6t40DejFfvcteW49MYhwMAAA9CdYBgI2mrb2Yx19YmiTZfuywJMmLSwXr68stj72Q5S1tuezmx8pdCvQ7tz7+YtqLyWs2r8/4hiG9vl1pzrqOdQAAuhGsAwAbzdMvLUtza3tqqiqy25YjkyQLBOvrRXNre+Yv7hir8/v75+bpl5aVuSLoX7rGwOy/7eg+3a4rWL//mcY0tbat97oAABiYBOsAwEbTdeDS14ypz+bDa5MYBbO+zGtckWKx43xbezGX3/JkWeuB/uaWx15M0vv56l22Hl2X0fU1aW5rz/3PNG6I0gAAGIAE6wDARtMVrG83dlhG19ckcfDS9eWZhcuTJJUVhSTJT++Yk8UrHMiU/qm9vZj7nl6UtvbiRnm8+Y0r8uj8JSkUkil97FgvFArZs7Nr/S7jYAAA6CRYBwA2mh7B+rCOYN0omPXjmZc6gvX9Jo/KdmOHZcn/z959x9dV138cf92Rm71ns5o23XuXMgqUsmQLCIIyRFQURXDiwAXqTxFFRVGWqIAIIsgeZRdo6d4jadOkSbP3vuv3x/ece5M0445zc2+Sz/Px6ONekpt7D2matO/zOe9Pj4MnP64I81EJMbhnt1VywR/f50f/2zUqr/eBNq0+NzeJlDib3x/vWWAqwboQQgghhNBIsC6EEEKIUVNSp4L14swE0rSJ9XqpgjFElTaxnp8ayw0nTwHgkfVlOJyucB6WEIP6uEwF1E9srKC8IfT7ANZr/er+1sDo9J71zUeacLtHZ8peCCGEEEJENgnWhRBCCDEq3G43pX0m1jMSVMd6o1TBGKKqRQXruSmxXLI4j7R4G5XNXbyyuzrMRybE8crqOwC1D+CPbx0M6Wu53W7PxPqJ0wIL1hfkJ2Mxm6hp7aGqpdvIwxNCCCGEEGOUBOtCCCGEGBV17T20djswm2BKRrxnYr2xo1cmQA1wtMkbrMdEWfjMCZMBeOC9w/L5FRHnsBasA/xnSyVHGjqGeXRwjjR0UtncRZTFxPKi1ICeI85mZc6kJAC2HJE6GCGEEEIIIcG6EEIIIUaJ3q9ekBZHTJTFE6zbnW5aux3hPLRxQa+CyUuJBeCzJ0zGZjWzvaJZeqFFROnsdVDdqqa+lxSmqKn1N0tC9nrrS1UNzOLCVOJs1oCfZ0lhCiA960IIIYQQQpFgXQghhBCjwlMDk5kAQEyUhYRoFXLJAtPguN1uqppVUKkH65mJ0VyyKA+AB949HLZjE2KgsnrVqZ4SF8UPz58DwDNbKz31MEb7oETVwATar65bMllNu28pbw72kIQQQgghxDggwboQQgghRkVJn351nT613tAuPevBaOq002V3ApCTHON5+w2nqCWmr+6pDmnVhhD+KNO+FqdkxLO4MJXTZmZqXevGT627XG4+0CbWT5qWHtRz6QtM91S10K39eRNCCCGEEBOXBOtCCCGEGBWldSpMK870BuvpCVqwLhPrQdFrYDISoomJsnjePiM7kdUzMnG74ZH1ZWE6OiH60/vVp6THA/D1tTMA+G8Iptb3VrfS1GknzmZhQX5KUM+VnxpLRkI0dqebnZUtxhygEEIIIYQYsyRYF0IIIcSo0CfWi/tMrKf3WWAqAlep96unxh73vhu1qfV/b6qgpcs+qsclxGA8wXqGCtYXFaRwuja1/vs3Dxr6Wh+WqhqYFVPSsFmD+6ePyWRi6eQUQBaYCiGEEEIICdaFEEIIMQrauu2eZYVSBWO8yiZ9cWnMce87eVoGM7MT6ex18sTG8tE+NCGOo0+lF2nBOnin1p/dWukJ3o2wvkSrgQmyX12n18HIAlMhhBBCCCHBuhBCCCFCTq+ByUyMJjk2yvP29IRoQKpggqVXweQmHz+xbjKZPF3rf1tfht3pGtVjE2KggRPrAAsLUlgzKwuXG/6wzpipdbvTxcbDjQCcGGS/uq7vAlO3223IcwohhBBCiLFJgnUhhBBChJxncWmffnXwVsE0tEuwHoyqlqGrYAAuWpRLRkI01a3dvLTz2GgemhD9tHTZPSfS+k6sA3x97XQAnt1WyaG69qBfa3tFMx29TtLibczOSQr6+QDm5yVjNZuoa+vhqHaliBBCCCGEmJgkWBdCCCFEyJXW6f3q/YM0fXmpdKwHR6+CyU0ZPFiPtlq4dtVkAB5475BM2oqw0WtgMhOjSYi29nvfgvwU1s7WptbfLAn6tdaXqH71VVPTMZtNQT8fQEyUhbl5yYDUwQghhBBCTHQSrAshhBAi5IaaWE+LV1Uw9dKxHpTKZtVfnzdEsA5w9QmTibaa2VXZygatHkOI0VbWcHwNTF+3nKG61p/bVuk5IReo9aWqX92oGhjdksIUQBaYiqH9++MKfvHyXnocznAfihBCCCFCSIJ1IYQQQoRcqR6sZyX2e7teBSMT64Hrtjs9JyaGC9bT4m1cujQfgAffOzwqxzZRPb+9il+8vJfWbnu4DyXiHNL2LUxJHzxYn5+fzNrZ2UF1rbvdbv78dqmnX92oxaU67wLTZkOfV4wP//zoCN/+zw7+8s4hfvXK/nAfjhBCCCFCSIJ1IYQQYgJo7OgN2+Rcr8PFkcZOAKZlDehY71MFI/UkgaluUdPqsVEWUuKihn3sDSerJabr9tUY0mEtjudyufneMzv5yzuHOP/377O9ojnchxRRPBPrmYMH6+DtWv/f9irP1S6+sjtd3P7MTv7vlX0AfHH11OO63IOlLzDde6yVrt6xPZHc3uPgP5uPyvcDg7yxp4Y7ntvl+e+H3j/MW/tqw3hEQgghhAglCdaFEEKIca6hvYcTf7mOzzy4ISyvX9bQgdPlJiHaSnZSdL/3pWkT6w6Xm9YuRzgOb8yrbNb71WMwmYbvkS7OTOCMWVm43fDweplaD4VD9e209aiv5fLGTi67/wMelF57j8Nax3rREBPrAPPykjlzjja1/qbvU+ut3XY+97eP+dfHFZhN8KML5nD7J2YHfcwD5SbHkJ0UjcPlZsfRZsOffzTd/ep+vvHUdtb85h0u+dN6/vnREVo65UqLQGyvaOarT2zF5YYrlhVw3YlFAHzzqe3UtnaH9+CEEEIIERISrAshhBDj3KH6DrrtLj4uawpLNYVeA1OcGX9c8BtttZCoLTBs6JCe9UDowXpeapxPj7/hFDW1/vTmozRJBY/htlW0ADA/L5lPzM/B7nRz54t7+fyjmyb859vtdnuC9anDTKwD3HKGf1PrR5s6uezPH/DewXpioyz89bPLuP6kKcEf9CBMJhNLtan1zWN4ganb7ebV3dWe/95a3swPnt3F8rve4MuPbWbd3hrsTlcYj3DsKG/o5IZHP6bL7mT1jEzuvGQe3z13FrMnJdHQ0cut/96GyyUn14QQQojxRoJ1IYQQYpxr7fKG6bsqW0b99fVQrHhADYwuTauDaZjgoWOgqvRgPSXGp8evmprOnElJdNtdPL6xPJSHNiHpE8wrp6Rx31VL+NnF87BZzazbV8snfv+ep/d7Imrs6KWt24HJBIVpw58ImpeXzFlzsnG74fcjdK1vr2jm4vs+4EBNO1mJ0Tz1pVWsnZNt5KEfx9OzfqQ5pK8TSnuPtXGspZuYKDPvfft0fnDebGblJNLrdPHSzmpueHQTq36xjp8+v4fdVaP/s2OsaOro5bpHNlLf3svc3CT+dPUSoixmYqIs/OHTi4mNsrC+pIH73y0N96EKIYQQwmASrAshhBDjXN8p9Z1HwxCs1+mLSwcP1vUFpg3tEqwHorJJq4JJHnpxaV8mk4kbV6tJ3r99UBa27v3xarv2Z2xBQQomk4nPnjCZZ798ElMz4jnW0s2Vf/2QP6w7iHMCTq/q0+q5ybHERFlGfPwtWtf68zuqKKltG/Qxr+6u5oq/fkh9ew+zchJ59isnMS8v2biDHsJiLVjfWt40Zmt+3tqvur9PnpZBQVocnz9lKq98fTUvfu1kbjh5ChkJNurbe3l4/WHO+/37nPO7d3ng3UPUtkmtia7b7uTzf9/EofoO8lJieeS65SRoV2GB+rn3kwvnAvCb1w6wZQxf4SCEEEKI40mwLoQQQoxzfftyd4RxYn1a5hAT6/Gqd71RJtYDUtWiV8H4FqwDnDc/l+ykaOraenh++7FQHdqE0+twsbeqFYCF+d5wd05uEs9/9WQ+uSQPlxt+8/oBPvvQhgnXu6wH61N8XCY6NzeZs+eqqfV715X0e5/b7ebB9w7xpX9uptvu4tQZmTz1pVXkpvj+5yAY8/KSsFnMNHT0Uq4tZx5r1u2tAeD0WVn93j43N5kfnj+HD28/g4euXcZ58ydhs5jZV93GXS/tZdUv3uRvsqMBp8vNrU9uY/ORJpJirPzt+uVkJR1/5dDly/K5YGEuTpebrz2xNSyVbJHC7XZTWtfOkx+X882ntnPBH97n6//ayr8/rqBijP45EkIIMbFZR36IEEIIIcay1m7vUtDRnlh3udQ/osGXifXx3bHe2m3n5se3cs7cHK5aWWjY81Y1q3DWn0DRZjVz7YlF/OqV/Tz43iEuXZI34uJTMbL91W30Ol2kxEUdV3USH23lnk8t4qTiDH743C4+KG3gE79/j3s+tYjVMzLDdMSjy99gHeCWM2bw6u4aXthRxdfWTGN6diIOp4ufPL+Hf3x0BICrVxbykwvnYrWM3sxQtNXC3LwktpY388TGCq5eWUh+auyY+XPU2NHL1opmANYMCNZ1URYzZ8zO5ozZ2bR02nl+RxVPbz7Ktopm7n7tAJcvKyA+euL+c/KuF/fy8q5qbBYzf71mGdOzEwd9nMlk4q5L5rGtoomKxi6+98xO/vDpxWPmayUYdqeL3VWtbCpr5OOyRjaVNR1X+7azsoVnt1UBkJ8ay4nF6awqTmfV1Axykn2rOBNCCCHCZeL+TUgIIYSYIFr6dKyXN3bS1NFLqhZmh1pVSxfddhdRFtOQncrpE6Rj/fXdNbx7oI7D9e2GBesul9u7vNTPSd2rVhTyh3Ul7Ktu44PSBk6almHIMU1k27V+9fl5yUOGZpcuzWdhQQo3P76FfdVtXPPwRm46rZjbzpxB1CgGw+FQ1qCC9SI/gvU5uUmcMzeHV3ZXc++6g/zy0gV89fEtvLW/DpMJvnfubD5/ypSwhJQritLYWt7M/e+Ucv87pWQmRrO4IIXFhaksKUxhfn4ycbbI/OfW2/trcbth9qQkJvlQI5UcF8VnTpjM1SsLWfObdzhc38H/tlfx6RXGnSQcSx56/zAPa1P7d39qISdMTR/28UkxUfz+ysVcfv+HvLDjGKunZ/Kp5QWjcagBaemy873/7mRTWSOpcTZS42ykxdtIjY8iLc5Garz2356320iLs+Fwudha3qwF6U1srWii295/Aa7NamZRQQrLi1KZPSmJ/drPoO0VzRxt6uLfm47y701HAZiaEa9C9uJ0TpiaTkZCdDg+HUIIIcSQIvNvekIIIYQwTN/lpaCmw0ZrQlavgSlKjx9ymjQtfmIE67u05X9Hm7rotjt96pgeSUNHL70OFyYTfk/2pcTZuHBhLk9uquDdA3USrBtguzYBvDA/ZdjHTctK4NmvnMSdL+7hnx+V8+e3S9l4uJGHr1tOcmxU6A80TA7V6RPrwy8uHeiWtdN5ZXc1L+48xr7qNkpq24mJMvO7KxZxzrxJoThUn3xh9VQcLjebyhrZXdVKXVsPr+2p4bU9qmLFYjYxKyeRJYWpLC5MYUlhKpPT4yJiUnndPtWvfsYQ0+pDMZlMXLWikLte2svjG8onZLD+8s5j3PniHgC+e+4sLlyY69PHLS5M5bazZvCrV/bzo//tZsnkFKZlDT7lHk5Hmzr53N8+5kCN+vld0xrc1WQpcVEsm5zG8qJUlhWlMS8viWir9+ff+QvgG0BHj4OPyxr5sLSBDw81sKuyhUP1HRyq7+CxDWrR9szsRNbOyeLSJflMHaJeTgghhBhNEqwLIYQQ45w+sW42gcsdnmB9qBoYwDOB1tgxvqtgdleq7m23G0rr2pmbG/yCxSptWj07MSagaecpmWpyuK5tfH/uR8sOrWppYUHKiI+NibJw58XzObE4g+88vYPNR5r4+wdlfPWM6SE+yvBwudwcaVAdylMy/AvEZk9K4tx5Oby8q5qS2nYyEmw8eO1yFvnweQ6l9IRofnj+HEAtsdxV2cLW8ma2lDexpbyJmtYedle1sruq1VNbkxZv45YzpnPtiUVhO26708W7B+oAWDPbv2Ad1FUXv351PzsrW9hxtJkFI5xIGk82lTVyy5PbcLvhsydM5ourp/r18V9aXcz6knrWlzTw1Se28d8vn2jISVaj7Dzawuce/Zi6th6yk6L55aULsJpNNHb00tTRS2OnXbvV/rujl6ZOdWt3qiW+hWlxLCtKZXmRCtOnZiRgNo98Mik+2sppM7M4bab6mmzpsrPxcCMflNbzYWkD+6rb2F+jft33VinLJqdy2dJ8zlswicSY8XtCUgghRGSTYF0IIYQY5/RFafPzktl+VAUho2WkfnXoM7HePn4n1l0uN7urvP32pXUdhgTreg1MbkpgPbSZ2kmNunHebz8aOnsdHKxtA/ovLh3JJ+ZPorXLznef2cm6fbXjNlivaeumy+7EYjaR78eiXd2tZ87g/ZJ68lJieeCaZRQMUS0VLjFRFpYVpbGsKM3ztqrmLk/QvrW8iV2VrTR29PLj53czMydxxPqQUNlU1kRbt4O0eNuIV1cMJi3exrnzc3huWxWPbyifMMF6aV07n//7JnodLtbOzubHF871++oDs9nEbz+1iHPufY+9x1r55cv7+PGFc0N0xP5Zt7eGmx/fSpfdyaycRB6+brnPuzvcbjcdvU7sDpdhVXPJsVGcOSebM+dkA2oPy/sl9Ty7tZJ3DtSx6UgTm4408ePnd3PuvElcvjSfE6am+xTiCyGEEEaRYF0IIYQY51q61PLSk6dnaMH66C0w9WVifSJUwRxp7KSj1+n5b/3zEqwqT7Duf1AJkJmoBesysR60XZWtuNyQkxRDVpJ/Jzr05ZHbjzZT19bj+X0ZT/TFpYVpcQFdXTEjO5EPbz+DeJslIqpUfJGbEktuSiznLVB1NT0OJz/47y6e2nyUb/x7Oy/dckpYqn/e2q9qYE6bmYklwBDy6pWTeW5bFf/bXsX3zptN0jifGK5r6+G6RzbS3GlnYUEKf/j04oA/d1lJMfzm8oVc/7eP+dsHZZw8LYO1WngcLv/4sIwf/W83LjecMj2DP129xK8pcJPJREK0FUL4rSs9IZqLFuVx0aI8alq7+e/WSp7aVEFpXQf/3VrJf7dWkpcSy6VL87lsST6F6ZF18k0IIcT4NL43JAkhhBDC07F+YnEGJhMca+mmtq17VF5bD5CLh+lC1atgmjp6cbnco3Jcg3G73bjdoXn9XZX9T2aUGhSsexaXBjABDN5gvV4m1oOmXwmywI9pdV1WUgwL8pNxu72h53hTVq9qYIqCCLsSoq1jJlQfTLTVwo8vnMvk9Dgqm7v40XO7wnIc6/aqDvg1fvar97W8KJVpWQl09jp5bmulUYcWkXZVtnDdIxupaOyiMC2Oh65dRqwtuPqW02dlccPJUwD41tPbqW4ZnZ/JA7lcbu56cQ8/fE6F6lcsK+Dh65ZHfLVKdlIMXzq1mDduO5X/fvlErlpZSGKMlcrmLn6/7iCrf/0WV/zlQ57aVEFjmP9uIYQQYnyTiXUhhBBjWnNnL6/truHixXnYrHK+eDB6sD4pOYZpmQkcrG1nV2ULa2YFVh/iK9W9ql57qtblPZjUePUPeIfLTWu3nZQ4Yy4j99cvXt7HYx8d4X9fPXnYEwGB0BeX5qXEUtnc5anICVZlU5fneQOhn9Ro6OjF4XQNuWBWjGy7H/3qg1kzK4sdR1t4c28tn1pWYOCRRYbD9epr3t9+9fEmPtrKb69YxOX3f8iz26o4fVYWFy3KG7XXP9LQQWldB1aziVOmB75rw2QycfXKQn7y/B4e21DOZ06YbNhJD5fLTa/TFfbu8ZLadu55fT8v7awGIDUuir9dv9zzfTNY3z5nJh8damB3VStff3Irj33+hICn4APRbXdy65PbeHmX+v/71tkz+fJpxWPq5JXJZGJxYSqLC1O54/w5vLq7mqc3H+X9kno2HG5kw+FG7XGqWiYlNoqUOBspcd77ybFRpMZp9+OiSIqJIjbKQkyUmZgoi/bLTIzVEpaamR6H09NfL8IvIVoiNCFEf/JdQQghxJj2sxf28p8tR6lt6+bmNeOzmzgYTpebth5VBZMUG8X8/GQO1raz42gLa2aF9tJzfVo9LyWWONvQf+WItlpIjLHS1u2goaM3LMG60+XmXxvL6eh18uruar582jRDn19fXHrRolz+9HYph+o7cLrcQYcoVS1aFUxyYMF6WrzNs9S2saPX7woT4bW9ohkIbGIdYO3sbH73xkHeO1hHj8NJtDVyFhoa4XC9vrhU6hmWFKby1TXT+N0bB/nBs7tYVpQW8Mkxf725T10RsbwoLegamk8uzueXL+9jX3UbW8qbWTo5Nejjc7vdfO7Rj9lc1sQfr17CqaO0aLuvyuYu7n3jAE9vPorLrULZixflcduZMwzt9o+2WvjDpxdz/h/e56NDjfzprRK/diy43e6AQ/D69h5u/PsmtpY3Y7OY+fXlC0b1BE8oxERZPFUxVc1d/HdrJf/ZfJRD9R243dDcaae50w7aEuVA2Cxmoj2BuwrbY20WYqyWPm+3EGPt8xhPOK/+O9pqocfhpL3bQVu3g/YeB63ddnW/20Fbj93zvrYeB70Ol4GfJRGMxGgrO39ydrgPQwgRYSRYF0IIMWY5XW7W7VOXtL+0s1qC9UG0dzs895NioliQl8wzWypHpWfdl351XXq8TQXr7b0Uj36Owq7KFlq1z9W28mZDn9vt9i4uPWtuDg+9f5geh4uKxk6KMoae5PdFVbOqDwi0CsZiNpGeEE1dWw+1bT0SrAeoqaOX8kYV1izISwnoOebmJpGdFE1Naw8bDjWyOgyBYijJxHp/N58+jbf317Gtoplv/Hsbj3/+hFGZhtWD9WBqYHTJcVFcsDCXpzcf5fEN5YYE6//bXsXb++sA+MLfN/Hwdcs5aVpG0M/ri/r2Hu57q4THPiqn16nCzLWzs/nm2TOYlZMUktecmpnATy+axzef2s49bxzgwfcPa7Vk4HK7cWm3btTPEs9/a4F/cWYCiwtSWFyYypLJKUzPShzxhG1pXTvXP/Ix5Y2dJMdG8dfPLmVlmBbphkpuSixfOX0aXzl9Gj0OJy1ddk+w3tzZS3OXnZZOO0197jd39dLcaaely0633UWP3Un3gGnxXqeLXqeLtj5/txJCCDGxSbAuhBBizNpa3qSmj4A9x1o52tRJfqpMQ/bVotXAxEZZsFnNLNBqKnYcbQlq2s0X/gTrafE2yho6aewIT9f3+yX1nvtbK5oN/dxUtXTT1GnHajYxe1IiUzLi2VfdRmlde1DBelevk0Zt4Wugy0sBMrVgvU561gO2Q+vQn5IRT3JcYFPAJpOJNbOyeGJjBW/uqx1XwbrT5faceCiSiXUArBYzv7tiEZ/4/Xt8dKiRB98/xBdWF4f0Ndt7HGw4pKox1swOPlgHuGplIU9vPsoLO6q44/w5AX/9g/qe9suX9wF4TjLd8OjHPHr9ipAGv63ddh589xAPvX/Ys2T6hKlpfOvsWYacLBjJpUvy+LC0gf9sOer5me0Lt1v9nC2pbeepzUcBVVOxsCCZxQUqaF9UkOpZEA6w8XAjN/59Ey1ddgrSYvnb9SsMrz6LNNFWC1mJFrISAztx7HS56bY71S+Hy3u/T/jebVdv77J77/cMeHyX3Xs/2mohKcZKQoyVxBgrCdFRJGr31a8oEqK996Ol6lAIISKWBOtCCCHGrIFL/tbtreXaE4vCczARqrVb/SNdv+R/zqQkLGYT9e09VLd2MynAChFflNSNvLhUl96n6zsc1vcJ1uvaeqhs7jLsJI2+uHR6diLRVgvTshLYV91GSW07Z8wOvI5HX1yaEG0lKSbwv9JlJkbDMfX/LQKzI8gaGN2aWdk8sbGCdftq+NEFc8ZU1/FwKpu6sDvd2KzmgGuLxqOijHjuOH8O331mJ79+dT8nT8tkTm5oJqMB3j9YT6/TxeT0OKYGebWMbnFBCrNyEtlX3cZ/thzlc9pCzkD89d1DHGvpJi8llpduOYWvPbGVdw7Ucf3fPuYfN6xg6eQ0Q45Z12138vcPy/jT26Wek/QL8pP51tkzOXlaxqj9+TOZTPz6sgXcvGYaTpcbswnMJhNmkwmTCcxmEyb0t6nHm01gd6qrobaUN7G1vJntFc209zhYX9LA+pIGz/NPyYhncUEKuSmx/PXdQ/Q6XSwqSOHBa5cZ1hc/nlnMJuKjrcRLt7YQQohByE8HIYQQY9ab+9Tl4vPykthV2crre2okWB9An35LilU/8mOiLMzITmTvsVZ2HG0JabBe6mcVDEBD++gH6129TjaVNQGQkWCjvr2XreXNhgXru6tUv/o8LTDTPx/6RH+gqpq9i0uDCYAyE1WwIsF64PTFpQvyU4J6npOmpWOzmqlo7KKktp3p2YkGHF34HW7oAKAoPS4sy/8i2RXLC1i3r5bX99Tw9Se38r+bTw7Z0s43teq0NbOyDAuNTSYTV58wmR8+u4vHN5Zz/UlFAT13dUs3979TCsDtn5hFcmwUf/nsUm549GPWlzRw3cMf84/Pr2RRgMuB+3K63Dz5cQW/X3eQ6lZVp1WcGc83z5rJOfNywnJCy2w2MSWAkx05yTGeE7ROl5sDNW2eoH1LeROH6jo4XK9+6c6em83vrlhMrG187XEQQgghwkGuKRJCCDEmHWvpYu+xVkwm+MmF8wD46FCDX5dRTwSterAe4708f0GemqrdcbQ5ZK/b2evwTFT7FKwnqGC9MQwT6x+XNdLrdDEpOYZPzJ8EwDZtAtkIu7WJ9Xna512f4C+tMyZYz00JrhddD9brpQomIG63m+3an6WFQU6sx9msnFSsKi/e2Fs7wqPHjsN1er+6MVPS44nJZOKXn5xPRkI0B2ra+b9X9oXkdVwuN29p3eVnGLy4+uJFucTZLJTUtrPxcGNAz/GrV/fRZXeybHIq52nfh2OiLDx4zXJWTkmjrcfBNQ9t8FwBFKhdlS1c8qf1fO+/O6luVdPxv75sAa9+fTXnzp80pq8SsZhNzJ6UxNUrJ3P35Qt58xunse2OM3nk+uV87YzprJ2dxXfOmcWfrl4qoboQQghhEAnWhRBCjEn6crNFBSksnZzKtKwEHC43b+8fP2GUEfQTDXoVDMD8fD1YD90C00N1ajouLd7Wr991KGnx4Qt39RqYk6dlsLgwBVD9/UbZVaUH68dPrLvd7iE/biSVnmA9uKsO9CoAmVgPTHVrN3VtPVjMJubmBhesA6zRpk/16eLxoKxB71eXYH0w6QnR/PryBQA8sr6Mdw/UGf4au6paqGvrId5mYcUUYytVEmOiuGhRLgCPbyz3++O3VzTzzJZKAH54fv8KpFibhYevW86yyam0djv4zEMb2Hus1e/X6Op18ouX9nLRfevZcbSFxBgrd5w/hze/eSqXLyvAahmf/yxOibNx+swsbjtzBg9eu5ybTisecbmpEEIIIXw3Pv8GIYQQYtx7c58K0NfMVAvYzpyjwqjX94yfMMoIesd6Up9gfaFWV7GzsiWoYHc4nsWlPi5FywjjxLq+uPTk6RksLlCL6nZVtdLrcAX93HVtPdS09mAywawcFaxPyYjHZILWbkdQC0P1YD0vNbhgXapggrO9QuvQz0owZAp0zSz1PW3zkSaawrRzwGiHtBqKKekSrA/l9JlZXLNqMgDffGq74b/3+s/MU6ZnYgvBIsSrVqhjf3lntV/fx91uNz97YQ8An1ySx8JBql7io608cv1yFhWk0Nxp5+oHN3Cgps3n13jnQB1n/e4d/vLuIZwuN+ctmMS6207lcydPIdoqk9tCCCGECJwE60IIIcacHofTM2V8+qz+wfo7++sMCUTHi8Em1mfkJGCzmGnutHO0qSskr6sH68VZvgVp+lT7aAfrjR29ng70E4szmJweR2pcFL0OV0BTkQPt1qbVp2bEexafxURZKND620trO4b82JH07VgPRqY+sS5VMAHRK5WM6H4G9fs5KycRl1sFguNBmR6sy8T6sG4/dzbFmfHUtvVw+zM7DT3x6TkZrf3MNNr8/GQW5CfT63Tx9OYKnz/uxZ3H2HSkidgoC98+e9aQj0uMieLRz61gXl4SjR29XPXAhhHrtOrbe/j6v7Zy7cMbqWjsIjc5hoeuXcZ9Vy0hKym4Ci0hhBBCCJBgXQghxBi08XAjnb1OshKjmasthFyUn0JmYjRtPQ4+OtQQ5iOMHK1dDgCSYrz7yqOtFmZNUksRt4eoZ10PPIp9nFjXg/X6UV5e+kGpOkEzKyeRzMRoTCaTJyA1og7Gs7g0r39FiKcOJoiedaOqYGRiPTg7DFpc2tdarQ7mjb1j/wqcXoeLo02qCkaC9eHF2izce+VirGYTr+yu5unNRw153trWbs/X6WmzMg15zsFctaIQgMc3lONyjXxSoNvu5BcvqU75L51aTE7y8GF3cmwU/7xhJbMnJVHf3sNVD3zkOWnTl9vt5qlNFay95x2e3VaF2QSfO2kKr992qmfRpxBCCCGEESRYF0IIMebok3enz8zydLGazSbWzlaTeFIH4zVYFQzAfC3o3RminnVPFYwPi0vB2/Pd1NnrUyBjFP3Kh5OmZXjetrhQ1cFsNWCBqb5oTz8BpNM/L6W1gQXrTpeb6pZuwICJdS1Yb+t20G13BvVcE43b7fZMrC8IcnFpX2u072XvHKjD7hzbV+CUN3bickO8zeL5WhNDm5eXzG1nzQDgx//bTbnWTx8MfSfJwvxkshJDN6l9wcJcEqKtlDV08qEPJ7gfev8wlc1dTEqO4Qurp/r0GilxNv55wwpmZCdQ06rC9YpG7+focH0HVz+4gW89vYPmTjuzJyXx3y+fxB0XzPFcNSSEEEIIYRQJ1oUQQow5b+nB+oDJO70O5o29NSHrDh9r9CqYgcG63rMeigWmDqeLsgY1RehrsJ4apybWnS6355hHw/t9Fpfq9AWm24wI1vXFpQOWWhZnqsndkaoMhlLf3oPd6cZiNpEVZFiZFGP1dC7L1Lp/yho6ae12EG01MzMn0bDnXZifQnq8jbZuB5vKjFukGw76RHFRRny/pZRiaF9cXcyKojQ6ep3c+u9tOII8ubJOW4R7eohqYHTx0VYuWZwHwGMbjgz72NrWbv70VgkA3zlnll/7CdITonns8ydQnBlPVUs3n37gI440dHDfWyWc/bt3+aC0gZgoM7efO4v/3XzSoL3tQgghhBBGkGBdCCHEmHK4voOyhk6iLCZOnt4/WD+xOIPYKAvHWrrZVRl8P/Z40DpIxzqoPlxQE9VGT4gfaezE7nQTG2UhN9m3aWqb1eypq2kYpZ718oZOKhq7sJpNrJiS5nn7woIUTCY40tBJQxC94y2ddioaVV3L3NwhqmACnFjXu/FzkmKwWoL765zJZPL0rNdLz7pftmsnX+bkJhEV5O9DXxazidO0xcxv7hvbV+Acln51v1nMJn7zqYUkRlvZfKSJP71dGvBz9TicvH9QnUA8Y1boa1CuWqnqYF7bXUNtW/eQj7v7tf109DpZVJDChQtz/X6dzMRoHr/xBIrS4zja1MXpd7/Nr1/dT6/DxSnTM3jt66fyxVOLDf1zKYQQQggxkPxNQwghxJii18CsmJJGwoDLumOiLKyeoSaPX99TPerHFok8E+sx/YP16VkJRFvNtPU4ONwQ+ALNwehh8dTMeMxm3ydU07VwN5gw2x/6tPqSwtR+FQFJMVGebvhgptZ3H1PT6gVpsSTH9f/8689/rKWb9h6H389t1OJSnfSsB0bfUbDQwH51nV5ttW5vreHPPZr07y8SrPunIC2On148F4B71x307IPw18bDjXT0Osnss5MklGZPSmJJYQoOl5unNg3eEb+rsoWntP74Oy6Y49fPib6yk2J4/MYTKEiLxeVWuzp+d8Ui/v65FRSmxwX8/yCEEEII4SsJ1oUQQowpb/XpVx/MmXNyAHhNetYBaO1Woe3AiXWrxewJWYzuWdfrTXytgdHpC0wbR2lifbB+dd1izwLT5oCff7d21cTcScd3b6fE2chIUP+/hwKog6nyLC41pi9Z77ivk4l1v3gXlxrXr647eXoGURYTh+o7AvoaiRSH6yRYD9TFi/L45OI8nC43X318q2dhsT/0k9FrZmYFHGD766qVkwF4YuPxS0zdbjc/e2EPbjdcuDCXJdpOi0DlpsTyn5tO5OeXzGfdbady8eI8qRwSQgghxKiRYF0IIcSY0dHjYMNhtRBtqK7YNbOyMJtgX3Vbv4VmE5W3Y/34pW0LQtSz7llcmulfsJ6uBeujUQXjcrlZr02Anjw9/bj36wtMg5pY1/vV8wafEtWn1gOpg6n0BOsysR4uDqfL83scig7nxJgoVk5RX5t6ODoW6fsWiiRY95vJZOKuS+YzNzeJho5ebvrnZr8WDLvdbs8VD/pC3NFw/oJJJMVYOdrUxbsH6/q979Xd1Ww43Ei01cx3zp1lyOtlJcZw1cpCUrWfIUIIIYQQo0WCdSGEEGPG+yX12J1uJqfHMXWIkCYt3sayItWX/cbeiT213m130utQS+8GLi8F75TtzspmQ1+3tDawifV0bYK7oT30wfqeY600d9pJiLZ6TjD0tUgLSrdVNOMMsIN+V5U2sZ43+DRzMD3rniqYVAnWw+VATTvddheJ0VampIcmNF4zS+9ZH5vBelevk2Mtqmd7qO/ZYnixNgv3f2YpqXFR7Djawg+e3eXzcu7Sug7KGzuxWcz9FjSHWkyUhUuX5gPw2IZyz9t7HE7uemkvAF9cPdWwKishhBBCiHCRYF0IIcSY0bcGZrhLvc+aoxa0vT7B62D0xaVmEyTYBptY1xeYtuJwugx5TbfbTalW/eB3sB6vwt3GjtCHu3q/+glT0wZdbjcjO4E4m4X2Hoen2sYfnb3ejxuq11ifWA/k+SubVVgpE+vhs0PrV5+fnxyyio0ztCnjjYcbae22h+Q1QkmfVk+JiyIlTqaJA1WQFscfPr0Eswme3nyUf350xKeP039mrpya1m+PxGi4Wlti+ua+Wo61qBOBj6wvo6Kxi6zEaL54avGoHo8QQgghRChIsC6EEGJMcLvdvLVfC9aHqIHRnakF6xsON9LSOfbCKKPoQVxiTNSgwd+UjATibRa67E5PGB6s6la1jNNiNjHZzylevWO9fhSqYIbrVwfVQa+feNgWQM/63mOtuN2QlRhNVuLgPeiGTKwbFaxLx7rf9MWlg13xYJTJ6fFMy0rA4XLz7oG6kT8gwhyu12pgQjTRP5GcPD2D75yjqlN+8vweNpU1jvgx6/apk8trRviZGQrTshJZMSUNp8vNkx9XUNfWwx/fLAHg2+fMGvWgXwghhBAiFCRYF0IIMSbsOdZKTWsPsVEWVk5JG/axk9PjmZGdgNPlDeMnIr1ffeDiUp3FbPLUlOjTt8EqrVVB2uS0OGxW//6aoVfBNIa4Cqbb7mTjYRVKDVePsKhA9axvrWjy+zV2azUw84aogQFvsH6koRO7H1cMtPc4PL+3Rk+s10uw7rPtFVq/eggWl/Z1hl4Hs3fsfS/Tg3WpgTHGF1ZP5bwFk3C43Nz02BZqWruHfGxLl51NZep7VziCdfBOrT/5cQW/fnUf7T0O5ucl88nFeWE5HiGEEEIIo43ZYP2+++6jqKiImJgYVq5cycaNG4d87GmnnYbJZDru13nnnTeKRyyEECIY+iXtJ01LJybKMuLjz5Q6GFq7HMDgi0t1C/L0nnVjFpiW1LYBUOxnDQz0rYIJbbC+5UgTPQ4XWYnRw9bVLC5MAWBrABPru7TP57whamAAJiXHEGez4HC5OdLg+6JdfVo9OTaKBIOmPrP6VMH42t88kXXbneyvUV/rC0KwuLQvPRR9a39twH3/4VJWL4tLjWQymfjVpQuYmZ1IXVsPN/1zs2ePxkDvHazD4XJTnBnv99VDRjlnXg5p8TaOtXTz701HAbjjgjkhq04SQgghhBhtYzJYf/LJJ7ntttv40Y9+xJYtW1i4cCFnn302tbWDT/I888wzHDt2zPNr165dWCwWLr/88lE+ciGEEIF6a7+qQRipBkZ35pwcAN7eX0uPwxmy44pkehXMUBPr4A0Fdxw1KFivC2xxKXirYBpC3LGu96ufPC1j2K7+xdrn5kBNG+09Dr9eY1elmlifkzv0NLPJZPL0rPtTB1OpBetGTasDZGhVMN12l9//rxPR7qpWnC43GQnR5CYPXvVjlKWTU0mOjaKp087Wcv+vnggnfWJ9igTrhomPtvKXzy4lKcbKlvJmfvL87kEfp1/hcMbs7NE8vH6irRYu05aYApw3fxLLi4a/4kwIIYQQYiwZk8H6Pffcw4033sj111/PnDlzuP/++4mLi+Phhx8e9PFpaWnk5OR4fr3++uvExcUNGaz39PTQ2tra75cQQojwaero9QRKp8/0LVhfkJdMVmI0Hb1OPixtCOXhRSy9LiQpZphgXZtY33OsdcjJR3/oAfG0TP+D9Qy9CqajF1cIJ3NH6lfXZSXFkJcSi8vtX1VOj8PJQW1yf17e0BPr4D0B4c8C08omvV/duEA31mbxTL/LAtOR6V8PC/OThz05YwSrxcxpMzMBWLdvbNXB6MtLJVg3VlFGPPdeuRiTCR7bUM6TH5f3e7/T5eZtrZM/XDUwuk+vKMRiNmGzmvnuubPCeixCCCGEEEYbc8F6b28vmzdvZu3atZ63mc1m1q5dy4cffujTczz00ENceeWVxMcP/pf8X/ziFyQnJ3t+FRQUGHLsQgghAvPOgTpcbpiVk+jzlK7ZbGLtBK+DaR2hYx1gcnociTFWeh0uDmjVFsEo0TrWA5lYT9Um1l1uaO4KzdLZlk47O7SalpGCdYBFAdTBHKxpx+50kxIXNeJy0eJM9XeRUj8m1o1eXKrL7FMHI4anX+ERysWlfa0Zgz3rrd126rV9CWOmCsbtVr/GgNNnZXHr2hkA/PDZ3WyraPa8b1tFM40dvSTGWFk6OTVMR6hMyYjn8c+v5OkvraIgLS6sxyKEEEIIYbQxF6zX19fjdDrJzu5/WWN2djbV1dUjfvzGjRvZtWsXn//854d8zO23305LS4vnV0VFRdDHLYQQInD6AlJfa2B0es/6G3trQjoBHak8E+vDBOsmk4kF+cb0rLd02j3LL6dm+h+kRVnMnpMAjSGqg/nwUD1utwr+c3yo8NDrYPwJ1vV+9bm5SSNOM+snIEr8mFivCkEVDECmVgdTJwtMR7RdCzEXFIR2canu1BmZWMwm9te0UdHoex9/OOn96pmJ0YbtAgi5134Ad02C+pJwH4lPbj59GmfOyabX6eKmf272fP/Vd5KcOiOTKEv4/7m3cmr6qJ2EEkIIIYQYTeH/m9Yoe+ihh5g/fz4rVqwY8jHR0dEkJSX1+yWEECI8nC437wR4SfuqqenE2SzUtPYYtpxzLPEsL40ZPtTSA49ge9b1cDgnKYbEYepnhpOu96y3h2aBad9+dV8sLlTTntsqmn1e6rmrSl9cOnLo6qmCqW33+flD0bEOMrHuq5YuO4e00HjhKIWFKXE2z+Txm2OkDmbM9au73bD9CXB0wYGXw300PjGbTdzzqYVMzYznWEs3X3lsC3any1MZFO4aGCGEEEKI8W7MBesZGRlYLBZqavpf1l9TU0NOTs6wH9vR0cG//vUvbrjhhlAeohBCCANtLW+iudNOcmyUZ3rYVzFRFk6dobqJJ2IdTIsPVTDg7Vn3p0d8MHqdSSA1MDrvAtPQBOvrS1Tfvi81MKCmzqMsJurbeziqdZuPZHeV2s0yN2/kYL0wLR6L2URHr5Pq1m6fnr+qWT0uLzU0wXq9TKwPS78iIT811vP1OhrWzlYh6VjpWfcE6+ljJFhvKIFObR/HsR3hPRY/JMZE8dfPLiXeZmHD4UZu+/d29h5rxWSC03zcSSKEEEIIIQIz5oJ1m83G0qVLWbdunedtLpeLdevWsWrVqmE/9qmnnqKnp4fPfOYzoT5MIYQQBtFrYFbPyMQawCXtfetgJprW7pGrYADma1Uw+6vb6LY7A349fWI9mGA9PSF0wfrRpk4O13dgMZtYOTXNp4+JibIwZ5K6cm1rnw7joTicLvYeU8H6vNyRr3izWc1MTle9wyU+9Kw7nC5PAC8d6+Gx3bO4NGVUX3fNLPW97KPSBjp6HKP62oHwBOsB1EKFRXmfXU3VYydYB5iWlchvPrUIgOe3VwGwpDB1VE/8CCGEEEJMRGMuWAe47bbbeOCBB3j00UfZu3cvN910Ex0dHVx//fUAXHPNNdx+++3HfdxDDz3ExRdfTHp6+mgfshBCiAC9uU+vgckM6OPXzMrCYjaxr3rsdBMbxZeOdVABbVq8DYfLzb7qwBeY6sFwcVAT6yrcbQjB1PQH2rT6ooIUkvyoqvHUwfjQs36ovoNuu4t4m4UiHyd1p2VqPes+BOs1bT04XW6iLCZPJ7pRPB3rEqwPa0eFmlhfOEr96rrizHgmp8fR63R5Ko0imd6x7uufg7Ar3+C9X38AesfWz4tz5uVw8+nTPP8tNTBCCCGEEKE3JoP1K664grvvvps77riDRYsWsW3bNl555RXPQtPy8nKOHTvW72P279/P+++/LzUwQggxhhxr6fJc0r56emDBekqcjeVFKhh9bYLVwXgm1kcIkU0mE/O12pKdQdTBlGoT68VBTKhmaBPrjSGYWNfDSF9rYHSLC1MA2FrRNOJj9ZqQOblJmM3DLy7V6SciSn1YYKovLp2UHOvz8/sqI1F97mV56fD0yqTRXsZoMpk8Yem6CL8Cx+12e3rox0zHet+JdbcLaveE71gCdOuZMzhv/iSSY6O4cGFuuA9HCCGEEGLcG36bWQS7+eabufnmmwd939tvv33c22bOnOnzUjAhhBCR4e39alp9UUEK6UFM5545J4ePDjXy+p5qbjh5ilGHF/H05aUjdawDLMxP5p0DdWw/2sJnA3itbrvTc0WAIR3rBi8vdbncrPdzcalukdbtv7uylR6Hk2irZcjHevrVfVhcqvNnYr3Ks7g0xufn91VmgnpOmVgfWm1bN1Ut3ZhMMM+HDn2jnTErm0fWl/HmvjpcLrfhJ1eM0tjRS1u3A5MJT9VRRGuvg8ZSdT9vGVRugmPbIX9ZeI/LTxaziT9etRiXW90XQgghhBChNSYn1oUQQkwMb2pL+k4PcgHbWVrP+sdlTTR3hmYpZqRxudx9OtZHPo8+X5u+3Xm0JaDXO1zfgcsNSTHWoCpKvMtLjQ1399e00dDRS5zN4gnKfVWYFkdavI1ep4u9x4avytEn1uf60K+u009ElNR2jPhYfYFqrsH96tB3eWkvLpcMIwxGr4GZlplAQvToz6esmJJGQrSV+vYedlYG9md1NJQ1qK/l3ORYYqKGPhEVMSo+UrdZc2DKKer+GOtZ15lMJgnVhRBCCCFGiQTrQgghIlKPw+mZMA62K7YgLY5ZOYk4XW5PWD/etfc60C/U8qVPfIG2wPRgbRudvf4vRtSnradlJWAyBR7qZGihvNFVMO8fVF9LK6ekYbP699cfk8nEYi2M31o+dB2My+Vmjzax7s8081StOqe+vYeWTvuwj9Un1vNDEKzri2OdLjdNE+QElL/CVQOjs1nNrJ6hrrhYF8Hfyw7VjbUaGC1YL1gJOQvU/eqd4TseIYQQQggxJkiwLoQQIiJtPNxIZ6+TrMRov6Z/h3KmNrX++gTpWdcD2mir2aeJ0eykGLISo3G58YTD/ugbrAcjVFUwgfar6xZ5gvXmIR9T0dRJW48Dm9Xs1+chMSaKnCRVw1IyQs+6twrG+GA9ymL2fP7rDf78jxfbj4ZncWlfa2ap72WR3LOuT6wXZYyBGhjwBuuFq2DSQnW/Zjc4/T/JKIQQQgghJg4J1oUQQkQkfbL8tJmZQU1A6/Rg/Z0DdXTbnUE/X6TTa2B86VfX6VO42/2sg3E4XXxY2gBAcaaPgXJnI7zza2g60u/N+tR0U2cvToPqSHocTjYebgTg5OmBBeuLC9UC3OEWmO6qVCckZuckEmXx769YehBfOkLPelVzNxCaYB3w1PhIz/rx3G63Z2J9YZgm1kH/nqj6/KtbusN2HMM57FlcGtyJtlHR26n61AEKV0LqFLAlgqMbGg6G99iEEEIIIUREk2BdCCFERHpLC9aDrYHRzc9LJicphs5epycEHs9auvR+dX+CdTWFu1MLD33hcrn59tM72FjWiM1i5nRff782PQRv3Qnrf9fvzalxKlh3uTGsD39reTNddicZCTZmZicG9BwLCpIxmaCisYv69sFD511V6oTEHD8Wl+qKtTqY0mEm1t1uN5XaxHpeaoiCda1nva49MgPbcDra1EVTp50oi4lZkwL7OjJCRkK0p5oolNVW//joCL96ZV9AJ7gO16tFxlPGwsR61RZw2SFxEqRMBrMZcuap9x0bmz3rQgghhBBidEiwLoQQIuIcru+grKGTKIsp4OqOgUwmE2vnqND3tQlQB9PapSoMkmJ8X7A4XwvWd/i4FNHtdvPD53bxzNZKLGYTf7xqMTN8Da5rdqvblqP93hxlMXum7I3qWV/fpwYm0KsfkmKimKZN428bog5GX1w6L8//6iLvAtOhg/XWbgftPer3NTc5NMF6hnbFgEysH29bRTMAsyclEW0N70LOM2arK3D+/mEZDqfL8Of/uKyRHz67iz+9XcqjH5T59bFut5uysTSx3rdfXf/+4OlZl2BdCCGEEEIMTYJ1IYQQEUefwlxelEaiD4s3fXXmnBwA3thbg8ugmpFI1drlfxXMfG3h5qG6Dtq6h1+i6Xa7+eXL+3hsQzkmE9zzqYWcNTfH9wOs269u248/yaHXwTQYFKwH26+uW1yYAgxeB+N291lcGsjEuh6sDzOxrverp8XbiLWFJtj1TKxLsH4c7+LS8PWr6z69opCUuCj2Vbfx6IdHRv4APzicLn747C7Pf9/92n4qGjt9/via1h667E4sZhP5IbqywlB9+9V1k7RgXa+IEUIIIYQQYhASrAshhIg4RtfA6E6YmkZCtJW6th62+1F3MhbpHev+VMFkJESTp3V37xxhav0Pb5bwl3cPAfCLS+Zz0aI83w/O6YCGEnW/ve64d6cbuMC0tdvOdm3SOPhgXfWs65PLfVW3dtPQ0YvFbGJmjv81Ifo0fEVj55A7ACqbtBqYEPWrgwTrw9F3DywIY7+6Li3exnfOmQXAb18/QE2rcdU9f//wCPuq20iOjWJxYQqdvU6+/+wu3G7fTkYeqlcnhwrT4vzeNTDqXC6o2KjuF670vr3vxLqP/99CCCGEEGLiifC/7QohhJhoOnocbDisOtB97uv2UbTVwqkzMgF4fZzXwbQEMLEOfXvWhw7WH3r/MPe8fgCAH54/hytXFPp3cE1l4NRC845aFW71kR6vwt3GjuDD3Y9KG3C5YWpGfNCBtD6xvr2i5bjeaX1x6fSsBGKi/J8mz0yMJjHGissNZQ0dgz6mqkUF67kpMX4/vz/HAVA3RI/8ROV0uT1VP+FcXNrXFcsKWFSQQnuPgztf3GvIc9a2dnv+bH/7nJncfflCbFYz7x6o49ltlT49R5nWr16UPgb61ev2Qk8LRMVD9nzv2zNngTkKuluguTx8xyeEEEIIISKaBOtCCCEiyvsl9didbgrT4piaEW/48589T9WVPLu1MqClfGOFXgWT5GeVzkg96//aWM7PXtgDwG1nzuCGk6f4f3B1+7z3XQ7obu737jStCqbegIn19QbVwABMz0ok3mahvcdxXBe6HrrODaAGBtQOgJF61vXFpbmhnFhPUKG9TKz3V1rXTmevkzibxfP7FG5ms4k7L56H2QTPb6/i/YP1QT/nz1/aS3uPg4X5yVy5vJDizARuOWM6AD99fg8NPpxw0U8MFYXg+7fh9BqY/GVg6bOPwmqDLHVFgPSsCyGEEEKIoUiwLoQQIqL0rYEJdNHkcM6ak01avI2qlm7W7R2/U+ut3WrJpd8T63kpwOAT689tq+T2/+4E4Iurp/LVNdMCO7i+wTpAe22//9SrYIxYXmpUvzqAxWzy1IBsG9CzvlvvVw9gcamuWKuDKa0dfGJ9NKtgjDipMZ7odULz8pKxmI3/vhSoeXnJXLOqCIA7nttFj2PwGiFffHSogWe3VWEywc8unuf5//zC6qnMykmkqdPOT7WTasM5VKe+fkNxYtRwg/Wr63IWqttjEqwLIYQQQojBSbAuhBAionx0SNXAnDozMyTPHxNl4VPLCgD4x0fGLv2LJHoVTFKsdYRH9qcvMC1v7KSpT7D92u5qbvv3dtxu+MwJhXz33FmBn/ioP9D/vwcsMPV0rAdZBXOspYvSug7MJlg1NT2o59J5FpiWN/d7++6q4CbWAe/E+hALTPXlpaMRrDd29GJ3ukZ49MSh72RYGAGLSwe67awZZCZGc6i+gwe0vQf+sjtd3PGcWlh61YrCfj3yURYz/3fpAswmeG5blefk51DG1MR6hR6srzz+ffoC0+qdo3c8QgghhBBiTJFgXQghRMSwO11UaFO5s3MCn/wdydUrCzGZ4L2D9RyuH3w6eKxrDbBjPTkuytONrC8wfe9gHTc/vhWny80nF+fx0wvnBXc1gWdiXXuOjv4LTNMSVLgb7PLS9SXqJM38/BSS4/z7PAxlUUEK0D9Yb2jv4ViLWh45Jzfwr1t9gelQVTBVzeo1QlkFkxIbhVWbVDZieex4sSOCFpcOlBQTxQ/Omw2opcIVjZ1+P8ff1pdxoKadtHgb3zp75nHvX1iQwudOUrVP3//vTtp7HIM+j9PlprxBvf6USA/WW6tUf7rJDPnLj39/3wWmQgghhBBCDEKCdSGEEBGjsqkLp8tNTJSZLG1yNhQK0uI4faZajPrYOJ1abwmwYx1UEA0qWP+4rJEv/H0zvU4X587L4VeXLcAcTBWGywV12sS6PhE6YGI9w6AqGL1f/eRpxkyrAyzSJtYP1LbR1q0+x3oNzNSMeBKi/btCoK9ibWL9UF07rgH9/70OFzVtoQ/WzWYT6VrHvfSsKz0OJ3uPqd/jSFlcOtCFC3NZNTWdHoeLnzy/26+PrW7p5ndvqD+T3z1nFilxtkEfd9tZMyhIi6WqpZu7X90/6GOqmrvodbqwWc3kJofu69QQeg1M9jyITjz+/TnzABO0VkJHw6gemhBCCCGEGBskWBdCCBExjmiTloVpccGFtz747AmTAXhq81G6egPvJY5Urd16FYz/wfoCrQ7mhR3H+NwjH9Nld3LqjEzuvXIxVkuQf3VoKQdHF1hsUKDVLwzoWNeXlzYEEay73W5D+9V1WYkx5KfG4nZ7e+h3aTUwwUyrAxSkxmKzmOlxuDyLSnU1rd243WCzmslIGDz4NIpeB1PX3h3S1xkrXtlVjd3pJiMhmoK0yAyLTSYTP7t4LlEWE2/sreX1Pb7vj7jzxT109DpZUpjCZUvzh3xcnM3Kzy+ZD8CjH5ax+UjTcY85pF0BVJQe+u/hQRuuXx1U2J42Vd2v3j46xySEEEIIIcYUCdaFEEJEjCNaN29hWugrBFbPyKQgLZaWLjvP76gK+euNtpYAq2AA5ms90nuPtdLW42DFlDTu/8xSbFYD/tqgT6unT4PEHHX/uOWlKtht6uzFOWBy21dHm7qoa+shymJiSWFqwIc7mMXa823VFlrurtQXlwbXv221mD31GQPrYCr79KuHYqlvX5laFY9MrKsTNH9+uxSAa1ZNDvnnPhjTshK58RQVBP/4f7vp7B28rqWv9SX1vLDjGGYT/PSieSOG4adMz+TSJfm43fDd/+yg19G/h7/ME6xHeA0MDN+vrtOvqpEFpkIIIYQQYhASrAshhIgYR7RuXr3jO5QsZhNXr1RT6//48Ahud2ABbiTqcTjptqvAK5AqmHl5yej54cL8ZB66dhmxNosxB6f3q2fOhIRsdb+jf7CeqvWhu90qXA+Evmxy9qQkYqIMOnaNt2ddTezqE+vzglhcqivOUoFk6YAFppXa7oHclJigX2Mknol1CdZ5+0Ad+6rbiLdZuGbV5HAfzohuXjONvJRYKpu7+OObJcM+ttfhXVj62RMm+3xi6AfnzSYjwcbB2nb+9Hb/19B3VkzJjPBgvafNu5S04IShH5ejJvSlZ10IIYQQQgxGgnUhhBARQ59YnzwKwTrAp5YVYLOa2VnZwnat1mM8aO1Sk6omEyTG+N/5nRBt5Yuri1k7O5tHP7eCxADC+SHVad3MmbMgXvXcD+xYt1rMpGjheqA969u1afIF+cGH3QMt1nrWt1U009pt95wQmhtkFQwMvcC0qs/EeqjpwXq9LC/lz2+pafWrVhYO2T0eSeJsVn50wRwAHnjv0JCLcAEeev8wpXUdZCTYuO2s4xeWDiU13saPLpgLwH1vlXCwps3zPk+wHukT60c3gdsFyYWQnDf043IWqluZWBdCCCGEEIOQYF0IIUTE0APKyaMUyqTF2zh/wSRATa2PF3q/ekK0NeCe4++eO4sHr11mfJjYb2JdD9brjntYurbAtCHAcFc/URKKZZNzc5OwWczUt/fy6q5qQAXeqfHBf670BabHBest+sT6KATrUgUDwKayRjaWNRJlMXHDyVPDfTg+O3NONmfMysLudHPHc7sGvRqnsrmL3687CMDt5872uzLq/AWTPK/xnf/s8Czb9QTrGREerHv61YeZVgdvFUxDCfQMfZJCCCGEEEJMTBKsCyGEiAgul5vyRj1YH52JdfAuMX1+RxVNQSzLjCStQfSrh5TbDfVax3pGn2C9ow5c/bua9Z71hg7/w12ny82uSi1Y12pbjBRttTBbm07/50fqhIwR0+oAxdrE+nFVMM1qkeioBOuJqm5mogfr97+jptUvXZJPTnLoK3iMYjKZ+PGFc4m2mvmgtIH/bT9+h8SdL+yhy+5keVEqn1wyzMT2MK/xs4vnkRBtZUt5M//46Ai9DhdHm9T38IgP1n3pVwf1PSohB3BDze6QH5YQQgghhBhbJFgXQggREWrauulxuLCYTaMSHuoWFaQwLy+JXoeLpzZXjNrrhpK+uDSQfvWQajsGPa1gskB6McRnqre7ndDV2O+h6Qlq+juQKpiS2nY6e53E2yyeoNpoi7XAXp+MD3Zxqa44MwGTCZo67TS0e4PtSi2wzB+FPxsZ2ue+rt2YYP3B9w7x+IZyQ55rtOyvbuONvbWYTPCF1WNnWl1XkBbHV9dMA+DOF/d6rmIBeOdAHS/vqsZiVuF4oAtZc1Ni+c45qkLmV6/sY8PhBlxuiLdZPHVCEcnpgIqP1f3CVSM/Xp9al551IYQQQggxgATrQgghIoJeA5OfGkuUZfR+PJlMJs/U+j8/KvdUGgTjlV3H+PPbpWFbiNrarTrWI25iXa+BSZsK1miwREFsmnpbe/8FpmlarUogPd96v/r8/GQsAVbhjETvWdcZNbEea7N4etT1Ohi3203VqE6sG1cFU9HYyZ0v7uV7/93JDm2h7FigT6ufOy+HqSE6ORNqN66eytSMeOraerjnNXWlSI/DyY+0haXXnVjErJzgvm6vXjmZpZNT6eh1cuuT2wEoyogPOKwfFTW7wN4B0cmQOXvkx+dIsC6EEEIIIQYnwboQQoiIUK4F64Vpo1cDo7twYR5JMVbKGzt55+Dxfd/++KC0npse28L/vbKP3VWtBh2hfzwT67H+Ly4NKc/i0j6LEhOy1e2ABaZ6x3pjAFUw27UANxT96rrFBan9/tuoiXXoWwej+qqbO+102Z0Ao1JJogfr7T0OOnsdQT3X/mrvYsu7tXA30lU0dnrqU246dVqYjyZw0VYLP7lILRn9+4dl7Kps4YF3D1HW0ElWYjRfXzs96Ncwm03836XztZ0D6s9qUaTXwOj96gUrwOzDP4X0iXVZYCqEEEIIIQaQYF0IIUREKGtQIWLRKC0u7SvWZuHyZQUA/DOIJaZ1bT3c8q9t6IPqemf8aIvYjvVBg3WtDqaj/wmNdG2BZiBVMJ5gPQT96rqCtFhP+J+REE2WgdUX0wYsMK1sVotLMxOjiYmyGPY6Q0mIthITpf6KWN8W3N6BA7XeYP3dA3VsPNw4zKMjwwPvHcLpcnPK9Azm5xt3wiQcTpmeyfkLJuFywzef2s4f3yoB4PvnzSbRoKqoaVmJ3LzGewJiaqQH6772q+v0ifXaPeC0D/9YIYQQQggxoUiwLoQQIiIcCcPi0r6uXlkIwJv7a6kIIBB3utzc+uS2fvUZ+iK/0dYaqR3rnmB9lvdt8doC0wET64FWwXTbnew7psLcBSEMRU0mk6cOZm5ukqHVF55gva5/sD5auwdMJpO3DibInvWDNer/Id6mTgjc/dr+sFUk+aK+vYcnP1a7Fm46tTjMR2OMH54/h4RoK/uq2+i2u1g1NZ0LF+Ya+hpfOrWYmdmJgHG1SCHhdnsn1n3pVwdILYLoJHD2er+HCSGEEEIIgQTrQgghIsQRbWI9HFUwAFMzEzhlegZuNzy+0f9Fi396q4T3S+qJjbJw5hxVb1LZ1GX0YfpEX1SYFEkT62431O1V9wetgunfsR7o8tI9x1pxuNxkJNg8XeWhctbcHADWzs4y9Hk9VTDaxHqVFqznpYS+BkaXmWBMz/qBGnWS49vnzMJmNbPxcCPvHawP+vhC5ZH1h+lxuFhYkMKq4vRwH44hspNiuPXMGQBYzSZ+etFcwzvQbVYzj924kvuuWsJZc3IMfW5DNZerJcpmK+Qu8e1jTCbIma/uS8+6EEIIIYToQ4J1IYQQYed2uz3LS8PZz/sZbYnpkx9X0ONw+vxxHx1q4LdvqP7on140l9NmqnqTo2EK1lsisQqmox66mgATpPfpdtarYAYG6/Eq2G3wc2JaX1y6ID8l5AsUL1+az/rvrvF83RhFn1ivbO6is9fhOUET6hMFfRkxse50uT11NqtnZPKZlerzFKlT623ddv6uVUHddGpxZC/g9NO1qyZzyxnT+e0Vi5iuTZYbLSMhmvMWTMIcooXBhtCn1SctBJsfJ3FzpGddCCGEEEIcT4J1IYQQYdfcaaetWy1JDNfEOsAZs7KYlBxDY0cvL++s9ulj6tt7+NoTW3G54dIl+Vy+rID8VPX/EK5gvbVLfS4janlpvVahkFLYP9DSJ9Y7+gfrehVMc5cdh9Pl88vsONoChHZxqc5kMpGXEmt4AJsWb/P8/x+q66CqZXSrYKBPsB7ExPrRpk56HC6irWYK0+L48unFxEZZ2HG0hdf21Iz8BKPs8Q3ltHU7KM6M5yztqpPxwmoxc+uZM7jA4AqYMafCzxoYnb7AVCbWhRBCCCFEHxKsCyGECDt9cWlOUsyoLGccitVi5qoVqmv97x+Wjfh4l9arXtvWw7SsBH528VwA8lNVAHq0qTMsk7kRObFet0/d9u1Xhz4d6/2Xl6bGRWEyqQaZpk7fFwZ6JtYLxvbSyeJMdeVGaV07lc3dwOgG6xkGVMEc0PrVizMTsJhNZCREc/1JRQDc89oBXK7ImVrvtjt58P3DgOoLj+ipaxE4fWK9wMfFpTp9Yr16J7h8P9EnhBBCCCHGNwnWhRBChF25tiy0MEyLS/u6YkUBURYTW8qb2VXZMuxj//xOKe8drCcmysx9Vy0hzqYmxPXKjo5eJ81+hMJG8XSsR9LyUs/i0pn9354w+PJSq8VMinZiwNee9ZYuO4fq1Uma0ZhYDyXPAtPadv+qYLY/CU9dB52NQb2+ERPrer/6jOwEz9u+uLqYxBgr+2vaeH5HVVDHaKRntlRS19bDpOQYLlqUF+7DEaHQ1QS12p6HwhP8+9jMmWCJhp5WaC4z/NCEEEIIIcTYJMG6EEKIsCurV8H65DDWwOiyEmM4Z94kAB7bcGTIx2083MhvXlNh8U8vnMfMHG9vcUyUxTPxW9k8+nUwrWNpYl0P1jvrwdW/1z5d+xw2dPgW7u7UamAK0mI9VSpjlb7AdE9VK/Vaz7lPwfo7v4Td/4W3fxnU63uWlwbRsX5QC9b7dnonx0XxhVOmAvC7Nw76VfMTKg6ni7+8WwrAjadMxWaVvx6PSxUfA25Im+r9vuMrSxRkzVb3pWddCCGEEEJo5F8OQgghwu5Io5oyDufi0r4+qy2jfHZrladWpa+GPr3qn1ycx+XL8o97TN86mNHkdrtp7dY71iMpWFfLXY+bWI/LAEzgdkFnQ7936eF4Q7tvE+vbjzYDY39aHaBYm1j/6JD6nMRGWUiJG+H302mH5nJ1f9ND0FAa8OvrE+v1QUysH9QWl07PSuj39utPnkJavI3D9R38Z8vRgJ/fKC/vquZIQycpcVFcuaIg3IcjQiXQfnXdpD51MEIIIYQQQiDBuhBCiAhQ3qBVwUTAxDrA8qJUZmYn0mV38syA4M/lcnPbv7dT3dpNcWY8P7t43qDLK73B+uhOrHf0OnFq3dURM7He1QTt2jLYjBn932exQly6ut/ef4FpRoIK1n2tgtH71RcVpAR6pBFjmjax3tGrpvjzUn1YktpSAS51UgWXA9b9JODX91TBtPcEtCfA6XJTogXrM/pMrAMkRFv58mnFAPx+XQk9DudxHz9a3G43f35bnYC47sQiT52TGIcC7VfX5cgCUyGEEEII0Z8E60IIIcKuTAvWi9IjY2LdZDLxmVVqav0fHx3pFyz+5d1DvHOgjmirmfuuXkJ89OBBXH6qOkkw2sG6PmFvs5iJjpRKC31aPSkPYpKOf39Ctrod0LPunVj3bWp6h1YFs2AcTKznpcQSE+X9/fNpcWmjWr5JbBqYzLDnOajYGNDr61VGvQ6X5woIf1Q0dtLjcBFtNVMwyAmzz5wwmeykaCqbu/jXxoqAjtEI7xyoY8+xVuJsFq5dVRS245iQOurh9Tu8+xdCydELlZvV/YAn1heqW6mCEUIIIYQQmgj5F7cQQoiJqqPH4emQjoTlpbpLFucRb7NwqK6DD0pVHcemskbu1nrVf3LhXGblDBISa8JVBaP3qyfFWkeecB4tnn71mYO/PyFT3XbU9XtzWrzesT7yxHp1SzfVrd2YTTAvb+jfl7HCbDYxNcNboZKXEjPyBzUeUrcFK2HRVer+az+EACbOY6IsJMWok0aBLDDVF5dOy0rAYj7+6zAmysLNa6YD8Me3SujqDc/Uuj6t/ukVhaSO8V7+MWfTI7D+Xvj7RdAa4kW2x7aDo1uddMqYHthzZM0BTOrqmwFX1wghhBBCiIlJgnUhhBBhVd6ogueUuKjIqS5B1VV8conqTv/Hh0do6ujlq09sxelyc9GiXK5YPnwXc7iqYFo8wXrkfC6p1ybWM4YI1uO1RYIDJtb9qYLR+9VnZCeOmzqPaVl9g3UfJtabytRt2lQ4/ftgjVW90vteCOj1M/Q6mACC9aH61fu6YlkB+amx1LX18OiHZQEdYzA2H2liw+FGoiwmPn/KlFF//QmvXptUbzsG/7oKekN4EtLTr34CBHrCMToB0qep+zK1LoQQQgghkGBdCCGEH2pau7n+kY08t63SsOc80qAWl06OkBqYvj6r1cG8vreGL/5zM8daupmaEc9dl8wfcRpcD9Yrm7oC6qgOlGdiPSaCgvURJ9b1YL3/FKg/y0t3jJXFpU1HwN7t00P7Buu+VcFoE+tpUyApF068Wf336z9Si039lJng7Vn310FtYn36gH71vmxWM19fqzr373+nlNZu/48xGPq0+iWL85iU7MPnVxhL/3oFqNoKz30loKsrfBJsv7rOs8B0e3DPI4QQQgghxgUJ1oUQQvjs7lf389b+Ov74Zolhz3lE61efHCGLS/uakZ3IyilpOF1uNh5uxGY188erlpAwRK96X3kp6v+nrcdBa5f/HdWB0ifWI2n639OhnDlr8PfrwfqAKph0TxXMyMHu9gqtX70gObBjHA01u+HehfCfG3x6eHFmEME6wIlfg7gMaCyFzX/z82D7LDANqApm8MWlA12yOI/izHiaO+08/P5hv18nUAdq2nhjbw0mE3zx1OJRe13RR4M6scG5vwKzFXY/A+/8yvjXcbu9wXqg/eo6fYGpTKwLIYQQQggkWBdCCOGjQ3XtPLNVTaqX1rUb1onsXVwaecE6eKfWAX50wRzm5PrW3x1rs3iqTCpGsWddXzQZMVUwPe3Qoi2nHHJiffDlpena52+kjnWXyz02JtbL3gfcUP6hTw/3qwrG5epfBQNqUexp31X33/4ldLf6dbh6sF7v58S60+WmtE4P1oeuggGwmE3ceqaaWn/wvcM0+VD7Y4T731Gh7jlzc/qdwBCjpLMRupvV/cWfgfPuUfff/jns/q+xr9V4CDrrwRINuYuCey7PxLoE60IIIYQQQoJ1IYQQPrp33UGcLnWZvssNe6v9C+mGUt6oqmAKI7AKBuDsuTl8alk+X10zjatWFPr1sXmp6mTBaPaseyfWI6RnXO9Xj8+EuLTBHxOvLS9tH7i8VAXrzZ12HE7XkC9R1tBBa7eDaKuZmTnDT0iHVc1uddvZoILFEUzJiCcjIZrMxGhykkdYXtp2TC1nNFkguU///9LrVC90Zz188Hu/DjfQifXyxk56HC6irWbyU0c+YfaJeZOYPSmJ9h4H979b6tdrBeJwfQf/26aWZX5JptXDQ7+6InES2OJh6bVwwpfV2/57k6qGMYp+IitvCVijg3uunIXqtvGQ3yeqhBBCCCHE+CPBuhBCiBEdqGnjf9tVEKVPlu+ubDHkuT1VMBE6sR5lMfOryxbyjbNmjtirPpB3gekoTqxHWsf6SDUwMOTEemqczbNnsKlz6P7tHUfV1+Lc3CSiLBH8V5vaPd77DSPXKdmsZl67dTWvfn31yP9fTVqNSkohWPr83luiYO2P1f0P/gitVT4frqdj3c9g/YDWrz4tKwGLeeQ/M2aziW+epabWH/2gjNpW3zroA3GkoYPPPLgBh8vNydMyWFiQErLXEsPw1BZN9b7tzJ/BtLXg6IInroK2amNey6h+dYD4dEjKU/drdgX/fEIIIYQQYkyL4H99CiGEiBS/e+MAbreqTbhgYS4AuyqDn9brdbioalbT3JEarAcjX6vvqGwevYl1fQFkxHSsj7S4FLwd650N4PT20VvMJlLj9DqYocPdbRXNACyI5BoYlwtq+gTr9Qd9+rC0eJtncn9YgwWVulnnQ8EJKrB86+c+vS4EPrFeUutbv3pfa2ZlsbgwhW67iz+9HZqp9ZLadj71lw+pbO5iakY8d1++MCSvA4CjB7qaQvf8Y91gX68WK1z2MGTMhLYqeOLTYDfge6dR/eo6vWe9eqcxzyeEEEIIIcYsCdaFEEIMa3dVCy/trMZkglvPnMFcrWN8V1XwE+tHmzpxuSHOZvFMx/qtahs8fYNfk7ijxTuxPorBuj6xHjHBug8T63HpYDIDblVZ0ke6Fio3tg/dva33qy+K5Onj5iNg7/D+d4NvwbrPBi4u7ctkgrN+pu5ve6x/wD8MT7DuZ8e6PrE+fYR+9f6HaOJbZ6mTL49vKDf8ZNT+6jau/OuH1LT2MCM7gX998YSR63WC8Y9L4HcLoHZf6F5jLNMXlw48ERSTDFf9C2JToWoLPPcVtXw0UB313j9rBSsCf56+JskCUyGEEEIIoUiwLoQQYli/fV11ZF+wIJeZOYnMzU0GVHjW4whugemRRlWRUpgW53fNisdrP4BdT8P6e4M6llDID0PHemuXtrw0Uqpg6rVgPWPG0I8xWyAuQ91vr+33Ln1au36IpZZ2p4tdVerqiYiu9agdEGb7OLHus0atCmawiXVQoeLsC8Htgtfv8Okp9ZNdDe09nv0KvjhQo02sZ/nXd3/itAxWTU2n1+ni928Y9/nZVdnClX/9kPr2XubmJvGvL6wiKzGEoXpXExxZDz2t8OI3gguGxyv9RFD6IB33aVPhU/8AsxV2/QfevTvw16nYoG4zZw2948FfOfPVbfV2Y55PCCGEEEKMWRKsCyGEGNK2imbe2FuL2QS3rJ0OqCns5Ngo7E43B7UALVBH6tUEb8A1MO11KsACKH0zqGMJhXB0rHuXl0ZAsG7vgqYydX+4iXXw1sEMCNbTE/SJ9cGnpvdXt9HrcJEUY/X0/0ckfXGp3ifvQ8e6X/SgMnWQiXXd2h+rsLLkdTj09ohPmRavOu5dbmgc4sTGQE6Xm9I69X3Bn4l13TfPVlPrT285yr1vHKTbHtzJu63lTXz6gY9o6rSzsCCFxz9/gm/VOsHou3jzyPuw86nQvt5Y1DjExLpuyinwCS1Qf+tO2PNcYK9jZL+6Tq+Cqd0HDt/+XIgJaNd/4O1f9qs3E0IIIcT4I8G6EEKIIf3mNTVtfMnifIozVUhmMpmYl6fVwQS5wFSfWJ+cHh/YE+x7Xk3gAtQfgOaKoI7HaHlasN7W7fAE3qGmd6wnxVr7vyMcU7MNJer3JybFG5wPRX9/x4BgPV6bmh4i2N2u1cAsLEgJ/KqH0aAH67MvVLeNh8AVXGjs4XZ7T2AMFVSCmg5edoO6/9oPVe/7MKwWs6eKp97HOpgjDR30OlzERJkpSPX/RMfSyalctjQfp8vNb984wNp73uG13dW4A/j63Xi4kc88uIG2bgfLi1L55w0rSI4bhRNOlZvVbZT2fe3V70O3Mcuex4XORm///HAngpZdDyu/pO7/90twzI8JcZcL6kug9C3130b1q4NaEByTAi471O017nnF+FF/EJ75Arz9C9j413AfjRBCCCFCSIJ1IYQQg/q4rJH3DtZjNZu45Yzp/d43T6uD2V0V3ALTIw16sB7gpPHAKcbSdUEdj9HibFZPMFk5SnUwg06s97TDH5epvuLR1LdffaTQO16fWK/p92Z9unjIYN2zuDQ54MMcFXoVzIyzwRINzl7Vu26EzgZVO4IJUouGf+yp34boJKjeoSqURpCR4N8C04Pa4tJpWQmYzYGd6Pj1ZQv4w6cXMyk5hqNNXXzhH5u59pGPPZPwvlhfUs+1D2+ko9fJicXpPPq5FSSOVj1SpTaxvvqbkD5NnSzyY2nsuKfXFiXkQPQIVzWcdRcUnwH2TrXMtK3m+Me43eqk6p7n4PUfwaMXwP8VwR+XQo22YLTwBOOO32Ty1sFIz7oYzGs/AJc2qf7WXdBSGd7jEUIIIUTISLAuhBDiOG63m7tfVaHo5csKKBwQfM/NUyFmsAtMjzRoVTBpAUysdzTA4ffU/YVXqduSyArWwTu1Php1MHani85eNQXdr2O9ZpeaHt/x79G9LN0TrA/Tr67zVMHU9XtzRsLwy0t3HFVfgwvzUwI6xFFh7/JWv+TM9/ZK1xtUB6PXwCTlQtQI3eHxGXDy19X9dT8Fe/ewD/csMPU1WNcWl/rbr96XyWTigoW5rPvGqXzl9GJsFjPvHqjjnN+9yy9e2kt7z/Bfw2/tr+X6v31Ml93JqTMyefi65cTZrMN+jKH0ifXJJ8Infq3ub/yrhLA6z6LdYa6u0FmscNnDkD4dWivhX1epEP3Aq/DWL+Cxy+Hu6fC7efDva2D97+Dwu9DTAtYYyF8BZ905+FLfYExaqG6r5fdUDFCyDg68omq3suZCbzu88p1wH5UQQgghQkSCdSGEEMf5oLSBDYcbsVnMfHXNtOPePy9XVcHsPdaKwzl8ncRQnC43FY1qijugifV9L4Dbqfpul2v1Fofeibg+U2/Peugn1lv71M0kxvQJEjvq1a2z1xtqjYa6fep2pH51GLIKJs1TBXN8sNvZ6+CAFuRG9OLSuv2qEic2TXWsp2t/phoMWtA50uLSgVbeBIm50FIBG/8y7EM9wbqPVTD64tLp2YEH67o4m5VvnT2L125dzRmzsrA73fzl3UOsuftt/rv16KD1MK/truYLf99Er8PFmXOy+es1S4mJsgR9LD5rrYL2ajBZ1Pem4jUw52L1+//iN0as35kQ9H71dB+/XmNT4KonVf1K5SYVoj/+KXjnl3DwNeioUyFmzgJYeh1c8Hv40vtw+1H4/Otw4leN/3/Qe9blZInoy+lQ1U8AK74Alz6gvhfsfR72vxLeYxNCCCFESEiwLoQQoh+32+3pVr9qZSG5KbHHPaYoPZ54m4Vuu4tD2gJSf1W3dtPrdBFlMQ36GiPa86y6nXsx5C6G2FQ1pahPi0aIfK1nelSC9W51UiEh2orV0udHfGe99/5odgJ7JtZnjvxYfanngCoYfXnpYFUwuypbcbkhJymG7KQRJrXDSa+ByZ6raiQytGqleqOCdX1xaZFvj7fFwZofqPvv/kZ1Xg/B34l1/UTH9Cz/F5cOpSgjnoeuW87D1y2jKD2O2rYebn1yO5ff/2G/PQ8v7Kjiy49twe50c978Sfzp6iVEW0cxVAfv95+sOerzDHD2z1Xf+tGNsO2x0T2eSOTPxLouvRg+9XdVo4QJMmbCgivh3F/BDW+oEP1L78EF98LSa9WVIZYQVv9M0oL1ml1yskR4bXlU/YyNTVW1W9lzYZVWwfbSt6A3sL8vCSGEECJySbAuhBCin7cP1LGlvJloq5kvn1Y86GPMZhNzcoNbYKrXwOSnxmHxt4u5s1FNp4OaBjVbYOpp6r9L3wzoeEIlfxSrYFoH61cH78Q6eMPuUHPavZOpvkysx2eq2wFVMHpHfcMgVTBjpl9dX1yaPVfdpmvBeoNBVTBNfk6sAyy8UtUU9LTAu3cP+bBMPzrWHU4Xh+rUn+sZBkysD7RmVjav3rqab509k9goC5uONHHhH9/n+//dyT8+OsLXntiKw+Xmk4vzuPfKRURZwvDX3Mot6jZvsfdtyXlw+u3q/ut3DHsiY0Jo0L4vpA3+82VIU0+F2/bCd8vh5o3wyb/Ayi9CwXKICuDkbDDSp6uqmd720b0KSESurmbVpw5w2vdUuA5w2nchuRBayuGd/wvb4QkhhBAiNCRYF0II4eF2u7nntQMAXHtiEVnDTAHP1RaY7qoMbIFpUItL972oamCy+/RVF69RtxG2wHQ0q2D0xaX9amCgf5BXO0oT642H1PI2WwIk5Y38+CEm1vXlpS1dduwDaoe2H20GIrwGBrzBetYcdZthcLAeyASw2QJn/VTd3/hXaCob9GH+TKyXN3bS63QRG2XxfN0bLdpq4SunT+PNb57KBQtzcbnhsQ3l/PDZXbjccOXyAu6+fGH/KzZGU5UerC/t//aVX4LM2dDVCOt+MvrHFUkC+XrVxadDTJKxxxMIi9V7omys96x31MP7v4VNj8Cht9X3ggirVBsT3v21WiSdMROWXe99uy3eu2vhw/u8Pw+EEEIIMS5IsC6EEMLjtT017KxsIc5m4Yurhw895mkLTHcHuMDUE6ynBRCs73lO3c65yPu24jPUbeVm6GoK6JhCIS9F/f9VNo9GFcwQE+v9qmD2hfw4+r1OxgxVfzISvWO9q1FNu2tS4mzoFzQ0dfafWteD9UWRHqz3rYIBb8d62zHoaQv++T1BpZ8LGovPgCmngssO2x4f9CGeiXUfOtb1fvVpWQmY/b0KxU+TkmP5w6cX88SNJzArR03HX3diET+/ZH7IX3tILhdUblX3c5f0f58lCs77jbq/+VE4GlmVVaOmq0n9GYfAgvVIkjNf3Y71YP2tu+CNH8MLX4e/XwT3LoS7suH3i+Efl8ALt8EHf1A94dW7oKc93EcceRpKYYO2r+Lsnx9fQzTzHJh1vjrZ/MKtUh8khBBCjCPWkR8ihBDh53a7eWZLJTNzEj2BrjCWy+Xmt6+rafXrTyoiXQvUhjIvT00N7qlqxeVy+x1mlTeqyojJ6fH+HWhXk5qqA9WvrkvOU5UjdfvU++de4t/zhkieNrnb0mWntdtOUkzoen/1ifWk4apg6g+q4DqU/cPQp1/dhxoYUIs9TRZ1JUJHHSTlAmAxm0iNs9HQ0UtDey9Zieoqiob2Hs/y24j+ntBRr03hm7yfi9gUVX3TUaem1nMXD/cMw+tuUVOSAKl+Busmk6pQOvwONB0Z9CH6xHq9D8H6Qb1fPdu4fvWRrCpO54Wvnsyxlm4KAjlJZ6TGQ6paxxoLWbOPf3/RSaoXfMe/4MVb4ca31JUDE4l+EighG6JH7+skJMbDAlO3Gw6+oe7nL1d1Js1HvIuuh6q5ic+C074Dyz8/aoca0V6/Q52gnLYWpq8d/DHn/kr93aRiA2z9h9oFIIQQQogxTybWhRBjwpbyZr7x1HY+/dePONYS+snfiejFncfYV91GYoyVL5wycvfttMwEoq1m2noclDf63x9eVh9gFcz+l9U/YLPmeCs1dPrUeknk1MEkRFtJjVMhdmWI62Bau9Tl+8eF930n1l12aDwc0uMAvBPrviwuBTCb+/Ss1/Z7l77AtLHPAtMdWrf/1Mz44yf0I4l+2X9qUf8gUe9Zrw+yDkb/vYzLCKwiIzlf3bZWDvpuPVhv7rTT43AO+1QHatUk6/Qs4/vVh2O1mMMfqoN3cemkBUOfuDrrZxCdDMe2w6aHR+/YIoX+9epvv3okmrRQ3VbvUAH1WNR4SHV/W2xwzXPw1U3w/Rq4dTdc+wJc+Ec45Rsw71J1FYbeG95RC6//SAXxE93hd2HfC+rE8Fl3Df245Dw4/Xvq/ut3HLdPRAghhBBjkwTrQogxYadW+dDW4+C7/9mJe6z+IzZCOZwufvuGmlb//MlTSY4bOai0WsyeCoZdftbBuN1uTxjvd7C++1l1O+fi4983Te9ZfzOigo78VPX/GOqe9ZahlpfqHesWFVBTNwo963Xq68nnYB0gYfBgXe9Z7zs1rS8uXZifEugRjo6BNTC6DK0OpuFgcM8fyOLSvvT++5ajg747OTaKKIu6GmWwBbJ96RPrM0ZxYj2iDNWv3ldCFqz5gbr/5s8mXrjmWVw6xmtgQJ3cNZnVlSdt1eE+msDoy74LVqoucFAnOZPzYcopsOSzcMYdcNnD8IW34Dtl8J0j6v+9t31inhzqy+WEV7SwfPkNkDXCFVorvqgqhLqb4fUfhvzwhBBCCBF6EqwLIcaE/TXeHuJ3DtTx1KbBQyARmOe2VXGoroOUuCg+d3KRzx83Ny+wBaaNHb209zgwmbyhs0+6W7xBQN9+dd3kk8ASraZv6w/4dUyh5F1g6v9kvz/0jvWk2AFNb3oVTP5ydVsb4p51l9P7+fcrWNcWmHYMmFiPV1PT/SbWj6qTOQvzI7gGBqBml7odGKzrPev1QQbrwSyCBDVFCdBaNejJKJPJ5O1ZH2aBqcPp4lCdqneakT26E+sRQ59YH9ivPtDyG1SNSHeLmlydSPSv1/RxEKzb4tQOCRi7Peulb6lbffm3L2JT4MSvqfsb7gd7t+GHNWZs/SfU7ISYZDjt9pEfb7HC+fcCJtj+hJp2F0IIIcSYJsG6EGJM2HtMBevLJqvLkH/2wh6qRmEZ5ERgd7q4d50K9764uphEPzrA5+UGtsC0TFtcOikphpgoPzqG9RqYzFmDT4ZFxcLkE9X9CKqD0YP1UFfBDDqx3tsBDu11J5+kbkM9sd5UBs4esMZAymTfPy5eW2A6RBWMPjHtdrs9E+sLjFxc6nRAt38niUZUo02sZ83p/3a9CibYifVAF5fqEnMBk/r90rvaB9DrYIYL1o80dtLrdBEbZSEvJTawYxnLnHZv13beCMG62QLn/xYVrj0ORz4M+eFFjMZxNLEO3p71134I793jvVJnLHDavcFu8en+fey8S9XVLu01sONJ449tLOhuVVedAJz6XYhL8+3j8peqk2ugFpk6Rt5fIYQQQojIJcG6ECLiuVxuDmgT63deMo/FhSmqEuYZqYQxwn82H6W8sZOMBBvXnuhHCIp3gemuyha/fi8CXly65zl1O9i0um6a1rNeGjnBuh40hroKplVfXtr35Ig+rW6Jhvxl6r6+WDRU9OfPmO7fcsaEwYN1vQqmQZtYr2zuoqGjF6vZxJxJAfSKD+Wlb8Kvi6F6pzHP53J6u+aPq4LRg/VScLkCf43GMnUbaFBptXk/70PUwWToE+vDLDDtu7jU30XG40LNbnVyIibZt9+L/GWw5Bp1/8XbVMg5EQR7hUWkmXOhqoOp3w/rfgL3LYc/LFP94xUfB/dnO9QqN0Nvm1ocnbPQv4+12uCEL6v7H/ze+P/PrY/B81+HnnZjn9dI7/1G1QClT/N/ieuaH6ortBpKYP29oTk+IzQehn0vQXO58fV6TrvaNbH9SfV3u8PvQvUuaKmE3s6IqvMTQgghhmMd+SFCCBFeR5u66Ox1YrOYmZaZwN2XL+QT977HuwfqePLjCq5cURjuQxyzehxOfq9Nq3/p1GLibP79WJiRnYjVbKKp086xlm5yfZxUDWhxaXerdwp9sH51XfEZwA+gbL26RD0qxvfXCBFPx3pziKtgBptY16eQ4zPUpD+o+hGnfegFi8Gq14N1P2pgwBvwDqyCSdCrYFSwu71CXSExa1Kif1c8DMfRCzufAmev6vHPmR/8czaVgb1TTe4PDBJTi8BsVe9vq/IuEfWXEUGlPnnaWgm5i457ty8T6wdqVAA2LWuC96vnLgGTjycW1v4Y9j6vevg3/AVOvDlkhxcRupq934/GS7A++wK4ba+6mmrfi3D4HXUVyvrfqV8J2TDzEzDrfJiyWgXSkUKvVZt6mupV99fSa+GdX6lweP9LMPt8Y46rvgSe/xq4HGDvgkvu9/3P1GhpKoOP/qTun3WX/7+vsSlw9s/hPzfAu3erKwDSI2yh7+7/wjNfUD8TQS0Xz12svsflLVH39Z/ZI3G51J+Lyi1QtVV9v6zeCY5haoQs0WpZblyauo1NVZ+32DSYcTYUnRz0/6IQQghhBAnWhRARb1+1qmaYlpWA1WKmODOBb509kztf3MudL+7llBmZE7N6wAB/W19GVUs32UnRfOYE/6bVAWKiLEzPTmTvsVZ2Vbb4HKzri0sL/QnWD7yqJkLTp0PW7KEflzVb1Vu0VUH5B/51x4ZIftooTax3OwBIGixYj0uH5AKIigd7hwpk/ek/94c+sZ45wiK3gfSO9YFVMPH9q2B2aMuMDV1cevRjtYwPjOu9rdmtbjNnHT+5b4lS4XpDiTrREUiwbu9SX+cAqQFWwYDqWa/aoiYFB6EH6/XDTKwf8CwuneD96sMtLh0oLk2F689/Dd7+Bcz7JCTlhuTwIoJ+Eig+C6LH0ddJYg4su1796m6Fkjdg3wtw4DV1wmrzI+pXdBJMPxNmnQczzlUd7eEUSL96X9GJqtLk/XvUSYRZ5xkTgL/2AxWqA+z4l1qiuvgzwT+vkV6/QwXOU09TIW8g5l0K2x5TJzhe/AZ89r+RcwJhw1/g5e8AbvX3hrZjajr/4Gvqly4pH/L6hO2TFqmrdpqP9AnRt0LVNnV1xEDRyZAzT13d1dUIXU3ql8uh/r7XXq1+DfTB72HxZ+GsO1XYLoQQQoSRBOtCiIi3v1r9ZXxWjvcf4tefNIVXdlWz6UgT33l6B/+4YQWmSPkHyRjxQWk9v3pVBaC3rp0R8OTv3NwkFaxXtXLW3ByfPuZIg6qCKfKnCmbPs9oLXjz8Pz5NJhUUbPunmnCPgGBdP/HT3GmnvcdBQnRofvwOOrGuV8HEZ6ipxMwZ6h+6tXtDGKxr9Sf+Pn98prodogpGX166TetXNzRYP/SW937VFlVBEB3k9HWt1q8+sAZGlz5dBesNJf53HIOamgQVTvja7zuYJC3Ubx28CsaXifWSWnVSYkb2BJ1Yr9yqbkfqVx9o8Wdh6z/UiZ1Xvw+XP2L8sUUKz+LSCJvMNVJMkjpBMu+Tqju77D01yb7vJRUQ7vqP+pU+DT79JGRMC89xdjVD5SZ1P5DvPbqVX4IP/6i+fss/gsmrgjuu0jfhwMvqap5FV8OWR+HFb6oTVsOdUB9NZetVdYnJrKbOA/27p8kEn7gb/rRK/fzZ9R+Yf5mxx+ovtxve+LE6UQKq4ubcX6kr3Gp2aWG5FpjX7Vc/M1qPqitvdNFJ0DPIrhJrLExaqE27a0F86pTjr5Zwu9VJbj1k7+wTuHc1qcXoO55U3zdL3lD7KmaeG6rPiBBCCDEiCdaFEBFvnzYJObNPsG4xm/jVZQs49973eL+knsc3lnP1Sv8nrieqisZOvvLYFpwuN5cszuOK5QUBP9e83CSe3gy7K31fYHpEW15amObjxF5PGxx8Xd0frl9dN00L1kvfGvmxoyAxJoqUuCiaO+1UNnX1+1o2itvt9iwvTYrt8+O9UwvW49LVbeZs7z+KQ8Hl8i7wC3hivabfmzO05aX17T04XW52al9rC41cXKrXIoCaliv/CKavDe45a3ap26GC9YxpcAA1sR4ITw1MUXCTjsl56naoifWE4YN1h9PFoTp1smx61jiaRPZVb4d3IbA/E+ugQqXzfgN/PQ12P6PqNaaeZvQRRobx1q8+Ems0TFurfn3iNyqQ3PcCbHtCnUx7cA1c9oh3L8hoKnsP3C7ImBF4DRVAYjYs/LQKwNffG1yw7nTAK99T95ffqELrlgr1vfmp6+DGN8Hm514Wo7lc8Ort6v7S64b+3u6r9GJY/S1460545Xb1tRKuCWynHf73Vdj+hPrvNT+AU76pfraYLWovhL6nBdTfy45t7x+2N5WpUN0cpSbR+1bHZMwEiw/Rg8mkroaIToSUIaoel14Hz92sliE/cSXMu0ydAIhPD/azIIQQQvhNlpcKISLevmNq8mVgGDk1M4Fvn6OCu5+/uJeKxtD2V48XHT0Obvz7Jpo67SzIT+YXn5wf1LT/vLxkAHZV+Rast3XbPUsofe5Y12tg0qZC9ryRHz/1dMAEtbuh9ZhvrxFi3gWmofk67bI7cbjUsq9Bl5fGZajbLC3s1oNAo7VWqqoZsxXS/Kwn0ftau5vVtKcmLV4Fu63dDvZVt9LZ6yTOZjGuz7uzUYUC4L3CocyAOpgabWI9a87g70/XF5gGGqwfVrfBBpV6/Ujr8FUwQy0vLWvopNfpIs5mmZi1XMe2q5AyMVfVgvhr0kLv8sOXv6sCxvFoogXrfZnNKpRc+2P40nuQvwK6W+Cxy+CjP4/+okZPv3oQ0+q6E78KmNSkee2+wJ9ny9/Uz6XYVDj12+pzdslfISFHXQX10reCP9ZgbX9C/XmPToLTv2/Mc570NfWzoKMW1v3UmOf0V087PH6F+v8zWeCi+1TgP9zfDaMTVc/5SV+Dy/8Gt2yHbx+Gmz6A71XCF95W0+RLPqtOQPgSqvtq8olw03o46RZ15cCup+G+FWrqX5aeCiGEGGUSrAshIlq33UmZNt08KyfpuPdff2IRy4tS6eh18p3/7MDlkr9QD8flcvPNp7azr7qNzMRo/vrZZUEvf5w9KQmTCWpae6htG2YRlUafVk+Pt5HYNwAezp7n1O2ci32bzI1LU5NS0H8SOYzyU0Pbs65Pq1vNJuJsfX5PPctL+0ysQ3AByHD0Sfj0af4vR41JUYE8qD5XTUpsFGbtt/3t/ert8/KSsZiDmNLu6/C7KhjNnAULrtDe9l5wz9nb6Q0Sh5xY14L1+pLAXsOooFKvghmhY32oifWD2lVF07ISMBv1ezKWePrV/ayB6ev076mlfHV7YdPDxhxXpGkoVbcTMVjvKyELrntBVZ24XfDKd9WksKN39I4h2H71vjKmq351gA/+ENhzdDXBm3ep+6d/31ttlZAJlz2kwtNtj8G2x4M/3kD1tMO6n6j7q7+l6tWMYI1WATSoP/uPX6mWwh58AzoajHmN4bTXwaPnQ+k6iIqDTz8ReKd9XJr6eWeNNvYYBxMVC2f+FD7/hjp53VkPT38O/nU1tA3Syy6EEEKEiATrQoiIVlLbjtPlJjk2iuyk4/+ibjab+PVlC4mJMvNBaQOPbSwPw1GOHX98q4SXd1Vjs5i5/zNLyUmOCfo546OtTM1Ql2fvrhqkV3MAvxeX9nb4VwOj0y+vL13n+8eEUH6q+v8N1cR6a5d3cWm/KxA8y0u1EEDvPW8oUZd+Gy3QfnVQE4rx2tR6n551s9nk6Vl/c596+yIja2D0fvWpp0PRKer+sW1qojRQdXsBt+qN1yfxB9In1lsq1CJSfzVpE+vBLC4FbxVMW5VaIjdAhlYF09nrpKPn+Gnqg1q/+oSsgQFVhQDBBeuxqbBGm4B96y51FcV4MxE61n1ljVZTwWfdpULjrf+Av1+oQs5QazysvneYrVB0kjHPedLX1e2OJ6G1yv+Pf+fXanll5ixYen3/9xWdDKdpFTEvfiN0J4VH8v5vVU1Z6hRY+UVjn3vKKbDsBsCtJv/fugseuxR+PRV+Nx/+fa2q2jn8nqpgMUrjIXjoTHXFVmwaXPt84MtYwyVvKXzhHTjtdvU1vf9FNb2+9TGZXhdCCDEqJFgXQkS0votLh6orKcqI5ztaJcwvXpJKmKG8truae15X3dd3XjyPpZNTDXtuvQ5mjw/Buj6x7vPi0oOvgaMLUotUZYKvivVg/a1Bw8LRNloT6/0Wl0KfKhhtYj25AKLiwWX3Bl1G8gTrfvar6xK0BaYd/QMmPVjfWt4EwIL85MCefyC323tVQ/EaFTKnTVWTpEc+CPx5R6qBATXxGJMMuL3TvP4wamI9IUeFey7HcYtjQZ0806+CGGxq/YA2sT5xF5fqE+t+9qsPtPR6VXXV3Qxv3hn0YUWU7hbvvoeJPrGuM5ngxJvhqn+rapHyD+GBNVC9K7Svq59ILFip6jyMULAcCk9UP1c++rN/H1tfAhv/ou6ffdfglSGn3KZ2D9g7Vd967yj+Pe/YDnjmCypYBzjrZ6GZyD7vN3DD63DOL2H+p9RVXwDN5Wp5++t3qMnyXxTAH5fDM1+EDX+Bqm2B/R2nais8dJY6yZJSqF67b4f6WGK1wWnfVQH7pEXq+81zX4Z/Xqo+f0IIIUQIyfJSIURE21/jDdaHc+2qIl7eVc3Gw418++kdPPb5lROzkmAIB2rauPXJbQBcd2IRnwpiWelg5uYm8dy2Knb5sMD0SINacujz4tLdz6rbORf5t6Axf5kKK7oa1fRxsKFXkLwT66EJ1lv1xaUxA36062GWftm62aymyau2QO3ewCbLh1OvLS7NmBHYxw+xwDQ9PhpoR297WpifEtjzD9R4SP3D2xzlnd6cslq9/fB7MPPcwJ63VgvWh1tuZzKpqfXKTeoKghwf9gfonHZorlD3/e2yH8hihcRJqmO9tRKSJh33kMzEaI40dFLf3kNRRv+TYgdr1MT6jOwJOLHe0QDNR9T9SYuCey6zBc79P/jbebD5EVj2Of++JiKZfhIoPsu4MHe8mH6mqrN44kptgvgs+ORfYfb5oXk9I/vV+zrpFij/ADY9Aqu/qZ009MFr31cn9aafrZZ3DsZsgU8+APefrK4GevlbauI/VFwuKHkdPvyjqgrTLbgCZoXo98VkgoIV6peuq1n9/UVfEFq5FVqPqp+z9Qdgx7/U46KT1ImSySfC5JNUFZ7VNvRrlbwBT16j9qHkzIer/6MW0Y51OfPg8+vU79tbP1dXLP5plVp2GuXj3zmFGI7VpqqghBCiDwnWhRARbZ82sT5zkH71vlQlzALO+d17fHiogX9uOMI1q4pG4QgjX3NnLzf+fRMdvU5WTU3n++fNNvw15uX6vsBUn1j3aXFpb6eaWAfVr+4PS5QKSPe9oIKEsAframK9sjm0E+tJAyfWB1bBAGTNVv9IrzP4knq3O/iJ9UGqYADSErwhQVq8zfP5DJo+vVl4Ati0wLjoFNj8t+AWmNZoU6fDBeug+okrN/m/wLS5HNxOsMaqifNgJeWpUL3l6KBTi5kJKlgfOLFud7o4VK+CdcOWyY4l+tLb9OkQmxL88xWdrL7X7XlWdW9f+7x/JxQjlfSrDy9zpgoEn74eDr0NT14Na34Ap3zT2N9/p8MbFBvRr97X9LPU9/26fSpcP/nrI39MyTo48Iqq8Dj7ruEfm5AFlz4If78Itv5TfZ9eeKUhh+5h71J1Nh/e5z1JbLLA3Etg1VeCq3sKRGyKmtSfepr3be216vtO5Rb1s6NiI/S0qhMBJVptnjUG8perkH3yieq+Tfs71/Z/wXNfUSczppwKV/wTYob/O/aYYrGqr71Z58FzN0PFRypoF8II0UkSrAshjiPBuhAiou07pqpFZo4wsQ4wOT2e7547ix/9bze/eGkfp83I8r3He5xyOF189YmtHGnoJD81lvuuXkKUxfgWsLlasF7R2EVLp53kuKGXVuod65N9qYIpeV1d+p1S6F1G6o9pZ6hgveTNsP9FOE8Lghs7eunocRAfbeyP4NbuQYJ1p93bE9530ZoeehsdrLfXqNczmb2XsfsrYfBgPSPeG6wvzE8eshrKb/oSv77Bhd6zXr1LdV3ri/T84UsVDHg/T/4uMG3U+tXTpqirEIKVnAdHUeH6IDwLTNv7B+tHGjqwO93E2SzkpRh0smMsMWJx6UBn/UyFjWXvqcXNcy827rnDRf96lX71ocWlqcnhV7+nqlHevFNdVXTRfWpRoxGqtqrv0TEpkLvImOfUmc1w4tdUBcdHf4YTbhq+MsXpgFe1vQIrvuBd5jycKavh1O/A27+AF26D3CWQGeDVUX2118HHD6pf+lVe0Umw5BpY+SVIMfYqv6AkZKkedL0L3eVUJ3KPfABH1qvbzgb1/aNMW8Jttqq/Q6VOgZ3/Vm+bdxlc/OfhJ9vHsozpcP3LsP0J7wlQIYI1Got5hRBjjgTrQoiI1dTRS602HelLsA7w2RMm8/KuY3x0qJFvPr2df914woSuhPnly/t472A9sVEWHrhmmaen2mjJcVEUpMVS0djF7qoWTpyWMejjehxOqlrUxLZPE+t7nlO3/tbA6PSJvKMbobs1rFNZSTFRJMVYae12UNncZXhthr68tF/Huj6tbjKrIEWnB+tGL4HTg/rUIogKcDGuHqx3DJhYj/f+Y2aBUTUw/aY3+9QiJGZDxkyo3w9l78OcC/173vZaLZwxjTy5r4dJ/k6sG7W4VJekLTBtGSFYHzCxrtfATM9KmJjfa6u0xaW5BgbrKYWqVuOd/4PXfqgCNKOC1XBp1CfWDfp6Ha8sVvjEr9RVRS99E3b9R037f/oJSMoN/vk9i5pPVfUqRpt/Obz5M2g7Bjv+DUs+O/RjNz+ial1iU+HUb/v+Gqu/pQLkw++qvvUb1wX+56Nuv5pO3/4vcGrf25IL1EmBxZ8dG5PcZovaPzNpoTput1tN2+sh+5EP1AnTox+rXwCrboYzf2bMSdlIZjbD4qvVLyGEECJExvlPUyHEWKbXwOSnxpLg43Sv2WziV5cuJM5mYePhRv7+YVkIjzCy/WfzUR58X4Vv93xqIbMnhfYfiL7UwVQ0duF2Q0K0lfSRQn57F+x/Rd33twZGl1oEacXqkufDQdR6GMTbs2784jVPFUxMn2BdX1wam9b/H9BZWtjbUKKm2o1Sp106H2gNDAw5sZ7epwpmUUFK4M/fV+VmdQl9bOrx/dhTtKl1feLPH3oNTHqx9/L7oaRrwXp9iQpEfOVZXGpQUJmcr26HmlhPGDxYP6AH6xOxX93tNm5x6UAnfR2S8qGlHD74g7HPHQ5GLdqdKJZdD9c8p753H9sGD5+tfiYGK1T96jqrDU74srr/we9VV/lguppUBzbA6d9X34N9ZbbAJx9UtWG1u+Hl7/h3jI2HYdPDarHlfStgy6MqVM9dApc9DF/bpmpfxkKoPhiTSVULLfucqs65dTfcsgMuvl8tSL7g96p2Z7yH6kIIIcQokZ+oQoiItb9a1cDMGqFffaDC9DhuP1cFe796dT9dvU7Djy3Sbato5vb/7gTga2umce7845cRGm1enhasV7YO+Zi+i0tHrPIoWacWayXlBxdaTTtD3ZauC/w5DKL3godigaleBTPoxHr8gCsIkgvAlgAuu7f72AiefvUgFqIO0bHe90TMgnwfl+KNRJ/enDLI9OaU1er2cCDBuo81MKAFjSboaYGOOt9fw+hgXZ9YH6kKZmCwXqtOgM7InoD96i1H1e+Z2aoWABrJFgdn/VTdf+8e9VpjmadjXapgfFZ0MnzhLfVns7lc9YoHo7vVO7FcHKJgHdSiyOhkNTV94JXBH/POr9Vi8cxZKuz1V2I2XPoAYFLB+I6nhn5sVzPs+R+8cCvcuwh+v0jdL3lDffys8+H6V+DGN2HepeqqgfHEZILUybDo03DB72DpteE+IiGEEGJckWBdCBGx9teowGaWjzUwfV29cjIJ0VY6e73VIxNFbWs3X/zHJnodLtbOzubraw3oH/XBnFx1AmT3MBPrfi0u3fOs9sQB1sDoirVgvWSdfxPBIaBPrFeGIFj3Li/tEwroXbFxA4J1faINjO1Zr9uvbkMwsZ6r9XcXpceRnmBQx6U+vTnYEr/JJ6vbur2qf9cftVqwPtLiUlCVOSmF6n69H3Uwno51gyaAk4evgsnQJ9bbB1bBqO/T07Mm4MS6Pq2ePTfw6qPhzP0kFJ4Iji54/Q7jn3+0dLd4vxfJxLp/UovglNvU/fX3BneFUdn76uqttKnqeUMlJgmWf07dX3/v8e+vP6g65AHO/nngQfbU07wVMi983fv902lX9Sdv3gkPnAG/mgL//qyaUm86rE6EFZ6oJuW/uhmufAwmrxofS4KFEEIIMeokWBdCRCy9CsbXfvW+zGaTZ8KyfsCE5XjW43DyxX9upqa1h+lZCfz2ioWj1nusV8Ecqu+go8cx6GN8Xlxq7/bWwAS7uK/oZDBHQfMR75RvmOSFcmJ90CoYfWI9/fgPCMUCU/25MoI4maMH6z0t6utAsyA/mbsumce9VwawxHYw3S1wdJO6P9j0Znw6ZM9T9/2tg9GrYHyZWAf/e9ZdLmgqU/eNCiqTtCqY9mrVPT/AYN9P7U4Xh+vVVSjTJ+LEeij61fsymeDcXwIm1bV95IPQvE6o6SeB4jPHbr1GOC36jLqSp6UCdg4zmT0ST796CKfVdSu/BBYbVHwE5R/1f99rP1AB//SzvVeUBerU76hl073t8K+r4PEr4P+K4JFz4d1fQ+UmcLvUz6QVX4RPPwnfKYPPvaxCeVmmK4QQQoggSbAuhIhILpeb/dWBT6wDZGidzPXtvYYdV6T71Sv72VreTFKMlQeuWUZi35A1xDITo8lOisbthr3HBq+DKdOqYEacWC99E3rbIDEX8pYFd2DRCVB4gvd5w8hbBRO6jvX+VTD6xPowwXrtXmMOoKPB+3rBBOsxKSqQgX7VKCaTiatXTmahUf3qh98DtxPSp3knxgcqCqBn3eX0Tu77MrEOfXrWfQzW26pUJ7DZ6g3EgxWfqU5AuV1q8eAAniqY9h7c2pUfRxo6sDvdxNss5KWM8eWagajUgnWj+9X7mrTQW93w8nfU19dY41lcKtPqAYmKgRNvVvffuyfwr4HhrtAxWmIOLLxS3V//e+/bS9apehizVfV8B8tsUT3i8Zne6pnedvUzb95lcNF9qmP85o/VUtiZ50D0BLy6RgghhBAhM2aD9fvuu4+ioiJiYmJYuXIlGzduHPbxzc3NfOUrX2HSpElER0czY8YMXnrppVE6WiGEv442ddHZ68RmMVOUMcJ08xD06oL69okzsf7G3hoAfv7J+QF/3oLhWWBaOXgdTLmvVTB7nlO3cy4yZsGWHiSUhLdnPZQd623daso4abCO9YFVMABZs9WtURPr9VqYnFyoTmYEymQasmfdUL5Mb+oLTP1ZfNt4CBzdEBUHqT72n2dMU7cNJb6/BkDKZOP6gM1mSMpV9wfpWdeXx9qdbs9JHH1x6bTsxJF3Jow3LidUbVP380I0sa5b80PVWV29A7b+I7SvFQqefQAyHRywZZ+DmGR1Vcu+F/z/+OZy9f3FZPF+Xwu1E78GmGD/i2qxtdMBr35fvW/FF7xX6gQrMQeufEJVJ639CXzxXfhmCVz2ECz+jHcxsxBCCCFECIzJYP3JJ5/ktttu40c/+hFbtmxh4cKFnH322dTWDv4P8N7eXs4880zKysp4+umn2b9/Pw888AB5eXmjfORCCF/t0xaXFmclEGUJ7FvVRAvWXS43x5pVdcbiwtSwHMNcfYFp1fET606Xm4omH6pgHD2wXzvxOeciYw5Mv9y87D1whO8KBr1jvaGjl87ewetyAjXoxHqHNkE+cHkpeCfWG0qC6+3VGbG4VJeQqW47Qhis+zK9OfkkMJnV56j1+CnuQek1MJmzfD8p5O/EutGLS3X6AtNBFmVGWy2kxKmvLX2B6QGtX31G1gSsgak/qK6qiYoPbqeAL+Iz4LTvqvvrfqqWMY4lDfrXq0ysByw6UdWrALz3G//3hZRqJxLzl6mAfjRkTIdZ56n7H/weNj+idlbEpnm70Y1SsBwufwRO/rq6ysOIE/JCCCGEED4Yk3/ruOeee7jxxhu5/vrrmTNnDvfffz9xcXE8/PDDgz7+4YcfprGxkWeffZaTTjqJoqIiTj31VBYuXDjo43t6emhtbe33S4ixzOly8+tX9/G/7VXhPhSf6TUwswOsgYGJF6zXt/fQ63RhMZvITjRouaOf5mkLTAebWK9q7sLudGOzmMlJGmbR36G3oacVEnKgYKUxB5Y9X10q3tsOFRuMec4AJMdGkRijJoyrmo2bWnc4XbRrvfZJMX2Xl+oT64NUwSTngy1Bdd02lAZ/EHUH1K0hwXq2um2vCf65BtNUpsJpk0V18A8lNgVyFqj7vtbB1PixuFSnT242lfl24sfoxaU6fYHpIBPrAJn6AlMtWD+oTaxP6H71SQtVHUWorbgRMmaqP9Pv/Cr0r2ck/URQugTrQVn5JXUi59h2KPXz6qvR7Ffv66Rb1O2OJ+Etrfrl9O9BbHhO/gshhBBCGG3MBeu9vb1s3ryZtWvXet5mNptZu3YtH3744aAf87///Y9Vq1bxla98hezsbObNm8fPf/5znM7BOwp/8YtfkJyc7PlVUFAQkv8XIUbLuwfquO+tUr799Ha6esdGP+u+msAXl+oyElV1QV3bxOhYP6oFtTlJMVgDnPIPlj6xfrC2nW57/681fXFpQVosluEWqu5+9AJNdAAArdJJREFUVt3OudC4qTOz2TuZ7G8gYTB9ar3CwDoYvQYGBlTBdAzTsW4yeUPwOgN61qu2qlsjpnfjtYn19rrhHxcofXqzYMXIyxT9rYOpDSBYT5ykTnK4nd6lpMNpDNEEsGdifYhgvU/POngn1qdnT8DO4srN6jbUNTA6SxSc83N1f+NfvD3+Y4F0rBsjLg2WXa/uv3eP7x/ncqoT1jA6/ep9FayAwlXg7IWuJvXzYen1o3sMQgghhBAhNOaC9fr6epxOJ9nZ2f3enp2dTXV19aAfc+jQIZ5++mmcTicvvfQSP/zhD/nNb37DnXfeOejjb7/9dlpaWjy/KioqDP//EGI0rdunpj677S7eOxiioMpg+sR6UMH6BJtY1yegc1OGmQYPsdzkGFLjonC63J7QTeddXDpC97seYM4639iDK9bqYMK8wFRf8mhkz3prt6pyibNZ+lcndQ5TBQOQqfWs1wbZs97ZCEe1XSdTTw3uuSD0E+v+TG8WrVa3Pk+sa1UwWXN8Px6TCdK1/ukGH+pgmrSJdV873H2ldxEPMbGe0Wdi3e50cbhe/ZmeMSGDdX1x6SgF6wDT1sKMc9VVJq/c7n8dSDh0t3qXEEuwHrxVX1HLnY+shyODDxQd59h2FWpHJ4V20e5Q9Kl1gLN/btxeCCGEEEKICDDmgvVAuFwusrKy+Otf/8rSpUu54oor+P73v8/9998/6OOjo6NJSkrq90uIscrtdvPmXm9P8Wt7QhRUGajb7vQENrNyAv/zN9GC9UotqNWD23AwmUzM03vWK/vXaPm0uNTeBa1av3P2PGMPrlgLUY9tD90ktA+8C0w7DXvO1i41sd6vX93lUoE3DL68FCBLmy4PdoHpwdfB7YKsuZBSGNxzASRoy0tD0bHucsKhd9R9X6Y3J69SlTFNZWoB4HB62r0T5/5MrAOkawtMR+pZd7tDVwUzTMc69J9YL6vvwOFykxBtJTc5fCfzwsLRA9U71f3RDirPvksFq6Xr4MCro/vagdCvrojLGL1u7/EsKRcWXaXuv+/j1Lp+MnnK6vCE2tPPhlU3w+k/8O47EUIIIYQYJ8ZcsJ6RkYHFYqGmpn84WFNTQ05OzqAfM2nSJGbMmIHF4u3AnD17NtXV1fT2ToyKCDFx7T3WRlVLt+e/1+2tweF0hfGIRlZS247T5SY5NorspMC7wjP7BOvusTDZF6RKz8R6+IJ1gLm5+gLT/j3rR/RgPW2YYF0PDKOT1WXvRkrIgpz56r4+sRwG3mDduIl1fXFpUkyfYL27WVWLwOBVMOCtbQk2WD/wsrqdeU5wz6PTg/X2EATrVdvU5yY6GXIXj/z46ETvVPLhEabW9c9jQvbQVwkMRV9gOtLEeked2hWACVIn+/caIxmpYz3RO7F+QOtXn5aVgMk0TLXTeFSzC1x2tYQxxeDfg5GkF8MJX1b3X71dhfyRzNOvXhze4xhPTrpFLVU++Boc2zHy4/UamKmnhfKohmY2qxNCp34rPK8vhBBCCBFCYy5Yt9lsLF26lHXrvB25LpeLdevWsWrVqkE/5qSTTqKkpASXyxsmHjhwgEmTJmGz2UJ+zEKE05taDcxpMzNJjYuiqdPOx2VNYT6q4fWtgQkmsNE71rvtLjrGSLd8MPQqmLzU8Abr8/LUVQa7Byww9akKpu+Su1CEdXodTEn4etb1jvVQVMEkxQ6yuDQ6CaxD/KzTg/WGEt+WZg7Gafd+PmecG9hzDBQfwmBdn96c6sf0ZpHWsz5SHUwgNTA6fYFpfcnwj9NPPiXng9XgJcVJWhVMR92ggW3f5aWefvWsCbi41FMDszQ036dGsvqb6uRN4yH46M+j//r+kH5146VNhXmXqvsjTa33tEP5R+r+aPerCyGEEEJMAGMuWAe47bbbeOCBB3j00UfZu3cvN910Ex0dHVx/vVqGc80113D77bd7Hn/TTTfR2NjILbfcwoEDB3jxxRf5+c9/zle+8pVw/S8IMWre0GpgzpqTwxmzVW/xa3sG30cQKfZrgc3sIPrVAeJsVuJs6kqV+rYIn+ozgB7UhntifZ42sb63ug27dnWE2+32LC8dtgom1CHMtD49667wXLmhT6xXhmBiPdnXxaW65HywJarOZv1z768jH0BPq6p6MKpv2lMFE4LKHn/61XWeBabvDd9rXRPA4lKdXgUz0sS6Z3Gpwf3qoK4SsWq1LoNMrfedWD9Yq75PS796GEQnwtofq/vv/cb7Zz0Shaq2aKI7+VZ1u/vZ4U/GHflAXV2RUii/B0IIIYQQITAmg/UrrriCu+++mzvuuINFixaxbds2XnnlFc9C0/Lyco4dO+Z5fEFBAa+++ioff/wxCxYs4Gtf+xq33HIL3/3ud8P1vyDEqKhr62H70WYA1szK4uy5qi7ptd01EV2Nss8zsR78foOJ1LOuT6znhzlYL0yLIzHaSq/DRUmtqouoa++hs9eJ2eSd2B6UJzQMUQBQcAJExavubn26eJTpwXp9ew/ddmOupGgdrApGn1gfrpLEZILMmep+oHUwB15RtzPOBrNl+Mf6Sg/We1pV775RetqgYoO678/0ZsEJYI5S/f/64tDB1BoQrHc2eLvxBxOqxaWgvh70nvXWquPerQfr9e09HNSqYKZnT8SJ9c3qNjdMwTrAgith0kL1Z+TtX4bvOEYS6u/pE1X2XJj5CcAN63879OP0K3SK14Tn6gohhBBCiHFuTAbrADfffDNHjhyhp6eHDRs2sHLlSs/73n77bf72t7/1e/yqVav46KOP6O7uprS0lO9973v9OteFGI/e3l+L262qOXKSYzhlegaxURYqm7vYc6x15CcIk/3V6thmBjmxDpCRoCowxnuw3tZtp7VbLbAM98S62Wxidq5WB1Olfi/1xaW5KbHYrMP86PGEMCHq47XavNPHeuAwypJjo0iIVhUkRtXBeDrW+06sd+oT6yN0fet1MLUBBOtuN+zX+tVnGNSvDqq+xqLVnBhZB1O2Xk3npxb5N/Fti4P8Zer+4XcHf4zbHVwVTHQCJOaq+w3DXD0Q6qBS71lvGXpivaGj17NgesJNrPe0Qf0BdT9cE+ugeqvPukvd3/Qw1B0I37EMp0GqYELm5NvU7fZ/QXPF4I8J5AodIYQQQgjhszEbrAshRrZOq4E5Y5a6miMmysLqGSpke213zZAfF05NHb3UtKoQ3JhgXasuaB/fi4qrmtWC2pS4KOKjfeyNDiG9DmaX1rPuWVw6XA0MQMMoTDfqPeu7nxm+1iNETCZTnwWmnYY8p7dj3c8qGIAsfYHpXv9fuP6gmqC22KDYwODGZFId0mBssN53etNfU1ar26EWmLZVQ1eTWiqon6zwV4YPdTChDtb1nvXWo8e9KzXOhsVswu0Gh8tNQrSVSckxoTmOSFW1DXBDcoH3yopwmXKKmlp2O+H1O8J7LIPpaVNXB4EE66FQsFx9X3I54MM/Hv/+lkp1JZLJ7P3+JYQQQgghDCXBuhDjVI/DyXsHVT/xGbO9//g/a46qg3l1d2T2rOs1MPmpsZ6p3mBk6NUF47xjvbJZmwhPDu+0us6zwLRKD9bVdGth2jCLS+1d3jAvPUQT66CWvllj4dj2kZdRhog3WDdqYl1drdCvY91TBTNCsJ45W90GMrF+QJtWLzpZ9T4bKSFT3XYYGKwHM73Zd4HpYCdkaner2/RpEBVg2JyuLzAdLljXO6tDUAUDw06sW8wm0uK9i3CnZSUEtWB6TNJrYMI5rd7X2p+AyaL+LA51NUW46CeB4tIhNiWshzJunfINdbv5UWgfsJPi0NvqNnex2p8ghBBCCCEMJ8G6EOPUhkONdPQ6yUyM9kwPg+pat5hN7Ktu89RzRBK9BmaWAdPqMHE61iu1ifW81EgJ1tXX3O6qVlwuN0e0xaVFw02sN5Wp2+ikkaesgxGfDouvVvc/+EPoXmcYes98ZbMxwbq3Y73PySg9WB+pCkafWG8sBYefV3bs1/vVz/Xv43zhmVg36OqalqOqwiPQ6c385aqepr1m8OC7RgvWA6mB0WVowfpQE+tdzdCl9a+HomMd+nSsHx+sA2Rq31MBZkzEfvUqbXFpOPvV+8qcAcs+p+6/+v2wLWUeVKirvQRMORXyloKjCzb8uf/7grlCRwghhBBC+ESCdSHGqTf36TUwWZjN3onC1HgbK4rU5NJreyJvan1/jb641JhgPTOEHeu9DhcflNQbtoAyGJXa5HNemPvVdVMz4omJMtPZ6+RwQ4dvVTB9Ky5CPQV7wpcBExx8DWoDqEAJktET63oVTPJgVTDDLS8FFaTaElWdQOMw3d4DdTZCxUfq/kwD+9V18drE+sApzECVatPqeUsDm56NioGCFep+2SCTwTX64tJ5AR0e0GdivWTw9+uLS+OzVCd7KCRrVTCDTKyDt2cdJmC/OkClFqznLQ3vcfR12nfVCcnqHbDjyXAfjZf0q4eeyeSdWt/4AHSrq8RwubwT69KvLoQQQggRMhKsCzEOud1u1u1TU55rZh3fAXvWXDUJ+tqeyOtZ16tgZuUkGfJ83ol14zvW//5hGVc9uIH73/EjjAyRqubICtatFjOzJ6nfw12VLZ4qmMnpw1TBjGYIk14Msy9Q9wfrpg0x/ffJqI714ZeXjjD9bzJB5kx135+TDCVvgNsFWXMhpdCPo/WR0RPreg1MMNObw/Ws61Uw2cFMrGsd642HwDXICbtQ96sDJGkLVAfpWIf+wfr0iRast9dCSwVggtxF4T4ar/gMb7i67qfQGyFXo+m1RaGs9hLqiqHM2dDTCh8/qN5Ws1P9DLAlqKtthBBCCCFESEiwLsQ4dLC2nYrGLmxWMydPP35a9ay5qmd9U1kjDRFUkeJyuTngCdYNqoJJDF0VzAFtun7zkSbDn9tfeqVIboQE6wBzc1Ww/tGhBpo6VfBbmObDxPpohTAnfk3d7vi3Wjw5ivQqGMMm1rWO9aSYvhPrPlbBQJ8Fpn70rO/X+tVnnO37x/hDXwxpRMe6UdOberBe9l7/yg2nHf6fvfsOj6ws+wf+PdMzKZPes733yrILUhdWeUGaAoLSFEQBC2Lh/SkoFl4VAQUURRFBELugIG3psOwuuyzbK7ub3U3dJDNJJmUy5ffHc54zk2Qm086UJN/PdeU6JzNnzjybzAS9zz3fu3WP2E8mCsZRJ+JmfP2As374/WkprKtRML0dYQu0gwrr5eMsCkZ2q5fN1H+mQLJW3AA4JgBdDcC6BzO9GqGdHetpYTAAJ39V7K/7pXjfyk/oTDoZMFkiP5aIiIiIksLCOtEYtHaXKEStmloCu2X4ANCawhzMqymAPwCs3a3jYMAkHXP2wu3xwWI0YFLpCJ3NcdA61lMwvLSpU5xzV2On7ueOl9axniUZ6wC0bP8XdoiO49I8K3JHGkib7iJM3XKg7kTA5wE2/CY9z6mSUTCtXf1JRwkFAgEtY91hT2B4KRAcYBprYd03AOxfK/ZnpiBfHdA3CqZpq/h5WPKB2mWJn6d6CWC2i3O1hnT3tx0QryNLHlA4MfHzG4zBC0ttYeJg2g+JbaoGlwKAzSH+HUDYnHWZsZ5vNaHKkeCQ1tEq2/LVQ5ltwOo7xP5b9wJdWfCJtHRcCCJh3sXik0M9x4H3H2e+OhEREVGasLBONAat3SX+D/WZYWJgpLPniK71F3dkT866LFBPLc+D2ajPn6dSNWPd7fGh16NvFnqzSwwMPd7tQWsKCvexGvD50dwp1lJdmD2FLjnAtN0tYnhGHFwKBGMD0jnobtXNYrvxd0B/d9qettBuRq7FCCB4USRR/V4/PD7RPa0NL/W4xTA7ILaO9TK1Y70lxsJ6/Tqg3yXOnaqsaT2jYGSRafJHAKN55GNHYrIAE04U+6FxMDIGpny26B5NRokaBxNuQGo6CpWKEuxadw2Pg5F/Y2ZW5kNJ9SyEbHNsk9jWZGFhHRDF1ZqlwIAbeO1HyZ2r1wn8+TPAHz6e2N/G/u7ge5eF9dQzmoCTviL23/45UK/Ov2C+OhEREVFKsbBONMa0uz3YXC+iSc6YXRHxOJmz/sa+43D3e9Oytmj26BwDAwB5VhOsJvGnTu84mCa1mA1ktmu9ydUHfwCwmAwozbVGf0CaTK/Ig9kYLLxNGKmwPtAXLOKlswgz82OikN/nBLY8kbanVRRFtzgYma9uUMTrHUBwcKnRClhi+PSHjIJpPwB4Y5hHsOd5sZ2xRnRZp4IWBaNDx7rMV9ejyDTpI2J7KKSw3iwL60nEwEil6gDTtjCFdTm8tCiFHesA4FAL62E61k+fVY6vrp6B28/T4d86mgQCIYNLs7SwrijAGrWgvvmx4EDdeLmOAb//GLDrGeDg68Cm38d/DnkRyF6S2LBgit+iK8QFyc5jIk6qoDb494SIiIiIUoKFdaIx5vW9LfAHRHF6pEGWMyvyMaHYDo/Xjzf36VC40sFuNbN8po6FdUVRtDiYVh0L630DPq2gCQC7mzJXWNfy1R02GAzZ00FqNRkxI2S44aSRBpd2HAIQAKwFYhBfuhiMwMobxf66BwFf+i4yydieZAvrnSGDS7UOYjm4NLdUFNuiKagRP3u/N3wESahAANib4nx1IFhY93SLDvxEeXqC3Zt6xCJoOetvBQeMygJmxbzkz1+iFsKGdqx73EBXo9hPZRQMENKxPrywbjUZ8eXV07GgtjC1a8g2HYeA3nbAaNHn95wqE04EZn9cDBZ+6TvxP75lN/C7s4CWnYBJ/QTUO/cDA3H+nWIMTPqZbcDKm4LfTz0ttr//RERERJQwFtaJxpiX1Xz11SN0qwOi4Hz2HHHMizuyIIsVqelYB0IGmOoY19Ic0q0OALsau3Q7d7yyMV9dkjnrADBxpI51LV99cvoLAQs/JboqnYeB3f9O29PWaoX14QMi4yEv8Dhywg0ujSFfHRA/87KZYj9azvrxfaJoZrSkNr/XkgeY1Nd0dxKzIA6/I/LPHXX6DMatWiSy2vucQNM2cZvsWK/Qs2N9yAWOjkNiaysE7MXJP89IHLVi2zk8CmbckvnqFfMAU/Z8Miiss74HGMzA/peDsxBicXgd8Mga0fFcOgP4wjvifdPdDLz/x/jWwMGlmbHsGvE3AmC+OhEREVEasLBONIYM+Px4Y4/oPj9jduR8densuSJnfe3uFgyoGc2Z0u/14eBx0ZU6q7JA13OXqTnrx7tjiLiIUZNraGE9gx3rHbJjPfsK63Nrgr/LCcUjFdZld2Ma89Ulix1Yfp3Yf/sXoiM7DWr16ljvUzvWbeEGl8bR/R9rYX2vGgMz6WTAqu9FsEEUJdi1nkxhXRvid7o+F22MJmDiKrF/6E2grxNw1Yvv9YiCkRnrXY1Af8gFO20GQYq71YERO9bHrWyPgQlVPAU44Xqx/+J3gp+sGMmufwOPXyAuGNWeAFz7grgQddKXxf1v3RdbTJSUyb/p45k1H7jkD8Cp3wRmn5/p1RARERGNeSysE40hGw+2o6vfi5JcCxbG8DH9pROLUJJrgat3ABsPtqd+gSPY39INnz8AR44ZFQX6dgPKKBg9M9ab1e53WRzd39INjzczFycaXNnbsT43pGN9xCiYtgx3Ny7/nIg9aNgsOpyT4fcDz35NDP3zRn7NBTPWU9CxLqNgYhlcKpXNFtuWXSMfJwvrMz4W+7kTpeWsJ1FY1zNfXZqs5qwffDP488qv0qeTPKcQyC0T+6Fd6+mM1hghY33c0grrKRrWq7dTbhWdyy07onebb/wd8JcrAW8fMPMc4Mqng6/lxZ8B8irFpxc++FPsz9/GKJiMmXIacPr/iouARERERJRSLKwTjSFrd4vi0+mzymGMIWvbaFC0yJgXdjSldG3R7G4M5qsrOkeBaBnrekbBqB3riycUId9mgtcfwP6Wbt3OHw/Z8Vw9Qqa+brwe4K17gcatMR0+t7oAE0vsWFRXiEK7OfKBmc7jzSsTkTCAyBNOxst3ABt/K4b+HdsU8TB5UUZm5Ceqs1fkwhfkhBRR5PDSWKNggOAA05E61nvag3nlqcxXl/LUSKvuBOOqOhtFVjQUUWzSixxgevgdoEl9L1TM1e/8Ws56SGE9XYNLATH0EAA6G1L/XKOBzws0bhH71aOgYx0QhfFTvyH2X/0h0B/mv0+BAPDKD4BnbxGZ7EuvBi55XHyKRzLbgJO+JPbfuif2ORTyb3oJC+tERERENHZlRWH9yJEjuPbaazO9DKJR7xW1sH7mrOgxMNLZc9Wc9Z3NCKQpAiOcPc2pyVcHgFItCka/wnqTmrFe5bBhthpdk6kBpjJjvTYdhfUNvwZe/i7w3NdjOtxmNmLtLafiH19YNfIFExlzoUcGdqJW3gRAEYM5W/cmdo5NjwLv/CL4vfNIxENlx3pzZz/6vTFENUTQOVLHem4chXXZsd52IHLkw/61QMAnIk+KJiaw2jjJzu3uBAcsf/ia2FYv0jeXvHK+6Ab2dAFbnhC36REDI5WqcTBtIQNMM9Gx3t8pom7Gu/cfBwZ6AJsjmIE/Giy/TlyI6W4e/HcJEAXyZ24C3vip+P6024Bz7wvf5bz0anGRruMQsP3v0Z+3vxvoVi/Ws2OdiIiIiMawrCist7e34w9/+EOml0Ep0uvxZSwiYzw50NqNg8fdMBsVfGRGWcyPO2laKewWIxpdfdh+LHMFlN1NwY51vWnDS1NQWC/Pt2J2lVhzJnLWA4GA1vGc8o51nxdY/xux3/hBbLm9AExGAwwjfYJioA9wqQXoTBZhSqcBs/5H7K97IP7HH3gV+M8tYl8Oj3PWRzy8yG5GjtkIAGhw9kU8LhoZBTM4Y12NdoonCqagGrAWiML50MGZ0t7/iu2Mjyaw0gQkGwWj5avrPMTPYBQZ8wDQ8L7YpqJjPVNRMJbc4Gt4vMfBdBwGXvy22D/lG+J3P1qYLGKQKSDmR8hPIHjcwFOXi4gYxQCc93PgtG9FnkFgyQVW3ij237xbxF2NRH66IqcYyClK/t9BRERERJSl0hK+98wzz4x4/4cffpiOZVAGdPd7sebeN+DIMePZL52se8QHBb2ySxSeTpxSgjxr7G9tm9mIU2eU4b/bm/DizibMr3VEf1AK7FG7vfUeXAqEZqzrN7y0RS2sVzpsyFV/3vLiQDp19Aygb0AUOaoKbal9sj3PBYc0entFZ3PZjOTP6zwMIABY8oMdypmy6mZg93+AD54Czvh2sLAbTctu4C9XiaL0gktF8fO1u9R/W3iKoqC2KAf7WrpxtKMHk0tHyKAfgTa8NLRjXUbBxDO8VFHEANOjG4HWXUDFkA5s3wCw72WxPzMN+epAcsNL/f5gx7qe+erSpI+I14qkZ2FddkUfVzvWvR7AdVTsp2N4KQA4asUgS9cxoHx2ep4z2/j9wNM3Ap5uYMIq4MQvZHpF8Zv9cWDCSqB+nYh9Oev7wJOfFDFVJhvwid8Ds86Jfp7l1wFv/xw4vhfY9Qww94LIx2Z6ZgYRERERUZqkpbB+wQUXQFGUEWMmWHAdm17Z3YJjzl4cc/aiubMflY4UF/7GsZd3iQzieGJgpLPnVojC+o5mfO3smXovLSpnjwfNnaKbPCUd67KwrmPGuuxYryywobZIfPgnEx3rx9R89bJ8K6ymFHdSrn9o8PdNW/UprGtFmMmROybTpW4FULtcFJc3PAyc8f+iP6a7FXjyEqDfJQpYH78/GJfgihwFAyCksJ54zrrWsR52eGkcUTAAUDZL/NtbwuSs168T/0Z7SfoGOOYmUVg//JbodLc5xO9Vb3KAKQAoRqBUh/eCpHWsHxA52M56kYFttgdz51OtoAZo3i6GVo5XGx8GDr0pfu4XPDi6utUlRQHO/gHw2zOBLU+Kf4+zXnSSf+rPwIQY3xu2AmDFDcDrPwbeuBuYc37kv9davnoGo72IiIiIiNIgLVEwVVVV+Mc//gG/3x/2a/PmzelYBmXAC9uDAzH3taS/m3e8cPUM4L3DHQCAM2bFX3Q5Y2YFTAYFe5q7cOi4W+/lRSU7vWuLcuLqto9VmVpY7+r3om8g8SxrKRAIaBcCKgpsmFGRB0URHfF6DkiNhYyBqUl1DEzjVuDw24DBBMxUuxvl0MZkZVMRRlFE1zogimqenpGPH+gTkQrOw0DRJODSJwCTFSicIO4fIQoGCOasH0uisK4NL7WFDi9tE9t4omCAYGdyuAGme18Q2+lr0ldgTGZ46dY/i+3cC0Ukht7KZgcvXJROF793vRRNFO+1AbeI7wgdXJqui08F1WLrGqdRMG0HgJfuEPtn3Tm6u69rlwHzLgagXqRx1AHXvhB7UV1acQNgyQOatwX/HoST6WHURERERERpkpbC+tKlS7Fp06aI90frZqfRqW/Ah1f3BLsM9zZ3Z3A1Y9tre1vg8wcwvTwPE0rscT/eYTfjxCmiQPTizqYoR+tvd6OMgdG/Wx0ACnJMsBjFnzs9ctadPQPa3IDyAivsFhMml4gYj3R3raetsC671eecD0w/W+w3bdPn3O1ZFhsw61xRJO/tCA6mDCcQAJ7+InB0g+iKvvyvwWGhWmH9yIh5xLVF4vd2tCNKAX8ErqHDS30DorMciC8KBhBRMED4wvoeNV99Zpry1QEgT40GcreKn3esBnqBnWoM3fxL9F8XABgMIg4G0DcGBgCMZvEaBMQAU61QmaYYGCA4wDTZjPVAAHjpduC1/4vvd5hJfh/wry+IyKvJpwLLPpvpFSVv9XfFpxBqlgKffSn4Xo+HvRhYrv4s3vhp5N+n9nrNgoulREREREQplJbC+te//nWsWrUq4v3Tpk3Dq6++mo6lUBq9ue84ejzB7uD97FhPmVd2iwsYZ85OPCLg7LnisS/uSKAzNEl7mlM3uBQQF+9K8kTHqh456zIGpjjXosWvzK4S2fDpLqw3yMJ6UQoL692twLa/iv0VXwCqFoj9xq36FMqyrQhjMAIrbxL76x6MPKT11R+JyBeDCbjk8cGxOPnVIh7EPwB0R75YVaMV1pPoWB+asd6jdqsrhuAAyliVqR3rbQcAb8hFqOP7xAUQg1n/QaAjkVEwAz0i5zpWe58H+jtFZ+6ElalZGyA6eIsmA4s/o/+5S0Jy1jPRAVxQK7auJKNgWnaKbO7X7gpGJGW7dQ8CR9aLuQ/nPyguoox2hROAL38AXPcKUFCV+HlW3gSYcoBj7wVnGAzFjnUiIiIiGifS8v8UampqsGbNmoj35+bm4tRTT03HUiiNXtghiklVaq76vnHUsd7S2YedDekpsHp9fry2pxUAcObs+PPVpdVqUX5TfUfa40xkFMzMFAwulfTMWZeF9YqC4MwA2W2f7gGmMkKkOpXzCzb9HvB5RKdj3XKgfI4o2vYcB7p0+IRDWxYWYRZdLjKIOw4OHlApbfkT8MZPxP659wFThvw3zGgS3aGA6FqPQEbB6JGxrnWsy8GlOcXxFwQLqgFrgRjCKrPvgWC3+qSTAWtqLoCFZc0DzOpQ13hy1rf+RWznfzK1RdGJK4EvbwGmpmA4auk0sW3bD7SrUTCjsWP9yPrg/nNfTywvP51adoshnwDw0R8BhXWZXY+ejObox0STVw4svVrsv3H38Ps9bqCrUeyXZNHfdCIiIiKiFEhLYX369OlobW3Vvr/00kvR3Jz+rlhKnwGfXxumecOpogt1b3PXuIn8ufr3G3HeA2+hydWX8ufadLgDrt4BFNrNWDKhKOHzVBfmYEGtA4EAsHZX+t6ffn8Ae9Vi9OwUdawDQKnWsZ58Yb3ZJQvrwUzljHWsu2THevwRQDHxeoCNvxX7K74gtuac4KDGZONgvP3BAZ/ZkLEuWXKB5Z8T+2//YnBn/qG3gWfUHPaTvwosidCtHEPOuoyCae7q0+KF4uH3B9DdLzPWZce6WliPNwYGEPndZbPEfuuu4O0yT3nmx+I/Z7Ly4hxg2tMO7HtR7C+4NDVrSoeMd6yrhXXXseQ+mVIfUljvbQeeuzW5daWSzysiYHz9IvIqFZ9EGAtW3QwYLWJA8OF3Bt8nLwLlFIkvIiIiIqIxLC2F9aHF1Oeeew5ud/oHJFL6bDjYDmfPAIpzLbhkWR0MCtDZ5017J3SmHGjths8fwKG21L/OZQzM6TPLYTQkN9Tu7DlqHMzO9BXWjzl74fb4YDEaMKk0N2XPo3Ws61FYVweXVoZ2rFeJiwL7W7oTKpAmSutYL0xRx/rOf4nBkXmVIl9dqlTjYJIdYNpxCEBADMTLLUvuXHo74XpRPDr2XrDrtu0A8OcrRMTL7I8DZ9we+fFaYf1wxENKci2wmQ0IBIBGV/xd6119Xq3mWZCjDi/tSXBwqSSzl1vUnPXeDqB+ndifkcZ8dUkW1t0xFtZ3/BPwe8VrtHxW6taVaqWysL43+BoqSmPHuiyse3vFayBR8r1z5h0iNmnn08COfyW9vJR4+16gYbOYmXDeL9I3KHa0cdQAi64Q+2/8dPB92syMLLpQSkRERESUImMgNJKy0fPbRTzEWbMrkGMxYpI62HFfy9iPg+n1+NCvFlY73MnneUcjPxlwxqzEY2Cks+dWAgDe2ndc64JNNRmdMrU8D2Zj6v4klebLwrp+GeuhUTA1hTnIt5ng9QewP02v874BH9rU11htYQo61gMB4N1fiv3lnwNMluB9lfPFNtnCemgnbrYVsfLKgYWXif137hed0E9eIoqM1UuAC389csxIDB3riqIkFQcj89VtZoOW9w+3WliXg1TjVa7mrMuO9X0vi2iY8jlA0cTEzpmMeDvWZQzMghQNLU0X2bHuOiKimAxmwFGbvuc324IXZxLNWe9uEXFKUMTQy5O/Km5/9mvB12m2aNoGvPZjsf+xnyaXQz4enPwVMUfiwCvA0U3B25mvTkRERETjSFoK64qiQBlSMBn6PY0dfn9Ay1f/6DxRqJ1WngcA2Nc89geYdvQEC7dtKS6sHzruxoFWN0wGBafMSL7bd3p5HiaV2OHx+fH6ntboD9DBniYRnTIrhTEwQLBjvVWXjvXhhXVFUTBbzYjf3ZSeOJhj6uDSXIsx2K2spyMbgIb3AaMVWHbN4Pu0wnqSUTAyxztbizByiOnuZ4HHLxR514464FNPAZYoFzNkNrMrcsY6EIyDOdrRE/fyZL66FgMDBKNg7AkW1rUomD1iu/d5sZ0ReVZKSuXGUVhvPwgceReAAsz7REqXlXK5paJzWiqaJAbrplOyOeuyW718jvi3nPJ1MSC35zjw32/os0Y9eD3AP78gPoky69zRf1EmHYomBaOW3gzJWs/2v+lERERERDpKWxTM1VdfjYsuuggXXXQR+vr6cMMNN2jfyy8aG7YcdaKlqx95VhNWTROFnRkVomi6dxx0rDt7BrT9VHesyxiY5ZOKg4MLk6AoCtaoXesv7tRhKGUMdmmDS1NdWFcz1nWII5KF9UqHddDts9U4mHTlrDc4Zb56TmouVq7/ldgu+OTwvG4ZBdP+IdCfxAUz2d2YTfnqocpmqvEnAaBxC2DJBy7/M5BfEf2xMXSsA+LTDkCCHetDB5cCweGliUbByI71tgNiEOH+l8T3MzKQrw7EFwWz7W9iO+XU0d9xrCjBrnUgvYNLpQK1Qz7RjvX6d8W27gSxNVmBCx4Uw4+3/01csMoGb/wUaN4mBv6ee2/2fXomW33kFgAKsOe54EVWmbGerX/TiYiIiIh0lJbC+lVXXYXy8nI4HA44HA58+tOfRnV1tfa9/KKx4QU1BuaMWeVaNMH0CtGxvr95PBTWg8X09p7UFtbX7hYxMGfOTj4GRjp7rigYvrK7JS1Z4XvSVFgv0zVjfXjHOhAcYCrjbVItmK+eo//JXUeBnc+IfTm0NFRuSTCDuWl74s/TPgq6G1d9SWwVA/DJ3wMVc2N7nFZYPwL4I7+X9IiCKcgJ07GeyPBSAMivAqwFIv7l/SeAPpfofq9dltj5khVrFEwgAGz9s9gfzUNLQ5WGFtYz8B7ROtYbEnv8kQ1iO+HE4G01S4Pvqf98VUQsZdKxzcCbPxP7594TfL1RdKXTgbkXin35MxwNf9OJiIiIiHSSguyA4X7/+9+n42koCwQCATyvxsDIzmcgGAWzt6ULgUBgTEcBdaSpY72rbwDrPxQFiTNnx9A9G6NFdUUozbPieHc/1h9sw0emp26gZL/Xh4PHxYBXGaOSKnplrHu8fu0clUMK67PUwnraO9ZTUVjf+FtRWJ30EaByXvhjKueLiIimbcDElYk9j5bHm8XdjZNOAi56WBSXp50Z++MKakQx3tcvuq3zK8MepkcUzKCOdVmoTDQKRlFEHMzRDSJbHgCmr0l/DIkUaxRM4xagbR9gyhFxHmNBybTgfiYKlQVJRMEM9InfCRDsWJdOu010OR/fC7zwv8CFDyW1zIQN9AH/+oL4Wzf3omCRmGJ3yq3Ajn+IgbQnbQG6GsXtLKwTERER0TjA4aWkq91NXTjc1gOLyYDTZgYLslPL8mBQREyKHsMjs1m6Mtbf2HscXn8AU8pyMbk0V7fzGg0KzpojClkv7mjW7bzh7G/phs8fgCPHjIoCa/QHhAoEAE+P6KRs2QUcXgfseR744Clg/a+B138CPP+/wAeig1VmrLt6B5LqxG/pEt3qZqOCIrtl0H0zK/KhKKJ4L49LpaPOFHWse3qATY+K/RU3RD4u2QGm3v5gxES2F2EWXBJfUR0AjGYgv1rsOyPnrMvC+rGEomDEkOECW8h1cneSGesAUK7mrLvUGJtM5asDQJ564TBaYV0OLZ35McCW2gt1aRPasV6UgSgYOSzVlUBhvXGLGLqaWz587WYbcP6DABTggz8Be19IdqWJee1HQOtuscb/+Vlm1jDaVcxVL2QFgH9/WdxmKwTsxZlcFRERERFRWqSlY53Gj+fVGJhTppch1xp8ednMRkwotuNQWw/2tXShLD/OIuooIjtIgcFFdr1pMTCzYvzYeuNWUWiTH+0fwdlzKvGnDUfw4s4m3HbOLNgtqflTERoDE/VTDAO9wN8/J4ogvU4RT+EfGPkxAAAFmLYahTnFMBoU+PwBtLn7UeVIrBjd3CmiZMrzbTAYBq85x2LE5JJcfHjcjd2NXSjPt4U7hW5kx7oszOpm21+A3g6gcKIoUkYic9YTLax3HAYCfsCSN3bjFwonAJ1HAedhoG552ENkFExTZx88Xj8sptiveYfvWE8yCgYQAyYlgxmYekbi50pWnnqR1t0iLqiF+1vh8wbz1cdKDAwwJGM9kx3rCWSsh+arh/ud1Z0AnPhF4N0HgX9/Bbjx3cHDWlOtfn3wExnn/ZyF4GR85GvA7v8EP6HAfHUiIiIiGifYsU66ekGNgfnovOGRB9PKRYb2/jE+wDQ0/qXDHUvhN34+fwCv7WkFAJwxK4YYmOP7gYdPB/54kShMRbFyagkKbCY0d/bjo/e9iXUH2pJdcliysD4rlnz1A6+I/+Petl8UDmVRXTGKgXPFU4DqxcCU04E5FwBLrhIFWwSAzmMwGBSU5MoBpolf8AgOLg1fNJ+dxjiYY6noWA8EgHfVWIYTrh85/kN2rLfsAnwJvNa1GJjJY3dYYAwDTEvzLLCaDPAHgCZXfJ90GJax7veHRMEkU1ifGdyfdHJmO8BlFIy3D+iP8L46+JoovOcUx//JgmxWMlVcELWXBl9L6RSasT7CnICwwuWrD3XGt8Xf7q4G4IX/l9gaE+HpUSNg/MDCTwGzzknfc49FNUuAaauD32f7J5CIiIiIiHTCwjrp5tBxN3Y3dcFoULA6zDBNOcB0b3N6BjtmSmjGenuKomC2HOlAu9uDApsJyyYVRX/Ah68Cfq/o9nYejnq4zWzEb65chmqHDfXtPfjUw+/iO//aDne/V4fVB+2KZ3Bpw/tiO+tc4AvvAF/dAdx2DLi9DfjmQeBL7wPXvwZc+S/gkj8AH/9FMH7ALSIkSnUYYCoLn5Gia+RFglQPMPX5A9paBmWs93cDLbsTP/HB14HWXYA5F1j86ZGPLZwohlz6PCIrOV7jYchdDIV1RVFQk2DO+rCO9T6nyIsGkoyCCelYn/HRxM+jB4sdsKh/I7pbwx8jY2DmXSwieMYKkxX4/JvA598ATJbox+stvwqAIt7j8pMQsQgEgCPrxX7disjHWezAxx8Q++8/Duxfm/BS47LpUfH3J78a+Oj/pec5x7pTvh7cz+aZGUREREREOmJhnXQju9VPnFKMQvvwAsAMtbC+r3lsd6w7Q+Jfegd86PX4dH+O9+udAIBVU0thNsbwNpYfyQeAw+/E9BwnTinBC189BZ86QRQGH3/3MM6+9w28vT+O4koUe5pE92lMHesNW8R2ymki09VRC1jzRu50lhESajazHGDamkRhvblLFtYz27He2tWPAV8ARoOC8tBopb9dC/xyBfDMzYDHHf+JZbf6osuBnMKRjzUYgl3rjQnEwYyGwaXJKqwTW1fkjHUgGAezvzW+v4+damG9wKYWk3vUT5dYHckVYvOrxEUBozU7unllVJA7TM66xw3s+o/YH0sxMJKjJqYIr5QwmoMZ96444mDaPxSFeKMVqFo48rGTThKfjgFERnd/ii++BwLA5j+I/VO+Fv3vHMVmwoniv8+A6GAnIiIiIhoHWFgn3TwvY2DmDo+BAYDp4yUKZkiuensKctblANiqwhgyvAMBoH5d8PtDb8f8PPk2M+66aD6e+NwK1BTm4JizF1f8dj1u+8c2dPUlF3Pj7PFoeeUzKqIU1gOBYHZr1aLYn0Qbeijy6Evz1CiYZArrapd4ZYTC+qyq4Os8mSGp0cgYmMoCG0yhF1fkz2nzY8BvTgOatsV+0rYDwN7nxf6Kz8f2GG2AaRzPE/p8wLjvWAeAlVNEd/mT6+sRiCGuSersU4eXyo51bXBpknnRigJc9W/g+lczE0EylCysd4cZqLz7OWDALT6hUrssvesaD7Q4mDgGmMqLudWLRdd9NGfeIT4B4zoCvHR7/GuMx9GN4tNbZjsw/5Opfa7x5pLHgSufAaafnemVEBERERGlBQvrpIsmV5/WRX12hML61LI8KArQ5vagLYnCZrZz9g4uOHekIA5G/vxktMmIXEcGF0QOx15Yl06aVooXvnoKPnPiRADAnzbUY829b+D1vRFiGWIgo1Jqi3KQb4sS3dB5DHC3ijz1ynmxP4lWjBPrLJNRMElkrDdFyVivKcxBgc0Erz+Q0otIsrA+KAZmoC9YeMwtE/EsD58hutBjKdZu+A2AADDtLKB0etTDAYQU1pPoWB/Lg+5CC+sj/A4uP2ECci1G7G7qwmtxvK9kFExBjjpgWI/BpVLRJPHpkGyQKz99EuZns/XPYrvgkrGb1Z9JcoCpK47CuoyBmTBCDEwoax7wcXWQ6HuPAB++HvtzxWuT2q0+98L0DksdD2wFwJRT+T4kIiIionGDhXXSxUs7Rbf6kgmFESMycixG1KlxB/vGcNe6U81YNxvF/7FMRc66PKccxjmiw2q3eulMQDEAHQfFILo45VlN+P4F8/Cn607EhGI7Glx9uOqRDfjG3z7QinsjOd7dj5d3NuPuF/bgM79bj+seew9AnDEw5bMBcxyDOnMHd7nqkbEuu+zL88O/zhVFwSw1DmZ3U+riYBpkYb0o5OchL6CY7cAX3xXZ2D4P8Pw3gScvDXYzh9PXCbz/hNg/8YbYF1K5QGybtsVWvJe8nmA8yljuWC+oBaCIwZvuyAVzh92My1eIIvyvXz8Q8+kjRsEkM7g0Gw359Immu0UMNgaA+Zekd03jhaNWbDvjiIKJJV99qCmnAkuvEfvP3CzmReitrxPY8Q+xv+Qq/c9PRERERETjCgvrpAstBmZe+G51aXq5mrM+Rgvrfn9Ay1ifUCwuIgyNhtHDcVlYj6VjXcbATD8r2F0cY856OCunluD5r3wEV6+aBEUB/vLeUay59w28sjtY8Or1+LDxUDt+++aHuPHJzTjp/17Bsh+8jM899h4eeHU/3tx3HF19XuSYjbhoSW30J00kBgYYHgWTn1wUTCAQQHOUjnUAmK1eLEhlzvqxDlFYrw6NA5JxI4460bH8qaeAj/1U5BzvewH41UnAh6+FP+GWJwBPF1A6A5h6ZuwLKZsFGMxiaGaUHPFBnIeBgF8MSZW/p7HIZAEKqsW+c+Sfz7UnT4bJoODdD9ux5YgzptMPG14qL57kJjG4NBtFyljf/g8xrLVmKVA6Lf3rGg/i7Vjv7RBRK0B8hXUAOOtOcTHKeRhYe2d8j43F9r8BAz3iQnPdCfqfn4iIiIiIxhVTphdAo1+H24N3P2wHAKyJEAMjTavIw9rdLdjfnOLhZBnS1eeFX23anVKWhwOt7pR0rMsomOJYOtZlYX3CSjWr/APg0FvA/E8k/Px2iwnf/fhcnDO/Ct/42wc41NaDax99D6fMKENbdz92N3XB5x/cvawowLSyPCyqK8TCukIsqivEzMr82Iavyo716kXxLVQrxolO4WQ71rv6vehRh9FWFES+qBEcYJq617nWsV5oD94oC9tyYKaiACuuByauFENNj+8FHrsAOPkrwOn/TwwmBAC/D1j/a7G/4vPxfYzfZBHF9eZtoms91jzu0Hz1sR4b4KgTnyZwHgZql0Y8rMqRg/MX1eDvm4/i168fwK8+HflYAOgb8KFfzfHXMta1jvUxWljvHlJYlzEw7FZPnXgz1o9sFNviqfFHEtkKgI//HPjjxcCGX4shyvH+3R/J5sfEdulVY//vDhERERERpRw71ilpL+9qhs8fwKzKfEwsyR3xWDnAdG/z2OxYd/aKIrrdYtSGW6YyCkYO44yopz3YOTjhRGDiKrGfRMd6qBMmF+O/Xz4Fnzt5MhQFeGNvK3Y0dMLnD6As34qz5lTg62tm4snPrcDWO87GS7ecip9+ciE+feJEzKtxxFZUDwSAhvfFfvXi+BaYFykKJrHfiRxcmm8zwW6JfF0yHVEwMmN9cMe6Wlh31A0+uHI+cP3rwNKrAQSAt+4FHlkDtB8U9+97UUQE2RzAwk/Fv5iqkDiYWGn56mM4BkaKcYApAHz+VPHzeH5HEz5sHfnvZKc6QFhRgHyr+nrUhpeOsSiY3DCF9eP7gYbNYvbCvIsys67xoED9VFGsHetavvqJiT3ftNXAvIvF/hs/Tewc4TRuFf8tMVqABZfpd14iIiIiIhq32LFOSXshxhgYAJhRMbajYDrUfPXCHLPWTa53Yb3HE+yajhoFU/+u2JbOEJ2DsrB+fI8YAphXlvR6cixGfPvcOTh3YTXe2teKKWpXepXDBkWPjsDOY2Igo2KMf5CijBjp7QC8/SjLFz+vjh4PvD4/TLEU9kNog0sjzBGQZlbkQ1FEAb+lqy9iHnsyZGG9NjRj3aVmIBfWDX+AxQ6c93NgyunAv78EHNsEPPQR4Nx7gfcfF8csuRKwjHxxLCwZMdQYxwDT9pCO9bEujsL6jIp8nDmrHGt3t+DhNw/irovmRzy2s9cLQBTVDQb1vSY71vUYXppNtFinkML6tr+I7dQzghfRSH+yY72rUXy6xWAc+XgtXz2JqJVTvylifnb/B2jaHt/Q6khkt/qsc8deVBIREREREWUEO9YpKe5+L97YJzokYymsTy0ThfXj3f3oSEEnd6bJPPVCu0UrrOudsd6mdltbTAbkWqIUOEJjYADAXgyUz1Hv06drXVpUV4ibzpiOc+ZXobowR5+iOpD44FIAsBWK/G8AcLeiyG6BQRFN8Ilc8JCDS0fKVwfExYbJ6qc3dqcgDqazbwBdfaKoWl0YWliP0LEeau4FwA1vi9eEpwv4x+eAg6+LwbYnXJ/YgmRhPZGO9eKpiT3naCIL6zFm0N9wmviZ/H3zUbR09UU8TstXt5uDN/aM0Y51eRHQ3SLewIFAMAZmwaWZW9d4kFcBGEwiy37o8NihfAPAUTGYGnUJdqwDQNlMYO6FYl+PrnVPD7BVvRCz5Mrkz0dERERERAQW1ilJr+1phcfrx8QSO2ZW5Ec9PtdqQo1aCNwfJeZgNJKDS4tyzShKUcd6m4yBybVEL14PLawDwMSTxFanOJiUk4NLE8nZNRiAXLUg190Co0FBca7oWm9NIGddDi6tiNKxDoTmrOsfByPz1Yvs5sGRNKHDS0dSWAdc9R/g1G+JgjogujhjzUcfqkLtJnXVi08HxKJtPHWsq7+PGDrWAWDZxCIsmVAIj9ePR98+FPE4GQVTYAsprLvHaMa6jILxeYA+F3B0I9BxSAy/nXVORpc25hmMQH6V2I8WB9O0DfD2ilip0hnJPe8pXxfbnU8DLbuSO9fOp4F+F1A4EZh8anLnIiIiIiIiUrGwTkl5XsbAzK2MuUN5uhoHs3cMDjB1yigYuwXF9tQU1tvdoiAcNQbG0xPs9p4YWlhX42AOva3rulJG5qtXLUrs8UOGHspc+kRy1ptcsrAe5WcPYFaluNC0u0n/1/mxDpmvHtKt7vcFhwuGi4IZymgCTr8NuPo50an+0f9LfEE5haJgBcTWte71BLu3S8ZDx7r6s3HWi07rKBRFwedPFT+Xx989jO5+b9jjOmXHuhxcGggEO9bHWtSF2QZYHWK/uyXYrT773MTiiyg+BXKA6dGRj9NiYFaIC5vJqJgDzP44gADwxt3JnUvGwCz5TPLrIiIiIiIiUvH/XVDC+gZ8eGWX+Fj4mhhiYKQZamf7vjE4wDQ0Y70oVxS72t0Duj6HLAjLqJmIjm0C/AOi01AW9oBgx3rz9ti7izMlEAheHIh3cKmkZTOL16rMWT/elXjHerSMdSA9Hes1oYX1ribA7xWRDbK7NBYTVwLn/DSYo5yoeOJgnPVAwA+Y7cHfz1jmUIc/DvQEM9CjOGt2BaaU5aKrz4s/rQ/f6S4L61rHuscNeNXomLEWBQMEL5J1HhP52wCw4JLMrWc8kX8fonWs65GvHkp2rW//O9C6N7FztO4V0WeKAVj0aX3WRUREREREBBbWKQnvHDgOt8eHigIrFtUWxvy4aeWiY33/GBxgqkXB2C0oyQ0OygzE0KUaK5mxXpIXpbAuB5dOWAmEfpogvwIomQYgEDwmWyUzuFSSxTi37FhXC+upjoKpFoX1/S3d6Pf64n6ukRx1hulYlx3gBdXRhwumQuUCsY2lsB46uFSvLP5sZrIGL3bEGAdjMCj4/CkiJud3bx2Ex+sfdkynmrNfkKPGAcmivck2Nru45Xv5gz8Bve0iHmbyaZlc0fihdayPUFgPBIB6WVhPIl89VNUCYOY5AALAmz9L7Bzvq93q09cABXFcdCQiIiIiIoqChXVK2Avb1W71uZUwGGIvjk0vH7tRMFrHut2MQnWgoM8f0ApgepBRMKXRomDkcNLQfHVJy1nP8jgYbXDpnPgHl0oRo2DiL6w3xVFYr3bYUGAzwesP4ECLO+7nGkmDU6yjtijkZ+KUg0sTzElPVpVaWG/cGv1YbXDpOMhXlxzx5awDwAWLa1Ceb0VTZx+e+aBh2P2uoVEw2uDSkrF5wULOS9j+d7Gd/wkRaUSpJz914RohCsZ1FOhqEBdCa5bo99yya33bX4KzGWLl9QBb/iT2l16l35qIiIiIiIjAwjolyOvz46VdwcJ6PKarUTAtXf1w9egbk5JpoR3rNrMRuRbROdyhY8661rE+UhSMzwsc2SD2J45QWM/2nHWZr169MPFzDImCCXasx/c78fr8aFXjYyod0QvriqJgVoriYI519ACI0LEuC2DpJqNgju8BBvpGPnY8DS6V5GDYOArrVpMR1548GQDw69cPwO8f/MmXYVEwY3VwqSTfy371QiVjYNKnoFpsR+pYlzEwVQv0/cREzRJg+tkiPurNe+J77J7nxAWn/Cpg2ln6rYmIiIiIiAgsrFOCNh7qQLvbg0K7GSdMLo7rsXlWE6rVwuT+1rHVtS6Hl8p89SK1+N2mY2H9uDuGjPXm7YCnG7AWiG7voSaphfXGD4D+LP4dNG4R20QHlwIhHeutABKPgmlze+APAEaDEv3TAqrZ2gBTfQvrsmO9JlxhPZbBpalQUAPkFImiZ+vukY+VHevjYXCplEBhHQAuXzEB+VYT9rV049U9LYPu0zrW7UM61nPHYL46AOSVBfdLpif3d4HiUxBDxvoRnWNgQp3yDbHd+hTQcSj2x23+g9guuoKfbiAiIiIiIt2xsE4JeWFHEwBg9ewKmI3xv4ymqV3re8fYANMOtWPdkSOK3rKrXN+O9RiiYGR2et2K8HnbjlpR6Av4gsWQbDNocGkSsQK5srCudqyrw0tb4xxe2uQSxeyyPCuMMUYfBQeY6nfxwuP1o7lLrGVQx7oWBZOhwrqixJ6z3j6OO9blBZAYFdjMuPxE8diHXh8cg9HZN6RjXWasj8XBpcDgQbcLLh2bcTfZSn4SprtZxKuEo/13R6fBpaHqlgNTzxAX7t66N7bHdBwGDrwq9hdzaCkREREREemPhXWKWyAQ0ArrH40zBkaSOev7dCisv1/fgRse34Qj7T1JnytZWse6FUCfS+tYb+/Rr7De7o5heKmWrz5C5+DEk8X28Ds6rUxnrqOiA9dgSnxwKRASBTM0Yz2+30kwXz22bnUAWhSMnh3rzZ19CAQAi8mg/VsAZL5jHQjGwTSNkLPu9QS7tovHU8d6/Bnr0rUnTYbFaMDGQx3YdLhdu31Yxro7JGN9LJIXyQCRr07pYy8FjBYAAaCrcfj9/d3ik1KAuKCbCqd+U2zffyJ4IXEk7/8RQACYchpQPDk1ayIiIiIionGNhXWK29ajLjS6+mC3GHHy9MQ6I2dUqIX1luQ7eX/y/B48v6MJP3txT9LnSobH60d3v8j+rXvuM8A9c1Fr6QWgX8d6IBDQMtYjRsEEAsHOwYmrIp9M3petOesyBqZsNmCOnmkekYyC8XQBnh6UqZ3+7e5++IZkVo+kJY7BpdLMinwoiijit3RFyR2P0dEO8ZqqKcyBIjt2A4HMDy8FYutYd9aLrGSzHchP7MLcqFQ4UWyd9eL3FYeKAhsuXCyiOH79+ofa7Z294u9NQY4acaFFwYzRwnrNEhFvNed8FkrTzWAYOWf92Hvife2oAxw1qVnDhBOBSR8B/APA2/eNfKzfpxbWASy5MjXrISIiIiKicY+FdYrb82q3+ukzy2Ezh4kZicG0chEFs78luY51Z48HGw6JDs7ntjVp3dyZILtHFQUwNW4GPF2YbjgKQL+O9a5+Lzw+PwCgJDdC53T7h+Lj+kbLyBEqMmf92CZgoFeX9elKi4FJYnApAFjzAZNaDHe3oDjXAkUB/IFgdE8sZMd6LINLpRyLEZNLxBC/3TrFwTQ4g4V1TW8HMOAW+5kaXgqIoYUA0LQd8PvDHyPz1YunjK8oD/l78XSL31ecrjtFxOa8tKtZ+7s5vGN9rEfBlAPfOAhc/EimVzI+Faiv4XA563JYdqq61SXZtb75MaCzIfJx+18GuhqAnGJg1rmpXRMREREREY1bLKxTXAKBAJ7fLgrra+Yl3m06TY2CaXT1aTnBiXh1T4vWdezx+fG3TfHlF+vJqRZpS60BKGqRs8IoIkDa44wdiUSeJ9diRI4lwkUN2a1evXjkTu+iyUB+lej+O7pRl/XpSnasVy9O7jyKEjLAtAUmowFFdhkHE3vOepNLHBtPxzoQmrOuTxzMMbWwXl0Ysg4ZA5Nbnlx3f7JKpgNGq/h0QMfB8Mdo+erjrOPYnBOMJXIejvvh08rzcNacCgQCwMNvfAi/P4CuSBnrY3V4KSAGUHIIZWbITvTOo8PvC53rkUqTTgYmrAJ8HuDtn0c+bvNjYrvwU4Ap9vguIiIiIiKieLCwTnHZ19KNg8fdsBgNOH1mWcLnceSYUakWKJPpWn9ppxhIObHEDgB4cn09/HHEe+ipQ81Xn5gTjPwohUu9T5/CeptbFHdLRhxcKvPVV458MkUBJqpd69mWsx4IAA3vi/2qJAvrQEjOujrAVOasd8X+e2lOIAoGAGZXiU9n6FVYD3as24M3ajEwGexWB0TBs2KO2I8UB6N1rI+jfHVJDpaNJR86jBtOFT+zf75/DB8ed0P+qSuQHesyCmasdqxTZhWohfWhHet+f/Di7IQUF9YVBTj1G2J/06NAV9PwY7qagD3/FfuMgSEiIiIiohRiYZ3i8u6HoiNyxZRi5MsuyQRNV3PW9yc4wLTf68Pre1oBAHddNB/5VhMOtfVgnbrGdJPF8xpbsLBe6HcCgG4RNcej5asDwc7BaIV1ICRn/a0kV6Yz11HRfZvs4FIpN9ixDgCl6oWJeDrWZWG9Ms7C+qxKOcBUnyiYETvWMzm4VNIGmEYorLfJjvUp6VlPNilU8+8TGGAKAEsnFmH5pCJ4fH7c9/JeAGKIrRbJpUXBjNGMdcosrWN9SGG9dRfQ3wmYc4FyHf5eRzPlNKD2BMDbB7xz//D7tzwJBHyie758VurXQ0RERERE4xYL6xSXbUdFB/biusKkzyXjYPY2J1ZwXHegDW6PD+X5Vpw4uQQXLhH/p/+J9fHHLOjBpXasV1t6tNsK1MK67GZPlhxcKjuuh+luAdr2i/1YOgcnnSy2RzcC3szl0w+j1+BSKS/5wnowYz2+WIHZ1aKwvr+lG/1eX1yPDUcW1muKQjLWtY71bCisy5z1reHvlx3rJeOwYz3JwjoQ7Fr/z9ZGACExMF4P0C/+Po/pKBjKHJmxPrSwfmS92NYuS09Mj6IEs9bfewTobg3e5/cHY2CWXJX6tRARERER0bjGwjrFZdsxUbiZV+NI+lzT1QGm+xKMgpExMKvnVMBgUHD5ClG0enFHM1q6+kZ6aErIjvUKU7Cwbh8Qg1Xb4ijgjqRdRsFEGlwqu9XL5wA5RdFPWDpDxEZ4+4CGzbqsURfa4NJF+pxvWBSM+Pm1xvh76fF40dXnBQCUx9mxXu2wocBmgtcfwIEWd1yPHSoQCIQfXupSC7WycJtJWmE9TMe6byBYVGbHekJOn1mO6epFSQBw5KiFzF7xtwaKEbAVJnx+oogcEaJg6tXCeqrz1UNNO1MM5x7oAdY9ELz90JtivoO1AJh7QfrWQ0RERERE4xIL6xSzXo9PK4IvqC1M+nwzZBRMAoV1vz+Al3eJIulZc0TRdFZlAZZOLILXH8Bf3wszXC3FZFd6qTH477H1iWiGzj4vBnz+pJ9Di4KJ1LFev05sY4mBAdScdTUO5vDbSa5ORzJfXbfCutqx7hadjaX58WWsN3eKArzdYkS+Nb6OTEVRMEunAabtbg/6BvxQFKDSEVLgz6aO9Yq5ABSgq3FwJykgCsoBH2DKEYNzx5vCiWLrSnzIssGg4PpTghcltHx1t8xXLwYM/E87pYDMWO85DgyEXLyWHeupzlcPFdq1vuHhYAyS7Faf/wnAkpu+9RARERER0bjE//dNMdvZ2AmfP4DSPCsqCuKLwwhHRsEcc/aiu98b12O3HXOhubMfuRYjVk0N5glfoXatP7m+Hr40DzF1qh3rxUqwsG7sbYWiyPuTj4NpU7PaSyJlrMdbWAeCA0wPZUlhPRAIRsHoMbgUCImCGdyxHmsUTJMrmK+uyF9oHGZXik9n7G5KrrAuY2DK8qywmozBO1zqhaRsyFi35gVjXpqHdK1rg0unAAn8HEc9+ftx1ovXeYLOX1SjZf07hg0uZb46pUhOEWBWhybLOJjuFtEhDgWoXZ7e9cxYIz4hM+AG3v0l0NMO7HpG3MehpURERERElAYsrFPMtqsxMAtqHQkVF4cqtFtQli8KnPF2rcsYmFNnlg0qMJ4zvwqOHDOOOXvxxt7WSA9PCVk4L0QwM15xt6JQLXzJqJhkyCgYWRgepL8baFRzrSfGUVifpBbWj6wHfPFd4EgJvQeXAsOiYMriLKzLwaUVccbASLO1jvXkBpg2hMtX9/QEi6qO2qTOrxs5wLRxSM66HFxaMg5jYIDgJwr6O4E+Z8KnsZgMuOFU8TOcUqrGwmgd68xXpxRRlGDXuiysy2718jmALfmIuLjXc+o3xP76X4svn0cU26t1uihLREREREQ0AhbWKWZbj+qXry7JOJh9cQ4wlYV1GQMj2cxGfGKpKC6me4ipLJzn+0P+LQM9qLKLCJh2d/KFdTm8tDhcx/rRjSJmw1EXX4FVFkQ83UDTB0mvMWmyW71cp8GlQEjHeisQCMTfsa4V1hP7pIYsrCfbsX60QxTWqwflq6vd6pb87MnWloX1oTnroR3r45HFDuSWif0kctYB4KpVk/DXG1bilrNniBt61CiMXHasUwoVVIutzFmXcz3qTsjMemb+D1A+F/B0Aa//n7htKYeWEhERERFRerCwTjHTOtbjKaz7vMCRjcEM6CHkANN4Otbr23qwp7kLRoOC02eWD7tfDjF9ZXeL1uGbDrJj3e4bXDydZBUDK/UorMuM9ZJwGeuJxMAAgMEITJA56+8ksTqdyHz1qkX6nTNXfZ14e4H+Li1jva3bA38MkUFax7ojsUL/jIp8GBTx+0tmsG6DUzy2Nuzg0rrsiVepXCi2wwrrasf6eC2sAyEDTBPPWQdEdv/yScXIk5n/srDOjnVKJXnRtlO9oKflq5+YmfUYDMCpXw9+b8oB5n8yM2shIiIiIqJxh4V1ikmPx4t9LaITe35tlMJ6fxew82ngH58H7p4G/G418NvVYWNGZM763jg61l/c2QQAOGFSMQrtwwvMU8vysHJKCfwB4KmNyRWv4iE71nO8zkG3T7CKiwbJFtb9/oD2HGGjYLTCegIFDjnANBty1hu2iK1eg0sB0SlsERdx0N2Cklzx8/P6A3D1Rs++l4X1ygSjYHIsRkwqFYP0komDOebsATCkYz2bBpdKsmO9bZ+IqpG0jvWp6V9TtnCE5Kzryc2MdUoDGQXjOiYGmMq/15nqWAeA2ecDZbPE/twL0x9JQ0RERERE4xYL6xSTXY2d8AeA8nxr+JzpzgZg4++AP14M/GQK8Jcrga1PAb0d4v7uJqB5+7CHTVcL6/vi6FiPFAMTSnatP7WhHgM+f8znTlQgEIBTLdCa+53iRkW8vapNopDakWRh3dU7oA1kLRp6QcE3ABx9T+zLInk8ZM56/TuAP/U/r4hCB5fqnZEr42DcLbCYDNrQx1jiYOTw0kQz1gFgdqUaB9OYeByM7FivGdSxrhbWs2FwqZRfIT4lEPADLTvFbb4BoEONZ2LHuv6FdZmzn8uOdUohR0jGesP7gH9AvNeLJmduTQYDcP4vgTkXAKfflrl1EBERERHRuMPCOsVE5qvPlzEwgYCIeXj9J8CvTwXumQ08ewuw/2UxPKxoMrDyJuDqZ4HJp4rHHN047LwzKkQX8dGOXvR4og/O7HB78N5hUawfqbC+Zm4lSnItaOnqx9pdLfH8UxPSO+CDxysK0qZ+9WJC0SQAQLlRFFLbkxxe2qYOLi2wmWAxDXnrNm4FBnpExnbpzPhPXrkQsOQBfS6gZUdS60yK60hwcGm5ToNLJS1nXVyYKVXjdFpjKKw3d4pjkiqsV4nX+q4kCuvHnGEy1rOxYx0AqhaIbZM6wNRZL2YAmHKA/KrMrSvTUlVYd8soGHasUwoVqFEwrmPBGJi6EzIfQ1W7FLjkD8H3FxERERERURqwsE4x2abmqy+qtgEvfhu4bwHw0MnAqz9UO4wVoPYE4Mw7gC+uB770PrDmh8Ckk4Md1Ec2DDtvUa5FK3DGkrP+6p4W+PwBzKrMR12xPeJxFpMBlywXhcYnN+hcwAqjQ81Xtxn9ojgNaB9NL4UopCbbsS7z1cPHwKjZ6BNWiu69eBlNQN0KsZ/JnHUZK6Dn4FJJK6yLCy3BAaYj/178/oCWi16ZYMY6AMySHetNiUXB9Hp8WpxQTVGY4aXZ1LEOBONgGtXCuhYDMzmx1+hYUThRbF16d6zL4aXsWKcU0jrWj2Y+X52IiIiIiCjDxnF1g+KxTe1YPz2wAXjnflEUMuUAM88BPn4/8LU9wOdeAj5yC1A+a3D3Wu1ysZX/J3wImbO+rzl6YT2WGBjpU8tF59obe1tR39YT5ejkyKJ5XY4HCtRhmKUzAACFAScAoC3JwrosqoYfXPqu2CZT4NBy1t9K/BzJkjEweg4ulfLU14zsWM9XC+tdI3est/d4MOATv9Py/DAXNWI0u1oU1ve3dKPf64v78bJbPc9qQoHNFLzDlaUd67KwLgeYaoX1cRwDAwQvgKQqCoYd65RKMmO9zxWcySEvyhIREREREY0zLKxTVO5+Lw60iqL3ZN8hceOcC4BvfAh86k/AkitFpnIktcsAKIDzsNYtHErGwUTLWe8b8OH1va0AYiusTyix45QZZQBS37XuVDvWJ9hEZzOsDqCgGgBQ4BPRMB3JRsGokSXFuUMK64FAcHBpIvnq0qSTxfbwO+KcmaANLtU5Xx0QOcCA9hos0zrWRy6sy3z10jwLzMbE/2RWO2wosJng9QdwoMUd9+Mb1MJ6TWEOFHnhyucV8w2ALCysLxTb5h2A3we0HRDfj/fCuvw99bmAXqc+5/T7gZ52sW9nxzqlkK0AsIqLhOh3AUYrULUws2siIiIiIiLKEBbWKaqd6uDSigIrcl37xY0TVwGWyFEsg9gcWixKuDgYbYBp88gRGesOtKHH40NlgS2Y9R7FFeoQ07++d0TLQE8FZ68omldbRfET9iIgVxT17R5R8OpwDyT1HDKypGRoFMzxfSIGwmRLrtO7erE4R89x4PjexM+TqEBADMMDgOpF+p9/WBSMuEARrbAuY2CSyVcHAEVRMKtKFKQSyVkP5quHrKOrQeSWGy3BjvxsUTwZMOcC3l6gbX+wY71kambXlWnWvGBXufy0QbL6nOJ1ALBjnVJPdq0D6n83Ev8kDxERERER0WjGwjpFtS10cGnLLnGjLJTHqu4EsT06vLA+rTy2jvUX1RiY1XPKgx27UZw5qxwVBVa0uT14YUdTHAuOj8xYrzSpkTM5xVph3dovso/bdYqCKR3asS671WuWAaYwMTGxMlmDsT2H3078PIlyHQF621MzuBQYHgUTY8Z6k0sU3iuTLKwDwJwqmbMef2Fd61gvCjO4tKAm+3LLDUagQv09Nm0D2tmxrtEGmOpUWJf56lZHcn8DiGLhCCmsT2AMDBERERERjV9ZVomhbCQHly6utAEdh8SN5bPjO4ksrB/ZOOyu6RWiY/1IRw96PeGzp/3+AF7eJfPVK2N+WpPRgEvVrPUn1h+OZ8VxcapF73KTGvFhL9Y6pE29Ivu4d8AX8d8XizZ3hCgYWVjXY4DcxJPE9lAGCuupHFwKBDvW3SJOqDTWKJhO0bFerkNhfValuIj0fr0z7sce65Ad66GDS9XCbLYNLpVkzvqxzcFM8eJx3rEOBONg9MpZd8t89WJ9zkc0ktCOdearExERERHROMbCOkUlC+vL81sBBAZ1Y8esVi2sN2wGvIM7hEvzrCjOtSAQgJblPtQHR51o7epHntWEE6fEVzy6bHkdDArw7oft2B+lKz5RsmO9xKCe316i/YyUfhdyjaKg3p5EznrEKBgtX31lwufWyIz2TOSsy8GlqchXBwZHwfj9MQ8vbVYz1vXoWF81tRQmg4L3Dndog3hjdSwkY12jDS6dkPTaUqJqgdjueRbwe0XUUH5VZteUDbSOdZ0K63JwaS7z1SkNHLXBfRbWiYiIiIhoHGNhnUbUHTK4dJbxmLixfDYQYxSLpmQaYCsEvH1A87Zhd0+TOest4XPWZRHy1JllsJqMcT11dWEOzpglYkD+lKIhpjJjvUhRC+s5xUBOEWAwAwCm2EVETEcScTByeGlJXkjHemej+BSBYghevEhG7XKx5q6G4KcT0kXmqyeTEz8SeTHIPwD0OUMy1j0IjHARoVnNWK90JJ8jPKHEjs9+ZDIA4LvP7ECPxxvzY8MW1mWUSGihK5vIjnX5Wiqekn2RNZlQOFFsnTp9ikbrWGdhndJAdqwXT+XFHCIiIiIiGtdY4aAR7WzoRCAgunULOtXBpfHmqwOimCbzu8PFwWgDTMN3lMvC+tlzEhvQKIeY/m3TUfQNJB7HEolT7VgvCKgXBuzF4uKDWsydZBURMcnkrGsZ66Ed67JbvWIeYCtI+Nwaix2oWSL205mzHggEo2BSMbgUEBnyOUViv7tF+zl6fH509kUucDe59BleKn35zOmoKczBMWcvfr52X0yP8fkD2joGZaxnexRM+RxACbkQxnx1QXas6zW8VGas53JwKaXB9LOBSR8BTrk10yshIiIiIiLKKBbWaURbjzoBAPNrHUDrbnFjvPnqkvzIeJgBpjMqIg8wPXTcjX0t3TAZFJw2szyhpz5lRhlqCnPg6h3As1sbEzrHSDrUiJdcnzqUUhZw1W6+Omv3oOPi5fX5tbiZQRnrWr66DjEwksxZP/yOfueMRhtcahYXCVIlV8bBNMNmNiLfagIwcs56c+cIhXXnEeCZLwHtH8a8BLvFhO99XAz1/N2bB2MaZNrS1QevPwCTQUF5fsg6tI71LC2sm3OA0hnB71lYFwp1zliXhXU7C+uUBnllwNX/ARZdnumVEBERERERZdSoLaw/+OCDmDRpEmw2G1asWIENG4YXa6VHH30UiqIM+rLZUjAccQzaruarz69xAC27xI2JdKwDQF0sHevDo2Dk0NIVU4rhyDEn9NRGg4LL1a71J1MQByM71u0+8fPShgiqud5VJlFYb+tOrLAus9kVBSiyhyms65GvLmkDTN/S75zRhA4uNSUfuRJRaM46EDVnvd/r0y5ohM1YX/8QsPkPwPO3xbWM1XMqcPacCnj9AXz7n9vh94+cZ9+gxsBUOmwwGtQYpkAAcB0V+9nasQ4E42AAFtYleSGktwPoi35hJSpGwRARERERERGl3agsrP/5z3/GLbfcgjvuuAObN2/GwoULsWbNGrS0tER8TEFBARobG7Wvw4d1yrYd47aqhfWFleZgHnCiHes1S0UWuKteZIOHmFYhCuv17T3DolpeVGNgzpqdWAyM9MlltTAZFGw63IFdjToUs0I41cK31eMUN8jOUbVDutIgni/RjnUZA1NstwQLq30uoGm72NezY33CCvF7ch4OFm5TTearpyoGRspTX0NutbAekrMeTkunKLhbTAYU2sNc1GlT45H2vQh0NsS1lDs+Phd2ixHvHe7A3zaN/HM+2iEK69Wh+eru44C3F4ACFGRpxjrAwno4toLgp1r0iIPh8FIiIiIiIiKitBuVhfV77rkH1113Ha655hrMmTMHDz30EOx2Ox555JGIj1EUBZWVldpXRUXkIm1/fz86OzsHfY1HXX0DOHhcZIMvtIriNuyliRdvrPkicxkYFgdTlmeFI8cMfwD4sNWt3d7u9uC9Q+0ARJdvMsrzbTh7rjjHk+v161r3+wNw9YquZlN/h7gxR3asi4z1EogLFIlmrMtO90ExMEc2AggARZOB/MqEzhuWNR+oWij20xUH07hFbFM1uFTKC0bBAMG8+khRME1aDIwVSriBvW0HxDbgB7Y8GddSagpz8NXVIiblR//dNeJro8Ep1lEbWlh3qa/h/ErAZAnzqCxRtSC4XzI1c+vINjJn3alDYZ0d60RERERERERpN+oK6x6PB5s2bcLq1au12wwGA1avXo1169ZFfFx3dzcmTpyIuro6nH/++dixY0fEY++66y44HA7tq64ui2MWUmiHOri02mFDYbdaQEy0W12qO0FsjwwurCuKghlq1/q+lmAczCu7W+APAHOqClBbZE/uuQFcsWIiAOCf7x+Duz/ywMp4dPYNQCR5BGDoUwvrMgpG7VgvDDgBJN6xLgu/JXmhMTBq0VvPbnVJy1lPwwDTQYNLF6f2uYZGwUQrrKsDQ8PGwPh9QMeh4Pfv/xHw++NaztUnTcKsynw4ewZw13O7Ih53zNkDYEjHuvw0gSOLu9UBcZHGbBfvhfzqTK8mezh0zFnvERcfObyUiIiIiIiIKH1GXWH9+PHj8Pl8wzrOKyoq0NTUFPYxM2fOxCOPPIKnn34af/zjH+H3+7Fq1SocPRo+fuG2226Dy+XSvo4c0aGjcBSS+erzakIGl5bNTO6ktWph/ejwnPVp5eoA0+bgANOXdorfabLd6tLKKSWYXJqL7n4v/v1BfNEdkcgM7nKLB4pfLdbLjvVc0bGe7xMF92Q71kvyQvLH698VWz3z1SUtZz0NhXVnfcjg0rmpfS4ZBRNjx/qIg0tdRwH/AGC0AJZ8oONg3BcizEYDfnihGNb6101Hsf7DtrDHyY71mqKQwnq2Dy6VcoqA614FPvsiYBh1/8lJnUJxkU+L2EpUIBCMguHwUiIiIiIiIqK0GRdVjpUrV+LKK6/EokWLcOqpp+If//gHysrK8Otf/zrs8VarFQUFBYO+xqOtR0VhfUGtA2iRhfUEB5dKsmO9YQvgHVzM1AaYqh3rfQM+vLFXFIzO1qmwbjAouHS5KES+sCP8hZh4yS70CTb132OyARa1u16NgrF7RME00cK6fFyJjILx9gNH3xP7KelYXwlAAdr2AVv/qv/5Q8kYmFQPLgW0TxCguxUAUJovfp6tXeF/L7KwHrZjvf1DsS2aBMy7SOy//3jcS1o6sRifOkG8Jr/9r+3weId3vR8Ll7Eus7mzeXCpVD4LKJ6c6VVkFy0KJsmOdY8b8IrXKaNgiIiIiIiIiNJn1BXWS0tLYTQa0dzcPOj25uZmVFbGljNtNpuxePFi7N+/PxVLHDMGd6yrMRXJRsEUTxFdlb5+oHHroLtmVKgd6y2iY/3t/cfRO+BDtcOGudX6XdyYUyXOdczZq8v5XGrHel2OiOsY1DWqFnKt/SKqod09kNBztLnVKJhctfDcsEX8DO2lQMm0hM45opwiYPEVYv8fnwNe/6nojE0FLQZmUWrOHyrujHVxe9iO9XY1Hql4CrDkKrG/82mg1xn3sr750VkozrVgX0s3fvvWh8Pub1BfqzWFo7BjncKThfVkh5fKbnWTDbDkJncuIiIiIiIiIorZqCusWywWLF26FGvXrtVu8/v9WLt2LVaujK1z1+fzYdu2baiqqkrVMke9zr4BfKgOLp1fZgx2VZYlWVhXFKB2udgfMsB0upqxfritB/1eH17aKYqfq+dUhB8cmaDqQlEkbVTjNZIlO9arzGqhXsbAAFoh19jXDiN86OjxIJBAgfq4FgWjdqw3bBbbuhXiZ5oK590PrLpZ7L/6A+DpGwFvYh33I5Id66nOVweCUTA9xwG/L/YoGEe4wvpBsS2eAtQsEYN5vX3Atvg7/AvtFvy/c8R76xdr9+FIe492n6t3AF3qPAD52hV3qO9JWaCl0UWvjvUeNT7IXpq6vwVERERERERENMyoK6wDwC233IKHH34Yf/jDH7Br1y584QtfgNvtxjXXXAMAuPLKK3Hbbbdpx99555148cUX8eGHH2Lz5s349Kc/jcOHD+Nzn/tcpv4JWW/HsU4AokO2pPeQuDG3TJ/heBEGmJbnW5FvM8HnD+BAixsv7xIDJs+KFgPjcQN7XxDDJGNQ5RBdv139XnT1JdZBHkrLWDeJCxGwFwXvtJcAUKAggGJ0wecPoLMv/qGpMgqmVBbWZbdyKuM1DAbg7B8A/3MPoBiBLU8Af7wI6O3Q7zkCAaDhfbFftUi/80aSWwooBiDgB3raUBZSWA93wSOmKJjiKaKgufgz4vsE4mAA4KIlNVgxuRh9A37c8cwObT2yW7041wK7xRR8ADvWRzcZ4dPTBvR3j3zsSNyysF488nFEREREREREpKtRWVi/9NJLcffdd+P222/HokWLsGXLFjz//PPaQNP6+no0NjZqx3d0dOC6667D7Nmzcc4556CzsxPvvPMO5syZk6l/QtbbdswJAJhfo2O+uhRhgKmiKFrO+l83HcHx7n7kW01YMTlKMf+/3wSevARY90BMT59rNaHAJgqUja7ku9adasd6mVHtMg7tWDcYtWiYOovIjk8kZ71N7agullEwnerg3XQUVZd/Frj8L4AlDzj0JvC7s4GOQ/qc21kvCvXpGFwKDPp9oLtZy1jvG/DD7Rl8YSYQCKDJJYeXhsl+Dy2sA8CCS8W/o/GDYTFHsVAUBT+8cB7MRgWv7G7BCzvEJzaC+eohxf3+LqDPKfYdtXE/F2UBm0N8AcnFwcgomFzmqxMRERERERGl06gsrAPATTfdhMOHD6O/vx/r16/HihUrtPtee+01PProo9r39957r3ZsU1MTnn32WSxenIbYiVFsm9qxPr9Wx3x1qWaJ6IDuPAa4jg66S+asP7VBFJpOm1UOi2mEl2l/F7D972J/82Mx54DLIZANOuSsO9WO9WJF7Tod2jmqxsFMUjPYEyusD4mCkT+3dBVVp68Grn0eKKgBju8FHj4TOLIx+uOikTEwFXNSP7hUknEw3c2wW0ywW4wAgONdg+NgXL0D6FcHiQ7LWPf7B0fBAOLTHLP+R+wn2LU+rTwf158izve9f+9Ad78XDa4w+ery929zALbxOVx5TNDiYJIorLvVwjoHlxIRERERERGl1agtrFNqbTvqBJCijnVLbrA7eUgczDS1Y713QHQPR42B2fk0MKB2irftB45timkJVWpmth4d6zJj3QFxMWLQ8FJAROgAqDWLwntHnIX1fq9Py9gulR3rrmNi66hJYMUJqpwPfG4tULlAdMn+4Vxgx7+SO6ccXJqOGBhJG2DaCiDyANNmdXBpod0Mm9k4+Bydx8TwWIN58KcGlqhxMFv/Agwk9tq66fTpqCvOQaOrD/e9tDekYz3c4FLmq49qhRPF1nk48XPIjHV2rBMRERERERGlFQvrNIyrdwCH2kSxen6NA2hVC+t6dawDwZz1IXEw09WOdQAwGxWcNrNs5PNseVI92C62H/wppqevUouUjTp2rOf7RdTLoCgYQCvkVpvVKJie+ArrssPdZFBQkGMCvP1Ad5O4M9352gVVwDX/BWZ8VAzq/OtVwNs/j/mTAsPIfPXqRbotMapcWVgXUSsyt35oYb0plnz1oomAMST3fMrpQEGtiGnZ/Z+ElpdjMeLOj88DAPz+nUN4fa+4ADC4Y10OLmW++qgm37/JDDCVUTDMWCciIiIiIiJKKxbWaZgdx1wAgNqiHBSZ+oP5v3p1rANAnRrdM6RjXWasA8CJU0pQYDNHPkf7QeDw2wAU4Jyfitu2/10UnqOoVjvWG3TsWM/1iZ/bsAKXWsgtV0RHe7wd6zIGpjjXAkVRgM4GcYfJNrw7Ph2secBlTwInfF58/9LtwH++AvjiHAQbCASjYDLSsS6G48qO9dbuwb+XZvW1UR62sH5AbGUMjGQwAos/LfY3P5bwEk+fVY5z5lfC5w9gd5O4IFMTtmOdhfVRTYuCSaKwrg0vZcc6ERERERERUTqxsE7DbFML66JbfY+4Ma9C347I2uVi2/jBoMiMKocNeVbRARw1BuaDp8R2yqnAwk8B+dViEObeF6I+fZVD7Vh36dexbvOqhfWhHetqREOJIu6PN2NddlKX5MnBpWoMTEENoCgJrFgHBiNwzk+Aj/4YUAzApkfFANk+V+znSPfgUklmrLvVwnq+GgXTNTQKRnasxzC4NNTiKwAowMHXgznsCbj93LnItQQjaGqKQjvW1cI6O9ZHNz0K6xxeSkRERERERJQRpuiH0HijFdZrHUDLOnGjnt3qAFA0SWSPu1tFcX2C6GBXFAWXr5iAN/a24twF1ZEf7/cHY18WXSEKvQsuAd6+TxTc53x8xKevKlQz1p3Jd6w71Y51i8cpbogwvLTQL+6Pt7Aujy/N1ODSkZx4g4hD+du1wIFXgN+cBpTFGBkkC4LpHFwKhHSsyyiY8BnrI0fByMGlU4ffVzgBmHIa8OGrwJYngDO+ndAyKx02fO3smbjzPzsBDMlY114DLKyParKw7kpieGkPO9aJiIiIiIiIMoGFdRpmUMf6hynIVwdEp3XtCcCeZ4GjG7TCOgD87zmz8b/nRHm++nVi4J8lH5h1rrht4WWisL7vBRGPkBs5JqVa7VhvcPUiEAiIiJUEeLx+uD1i0Kqxr0PcGCEKJt/nBBCMjolVaBQMgGARLhsK6wAw82PANc8BT14mOrllN3esJqxKzboiGRIFUxYhY112rIePghmhYx0QQ0w/fFXMADjtNnHhJwFXrpyIHQ2dsFuM2gUAAIyCGSvkJw7crYCnB7DY4z+HFgWTgVgoIiIiIiIionGMhXUaxNUzgMOhg0vf3SXu0LtjHRADTPc8CxxZD+Dm+B4rh5bOPT9YjCqfLbK6G7eIrPUV10d8eKWasd434IezZwBFsmgdJ9mtblM8UAbEz2348FIxgNXuEQWwuKNg3GoUTK5aWHWpUTDZUlgHgOrFwA1vAfteBPxxZK2bbMDMc1K3rnBkFMywjvXBv5eIHet+f0hhfXL455h1LpBTJGJ7DrwCTD8roaWajAb87JKFg2/0eoCuRrHPKJjRzVYIWAuA/k5xwaxsZnyP93qAfjV+iVEwRERERERERGnFwjoNsr1BFGnqinNQaLcArSnqWAdEYR0AjmwUgyxj7Rr3uIGd/xL7i64YfN+iy0Vh/YM/jVhYt5mNKMm1oM3tQYOrN/HCeq8oIk+w9QEBAIoRsDkGH6R2rFv626HAj46e+IZ8tqsF35JsjIIJlVem5otnOfX3gd4OwOsJZqwP61gX38uLMJquRsDbJ37XMspjKJMVWHApsP4hMcQ0wcJ6WJ3HAATERYncMv3OS+mnKOI11Lxd5KzHW1iXMTCKURTpiYiIiIiIiChtOLyUBpExMAtqCsUgSjkoMxUd69WLAYMJ6G6KL2N4178BT7fIaZ+wcvB98y4W52zYHBy8GoHMWW9yJZ6z3qF2n9fZ1KJsTtHwCwRqJ6kh4IUDbrQNKeBG06Y+R4ks/ocOL6X45RSJ1wgAuFuDHeshw0sHfH6t0F4xtGNddqsXTQSM5sjPs/gzYrvnv0B3qy5LBzA4CihTw2tJP8kMMJVzCuzFgIH/OSciIiIiIiJKJ/4/cRpk21FRWJ9X4wgWpvOrgJxC/Z/MnANUzhf7RzbE/jgZA7Pw8vBF7Olni3053DSCKi1nPYnCutp9XmN1ixvC5RybrFoXe6niQmefFwM+f8zPIQvxJTJjm4Mrk2MwBLvWu5u1obBujw+9al5+a1c/AgHAZFCCFzSkaPnqUuU8oHqJiMbZ+pR+62e++tgif48JFdaZr05ERERERESUKSys0yBax3qtA2hJYb66VKvGwRzdGNvxziPAwTfE/sLLwh8jb9/6F8Dvi3iqajXio9HZG9tzh1uOmrFeaVbPMXRwqaQWckuVTvVxscfBHA+NgulziTxmAHCwYz1hcoCpuxV5VhOsJvGnUHapa4NL860wGIZcvGk/ILbRCuuAGGIKAJsfF3FHepAd68xXHxuS6Vh3y4515qsTERERERERpRsL66Rx9nhQ3y4GcM6rdqQ2X13SctbXx3b81qcABIBJHxFRHOHM+KjoEO88Bhx6M+KpqgpFx3qjDh3rZUa1Y33o4FJJLeROtHarj4t9gGl7aBSMHFyaUwRYchNYMQEIFta7m6EoihYH0zqksF4xNF8dCOlYnxr9eeZdDJhygON7Yr94FI2LHetjSlJRMGrHei471omIiIiIiIjSjYV10mw/JjqhJ5bY4bCbg4X1VHasy8J60zZgIErneCAAbFHjXRZ+KvJxJqsoaALAB5EjOKrUomlDMh3rvaLoXWqQUTBF4Q9Uh0zWWkRhva07tsJ6j8eL3gHRdV+SZ83ewaWjTUhhHUBwgKmasy5z9yvywxXWD4ptLB3rNgcw9wKxv/mxhJc7CKNgxhZZWI9nzoTEjnUiIiIiIiKijGFhnTRbjzkBqPnqANCSho51Rx2QVwn4vUDD+yMfe2SDiOEw5wJzzh/52IWXi+3OZ4D+7rCHVOvQse50i471QqVL3BCpY10trNeYxHGxdqzLArzVZECuxRgsvhWwsJ6UvAqxVYeKlqk56zJ2p6lTFNgrh3asBwKxZ6xLcojpjn9GfC3GhVEwY4ssrHc3R7+4OJTWsc7COhEREREREVG6sbBOmu0yX73GAfQ6ga4GcUfpjNQ9qaIAdcvFfrQBplueENs5HweseSMfW7tMRHUMuIFd/w57SGWBKJo2ufrg9yeWfy0L5AUBNfc80hBBtUO6zCCOk/Eu0bSpx5XmWaEoioi3AdixnqzcIR3rahSMzFhvkVEwBUMK611NwEAPoBiCBdFoJq4Sr0VPtyiuJ8Pv5/DasSanCLCof8/k7zZWPbJjnVEwREREREREROnGwjppth4VhfX5NQ6gdY+4Mb8ayClM7RPHMsB0oDdYlFx0efRzKkowLuaDP4U9pNJhg6IAHp9fK2DHSw4hzfPJwvrIHeslEMd1xFpYVwu9xbmio5pRMDrRomBaAAwvrDephfVKh3Xw42S3euEEwGSJ7bkUBVj8abGfbByMuwXweURhv6A6uXNRdlCUkJz1w/E91q12rLOwTkRERERERJR2LKwTAFHoPdohYgjm1jiA1l3ijvIU5qtLdSvE9sh6EbURzu5ngf5OwDEBmHhybOddcInYHnwjbCeo2WhAmVpQbXQllrMuM9btsrAeZXhpod8JAGiPMwqmJE8W1tmxrgsZBeOWhXUZBTO4sD4sYz3eGBhp0eWAYgSObghetEqEzFfPrwaM5sTPQ9lFK6zHmbMuO9YZBUNERERERESUdiysEwBgmxoDM6nEDkeOOZivXpbCfHWpaiFgMAPuVqDjUPhjtjwptgsvAwwxvmyLJqpF+ACw9c/hn1rNWW9wJpaz3qF2rFs8TnFDxI51UVjP97UDiD0K5rhbFHpLctXOaZmvzcJ6coZ2rGvDS8XvpUXNWK8YmrHefkBs4y2s51cCM9aI/WS61pmvPjZphfX6+B4nM9Y5vJSIiIiIiIgo7VhYJwDBwvr82kJxQzo71s02UVwHwsfBdDYAH74q9hdeFt+5F8k4mKfCdsNXq4XTRDrWA4EAnGrnubnfKW6MOLxUFL7snnYAgZgL6+2hHet+v/hZAEBBTdzrpRCysN7fCXh6BkXBdPd70d3vBRAmY13rWJ8a/3PKIaYfPAV4E4se4oWVMUrm5cdTWPf7gR5xoY5RMERERERERETpx8I6AQC2afnqBeKGdHasA0CdmrMeboDp1j8DAT8wYSVQEmdBc/bHAVMOcHwv0LB52N1VDtGx3uiKv2Pd7fFhwBeAET4YZGE9yvBSo78feejVhp5GI7PfS3ItIrbEPyDytfOr4l4vhbAWACa1aO5u0Qrrrd39aFJfC3lWE/KspsGPSzQKBgCmny0iaHqOA3ufT2zdMiqEg0vHlkQ61vucQMAn9llYJyIiIiIiIko7FtYJQEjHek0h0NsBdDeJO8pmpmcBsrB+dEhhPRAAtqjDR+Uw0njYCoDZ54r9D54adnd1oSiuNjjj71iX3eqlppDH5hSFP9iSC5hzAQAlSic63AMxPYfM/C7JswZz4vOrAaNphEdRVIqixfOgu1XL2u/q8+JIew8AoKJgyODSQABoPyj2EymsG03B1/DmPySyakbBjFWysO6KI2PdrearWx2xD9IlIiIiIiIiIt2wsE5od3twzCkHlxYEu9ULakVhOh1q1cJ603bA4w7efmwzcHyP6Dqfe0Fi55bxMdv+NiyCI5mOdaearz7BphbWrY6RC955ZQCAUrjQpmanR9Me2rGuxYAwBkYXWs56MwpyTLAYxZ/DHQ3iIlPl0Hz17hbA0y0+MVA0MbHnXHIlAAXY/3JiQ0y1jvUJiT0/ZadC9fXU1Qh4Y/vboOWr57JbnYiIiIiIiCgTWFgnrVt9SmkuCmzm9OarS44akRse8IliuvSBOrR09rmAzZHYuSefBuRVAr3twL4XB91VpXasNybQsS7jXGqtalHeHqFbXVI7pEsVF/oG/Oj1+KI+R1toxrrrmLiR+dr6yKsQ2+5mKIoifsYAdjR0AhghX91RC5iGdLPHqmQqMPMcsf/uL+N/PDvWxyZ7MWC2i335yZSR+H3B1wJjYIiIiIiIiIgygnkShG1HnQCAeTVq4VrLV09jYR0AapcDO4+JOJjJHxGdm9v+Ju5bdHni5zWagAWfBN65H/jgT8FoGADVasd6c1c/fP4AjAYl5tN2qB3rVZYewI3Ig0ulXNGxXmnsBPxAe48HNZaciIcHAgGts31QFAwHl+pDdqy7WwEApXlWNLr6sF3tWI88uDSBGJhQq24C9jwroonO+I422DaqPpcYtgrw4spYoygiDqZ1N/D+4+KiT29H8KunffD3fS4A6jBme4yvHyIiIiIiIiLSFQvrpHWsL6hVC+tax3qaBpdKdScAO/8FHNkovt/zXzGgL78amHxqcudeeLkorO99QRSp7KIIXpZvhcmgwOsPoKWrT4uGiYXMWK80iUzuqJ2jahRMrbkbGAA63B7UFEZ+vq5+LwZ8onhWkmsBOtXCOgdX6iMkCgYAStWO9SPt4tMLlcMK6wfENtnC+oSVQPVioOF9YOPvgNO+GdvjZAxMTrHI7KexRRbW37o39sfYCoH5n0jZkoiIiIiIiIgoMhbWCduOisL68I71dBfWV4jt0Q3q0FI1BmbhZYDBmNy5K+YAlQuApq3A9r8DJ1wHADAaFFQU2HDM2YsGZ7yFddGxXmrsFjfYo3Wsi0JupakLQDA/PRIZA5NrMcJmNgY71tmtrA+tsN4CQHSsh0pZx7qiACtvAv7+WWDjw8BJXwbMtuiPYwzM2HbiF4D+LhEzlFMsBiHLL/uQ7+WX0ZzpVRMRERERERGNWyysj3PHu/vR4OqDogBzqwtEN7dbFBpRNjO9i6lcABitYihf/btiwCOQXAxMqIWfEoX1D57SCusAUOUQhfVGVy+AKDnpIWTGepGiDluNFgWjFnIrDCLOI3phPSQGBggprDMKRhdaxrpaWM8fWlgfkqOuFdanJv/cc84HXrod6DwGbPsrsOQz0R+jDS5lYX1MmnqG+CIiIiIiIiKiUYHDS8c5GQMzuTQX+TYz0KLGwDgmANa89C7GZAGqF4n9/35DDDKtXQ6UTtfn/PM/AShG4Nh7wPF92s1VahxLo7MvrtPJjvVCiA706B3rIgqmBOJnHq2wfjx0cOlAn5YFzsKqTnKHRsEMLqRXOkK6yAMBoP2g2E+2Yx0QncYrbhD76x4U54/GVS+2hROSf34iIiIiIiIiIkoKC+vj3HY1BmZBzdB89TQPLpVql4tt01axXfgp/c6dVw5MWy32P/iTdnO1WkBtcPXGdTrZsZ7vVwdK5kTpdlcL64UB56DHRyIL7yJf/Zi40WyP/jwUm9AomEBAy1gHAIMClIUW2t3H1cGhClA0SZ/nX3IlYMkT77kDa6Mfz451IiIiIiIiIqKswcL6OHeoTQzeHJ6vnqHCusxZB0QszLyL9D3/IrVQ/8GfAb8fgIiCAYAmV2Id67k+cXEi+vBSUcjN83YAiCMKJtcajIEpqBEZ3ZQ8WVj39gKe7kGF9NI8K0zGkD+PMgbGURtbHnoscgqBxWoEzLoHox/PjHUiIiIiIiIioqzBwvo497NLFmL9/56Ji5eoAzFb1cJ6eZoHl0p1JwT3Z52jf3f2jI8BVgfQeRQ4/BaAYBRMQ9yFdVEYtw7IwnpsUTBWnxtWeKJ2rLe5Q6JgZMc6B5fqx5IrOsYBoLtlUMZ65MGlk/Vdw4k3AIoBOPAK0Lxz5GO1jH0W1omIiIiIiIiIMo2FdUJFgQ1FuWoMRmuGO9bzK4PDIRd9Wv/zm23AvAvF/ta/AACqHTJjPd4oGNGxbvE4xQ3RhpfaHIBR/JxL4UJbd2yF9eJcCweXpkpeMGc9NGN9eGH9gNjqka8eqmgSMPs8sf/uCF3rA31aFjwL60REREREREREmcfCOgW524IDMstmZm4dlzwGfOL3wPTVqTm/zFlv3gEAqCoURdTW7n54vP6YTuHzB9DZNwAgAGO/iHaJ2rGuKNrAzFLFFb1jXY2CKc2zsls5VfIqxLa7GYU5ZhgNIman0jF4kGmwY13nwjoArLxJbLf+BehqDn9MaMZ+tNcZERERERERERGlHAvrFCQHlxZOFDEZmVI5T/9s9VCFE8XWeRiAGA5qMRkQCADNnbHFwXT2DiAQAPLQC8XvFTdG61gHgNxSAKKw3u4eGPFQ2dFekhfasc4oGF1pHeutMBgUMSgWQGXEKJip+q+h7gQxtNfnATb+NvwxznqxddQxY5+IiIiIiIiIKAuwsE5BLWphPVMxMOlSOEFse9qA/m4oiqINMG2MMWdddpvXWtX4GFMOYLFHf2Ce7FjvREePB4FAIOKhbe4Iw0tJP7nBKBgAKFNz1stDC+uBANCWwo51INi1vvG3wECYSCIOLiUiIiIiIiIiyiosrFOQNrh0jBfWcwpF3jmgdQIHC+ux5azLfPVam3p8rPEcaiG3BC4RJ9PrDXuY3x9AuxxemmsOGV7KwqquQqJgAODcBdWoKczBqqklwWN62oF+dUBt0aTUrGPWueKCT2878MFTw+93qoV1/v6JiIiIiIiIiLICC+sU1CIHl87O7DrSYUgcTJU6wLTBGVvHulPtWK+xqIX1WGJgACCvDABQbeoCALRHyFl39g7ArzazFxl6AU+3+IbDS/Ulo2DU2QJfOG0q3v7WGagtCvn0gYyBya+O7VMJiTCagBVfEPvrHgT8Q7L+5ScW2LFORERERERERJQVWFinIJmxPtY71gGgSBbWE+tYd6od65XmHnGDvSi251U71qtkYd0dvrAuB5c6csywuNVudXsJYM6J7XkoNnmDo2DCkoX1khTkq4da/GnAWgC07QP2vzT4Phc71omIiIiIiIiIsgkL6yR0t4rMcShA6cxMryb1ZMd6h9qxXhhfx7rMWC8zucUN9pIRjg6RKzrWyw0iWqQjUmFdi4GxAC4ZA8PBpbrTCustkY/RBpdOTu1abAXA0qvE/roHBt8XOryUiIiIiIiIiIgyjoV1EmS3etHE1MVdZJMhUTDVCXaslxrUiJY4o2CKIQrrkaJg2rrVwnqeJditXMDCuu60jPUWMaQ0nPYDYpuqwaWhTvg8oBiBg28AjVvFbX5fMGOfUTBERERERERERFmBhXUSxlO+OiAGRQLDMtYbXfF1rBdCLazHObzU4VcL6xE71kUUTEmuNZivzY51/amfIIB/AOjtCH+M1rGe4igYQBTO514g9tc9KLZdTYDfCxhMQH5V6tdARERERERERERRsbBOwnjKVweCGesdImKjulB0rLe7Pegb8EV9uOxYLwiIrPTYO9ZFYT3P54IJ3ohRMMdDO9Y7GQWTMiYrYCsU+5HiYLTCeho61gFg5Y1iu/1vQGdDyCcWqgGDMT1rICIiIiIiIiKiEbGwTsJ47VjvdwG9TjhyzMgxi6JlLF3rzl5R+M7zd4obYu1YzykGFPG2K0ZXxI71dq1j3RLSsV4T23NQfGQcjDtMYb2nPdjJnuqMdalmKTBhlehS3/BwyO9/Qnqen4iIiIiIiIiIomJhnUS29HjrWLfkAvZSse88DEVRUKV2rTc6o+esd7hFx3rOgFPcEOvwUoNBe94yxaVFygwVzFi3hgwvZb52Sow0wLT9oHpMpXjNpIvsWn/vEaBVvejFTywQEREREREREWUNFtZJFBR7O0QndemMTK8mfbQ4GDnAVOSsN8TSsa4WxK0DIis95igYQCvkliquyBnrsrCeawxGwRSwYz0ltMJ68/D7ZAxMSRry1UPN/BhQNBnocwIbfiNu4+BSIiIiIiIiIqKswcI6BbvViyYB5pyMLiWttAGmIme9yhFHx7qasW7qV2NC7EWxP686MLMEIxTW1SiYSsUJBHyAYgTyK2N/DoqdjIIZqbCerhgYyWAETvyi2O9TL97wEwtERERERERERFmDhXUaf/nqUqHase4UHetVhbF1rPcN+NA74IMVHhi8ahFe74519fbyQJu4oaCGgytTRetYbx1+X/sBsU3X4NJQi68IDlYF2LFORERERERERJRFWFin8ZevLskoGLVjvVp2rLtG7lh39Ypu9WKlW9ygGAGbI/bnVTvWS5VOdPZ5MeDzD7p7wOeHU+2ILxpQu6g5uDR1cmOIgslEYd2SCyy7Jvg9h5cSEREREREREWUNFtZpHHesq4XKjsEd643OkTvW5cDRCTnqcfZiQFFif161sF6miIgPWUQfen5FAXL7msSNHFyZOloUTLjhpbKwnuaMdemE6wFTjrhww9cAEREREREREVHWMGV6AZRhgQDQqhbWx1vHeuEksXUeBgKBmDvWO9yiEF5r7QV6EV8MDKBFj1QaO4EBoN3tQVm+VbtbDi4ttltgkINLWVRNHRkF4x5SWO91Aj1qFE+6M9algmrgulfEvtmWmTUQEREREREREdEwLKyPd93NQJ8TUAxAyfRMrya9ZLF6oAfoaUNVYSEAoLPPC3e/F7nW8G8PV68ofFdZ1MK6Pc7Cuho9UmboAoBhOeuysF6SZwFcR8WNBYyCSRmtsN4K+H3BLHvZrZ5bDljzM7M2AKiYk7nnJiIiIiIiIiKisBgFM961qPnqxVPGX0es2QbkV4n9jsPIs5qQbxPF9JG61jvU6JZyo1vcEHfHuoiCKYFTPd+Qwrq7X9yfawU61cK6g4MrU8ZeCkABAv5ghzoQLKyXZCgGhoiIiIiIiIiIshYL6+OdjIEpG2cxMFKhHGAqctarHSJnvWGEnHVZCC+VhXV7UXzPqXasF/hdUOCP2LFeHNqxzuGlqWM0AbmlYj80Z739oNhmYnApERERERERERFlNRbWxzvZsV4+zgaXSnKAqVMOMI2esy6HjZYoIsoF9pL4nlMt4hrhRxG60TG0sK52rFflhHRQM2M9tdSLHehuDt7WfkBsM5WvTkREREREREREWYsZ6+PdostF4XDiSZleSWYUyY71egBAVQwd6061Y92hdIsb4o2CMZqBnCKgtwOligttETrWJ5ic4gZLHmArjO85KD555UDLjiEd62oUDDvWiYiIiIiIiIhoCBbWx7sJJ4qv8UpGwXTIKJjoHesyYz3f3yluiHd4KSA6pHs7UKJ0hslYF99XK2q3ekENoCjxPwfFLq9CbN3hCuvMWCciIiIiIiIiosEYBUPj27AoGNGx3uiK3rFu96qF9Xg71gHRIQ2gDK4wGesiCqYi0CpuYAxM6qkDZbWO9b5OwK3+/BkFQ0REREREREREQ7CwTuObFgVzBPD7tY71Bmf0jnWb1yluSKhjXRRySxVXxI71Yh8L62kjO9ZlxrrsVreXAjZHZtZERERERERERERZi4V1Gt8KagDFAPj6ge5mVGpRMH0IBAJhHyI71i39TnFDvMNLAa1jvVRxocM9MOgumbGe398kbmBhPfUiFdZLGANDRERERERERETDsbBO45vRDBSohWtnvTa8tMfjQ2evd9jhgUAAzp4BGOGD0eMSNyYSBZNbCgAohQtt7n7t5r4BH7r7xfPm9LKwnjbqhQ50q58S4OBSIiIiIiIiIiIaAQvrRFoczGHkWIwospsBAA1hBph293vh9QfggDt4Y05R/M+ZKwq5JUon+gb86PX4AEDLWzcZFJi6joljC2riPz/FJ1cW1mXH+kGxZWGdiIiIiIiIiIjCYGGdSA4w7VAHmDrkANPhhXWnmq9eYVIL61YHYDTF/5xqh3S5Irre29V4GRkDU5JrhuI6Ko5lx3rqySiY3nbANwC0HxDfs7BORERERERERERhsLBOVBjsWAeA6kI5wLRv2KFy0GidTb3PnkC3OqB1SJcZOsV51U51GQsz0e4BvGphnx3rqZdTBBjUCyTuVkbBEBERERERERHRiFhYJ5Id687YO9arrT3ihkQGlwJAXhkAoAROAAG0uQd3rM+wOcVxuWWA2ZbYc1DsDAbxswZEUV1GwhRPztyaiIiIiIiIiIgoa7GwTqRlrNcDAKrUjvXGETrWK81qYT2RwaWAVsQ1w4sC9AzrWJ9kahfHMQYmfeQA0/p3xTanOLH8fCIiIiIiIiIiGvNYWCeSUTCuo4Dfh2q1Yz3c8FLZsV5uVDPW7QkW1s05gCUfgBhg2j6kY73GoBbWGQOTPjJnXRbWGQNDREREREREREQRsLBOlF8JGMyA3wt0HkOVQ+1Yd0XuWC82qIX1RDvWAS0OphQu7bwyEqYicFwc46hL/PwUHzX3Hkc2iG3J1MythYiIiIiIiIiIshoL60QGI1CoFrCd9agulBnrfQgEAoMOlR3rxegSNyTasQ5ohdxSxRXSsS6iYEr8reIYRsGkj4yC6XeJLTvWiYiIiIiIiIgoAhbWiYDgANOOw6gosEFRAI/Xr3WQS061szw/oENhXXashxbW1a2jXx2e6WAUTNrIKBiJhXUiIiIiIiIiIoqAhXUiIJiz7qyHxWRAaZ4VwPABph1qx3qev1PckEwUTG6YwrqasW7vaxTHMAomfdQLHZpiRsEQEREREREREVF4LKwTAUCRLKwfBgBUaznrgweYyo71HK8aF6JHFAw60dHjQSAQQJu7H0b4YO6RHeuMgkmbYR3rkzOzDiIiIiIiIiIiynosrBMBwY71DlFYr3IEc9ZDyY51q8cpbtBjeKniQrt7AD0eH/oG/KhAB5SAXwxUlQM1KfVCC+u2wuQumhARERERERER0ZjGwjoRMCgKBgCqCkXHekPYjvUATLKwbi9J/DlDhpd29Hi0GJhJpnZxf0E1YOBbNG3yQi5iMF+diIiIiIiIiIhGwKodERAcXtp5DPB6UC071kMy1r0+Pzr7vMhHLxS/V9yY1PBSGQXjgs8fwIfHuwEA03PU/HbGwKSXtQAwimx9lDBfnYiIiIiIiIiIImNhnQgQRW5TDoAA0HlU61gPzVh39YoYmEKlS9xgygHMOYk/pzq8tEwRee37W0RhfbJZ7VhnYT29FCUYB8OOdSIiIiIiIiIiGgEL60SAKKrKrvWOw1rGekNIx7rMV6+xqsX2ZDO41cK6XelHDvqwr1kU1muNLKxnTH6l2BazY52IiIiIiIiIiCIbtYX1Bx98EJMmTYLNZsOKFSuwYcOGmB731FNPQVEUXHDBBaldII0+srDurEe12rHe3NkHnz8AAHD1igx0rbCezOBSALDmAybxPCVKJ/a3isJ6VeC4uL+gJrnzU/xO/Qaw6Apg1v9keiVERERERERERJTFRmVh/c9//jNuueUW3HHHHdi8eTMWLlyINWvWoKWlZcTHHTp0CLfeeis+8pGPpGmlNKoUyQGmh1Geb4PRoMDrD+B4dz8AoMOtdqybe8RxyXasK4o2wLQMLuxrFhEzpf5Wcb+jLrnzU/ymnwVc8EvAmpfplRARERERERERURYblYX1e+65B9dddx2uueYazJkzBw899BDsdjseeeSRiI/x+Xy44oor8L3vfQ9Tpoycn9zf34/Ozs5BXzQOFKqF9Y7DMBoUVOSLQZYNTtGh3tEjOtbLTToV1gEgT8TBlCoudPaJgagOj3qBiFEwREREREREREREWWnUFdY9Hg82bdqE1atXa7cZDAasXr0a69ati/i4O++8E+Xl5fjsZz8b9TnuuusuOBwO7auujp3D40JIFAwAVBWKnPVGl8hZd6oZ66VGtzgu2SgYQMtZL1UHmNrRB5tX7MPBKBgiIiIiIiIiIqJsNOoK68ePH4fP50NFRcWg2ysqKtDU1BT2MW+99RZ+97vf4eGHH47pOW677Ta4XC7t68iRI0mvm0aBkCgYAKhyiPxz2bHuVDPWixWRha5Lx7paWC+B+FREldImbrcWADZH8ucnIiIiIiIiIiIi3ZkyvYBU6+rqwmc+8xk8/PDDKC0tjekxVqsVVqs1xSujrCOjYLqbgYFerbAuO9Y71I51B0QWui4d63kiY112rFfLwjoHlxIREREREREREWWtUVdYLy0thdFoRHNz86Dbm5ubUVlZOez4AwcO4NChQzjvvPO02/x+PwDAZDJhz549mDp1amoXTaNDThFgyQc8XYDzCKocMgpG7VhXM9bz/Wrmvr0k+efMjVBYZ746ERERERERERFR1hp1UTAWiwVLly7F2rVrtdv8fj/Wrl2LlStXDjt+1qxZ2LZtG7Zs2aJ9ffzjH8fpp5+OLVu2MD+dghRlUBxMdaGMglE71t2iY93uUzPQdRxeWsbCOhERERERERER0agx6jrWAeCWW27BVVddhWXLluGEE07AfffdB7fbjWuuuQYAcOWVV6KmpgZ33XUXbDYb5s2bN+jxhYWFADDsdiIUTgCatwPOw6iqXA4gpGO9VxTWbQNqYV3P4aUQ55xgbBe3c3ApERERERERERFR1hqVhfVLL70Ura2tuP3229HU1IRFixbh+eef1waa1tfXw2AYdc34lA1kznrHYVTNEh3rLV39GPD5tSgYs8cpjrEXJf98ahRMiSLiZSYY2wA/AAc/SUFERERERERERJStRmVhHQBuuukm3HTTTWHve+2110Z87KOPPqr/gmhsKJwgts56lOZaYTYqGPAF0NzZh44eD6zwwOgVHex6Di8tVNwww4sqDi8lIiIiIiIiIiLKemzrJgoVkrFuMCiodIiu9YPH3egb8KMIXeJ+xQjYHMk/n60QMIjrW6VwoczfKm5nxjoREREREREREVHWYmGdKFRIFAwAVDlyAAC7GkVUS4nRLe63F4thp8kyGLSc9RmGozAHBgAoQEF18ucmIiIiIiIiIiKilGBhnSiUjILpbQf6u1CtdqzvbBCF9TqrjjEwUm4pAGCBckB8n1cOmKz6nZ+IiIiIiIiIiIh0xcI6UShbAZCjDiV11qOqUHasiwiYKotaWLfrWVgXOesLDAfF94yBISIiIiIiIiIiymosrBMNFTLAVHas72/tBgBUmdUoGD071tUBpidYRfwMC+tERERERERERETZjYV1oqFCctZlxrrPHwAAlBl7xH26dqyLjHWH97j4voCFdSIiIiIiIiIiomzGwjrRUEVqYd15GFWFtkF3lRhE57quhXW1Y13DjnUiIiIiIiIiIqKsxsI60VCyY91Zj2q1Y127SxFZ6/oOLy0b/L2jRr9zExERERERERERke5YWCcaKiQKptBuhs0cfJsUBNTCegqiYDTsWCciIiIiIiIiIspqLKwTDRUyvFRRlEFd63m+TrGTguGlGkedfucmIiIiIiIiIiIi3bGwTjSULKz3u4DejkE56zlel9ixl+j3fLkhhXWjBbCX6nduIiIiIiIiIiIi0h0L60RDWezBYrezHlUhHevWAafY0TMKxl4CQBH7BTWAgW9LIiIiIiIiIiKibMYKHlE4smu94zCqHaJj3QgfTJ4URMEYTcFCPfPViYiIiIiIiIiIsh4L60ThFKkDTJ2HUVUoOtYdcAfvzynS9/lkhzwL60RERERERERERFnPlOkFEGWlkAGmVVNFx3qR0iVuszpEl7me8sqA1l0srBMRERERERERxSEQCMDr9cLn82V6KZTFzGYzjEajrudkYZ0onEK1Y73jMGqLRMd6tblH3KZnvrpUMQ84+AZQtVD/cxMRERERERERjUEejweNjY3o6enJ9FIoyymKgtraWuTl5el2ThbWicLRomDqMbUsD184bSpWDrQCm5Cawvrq7wGLLhcFdiIiIiIiIiIiGpHf78fBgwdhNBpRXV0Ni8UCRVEyvSzKQoFAAK2trTh69CimT5+uW+c6C+tE4RQGM9YVAN/86Cxg80ZRWNdzcKlksgCV8/U/LxERERERERHRGOTxeOD3+1FXVwe73Z7p5VCWKysrw6FDhzAwMKBbYZ3DS4nCcdQCUICBHsB9XNzW2y62qehYJyIiIiIiIiKiuBkMLG9SdKn4NANfeUThmKxAQbXYd9aLbU+b2NpLMrMmIiIiIiIiIiIiygosrBNFUjhBbJ2HxLZH7VhPRRQMERERERERERERjRosrBNFUhgcYAoA6O0QW3tRZtZDREREREREREQ0xKRJk3DffffFfPxrr70GRVHgdDpTtqbxgIV1okhkx3rHYbFlxzoRERERERERESXptNNOw1e+8hXdzrdx40Zcf/31MR+/atUqNDY2wuFw6LaGcGQBf+jXt7/9bQBAX18frr76asyfPx8mkwkXXHBBStejN1OmF0CUtYpkx7paWOfwUiIiIiIiIiIiSoNAIACfzweTKXr5tqysLK5zWywWVFZWJrq0uO3ZswcFBQXa93l5eQAAn8+HnJwcfOlLX8Lf//73tK1HL+xYJ4pkaBQMh5cSEREREREREWWtQCCAHo83I1+BQCCmNV599dV4/fXX8fOf/1zr4D506JDW3f3f//4XS5cuhdVqxVtvvYUDBw7g/PPPR0VFBfLy8rB8+XK8/PLLg845NApGURT89re/xYUXXgi73Y7p06fjmWee0e4fGgXz6KOPorCwEC+88AJmz56NvLw8fPSjH0VjY6P2GK/Xiy996UsoLCxESUkJvvnNb+Kqq66Kqcu8vLwclZWV2pcsrOfm5uJXv/oVrrvuurQW+vXCjnWiSLThpfWA3xfMWGcUDBERERERERFR1ukd8GHO7S9k5Ll33rkGdkv0UuvPf/5z7N27F/PmzcOdd94JQHScHzp0CADwrW99C3fffTemTJmCoqIiHDlyBOeccw5++MMfwmq14rHHHsN5552HPXv2YMKECRGf53vf+x5+8pOf4Kc//Snuv/9+XHHFFTh8+DCKi8PXtXp6enD33Xfj8ccfh8FgwKc//WnceuuteOKJJwAAP/7xj/HEE0/g97//PWbPno2f//zn+Ne//oXTTz89zp/U2MGOdaJICmoAxQj4PEDbfsDvFbczCoaIiIiIiIiIiBLgcDhgsVhgt9u1Dm6j0ajdf+edd+Kss87C1KlTUVxcjIULF+Lzn/885s2bh+nTp+P73/8+pk6dOqgDPZyrr74an/rUpzBt2jT86Ec/Qnd3NzZs2BDx+IGBATz00ENYtmwZlixZgptuuglr167V7r///vtx22234cILL8SsWbPwwAMPoLCwMKZ/c21tLfLy8rSvtra2mB6X7dixThSJ0QQ4akTHesMWcZspBzDnZHRZREREREREREQ0XI7ZiJ13rsnYc+th2bJlg77v7u7Gd7/7XTz77LNobGyE1+tFb28v6uvrRzzPggULtP3c3FwUFBSgpaUl4vF2ux1Tp07Vvq+qqtKOd7lcaG5uxgknnKDdbzQasXTpUvj9/qj/pjfffBP5+fna90VFRVEfMxqwsE40ksKJamF9s/ie3epERERERERERFlJUZSY4liyWW5u7qDvb731Vrz00ku4++67MW3aNOTk5OATn/gEPB7PiOcxm82DvlcUZcQieLjjY82Nj2by5Mkxd7ePJoyCIRpJkTrAtOF9sWVhnYiIiIiIiIiIkmCxWODz+WI69u2338bVV1+NCy+8EPPnz0dlZaWWx54uDocDFRUV2Lhxo3abz+fD5s2b07qObDO6L+EQpVqhWlhv2ia2HFxKRERERERERERJmDRpEtavX49Dhw4hLy8v4kBRAJg+fTr+8Y9/4LzzzoOiKPjOd74TU/yK3m6++WbcddddmDZtGmbNmoX7778fHR0dUBQlqfPu3LkTHo8H7e3t6OrqwpYtWwAAixYtSn7RKcbCOtFIZGF9oEds2bFORERERERERERJuPXWW3HVVVdhzpw56O3txcGDByMee8899+Daa6/FqlWrUFpaim9+85vo7OxM42qFb37zm2hqasKVV14Jo9GI66+/HmvWrBk0eDUR55xzDg4fPqx9v3jxYgDQLYYmlZTAaFhlhnV2dsLhcMDlcqGgoCDTy6F0OrwO+P1Hg98v+yxw7j2ZWw8RERERERERkQ5Ge72rr68PBw8exOTJk2Gz2TK9nHHH7/dj9uzZuOSSS/D9738/08uJKhWvF3asE41EZqxL7FgnIiIiIiIiIqJx5vDhw3jxxRdx6qmnor+/Hw888AAOHjyIyy+/PNNLyxgOLyUaSV4lYLQEv7eXZG4tREREREREREREGWAwGPDoo49i+fLlOOmkk7Bt2za8/PLLmD17dqaXljHsWCcaicEAOOqA9gPiew4vJSIiIiIiIiKicaaurg5vv/12ppeRVdixThRNaBwMo2CIiIiIiIiIiIjGPRbWiaIpnBDcZ8c6ERERERERERHRuMfCOlE0haEd60WZWwcRERERERERERFlBRbWiaIZFAXD4aVERERERERERETjHQvrRNHIjnWDCbAWZHYtRERERERERERElHGmTC+AKOtVzAWKpwBlswBFyfRqiIiIiIiIiIiIKMPYsU4UjTkHuGkTcNmTmV4JERERERERERERJk2ahPvuu0/7XlEU/Otf/4p4/KFDh6AoCrZs2ZLU8+p1nrGAhXWiWBgM7FYnIiIiIiIiIqKs1NjYiI997GO6nvPqq6/GBRdcMOi2uro6NDY2Yt68ebo+11Df/e53oSjKsK+XX34ZALBjxw5cfPHFmDRpEhRFGXSRIV0YBUNEREREREREREQ0ilVWVqbleYxGY9qea+7cuVohXSouLgYA9PT0YMqUKfjkJz+Jr371q2lZz1DsWCciIiIiIiIiIqLRLxAAPO7MfAUCMS3xN7/5Daqrq+H3+wfdfv755+Paa68FABw4cADnn38+KioqkJeXh+XLlw8rMA81NApmw4YNWLx4MWw2G5YtW4b3339/0PE+nw+f/exnMXnyZOTk5GDmzJn4+c9/rt3/3e9+F3/4wx/w9NNPa93ir732WtgomNdffx0nnHACrFYrqqqq8K1vfQter1e7/7TTTsOXvvQlfOMb30BxcTEqKyvx3e9+N+rPymQyobKyctCXxWIBACxfvhw//elPcdlll8FqtUY9VyqwY52IiIiIiIiIiIhGv4Ee4EfVmXnu/20ALLlRD/vkJz+Jm2++Ga+++irOPPNMAEB7ezuef/55PPfccwCA7u5unHPOOfjhD38Iq9WKxx57DOeddx727NmDCRMmRH2O7u5unHvuuTjrrLPwxz/+EQcPHsSXv/zlQcf4/X7U1tbir3/9K0pKSvDOO+/g+uuvR1VVFS655BLceuut2LVrFzo7O/H73/8egOgWb2hoGHSeY8eO4ZxzzsHVV1+Nxx57DLt378Z1110Hm802qHj+hz/8AbfccgvWr1+PdevW4eqrr8ZJJ52Es846K+q/J1uxsE5ERERERERERESUBkVFRfjYxz6GJ598Uius/+1vf0NpaSlOP/10AMDChQuxcOFC7THf//738c9//hPPPPMMbrrppqjP8eSTT8Lv9+N3v/sdbDYb5s6di6NHj+ILX/iCdozZbMb3vvc97fvJkydj3bp1+Mtf/oJLLrkEeXl5yMnJQX9//4jRL7/85S9RV1eHBx54AIqiYNasWWhoaMA3v/lN3H777TAYRGDKggULcMcddwAApk+fjgceeABr164dsbC+bds25OXlad/PmTMHGzZsiPrvTxcW1omIiIiIiIiIiGj0M9tF53imnjtGV1xxBa677jr88pe/hNVqxRNPPIHLLrtMK0J3d3fju9/9Lp599lk0NjbC6/Wit7cX9fX1MZ1/165dWLBgAWw2m3bbypUrhx334IMP4pFHHkF9fT16e3vh8XiwaNGimP8d8rlWrlwJRVG020466SR0d3fj6NGjWof9ggULBj2uqqoKLS0tI5575syZeOaZZ7TvMxX5EgkL60RERERERERERDT6KUpMcSyZdt555yEQCODZZ5/F8uXL8eabb+Lee+/V7r/11lvx0ksv4e6778a0adOQk5ODT3ziE/B4PLqt4amnnsKtt96Kn/3sZ1i5ciXy8/Px05/+FOvXr9ftOUKZzeZB3yuKMixnfiiLxYJp06alZD16YGGdiIiIiIiIiIiIKE1sNhsuuugiPPHEE9i/fz9mzpyJJUuWaPe//fbbuPrqq3HhhRcCEB3shw4divn8s2fPxuOPP46+vj6ta/3dd98ddMzbb7+NVatW4Ytf/KJ224EDBwYdY7FY4PP5oj7X3//+dwQCAa1r/e2330Z+fj5qa2tjXvNoZMj0AoiIiIiIiIiIiIjGkyuuuALPPvssHnnkEVxxxRWD7ps+fTr+8Y9/YMuWLfjggw9w+eWXR+3uDnX55ZdDURRcd9112LlzJ5577jncfffdw57jvffewwsvvIC9e/fiO9/5DjZu3DjomEmTJmHr1q3Ys2cPjh8/joGBgWHP9cUvfhFHjhzBzTffjN27d+Ppp5/GHXfcgVtuuUWLtkkFj8eDLVu2YMuWLfB4PDh27Bi2bNmC/fv3p+w5h2JhnYiIiIiIiIiIiCiNzjjjDBQXF2PPnj24/PLLB913zz33oKioCKtWrcJ5552HNWvWDOpojyYvLw///ve/sW3bNixevBj/7//9P/z4xz8edMznP/95XHTRRbj00kuxYsUKtLW1DepeB4DrrrsOM2fOxLJly1BWVoa333572HPV1NTgueeew4YNG7Bw4ULccMMN+OxnP4tvf/vbcfw04tfQ0IDFixdj8eLFaGxsxN13343Fixfjc5/7XEqfN5QSCAQCaXu2UaqzsxMOhwMulwsFBQWZXg4REREREREREVFSRnu9q6+vDwcPHsTkyZMHDekkCicVrxd2rBMRERERERERERERxYGFdSIiIiIiIiIiIiKiOLCwTkREREREREREREQUBxbWiYiIiIiIiIiIiIjiwMI6ERERERERERERjUqBQCDTS6BRIBWvExbWiYiIiIiIiIiIaFQxm80AgJ6engyvhEYDj8cDADAajbqd06TbmcYweUWjs7MzwyshIiIiIiIiIiJKnqxzjdaOb6PRiMLCQrS0tAAA7HY7FEXJ8KooG/n9frS2tsJut8Nk0q8czsJ6DLq6ugAAdXV1GV4JERERERERERGRfrq6uuBwODK9jIRUVlYCgFZcJ4rEYDBgwoQJul58UQKj9bJUGvn9fjQ0NCA/P3/MXfnq7OxEXV0djhw5goKCgkwvh4iSxPc00djC9zTR2ML3NNHYwvc0jXaBQABdXV2orq6GwTC606J9Ph8GBgYyvQzKYhaLRffXOTvWY2AwGFBbW5vpZaRUQUEB/4cA0RjC9zTR2ML3NNHYwvc00djC9zSNZqO1U30oo9Goa3Y2USxG9+UoIiIiIiIiIiIiIqI0Y2GdiIiIiIiIiIiIiCgOLKyPc1arFXfccQesVmuml0JEOuB7mmhs4XuaaGzhe5pobOF7mohofOPwUiIiIiIiIiIiIiKiOLBjnYiIiIiIiIiIiIgoDiysExERERERERERERHFgYV1IiIiIiIiIiIiIqI4sLBORERERERERERERBQHFtbHuQcffBCTJk2CzWbDihUrsGHDhkwviYiiuOuuu7B8+XLk5+ejvLwcF1xwAfbs2TPomL6+Ptx4440oKSlBXl4eLr74YjQ3N2doxUQUj//7v/+Doij4yle+ot3G9zTR6HLs2DF8+tOfRklJCXJycjB//ny899572v2BQAC33347qqqqkJOTg9WrV2Pfvn0ZXDERReLz+fCd73wHkydPRk5ODqZOnYrvf//7CAQC2jF8TxMRjU8srI9jf/7zn3HLLbfgjjvuwObNm7Fw4UKsWbMGLS0tmV4aEY3g9ddfx4033oh3330XL730EgYGBnD22WfD7XZrx3z1q1/Fv//9b/z1r3/F66+/joaGBlx00UUZXDURxWLjxo349a9/jQULFgy6ne9potGjo6MDJ510EsxmM/773/9i586d+NnPfoaioiLtmJ/85Cf4xS9+gYceegjr169Hbm4u1qxZg76+vgyunIjC+fGPf4xf/epXeOCBB7Br1y78+Mc/xk9+8hPcf//92jF8TxMRjU9KIPQyK40rK1aswPLly/HAAw8AAPx+P+rq6nDzzTfjW9/6VoZXR0Sxam1tRXl5OV5//XWccsopcLlcKCsrw5NPPolPfOITAIDdu3dj9uzZWLduHU488cQMr5iIwunu7saSJUvwy1/+Ej/4wQ+waNEi3HfffXxPE40y3/rWt/D222/jzTffDHt/IBBAdXU1vva1r+HWW28FALhcLlRUVODRRx/FZZddls7lElEU5557LioqKvC73/1Ou+3iiy9GTk4O/vjHP/I9TUQ0jrFjfZzyeDzYtGkTVq9erd1mMBiwevVqrFu3LoMrI6J4uVwuAEBxcTEAYNOmTRgYGBj0/p41axYmTJjA9zdRFrvxxhvxP//zP4PeuwDf00SjzTPPPINly5bhk5/8JMrLy7F48WI8/PDD2v0HDx5EU1PToPe0w+HAihUr+J4mykKrVq3C2rVrsXfvXgDABx98gLfeegsf+9jHAPA9TUQ0npkyvQDKjOPHj8Pn86GiomLQ7RUVFdi9e3eGVkVE8fL7/fjKV76Ck046CfPmzQMANDU1wWKxoLCwcNCxFRUVaGpqysAqiSiap556Cps3b8bGjRuH3cf3NNHo8uGHH+JXv/oVbrnlFvzv//4vNm7ciC996UuwWCy46qqrtPdtuP8dzvc0Ufb51re+hc7OTsyaNQtGoxE+nw8//OEPccUVVwAA39NEROMYC+tERKPYjTfeiO3bt+Ott97K9FKIKEFHjhzBl7/8Zbz00kuw2WyZXg4RJcnv92PZsmX40Y9+BABYvHgxtm/fjoceeghXXXVVhldHRPH6y1/+gieeeAJPPvkk5s6diy1btuArX/kKqqur+Z4mIhrnGAUzTpWWlsJoNKK5uXnQ7c3NzaisrMzQqogoHjfddBP+85//4NVXX0Vtba12e2VlJTweD5xO56Dj+f4myk6bNm1CS0sLlixZApPJBJPJhNdffx2/+MUvYDKZUFFRwfc00ShSVVWFOXPmDLpt9uzZqK+vBwDtfcv/HU40Onz961/Ht771LVx22WWYP38+PvOZz+CrX/0q7rrrLgB8TxMRjWcsrI9TFosFS5cuxdq1a7Xb/H4/1q5di5UrV2ZwZUQUTSAQwE033YR//vOfeOWVVzB58uRB9y9duhRms3nQ+3vPnj2or6/n+5soC5155pnYtm0btmzZon0tW7YMV1xxhbbP9zTR6HHSSSdhz549g27bu3cvJk6cCACYPHkyKisrB72nOzs7sX79er6nibJQT08PDIbBpROj0Qi/3w+A72kiovGMUTDj2C233IKrrroKy5YtwwknnID77rsPbrcb11xzTaaXRkQjuPHGG/Hkk0/i6aefRn5+vpbd6HA4kJOTA4fDgc9+9rO45ZZbUFxcjIKCAtx8881YuXIlTjzxxAyvnoiGys/P12YkSLm5uSgpKdFu53uaaPT46le/ilWrVuFHP/oRLrnkEmzYsAG/+c1v8Jvf/AYAoCgKvvKVr+AHP/gBpk+fjsmTJ+M73/kOqqurccEFF2R28UQ0zHnnnYcf/vCHmDBhAubOnYv3338f99xzD6699loAfE8TEY1nLKyPY5deeilaW1tx++23o6mpCYsWLcLzzz8/bOgKEWWXX/3qVwCA0047bdDtv//973H11VcDAO69914YDAZcfPHF6O/vx5o1a/DLX/4yzSslIr3wPU00eixfvhz//Oc/cdttt+HOO+/E5MmTcd9992mDDgHgG9/4BtxuN66//no4nU6cfPLJeP755zlngSgL3X///fjOd76DL37xi2hpaUF1dTU+//nP4/bbb9eO4XuaiGh8UgKBQCDTiyAiIiIiIiIiIiIiGi2YsU5EREREREREREREFAcW1omIiIiIiIiIiIiI4sDCOhERERERERERERFRHFhYJyIiIiIiIiIiIiKKAwvrRERERERERERERERxYGGdiIiIiIiIiIiIiCgOLKwTEREREREREREREcWBhXUiIiIiIiIiIiIiojiwsE5ERPT/27mfkKrWPYzjX7FSQ0VMCUFLQRSNNKygMohNNpIgLDQwNEQaNAkrK5TCdLCdOLHozyASJ0UUDcIGJmEDKVDBKNukFRRNMsyCUCTSMziw79lcOOesWyFdvx9YsNjrXe9+eIcPi5+k387g4CBxcXF8/vx5qaNIkiRJWoYs1iVJkiRJkiRJCsBiXZIkSZIkSZKkACzWJUmSFNjCwgLhcJi8vDySkpIoLS3l9u3bwH/GtPT19VFSUkJiYiLbtm3j+fPnMXvcuXOHDRs2kJCQQG5uLl1dXTHP5+fnOX36NDk5OSQkJJCfn8+1a9di1oyOjrJlyxZWr17Njh07ePnyZfTZ06dPCYVCpKSkkJqayubNmxkZGflFJyJJkiRpObFYlyRJUmDhcJje3l6uXLnC+Pg4TU1NHDp0iEePHkXXNDc309XVxfDwMJmZmezdu5dv374Bfxbi1dXVHDx4kGfPntHW1sbZs2fp6emJvl9XV8eNGzfo7u4mEolw9epVkpOTY3K0trbS1dXFyMgIK1asoKGhIfqstraW7OxshoeHGR0d5cyZM6xcufLXHowkSZKkZSFucXFxcalDSJIk6fcxPz9Peno6AwMDbN++Pfp7Y2Mjs7OzHDlyhFAoxM2bN6mpqQHg06dPZGdn09PTQ3V1NbW1tXz8+JH+/v7o+6dOnaKvr4/x8XEmJiYoLCzkwYMHVFRU/FeGwcFBQqEQAwMD7N69G4D79+9TWVnJ3NwciYmJpKamcuHCBerr63/xiUiSJElabvxiXZIkSYG8evWK2dlZ9uzZQ3JycvTq7e3l9evX0XV/Ld3T09MpLCwkEokAEIlEKC8vj9m3vLycyclJvn//ztjYGPHx8ezatetvs5SUlETvs7KyAJiamgLg+PHjNDY2UlFRQWdnZ0w2SZIkSfoRFuuSJEkK5OvXrwD09fUxNjYWvV68eBGds/6jkpKS/tW6v452iYuLA/6c/w7Q1tbG+Pg4lZWVPHz4kOLiYu7evftT8kmSJEla3izWJUmSFEhxcTEJCQm8e/eO/Pz8mCsnJye67smTJ9H7mZkZJiYmKCoqAqCoqIihoaGYfYeGhigoKCA+Pp6NGzeysLAQM7P9f1FQUEBTUxP9/f1UVVVx/fr1H9pPkiRJkgBWLHUASZIk/V5SUlI4efIkTU1NLCwssHPnTr58+cLQ0BCpqamsX78egPb2dtasWcPatWtpbW0lIyODffv2AXDixAm2bt1KR0cHNTU1PH78mIsXL3Lp0iUAcnNzqa+vp6Ghge7ubkpLS3n79i1TU1NUV1f/Y8a5uTmam5s5cOAAeXl5vH//nuHhYfbv3//LzkWSJEnS8mGxLkmSpMA6OjrIzMwkHA7z5s0b0tLSKCsro6WlJTqKpbOzk2PHjjE5OcmmTZu4d+8eq1atAqCsrIxbt25x7tw5Ojo6yMrKor29ncOHD0f/4/Lly7S0tHD06FGmp6dZt24dLS0t/ypffHw809PT1NXV8eHDBzIyMqiqquL8+fM//SwkSZIkLT9xi4uLi0sdQpIkSf8/BgcHCYVCzMzMkJaWttRxJEmSJOmnc8a6JEmSJEmSJEkBWKxLkiRJkiRJkhSAo2AkSZIkSZIkSQrAL9YlSZIkSZIkSQrAYl2SJEmSJEmSpAAs1iVJkiRJkiRJCsBiXZIkSZIkSZKkACzWJUmSJEmSJEkKwGJdkiRJkiRJkqQALNYlSZIkSZIkSQrAYl2SJEmSJEmSpAD+AGKztMdvuTkwAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 1500x1000 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "100%|██████████| 79/79 [00:20<00:00,  3.78it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:43,714 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n",
            "100%|██████████| 10/10 [00:02<00:00,  3.93it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:49,079 Evaluating as a multi-label problem: False\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n",
            "100%|██████████| 10/10 [00:04<00:00,  2.28it/s]"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "2023-04-21 07:40:53,567 Evaluating as a multi-label problem: True\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "\n",
              "  <div id=\"df-bacb1f8b-e29e-415b-bdbc-7505c8ee48ff\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>split</th>\n",
              "      <th>index</th>\n",
              "      <th>I-ADR</th>\n",
              "      <th>E-ADR</th>\n",
              "      <th>B-ADR</th>\n",
              "      <th>S-ADR</th>\n",
              "      <th>S-Drug</th>\n",
              "      <th>B-Finding</th>\n",
              "      <th>E-Finding</th>\n",
              "      <th>B-Drug</th>\n",
              "      <th>...</th>\n",
              "      <th>I-Finding</th>\n",
              "      <th>B-Symptom</th>\n",
              "      <th>E-Symptom</th>\n",
              "      <th>I-Symptom</th>\n",
              "      <th>I-Disease</th>\n",
              "      <th>I-Drug</th>\n",
              "      <th>micro avg</th>\n",
              "      <th>macro avg</th>\n",
              "      <th>weighted avg</th>\n",
              "      <th>samples avg</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>train</td>\n",
              "      <td>precision</td>\n",
              "      <td>0.747656</td>\n",
              "      <td>0.793743</td>\n",
              "      <td>0.777483</td>\n",
              "      <td>0.845815</td>\n",
              "      <td>0.986923</td>\n",
              "      <td>0.890000</td>\n",
              "      <td>0.824742</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.951220</td>\n",
              "      <td>0.968750</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.976190</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.811251</td>\n",
              "      <td>0.923504</td>\n",
              "      <td>0.814371</td>\n",
              "      <td>0.122043</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>train</td>\n",
              "      <td>recall</td>\n",
              "      <td>0.627332</td>\n",
              "      <td>0.781517</td>\n",
              "      <td>0.787698</td>\n",
              "      <td>0.845194</td>\n",
              "      <td>0.997667</td>\n",
              "      <td>0.445000</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.936364</td>\n",
              "      <td>...</td>\n",
              "      <td>0.167785</td>\n",
              "      <td>0.354545</td>\n",
              "      <td>0.271930</td>\n",
              "      <td>0.173077</td>\n",
              "      <td>0.820000</td>\n",
              "      <td>0.820000</td>\n",
              "      <td>0.735657</td>\n",
              "      <td>0.639484</td>\n",
              "      <td>0.735657</td>\n",
              "      <td>0.122001</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>train</td>\n",
              "      <td>f1-score</td>\n",
              "      <td>0.682229</td>\n",
              "      <td>0.787583</td>\n",
              "      <td>0.782557</td>\n",
              "      <td>0.845505</td>\n",
              "      <td>0.992266</td>\n",
              "      <td>0.593333</td>\n",
              "      <td>0.538721</td>\n",
              "      <td>0.967136</td>\n",
              "      <td>...</td>\n",
              "      <td>0.287356</td>\n",
              "      <td>0.516556</td>\n",
              "      <td>0.424658</td>\n",
              "      <td>0.295082</td>\n",
              "      <td>0.891304</td>\n",
              "      <td>0.901099</td>\n",
              "      <td>0.771607</td>\n",
              "      <td>0.721410</td>\n",
              "      <td>0.763643</td>\n",
              "      <td>0.122015</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>train</td>\n",
              "      <td>support</td>\n",
              "      <td>4449.000000</td>\n",
              "      <td>3506.000000</td>\n",
              "      <td>3349.000000</td>\n",
              "      <td>1363.000000</td>\n",
              "      <td>1286.000000</td>\n",
              "      <td>200.000000</td>\n",
              "      <td>200.000000</td>\n",
              "      <td>110.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>149.000000</td>\n",
              "      <td>110.000000</td>\n",
              "      <td>114.000000</td>\n",
              "      <td>104.000000</td>\n",
              "      <td>50.000000</td>\n",
              "      <td>50.000000</td>\n",
              "      <td>15722.000000</td>\n",
              "      <td>15722.000000</td>\n",
              "      <td>15722.000000</td>\n",
              "      <td>15722.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>dev</td>\n",
              "      <td>precision</td>\n",
              "      <td>0.635731</td>\n",
              "      <td>0.693750</td>\n",
              "      <td>0.707006</td>\n",
              "      <td>0.758621</td>\n",
              "      <td>0.940828</td>\n",
              "      <td>0.571429</td>\n",
              "      <td>0.375000</td>\n",
              "      <td>0.615385</td>\n",
              "      <td>...</td>\n",
              "      <td>1.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.444444</td>\n",
              "      <td>0.666667</td>\n",
              "      <td>0.701404</td>\n",
              "      <td>0.494253</td>\n",
              "      <td>0.670453</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>dev</td>\n",
              "      <td>recall</td>\n",
              "      <td>0.529981</td>\n",
              "      <td>0.741648</td>\n",
              "      <td>0.753394</td>\n",
              "      <td>0.736842</td>\n",
              "      <td>0.969512</td>\n",
              "      <td>0.102564</td>\n",
              "      <td>0.088235</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.050000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.307692</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.628751</td>\n",
              "      <td>0.318609</td>\n",
              "      <td>0.628751</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>dev</td>\n",
              "      <td>f1-score</td>\n",
              "      <td>0.578059</td>\n",
              "      <td>0.716900</td>\n",
              "      <td>0.729463</td>\n",
              "      <td>0.747573</td>\n",
              "      <td>0.954955</td>\n",
              "      <td>0.173913</td>\n",
              "      <td>0.142857</td>\n",
              "      <td>0.484848</td>\n",
              "      <td>...</td>\n",
              "      <td>0.095238</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.363636</td>\n",
              "      <td>0.400000</td>\n",
              "      <td>0.663093</td>\n",
              "      <td>0.358101</td>\n",
              "      <td>0.637441</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>dev</td>\n",
              "      <td>support</td>\n",
              "      <td>517.000000</td>\n",
              "      <td>449.000000</td>\n",
              "      <td>442.000000</td>\n",
              "      <td>209.000000</td>\n",
              "      <td>164.000000</td>\n",
              "      <td>39.000000</td>\n",
              "      <td>34.000000</td>\n",
              "      <td>20.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>20.000000</td>\n",
              "      <td>12.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>23.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>7.000000</td>\n",
              "      <td>2066.000000</td>\n",
              "      <td>2066.000000</td>\n",
              "      <td>2066.000000</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>test</td>\n",
              "      <td>precision</td>\n",
              "      <td>0.671717</td>\n",
              "      <td>0.763593</td>\n",
              "      <td>0.744578</td>\n",
              "      <td>0.700000</td>\n",
              "      <td>0.915842</td>\n",
              "      <td>0.454545</td>\n",
              "      <td>0.333333</td>\n",
              "      <td>0.842105</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.875000</td>\n",
              "      <td>0.739412</td>\n",
              "      <td>0.456980</td>\n",
              "      <td>0.703640</td>\n",
              "      <td>0.106219</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>test</td>\n",
              "      <td>recall</td>\n",
              "      <td>0.506667</td>\n",
              "      <td>0.730769</td>\n",
              "      <td>0.739234</td>\n",
              "      <td>0.753165</td>\n",
              "      <td>0.963542</td>\n",
              "      <td>0.208333</td>\n",
              "      <td>0.074074</td>\n",
              "      <td>0.592593</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.280000</td>\n",
              "      <td>0.631976</td>\n",
              "      <td>0.315600</td>\n",
              "      <td>0.631976</td>\n",
              "      <td>0.106135</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>test</td>\n",
              "      <td>f1-score</td>\n",
              "      <td>0.577633</td>\n",
              "      <td>0.746821</td>\n",
              "      <td>0.741897</td>\n",
              "      <td>0.725610</td>\n",
              "      <td>0.939086</td>\n",
              "      <td>0.285714</td>\n",
              "      <td>0.121212</td>\n",
              "      <td>0.695652</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.424242</td>\n",
              "      <td>0.681485</td>\n",
              "      <td>0.358437</td>\n",
              "      <td>0.658686</td>\n",
              "      <td>0.106163</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>test</td>\n",
              "      <td>support</td>\n",
              "      <td>525.000000</td>\n",
              "      <td>442.000000</td>\n",
              "      <td>418.000000</td>\n",
              "      <td>158.000000</td>\n",
              "      <td>192.000000</td>\n",
              "      <td>24.000000</td>\n",
              "      <td>27.000000</td>\n",
              "      <td>27.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>9.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>13.000000</td>\n",
              "      <td>20.000000</td>\n",
              "      <td>3.000000</td>\n",
              "      <td>25.000000</td>\n",
              "      <td>1989.000000</td>\n",
              "      <td>1989.000000</td>\n",
              "      <td>1989.000000</td>\n",
              "      <td>1989.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>12 rows × 26 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bacb1f8b-e29e-415b-bdbc-7505c8ee48ff')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-bacb1f8b-e29e-415b-bdbc-7505c8ee48ff button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-bacb1f8b-e29e-415b-bdbc-7505c8ee48ff');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ],
            "text/plain": [
              "    split      index        I-ADR        E-ADR        B-ADR        S-ADR  \\\n",
              "0   train  precision     0.747656     0.793743     0.777483     0.845815   \n",
              "1   train     recall     0.627332     0.781517     0.787698     0.845194   \n",
              "2   train   f1-score     0.682229     0.787583     0.782557     0.845505   \n",
              "3   train    support  4449.000000  3506.000000  3349.000000  1363.000000   \n",
              "4     dev  precision     0.635731     0.693750     0.707006     0.758621   \n",
              "5     dev     recall     0.529981     0.741648     0.753394     0.736842   \n",
              "6     dev   f1-score     0.578059     0.716900     0.729463     0.747573   \n",
              "7     dev    support   517.000000   449.000000   442.000000   209.000000   \n",
              "8    test  precision     0.671717     0.763593     0.744578     0.700000   \n",
              "9    test     recall     0.506667     0.730769     0.739234     0.753165   \n",
              "10   test   f1-score     0.577633     0.746821     0.741897     0.725610   \n",
              "11   test    support   525.000000   442.000000   418.000000   158.000000   \n",
              "\n",
              "         S-Drug   B-Finding   E-Finding      B-Drug  ...   I-Finding  \\\n",
              "0      0.986923    0.890000    0.824742    1.000000  ...    1.000000   \n",
              "1      0.997667    0.445000    0.400000    0.936364  ...    0.167785   \n",
              "2      0.992266    0.593333    0.538721    0.967136  ...    0.287356   \n",
              "3   1286.000000  200.000000  200.000000  110.000000  ...  149.000000   \n",
              "4      0.940828    0.571429    0.375000    0.615385  ...    1.000000   \n",
              "5      0.969512    0.102564    0.088235    0.400000  ...    0.050000   \n",
              "6      0.954955    0.173913    0.142857    0.484848  ...    0.095238   \n",
              "7    164.000000   39.000000   34.000000   20.000000  ...   20.000000   \n",
              "8      0.915842    0.454545    0.333333    0.842105  ...    0.000000   \n",
              "9      0.963542    0.208333    0.074074    0.592593  ...    0.000000   \n",
              "10     0.939086    0.285714    0.121212    0.695652  ...    0.000000   \n",
              "11   192.000000   24.000000   27.000000   27.000000  ...    9.000000   \n",
              "\n",
              "     B-Symptom   E-Symptom   I-Symptom  I-Disease     I-Drug     micro avg  \\\n",
              "0     0.951220    0.968750    1.000000   0.976190   1.000000      0.811251   \n",
              "1     0.354545    0.271930    0.173077   0.820000   0.820000      0.735657   \n",
              "2     0.516556    0.424658    0.295082   0.891304   0.901099      0.771607   \n",
              "3   110.000000  114.000000  104.000000  50.000000  50.000000  15722.000000   \n",
              "4     0.000000    0.000000    0.000000   0.444444   0.666667      0.701404   \n",
              "5     0.000000    0.000000    0.000000   0.307692   0.285714      0.628751   \n",
              "6     0.000000    0.000000    0.000000   0.363636   0.400000      0.663093   \n",
              "7    12.000000   13.000000   23.000000  13.000000   7.000000   2066.000000   \n",
              "8     0.000000    0.000000    0.000000   0.000000   0.875000      0.739412   \n",
              "9     0.000000    0.000000    0.000000   0.000000   0.280000      0.631976   \n",
              "10    0.000000    0.000000    0.000000   0.000000   0.424242      0.681485   \n",
              "11   13.000000   13.000000   20.000000   3.000000  25.000000   1989.000000   \n",
              "\n",
              "       macro avg  weighted avg   samples avg  \n",
              "0       0.923504      0.814371      0.122043  \n",
              "1       0.639484      0.735657      0.122001  \n",
              "2       0.721410      0.763643      0.122015  \n",
              "3   15722.000000  15722.000000  15722.000000  \n",
              "4       0.494253      0.670453           NaN  \n",
              "5       0.318609      0.628751           NaN  \n",
              "6       0.358101      0.637441           NaN  \n",
              "7    2066.000000   2066.000000           NaN  \n",
              "8       0.456980      0.703640      0.106219  \n",
              "9       0.315600      0.631976      0.106135  \n",
              "10      0.358437      0.658686      0.106163  \n",
              "11   1989.000000   1989.000000   1989.000000  \n",
              "\n",
              "[12 rows x 26 columns]"
            ]
          },
          "execution_count": 17,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "print(suf)\n",
        "\n",
        "os.mkdir('flair_ner_model'+suf+'/classification_report')\n",
        "plotter.plot_training_curves('flair_ner_model'+suf+'/loss.tsv')\n",
        "\n",
        "# evaluate the tagger on the test set\n",
        "result = tagger.evaluate(corpus.train, \n",
        "                         embeddings_storage_mode='gpu',\n",
        "                         mini_batch_size=12,\n",
        "                         main_evaluation_metric = (\"weighted avg\", \"f1-score\"),\n",
        "                         gold_label_type='ner',\n",
        "                         gold_label_dictionary=tag_dictionary)\n",
        "open('flair_ner_model'+suf+'/classification_report/train.txt', 'w').write(str(result))\n",
        "res1 = pd.DataFrame.from_dict(result.classification_report).reset_index()\n",
        "res1.insert(loc=0, column='split', value='train')\n",
        "\n",
        "result = tagger.evaluate(corpus.dev, \n",
        "                         embeddings_storage_mode='gpu',\n",
        "                         mini_batch_size=12,\n",
        "                         main_evaluation_metric = (\"weighted avg\", \"f1-score\"),\n",
        "                         gold_label_type='ner',\n",
        "                         gold_label_dictionary=tag_dictionary)\n",
        "open('flair_ner_model'+suf+'/classification_report/dev.txt', 'w').write(str(result))\n",
        "res2 = pd.DataFrame.from_dict(result.classification_report).reset_index()\n",
        "res2.insert(loc=0, column='split', value='dev')\n",
        "\n",
        "result = tagger.evaluate(corpus.test, \n",
        "                         embeddings_storage_mode='gpu',\n",
        "                         mini_batch_size=12,\n",
        "                         main_evaluation_metric = (\"weighted avg\", \"f1-score\"),\n",
        "                         gold_label_type='ner',\n",
        "                         gold_label_dictionary=tag_dictionary)\n",
        "open('flair_ner_model'+suf+'/classification_report/test.txt', 'w').write(str(result))\n",
        "res3 = pd.DataFrame.from_dict(result.classification_report).reset_index()\n",
        "res3.insert(loc=0, column='split', value='test')\n",
        "\n",
        "res = pd.concat([res1,res2,res3],ignore_index=True)\n",
        "res.to_csv('flair_ner_model'+suf+'/classification_report.csv',index=False)\n",
        "res"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "VvIDCtYJkY-G"
      },
      "outputs": [],
      "source": [
        "# import time\n",
        "# start_time = time.time()\n",
        "# while time.time() - start_time < 1:\n",
        "#     time.sleep(1)\n",
        "# print('Session has timed out (2 minutes). Disconnecting...')\n",
        "# !kill -9 -1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tGILL4RFtGil"
      },
      "source": [
        "Layering can be a good idea if the NER tags in the first layer have their own set of dependent values in the second layer. This can help capture more information about the entities being recognized and provide more context for downstream tasks. However, it also increases the complexity of the annotation and the training process, so it should be used judiciously. In general, it's a good idea to start with a simple tag scheme and gradually increase complexity as needed."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JoZ5D6aEtGin"
      },
      "source": [
        "    does it really matter if u use the E header\n",
        "\n",
        "It depends on the specific task and the requirements of the downstream model. In some cases, using the E tag can help disambiguate between overlapping entities, while in other cases it may not be necessary or even desired. Ultimately, it's up to the developer to determine the appropriate tagging scheme for their specific use case."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BGt2RhM6tGin"
      },
      "source": [
        "    would adding POS as an extra NER layer help model performance or not\n",
        "\n",
        "Adding POS as an extra NER layer may or may not help model performance, depending on the specific use case and data being analyzed.\n",
        "\n",
        "On one hand, incorporating POS tags could provide additional context to the model and help it better distinguish between named entities and other parts of speech. For example, knowing that a word is a noun or verb could help the model determine if it is likely to be a named entity or not.\n",
        "\n",
        "On the other hand, adding an additional layer could also increase model complexity and training time without necessarily improving performance. It may also require additional manual annotation efforts to create a labeled dataset with both NER and POS tags.\n",
        "\n",
        "Ultimately, whether to include POS as an additional layer will depend on the specific requirements and goals of the project, as well as the available data and resources."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6zpEn1bgtGin"
      },
      "source": [
        "    besides bioes ordering (which is up-to-down), it doesn't matter what order the tags in the same domain appear in left-to-right?\n",
        "\n",
        "Yes, that is correct. As long as the BIOES order is maintained vertically, the order of tags in the same domain from left-to-right should not impact training in flai"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yrmL98VatGin"
      },
      "source": [
        "    different typenames must be treated as separate tasks?\n",
        "\n",
        "Yes, that's correct. Different typenames should be treated as separate tasks because they represent different types of annotations that you want to predict. For example, in a biomedical NER task, you might want to predict the entity types of both drugs and diseases, which would be represented by two different typenames in your annotation data. Predicting both types of entities would require different models and possibly different evaluation metrics."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "A100",
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "flair",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.16"
    },
    "vscode": {
      "interpreter": {
        "hash": "381c8abc3392c5f20473858f182d9ac4a53ef053fb1f276fcb4936938616c3f3"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}