dcunet.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lAPS89OQCOS2"
   },
   "source": [
    "The task completed by [Abapolov Philipp](https://vk.com/pheepa) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "RUzzX1lpCOS3"
   },
   "source": [
    "![header](img/header.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9oX5DLS3COS5"
   },
   "source": [
    "# Phase-aware speech enchancement with Deep Complex U-Net #\n",
    "For research I chose this article [\"Phase-aware speech enchancement with Deep Complex U-Net\"](https://openreview.net/pdf?id=SkeRTsAcYm), which describes the architecture and training of a convolutional neural network for improving speech, so-called denoising, and set up an experiment. \n",
    "![architecture](img/dcunet.png)\n",
    "\n",
    "## The task ##\n",
    "The main task is to develop a complex variation of the architecture of the well known UNet network to eliminate unwanted noise from the audio.\n",
    "\n",
    "## Method's features ##\n",
    "Its speciality and difference from other networks, such as SegNet, for semantic segmentation (not only that) lies in Skip-Connections and that the values of both input data and all of network parameters (convolution filters, etc.) are complex.\n",
    "\n",
    "### Skip-Connections ###\n",
    "The main idea is that the early layers of the Encoder are concatenated with the \" parallel \" layers of the Decoder.\n",
    "\n",
    "![skip-connection](img/skip-connection.png)\n",
    "\n",
    "### Mask ###\n",
    "As a result of the convolution layers, we get a mask, which we multiply by the input time-frequency signal with noise and get a cleared time-frequency signal, which then passes the inverse Short-time Fourier transform.\n",
    "![arch](img/arch.png) \n",
    "\n",
    "\n",
    "### Alternative solution ###\n",
    "* [Improved Speech Enhancement with the Wave-U-Net](https://arxiv.org/abs/1811.11307)\n",
    "\n",
    "## The Experiment ##\n",
    "For training we will use [Noisy speech database for training speech enhancement algorithms and TTS models](https://datashare.is.ed.ac.uk/handle/10283/2791), which contains a data set for training and testing with 28 and 56 speakers in .wav audio files are 48 KHz. The 10-layer network architecture will be implemented, which looks like this:\n",
    "![10-layers](img/layers.png)\n",
    "\n",
    "A graph of changes in the value of the loss function during training and validation will be shown. \n",
    "\n",
    "The PESQ metric will also be calculated.\n",
    "\n",
    "### Issues ###\n",
    "Due to my lack of equipment with proper GPU (a laptop with 2 GB of GPU, so the model does not fit into the given memory, not to mention training) I had to consider alternatives for training:\n",
    "\n",
    "* Training on Google Colab or another cloud service. \n",
    "Cloud services have strict session time limits, so it was decided to train on a small number of epochs. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FiXUioRtCOS6"
   },
   "source": [
    "## Import of libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D0bWtt2J5i9F"
   },
   "outputs": [],
   "source": [
    "import time\n",
    "import pickle\n",
    "import numpy as np\n",
    "\n",
    "from tqdm import tqdm, tqdm_notebook\n",
    "from pathlib import Path\n",
    "\n",
    "import torch\n",
    "from torch.utils.data import Dataset, DataLoader\n",
    "import torch.nn as nn\n",
    "import torchaudio\n",
    "import copy\n",
    "\n",
    "from pypesq import pesq\n",
    "from matplotlib import colors, pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# not everything is smooth in sklearn, to conveniently output images in colab\n",
    "# we will ignore warnings\n",
    "import warnings\n",
    "warnings.filterwarnings(action='ignore', category=DeprecationWarning)\n",
    "\n",
    "from IPython.display import clear_output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "-xnlgTsICOTA"
   },
   "source": [
    "Checking whether the GPU is available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "byUnPtQ25i9O",
    "outputId": "91548102-17b7-42a1-896b-7f8b52f4d3ae"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training on GPU.\n"
     ]
    }
   ],
   "source": [
    "# First checking if GPU is available\n",
    "train_on_gpu=torch.cuda.is_available()\n",
    "\n",
    "if(train_on_gpu):\n",
    "    print('Training on GPU.')\n",
    "else:\n",
    "    print('No GPU available, training on CPU.')\n",
    "       \n",
    "DEVICE = torch.device('cuda' if train_on_gpu else 'cpu')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 381
    },
    "colab_type": "code",
    "id": "qacrfNwA6vw_",
    "outputId": "83374e21-bed4-448a-b7a2-5791139f5375"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sun Jun 14 15:10:26 2020       \n",
      "+-----------------------------------------------------------------------------+\n",
      "| NVIDIA-SMI 450.36.06    Driver Version: 418.67       CUDA Version: 10.1     |\n",
      "|-------------------------------+----------------------+----------------------+\n",
      "| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\n",
      "| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\n",
      "|                               |                      |               MIG M. |\n",
      "|===============================+======================+======================|\n",
      "|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |\n",
      "| N/A   40C    P0    26W / 250W |     10MiB / 16280MiB |      0%      Default |\n",
      "|                               |                      |                 ERR! |\n",
      "+-------------------------------+----------------------+----------------------+\n",
      "                                                                               \n",
      "+-----------------------------------------------------------------------------+\n",
      "| Processes:                                                                  |\n",
      "|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |\n",
      "|        ID   ID                                                   Usage      |\n",
      "|=============================================================================|\n",
      "|  No running processes found                                                 |\n",
      "+-----------------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JELiNbPtCOTH"
   },
   "source": [
    "# Loading and creating dataset objects #"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2YHdvu2FCRkA"
   },
   "source": [
    "### Loading ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DoZKpT7iCojJ"
   },
   "outputs": [],
   "source": [
    "!wget https://datashare.is.ed.ac.uk/bitstream/handle/10283/2791/clean_trainset_28spk_wav.zip?sequence=2&isAllowed=y\n",
    "!wget https://datashare.is.ed.ac.uk/bitstream/handle/10283/2791/noisy_trainset_28spk_wav.zip?sequence=6&isAllowed=y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZI0BHsnDCVIi"
   },
   "outputs": [],
   "source": [
    "!wget https://datashare.is.ed.ac.uk/bitstream/handle/10283/2791/clean_testset_wav.zip?sequence=1&isAllowed=y\n",
    "!wget https://datashare.is.ed.ac.uk/bitstream/handle/10283/2791/noisy_testset_wav.zip?sequence=5&isAllowed=y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "yxkIsMa20pxp"
   },
   "outputs": [],
   "source": [
    "!unzip clean_trainset_28spk_wav.zip?sequence=2.3\n",
    "!unzip noisy_trainset_28spk_wav.zip?sequence=6\n",
    "!unzip clean_testset_wav.zip?sequence=1\n",
    "!unzip nnoisy_testset_wav.zip?sequence=5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mDW1Vj2W05zX"
   },
   "outputs": [],
   "source": [
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IWvlIABzCOTM"
   },
   "source": [
    "The sampling frequency and the selected values for the Short-time Fourier transform."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8ngFJtPj5i9V"
   },
   "outputs": [],
   "source": [
    "SAMPLE_RATE = 48000\n",
    "N_FFT = SAMPLE_RATE * 64 // 1000 + 4\n",
    "HOP_LENGTH = SAMPLE_RATE * 16 // 1000 + 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "x8suREWkb5Se"
   },
   "source": [
    "### The declaration of datasets and dataloaders ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cZ0wb9EN5i9f"
   },
   "outputs": [],
   "source": [
    "class SpeechDataset(Dataset):\n",
    "    \"\"\"\n",
    "    A dataset class with audio that cuts them/paddes them to a specified length, applies a Short-tome Fourier transform,\n",
    "    normalizes and leads to a tensor.\n",
    "    \"\"\"\n",
    "    def __init__(self, noisy_files, clean_files, n_fft=64, hop_length=16):\n",
    "        super().__init__()\n",
    "        # list of files\n",
    "        self.noisy_files = sorted(noisy_files)\n",
    "        self.clean_files = sorted(clean_files)\n",
    "        \n",
    "        # stft parameters\n",
    "        self.n_fft = n_fft\n",
    "        self.hop_length = hop_length\n",
    "        \n",
    "        self.len_ = len(self.noisy_files)\n",
    "        \n",
    "        # fixed len\n",
    "        self.max_len = 165000\n",
    "\n",
    "    \n",
    "    def __len__(self):\n",
    "        return self.len_\n",
    "      \n",
    "    def load_sample(self, file):\n",
    "        waveform, _ = torchaudio.load(file)\n",
    "        return waveform\n",
    "  \n",
    "    def __getitem__(self, index):\n",
    "        # load to tensors and normalization\n",
    "        x_clean = self.load_sample(self.clean_files[index])\n",
    "        x_noisy = self.load_sample(self.noisy_files[index])\n",
    "        \n",
    "        # padding/cutting\n",
    "        x_clean = self._prepare_sample(x_clean)\n",
    "        x_noisy = self._prepare_sample(x_noisy)\n",
    "        \n",
    "        # Short-time Fourier transform\n",
    "        x_noisy_stft = torch.stft(input=x_noisy, n_fft=self.n_fft, \n",
    "                                  hop_length=self.hop_length, normalized=True)\n",
    "        x_clean_stft = torch.stft(input=x_clean, n_fft=self.n_fft, \n",
    "                                  hop_length=self.hop_length, normalized=True)\n",
    "        \n",
    "        return x_noisy_stft, x_clean_stft\n",
    "        \n",
    "    def _prepare_sample(self, waveform):\n",
    "        waveform = waveform.numpy()\n",
    "        current_len = waveform.shape[1]\n",
    "        \n",
    "        output = np.zeros((1, self.max_len), dtype='float32')\n",
    "        output[0, -current_len:] = waveform[0, :self.max_len]\n",
    "        output = torch.from_numpy(output)\n",
    "        \n",
    "        return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8NLV2lcv5i9k"
   },
   "outputs": [],
   "source": [
    "TRAIN_NOISY_DIR = Path('noisy_trainset_28spk_wav')\n",
    "TRAIN_CLEAN_DIR = Path('clean_trainset_28spk_wav')\n",
    "\n",
    "TEST_NOISY_DIR = Path('noisy_testset_wav')\n",
    "TEST_CLEAN_DIR = Path('clean_testset_wav')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UBp3ws355i9o"
   },
   "outputs": [],
   "source": [
    "train_noisy_files = sorted(list(TRAIN_NOISY_DIR.rglob('*.wav')))\n",
    "train_clean_files = sorted(list(TRAIN_CLEAN_DIR.rglob('*.wav')))\n",
    "\n",
    "test_noisy_files = sorted(list(TEST_NOISY_DIR.rglob('*.wav')))\n",
    "test_clean_files = sorted(list(TEST_CLEAN_DIR.rglob('*.wav')))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "7GHaKjYS5i9u"
   },
   "outputs": [],
   "source": [
    "test_dataset = SpeechDataset(test_noisy_files, test_clean_files, N_FFT, HOP_LENGTH)\n",
    "train_dataset = SpeechDataset(train_noisy_files, train_clean_files, N_FFT, HOP_LENGTH)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "79nIQjht5i9y"
   },
   "outputs": [],
   "source": [
    "test_loader = DataLoader(test_dataset, batch_size=2, shuffle=True)\n",
    "train_loader = DataLoader(train_dataset, batch_size=2, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "k73YEkgQCOTj"
   },
   "source": [
    "# Declaring the class layers #"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Znx7QM3h5i92"
   },
   "outputs": [],
   "source": [
    "class CConv2d(nn.Module):\n",
    "    \"\"\"\n",
    "    Class of complex valued convolutional layer\n",
    "    \"\"\"\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, stride, padding=0):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.in_channels = in_channels\n",
    "        self.out_channels = out_channels\n",
    "        self.kernel_size = kernel_size\n",
    "        self.padding = padding\n",
    "        self.stride = stride\n",
    "        \n",
    "        self.real_conv = nn.Conv2d(in_channels=self.in_channels, \n",
    "                                   out_channels=self.out_channels, \n",
    "                                   kernel_size=self.kernel_size, \n",
    "                                   padding=self.padding, \n",
    "                                   stride=self.stride)\n",
    "        \n",
    "        self.im_conv = nn.Conv2d(in_channels=self.in_channels, \n",
    "                                 out_channels=self.out_channels, \n",
    "                                 kernel_size=self.kernel_size, \n",
    "                                 padding=self.padding, \n",
    "                                 stride=self.stride)\n",
    "        \n",
    "        # Glorot initialization.\n",
    "        nn.init.xavier_uniform_(self.real_conv.weight)\n",
    "        nn.init.xavier_uniform_(self.im_conv.weight)\n",
    "        \n",
    "        \n",
    "    def forward(self, x):\n",
    "        x_real = x[..., 0]\n",
    "        x_im = x[..., 1]\n",
    "        \n",
    "        c_real = self.real_conv(x_real) - self.im_conv(x_im)\n",
    "        c_im = self.im_conv(x_real) + self.real_conv(x_im)\n",
    "        \n",
    "        output = torch.stack([c_real, c_im], dim=-1)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GgtxJbSQ5i96"
   },
   "outputs": [],
   "source": [
    "class CConvTranspose2d(nn.Module):\n",
    "    \"\"\"\n",
    "      Class of complex valued dilation convolutional layer\n",
    "    \"\"\"\n",
    "    def __init__(self, in_channels, out_channels, kernel_size, stride, output_padding=0, padding=0):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.in_channels = in_channels\n",
    "\n",
    "        self.out_channels = out_channels\n",
    "        self.kernel_size = kernel_size\n",
    "        self.output_padding = output_padding\n",
    "        self.padding = padding\n",
    "        self.stride = stride\n",
    "        \n",
    "        self.real_convt = nn.ConvTranspose2d(in_channels=self.in_channels, \n",
    "                                            out_channels=self.out_channels, \n",
    "                                            kernel_size=self.kernel_size, \n",
    "                                            output_padding=self.output_padding,\n",
    "                                            padding=self.padding,\n",
    "                                            stride=self.stride)\n",
    "        \n",
    "        self.im_convt = nn.ConvTranspose2d(in_channels=self.in_channels, \n",
    "                                            out_channels=self.out_channels, \n",
    "                                            kernel_size=self.kernel_size, \n",
    "                                            output_padding=self.output_padding, \n",
    "                                            padding=self.padding,\n",
    "                                            stride=self.stride)\n",
    "        \n",
    "        \n",
    "        # Glorot initialization.\n",
    "        nn.init.xavier_uniform_(self.real_convt.weight)\n",
    "        nn.init.xavier_uniform_(self.im_convt.weight)\n",
    "        \n",
    "        \n",
    "    def forward(self, x):\n",
    "        x_real = x[..., 0]\n",
    "        x_im = x[..., 1]\n",
    "        \n",
    "        ct_real = self.real_convt(x_real) - self.im_convt(x_im)\n",
    "        ct_im = self.im_convt(x_real) + self.real_convt(x_im)\n",
    "        \n",
    "        output = torch.stack([ct_real, ct_im], dim=-1)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "OJSmVrxp5i9-"
   },
   "outputs": [],
   "source": [
    "class CBatchNorm2d(nn.Module):\n",
    "    \"\"\"\n",
    "    Class of complex valued batch normalization layer\n",
    "    \"\"\"\n",
    "    def __init__(self, num_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.num_features = num_features\n",
    "        self.eps = eps\n",
    "        self.momentum = momentum\n",
    "        self.affine = affine\n",
    "        self.track_running_stats = track_running_stats\n",
    "        \n",
    "        self.real_b = nn.BatchNorm2d(num_features=self.num_features, eps=self.eps, momentum=self.momentum,\n",
    "                                      affine=self.affine, track_running_stats=self.track_running_stats)\n",
    "        self.im_b = nn.BatchNorm2d(num_features=self.num_features, eps=self.eps, momentum=self.momentum,\n",
    "                                    affine=self.affine, track_running_stats=self.track_running_stats) \n",
    "        \n",
    "    def forward(self, x):\n",
    "        x_real = x[..., 0]\n",
    "        x_im = x[..., 1]\n",
    "        \n",
    "        n_real = self.real_b(x_real)\n",
    "        n_im = self.im_b(x_im)  \n",
    "        \n",
    "        output = torch.stack([n_real, n_im], dim=-1)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "N7W37XMO5i-B"
   },
   "outputs": [],
   "source": [
    "class Encoder(nn.Module):\n",
    "    \"\"\"\n",
    "    Class of upsample block\n",
    "    \"\"\"\n",
    "    def __init__(self, filter_size=(7,5), stride_size=(2,2), in_channels=1, out_channels=45, padding=(0,0)):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.filter_size = filter_size\n",
    "        self.stride_size = stride_size\n",
    "        self.in_channels = in_channels\n",
    "        self.out_channels = out_channels\n",
    "        self.padding = padding\n",
    "\n",
    "        self.cconv = CConv2d(in_channels=self.in_channels, out_channels=self.out_channels, \n",
    "                             kernel_size=self.filter_size, stride=self.stride_size, padding=self.padding)\n",
    "        \n",
    "        self.cbn = CBatchNorm2d(num_features=self.out_channels) \n",
    "        \n",
    "        self.leaky_relu = nn.LeakyReLU()\n",
    "            \n",
    "    def forward(self, x):\n",
    "        \n",
    "        conved = self.cconv(x)\n",
    "        normed = self.cbn(conved)\n",
    "        acted = self.leaky_relu(normed)\n",
    "        \n",
    "        return acted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fuugYDZs5i-G"
   },
   "outputs": [],
   "source": [
    "class Decoder(nn.Module):\n",
    "    \"\"\"\n",
    "    Class of downsample block\n",
    "    \"\"\"\n",
    "    def __init__(self, filter_size=(7,5), stride_size=(2,2), in_channels=1, out_channels=45,\n",
    "                 output_padding=(0,0), padding=(0,0), last_layer=False):\n",
    "        super().__init__()\n",
    "        \n",
    "        self.filter_size = filter_size\n",
    "        self.stride_size = stride_size\n",
    "        self.in_channels = in_channels\n",
    "        self.out_channels = out_channels\n",
    "        self.output_padding = output_padding\n",
    "        self.padding = padding\n",
    "        \n",
    "        self.last_layer = last_layer\n",
    "        \n",
    "        self.cconvt = CConvTranspose2d(in_channels=self.in_channels, out_channels=self.out_channels, \n",
    "                             kernel_size=self.filter_size, stride=self.stride_size, output_padding=self.output_padding, padding=self.padding)\n",
    "        \n",
    "        self.cbn = CBatchNorm2d(num_features=self.out_channels) \n",
    "        \n",
    "        self.leaky_relu = nn.LeakyReLU()\n",
    "            \n",
    "    def forward(self, x):\n",
    "        \n",
    "        conved = self.cconvt(x)\n",
    "        \n",
    "        if not self.last_layer:\n",
    "            normed = self.cbn(conved)\n",
    "            output = self.leaky_relu(normed)\n",
    "        else:\n",
    "            m_phase = conved / (torch.abs(conved) + 1e-8)\n",
    "            m_mag = torch.tanh(torch.abs(conved))\n",
    "            output = m_phase * m_mag\n",
    "            \n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "HO3p2zrOcn_z"
   },
   "source": [
    "10-layer model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "j8XCrVIg5i-K"
   },
   "outputs": [],
   "source": [
    "class DCUnet10(nn.Module):\n",
    "    \"\"\"\n",
    "    Deep Complex U-Net class of the model.\n",
    "    \"\"\"\n",
    "    def __init__(self, n_fft=64, hop_length=16):\n",
    "        super().__init__()\n",
    "        \n",
    "        # for istft\n",
    "        self.n_fft = n_fft\n",
    "        self.hop_length = hop_length\n",
    "        \n",
    "        # downsampling/encoding\n",
    "        self.downsample0 = Encoder(filter_size=(7,5), stride_size=(2,2), in_channels=1, out_channels=45)\n",
    "        self.downsample1 = Encoder(filter_size=(7,5), stride_size=(2,2), in_channels=45, out_channels=90)\n",
    "        self.downsample2 = Encoder(filter_size=(5,3), stride_size=(2,2), in_channels=90, out_channels=90)\n",
    "        self.downsample3 = Encoder(filter_size=(5,3), stride_size=(2,2), in_channels=90, out_channels=90)\n",
    "        self.downsample4 = Encoder(filter_size=(5,3), stride_size=(2,1), in_channels=90, out_channels=90)\n",
    "        \n",
    "        # upsampling/decoding\n",
    "        self.upsample0 = Decoder(filter_size=(5,3), stride_size=(2,1), in_channels=90, out_channels=90)\n",
    "        self.upsample1 = Decoder(filter_size=(5,3), stride_size=(2,2), in_channels=180, out_channels=90)\n",
    "        self.upsample2 = Decoder(filter_size=(5,3), stride_size=(2,2), in_channels=180, out_channels=90)\n",
    "        self.upsample3 = Decoder(filter_size=(7,5), stride_size=(2,2), in_channels=180, out_channels=45)\n",
    "        self.upsample4 = Decoder(filter_size=(7,5), stride_size=(2,2), in_channels=90, output_padding=(0,1),\n",
    "                                 out_channels=1, last_layer=True)\n",
    "        \n",
    "        \n",
    "    def forward(self, x, is_istft=True):\n",
    "        # downsampling/encoding\n",
    "        d0 = self.downsample0(x)\n",
    "        d1 = self.downsample1(d0) \n",
    "        d2 = self.downsample2(d1)        \n",
    "        d3 = self.downsample3(d2)        \n",
    "        d4 = self.downsample4(d3)\n",
    "        \n",
    "        # upsampling/decoding \n",
    "        u0 = self.upsample0(d4)\n",
    "        # skip-connection\n",
    "        c0 = torch.cat((u0, d3), dim=1)\n",
    "        \n",
    "        u1 = self.upsample1(c0)\n",
    "        c1 = torch.cat((u1, d2), dim=1)\n",
    "        \n",
    "        u2 = self.upsample2(c1)\n",
    "        c2 = torch.cat((u2, d1), dim=1)\n",
    "        \n",
    "        u3 = self.upsample3(c2)\n",
    "        c3 = torch.cat((u3, d0), dim=1)\n",
    "        \n",
    "        u4 = self.upsample4(c3)\n",
    "        \n",
    "        # u4 - the mask\n",
    "        output = u4 * x\n",
    "        if is_istft:\n",
    "            output = torch.squeeze(output, 1)\n",
    "            output = torch.istft(output, n_fft=self.n_fft, hop_length=self.hop_length, normalized=True)\n",
    "        \n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3G5CqR3-COT8"
   },
   "source": [
    "## Loss function ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "J71ny6expQeW"
   },
   "outputs": [],
   "source": [
    "def wsdr_fn(x_, y_pred_, y_true_, eps=1e-8):\n",
    "    # to time-domain waveform\n",
    "    y_true_ = torch.squeeze(y_true_, 1)\n",
    "    x_ = torch.squeeze(x_, 1)\n",
    "    y_true = torch.istft(y_true_, n_fft=N_FFT, hop_length=HOP_LENGTH, normalized=True)\n",
    "    x = torch.istft(x_, n_fft=N_FFT, hop_length=HOP_LENGTH, normalized=True)\n",
    "\n",
    "    y_pred = y_pred_.flatten(1)\n",
    "    y_true = y_true.flatten(1)\n",
    "    x = x.flatten(1)\n",
    "\n",
    "\n",
    "    def sdr_fn(true, pred, eps=1e-8):\n",
    "        num = torch.sum(true * pred, dim=1)\n",
    "        den = torch.norm(true, p=2, dim=1) * torch.norm(pred, p=2, dim=1)\n",
    "        return -(num / (den + eps))\n",
    "\n",
    "    # true and estimated noise\n",
    "    z_true = x - y_true\n",
    "    z_pred = x - y_pred\n",
    "\n",
    "    a = torch.sum(y_true**2, dim=1) / (torch.sum(y_true**2, dim=1) + torch.sum(z_true**2, dim=1) + eps)\n",
    "    wSDR = a * sdr_fn(y_true, y_pred) + (1 - a) * sdr_fn(z_true, z_pred)\n",
    "    return torch.mean(wSDR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "kYvWz_6jRZ3e"
   },
   "source": [
    "Description of the training of epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VukJTCGIZ8ZU"
   },
   "outputs": [],
   "source": [
    "def train_epoch(net, train_loader, loss_fn, optimizer):\n",
    "    net.train()\n",
    "    train_ep_loss = 0.\n",
    "    counter = 0\n",
    "    for noisy_x, clean_x in train_loader:\n",
    "\n",
    "        noisy_x, clean_x = noisy_x.to(DEVICE), clean_x.to(DEVICE)\n",
    "\n",
    "        # zero  gradients\n",
    "        net.zero_grad()\n",
    "\n",
    "        # get the output from the model\n",
    "        pred_x = net(noisy_x)\n",
    "\n",
    "        # calculate loss\n",
    "        loss = loss_fn(noisy_x, pred_x, clean_x)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        train_ep_loss += loss.item() \n",
    "        counter += 1\n",
    "\n",
    "    train_ep_loss /= counter\n",
    "\n",
    "    # clear cache\n",
    "    gc.collect()\n",
    "    torch.cuda.empty_cache()\n",
    "    return train_ep_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zYCJWaTMRgS3"
   },
   "source": [
    "Description of the validation of epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-JKrTMpPhw19"
   },
   "outputs": [],
   "source": [
    "def test_epoch(net, test_loader, loss_fn):\n",
    "    net.eval()\n",
    "    test_ep_loss = 0.\n",
    "    counter = 0.\n",
    "    for noisy_x, clean_x in test_loader:\n",
    "        # get the output from the model\n",
    "        noisy_x, clean_x = noisy_x.to(DEVICE), clean_x.to(DEVICE)\n",
    "        pred_x = net(noisy_x)\n",
    "\n",
    "        # calculate loss\n",
    "        loss = loss_fn(noisy_x, pred_x, clean_x)\n",
    "        test_ep_loss += loss.item() \n",
    "        \n",
    "        counter += 1\n",
    "\n",
    "    test_ep_loss /= counter\n",
    "\n",
    "    # clear cache\n",
    "    gc.collect()\n",
    "    torch.cuda.empty_cache()\n",
    "    \n",
    "    return test_ep_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "879vq_uBRm_2"
   },
   "source": [
    "To understand whether the network is being trained or not, we will output a train and test loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "I4gdVmhRr1Qi"
   },
   "outputs": [],
   "source": [
    "def train(net, train_loader, test_loader, loss_fn, optimizer, scheduler, epochs):\n",
    "    \n",
    "    train_losses = []\n",
    "    test_losses = []\n",
    "\n",
    "    for e in tqdm(range(epochs)):\n",
    "\n",
    "        # first evaluating for comparison\n",
    "        if e == 0:\n",
    "            with torch.no_grad():\n",
    "                test_loss = test_epoch(net, test_loader, loss_fn)\n",
    "                \n",
    "            test_losses.append(test_loss)\n",
    "            print(\"Loss before training:{:.6f}\".format(test_loss))\n",
    "          \n",
    "\n",
    "        train_loss = train_epoch(net, train_loader, loss_fn, optimizer)\n",
    "        scheduler.step()\n",
    "        with torch.no_grad():\n",
    "          test_loss = test_epoch(net, test_loader, loss_fn)\n",
    "\n",
    "        train_losses.append(train_loss)\n",
    "        test_losses.append(test_loss)\n",
    "\n",
    "        # clear cache\n",
    "        torch.cuda.empty_cache()\n",
    "        gc.collect()\n",
    "\n",
    "        print(\"Epoch: {}/{}...\".format(e+1, epochs),\n",
    "                      \"Loss: {:.6f}...\".format(train_loss),\n",
    "                      \"Test Loss: {:.6f}\".format(test_loss))\n",
    "    return train_losses, test_losses"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tHJZXeUQcsrq"
   },
   "source": [
    "## Metrics ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_gDZAOnGdeB0"
   },
   "outputs": [],
   "source": [
    "def pesq_score(net, test_loader):\n",
    "    net.eval()\n",
    "    test_pesq = 0.\n",
    "    counter = 0.\n",
    "\n",
    "\n",
    "    for noisy_x, clean_x in test_loader:\n",
    "        # get the output from the model\n",
    "        noisy_x = noisy_x.to(DEVICE)\n",
    "        with torch.no_grad():\n",
    "            pred_x = net(noisy_x)\n",
    "        clean_x = torch.squeeze(clean_x, 1)\n",
    "        clean_x = torch.istft(clean_x, n_fft=N_FFT, hop_length=HOP_LENGTH, normalized=True)\n",
    "        \n",
    "        \n",
    "        psq = 0.\n",
    "        for i in range(len(clean_x)):\n",
    "            clean_x_16 = torchaudio.transforms.Resample(48000, 16000)(clean_x[i, 0,:].view(1,-1))\n",
    "            pred_x_16 = torchaudio.transforms.Resample(48000, 16000)(pred_x[i, 0,:].view(1,-1))\n",
    "\n",
    "            clean_x_16 = clean_x_16.cpu().cpu().numpy()\n",
    "            pred_x_16 = pred_x_16.detach().cpu().numpy()\n",
    "            \n",
    "            \n",
    "            \n",
    "            psq += pesq(clean_x_16.flatten(), pred_x_16.flatten(), 16000)\n",
    "            \n",
    "        psq /= len(clean_x)\n",
    "        test_pesq += psq\n",
    "        counter += 1\n",
    "\n",
    "    test_pesq /= counter \n",
    "    return test_pesq"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "G6bIE9iOj8pq"
   },
   "source": [
    "## Training ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QyBc1awQkI-D"
   },
   "outputs": [],
   "source": [
    "# clear cache\n",
    "import gc\n",
    "gc.collect()\n",
    "torch.cuda.empty_cache()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "DIhn5cn85i-O"
   },
   "outputs": [],
   "source": [
    "dcunet10 = DCUnet10(N_FFT, HOP_LENGTH).to(DEVICE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PR7dL8sJ5i-h"
   },
   "outputs": [],
   "source": [
    "loss_fn = wsdr_fn\n",
    "optimizer = torch.optim.Adam(dcunet10.parameters(), lr=1e-4)\n",
    "scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=1, gamma=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 472
    },
    "colab_type": "code",
    "id": "ppXkJUsY55vI",
    "outputId": "260b6225-8621-4927-9702-093155e0c983"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "  0%|          | 0/3 [00:00<?, ?it/s]\u001b[A\u001b[A\u001b[A\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss before training:0.517480\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " 33%|███▎      | 1/3 [22:23<44:47, 1343.54s/it]\u001b[A\u001b[A\u001b[A\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1/3... Loss: -0.923392... Test Loss: -0.976580\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      " 67%|██████▋   | 2/3 [44:25<22:17, 1337.15s/it]\u001b[A\u001b[A\u001b[A\u001b[A\u001b[A"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 2/3... Loss: -0.944345... Test Loss: -0.976268\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "100%|██████████| 3/3 [1:06:28<00:00, 1329.46s/it]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 3/3... Loss: -0.946829... Test Loss: -0.976848\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "train_losses, test_losses = train(dcunet10, train_loader, test_loader, loss_fn, optimizer, scheduler, 3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.save(dcunet10.state_dict(), '__weights.pth')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5R43OUHZYbKm"
   },
   "source": [
    "## PESQ метрика ##\n",
    "Let's calculate the average value of the metric in the test sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bu3CHNxdYoKR"
   },
   "outputs": [],
   "source": [
    "pesq_metric = pesq_score(dcunet10, test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "j6rD0yrSbx1i"
   },
   "source": [
    "# The Results #"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "mw7_CKy2WeI_"
   },
   "source": [
    "## Loss ##\n",
    "Let's look at the graph of loss reduction for 3 epochs on the test, the loss for the first epoch decreases very quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 388
    },
    "colab_type": "code",
    "id": "wCf59OBzoUon",
    "outputId": "e66330cc-a9b9-42fb-b287-b0258f990cc9"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfoAAAFzCAYAAADWqstZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXzV9b3v+9cnM5AwQxgCJEFsxQkhIgjEsNWjdpC6a9Vap1qM+97T3u67b3trb4dzT88+97r3vqd7n3PavWtEW6xt0dptpdVup00ELMhUHFGBMIUZZEgCCRk+948sCyssYAEr67uG9/PxyIM1fPNb73xc8s4a+C5zd0RERCQz5YQOICIiIr1HRS8iIpLBVPQiIiIZTEUvIiKSwVT0IiIiGUxFLyIiksHyQgdItKFDh3p5eXlCj9nS0kK/fv0Sesx0pVlE0zyiaR7HaRbRNI9oiZ7H6tWr97n7sFjXZVzRl5eXs2rVqoQes76+npqamoQeM11pFtE0j2iax3GaRTTNI1qi52FmW051nZ66FxERyWAqehERkQymohcREclgGfcavYiISHt7O42NjbS2toaOEtOAAQNYt27dWX9fUVERZWVl5Ofnx/09KnoREck4jY2NlJSUUF5ejpmFjnOSpqYmSkpKzup73J39+/fT2NhIRUVF3N+np+5FRCTjtLa2MmTIkJQs+XNlZgwZMuSsn6VQ0YuISEbKpJL/2Ln8TCp6ERGRBDt48CD//M//fE7f+0//9E8cOXIkYVlU9CIiIgmWSkWvN+OJiIgk2EMPPcTGjRuZNGkS119/PcOHD+fpp5+mra2NW265hW984xu0tLRw22230djYSGdnJ9/73vfYvXs3O3bsYPbs2QwdOpRFixaddxYVvYiIZLT//Lt3eW/H4YQec+Ko/vynz158yusffvhh3nnnHdauXctLL73EM888w4oVK3B3br75Zl5//XVaWloYNWoUzz//PACHDh1iwIAB/PCHP2TRokUMHTo0IVn11P1puDvLNu5n39Gu0FFERCRNvfTSS7z00ktcccUVTJ48mffff5+NGzdy6aWX8vLLL/Otb32LJUuWMGDAgF65fT2iP439Lce45/E3mDU6l1tDhxERkXNyukfeyeDufPvb3+bBBx/882Uf/zv6NWvW8MILL/Dd736Xa6+9lu9///sJv309oj+NocWFfG7SaJY2dvBRy7HQcUREJE2UlJTQ1NQEwA033MDjjz9Oc3MzANu3b2fv3r3s2LGDvn37ctddd/HNb36TNWvWnPS9iaBH9GdQW13Jr1c38sSyzfz1dReGjiMiImlgyJAhzJgxg0suuYSbbrqJO++8k+nTpwNQXFzMT37yE9avX883v/lNcnJyyM/P51/+5V8AqK2t5cYbb2TUqFF6M14yTCgt4fJhuTyxbAt/dc14ivJzQ0cSEZE08Mtf/jLq/Ne//vU/n25qauLyyy/nhhtuOOn7vva1r/G1r30tYTn01H0cbqrI56OWYzyzujF0FBERkbOioo/DJwblcHnZAOYtaaCzy0PHERERiZuKPg5mxgPVlWzef4SX39sVOo6IiEjcVPRxuvHiEYwZ3IdHFjfgrkf1IiKpLhP/rj6Xn0lFH6e83BzmzqzkT1sPsnrLgdBxRETkNIqKiti/f39Glf3Hn0dfVFR0Vt+nd92fhS9UlfGPr3zII4sbqCofHDqOiIicQllZGY2Njezduzd0lJhaW1vPurCh+xeYsrKys/oeFf1Z6FuQxz3TxvE/F21g495mxg8rDh1JRERiyM/Pp6KiInSMU6qvr+eKK65Iym3pqfuzdPf0cvJzc5i3pCF0FBERkTNS0Z+lYSWFfH5yGb9Zs529TW2h44iIiJyWiv4cPDCrgvbOLp5Ytjl0FBERkdNS0Z+DymHFXH9RKT9fvoUjxzpCxxERETklFf05evCaSg4eaefpldtCRxERETklFf05mjJuMFPGDWLe0k10dHaFjiMiIhKTiv48PDCrksYDR/nDO9oWV0REUpOK/jxcP7GUiqH9qNO2uCIikqJU9OchN8eYO6uCt7cfYnnDR6HjiIiInCRo0ZvZjWb2gZltMLOHTrPu82bmZlaVzHzx+PzkMob0K6Bu8cbQUURERE4SrOjNLBf4MXATMBH4oplNjLGuBPg68EZyE8anKD+Xe68uZ9EHe/lwd1PoOCIiIlFCPqKfCmxw9wZ3PwYsAObEWPdfgL8DWpMZ7mzcNW0cRfk51C3WtrgiIpJaQhb9aODEf4TeGLnsz8xsMjDG3Z9PZrCzNbhfAbdVjeG5tdvZdShlfx8REZEslLKfXmdmOcAPgfviWFsL1AKUlpZSX1+f0CzNzc1nPOYl+V10dDo/WLCY2z5RkNDbTyXxzCKbaB7RNI/jNItomke0ZM4jZNFvB8accL4sctnHSoBLgHozAxgBLDSzm9191YkHcvc6oA6gqqrKa2pqEhq0vr6eeI752oE1LF6/l7+/bybFhSn7O9R5iXcW2ULziKZ5HKdZRNM8oiVzHiGful8JTDCzCjMrAO4AFn58pbsfcveh7l7u7uXAcuCkkk8ltdWVNLV2sGDF1tBRREREgIBF7+4dwFeBF4F1wNPu/q6Z/cDMbg6V63xcPmYgV1UM5vGlm2jXtrgiIpICgv47end/wd0vdPfx7v5fI5d9390Xxlhbk8qP5j9WW13JjkOt/P6tHaGjiIiIaGe8RJv9ieFcMLyYusWbtC2uiIgEp6JPsJwco3ZWJet2Hmbphn2h44iISJZT0feCOVeMYnhJoTbQERGR4FT0vaAwL5f7ZpSzZP0+3t1xKHQcERHJYir6XvKlqePoW5DLo3pULyIiAanoe8mAvvncceVYfvfWTrYfPBo6joiIZCkVfS+6f2Y5AD9duilsEBERyVoq+l5UNqgvn7lsJL9asZVDR9tDxxERkSykou9lD8yqpOVYJ798Q9viiohI8qnoe9klowcw44Ih/PT1TbR1dIaOIyIiWUZFnwS11ePZ09TGc2u1La6IiCSXij4JqicM5ZMjSnh0cYO2xRURkaRS0SeBmVFbXcn6Pc3Uf7A3dBwREckiKvok+ezloxg5oIhHFm8MHUVERLKIij5J8nNzuH9GBcsbPuKtxoOh44iISJZQ0SfRHVPHUFKYxyPaFldERJJERZ9EJUX53HnVWP7w9k62fXQkdBwREckCKvok+/KMCnJzjMe0La6IiCSBij7JRgwo4ubLR/PUym0caDkWOo6IiGQ4FX0AtdWVHG3v5MnlW0JHERGRDKeiD+ATI0q45sJhzF+2mdZ2bYsrIiK9R0UfyIPVlexrPsa/rtkeOoqIiGQwFX0g08cP4ZLR/Zm3pIGuLm2LKyIivUNFH0j3trjjadjXwivrdoeOIyIiGUpFH9CnLhnB6IF9qNMGOiIi0ktU9AHl5ebwlZkVrNpygNVbDoSOIyIiGUhFH9jtV45hQJ986vRhNyIi0gtU9IH1K8zjrmljeem93Wza1xI6joiIZBgVfQq49+py8nNymLdEr9WLiEhiqehTwPCSIv5y8mieWd3Ivua20HFERCSDqOhTxNxZlbR1dPHEMm2LKyIiiRO06M3sRjP7wMw2mNlDMa7/GzN7z8zeMrNXzWxciJzJcMHwYq67aDg/X7aZo8e0La6IiCRGsKI3s1zgx8BNwETgi2Y2sceyPwFV7n4Z8Azw98lNmVy11eM5cKSdZ1ZvCx1FREQyRMhH9FOBDe7e4O7HgAXAnBMXuPsidz8SObscKEtyxqS6snwQk8YMZN7STXRqW1wREUkAcw9TKGZ2K3Cju8+NnL8buMrdv3qK9T8Cdrn738a4rhaoBSgtLZ2yYMGChGZtbm6muLg4occ8lZW7Ovjx2jb+46RCrhyRl5TbPBvJnEU60DyiaR7HaRbRNI9oiZ7H7NmzV7t7VazrUq9JYjCzu4Aq4JpY17t7HVAHUFVV5TU1NQm9/fr6ehJ9zFOZ1eX8fls9S/cX8I3br8bMknK78UrmLNKB5hFN8zhOs4imeURL5jxCPnW/HRhzwvmyyGVRzOw64DvAze6e8f/2LDfHmDuzgje3HWTFpo9CxxERkTQXsuhXAhPMrMLMCoA7gIUnLjCzK4BH6C75PQEyBnHrlDEM7lfAo9pAR0REzlOwonf3DuCrwIvAOuBpd3/XzH5gZjdHlv0DUAz82szWmtnCUxwuo/QpyOXuaeN4Zd0eNuxpCh1HRETSWNB/R+/uL7j7he4+3t3/a+Sy77v7wsjp69y91N0nRb5uPv0RM8c908dRmJfDo4s3hY4iIiJpTDvjpaghxYV8oaqMZ/+0nT2HW0PHERGRNKWiT2FfmVlJe1cXP/vj5tBRREQkTanoU1jF0H7cMHEETy7fQnNbR+g4IiKShlT0Ka72mkoOt3bw9EptiysiImdPRZ/iJo8dxJXlg3hs6SY6OrtCxxERkTSjok8DtdXj2X7wKM+/vTN0FBERSTMq+jRw7SeHUzmsH3WLGwj12QQiIpKeVPRpICfHeGBWJe/uOMwfN+4PHUdERNKIij5N3HLFaIYWF1K3WNviiohI/FT0aaIoP5f7rh7Hax/u5f1dh0PHERGRNKGiTyN3TRtH34JcPaoXEZG4qejTyMC+BdxWNYaFa3ew89DR0HFERCQNqOjTzFdmVtDlzk9f3xw6ioiIpAEVfZoZM7gvn7p0JL98YyuHW9tDxxERkRSnok9DD1aPp7mtgwUrtoaOIiIiKU5Fn4YuLRvA9MohPL50M8c6tC2uiIicmoo+TdVeU8muw6387s0doaOIiEgKU9GnqZoLh3FhaTGPLtG2uCIicmoq+jRl1r0t7vu7mnjtw72h44iISIpS0aexOZNGU9q/kEeXaAMdERGJTUWfxgrycvjyjApe37Cfd7YfCh1HRERSkIo+zd151ViKC/O0La6IiMSkok9z/Yvy+eLUMTz/9k4aDxwJHUdERFKMij4DfHlGBQY8tnRT6CgiIpJiVPQZYNTAPnz28lE8tXIbh45oW1wRETlORZ8hHphVyZFjnTz5xpbQUUREJIWo6DPExFH9mTVhKD/742baOjpDxxERkRShos8gD1aPZ29TG7/90/bQUUREJEWo6DPIjAuGMHFkf+oWN9DVpW1xRURERZ9RzIza6ko27m1h0Qd7QscREZEUoKLPMJ++bCSjBhTxiDbQERERAhe9md1oZh+Y2QYzeyjG9YVm9lTk+jfMrDz5KdNLfm4O98+sYMWmj1i77WDoOCIiEliwojezXODHwE3AROCLZjaxx7KvAAfc/QLgH4G/S27K9HTH1LGUFOVRt3hj6CgiIhJYyEf0U4EN7t7g7seABcCcHmvmAPMjp58BrjUzS2LGtFRcmMdd08bxb+/sYsv+ltBxREQkIHMP8+5sM7sVuNHd50bO3w1c5e5fPWHNO5E1jZHzGyNr9vU4Vi1QC1BaWjplwYIFCc3a3NxMcXFxQo/Z2w60dvGN145yzZg87plYmLDjpuMsepPmEU3zOE6ziKZ5REv0PGbPnr3a3atiXZeXsFsJyN3rgDqAqqoqr6mpSejx6+vrSfQxk+GPTW/yu7d28P/dezWD+xUk5JjpOoveonlE0zyO0yyiaR7RkjmPkE/dbwfGnHC+LHJZzDVmlgcMAPYnJV0GqK2upLW9i58v07a4IiLZKmTRrwQmmFmFmRUAdwALe6xZCNwbOX0r8O8e6rWGNDShtIS/+ORwnli2mdZ2bYsrIpKNghW9u3cAXwVeBNYBT7v7u2b2AzO7ObLsMWCImW0A/gY46Z/gyenVVleyv+UYz6xuDB1FREQCCPoavbu/ALzQ47Lvn3C6FfhCsnNlkqsqBnNZ2QDmLWngi1PHkpujf7QgIpJNtDNehvt4W9zN+4/w8nu7Q8cREZEkU9FngRsvHsGYwX20gY6ISBZS0WeBvNwc5s6sZM3Wg6za/FHoOCIikkQq+izxhaoyBvbN14fdiIhkGRV9luhbkMfd08bxyrrdbNzbHDqOiIgkiYo+i9wzvZz83BzmLdkUOoqIiCSJij6LDCsp5POTy/jNmkb2NrWFjiMiIkmgos8yD8yqoL2ziyeWbQ4dRUREkkBFn2UqhxVz/UWl/Hz5Fo4c6wgdR0REepmKPgs9eE0lB4+08/TKbaGjiIhIL1PRZ6Ep4wYzeexA5i3dREdnV+g4IiLSi1T0Waq2ejyNB47yb+/uCh1FRER6kYo+S10/sZSKof2oW9yAPvlXRCRzqeizVG6OMXdWBW81HmJ5g7bFFRHJVCr6LPb5yWUM6VegD7sREclgKvosVpSfyz3Ty1n0wV4+3N0UOo6IiPQCFX2Wu3v6OIryc3hUH3YjIpKRVPRZbnC/Am6rGsNv125n9+HW0HFERCTBVPTC3JmVdHY5P319c+goIiKSYCp6YeyQvtx0yUh+8cYWmtu0La6ISCZR0QsAD1RX0tTawYIVW0NHERGRBFLRCwCTxgxkasVgHl+6iXZtiysikjFU9PJnD1ZXsuNQK8+/tTN0FBERSRAVvfzZ7E8M54LhxTyibXFFRDKGil7+LCfHqJ1Vybqdh1m6YV/oOCIikgAqeoky54pRDCsppE4b6IiIZAQVvUQpzMvlvqvLWbJ+H+/uOBQ6joiInCcVvZzkrqvG0bcgl3lLNoWOIiIi50lFLycZ0DefO64cy+/e3MGOg0dDxxERkfOgopeY7p9ZjgOPL9WjehGRdBak6M1ssJm9bGbrI38OirFmkpktM7N3zewtM7s9RNZsVTaoL5+5bCS/WrGVQ0fbQ8cREZFzFOoR/UPAq+4+AXg1cr6nI8A97n4xcCPwT2Y2MIkZs94DsyppOdbJL9/QtrgiIukqVNHPAeZHTs8HPtdzgbt/6O7rI6d3AHuAYUlLKFwyegAzLhjCT1/fxLEObYsrIpKO4ip6M/u6mfW3bo+Z2Roz+w/ncbul7v7xPqu7gNIz3P5UoADYeB63Keegtno8e5raeG7t9tBRRETkHFg8W52a2ZvufrmZ3QA8CHwP+Lm7Tz7N97wCjIhx1XeA+e4+8IS1B9z9pNfpI9eNBOqBe919+SnW1AK1AKWlpVMWLFhwxp/pbDQ3N1NcXJzQY6YLd+f7f2yly52/ndGHlpaWrJ1FLNl834hF8zhOs4imeURL9Dxmz5692t2rYl2XF+cxLPLnp+gu+HfNzE73De5+3SkPZrbbzEa6+85Ike85xbr+wPPAd05V8pHbqgPqAKqqqrympua0P8zZqq+vJ9HHTCf/+4BG/ubpN2HkxRTvei+rZ9FTtt83etI8jtMsomke0ZI5j3hfo19tZi/RXfQvmlkJcD4v2i4E7o2cvhd4rucCMysAngWecPdnzuO25Dx95rJRjOhfxCOL9cqJiEi6ibfov0L3O+OvdPcjQD7w5fO43YeB681sPXBd5DxmVmVm8yJrbgOqgfvMbG3ka9J53Kaco4K8HO6fWc7yho/YdKgzdBwRETkL8Rb9dOADdz9oZncB3wXOeSN0d9/v7te6+wR3v87dP4pcvsrd50ZOP+nu+e4+6YSvted6m3J+vjh1LCWFefxhk/5NvYhIOom36P8FOGJmlwP/B93vfn+i11JJyikpyufOq8ayclcn2z46EjqOiIjEKd6i7/Dut+fPAX7k7j8GSnovlqSiL8+oIMfgMW2LKyKSNuIt+iYz+zZwN/C8meXQ/Tq9ZJERA4qYNjKPp1Zu40DLsdBxREQkDvEW/e1AG3C/u+8CyoB/6LVUkrJurMjnaHsnTy7fEjqKiIjEIa6ij5T7L4ABZvYZoNXd9Rp9FhpTksM1Fw5j/rLNtLbrHfgiIqku3i1wbwNWAF+g+5+9vWFmt/ZmMEldD1ZXsq/5GM/+Sdviioikunh3xvsO3f+Gfg+AmQ0DXgG0kU0Wmj5+CJeM7s+jSxq4vWoMOTmn3SRRREQCivc1+pyPSz5i/1l8r2QYM6O2ejwNe1t4Zd3u0HFEROQ04i3rfzOzF83sPjO7j+7951/ovViS6j51yQhGD+xD3eKG0FFEROQ04n0z3jfp/tCYyyJfde7+rd4MJqktLzeHr8ysYNWWA6zeciB0HBEROYW4n35399+4+99Evp7tzVCSHm6/cgwD+uTzqB7Vi4ikrNMWvZk1mdnhGF9NZnY4WSElNfUrzOOuaWN58b1dbNrXEjqOiIjEcNqid/cSd+8f46vE3fsnK6SkrnuvLic/J4d5S/SoXkQkFemd83JehpcU8ZeTR/PM6kb2NbeFjiMiIj2o6OW8zZ1VQVtHFz9fpm1xRURSjYpeztsFw0u47qLhPLFsM0ePaVtcEZFUoqKXhKitHs+BI+08s3pb6CgiInICFb0kxJXlg5g0ZiDzlm6is8tDxxERkQgVvSSEmfFgdSVb9h/hxXd3hY4jIiIRKnpJmP9w8QjGDenLI4sbcNejehGRVKCil4TJzTHmzqzgzW0HWblZ2+KKiKQCFb0k1K1TxjC4XwF1izeGjiIiIqjoJcH6FORy97RxvLJuDxv2NIWOIyKS9VT0knD3TB9HYV4Ojy7eFDqKiEjWU9FLwg0pLuTWKWU8+6ft7DncGjqOiEhWU9FLr5g7q5L2ri7mL9scOoqISFZT0UuvqBjajxsmjuDJ5VtpaesIHUdEJGup6KXX1F5TyaGj7Ty1UtviioiEoqKXXjN57CCuLB/EY0s30dHZFTqOiEhWUtFLr6qtHs/2g0d5/u2doaOIiGQlFb30qms/OZzKYf14dIm2xRURCSFI0ZvZYDN72czWR/4cdJq1/c2s0cx+lMyMkhg5OcYDsyp5Z/thlm3cHzqOiEjWCfWI/iHgVXefALwaOX8q/wVYnJRU0ituuWI0Q4sLeWRxQ+goIiJZJ1TRzwHmR07PBz4Xa5GZTQFKgZeSlEt6QVF+LvddPY7XPtzL+7sOh44jIpJVQhV9qbt//O6sXXSXeRQzywH+G/CNZAaT3nHXtHH0yc+lTo/qRUSSynrrDVJm9gowIsZV3wHmu/vAE9YecPeo1+nN7KtAX3f/ezO7D6hy96+e4rZqgVqA0tLSKQsWLEjQT9GtubmZ4uLihB4zXZ3PLJ58r41F2zr4h2v6MLgoM94HqvtGNM3jOM0imuYRLdHzmD179mp3r4p5pbsn/Qv4ABgZOT0S+CDGml8AW4HNwD7gMPDwmY49ZcoUT7RFixYl/Jjp6nxmsXV/i1c89Hv/f55/L3GBAtN9I5rmcZxmEU3ziJboeQCr/BS9GOph1ULg3sjpe4Hnei5w9y+5+1h3L6f76fsn3P10b9qTFDdmcF8+delIfvnGVppa20PHERHJCqGK/mHgejNbD1wXOY+ZVZnZvECZJAkerB5PU1sHv1qxNXQUEZGsEKTo3X2/u1/r7hPc/Tp3/yhy+Sp3nxtj/c/8FK/PS3q5tGwA0yuH8PjSzRzr0La4IiK9LTPeESVppba6kl2HW/ndmztCRxERyXgqekm6mk8M48LSYm2LKyKSBCp6STqz7m1x39/VxOL1+0LHERHJaCp6CWLOpNGU9i+kbvHG0FFERDKail6CKMjL4cszKnh9w37e2X4odBwRkYylopdg7rxqLMWFedoWV0SkF6noJZj+RfncceUYnn97J40HjoSOIyKSkVT0EtT9Mysw4PGlm0NHERHJSCp6CWrUwD589vJRLFi5lUNHtC2uiEiiqegluAdmVXLkWCdPvrEldBQRkYyjopfgJo7qz6wJQ/nZHzfT1tEZOo6ISEZR0UtKqK2uZG9TG8/9SdviiogkkopeUsLMC4YycWR/6pY00NWlbXFFRBJFRS8pwcyora5kw55mFn2wJ3QcEZGMoaKXlPHpy0YyakARj2gDHRGRhFHRS8rIz83h/pkVrNj0EWu3HQwdR0QkI6joJaXcMXUsJUV5+rAbEZEEUdFLSikuzONLV43j397ZxZb9LaHjiIikPRW9pJwvzygnN8d4bOmm0FFERNKeil5STmn/Ij43aTRPr9rGRy3HQscREUlrKnpJSbXVlbS2d/HzZdoWV0TkfKjoJSVNKC3hLz45nCeWbaa1XdviioicKxW9pKwHZlWyv+UYv1nTGDqKiEjaUtFLyppWOZjLygYwb8kmOrUtrojIOVHRS8r6eFvcTftaePm93aHjiIikJRW9pLQbLx7BmMF9tIGOiMg5UtFLSsvLzWHuzErWbD3Iqs0fhY4jIpJ2VPSS8r5QVcbAvvnU6cNuRETOmopeUl7fgjzunjaOl9ftpmFvc+g4IiJpRUUvaeGe6eXk5+bw6BJtiysicjZU9JIWhpUU8vnJZfxmTSN7m9pCxxERSRtBit7MBpvZy2a2PvLnoFOsG2tmL5nZOjN7z8zKk5tUUsncWRW0d3bxxLLNoaOIiKSNUI/oHwJedfcJwKuR87E8AfyDu18ETAX2JCmfpKDxw4q57qJSfr58C0eOdYSOIyKSFkIV/RxgfuT0fOBzPReY2UQgz91fBnD3Znc/kryIkooerK7k4JF2fr1K2+KKiMTD3JO/taiZHXT3gZHTBhz4+PwJaz4HzAWOARXAK8BD7n7SJ5yYWS1QC1BaWjplwYIFCc3b3NxMcXFxQo+ZrlJhFn+7/CiH2pyHZ/UhN8eCZkmFeaQSzeM4zSKa5hEt0fOYPXv2anevinVdXsJupQczewUYEeOq75x4xt3dzGL9tpEHzAKuALYCTwH3AY/1XOjudUAdQFVVldfU1JxP9JPU19eT6GOmq1SYRevQXfzVk6s5OvQTfOayUUGzpMI8UonmcZxmEU3ziJbMefRa0bv7dae6zsx2m9lId99pZiOJ/dp7I7DW3Rsi3/NbYBoxil6yy/UTS6kY2o+6xQ18+tKRdD8pJCIisYR6jX4hcG/k9L3AczHWrAQGmtmwyPm/AN5LQjZJcbk5xtxZFbzVeIjlDdoWV0TkdEIV/cPA9Wa2Hrguch4zqzKzeQCR1+K/AbxqZm8DBjwaKK+kmM9PLmNIvwIeXaJtcUVETqfXnro/HXffD1wb4/JVdL8B7+PzLwOXJTGapImi/FzumV7OP77yIet3NzGhtCR0JBGRlKSd8SRt3T19HEX5OfqwGxGR01DRS9oa3K+A26rG8Nu129l9uDV0HBGRlKSil7T2lZkVdHY5P319c1zpHKsAAA83SURBVOgoIiIpSUUvaW3ckH7ceMkIfvHGFprbtC2uiEhPKnpJe7XV42lq7WDBiq2ho4iIpBwVvaS9SWMGMrViMI8v3UR7Z1foOCIiKUVFLxnhwepKdhxq5fm3doaOIiKSUlT0khFmf2I4Fwwv5pHFDYT4oCYRkVSlopeMkJNj1M6qZN3OwyzdsC90HBGRlKGil4wx54pRDCsp1AY6IiInUNFLxijMy+W+q8tZsn4f7+04HDqOiEhKUNFLRrnrqnH0LcjVh92IiESo6CWjDOibzx1XjuV3b+5gx8GjoeOIiASnopeMc//Mchx4fOmm0FFERIJT0UvGKRvUl09fOpJfrdjKoaPtoeOIiASlopeMVFtdScuxTn6lbXFFJMup6CUjXTJ6ADMuGMJPX9/EsQ5tiysi2UtFLxmrtno8uw+38dza7aGjiIgEo6KXjFU9YSifHFHCo0u0La6IZC8VvWQsM+OBWZV8uLuZ+g/3ho4jIhKEil4y2mcvH8WI/kXUvaYNdEQkO6noJaMV5OVw/8xyljXs5+3GQ6HjiIgknYpeMt4Xp46lpDCPRxZvDB1FRCTpVPSS8UqK8rnzqrG88PZOtn10JHQcEZGkUtFLVvjyjApyzHhM2+KKSJZR0UtWGDGgiJsnjeKplds4eORY6DgiIkmjopesUVtdydH2Tp5cviV0FBGRpFHRS9b45Ij+XHPhMH72xy20tneGjiMikhQqeskqD1ZXsq+5jWf/pG1xRSQ7qOglq0wfP4RLRvfn0SUNdHVpW1wRyXxBit7MBpvZy2a2PvLnoFOs+3sze9fM1pnZ/zAzS3ZWySwfb4vbsLeFV9/fEzqOiEivC/WI/iHgVXefALwaOR/FzK4GZgCXAZcAVwLXJDOkZKZPXzqS0QP7UKcNdEQkC4Qq+jnA/Mjp+cDnYqxxoAgoAAqBfGB3UtJJRsvLzeErMytYufkAa7YeCB1HRKRXhSr6UnffGTm9CyjtucDdlwGLgJ2RrxfdfV3yIkomu/3KMQzok68PuxGRjGe99TndZvYKMCLGVd8B5rv7wBPWHnD3qNfpzewC4L8Dt0cuehn4P919SYzbqgVqAUpLS6csWLAgMT9ERHNzM8XFxQk9ZrrKpFk88+Exnm9o5/+d1YcR/c7td95MmkciaB7HaRbRNI9oiZ7H7NmzV7t7Vazr8hJ2Kz24+3Wnus7MdpvZSHffaWYjgVjviroFWO7uzZHv+QMwHTip6N29DqgDqKqq8pqamgT8BMfV19eT6GOmq0yaxcTJrbz0d4t4p30Yd9Rcek7HyKR5JILmcZxmEU3ziJbMeYR66n4hcG/k9L3AczHWbAWuMbM8M8un+414eupeEmZ4/yJuuWI0v17VyP7mttBxRER6Raiifxi43szWA9dFzmNmVWY2L7LmGWAj8DbwJvCmu/8uRFjJXA9UV9DW0cUTy7Qtrohkpl576v503H0/cG2My1cBcyOnO4EHkxxNsswFw0u47qLhPLFsM391zXj6FOSGjiQiklDaGU+yXm31eA4caeeZ1dtCRxERSTgVvWS9K8sHMWnMQOYt3USntsUVkQyjopesZ2bUVleyZf8RXnp3V+g4IiIJpaIXAW64eATjhvTlkcUN9NbeEiIiIajoRYDcHGPuzArWbjvIys3aFldEMoeKXiTi1iljGNyvQB92IyIZRUUvEtGnIJe7p43jlXV72LCnKXQcEZGEUNGLnOCe6eMozMth3pJNoaOIiCSEil7kBEOKC7l1Shn/umY7e5paQ8cRETlvKnqRHubOqqS9q4v5f9wcOoqIyHlT0Yv0UDG0HzdMHMGTy7fS0tYROo6IyHlR0YvEUHtNJYeOtvPUSm2LKyLpTUUvEsPksYOoGjeIx5ZuoqOzK3QcEZFzpqIXOYXa6kq2HzzKC+9oW1wRSV8qepFTuO6iUiqH9aNu8UZtiysiaUtFL3IKOTnGA7MqeWf7YZZt3B86jojIOVHRi5zGLVeMZmhxIY8sbggdRUTknKjoRU6jKD+X+64ex2sf7uWDXdoWV0TSj4pe5Ay+dNU4+uTnUqdH9SKShlT0ImcwqF8Bt185hoVvbmfXIW2LKyLpRUUvEoevzKygs8v56ev6sBsRSS8qepE4jBncl09dOpJfvrGVptb20HFEROKmoheJU211JU1tHfxqxdbQUURE4qaiF4nTZWUDmVY5mMeXbuZYh7bFFZH0oKIXOQsPVo9n1+FWfv/WjtBRRETioqIXOQs1nxjGhaXF1C1u0La4IpIWVPQiZ8Gse1vc93c1sXj9vtBxRETOSEUvcpbmTBpNaf9C6hZvDB1FROSMVPQiZ6kgL4f7rq7g9Q372XK4M3QcEZHTygsdQCQd3XnVWH707+v5b6taeXLDa6dcF8+r+Gd6rT+udwKcYVEycrjD0aNH6bNi0WmOcYbbiCNoIt4akYif9Uza2too/OOrUZeZnbyu50UWa9FJa+K8rMfRz/X2T7rkHI5zpOUI/da81mNNrOOcOfNJ3xNP5hjHStTMYi06eR7R528c0UlNjIy9QUUvcg4G9Mnnb2+5hF/Uv8Pw4cWnXdvzL44YC87oTEviKoczHuP8c+ze08aI0kHndYz45nH6RWf6WeK4iTiOcfoFO3ftZOSIYX8+H+uXnJ6/MMT6/eHkNTFWxXFRrF9uTl6TmOPEyrN7z9Go/1di3lYcP+u5zCzWykT9rOd6nJx47oQJEqTozewLwP8NXARMdfdVp1h3I/DfgVxgnrs/nLSQImdwyxVlDDq0gZqaKaGjpIz6+npqaiaFjpES6us/oqbmstAxUkb3fUP/r3ysvr4+abcV6jX6d4C/BBafaoGZ5QI/Bm4CJgJfNLOJyYknIiKSGYI8onf3dXDGpxunAhvcvSGydgEwB3iv1wOKiIhkiFR+1/1oYNsJ5xsjl4mIiEiceu0RvZm9AoyIcdV33P25BN9WLVALUFpamvDXPpqbm5P6ekoq0yyiaR7RNI/jNItomke0ZM6j14re3a87z0NsB8accL4sclms26oD6gCqqqq8pqbmPG86WvebSBJ7zHSlWUTTPKJpHsdpFtE0j2jJnEcqP3W/EphgZhVmVgDcASwMnElERCStBCl6M7vFzBqB6cDzZvZi5PJRZvYCgLt3AF8FXgTWAU+7+7sh8oqIiKSrUO+6fxZ4NsblO4BPnXD+BeCFJEYTERHJKKn81L2IiIicJxW9iIhIBlPRi4iIZDAVvYiISAZT0YuIiGQwO9PnMqcbM9sLbEnwYYcC+xJ8zHSlWUTTPKJpHsdpFtE0j2iJnsc4dx8W64qMK/reYGar3L0qdI5UoFlE0zyiaR7HaRbRNI9oyZyHnroXERHJYCp6ERGRDKaij09d6AApRLOIpnlE0zyO0yyiaR7RkjYPvUYvIiKSwfSIXkREJIOp6CPM7EYz+8DMNpjZQzGuLzSzpyLXv2Fm5clPmTxxzOM+M9trZmsjX3ND5EwGM3vczPaY2TunuN7M7H9EZvWWmU1OdsZkimMeNWZ26IT7xveTnTFZzGyMmS0ys/fM7F0z+3qMNVlz/4hzHllx/zCzIjNbYWZvRmbxn2OsSU6vuHvWfwG5wEagEigA3gQm9ljzvwI/iZy+A3gqdO7A87gP+FHorEmaRzUwGXjnFNd/CvgDYMA04I3QmQPPowb4feicSZrFSGBy5HQJ8GGM/1ey5v4R5zyy4v4R+e9dHDmdD7wBTOuxJim9okf03aYCG9y9wd2PAQuAOT3WzAHmR04/A1xrZpbEjMkUzzyyhrsvBj46zZI5wBPebTkw0MxGJidd8sUxj6zh7jvdfU3kdBOwDhjdY1nW3D/inEdWiPz3bo6czY989XxTXFJ6RUXfbTSw7YTzjZx85/zzGnfvAA4BQ5KSLvnimQfA5yNPRT5jZmOSEy0lxTuvbDI98pTlH8zs4tBhkiHytOsVdD9yO1FW3j9OMw/IkvuHmeWa2VpgD/Cyu5/yvtGbvaKil3P1O6Dc3S8DXub4b6Uia+jejvNy4H8Cvw2cp9eZWTHwG+Cv3f1w6DyhnWEeWXP/cPdOd58ElAFTzeySEDlU9N22Ayc+Ii2LXBZzjZnlAQOA/UlJl3xnnIe773f3tsjZecCUJGVLRfHcf7KGux/++ClLd38ByDezoYFj9Rozy6e71H7h7v8aY0lW3T/ONI9su38AuPtBYBFwY4+rktIrKvpuK4EJZlZhZgV0vyliYY81C4F7I6dvBf7dI++gyEBnnEeP1xhvpvu1uGy1ELgn8u7qacAhd98ZOlQoZjbi49cZzWwq3X/PZOQvxZGf8zFgnbv/8BTLsub+Ec88suX+YWbDzGxg5HQf4Hrg/R7LktIreYk+YDpy9w4z+yrwIt3vOH/c3d81sx8Aq9x9Id133p+b2Qa634h0R7jEvSvOefxvZnYz0EH3PO4LFriXmdmv6H6n8FAzawT+E91vrMHdfwK8QPc7qzcAR4Avh0maHHHM41bgfzGzDuAocEcG/1I8A7gbeDvyWizA/wWMhay8f8Qzj2y5f4wE5ptZLt2/zDzt7r8P0SvaGU9ERCSD6al7ERGRDKaiFxERyWAqehERkQymohcREclgKnoREZEMpqIXkaSJfHLZ70PnEMkmKnoREZEMpqIXkZOY2V2Rz9Jea2aPRD6co9nM/jHy2dqvmtmwyNpJZrY88gFHz5rZoMjlF5jZK5EPL1ljZuMjhy+OfBDS+2b2iwz+FEiRlKCiF5EoZnYRcDswI/KBHJ3Al4B+dO/odTHwGt074gE8AXwr8gFHb59w+S+AH0c+vORq4ONtX68A/hqYCFTSvZuaiPQSbYErIj1dS/eHFK2MPNjuQ/fHbHYBT0XWPAn8q5kNAAa6+2uRy+cDvzazEmC0uz8L4O6tAJHjrXD3xsj5tUA5sLT3fyyR7KSiF5GeDJjv7t+OutDsez3Wnev+2W0nnO5Efw+J9Co9dS8iPb0K3GpmwwHMbLCZjaP774tbI2vuBJa6+yHggJnNilx+N/CauzcBjWb2ucgxCs2sb1J/ChEB9Ju0iPTg7u+Z2XeBl8wsB2gH/iPQAkyNXLeH7tfxoftjNn8SKfIGjn86293AI5FP62oHvpDEH0NEIvTpdSISFzNrdvfi0DlE5OzoqXsREZEMpkf0IiIiGUyP6EVERDKYil5ERCSDqehFREQymIpeREQkg6noRUREMpiKXkREJIP9/xQJEdloVmndAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "f = plt.figure(figsize=(8, 6))\n",
    "plt.grid()\n",
    "plt.plot(test_losses, label='test')\n",
    "plt.xlabel('epoch')\n",
    "plt.ylabel('loss')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 150,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "VMGZjcpGorvU",
    "outputId": "7f016842-4e28-4f1f-bf40-76d077b5347f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Min test loss: -0.976848, min train loss: -0.946829\n"
     ]
    }
   ],
   "source": [
    "print(\"Min test loss: {:.6f}, min train loss: {:.6f}\".format(min(test_losses), min(train_losses)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "IRe-Iv84orJU"
   },
   "source": [
    "We managed to achieve a great loss, taking into account that the values lie on the segment [-1, 1]."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TLmET_EupdT_"
   },
   "source": [
    "## PESQ metrics ##"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "0bjGE7-7pgAJ",
    "outputId": "3cfa6a99-e103-4d6b-8a41-ccd8dab4b2d6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Value of PESQ: 2.818076\n"
     ]
    }
   ],
   "source": [
    "print(\"Value of PESQ: {:.6f}\".format(pesq_metric))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9vaApoDWpqGN"
   },
   "source": [
    "Values of the PESQ metric calculated on different networks from the article: \n",
    "\n",
    "![pesq](img/metric.png)\n",
    "\n",
    "Taking into account the fact that our 10-layer model was trained for only 2 epochs, we have achieved decent results. Our moel even shows a greater metric value than Wiener, SALON, MEGAN. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5nvZcN3WYpmb"
   },
   "source": [
    "## Visualization ##\n",
    "Let's take a look at the time-domain waveform of a pure signal, with noise and predicted :\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "IhZKPirKvO3q"
   },
   "outputs": [],
   "source": [
    "dcunet10.eval()\n",
    "x_n, x_c = iter(test_loader).next()\n",
    "x_est = dcunet10(x_n.cuda(), is_istft=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "TWySU5oOXraV"
   },
   "source": [
    "Purified"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 151,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "colab_type": "code",
    "id": "156VyDi1vOCe",
    "outputId": "8f80a35a-b4dd-492a-c97f-f768b154294c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f04c3f7b240>]"
      ]
     },
     "execution_count": 151,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAdPUlEQVR4nO3de5gU5Zn38e89DAcBUU5B5DQguGoWRZ0gKmoMeEDzyiYmBjdZzWuM63pYdzUHjNHX18SE6CZXssZsoonR6CYeN5EIHlGjJiIOiiggMCAiyBlFEIE53PtH14w9Q3fPdHd1V83U73NdXFRXVddzT9H8pvqpqqfM3RERkc6vIuoCRESkPBT4IiIJocAXEUkIBb6ISEIo8EVEEqIy6gKyGTBggFdVVUVdhohIhzJ//vzN7j4w07LYBn5VVRU1NTVRlyEi0qGY2dvZlqlLR0QkIRT4IiIJocAXEUkIBb6ISEIo8EVEEkKBLyKSEAp8EZGEUOCLSKje3vIhLyzfHHUZkkFsb7wSkY7ppJufBWDZ96fQrVLHlHGifw0RKYnfvPBW1CVIKwp8EQnN029uaJ7evqsuwkokEwW+iIRmzpKNzdO/eHZFhJVIJgp8EZGEUOCLiCSEAl9ESuaT1z3GwjXvR12GBBT4IlIyH+5p4A5drRMbCnwRkYRQ4ItIaBp973nvvr+r/IVIRgp8EQnNH+at3mvevFVbI6hEMlHgi4gkhAJfRCQhFPgiIgmhwBcRSQgFvohIQijwRUQSQoEvIpIQoQS+mZ1uZkvNrNbMpudY72wzczOrDqNdEekY3DPckSVlV3Tgm1kX4FZgCnAYcK6ZHZZhvX2BK4CXim1TRDqWu+e+HXUJQjhH+OOBWndf6e57gHuBqRnW+x7wI0D3WYskzCtvvxd1CUI4gT8EeCft9ZpgXjMzOwoY5u6zcm3IzC4ysxozq9m0aVMIpYmISJOSn7Q1swrgJ8BVba3r7re5e7W7Vw8cOLDUpYmIJEoYgb8WGJb2emgwr8m+wN8Dz5rZKmACMFMnbkWSw8yiLkEIJ/BfBsaY2Ugz6wZMA2Y2LXT3be4+wN2r3L0KmAuc5e41IbQtIiLtVHTgu3s9cBnwOLAEuN/dF5nZDWZ2VrHbFxGRcFSGsRF3nw3MbjXvuizrfjqMNkUkXjbv2B11CdIG3WkrIqGY/tDCrMvUgx8PCnwRCcWuusasy1Zs2lHGSiQbBb6IlNxra7ZFXYKgwBcRSQwFvoiEoqFRA6TFnQJfREKhwI8/Bb6ISEIo8EUkFI6O8ONOgS8ikhAKfBEJhR5qFX8KfBEJhfI+/hT4IhIKPbc2/hT4IiIJocAXEUkIBb6IhEIdOvGnwBcRSQgFvoiEQuds40+BLyLNajfu4N33Pyrovcr7+AvlEYci0jlM/slfAFg148z839zGIf7u+ga6V3YppCwJiY7wRSQUbR3h3zKntix1SHYKfBEpiy0f7om6hMRT4ItIKHTSNv4U+CJSFmZRVyAKfBEBYP22XUW9vyOMh//YG+uoWbU16jIio6t0RASA2a+va57eVddAj67hXlEThwP8i+95BSjwKqROQEf4IrKXTdt35/0e9eHHnwJfRIDS3zgVZR9+Q6Nz6zMfXxa6q66BxgQ+dF2BLyJ7qajIP53jfIT/2Bvrufnxpc2vD7n2Mf7/nxdFWFE0FPgispc49LeHaXd9w17z7n35nQgqiZYCX0SAlk+s+t4ji9m+qy6/97ex/J65q6maPquAyoqX6dtHXUNj+QuJmAJfRABYteXD5ulH31jPLU/nNxTCknUfhF1SSSWwCz+cwDez081sqZnVmtn0DMuvNLPFZrbQzOaY2Ygw2hWR8Nwzd3WL1/UNnScRs/0kX/n1S0Xff9CRFB34ZtYFuBWYAhwGnGtmh7Va7VWg2t0PBx4Ebiq2XREprTv++lbUJZTcC7WbmfDDOVGXUTZhHOGPB2rdfaW77wHuBaamr+Duz7j7zuDlXGBoCO2KSIn99KllUZcQCo/zJURlFEbgDwHST3evCeZl8zXg0UwLzOwiM6sxs5pNmzaFUJqIFOOnTy0PfZt3vND5vznEVVlP2prZV4Bq4OZMy939NnevdvfqgQMHlrM0kUQr5xHwDY8sLltb7VWfkCt2wgj8tcCwtNdDg3ktmNlk4BrgLHfP/75tESmZYvM+7l0mtz+/MufyK+9/rUyVRCuMwH8ZGGNmI82sGzANmJm+gpkdCfyKVNhvDKFNEQnRDx9dUtT731y/PaRKSmPZhh05l8987d0yVRKtogPf3euBy4DHgSXA/e6+yMxuMLOzgtVuBnoDD5jZAjObmWVzIhKB258vrl+9oRNc1P70mxuiLqHkQhke2d1nA7NbzbsubXpyGO2ISPmt3rKT4f17Rl1GyV1wZw1Txx3Iz6YdGXUpJaM7bUUkp/Zcj//ZW14oQyWl9/CCzt21o8AXEUmzNObnI4qhwBcRSXPaT5+LuoSSUeCLSE5tPbjk4QV7XYUtMaXAF5Gc2rrE/op7F5SnkDKK+30FhVLgi0jBOmsw7umkd94q8EUkp1xdOnf8dVXZ6mhtx+56qqbP4rO3PB/6tq3TPfMrRYEvIjnlGhf/pZVbCtpmGN8MfhsMwvbG2o714JUoKfBFJKe7574d+jZfLPAXRbofP9k5hm4uJwW+iBTsicWFDUcQ96EYjpsxh8aY11gIBb6IlF1FW9d6Rmzzjj2d8sStAl8k4aJ4+Hi84z7lkGsf4/nlnetBTKEMniYiHdeUn4V/lUtbLOZH+E0uuPNl6hqcm79wOK+v3cbUcUPo06OSMYP2jbq0gijwRaQgO/fUF/zeio6R99QFVyh988GFAPzuxdQJ7FUzzoyspmKoS0dECrJy04cFvzfsI/y3NhdeS5Io8EWkXeobGnn3/Y8AeG7ZpqKGRA77CL+prnLZXd9Q1vbCosAXkXb5wew3OW7G02zesZvHF60valsdpQ8/m+v+tIjVW3ZGXUbeFPgi0i7PLks9jvr9nXUUe4X632o3F19QhO6reYcTb34m6jLypsAXkTx50UMjhH2XbFTfF2YtXFfUyety01U6ItIuTaHqDo0xuydp3bZdkbR76e9fATrOVTs6wheRNu2pb5nw81e/F1ElKTt2tzyqvuqB1yKqJOWrv51H1fRZsR8uWoEvIm1qfTdu7cYdEVWS0hizYH12aeqO3Adq1kRcSW4KfBFpk/PxlTUbt++OthhyP4Vr7fsfsXF7NF08KzZF+4uwLerDF5E2Nbqz9cM9ANz5t1XRFpPFrroGlm3Yzlk//ysA/3ziKKZPOaSsl4DG7ZtHawp8EWlTQ+PHgf9kgUMihypDrh5y7WMtXv/quZWcO344I/r3LFNR8acuHRFpU9wOXL2ddwK8/1FdWWuP+txGWxT4ItKmuN0Y295nk9Ss2lrWbpZnlsZ7OGUFvoi06Ykih1LI5cr7FnB/zTt5vae9If79WUuKvis4X3EeZ0eBLyJtuv35t0qyXXfnf15dy7ceXMireVzbn89R+/oy35T1q7+sLGt7+VDgiyTQxg928cu/rODFFcU/TLxQ7s7Iq2c3v/7cL/6WcZ2q6bO47+XVLeavfa/9o2OecFN5x7z5SYwfrq6rdEQS6NP/8Sw790Tb9ZAe9k3qGhrp2qWCXXUNzHztXb7358UAfPuh1xnRvxf3zH2bzx4+mIvveaXc5ebF3WM5IqjF9Vbg6upqr6mpiboMkZw279jNrc/U8p0zDqVrl47zhblq+qyoS8hoUJ/u/PnyiYy/cU7UpRRteL+eXHXqwUwdN6Ss7ZrZfHevzrQslE+omZ1uZkvNrNbMpmdY3t3M7guWv2RmVWG0KxK1//fwIn7711V8+6GFeb2vodF5ZOG7e40J89gb66maPiu0ERjf+3APVdNnMe+trc3z4nxSccMHuztF2AOs3rqTK+5dwPIN25ufH+DubPhgF/UNjTw0f02LsXfqGhppaO/lRwUq+gjfzLoAy4BTgDXAy8C57r44bZ1LgMPd/WIzmwZ8zt2/lGu7OsKXONhT38hHdQ38+30LOHZUf6aOO5BP9OmBu7O7vrHFzT5Txx3Iz6Ydibsz762tfOm2uc3LHrl8In8/ZD8Anl++iX/6zbzmZQ/9y7EcPaIfB31ndov/8E0jMH7jgdd4cP6aFvNeXLGFc2+fy749Ktm+6+NfDo9cPpGDBvZm55566hqcCT+c02J7r65+L2NfuZRe7Y1TGHv9E3xU9/Ev3OH9evLct05m3ltbOedXLwLw1g/PKKo7KNcRfhiBfyxwvbufFry+GsDdf5i2zuPBOi+aWSWwHhjoORovNPC3fVTHxXfPb9e67b15A9p/40lee7Pd24y2znw+I+1dM5+PXbtXLUGdC9dsa/c2AYb124d3tmY/oXjpyQdx6zMr9ppfe+MURl/zaF5tSedW6JDLuQI/jJO2Q4D0i2jXAMdkW8fd681sG9AfaPHYGzO7CLgIYPjw4QUXlNfXojx+kbZnVSOPm1QMrJ1bzecXfnvXbV/b+W0zH/kcxbR7l4b879ke8787maO//xRAzrAHWoT936Z/huNmPA3QIuzvvWgC09K+HYiEJVZX6bj7bcBtkDrCL2Qb++3TlfsvPjbUukRg78sIAS6cOJL+vbuzasaZLU6EdutSwdLvn978S+3Qax9r8VW+6egt/ZcFwFWnHMyEUf05YcwAnl+eOh5acN0p7N+zG8s2bGfH7no+H3TJDOjdnZrvTubtLR8yZ8lG/unYEYxp9S3hoIG9eOrKkzJeESOld80ZhzJt/DDGXv9E87w/XzaRsUP3a/F5uezk0XSvrODcY4Zzf807fKqqX0nq6XRdOiJRWbX5Q55cvIGvnzgqr/fd8cJb3PBI6pRX66/xDY1Ol4qW30WWrPuARnc+eeB+7W5j0/bd/OnVtdw4e0nz+YQb/ryYO/5amhuqinX5Z0Zzy9O1UZdRtMU3nEbPbpXN4b7yB2dQEfx7/vt9C/jjq2uZ/93J9O/dPbQ2S92HX0nqpO0kYC2pk7b/6O6L0ta5FBibdtL28+5+Tq7tKvAlKT7YVcfX7nyZ//jiEYzo36ts7TY0Ogd9J35H/kP77sNz3zyZUVlq+9rEkeyub+CeuaszLo+LLx8znBs/Nzbr8vqGRjZs382Q/fcJtd2S9uEHffKXAY8DXYA73H2Rmd0A1Lj7TOA3wN1mVgtsBaYV265IZ9GnR1ceuPi4srfbpcK45dwjufwPr5a97UyG9duHCyeO4vzjqrKus+IHZ9ClwrjjhXh+M0mXK+wBKrtUhB72bQmlD9/dZwOzW827Lm16F/DFMNoSkfBs3hH906uaPP+tz7R4vez7U7j9+ZVc8umD9jrBf96xI5q7weJoyQ2nR11CRh3n1kARCd2pnzwgsrYPHtSbHwRHwY9eccJey7tVVnDpyaMzXs1VGfO7mvfp1iXqEjKK914TkZIqd5dCuj9dejznjh/G4htO49DBfYre3lHD9884/83vlfdo+wdtdOVESYEvIpHo2a0SM6Nnt3CuDh81sHfG+T26lvdo+5zqoWVtLx8KfBHJy2cO+UTUJWQUl3EgW19GGycKfBFpl198+SggdVNZsU46eGDR22gtnyFISmXedybFcljkJgp8EclLGHn2qaq+xW8kzcgBvfIcyCp8x4/uzyf69Ii2iDYo8EWkXcLsMgn7KPinXxrH6EGpPvzPHzmEeddMCnX77fHr8z5V9jbzFauxdEQk/sLI6rB7PY4Ytj9jh+zH+Kp+VFf1o76hEYBSd6dfOHEkY4fuV/aHnBRKgS8i7XLEsNTYPV84eiizX19f1LaqR4Q/OFhFhVEdDDrWpcL455NGcebYwaG30+Smsw/nnE8NK9n2S0GBLyLtMrRvz4LHaG9t/MjSjAbZxMy4esqhJdt+0wimHY368EWkQ/pqjjF3SuneiyZ0yLAHBb6ItMP+PbtGXcJerv3sYWVv84apn2TCqP5lbzcsCnwRadPcq1te9TJ4v+gvPyzXDU6fOzJ1Qva5b57MecdWlaXNUlEfvoi0qfXwBL26xyc6wv72cdnJo7nq1IN5d9suBvfpgQP/OmkMw/v3DLWdKMTnX01EOoymY+v0RzFG4a4LxjPmE5nH0GmvA/r0YP0Hu5pfTx13IGbWYmC5kQPK92CaUlKXjogUbPKhgyJt/6SDB3JgESN+XjFpDHO/83F31aoZZzJm0L5hlBZLCnwRKdixB3XcE5hJpMAXkaJEfZRfjKbRIm46+3C+fsLISGspB/Xhi0hRfjZtHL/6ywr+8+naqEvJ2wHBYGcd7Y7ZQukIX0SK0qt7JVee+ndRl5G37555KNMSEvRNFPgiCffFo/N/QtPAfbsD0LWAsfHnXHVS3u8phdM+eQAVMX5YSSko8EUS7uYvHpH3e37+j0dx09mHF3S54kFZHkVYbn17dczhEYqhwBeRvPXr1a1D93v/4stH0TtGN4+ViwJfRBInWR05H1Pgi4gkhAJfRHL6/YXHtGu9vjEcUbPJ6FbDL8T92bOlkrxOLBHJy8iBHX8cmR5dU8e2f7r0eNydI4eH+xD1jkJH+CKSk3WCHu9RA1JH+H16VCY27EFH+CLShvY+cNzCfjJ5iGacPZazjx7KqJhcEhoVHeGLyF7+ddKY5ukwY/zwofuFuLX269mtkpMOHhhJ23GiwBeRvVx5ysEfvwgx8b97ZvkfSygfK6pLx8z6AfcBVcAq4Bx3f6/VOuOA/wL6AA3Aje5+XzHtikhp3PO1Y1i9dSeQepLU+zvr2v3e9vxeKNdjCSWzYvvwpwNz3H2GmU0PXn+71To7gfPcfbmZHQjMN7PH3f39ItsWkZBNHDOgefrBi4/jj6+uYWDv7qFtf1Cf8LYl+Ss28KcCnw6m7wKepVXgu/uytOl3zWwjMBBQ4IvExMOXHr/XydnRn+jNN087JNR2hvbt+M+F7ciKDfxB7r4umF4P5HwSgpmNB7oBK4psV0RCdMSw/aMuQcqgzcA3s6eAAzIsuib9hbu7mXmG9Zq2Mxi4Gzjf3RuzrHMRcBHA8OHD2ypNRGIkxldlSqDNwHf3ydmWmdkGMxvs7uuCQN+YZb0+wCzgGnefm6Ot24DbAKqrq7P+8hARkfwVe1nmTOD8YPp84OHWK5hZN+CPwO/c/cEi2xOR2Mp9iH/XBePLVIdkU2zgzwBOMbPlwOTgNWZWbWa/DtY5BzgR+KqZLQj+jCuyXRHpYCaOHtD2SlJSRZ20dfctwKQM82uAC4Ppe4B7imlHROLvikmjufbhRVGXITnoTlsRCcVXJoyIugRpgwJfRELR1uBpusk2egp8ESmLOI+mmRQKfBGRhFDgi4gkhAJfRCQhFPgiIgmhwBcRSQgFvohIQijwRUQSQoEvIpIQCnwRKTk92jAeFPgiUnJV/XtFXYKgwBeRMrj05NFRlyAo8EWkDKqr+kZdgqDAFxFJDAW+iJRc1y6KmjjQv4KIlFT3ygoFfkzoX0FESuqIYftHXYIEFPgiUlJ67El8KPBFpKT0oKv4UOCLiCSEAl9EJCEU+CISmguOHxl1CZKDAl9EQtO7e5eoS5AcFPgiUlKm63RiQ4EvIiX19RPVzRMXCnwRKanjRw+IugQJKPBFJDQedQGSkwJfRCQhFPgiIgmhwBeRkupaoZiJi6L+Jcysn5k9aWbLg7+zPtbGzPqY2Roz+3kxbYpIx1JRocsy46LYX73TgTnuPgaYE7zO5nvAc0W2JyIiBSo28KcCdwXTdwH/kGklMzsaGAQ8UWR7IiJSoGIDf5C7rwum15MK9RbMrAL4MfCNtjZmZheZWY2Z1WzatKnI0kREJF1lWyuY2VPAARkWXZP+wt3dzDJdhnsJMNvd11gbA2O7+23AbQDV1dW6pFdEJERtBr67T862zMw2mNlgd19nZoOBjRlWOxY4wcwuAXoD3cxsh7vn6u8XkQ7o2FH9ueXp2ubXMz4/NsJqpLU2A78NM4HzgRnB3w+3XsHdv9w0bWZfBaoV9iKd03GthlGYNn54RJVIJsX24c8ATjGz5cDk4DVmVm1mvy62OBERCU9RR/juvgWYlGF+DXBhhvl3AncW06aIiBRGt8CJiCSEAl9EJCEU+CIiCaHAF5GSGNZvn6hLkFYU+CJSEhNHD4y6BGlFgS8ikhAKfBEpiXPHD4u6BGlFgS8iJXH40P2jLkFaUeCLiCREsWPpiIi0MOPzYxkzaN+oy5AMFPgiEioNmBZf6tIREUkIBb6ISEIo8EVEEkKBLyKSEAp8EZGEUOCLiCSEAl9EJCEU+CIiCWHuHnUNGZnZJuDtIjYxANgcUjnlpLrLq6PWDR23dtVdWiPcPePY1LEN/GKZWY27V0ddR75Ud3l11Lqh49auuqOjLh0RkYRQ4IuIJERnDvzboi6gQKq7vDpq3dBxa1fdEem0ffgiItJSZz7CFxGRNAp8EZGE6HSBb2anm9lSM6s1s+kR1TDMzJ4xs8VmtsjMrgjmX29ma81sQfDnjLT3XB3UvNTMTmvr5zGzkWb2UjD/PjPrFlLtq8zs9aC+mmBePzN70syWB3/3Deabmf1nUMNCMzsqbTvnB+svN7Pz0+YfHWy/NnivhVT336Xt1wVm9oGZ/Vsc97mZ3WFmG83sjbR5Jd/H2doosu6bzezNoLY/mtn+wfwqM/sobb//stD6cu2DImsv+WfDzLoHr2uD5VX51h4qd+80f4AuwApgFNANeA04LII6BgNHBdP7AsuAw4DrgW9kWP+woNbuwMjgZ+iS6+cB7gemBdO/BP4lpNpXAQNazbsJmB5MTwd+FEyfATwKGDABeCmY3w9YGfzdN5juGyybF6xrwXunlOhzsB4YEcd9DpwIHAW8Uc59nK2NIus+FagMpn+UVndV+nqttpNXfdn2QQi1l/yzAVwC/DKYngbcF/bnPZ8/ne0IfzxQ6+4r3X0PcC8wtdxFuPs6d38lmN4OLAGG5HjLVOBed9/t7m8BtaR+low/T3BE9BngweD9dwH/UJqfprm+uzK0NRX4nafMBfY3s8HAacCT7r7V3d8DngROD5b1cfe5nvof8LsS1T0JWOHuue7Ujmyfu/tzwNYM9ZR6H2dro+C63f0Jd68PXs4FhubaRoH1ZdsHRdWeQ5ifjfSf6UFgUtM3mih0tsAfAryT9noNuYO25IKvcEcCLwWzLgu+lt6R9pU6W93Z5vcH3k/7jxbmz+nAE2Y238wuCuYNcvd1wfR6YFCBdQ8JplvPD9s04A9pr+O+z6E8+zhbG2G5gNSReJORZvaqmf3FzE4I5hVSXyn/X5f6s9H8nmD5tmD9SHS2wI8VM+sNPAT8m7t/APwXcBAwDlgH/DjC8rKZ6O5HAVOAS83sxPSFwVFZbK/lDfpOzwIeCGZ1hH3eQjn2cdhtmNk1QD3w38GsdcBwdz8SuBL4vZn1iaq+LDrcZ6NYnS3w1wLD0l4PDeaVnZl1JRX2/+3u/wPg7hvcvcHdG4HbSX1FhOx1Z5u/hdTX2spW84vm7muDvzcCfwxq3ND0FTr4e2OBda+l5Vf+Uvz7TAFecfcNwc8R+30eKMc+ztZGUczsq8BngS8HQU3QHbIlmJ5Pqu/74ALrK8n/6zJ9NprfEyzfL1g/Ep0t8F8GxgRnzLuR+mo/s9xFBH10vwGWuPtP0uan9zt+Dmi6YmAmMC04oz8SGEPqxFbGnyf4T/UM8IXg/ecDD4dQdy8z27dpmtQJuTeC+pquAklvayZwXnAVxQRgW/CV/HHgVDPrG3xNPhV4PFj2gZlNCPbReWHU3cq5pHXnxH2fpynHPs7WRsHM7HTgW8BZ7r4zbf5AM+sSTI8itX9XFlhftn1QbO3l+Gyk/0xfAJ5u+qUYiVKfFS73H1Jn9JeROqK4JqIaJpL6OroQWBD8OQO4G3g9mD8TGJz2nmuCmpeSduVKtp+H1JUC80idUHoA6B5C3aNIXXnwGrCoqT1SfY5zgOXAU0C/YL4Btwa1vQ5Up23rgqC2WuD/ps2vJvUfawXwc4K7vUPa771IHT3tlzYvdvuc1C+kdUAdqf7er5VjH2dro8i6a0n1UTd9zpuuSDk7+AwtAF4B/k+h9eXaB0XWXvLPBtAjeF0bLB8Vdt7k80dDK4iIJERn69IREZEsFPgiIgmhwBcRSQgFvohIQijwRUQSQoEvIpIQCnwRkYT4X97WMTVSyqu9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(x_est[1].view(-1).detach().cpu().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5jBeTpGiXukt"
   },
   "source": [
    "Clear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 152,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "colab_type": "code",
    "id": "Hh25gt1DvaO_",
    "outputId": "5a1aabae-f7ec-4e09-faca-10a44d61e665"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f04c3f4e6d8>]"
      ]
     },
     "execution_count": 152,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAeRklEQVR4nO3de3wU9b3/8dcniYAHAUEiUm4BwVpEqxCp1lsr3sALbe0Fe5FWe2xr9bQ/29+RllZtrRV7Pb34ULH1Z0U92tbachQKgopHK0rwAnIVMCjIJYCAgNySz++PncRNsptks7M7s9n38/HIg9mZ2ZnPLpt3Zr/zne+YuyMiIh1fSdQFiIhIfijwRUSKhAJfRKRIKPBFRIqEAl9EpEiURV1AOr179/aKioqoyxARKSgLFy7c4u7lqZbFNvArKiqoqqqKugwRkYJiZmvTLVOTjohIkVDgi4gUCQW+iEiRUOCLiBQJBb6ISJFQ4IuIFAkFvohIkVDgi0io5q2s4a1te6IuQ1JQ4ItIqCbe8yJjfjkv6jIkBQW+iIRuf20dz6/eGnUZ0oQCX0Ry4rK750ddgjShwBeR0OzYcyDqEqQFCnwRCc0vn1gRdQnSAgW+iISmts4bPf7aNI14GycKfBHJmVlLNrFw7TtRlyEBBb6I5NR9z1dHXYIEFPgiIkVCgS8iObVdPXdiQ4EvIqHZve9gs3nzVtZEUImkosAXkdD8/ZW3oy5BWqDAFxEpEgp8EZEiocAXESkSCnwRkSKhwBcRKRKhBL6ZXWBmK8xslZlNamG9S83MzawyjP2KSGHYd7A26hKEEALfzEqB24GxwHDgMjMbnmK9bsC3gBey3aeIFJZfPbEy6hKEcI7wRwOr3H2Nu+8HHgLGp1jvZuA2YG8I+xSRArJph37t4yCMwO8HvJX0eF0wr4GZjQQGuPvjIexPRETaIecnbc2sBPgV8J02rHuVmVWZWVVNjS7HFhEJUxiBvx4YkPS4fzCvXjdgBPC0mVUDpwDTU524dfep7l7p7pXl5eUhlCYicWBmUZcghBP4C4BhZjbYzDoBE4Dp9QvdfYe793b3CnevAOYDl7i7boUjIpJHWQe+ux8ErgFmAcuAP7v7EjP7sZldku32RUQkHGVhbMTdZwAzmsy7Ic26HwtjnyJSOF55a3vUJQi60lZEQrJ5Z/qul29s2Z3HSiQdBb6IhOL6RxZFXYK0QoEvIqE4UOtRlyCtUOCLiBQJBb6ISJFQ4ItIKA7W1UVdgrRCgS8ioahTE37sKfBFJBwK/NhT4ItIKFyJH3sKfBGRIqHAF5FQuA7wY0+BLyKhUN7HnwJfRKRIKPBFJBSuNp3YU+CLiBQJBb6IhELH9/GnwBeRUKhFJ/4U+CLSYNaSjSyo3hZ1GZIjodziUEQ6hq9NWwhA9ZQLI65EckFH+CISCrXoxJ8CX0Sa2XugNuPntNYt882te9pbjoREgS8izdS8uy/0bd4xb3Xo25TMKPBFJBTqpRN/CnwRAWD99veiLkFyTIEvIgA8sWRjVs9vbTx8s6w2LyFQ4ItIKNSkE38KfBEBct+tMg4H+P9+XxU/n7U86jIio8AXkVAUwhH+E0s3cftTxdtbSIEvInkRdRv+nv0HG6YXrdseYSXRUeCLCND4CH3S3xaxY8+B6IoJ2fKNOxl+w6yGx5f8/jl27zvYwjM6JgW+iDTz3Kqt/Gbu6xk9p7UWnfvnv8mMxRvaX1QWlr69s9m8426cxeadeyOoJjoKfBEJRVvueHX1Ay/loZLm0pV2+T0v5reQiIUS+GZ2gZmtMLNVZjYpxfLrzGypmS0ys7lmNiiM/YpIeJLbuAHuee4NNuzo2BdjLd/4Lis3vRt1GXmTdeCbWSlwOzAWGA5cZmbDm6z2MlDp7icAfwV+lu1+RSRcv5i9stm8C3/7bJufv3xjfIOzpe8ev0rxujuqMI7wRwOr3H2Nu+8HHgLGJ6/g7k+5e/1QefOB/iHsV0RCkq45Ztvu/aHvK4oeMlt2hT8YXCEKI/D7AW8lPV4XzEvnSmBmqgVmdpWZVZlZVU1NTQiliUjc3Ptcdd73OWVm+out/pnlkBKFJK8nbc3si0Al8PNUy919qrtXuntleXl5PksTKWr5vGjqby+vz9/O2uiNLbujLiEvwgj89cCApMf9g3mNmNk5wGTgEnfX9ysRiY2P/+LpqEvIizACfwEwzMwGm1knYAIwPXkFMzsJuItE2G8OYZ8iEqJ5K9WEWgyyDnx3PwhcA8wClgF/dvclZvZjM7skWO3nwGHAX8zsFTObnmZzIhKBr9y7ILvn/7/C788+6ZFFTH50cdRl5FRZGBtx9xnAjCbzbkiaPieM/YhI/tXVOSUlLQ+E89SKwv+G8NCCRN+Ta88exlE9ukRcTW7oSlsRadG9/6qOuoS8em7VlqhLyBkFvoi0aN07LV9te93Dr+Spkvz4zl9ejbqEnFHgi0hW4tjNUlJT4IuINFFXVwB3c2kHBb6ISBMH6uqiLiEnFPgiUpD2HqhlyszlzUb5DIPF4g684VPgi0i7TX/17cj2ff/8tdw5bzV3PF2896jNlAJfRNrtsQgD/739tQC8vb247lqVDQW+iLTo5bfeSbtsSYpbB7bFjveyv1/uL59IjGP/yEvrst5WsVDgi0iLXn4z/fj167e3745YL7+Z/o9IHKyI8c1csqHAFyly+w/mv0dKicX7pOjFv3+WA7Xvvy/rt79HxaTHuXXmsgiryp4CX6TIXdHOgdN27m1/s0zYgf/a+h2hbg/gvufXNvTHX7wusf275q1hdgHfMEWBL1Lknm3H2DH/fG0DJ9w0u937bGUstoy9syf8WzHe/NhShnx/BiNunMVLSU1QV01bmJOuoPmgwBeRjD23amt2G4h3i04ju/YdZOozaxrN+83c1yOqJjsKfBHJWLYtMnFvw2/N3qBLaKFR4ItIm2zfs58HX3gzlG0drC3ssWqsQP9gKfBFpE2+8+dX+f6ji1nydvYnSK9/ZFEIFUVn5msbuPqBhew9UFhH+gp8EWmTLbv2AYlunNke37a3/35cbNq5jxmLN/LrOSujLiUjCnwRaZP6RphCbc7IhbvmreFfqwvnDlkKfBFp1b6DjZsuZi3ZFFElCe7xOQfw+btfiLqENlPgi0irVm3exaLg4qP/mrOSjTujHbDs3X2N+8H/n4hvs/j1aQt5cvmm2N84RYEvIq1Kvh/I0ytqoisk0LRRacuu/bg7yzbsZEH1tpxceduSfy7ZyBX3VjFt/tq87jdTZVEXICLxVxejJpR07nmumpsfW9rwuHrKhXmvYd07e/K+z0zoCF9EWhW3wE/VctL0qD7fR/kAd//vG3nfZyYU+CLSqq27wh+rJhupTto++vL6Ro8v+t2z1Ly7L1YneKOmwBeRVm3bHa/Ab+u50ZNvmRP5CeY4UeCLSKsO1OVmzPzHFr1NxaTHubvJ4GStqc2gN8z0V/J7G8aHF4Qz/EQuKPBFpFW5GvvmmgdfBuCWGcv47J3Pp1zn3RTj7mfSTHMwz10lr39kcV73lwn10hGRVq3YFP4t/+atbNy988XqbQ3Tc5Zu4qv3VXHWMeXMW1nD6UN7c/9XP9KwPJMML/SB2sKkI3wRaVVYo2Qmm3jPi2mXPRQ0i9T/UWh6k5YHX2h7f/dCG+8mlxT4Inl2sLaOikmPUzHp8chq2LHnABWTHufPVW9FVkNLXl3XvEtl5U/mcKC2jtlLNvLbJ1dFUFXbzVmaGHpi74HaZsNSREmBL5KFrbv2NYR308vqD9bWsXDtO82eM3f55rTbq5j0OOf/+plGbdTuzi2PL+VAbR13zlvN/DXN7zb1yMJ1LFq3PeU2U7V3f/jHidsT/udf4zVM8eqaXby3v5aad/c1W7Zl1z6GTZ7JVdMWRlBZZpZu2AnAsT/8Jx/8wT8jruZ9FkYfVTO7APgNUAr8wd2nNFneGbgPGAVsBT7n7tUtbbOystKrqqqyrk0kDHsP1DLt+bV85bQKykrfP05KPkrvceghvHrjec2WTTh5AFMuPSHlc578zlkMKT8Md+e4G2exJ7iT0tfPOppJY48F4IL/eoblGxu3oVdPuZDd+w7ywhtbueLeqkbzIXFj8iO7deahBYkj+MU3nUe3LocAiT9EQyfPzOLdkNYM79udHe8daDQMdP3/zZ79B5n2/Fo+eVI/juzepWH5wwveZM/+Wn70P0u57dLj+dzJA9u1bzNb6O6VqZZlfYRvZqXA7cBYYDhwmZkNb7LalcA77j4U+DVwW7b7FcmnPz77BrfMWMbQyTNxd2Yv2Uj1lt2N1tnxXqI3yczFGxqF+kML3mp43PQA6+LfPQvAvf+qbgh7gDvnrebNrYnL9NdubX65/lvb9nDcjbMahT0kuivuPVDLk8s3N4Q9wPE3zW7Yt8I+95Zu2NlszP8DtXXcOnMZw2+Yxa0zlzP6p3MbfU6uf2QxP/qfpQ3TuZD1Eb6ZnQrc5O7nB4+/B+DutyatMytY53kzKwM2AuXews7be4S/bfd+zvrZUxk/T6QlTUdnTOeOL4zkGw+8lNG2zxveh9lLUw83fOxR3Zod3bdX5aCeVKVoYpL4+ejRR/Dgv5/Srue2dIQfRrfMfkDymZ91wEfSrePuB81sB3AE0OjUu5ldBVwFMHBg+77OdC4r4TOVA9r1XJGW3PNc+nFS7r/yI3zxjy80C/upXxqVss35xe+PYfRP5wI0CvvqKRfy8II3G47wmob98L7dG9qHk40dcRQzX9uYsrbvnncMv5i9slHY/8fZQ/nKaYM56eYn0r4mCcdvLzuJ//jvl9u8fmmJtTvsWxOrfvjuPhWYCokj/PZso2vnMm64uGmLkkj2brh4OAvXbuPSO96/QKi0xFj903Ep169vs/3Gx47mjqdXN1p2ZPcuzLnuLM751bxmz/vcyQObfaVPHvmxae+e+64YzelDezPk+zMa5n3hIwN54IU3Gd63O1eePoRfzG7cNfFzowfSs2unll6uhGDNT8c13Bqy3vPfO5u+PQ7lmZU1XJ7UNfXas4fynfM+mNN6wgj89UDyIXX/YF6qddYFTTo9SJy8FSkoowb1At5vhll1y9iGZb+ZcCLfeihxI44Hki4S2pLU4yQ5uI8u79po25PHfahh+mtnDeGueWuo+sE59D6sc9p6pl05mjOGlTdsu2LS45zzoT7c8snjueWTxzdb/41bx7G/to7OZaUA3PyJEfzw76+1/sLz6MQBh1NWYh2i+amkxBqdmF3903GUliRG8z/zmHKqp1zI5p176X1YZ0pKcn/ryDDa8MuAlcAYEsG+APi8uy9JWuebwPHu/nUzmwB8yt0/29J21UtHOoraOmf87c/y6NWncUhp434SH711Lm/v2JvR2O17D9Ry7A8TXf2yHfN92YadjP3N/2a1jbCt/MlYnl6xOW33y6O6d2FEvx7MWRbtbRZbc98VoznzmMQf4ze27OawzmWUd0v/xzssOW3DD9rkrwFmkeiWeY+7LzGzHwNV7j4d+CMwzcxWAduACdnuV6RQlJYYj117Rspl//remIy31+WQ0tBu7lES8Q3Jb7v0+GbNV53KSjjl6CNSrn9E1048+d2z6FJW2qgJK47qwx5gcO+uLayZP6G04bv7DGBGk3k3JE3vBT4Txr5EJDw9ux4S6f6bnq/4UN/uAHTv8n5dXztrCMs3vMu1Zw+lsqJX3mtsj+vOPSbqElKK1UlbEcmvI7t1aX2lHKn/lnJyRU9OG9qbE/r34Oxj+zRbb9IFx2IRfxPJxJDyrvzHmGFRl5GSAl9EIvWXr3805fwlPzofoKDCHuDBr+amS2UYNJaOiMRS185ldO3c9mPSi07om3L++BM/EFZJbXJUj+i+NbVGgS8iHUKXQ0pTzv/x+BF5q6H+W0lcKfBFJO+u+fjQ0LeZrod5j0Pzd2I6k28kUVDgi0hGzj+u+YnVTOXiGqP6a4o+NbJf+Btvg3HHHxXJfjOhwBeRNvleMFxzKEI+Edu/56ENbefp2vJz6c4vjuJ3l43M+34zFe/vHyISGwN7/Vto2wr7AP9TI/tzzceHMvwD3fn4B4/kzGPKeabJPXNz4fvjjuXyUyvSnj+IGwW+iLRJfRO5hRDXyVehhuHbY4ZRUmJcdEKiR84fLq/kmB/MbBi3Jmx3X17JKUN6NdxUplAo8EUkI2G0xowa1DP7jSRpOvDYIaWJx9/MwclhgHOHZ38eIwoKfBHpcMwstPGGPjzgcF596/37Bee7X3+YFPgi0iYh3P664Lxx6zjMjC279rFy07uMHNiz2YinhUSBLyIZictIBydX9GRBdW7GzJ9w8gCuHTOsYViH3od1bvG+BIWicP9UiUheDToi0Utn1KBedOsS/bHiiQMOD2U7X/5oRcN0/WidEz9aQb/DDw1l+3GiwBeRNhnRrwfP/N+Pc8VpFQ3hf9no6O4fff0FiesCbsrylqY3XXJcw/TMb51B9ZQLG4K/o4n+z7SIxN6im84DYGAQ9PXt+RNOHsjsJZvYunt/3msqKy3J+sTsjcEfi79/8zQ27tgbRlmxpiN8EWlV9yb9zesDv7TEqPrBORFUFI7LT60AEs1DF4yI/9AI2VLgi0jGkjvsmBkv/fDcyGrJRq4uzIorBb6IZKx+oLL6Hju9unaKsBppKwW+iLRbGMMsROVbMb0NYS4p8EUkY/Vt+HHpk98el47sH3UJeafAFylys759ZsbPcRo36UhhUOCLFLkPHtUt4+eUBEnfniadOdedlfFzcqH7ocXXK734XrGIZO2uL41i2vNrGXbkYRk/d2g7nhO233/+JA7/t+I70azAF5GMDTqiKz+4KLsrXKNUP25+sVGTjoi0qD1H8XEzol/HHCohUzrCF5EW3fWlUVGXEJq7L69k5MBwBl0rRDrCF5EWdT+0sG7j15I+3TtzRAcY5ri9FPgi0uAXn/kwE08d1GheWZENP9CRKfBFpMGnR/XnR+NHNBqFMszeLJ+rjG44ZSjsK4PDoMAXkZQ6lZUw5tgjQ93mZ0+OJvCHlidOPB8Wgxu3RCmrV29mvYCHgQqgGvisu7/TZJ0TgTuA7kAtcIu7P5zNfkUk91b+ZGxG6/fp3plNO/flqJrs3PqpE7h0VH8G9+4adSmRyvbP3SRgrrtPMbNJwePrm6yzB7jc3V83sw8AC81slrtvb7oxEYnGTz4xIuthEkrasIGoRtU8tFMpZwwrj2TfcZJt4I8HPhZM/wl4miaB7+4rk6bfNrPNQDmgwBeJiS+eMqj1lVrh3vo6xX6EHbVs2/D7uPuGYHoj0Kellc1sNNAJWJ1m+VVmVmVmVTU1NVmWJiL55LQh8SVSrR7hm9kcINW9vyYnP3B3N7O0/+Nm1heYBkx097pU67j7VGAqQGVlpT49IgWkLUf4Eq1WA9/d096w0sw2mVlfd98QBPrmNOt1Bx4HJrv7/HZXKyIF67FrT4+6hKKXbZPOdGBiMD0R+EfTFcysE/AocJ+7/zXL/YlITPVspb/+h/pqPJuoZRv4U4Bzzex14JzgMWZWaWZ/CNb5LHAm8GUzeyX4OTHL/YpIzHzz7KFRlyCtyKqXjrtvBcakmF8FfDWYvh+4P5v9iEj8dSrVdZxxp/8hEQlFebfiu6FIoVHgi0goRg3q1eLy4h7FJh4U+CKSFyUadTNyCnwRkSKhwBcRKRIKfBGRIqHAFxEpEgp8EZEiocAXESkSCnwRkSKhwBcRKRIKfBHJuY8MbvkqXMkPBb6ISJFQ4IuIFAkFvojk3OQLPxR1CYICX0Ty4IT+h0ddgqDAFxEpGgp8EcmpU4aoh05cKPBFJKdMtz6JDQW+iEiRUOCLiBQJBb6I5JSpRSc2FPgiIkVCgS8iUiQU+CISmjOG9W42T0068aHAF5HQnDRAV9TGmQJfRELjKeYN7t0173VIagp8EcmpSWM1cFpcKPBFJKcOKVUjflwo8EVEioQCX0RyqlOpYiYu9D8hIjll6pcZG1kFvpn1MrMnzOz14N+eLazb3czWmdnvs9mniIi0T7ZH+JOAue4+DJgbPE7nZuCZLPcnIjHmqfplSmxkG/jjgT8F038CPpFqJTMbBfQBZme5PxEpIBNPHRR1CZIk28Dv4+4bgumNJEK9ETMrAX4JfLe1jZnZVWZWZWZVNTU1WZYmIlG78eLjoi5BkpS1toKZzQGOSrFocvIDd3czS/WF7mpghruva+3kjbtPBaYCVFZW6suhSIFp+iteUqITtnHSauC7+znplpnZJjPr6+4bzKwvsDnFaqcCZ5jZ1cBhQCcz2+XuLbX3i4hIyFoN/FZMByYCU4J//9F0BXf/Qv20mX0ZqFTYi4jkX7Zt+FOAc83sdeCc4DFmVmlmf8i2OBERCU9Wge/uW919jLsPc/dz3H1bML/K3b+aYv173f2abPYpIvF18Yc/EHUJ0gJdaSsioTmmT7eoS5AWKPBFRIqEAl9EpEgo8EVEioQCX0Ry4rLRA6MuQZpQ4IuIFAkFvojkRLcu2V7XKWFT4ItITlx37jFRlyBNKPBFJCe6HFIadQnShL5ziUiobrhoOBt37o26DElBgS8iobri9MFRlyBpqElHRKRIKPBFRIqEAl9EpEgo8EVEioQCX0SkSCjwRUSKhAJfRKRIKPBFRIqEuXvUNaRkZjXA2iw20RvYElI5+aS686tQ64bCrV1159Ygdy9PtSC2gZ8tM6ty98qo68iU6s6vQq0bCrd21R0dNemIiBQJBb6ISJHoyIE/NeoC2kl151eh1g2FW7vqjkiHbcMXEZHGOvIRvoiIJFHgi4gUiQ4X+GZ2gZmtMLNVZjYpohoGmNlTZrbUzJaY2beC+TeZ2XozeyX4GZf0nO8FNa8ws/Nbez1mNtjMXgjmP2xmnUKqvdrMFgf1VQXzepnZE2b2evBvz2C+mdlvgxoWmdnIpO1MDNZ/3cwmJs0fFWx/VfBcC6nuDya9r6+Y2U4z+3Yc33Mzu8fMNpvZa0nzcv4ep9tHlnX/3MyWB7U9amaHB/MrzOy9pPf9zvbW19J7kGXtOf9smFnn4PGqYHlFprWHyt07zA9QCqwGhgCdgFeB4RHU0RcYGUx3A1YCw4GbgO+mWH94UGtnYHDwGkpbej3An4EJwfSdwDdCqr0a6N1k3s+AScH0JOC2YHocMBMw4BTghWB+L2BN8G/PYLpnsOzFYF0Lnjs2R5+DjcCgOL7nwJnASOC1fL7H6faRZd3nAWXB9G1JdVckr9dkOxnVl+49CKH2nH82gKuBO4PpCcDDYX/eM/npaEf4o4FV7r7G3fcDDwHj812Eu29w95eC6XeBZUC/Fp4yHnjI3fe5+xvAKhKvJeXrCY6Izgb+Gjz/T8AncvNqGur7U4p9jQfu84T5wOFm1hc4H3jC3be5+zvAE8AFwbLu7j7fE78B9+Wo7jHAandv6UrtyN5zd38G2Jainly/x+n20e663X22ux8MHs4H+re0jXbWl+49yKr2FoT52Uh+TX8FxtR/o4lCRwv8fsBbSY/X0XLQ5lzwFe4k4IVg1jXB19J7kr5Sp6s73fwjgO1Jv2hhvk4HZpvZQjO7KpjXx903BNMbgT7trLtfMN10ftgmAP+d9Dju7znk5z1Ot4+wXEHiSLzeYDN72czmmdkZwbz21JfL3+tcfzYanhMs3xGsH4mOFvixYmaHAY8A33b3ncAdwNHAicAG4JcRlpfO6e4+EhgLfNPMzkxeGByVxbYvb9B2egnwl2BWIbznjeTjPQ57H2Y2GTgIPBDM2gAMdPeTgOuAB82se1T1pVFwn41sdbTAXw8MSHrcP5iXd2Z2CImwf8Dd/wbg7pvcvdbd64C7SXxFhPR1p5u/lcTX2rIm87Pm7uuDfzcDjwY1bqr/Ch38u7mdda+n8Vf+XPz/jAVecvdNweuI/XseyMd7nG4fWTGzLwMXAV8IgpqgOWRrML2QRNv3Me2sLye/13n6bDQ8J1jeI1g/Eh0t8BcAw4Iz5p1IfLWfnu8igja6PwLL3P1XSfOT2x0/CdT3GJgOTAjO6A8GhpE4sZXy9QS/VE8Bnw6ePxH4Rwh1dzWzbvXTJE7IvRbUV98LJHlf04HLg14UpwA7gq/ks4DzzKxn8DX5PGBWsGynmZ0SvEeXh1F3E5eR1JwT9/c8ST7e43T7aDczuwD4T+ASd9+TNL/czEqD6SEk3t817awv3XuQbe35+Gwkv6ZPA0/W/1GMRK7PCuf7h8QZ/ZUkjigmR1TD6SS+ji4CXgl+xgHTgMXB/OlA36TnTA5qXkFSz5V0r4dET4EXSZxQ+gvQOYS6h5DoefAqsKR+fyTaHOcCrwNzgF7BfANuD2pbDFQmbeuKoLZVwFeS5leS+MVaDfye4GrvkN73riSOnnokzYvde07iD9IG4ACJ9t4r8/Eep9tHlnWvItFGXf85r++RcmnwGXoFeAm4uL31tfQeZFl7zj8bQJfg8apg+ZCw8yaTHw2tICJSJDpak46IiKShwBcRKRIKfBGRIqHAFxEpEgp8EZEiocAXESkSCnwRkSLx/wG06ZTIjMASggAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_c[1] = torch.squeeze(x_c[1], 1)\n",
    "plt.plot(torch.istft(x_c[1], n_fft=N_FFT, hop_length=HOP_LENGTH, normalized=True).view(-1).detach().cpu().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xgcJTi6sXwE1"
   },
   "source": [
    "With Noise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 153,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 283
    },
    "colab_type": "code",
    "id": "6xZ7N06evc7u",
    "outputId": "7670ecd6-d269-430e-ee8a-74513651cee4"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f04c3f23940>]"
      ]
     },
     "execution_count": 153,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAfaElEQVR4nO3deXxU9b3/8dcnYV9lExDBoIIKVgUjWhfkKihoC7bVirWKS/XW5Xdd2tvGH9VaUS+tv1av1qullutetS4VyyKLWtTKEhDZlYAgIEsEZBXC8v39MSdhEmYmmcyZOSc57+fjkUfmLHPOZyaT95z5nu/5jjnnEBGR+i8v6AJERCQ3FPgiIhGhwBcRiQgFvohIRCjwRUQiokHQBSTTvn17V1BQEHQZIiJ1ypw5c75yznVItCy0gV9QUEBxcXHQZYiI1ClmtirZMjXpiIhEhAJfRCQiFPgiIhGhwBcRiQgFvohIRCjwRUQiQoEvIhIRCnwR8dWqTTuZvGh90GVIAgp8EfHVuQ+9x43PzWHSwnVBlyJVKPBFJCt++vzcoEuQKhT4IuKbd5duDLoESUGBLyK+mbpkQ9AlSAoKfBGRiFDgi0jWvDlvLXv27Q+6DPEo8EUka257aR5/mPxZ0GWIR4EvIln1xeZdQZcgHgW+iGTVxIW6CCssFPgiIhGhwBcRiQgFvoj45oWZXwRdgqSgwBcRiQgFvohIRCjwRUQiQoEvIhIRCnwRkYjwJfDNbLCZfWpmJWZWlGK9H5iZM7NCP/YrIiI1l3Hgm1k+8DgwBOgFXGFmvRKs1xK4DZiZ6T5FpG75x/wvgy5B8OcIvx9Q4pxb4ZwrA14ChiVYbxTwW2C3D/sUkTpk6mKNkx8GfgR+F2B13PQab14FM+sLdHXOjU+1ITO70cyKzay4tLTUh9JERKRc1k/amlke8AfgZ9Wt65wb45wrdM4VdujQIduliYhEih+BvxboGjd9pDevXEvgROA9M1sJnAGM04lbEZHc8iPwZwM9zKy7mTUChgPjyhc657Y659o75wqccwXADGCoc67Yh32LSB1gZkGXIPgQ+M65fcCtwNvAEuAV59wiM7vPzIZmun0REfFHAz824pybAEyoMu+eJOsO8GOfIhIub32irpdhpyttRcQXL89enXSZGnTCQYEvIlk3f+3WoEsQFPgi4hOHS7qsZOOOHFYiySjwRcQXe/cnD3wJBwW+iEhEKPBFxB86wA89Bb6I+CJVG76EgwJfRCQiFPgi4gunA/zQU+CLiC+U9+GnwBcRiQgFvohIRCjwRcQXTo34oafAFxFfKO7DT4EvIhIRCnwRqXDybybzoz/PqNV91aITfgp8Eamw9Zu9/Gv5pqDLkCxR4IuIL6o7wC/ZuD0ndUhyCnwR8Uc1bTp/+WBlbuqQpBT4IuILNeGHnwJfRCQiFPgicojde/enfZ/qeumYvsk8cAp8EQFg6uINFbdLt+9J+/4aDz/8FPgiAsCaLbuyuv0wHODf8+ZCxn7wedBlBKZB0AWISP1QFy68evajVQBcd3b3gCsJho7wRQTIfi8bteEHT4EvIodY8dXOej36ZVSbdRT4IgJUbpIZMXYWr89dW+v7J2IBt+LPXHFwyIj7/rE4wEqCo8AXkYQWfbktrfXD/Hngo+WbuHxM5UHhXp79RUDVBEeBLyJA5oG9ZF3qN4i5X2xh0sJ1Ge6ldjZs233IvF++toCCovG8v6w0gIqCocAXEQBGZbmZY9GX2/jp83Ozuo9kUl0j8MKM6Bzp+xL4ZjbYzD41sxIzK0qw/E4zW2xm881smpkd5cd+RUQyNWnR+qBLyJmMA9/M8oHHgSFAL+AKM+tVZbWPgULn3EnAq8DvMt2viGRXSemOoEvwTT3ucJQWP47w+wElzrkVzrky4CVgWPwKzrl3nXPll/HNAI70Yb8ikkXTPyvNStfMhWu3+r5NqRk/Ar8LsDpueo03L5nrgYk+7FdEsuyl2aurXylN33nsA9+3WZ2yfQdyvs8wyulJWzP7MVAIPJRk+Y1mVmxmxaWl0TlzLhJWd72+IOgSfFFUzePYsy/90UHrIj8Cfy3QNW76SG9eJWY2EBgJDHXOJRyKzzk3xjlX6Jwr7NChgw+liUhN1OeramviuF9NCrqEnPAj8GcDPcysu5k1AoYD4+JXMLM+wJ+Ihf1GH/YpIiFSm/HzJfcyDnzn3D7gVuBtYAnwinNukZndZ2ZDvdUeAloAfzOzeWY2LsnmRCQAezJs415QD07EFhSN5+kP6/cYO74Mj+ycmwBMqDLvnrjbA/3Yj4hkx/F3Z9akcdmTH/lUSbDufWsx15xVf4dO1pW2IpJS1Nr3vymrv81TCnwRSenFWdEZegDghHsmceBA/XyTU+CLSErLN+5MuXxX2b4cVZI7++vppxoFvohkpNc9bwddgu/y6unXcynwRUSqqJ9xr8AXkQys2bKr+pUS2LnHn2agotfm84fJn/qyrXj19ABfgS8iqaUKv/veqt0Y+sWrttSymoO27CzjpdmrefSdkoy3VZXV08RX4ItInfTI1M+CLqHOUeCLSEoflnwVdAkJPfPRqqxtu74OFaHAF5GUlq7fnnTZ5MUbclhJ7hx/9yT27Y8NN7F999560y/fl6EVRETqm2NHTuSmAcfwxHvLuWnAMezdd4CnPvicl288g9OPbhd0ebWiI3yRiKvtidcoeOK95RW/n/ogNrDa5WNmBFlSRhT4IhE3toYjRG7asYdX56zJcjWSTWrSEZEaueXFucxYsZnTu7ela9tmGW3L706PBw448vLqZ1dKP+kIX0RqZOO22BfVle0/wNZde/nZK5/Uelt+nwKd8fkmn7eY2rbde3O6P78o8EUkLc7Bw1M/47W54WneyfVYZz9/5ZM6OWy0Al9EaqaixcSxP8Nuiss2JO/qWRu5bsyZvHgD1z09m1mfb87xnjOjwBeRGvEzVO8fv8THrQXj3U9L+eGfPmJKHboWQYEvImn58uvd7KyHY+DX1g3PFjNv9dcUFI0P/RG/Al9EamR5aeyLUK4eO4vX564NtJaq7eeLvtwWUCUxlzz+IQDvfrox0Dqqo8AXkTpn886yStMPTFjCF5t28eCEJYGeTP1q+57A9l0TCnwRqZZf49f7pUHeodF10wtzGDN9BUvXb+eT1V9z47PFFePh5Er7lo1zur906cIrEanW+m27gy6hEpegJ395s86K0p3c8uJcAFZv+Ybu7ZvnrK6wX/qlI3wRqdbj7/r/JSOZSNVqUx72kPtPJmH/3hQFvohUK+iTtFUdqGE7/fbd+3I6tPFHy3N7xW+6FPgiGdh/wHH33xdyy4tz6+2XZoRRTSN89MQl7Mth4M/94uuc7as2FPgiGRj1j8U8N2MV4+ev472AuuRt372X+WvSC5qN23czZvryLFVUc4+/W0JB0XiOvms8/0rwzVrOOW5+YQ5L12+rMr9m2/9kzVb2HcjtidswU+CLZODpf62suP3u0tIa32/8/HUJhxfYuG0381anF95nPDiNoX/8kML7p9T4Pv0emMaDE5ZSUDQ+rX35afXmXTz09qcAHHDwo6dmVizbs28/f/94LfeOW8SEBesZ/Mj7lYZzqGmTDsDKr3b5V3QNjJ64NKf7S4cCX8QnLxevrjR93dOzKSgaz9ZvDh1Z8ZYX5zLo4ems2rSz0vx+D07jksc/ZGOVXjEL125N2L/8uRmr2FkWa0r6akcZC9duPWSdLVX6rIfFOb97N+myK8bM4PaX51X63tor/jyDgqLxrPxqJ+u21rzX0EWPvp9Rnel68p/Lmf5Zzd/8c0mBL1JLqYJ0V9k+3lkaa+IZMXZWpXXjg/vch95jr9dXfOaKgyf8+j04DYCSjTs4+TeT+c5jH3DzCwd7n5S7++8LK01v3F45CD9Z/TV9Rk3hzXnhOumayvbdexO2hZcPWzDg/71XcWVrWK3ekvhThXOOrbtiBwA79lQ+obzoy62s2/oNS9Ztq3hN+E398EVqqc+o5E0ou8oOnsCdt/prrho7k7duPRszY9s3lbsK9hg5kf8efgq3vTSv0vxpSzZw/TPFFdPxB/gTF6zjpgRvANc9XczK0RcDsHf/Aa70mkkee6eEYad0wTlH97sm1PxB5tikheu4/eV51a8YciPfWMjIN2Jvxg3zjXn3XECzRvn818SljJm+omK9Qb068uerCwG4+NEPKuZfc2YB9w7t7XtdOsIX8dEHy2InHtdu+abS/IVrt1UE7cn3TT7kfn8rPnRs+fiwByhedXBgrkRhH2/rN3vpMXIiO7x+6CUbd+CcC90FVFX99Pm57N5bv06y7t3v6P3rt/nrrNWVwh5gyuINzF/zNTc8W/lvPfeLLVmpxZcjfDMbDPw3kA885ZwbXWV5Y+BZ4FRgE3C5c26lH/sWCZMf/2UmQ07sxMSF6xMuT3aS9IMEPVSq+mpHGVt2ltGscX616w55ZPoh8/7jpXm89cmX1d5XsuP/vrEg4fyhfzy0eSpbwwFlfIRvZvnA48AQoBdwhZn1qrLa9cAW59yxwMPAbzPdr0gufVO2n7teX1BxAnbakuRjoCcLez/0GTWF4341KeU6m3eW8WWCk5oK+7pjQYKT737w4wi/H1DinFsBYGYvAcOAxXHrDAPu9W6/CvzRzMxlYVi7HXv28YBvX67gX3l+PlJftxXCx+jni8Kvmsq/zu+vs75gcO9OTFqUvVDP1NVjZ1a/kkSSH4HfBYjvj7YGOD3ZOs65fWa2FWgHVPoca2Y3AjcCdOvWrVbF7N13gKkpjr7S5efQGH6Os2E+VeZvTT5uK8SDkoQx7M/p0Z73vfMHC9cGOza8hFeoeuk458YAYwAKCwtrdWzWpnkjZo8c6GtdIve9tZixH34edBlJ/WXEafT81cSgy5CQ86OXzlqga9z0kd68hOuYWQOgNbGTtyJ1wuWnda1+pTSdd/zhCef36twq7W01aqAOd2H03ZOPCLqESvx4lcwGephZdzNrBAwHxlVZZxwwwrt9KfBONtrvRbLluE4tUy4fdkrif+xX/v3bSe/zxI/7Jpw/4bZzKk1fe1ZByu3MuOt8AJo2rL73TrnbB/bg/V/8W43Xl9p5dPgpfPDLmj/P5W/2Kx68KCv1ZBz4zrl9wK3A28AS4BXn3CIzu8/Mhnqr/QVoZ2YlwJ1AUab7FQmLefcM4pHLT+Gn5x5zyLLjOyd/o2jcIJ+LT+pcaV55CB8e981Jg07oSL/ubRNuY9QlJ9KpdRMApv7s3ITr3Dmo5yHzbh/Yk65tm4V2/PaJVd704i0dNZg7Bh76mMLIzOhyWNOK6X8/92jm3TOIRb+5sNJ6n90/hJWjL2bCbeewcvTF5OVl5w/jy+dA59wE51xP59wxzrkHvHn3OOfGebd3O+cuc84d65zrV96jR6Quad4o8RH0Yc0aYWYUDTme7/XpAsDQk4/g9oE9aNWkIcseGMKYq07l2ev6AfD89afzxs1nAvD4jyof5Xdt2wyAWSMH8uINp9O0YT69u7QG4I2bz+RfRedVWn9B3CiZ8cESr2+3NpWm47dx/yUnpn7QATkhRbNWk4b5XHd2Qe6KqaXv9429FuI7INwxsCeHNWtE88YHT5/e+91eOWuSC9VJW5Ewm3P3IKYt2cjoSUsYcmJnxkxfwfAqbfv/9f1v0feoNlzZr1vFUVrD/Dwu6N0JoGLYg5o485j2LBk1uGK6jxfcK0dfXHEB14XedssN6tWRKYtjvdT6FbRl1srNlB8s9j6iFdee1Z0j4t4YUgVrLhzfqSVL1x8cNbRd80a8n6IJZLnX1NGyScOs15aJC3t35PeXnVwx/dz1/WjRuAFN4prdXrvp2/zgiY/4Xt8jc1aXAl+khpo0jDXBXHxSZw4ccHRo0Zgfnd7tkHWuOuOotLb78OUnc8fLn6R1nz7dDuPjL77mzGPaV5r/56sLmbNqMyd0bsW6rbv53aSlnFrQJq03mlx6/ienc8HD0ynbd4D5v74gaVNG2+aN2LyzjPwsNXX47YkrT610ZH9Ojw6HrHPqUW1z/ndR4IvUQl6ecUP/o33Z1iWndKFpw3z6HtWm+pU9b9x8VtJlpx4Va+8/pkML/nRVYcrttGgcXATcPrAH7Vs0Zu7dgxIuf+PmM3nsnRL+58q+bNlVVumTQJi1b9E4a23wmVJfLpGAmRmDT+zM4S2b5HzfPTum7n2UTbdXc+K1T7c2jL3mNJo0zKdz66b823GJu7GW69/z0KNoONgMlCuv3ZS8R1XQFPgikpZkJ4fT8bMEPYcy1a55o4Tzc90MdFS75jndXzoU+CKSlhNSdDWtqbA2edR3asMXkTQprBP5nyv7cm6SZqWw0BG+iKTFj4u1Gvh8hD/nV5XHz3rGu+Yhl/r37FCpf30YKfBFJC1+RPXV3y7wYSsHtWvRmGbehXGPXdGHM49pB+S+/T7sFPgiUiOjhsW+Y9WPEG2a5KrlTBQNOZ47Bvbkom91rnhTytaQXUVDjmfZA0NY9JsL6dvtMADywzpORZxwf/4QkdBo27xx9SsFqGWThtw2sAcAB7yvxa06rIQfLjv1yIpxkxrm5/G/1/bj0/Xbs/Im5jcd4YtInXT92d2TLsvLM/7xf87mf689zff93v+9yuMPtW7aMOngdmGjwBeRtISl5eLu71T96uzKTuzS2vcxd/5VdB6NG4T/SD4ZBb6I1Mixh7cAEo8LU5+VX2j2z/8cUGngubpIgS8iNXJcp5Z8fPcghp/Wlc7eGPwN80NyuO+jmwcc/F6D4ad15Z//OYDJd/QP9RW0NaXAF5Eaa9M8NvZ/+xaxE7i3nd8j0Hp6dW6V0Zj+Vc8DdG3blCv6dWPFgxcx9c7+jP7BSTTIzwt0zCE/KfBFJG2OWHfHZAOW5cqE287hx2kORx2vReMGXNi7Y8X0+784j65tm5GXZxx7eP0I+XgKfBGp1iOXn1Jpurx7e55Z6L6oO11/uqqQi7/VmbZJBl+rT9QPX0SqdYn31Y3l4q9neujSkxh68hHc8GxxjqvKXPnDePzKxF8oX9/oCF9E0lYelGaxb/ka1KtjyvUlHBT4IpK28iELrBYj60y+o7/f5dRa+bAIUaEmHRGptdpchBWWHi9v3nIWJ3eNVuDrCF8k4h669KS073Og/Ai/DnfDD8sbTy4p8EUi7rLCrrW+b22adMLgiSv71onBzvymwBeRtA3wvlC8TfODY9W0aqIW4rDTX0hE0vbLwcdz3VndObxlk6BLqZXWTf0dVK2uUOCLSNry84xOrSuHvYW4Qb/3Ea1Y9OU2nr72NHaV7efMY9sHXVIgFPgiktLPBvUMuoSMlb8XtWvemAHHtQ62mACpDV9EUrq08MigS8hYlr7psM7REb6IVJh8R39Kt++hU+smnP/7f6Z13xC36IhHgS8iFXp2bHlI//S62vUynt6MYtSkIyIp1TQsa7LafcN6Z1RLbTVrFDu2zYt44mX08M2srZlNMbNl3u9DviLezE4xs4/MbJGZzTezyzPZp4jkVksf+9cP7t3Jt22l47Er+vDzC3rSq3OrQPYfFpm+3xUB05xzPYBp3nRVu4CrnXO9gcHAI2YWrQEsROqgwb07cdOAYyqOjqtTk26Zh7cKpt9+x1ZNuPW8HqHuOpoLmb51DwMGeLefAd4Dfhm/gnPus7jbX5rZRqAD8HWG+xaRLHryqlODLkF8lukRfkfn3Drv9nog5aDYZtYPaAQsT7L8RjMrNrPi0tLSDEsTkVyK9rFz3VDtEb6ZTQUSNbyNjJ9wzjkzS9rb1cw6A88BI5xzBxKt45wbA4wBKCwsVM9ZkRwZ/f1vqSdLBFQb+M65gcmWmdkGM+vsnFvnBfrGJOu1AsYDI51zM2pdrYhkxfB+3TLeRnVvGB/fPSjjfUhmMm3SGQeM8G6PAN6suoKZNQLeAJ51zr2a4f5EJKSqjq1TVauIDlgWJpkG/mhgkJktAwZ605hZoZk95a3zQ6A/cI2ZzfN+TslwvyISMg//UP/WYZdRLx3n3Cbg/ATzi4GfeLefB57PZD8iEn49IvgNUnVNxK87ExGJDgW+iOREfp66AQVNgS8iEhEKfBGRiFDgi4hEhAJfRCQiFPgiIhGhwBcRiQgFvohIRCjwRUQiQoEvIhIRCnwRkYhQ4ItI1g08IeWX4UmOKPBFJOvuv+TEoEsQFPgikgOtmmY0Erv4RIEvIlnXpEF+0CUICnwRybIzjm5LnoZGDgUFvohIRCjwRSSrDB3dh4UCX0QkIhT4IiIRocAXkawyteiEhgJfRCQiFPgi4pvvnnxE0CVICgp8EfFN93bNDpmnJp3wUOCLiESEAl9EfOMSzPv1d3vnvA5JTIEvIll1VIJmHgmGAl9EJCIU+CIiEaHAF5GsaqyhkUMjo8A3s7ZmNsXMlnm/26RYt5WZrTGzP2ayTxERqZ1Mj/CLgGnOuR7ANG86mVHA9Az3JyIitZRp4A8DnvFuPwNckmglMzsV6AhMznB/IiJSS5kGfkfn3Drv9npioV6JmeUBvwd+Xt3GzOxGMys2s+LS0tIMSxMRkXjVfrOwmU0FOiVYNDJ+wjnnzCzRdRc3AxOcc2usmmusnXNjgDEAhYWFibYlInVI4wbqFxIm1Qa+c25gsmVmtsHMOjvn1plZZ2BjgtW+DZxjZjcDLYBGZrbDOZeqvV9E6oHZv0oaHxKAagO/GuOAEcBo7/ebVVdwzl1ZftvMrgEKFfYi9VOj/MpH9K2aNAyoEkkk089bo4FBZrYMGOhNY2aFZvZUpsWJSN1yQ/+jgy5BUsjoCN85twk4P8H8YuAnCeY/DTydyT5FJLyaNNRFVmGmMyoiIhGhwBcRiQgFvohIRCjwRSQrrujXLegSpAoFvohIRCjwRUQiQoEvIlnRIC/1UCqSewp8EcmKn194XNAlSBUKfBHJitZNNaxC2CjwRUQiItPB00REKvmP844lP0/HkmGkwBcRX915gdruw0pvwyIiEaHAFxGJCAW+iEhEKPBFRCJCgS8iEhEKfBGRiFDgi4hEhAJfRCQizDkXdA0JmVkpsCqDTbQHvvKpnFxS3blVV+uGulu76s6uo5xzHRItCG3gZ8rMip1zhUHXkS7VnVt1tW6ou7Wr7uCoSUdEJCIU+CIiEVGfA39M0AXUkurOrbpaN9Td2lV3QOptG76IiFRWn4/wRUQkjgJfRCQi6l3gm9lgM/vUzErMrCigGrqa2btmttjMFpnZbd78e81srZnN834uirvPXV7Nn5rZhdU9HjPrbmYzvfkvm1kjn2pfaWYLvPqKvXltzWyKmS3zfrfx5puZPerVMN/M+sZtZ4S3/jIzGxE3/1Rv+yXefc2nuo+Le17nmdk2M7s9jM+5mY01s41mtjBuXtaf42T7yLDuh8xsqVfbG2Z2mDe/wMy+iXven6xtfamegwxrz/prw8wae9Ml3vKCdGv3lXOu3vwA+cBy4GigEfAJ0CuAOjoDfb3bLYHPgF7AvcDPE6zfy6u1MdDdewz5qR4P8Aow3Lv9JHCTT7WvBNpXmfc7oMi7XQT81rt9ETARMOAMYKY3vy2wwvvdxrvdxls2y1vXvPsOydLrYD1wVBifc6A/0BdYmMvnONk+Mqz7AqCBd/u3cXUXxK9XZTtp1ZfsOfCh9qy/NoCbgSe928OBl/1+vafzU9+O8PsBJc65Fc65MuAlYFiui3DOrXPOzfVubweWAF1S3GUY8JJzbo9z7nOghNhjSfh4vCOi84BXvfs/A1ySnUdTUd8zCfY1DHjWxcwADjOzzsCFwBTn3Gbn3BZgCjDYW9bKOTfDxf4Dns1S3ecDy51zqa7UDuw5d85NBzYnqCfbz3GyfdS6bufcZOfcPm9yBnBkqm3Usr5kz0FGtafg52sj/jG9Cpxf/okmCPUt8LsAq+Om15A6aLPO+wjXB5jpzbrV+1g6Nu4jdbK6k81vB3wd94/m5+N0wGQzm2NmN3rzOjrn1nm31wMda1l3F+921fl+Gw78NW467M855OY5TrYPv1xH7Ei8XHcz+9jM/mlm53jzalNfNv+vs/3aqLiPt3yrt34g6lvgh4qZtQBeA253zm0DngCOAU4B1gG/D7C8ZM52zvUFhgC3mFn/+IXeUVlo+/J6badDgb95s+rCc15JLp5jv/dhZiOBfcAL3qx1QDfnXB/gTuBFM2sVVH1J1LnXRqbqW+CvBbrGTR/pzcs5M2tILOxfcM69DuCc2+Cc2++cOwD8mdhHREhed7L5m4h9rG1QZX7GnHNrvd8bgTe8GjeUf4T2fm+sZd1rqfyRPxt/nyHAXOfcBu9xhP459+TiOU62j4yY2TXAd4ArvaDGaw7Z5N2eQ6ztu2ct68vK/3WOXhsV9/GWt/bWD0R9C/zZQA/vjHkjYh/tx+W6CK+N7i/AEufcH+Lmx7c7fg8o7zEwDhjundHvDvQgdmIr4ePx/qneBS717j8CeNOHupubWcvy28ROyC306ivvBRK/r3HA1V4vijOArd5H8reBC8ysjfcx+QLgbW/ZNjM7w3uOrvaj7iquIK45J+zPeZxcPMfJ9lFrZjYY+AUw1Dm3K25+BzPL924fTez5XVHL+pI9B5nWnovXRvxjuhR4p/xNMRDZPiuc6x9iZ/Q/I3ZEMTKgGs4m9nF0PjDP+7kIeA5Y4M0fB3SOu89Ir+ZPieu5kuzxEOspMIvYCaW/AY19qPtoYj0PPgEWle+PWJvjNGAZMBVo68034HGvtgVAYdy2rvNqKwGujZtfSOwfaznwR7yrvX163psTO3pqHTcvdM85sTekdcBeYu291+fiOU62jwzrLiHWRl3+Oi/vkfID7zU0D5gLfLe29aV6DjKsPeuvDaCJN13iLT/a77xJ50dDK4iIRER9a9IREZEkFPgiIhGhwBcRiQgFvohIRCjwRUQiQoEvIhIRCnwRkYj4/+yQgUtycHuKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_n[1] = torch.squeeze(x_n[1], 1)\n",
    "plt.plot(torch.istft(x_n[1], n_fft=N_FFT, hop_length=HOP_LENGTH, normalized=True).view(-1).detach().cpu().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "zFdga4o-X-yo"
   },
   "source": [
    "In General, it can be seen that the cleared signal is close to the purified one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qi7uvNVWqsvO"
   },
   "source": [
    "## Conclusions ##\n",
    "\n",
    "We conducted an experiment: we implemented the Deep Complex UNet architecture, trained it, and tested it.\n",
    "\n",
    "The obtained minimum loss value on validation: -0.976848\n",
    "The resulting PESQ metric value: 2.818076\n",
    "\n",
    "The value of the PESQ metric on a 20-layer network from the article: 3.24\n",
    "\n",
    "We were able to get close enough to the results of the study described in [the article](https://openreview.net/pdf?id=SkeRTsAcYm). But the values still turned out a little worse for the following reasons\n",
    "\n",
    "1.The possile minimum size of network: 10 layers instead of 20.\n",
    "\n",
    "2. A small number of epochs. 3 epochs is too little, but the results are enough for us.\n",
    "\n",
    "\n",
    "## The prospects ##\n",
    "* You can implement a deeper variation of Deep Complex U-Net using the same principle\n",
    "* The use of more powerful in terms of computing hardware(more memory) and increase the size of the batch to increase the speed of learning\n",
    "*Training on a larger number of epochs.\n",
    "* Augmentation of a dataset to increase its size by adding synthetic noise\n",
    "\n",
    "## In practice ##\n",
    "This technology can be used to improve the sound of phone calls, in the same voice messages in messengers and social networks, for professional video and audio processing.  "
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "machine_shape": "hm",
   "name": "dcunet.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}