16_Autoencoder/autoencoder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AutoEncoder\n",
    "\n",
    "![autoencoder](./images/autoencoder2.png)\n",
    "\n",
    "지도학습은 단순히 말해 입력 x와 정답 y 사이의 관계를 찾는 것이고, 비지도 학습은 정답이 없는 채로 x를 예측하는 것이다.\n",
    "\n",
    "즉 오토인코더에서는 입력도 x, 정답도 x로 설계된 신경망이다.\n",
    "\n",
    "단, 신경망은 범용근사자(univeral function approximator)로서 근사치를 출력하므로 x와 똑같은 출력을 내긴 힘들다.\n",
    "\n",
    "그러므로 오찻값에도 x를 얼마나 복원했는지를 뜻하는 복원오차, 혹은 정보손실값(reconstruction loss)이라는 용어를 사용한다.\n",
    "\n",
    "오토인코더는 입력과 출력의 크기는 같지만 중간으로 갈수록 신경망의 차원이 줄어든다.\n",
    "\n",
    "작은 차원으로 압축된 표현을 잠재변수(latent variable)라 하고, 간단히 z라고 한다.\n",
    "\n",
    "잠재 변수의 앞뒤를 구분하여 앞부분을 인코더(encoder), 뒷부분을 디코더(decoder)라고 한다.\n",
    "\n",
    "인코더는 정보를 받아 압축하고, 디코더는 압축된 표현을 풀어 입력을 복원하는 역할을 한다.\n",
    "\n",
    "오토인코더에서는 필연적으로 정보의 손실이 일어나지만 이는 중요한 정보만 남겨두는 일종의 데이터 가공이라고 볼 수 있다. \n",
    "\n",
    "이런 특징으로 인해 오토인코더는 주로 복잡한 비선형 데이터의 차원을 줄이는 용도로 쓰인다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\pinb\\AppData\\Local\\Packages\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\LocalCache\\local-packages\\Python310\\site-packages\\tqdm\\auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torchvision\n",
    "import torch.nn.functional as F\n",
    "from torch import nn, optim\n",
    "from torchvision import transforms, datasets\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D #생성되는 이미지를 관찰하기 위함입니다. 3차원 플롯을 그리는 용도입니다.\n",
    "from matplotlib import cm # 데이터포인트에 색상을 입히는 것에 사용됩니다.\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hyper Params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using Device: cuda\n"
     ]
    }
   ],
   "source": [
    "# 하이퍼파라미터 준비\n",
    "EPOCH = 10\n",
    "BATCH_SIZE = 64\n",
    "USE_CUDA = torch.cuda.is_available()\n",
    "DEVICE = torch.device(\"cuda\" if USE_CUDA else \"cpu\")\n",
    "print(\"Using Device:\", DEVICE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Dataset (Fashion-MNIST)\n",
    "\n",
    "![fation-mnist](./images/fashionmnist.jpg)\n",
    "* Fashion-MNIST는 총 70,000개의 그레이스케일 이미지로 구성된다. \n",
    "* 60,000개는 학습용 데이터, 10,000개는 테스트용 데이터 \n",
    "* 데이터셋은 10가지 범주로 구성 ('티셔츠/탑', '바지', '풀오버', '드레스', '코트', '샌들', '셔츠', '스니커즈', '가방', '앵클 부츠' 등)\n",
    "* 각 이미지는 28x28x1 크기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ./.data/FashionMNIST\\raw\\train-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 26421880/26421880 [00:06<00:00, 4076925.90it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./.data/FashionMNIST\\raw\\train-images-idx3-ubyte.gz to ./.data/FashionMNIST\\raw\n",
      "\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to ./.data/FashionMNIST\\raw\\train-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 29515/29515 [00:00<00:00, 118201.02it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./.data/FashionMNIST\\raw\\train-labels-idx1-ubyte.gz to ./.data/FashionMNIST\\raw\n",
      "\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to ./.data/FashionMNIST\\raw\\t10k-images-idx3-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 4422102/4422102 [00:03<00:00, 1360040.16it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./.data/FashionMNIST\\raw\\t10k-images-idx3-ubyte.gz to ./.data/FashionMNIST\\raw\n",
      "\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz\n",
      "Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to ./.data/FashionMNIST\\raw\\t10k-labels-idx1-ubyte.gz\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 5148/5148 [00:00<?, ?it/s]"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./.data/FashionMNIST\\raw\\t10k-labels-idx1-ubyte.gz to ./.data/FashionMNIST\\raw\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "# Fashion MNIST 데이터셋 불러오기 (학습데이터만 사용)\n",
    "trainset = datasets.FashionMNIST(\n",
    "    root      = './.data/', \n",
    "    train     = True,\n",
    "    download  = True,\n",
    "    transform = transforms.ToTensor()\n",
    ")\n",
    "train_loader = torch.utils.data.DataLoader(\n",
    "    dataset     = trainset,\n",
    "    batch_size  = BATCH_SIZE,\n",
    "    shuffle     = True,\n",
    "    num_workers = 2\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Autoencoder Network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 오토인코더 모듈 정의\n",
    "class Autoencoder(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Autoencoder, self).__init__()\n",
    "\n",
    "        #인코더는 간단한 신경망으로 분류모델처럼 생겼습니다.\n",
    "        self.encoder = nn.Sequential( # nn.Sequential을 사용해 encoder와 decoder 두 모듈로 묶어줍니다.\n",
    "            nn.Linear(28*28, 128), #차원을 28*28에서 점차 줄여나갑니다.\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(128, 64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, 12),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(12, 3),   # 입력의 특징을 3차원으로 압축합니다 (출력값이 바로 잠재변수가 됩니다.)\n",
    "        )\n",
    "        self.decoder = nn.Sequential(\n",
    "            nn.Linear(3, 12), #디코더는 차원을 점차 28*28로 복원합니다.\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(12, 64),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(64, 128),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(128, 28*28),\n",
    "            nn.Sigmoid(),       # 픽셀당 0과 1 사이로 값을 출력하는 sigmoid()함수를 추가합니다.\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        encoded = self.encoder(x) # encoder는 encoded라는 잠재변수를 만들고\n",
    "        decoded = self.decoder(encoded) # decoder를 통해 decoded라는 복원이미지를 만듭니다.\n",
    "        return encoded, decoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "autoencoder = Autoencoder().to(DEVICE)\n",
    "optimizer = torch.optim.Adam(autoencoder.parameters(), lr=0.005) \n",
    "# Adam()을 최적화함수로 사용합니다. Adam은 SGD의 변형함수이며 학습중인 기울기를 참고하여 학습 속도를 자동으로 변화시킵니다.\n",
    "criterion = nn.MSELoss() #원본값과 디코더에서 나온 값의 차이를 계산하기 위해 평균제곱오차(Mean Squared Loss) 오차함수를 사용합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Normalize"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 원본 이미지를 시각화 하기 (첫번째 열)\n",
    "view_data = trainset.data[:5].view(-1, 28*28)\n",
    "# 복원이 어떻게 되는지 관찰하기 위해 5개의 이미지를 가져와 바로 넣어보겠습니다.\n",
    "view_data = view_data.type(torch.FloatTensor)/255.\n",
    "#픽셀의 색상값이 0~255이므로 모델이 인식하는 0부터 1사이의 값으로 만들기 위해 255로 나눠줍니다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 학습하기 위한 함수\n",
    "def train(autoencoder, train_loader):\n",
    "    autoencoder.train()\n",
    "    for step, (x, label) in enumerate(train_loader):\n",
    "        x = x.view(-1, 28*28).to(DEVICE)\n",
    "        y = x.view(-1, 28*28).to(DEVICE) #x(입력)와 y(대상 레이블)모두 원본이미지(x)인 것을 주의해야 합니다.\n",
    "        label = label.to(DEVICE)\n",
    "\n",
    "        encoded, decoded = autoencoder(x)\n",
    "\n",
    "        loss = criterion(decoded, y) # decoded와 원본이미지(y) 사이의 평균제곱오차를 구합니다\n",
    "        optimizer.zero_grad() #기울기에 대한 정보를 초기화합니다.\n",
    "        loss.backward() # 기울기를 구합니다.\n",
    "        optimizer.step() #최적화를 진행합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 1]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5VElEQVR4nO192W5k2ZXdinkeGJyZzKlKpcoqlSyVSkZLLXXDrTaEfvBL237zB/gX/Cf+Aj8ZsGHAgJ4Mww0DsqVu1aBWSZVdlVWVzIFMDjHPkx8S63DF5gkmmYwIMtJ3AQTJiBs37t33nD2svc8+ofF4PEaAAAECBAgwZ4Sv+wICBAgQIMD/HwgMToAAAQIEWAgCgxMgQIAAARaCwOAECBAgQICFIDA4AQIECBBgIQgMToAAAQIEWAgCgxMgQIAAARaC6Ot+cDQa4dmzZ8jlcgiFQrO8pqXDeDxGvV7Hzs4OwuGL2fBAfqcI5Hc1BPK7Gl5HfkAgQ+JS8hu/Jvb29sYAgh/52dvbC+QXyC+Q35L+XEZ+gQxfT36vHeHkcrnX/egbi8vIZJbyC4VCCIfDE7/T6TQ2NjaQy+Xwr/7Vv8JHH32EJ0+e4Pe//z1arRaePHmCVquF4XCI4XCI8XiM0WiEcDiMzc1N5PN53L59G++++y7G4zGOj4/RarXwm9/8Bp988gn6/T7a7TZGoxFGo5H7PP9+HVyX/BSZTAaZTAabm5v4yU9+gmQyiU8++QTffPMN2u02Go0GRqPRhc5VKpWwvb2NRCKBtbU1JBIJ/OlPf8JXX32FwWCAwWAw02tfpPw41iKRCKLRKEqlEpLJJHZ3d3Hr1i2srq7igw8+QDqdRqvVQq/Xw97eHj755BM0m008ffoUzWYTKysrWFlZwdraGr7//e+jWCzirbfewvr6OobDIXq9Hmq1Gv7hH/4Bh4eH+Oyzz/CHP/zBjbPxeDwzOV5WJrMeg9Fo1M1fgnNpMBhceNwBQCwWQzgcRjqdRi6Xw3A4RLVaRa/Xu/S5LoqLyOO1Dc7/zyHkNFxGJrOQXyqVQrFYRCqVwp07d5DP57G5uYmtrS1Eo1Gk02kkEgm8++672NnZwfr6Ou7evYvBYIBer+cMjf6EQiFkMhkkEgnEYjHE43GMx2O0Wi30+308ePAAf/u3f4ter4dGo4Fer4f9/X3UajU8ffoUX375JbrdLiqVCvr9/qXuZ9HyU3CCbm1t4c6dO7h16xb+7M/+DLlcDt/5zndQLpfR6XRQr9cnDKsaaioM/uTzeaytrWE0GuHk5ASdTgcAkEwmUa1W8fjxY/R6vZndwyLkR4ekUChga2sL77zzjhsrkUgE+XwehUIByWQSKysriEajyGQyAID19XW8//77GI1G6Pf7TumNx2OkUimsrq4iHo8jmUwiHo+j3+9jOBwinU7jwYMHuHfvHr773e/il7/8JVqtFg4ODtBoNPDxxx/j6dOnr3U/V5HJLMdgPp/HX/3VX+H27duIxWKIxWLO4Ha7Xfzud7/Do0eP0O120Wq1vE5dJBJBKpVCJpPBX/7lX+Ldd99FIpFAKpXCYDBAuVxGu93Gb37zG3z88ccYDoczdXouIo/XNjgBrh+JRAKrq6soFov44Q9/iO3tbbz77rt47733EAqFMBqNEAqFkEgkEIlEsLa2hrffftspgVgsdu75q9UqDg8PJ6KWtbU1rKysOM+z3W7jj3/8I/b39/Hpp5+iVquhVquh0Whc2uBcF0KhECKRCGKxGFZXV3Hv3j0X3RWLRbz77rsYDofodDpoNptOYQJwEWIkEkEymXSGJxKJIJPJoFAooNls4pNPPsHR0RF6vR7C4TCeP3+OZ8+ezdTgLAKhUAilUsnJ5xe/+AWSySR6vd6E4VUZ0SAVi0Vsb28jHo8jn88jFouhWq2iUqk4wz0cDt24orIFgJ2dHUQiEeRyOWQyGZTLZfzpT3/C4eEhnjx5MhODc51Ip9P42c9+ho8++gjJZBLJZBKDwQDNZhP1eh31eh1HR0doNBpot9tTDU46ncbKygr+xb/4F/iX//JfunE9GAxwcnKCRqOBer2Of/zHfwQA53QuCoHBWSKQvtjd3cX6+jpWVlZw69YtZLNZvP322yiVSsjn887zHg6HAF56kJFIxHnekUgE/X7feeSRSMRNeA6+8XiMZrPpvEyi2WwiFAphOByi2+1iMBggk8k4Y9btdtFoNPD48WPU63U8ffoUBwcH1yKvy2A4HCIUCjmqq16v4+HDh8jn81hdXUUul0M0GkU2m3V0EoCJyUo6hJ7eeDzGyckJarUaDg4OcHR0hGazudAJPiuEw2EkEgnE43FsbGzgzp07zpjS2PC4cDg8QXUNBgNEIhHn/MRiMbTbbUQiETSbTbRaLRc1jkYjNBoNdDodDAYDZ3B6vR5CoZBzdFqtFoCXxoxR+8nJCfb39+dCFy0CHDvRaBTJZBKj0cgZjA8//NDRk7VaDb1eD+VyGf1+H4VCwRnwTCaDXC6H3d1dxONxAHD6gPoDeDner0J/vy4Cg7Mk4IRPp9P4q7/6K/zsZz9DsVjErVu3HPVFw8HBRNrCGhJ69OFwGLFYzA1uKg6l2NTYAC8n/vHxsTNUALCysuIig7/4i79As9nEl19+iXK5jP/23/7bjTc44/HYGdZOp4N2u41nz57h+fPnSCaT+Mu//Eu8//77SCaTyGaziMVi7nckEnEGvNPpODkOBgMcHBzg66+/xsnJCT7//HMcHR05uV/HZL8KotEoVlZWkM1m8eDBA3z44YcYDoc4PDwEACcLKk1L1Y7HY2co1DAPBoMzx5G+ZcSjzlC320W3251Qrj/96U/xgx/8AL/97W9dRL6MoCOTSCScY0OZ3r17F8Ph0I2zer2O3//+9yiXy3jw4AEePHjgxmI4HEY8Hkc0GnVR4nA4dHMdgKPUF403yuDQsySlwQf0JiAWi6FUKiGbzWJ9fd0VBBQKBTcoAUwYGfsDnHrkpD/oNVqDQ1ilqJ7seDx2RouDnIZofX0d0WgUuVwOsVjMKY+bDCo2AC7P1e12HaUBYCLCocdIw9vr9ZyRHgwGaLVaKJfLqFarLhoE4OjOZQKfZS6Xc7lBKjKOA+A0yrNOy3g8Rq/XQ7vddu8DmPjNn16vh36/P2Fo9Bzdbhfj8dhRmLyefD6PVCqFUCjkPr9M4P0Dp9EOI8ZUKjWh05LJJKLRKAaDAZLJJEqlkouGgJd6gOPNyhjAtY2/N8bgUPHFYjFsbW2hUCjg6OgIe3t7XuEq9bEMk39jYwP/9t/+W+zu7uK73/0udnZ2AMBRC9ZI6CRlVEPovevktufQ4yxGo9EEZWKP3d7extraGu7cuYPt7W2nfG+60aGXyPsbDAb4/PPPsb+/j/v37+OHP/yhS8LGYjFncGhg+v0+Dg4OUKvV8NVXX+Hjjz92hjyfz6PX67kIaJlQKBTw05/+FBsbG1hZWUGtVnPGgM+dRthStByLpMloOJj7orHiPFRKkr+1EpIGpVKpIBwOo1gsIplM4vbt2/jggw9QrVbxzTffoNlsXoOkXg/j8dhVQabTaSe3drsN4KVsON5yuRzK5TL+1//6X/j000/R7/exsbHhqO1oNOoiQeDlcxkOh2i3226MXhfeGIMTCoUctVQsFrG+vo5ut+voi2mf4e+b7g1lMhl873vfw1tvveWojV6v55LYHKDKoet9673yN3MxOtFtSeY0g6N0m37PeDxGLBZDoVDAeDxGoVBANpvFeDxGpVKZqUzmBcqPMnzx4oVTBPTQw+Gwy03Q4DDRXS6XcXJygoODA+zt7QF4WaGVSCTOUEjLgkQigVu3bmFnZweDwcApLY4RGg3+rZ60OnYspW+32xgMBohGoy6vwGOZWyQs5cY53el0nMFKpVLI5/NYX19HOBzGkydPFimeK4M5L1K7vFc6g2QRqOMA4PHjx/jDH/6Ajz76CLVazUXOnJv9ft/pAwAT578uLJ3Bsd4PKzry+Tzu3bvnSoXT6TSGw6GrluKDqNfrzvOxNJPvexTTjpun4kin08hms9jY2EChUHC17u122w0cneCWx6XXqQqB92KvW8N
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 2]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5GElEQVR4nO29WW9k2ZUdvGKeRw7JZDKHqpJUWSWpNbXRklrdcMuG0A9+aXe/+Qf4L/if+Bf4yYANwwb6yTDcMCxBaqkGWSWpOjOrKkdOyZjn6XvIbx2u2DwRnIJBRvZdAEEy4saNe/c9Zw9r77NPaDKZTBAgQIAAAQJcMcLXfQEBAgQIEOCfBwKDEyBAgAABloLA4AQIECBAgKUgMDgBAgQIEGApCAxOgAABAgRYCgKDEyBAgAABloLA4AQIECBAgKUgetEPjsdjvHz5ErlcDqFQaJHXtHKYTCZoNBrY3t5GOHw2Gx7I7xiB/C6HQH6Xw0XkBwQyJM4lv8kF8ezZswmA4Ed+nj17FsgvkF8gvxX9OY/8AhleTH4XjnByudxFP/rW4jwyWaT8QqEQwuHw1O90Oo3NzU3kcjn8m3/zb/CDH/wAz58/x29/+1u02208f/4c7XYbo9EIo9EIk8kE4/EY4XAYt27dQj6fx927d/H+++9jMpng9evXaLfb+OUvf4mPP/4Yg8EAnU4H4/EY4/HYfZ5/XwTXJT9FJpNBJpPBrVu38MMf/hDJZBIff/wxvvzyS3Q6HTSbTYzH4zOdq1wu4/bt20gkElhfX0cikcAf/vAHPH78GMPhEMPhcKHXvkz5caxFIhFEo1GUy2Ukk0ns7Ozgzp07WFtbw7e+9S2k02m02230+308e/YMH3/8MVqtFl68eIFWq4VSqYRSqYT19XV8+9vfRrFYxLvvvouNjQ2MRiP0+33U63X8+te/xsHBAT799FP87ne/c+NsMpksTI7nlcmix2A0GnXzl+BcGg6HZx53ABCLxRAOh5FOp5HL5TAajVCr1dDv9899rrPiLPK4sMH55xxCzsJ5ZLII+aVSKRSLRaRSKdy7dw/5fB63bt3C1tYWotEo0uk0EokE3n//fWxvb2NjYwP379/HcDhEv993hkZ/QqEQMpkMEokEYrEY4vE4JpMJ2u02BoMBHj58iL/5m79Bv99Hs9lEv9/H7u4u6vU6Xrx4gUePHqHX66FarWIwGJzrfpYtPwUn6NbWFu7du4c7d+7gz/7sz5DL5fC1r30NlUoF3W4XjUZjyrCqoabC4E8+n8f6+jrG4zGOjo7Q7XYBAMlkErVaDU+fPkW/31/YPSxDfnRICoUCtra28PWvf92NlUgkgnw+j0KhgGQyiVKphGg0ikwmAwDY2NjAhx9+iPF4jMFg4JTeZDJBKpXC2toa4vE4kskk4vE4BoMBRqMR0uk0Hj58iAcPHuAb3/gGfvazn6HdbmNvbw/NZhMfffQRXrx4caH7uYxMFjkG8/k8/uqv/gp3795FLBZDLBZzBrfX6+E3v/kNnjx5gl6vh3a77XXqIpEIUqkUMpkM/vIv/xLvv/8+EokEUqkUhsMhKpUKOp0OfvnLX+Kjjz7CaDRaqNNzFnlc2OAEuH4kEgmsra2hWCziu9/9Lm7fvo33338fH3zwAUKhEMbjMUKhEBKJBCKRCNbX1/Hee+85JRCLxeaev1ar4eDgYCpqWV9fR6lUcp5np9PB73//e+zu7uKTTz5BvV5HvV5Hs9k8t8G5LoRCIUQiEcRiMaytreHBgwcuuisWi3j//fcxGo3Q7XbRarWcwgTgIsRIJIJkMukMTyQSQSaTQaFQQKvVwscff4zDw0P0+32Ew2G8evUKL1++XKjBWQZCoRDK5bKTz09/+lMkk0n0+/0pw6syokEqFou4ffs24vE48vk8YrEYarUaqtWqM9yj0ciNKypbANje3kYkEkEul0Mmk0GlUsEf/vAHHBwc4Pnz5wsxONeJdDqNP//zP8cPfvADJJNJJJNJDIdDtFotNBoNNBoNHB4eotlsotPpzDQ46XQapVIJ//Jf/kv863/9r924Hg6HODo6QrPZRKPRwP/7f/8PAJzTuSwEBmeFQPpiZ2cHGxsbKJVKuHPnDrLZLN577z2Uy2Xk83nneY9GIwBvPMhIJOI870gkgsFg4DzySCTiJjwH32QyQavVcl4m0Wq1EAqFMBqN0Ov1MBwOkclknDHr9XpoNpt4+vQpGo0GXrx4gb29vWuR13kwGo0QCoUc1dVoNPD5558jn89jbW0NuVwO0WgU2WzW0UkApiYr6RB6epPJBEdHR6jX69jb28Ph4SFardZSJ/iiEA6HkUgkEI/Hsbm5iXv37jljSmPD48Lh8BTVNRwOEYlEnPMTi8XQ6XQQiUTQarXQbrdd1Dgej9FsNtHtdjEcDp3B6ff7CIVCztFpt9sA3hgzRu1HR0fY3d29ErpoGeDYiUajSCaTGI/HzmB873vfc/RkvV5Hv99HpVLBYDBAoVBwBjyTySCXy2FnZwfxeBwAnD6g/gDejPfL0N8XRWBwVgSc8Ol0Gn/1V3+FP//zP0exWMSdO3cc9UXDwcFE2sIaEnr04XAYsVjMDW4qDqXY1NgAbyb+69evnaECgFKp5CKDv/iLv0Cr1cKjR49QqVTw3/7bf7vxBmcymTjD2u120el08PLlS7x69QrJZBJ/+Zd/iQ8//BDJZBLZbBaxWMz9jkQizoB3u10nx+FwiL29PXzxxRc4OjrCZ599hsPDQyf365jsl0E0GkWpVEI2m8XDhw/xve99D6PRCAcHBwDgZEGlaanayWTiDIUa5uFweOI40reMeNQZ6vV66PV6U8r1Rz/6Eb7zne/gV7/6lYvIVxF0ZBKJhHNsKNP79+9jNBq5cdZoNPDb3/4WlUoFDx8+xMOHD91YDIfDiMfjiEajLkocjUZurgNwlPqy8VYZHHqWpDT4gN4GxGIxlMtlZLNZbGxsuIKAQqHgBiWAKSNjf4Bjj5z0B71Ga3AIqxTVk51MJs5ocZDTEG1sbCAajSKXyyEWiznlcZNBxQbA5bl6vZ6jNABMRTj0GGl4+/2+M9LD4RDtdhuVSgW1Ws1FgwAc3blK4LPM5XIuN0hFxnEAHEd51mmZTCbo9/vodDrufQBTv/nT7/cxGAymDI2eo9frYTKZOAqT15PP55FKpRAKhdznVwm8f+A42mHEmEqlpnRaMplENBrFcDhEMplEuVx20RDwRg9wvFkZA7i28ffWGBwqvlgshq2tLRQKBRweHuLZs2de4Sr1sQqTf3NzE3/3d3+HnZ0dfOMb38D29jYAOGrBGgmdpIxqCL13ndz2HHqcxXg8nqJM7LG3b9/G+vo67t27h9u3bzvle9ONDr1E3t9wOMRnn32G3d1dvPPOO/jud7/rkrCxWMwZHBqYwWCAvb091Ot1PH78GB999JEz5Pl8Hv1+30VAq4RCoYAf/ehH2NzcRKlUQr1ed8aAz51G2FK0HIukyWg4mPuiseI8VEqSv7USkgalWq0iHA6jWCwimUzi7t27+Na3voVarYYvv/wSrVbrGiR1MUwmE1cFmU6nndw6nQ6AN7LheMvlcqhUKvjf//t/45NPPsFgMMDm5qajtqPRqIsEgTfPZTQaodPpuDF6XXhrDE4oFHLUUrFYxMbGBnq9nqMvZn2Gv2+6N5TJZPDNb34T7777rqM2+v2+S2JzgCqHrvet98rfzMXoRLclmbMMjtJt+j2TyQSxWAyFQgGTyQSFQgHZbBaTyQTVanWhMrkqUH6U4f7+vlME9NDD4bDLTdDgMNFdqVRwdHSEvb09PHv2DMCbCq1EInGCQloVJBIJ3LlzB9vb2xgOh05pcYzQaPBv9aTVsWMpfafTwXA4RDQadXkFHsvcImEpN87pbrfrDFYqlUI+n8fGxgbC4TCeP3++TPFcGsx5kdrlvdIZJItAHQcAT58+xe9+9zv84Ac/QL1ed5Ez5+ZgMHD6AMDU+a8LK2dwrPfDio58Po8HDx64UuF0Oo3RaOSqpfggGo2G83wszeT7HsWs465ScaTTaWSzWWxubqJQKLha906n4waOTnDL49LrVIXAe7H
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 3]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3DElEQVR4nO192XJk2XXdynkekBgKQKFQVV0iu3qg2STlEClRCotWMPTgF9l60wf4F/wn/gI/OcIORzhCL3bYVthBBSmxB5LdzVZ3dXeNmBM5z5l+KK+DlRsnMSYSmeW7IhAAMm/evHffc/aw9j77hEaj0QgBAgQIECDADSN82xcQIECAAAH+/0BgcAIECBAgwEwQGJwAAQIECDATBAYnQIAAAQLMBIHBCRAgQIAAM0FgcAIECBAgwEwQGJwAAQIECDATRK/6weFwiJcvXyKXyyEUCk3zmhYOo9EItVoNm5ubCIcvZsMD+Z0gkN/1EMjveriK/IBAhsSl5De6Ip49ezYCEPzIz7NnzwL5BfIL5LegP5eRXyDDq8nvyhFOLpe76kffWFxGJtOUXygUQjgcHvudTqextraGXC6Hf/Wv/hV+9KMf4fnz5/jNb36DZrOJ58+fo9lsYjAYYDAYYDQaYTgcIhwO486dO8jn87h37x7efvttjEYjHB4eotls4pe//CU++ugj9Ho9tFotDIdDDIdD93n+fRXclvwUmUwGmUwGd+7cwY9//GMkk0l89NFH+Oabb9BqtVCv1zEcDi90rlKphI2NDSQSCaysrCCRSODzzz/HV199hX6/j36/P9Vrn6X8ONYikQii0ShKpRKSySS2trZw9+5dLC8v4/3330c6nUaz2US328WzZ8/w0UcfodFo4MWLF2g0GlhaWsLS0hJWVlbwve99D8ViEW+99RZWV1cxGAzQ7XZRrVbxj//4j9jf38cnn3yC3/3ud26cjUajqcnxsjKZ9hiMRqNu/hKcS/1+/8LjDgBisRjC4TDS6TRyuRwGgwEqlQq63e6lz3VRXEQeVzY4/z+HkJNwGZlMQ36pVArFYhGpVArb29vI5/O4c+cO1tfXEY1GkU6nkUgk8Pbbb2NzcxOrq6u4f/8++v0+ut2uMzT6EwqFkMlkkEgkEIvFEI/HMRqN0Gw20ev18PjxY/zVX/0Vut0u6vU6ut0udnZ2UK1W8eLFC3z55ZfodDo4Pj5Gr9e71P3MWn4KTtD19XVsb2/j7t27+KM/+iPkcjn8wR/8AcrlMtrtNmq12phhVUNNhcGffD6PlZUVDIdDHB0dod1uAwCSySQqlQqePn2Kbrc7tXuYhfzokBQKBayvr+M73/mOGyuRSAT5fB6FQgHJZBJLS0uIRqPIZDIAgNXVVbz77rsYDofo9XpO6Y1GI6RSKSwvLyMejyOZTCIej6PX62EwGCCdTuPx48d48OABvvvd7+LnP/85ms0mdnd3Ua/X8eGHH+LFixdXup/ryGSaYzCfz+PP//zPce/ePcRiMcRiMWdwO50Ofv3rX+PJkyfodDpoNptepy4SiSCVSiGTyeDP/uzP8PbbbyORSCCVSqHf76NcLqPVauGXv/wlPvzwQwwGg6k6PReRx5UNToDbRyKRwPLyMorFIj744ANsbGzg7bffxjvvvINQKIThcIhQKIREIoFIJIKVlRU8evTIKYFYLHbm+SuVCvb398eilpWVFSwtLTnPs9Vq4bPPPsPOzg4+/vhjVKtVVKtV1Ov1Sxuc20IoFEIkEkEsFsPy8jIePHjgortisYi3334bg8EA7XYbjUbDKUwALkKMRCJIJpPO8EQiEWQyGRQKBTQaDXz00Uc4ODhAt9tFOBzGq1ev8PLly6kanFkgFAqhVCo5+fzsZz9DMplEt9sdM7wqIxqkYrGIjY0NxONx5PN5xGIxVCoVHB8fO8M9GAzcuKKyBYDNzU1EIhHkcjlkMhmUy2V8/vnn2N/fx/Pnz6dicG4T6XQaf/Inf4If/ehHSCaTSCaT6Pf7aDQaqNVqqNVqODg4QL1eR6vVmmhw0uk0lpaW8C/+xb/AX/zFX7hx3e/3cXR0hHq9jlqtht/+9rcA4JzOWSEwOAsE0hdbW1tYXV3F0tIS7t69i2w2i0ePHqFUKiGfzzvPezAYAHjtQUYiEed5RyIR9Ho955FHIhE34Tn4RqMRGo2G8zKJRqOBUCiEwWCATqeDfr+PTCbjjFmn00G9XsfTp09Rq9Xw4sUL7O7u3oq8LoPBYIBQKOSorlqthi+++AL5fB7Ly8vI5XKIRqPIZrOOTgIwNllJh9DTG41GODo6QrVaxe7uLg4ODtBoNGY6waeFcDiMRCKBeDyOtbU1bG9vO2NKY8PjwuHwGNXV7/cRiUSc8xOLxdBqtRCJRNBoNNBsNl3UOBwOUa/X0W630e/3ncHpdrsIhULO0Wk2mwBeGzNG7UdHR9jZ2bkRumgW4NiJRqNIJpMYDofOYPzgBz9w9GS1WkW320W5XEav10OhUHAGPJPJIJfLYWtrC/F4HACcPqD+AF6P9+vQ31dFYHAWBJzw6XQaf/7nf44/+ZM/QbFYxN27dx31RcPBwUTawhoSevThcBixWMwNbioOpdjU2ACvJ/7h4aEzVACwtLTkIoM//dM/RaPRwJdffolyuYz/8l/+y9wbnNFo5Axru91Gq9XCy5cv8erVKySTSfzZn/0Z3n33XSSTSWSzWcRiMfc7Eok4A95ut50c+/0+dnd38fXXX+Po6AiffvopDg4OnNxvY7JfB9FoFEtLS8hms3j8+DF+8IMfYDAYYH9/HwCcLKg0LVU7Go2coVDD3O/3Tx1H+pYRjzpDnU4HnU5nTLn+5Cc/wfe//3386le/chH5IoKOTCKRcI4NZXr//n0MBgM3zmq1Gn7zm9+gXC7j8ePHePz4sRuL4XAY8Xgc0WjURYmDwcDNdQCOUp813iiDQ8+SlAYf0JuAWCyGUqmEbDaL1dVVVxBQKBTcoAQwZmTsD3DikZP+oNdoDQ5hlaJ6sqPRyBktDnIaotXVVUSjUeRyOcRiMac85hlUbABcnqvT6ThKA8BYhEOPkYa32+06I93v99FsNlEul1GpVFw0CMDRnYsEPstcLudyg1RkHAfASZRnnZbRaIRut4tWq+XeBzD2mz/dbhe9Xm/M0Og5Op0ORqORozB5Pfl8HqlUCqFQyH1+kcD7B06iHUaMqVRqTKclk0lEo1H0+30kk0mUSiUXDQGv9QDHm5UxgFsbf2+MwaHii8ViWF9fR6FQwMHBAZ49e+YVrlIfizD519bW8Nd//dfY2trCd7/7XWxubgKAoxaskdBJyqiG0HvXyW3PocdZDIfDMcrEHruxsYGVlRVsb29jY2PDKd95Nzr0Enl//X4fn376KXZ2dvDw4UN88MEHLgkbi8WcwaGB6fV62N3dRbVaxVdffYUPP/zQGfJ8Po9ut+sioEVCoVDAT37yE6ytrWFpaQnVatUZAz53GmFL0XIskiaj4WDui8aK81ApSf7WSkgalOPjY4TDYRSLRSSTSdy7dw/vv/8+KpUKvvnmGzQajVuQ1NUwGo1cFWQ6nXZya7VaAF7LhuMtl8uhXC7jf/2v/4WPP/4YvV4Pa2trjtqORqMuEgReP5fBYIBWq+XG6G3hjTE4oVDIUUvFYhGrq6vodDqOvpj0Gf6ed28ok8ngvffew1tvveWojW6365LYHKDKoet9673yN3MxOtFtSeYkg6N0m37PaDRCLBZDoVDAaDRCoVBANpvFaDTC8fHxVGVyU6D8KMO9vT2nCOihh8Nhl5ugwWGiu1wu4+joCLu7u3j27BmA1xVaiUTiFIW0KEgkErh79y42NzfR7/ed0uIYodHg3+pJq2PHUvpWq4V+v49oNOryCjyWuUXCUm6c0+122xmsVCqFfD6P1dVVhMNhPH/+fJbiuTaY8yK1y3ulM0gWgToOAJ4+fYrf/e53+NGPfoRqteoiZ87NXq/n9AGAsfPfFhbO4FjvhxUd+XweDx48cKXC6XQag8HAVUvxQdRqNef5WJrJ9z2KScfdpOJIp9PIZrNYW1tDoVBwte6tVssNHJ3glsel16kKgfdir1v
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 4]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5y0lEQVR4nO29WW9kWXYdvGKeRw7JKYeqVHdlVXerh5KhbqklWG2jIRh+keQ3/wD/Bf8T/wI/GfAAGNCTYVgw3Fa31DW0u6qrOiurKpkDmZxinofvIb91uGLz3CCZDAYZpVgAQTLixo179z1nD2vvs09oPB6PscQSSyyxxBLXjPBNX8ASSyyxxBL/NLA0OEssscQSS8wFS4OzxBJLLLHEXLA0OEssscQSS8wFS4OzxBJLLLHEXLA0OEssscQSS8wFS4OzxBJLLLHEXBB90w+ORiO8ePECuVwOoVBolte0cBiPx6jX69ja2kI4fDEbvpTfKZbyuxqW8rsa3kR+wFKGxKXkN35D7O7ujgEsf+Rnd3d3Kb+l/JbyW9Cfy8hvKcM3k98bRzi5XO5NP/qNxWVkMkv5hUIhhMPhid/pdBrr6+vI5XL41//6X+P999/Hs2fP8Jvf/AatVgvPnj1Dq9XCcDjEcDjEeDzGaDRCOBzGnTt3kM/ncffuXbzzzjsYj8c4OjpCq9XCL3/5S3z44Yfo9/tot9sYjUYYjUbu8/z7TXBT8lNkMhlkMhncuXMHP/7xj5FMJvHhhx/iq6++QrvdRqPRwGg0utC5yuUyNjc3kUgksLq6ikQigd/97nf44osvMBgMMBgMZnrt85Qfx1okEkE0GkW5XEYymcTOzg62t7exsrKC7373u0in02i1Wuj1etjd3cWHH36IZrOJ58+fo9lsolQqoVQqYXV1Fd/73vdQLBbx9ttvY21tDcPhEL1eD7VaDf/4j/+Ig4MDfPzxx/jtb3/rxtl4PJ6ZHC8rk1mPwWg06uYvwbk0GAwuPO4AIBaLIRwOI51OI5fLYTgcolqtotfrXfpcF8VF5PHGBuefcggZhMvIZBbyS6VSKBaLSKVSuHfvHvL5PO7cuYONjQ1Eo1Gk02kkEgm888472NrawtraGu7fv4/BYIBer+cMjf6EQiFkMhkkEgnEYjHE43GMx2O0Wi30+308evQIf/VXf4Ver4dGo4Fer4e9vT3UajU8f/4cjx8/RrfbRaVSQb/fv9T9zFt+Ck7QjY0N3Lt3D9vb2/jjP/5j5HI5/MEf/AFOTk7Q6XRQr9cnDKsaaioM/uTzeayurmI0GuH4+BidTgcAkEwmUa1W8fTpU/R6vZndwzzkR4ekUChgY2MD3/rWt9xYiUQiyOfzKBQKSCaTKJVKiEajyGQyAIC1tTW89957GI1G6Pf7TumNx2OkUimsrKwgHo8jmUwiHo+j3+9jOBwinU7j0aNHePDgAb797W/j5z//OVqtFvb399FoNPDBBx/g+fPnb3Q/V5HJLMdgPp/HX/zFX+Du3buIxWKIxWLO4Ha7Xfz617/GkydP0O120Wq1vE5dJBJBKpVCJpPBn//5n+Odd95BIpFAKpXCYDDAyckJ2u02fvnLX+KDDz7AcDicqdNzEXm8scFZ4uaRSCSwsrKCYrGIH/zgB9jc3MQ777yDd999F6FQCKPRCKFQCIlEApFIBKurq3j48KFTArFYbOr5q9UqDg4OJqKW1dVVlEol53m22218+umn2Nvbw0cffYRarYZarYZGo3Fpg3NTCIVCiEQiiMViWFlZwYMHD1x0VywW8c4772A4HKLT6aDZbDqFCcBFiJFIBMlk0hmeSCSCTCaDQqGAZrOJDz/8EIeHh+j1egiHw3j58iVevHgxU4MzD4RCIZTLZSefn/3sZ0gmk+j1ehOGV2VEg1QsFrG5uYl4PI58Po9YLIZqtYpKpeIM93A4dOOKyhYAtra2EIlEkMvlkMlkcHJygt/97nc4ODjAs2fPZmJwbhLpdBp/+qd/ivfffx/JZBLJZBKDwQDNZhP1eh31eh2Hh4doNBpot9uBBiedTqNUKuGf//N/jn/5L/+lG9eDwQDHx8doNBqo1+v4f//v/wGAczrnhaXBWSCQvtjZ2cHa2hpKpRK2t7eRzWbx8OFDlMtl5PN553kPh0MArz3ISCTiPO9IJIJ+v+888kgk4iY8B994PEaz2XReJtFsNhEKhTAcDtHtdjEYDJDJZJwx63a7aDQaePr0Ker1Op4/f479/f0bkddlMBwOEQqFHNVVr9fx+eefI5/PY2VlBblcDtFoFNls1tFJACYmK+kQenrj8RjHx8eo1WrY39/H4eEhms3mXCf4rBAOh5FIJBCPx7G+vo579+45Y0pjw+PC4fAE1TUYDBCJRJzzE4vF0G63EYlE0Gw20Wq1XNQ4Go3QaDTQ6XQwGAycwen1egiFQs7RabVaAF4bM0btx8fH2Nvbuxa6aB7g2IlGo0gmkxiNRs5g/PCHP3T0ZK1WQ6/Xw8nJCfr9PgqFgjPgmUwGuVwOOzs7iMfjAOD0AfUH8Hq8X4X+flMsDc6CgBM+nU7jL/7iL/Cnf/qnKBaL2N7edtQXDQcHE2kLa0jo0YfDYcRiMTe4qTiUYlNjA7ye+EdHR85QAUCpVHKRwZ/92Z+h2Wzi8ePHODk5wX/7b//t1huc8XjsDGun00G73caLFy/w8uVLJJNJ/Pmf/znee+89JJNJZLNZxGIx9zsSiTgD3ul0nBwHgwH29/fx5Zdf4vj4GJ988gkODw+d3G9isl8F0WgUpVIJ2WwWjx49wg9/+EMMh0McHBwAgJMFlaalasfjsTMUapgHg8GZ40jfMuJRZ6jb7aLb7U4o15/85Cf4/ve/j1/96lcuIl9E0JFJJBLOsaFM79+/j+Fw6MZZvV7Hb37zG5ycnODRo0d49OiRG4vhcBjxeBzRaNRFicPh0M11AI5Snze+UQaHniUpDT6gbwJisRjK5TKy2SzW1tZcQUChUHCDEsCEkbE/wKlHTvqDXqM1OIRViurJjsdjZ7Q4yGmI1tbWEI1GkcvlEIvFnPK4zaBiA+DyXN1u11EaACYiHHqMNLy9Xs8Z6cFggFarhZOTE1SrVRcNAnB05yKBzzKXy7ncIBUZxwFwGuVZp2U8HqPX66Hdbrv3AUz85k+v10O/358wNHqObreL8XjsKExeTz6fRyqVQigUcp9fJPD+gdNohxFjKpWa0GnJZBLRaBSDwQDJZBLlctlFQ8BrPcDxZmUM4MbG3zfG4FDxxWIxbGxsoFAo4PDwELu7u17hKvWxCJN/fX0d/+bf/Bvs7Ozg29/+Nra2tgDAUQvWSOgkZVRD6L3r5Lbn0OMsRqPRBGVij93c3MTq6iru3buHzc1Np3xvu9Ghl8j7GwwG+OSTT7C3t4e33noLP/jBD1wSNhaLOYNDA9Pv97G/v49arYYvvvgCH3zwgTPk+XwevV7PRUCLhEKhgJ/85CdYX19HqVRCrVZzxoDPnUbYUrQci6TJaDiY+6Kx4jxUSpK/tRKSBqVSqSAcDqNYLCKZTOLu3bv47ne/i2q1iq+++grNZvMGJPVmGI/HrgoynU47ubXbbQCvZcPxlsvlcHJygv/1v/4XPvroI/T7fayvrztqOxqNukgQeP1chsMh2u22G6M3hW+MwQmFQo5aKhaLWFtbQ7fbdfRF0Gf4+7Z7Q5lMBt/5znfw9ttvO2qj1+u5JDYHqHLoet96r/zNXIxOdFuSGWRwlG7T7xmPx4jFYigUChiPxygUCshmsxiPx6hUKjOVyXWB8qMMX7165RQBPfRwOOxyEzQ4THSfnJzg+PgY+/v72N3dBfC6QiuRSJyhkBYFiUQC29vb2NrawmAwcEqLY4RGg3+rJ62OHUvp2+02BoMBotGoyyvwWOYWCUu5cU53Oh1nsFKpFPL5PNbW1hAOh/Hs2bN5iufKYM6L1C7vlc4gWQTqOAB4+vQpfvvb3+L9999HrVZzkTPnZr/fd/oAwMT5bwoLZ3Cs98OKjnw+jwcPHrhS4XQ6jeFw6Kql+CDq9brzfCzN5PseRdBx16k40uk0stks1tfXUSgUXK17u912A0cnuOV
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 5]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB4RklEQVR4nO29WY9k2XUdvGKeh4wcK7PGLpJd3STFoWWIlCjZog1CD36R7Tf/AP8F/xP/Aj8ZsGHAgJ4Mw7IACqTEHuhudre6q7przKkyM+Y54nuob51csfPcyCkyMqN8FxCIzBhu3LvvOXtYe599IuPxeIwQIUKECBHiihG97hMIESJEiBD/byA0OCFChAgRYi4IDU6IECFChJgLQoMTIkSIECHmgtDghAgRIkSIuSA0OCFChAgRYi4IDU6IECFChJgL4hf94mg0wsuXL1EoFBCJRGZ5TguH8XiMer2Ozc1NRKNns+Gh/I4Ryu9yCOV3OVxEfkAoQ+Jc8htfEM+ePRsDCB/yePbsWSi/UH6h/Bb0cR75hTK8mPwuHOEUCoWLfvWtxXlkMkv5RSIRRKPRiedsNou1tTUUCgX863/9r/HBBx/g+fPn+P3vf49Wq4Xnz5+j1WphOBxiOBxiPB5jNBohGo1ifX0dxWIRd+7cwbvvvovxeIzXr1+j1WrhN7/5DT766CP0+320222MRiOMRiP3ff59EVyX/BS5XA65XA7r6+v42c9+hnQ6jY8++gjffPMN2u02Go0GRqPRmY5VqVRw69YtpFIprKysIJVK4fPPP8fXX3+NwWCAwWAw03Ofp/w41mKxGOLxOCqVCtLpNG7fvo2trS0sLy/jBz/4AbLZLFqtFnq9Hp49e4aPPvoIzWYTL168QLPZxNLSEpaWlrCysoIf/vCHKJfLeOedd7C6uorhcIher4darYZ//Md/xN7eHj755BN8+umnbpyNx+OZyfG8Mpn1GIzH427+EpxLg8HgzOMOABKJBKLRKLLZLAqFAobDIarVKnq93rmPdVacRR4XNjj/L4eQQTiPTGYhv0wmg3K5jEwmg7t376JYLGJ9fR0bGxuIx+PIZrNIpVJ49913sbm5idXVVdy7dw+DwQC9Xs8ZGn1EIhHkcjmkUikkEgkkk0mMx2O0Wi30+308evQIf/3Xf41er4dGo4Fer4ft7W3UajW8ePECX331FbrdLo6OjtDv9891PfOWn4ITdGNjA3fv3sXW1hb+5E/+BIVCAd/5zndweHiITqeDer0+YVjVUFNh8FEsFrGysoLRaISDgwN0Oh0AQDqdRrVaxdOnT9Hr9WZ2DfOQHx2SUqmEjY0NfPe733VjJRaLoVgsolQqIZ1OY2lpCfF4HLlcDgCwurqK999/H6PRCP1+3ym98XiMTCaD5eVlJJNJpNNpJJNJ9Pt9DIdDZLNZPHr0CPfv38f3vvc9/OpXv0Kr1cLOzg4ajQY+/PBDvHjx4kLXcxmZzHIMFotF/OVf/iXu3LmDRCKBRCLhDG6328Xvfvc7PH78GN1uF61Wy+vUxWIxZDIZ5HI5/MVf/AXeffddpFIpZDIZDAYDHB4eot1u4ze/+Q0+/PBDDIfDmTo9Z5HHhQ1OiOtHKpXC8vIyyuUyfvzjH+PWrVt499138d577yESiWA0GiESiSCVSiEWi2FlZQUPHz50SiCRSEw9frVaxd7e3kTUsrKygqWlJed5tttt/OEPf8D29jY+/vhj1Go11Go1NBqNcxuc60IkEkEsFkMikcDy8jLu37/vortyuYx3330Xw+EQnU4HzWbTKUwALkKMxWJIp9PO8MRiMeRyOZRKJTSbTXz00UfY399Hr9dDNBrFq1ev8PLly5kanHkgEomgUqk4+fzyl79EOp1Gr9ebMLwqIxqkcrmMW7duIZlMolgsIpFIoFqt4ujoyBnu4XDoxhWVLQBsbm4iFouhUCggl8vh8PAQn3/+Ofb29vD8+fOZGJzrRDabxZ/92Z/hgw8+QDqdRjqdxmAwQLPZRL1eR71ex/7+PhqNBtrtdqDByWazWFpawr/4F/8C/+pf/Ss3rgeDAQ4ODtBoNFCv1/F//+//BQDndM4LocFZIJC+uH37NlZXV7G0tIStrS3k83k8fPgQlUoFxWLRed7D4RDAGw8yFos5zzsWi6Hf7zuPPBaLuQnPwTcej9FsNp2XSTSbTUQiEQyHQ3S7XQwGA+RyOWfMut0uGo0Gnj59inq9jhcvXmBnZ+da5HUeDIdDRCIRR3XV63V8+eWXKBaLWF5eRqFQQDweRz6fd3QSgInJSjqEnt54PMbBwQFqtRp2dnawv7+PZrM51wk+K0SjUaRSKSSTSaytreHu3bvOmNLY8HPRaHSC6hoMBojFYs75SSQSaLfbiMViaDabaLVaLmocjUZoNBrodDoYDAbO4PR6PUQiEefotFotAG+MGaP2g4MDbG9vXwldNA9w7MTjcaTTaYxGI2cwfvKTnzh6slarodfr4fDwEP1+H6VSyRnwXC6HQqGA27dvI5lMAoDTB9QfwJvxfhn6+6IIDc6CgBM+m83iL//yL/Fnf/ZnKJfL2NractQXDQcHE2kLa0jo0UejUSQSCTe4qTiUYlNjA7yZ+K9fv3aGCgCWlpZcZPDnf/7naDab+Oqrr3B4eIj//t//+403OOPx2BnWTqeDdruNly9f4tWrV0in0/iLv/gLvP/++0in08jn80gkEu45Fos5A97pdJwcB4MBdnZ28OTJExwcHOCzzz7D/v6+k/t1TPbLIB6PY2lpCfl8Ho8ePcJPfvITDIdD7O3tAYCTBZWmpWrH47EzFGqYB4PBic+RvmXEo85Qt9tFt9udUK4///nP8aMf/Qi//e1vXUS+iKAjk0qlnGNDmd67dw/D4dCNs3q9jt///vc4PDzEo0eP8OjRIzcWo9Eokskk4vG4ixKHw6Gb6wAcpT5vvFUGh54lKQ3eoLcBiUQClUoF+Xweq6urriCgVCq5QQlgwsjYB3DskZP+oNdoDQ5hlaJ6suPx2BktDnIaotXVVcTjcRQKBSQSCac8bjKo2AC4PFe323WUBoCJCIceIw1vr9dzRnowGKDVauHw8BDVatVFgwAc3blI4L0sFAouN0hFxnEAHEd51mkZj8fo9Xpot9vufQATz3z0ej30+/0JQ6PH6Ha7GI/HjsLk+RSLRWQyGUQiEff9RQKvHziOdhgxZjKZCZ2WTqcRj8cxGAyQTqdRqVRcNAS80QMcb1bGAK5t/L01BoeKL5FIYGNjA6VSCfv7+3j27JlXuEp9LMLkX1tbw7/7d/8Ot2/fxve+9z1sbm4CgKMWrJHQScqohtBr18ltj6GfsxiNRhOUif3srVu3sLKygrt37+LWrVtO+d50o0Mvkdc3GAzw2WefYXt7Gw8ePMCPf/xjl4RNJBLO4NDA9Pt97OzsoFar4euvv8aHH37oDHmxWESv13MR0CKhVCrh5z//OdbW1rC0tIRareaMAe87jbClaDkWSZPRcDD3RWPFeaiUJJ+1EpIG5ejoCNFoFOVyGel0Gnfu3MEPfvADVKtVfPPNN2g2m9cgqYthPB67KshsNuvk1m63AbyRDcdboVDA4eEh/vf//t/4+OOP0e/3sba25qjteDzuIkHgzX0ZDodot9tujF4X3hqDE4lEHLVULpexurqKbrfr6Iug7/D5pntDuVwO3//+9/HOO+84aqPX67kkNgeocuh63XqtfGYuRie6LckMMjhKt+nvjMdjJBIJlEoljMdjlEol5PN5jMdjHB0dzVQmVwXKjzLc3d11ioAeejQadbkJGhwmug8PD3FwcICdnR08e/YMwJsKrVQqdYJCWhSkUilsbW1hc3MTg8HAKS2OERoN/q2etDp2LKVvt9sYDAaIx+Mur8DPMrdIWMqNc7rT6TiDlclkUCwWsbq6img0iufPn89TPJcGc16kdnmtdAbJIlDHAcDTp0/x6aef4oMPPkCtVnORM+dmv993+gDAxPGvCwtncKz3w4qOYrGI+/fvu1LhbDaL4XDoqqV4I+r1uvN8LM3k+x1F0OeuUnFks1nk83msra2hVCq5Wvd2u+0Gjk5wy+PS61S
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 6]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB1OElEQVR4nO192Y5k2XXdinkec6gcauxSd1eTTXOSIVKiBIsyaD34Rbbf/AH+Bf+Jv8BPBmwYMKAnw7AggAIpsQeym2Sxqrq7xpwz5nnwQ3mdXLHzRGZWZWRkROkuIBFVETdu3LvvOXtYe599QuPxeIwAAQIECBDgihG+7gsIECBAgAD/PBAYnAABAgQIMBcEBidAgAABAswFgcEJECBAgABzQWBwAgQIECDAXBAYnAABAgQIMBcEBidAgAABAswF0bf94mg0wsuXL5HL5RAKhWZ5TUuH8XiMer2Ora0thMMXs+GB/E4QyO9yCOR3ObyN/IBAhsQbyW/8lnj27NkYQPAnf8+ePQvkF8gvkN+S/r2J/AIZvp383jrCyeVyb/vVdxZvIpNZyi8UCiEcDk+8ptNprK+vI5fL4d/+23+LH/7wh3j+/Dl+/etfo9Vq4fnz52i1WhgOhxgOhxiPxxiNRgiHw7hx4wby+Txu3bqFDz/8EOPxGIeHh2i1WvjFL36BTz/9FP1+H+12G6PRCKPRyH2f/34bXJf8FJlMBplMBjdu3MCPfvQjJJNJfPrpp/j666/RbrfRaDQwGo0udK5yuYzNzU0kEgmsrq4ikUjgd7/7HR4/fozBYIDBYDDTa5+n/DjWIpEIotEoyuUykskkbt68ie3tbaysrODjjz9GOp1Gq9VCr9fDs2fP8Omnn6LZbOLFixdoNpsolUoolUpYXV3Fd77zHRSLRbz33ntYW1vDcDhEr9dDrVbDP/3TP2F/fx+ff/45vvjiCzfOxuPxzOT4pjKZ9RiMRqNu/hKcS4PB4MLjDgBisRjC4TDS6TRyuRyGwyGq1Sp6vd4bn+uiuIg83trg/HMOIafhTWQyC/mlUikUi0WkUincvn0b+XweN27cwMbGBqLRKNLpNBKJBD788ENsbW1hbW0Nd+7cwWAwQK/Xc4ZG/0KhEDKZDBKJBGKxGOLxOMbjMVqtFvr9Ph48eIC/+Zu/Qa/XQ6PRQK/Xw87ODmq1Gl68eIFHjx6h2+2iUqmg3++/0f3MW34KTtCNjQ3cvn0b29vb+JM/+RPkcjn80R/9EY6Pj9HpdFCv1ycMqxpqKgz+5fN5rK6uYjQa4ejoCJ1OBwCQTCZRrVbx9OlT9Hq9md3DPORHh6RQKGBjYwPvv/++GyuRSAT5fB6FQgHJZBKlUgnRaBSZTAYAsLa2hm9961sYjUbo9/tO6Y3HY6RSKaysrCAejyOZTCIej6Pf72M4HCKdTuPBgwe4e/cuPvjgA/zsZz9Dq9XC7u4uGo0GPvnkE7x48eKt7ucyMpnlGMzn8/jLv/xL3Lp1C7FYDLFYzBncbreLX/3qV3jy5Am63S5arZbXqYtEIkilUshkMviLv/gLfPjhh0gkEkilUhgMBjg+Pka73cYvfvELfPLJJxgOhzN1ei4ij7c2OAGuH4lEAisrKygWi/je976Hzc1NfPjhh/joo48QCoUwGo0QCoWQSCQQiUSwurqK+/fvOyUQi8XOPH+1WsX+/v5E1LK6uopSqeQ8z3a7jd/+9rfY2dnBZ599hlqthlqthkaj8cYG57oQCoUQiUQQi8WwsrKCu3fvuuiuWCziww8/xHA4RKfTQbPZdAoTgIsQI5EIksmkMzyRSASZTAaFQgHNZhOffvopDg4O0Ov1EA6H8erVK7x8+XKmBmceCIVCKJfLTj4//elPkUwm0ev1JgyvyogGqVgsYnNzE/F4HPl8HrFYDNVqFZVKxRnu4XDoxhWVLQBsbW0hEokgl8shk8ng+PgYv/vd77C/v4/nz5/PxOBcJ9LpNP7sz/4MP/zhD5FMJpFMJjEYDNBsNlGv11Gv13FwcIBGo4F2uz3V4KTTaZRKJfyrf/Wv8K//9b9243owGODo6AiNRgP1eh2/+c1vAMA5nfNCYHCWCKQvbt68ibW1NZRKJWxvbyObzeL+/fsol8vI5/PO8x4OhwBee5CRSMR53pFIBP1+33nkkUjETXgOvvF4jGaz6bxMotlsIhQKYTgcotvtYjAYIJPJOGPW7XbRaDTw9OlT1Ot1vHjxAru7u9cirzfBcDhEKBRyVFe9XsfDhw+Rz+exsrKCXC6HaDSKbDbr6CQAE5OVdAg9vfF4jKOjI9RqNezu7uLg4ADNZnOuE3xWCIfDSCQSiMfjWF9fx+3bt50xpbHhceFweILqGgwGiEQizvmJxWJot9uIRCJoNptotVouahyNRmg0Guh0OhgMBs7g9Ho9hEIh5+i0Wi0Ar40Zo/ajoyPs7OxcCV00D3DsRKNRJJNJjEYjZzC+//3vO3qyVquh1+vh+PgY/X4fhULBGfBMJoNcLoebN28iHo8DgNMH1B/A6/F+Gfr7bREYnCUBJ3w6ncZf/uVf4s/+7M9QLBaxvb3tqC8aDg4m0hbWkNCjD4fDiMVibnBTcSjFpsYGeD3xDw8PnaECgFKp5CKDP//zP0ez2cSjR49wfHyM//k//+fCG5zxeOwMa6fTQbvdxsuXL/Hq1Sskk0n8xV/8Bb71rW8hmUwim80iFou510gk4gx4p9NxchwMBtjd3cVXX32Fo6MjfPnllzg4OHByv47JfhlEo1GUSiVks1k8ePAA3//+9zEcDrG/vw8AThZUmpaqHY/HzlCoYR4MBqeOI33LiEedoW63i263O6Fcf/zjH+O73/0ufvnLX7qIfBlBRyaRSDjHhjK9c+cOhsOhG2f1eh2//vWvcXx8jAcPHuDBgwduLIbDYcTjcUSjURclDodDN9cBOEp93ninDA49S1IafEDvAmKxGMrlMrLZLNbW1lxBQKFQcIMSwISRsX/AiUdO+oNeozU4hFWK6smOx2NntDjIaYjW1tYQjUaRy+UQi8Wc8lhkULEBcHmubrfrKA0AExEOPUYa3l6v54z0YDBAq9XC8fExqtWqiwYBOLpzmcBnmcvlXG6QiozjADiJ8qzTMh6P0ev10G633ecAJl751+v10O/3JwyNnqPb7WI8HjsKk9eTz+eRSqUQCoXc95cJvH/gJNphxJhKpSZ0WjKZRDQaxWAwQDKZRLlcdtEQ8FoPcLxZGQO4tvH3zhgcKr5YLIaNjQ0UCgUcHBzg2bNnXuEq9bEMk399fR3/4T/8B9y8eRMffPABtra2AMBRC9ZI6CRlVEPovevktufQ4yxGo9EEZWKP3dzcxOrqKm7fvo3NzU2nfBfd6NBL5P0NBgN8+eWX2NnZwb179/C9733PJWFjsZgzODQw/X4fu7u7qNVqePz4MT755BNnyPP5PHq9nouAlgmFQgE//vGPsb6+jlKphFqt5owBnzuNsKVoORZJk9FwMPdFY8V5qJQkX7USkgalUqkgHA6jWCwimUzi1q1b+Pjjj1GtVvH111+j2Wxeg6TeDuPx2FVBptNpJ7d2uw3gtWw43nK5HI6Pj/F//+//xWeffYZ+v4/19XVHbUejURcJAq+fy3A4RLvddmP0uvDOGJxQKOSopWKxiLW1NXS7XUdfTPsOXxfdG8pkMvj2t7+N9957z1EbvV7PJbE5QJVD1/vWe+UrczE60W1J5jSDo3Sb/s54PEYsFkOhUMB4PEahUEA2m8V4PEalUpmpTK4KlB9luLe35xQBPfRwOOxyEzQ4THQfHx/j6OgIu7u7ePbsGYDXFVqJROIUhbQsSCQS2N7extbWFgaDgVNaHCM0Gvy3etLq2LGUvt1uYzAYIBqNurwCj2VukbCUG+d0p9NxBiuVSiGfz2NtbQ3hcBjPnz+fp3guDea8SO3yXukMkkWgjgOAp0+f4osvvsAPf/hD1Go1Fzlzbvb7facPAEyc/7qwdAbHej+s6Mjn87h7964rFU6n0xgOh65aig+iXq87z8fSTL7fUUw77ioVRzqdRjabxfr6OgqFgqt1b7fbbuDoBLc8Lr1OVQi8F3vdGs7
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 7]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB2GklEQVR4nO19145k2ZXdCu9tmkpXrmuaXd1siqZHGnKGHGgogZgHvYykN32AfkF/oi/QkwAJAgTMkyBpIIACOcN2alesru7y6cN7p4fCOrli573pMzKi5i4gEJkRNyLu3fecbdbeZ5/QZDKZIECAAAECBLhmhG/6BAIECBAgwD8OBAYnQIAAAQLMBIHBCRAgQIAAM0FgcAIECBAgwEwQGJwAAQIECDATBAYnQIAAAQLMBIHBCRAgQIAAM0H0oh8cj8d49eoVcrkcQqHQVZ7TwmEymaDRaGBjYwPh8NlseCC/IwTyuxwC+V0OF5EfEMiQOJf8JhfE8+fPJwCChzyeP38eyC+QXyC/BX2cR36BDC8mvwtHOLlc7qIffWtxHplcpfxCoRDC4fDUczqdxurqKnK5HP7Vv/pX+Oijj/DixQt8/vnnaLfbePHiBdrtNkajEUajESaTCcbjMcLhMG7duoV8Po/bt2/jvffew2QywcHBAdrtNn73u9/hk08+wWAwQKfTwXg8xng8dp/n3xfBTclPkclkkMlkcOvWLfz85z9HMpnEJ598gu+//x6dTgfNZhPj8fhM31Uul7G+vo5EIoHl5WUkEgl8/fXX+PbbbzEcDjEcDq/03GcpP461SCSCaDSKcrmMZDKJra0tbG5uYmlpCR9++CHS6TTa7Tb6/T6eP3+OTz75BK1WCy9fvkSr1UKpVEKpVMLy8jJ+9KMfoVgs4p133sHKygpGoxH6/T7q9Tr+4R/+AXt7e/jss8/wxRdfuHE2mUyuTI7nlclVj8FoNOrmL8G5NBwOzzzuACAWiyEcDiOdTiOXy2E0GqFWq6Hf75/7u86Ks8jjwgbnH3MI6YfzyOQq5JdKpVAsFpFKpXDnzh3k83ncunULa2triEajSKfTSCQSeO+997CxsYGVlRXcvXsXw+EQ/X7fGRp9hEIhZDIZJBIJxGIxxONxTCYTtNttDAYDPHz4EH/zN3+Dfr+PZrOJfr+P7e1t1Ot1vHz5Eo8fP0av10O1WsVgMDjX9cxafgpO0LW1Ndy5cwebm5v4sz/7M+RyOfzJn/wJKpUKut0uGo3GlGFVQ02FwUc+n8fy8jLG4zEODw/R7XYBAMlkErVaDc+ePUO/37+ya5iF/OiQFAoFrK2t4d1333VjJRKJIJ/Po1AoIJlMolQqIRqNIpPJAABWVlbwwQcfYDweYzAYOKU3mUyQSqWwtLSEeDyOZDKJeDyOwWCA0WiEdDqNhw8f4t69e/jBD36A3/zmN2i329jZ2UGz2cTHH3+Mly9fXuh6LiOTqxyD+Xwef/VXf4Xbt28jFoshFos5g9vr9fCHP/wBT548Qa/XQ7vd9nTqIpEIUqkUMpkM/vIv/xLvvfceEokEUqkUhsMhKpUKOp0Ofve73+Hjjz/GaDS6UqfnLPK4sMEJcPNIJBJYWlpCsVjET37yE6yvr+O9997D+++/j1AohPF4jFAohEQigUgkguXlZTx48MApgVgsduL312o17O3tTUUty8vLKJVKzvPsdDr46quvsL29jU8//RT1eh31eh3NZvPcBuemEAqFEIlEEIvFsLS0hHv37rnorlgs4r333sNoNEK320Wr1XIKE4CLECORCJLJpDM8kUgEmUwGhUIBrVYLn3zyCfb399Hv9xEOh/H69Wu8evXqSg3OLBAKhVAul518fv3rXyOZTKLf708ZXpURDVKxWMT6+jri8Tjy+TxisRhqtRqq1aoz3KPRyI0rKlsA2NjYQCQSQS6XQyaTQaVSwddff429vT28ePHiSgzOTSKdTuMv/uIv8NFHHyGZTCKZTGI4HKLVaqHRaKDRaGB/fx/NZhOdTsfX4KTTaZRKJfzzf/7P8S//5b9043o4HOLw8BDNZhONRgP/7//9PwBwTuesEBicBQLpi62tLaysrKBUKmFzcxPZbBYPHjxAuVxGPp93nvdoNALwxoOMRCLO845EIhgMBs4jj0QibsJz8E0mE7RaLedlEq1WC6FQCKPRCL1eD8PhEJlMxhmzXq+HZrOJZ8+eodFo4OXLl9jZ2bkReZ0Ho9EIoVDIUV2NRgOPHj1CPp/H0tIScrkcotEostmso5MATE1W0iH09CaTCQ4PD1Gv17Gzs4P9/X20Wq2ZTvCrQjgcRiKRQDwex+rqKu7cueOMKY0NjwuHw1NU13A4RCQScc5PLBZDp9NBJBJBq9VCu912UeN4PEaz2US328VwOHQGp9/vIxQKOUen3W4DeGPMGLUfHh5ie3v7WuiiWYBjJxqNIplMYjweO4Px05/+1NGT9Xod/X4flUoFg8EAhULBGfBMJoNcLoetrS3E43EAcPqA+gN4M94vQ39fFIHBWRBwwqfTafzVX/0V/uIv/gLFYhGbm5uO+qLh4GAibWENCT36cDiMWCzmBjcVh1JsamyANxP/4ODAGSoAKJVKLjL41a9+hVarhcePH6NSqeC//bf/NvcGZzKZOMPa7XbR6XTw6tUrvH79GslkEn/5l3+JDz74AMlkEtlsFrFYzD1HIhFnwLvdrpPjcDjEzs4OvvvuOxweHuLLL7/E/v6+k/tNTPbLIBqNolQqIZvN4uHDh/jpT3+K0WiEvb09AHCyoNK0VO1kMnGGQg3zcDg8dhzpW0Y86gz1ej30er0p5fqLX/wCP/7xj/H73//eReSLCDoyiUTCOTaU6d27dzEajdw4azQa+Pzzz1GpVPDw4UM8fPjQjcVwOIx4PI5oNOqixNFo5OY6AEepzxpvlcGhZ0lKgzfobUAsFkO5XEY2m8XKyoorCCgUCm5QApgyMvYBHHnkpD/oNVqDQ1ilqJ7sZDJxRouDnIZoZWUF0WgUuVwOsVjMKY95BhUbAJfn6vV6jtIAMBXh0GOk4e33+85ID4dDtNttVCoV1Go1Fw0CcHTnIoH3MpfLudwgFRnHAXAU5VmnZTKZoN/vo9PpuPcBTD3z0e/3MRgMpgyNfkev18NkMnEUJs8nn88jlUohFAq5zy8SeP3AUbTDiDGVSk3ptGQyiWg0iuFwiGQyiXK57KIh4I0e4HizMgZwY+PvrTE4VHyxWAxra2soFArY39/H8+fPPYWr1MciTP7V1VX823/7b7G1tYUf/OAH2NjYAABHLVgjoZOUUQ2h166T236HHmcxHo+nKBN77Pr6OpaXl3Hnzh2sr6875TvvRodeIq9vOBziyy+/xPb2Nu7fv4+f/OQnLgkbi8WcwaGBGQwG2NnZQb1ex7fffouPP/7YGfJ8Po9+v+8ioEVCoVDAL37xC6yurqJUKqFerztjwPtOI2wpWo5F0mQ0HMx90VhxHiolyWethKRBqVarCIfDKBaLSCaTuH37Nj788EPUajV8//33aLVaNyCpi2EymbgqyHQ67eTW6XQAvJENx1sul0OlUsH//t//G59++ikGgwFWV1cdtR2NRl0kCLy5L6PRCJ1Ox43Rm8JbY3BCoZCjlorFIlZWVtDr9Rx94fcZPs+7N5TJZPDDH/4Q77zzjqM2+v2+S2JzgCqHrtet18pn5mJ0otuSTD+Do3Sb/s5kMkEsFkOhUMBkMkGhUEA2m8VkMkG1Wr1SmVwXKD/KcHd31ykCeujhcNjlJmhwmOiuVCo4PDzEzs4Onj9/DuBNhVYikThGIS0KEokENjc3sbGxgeFw6JQWxwiNBv9WT1odO5bSdzodDIdDRKNRl1fgscwtEpZy45zudrvOYKVSKeTzeaysrCAcDuPFixezFM+lwZwXqV1eK51BsgjUcQDw7NkzfPHFF/joo49Qr9dd5My5ORgMnD4AMPX9N4WFMzjW+2FFRz6fx71791ypcDqdxmg0ctVSvBGNRsN5PpZm8vodhd9x16k40uk0stksVldXUSgUXK17p9NxA0cnuOVx6XWqQuC12PPWcJ7PmhznQOaDOQv7WZX
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 8]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3ZklEQVR4nO192Y5j2ZXd4jyPEYyMjByrUlJlabBGQ1JPcMuG0A9+adtv/gD/gv/EX+AnAzYMGOgnw7BgQLbUUg3qKqmys7Iq55gZnGfSD+l1YnHHISMyBgaZvgsgIpO8vLx333P2sPY++4Qmk8kEAQIECBAgwBUjfN0XECBAgAAB/v9AYHACBAgQIMBCEBicAAECBAiwEAQGJ0CAAAECLASBwQkQIECAAAtBYHACBAgQIMBCEBicAAECBAiwEETP+8XxeIxXr14hl8shFApd5jWtHCaTCRqNBra2thAOn82GB/I7RiC/iyGQ38VwHvkBgQyJt5Lf5Jx4/vz5BEDwktfz588D+QXyC+S3oq+3kV8gw/PJ79wRTi6XO+9X31m8jUwuU36hUAjhcHjqbzqdxsbGBnK5HP7lv/yX+PGPf4wXL17gD3/4A9rtNl68eIF2u43RaITRaITJZILxeIxwOIwbN24gn8/jzp07+OCDDzCZTHBwcIB2u43f/OY3+PjjjzEYDNDpdDAejzEej933+e/z4Lrkp8hkMshkMrhx4wZ+9rOfIZlM4uOPP8bXX3+NTqeDZrOJ8Xh8pnOVy2XcvHkTiUQC6+vrSCQS+NOf/oQvv/wSw+EQw+HwUq99kfLjWItEIohGoyiXy0gmk7h9+zZu3bqFtbU1fPe730U6nUa73Ua/38fz58/x8ccfo9Vq4eXLl2i1WiiVSiiVSlhfX8f3vvc9FItFvP/++6hUKhiNRuj3+6jX6/jd736Hvb09fPrpp/jss8/cOJtMJpcmx7eVyWWPwWg06uYvwbk0HA7PPO4AIBaLIRwOI51OI5fLYTQaoVarod/vv/W5zoqzyOPcBuf/5xByFt5GJpchv1QqhWKxiFQqhbt37yKfz+PGjRvY3NxENBpFOp1GIpHABx98gK2tLVQqFdy7dw/D4RD9ft8ZGn2FQiFkMhkkEgnEYjHE43FMJhO0220MBgM8fPgQf/u3f4t+v49ms4l+v4/t7W3U63W8fPkSjx8/Rq/Xw9HREQaDwVvdz6Llp+AE3dzcxN27d3Hr1i389Kc/RS6Xwze+8Q1Uq1V0u100Go0pw6qGmgqDr3w+j/X1dYzHYxweHqLb7QIAkskkarUanj17hn6/f2n3sAj50SEpFArY3NzEN7/5TTdWIpEI8vk8CoUCkskkSqUSotEoMpkMAKBSqeDb3/42xuMxBoOBU3qTyQSpVApra2uIx+NIJpOIx+MYDAYYjUZIp9N4+PAh7t+/j29961v45S9/iXa7jZ2dHTSbTXz00Ud4+fLlue7nIjK5zDGYz+fx13/917hz5w5isRhisZgzuL1eD7///e/x5MkT9Ho9tNttr1MXiUSQSqWQyWTwV3/1V/jggw+QSCSQSqUwHA5RrVbR6XTwm9/8Bh999BFGo9GlOj1nkce5DU6A60cikcDa2hqKxSJ+8IMf4ObNm/jggw/w4YcfIhQKYTweIxQKIZFIIBKJYH19HQ8ePHBKIBaLzT1/rVbD3t7eVNSyvr6OUqnkPM9Op4M//vGP2N7exieffIJ6vY56vY5ms/nWBue6EAqFEIlEEIvFsLa2hvv377vorlgs4oMPPsBoNEK320Wr1XIKE4CLECORCJLJpDM8kUgEmUwGhUIBrVYLH3/8Mfb399Hv9xEOh/H69Wu8evXqUg3OIhAKhVAul518fvGLXyCZTKLf708ZXpURDVKxWMTNmzcRj8eRz+cRi8VQq9VwdHTkDPdoNHLjisoWALa2thCJRJDL5ZDJZFCtVvGnP/0Je3t7ePHixaUYnOtEOp3Gn//5n+PHP/4xkskkkskkhsMhWq0WGo0GGo0G9vf30Ww20el0ZhqcdDqNUqmEf/bP/hn+xb/4F25cD4dDHB4eotlsotFo4B/+4R8AwDmdi0JgcFYIpC9u376NSqWCUqmEW7duIZvN4sGDByiXy8jn887zHo1GAN54kJFIxHnekUgEg8HAeeSRSMRNeA6+yWSCVqvlvEyi1WohFAphNBqh1+thOBwik8k4Y9br9dBsNvHs2TM0Gg28fPkSOzs71yKvt8FoNEIoFHJUV6PRwKNHj5DP57G2toZcLodoNIpsNuvoJABTk5V0CD29yWSCw8ND1Ot17OzsYH9/H61Wa6ET/LIQDoeRSCQQj8exsbGBu3fvOmNKY8PjwuHwFNU1HA4RiUSc8xOLxdDpdBCJRNBqtdBut13UOB6P0Ww20e12MRwOncHp9/sIhULO0Wm32wDeGDNG7YeHh9je3r4SumgR4NiJRqNIJpMYj8fOYPzwhz909GS9Xke/30e1WsVgMEChUHAGPJPJIJfL4fbt24jH4wDg9AH1B/BmvF+E/j4vAoOzIuCET6fT+Ou//mv8+Z//OYrFIm7duuWoLxoODibSFtaQ0KMPh8OIxWJucFNxKMWmxgZ4M/EPDg6coQKAUqnkIoO//Mu/RKvVwuPHj1GtVvFf/+t/XXqDM5lMnGHtdrvodDp49eoVXr9+jWQyib/6q7/Ct7/9bSSTSWSzWcRiMfc3Eok4A97tdp0ch8MhdnZ28NVXX+Hw8BCff/459vf3ndyvY7JfBNFoFKVSCdlsFg8fPsQPf/hDjEYj7O3tAYCTBZWmpWonk4kzFGqYh8PhieNI3zLiUWeo1+uh1+tNKdef//zn+P73v4/f/va3LiJfRdCRSSQSzrGhTO/du4fRaOTGWaPRwB/+8AdUq1U8fPgQDx8+dGMxHA4jHo8jGo26KHE0Grm5DsBR6ovGO2Vw6FmS0uADehcQi8VQLpeRzWZRqVRcQUChUHCDEsCUkbEv4NgjJ/1Br9EaHMIqRfVkJ5OJM1oc5DRElUoF0WgUuVwOsVjMKY9lBhUbAJfn6vV6jtIAMBXh0GOk4e33+85ID4dDtNttVKtV1Go1Fw0CcHTnKoHPMpfLudwgFRnHAXAc5VmnZTKZoN/vo9PpuM8BTP3lq9/vYzAYTBkaPUev18NkMnEUJq8nn88jlUohFAq5768SeP/AcbTDiDGVSk3ptGQyiWg0iuFwiGQyiXK57KIh4I0e4HizMgZwbePvnTE4VHyxWAybm5soFArY39/H8+fPvcJV6mMVJv/Gxgb+zb/5N7h9+za+9a1vYWtrCwActWCNhE5SRjWE3rtObnsOPc5iPB5PUSb22Js3b2J9fR13797FzZs3nfJddqNDL5H3NxwO8fnnn2N7exvvvfcefvCDH7gkbCwWcwaHBmYwGGBnZwf1eh1ffvklPvroI2fI8/k8+v2+i4BWCYVCAT//+c+xsbGBUqmEer3ujAGfO42wpWg5FkmT0XAw90VjxXmolCT/aiUkDcrR0RHC4TCKxSKSySTu3LmD7373u6jVavj666/RarWuQVLnw2QycVWQ6XTaya3T6QB4IxuOt1wuh2q1iv/5P/8nPvnkEwwGA2xsbDhqOxqNukgQePNcRqMROp2OG6PXhXfG4IRCIUctFYtFVCoV9Ho9R1/M+g7/Lrs3lMlk8J3vfAfvv/++ozb6/b5LYnOAKoeu9633yr/MxehEtyWZswyO0m36O5PJBLFYDIVCAZPJBIVCAdlsFpPJBEdHR5cqk6sC5UcZ7u7uOkVADz0cDrvcBA0OE93VahWHh4fY2dnB8+fPAbyp0EokEicopFVBIpHArVu3sLW1heFw6JQWxwiNBv+tnrQ6diyl73Q6GA6HiEajLq/AY5lbJCzlxjnd7XadwUqlUsjn86hUKgiHw3jx4sUixXNhMOdFapf3SmeQLAJ1HAA8e/YMn332GX784x+jXq+7yJlzczAYOH0AYOr814WVMzjW+2FFRz6fx/37912pcDqdxmg0ctVSfBCNRsN5PpZm8v2OYtZxV6k40uk0stksNjY2UCgUXK17p9NxA0cnuOVx6XWqQuC92OvWcJ5
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 9]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB21UlEQVR4nO192W5k2ZXdinkeGGQwSSZzqiwps0qllkrVRkvdasGtNoR+8EvbfvMH+Bf8J/4CPxmwYcBAAwaMhhsGZEhq1aQalMrMqsqBQ3IIxjxH+CGxDldsnuCQJIOM9F1AgJkRN27cu+85e1h7n31C4/F4jAABAgQIEOCSEb7qCwgQIECAAP9/IDA4AQIECBBgJggMToAAAQIEmAkCgxMgQIAAAWaCwOAECBAgQICZIDA4AQIECBBgJggMToAAAQIEmAmib/rF0WiEjY0N5HI5hEKhi7ymucN4PEa9Xsfa2hrC4dPZ8EB+hwjkdz4E8jsf3kR+QCBD4kzyG78hnj9/PgYQvOT1/PnzQH6B/AL5zenrLPILZPhm8nvjCCeXy73pV99anEUmFym/UCiEcDg88TedTmN5eRm5XA7/+l//a3z00Ud48eIFPv/8c7RaLbx48QKtVgvD4RDD4RDj8Rij0QjhcBg3btxAPp/HrVu38ODBA4zHY+zt7aHVauE3v/kNPvnkE/T7fbTbbYxGI4xGI/d9/vtNcFXyU2QyGWQyGdy4cQM//elPkUwm8cknn+Dbb79Fu91Go9HAaDQ61blKpRJWV1eRSCSwtLSERCKBr7/+Gk+ePMFgMMBgMLjQa5+l/DjWIpEIotEoSqUSkskk1tfXcfPmTSwuLuKDDz5AOp1Gq9VCr9fD8+fP8cknn6DZbOLly5doNptYWFjAwsIClpaW8MMf/hDFYhHvvPMOyuUyhsMher0earUa/vmf/xk7Ozv47LPP8MUXX7hxNh6PL0yOZ5XJRY/BaDTq5i/BuTQYDE497gAgFoshHA4jnU4jl8thOByiWq2i1+ud+VynxWnk8cYG5//nEHIaziKTi5BfKpVCsVhEKpXC7du3kc/ncePGDaysrCAajSKdTiORSODBgwdYW1tDuVzGnTt3MBgM0Ov1nKHRVygUQiaTQSKRQCwWQzwex3g8RqvVQr/fx8OHD/H3f//36PV6aDQa6PV62NraQq1Ww8uXL/H48WN0u10cHByg3++f6X5mLT8FJ+jKygpu376Nmzdv4i/+4i+Qy+Xw7rvvolKpoNPpoF6vTxhWNdRUGHzl83ksLS1hNBphf38fnU4HAJBMJlGtVvHs2TP0er0Lu4dZyI8OSaFQwMrKCr73ve+5sRKJRJDP51EoFJBMJrGwsIBoNIpMJgMAKJfLeP/99zEajdDv953SG4/HSKVSWFxcRDweRzKZRDweR7/fx3A4RDqdxsOHD3H37l18//vfx69+9Su0Wi1sb2+j0Wjg448/xsuXL9/ofs4jk4scg/l8Hn/zN3+DW7duIRaLIRaLOYPb7Xbx+9//Hk+fPkW320Wr1fI6dZFIBKlUCplMBr/4xS/w4MEDJBIJpFIpDAYDVCoVtNtt/OY3v8HHH3+M4XB4oU7PaeTxxgYnwNUjkUhgcXERxWIRP/7xj7G6uooHDx7gvffeQygUwmg0QigUQiKRQCQSwdLSEu7fv++UQCwWO/b81WoVOzs7E1HL0tISFhYWnOfZbrfx1VdfYWtrC59++ilqtRpqtRoajcaZDc5VIRQKIRKJIBaLYXFxEXfv3nXRXbFYxIMHDzAcDtHpdNBsNp3CBOAixEgkgmQy6QxPJBJBJpNBoVBAs9nEJ598gt3dXfR6PYTDYWxubmJjY+NCDc4sEAqFUCqVnHx++ctfIplMotfrTRhelRENUrFYxOrqKuLxOPL5PGKxGKrVKg4ODpzhHg6HblxR2QLA2toaIpEIcrkcMpkMKpUKvv76a+zs7ODFixcXYnCuEul0Gn/1V3+Fjz76CMlkEslkEoPBAM1mE/V6HfV6Hbu7u2g0Gmi321MNTjqdxsLCAv7lv/yX+Ff/6l+5cT0YDLC/v49Go4F6vY4//OEPAOCczlkhMDhzBNIX6+vrKJfLWFhYwM2bN5HNZnH//n2USiXk83nneQ+HQwCvPchIJOI870gkgn6/7zzySCTiJjwH33g8RrPZdF4m0Ww2EQqFMBwO0e12MRgMkMlknDHrdrtoNBp49uwZ6vU6Xr58ie3t7SuR11kwHA4RCoUc1VWv1/Ho0SPk83ksLi4il8shGo0im806OgnAxGQlHUJPbzweY39/H7VaDdvb29jd3UWz2ZzpBL8ohMNhJBIJxONxLC8v4/bt286Y0tjwuHA4PEF1DQYDRCIR5/zEYjG0221EIhE0m020Wi0XNY5GIzQaDXQ6HQwGA2dwer0eQqGQc3RarRaA18aMUfv+/j62trYuhS6aBTh2otEokskkRqORMxgffvihoydrtRp6vR4qlQr6/T4KhYIz4JlMBrlcDuvr64jH4wDg9AH1B/B6vJ+H/n5TBAZnTsAJn06n8Td/8zf4q7/6KxSLRdy8edNRXzQcHEykLawhoUcfDocRi8Xc4KbiUIpNjQ3weuLv7e05QwUACwsLLjL467/+azSbTTx+/BiVSgX//b//92tvcMbjsTOsnU4H7XYbGxsb2NzcRDKZxC9+8Qu8//77SCaTyGaziMVi7m8kEnEGvNPpODkOBgNsb2/jm2++wf7+Pr788kvs7u46uV/FZD8PotEoFhYWkM1m8fDhQ3z44YcYDofY2dkBACcLKk1L1Y7HY2co1DAPBoMjx5G+ZcSjzlC320W3251Qrj/72c/wox/9CL/97W9dRD6PoCOTSCScY0OZ3rlzB8Ph0I2zer2Ozz//HJVKBQ8fPsTDhw/dWAyHw4jH44hGoy5KHA6Hbq4DcJT6rPFWGRx6lqQ0+IDeBsRiMZRKJWSzWZTLZVcQUCgU3KAEMGFk7As49MhJf9BrtAaHsEpRPdnxeOyMFgc5DVG5XEY0GkUul0MsFnPK4zqDig2Ay3N1u11HaQCYiHDoMdLw9no9Z6QHgwFarRYqlQqq1aqLBgE4unOewGeZy+VcbpCKjOMAOIzyrNMyHo/R6/XQbrfd5wAm/vLV6/XQ7/cnDI2eo9vtYjweOwqT15PP55FKpRAKhdz35wm8f+Aw2mHEmEqlJnRaMplENBrFYDBAMplEqVRy0RDwWg9wvFkZA7iy8ffWGBwqvlgshpWVFRQKBezu7uL58+de4Sr1MQ+Tf3l5Gf/u3/07rK+v4/vf/z7W1tYAwFEL1kjoJGVUQ+i96+S259DjLEaj0QRlYo9dXV3F0tISbt++jdXVVad8r7vRoZfI+xsMBvjyyy+xtbWFe/fu4cc//rFLwsZiMWdwaGD6/T62t7dRq9Xw5MkTfPzxx86Q5/N59Ho9FwHNEwqFAn72s59heXkZCwsLqNVqzhjwudMIW4qWY5E0GQ0Hc180VpyHSknyr1ZC0qAcHBwgHA6jWCwimUzi1q1b+OCDD1CtVvHtt9+i2WxegaTeDOPx2FVBptNpJ7d2uw3gtWw43nK5HCqVCv73//7f+PTTT9Hv97G8vOyo7Wg06iJB4PVzGQ6HaLfbboxeFd4agxMKhRy1VCwWUS6X0e12HX0x7Tv8e929oUwmgx/84Ad45513HLXR6/VcEpsDVDl0vW+9V/5lLkYnui3JnGZwlG7T3xmPx4jFYigUChiPxygUCshmsxiPxzg4OLhQmVwWKD/K8NWrV04R0EMPh8MuN0GDw0R3pVLB/v4+tre38fz5cwCvK7QSicQRCmlekEgkcPPmTaytrWEwGDilxTFCo8F/qyetjh1L6dvtNgaDAaLRqMsr8FjmFglLuXFOdzodZ7BSqRTy+TzK5TLC4TBevHgxS/GcG8x5kdrlvdIZJItAHQcAz549wxdffIGPPvoItVrNRc6cm/1+3+kDABPnvyrMncGx3g8rOvL5PO7evetKhdPpNIbDoauW4oOo1+vO87E0k+93FNOOu0zFkU6nkc1msby8jEKh4Grd2+22Gzg6wS2PS69TFQLvxV63hvP8q8l
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Epoch 10]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAACvCAYAAADJ2GiOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB110lEQVR4nO29WW9k2ZUdvGKeRzKYHHKsrFJllUrWaLSkVjfcakPoB7+07Tf/AP8F/xP/Aj8ZsGHAQL/YMNywIUPqVg1SlUpVmVlVOXIOxjxHfA/5rcMVmyeYZJIMRlTfBRAkI27cuHffc/aw9j77hCaTyQQBAgQIECDAFSN83RcQIECAAAH+aSAwOAECBAgQYC4IDE6AAAECBJgLAoMTIECAAAHmgsDgBAgQIECAuSAwOAECBAgQYC4IDE6AAAECBJgLom/6wfF4jBcvXiCXyyEUCl3mNS0dJpMJGo0GNjc3EQ6fzYYH8jtGIL+LIZDfxfAm8gMCGRLnkt/kDfH06dMJgOBHfp4+fRrIL5BfIL8l/TmP/AIZvpn83jjCyeVyb/rRby3OI5PLlF8oFEI4HJ76nU6nsba2hlwuh3/1r/4VfvzjH+PZs2f4/e9/j3a7jWfPnqHdbmM0GmE0GmEymWA8HiMcDuPGjRvI5/O4desW3n33XUwmExwcHKDdbuM3v/kNPvroIwwGA3Q6HYzHY4zHY/d5/v0muC75KTKZDDKZDG7cuIGf/vSnSCaT+Oijj/D111+j0+mg2WxiPB6f6VzlchkbGxtIJBJYXV1FIpHA559/jkePHmE4HGI4HF7qtc9TfhxrkUgE0WgU5XIZyWQSN2/exNbWFlZWVvDBBx8gnU6j3W6j3+/j6dOn+Oijj9BqtfD8+XO0Wi2USiWUSiWsrq7ie9/7HorFIt566y1UKhWMRiP0+33U63X84z/+I/b29vDJJ5/g008/deNsMplcmhzPK5PLHoPRaNTNX4JzaTgcnnncAUAsFkM4HEY6nUYul8NoNEKtVkO/3z/3uc6Ks8jjjQ3OP+UQchbOI5PLkF8qlUKxWEQqlcLt27eRz+dx48YNrK+vIxqNIp1OI5FI4N1338Xm5iYqlQru3LmD4XCIfr/vDI3+hEIhZDIZJBIJxGIxxONxTCYTtNttDAYDPHjwAH/7t3+Lfr+PZrOJfr+P7e1t1Ot1PH/+HA8fPkSv18PR0REGg8G57mfe8lNwgq6vr+P27dvY2trCn/3ZnyGXy+Htt99GtVpFt9tFo9GYMqxqqKkw+JPP57G6uorxeIzDw0N0u10AQDKZRK1Ww5MnT9Dv9y/tHuYhPzokhUIB6+vreOedd9xYiUQiyOfzKBQKSCaTKJVKiEajyGQyAIBKpYL3338f4/EYg8HAKb3JZIJUKoWVlRXE43Ekk0nE43EMBgOMRiOk02k8ePAAd+/exXe+8x386le/Qrvdxs7ODprNJj788EM8f/78je7nIjK5zDGYz+fxV3/1V7h16xZisRhisZgzuL1eD7/73e/w+PFj9Ho9tNttr1MXiUSQSqWQyWTwl3/5l3j33XeRSCSQSqUwHA5RrVbR6XTwm9/8Bh9++CFGo9GlOj1nkccbG5wA149EIoGVlRUUi0X84Ac/wMbGBt5991289957CIVCGI/HCIVCSCQSiEQiWF1dxf37950SiMVip56/Vqthb29vKmpZXV1FqVRynmen08Ef//hHbG9v4+OPP0a9Xke9Xkez2Ty3wbkuhEIhRCIRxGIxrKys4O7duy66KxaLePfddzEajdDtdtFqtZzCBOAixEgkgmQy6QxPJBJBJpNBoVBAq9XCRx99hP39ffT7fYTDYbx8+RIvXry4VIMzD4RCIZTLZSefX/7yl0gmk+j3+1OGV2VEg1QsFrGxsYF4PI58Po9YLIZarYajoyNnuEejkRtXVLYAsLm5iUgkglwuh0wmg2q1is8//xx7e3t49uzZpRic60Q6ncaf//mf48c//jGSySSSySSGwyFarRYajQYajQb29/fRbDbR6XRmGpx0Oo1SqYR/8S/+Bf7lv/yXblwPh0McHh6i2Wyi0WjgD3/4AwA4p3NeCAzOEoH0xc2bN1GpVFAqlbC1tYVsNov79++jXC4jn887z3s0GgF45UFGIhHneUciEQwGA+eRRyIRN+E5+CaTCVqtlvMyiVarhVAohNFohF6vh+FwiEwm44xZr9dDs9nEkydP0Gg08Pz5c+zs7FyLvM6D0WiEUCjkqK5Go4EvvvgC+XweKysryOVyiEajyGazjk4CMDVZSYfQ05tMJjg8PES9XsfOzg729/fRarXmOsEvC+FwGIlEAvF4HGtra7h9+7YzpjQ2PC4cDk9RXcPhEJFIxDk/sVgMnU4HkUgErVYL7XbbRY3j8RjNZhPdbhfD4dAZnH6/j1Ao5ByddrsN4JUxY9R+eHiI7e3tK6GL5gGOnWg0imQyifF47AzGD3/4Q0dP1ut19Pt9VKtVDAYDFAoFZ8AzmQxyuRxu3ryJeDwOAE4fUH8Ar8b7RejvN0VgcJYEnPDpdBp/9Vd/hT//8z9HsVjE1taWo75oODiYSFtYQ0KPPhwOIxaLucFNxaEUmxob4NXEPzg4cIYKAEqlkosM/uIv/gKtVgsPHz5EtVrFf/tv/23hDc5kMnGGtdvtotPp4MWLF3j58iWSyST+8i//Eu+//z6SySSy2SxisZj7HYlEnAHvdrtOjsPhEDs7O/jqq69weHiIzz77DPv7+07u1zHZL4JoNIpSqYRsNosHDx7ghz/8IUajEfb29gDAyYJK01K1k8nEGQo1zMPh8MRxpG8Z8agz1Ov10Ov1ppTrz372M3z/+9/Hb3/7WxeRLyPoyCQSCefYUKZ37tzBaDRy46zRaOD3v/89qtUqHjx4gAcPHrixGA6HEY/HEY1GXZQ4Go3cXAfgKPV541tlcOhZktLgA/o2IBaLoVwuI5vNolKpuIKAQqHgBiWAKSNjf4Bjj5z0B71Ga3AIqxTVk51MJs5ocZDTEFUqFUSjUeRyOcRiMac8FhlUbABcnqvX6zlKA8BUhEOPkYa33+87Iz0cDtFut1GtVlGr1Vw0CMDRncsEPstcLudyg1RkHAfAcZRnnZbJZIJ+v49Op+PeBzD1mz/9fh+DwWDK0Og5er0eJpOJozB5Pfl8HqlUCqFQyH1+mcD7B46jHUaMqVRqSqclk0lEo1EMh0Mkk0mUy2UXDQGv9ADHm5UxgGsbf98ag0PFF4vFsL6+jkKhgP39fTx9+tQrXKU+lmHyr62t4d/+23+Lmzdv4jvf+Q42NzcBwFEL1kjoJGVUQ+i96+S259DjLMbj8RRlYo/d2NjA6uoqbt++jY2NDad8F93o0Evk/Q2HQ3z22WfY3t7GvXv38IMf/MAlYWOxmDM4NDCDwQA7Ozuo1+t49OgRPvzwQ2fI8/k8+v2+i4CWCYVCAT/72c+wtraGUqmEer3ujAGfO42wpWg5FkmT0XAw90VjxXmolCR/ayUkDcrR0RHC4TCKxSKSySRu3bqFDz74ALVaDV9//TVardY1SOrNMJlMXBVkOp12cut0OgBeyYbjLZfLoVqt4n//7/+Njz/+GIPBAGtra47ajkajLhIEXj2X0WiETqfjxuh14VtjcEKhkKOWisUiKpUKer2eoy9mfYa/F90bymQy+O53v4u33nrLURv9ft8lsTlAlUPX+9Z75W/mYnSi25LMWQZH6Tb9nslkglgshkKhgMlkgkKhgGw2i8lkgqOjo0uVyVWB8qMMd3d3nSKghx4Oh11uggaHie5qtYrDw0Ps7Ozg6dOnAF5VaCUSiRMU0rIgkUhga2sLm5ubGA6HTmlxjNBo8G/1pNWxYyl9p9PBcDhENBp1eQUey9wiYSk3zulut+sMViqVQj6fR6VSQTgcxrNnz+YpnguDOS9Su7xXOoNkEajjAODJkyf49NNP8eMf/xj1et1Fzpybg8HA6QMAU+e/LiydwbHeDys68vk87t6960qF0+k0RqORq5big2g0Gs7zsTST73sUs467SsWRTqeRzWaxtraGQqHgat07nY4bODrBLY9Lr1MVAu/FXreG8/y
      "text/plain": [
       "<Figure size 500x200 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#학습하기\n",
    "for epoch in range(1, EPOCH+1):\n",
    "    train(autoencoder, train_loader)\n",
    "\n",
    "    # 디코더에서 나온 이미지를 시각화 하기\n",
    "    # 앞서 시각화를 위해 남겨둔 5개의 이미지를 한 이폭만큼 학습을 마친 모델에 넣어 복원이미지를 만듭니다.\n",
    "    test_x = view_data.to(DEVICE)\n",
    "    _, decoded_data = autoencoder(test_x)\n",
    "\n",
    "    # 원본과 디코딩 결과 비교해보기\n",
    "    f, a = plt.subplots(2, 5, figsize=(5, 2))\n",
    "    print(\"[Epoch {}]\".format(epoch))\n",
    "    for i in range(5):\n",
    "        img = np.reshape(view_data.data.numpy()[i],(28, 28)) #파이토치 텐서를 넘파이로 변환합니다.\n",
    "        a[0][i].imshow(img, cmap='gray')\n",
    "        a[0][i].set_xticks(()); a[0][i].set_yticks(())\n",
    "\n",
    "    for i in range(5):\n",
    "        img = np.reshape(decoded_data.to(\"cpu\").data.numpy()[i], (28, 28)) \n",
    "        # CUDA를 사용하면 모델 출력값이 GPU에 남아있으므로 .to(\"cpu\") 함수로 일반메모리로 가져와 numpy행렬로 변환합니다.\n",
    "        # cpu를 사용할때에도 같은 코드를 사용해도 무방합니다.\n",
    "        a[1][i].imshow(img, cmap='gray')\n",
    "        a[1][i].set_xticks(()); a[1][i].set_yticks(())\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "참고: 도서 '펭귄브로의 3분 딥러닝 파이토치맛' ( 깃허브 : https://github.com/keon/3-min-pytorch )"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}