{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "到目前，你已经知道了如何定义神经网络、计算损失、更新网络权重。\n", "\n", "那么，数据呢？\n", "\n", "# 数据\n", "\n", "通常，当你处理图像，文本，音频或视频数据时，可以使用标准python包将数据加载到numpy数组中。然后便可以将数组转换为`torch.*Tensor`。\n", "\n", "* 对于图像，Pillow，OpenCV等软件包很有用\n", "* 对于音频，可以使用scipy和librosa等包\n", "* 对于文本，基于Python或Cython的加载、NLTK和SpaCy都很有用\n", "\n", "特别地，对于图像领域，我们创建了一个叫做`torchvision`的包，这个包包含了支持常见数据集（如Imagenet, CIFAR10, MNIST等）的数据加载器以及图像的数据转换器，即`torchvision.datasets`和`torch.utils.data.DataLoader`。\n", "\n", "这提供了极大的便利，并且避免了编写样板代码。\n", "\n", "在这个教程中，我们将会使用CIFAR10数据集。它具有以下类别：“飞机”，“汽车”，“鸟”，“猫”，“鹿”，“狗”，“青蛙”，“马”，“船”，“卡车”。CIFAR-10中的图像尺寸为3×32×32，即尺寸为32x32像素的3通道彩色图像。\n", "\n", "![](https://pytorch.org/tutorials/_images/cifar10.png)\n", "\n", "# 训练图像分类器\n", "\n", "我们将按顺序执行以下步骤：\n", "\n", "1. 使用以下命令加载和标准化CIFAR10训练和测试数据集 `torchvision`\n", "2. 定义卷积神经网络\n", "3. 定义损失函数\n", "4. 根据训练数据训练网络\n", "5. 在测试数据上测试网络\n", "\n", "## 1. 加载CIFAR10并将其规则化\n", "\n", "在`torchvision`的帮助下，加载CIFAR10非常容易。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import torch\n", "import torchvision\n", "import torchvision.transforms as transforms" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "torchvision数据集的输出是在[0,1]范围的PILImage。我们将它们转换成范围是[-1,1]的张量。\n", "\n", "Note:\n", "> 如果在Windows上运行时你遇到了一个BrokenPipeError，请尝试将`torch.utils.data.DataLoader()`的`num_worker`设置为0。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Files already downloaded and verified\n", "Files already downloaded and verified\n" ] } ], "source": [ "transform = transforms.Compose(\n", " [transforms.ToTensor(),\n", " transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])\n", "\n", "trainset = torchvision.datasets.CIFAR10(root='./data', train=True,\n", " download=True, transform=transform)\n", "trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,\n", " shuffle=True, num_workers=2)\n", "\n", "testset = torchvision.datasets.CIFAR10(root='./data', train=False,\n", " download=True, transform=transform)\n", "testloader = torch.utils.data.DataLoader(testset, batch_size=4,\n", " shuffle=False, num_workers=2)\n", "\n", "classes = ('plane', 'car', 'bird', 'cat',\n", " 'deer', 'dog', 'frog', 'horse', 'ship', 'truck')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "让我们来展示一些训练图像：" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO29eZBd53Uf+PvevnS/3hd0Y2lsBEhC4L5osUSLpkUt1hbbka3ycBzZnJrKTOSUU7EcJxUpNZOykhk78ZTtjGwrUlyOJVlRTEqWbNEUSVGkKREkSILYtwYaQO/727cvf5zz3XMa/V53A6AAvPj7VZH98N377v22e98553cWY62Fh4eHh0frIXSjO+Dh4eHhcXXwL3APDw+PFoV/gXt4eHi0KPwL3MPDw6NF4V/gHh4eHi0K/wL38PDwaFFc0wvcGPOoMea4MeaUMeYzb1WnPDw8PDzWh7laP3BjTBjACQCPALgA4GUAv2CtPfLWdc/Dw8PDoxki1/Dd+wGcstaeAQBjzFcAfARA0xd4KpWynZ2d13BLDw8Pj79/GB8fn7HW9l3efi0v8GEAY+rfFwA8sNYXOjs78fjjj1/DLT08PDz+/uFzn/vcuUbt12IDNw3aVtljjDGPG2MOGGMO5PP5a7idh4eHh4fGtbzALwDYov69GcCly0+y1n7BWnuvtfbeVCp1Dbfz8PDw8NC4lhf4ywB2G2O2G2NiAD4B4Mm3plseHh4eHuvhqm3g1tqqMeb/APA3AMIAvmitPXyl13lm6W4AQEhZX4wh64xdbZG5Jhi2+pjQauuPrdsG93TnNWpbjXrD/vL5ytvncs8ffUXb8NPqez7c8dqKf7/x2ovB59NHzgAAhjZvD9p++kM/AwCIGlnyc+fOAgBOnTsJACjlF4NjnYkYACAZkXvvGCGFq7M9I/c6foI+1Ki/mU7Rsga3DgAAFpYLQdvk7DIAYGJ2Qc4bGAQAbBkeogYj93zu+efonE1DQdtAXy8AoJjPBm1dXT0AgGqd/m1D4eCYtdR46dJo0JbsIjK9M7MNl+Ozn/0sDalWC9qMabQXVq6jXWONuZH6aERuCgXXNau+F67bFceawZqVf+vq1u56Rs2prVapH+WK3CtG+yIScfOm+sFzaVS/3XXDYZlnN28OC9FS8LmX1+cffvjjQVtfe+eqa9R5rVx3FxZkn6TTaXfzoG0pnwMAfP8HzwVtz337LwAA5docAKCWaA+OlXlJTTgRtG3augMA0Dk4jMtR5Q2VK8heixrqb7Us+6NYprEmk8mgza1fvUbzbULy7JloFABQKpZXjS+5KPO2Hq6FxIS19tsAvn0t1/Dw8PDwuDpc0wv8rUDINJAuuM2sI3lcMdx1G9zTmvoa92zeDy1nhTbY3/oax0TOM6va1sLhN44Gn+cmSJJ+6N0/E7RtGdxFx+ZFyt6yjaSAxRxJxedPTwfH0u0kSfR2p4O2ZIokj2JFpJG+4W4AwMI8SUrFwnJwbPosSRfLStIr1mn0t++/Ra7bQZLY3CJ9d3zsYnCsUqHvdqdFihruIW+qMwtzQdvEhfMAgNGLRMMUqyId1VkS6t/UE7Qt1+i6jSRwBy0Zro1Gmhe15XNC3OeW6HM8GQ/a0hkaV2W1ooYIf66psYC1R923qtMUQk7DVJZRHrubRwAYO3UKADB/USir/j7SlrbsoXUJt4sk6Xbgldpbe3rFZdg9cuWaSJcVS/2uqQfCcpvlfRKOyCtKa0RyXbrwyM5dQdurW0nznJkizc/U5XuxEH3OFopB23neO8W4jLlwgZ6FcBu1zRREE8gk+DwrT2adF24pJ89Xkvse4pmrVOX8quX1UFpNvkJ92mraVo2zGXwovYeHh0eLwr/APTw8PFoUN9yE4qDNGoEK2DDMfy1iUf8e2dXnrWGLaHzP1fcKrtqAHAr0xEZkljrPmVoC0lOfb9wxQcgdDzUndeORWPC5K9MBAKgUhTx03NTAwEDQNr9M39l+C6nNceF1ECoTOVSuiQlgfHycjikVPZ6gL0XTRF7GEmJyKbLJIJZUKmGR2qIJMSNkS9TPSTaJRJSJYdMQEUvJhKi3ne1kduhsk3vNLy0BAIYG++lYv4wzysRSd09H0DaTZVPPaq08wPppJpobwxbm5wEAh46IaevSxUkAwLZeMeXsv/NtAABnWCjmRbUvl+jzclbMUtUKEWK1uty7VGBTAdsiwg1MKFUmLgFg7AyZUOaUqSpqaYO862c+BADY9fZ7g2M1NmuEzGp5r5E50uHocQnKHh4iAvzg6weDtrtuuxMA0NsjAYalcpGvy92vSb+Ne1xUN0oVMtNlFLGe6KB1zpRoL8xPi2nQcfJhRea3d9J5e/feFbTVl2iOxhNk6hhbVmQq96NUlGejxuRrMiEP0RKvS507XK2K2StXpjXVJmSzQGPdOrQfG4WXwD08PDxaFDdcAncuffp3PPi8xq97w2/YBueb9aSojdxTtV0mUTckRDd4M5HEV3/bhKS1xj+zIdv89zZs5dc9ZOmXfHr6TNA2Pk6S4P473hG0RZJ0vf4dJAE99MGHg2Pf/Aa5Yp195YdB20gfSY6hqGybWIIk73QHScVzUyKpLFqSjsbPi6QXY1epbE65SrELWy5L5KhRpF1HG0nvZy9J1oZykbQDk18K2uZmSMpaYkkz2i7S+d6dIzTOAZH0zr78MgCgK9E8N89a0iUAWOv2ALuLKf+9s6Oj1J+cEL4I03yfOCBS6MyJYwCAxWWS5hanhJjN5klKW1qScRaLJKHWlETtpNA6k3W1shxDhdqiEeVWyXurVhRp32lLc6wF/fJQb3Csf2SEP12ZU8GZEyeCz7fsvI0+xERCvTQ1RdfvHwzaFpdprIkUaU1ac0iEaX61Irq8QHNk03Lde+6gjB7PPU9aUMWIBhM2NFexkGh5TmKPQ+aoNEB7vf00udpmMqI1pcK8xxQ5muM5j8dEUyyxxlIs0T2r6l1UZhI6osZSLqp12yC8BO7h4eHRovAvcA8PD48WxY03oThixK7lHX0Twaw2+VwrVpBODFuTOxTmSA0PKyIPl6WVSaXEZNDfQ2aBcl1IlkNHXgEA9GwSv+c7HnwQAJDoIQKoqEa19z2PAADGTp8O2ooFUgVnJmaCtiFWrxOddP8jo8eDYxOTpI7Pzc8GbV3drE6GxSSSYl/bepn1SWWKmC2QGmwUoXhhivyXBxPRoC3DphYX/ZZbFLV5ZpbMK+dmLwRtpdAa7OWGsTJSslCQ+XZ+zFs2bw7aps6SKel7zz0ftJ058SYAYNlF5GXFtGSr1FZRZgT3vERVVF/Z0FgqYfc9GVuN1fewmtMyxzx09nYFbdEIzeVz3/wGAODue28Pjj3yy79C1zViYtiImXPz0Nbg88QkEeAzs7J3Pvo+IkxLuVzQtrhIftQnR2nfdXQI8TzUQ2RjQkXZZpn+nT1ySNrO0Tx39u8BACzk5Pna/XeU7SOvYhMy7Lsd3/euoG0hR/0ce+W7AIDkPfcEx+K9ZC7ctU3Gd/YCjS+ZludwboH2boTNWNrEFo+SA0FUydCRq6jN4CVwDw8PjxbFDZfAQ0Hek7Vl2iA/yhq/UkaRBBvKo6JdeBocdve6lows6xFhq+4eps+XDo4GTYtn6Nd90x0jct5lqd0jiqSKsxvhUkEkGztJUsZTP/hB0DbDkti+d7wTAHDwrLhb9Wwm18L2LZJwMnuSpMVKWLbNyXNERL16jPp48ZJI28tZIslyORUFOEFRb68fFsm+jSXwXpYINw+LC2AiSjJGMiH3jMRJeqmLAI72JElF1QUac7UoUujBg6R9pPYIERWNSmTnVYPXtsZk1qLK25HgzJsRNVe50zT26TfeDNpmxmkeyuzqGKvIXqjWyivuAwCxGA9a5agJuyhil0dE+UbW+VhEiWqGtYP2mLiebt5O0uSxI5TO6MizzwTH3vVBiuhNqlwhgRstmkPv/dOniNC8/4H7g7ZNQ7TOU7NTQVulTtrGD14iLWXPLXuCY11ttK9DKdFEHXEbff7loC107HUAwL7/7dMAgJGIbJShSdrjL1WELJ6N01qFcrL/R+ZorUbYhfLIyKbg2NwM7fHx8YmgzeVSqpVkr/d0UJRyJEQadFm59VaZ0A6S9wAoVz2J6eHh4fH3Bv4F7uHh4dGiuOEmlLXQyPzQ2JTifLPVee5Iw2DOBtdtRCS6aLcrJBc2ZjZpfNniHKlxy6fOBm35OVKt6mtct1QStXmBzQgmLGOanyb1fjn7StA2NUk+txdqZCZZ2jESHJudIkIum1PkF0jlni+JKjg/R6aZQoHmqqwS9pgwqa7pjPhaJ2qk/i4tSdKf6Vnqx9QEjX38kqim27dRGtnhof6gLR4mH16TFN9fR/SVOZIvl5c+ltj3d1NMCKaXDxDpNXLfTjSDra9eIL22bsYXskR4VXPir+0I55BKwDR/giITc/OTQVuITSfbbyfScHlSTFDTnGyqVJeUo539NA8jt+wI2pzZIb9EqvrEqdHgWJ2jM1OKDKwuUj/nZ+eDti728U/GaG5PH5UoSmf+2NcvJpQyPxvxSPOEX7Nzkiyrt38EAHDvPW8P2mY4uvHSRSG0ExzJa9nZ+/AxMTfduot9ycOyBguTNPdzW4RQnMnQNaJHKZrSJQwDgOV3/SSNZYv4nld4zFFlprvwPJmQljppfZIx8Rqol8m8kq8I4Vzg569DkdYdXeRL35ai+18clbiMEJvWYuq9k1cJxzYKL4F7eHh4tCjWlcCNMV8E8CEAU9bafdzWDeCrAEYAjAL4eWvtfLNr/Dig83GsRWyuOG8D52s4Ycvo37kNCNfrJvYPrs/aRF1+yduXSTK9e69Ewo3NkDQZz6TRDPGESAhtnHtkbl6ImgiTge0ZJTEV6XoHj3O61b1C8mUPkQR25qjUUl1eIMkxnxVJwTCTWMySlFisiLToAkejKnLThEmKT7eLC1s4SpJ0xeX+UClYjxwbBQDMzonEfvfb9gIAtvSKZBWJ0n1DYed2KH1MMYl1+lVxccxNiJthM6xILey2gI6cYze/mQUi4TpCilXlVKKFokRihsK0zvW0zMfmEUqDuudeKmxy8uAbwbEwM49VI1J8/1aSHLuGRYJ0kZgFzqOSzYr2EecI3VxB5sMa6mddkWaX2A0uxuRoWZFrkxfILe+2ipwfWkPydtApfbdtpxSv586dD9qWlonkO3NGIjYTnEukwIRftaRzGXHK1jnZ19/n/fG9VyXnTJ5TwZYnSAMwVXm+tt1KGtenPvmLQdvF1+m7d9x7X9A2v5M0ouOcb2dgUeYvlCEtKJSXPfTaCz8CACzvkbah7SSNDwzSWrV1KOJ8nrQgq9vKzZ/vZtiIBP4lAI9e1vYZAE9ba3cDeJr/7eHh4eFxHbGuBG6t/b4xZuSy5o8AeIg/fxnAswB+46p6EEi5ze3d615io+fJF1Yfq7M0rAULlqhMvZF7zwZDeVhqMCpPi5PKnU27tCDuS2aCJMiFcfnFH9hN0ni+TaULvAwxJfxVSiQFdKrz29vYrS0pv9mZvZQXJXobSR6pZbHhvvq3fwMAOPTCS3LhJPU3Ehb3s2rFlY2iY7GEHEuwjTqm8l84bSakpLlanRPfG7JFhkJyjXKRxjI2JnO0wOXY5veI3fPWnSTl9PSQRNPRKXb3fI0W9eAZkcDf8/Z3Yz1oV1RnS9aFFEosJQYFF2IiTdVcThtVwGDffZR5UJf/uu9hCh7ZvJNs2sWiuCJuZbfRQlGuEeL7F1Xgm3HZ7thdM1qXxzrFLnK1qtYKeQyqtGChQPcwKZr75QXRHJbOUABUoSSaUTIm2f+aIRwRifLwMXJPVOZrZHhPnj19KmjLB/wKnXjPnQ8Ex3Kcu+XEski53+EgsUX1ANQXqK3EPIjpkP3UE6d+/9t/9ztB2+kTdP9HH5ZcQB9//DEAQLGHXB1LZdEsU4fo/MkzYtPOzxLv9Obrrwdto5xHJW1p/6e6hIfo5iyLpW5pG76L+ZiNV1S7ahv4gLV2HAD4b/8653t4eHh4vMX4sZOYxpjHjTEHjDEH8vn8+l/w8PDw8NgQrtaNcNIYs8laO26M2QRgqtmJ1tovAPgCAAwNDa1i9IzYUKRtDZNIEB2puQ13CbPaTNGounujCuOOcAurBPLFGRpWMSQFCTJ9HXz95rlbTIOxrOQySYUts9vX5CtC7OzsJDNCPCUqYbyd2qrJ5suVSMtNndqs8yxEOYdG/9Z90nY/uXRVOUN91wHJ0WF/9D0AQGVJTBco0nVtXFJxGo6KjHF0nDahxPhYVJlQgumtqsrsTqNnN8WQyrlh2Q0vqiIaC5xO9sw5cVPrzrBpht0Uk0khSRcWSf3duUPqcN6yi4jQxQUh/C6H0ZXZw6v3jMvhkYjSPUtqS4TYhdMsCvm6OEaEsM7D0bOV3CRjPXSN+KCYJkyU7tkZFwU3xa6Qp0+I+j43zvuUc9VEVGWOCJOCOj9KmM0ptib9KFVdSlr6d3ZJzBQnXyQz2j2f+FjQNpzhfq6RwigZlT2ciNPniQlJLXzyJLlyhlRelwmOlIzzHhsbHw2OtfWRG+NTs+JqWWRTXH+fmMyy22i+yiEaX1KZvW6/jcwULx55Tfo2QCbKc1lZq/4k5/YpUn+70rIubZx3KL5FXFCXeO+Gh8WNsMTurhfGyJQyp4j4UUP7rqLSJPe5MfTJNdbD1UrgTwJ4jD8/BuCJq7yOh4eHh8dVYiNuhH8OIix7jTEXAPxrAL8N4GvGmE8BOA/g534cnWtU3MwEhRQaHVUtgRi8Ot+JlEOT3y8nn8yf/qugbfkc/XJ23fEP5Roh+pW0jhTS4rYr+aQlcEdYqu66IIU5doGaOi6/zMP3kZQxtFPyPVzkIIHYGj+3FqI5VGpclkq5tUU4qKZ9YEjaOsmNqzpNEk3sjR8Fx3qWKUBCcaMos0tmQmVcgwtIYDfGiMqvEWEysqJIuGqFRDarK3pzzo8IXz+kCKlCgdrqqgRWiKXQ2ZwUJHj9GEnjdheNr7tNrl/K0/WNqnB+4jC5SQ4Mb0czXBwVF8o5DrDaNiLEaYVLZkV5zKWaSHoRl6HwnFzj3POk4VSUm1/NZU/kiuRGV6znbIuDPZIbJsS5UhbSQpjmQrR+gdQfl3EWOZdH1YoEHqo7CVyXg6fPrlhCRRV7GHuN8ozMnxMSePOO3Xyx5ptyelIUc8uVC3oVkTfSQy6Uutp8iF0+c+xKeuqsBLSdjtCzdzIq81yukxY7Py+ka/k87YXwFDkEbB6SPCZnLpDEW1Fuksluum6pV6TsNBPwiQid19+jtPD3fRQAsE0903v++P+msUyIO2M8QQmLTvfTWk1sFdfPrKU9XMyK48By5ArYS8ZGvFB+ocmhh5u0e3h4eHhcB/hITA8PD48WxY3PheIISLu6MaSIwnpQSIGPqfp1zl/XNmBUVlo4Vvpfh5SRJnfmWQDA7LGng7b2rVSlOjUoqk8tqIkpVw3uFbrsEADL7GhUqX2L46Tyzh4ngiQUk28cZ3PK2DkJbB24h1JqJsPNl0un481xRF60XcwZLjKwqgoZzHG+i+xxisLbsij3XOAovboiIBOdpP6mlY91lXkwVx6wUhBTjuXu6mjYOn9Bq811jpSrMVEYU2aYKJtTSoohNBy5WVBtY9OkcrclSCWNDosZxkVlFuaEkA3XaN0amVBmmUj7L3/0R0HbmROkGr/v0fcFbcM76LttnJ/ExMU8EGLzSvnsaNBWOMvEo2T5xck3yCwRaaP9sbgsposCb5mpw0JyczlGFJaUyYBzmoSDyuhihqlwetOaipCtl/giqrJ9mOtkVtl0ElHHkjmKfMwflDw6tQd+gsaZErL4cpRLMpYF9tceV+axHffTvt6yTcx6T3zrm3R/3nfLWen37CKNuabS9tZPnaS+qYrvBR5+uZPudcs9Qty/9nc0BqMiUzftIbNlpVein51N1ZZpsZZmpBhI3y1EhkdU3p+2JerTiarkuenO0XoPXOJq82XpY4Jz5VTUfCxZIkVnuprn57kcXgL38PDwaFHccAk8xGKrlp1d6aawFbKnjYmZNLumKQEukIFXZOtzpIz6iaozkeM8wiZGJePa7AnKPhZRHWlPkETVoySasKuMzTkjQooIdQJHVZFD1Tr90peUxLR4lCXvIo0plZQ8JjbE7lzRk0FbJcyER13ImMuxZZtkpytyBFpNuavF2C0rOymZ/o4ffgoAcOYISRcPKi/P+i6SWjLq/ATPfTwuBGuFJ6zGpFChIFKX8/t3+S2Axi6ijlSusEuYltidNK6/F+Y8HLWKSpDPGsN5zuDX0yZrFrU0p+0dQkT1d8rny3HhPEm8B16SKNTFS7RmfzktUvxd730IAPDAe38KAJDoUBnrOBowvigkVZrVzFpBpMo3v3+AvsvzWKoqsjZK51dVqbYQj7OqSMaCyx3D5GQkolw5eX8apc06TbWucqxYFzHMknpYPZGDnMtm7rtPBW0X3/FeAMDIg+9BM8RTMpYwR+/G26USSf82IjFdARIAePTRfwAA6OogQjGWkHWKcCX5kKoGny2Qu2NR7bsf/JDI+HMc4fnzP/Ph4Nj3vk9Eci0sz8bMLEnNtSHRLJc5q2UnZ4t89rt/Gxw7fnIUALDv1luDtlv5nbWtKFJ2jKcwZ2kM+Zis43yd9u5iQdw1ty8R2SlF59aHl8A9PDw8WhT+Be7h4eHRorjhJhTDCXV0ObjhdlKRfmK/RCTt7CUVrKeT1AyrKlNXmXCpK0IxIMkUOZpk386Txyh95f/7vSelH1VSZVJtorI98gCRVO//6N6gzTIJF2HSJyaaOrLLTAQpf+AaO9Yuz4oqnXuASFEXCDeqEtr/2de+ROcXxJ902yAlnYpkVDIr0cIBAP2bJNn+6GkiT0oVVZ3cJVW6JJFwbVUiaCo5MqG8cEpMEoU4F0hQaUOjLgGPIqdqTOQ4zkubP6IciVdWiYBcW0T5ZFeZgKrVVpPQ7nydRKrKa9BI+lgu0TgvzQgh29/JKUHjYlpIhpvLLoObiFR76NGfDtqmzpNJ6/wR8YWOsW9939A2AMDiovjxtjGJOdAuPvOTbLYZXhQSLrtIJpkEz23IKsKXx5xU9sIKq/axqpgA2jkRm0tqVVMJtMps8kuFZW6jvGeNeoZqjtjnv2HlN55mM0z4lKR9XThEph+sYUIZvyB+4A8+ci8AYNfeB4O2yQUaw8SURNR+7L2UZGxLH9eTVMnl3HNeq8scJbm6e13tuw89TESzLdMaHDkua+bMMAkV4JDI0VjvHRoJ2jo5Mdmdt1Ja2Qvj0sc8z1s2L2bRTpB5ZFNFzCSv95BJra1A58W0o0aYju0qy/mlqHI62CC8BO7h4eHRorjhErjLjdFRE6nk7YP0u/JzD+5TJ5JUEWXJKbssvlh5TuvZ0SUkRCyW5PPkV7JYIneop/77lwEAs5ckSs5FLcZVno8H7twPALilVwi0YpE6bFlCUXwKIky69gxIRFdPH/2SV8pC/MXiVMJsltO3fv53/zI4Vq9Tn955r5Se+pe/SjFTZyQFBF59Fitw4pTkxpidIsmnTRV5CDMRFVKqTrcll8VMnWiTc4potUmSMqJKcquy62axqMR/9hWMslinJWWHRlJ5VfXDEZWVyuriF+67ujBGrerIThVly1KRrdH5WgJPseY1OyeE0XKeJOr9W25fdc+BYSKLP/mp/zVoKy2Tq+Wx54XYbB8kyTvJFehrWdmTXQuk6ZQvChkd5wrxt24R0i5bofkLMyk5qxK+zbFLYV0z9kyeV9U8u8y8Lk3sCsmaydFIVdbs7p2kWVZVlfTTLq+MSzWr3AgLNepjvSKaV3ZcXBubQdUpweGDJAW/+oJolglLJ3R2i9bbzprwJz72swCAVFJH/bo9KU12iearnJO1rVdoX4ertCd7l2ROf/F9lL735TFx9/uJO0g7+N8/8fNBm2Gi/uhrRIju3LZN7pnmKE1oV1XadyWV16XM65ItUVuyLvt1voPOT+VXuG/gSuElcA8PD48WhX+Be3h4eLQobrgJZaSb1JB3bJd6jDv7qVuLS0L8FTmirM5pSIsFFcnHvtYHjwpZMT1N5oBoWM77EadLffLbFG2ZbBdSsMi+tnfdcUfQNjxIZNbSoqjGZct+uGw7qSmisFglFTNRFtWqOsvnlxUplCaV8e9eofqHrx86HRzbto2i0zZv3RO0vXiAqpkcmxA1+PJ6KPGEmGj6B8gEUFAqfaFE3+2Ji1mlh7Nj3TZMKn22LCr1Yp7mPp6W61pONKRc3xFmldGZnrS/tjOTGG3q4OMrEhixyu986lceozZNhNbY7hZWphlnTqkzgVwoi4lmmc1eI1skanDNekp8sLtbZnl+mUwGkbqYEcpZIiCjJfrbXhHSbuYI7bWZV54J2i5yWttcVNagwARujNO95nKyBuPzZGLYukfS4O559/0AgFCPmGEGNjmyn661NC/myBT7ei++IdXd3zVIBOGhlw8EbUdy7P/NJsq6yr42GWMzjyqaPpSVeWiGX/nlx4PP33v2RQDAq2/KPWNsdljMybP/9KtksrjznjsBALsHhZw//zKZr8pHJRVs5jgRq5XJ8aAtlKVn+YU07ckhFTH5y1l+L6gKRelpMnctvPhC0PZMnOatbxeZ2BL75Xl0cQhplSwuwbJwUlWWf8cs9eNSFyVAi1TFlLNrmp6vakTm8XBt/Tm9HF4C9/Dw8GhR3HAJvNeSC11pTv0SzRNxceasxCQZjgxLctRiWLmhlTi3w3/9qvwyH36dSI14TH4RpxboFzzEqS2rVfWLxzlCEnEhVMbH6dd67LxINJXKylweRuUWqbF0fvrMaNBW5JqOS8p1zLn3nThLUl28TaQppCivxgmu2A0Ar73G91SFJd6jvgJIfUEAyDPht1QSqdWwRFVR0ZkJjky8622UGjSvEtI8f5CkynxUJPAw52JxUagAEOFcDjVeH1tvEPGnogArLEnXlJuadW5hQZoZHd1KUmhZ9dtVU9f1Jl3NShfFqwQszC3TeeGUSsqfbFTndCWy87L/jvyQpLPpi0IWRxdJ6uvtIU1u+vRocOzsGyRpppVGMsPa4/SiEG6hCEmJeY4kzBdU6l0ew6btI0Hbg488AgAoZrXhApsAACAASURBVGS9d3FxClfI4/DhQ3KNOSa052T/TY9ShOLStIwv4AX5njr9cY6fObtTUum2baLPil5dhX/1z34z+GzHfgUAcGlSyMBNHHXclpZn2fIztFikK1+8KNrpmf/02wCA7rNCoJoKzVdK7ZkCR4AOztPcx0MymGyezh9QkbizOdo757OS7+TCAknLlrXUuc0SQdpp+R2UljVwEzaVkuelHungv6yd1sXJAlHWNo2kki62S06YjcJL4B4eHh4tihsugZ84S7+wFyviKB+PkwTeoaSMFFfLdmWu4qq6QY4zlr1xUFyUpiboV89acY2rhehXdcsI/fpGVRBOlMtQLS6Kr16mgzPWqSIIJzm/wptvkpRTVWWp8pyDZFGVo1ri5P0Lyp1xvkL3KqdGAACJHpFsbJx+paMRlX+FS2tFas1/b+MxWcpFruxtlR9XOkljjkWVdMt5L+aWyeUuYkQq7eAAlDll47fsipZQ2QLLHFji5My6CsaJstuXtjc7s3VNlfgqukrknE8lrKI3KhyYU1E2cFtnV86KjO/yIh26hN0S25XPjYmEtf/25nllgn4ti9SamyZJNmRljiIV4hhOHCS77RuvHguO7dtJXMo7Pyzp9P/2O1SmrnBIuJrODrLLT54nTXQxJ2NKsEZUUoEuc4eJDylEZb0XXiLNM875RqbmZQ/XSrQXc4cl70+ES7vNL8g+dajzalnFW+Q4YOlMSdYsM09jr9vmMnhFuS7O877QsSoZ3ltx5V4H1pbKs+R+Wa+LRDtQIDfgaEQWNw56lsZC8hx2sbF+O383pjiVuQw9Bz2dcs80a5lTyrX1/io9hxOcV2Y2L2NJdpBrcKqu8tYY0piNCsQ6nhgBANw5Tf2ebpP3yFKMgvm21cVNsqhyLm0U60rgxpgtxphnjDFHjTGHjTGf5vZuY8xTxpiT/Ld5XkkPDw8Pj7ccGzGhVAH8urX2VgAPAvjHxpjbAHwGwNPW2t0AnuZ/e3h4eHhcJ2ykpNo4gHH+vGyMOQpgGMBHQLUyAeDLAJ4F8BtX2oFihtx0ikuiKkXyRA7N5UWFDVuOSmOVLaJMDFWuqZjsEZUwVaI0qJWiEDXJNH03lSK1q0vXwIuRClRWaSm/8zd/xd+TvmU5x8XMFJEPhZIiw5hknK+J6accI9ImNCyVxeNRTpUZYXJI6fuu2r1zjaSLcEpQ0zxSa3ZaIsvm5uhzNKJqflZYlVauTH0ZUukcoZhQuUIcybM8LulTi8v03ZzyIwwxmRzjyLVSSUwAtcjq6FYb5KhZbZop89zq9LNVjv6rlOW6cnc9H1zUw9UlVceq7HZYUeagen192WV2QuY0z8UEihVVBIHNGMscwZrLy5rNW5qPN2bFLfAi1/CcnJIo0dkJUq9dSLIu2tHFJq2Z42Ka+fYFIvDKqqYjeA8abqtpV06OpE2o3CmdfLxcU3MUcqQa/Vu7fuZ5TQ8dln7YQTL7fbJyWVIehSee+37w+aXTZCK6eFbVGeVbpFXNyvsS1N+Rb/wuACCl9lO8QGasGUXmf5Ejsp9ekPl4iEnMXx2mMVtlGuwu0B5P15TBIExrFSmJ222Ko40n2YSS7pZiD1GODg3pSMwqHe9ZkPnYF6Fn7pVecpOs52S+M+xWHFdj6Zym9112CzaMKyIxjTEjAO4C8EMAA/xydy/5/ibfedwYc8AYcyCvwoQ9PDw8PK4NGyYxjTFtAP4bgF+z1i41SszfCNbaLwD4AgAMDQ2tEiFDMfp1qqqgiWqFpHKdGiPG5YnKHDhS0+WrWGrd+ZD8HrWfJdLm9Itfk2twUYhcln6Zy+W54Fh3F1fvjomrz18fJCk+HZNf8Ft3UBL6RA+5FeWKyp+vLc3XlWs4ibBuhJhzZeGsI/KUm51Mq1qalV52DbE4K8RVG1c2b0spxojJ1rgKNIjF6cJtYZr7nJIuRwZJoqioogmX5khynFPkV50JrjLnngmF1T1dkg5FDtVcRjmdYY+zG5Y4ub1V7oGBu6ZKOuPmQe/AeqDFODJTjoYsfc7nRbuanab52iI5+Vdh/JIEh+Q5cKVjSDJkTkyQRHhxlM5LqyCpi5dI4jx+4nDQNnaM9nBCaW2pQGXgYiOK0Eu5vCSqTF2ltMhjUh3l/eMUtFhd9lrJBTgp7W2B71lW16jw/nQunyG1PpGQK7qiqsGzNhEkM2qA508IaWx6ad46t4iEWlii568ImbfXz9Haf76NAnjao0L+zzhCcULyy/xwjufNyvPydZae93bTsQ8oDbCcoEHP9+kiLfQ5WZA2w6R1hl0F4yvIf3omwsr9MVSn82t1EVInO0nK/48nuLRaUebq33YMUH+yopF0brsbACCztj42JIEbY6Kgl/efWWu/4fpnjNnExzcBmGr2fQ8PDw+Ptx4b8UIxAP4EwFFr7e+oQ08CeIw/Pwbgibe+ex4eHh4ezbARE8o7AfwSgEPGGBfq+C8A/DaArxljPgXgPICfu6oesDqn81pY9smuK7XIlsnYH0mTah+NqJqGTv1Ujt3hJKm18zOST2V+mk0EhlSseEKlieUK5z233R+09d1KyepNSFSfWVbjIqyiF5NiMnAuwqGQntb6inHSR6fmM/G2dmaODSFkxPzRxqTr9q2DQVuUE/tHFSkEJvKOHD0LAJhW5FoHp2DdvVnImw7OHXNaqeOXZoh4cblWEsl2ubwzGynzh8uPUlcRm6Ui+xSz33q5pAjchmCzgG0ufxg1p86cYpQJwJj1lU8TEzK1hBj3TcY+MUV76+IkKZ/datnb2Ud8U0bm4zwTlBOqnmUnk8CuZmVYEYvmsr8AUGETi1Vr4EhLy3+j2jTCf+tWruJmvqwLoPD9A/JX7dcYR8PWFMHf3kGmw6ginC/H8GYhCnt+/oMAgHxO1vbkaSKJp8fFlJnj/CHjeyn/y1harr/cQ8aFIwf/IGjL8DwU42K6sBzxerhE5r2fhJhhUpyPZmBSTI4VjqyMVGVPGH6X1DhqNq22S5r3cFKtd4VTQ6fVHGWX6F5ZvmfUqHw+24mpzPRLgYvEnSP04dLGjRkb8UL5AZrn/nl4w3fy8PDw8HhLccMjMQPmRUkDxkmLVckTUC3z5xS5+4WVxOm+W1PXSLWT5NO3RbKZOS+YMBNvqS6RLvu2vw0AUK4owo0z2iWZcACAJZaQXAqUupLOw5bLf0FFCAbSopIIA+km5BqwCvUGbWtgeEiivCKcgS63LGRjjIXPhCKn8gWSEvsGOM+DIlrrRZJaknFp6+0gkrY6pOaDJY5ZTppfUUn/Q4alSy0t8v2rikytBS6TXCRjRXSfk56VdMSfLeoNzmNpVB1xUqUuIpFMq7wUTdDdLxrMaJokzqVJlZ+nRGNOt7FWlpX5jrBUXpkUaSpR5myBMXE/y7ImEuJ+K0W04bZwkrIeYN1Fn/JfndMm4EgVqeuO1pRY5rhTG8yfdKTMGoOubJ/uYs3CqNpkl+H3/81n5XzOCNg1JD5ybX00pwODkvdn3zBJqcdPfxMAMBmXvVY/NwoAaA/JPIcHScrvqkhGwzyXKTvN7oHJtz0QHAu1E2FvkvLstyeprZIUMjWa4MhvVgD6VXRr2107aUwluUbnZ/8/ulZG5j734kG61vQf8Y3kXZT92PsBAPFu0VJii+Q0AQm8XRc+F4qHh4dHi8K/wD08PDxaFDfehOL8dlWLcVFyYVFpIp0U0Vjj6MWVTtFBhqSgJdM/AgC47+MS4V926r2rrZcSNbrO1zj13JeCtmKOzDbJTlGlwy6qy1XvXhEX6O6/ui6kUR1e5cesfeobRBJuBDNToqonWUXXyZ7akqROdqWFVMsyWWjS1N+d+6RwwBT7QIdVN0occZhsFxNHdx+pkfNFOr+i1sAF86n8TwGBq8/TfvCr4cwlitCW1Fmrzg78v7UfOC9RtiCq97kxmq9b7m5+5w5FQHZkyHxUU5XI+9hHuL2NzAJLZyTVbJWnvqiKakRLNM+darfXA87QrhqRI4FXFDN35KsiGV0hDDGlqJS+bjrUNWp2tdnS7eI6VvejwmYefY1Mx/omqL6UOBrU2Qf6zClJqhU+Qm2Dt+4M2vb/wscBAJsylOK4TxVC+cGblAQsmZB7FznaN6rIzlCek5310Lp0/Jv/EBwbSNE6mrIqzLFE5hGzIJHfhRmKQM6y2W13SM4Pp8lck9SRzq++TNcaF7/uGZ6w+xNkFqqElU/7HxAROz4tkc4zj36APnRvPK2Ul8A9PDw8WhQ3gQTeCOxep6LSbITJB7PynKZwpb7aJcI/6ooOOAFfEYU1Jv5G7v6AnN/exf1YTZYFvbBXJin/uOAISQDobKfcEgPDkqZ2dobIt7E5Va2df+kzvTRH8YyQSUmupB1ShOLWISJ2toTEdbL7IrmCleskgUwq0q7IuU1qK6rHM9m4glTbyBw2Oke1OUmTI+ZCyk0wEuZ0vKqSezi8/taPqRS9USYGa/WyOk7XS7aTRDinCg0UmNAuK9e0Oke6GrXvXOkyJ/nqQhSVBqlaG2kYziXTXcMqjabeYH/aBlK2SN7OIUDg3BR1zGUhl8N6eMe73xF8PsraSfmCkNcVlpSLSyKZ/vUT3wEA9HeT9jM3J27AUxMkrbZ1iwR+G6cWfqBb1vZblksEcjTs6Od+PTh2Zokk6cWskNG5HEnecxXpRyhGkvr5D/8SNdx7W3BsO5OjiaiMJfdX36W/EXk2TnfQveKdRMQOKIa68vBHAQAzfTKWbfspAv3EE9/FRuElcA8PD48WhX+Be3h4eLQobrgJxXBFcqPMFCbw6V3tz9pQlXaRaCsanUqqFL+A+2J1UUdHssqd6RE/1RpWJ5u6/O6Nkno1MqusnfyrAcG5wWRhDt1dYv7Ic+RcRf0+d/WTWSWqKgilerpXHJueEhJnYo4T9ihluo05qURSfH9Zg0VXhtTWVFIqjCznSCVdVORQmf2/bUir+bQNG3OZzuSyOuXuitVwScO4u+Go9DGVILIxpSoJmQ342adSMhaXOjYcl+vGOP1urJtIrXxEiLQZttNllbN1nqvS5FVa26pdabrQ5iTTYK+vZbFba0TrGKCCz6vpTUVGq8aZSU61uwYB/dMf+lDwefcZivb94u/9XtBWZbNlCLInZ05RlfmLfFlV1AehKu3JkFqDt/EGfGRJIjEXOihx1jdKFOH5+ZeeD47NciX5YloI1lCa17ld/LozSXo2drJN63YjhHaZk2VNFOWexQ+/HQDwymmJKj3yEtVRHbmLapZ2KkL2nR/7MABg105V7Z7/PuVNKB4eHh7/8+OGS+CObFrhRhj80ut8Fu7TahmhkdQg19f/WpldQueTcClHa0piD/JpqGu4fqwl7eg8HI0IukC6tiuLEKy41RVK4AP9QtbmiyTR5FXa0kFOmq+J21mO1JyaJ4Jp8qKkT12eJEnCqkIKiRTlh0ioiDWX7H/nDiJM998pfnmzC0RAnT8nrlUFjupbmBep/OQJcg9zkaPW6twpJO1YpQnEWBpOqAKLCc6f4+pphjVh6b6niM3w+gI44kmR0myUrj+bFXeyBCfqT3Cy/9mizNU5Trm7Is8NS/RZ1Q+nkTgJPKQk2kh9NYnZEKt5Tel3sMdWaysrJHD+br3BNaIN2koFGmu90jxvTUK5rA5vITfg3LKI1HPsqqe1mjrvN8PaUlWlYI1z8pGkihh+uI0k5fODslaZedL8wob26djOvcExmyHSMGQkb5LTtIuqbu3cRYqKHDv1VQDAxVcOBsdmc7Sv92TE3a+XUyD//8++ELSluuj+txwjzaHatS049tKBVwEAyyp3SscGXDMvh5fAPTw8PFoUN1wCD69O7fBjxMq7mAbHNpKlbv27bGw0b+WY021ir52eJ+khHBNJ5QKXRivmRaorc5mtiiUJobIgrlX9nKy+qOx2MbYHRpSUOMgZD2+9Yx8AoKNHymN1zLGNUAedsGQ8NjYWtF28eBGAuNlZXfmd3bnicZU8n6/XFZcxD/RSPpeePvprlaYxM8X2WlVWLK7s4c2gi3DMcbX4E+dFS+ll+3b3Fgo6KRdkXrJcxTym+IK2DLmfxdulRN/iIgWL1Vxxj5pcI1RfK8BJ4DgBW29QXMG4XDxKY3WBOTqfCv8NX/bvFZfS/6i7ezbfxS8fei347Fwd44qbCDn3RJUhs8725R7OQ7MwK26vUcdDlKTISCZM83Z+8z1y3QKVfouwm9/yrNiq79hF+/SF774YtBUXaE1rKkskyi7/C43vwgkJ0orxHh5VWm+S5yEJ4UFc7pso750z7TLOH/zVtwAAbzt3Omjbe5sE0m0UXgL38PDwaFH4F7iHh4dHi2JdE4oxJgHg+wDifP7XrbX/2hizHcBXAHQDeBXAL1lrm5eobnb91fnjm2cfX/lN+Wibf60RsVNfi1hcF2uZYVybbrXq/43Pa3SsYR/X6NXp02KSuHCJIgK7eiQVZzJF6mTMCnkTZdV7mOtfLiurQpmJmqT6jY/EiJQpKxfArSMjAICBYXLdOnFMVML8Aqmuk+fOynXLtEV0getOzt3iUniGIiqFbS+5J27nWqQAMMcReUWVLreNv5tgNTupTEplVofz8+LiFQmtzldzOVyRCgCY4DSyJ0+MBm3zWVLlBzYROVXKiWrvNnQpL20lV8Fd2ScC80Hd5QSSVXZpj3UEaZXT8dZqmtwjtb2th0i1mkqbWylX3BeDtiJHUWpTn0tnG2HTUkWRk86MsMLRYNWH1fj2M38r9+TCCEtqzco8dl3MJcQRsguzRHLrrNERXtuicsM8XKV9NKRyipzhHLCLnPsmNC2E+cHJvwYA5GbENAN+bYWVyarqUhsHN5c+FvmlVZ+SFLM2RWtUVcVccvN03aejdGz7PnH1/clHHqLvKXPhPEdJX4n7wkYk8BKA91pr7wBwJ4BHjTEPAvg8gN+11u4GMA/gU1dwXw8PDw+Pa8RGKvJYIKhJFOX/LID3AvhFbv8ygM8C+MMr7YC9Mm85NPp9CpwDG0jx+hfKHb4WCVzuYVfds/EVNhDo0yDZfqMcFqHmfBFCYZEe7r+fcirUVXXyQpbLbSn3KVeGLcTBPe0pcQ8ss8SRi6gCDVy9u6MjE7R19ZKUf26MiMKZCcmFMnV2lK6lXKWcRBhSAUVDXXS9cFAkQM6PMhFrCiLJbubAo9CgkEiTLIFluRBFVfkJGvaDU4LsisITzVBXhGKFq7CfPyXaxBS7we1i7aCmr+mSBqrMlJu2b6f+bxV3MkemOkIvHhcSLNlG89HeJu548/Mkpc3MCOG8Yw/df2Qv/S0qMq7GpOClU6NB2zf/9E/pvIJoQaEEre39734PAODwa68HxxaZBF4hgfP+WIvzj7bJfmpnbWK+TwovVBdpLKVsVn2HxhpnF9GwSg5T5WIqeVVkZIaLgVTe+Lug7cTADgBAbZY0rlxJ1nFphsccUaPhe7V1dwRNbZyLpcb3jCrSu3+Yiqd094lEHcpQn/JF5QKbpbkvsSPA7tt3B8cyvK8zabnn/lvouf3umb/ARrHRqvRhroc5BeApAKcBLFiR/y8AGG7y3ceNMQeMMQe02uzh4eHhcW3Y0AvcWluz1t4JYDOA+wHc2ui0Jt/9grX2XmvtvSkl4Xl4eHh4XBuuyA/cWrtgjHkWwIMAOo0xEZbCN+OKKrkpOE3mLXYEv/Y6743hzBiBhePHdaMrxD33bQ8+38E+2Qd+9GbQ9uYEkZyJiPLr5nSpiQSnjlX6cIwJy7ry852cZuJHpVk9c+okAGBmgfyZF6aETLo0SuaGtrQQijH2A44nxZTTy7lE2vgHvlgUlXp+gdTg5SVVB7GL6xUqMtXywiTZ79pCCMgI+wqHwjL2ULh5LUcHTeSNs2loWZkd6mxGyM4T4VtSpqJckQlOJncB4Bd/7f8EANyyVyIDnX90lcldXfsz7CrWq022uEiE3OysELI9nRTBl06Rf3lO9THC1yssiwmqytfTMZQRbktwjhCr7E0FV4RDnV9jotmEm5PBP/vIB4PPd+0hH+dzs1Ln9s0XyBf7xWeeCdou8jzPsqmoop4v5ysfsTJHZw3trc5BMQAccrUz2Vwz3Cs5TjZtIvPb/nslYvjO/fsBAD1DfUFbW4LmMpMkU4pO+VLiqEujzDBlrp+bL8q+Gxwg86LlfsdVLdR2Jp6NipZ2JO2VYF0J3BjTZ4zp5M9JAD8F4CiAZwD8LJ/2GIAnrvjuHh4eHh5XjY288jcB+LIxJgx64X/NWvstY8wRAF8xxvxfAA4C+JOr6sFVSt4bJR1tg9Le5hrEZl0bHVhJSK5JyF5hFrkr7eH50ePB50ScxIWFOZHSFuaI9GpLiSjR1pbkvySpZPOSPL/Amdb0/GU6SDrLZeW8JZaMS+y6Njc1ERyLRBx5KKPJdJJE06WyJ2baqc0waZjNyj3LTAplOoTsSaZJel/MijTXwQRoIsmSofI/izspsSTbvb4B9jwaEy0h3UVS7i379gdt3b00hhq7JM4pN0Ini6cy0u+tgySNL06K+9mhgxStODZK+WIiK0r0EcrKndFpBUUVvWg5wtS5+5VVxGmMCbqKytPSy1KoFt8iXHJvfIb61rtZsnJ2baHzK+qeI7eSFdWEmr9Cdig3VlOl9XjbZpFy93/yHwAAPvZxyVp44TxF5V64QAp9flm0iZkpipj886/+adD2xBztu/iyFNPo2kwk8ef/xT8DAGzZvUPGyVrNQLeQqakIrXM0ImOJ85o6jVu/M5wLp3Y0cNrSirxG/JUSk8oh5brqpHEjKVxQu4qX4Ua8UN4AcFeD9jMge7iHh4eHxw2Aj8T08PDwaFHc8GRWku5yY4l7Nmw6Cf5uNCHQxtQXE6hUwRfl4AaKBOjvBj7fK4bE/uUrrtU8mtOhsCgXuXCWiMSZSTF19PdQcqCkqhQei7Mvb9gl8lKkIKd0TabFcyjTSWp2sSQqerFA6v1yllTdbFz8ZcNpUhM7OyTtZnuGfL4jKrItmyViLse1EatVGUsqlVnRVwBIcQXymvJ9z+fJlBON0lj2KLW5LUXnv/n6oaBtOSvmgGZoV2abX/30rwEASv9ICFZHxJ46TkUIamquymyyGBwaCtoiXBEjpooJjOwi3+AOTk3a2SlzlWTyV+/NOj8n9ZoQspZzJkcSrJarZyTqolyVH3OYz9cmgyj7n7tIyKhK1RuKcVEN5Rffxmu61p6cLwnxPHGRzHkpsUphyyYiHjNtysw0Qm2b2WyTTsicZtlc15GRvo1xDdZ0u8QmPPLTZJK5bx/VsUwrctBFAucU4Rxp4zlSr4oQz68zIWqLmyP7dTSvi2TVhUdqTFC76OCQjqh15LV674V/HCSmh4eHh8fNCXM9q6oPDQ3Zxx9//Lrdz8PDw+N/Bnzuc597xVp77+XtXgL38PDwaFH4F7iHh4dHi8K/wD08PDxaFP4F7uHh4dGiuK4kpjFmGkAOwMx6597k6EVrj6HV+w+0/hhavf9A64+hlfq/zVrbd3njdX2BA4Ax5kAjNrWV0OpjaPX+A60/hlbvP9D6Y2j1/gPehOLh4eHRsvAvcA8PD48WxY14gX/hBtzzrUarj6HV+w+0/hhavf9A64+h1ft//W3gHh4eHh5vDbwJxcPDw6NFcV1f4MaYR40xx40xp4wxn7me974aGGO2GGOeMcYcNcYcNsZ8mtu7jTFPGWNO8t+u9a51I8FFqQ8aY77F/95ujPkh9/+rxpjYete4kTDGdBpjvm6MOcZr8fYWXIN/ynvoTWPMnxtjEjfzOhhjvmiMmTLGvKnaGs65IfweP9dvGGPubn7l64cmY/j3vI/eMMb8d1dtjI/9Jo/huDHmfTem11eG6/YC54o+vw/g/QBuA/ALxpjbrtf9rxJVAL9urb0VVAf0H3OfPwPgaWvtbgBP879vZnwaVAbP4fMAfpf7Pw/gUzekVxvHfwTw19bavQDuAI2lZdbAGDMM4J8AuNdauw9AGMAncHOvw5cAPHpZW7M5fz+A3fzf4wD+8Dr1cT18CavH8BSAfdba/QBOAPhNAODn+hMAbufv/AG/s25qXE8J/H4Ap6y1Z6y1ZQBfAfCR63j/K4a1dtxa+yp/Xga9OIZB/f4yn/ZlAB+9MT1cH8aYzQA+COCP+d8GwHsBfJ1Pudn7nwHwbnDJPmtt2Vq7gBZaA0YEQNIYEwGQAjCOm3gdrLXfBzB3WXOzOf8IgP9iCS+BCp5vuj49bY5GY7DWfpcLsQPAS6CC7ACN4SvW2pK19iyAU2iBimPX8wU+DGBM/fsCt7UEjDEjoNJyPwQwYK0dB+glD6D/xvVsXfwHAP8cCCpb9ABYUJv4Zl+HHQCmAfxnNgP9sTEmjRZaA2vtRQD/D4DzoBf3IoBX0FrrADSf81Z9tv8RgO/w55Ycw/V8gTcqpdMSLjDGmDYA/w3Ar1lrl9Y7/2aBMeZDAKasta/o5gan3szrEAFwN4A/tNbeBUrFcNOaSxqBbcUfAbAdwBCANMjscDlu5nVYC622p2CM+S2QifTPXFOD027qMQDX9wV+AcAW9e/NAC5dx/tfFYwxUdDL+8+std/g5kmnIvLfqRvVv3XwTgAfNsaMgkxW7wVJ5J2sygM3/zpcAHDBWvtD/vfXQS/0VlkDAPgpAGettdPW2gqAbwB4B1prHYDmc95Sz7Yx5jEAHwLwSSt+1C01Bofr+QJ/GcBuZt5jIMLgyet4/ysG24v/BMBRa+3vqENPAniMPz8G4Inr3beNwFr7m9bazdbaEdB8f89a+0kAzwD4WT7tpu0/AFhrJwCMGWP2cNPDAI6gRdaAcR7Ag8aYFO8pN4aWWQdGszl/EsD/wt4oDwJYdKaWmw3GmEcB/AaAD1tr8+rQkwA+YYyJG2O2gwjZH92IPl4RrLXX7T8AHwAxv6cB/Nb1vPdV9vddIDXqDQCv8X8fANmRnwZwkv923+i+bmAsDwH4Fn/eAdqcpwD8BYD4je7fOn2/E8ABXoe/jim+JAAAAJdJREFUBNDVamsA4HMAjgF4E8CfAojfzOsA4M9B9voKSDr9VLM5B5kffp+f60Mgb5ubdQynQLZu9zz/J3X+b/EYjgN4/43u/0b+85GYHh4eHi0KH4np4eHh0aLwL3APDw+PFoV/gXt4eHi0KPwL3MPDw6NF4V/gHh4eHi0K/wL38PDwaFH4F7iHh4dHi8K/wD08PDxaFP8DLX9KzSaxvEsAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "plane bird car truck\n" ] } ], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "# functions to show an image\n", "\n", "\n", "def imshow(img):\n", " img = img / 2 + 0.5 # unnormalize\n", " npimg = img.numpy()\n", " plt.imshow(np.transpose(npimg, (1, 2, 0)))\n", " plt.show()\n", "\n", "\n", "# get some random training images\n", "dataiter = iter(trainloader)\n", "images, labels = dataiter.next()\n", "\n", "# show images\n", "imshow(torchvision.utils.make_grid(images))\n", "# print labels\n", "print(' '.join('%5s' % classes[labels[j]] for j in range(4)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 2. 定义卷积神经网络\n", "复制之前的神经网络，然后对其进行修改以获取3通道图像（而不是原来定义的1通道图像）。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "import torch.nn as nn\n", "import torch.nn.functional as F\n", "\n", "\n", "class Net(nn.Module):\n", " def __init__(self):\n", " super(Net, self).__init__()\n", " self.conv1 = nn.Conv2d(3, 6, 5)\n", " self.pool = nn.MaxPool2d(2, 2)\n", " self.conv2 = nn.Conv2d(6, 16, 5)\n", " self.fc1 = nn.Linear(16 * 5 * 5, 120)\n", " self.fc2 = nn.Linear(120, 84)\n", " self.fc3 = nn.Linear(84, 10)\n", "\n", " def forward(self, x):\n", " x = self.pool(F.relu(self.conv1(x)))\n", " x = self.pool(F.relu(self.conv2(x)))\n", " x = x.view(-1, 16 * 5 * 5)\n", " x = F.relu(self.fc1(x))\n", " x = F.relu(self.fc2(x))\n", " x = self.fc3(x)\n", " return x\n", "\n", "\n", "net = Net()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 3. 定义损失函数和优化器\n", "这里我们使用分类交叉熵损失和带有动量的SGD。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "import torch.optim as optim\n", "\n", "criterion = nn.CrossEntropyLoss()\n", "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 4. 训练网络\n", "我们只需要遍历数据迭代器，然后将输入喂给网络并进行优化。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[1, 2000] loss: 2.192\n", "[1, 4000] loss: 1.815\n", "[1, 6000] loss: 1.630\n", "[1, 8000] loss: 1.541\n", "[1, 10000] loss: 1.475\n", "[1, 12000] loss: 1.446\n", "[2, 2000] loss: 1.370\n", "[2, 4000] loss: 1.350\n", "[2, 6000] loss: 1.302\n", "[2, 8000] loss: 1.292\n", "[2, 10000] loss: 1.268\n", "[2, 12000] loss: 1.253\n", "Finished Training\n" ] } ], "source": [ "for epoch in range(2): # loop over the dataset multiple times\n", "\n", " running_loss = 0.0\n", " for i, data in enumerate(trainloader, 0):\n", " # get the inputs; data is a list of [inputs, labels]\n", " inputs, labels = data\n", "\n", " # zero the parameter gradients\n", " optimizer.zero_grad()\n", "\n", " # forward + backward + optimize\n", " outputs = net(inputs)\n", " loss = criterion(outputs, labels)\n", " loss.backward()\n", " optimizer.step()\n", "\n", " # print statistics\n", " running_loss += loss.item()\n", " if i % 2000 == 1999: # print every 2000 mini-batches\n", " print('[%d, %5d] loss: %.3f' %\n", " (epoch + 1, i + 1, running_loss / 2000))\n", " running_loss = 0.0\n", "\n", "print('Finished Training')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5. 在测试集上测试网络\n", "\n", "我们已经使用训练集在网络上进行了两轮的训练，但我们需要确认一下网络是否真的学到了些什么。\n", "\n", "我们将对测试集上的数据进行预测，并根据实际情况进行检查。如果预测正确，则将样本添加到正确预测列表中。\n", "\n", "第一步，让我们显示一些测试集中的图像。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAB5CAYAAAAgYXpDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO19aZAl2VXedzPz7a9e7V1d1XtPd88uzYxGIwmEEBLYIwESYQssTMCELcdEOFAYHEQYYX5gRfgHhB1gHIHlmEBCAhMIWRJIFjJGjHZgpOlZpZmeXqbX6q6u6tqr3v4yr3+cc/OcV0t39UJXP7hfREdl38yXee/Nm5nnnO8sxloLDw8PD4/eQ7DdHfDw8PDwuDH4F7iHh4dHj8K/wD08PDx6FP4F7uHh4dGj8C9wDw8Pjx6Ff4F7eHh49Chu6gVujHncGHPcGHPKGPORW9UpDw8PD49rw9yoH7gxJgRwAsCPAZgE8CyAn7XWvnrruufh4eHhsRmim/jtYwBOWWtPA4Ax5tMA3g9g0xd4sVi0AwMDN3FJDw8Pj398mJqamrXWjq5tv5kX+C4AF9T/JwG85Wo/GBgYwJNPPnkTl/Tw8PD4x4ePfvSj5zZqvxkbuNmgbZ09xhjzpDHmqDHmaK1Wu4nLeXh4eHho3MwLfBLAHvX/3QAurT3IWvuUtfZRa+2jxWLxJi7n4eHh4aFxMy/wZwEcNsYcMMZkAXwQwBdvTbc8PDw8PK6FG7aBW2s7xpgPA/h/AEIAn7DWvnK959m39AUAgLFJ2pbNULdMIN+XVqsJAOjEbTomm033xQn91iZiwTFBDAAIQtXndon2gfZlso10Xwh3TTlHnHQAAO2O9C1J2HJkIu6PWJKavE/blhIelzHS2mrRGOI4Wjf2gPvWSqStSt1ArRWnbaX7noDGhz/84XS70+msu+atwHWfz675q5sC3UatgWvUhjjj5i9Rx7t5lpNczZtqo3674z/2sY+t27fvh3lu407aNnflMgCg2ZA1c/CuQwCAgf4KACATSn+yGVp4Wd3G6zkyao116gCAcinD55C+RrwdqkW8sDAPAOjr60vbMpkMn5eOM4Gco5O0AADBBqJaYKSxViXzZhTRmszn8+m+VovO0eFnEAAK+QJfS/r2u7/9W13n371nR7pdHjlCvwvlua30lQEAK01Z19XlOe4v3e9ELYaIB1GIcmlbPuRXmHpu0weQm+JEzu/aEtXmruHGTtfnudxg7Ri+fybQ74V4g+Pot7kc9TcbSL9hadtkZf5qc8cAAF975vvrzrUZbobEhLX2ywC+fDPn8PDw8PC4MdzUC/xWoMVSlLV1aWTpM4dS2hSAvlRRxJK1lij4q2oy0th0UkMiX7iIJbyQmyJ1DpOQVIyOSBlOGk7UOVqGJJM4pC9oS++LAz6XfI0NS/F51beIJZ8goo7H7bbqSIeHJOdwEmcYbm7xCsNw0323Cjcq0ev5SOUkJSUmTmSyPAYr+5xGZCDSjpzl5iXwjVAu0r0NrDwezSq1JS0h4vNZOm+pQMdF6jJu7eTUIitk+b6rsTRjdxytq6xaJ26KokjurZPsAyXFu7nJsVaql0m11uZrCpz2aiHnDfhiGZZCnVQPAO1mk8enxsJSJa6yJhIrUnwnHKRzZeSZjkOSwIOMksDrq9S3uMr9kPM1LR3XVpJvg+dXCeVotUlLCviZqNfk3eKeEz0+pxEHgTyH1mkuPJla4+90Yj5GrmmMez/JmhkcpDHnCn18frlniVvXOelHvFrG9cKH0nt4eHj0KPwL3MPDw6NHse0mFMsmBlgxXVgmj0wsKl7SJpUmLLCZQqmhznqgiYQsq0gdKypK0g67jnOqEAAYu4ZIA2CYcLGhqIL1mHS1y3OkblVbohatrlJbaOW8fXkmsxQJVykSAVTI0TiToJXuC1JziYzdjaCdbK72a5PA31eZvK2ct8tc4Y7v0jXdLm3yoTlvtmk+Iq03x/Tb0Gx07WSDtq3hamOJ2IwVKDNWNqRrZQJpywVsHnP7FAHZrJOpJQwV4RbRfW83hQgNwCazDrVZI49kzKaibKYgx7t5UGvMkbkxmwF1vMXclSsAgLGRQTmezSVhVq4V8rXcPCtLDiI+vqlIXUewttvSthaBlX0x9zdWz0FsaMz5PunH8L4x+u3SAgCgXFtN97Ua9I6Iy/I8Jv0U2d2Xlbl31w3YztpqyvPlHB7yebkv6ZSqNeHWsfsbKJtth8ec6OXHl89GsnYLBSZ64cyAYqJJnHlWy9A3YKL0EriHh4dHj2LbJfAoZsk7lK9fwJJELlRfd8cQ8Zcw0EwN/7SjJVRHymRFetm5/24AwPLiLABgdk4klUxE0nYA+TK3OjQ9dSsBSMfOkURjc8MAgHYopEyLJYPVpfm07eI0SxJ5JVlNLQIA9u6kaw73aSnNuRbK2J1wEdv1rkoOWvK9Fe6Dt0SKT/uttAN2tewo8aXNmtDJ06cBAGM7xf0sYTJ6dEgkyDwTP8lN9PFqc5RlKTvpiOQWsvSUUQRahtuCmNZRNqOkupBdVZV2lQno3iZGaVwJu8c2mMxU66nBYy8WZQ2HjtnU4h/PQ5VdHJ977vl0V5s1gcHKm9O2XI7JfDUFqSsra6eBct8z1pH5siZt4oi8zSXwDsTVMQCt9SRUBC5rYaHSxkrMRlaKfI+ffzbd15olaXz8gbulb1fomWsambcyD2ylTkRoXo0lxxp5MCyEYcAkpn6lNIt03qjNmklbJmulRPclt7SUtkV77gMA1Ab607aEtaqY71k+ESI01fhjaQvj65envQTu4eHh0aPwL3APDw+PHsW2m1Ccnm0iSTPr1NuOjlBkwqjFam1WkUNx7NQ5ZWLgc2i/2rf86I8BAJ77278DAFxiUwoAVDsuslJUq3OTMwCAM5MX07bc4DgAYPfYAbpmTtTEFqt/mbJkfew0SO2bm5E0McVBMr9MrlJ0X0Opw2N9pOIVM6JWxm1Sg3Ww2Vr6biMS83ZEYl7d1MJkWUZFzbKPd31VSOvFJVJ1p2fJ9FToE3V4mCMOddSgI+10dOYGnV3Ti60jy+Y6q86RcZMfS79DOLKd2jLKr7rt1OdEzhFWaB6MVX7/7G+cuGjfWNb16jKZ2spFIe0Cnm8dFRlx5PIik5fzy2IaLLCfdEtZOlptulaU1WuG2mKOdO4o85GLgs4qH2fLazaJNzfr6Zl3JsFAjT3u8FiV7cKwiaNh6L5nElkLZoRMa7UV6Vv7zAnqrxEzU8LTVXX+5er5yrY5fuOCItF5PrRjRIPNoWGD50ouieZO6mP9sphK+ww986Z/RMbH120HjhhWsQ8836EixaPg+k2CXgL38PDw6FFsuwTeDOhLu1RTEVosvQyWRWyoMCkUsQSiCabUDUgRKo7krNUW0ravfonyrkwvkkQxvSrfr3MX6bhzlyTFeZgnaTwOK2lbqUJf2kyR9kV5+fLnWErMBzKW2RZFgY3v3pu2NZhcOX2aJPD5RZWTZRedd/+oaAIZdqUzyo1L5C8er/q62+T6ZM408HEDAUBL3cEGEnjMUlbC0oaOFnURblfmltO25SqNta7zX9RoNEGOyOJqXe5tucgSp+qbk+e3qmBcryaSM87lTebbkZcbugAmHPmnXAAj1hgjxRSGhubDxvru8fiYuI+Vq9nqCs3beX3NyEUui7S4p0Lz5lwGX3r55XTfG+6/HwCQaBfHmOY3r11sWROo11jDjeT8HdYAw0jI/Dbn22k2N08RHSvpPOE1bLXMyE4HLe1uyNftX+G5Gh1L9xV27KP+WCEPwa6QdmRn2lTPcG6Ty5RXBcolt8rPqx0bTtsyCfWpoTT4EmuBrRUaX1PnqClwxGtV7ks0TNqBySg3Sc530sc/DZWE3zE09yZQLrO4/mhqL4F7eHh49Cj8C9zDw8OjR7HtJpQrdVIb5ttCYn7jb74OALjviJgifuR+IgcG2V9ckycuaU2g1JGYyRLFfeHMOfIznq+TamOLQ+m+sMxk2ZCo+wWu39lSKURbTJxVBqlvlbL0ceYymUSWFxS5wSpeviCmlvMLRJ5mKqQezkxJtaTy5RUAwM6KHF9wqWsTRX6tQbWmk4GxCqlUR5dqN1SJkdy2S4+pckghSNZ/212UqLZdrLJ678jMgiK6GhyxNqVMKDMLtJ0ogqvN9pHaChG+M7Myf5MXpwAA9x0+mLbdtX839V/5xadkqouk1VYT120dJnAVajNkE17SFvNAwCa7+pKMBWw+sJwEKSzI2LN8r7Jqvk2bTGexNjtwtLFJiVMxH1WrZCqYnpbjS5UyX1Ml8uI5b63ScXnlj35lkYjQ578vZpVSjq556KDMacSmnGaN1l8hUomXmrS2YpVWOXaPWkPNx1qoKXYpXZOuWA3ep57lDJuvcqdO0umf+1a6r/NmNj2ptKyWYzSyK/JsNEDzUOZ4izAnxyclOr+xiljnZHJ9w/IOylxk88sqrcnMmDgr4ALtiypi5mxcofkNi9KWHCHf8AYnwgoU6Z7t0OREyjZor8LJbwYvgXt4eHj0KK4pgRtjPgHgJwDMWGsf4LYhAH8KYD+AswB+xlq7sNk5rtqBfpICanPyLWlniSicr6lk5y1y66lk2e1KER9O4gxDIVkaLZJgryi+aHaFvr7FASIwBkeFWKwmJEmMQEW9MeHRyohU1KiShNJYpeP3KTKkxtL2TEukYcPS0NK8krpYGqnz1z3MSr+nl2kap5ZE6t83whrGVb7Qi3UZaLlIWkGg8jK44hRdgrUjV1yQa1ca1w2+7Ru4J16eIhfLoSHSZgp5kWyaDRpzMSdtO0dJk7JKPKvWaKwlllRaDZX+kwe92pTxddI8FcqtLXVndPvWDbNLIrya92PeJexXBzkJPKek/jKTxf1MPgXsDgkAOb7HeS1wspYUNGQtpEn+uTBIa1nWWl+J9g0OiaZ4ZpK0vNMXLqdtJ049DQBYmCWJc7Uh56i1qcZKBOUWyJL9g3cfSdve9+OPAwB28Xpu5mWcjWqVfyfXrHCBdFNfwWbIhLL+XDpoR2YCklI1UnJkeYGu1Zkkt9uK0iZWLtH1W3mJdrSg94K5PJO2lSaYgKywZgl5lgrsvppdlH43mDjuzE6lbVmew84yzVVuXhwZ2nXWlgqiwSyeIeeHbEEk8L5xIl1dKiWrXAabjrxWa7iVXL8IvhUJ/JMAHl/T9hEAT1trDwN4mv/v4eHh4XEbcU0J3Fr7TWPM/jXN7wfwTt7+FICvA/jVG+nA3W94DAAw+czxtK3cT1/3x972lrStGJKduMUSsJYuDWdri63ky+jbQfWWX3z5pJx3gKS/XfvItcoqW1qGpeykOZe2tVrJumuF/MV85aWXAAAVlZC9WKIvf0nZwS5dngbQnaclZKliiN2/FhfEfrcwT9tnpsRVamKMXKSirIomWIOoIppAzNJzW9eTY9ti+hdil3TBIVritBv4FDoBXXkspgElLl8GlCvnALtitdvqXCyVFctiU3QSuOHgLKNctnIF526lyoQxsdFlM1zXN7lmpvsQ3r25CH7h7Fnut8z3yjKtu7gtmsDFi6R9LPAaqK6KPXjHMEnN5ZIE4YRcjKSlMvhFnKsn4Fw8VSWdN9xgVGGJ85eIPzkzKTxBtUW/zfezK1tJJsatxFJWZLWpcxT8cunSdNr2rW/9DQDgXuYaRgdE4qyvkmTvyp0BQPteykeyurS54p3Lytitk8YTpRKzBhMot9dVDrxbffSNAIBK9KZ0X22F7kFb5U0yOZ4bVW4wU6DrVtldUru/tjnfSEY9G3WeG+3EV2e7fG2VrlkqyFgafHyuLM/5UB+9e2L1rljltQt2ayy0VUZD7pP2+G3fQG6fG7WBj1lrpwCA/+64xvEeHh4eHrcYf+8kpjHmSWPMUWPMUZ2n2MPDw8Pj5nCjboTTxphxa+2UMWYcwMxmB1prnwLwFABMTEys0xGK/aT67zsohEqdLQp7DxxK20ZYDV88cxYA0NbRWx0yRTz2jp9K2/YefBQAcODBs2nbcy+Q2WOwTCaJSzOSCyVit6KcLibAvV2tCjm1OE9q5FA5ow+hfrCZZGRUcqG4IgWzC2ISMRyt2McuiFGoiAxWoV+/MJm2jQ6Smn14t3JlWoNP/OH/kvNzPzJKnSv3kQp46IAQt29+A7k5ubKNVpl5HClotb3E5ahRZhJHsGVzdH5NTmazZBIZHlTujK62qaoxmObYyNA5Gh05/yKTuosqdefKEqn0be06ycTjMLuCHT4kBFPGRevpwuVBl0GlC9/622d4uKqgiCOe67IWzl4moi2tXanEoUGuVF9SpG6Oj8so18KIXdwCrolZUwRkxOewKu/P5XkivtuKjS72Ofc3zhe0qtwf+X40GtLvSh+d961vejBtq3IK5Aa7zJ4/L6aR119/ncauXN7OzdHc12ty3ignZDwAlEriENDheWjH+p5xYRVF3hk2KRXGiKhcrspYrizR2I1yj21xzc+sJgMX6Tcul1IuK8/BMq/xfEa9+lyaXxWJ2eToYHDN26W6rEmXhqaoolX7dpPJNtRmvbSeK98rXbvBvTnUokxuwI/wRiXwLwJ4grefAPCFGzyPh4eHh8cNYituhH8CIixHjDGTAH4DwG8C+Iwx5kMAzgP46RvtQJgjIuDS9LG07aE3UfL5Ur980cMVIoxilgIiVQ7q9AUiGt4+eEBOXKRgj76SqiIe0bUK7LaXz6pS1vz13TUxnja9ypJHVpExy0ykHNhDGsORe+5L983Pc/GGigQEXGL3JqNIk4FBklqXWLrU+UMKRfptfUX6ffI8B1coImpMUj/Q8TUVbFSn7YwKqllhAbao2uJ77wEANCyTPUoCz7EkpKVWV5hBZ+nrHyJtIyWKlPuhc4sKlbTtIqu0rJGwNHKWA60uzohCNz9HGk+9LpJb3GRJU+VMcTk5du+h4Ki9e3an+0rpWtEk7eYS+IsnqR/Fgmg8ljW+ZkfuSz9nlXRkXUtJuVdW6R6Eaq768qRxdWIhrQ2TdiH7mplIAsNyVZIcW20hR+fnHXmpy3/R3xbnWFmpyly12L10z6i4Ig4P0uJxgUIAML9AeVSGB6gfj77x/nTfJLuKLtVlDb82SfclUOv6gKQtAQBEKhNooY+euVVVIi1ilSVWWfgiDnYJeE0myv3RcIGXSF3TbbVbKgMja9ERS9Za43HkZay0PFeqraNWZabAJGO8Pqupy52S6ShNgBl+ndEwH7sMlnwtteRcIFu3V+/1Zw/dihfKz26y693XfTUPDw8Pj1sGH4np4eHh0aPY9lwomTwRKo2GVoe5/qCKUCyWHClEqr2ul1mOSAX65FMfT9t+8l98mM6hoseyXAvQFYc4cHBXum9mngipxqqowTt3kN+4TpDf5DqFBw8RwXrXISFfl16gWoTVFVETHQnTURFodTZxDHD9vNhKVFj/IKl/HZWBPwxofJOXxLQw9gZ04Wf+2T+XPjK5V1L5VxxpUlCmJ5eaYXmZ85N0RLXPMKkWKf9Xy6poXflH24TO56p2a+I04uMzGR3hud4M4/xfG5w/pKRyTAxyPpq4JX3LhzSuxTkxAUxePAsAOMTEdxgoU5F1FddVyt2ruNwus5nOaqKQffsLoczH7j13Uf9d2tzLstZm2fQzNiYetrkRMutUF8WfOuFI0/5Bsj/kchLL0OAh1zpiQsnzcxC3ZY2FTAa6IieZrCoskaftxx4Rk8iRfRN0/pas9TOv07heP/4qAOBtbxaCc88eOv78y5Kzpx27nESb18TMqn5kuSZsYsVsWWDSuqPS9q5wJGrMRGW+X0w/YyU2aSmyTyq+q7S9cDU/6a8uRLERLD+b2oQSs6+5S9sbqGtmneFGJVpq8jtF516K2IQYcwX6rrq1/NzouqTalLpVeAncw8PDo0ex7RK44QitmpJ8GyxBZnQehDl28eF8JxkspvvGB+iLePKYRF1emjxFGzUpZXZu8iwA4OGdFP25a58wgRMzJAFVT4mUMZQj6a9vQMokvf76GbrmBEnvi8siHbX5Sz59RUlYjtxQroI1lsAN50bQ1EXJZTdMJLIya2g+WrOXsRmStkgIqQSi9pezdN5CXua0zpnkam3qx9nTZ+WaTGLuPbAvbTtzgebyS3/5dNrW5gyQec53UlTnd9Fr/RWJ6hvoJynq4YdFhRgdIanzrt00p4Fy33NSlCOaACGn6jtEOpsYp3s1sYtIaJ3hrsauZl0ayVVElwwT66M7JtK2PBPIs7Pi3lnlqGAXTtdQEZb9o7S2dilX2L5+GmdlRKTyOSa+Y5bI2qpCmXNZrCnir9V2BKVoJFmX8TJH9zhjRUPawXM/Oij3IM+E3OigsI4VdrWbO38eAHDu9bPpvp1DtP6Xpp9J2zJMXrfCzV8hkcr9EXKWxbzKj7I4Q4Ts/KrkILkyRfM72Efr/4H7RBPIsPbdVARumzUATcC79e+KnASKWHdSsC4FGKfEqWYZu3Pr6EynSM8hz1zEx+u1636TcZqRftD59IFyiYyv4tq6GbwE7uHh4dGj8C9wDw8Pjx7FtptQ0lSwSh0ZHyH1SavjX32ZfLIHOan84SFRafI5JnEi8YW+MnOWTt+UiLK9d5GfeMjnLVaEMBoZI4Jpbl7U1SUmL3Xh7R07SP2N2LzTUGSjS1JUV+p+h3/cUSdpNDlVZYe+n8NKpTZcKy9rZCw5Jnli2x3ppvHn/+ev0u2EE9QHyoe2zIRwnzJn7D9MYx4dJpPB8LhEaQ5xn/IqGdPiMTIvfe+Y1A2tW1c8gv4fKfW2wr89tFfMMG977BG6Vkl8rEushjsNtqXmtMO+zbUlMZm12Y+6oKq1DwyQ+WCak4fNqqIQBY4IHNsp81wsqhiANRhkk1mozANNLlxhlMwzP0d9Wl7mtMDK5BdyBN+5i5IwqrJM5o/+fokTcP7fTSbxjSL0ci5asCT3vWBd5KbOjUvPRKnA5kVV+X33MM1LURGKVa5231GmGVfs4gCbfI69djrdd+QIJa6CIiwvXSLf8PygmLEAvd1N2rniIokyZ6xwTMWVK2IaXFyg8554+bsAgNde+rt036FDFHOx/9C9advgCJuBlPnBpU52xT20YSJMfchV39LCJqpqPBOQUjhGkaR8vObB08jlDdjxlCTtShbHZ1X3W79LtgovgXt4eHj0KLZdAndRUv1lIZgG+mjbqJwby5YkidkF+hKO9EnXS0zAxIFIHmcvnQUAjA1K8vd9/AV37lnffU6iPy9OkaTeVxapPMNuTq+cOq967CIJ6W9TfTVXOQJuQCXg77BYOTWtEs73UZ8idlUqFkXCcvlD0BYiNK5S38Z2bJ4L5dkXvp9uFzJEKDabQrBmmYR7y1vfnLadu0iS9BxzSA/cL65mWSYga02R4jOsuTzyiBCQDY70y7K0ePigRMPezylHJ0ZE4qwU6d4mym30wmWKApxZ4GIWs1fSfVUmtxcXRQJvcUrXjHKJdLlYXKRuWxGKxQGatwcg4+vv33wunSRdU5GeoXEl6UTqjzk1acQRvokVeSibo/OPjEhkb5nXeF65ZvZzvyO+Z9q90rKrXke5d/azi2WgohcTTpsauejFpkjW/ZyAxXZEK4xZq2mpSMI6348ir81zl2X9vfo6aXfNpkR4ths0vzbUVPnmcFJrPi9jv+duigQ+dK+489ZWSBp/5XlyyX3hqBCn3/omaYDHXpW1fuTehwAAh+8WqXxgkNabI3fDrj66+d0gF7EmR10JuM76MoYuOjNWpGeSujNujq50zcaVgZQ1rFNObxVeAvfw8PDoUfgXuIeHh0ePYttNKC46bucO8cl2NfISRQaO7ybV/CibRhaNpGy1IanZ/SNCFPZX2AczL6ryfjahlDmF7R984o/SfTW+1nJdyK8a++HqzJM7OVKyMU/qXDWnr0lmnteOiz/69DSZA5ZVdObAAJ2wUiJ1OFSkU4aj48LaxbRttET7+/OioKmknACAKxeU//oQmYF27xbS7r43HKbz5+Qcr7xIRNEYq7VlVa1nhusDlipighqu0HHve/wdaVvADtX9/XTcyLD4r89z6t0z52Q+lhbJrLO8JNGnK0wWL3La3vllibDsMCGbUWl+s1wBJ1CRa/0VGtcAR24OKnNTjk1U2YKYqlbrQhKvxTD7cGvf+jJXV0lUOtRMQPOxg/3FjYpCzbLPsjPtAECeoxFDlXfWmUzSKkTKhOJ84GtVWTsuIjCnFqVlc0ptieb74lmZ73l2Ph4oyPFjnHI3n9c1ZNkkEpH5KCoK2X2F61PuGZdnro+rVS03NyfeEpUm1iW9soFuo76Fyjd8YJjSsr79nbR2Dx0Sk9y3v/F1AMCZM/JsVF/g53ZZTGwPvoGq+ezZQ+fS6ZrjDq3xWPUtYVNtVxWqtP6r+yu7XL1YTWg764f2OXeEZnqtLhKT33HKDKNNMluFl8A9PDw8ehTbLoE70q4yKBJ4J6Zu5SJxyzrChQiOPkeS1XJGItwSQ9Lc2C75kr96jNyPfuCH/1Xa9necqL9aJSmw3ZKCDjOXnWucfNNWuYZdpKLeBgOS0HcV6BxLV0Ta6YQk+Y7tECI0ZterupL4GnWSOKtMlnUSkbDaDYpE25ERSW+iTJJSsyNtayXwiydeSbeXmej6yX/yb9O2xx+n5JF//VVxN9zB5N4OrmJfUK5peY5OG+sXSayPt/PKfa/DUouTNHXOl8vHSVI6PyOudC0uzBHlJW1qXx+RvjtYImy31hNHGZWU3+WM0Lkj+vpoLJVKH+9TdRY5H830tNzvRmPz6lBFlj7bimgtsEvkQEW0miRNbUwEZEHV+UxJKiX9JZbbtNzkimm4v4pc6/D97sTS1+U5GoN+cDMsga8ukbY3dUmij8eGaCwDJYkmrrH0nChNoMNndMTpLi5QAAB3c53Mh+6TIhknTtPz8sL3xBFgLXQK5YALLgSRaNUZJvFjFb3o0rEGTOoePiKEecJut1NTn0vbFmZprCeborVNX6T6uncdJpL03vvlHDvGiFSO1Lul0+ZiEyrFbMw1Xt193LAASFdOlvX705TFPA/6FGnxFCXad0V7bhFeAvfw8PDoUWy7BO5yfwyOiITQ4a91I5BCAPkySxKcwe/8BXH+f/ubyT2ssSpfxGIfue1NXZTcFYSO61cAACAASURBVKdOUDXujqtWrbyLqmx37RsWt6+lJZJ8+ssicd59hHIzPPvSawCA54+dkX78yHsBdGdRPH2KJPRFldHQuSA26iR57xsTya3AQRtDQyL52ogkg05rczejhipt9eAbqY/veve70rbhAbJN/+BblP2aJbc+1gQqZZGKQy5S4KqmA2Jr1Un2lxbI7lphiSZRGVgO3v0AAGDHbsnYOL9AmkvfgLgWusx2xq6vGO7sqK7UFwCssk3YqhJYrlDAhSmy3TstBwDaXOxC50cpljYP5KmyttSnCjq4oJ4ZledmmYOLEs5aeMgFvAAY4PwhYUZLl7SttZQW1+eqMffRaEq/Oy2aK6MKQNgmHV9SGsnAAGkwhSzZqCMj62SAtbf+PlmTLT5HTWVbbHEG0IADSwaV5lXkLJ6TimdxheHvv/tw2nZFuX/SubQ9n+3dqm9Z3p3oB5ElU2cjbiltbPee/QCA/fv3p23PTtP97qhyb1dmFrk/JJ0fO/Zyus8FKt11l/R7bIzcGPv6hO8BB9Q1uNp9rJ69DGtcOmjHuRHqOB5rtKsijSo9fVoAQhDeQEGHa0rgxpg9xpivGWOOGWNeMcb8ErcPGWO+Yow5yX8Hr3UuDw8PD49bh62YUDoAfsVaey+AtwL4RWPMfQA+AuBpa+1hAE/z/z08PDw8bhO2UlJtCsAUb68YY44B2AXg/aBamQDwKQBfB/Cr19uBhGsM9g9JEv9qndSWWiwqhyOsXK3DE68o17QaqSrlkuTy4Fz7OHdC1L6LTO687W2UTlan6ezj9LBDE+K2dH6ezCT1pkrmXiJ1tTJKJM/DfVJ78Qqr12fPvShjqZG5YXFJrrWDq9b3W+rPvrK43u2ocBEEIyYRl0K0pFRSccIjHLznoXT7g7/wb2h8sajZx08RkZgYlUOGyc42q3PziyrpS+LywAhd6gp/JxAiamWZehJOk6p7SdWzdIU5koaQQyUmTE+fFNPWGU5h6tzwhkZkPpy6v6Sq0s/NEpFnlUkkYPc0E7i8ICqylwnTvE6lu7qWBhbk2GVxblbG8voCXdNFMQLAwCApnePjlI+jpaL22i0ywyRW+rjMZq66Mu/EHCEZsnlK1150ZpK8qu5eYPfBhlq7CRN/pTK7pap1kuUoRE34OkK4oUg7V+ndkYhtVbRjco4iZGuqhqYjAXeOy/pfi1CZENJtdU0Ynq8u9zr3G7Nun4vi7OsT805KLnYV63AmObrWyoLcxxc4JfMrLz2btg0N033cuVOI253j+/maZFYZVqbVUS5IaxRR7u5zR5n1Okxypm6E2hWRzVdWmdNsstbkcm1cF4lpjNkP4GEA3wEwxi9395LfsclvnjTGHDXGHK3VNmf+PTw8PDyuD1smMY0xZQCfA/DL1tplnbnrarDWPgXgKQCYmJhYx8KtcCKOgsrklmZmS1T5LyY/RoZIOjsRSLa0mXmSbOZC+YL1l+krec8DQkycPkuSnkuar4nFw4eJ1Dh84K607dwUSRyvvPK9tG1uloNCOOn/oHIdm3yFJPapWclBYpiIDVVA0fgecsfax1O4t08krDyXZmo2dKABSUzazWktPvBz/zLdHtxJUtFL3xcp15FBLfWVj5lUc6XDNIniSlXFWkLgtqDrs8+5RzhL5OycuAw6NzgVu4GBygD3RyTZ+TnWNlgKnJ0VwrLJ2kdHuWHGXNYuVLlQinma55xzMdQVw13yG4h0VFBZFtdikYnZSxfFHa/E5PI9qsCAy9hY5PwujbpoTQsL5G7abss4a5yrpKjcMPsrtO5LOfpbUORkxM9YrEjMTqfF51XZLV05r7T4gCoSwFpsWz15UcgkXKJcWznb4twV0jRm58Tl0mUNXFD5aJwmlesTbWktjNUSOP3VxJ5hqVXnCEklaf7rCEMAqK9SPy5flgIQly7R9lJRjsvwOnKkfEnlXylGdJwmtC9yEYmTZ+WdUq9T0ZJOTOcaGZXiHg8+SAGBhw+JxD46Smuh0i/OGLkCaQoWfH317HXSJIeKSP77IDEBwFCO088B+GNr7ee5edoYM877xwHMbPZ7Dw8PD49bj614oRgAHwdwzFr722rXFwE8wdtPAPjCre+eh4eHh8dm2IoJ5QcB/DyA7xljHDv3HwH8JoDPGGM+BOA8gJ++kQ6cPkVqy97Dkg4yH3BazJYQTRGrQUJkCOlZ5iIF99wjfrh//VdfBgDUlsRfvDhMZNOpSVIW9uwW0vPA3VRoIKfU8oN7af/ivBSFeJXrbiZMkEwuCNmzzORrIxZz0PIimWl2KILk3By1De0hc8JcTvkkJ0x6KnOJjbgWYCLq+Fov5hdePJpuv/w9uk0GYppx+SYiXXQgTY2a4WNE9Y44/axO/+nykWRVfwP2Ew8t7atkxZs0YDNTO1TqPkemKrddZDlXSbvG/slVMUG1mOQzbRWdyTacliK5Y462rK7Q8UV1H0f7qR+RMl04S8VGVObQKK2TQVVowxUkiNR8rKwSkbi6Sv3N5cT84UhAnY50YozI61xe1H1HXlrOx1FtSI8aTBAvLkh+nrl58rWuK3PNvZy2N8O+9d0FDLhep1pPTa7lOZlGH4sPd4vNU7WqnH9pkUyJWRVV6sb+9Fe/mra94y0PowuqWEHi/Ls7KgKSTSzKHR0mNe/QvlBFpr70/HMAgNUF8TcfZv/2C1PSVmEf9iw/N4mKYK6U2R9d+ednIy6EkVNxEAGbZRfIbHT2jEQ6Ly7QvD1/VOW+4biJPXskWnWCC6SMT9CzPzEm75sSp602BVWvM9g8NmEzbMUL5dvYPM3tu6/7ih4eHh4etwTbHon54imShvc+8FjaloC+fkaTdvwFX2ZCZXFRSJbhIXKhe+/jP5K2PfRGyoPwmc//WdpmOK9BP1cH3zUhLlBlJtfCjkgeQztpesYPiBS1xMn4n3+RpNypVeW+lCHCtH9ciJ2RQ9TWVQiA3faOc5GKU5dFQs0y21NXkYdVnoZOIlLDe9b4/HzrG19Jt2ucmS2bUaW4io5ElVseWs5/4ap4Z7QETv3I5xTBym54WZXFLirRWPNZGmdO5XNwqTaMyqLoyOi2KhTRYIIylVp1BBsfr0u1pSG0SuIdKNF2f4nGVC6IlJvL0PkyRu6jUe6Aa9FmUk27HUbs4hh3EXOunBzPnxJz8ixl16syzjpnYKwrH1Cn6QQZ51Yma/74sVcBAOfOnk3bXBSxVe6JE+NE2A9xRsi68vZy24sLQkDOMUlbVxquy9njPMUWl0ULCnjui5GsHZdv5fJl0XDXSuBtVUTCkeimI+dwUZ/aec6C2hzpuboqk+WKh9x9RLT1Rx56FADw3MtS5OGZZynL5iIXA4k7cg92jBMZ+fa3vz1ti/g+nz0nLsfPPEO5lB64j6K8K/3iDDHNY56eFsLerd2dY+JueODAfro+OwJUV8QN0zkEZCKR+hsb5AC6FnwuFA8PD48ehX+Be3h4ePQott2EcmKJVPTZWKXizJBKHbSUypG4GnL0d2JcbAg/9ANEQOYzolYe2EeRlT/+gQ+mbZ/9s7+ga12m804tifLWaJwCAGQhKux8nbZPnRM1Eazm2FEy0QyOiTkhrYunoh0TNjckRlR6l7xpiSMl8xmVtItTulaNSsbE5KFNtIrVrW6NjUp02lSdCJ04FrW5wnU6I9W35VkiZ1eWq9wvUTUTp/5uFB2mzCSZAt0Hm6Hru0RkABCwDaWoknu5yulxe715DJw0yWTFFpFnMrKgzBlDfaR27lE++LvHyf/W8ZTNhqjegaX1FKnIuYEKrbua5KZKceIEpUi9//770rYCm0T0dARMDSUcfTetolBdcrRmXZkp2CQYKzPJwUP7AQCjO6j/utBAhs02AyqxlCNAdZlH58P92nFKo7qqCkC4fTqGIGETUXVF5qjG/axxtGhLmbhc8Yjz00IUuhql8VXqONquCEvrNlK4KEoVJIrEEZ98qwqqXuwPvfPdvEt+4Io1HHlITLAPvInqvrqyoYGi8FzBkYMHJd4j4jndf1jSzk7sJWK4wBG9/cqE4sblCpYAYibZMSppsV1yrJBNT4Fia2N2SGgru1tiNp/LzeAlcA8PD48exbZL4McX6RvyhW9LtOND+0ga2ZkVA3+RpYDxnfSFGx8RqeSug0xGWpEapjgvySc+/Rdp23MvEinkIj27AhutI5HkHHGOrhFrYo5d8zpMiHYCRfK52VSlkRotPq/60kZMaIYsbVmVK6TDlE5Gfa1daa1We/NILdsWib2/RBLFiiJC2zFJZffc+4D8ZoKkkRmOvptR0XernBdFpz9wkqON5byliKSMe95IaTovqVJpV5ZJwq+3RCKscyEFHfWZY9fGEmsaAyr3xyhXGB+fEMnm0C5y89uREzF0lV0P59nNLszK/BVLRFqXVcTrMOe/uHRGiCuHNkvvjVXRYAJHHioR0hVriNlV8OTJE+m+lSVHJMsj5opeREp8TjgkL+BIVijXyGHWmjQ5WuMUxPW6zOmFC5Ndx6ngPlh2uay15J456bk6KxpuhvvpSth1VKRild0IO8p1USIZN5ca60r7CNklMrIqQpaf146KkO3wPLjz67JsTqDvKA3GlTdrqRwkE3s5n1HCKVsTVTSBn/Mz58U1s95yeXRUgZD+A13XX1iSa0YsUZcq+2WwLp/Qkoz50vQ8n4M6nlPpsV2AqSnL+mgsbF7mbzN4CdzDw8OjR+Ff4B4eHh49im03oayyWvHXz4v6eeJ1is58z5uERLprglT1M6cpEvIdbxZTQJ5V75WWqGef+UtKF/n8q5KQqOaiwNiEEajUnU7NCVT0mDN7xEo9a7Jpo80qnlG+xU2OaNTkTRStr99Y5MQ7WbgK2ekuxEwC6iRSHSb8sn1SxWZt6pm5S5K4Km6TKlZX6m3tAiXyGlIVwEc5zWqGq8AUVNapeugqjGg703q1uVYns8s7uCrS/fdKsqfz58k8MbcokaxNR44p8itiYrrArNOIIiwHSiW+styDy7M0luOzktTIMBFV2UFmoUJFCM4ik546TW1ZkVJrUeB71lJmCkcud9V5dP7fbH6oVCQ6OM8+9eWSkHAhj6uoojmdyeLka5QIbWleVPsljpiMlc93JssRoWo95VgfN646vYrmnGGirdYU9TzkMQz2y3pqsbmtxk7qHZUsK0nNJTofKs+H2VwG/OY3vyZj6VBVnFIk8xHzumsrM4kj0l0CL/0stdlUpZ9HRxA2mtIWpxWeODWzqn85NEDm2XJZV4RyFeL18EzXX11t3o05UCaRiJNkBWb9cW4IXeENht8fRTk+aLD5TxHU14KXwD08PDx6FNsugQ+PUH6I+QX5/E1x1Njfct1JAIjb+3iLvnSjOyWK0oT0hf3uUYnG+ouvUiRVM5EvPvhLHATrv1sxS4ZWfYade5iWAlwUZYa//EZ/LjmPgyapXC1Fnbsl5OuHliUKqzQBluK1WD6+k6TFvoqSGmvdEvjO8aF0e/L8JI9JJ8+n7TMnjqdNS+ze565eVW6KVZZ2kriL6aXjVSrhVpMktue/TdXu31mScT7A46z3izTsSDsdZdtggm2JoyM1mXruNYp2m61LZGAjQ9cv7JAxD+4kiSpXoTGFKhKzyG54uaKQ4ibcfOk7V9W4I/fARfEmHaWN8dgdiVlQkYoBa4V1lVOkOU/a4HldjIHnwaVUdflmACG7M3kl9fMlWi2Zv5UFkrgbjVX+K8Szu1N5tebbdU5Jq+qXOsLR/dXkoXP36yjtw7LUms1sTqznVSRwO+T7olJE59hJIFGup86NMuBratI44XwxWup3EamJVVG2PGrr6k6qqvdOeA9UXdco5BTOTYkcTQlNHp6uudlmjVhr1W7NmK4q893vmZaKKrV8joZ6feRC0pYmJvZhq/ASuIeHh0ePYtslcCetZlSWvE6DpKcz0yJ1NasUXPGOR6jCeWFAVY/n4gff+I5k5Kuz7batssHl2I3LSRcbVQgKlTSQfkyVbSzHkptxolCgjs+RlFFQ5bycy1FbBa6ssFTmgiCaStLrH2QXynFJDF9m/8S6CrxY++nde0QynS2zS111clYdwVnplHvYPF83y2NuKXu32F3Xu4l1JeBnnHyZ8k9cWBHJZjSg+ejSYFgqWVX29suWpL5TbBOdVDk0akXWYPZKQv2xAySh5AfElTS9DywVlcuiCRTZHh6oNWavYrtd5jw7tRVxI5y5RGuy0ZC+uXJoLg+GvsdOkwtU8FCGA80cLwJIBsiIbebaZbDNdmCdT6XZpLWzotzV3G0rVdg9VUl+tk3z3FxV1e45N8iSkjid5O3sy0bZuxO7PpjL5YYxyeZFRhJ1H1erxIMUQ30P6G+sFrMLOGqxW2yno1zruHCFVdK2ZH2U57DDNvDYaXvqXrsgJi0cW0v9bDZ0bpi463itmduUj4lVmwvi00VRuq8ZtnS/OffMoC70QtsT8BK4h4eHxz94+Be4h4eHR4/imiYUY0wewDdBNQQiAJ+11v6GMeYAgE8DGALwPICft1aFQm4RKSmkibyQVMGWIlmmV0nNef44EUHvrYlKs2LJtHBxQUwMeVahOzU5R4NVRlfDMFJRcm5fl5uYcW5IcpwNulOwZnLiErbKrlctlZLWmVO0GcGZTKocEVoeEHPJIOdSaKkUmK+xi1lGuU+9aY2WVRkUQm90jPKTTCkTSqrOqd802Uzi6iVqV734KhF2XXv4xG1Wwauzki8jyHGKXuXCdomv8aKqbH8q4vkok1pe2iNFIUYnKKfNMBdZAIAcu+a1VE8sq/m5iKuwR5pIdm2KZLyKr9bls+TSqquEO5Xa6IhaTmfrqpNr9TnL5hqdB8bt1wRhh00Gq6tcs7Spc5awC5vRLn20LrKq+MDYrgk+B0VMLi+I22aHCzRYXYGeb1qtpc0qzjzhfN6w7viMGrsrtFCrKbPeGly4IE4FJ6eoHyVV4zJi20/cVW6A5tRFWyaKWM9yrhzd5kwusU4NxPPsSEajcow4clTbqlw+FX1fnLtrErsoTUVOssmxK+eRK1hh10eOul+2VZ6leIjWxa4HxVW6393S60iJshUJvAngXdbaNwJ4CMDjxpi3AvgtAL9jrT0MYAHAh7Z+WQ8PDw+Pm8VWKvJYAM7vKcP/LIB3AXCl0D8F4D8B+Nh198CRAzpRPgebJCpvgstHcmaGvvif+MyX033veicldT9zSaS/qnPOV9+ojMvkxlJAUbkBZblQQ31FpGdHNFhFMmaYUHQSniaunKSXKMKjzi5jus0dN8BS87BKAn9ljgI5FmclA+LiOQpeOnTwADZDIS8SWY4DRjIqH0jMZJb+uHdSyYTHp3deRQroorRY2lnl8b2mpLp+Lrf2WkMS37/C2slcRSTT4T00rvEDJG0PKJfIHLslBiqfRZvXShip0mQs8UZpUIscn0rP2sXrKiRmmLArnXLlTN399HlZGwusk8jkHE12iey0ZT05iVpXRHdwZHcmq0vecRk8TQLzWsznlDtegX4zP0fX1FkGM6xRhrr6OWubHS0triHhugJXXIELpdWsctGQWlXyqaxFYFU5PieNxiK1Omm/KxgoZDdC61z1lCbFkq+Ka0rn3ipXQXcjrPgMpnBStnb17fD124rET/gdZF3JO/U8pHmNVEcM1o/FMlnd4YDBisrns/tBcsaIjNzvxROcD2q3aJvXwlar0odcD3MGwFcAvA5g0UqY3iSAXZv89kljzFFjzNGNvD48PDw8PG4MW3qBW2tja+1DAHYDeAzAvRsdtslvn7LWPmqtfbSocvt6eHh4eNwcrssP3Fq7aIz5OoC3AhgwxkQshe8GcOmqP94Ew1xJu6ES8Fc5Uiwbij+1SzPpfHm/8d2X031nuD7fYlWYjPlVUoMVF4gSq+MdVqNyqrq6U73zBZVnIXA+uqKqO5/VDpsMjPYPZZUqVhXUW+ynWlD5L1xS+aERMp20FIHb5AIG9ZxcM+HoPF2xfC3aKmKyyvks+gbkmo0qqc26YEDM6l6awVSlMjXrtfwUVqXLtUwAVdlH91uqCMe5GrXNqXwP0RhV6B7fPZq2HRil7eF+mpdARXNWWS5oKCIqYlVe16zMc5RlxNXB8wURFnI89zrK8WpINsjD4ZRNq0w5ltnf1ESjzuEi+WJtAuB1pNedW2OOVO2yYiVuPQkJHDNZ3MrIvXUV6p3pJNGEJedOaSjt143Lal9od7wzP6h+RDwW2xLieWGOzGLt1uZrsqP8wGM+rhVoAtflxdFFQLiJn6VA3QOXMjbRpg42cyUq/bIjkJ01Qx/vTGDaapM4/2xlMnNmo9TUov272cwDTbA6M4x6H7Q5rfPQ3VQ8Ytf+Pem+BtfTfP01iV0ptNlSLUHm18Q1JXBjzKgxZoC3CwB+FMAxAF8D8AE+7AkAX9j6ZT08PDw8bhZbkcDHAXzKUEKBAMBnrLVfMsa8CuDTxpj/DOAFAB+/kQ40WKrMqU9JkyWgTChSaIc/hC5BfVAQKe0sk5eBIlk6LB11FAHZ4IxrVY6E1ESNk4pKWZHSCkxsBkpqcARhoUjX1zkprnAmuUS5C0VMYAxWhGTcOURax86dRNYtVkVSWebMfatLEgU4wIn9Z6/oyMoRaLRVlfUwS2MfHJVrtss0l522yvyWuL9McCoJ3A1ZR+Sl0plm6xzRxtn62ioHSbOf+n3XgJAyg0MUPVmuyNIrF+m+5Zggbqh8Iy12O7RKeg6d+6fuB29nWJPSboSuWIEmxOxVWNoGu95F2n3UuaZpV0QeuyvsoNfTWsmaO0Bd1ZGSPPfOjS9WkY1tnodQaV5tzqcRK3fXUpM0Fyd561w1zTpL7xuUPks2iKh1/Yj0fHO/56cl/06bI0L1LVgHPXTOmRJk5ZoZlw007qpAwT/luVKnsy6Dn9IA86xhDFaE+HYl1FwBEj2nIbt85pSG6/KcdEWf8n1xkakryyqPCS/PJJI5WuJUg9GI9GPfESIqBzm6+uJrp9J9s6co42qk+pa/Sl6ZzbAVL5SXATy8QftpkD3cw8PDw2Mb4CMxPTw8PHoU257Myql4OZX0p+iIjLaojs7NM2EvZJ1gJ2F1q9NSpFPsUkpqIoq2kzRlpXy/FubJdDGvrlnhQgD9Ksqxwr7jeZB5xVWXBoCIVbxQ1WpscvIjVxBAH9epca3Bmkr6szjHYxf2Nc8Rf42rRA+GSv0aGCbzTrmk/MCbbFJSJpRO7HzDne+vSszF3/agKz0mmwVUMqaIVeIimyz6+lSEICfNL+eEjC6xb3g2J+pnizdX2W+9rghZR7TmlbqaDZ3PtKjBwRrzhL7vLSapsllFOmU2n0sXXRsoM0XGme60+YP75maoq6h4Gpmnkj3F64lkF4nsCju0WnLf62w6iesqYpJJzJIyMxX6SUXv8DjbDTlHsIGNI/WH14R2WjSeNkoqRqLKtU2Xl8Ws5yxQes2sRdhRc8x1JxMVgWtB/Q2hUujytkStKgLS2K6/AJBwsrpaJInvJJrapYNW883R0o229M2tddPlS552ks+kQj35+pqgrnBq49EjEqsR8Lvq+LPfoWvOiAk05PunC3NsZNK6FrwE7uHh4dGjMPYG3vo3iomJCfvkk0/etut5eHh4/EPARz/60eestY+ubfcSuIeHh0ePwr/APTw8PHoU/gXu4eHh0aPwL3APDw+PHsVtJTGNMVcAVAHMXuvYOxwj6O0x9Hr/gd4fQ6/3H+j9MfRS//dZa0fXNt7WFzgAGGOObsSm9hJ6fQy93n+g98fQ6/0Hen8Mvd5/wJtQPDw8PHoW/gXu4eHh0aPYjhf4U9twzVuNXh9Dr/cf6P0x9Hr/gd4fQ6/3//bbwD08PDw8bg28CcXDw8OjR3FbX+DGmMeNMceNMaeMMR+5nde+ERhj9hhjvmaMOWaMecUY80vcPmSM+Yox5iT/Hdzuvl4NXJT6BWPMl/j/B4wx3+H+/6kxJnutc2wnjDEDxpjPGmNe43vxth68B/+e19D3jTF/YozJ38n3wRjzCWPMjDHm+6ptwzk3hP/Oz/XLxphHtq/ngk3G8F94Hb1sjPkzV22M9/0aj+G4Meafbk+vrw+37QXOFX1+D8B7ANwH4GeNMffdruvfIDoAfsVaey+oDugvcp8/AuBpa+1hAE/z/+9k/BKoDJ7DbwH4He7/AoAPbUuvto7fBfCX1tp7ALwRNJaeuQfGmF0A/h2AR621D4Bq1XwQd/Z9+CSAx9e0bTbn7wFwmP89CeBjt6mP18InsX4MXwHwgLX2DQBOAPg1AODn+oMA7uff/A/TlV/2zsTtlMAfA3DKWnvaWtsC8GkA77+N179uWGunrLXP8/YK6MWxC9TvT/FhnwLwU9vTw2vDGLMbwI8D+H3+vwHwLgCf5UPu9P5XALwDXLLPWtuy1i6ih+4BIwJQMMZEAIoApnAH3wdr7TcBzK9p3mzO3w/gDy3hGVDB8/Hb09PNsdEYrLV/ZSVJ+zOQEsLvB/Bpa23TWnsGwCn0QMWx2/kC3wXggvr/JLf1BIwx+0Gl5b4DYMxaOwXQSx7Aju3r2TXx3wD8BwAuq/0wgEW1iO/0+3AQwBUAf8BmoN83xpTQQ/fAWnsRwH8FcB704l4C8Bx66z4Am895rz7b/xrA/+XtnhzD7XyBb1SxsydcYIwxZQCfA/DL1trlax1/p8AY8xMAZqy1z+nmDQ69k+9DBOARAB+z1j4MSsVwx5pLNgLbit8P4ACACQAlkNlhLe7k+3A19NqagjHm10Em0j92TRscdkePAbi9L/BJAHvU/3cDuHQbr39DMMZkQC/vP7bWfp6bp52KyH9nNvv9NuMHAbzPGHMWZLJ6F0giH2BVHrjz78MkgElr7Xf4/58FvdB75R4AwI8COGOtvWKtbQP4PIAfQG/dB2DzOe+pZ9sY8wSAnwDwc1b8qHtqDA638wX+LIDDzLxnQYTBF2/j9a8bbC/+OIBj1trfVru+COAJ3n4CwBdud9+2AmvtUPp/igAAAUpJREFUr1lrd1tr94Pm+6vW2p8D8DUAH+DD7tj+A4C19jKAC8aYu7np3QBeRY/cA8Z5AG81xhR5Tbkx9Mx9YGw2518E8AvsjfJWAEvO1HKnwRjzOIBfBfA+a21N7foigA8aY3LGmAMgQva729HH64K19rb9A/BeEPP7OoBfv53XvsH+vh2kRr0M4EX+916QHflpACf579B293ULY3kngC/x9kHQ4jwF4H8DyG13/67R94cAHOX78OcABnvtHgD4KIDXAHwfwB8ByN3J9wHAn4Ds9W2QdPqhzeYcZH74PX6uvwfytrlTx3AKZOt2z/P/VMf/Oo/hOID3bHf/t/LPR2J6eHh49Ch8JKaHh4dHj8K/wD08PDx6FP4F7uHh4dGj8C9wDw8Pjx6Ff4F7eHh49Cj8C9zDw8OjR+Ff4B4eHh49Cv8C9/Dw8OhR/H86g/sGL68EWQAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "GroundTruth: cat ship ship plane\n" ] } ], "source": [ "dataiter = iter(testloader)\n", "images, labels = dataiter.next()\n", "\n", "# print images\n", "imshow(torchvision.utils.make_grid(images))\n", "print('GroundTruth: ', ' '.join('%5s' % classes[labels[j]] for j in range(4)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "那么，我们的神经网络是如何预测上面的图片的呢？" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [], "source": [ "outputs = net(images)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "outputs给出十个类的置信度，置信度越高，代表网络认为这张图片越可能属于这一类别，所以我们现在获取置信度最高的类别的下标。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Predicted: cat ship ship ship\n" ] } ], "source": [ "_, predicted = torch.max(outputs, 1)\n", "\n", "print('Predicted: ', ' '.join('%5s' % classes[predicted[j]]\n", " for j in range(4)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "结果似乎还行，我们来看一下网络在整个数据集上的表现。" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy of the network on the 10000 test images: 53 %\n" ] } ], "source": [ "correct = 0\n", "total = 0\n", "with torch.no_grad():\n", " for data in testloader:\n", " images, labels = data\n", " outputs = net(images)\n", " _, predicted = torch.max(outputs.data, 1)\n", " total += labels.size(0)\n", " correct += (predicted == labels).sum().item()\n", "\n", "print('Accuracy of the network on the 10000 test images: %d %%' % (\n", " 100 * correct / total))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "看起来，这个准确率比随机选择10%的准确率要高得多，看来我们的网络学到了一些东西。\n", "\n", "让我们看下哪些类的准确率高，哪些类的准确率低：" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy of plane : 57 %\n", "Accuracy of car : 58 %\n", "Accuracy of bird : 57 %\n", "Accuracy of cat : 49 %\n", "Accuracy of deer : 49 %\n", "Accuracy of dog : 19 %\n", "Accuracy of frog : 41 %\n", "Accuracy of horse : 67 %\n", "Accuracy of ship : 73 %\n", "Accuracy of truck : 61 %\n" ] } ], "source": [ "class_correct = list(0. for i in range(10))\n", "class_total = list(0. for i in range(10))\n", "with torch.no_grad():\n", " for data in testloader:\n", " images, labels = data\n", " outputs = net(images)\n", " _, predicted = torch.max(outputs, 1)\n", " c = (predicted == labels).squeeze()\n", " for i in range(4):\n", " label = labels[i]\n", " class_correct[label] += c[i].item()\n", " class_total[label] += 1\n", "\n", "\n", "for i in range(10):\n", " print('Accuracy of %5s : %2d %%' % (\n", " classes[i], 100 * class_correct[i] / class_total[i]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "那么接下来，我们该如何在GPU上运行这个神经网络呢？\n", "\n", "# 在GPU上训练\n", "\n", "就像把Tensor转移到GPU上一样，神经网络也可以转移到GPU上。\n", "\n", "如果CUDA可用，首先让我们将设备定义为第一个可见的CUDA设备：" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "cuda:0\n" ] } ], "source": [ "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n", "\n", "# Assuming that we are on a CUDA machine, this should print a CUDA device:\n", "\n", "print(device)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "接下来，我们假定`device`为CUDA设备。\n", "\n", "下面这些方法将逐一将模块转移到CUDA设备上。" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Net(\n", " (conv1): Conv2d(3, 6, kernel_size=(5, 5), stride=(1, 1))\n", " (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n", " (conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))\n", " (fc1): Linear(in_features=400, out_features=120, bias=True)\n", " (fc2): Linear(in_features=120, out_features=84, bias=True)\n", " (fc3): Linear(in_features=84, out_features=10, bias=True)\n", ")" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "net.to(device)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "请记得你需要将输入和每一步的目标都设定在GPU。" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "inputs, labels = data[0].to(device), data[1].to(device)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "为什么相比CPU，GPU在这里没有表现出很大的加速？这是因为这里的网络真的很小。\n", "\n", "**至此，我们实现了：**\n", "* 全面了解PyTorch的Tensor库和神经网络。\n", "* 训练一个小型神经网络对图像进行分类。" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.0" } }, "nbformat": 4, "nbformat_minor": 2 }