From 41a072972a3caa29e4fc5dff7eaa05bcb32b1f08 Mon Sep 17 00:00:00 2001 From: Vasily Griaznov Date: Tue, 18 Feb 2025 16:27:56 +0100 Subject: [PATCH 1/2] Removed the metadata of the notebook --- notebooks/1-intro-to-linear-algebra.ipynb | 9306 ++++++++------------- 1 file changed, 3716 insertions(+), 5590 deletions(-) diff --git a/notebooks/1-intro-to-linear-algebra.ipynb b/notebooks/1-intro-to-linear-algebra.ipynb index 13baf0e..826a278 100644 --- a/notebooks/1-intro-to-linear-algebra.ipynb +++ b/notebooks/1-intro-to-linear-algebra.ipynb @@ -1,5591 +1,3717 @@ { - "nbformat": 4, - "nbformat_minor": 0, - "metadata": { - "colab": { - "name": "1-intro-to-linear-algebra.ipynb", - "provenance": [], - "include_colab_link": true - }, - "kernelspec": { - "display_name": "Python 3", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.7.6" - } - }, - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "view-in-github", - "colab_type": "text" - }, - "source": [ - "\"Open" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "aTOLgsbN69-P" - }, - "source": [ - "# Intro to Linear Algebra" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "yqUB9FTRAxd-" - }, - "source": [ - "This topic, *Intro to Linear Algebra*, is the first in the *Machine Learning Foundations* series.\n", - "\n", - "It is essential because linear algebra lies at the heart of most machine learning approaches and is especially predominant in deep learning, the branch of ML at the forefront of today’s artificial intelligence advances. Through the measured exposition of theory paired with interactive examples, you’ll develop an understanding of how linear algebra is used to solve for unknown values in high-dimensional spaces, thereby enabling machines to recognize patterns and make predictions.\n", - "\n", - "The content covered in *Intro to Linear Algebra* is itself foundational for all the other topics in the Machine Learning Foundations series and it is especially relevant to *Linear Algebra II*." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "d4tBvI88BheF" - }, - "source": [ - "Over the course of studying this topic, you'll:\n", - "\n", - "* Understand the fundamentals of linear algebra, a ubiquitous approach for solving for unknowns within high-dimensional spaces.\n", - "\n", - "* Develop a geometric intuition of what’s going on beneath the hood of machine learning algorithms, including those used for deep learning.\n", - "* Be able to more intimately grasp the details of machine learning papers as well as all of the other subjects that underlie ML, including calculus, statistics, and optimization algorithms." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Z68nQ0ekCYhF" - }, - "source": [ - "**Note that this Jupyter notebook is not intended to stand alone. It is the companion code to a lecture or to videos from Jon Krohn's [Machine Learning Foundations](https://github.com/jonkrohn/ML-foundations) series, which offer detail on the following:**\n", - "\n", - "*Segment 1: Data Structures for Algebra*\n", - "\n", - "* What Linear Algebra Is \n", - "* A Brief History of Algebra\n", - "* Tensors\n", - "* Scalars\n", - "* Vectors and Vector Transposition\n", - "* Norms and Unit Vectors\n", - "* Basis, Orthogonal, and Orthonormal Vectors\n", - "* Arrays in NumPy \n", - "* Matrices\n", - "* Tensors in TensorFlow and PyTorch\n", - "\n", - "*Segment 2: Common Tensor Operations*\n", - "\n", - "* Tensor Transposition\n", - "* Basic Tensor Arithmetic\n", - "* Reduction\n", - "* The Dot Product\n", - "* Solving Linear Systems\n", - "\n", - "*Segment 3: Matrix Properties*\n", - "\n", - "* The Frobenius Norm\n", - "* Matrix Multiplication\n", - "* Symmetric and Identity Matrices\n", - "* Matrix Inversion\n", - "* Diagonal Matrices\n", - "* Orthogonal Matrices\n" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "2khww76J5w9n" - }, - "source": [ - "## Segment 1: Data Structures for Algebra\n", - "\n", - "**Slides used to begin segment, with focus on introducing what linear algebra is, including hands-on paper and pencil exercises.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "niG_MgK-iV6K" - }, - "source": [ - "### What Linear Algebra Is" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "LApX90aliab_" - }, - "source": [ - "import numpy as np\n", - "import matplotlib.pyplot as plt" - ], - "execution_count": 1, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "E4odh9Shic1S" - }, - "source": [ - "t = np.linspace(0, 40, 1000) # start, finish, n points" - ], - "execution_count": 2, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "N-tYny12nIyO" - }, - "source": [ - "Distance travelled by robber: $d = 2.5t$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "e_zDOxgHiezz" - }, - "source": [ - "d_r = 2.5 * t" - ], - "execution_count": 3, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "djVjXZy-nPaR" - }, - "source": [ - "Distance travelled by sheriff: $d = 3(t-5)$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "JtaNeYSCifrI" - }, - "source": [ - "d_s = 3 * (t-5)" - ], - "execution_count": 4, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "SaaIjJSEigic", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 472 - }, - "outputId": "c354282a-095b-4318-de3a-f26e4c45e354" - }, - "source": [ - "fig, ax = plt.subplots()\n", - "plt.title('A Bank Robber Caught')\n", - "plt.xlabel('time (in minutes)')\n", - "plt.ylabel('distance (in km)')\n", - "ax.set_xlim([0, 40])\n", - "ax.set_ylim([0, 100])\n", - "ax.plot(t, d_r, c='green')\n", - "ax.plot(t, d_s, c='brown')\n", - "plt.axvline(x=30, color='purple', linestyle='--')\n", - "_ = plt.axhline(y=75, color='purple', linestyle='--')" - ], - "execution_count": 5, - "outputs": [ - { - "output_type": "display_data", - "data": { - "text/plain": [ - "
" - ], - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkQAAAHHCAYAAABeLEexAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAACLqklEQVR4nOzdd1hT99/G8XcSNjIUEcG9t1L3BnfVVq0DZ6u2Kirura1arbtqHVXEuhX3aNWqrRP3XnXvLUMRkB2S8/zhr3lK1QoYCOPzui6uJiffc3LH0OTmTJWiKApCCCGEEFmY2tQBhBBCCCFMTQqREEIIIbI8KURCCCGEyPKkEAkhhBAiy5NCJIQQQogsTwqREEIIIbI8KURCCCGEyPKkEAkhhBAiy5NCJIQQQogsTwqRECLZChYsyGeffWbqGAZJzbNixQpUKhVnz55Ng1QZX3p7n4VITVKIhDChhQsXolKpqFatWrLm8/T0RKVSGX4sLCwoVKgQvXr14vHjx6mU1jj+nd3a2pry5cszZ84c9Hq9qeOlmbt37+Lt7U3hwoWxsrLC3t6eWrVqMXfuXGJiYkwdL1muXbvG999/z4MHD0wdRYgUMzN1ACGyMn9/fwoWLMjp06e5c+cORYsWTfK8efPmZerUqQDEx8dz7do1Fi1axB9//MH169exsbFJrdgf7Z/ZX7x4wdq1axk8eDAhISFMnjzZxOlS3++//067du2wtLTkq6++omzZssTHx3P06FGGDx/O1atXWbx4saljJtm1a9eYMGECnp6eFCxY0NRxhEgRKURCmMj9+/c5fvw4W7duxdvbG39/f8aPH5/k+R0cHOjSpUuiaYUKFaJfv34cO3aMRo0aGTuy0fw7e+/evSlZsiTz589n4sSJaDQaE6b7eFFRUdja2r7zsfv379OhQwcKFCjAgQMHcHV1NTzm4+PDnTt3+P3339MqqhDif2STmRAm4u/vT/bs2WnevDlt27bF39//o5eZO3duAMzM/v9vnYcPH9K3b19KlCiBtbU1Tk5OtGvX7q3NG3/vX3Ps2DGGDBmCs7Mztra2fPHFF4SEhHzwuVeuXImZmRnDhw9Pdm4rKyuqVKnC69evCQ4ONkxPSEjghx9+oEiRIlhaWlKwYEHGjBlDXFzcO5fz559/4u7ujpWVFaVLl2br1q3vHBcdHY23tzdOTk7Y29vz1Vdf8erVq7fG7d69mzp16mBra4udnR3Nmzfn6tWricZ069aNbNmycffuXZo1a4adnR2dO3d+72udMWMGkZGRLF26NFEZ+lvRokUZOHCg4f7y5cupX78+uXLlwtLSktKlS+Pr6/vWfCqViu+///6t6QULFqRbt26Jpl2+fBkPDw+sra3JmzcvkyZNYvny5ahUqndu9jp69ChVq1bFysqKwoULs2rVKsNjK1asoF27dgDUq1fPsCn00KFD7/03ECI9kjVEQpiIv78/rVu3xsLCgo4dO+Lr68uZM2eoUqVKkubX6XS8ePECAK1Wy/Xr1xk/fjxFixalVq1ahnFnzpzh+PHjdOjQgbx58/LgwQN8fX3x9PTk2rVrb21a69+/P9mzZ2f8+PE8ePCAOXPm0K9fPzZs2PDeLIsXL6Z3796MGTOGSZMmpeBfAx48eIBKpcLR0dEwrUePHqxcuZK2bdsydOhQTp06xdSpU7l+/Trbtm1LNP/t27dp3749vXv3pmvXrixfvpx27dqxZ8+et9aW9evXD0dHR77//ntu3ryJr68vDx8+5NChQ6hUKgBWr15N165dadKkCdOnTyc6OhpfX19q167NhQsXEm0aSkhIoEmTJtSuXZuZM2f+5+bKHTt2ULhwYWrWrJmkfxdfX1/KlClDixYtMDMzY8eOHfTt2xe9Xo+Pj0+SlvFPT58+NRSX0aNHY2try5IlS7C0tHzn+Dt37tC2bVu++eYbunbtyrJly+jWrRuVKlWiTJky1K1blwEDBjBv3jzGjBlDqVKlAAz/FSLDUIQQae7s2bMKoOzdu1dRFEXR6/VK3rx5lYEDByZpfg8PDwV466dUqVLKvXv3Eo2Njo5+a/4TJ04ogLJq1SrDtOXLlyuA0rBhQ0Wv1xumDx48WNFoNEpYWJhhWoECBZTmzZsriqIoc+fOVVQqlfLDDz8kOXvJkiWVkJAQJSQkRLlx44YyfPhwBTAsU1EU5eLFiwqg9OjRI9H8w4YNUwDlwIEDifIAypYtWwzTwsPDFVdXV+WTTz556zVWqlRJiY+PN0yfMWOGAii//faboiiK8vr1a8XR0VHp2bNnoucODAxUHBwcEk3v2rWrAiijRo364GsPDw9XAKVly5YfHPu3d71/TZo0UQoXLpxoGqCMHz/+rbEFChRQunbtarjfv39/RaVSKRcuXDBMe/nypZIjRw4FUO7fv59oXkA5fPiwYVpwcLBiaWmpDB061DBt06ZNCqAcPHgwya9LiPRGNpkJYQL+/v64uLhQr1494M3mjvbt27N+/Xp0Ol2SllGwYEH27t3L3r172b17N3PmzCE8PJymTZsm2sRlbW1tuK3Vann58iVFixbF0dGR8+fPv7XcXr16GdaSANSpUwedTsfDhw/fGjtjxgwGDhzI9OnT+e6775L8+m/cuIGzszPOzs6ULFmSH3/8kRYtWrBixQrDmF27dgEwZMiQRPMOHToU4K39bNzc3Pjiiy8M9//eFHbhwgUCAwPfeo3m5uaG+3369MHMzMzwnHv37iUsLIyOHTvy4sULw49Go6FatWocPHjwrdfUp0+fD77uiIgIAOzs7D449m//fP/Cw8N58eIFHh4e3Lt3j/Dw8CQv52979uyhRo0auLu7G6blyJHjvZv5SpcuTZ06dQz3nZ2dKVGiBPfu3Uv2cwuRnskmMyHSmE6nY/369dSrV4/79+8bplerVo1Zs2axf/9+Gjdu/MHl2Nra0rBhQ8P9Tz/9lNq1a1O5cmWmTZvGrFmzAIiJiWHq1KksX76cp0+foiiKYZ53faHmz58/0f3s2bMDvLWPTUBAAL///jsjR45M9n5DBQsW5JdffkGv13P37l0mT55MSEgIVlZWhjEPHz5ErVa/deRd7ty5cXR0fKugFS1aNFGRAyhevDjwZnPc3/tXARQrVizRuGzZsuHq6mrYf+b27dsA1K9f/5357e3tE903MzMjb968H3rZhvlev379wbF/O3bsGOPHj+fEiRNER0cneiw8PBwHB4ckLwve/LvWqFHjrenvO8Lx378P8OZ34l37XAmRkUkhEiKNHThwgOfPn7N+/XrWr1//1uP+/v5JKkTvUqlSJRwcHDh8+LBhWv/+/Vm+fDmDBg2iRo0aODg4oFKp6NChwzvP+/O+I7z+WaQAypQpQ1hYGKtXr8bb25tChQolOee/y1ytWrWoWLEiY8aMYd68eYnG/rvkpIW//11Wr16dqEj97Z87rQNYWlqiVn94hbu9vT1ubm5cuXIlSTnu3r1LgwYNKFmyJLNnzyZfvnxYWFiwa9cufvrppySdtympaxzfJ6m/D0JkdFKIhEhj/v7+5MqViwULFrz12NatW9m2bRuLFi1KtKkkOXQ6HZGRkYb7mzdvpmvXroY1RgCxsbGEhYWlaPl/y5kzJ5s3b6Z27do0aNCAo0eP4ubmlqJllS9fni5duuDn58ewYcPInz8/BQoUQK/Xc/v27UQ76AYFBREWFkaBAgUSLePOnTsoipKoQN26dQvgrXPj3L5927C5EiAyMpLnz5/TrFkzAIoUKQJArly5EhU3Y/jss89YvHgxJ06ceOeamn/asWMHcXFxbN++PdGamndtssuePftb72l8fDzPnz9PNK1AgQLcuXPnrfnfNS2pTFFahTA22YdIiDQUExPD1q1b+eyzz2jbtu1bP/369eP169ds3749Rcs/ePAgkZGRVKhQwTBNo9G89df8/PnzP3rNAbw5weK+ffuIiYmhUaNGvHz5MsXLGjFiBFqtltmzZwMYysmcOXMSjfv78ebNmyea/uzZs0RHnkVERLBq1Src3d3fWsuzePFitFqt4b6vry8JCQk0bdoUgCZNmmBvb8+UKVMSjftbUk5D8D4jRozA1taWHj16EBQU9Nbjd+/eZe7cucD/r53592bO5cuXvzVfkSJFEq0ZhDev89/vc5MmTThx4gQXL140TAsNDf2o0z78fc6ljy3ZQpiSrCESIg1t376d169f06JFi3c+Xr16dZydnfH396d9+/b/uazw8HDWrFkDvDns++/Dx62trRk1apRh3Geffcbq1atxcHCgdOnSnDhxgn379uHk5GSU11S0aFH+/PNPPD09adKkCQcOHHhrH5ukKF26NM2aNWPJkiWMHTuWChUq0LVrVxYvXkxYWBgeHh6cPn2alStX0qpVq0RreODN/kLffPMNZ86cwcXFhWXLlhEUFPTO8hAfH0+DBg3w8vLi5s2bLFy4kNq1axveF3t7e3x9ffnyyy+pWLEiHTp0wNnZmUePHvH7779Tq1Ytfv755xT9exUpUoS1a9fSvn17SpUqlehM1cePH2fTpk2G8wY1btwYCwsLPv/8c7y9vYmMjOSXX34hV65cb6356dGjB71796ZNmzY0atSIS5cu8ccff5AzZ85E40aMGMGaNWto1KgR/fv3Nxx2nz9/fkJDQ1O0tsfd3R2NRsP06dMJDw/H0tLScO4kITIMUx7iJkRW8/nnnytWVlZKVFTUe8d069ZNMTc3V168ePHeMf8+7F6lUik5cuRQWrRooZw7dy7R2FevXindu3dXcubMqWTLlk1p0qSJcuPGjbcOx/77kPQzZ84kmv/gwYNvHVL9z8Pu/3bq1CnFzs5OqVu37jsPFf9n9jJlyrzzsUOHDiU6fFyr1SoTJkxQChUqpJibmyv58uVTRo8ercTGxiaa7+88f/zxh1K+fHnF0tJSKVmypLJp06ZE4/5+jQEBAUqvXr2U7NmzK9myZVM6d+6svHz58q08Bw8eVJo0aaI4ODgoVlZWSpEiRZRu3bopZ8+eNYzp2rWrYmtr+97X+z63bt1SevbsqRQsWFCxsLBQ7OzslFq1ainz589P9Pq2b9+ulC9fXrGyslIKFiyoTJ8+XVm2bNlbh8jrdDpl5MiRSs6cORUbGxulSZMmyp07d956nxVFUS5cuKDUqVNHsbS0VPLmzatMnTpVmTdvngIogYGBb/27/puHh4fi4eGRaNovv/yiFC5cWNFoNHIIvsiQVIoie8YJIURWN2jQIPz8/IiMjMzwl04RIiVkHyIhhMhiYmJiEt1/+fIlq1evpnbt2lKGRJYl+xAJIUQWU6NGDTw9PSlVqhRBQUEsXbqUiIgIxo4da+poQpiMFCIhhMhimjVrxubNm1m8eDEqlYqKFSuydOlS6tata+poQpiMSTeZHT58mM8//xw3NzdUKhW//vproscVRWHcuHG4urpibW1Nw4YNDWeQ/VtoaCidO3fG3t4eR0dHvvnmm0TnYBFCCJHYlClTuHXrFtHR0URFRXHkyBGjn29JiIzGpIUoKiqKChUqvPMEdfDmOknz5s1j0aJFnDp1CltbW5o0aUJsbKxhTOfOnbl69Sp79+5l586dHD58mF69eqXVSxBCCCFEJpBujjJTqVRs27aNVq1aAW/WDrm5uTF06FCGDRsGvDnviouLCytWrKBDhw5cv36d0qVLc+bMGSpXrgy8uXBhs2bNePLkSYrPmiuEEEKIrCXd7kN0//59AgMDE63GdXBwoFq1apw4cYIOHTpw4sQJHB0dDWUIoGHDhqjVak6dOpXoytf/FBcXR1xcnOG+Xq8nNDQUJycnOQW9EEIIkUEoisLr169xc3NL0vUE/0u6LUSBgYEAuLi4JJru4uJieCwwMPCtM6GamZmRI0cOw5h3mTp1KhMmTDByYiGEEEKYwuPHj8mbN+9HLSPdFqLUNHr0aIYMGWK4Hx4eTv78+Xn8+HGKLjkghBDC9OKj4pnl9uYixkOfDcXC1sLEicTLK1c4OmQIcWFhZMublzpz5mD3jwsVJ9fd0Lt0+7Ubl4MuA9C7XG8WdV6EnZ3dR2dNt4Xo74sxBgUF4erqapgeFBSEu7u7YUxwcHCi+RISEggNDX3rYo7/ZGlpiaWl5VvT7e3tpRAJIUQGFa+Jxwor4M3nuRQi03p25AinBw9GExNDnnLl8PT1xfpf19ZLjvVX1tNrRy9ex7/GKbsTq75YRW2X2ixikVF2d0m3Z6ouVKgQuXPnZv/+/YZpERERnDp1iho1agBvTi4WFhbGuXPnDGMOHDiAXq+nWrVqaZ5ZCCGEEHB32zYCfHzQxcTgWqsWDVesSHEZitHG4L3Dm45bOvI6/jV18tfhYu+LNCvWzKiZTbqGKDIykjt37hju379/n4sXL5IjRw7y58/PoEGDmDRpEsWKFaNQoUKMHTsWNzc3w5FopUqV4tNPP6Vnz54sWrQIrVZLv3796NChgxxhJoQQQqQxRVG46ufH5fnzASjUogXVJk5EbW6eouXdfHETr81eXA66jAoVY+qM4XvP7zFTG7++mLQQnT17lnr16hnu/71fT9euXVmxYgUjRowgKiqKXr16ERYWRu3atdmzZw9WVlaGefz9/enXrx8NGjRArVbTpk0b5s2bl+avRQghhGmpzdRU6FrBcFukLb1Ox7nJk7m9YQMApXv0oMKgQSnenLXm8hp67+xNlDaKXLa5WPPFGhoVaWTMyImkm/MQmVJERAQODg6Eh4fLPkRCCCFEMiXExnJ8xAie7N8PKhWVRo+mROfOKVpWtDaafrv6sfzicgDqFayHf2t/XO1c3xprzO/vdLtTtRBCCCHSv7iwMA7360fIhQuoLSyoOW0a+Zs0SdGyrgZfxWuzF9dCrqFCxXiP8XxX9zs0ao2RU79NCpEQQohMQVEUtNFaAMxtzOVEu2kg6tkzDnp7E3HvHuZ2dnj8/DO5/nGy5KRSFIUVF1fgs8uHmIQYcmfLzdrWa6lXqN6HZzYSKURCCCEyBW20lqnZpgIwOnK0HHafysJu3eKgtzcxwcFYu7hQz88Px2LFkr2cyPhI+v7el9WXVwPQqHAj1rReQy7bXB+Y07ikEAkhhBAiWYJOn+Zw//5oIyNxKFoUz0WLsHV9ex+fD7kcdBmvTV7cfHkTtUrND/V+YFTtUahVab9TvBQiIYQQQiTZwz17ODFqFHqtFudKlfCYPx8LB4dkLUNRFH45/wsDdg8gThdHHrs8rGuzjjoF6qRS6g+TQiSEEEKIJLmxejXnp08HRSFfo0bUnD4dzTuu/PBfIuIi8N7pzfor6wFoWrQpq75YRU6blJ/F2hikEAkhhBDiPyl6PRdnz+b68jeHwhfr2JFKo0ej1iTv6K8Lzy/gtdmLO6F30Kg0TG0wlaE1h5pkE9m/SSESQgghxHvp4uM5NXYsD3buBKDCoEGU7tEjWUfxKYrCwjMLGfLnEOJ18eR3yM/6Nuupka9GasVONilEQgghhHgnbVQURwYOJPDECVRmZlSbOJHCLVsmaxlhsWH02N6DLde3ANCiRAuWt1xODuscqRE5xaQQCSGEyBTUGjWl25Y23BYfJyYkhEN9+vDq+nXMrK2p/dNPuNVJ3k7PZ56eof3m9twPu4+52pwZjWYwsNrAdHmOKClEQgghMgUzKzPabWpn6hiZQsSDBxzs1Yuop0+xcnLCY+FCnMqWTfL8iqIw99RcRuwdgVavpZBjITa03UCVPFVSMfXHkUIkhBBCCIMXly8T0KcPcWFhZMuXj3qLF2OXP3+S5w+NCaX7b93ZfnM7AG1KtWFJiyU4WjmmUmLjkEIkhBBCCACeHjrE0aFD0cXGkqNMGTx9fbFyckry/Ccen6DDlg48Cn+EhcaC2Y1n07dK33S5iezfpBAJIYTIFOKj4uXSHR/hzubNnJk4EUWnw7VOHWrPmoW5rW2S5tUremYdn8WYA2NI0CdQJHsRNrbbSEXXiqmc2nikEAkhhBBZmKIoXPH15a8FCwAo3KoVVb//HrW5eZLmfxH9gq6/dmXX7V0AtC/TnsWfL8be0j7VMqcGKURCCCFEFqVPSODspEnc2bQJgDK9elF+wIAkb+I68vAIHbd05Onrp1hqLJnXdB49K/bMEJvI/k0KkRBCCJEFJcTEcGz4cJ4ePAgqFZW//ZbiHTsmaV69omfa0WmMOzgOnaKjhFMJNrbbSHmX8qmcOvVIIRJCCCGymLiwMAJ8fHhx8SJqCwtq/fgj+Ro2TNK8wVHBdNnahb339gLQpXwXfJv7ks0iW2pGTnVSiIQQQogsJPLpUw55exNx/z7m9vZ4/PwzuSpVStK8B+8fpNPWTgRGBmJtZs2CZgvo5t4tQ24i+zcpREIIIUQW8erGDQ717k1MSAg2uXNTz88Ph6JFPzifTq9j0uFJTDw8Eb2ip7RzaTa23UiZXGXSIHXakEIkhBAiU1Br1BRrVsxwWyQWePIkhwcMICEqCodixai3aBE2uXN/cL7nr5/TZVsXDtw/AEB39+7MbzofW4ukHZKfUUghEkIIkSmYWZnR6fdOpo6RLj3YtYuTo0ejT0ggV5Uq1J03Dwv7Dx8Wv/fuXrps60JwVDC25rb4NvflywpfpkHitCeFSAghhMjErq9YwYUffwQgf5Mm1Jg6FY2l5X/Ok6BP4PtD3zPlyBQUFMrlKsfGdhspmbNkWkQ2CSlEQgghRCak6PVcmDmTGytXAlC8SxcqjRyJSv3fmxOfRDyh05ZOHHl0BADvSt781OQnrM2tUz2zKUkhEkIIkSnER8UzM9dMAIYFD8vSl+7Qxcdz8ttvebjrzdmj3YcOpVT37h88Gmz37d18ue1LXsa8xM7CjsWfL6ZD2Q5pEdnkpBAJIYTINLTRWlNHMDltZCSHBwwg6NQpVGZmVJ80iUKff/7f8+i0fHfgO2YcnwHAJ7k/YUPbDRRzKpYWkdMFKURCCCFEJhETEsJBb2/Cbt7EzMaGOnPm4Fqr1n/O8yj8ER02d+DEkxMA+FTxYWbjmViZWaVF5HRDCpEQQgiRCYTfu8chb2+inj3DyskJz0WLyFG69H/Os/3mdrr92o1Xsa9wsHRgaYultCndJo0Spy9SiIQQQogMLuTiRQL69iU+PBy7AgWo5+dHtnz53js+XhfPqH2j+OnkTwBUcavC+rbrKZy9cFpFTnekEAkhhBAZ2JMDBzg2fDi62FicypXDY+FCrHLkeO/4+6/u02FLB04/PQ3AoGqDmN5oOhaarLsTOkghEkIIITKsO5s2cWbiRBS9Hre6dak9axZmNjbvHb/1+la+/u1rwuPCyW6VnRWtVtCiRIs0TJx+SSESQgiRKajUKgp4FDDczswUReGvhQu5snAhAIVbt6bq+PGozd79tR6XEMewP4fx85mfAaietzrr26yngGOBNMuc3kkhEkIIkSmYW5vT7VA3U8dIdfqEBM5MnMjdLVsAKNu7N+X69XvvOYbuhN6h/eb2nH9+HoARNUcwqf4kzDXmaZY5I5BCJIQQQmQQCTExHB06lGcBAajUaiqPHUsxL6/3jt9wZQM9d/TkdfxrnKydWPXFKpoVa5aGiTMOKURCCCFEBhD76hUBffvy8vJlNJaW1Jo5k7z1679zbIw2hsF/DMbvnB8AtfPXZl2bdeS1z5uWkTMUKURCCCEyhfioeOYWnAvAwAcDM9WlOyKfPOGgtzevHzzAwsEBjwULcP7kk3eOvfniJl6bvbgcdBkVKkbXHs2EehMwU8tX/n+Rfx0hhBCZRvSLaFNHMLrQ69c51Ls3sS9eYOPqSj0/PxyKFHnn2DWX19B7Z2+itFE42zizpvUaGhdpnMaJMyYpREIIIUQ6FXjiBIcHDiQhKgrH4sXx9PPDJleut8ZFa6Ppv6s/yy4uA8CzoCdrW6/F1c41rSNnWFKIhBBCiHTo/s6dnPz2W5SEBFyqVqXOvHlY2Nm9Ne5ayDW8NnlxNeQqKlSM8xjH2Lpj0ag1JkidcUkhEkIIIdIRRVG4sWIFF2bOBCB/06bUmDIFjcXb+0StuLiCvr/3JSYhhtzZcuPf2p/6hd69o7X4b1KIhBBCiHRC0es5P2MGN1evBqDEV19RcfhwVGp1onGR8ZH47PJh1aVVADQs3JA1X6zBJZtLmmfOLKQQCSGEEOmALj6eE6NH82jPHgA+GT6cUt26vTXur6C/8NrsxY0XN1Cr1Ez0nMjoOqNRq9RvjRVJJ4VICCFEpqBSq3Cr7Ga4nZHEv37N4f79CT5zBrWZGdUnT6bgZ58lGqMoCkvOL2HAngHEJsTiZufGujbrqFugrolSZy5SiIQQQmQK5tbm9DzT09Qxki06KIhDvXsTdusWZra21J03j9zVqycaExEXgfdOb9ZfWQ9A06JNWdlqJc62zqaInClJIRJCCCFMJPzuXQ56exP9/DlWOXNSb9EispcqlWjMhecX8NrsxZ3QO2hUGqY0mMKwmsNkE5mRSSESQgghTCDk/HkCfHyIj4jAvlAhPP38yJYnj+FxRVHwPevL4D8GE6+LJ599Pta3XU/NfDVNmDrzkkIkhBAiU9BGa1lQegEAPtd8MLdJv1dzf7x/P8eHD0cXF4dThQp4LFiAVfbshsfDY8PpsaMHm69tBuDz4p+zotUKcljnMFXkTE8KkRBCiExBURTCH4YbbqdXtzds4OykSSh6PW4eHtSeNQsza2vD42eenqH95vbcD7uPudqc6Q2nM6j6IFSqjLWjeEYjhUgIIYRIA4qicHn+fK76vbkCfZG2bakydixqMzPD4/NOzWP43uFo9VoKOhZkQ9sNVM1T1ZSxswwpREIIIUQq02u1nJ44kXtbtwJQzseHsn36GNb6hMaE8vVvX/Pbzd8AaF2qNUtbLMXRytFUkbMcKURCCCFEKkqIjubo0KE8O3wYlVpNlXHjKNquneHxk09O0n5zex6FP8JCY8GsxrPwqeIjm8jSmBQiIYQQIpXEhoZyqE8fQq9cQWNlRa2ZM8lbrx4AekXP7BOzGb1/NAn6BIpkL8LGdhup6FrRxKmzJilEQgghRCqIfPyYA716EfnoEZaOjngsXEjOChUAeBH9gm6/duP3278D0L5MexZ/vhh7S3tTRs7SpBAJIYTIFFQqFc6lnQ23TSn06lUO9elD7MuX2Lq5UW/xYuwLFQLg6KOjdNjcgaevn2KpsWTup3PpVamXyTNndVKIhBBCZArmNub0vdrX1DF4fuwYRwYNIiE6GscSJajn54e1szN6Rc/0o9MZe3AsOkVHcafibGy7kQq5K5g6skAKkRBCCGE097dv5+TYsSgJCbhUr07duXMxz5aN4Khgvtz2JX/e/ROALuW74Nvcl2wW2UycWPxNCpEQQgjxkRRF4frSpVz86ScACjRvTvVJk9BYWHDowSE6benE88jnWJtZ83Ozn+nu3l02kaUzUoiEEEJkCtpoLb9U+QWAnmd6ptmlO/Q6HeenT+eWvz8AJbt145OhQ9GjMDFgIhMCJqBX9JTKWYpN7TZRJleZNMklkkcKkRBCiExBURRCroUYbqcFXVwcx0eN4vGfbzaFVRwxgpJduxIYGUjnrZ05cP8AAN3duzO/6XxsLWzTJJdIPilEQgghRArER0RwuH9/gs+eRW1uTo2pUynQtCn77u2j89bOBEcFY2tui29zX76s8KWp44oPkEIkhBBCJFN0YCAHe/cm/PZtzLNlo+68eThVqcTYA2OZfGQyCgrlcpVjY7uNlMxZ0tRxRRJIIRJCCCGSIezOHQ55exMdGIi1szOeixYR7WZHg1UNOPzwMAC9KvZizqdzsDa3/sDSRHohhUgIIYRIouBz5wjo1w9tRAT2hQtTz8+PI9GX+dLvS15EvyCbRTZ++fwXOpTtYOqoIpnUpg7wX3Q6HWPHjqVQoUJYW1tTpEgRfvjhh0Q7yymKwrhx43B1dcXa2pqGDRty+/ZtE6YWQgiRGT3eu5cDPXqgjYggp7s7niuW8cO1eTT1b8qL6Be453bnfK/zUoYyqHS9hmj69On4+vqycuVKypQpw9mzZ+nevTsODg4MGDAAgBkzZjBv3jxWrlxJoUKFGDt2LE2aNOHatWtYWVmZ+BUIIYRIKyqVCocCDobbxnRr7VrOTpkCikLe+vXJN3YATX5rxfHHxwHwqeLDzMYzsTKT752MSqWk1bGJKfDZZ5/h4uLC0qVLDdPatGmDtbU1a9asQVEU3NzcGDp0KMOGDQMgPDwcFxcXVqxYQYcOSWvpERERODg4EB4ejr29XFhPCCHEG4qicGnuXK798ub8RkW9vAhsV4HuO78hNCYUe0t7lrZYStvSbU2cNGsy5vd3ut5kVrNmTfbv38+tW7cAuHTpEkePHqVp06YA3L9/n8DAQBo2bGiYx8HBgWrVqnHixIn3LjcuLo6IiIhEP0IIIcQ/6bVaTn77raEMlenXlw3Vwmm56QtCY0Kp7FaZC94XpAxlEul6k9moUaOIiIigZMmSaDQadDodkydPpnPnzgAEBgYC4OLikmg+FxcXw2PvMnXqVCZMmJB6wYUQQmRo2qgojg4ZwvOjR1FpNBQa7kOvuGWcPnUagEHVBjGt4TQszSxNnFQYS7ouRBs3bsTf35+1a9dSpkwZLl68yKBBg3Bzc6Nr164pXu7o0aMZMmSI4X5ERAT58uUzRmQhhBAmoo3RsqLuCgC6He6GuXXKLt0R+/Ilh/r0IfTqVTRWVmgGt+PTJyMJjwvH0cqRFS1X0LJkSyMmF+lBui5Ew4cPZ9SoUYZ9gcqVK8fDhw+ZOnUqXbt2JXfu3AAEBQXh6upqmC8oKAh3d/f3LtfS0hJLS2n1QgiRmSh6hWdnnxlup8Trhw856O1N5OPHWGR35HyXoky9OxqA6nmrs77Nego4FjBaZpF+pOt9iKKjo1GrE0fUaDTo9XoAChUqRO7cudm/f7/h8YiICE6dOkWNGjXSNKsQQoiM7eWVK/zZpQuRjx9j6erCos9jmBq0CoDhNYdzuNthKUOZWLpeQ/T5558zefJk8ufPT5kyZbhw4QKzZ8/m66+/Bt4cVjlo0CAmTZpEsWLFDIfdu7m50apVK9OGF0IIkWE8O3KEo4MHkxATA4VyM7DiOZ7GhuNk7cTKVitpXry5qSOKVJauC9H8+fMZO3Ysffv2JTg4GDc3N7y9vRk3bpxhzIgRI4iKiqJXr16EhYVRu3Zt9uzZI+cgEkIIkST3fv2VU+PHoyQk8KqYI8PKHiJWo6d2/tqsa7OOvPZ5TR1RpIF0fR6itCLnIRJCiIwvPiqeqdmmAjA6cjQWthb/OV5RFK798guX5s4F4EoJFTPKXkWngdG1RzOx3kTM1Ol6vUGWZ8zvb3mnhRBCZDl6nY5zU6Zwe/16APaUDmdNmafktHVm9ReraVK0iYkTirQmhUgIIUSmYZPT5oNjEmJjOTFyJI/37UNRweoKgfxRPBTPgp74t/bHzc4tDZKK9EYKkRBCiEzBwtaC4SHD/3NMfHg4Af36EXL+PAlqWFDtMWfyRTKu7jjGeYxDo9akUVqR3kghEkIIkSVEPX/OIW9vwu/eJdpcz+xajwgtZMe+Nr9Sv1B9U8cTJiaFSAghRKYXdvs2B3r1JDY4hFBrLTPqPKJExToc/GINLtlcPrwAkelJIRJCCJEpaGO0+Df1B6Dz7s6GS3cEnTnDQZ++6KOieWIfy8y6TxjYfByjao+STWTCQAqREEKITEHRKzwMeGi4DfBwzx6OjhyBKkHHjZzRrP1Uy+ZOe6lboK4po4p0SAqREEKITOnyyqX89eNsVAqcyRPB7U6lOdFuDc62zqaOJtIhKURCCCEynX1TxxK2bScqYF/RV5QY2p9ZdUaiVqXrS3gKE5JCJIQQItMJ3bIbtRr2VI7D+4dN1Mpfy9SRRDonhegf4qPiidfEvzVdrVFjZmWWaNz7qNQqw458yR2rjdbyviupqFQqzG1SODZGa9ie/i7/PL19csYmxCag1+mNMtbcxhyVSvVmbFwC+gQjjbU2R6V+M1YXr0On1RllrJmVGWqNOvljtTp08f8x1tIMtVnyx+oT9CTEJbx3rMZCg8Zck/yxOj0Jsf8x1lyDxiL5YxW9gjZGa5SxajM1ZpZv/v9UFAVttJHGJuP/e/mMePfYtP6MCHr8yHBbp1I487kzM0duxNHC8b3/zvIZ8UZG/owwFilE/zDLbRZWvH1R2GLNitHp906G+zNzzXzvB2kBjwJ0O9TNcH9uwblEv4h+51i3ym70PNPTcH9B6QWEPwx/51jn0s70vdrXcP+XKr8Qci3knWMdCjgw6MEgw/0VdVfw7Oyzd461yWmT6ERm/k39DTsl/pu5jTljosYY7m9ss5Hbu26/cyzAeGW84fa2L7dxbfO1947953WHdnrv5NLKS+8dOyx4GLbOtgD8MeQPzi48+96xA+8PxLGgIwD7v93PiZkn3ju2z5U+5CqTC4AjU44QMCHgvWN7nO5Bnip5ADg59yT7Rux779iuB7tS0LMgAOcWn2N3v93vHdtxZ0eKNy8OwF/+f/Fb99/eO7btxraUaVcGgOvbrrPZa/N7x7Zc3hL3bu4A3PnjDus+W/fesU1/bkpVn6oAPDryiJX1Vr53bMMZDak1/M1f3s/PP2dJ1SXvHesx3gPP7z0BCLkegm9Z3/eOrTGsBo1/bAxA+KNw5haa+96xlftWpvmCN1cij34RzcxcM987tkLXCrRa0Qp4Uxj+vu7Vu5RuW5p2m9oZ7v/XWPmMeCO9fEYARPRpyJx+s9nVb5d8RpC5PyOMRTamCiGEyFRUluD99XTDGiUhkkKuds//Xy035FnIO6+WK6vD3z1WNpnJ6nDZZJb8sfIZkbKx7/r/PiwmjNG/dKPy+gfYxZsR6WhOE98lFKzgLp8RZI3PCGNe7V4KERj1H1QIIUTqO/XkFGNnd6LDPgusdGoSCjjTduUmbJzlkPqsxJjf37IPkRBCiAxDr+j56cRP/P7LFLqfcUGjqLCpXI7mC5dibmtr6ngiA5NCJIQQIkN4Gf2Srtu+wnz7aXpcyQ1AnuZNqTN5KmpzcxJiE9jYZiMAXlu8Em3GFOJD5LdFCCFEunfs0TE6bupAw4AEGt59c6RX6R49qDBokGF/Ib1Obziq7b/2RxLiXeQoMyGEEOmWXtEz9chUGi7xpM0f0PBuDlCpqDRmDO6DB8uRZMJoZA2REEKIdCk4Kpivtn3F0Wt7GXk0P8Vf2qC2sKDm9Onkb2zcc9AIIYVICCFEuhPwIICOWzqiDXrBhCOFcY2wwNzeHo+ffyZXpUqmjicyIdlkJoQQIt3Q6XVMDJhI/VX1MXsSyg+HiuIaYYFN7tw0Wr1aypBINbKGSAghRLoQGBlIl61d2H9/P6WDbRhxogjmcTocihbFc9EibF1dTR1RZGJSiIQQQpjc/nv76by1M0FRQdR9mpNep1xRJejIVbkydefPx0JOmitSmRQiIYQQJpOgT2BiwEQmHZ6EgkL3Z2VoeEwBdORr1Iia06ejsbRM0rIsbC0SXTBWiOSQQiSEEMIknr1+RsctHTn88DAqBaYGNST/0WcAFO/UiYqjRqHWaEycUmQVUoiEEEKkuT139vDlti95Ef0CB002Fj5uBkevAOA+eDClvvlGzjEk0pQUIiGEEGlGq9My7uA4ph2bBkDV7BX47lxpXp+9hMrMjOo//EChFi1StOyE2AS2fbkNgC9WfyGX7hDJIr8tQggh0sTj8Md02NKB44+PAzCwxDc02hhK+M1LmFlbU3vOHNxq107x8vU6Pdc2XwOg5YqWRskssg4pREIIIVLdzls76fprV0JjQrG3tOeXStOw/HEb4U+fYuXkhKevLznKlDF1TJGFyYkZhRBCpJp4XTxD/xjK5+s+JzQmlEqulQiotRbN92uIevqUbPnz02jNGilDwuRkDZEQQohU8SDsAR02d+DU01MADKw2kEHmzTk1aBS62FhylC2L58KFWDk5mTipEFKIhBBCpIJfb/xK99+6ExYbhqOVI8tbLqfslQROTBiCotfjWqcOtWfNwtzW1tRRhQBkk5kQQggjikuIY+DugXyx4QvCYsOolqca53udp8iBZ5wePx5Fr6fwF1/gMX++lCGRrsgaIiGEEEZxN/Qu7Te359zzcwAMqzGMSR4TuThlGnc3bwagTK9elB8wQM4xJNIdKURCCCE+2qarm+ixowcRcRHksM7BqlaraJKvPseGDufpwYOgUlHlu+8o1qFDqmUwtzFndORow20hkkMKkRBCiBSLTYhlyB9D8D3rC0CtfLVY12YduRQ7DvTowYuLF9FYWlJzxgzyNWyYqllUKhUWthap+hwi85JCJIQQIkVuvbyF1yYvLgVdAmB07dFMrDeR2OdB7PXuQsT9+1jY2+OxYAHOFSuaOK0Q/00KkRBCiGRb+9davHd6ExkfibONM6u/WE2Tok14df06B3v3JvbFC2xy56aenx8ORYumSaaEuAR2eu8E4DO/zzCzlK84kXTy2yKEECLJorXRDNw9kCUXlgDgUcCDtW3W4mbnRuDJkxweMICEqCgcixfHc9EibFxc0iybPkHPpZVv1lY1W9AMLNPsqUUmIIVICCFEklwPuY7XZi+uBF9BhYqxdccy1mMsZmozHvz+OyfHjEGfkECuKlWoO28eFvb2po4sRJJJIRJCCPFBKy+upO+uvkRro3GxdcG/tT8NCjcA4PqKFVz48UcA8n/6KTWmTkVjITs3i4xFCpEQQoj3ioqPwmeXDysvrQSgQaEGrGm9htzZcqPo9VyYOZMbK988VqJLFyqOHIlKLef8FRmPFCIhhBDvdCX4Cu02tePGixuoVWq+9/ieMXXGoFFr0MXHc3LMGB7u3g2A+9ChlOreXU64KDIsKURCCCESURSFpReW0n93f2ITYnGzc2Nt67V4FPQAIP71a44MGEDQ6dOozMyoPnkyhT77zMSphfg4UoiEEEIYvI57Te/fe7P2r7UAfFr0U1a1WoWzrTMA0cHBHPL2JuzWLcxsbKgzdy6uNWuaMrIQRiGFSAghBAAXAy/SfnN7br28hUalYXL9yQyvNRy16s0+QeH37nHI25uoZ8+wcnLC08+PHKVKmTj1/zO3MWdY8DDDbSGSQwqREEJkcYqisOjsIgb/MZg4XRx57fOyvs16auWvZRgTcuECAT4+xIeHY1egAPUWLyZb3rwmTP02lUqFrbOtqWOIDEoKkRBCZGHhseH03NGTTdc2AfBZ8c9Y0XIFTjZOhjFPDhzg2LBh6OLicCpfHo+FC7HKnt1UkYVIFVKIhBAiizr77CztN7fn3qt7mKnNmN5wOoOrD050pNidjRs588MPKHo9bh4e1J45EzMbGxOmfr+EuAT+GPIHAE1mN5FLd4hkSdFvy/379zly5AgPHz4kOjoaZ2dnPvnkE2rUqIGVlZWxMwohhDAiRVGYf3o+w/4chlavpYBDATa03UC1vNUSjflrwQKu+L65in2RNm2oMm4carP0WzL0CXrOLjwLQKMZjeTSHSJZkvWb7e/vz9y5czl79iwuLi64ublhbW1NaGgod+/excrKis6dOzNy5EgKFCiQWpmFEEKk0KuYV3yz/Ru23dgGQKuSrVjWYhnZrf9/E5g+IYEzEydyd8sWAMr26UM5Hx85x5DI1JJciD755BMsLCzo1q0bW7ZsIV++fIkej4uL48SJE6xfv57KlSuzcOFC2rVrZ/TAQgghUubUk1O039yeh+EPMVebM7PxTPpX7Z+o6CRER3N02DCeBQSgUqupPHYsxby8TJhaiLSR5EI0bdo0mjRp8t7HLS0t8fT0xNPTk8mTJ/PgwQNj5BNCCPGRFEVh9onZjNo/igR9AoWzF2ZD2w1UdqucaFxsaCgBPj68vHwZjaUltWbOJG/9+iZKLUTaSnIh+q8y9G9OTk44OTl9eKAQQohU9TL6Jd1+68bOWzsBaFe6Hb98/gsOVg6JxkU+ecLBXr14/fAhFg4OeCxYgPMnn5gishAm8VF7xwUHBxMcHIxer080vXz58h8VSgghxMc79ugYHbd05HHEYyw1lsz5dA7elbzf2hco9Pp1Dnl7E/vyJTaurtRbvBiHwoVNlFoI00hRITp37hxdu3bl+vXrKIoCvDkhlqIoqFQqdDqdUUMKIYRIOr2iZ8axGXx34Dt0io5iOYqxsd1G3HO7vzX2+fHjHBk4kIToaByLF8fTzw+bXLnSPrQQJpaiQvT1119TvHhxli5diouLixx5IIQQ6URIVAhf/foVe+7sAaBTuU4sar4IO0u7t8be37mTk99+i5KQgEvVqtSZNw8Lu7fHZRTm1uYMvD/QcFuI5EhRIbp37x5btmyhaNGixs4jhBAihQIeBNBpayeevX6GlZkVPzf9ma8/+fqtP1oVReH68uVcnDULgAJNm1J9yhQ0FhamiG00KrUKx4KOpo4hMqgUFaIGDRpw6dIlKURCCJEO6PQ6phyZwvcB36NX9JTMWZJN7TZRNlfZt8Yqej3np0/n5po1AJTs2pVPhg1DpVandWwh0pUUFaIlS5bQtWtXrly5QtmyZTE3T7xqskWLFkYJJ4QQ4r8FRgbSZWsX9t/fD0DXCl1Z0GwBthZvX+RUFxfHiTFjeLTnzea0T4YPp1S3bmkZN1Xp4nXs//bNv0ODyQ3QWGhMnEhkJCkqRCdOnODYsWPs3r37rcdkp2ohhEgb++/tp/PWzgRFBWFjbsPCZgvp6t71nWPjIyI4PGAAwWfOoDYzo/qUKRRs3jyNE6cunVbHiZknAPD83lMKkUiWFK0j7d+/P126dOH58+fo9fpEP1KGhBAiden0OsYfHE+j1Y0IigqibK6ynOl55r1lKDooiH1duxJ85gxmtrZ4+vllujIkxMdKUSF6+fIlgwcPxsXFxdh53vL06VO6dOmCk5MT1tbWlCtXjrNnzxoeVxSFcePG4erqirW1NQ0bNuT27dupnksIIUzh2etnNFjVgImHJ6Kg0OOTHpzqcYrSzqXfOT787l3+7NSJsFu3sMqZk0YrV5K7evU0Ti1E+peiQtS6dWsOHjxo7CxvefXqFbVq1cLc3Jzdu3dz7do1Zs2aRfbs/38RwhkzZjBv3jwWLVrEqVOnsLW1pUmTJsTGxqZ6PiGESEt/3PkD90XuBDwMIJtFNvxb+/NLi1+wMbd55/jgc+fY26UL0YGB2BcqROO1a8leqlQapxYiY0jRPkTFixdn9OjRHD16lHLlyr21U/WAAQOMEm769Onky5eP5cuXG6YVKlTIcFtRFObMmcN3331Hy5YtAVi1ahUuLi78+uuvdOjQwSg5hBDClBL0CYw9MJZpx6YBUMGlAhvbbaS4U/H3zvN43z6OjxiBLi4OpwoV8Fy4EEtHxzRKLETGo1L+PtV0MvyzlLy1QJWKe/fufVSov5UuXZomTZrw5MkTAgICyJMnD3379qVnz57Am/MhFSlShAsXLuDu7m6Yz8PDA3d3d+bOnfvO5cbFxREXF2e4HxERQb58+QgPD8fe3t4o2YUQwhgehz+m45aOHHt8DIA+lfswu8lsrMys3jvP7fXrOTt5MopeTx5PT2rNnImZtXVaRTaZ+Kh4pmabCsDoyNFY2Gbs8yqJD4uIiMDBwcEo398pWkN0//79j3rSpLp37x6+vr4MGTKEMWPGcObMGQYMGICFhQVdu3YlMDAQ4K19mVxcXAyPvcvUqVOZMGFCqmYXQoiP9fut3/nq168IjQnF3tKeXz7/Ba8yXu8drygKl+fN4+rixQAUaduWKmPHojb7qMtWCpElpGgfoitXrrz3sV9//TWlWd6i1+upWLEiU6ZM4ZNPPqFXr1707NmTRYsWfdRyR48eTXh4uOHn8ePHRkoshBAfT6vTMuzPYXy27jNCY0Kp5FqJ873O/2cZ0mu1nBo71lCGyvn4UPX777NUGTK3NqfPlT70udJHLt0hki1FhahJkybvXEu0ZcsWOnfu/NGh/ubq6krp0omPnChVqhSPHj0CIHfu3AAEBQUlGhMUFGR47F0sLS2xt7dP9COEEOnBg7AH1Fleh1kn3lxWY0DVARz7+hhFchR57zwJ0dEE9O/PvW3bUKnVVJ0wgXJ9+2a560yq1CpylclFrjK5UKmz1msXHy9FhahHjx40bNgw0WapDRs28NVXX7FixQpjZaNWrVrcvHkz0bRbt25RoEAB4M2+TLlz52b//v2GxyMiIjh16hQ1atQwWg4hhEgLv974lU/8PuHU01M4Wjmy1Wsrc5vOxdLM8r3zxIaGsq97d54fOYLGyoo68+ZRtG3bNEwtROaQonWpEyZMIDQ0lIYNG3L48GH27NlDjx49WL16NW3atDFauMGDB1OzZk2mTJmCl5cXp0+fZvHixSz+3yphlUrFoEGDmDRpEsWKFaNQoUKMHTsWNzc3WrVqZbQcQgiRmuIS4hi5byRzT705EKRqnqpsaLuBgo4F/3O+148ecdDbm8hHj7B0dMRj4UJyVqiQBonTJ128jiNTjgBQZ0wdOVO1SJYUHWX2t86dO3PmzBmePn3K2rVrDYe+G9POnTsZPXo0t2/fplChQgwZMsRwlBm82Ylw/PjxLF68mLCwMGrXrs3ChQspXvz9h6P+mzH3UhdCiOS49+oeXpu8OPf8HABDawxlSoMpWGj++wip0KtXOdSnD7EvX2KbJw/1/Pyw/48jgLMCOcos6zHm93eSC9H27dvfmqbVahk8eDCNGzdOdEHXjHZxVylEQghT2HxtM99s/4aIuAhyWOdgRcsVfF7i8w/O9+zoUY4OGkRCTAzZS5bEc9EirJ2d0yBx+iaFKOsxyWH3/7UJatmyZSxbtgyQi7sKIcSHxCbEMvSPoSw8uxCAWvlqsa7NOvI55PvgvPd++41T48ahJCSQu0YN6syZg3m2bKkdWYhML8mFSK/Xp2YOIYTIEm6/vI3XZi8uBl4EYFStUUysNxFzzX8fJq4oCteWLOHSnDkAFGjenOqTJqGxkLUgQhhD1jlBhRBCmNi6v9bRa2cvIuMjyWmTk9VfrObTop9+cD69Tsf5adO4tXYtAKW6d8d9yBBU6hQdKCyEeAcpREIIkcpitDEM2D2AJReWAFC3QF3Wtl5LHvs8H5xXFxfH8VGjePznnwBUHDmSkl99lap5hciKpBAJIUQquh5yHa/NXlwJvoIKFd/V/Y5xHuMwU3/44zc+PJzDAwYQfPYsanNzakydSoGmTdMgtRBZjxQiIYRIJasuraLP732I1kbjYuvCmtZraFi4YZLmjQ4M5KC3N+F37mCeLRt1583DpVq1VE6csZlZmdHjdA/DbSGSQ35jhBDCyKLio+i3ux8rLq4AoH6h+vi39id3tvdfUuifwu7c4ZC3N9GBgVg7O+Pp50f2EiVSMXHmoNaoyVPlw5shhXiXFBcivV7PnTt3CA4OfusItLp16350MCGEyIiuBF/Ba5MX119cR61S873H94ypMwaNOmlnTQ4+d46Afv3QRkRgX7gw9fz8sHVzS+XUQogUFaKTJ0/SqVMnHj58yL/P6yjnIRJCZEWKorDswjL67+5PTEIMrtlcWdtmLZ4FPZO8jEd//snxkSPRx8eT090djwULsHR0TLXMmY0uXsfJuScBqD6wuly6QyRLigpR7969qVy5Mr///juurq5Z7orKQgjxT6/jXtPn9z74/+UPQJMiTVj1xSpy2eZK8jJu+vtzbupUUBTyNmhAzRkzMLOySq3ImZJOq2PfiH0AVOlbRQqRSJYUFaLbt2+zefNmihYtauw8QgiRoVwKvITXZi9uvbyFRqVhUv1JjKg1ArUqaecIUhSFS3Pncu2XXwAo2r49lb/9FrVGvsyFSEspKkTVqlXjzp07UoiEEFmWoij4nfNj0J5BxOniyGufl3Vt1lE7f+0kL0Ov1XJq3Dju/+9akeUHDKBMr16y1l0IE0hRIerfvz9Dhw4lMDCQcuXKYW6e+JTz5cuXN0o4IYRIjyLiIui5oycbr24EoHmx5qxstRInG6ckL0MbFcXRwYN5fuwYKo2Gqt9/T5HWrVMrshDiA1JUiNq0aQPA119/bZimUqlQFEV2qhZCZGrnnp2j/eb23H11FzO1GdMaTGNwjcFJ3kQGEPPiBQF9+xJ69Soaa2tqz5pFHg+PVEwthPiQFBWi+/fvGzuHEEKka4qi8PPpnxm2dxjxungKOBRgfdv1VM9bPVnLef3wIQe9vYl8/BjL7NnxWLiQnLJWXQiTS1EhKlCggLFzCCFEuvUq5hXfbP+GbTe2AdCqZCuWtVhGduvsyVrOy7/+4lDfvsSFhmKbNy/1Fi/GXj5PhUgXklyItm/fTtOmTTE3N2f7/3YAfJ8WLVp8dDAhhEgPTj89TfvN7XkQ9gBztTkzG8+kf9X+yd7x+dmRIxwZPBhdTAzZS5fG09cX65w5Uyl11mRmZUbXg10Nt4VIDpXy7zMrvodarSYwMJBcuXKhVr9/W3lG3IcoIiICBwcHwsPDsbe3N3UcIUQ6oCgKP538iZH7RpKgT6Bw9sJsaLuBym6Vk72se9u2cWr8eBSdjtw1a1JnzhzMbW1TIbUQWYsxv7+TXKH/eXmOf1+qQwghMpPQmFC6/dqNHbd2ANC2dFuWfL4EByuHZC1HURSuLl7M5XnzACjYogXVJkxAY2Fh9MxCiI8j6xSFEOIfjj8+TofNHXgc8RhLjSU/NfmJ3pV7J3sTmV6n49yUKdxevx6A0t98Q4XBg+UcQ6lIp9VxbvE5ACr1qoTGXE5uKZIuyceJrv/f/9RJ8fjxY44dO5aiQEIIYQp6Rc/0o9Opu7wujyMeUyxHMU72OEmfKn2SXWISYmM5NmTImzKkUlFpzBjchwyRMpTKdPE6dvfbze5+u9HFZ6xdN4TpJbkQ+fr6UqpUKWbMmMH169ffejw8PJxdu3bRqVMnKlasyMuXL40aVAghUktIVAifrf2MUftHoVN0dCzbkXO9zuGe2z3Zy4oLC+Ngz5483rcPtbk5tWfNokTnzsYPLYQwqiRvMgsICGD79u3Mnz+f0aNHY2tri4uLC1ZWVrx69YrAwEBy5sxJt27duHLlCi4uLqmZWwghjOLww8N03NKRZ6+fYWVmxfym8/nmk29StDYn6tkzDnp7E3HvHuZ2dtSdPx+XKlVSIbUQwtiStQ9RixYtaNGiBS9evODo0aM8fPiQmJgYcubMySeffMInn3zyn0egCSFEeqHT65h6dCrjD41Hr+gpmbMkG9tupJxLuRQtL+zWLQ727k1MUBDWLi7UW7QIx+LFjZxaCJFaUrRTdc6cOWnVqpWRowghRNoIigyiy7Yu7Lu3D4CvKnzFgmYLyGaRLWXLO32awwMGoH39GociRfD088PW1dWYkYUQqUyOMhNCZCkH7h+g05ZOBEUFYWNuw4JmC+jm3i3Fy3v0xx8cHzkSvVaLc6VKeMyfj4VD8g7PF0KYnhQiIUSWoNPrmBgwkR8O/4CCQhnnMmxst5HSzqVTvMyba9Zwbto0UBTyNWxIzRkz0FhaGjG1ECKtSCESQmR6z14/o/PWzhx6cAiAbz75hnlN52FjbpOi5Sl6PRd/+onry5YBUKxDByqNGYNaI+e9MSUzSzM67uxouC1EcshvjBAiU/vz7p902dqFkOgQbM1t8fvMj87lU34YvC4+nlPjxvFgx5uzWFcYNIjSPXrIOYbSAbWZmuLNZUd2kTIfVYji4+O5f/8+RYoUwcxMupUQIv1I0Ccw7uA4ph6dCkAFlwpsbLeR4k4p/8LURkVxZNAgAo8fR6XRUG3iRArLASZCZAopOkY+Ojqab775BhsbG8qUKcOjR48A6N+/P9OmTTNqQCGESK4nEU+ot7KeoQz1rtSbkz1OflQZigkJYV+3bgQeP46ZtTUeCxZIGUpndFodF1dc5OKKi+i0cqZqkTwpKkSjR4/m0qVLHDp0CCsrK8P0hg0bsmHDBqOFE0KI5Np1exfui9w5+ugodhZ2bGi7Ad/PfLEys/rwzO8R8eABf3bpwqtr17DMkYMGy5fjVqeOEVMLY9DF6/it+2/81v03uXSHSLYUbef69ddf2bBhA9WrV0+03bxMmTLcvXvXaOGEECKptDot3x74lh+P/whARdeKbGy7kSI5inzUcl9cvkxA377EvXpFtnz5qOfnh12BAsaILIRIR1JUiEJCQsiVK9db06OiomTHQiFEmnsY9pAOWzpw8slJAPpX7c+PjX7E0uzjDoF/GhDA0aFD0cXEkKNMGTx9fbFycjJGZCFEOpOiTWaVK1fm999/N9z/uwQtWbKEGjVqGCeZEEIkwW83fsPdz52TT07iYOnAFq8tzGs676PL0N2tWzncvz+6mBhca9WiwfLlUoaEyMRStIZoypQpNG3alGvXrpGQkMDcuXO5du0ax48fJyAgwNgZhRDiLfG6eEbsHcHcU3MBqJqnKuvbrKdQ9kIftVxFUbiyaBF//fwzAIVatqTahAmozc0/OrMQIv1K0Rqi2rVrc/HiRRISEihXrhx//vknuXLl4sSJE1SqVMnYGYUQIpF7r+5Ra1ktQxkaUn0IR7of+egypNfpODNxoqEMlenVi+qTJ0sZEiILSPHJg4oUKcIvv/xizCxCCPFBm69t5pvt3xARF0F2q+ysbLWSz0t8/tHLTYiN5fjw4Tw5cABUKiqPGUPxTp2MkFgIkRGkqBDt2rULjUZDkyZNEk3/448/0Ov1NG3a1CjhhBDib7EJsQz9YygLzy4EoGa+mqxrs478Dvk/etlxYWEE+Pjw4uJF1BYW1Joxg3yNGn30ckXaMrM0o+3GtobbQiRHijaZjRo1Cp3u7XM8KIrCqFGjPjqUEEL80+2Xt6m5tKahDI2sNZJDXQ8ZpQxFPXvG3i+/5MXFi5jb21N/yRIpQxmU2kxNmXZlKNOuDGqzFH29iSwsRRX69u3blC799hWiS5YsyZ07dz46lBBC/G39lfX03NGTyPhIctrkZPUXq/m06KdGWfarGzc41Ls3MSEh2OTOjaefH45Fixpl2UKIjCVFFdrBwYF79+69Nf3OnTvY2tp+dCghhIjRxuC9w5uOWzoSGR9J3QJ1ueh90WhlKOjUKfZ17UpMSAgOxYrR2N9fylAGp0/Qc3XTVa5uuoo+QW/qOCKDSVEhatmyJYMGDUp0Vuo7d+4wdOhQWrRoYbRwQois6caLG1RbUo3F5xejQsV3db5j/1f7yWOfxyjLf7h7Nwe9vdFGRpKrcmUarVqFTe7cRlm2MJ2EuAQ2e21ms9dmEuISTB1HZDApKkQzZszA1taWkiVLUqhQIQoVKkSpUqVwcnJi5syZxs4ohMhCVl1aRaXFlfgr+C9y2ebizy//5If6P2CmNs5OsjdWruTYsGHotVryNW5MvcWLsbC3N8qyhRAZV4o+YRwcHDh+/Dh79+7l0qVLWFtbU758eerWrWvsfEKILCIqPop+u/ux4uIKAOoXqs+aL9bgaudqlOUrej0XZs3ixoo3yy/euTMVR45ErdEYZflCiIwtxX9yqVQqGjduTOPGjY2ZRwiRBV0NvorXZi+uhVxDrVIz3mM839b5Fo3aOGVFFx/Pye++4+H/LjnkPmQIpb7+Wq69KIQwSHEh2r9/P/v37yc4OBi9PvHOa8uWLfvoYEKIzE9RFJZfXE6/Xf2ISYjBNZsra9usxbOgp9GeQxsZyeGBAwk6eRKVmRnVf/iBQrKvoxDiX1JUiCZMmMDEiROpXLkyrq6u8leWECLZIuMj6b2zN/5/+QPQuEhjVn+xmly2uYz2HDEhIRzq3ZtXN25gZm1Nnblzca1Vy2jLF0JkHikqRIsWLWLFihV8+eWXxs4jhMgCLgVewmuzF7de3kKj0vBDvR8YWXskapXxTqYXcf8+B729iXr6FCsnJzx9fclRpozRli+EyFxSVIji4+OpWbOmsbMIITI5RVFYfG4xA/cMJE4XRx67PKxvu57a+Wsb9XleXLpEQN++xIWFkS1/fuovXky2fPmM+hwi/dFYaGi5vKXhthDJkaI/x3r06MHatWuNnUUIkYlFxEXQcUtHev/emzhdHM2LNedi74tGL0NPDh5k/9dfExcWRo6yZWns7y9lKIvQmGtw7+aOezd3NOZSiETypGgNUWxsLIsXL2bfvn2UL18ec3PzRI/Pnj3bKOGEEJnD+efn8drkxd1XdzFTmzG1wVSG1Bhi1E1kAHc2b+bMhAkoej1udetSe9YszGxsjPocQojMKUWF6PLly7i7uwNw5cqVRI/JDtZCiL8pisKCMwsY+udQ4nXx5HfIz4a2G6iet7rRn+eKry9/LVgAQOHWrak6bhzqf/2xJjI3fYKeO3+8uZ5m0SZF5QKvIllSVIgOHjxo7BxCiEwmLDaMb7Z/w9brWwFoWaIly1ouI4d1DqM+jz4hgTM//MDdzZsBKOPtTfn+/eWPsywoIS6BdZ+tA2B05GgszCxMnEhkJMY5F74QQvzD6aenab+5PQ/CHmCuNufHRj8yoNoAo5eUhJgYjg0bxtNDh1Cp1VT+7juKtW9v1OcQQmQNKS5EZ8+eZePGjTx69Ij4+PhEj23duvWjgwkhMh5FUZhzcg4j941Eq9dSyLEQG9puoEqeKkZ/rthXrwjw8eHlpUtoLC2p+eOP5GvQwOjPI4TIGlK0gXX9+vXUrFmT69evs23bNrRaLVevXuXAgQM4ODgYO6MQIgMIjQml5fqWDPlzCFq9ljal2nDe+3yqlKHIp0/Z26ULLy9dwsLenvpLlkgZEkJ8lBQVoilTpvDTTz+xY8cOLCwsmDt3Ljdu3MDLy4v8+fMbO6MQIp07/vg47ovc2XFrBxYaCxY0W8CmdptwtHI0+nO9un6dPzt14vWDB9i4utJozRqcK1Y0+vMIIbKWFBWiu3fv0rx5cwAsLCyIiopCpVIxePBgFi9ebNSAQoj0S6/omXFsBnWX1+VxxGOK5ijKyW9O0rdK31TZqTnw5En2du1K7IsXOBYvTmN/fxyKFDH68wghsp4UFaLs2bPz+vVrAPLkyWM49D4sLIzo6GjjpRNCpFshUSF8tvYzRu4biU7R0bFsR873Os8nrp+kyvM92LmTQ97eJERFkatKFRquWoWNi0uqPJcQIutJ0U7VdevWZe/evZQrV4527doxcOBADhw4wN69e2kg2/GFyPSOPDxChy0dePb6GVZmVsz7dB49KvZItUPdr69YwYUffwQg/6efUmPqVDQWcki1SExjoaHpz00Nt4VIDpWiKEpyZwoNDSU2NhY3Nzf0ej0zZszg+PHjFCtWjO+++47s2bOnRtZUExERgYODA+Hh4djb25s6jhDpll7RM/XIVMYdGode0VPCqQQb222kvEv5VHk+Ra/n/I8/cnPVKgBKfPklFUeMQKWWE+4JIYz7/Z2iQpTZSCES4sOCIoP4ctuX7L23F4Avy3/JwuYLyWaRLVWeTxcfz4kxY3i0ezcAnwwbRslu3eSEi0IIA2N+f6fozyyNRkNwcPBb01++fIlGk3qrKadNm4ZKpWLQoEGGabGxsfj4+ODk5ES2bNlo06YNQUFBqZZBiKzowP0DuPu5s/feXqzNrFnecjmrvliVamUo/vVrDnl782j3btRmZtSYPp1S3btLGRL/Sa/T8+DQAx4ceoBepzd1HJHBpKgQvW+lUlxcHBaptF3/zJkz+Pn5Ub584lXzgwcPZseOHWzatImAgACePXtG69atUyWDEFmNTq/j+0Pf03BVQwIjAynjXIazvc7Szb1bqj1ndHAw+776iqDTpzGzscFz0SIKffZZqj2fyDwSYhNYWW8lK+utJCE2wdRxRAaTrJ2q582bB7y5gOuSJUvIlu3//zrU6XQcPnyYkiVLGjchEBkZSefOnfnll1+YNGmSYXp4eDhLly5l7dq11K9fH4Dly5dTqlQpTp48SfXqxr2ApBBZyfPXz+m0tROHHhwC4Gv3r5nfbD425ql39fjwu3c56O1N9PPnWDk54ennR45SpVLt+YQQ4m/JKkQ//fQT8GYN0aJFixJtHrOwsKBgwYIsWrTIuAkBHx8fmjdvTsOGDRMVonPnzqHVamnYsKFhWsmSJcmfPz8nTpx4byGKi4sjLi7OcD8iIsLomYXIyP68+yddtnYhJDoEW3NbFn22iC7lu6Tqc4ZcuECAjw/x4eHYFSxIPT8/suXNm6rPKYQQf0tWIbp//z4A9erVY+vWrWlyNNn69es5f/48Z86ceeuxwMBALCwscHR0TDTdxcWFwMDA9y5z6tSpTJgwwdhRhcjwEvQJjD84nqlHp6KgUN6lPBvbbqREzhKp+rxPDhzg2LBh6OLicCpfHo+FC7HKYEerCiEythTtQ3Tw4MFEZUin03Hx4kVevXpltGAAjx8/ZuDAgfj7+2NlZWW05Y4ePZrw8HDDz+PHj422bCEyqicRT6i/sj5Tjk5BQcG7kjcnvzmZ6mXo9saNHBk4EF1cHG4eHjRYulTKkBAizaWoEA0aNIilS5cCb8pQ3bp1qVixIvny5ePQoUNGC3fu3DmCg4OpWLEiZmZmmJmZERAQwLx58zAzM8PFxYX4+HjCwsISzRcUFETu3Lnfu1xLS0vs7e0T/QiRle26vQv3Re4ceXQEOws71rdZz6LPFmFtbp1qz6koCpfnz+fMhAkoej1F2rSh7rx5mNmk3j5KQgjxPikqRJs2baJChQoA7NixgwcPHnDjxg0GDx7Mt99+a7RwDRo04K+//uLixYuGn8qVK9O5c2fDbXNzc/bv32+Y5+bNmzx69IgaNWoYLYcQmZVWp2XE3hE0X9uclzEvqehakfPe52lftn2qPq8+IYFT48Zx5X/7HJbt25eqEyagNkvRyfOFEOKjpejT5+XLl4Y1MLt27aJdu3YUL16cr7/+mrlz5xotnJ2dHWXLlk00zdbWFicnJ8P0b775hiFDhpAjRw7s7e3p378/NWrUkCPMhPiAR+GP6LC5AyeenACgf9X+/NjoRyzNLFP1eROiozk6bBjPAgJQqdVUGTuWol5eqfqcImvQmGtoOKOh4bYQyZGiQuTi4sK1a9dwdXVlz549+Pr6AhAdHZ2qJ2Z8l59++gm1Wk2bNm2Ii4ujSZMmLFy4ME0zCJHRbL+5nW6/duNV7CscLB1Y1nIZrUul/vm7YkNDCejbl5d//YXG0pJaM2eS93+nzBDiY2ksNNQaXsvUMUQGlaJC1L17d7y8vHB1dUWlUhkOez916lSqnIfon/69j5KVlRULFixgwYIFqfq8QmQG8bp4Ru4dyZxTcwCo4laFDW03UCh7oVR/7sjHjzno7c3rhw+xcHDAY+FCnN3dU/15hRAiKVJUiL7//nvKli3L48ePadeuHZaWb1axazQaRo0aZdSAQgjjuP/qPu03t+fMszensBhcfTDTGk7DQpP6V40PvXaNQ717E/vyJbZubnj6+eFQuHCqP6/IWvQ6Pc/PPwfAtaIrao1cBFgknVzcFbm4q8j8tlzbwjfbvyE8LpzsVtlZ0WoFLUq0SJPnfn78OEcGDiQhOhrHEiXwXLQIm1y50uS5RdYSHxXP1GxTARgdORoL29Qv+8K0jPn9neQ1RPPmzaNXr15YWVkZLuHxPgMGDPioUEII44hNiGXYn8NYcObNJuUaeWuwvu168jvkT5Pnv79jBye/+w4lIQGXatWoM3cuFnZ2afLcQgiRHEkuRD/99BOdO3fGysrKcAmPd1GpVFKIhEgH7oTewWuTFxcCLwAwouYIJtWfhLnGPNWfW1EUri9bxsXZswEo0KwZ1SdPRpNKF38WQoiPleRC9PdlO/59WwiR/qy/sp5eO3rxOv41OW1ysqrVKpoWa5omz63o9ZybPp1ba9YAULJbNz4ZOhSVWvbnEEKkX3IWNCEykRhtDIP2DGLx+cUA1Mlfh3Vt1pHHPk+aPL8uLo4To0fz6I8/APhk+HBKdeuWJs8thBAfI8mFaMiQIUle6Oz/rSYXQqSdmy9u4rXZi8tBl1Gh4ts63zLeczxm6rT5uyc+IoLDAwYQfOYMajMzqk+dSsFmzdLkuYUQ4mMl+ZPywoULie6fP3+ehIQESpR4c+HHW7duodFoqFSpknETCiE+aM3lNfTe2ZsobRS5bHOx5os1NCrSKM2ePzooiIPe3oTfvo2ZrS11580jt5wtXgiRgSS5EB08eNBwe/bs2djZ2bFy5UrDVe9fvXpF9+7dqVOnjvFTCiHeKSo+iv67+7P84nIA6hWsh39rf1ztXNMsQ/idOxz09iY6MBBrZ2c8Fy0ieyqfoFWId9GYa/AY72G4LURypOg8RHny5OHPP/+kTJkyiaZfuXKFxo0b8+zZM6MFTAtyHiKREV0NvorXZi+uhVxDhYrxHuP5ru53aNRp90UQfO4cAf36oY2IwL5QITz9/MiWJ232VxJCCJOch+jfAUJCQt6aHhISwuvXrz8qkBDivymKwoqLK/DZ5UNMQgy5s+Vmbeu11CtUL01zPN67l2MjRqCPjyenuzseCxZg6eiYphmEEMJYUlSIvvjiC7p3786sWbOoWrUq8OY6ZsOHD6d169S/QKQQWVVkfCR9fu/DmstvDmlvVLgRa1qvIZdt2p75+da6dZydPBkUhTz16lHrxx8xs7ZO0wxC/JuiVwi5/uaPdedSzqjUKhMnEhlJigrRokWLGDZsGJ06dUKr1b5ZkJkZ33zzDT/++KNRAwoh3rgcdBmvTV7cfHkTtUrND/V+YFTtUahVaXd+H0VRuDxvHlcXvzmsv2i7dlT+7jvUZnIGD2F62hgtvmV9Abl0h0i+FH2K2djYsHDhQn788Ufu3r0LQJEiRbC1tTVqOCHEmxLyy/lfGLB7AHG6OPLY5WFdm3XUKZC2BzDotVpOf/899379FYBy/fpRtndvVCr5K1wIkfF91J91tra2lC9f3lhZhBD/EhEXgfdOb9ZfWQ9As2LNWNlqJTltcqZpDm1UFEeHDuX5kSOoNBqqjBtH0bZt0zSDEEKkJlnPLUQ6deH5Bbw2e3En9A5majOm1J/C0JpD03QTGUDsy5cc6tOH0KtX0VhZUXvWLPJ4eqZpBiGESG1SiIRIZxRFYeGZhQz5cwjxunjyO+RnfZv11MhXI82zvH70iIPe3kQ+eoSloyMevr7klLXCQohMSAqREOlIWGwYPbb3YMv1LQC0KNGC5S2Xk8M6R5pneXnlCof69CEuNBTbPHmot3gx9gULpnkOIYRIC1KIhEgnzjw9Q/vN7bkfdh9ztTkzGs1gYLWBJtlp+dnRoxwdNIiEmBiylyqFp68v1s7OaZ5DCCHSihQiIUxMURTmnprLiL0j0Oq1FHIsxIa2G6iSp4pJ8tz77TdOjRuHkpBA7ho1qDNnDubZspkkixDJoTHXUGNYDcNtIZJDCpEQJhQaE0r337qz/eZ2ANqUasOSFktwtHJM8yyKonBtyRIuzZkDQMHPPqPaDz+gsZBzuYiMQWOhofGPjU0dQ2RQUoiEMJETj0/QYUsHHoU/wkJjwezGs+lbpa9JNpHpdTrOTZ3K7XXrACjVvTvuQ4agUqftEW1CCGEqUoiESGN6Rc+s47MYc2AMCfoEimQvwsZ2G6noWtEkeXRxcRwfOZLHe/eCSkXFkSMp+eWXJskixMdQ9Arhj8IBcMjvIJfuEMkihUiINPQi+gVdf+3Krtu7AGhfpj2LP1+MveXHXaU5peLDwwno35+Qc+dQm5tTY9o0Cnz6qUmyCPGxtDFa5haaC8ilO0TySSESIo0ceXiEjls68vT1U6zMrJj76Vx6VuxpsktfRD1/zqHevQm/cwfzbNmoO38+Lv+7WLMQQmQ1UoiESGV6Rc+0o9MYd3AcOkVHCacSbGy3kfIupjvBYdjt2xz09iYmKAjrXLnwXLSI7CVKmCyPEEKYmhQiIVJRcFQwXbZ2Ye+9vQB8Wf5LFjZfSDYL0x3GHnz2LAH9+6ONiMC+cGHq+flh6+ZmsjxCCJEeSCESIpUcvH+QTls7ERgZiLWZNQuaLaCbezeTXh3+0R9/cHzkSPRaLc6ffELdn3/G0tHRZHmEECK9kEIkhJHp9DomHZ7ExMMT0St6SjuXZmPbjZTJVcakuW76+3Nu6lRQFPI2aEDNGTMws7IyaSYhhEgvpBAJYUTPXz+n89bOHHxwEICv3b9mfrP52JjbmCyToihcmjOHa0uWAFCsfXsqffstao2cyVcIIf4mhUgII9l7dy9dtnUhOCoYW3NbfJv78mUF057PR6/VcmrcOO5vf3Mm7PIDBlCmVy+TbrYTIrWozdRU7lvZcFuI5JBCJMRHStAn8P2h75lyZAoKCuVylWNju42UzFnSpLm0UVEcGTSIwOPHUWk0VJ0wgSJffGHSTEKkJjNLM5ovaG7qGCKDkkIkxEd4EvGETls6ceTREQC8K3nzU5OfsDa3NmmumBcvCOjbl9CrV9FYW1N79mzy1K1r0kxCCJGeSSESIoV2397Nl9u+5GXMS+ws7Fj8+WI6lO1g6lhEPHzIIW9vIh8/xjJ7djwWLiRnedOd80iItKIoCtEvogGwyWkjm4ZFskghEiKZtDot3x34jhnHZwDwSe5P2NhuI0VzFDVxMnhx+TIBffsS9+oV2fLlo56fH3YFCpg6lhBpQhutZWaumYBcukMknxQiIZLhUfgjOmzuwIknJwDoV6UfPzb+ESsz0x++/uzIEY4MHowuJobspUvj6euLdc6cpo4lhBAZghQiIZJo+83tdPu1G69iX+Fg6cDSFktpU7qNqWMBcG/bNk6NH4+i05G7Zk3qzJmDua2tqWMJIUSGIYVIiA+I18Uzat8ofjr5EwBV3Kqwvu16CmcvbOJkb/aZuLp4MZfnzQOgYIsWVJswAY2FbCoQQojkkEIkxH+4/+o+7Te358yzMwAMqjaI6Y2mY6ExfeHQ63ScmzyZ2xs2AFC6Rw8qDBokO5IKIUQKSCES4j22Xt/K1799TXhcONmtsrOi1QpalGhh6lgAJMTGcnzkSJ7s2wcqFZVGj6ZE586mjiWEEBmWFCIh/iUuIY5hfw7j5zM/A1A9b3XWt1lPAcf0cbRWXFgYh/v1I+TCBdQWFtScNo38TZqYOpYQQmRoUoiE+Ic7oXdov7k955+fB2BEzRFMqj8Jc425iZO9EfXsGQe9vYm4dw9zOzs8fv6ZXJUrmzqWEOmC2kxNha4VDLeFSA4pREL8z4YrG+i5oyev41/jZO3Eqi9W0axYM1PHMgi7dYuD3t7EBAdj7eJCPT8/HIsVM3UsIdINM0szWq1oZeoYIoOSQiSyvBhtDIP/GIzfOT8Aauevzbo268hrn9fEyf5f0OnTHO7fH21kJA5FiuDp54etq6upYwkhRKYhhUhkaTdf3MRrsxeXgy6jQsWYOmP43vN7zNTp53+Nh3v2cGLUKPRaLc6VKuExfz4WDg6mjiVEuqMoCtpoLQDmNuZyxKVIlvTzqS9EGltzeQ29d/YmShuFs40z/q39aVSkkaljJXJj9WrOT58OikK+Ro2oOX06GktLU8cSIl3SRmuZmm0qIJfuEMknhUhkOdHaaPrv6s+yi8sAqFewHv6t/XG1Sz+boBS9nouzZ3N9+XIAinXsSKXRo1FrNCZOJoQQmZMUIpGlXAu5htcmL66GXEWFinEe4xhbdywadfopGrr4eE6NHcuDnTsBqDBoEKV79JDV/0IIkYqkEIksQVEUVlxcgc8uH2ISYsidLTf+rf2pX6i+qaMloo2K4sjAgQSeOIHKzIxqEydSuGVLU8cSQohMTwqRyPQi4yPp+3tfVl9eDUCjwo1Y/cVqXLK5mDhZYjEhIRzq04dX169jZm1N7Z9+wq1OHVPHEkKILEEKkcjULgddpv3m9tx4cQO1Ss1Ez4mMrjMatSp9nbQt4sEDDnp7E/XkCZY5cuDp64tT2bKmjiWEEFmGFCKRKSmKwi/nf2HgnoHEJsTiZufGujbrqFugrqmjveXF5csE9O1L3KtXZMuXj3qLF2OXP7+pYwkhRJYihUhkOhFxEXjv9Gb9lfUANC3alJWtVuJs62ziZG97eugQR4cORRcbS44yZfD09cXKycnUsYTIkNQaNaXbljbcFiI5pBCJTOXC8wt4bfbiTugdNCoNUxtMZWjNoeluExnA3S1bOD1hAopOh2vt2tSePRtzW1tTxxIiwzKzMqPdpnamjiEyKClEIlNQFAXfs74M/mMw8bp48tnnY0PbDdTIV8PU0d6iKApXfH35a8ECAAq3akXV779HbZ4+LiArhBBZkRQikeGFx4bTY0cPNl/bDECLEi1Y3nI5OaxzmDjZ2/QJCZydNIk7mzYBUKZXL8oPGCDnGBJCCBOTQiQytDNPz9B+c3vuh93HXG3O9IbTGVR9ULosGAkxMRwbPpynBw+CSkXlb7+leMeOpo4lRKYRHxUvl+4QKSaFSGRIiqIw99RcRuwdgVavpaBjQTa03UDVPFVNHe2d4sLCCPDx4cXFi6gtLKg1Ywb5GqWv66YJIURWJoVIZDihMaF8/dvX/HbzNwBal2rN0hZLcbRyNG2w94h69oyD3t5E3LuHub09Hj//TK5KlUwdSwghxD9IIRIZysknJ2m/uT2Pwh9hobFgVuNZ+FTxSZebyABe3bjBod69iQkJwSZ3bur5+eFQtKipYwkhhPgXKUQiQ9AremYdn8WYA2NI0CdQJHsRNrbbSEXXiqaO9l6BJ09yZOBAtJGROBQrRr1Fi7DJndvUsYQQQryDFCKR7r2IfkG3X7vx++3fAWhfpj2LP1+MvaW9iZO934Nduzg5ejT6hARyValC3XnzsLBPv3mFECKrk0Ik0rWjj47SYXMHnr5+iqXGknlN59GzYs90u4kM4PqKFVz48UcA8jdpQo2pU9FYWpo4lRBCiP+S/k7f+w9Tp06lSpUq2NnZkStXLlq1asXNmzcTjYmNjcXHxwcnJyeyZctGmzZtCAoKMlFiYSx6Rc/UI1PxXOHJ09dPKe5UnNM9T9OrUq90W4YUvZ7zM2YYylDxLl2oNXOmlCEh0ohao6ZYs2IUa1ZMLt0hkk2lKIpi6hDv8+mnn9KhQweqVKlCQkICY8aM4cqVK1y7dg3b/13ioE+fPvz++++sWLECBwcH+vXrh1qt5tixY0l+noiICBwcHAgPD8deNmuYXHBUMF9u+5I/7/4JQJfyXfBt7ks2i2wmTvZ+uvh4Tn77LQ937QLAfehQSnXvnm7LmxBCZAbG/P5O14Xo30JCQsiVKxcBAQHUrVuX8PBwnJ2dWbt2LW3btgXgxo0blCpVihMnTlC9evUkLVcKUfpx6MEhOm3pxPPI51ibWfNzs5/p7p6+i4U2MpLDAwYQdOoUKjMzqk+aRKHPPzd1LCGEyPSM+f2dofYhCg8PByBHjjeXZDh37hxarZaGDRsaxpQsWZL8+fP/ZyGKi4sjLi7OcD8iIiIVU4uk0Ol1TD4ymQkBE9Areko7l2Zj242UyVXG1NH+U0xICAe9vQm7eRMzGxvqzJmDa61apo4lhBAimTJMIdLr9QwaNIhatWpRtmxZAAIDA7GwsMDR0THRWBcXFwIDA9+7rKlTpzJhwoTUjCuSITAykM5bO3Pg/gEAurt3Z37T+dhapO8rv0fcv8/BXr2IevYMKycnPBctIkfp0qaOJUSWFR8Vz8xcMwEYFjxMLt0hkiXD7HXm4+PDlStXWL9+/Ucva/To0YSHhxt+Hj9+bISEIiX23dtHhUUVOHD/ALbmtqxqtYplLZel+zIUcvEif3buTNSzZ9gVKEBjf38pQ0KkA9poLdporaljiAwoQ6wh6tevHzt37uTw4cPkzZvXMD137tzEx8cTFhaWaC1RUFAQuf/jBHiWlpZYypE/JpWgT+D7Q98z5cgUFBTK5SrHxnYbKZmzpKmjfdCTAwc4Nnw4uthYnMqVw2PhQqz+txlXCCFExpSu1xApikK/fv3Ytm0bBw4coFChQoker1SpEubm5uzfv98w7ebNmzx69IgaNWqkdVyRRE8jntJgVQMmH5mMgkKvir041eNUhihDdzZt4sjAgehiY3GrW5cGy5ZJGRJCiEwgXa8h8vHxYe3atfz222/Y2dkZ9gtycHDA2toaBwcHvvnmG4YMGUKOHDmwt7enf//+1KhRI8lHmIm0tefOHr7c9iUvol+QzSIbv3z+Cx3KdjB1rA9SFIW/Fi7kysKFABRu3Zqq48ejNkvX/wsJIYRIonT9ae7r6wuAp6dnounLly+nW7duAPz000+o1WratGlDXFwcTZo0YeH/vrRE+qHVaRl7cCzTj00HwD23OxvbbqSYUzETJ/swfUICZyZO5O6WLQCU7d2bcv36petTAQghhEiedF2IknKKJCsrKxYsWMCCBQvSIJFIiUfhj+i4pSPHHx8HwKeKDzMbz8TKzMrEyT4sISaGo0OH8iwgAJVaTeWxYynm5WXqWEIIIYwsXRcikfHtuLmDbr91IzQmFHtLe5a2WErb0m1NHStJYl+9IsDHh5eXLqGxtKTmjz+Sr0EDU8cSQryHSq2igEcBw20hkkMKkUgV8bp4Ru8bzeyTswGo7FaZDW03UDh7YRMnS5rIJ0846O3N6wcPsLC3x2PBApwrVjR1LCHEfzC3NqfboW6mjiEyKClEwujuv7pPhy0dOP30NACDqg1ieqPpWGgyxknSQq9f51Dv3sS+eIGNqyv1/PxwKFLE1LGEEEKkIilEwqi2Xt/K1799TXhcOI5WjqxouYKWJVuaOlaSBZ44weGBA0mIisKxeHE8/fywyZXL1LGEEEKkMilEwijiEuIY9ucwfj7zMwDV81ZnfZv1FHAsYOJkSXd/505Offst+oQEXKpWpc68eVjY2Zk6lhAiieKj4plbcC4AAx8MlEt3iGSRQiQ+2p3QO7Tf3J7zz88DMLzmcCbXn4y5xtzEyZJGURRurFjBhZlvroGUv2lTakyZgsZCPkyFyGiiX0SbOoLIoKQQiY+y8epGemzvwev41zhZO7Gy1UqaF29u6lhJpuj1nJ8xg5urVwNQ4quvqDh8OCp1uj6JuxBCCCOTQiRSJEYbw5A/hrDo3CIAauevzbo268hrn/cDc6Yfuvh4TowZw6PduwH4ZPhwSv3vhJ9CCCGyFilEItluvriJ12YvLgddRoWK0bVHM6HeBMzUGefXKf71aw7370/wmTOozcyoPnkyBT/7zNSxhBBCmEjG+QYT6YL/ZX+8d3oTpY3C2caZNa3X0LhIY1PHSpbooCAO9e5N2K1bmNnaUnfePHLLte+EECJLk0IkkiRaG82A3QNYemEpAJ4FPfFv7Y+bnZuJkyVP+N27HPT2Jvr5c6xy5qTeokVkL1XK1LGEEEKYmBQi8UHXQq7htcmLqyFXUaFinMc4xtYdi0atMXW0ZAk5f54AHx/iIyKwK1iQen5+ZMubcfZ5EkL8N5VahVtlN8NtIZJDCpH4TysursBnlw/R2mhyZ8uNf2t/6heqb+pYyfZ4/36ODx+OLi4OpwoV8FiwAKvs2U0dSwhhRObW5vQ809PUMUQGJYVIvFNkfCQ+u3xYdWkVAA0LN2TNF2twyeZi4mTJd3vDBs5OmoSi1+Pm4UHtWbMws7Y2dSwhhBDpiBQi8Za/gv7Ca7MXN17cQK1SM9FzIqNqj8pwm8gUReHy/Plc9fMDoEibNlQZNw61mfzaCyGESEy+GYSBoigsOb+EAXsGEJsQi5udG+varKNugbqmjpZseq2W0xMncm/rVgDK+fhQtk8fVCrZr0CIzEobrWVB6QUA+FzzwdwmY5wtX6QPUogEAK/jXuO905t1V9YB8GnRT1nVahXOts4mTpZ8CdHRHB06lGeHD6NSq6kybhxF27UzdSwhRCpTFIXwh+GG20IkhxQiwYXnF/Da7MWd0DtoVBom15/M8FrDUasy3uUrYkNDCejbl5d//YXGyopaM2eSt149U8cSQgiRzkkhysIURcH3rC9D/hhCnC6OfPb5WN92PTXz1TR1tBSJfPyYg97evH74EEtHR+ouWICzu7upYwkhhMgApBBlUeGx4fTY0YPN1zYD8Hnxz1necjlONk4mTpYyoVevcqhPH2JfvsTWzY16ixdjX6iQqWMJIYTIIKQQZUFnn52l/eb23Ht1DzO1GTMazmBQ9UEZdofj58eOcWTQIBKio3EsUYJ6fn5YO2e8fZ+EEEKYjhSiLERRFOadmsfwvcPR6rUUdCzIhrYbqJqnqqmjpdj9HTs4+d13KAkJuFSvTt25czHPls3UsYQQQmQwUoiyiFcxr/h6+9f8euNXAL4o+QXLWi7D0crRpLlSSlEUri9bxsXZswEo0KwZ1SdPRmNhYeJkQghTUalUOJd2NtwWIjmkEGUBJ5+cpMPmDjwMf4iFxoJZjWfhU8Unw35g6HU6zk+fzi1/fwBKduvGJ0OHolJnvKPihBDGY25jTt+rfU0dQ2RQUogyMb2iZ/aJ2YzeP5oEfQJFshdhQ9sNVHKrZOpoKaaLi+P4qFE8/vNPACqOGEHJrl1NnEoIIURGJ4Uok3oZ/ZKuv3bl99u/A+BVxovFny3GwcrBxMlSLj4igsMDBhB85gxqc3OqT5lCwWbNTB1LCCFEJiCFKBM6+ugoHbd05EnEEyw1lsz5dA7elbwz7CYygOjAQA727k347duYZ8tG3XnzcKlWzdSxhBDpiDZayy9VfgGg55mecukOkSxSiDIRvaJn+tHpjD04Fp2io7hTcTa23UiF3BVMHe2jhN25wyFvb6IDA7F2dsZz0SKylyxp6lhCiHRGURRCroUYbguRHFKIMongqGC+2vYVf9z9A4DO5Trj29wXO0s7Eyf7OMHnzhHQrx/aiAjsCxemnp8ftm5upo4lhBAik5FClAkEPAig45aOPI98jrWZNT83+5nu7t0z9CYygMd793JsxAj08fHkdHfHY8ECLB0dTR1LCCFEJiSFKAPT6XVMPjKZCQET0Ct6SuUsxcZ2Gymbq6ypo320W2vXcnbKFFAU8tavT80ff8TMysrUsYQQQmRSUogyqMDIQLps7cL++/sB6ObejZ+b/oytha2Jk30cRVG4PG8eVxcvBqColxeVv/0WtZn8qgohhEg98i2TAe27t48uW7sQFBWEjbkNvs19+arCV6aO9dH0Wi2nxo/n/m+/AVCuXz/K9u6d4Tf9CSGESP+kEGUgCfoEJhyawOQjk1FQKJurLJvabaJkzox/xJU2KoqjQ4bw/OhRVBoNVcePp0ibNqaOJYTIQFQqFQ4FHAy3hUgOKUQZxNOIp3Ta2onDDw8D0LNiT+Z+Ohdrc2sTJ/t4sS9fcqhPH0KvXkVjZUXt2bPJ4+Fh6lhCiAzG3MacQQ8GmTqGyKCkEGUAe+7s4cttX/Ii+gXZLLKx+LPFdCzX0dSxjOL1w4cc9PYm8vFjLLNnx2PhQnKWL2/qWEIIIbIYKUTpmFanZdzBcUw7Ng0A99zubGi7geJOxU2czDheXrnCoT59iAsNxTZvXur5+WFfsKCpYwkhhMiCpBClU4/DH9NhSweOPz4OQN/KfZnVZBZWZpnj0PNnR45wdPBgEmJiyF6qFJ6+vlg7O5s6lhAiA9PGaFlRdwUA3Q53w9xaLt0hkk4KUTq089ZOuv7aldCYUOwt7VnaYiltS7c1dSyjuffrr5waPx4lIYHcNWtSZ84czG0z9ukChBCmp+gVnp19ZrgtRHJIIUpH4nXxjN43mtknZwNQ2a0yG9puoHD2wiZOZhyKonBtyRIuzZkDQMHPP6faxIloLCxMG0wIIUSWJ4UonXgQ9oAOmztw6ukpAAZWG8j0htOxNLM0cTLj0Ot0nJsyhdvr1wNQ6uuvcR88GJVabeJkQgghhBSidOHXG7/S/bfuhMWG4WjlyPKWy2lVspWpYxmNLi6O4yNG8HjfPlCpqDRqFCW6dDF1LCGEEMJACpEJxSXEMWLvCOadngdAtTzV2NB2AwUcC5g4mfHEh4cT0L8/IefOoTY3p8a0aRT49FNTxxJCCCESkUJkIndD79J+c3vOPT8HwLAaw5jSYArmmsxzVETU8+cc8vYm/O5dzO3sqDtvHi5Vq5o6lhBCCPEWKUQmsOnqJnrs6EFEXAQ5rHOwqtUqmhdvbupYRhV2+zYHvb2JCQrCOlcu6vn54Vg8c5w/SQiRftnktDF1BJFBSSFKQ7EJsQz5Ywi+Z30BqJWvFuvarCOfQz4TJzOuoDNnONy/P9rXr7EvXJh6fn7YurmZOpYQIpOzsLVgeMhwU8cQGZQUojRy6+UtvDZ5cSnoEgCja49mYr2JmKkz11vw6I8/OD5yJHqtFudPPqHuzz9j6eho6lhCCCHEf8pc38bp1Nq/1uK905vI+EicbZxZ/cVqmhRtYupYRnfT359zU6eCopC3YUNqTp+OmVXmOLO2EEKIzE0KUSqK1kYzcPdAllxYAoBnQU/8W/vjZpe5Nh8pisKln37i2tKlABTr0IFKY8ag1mhMnEwIkZVoY7T4N/UHoPPuznLpDpEsUohSyfWQ63ht9uJK8BVUqBhbdyzjPMahUWeukqCLj+fU+PE82L4dgAoDB1K6Z09UKpWJkwkhshpFr/Aw4KHhthDJIYUoFay8uJK+u/oSrY3GxdYF/9b+NCjcwNSxjE4bFcWRQYMIPH4clUZDtQkTKPzFF6aOJYQQQiSbFCIjioqPwmeXDysvrQSgQaEGrGm9htzZcps4mfHFvHjBoT59eHXtGhpra+r89BNudeqYOpYQQgiRIlKIjOSvoL/w2uzFjRc3UKvUTPCcwOjaozPdJjKAiIcPOeTtTeTjx1jmyIHnwoU4lStn6lhCCCFEikkh+kiKorD0wlL67+5PbEIsbnZurG29Fo+CHqaOlipeXL5MQN++xL16RbZ8+ajn54ddgcxzqREhhBBZkxSij/A67jW9f+/N2r/WAvBp0U9Z1WoVzrbOJk6WOp4ePszRIUPQxcSQo0wZPBYuxDpnTlPHEkIIIT6aFKIUuhh4Ea9NXtwOvY1GpWFy/ckMrzUctUpt6mip4u62bZwePx5Fp8O1Vi1q//QT5ra2po4lhBCJmNvIofYiZaQQJZOiKCw6u4jBfwwmThdHXvu8rG+znlr5a5k6WqpQFIWrfn5cnj8fgEItWlBt4kTU5vKhI4RIXyxsLRgTNcbUMUQGJYUoGcJjw+m5oyebrm0C4LPin7Gi5QqcbJxMnCx16HU6zk6ezJ0NGwAo3bMnFQYOlHMMCSGEyHSkECXR2Wdnab+5Pfde3cNMbcb0htMZXH1wpi0HCbGxHB8xgif794NKRaXRoynRubOpYwkhhBCpQgrRByiKwvzT8xn25zC0ei0FHAqwoe0GquWtZupoqSYuLIzD/foRcuECagsLak6bRv4mme/aa0KIzCUhNoGNbTYC4LXFCzMr+YoTSSe/Lf/hVcwrvtn+DdtubAOgVclWLGuxjOzW2U2cLPVEPXvGQW9vIu7dw9zeHo/588lVubKpYwkhxAfpdXpu77ptuC1Eckgheo9TT07RfnN7HoY/xEJjwcxGM+lXtV+m3UQG8OrmTQ717k1McDA2uXPj6eeHY9Gipo4lhBBCpLpMc4z4ggULKFiwIFZWVlSrVo3Tp0+naDmKojDr+CxqL6/Nw/CHFM5emONfH6d/tf6ZugwFnT7Nvq++IiY4GIeiRWm0Zo2UISGEEFlGpihEGzZsYMiQIYwfP57z589ToUIFmjRpQnBwcLKWExodSov1LRi2dxgJ+gTalW7H+V7nqeRWKZWSpw8Pd+/mYK9eaCMjyVW5Mo1WrcLW1dXUsYQQQog0kykK0ezZs+nZsyfdu3endOnSLFq0CBsbG5YtW5as5dRaXoudt3ZiqbHEt7kvG9puwMHKIZVSpw83Vq3i2LBh6LVa8jVqRL3Fi7FwyNyvWQghhPi3DF+I4uPjOXfuHA0bNjRMU6vVNGzYkBMnTiRrWc8inlEsRzFO9jhJ78q9M/UmMkWv58LMmZyfPh2A4p06UWvWLDSWliZOJoQQQqS9DL9T9YsXL9DpdLi4uCSa7uLiwo0bN945T1xcHHFxcYb74eHhALQq3IqfW/2MnaUdERERqRfaxHTx8ZydNImHe/YAUK5vX4p17UpkVJSJkwkhRMrFR8UTSywAERERWOgsTJxIpLa/v6sVRfn4hSkZ3NOnTxVAOX78eKLpw4cPV6pWrfrOecaPH68A8iM/8iM/8iM/8pMJfu7evfvRfSLDryHKmTMnGo2GoKCgRNODgoLInTv3O+cZPXo0Q4YMMdwPCwujQIECPHr0CIcstP9MREQE+fLl4/Hjx9jb25s6TpqR1y2vOyuQ1y2vOysIDw8nf/785MiR46OXleELkYWFBZUqVWL//v20atUKAL1ez/79++nXr98757G0tMTyHfvKODg4ZKlfpL/Z29vL685C5HVnLfK6s5as+rrV6o/fJTrDFyKAIUOG0LVrVypXrkzVqlWZM2cOUVFRdO/e3dTRhBBCCJEBZIpC1L59e0JCQhg3bhyBgYG4u7uzZ8+et3a0FkIIIYR4l0xRiAD69ev33k1kH2Jpacn48ePfuRktM5PXLa87K5DXLa87K5DX/fGvW6UoxjhWTQghhBAi48rwJ2YUQgghhPhYUoiEEEIIkeVJIRJCCCFElieFSAghhBBZXpYvRAsWLKBgwYJYWVlRrVo1Tp8+bepIqer7779HpVIl+ilZsqSpYxnd4cOH+fzzz3Fzc0OlUvHrr78melxRFMaNG4erqyvW1tY0bNiQ27dvmyasEX3odXfr1u2t9//TTz81TVgjmjp1KlWqVMHOzo5cuXLRqlUrbt68mWhMbGwsPj4+ODk5kS1bNtq0afPWGe4zmqS8bk9Pz7fe8969e5sosXH4+vpSvnx5w0kIa9Sowe7duw2PZ8b3Gj78ujPje/0u06ZNQ6VSMWjQIMM0Y7znWboQbdiwgSFDhjB+/HjOnz9PhQoVaNKkCcHBwaaOlqrKlCnD8+fPDT9Hjx41dSSji4qKokKFCixYsOCdj8+YMYN58+axaNEiTp06ha2tLU2aNCE2NjaNkxrXh143wKeffpro/V+3bl0aJkwdAQEB+Pj4cPLkSfbu3YtWq6Vx48ZE/eOCxYMHD2bHjh1s2rSJgIAAnj17RuvWrU2Y+uMl5XUD9OzZM9F7PmPGDBMlNo68efMybdo0zp07x9mzZ6lfvz4tW7bk6tWrQOZ8r+HDrxsy33v9b2fOnMHPz4/y5csnmm6U9/yjr4aWgVWtWlXx8fEx3NfpdIqbm5sydepUE6ZKXePHj1cqVKhg6hhpClC2bdtmuK/X65XcuXMrP/74o2FaWFiYYmlpqaxbt84ECVPHv1+3oihK165dlZYtW5okT1oKDg5WACUgIEBRlDfvr7m5ubJp0ybDmOvXryuAcuLECVPFNLp/v25FURQPDw9l4MCBpguVRrJnz64sWbIky7zXf/v7dStK5n+vX79+rRQrVkzZu3dvotdqrPc8y64hio+P59y5czRs2NAwTa1W07BhQ06cOGHCZKnv9u3buLm5UbhwYTp37syjR49MHSlN3b9/n8DAwETvvYODA9WqVcv07z3AoUOHyJUrFyVKlKBPnz68fPnS1JGMLjw8HMBwwcdz586h1WoTveclS5Ykf/78meo9//fr/pu/vz85c+akbNmyjB49mujoaFPESxU6nY7169cTFRVFjRo1ssx7/e/X/bfM/F77+PjQvHnzRO8tGO//70xzpurkevHiBTqd7q3Le7i4uHDjxg0TpUp91apVY8WKFZQoUYLnz58zYcIE6tSpw5UrV7CzszN1vDQRGBgI8M73/u/HMqtPP/2U1q1bU6hQIe7evcuYMWNo2rQpJ06cQKPRmDqeUej1ev6vvfuPibr+4wD+PA4OPQ859E4OLTjAOMEEgZt0uUxDhUYOTIPyNmU5qIRRy3PptK3aynStCWXlVgJZeTaH0QyZhncXsgt/xHWk65LbAbKBZCbq+M29v384PvueckfGycV9Xo+N7fjc+/P+PN+f1xwvP5/PwWuvvYalS5fi0UcfBXCn5iKRCFKp1GWsP9V8rHUDwIYNGxAVFYW5c+fCarXijTfegM1mQ1VVlQ/TTlxzczM0Gg36+/shkUhw7NgxJCQkwGKx+HWt3a0b8N9aA4Ber8cvv/yCc+fO3fOet/5987Yh4qunn36ae52YmIi0tDRERUXh22+/xebNm32YjEyG559/nnu9aNEiJCYmIjY2FkajEenp6T5M5j1FRUX47bff/PLZOE/crbuwsJB7vWjRIkRERCA9PR12ux2xsbGTHdNrVCoVLBYLenp6cPToUWzatAkmk8nXsR44d+tOSEjw21pfuXIFr776Kk6dOoVp06Y9sOPw9paZTCaDUCi85yn0q1evQqFQ+CjV5JNKpYiLi0NLS4uvo0ya0fryvfYAEBMTA5lM5jf1Ly4uxvHjx2EwGPDQQw9x2xUKBQYHB3Hjxg2X8f5Sc3frHktaWhoATPmai0QizJ8/H6mpqdi9ezeSkpJQWlrq97V2t+6x+EutL1y4gO7ubqSkpCAwMBCBgYEwmUwoKytDYGAgwsPDvVJz3jZEIpEIqampqKur47Y5nU7U1dW53I/1d7dv34bdbkdERISvo0ya6OhoKBQKl9rfvHkTjY2NvKo9AHR0dOCvv/6a8vVnjKG4uBjHjh3D6dOnER0d7fJ+amoqgoKCXGpus9nQ3t4+pWs+3rrHYrFYAGDK1/xuTqcTAwMDfltrd0bXPRZ/qXV6ejqam5thsVi4L7VaDa1Wy732Ss29+wz41KLX61lwcDCrqKhgly5dYoWFhUwqlbKuri5fR3tgtm7dyoxGI3M4HKyhoYGtXLmSyWQy1t3d7etoXnXr1i3W1NTEmpqaGAD24YcfsqamJtbW1sYYY+z9999nUqmUVVdXM6vVyrKzs1l0dDTr6+vzcfKJ8bTuW7duMZ1Ox8xmM3M4HOzHH39kKSkp7JFHHmH9/f2+jj4hr7zyCgsNDWVGo5F1dnZyX729vdyYl19+mUVGRrLTp0+z8+fPM41GwzQajQ9TT9x4625paWHvvPMOO3/+PHM4HKy6uprFxMSwZcuW+Tj5xGzfvp2ZTCbmcDiY1Wpl27dvZwKBgJ08eZIx5p+1Zszzuv211u7c/Yk6b9Sc1w0RY4x99NFHLDIykolEIrZkyRL2888/+zrSA5WXl8ciIiKYSCRi8+bNY3l5eaylpcXXsbzOYDAwAPd8bdq0iTF256P3b775JgsPD2fBwcEsPT2d2Ww234b2Ak/r7u3tZatXr2ZyuZwFBQWxqKgoVlBQ4Bf/ARhrzQBYeXk5N6avr49t2bKFhYWFMbFYzNauXcs6Ozt9F9oLxlt3e3s7W7ZsGZs1axYLDg5m8+fPZ9u2bWM9PT2+DT5BL774IouKimIikYjJ5XKWnp7ONUOM+WetGfO8bn+ttTt3N0TeqLmAMcYmcCWLEEIIIWTK4+0zRIQQQggho6ghIoQQQgjvUUNECCGEEN6jhogQQgghvEcNESGEEEJ4jxoiQgghhPAeNUSEEEII4T1qiAghhBDCe9QQEUI4RqMRAoHgnj+SOFnq6uoQHx+PkZERAMBbb72FxYsX+ySLQCDAd99955Nj34/PPvsMa9as8XUMQqY8+k3VhPDU8uXLsXjxYuzbt4/bNjg4iOvXryM8PBwCgWDSM6WmpuL111+HVqsFcOePDw8MDGD27NmTnqWrqwthYWEIDg722pz5+fm4ceOGVxutwcFBREdHQ6/X44knnvDavITwDV0hIoRwRCIRFAqFT5qhM2fOwG63Y926ddw2iUTik2YIABQKhVeboQdFJBJhw4YNKCsr83UUQqY0aogI4aH8/HyYTCaUlpZCIBBAIBCgtbX1nltmFRUVkEqlOH78OFQqFcRiMdavX4/e3l5UVlZCqVQiLCwMJSUl3G0uABgYGIBOp8O8efMwY8YMpKWlwWg0esyk1+uxatUqTJs2jdt29y2z/Px85OTk4IMPPkBERARmz56NoqIiDA0NuZ13dI6DBw8iMjISEokEW7ZswcjICPbu3QuFQoE5c+bg3Xffddnv/2+Ztba2QiAQoKqqCitWrIBYLEZSUhLMZrPbrACwb98+KJVK7v3KykpUV1dz53z0nFy5cgW5ubmQSqWYNWsWsrOz0drays1jNBqxZMkSzJgxA1KpFEuXLkVbWxv3/po1a/D999+jr6/P4zkmhLhHDREhPFRaWgqNRoOCggJ0dnais7MTDz/88Jhje3t7UVZWBr1ej9raWhiNRqxduxY1NTWoqanBoUOHcODAARw9epTbp7i4GGazGXq9HlarFc899xwyMzNx+fJlt5nq6+uhVqvHzW4wGGC322EwGFBZWYmKigpUVFR43Mdut+PEiROora3F4cOH8cUXXyArKwsdHR0wmUzYs2cPdu3ahcbGRo/z7Ny5EzqdDhaLBXFxcXjhhRcwPDw8bmYA0Ol0yM3NRWZmJnfOH3/8cQwNDSEjIwMhISGor69HQ0MDJBIJMjMzMTg4iOHhYeTk5ODJJ5+E1WqF2WxGYWGhy1U8tVqN4eHhcfMTQtwL9HUAQsjkCw0NhUgkglgshkKh8Dh2aGgIn376KWJjYwEA69evx6FDh3D16lVIJBIkJCRgxYoVMBgMyMvLQ3t7O8rLy9He3o65c+cCuNMM1NbWory8HO+9996Yx2lra+PGexIWFoaPP/4YQqEQCxYsQFZWFurq6lBQUOB2H6fTiYMHDyIkJITLa7PZUFNTg4CAAKhUKuzZswcGgwFpaWlu59HpdMjKygIAvP3221i4cCFaWlqwYMGCcXNLJBJMnz4dAwMDLuf8q6++gtPpxOeff841OeXl5ZBKpTAajVCr1ejp6cEzzzzD1SA+Pt5lbrFYjNDQUJerRoSQ+0MNESHEI7FYzP0gBoDw8HAolUpIJBKXbd3d3QCA5uZmjIyMIC4uzmWe8R6O7uvrc7ld5s7ChQshFAq57yMiItDc3OxxH6VSiZCQEJe8QqEQAQEBLttG1+BOYmKiy3EBoLu7+x81RO78+uuvaGlpcckHAP39/bDb7Vi9ejXy8/ORkZGBVatWYeXKlcjNzeWOP2r69Ono7e391zkI4TtqiAghHgUFBbl8LxAIxtzmdDoB3PlkmFAoxIULF1waFwAuTdTdZDIZ/v7773+VZ/TY3lrDP5ln9GrO6D4BAQG4+0O7np5tGnX79m2kpqbi66+/vuc9uVwO4M4Vo5KSEtTW1uLIkSPYtWsXTp06hccee4wbe/36dW48IeT+UUNECE+JRCKXB6G9JTk5GSMjI+ju7r6vj4EnJyfj0qVLXs8zWeRyObq6usAY45oli8XiMmasc56SkoIjR45gzpw5mDlzptv5k5OTkZycjB07dkCj0eCbb77hGiK73Y7+/n4kJyd7d1GE8Ag9VE0ITymVSjQ2NqK1tRXXrl0b9+rIPxUXFwetVouNGzeiqqoKDocDZ8+exe7du/HDDz+43S8jIwNnzpzxSgZfWL58Of7880/s3bsXdrsd+/fvx4kTJ1zGKJVKWK1W2Gw2XLt2DUNDQ9BqtZDJZMjOzkZ9fT0cDgeMRiNKSkrQ0dEBh8OBHTt2wGw2o62tDSdPnsTly5ddniOqr69HTEyMy61NQsj9oYaIEJ7S6XQQCoVISEiAXC5He3u71+YuLy/Hxo0bsXXrVqhUKuTk5ODcuXOIjIx0u49Wq8XFixdhs9m8lmMyxcfH45NPPsH+/fuRlJSEs2fPQqfTuYwpKCiASqWCWq2GXC5HQ0MDxGIxfvrpJ0RGRuLZZ59FfHw8Nm/ejP7+fsycORNisRi///471q1bh7i4OBQWFqKoqAgvvfQSN+/hw4c9PlROCBkf/aZqQsh/xrZt23Dz5k0cOHDA11GmjIsXL+Kpp57CH3/8gdDQUF/HIWTKoitEhJD/jJ07dyIqKsprt+/4oLOzE19++SU1Q4RMEF0hIoQQQgjv0RUiQgghhPAeNUSEEEII4T1qiAghhBDCe9QQEUIIIYT3qCEihBBCCO9RQ0QIIYQQ3qOGiBBCCCG8Rw0RIYQQQniPGiJCCCGE8N7/AF+V/l4QH97dAAAAAElFTkSuQmCC\n" - }, - "metadata": {} - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kpwZw64EYfs6" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "NgGMhK4B51oe" - }, - "source": [ - "### Scalars (Rank 0 Tensors) in Base Python" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ZXnTHDn_EW6b", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "02e2301e-5913-47d0-a31d-aaf344084770" - }, - "source": [ - "x = 25\n", - "x" - ], - "execution_count": 6, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "25" - ] - }, - "metadata": {}, - "execution_count": 6 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "VF8Jam76R4KJ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d5bb3326-7b99-4489-d7dc-386cb76ba951" - }, - "source": [ - "type(x) # if we'd like more specificity (e.g., int16, uint8), we need NumPy or another numeric library" - ], - "execution_count": 7, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "int" - ] - }, - "metadata": {}, - "execution_count": 7 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ZBzYlL0mRd-P" - }, - "source": [ - "y = 3" - ], - "execution_count": 8, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "1i-hW0bcReyy", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ae377616-ba20-4b4b-938b-178a10bf8ff4" - }, - "source": [ - "py_sum = x + y\n", - "py_sum" - ], - "execution_count": 9, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "28" - ] - }, - "metadata": {}, - "execution_count": 9 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "CpyUxB6XRk6y", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "f5fe891a-b958-4b2a-d87d-1e103ca2ca76" - }, - "source": [ - "type(py_sum)" - ], - "execution_count": 10, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "int" - ] - }, - "metadata": {}, - "execution_count": 10 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "V2UiLj-JR8Ij", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "8a1a49bb-88e3-47ba-a1c9-a03136330b6d" - }, - "source": [ - "x_float = 25.0\n", - "float_sum = x_float + y\n", - "float_sum" - ], - "execution_count": 11, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "28.0" - ] - }, - "metadata": {}, - "execution_count": 11 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ikOwjp6ASCaf", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "691b7803-b978-4598-c804-3ef53193f0cd" - }, - "source": [ - "type(float_sum)" - ], - "execution_count": 12, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "float" - ] - }, - "metadata": {}, - "execution_count": 12 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "SgUvioyUz8T2" - }, - "source": [ - "### Scalars in PyTorch\n", - "\n", - "* PyTorch and TensorFlow are the two most popular *automatic differentiation* libraries (a focus of the [*Calculus I*](https://github.com/jonkrohn/ML-foundations/blob/master/notebooks/3-calculus-i.ipynb) and [*Calculus II*](https://github.com/jonkrohn/ML-foundations/blob/master/notebooks/4-calculus-ii.ipynb) subjects in the *ML Foundations* series) in Python, itself the most popular programming language in ML.\n", - "* PyTorch tensors are designed to be pythonic, i.e., to feel and behave like NumPy arrays.\n", - "* The advantage of PyTorch tensors relative to NumPy arrays is that they easily be used for operations on GPU (see [here](https://pytorch.org/tutorials/beginner/examples_tensor/two_layer_net_tensor.html) for example).\n", - "* Documentation on PyTorch tensors, including available data types, is [here](https://pytorch.org/docs/stable/tensors.html)." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "A9Hhazt2zKeD" - }, - "source": [ - "import torch" - ], - "execution_count": 13, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "a211IRW_0-iY", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "18107a49-adb2-45c8-ba6f-1851beeb75af" - }, - "source": [ - "x_pt = torch.tensor(25) # type specification optional, e.g.: dtype=torch.float16\n", - "x_pt" - ], - "execution_count": 14, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(25)" - ] - }, - "metadata": {}, - "execution_count": 14 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "LvxzMa_HhUNB", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "9a3a10fe-7a4f-427c-cfb3-1311fe2ea511" - }, - "source": [ - "x_pt.shape" - ], - "execution_count": 15, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "torch.Size([])" - ] - }, - "metadata": {}, - "execution_count": 15 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "eUyuZXlWS8T9" - }, - "source": [ - "### Scalars in TensorFlow (version 2.0 or later)\n", - "\n", - "Tensors created with a wrapper, all of which [you can read about here](https://www.tensorflow.org/guide/tensor): \n", - "\n", - "* `tf.Variable`\n", - "* `tf.constant`\n", - "* `tf.placeholder`\n", - "* `tf.SparseTensor`\n", - "\n", - "Most widely-used is `tf.Variable`, which we'll use here.\n", - "\n", - "As with TF tensors, in PyTorch we can similarly perform operations, and we can easily convert to and from NumPy arrays.\n", - "\n", - "Also, a full list of tensor data types is available [here](https://www.tensorflow.org/api_docs/python/tf/dtypes/DType)." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "CHBYse_MEqZM" - }, - "source": [ - "import tensorflow as tf" - ], - "execution_count": 16, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "sDv92Nh-NSOU", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "33521159-174d-4b14-89b3-495c8bf205da" - }, - "source": [ - "x_tf = tf.Variable(25, dtype=tf.int16) # dtype is optional\n", - "x_tf" - ], - "execution_count": 17, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 17 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "EmPMBIV9RQjS", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d104c2a4-e776-4b61-82d4-8b23c9ada5f7" - }, - "source": [ - "x_tf.shape" - ], - "execution_count": 18, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "TensorShape([])" - ] - }, - "metadata": {}, - "execution_count": 18 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "mEILtO9pPctO" - }, - "source": [ - "y_tf = tf.Variable(3, dtype=tf.int16)" - ], - "execution_count": 19, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "dvvWuaw6Ph_D", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "928a5ae6-d456-453f-a695-1704f9654b3d" - }, - "source": [ - "x_tf + y_tf" - ], - "execution_count": 20, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 20 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "JZVhRnX9RUGW", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "7170b45f-858e-4497-848a-92bada682572" - }, - "source": [ - "tf_sum = tf.add(x_tf, y_tf)\n", - "tf_sum" - ], - "execution_count": 21, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 21 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "sVbMxT1Ey6Y3", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "63b9e20e-724f-4b10-960b-79635d034d4c" - }, - "source": [ - "tf_sum.numpy() # note that NumPy operations automatically convert tensors to NumPy arrays, and vice versa" - ], - "execution_count": 22, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "28" - ] - }, - "metadata": {}, - "execution_count": 22 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "LXpv69t0y-f6", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d728326e-38d4-48ae-9224-925febcec085" - }, - "source": [ - "type(tf_sum.numpy())" - ], - "execution_count": 23, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "numpy.int16" - ] - }, - "metadata": {}, - "execution_count": 23 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "VszuTUAg1uXk", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "cf5b7af9-3f1c-4ed4-dff1-dc8459d90567" - }, - "source": [ - "tf_float = tf.Variable(25., dtype=tf.float16)\n", - "tf_float" - ], - "execution_count": 24, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 24 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "B5VRGo1H6010" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4CURG9Er6aZI" - }, - "source": [ - "### Vectors (Rank 1 Tensors) in NumPy" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "T9ME4kBr4wg0", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "fe23648e-9fda-4e84-8c45-d05852642b1f" - }, - "source": [ - "x = np.array([25, 2, 5]) # type argument is optional, e.g.: dtype=np.float16\n", - "x" - ], - "execution_count": 25, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 25 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ZuotxmlZL2wp", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "46573b45-8ff1-4d80-a0be-11322fc7a7e6" - }, - "source": [ - "len(x)" - ], - "execution_count": 26, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "3" - ] - }, - "metadata": {}, - "execution_count": 26 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "OlPYy6GOaIVy", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b9e7b1d6-c780-4fad-8737-0e7c7dfc3353" - }, - "source": [ - "x.shape" - ], - "execution_count": 27, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(3,)" - ] - }, - "metadata": {}, - "execution_count": 27 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "sWbYGwObcgtK", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "489b8c9a-0a29-40e2-f294-09216be93fac" - }, - "source": [ - "type(x)" - ], - "execution_count": 28, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "numpy.ndarray" - ] - }, - "metadata": {}, - "execution_count": 28 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ME_xuvD_oTPg", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "36265cc4-43b2-4510-d296-a0b7d6846979" - }, - "source": [ - "x[0] # zero-indexed" - ], - "execution_count": 29, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "25" - ] - }, - "metadata": {}, - "execution_count": 29 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "hXmBHZQ-nxFw", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ddb7f5f0-a04b-437d-fda8-5d9c8503ab2e" - }, - "source": [ - "type(x[0])" - ], - "execution_count": 30, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "numpy.int64" - ] - }, - "metadata": {}, - "execution_count": 30 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "NiEofCzYZBrQ" - }, - "source": [ - "### Vector Transposition" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "hxGFNDx6V95l", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c76eb8db-2df5-41d3-ef83-87df72f7a4d7" - }, - "source": [ - "# Transposing a regular 1-D array has no effect...\n", - "x_t = x.T\n", - "x_t" - ], - "execution_count": 31, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 31 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "_f8E9ExDWw4p", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "024104e6-7d16-49f9-e886-e1c9a08c756f" - }, - "source": [ - "x_t.shape" - ], - "execution_count": 32, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(3,)" - ] - }, - "metadata": {}, - "execution_count": 32 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "AEd8jB7YcgtT", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b5489d91-8f91-4582-e990-0d02ecdd4095" - }, - "source": [ - "# ...but it does we use nested \"matrix-style\" brackets:\n", - "y = np.array([[25, 2, 5]])\n", - "y" - ], - "execution_count": 33, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2, 5]])" - ] - }, - "metadata": {}, - "execution_count": 33 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "UHQd92oRcgtV", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "19065b81-34fa-41ba-b65c-7f65efca1478" - }, - "source": [ - "y.shape" - ], - "execution_count": 34, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(1, 3)" - ] - }, - "metadata": {}, - "execution_count": 34 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "SPi1JqGEXXUc", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "12f118dd-a5bd-491e-9e1f-73f4a265426c" - }, - "source": [ - "# ...but can transpose a matrix with a dimension of length 1, which is mathematically equivalent:\n", - "y_t = y.T\n", - "y_t" - ], - "execution_count": 35, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25],\n", - " [ 2],\n", - " [ 5]])" - ] - }, - "metadata": {}, - "execution_count": 35 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "6rzUv762Yjis", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "39c8296e-f074-4010-dc21-b79325a8ac3a" - }, - "source": [ - "y_t.shape # this is a column vector as it has 3 rows and 1 column" - ], - "execution_count": 36, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(3, 1)" - ] - }, - "metadata": {}, - "execution_count": 36 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "xVnQMLOrYtra", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "2117f2c2-dc95-42c9-af92-507f5b1a97ec" - }, - "source": [ - "# Column vector can be transposed back to original row vector:\n", - "y_t.T" - ], - "execution_count": 37, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2, 5]])" - ] - }, - "metadata": {}, - "execution_count": 37 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "QIAA2NLRZIXC", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "8f77c7c5-7fe1-49ae-d4cd-dc232ef944f4" - }, - "source": [ - "y_t.T.shape" - ], - "execution_count": 38, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(1, 3)" - ] - }, - "metadata": {}, - "execution_count": 38 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Voj26mSpZLuh" - }, - "source": [ - "### Zero Vectors\n", - "\n", - "Have no effect if added to another vector" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-46AbOdkZVn_", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "48e0ffe6-6c0b-40b9-ca08-1a68178da82b" - }, - "source": [ - "z = np.zeros(3)\n", - "z" - ], - "execution_count": 39, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([0., 0., 0.])" - ] - }, - "metadata": {}, - "execution_count": 39 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "c6xyYiwSnSGC" - }, - "source": [ - "### Vectors in PyTorch and TensorFlow" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "s2TGDeqXnitZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "dca2930e-9dc8-4db3-bff1-7a0771b0be76" - }, - "source": [ - "x_pt = torch.tensor([25, 2, 5])\n", - "x_pt" - ], - "execution_count": 40, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 40 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-0jbHgc5iijG", - "scrolled": true, - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "6c7b8c17-b848-44d2-8ba0-da1296a2d0c4" - }, - "source": [ - "x_tf = tf.Variable([25, 2, 5])\n", - "x_tf" - ], - "execution_count": 41, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 41 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "rTDDta1Ro4Pf" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "8fU5qVTI6SLD" - }, - "source": [ - "### $L^2$ Norm" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "lLc2FbGG6SLD", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d2baf173-70e9-4f1f-f773-64803527483d" - }, - "source": [ - "x" - ], - "execution_count": 42, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 42 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "AN43hsl86SLG", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c821c7fd-5aa7-4c77-ccee-feeb90c1722f" - }, - "source": [ - "(25**2 + 2**2 + 5**2)**(1/2)" - ], - "execution_count": 43, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "25.573423705088842" - ] - }, - "metadata": {}, - "execution_count": 43 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "D9CyWo-l6SLI", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "936d1404-2538-4511-9db8-c5b5806bc5e4" - }, - "source": [ - "np.linalg.norm(x)" - ], - "execution_count": 44, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "25.573423705088842" - ] - }, - "metadata": {}, - "execution_count": 44 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "TNEMRi926SLK" - }, - "source": [ - "So, if units in this 3-dimensional vector space are meters, then the vector $x$ has a length of 25.6m" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "ugQC6k4h6SLK" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "PwiRlMuC6SLK" - }, - "source": [ - "### $L^1$ Norm" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "lcYKyc5H6SLL", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "905fe7dc-6b8e-4e8b-a65d-839c3564e1c5" - }, - "source": [ - "x" - ], - "execution_count": 45, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 45 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "8jNb6nYl6SLM", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "629afdb9-02c7-4f3b-ea4d-210269cffb00" - }, - "source": [ - "np.abs(25) + np.abs(2) + np.abs(5)" - ], - "execution_count": 46, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "32" - ] - }, - "metadata": {}, - "execution_count": 46 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WTPz0EBSAVee" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "lQP73B916SLP" - }, - "source": [ - "### Squared $L^2$ Norm" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Qv1ouJ8r6SLP", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "6cb6eebf-e88a-4b41-aa9f-ac82a78f99e6" - }, - "source": [ - "x" - ], - "execution_count": 47, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 47 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "eG3WSB5R6SLT", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b1971235-6180-4f41-f4fb-d539583f5748" - }, - "source": [ - "(25**2 + 2**2 + 5**2)" - ], - "execution_count": 48, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "654" - ] - }, - "metadata": {}, - "execution_count": 48 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "bXwzSudS6SLV", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "264e1226-1066-4b64-db16-0070ba65b245" - }, - "source": [ - "# we'll cover tensor multiplication more soon but to prove point quickly:\n", - "np.dot(x, x)" - ], - "execution_count": 49, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "654" - ] - }, - "metadata": {}, - "execution_count": 49 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "q3CIH9ba6SLX" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "BHWxVPFC6SLX" - }, - "source": [ - "### Max Norm" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "vO-zfvDG6SLX", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "e3c17461-b374-47ed-e95b-f7ef61f1bfca" - }, - "source": [ - "x" - ], - "execution_count": 50, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 50 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "vXXLgbyW6SLZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "eae3e34d-ae6f-46f0-bb0b-a518792c2cb7" - }, - "source": [ - "np.max([np.abs(25), np.abs(2), np.abs(5)])" - ], - "execution_count": 51, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "25" - ] - }, - "metadata": {}, - "execution_count": 51 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "3MVTsXA8nNR0" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "JzKlIpYZcgt9" - }, - "source": [ - "### Orthogonal Vectors" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "4jHg9La-cgt9", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ece7c34c-c185-44bf-c5cd-218ab3e80c87" - }, - "source": [ - "i = np.array([1, 0])\n", - "i" - ], - "execution_count": 52, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([1, 0])" - ] - }, - "metadata": {}, - "execution_count": 52 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "3FyLhPK3cguA", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "580fea73-ce7a-406e-c9b5-70ebe1f26a79" - }, - "source": [ - "j = np.array([0, 1])\n", - "j" - ], - "execution_count": 53, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([0, 1])" - ] - }, - "metadata": {}, - "execution_count": 53 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "7eQtKhaDcguC", - "scrolled": false, - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "4422976d-b8de-4526-88ba-be0e7aa43754" - }, - "source": [ - "np.dot(i, j) # detail on the dot operation coming up..." - ], - "execution_count": 54, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "0" - ] - }, - "metadata": {}, - "execution_count": 54 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "C6eMVPu4nNR7" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "mK3AZH53o8Br" - }, - "source": [ - "### Matrices (Rank 2 Tensors) in NumPy" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "stk57cmaESW1", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3c147c9a-7d8a-4b1b-aebd-9656a7c1f368" - }, - "source": [ - "# Use array() with nested brackets:\n", - "X = np.array([[25, 2], [5, 26], [3, 7]])\n", - "X" - ], - "execution_count": 55, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2],\n", - " [ 5, 26],\n", - " [ 3, 7]])" - ] - }, - "metadata": {}, - "execution_count": 55 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "IhDL4L8S6SLc", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "599b1ac9-bae5-4bd0-ca42-2e437b22dcbf" - }, - "source": [ - "X.shape" - ], - "execution_count": 56, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "(3, 2)" - ] - }, - "metadata": {}, - "execution_count": 56 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "q3oyaAK36SLe", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b12026d0-5614-4261-cdaa-15e06d33289f" - }, - "source": [ - "X.size" - ], - "execution_count": 57, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "6" - ] - }, - "metadata": {}, - "execution_count": 57 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "YN9CHzja6SLg", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "fd31c5a0-75ea-4274-9d5e-43e07464edbe" - }, - "source": [ - "# Select left column of matrix X (zero-indexed)\n", - "X[:,0]" - ], - "execution_count": 58, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 5, 3])" - ] - }, - "metadata": {}, - "execution_count": 58 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ih7nh4qC6SLi", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "30cd3a8b-0daa-4137-e6d6-0464eb263353" - }, - "source": [ - "# Select middle row of matrix X:\n", - "X[1,:]" - ], - "execution_count": 59, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([ 5, 26])" - ] - }, - "metadata": {}, - "execution_count": 59 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "pg7numxP6SLl", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "83d7da30-ec2a-448a-f3d0-c6a87736b5df" - }, - "source": [ - "# Another slicing-by-index example:\n", - "X[0:2, 0:2]" - ], - "execution_count": 60, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2],\n", - " [ 5, 26]])" - ] - }, - "metadata": {}, - "execution_count": 60 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "HGEfZiBb6SLt" - }, - "source": [ - "### Matrices in PyTorch" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-bibT9ye6SLt", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c0b079b1-6fa6-48c5-85cd-2713dc27d8d6" - }, - "source": [ - "X_pt = torch.tensor([[25, 2], [5, 26], [3, 7]])\n", - "X_pt" - ], - "execution_count": 61, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[25, 2],\n", - " [ 5, 26],\n", - " [ 3, 7]])" - ] - }, - "metadata": {}, - "execution_count": 61 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "TBPu1L7P6SLv", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "7f0dcc6a-237e-409e-f6c2-e2bb267208be" - }, - "source": [ - "X_pt.shape # pythonic relative to TensorFlow" - ], - "execution_count": 62, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "torch.Size([3, 2])" - ] - }, - "metadata": {}, - "execution_count": 62 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "4mTj56M16SLw", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "1b03c6e8-10ce-40d5-97e0-357936b1c510" - }, - "source": [ - "X_pt[1,:] # N.B.: Python is zero-indexed; written algebra is one-indexed" - ], - "execution_count": 63, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([ 5, 26])" - ] - }, - "metadata": {}, - "execution_count": 63 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "E026fQlD6SLn" - }, - "source": [ - "### Matrices in TensorFlow" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "1gtGH6oA6SLn", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c0ce4bd3-5e6d-48f4-d2f6-6a89fb2be5fa" - }, - "source": [ - "X_tf = tf.Variable([[25, 2], [5, 26], [3, 7]])\n", - "X_tf" - ], - "execution_count": 64, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 64 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "4CV_KiTP6SLp", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "aa1c1260-ad22-47df-ade8-acb92b37a406" - }, - "source": [ - "tf.rank(X_tf)" - ], - "execution_count": 65, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 65 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "vUsce8tC6SLq", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "6a67de12-22c8-45be-f757-2abc7a07e56e" - }, - "source": [ - "tf.shape(X_tf)" - ], - "execution_count": 66, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 66 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "QNpfvNPj6SLr", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "af5fc194-5874-4894-dfc9-0d8a11087bcf" - }, - "source": [ - "X_tf[1,:]" - ], - "execution_count": 67, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 67 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "CodS4evY6SLy" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "cMpfujF_6SLy" - }, - "source": [ - "### Higher-Rank Tensors\n", - "\n", - "As an example, rank 4 tensors are common for images, where each dimension corresponds to:\n", - "\n", - "1. Number of images in training batch, e.g., 32\n", - "2. Image height in pixels, e.g., 28 for [MNIST digits](http://yann.lecun.com/exdb/mnist/)\n", - "3. Image width in pixels, e.g., 28\n", - "4. Number of color channels, e.g., 3 for full-color images (RGB)" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "KSZlICRR6SL1" - }, - "source": [ - "images_pt = torch.zeros([32, 28, 28, 3])" - ], - "execution_count": 68, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "6Dqj0vmh6SL2" - }, - "source": [ - "# images_pt" - ], - "execution_count": 69, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "7TASTVD96SLy" - }, - "source": [ - "images_tf = tf.zeros([32, 28, 28, 3])" - ], - "execution_count": 70, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "ftOliyru6SL0" - }, - "source": [ - "# images_tf" - ], - "execution_count": 71, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "O3sgkdXZ6SL3" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "lmG3LEZK6SL4" - }, - "source": [ - "## Segment 2: Common Tensor Operations" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "iSHGMCxd6SL4" - }, - "source": [ - "### Tensor Transposition" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "1YN1narR6SL4", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "9d2f2223-2e97-4066-e774-b7b2990a4387" - }, - "source": [ - "X" - ], - "execution_count": 72, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2],\n", - " [ 5, 26],\n", - " [ 3, 7]])" - ] - }, - "metadata": {}, - "execution_count": 72 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "5hf3M_NL6SL5", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "a1e37e11-2dd8-47aa-e6fc-d8f41fc9c94f" - }, - "source": [ - "X.T" - ], - "execution_count": 73, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 5, 3],\n", - " [ 2, 26, 7]])" - ] - }, - "metadata": {}, - "execution_count": 73 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "vyBFN_4g6SL9", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "18d89c80-72b3-4d8f-a343-5319478b103b" - }, - "source": [ - "X_pt.T" - ], - "execution_count": 74, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[25, 5, 3],\n", - " [ 2, 26, 7]])" - ] - }, - "metadata": {}, - "execution_count": 74 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "K2DuDJc_6SL6", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "fe0b51cb-89c0-47fd-eeef-c99ef54e8b72" - }, - "source": [ - "tf.transpose(X_tf) # less Pythonic" - ], - "execution_count": 75, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 75 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Hp9P1jx76SL_" - }, - "source": [ - "### Basic Arithmetical Properties" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WxaImEUc6SMA" - }, - "source": [ - "Adding or multiplying with scalar applies operation to all elements and tensor shape is retained:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "yhXGETii6SMA", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3b84b86c-40e4-4a9d-ba98-29400df1e3b4" - }, - "source": [ - "X*2" - ], - "execution_count": 76, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[50, 4],\n", - " [10, 52],\n", - " [ 6, 14]])" - ] - }, - "metadata": {}, - "execution_count": 76 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "KnPULtDO6SMC", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "7799494b-fdd7-40a9-a25d-6fae2c3dafba" - }, - "source": [ - "X+2" - ], - "execution_count": 77, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[27, 4],\n", - " [ 7, 28],\n", - " [ 5, 9]])" - ] - }, - "metadata": {}, - "execution_count": 77 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "MkfC0Gsb6SMD", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "729eef68-a48e-45d9-e91f-e07e976c7dc2" - }, - "source": [ - "X*2+2" - ], - "execution_count": 78, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[52, 6],\n", - " [12, 54],\n", - " [ 8, 16]])" - ] - }, - "metadata": {}, - "execution_count": 78 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "04bIDpGj6SMH", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "4a021c90-7e73-4939-eac7-e22f7a52135e" - }, - "source": [ - "X_pt*2+2 # Python operators are overloaded; could alternatively use torch.mul() or torch.add()" - ], - "execution_count": 79, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[52, 6],\n", - " [12, 54],\n", - " [ 8, 16]])" - ] - }, - "metadata": {}, - "execution_count": 79 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "2oRBSmRL6SMI", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "5626b6c5-f89b-400f-fb19-2b550056065f" - }, - "source": [ - "torch.add(torch.mul(X_pt, 2), 2)" - ], - "execution_count": 80, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[52, 6],\n", - " [12, 54],\n", - " [ 8, 16]])" - ] - }, - "metadata": {}, - "execution_count": 80 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "OMSb9Otd6SMF", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ffac929e-7e5c-44a6-f792-9add7acc3ddc" - }, - "source": [ - "X_tf*2+2 # Operators likewise overloaded; could equally use tf.multiply() tf.add()" - ], - "execution_count": 81, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 81 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "5ya2xZ4u6SMG", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "2b473b86-0a90-414d-9e5b-d19752d64cc9" - }, - "source": [ - "tf.add(tf.multiply(X_tf, 2), 2)" - ], - "execution_count": 82, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 82 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "wt8Ls4076SMK" - }, - "source": [ - "If two tensors have the same size, operations are often by default applied element-wise. This is **not matrix multiplication**, which we'll cover later, but is rather called the **Hadamard product** or simply the **element-wise product**.\n", - "\n", - "The mathematical notation is $A \\odot X$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "KUMyU1t46SMK", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "1ccfbb86-871e-4a8a-cc4c-d5f332418d94" - }, - "source": [ - "X" - ], - "execution_count": 83, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2],\n", - " [ 5, 26],\n", - " [ 3, 7]])" - ] - }, - "metadata": {}, - "execution_count": 83 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "UNIbp0P36SML", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ac9a00b6-fc5b-46ed-b58f-a194efe0a9e2" - }, - "source": [ - "A = X+2\n", - "A" - ], - "execution_count": 84, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[27, 4],\n", - " [ 7, 28],\n", - " [ 5, 9]])" - ] - }, - "metadata": {}, - "execution_count": 84 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "HE9xPWPdcgu4", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "0e7fb59b-eabb-4099-9997-6a0884aa1fde" - }, - "source": [ - "A + X" - ], - "execution_count": 85, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[52, 6],\n", - " [12, 54],\n", - " [ 8, 16]])" - ] - }, - "metadata": {}, - "execution_count": 85 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "xKyCwGia6SMP", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b26a192f-6edf-4c4e-d7e3-1010ab17a565" - }, - "source": [ - "A * X" - ], - "execution_count": 86, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[675, 8],\n", - " [ 35, 728],\n", - " [ 15, 63]])" - ] - }, - "metadata": {}, - "execution_count": 86 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "B5jXGIBp6SMT" - }, - "source": [ - "A_pt = X_pt + 2" - ], - "execution_count": 87, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "A7k6yxu36SMU", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "a5057174-9c44-4234-8db6-0cc296cc67dd" - }, - "source": [ - "A_pt + X_pt" - ], - "execution_count": 88, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[52, 6],\n", - " [12, 54],\n", - " [ 8, 16]])" - ] - }, - "metadata": {}, - "execution_count": 88 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "r8vOul0m6SMW", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "828619be-624e-4924-aa95-c9e314f48fe8" - }, - "source": [ - "A_pt * X_pt" - ], - "execution_count": 89, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[675, 8],\n", - " [ 35, 728],\n", - " [ 15, 63]])" - ] - }, - "metadata": {}, - "execution_count": 89 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "rQcBMSb76SMQ" - }, - "source": [ - "A_tf = X_tf + 2" - ], - "execution_count": 90, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "x6s1wtNj6SMR", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "8904d6bb-ba2f-4a8d-eed3-fa97ab5f3b48" - }, - "source": [ - "A_tf + X_tf" - ], - "execution_count": 91, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 91 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "J1D7--296SMS", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "da919529-d302-4f0f-c4df-3379722d20c2" - }, - "source": [ - "A_tf * X_tf" - ], - "execution_count": 92, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 92 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "FE5f-FEq6SMY" - }, - "source": [ - "### Reduction" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WPJ9FVQF6SMY" - }, - "source": [ - "Calculating the sum across all elements of a tensor is a common operation. For example:\n", - "\n", - "* For vector ***x*** of length *n*, we calculate $\\sum_{i=1}^{n} x_i$\n", - "* For matrix ***X*** with *m* by *n* dimensions, we calculate $\\sum_{i=1}^{m} \\sum_{j=1}^{n} X_{i,j}$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "rXi2stvz6SMZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "7ea95647-da4e-4293-c668-13d3bad8e1d0" - }, - "source": [ - "X" - ], - "execution_count": 93, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[25, 2],\n", - " [ 5, 26],\n", - " [ 3, 7]])" - ] - }, - "metadata": {}, - "execution_count": 93 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "W9FKaJbf6SMZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "01308bf7-b99e-44cd-cc93-76d504affaf7" - }, - "source": [ - "X.sum()" - ], - "execution_count": 94, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "68" - ] - }, - "metadata": {}, - "execution_count": 94 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "3y9aw7t66SMc", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "234c1149-b78d-4d2d-9724-b25434e20760" - }, - "source": [ - "torch.sum(X_pt)" - ], - "execution_count": 95, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(68)" - ] - }, - "metadata": {}, - "execution_count": 95 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "wcjRtFml6SMb", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c1b1f84d-83b0-4c45-ccd2-b312f6851305" - }, - "source": [ - "tf.reduce_sum(X_tf)" - ], - "execution_count": 96, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 96 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "awjH9bOz6SMc", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "12d3fb04-b017-413c-cb8a-72c9795b9fa4" - }, - "source": [ - "# Can also be done along one specific axis alone, e.g.:\n", - "X.sum(axis=0) # summing over all rows (i.e., along columns)" - ], - "execution_count": 97, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([33, 35])" - ] - }, - "metadata": {}, - "execution_count": 97 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "n2SASjsn6SMd", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ae990ba3-eca0-4ed6-dcbf-bc929e7f0c20" - }, - "source": [ - "X.sum(axis=1) # summing over all columns (i.e., along rows)" - ], - "execution_count": 98, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([27, 31, 10])" - ] - }, - "metadata": {}, - "execution_count": 98 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "uVnSxvSJ6SMh", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "396683c4-e825-4287-b5d2-e97a8cef5a9f" - }, - "source": [ - "torch.sum(X_pt, 0)" - ], - "execution_count": 99, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([33, 35])" - ] - }, - "metadata": {}, - "execution_count": 99 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "IO8drxz36SMe", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "0a0f18a6-d220-4ecb-94a1-68f1978920f7" - }, - "source": [ - "tf.reduce_sum(X_tf, 1)" - ], - "execution_count": 100, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 100 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gdAe8S4A6SMj" - }, - "source": [ - "Many other operations can be applied with reduction along all or a selection of axes, e.g.:\n", - "\n", - "* maximum\n", - "* minimum\n", - "* mean\n", - "* product\n", - "\n", - "They're fairly straightforward and used less often than summation, so you're welcome to look them up in library docs if you ever need them." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "r2eW8S_46SMj" - }, - "source": [ - "### The Dot Product" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "LImETgD76SMj" - }, - "source": [ - "If we have two vectors (say, ***x*** and ***y***) with the same length *n*, we can calculate the dot product between them. This is annotated several different ways, including the following:\n", - "\n", - "* $x \\cdot y$\n", - "* $x^Ty$\n", - "* $\\langle x,y \\rangle$\n", - "\n", - "Regardless which notation you use (I prefer the first), the calculation is the same; we calculate products in an element-wise fashion and then sum reductively across the products to a scalar value. That is, $x \\cdot y = \\sum_{i=1}^{n} x_i y_i$\n", - "\n", - "The dot product is ubiquitous in deep learning: It is performed at every artificial neuron in a deep neural network, which may be made up of millions (or orders of magnitude more) of these neurons." - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "HveIE3IDcgvP", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "7e47ff8a-3d70-4480-b585-b0cac5bea84e" - }, - "source": [ - "x" - ], - "execution_count": 101, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 101 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "3ZjkZcvVcgvQ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3d4f0992-086a-4789-912d-6899b31dab37" - }, - "source": [ - "y = np.array([0, 1, 2])\n", - "y" - ], - "execution_count": 102, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([0, 1, 2])" - ] - }, - "metadata": {}, - "execution_count": 102 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Xu8z0QB0cgvR", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "684539d6-0ebb-47f3-d2f4-6ec21a14e1f8" - }, - "source": [ - "25*0 + 2*1 + 5*2" - ], - "execution_count": 103, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "12" - ] - }, - "metadata": {}, - "execution_count": 103 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ThehRrr8cgvS", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "23b6fb0b-f5c8-414a-c09d-1552a44a5591" - }, - "source": [ - "np.dot(x, y)" - ], - "execution_count": 104, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "12" - ] - }, - "metadata": {}, - "execution_count": 104 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "J5Zdua4xcgvT", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d59bc178-eccd-46f2-d889-32b266eee56f" - }, - "source": [ - "x_pt" - ], - "execution_count": 105, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 105 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "b3vEdroXcgvU", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "63ef19a6-2bbf-4336-d216-2b941658e9fe" - }, - "source": [ - "y_pt = torch.tensor([0, 1, 2])\n", - "y_pt" - ], - "execution_count": 106, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([0, 1, 2])" - ] - }, - "metadata": {}, - "execution_count": 106 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "F741E5imcgvV", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "754c34c3-548f-4674-8cbd-925cb0d78a6d" - }, - "source": [ - "np.dot(x_pt, y_pt)" - ], - "execution_count": 107, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "12" - ] - }, - "metadata": {}, - "execution_count": 107 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-W5loHc8cgvX", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "18e71dbb-163f-4291-8090-366a21c35642" - }, - "source": [ - "torch.dot(torch.tensor([25, 2, 5.]), torch.tensor([0, 1, 2.]))" - ], - "execution_count": 108, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(12.)" - ] - }, - "metadata": {}, - "execution_count": 108 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "jUwKBiqzcgvY", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "710897b4-cdc5-48f8-ba45-a536a3d1c686" - }, - "source": [ - "x_tf" - ], - "execution_count": 109, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 109 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Xqt3Rac7cgvZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "50214346-8c20-42aa-d25d-1eb711f35330" - }, - "source": [ - "y_tf = tf.Variable([0, 1, 2])\n", - "y_tf" - ], - "execution_count": 110, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 110 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "x4pgc5JEcgvc", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "a3e872f5-c429-4e0f-ea1a-7751d9e289d5" - }, - "source": [ - "tf.reduce_sum(tf.multiply(x_tf, y_tf))" - ], - "execution_count": 111, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 111 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "mSmvC1cc6SMj" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "9dfTV7hQR6zn" - }, - "source": [ - "### Solving Linear Systems" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "pCoSpqtWSyFd" - }, - "source": [ - "In the **Substitution** example, the two equations in the system are:\n", - "$$ y = 3x $$\n", - "$$ -5x + 2y = 2 $$\n", - "\n", - "The second equation can be rearranged to isolate $y$:\n", - "$$ 2y = 2 + 5x $$\n", - "$$ y = \\frac{2 + 5x}{2} = 1 + \\frac{5x}{2} $$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "1T0_m11kTh82" - }, - "source": [ - "x = np.linspace(-10, 10, 1000) # start, finish, n points" - ], - "execution_count": 112, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "_K6IyUWcTh85" - }, - "source": [ - "y1 = 3 * x" - ], - "execution_count": 113, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "DYkQdUJ5Th86" - }, - "source": [ - "y2 = 1 + (5*x)/2" - ], - "execution_count": 114, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 455 - }, - "id": "aBL9NDoHTh86", - "outputId": "dada3255-d947-4809-95e9-dc2485cb7a34" - }, - "source": [ - "fig, ax = plt.subplots()\n", - "plt.xlabel('x')\n", - "plt.ylabel('y')\n", - "ax.set_xlim([0, 3])\n", - "ax.set_ylim([0, 8])\n", - "ax.plot(x, y1, c='green')\n", - "ax.plot(x, y2, c='brown')\n", - "plt.axvline(x=2, color='purple', linestyle='--')\n", - "_ = plt.axhline(y=6, color='purple', linestyle='--')" - ], - "execution_count": 115, - "outputs": [ - { - "output_type": "display_data", - "data": { - "text/plain": [ - "
" - ], - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAG2CAYAAACH2XdzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABoPUlEQVR4nO3dd3QU5eLG8e+WdEhooUR6770FCxYUFUJvwQKo2OAKIkoCKCAlwYLgFUUsgIXQW1BEUUGUXqX33mtC+mZ3fn94b+4PKRJIMpvN8zkn5+wu7848jMvkcfadGYthGAYiIiIiuZzV7AAiIiIiWUGlRkRERDyCSo2IiIh4BJUaERER8QgqNSIiIuIRVGpERETEI6jUiIiIiEdQqRERERGPoFIjIiIiHkGlRkRERDyCqaXG6XTy5ptvUq5cOfz8/KhQoQIjR45Ed24QERGRzLKbufKxY8fyySefMG3aNGrUqMGGDRvo1asXQUFBvPLKK2ZGExERkVzGYuYNLVu3bk2xYsX44osvMl7r2LEjfn5+fPPNN2bFEhERkVzI1CM1zZo1Y/Lkyezdu5fKlSuzdetWfv/9d8aNG3fd8ampqaSmpmY8d7lcXLx4kcKFC2OxWHIqtoiIiNwBwzC4cuUKISEhWK1ZOBPGMJHT6TQGDRpkWCwWw263GxaLxRgzZswNxw8bNswA9KMf/ehHP/rRjwf8HDt2LEt7halfP82YMYPXX3+dd999lxo1arBlyxb69+/PuHHj6NGjxzXj/36kJi4ujtKlS3Ps2DECAwNzMrqISK6QlpjG+yHvA/DaydfwDvA2OZHczE8HfqLngp4kpCVQrmA5ZnaaSZUiVe5omWlxcWx6912O/vgjAIVq1KDx8OEElimTFZFvS3x8PKVKleLy5csEBQVl2XJN/frp9ddfJyIigm7dugFQq1Ytjhw5QlRU1HVLjY+PDz4+Pte8HhgYqFIjInIdabY0fPEF/tpXqtS4J8Mw+HDthwz4cQAuq4vmVZozt8tcCvsXvqPlnl6zhjVDhpB0+jQB3t7UfPFFavTujdXLK4uS35msnjpiaqlJSkq65rs0m82Gy+UyKZGIiEjOcjgd9P2+L5M3TQbg2XrP8nGrj/G23X4BdaamsmX8ePZ89RUA+UqXptnYsRSpXTtLMrsrU0tNWFgYo0ePpnTp0tSoUYPNmzczbtw4nnnmGTNjiYiI5IiLyRfpPLszvxz6BQsW3n34XQaEDrijIxiXdu9mVUQEcfv2AVCxSxfqv/46dn//rIrttkwtNf/+97958803efnllzl79iwhISG88MILvPXWW2bGEhHxGFa7lTo96mQ8Fvex98JeWk9vzb6L+8jnnY+YjjG0rtz6tpfncjrZPW0af374IS6HA9/ChWny9tvcdf/9WRfazZk6UfhOxcfHExQURFxcnObUiIhIrvHzwZ/pNLsTl1MuUzqoNLHhsdQudvtfDSWePMnqwYM5u349AHc98ABNRozAt/CdzcnJLtn1+9vUIzUiIiJ5zacbPqXP931wGk5CS4Yyv+t8iuUrdlvLMgyDw7GxbBg9GkdCAnY/PxpERlK+Q4c8ef02lRoREQ9mGAaOJAcAXv5eefIXnbtId6Xz2tLX+HDdhwA8UesJPm/zOb5239taXurly6x/+22OLl0KQJG6dQmNiiJ/6dJZljm3UakREfFgjiQHUfmiAIhMiNQp3SaJS4mj29xu/LD/BwBGPziayHsib7tknlq1ijVDhpB89iwWu51aL71E9eeew2rP27/W8/bfXkREJJsdvHSQsJgwdp7biZ/dj6/bf03H6h1va1npKSls+eAD9v7n/oiB5coRGh1N4Zo1szJyrqVSIyIikk1WHllJ+5ntuZB8gZD8IcSGx1K/RP3bWtbFXbtYPWgQcQcOAFApPJx6r72G3c8vKyPnaio1IiIi2WDqlqk8H/s8DpeDBiUasCh8ESH5QzK9HJfTya4vv2TbRx/hSk/Ht0gRmo4aRci992ZD6txNpUZERCQLOV1OBv88mHdWvQNAp+qdmNZuGv5emb/4XcLx46yOjOTcpk0AlGrRgkbDh+NbsGCWZvYUKjUiIiJZJCEtgSfmPcGiPYsAePO+Nxl+/3Cslsxd+NAwDA4tWMCGqCjSExOxBwTQMDKScu3a6Qy2m1CpERERyQJH447SJqYNW89sxcfmw5dtv6R7re6ZXk7KpUusHz6cY8uWARBcvz6hUVHkK1kyqyN7HJUaEREPZrVZqd6pesZjyR5rjq+h3Yx2nEk8Q7GAYizotoCmJZtmejknV65kzdChpJw/j9Vup1bfvlR75hmsNls2pPY8KjUiIh7M7mun8+zOZsfwaDHbYui1sBepzlRqF6tNbHgspYMydwG89ORkNr/3HvtmzAAgsHx5mr3zDoWqVcuOyB5LpUZEROQ2uAwXI5aP4O3f3gagTZU2fNvhW/J558vUci5s387qiAjiDx0CoPKTT1L31Vex+97elYbzMpUaERGRTEpyJNFzQU9m75wNwOvNXifqoShs1lv/msiVns7Ozz9n2yefYKSn41e0KE1Hj6ZEs2bZFdvjqdSIiHiwtMQ03SYhi528cpK2M9qy4eQGvKxefNr6U3rV65WpZVw5epTVkZGc37IFgNKPPkqjN9/Ep0CBrA+ch6jUiIiI3KJNpzbRJqYNJ66coLBfYeZ1ncd9Ze675fcbhsGBuXPZFB1NenIyXvny0XDoUMq2bq1TtbOASo2IiMgtmLdrHk/Oe5Lk9GSqFanG4u6LKV+w/C2/P+XCBdYNH87xX34BoGijRoSOGUNASOavMizXp1IjIiJyE4ZhEPV7FEN+GQJAywotmdlpJkG+Qbe8jBMrVrD2zTdJuXABq91O7X79qNqjh07VzmIqNSIiIjeQkp5C79jefPPnX3fFfqXxK7zf8n3s1lv79ZmelMSmd99l/6xZAARVqkSz6GgKVq2abZnzMpUaERGR6zibeJb2M9uz6tgqbBYbHz3+ES82fPGW33/+zz9ZNWgQCUePAlC1Rw/q9OuHzccnuyLneSo1IiIif7PtzDbCYsI4EneEAr4FmN15Ni3Kt7il97ocDrZPnsyOTz/FcDrxL16cpqNHU7xp5q8wLJmjUiMi4sGsNiuVHq+U8Vj+2eK9iwmfG05CWgKVClUiNjyWKkWq3NJ7448cYfWgQVzYtg2AMo8/TqOhQ/EOuvX5N3L7VGpERDyY3ddO9+8yf1PFvMgwDD5Y8wEDfxyIgcEDZR9gTpc5FPIrdEvv3T97NpveeQdncjJegYE0GjqUsq1a5UBy+S+VGhERyfPSnGn0+a4Pn2/+HIDe9Xsz8fGJeNm8/vG9yefPs/attzi5YgUAxZo0oeno0QSUKJGtmeVaKjUiIpKnXUi6QMdZHVlxZAVWi5X3H3mffk363dLF8I7/8gtr33qL1EuXsHp7U7d/f6o89RQWq77qM4NKjYiIB0tLTOO9ou8BMPDsQN0m4W92n99N6+mtOXDpAPm98zOj0wwer/T4P77PkZjIprFjOTB3LgAFKlem2TvvUKBSpeyOLDehUiMi4uEcSQ6zI7ilHw/8SJfZXYhLjaNsgbIsDl9MjaI1/vF957ZsYXVEBAnHjoHFQrWePan9yivYvFUYzaZSIyIiec7EdRPp90M/nIaTu0vdzfyu8wkOCL7pe1wOB9s++YSdn32G4XLhX6IEoVFRFGvUKIdSyz9RqRERkTwj3ZVO/x/6M3H9RACervM0k1tPxsd+8wvixR08yOqICC7u2AFA2TZtaDh4MN7582d7Zrl1KjUiIpInXE65TJfZXfjp4E9YsBD1UBRv3P3GTScEG4bBvpgYNr//Ps6UFLwDA2k8fDilW7bMweRyq1RqRETE4+2/uJ+wmDB2n9+Nv5c/33b4lnZV2930PcnnzrFm6FBO/f47AMWbNaPpqFH4FyuWA4nldqjUiIiIR1txeAUdZnXgYvJFSgaWZFG3RdQrUe+m7zn200+sGz6c1MuXsfn4UPe116gcHq5Ttd2cSo2IiAezWC2UaV4m43Fe88WmL3jxuxdJd6XT+K7GLOi6gBL5b3xRPEdCAhvGjOHQwoUAFKxWjWbR0QRVrJhTkeUOqNSIiHgwLz8vei7vaXaMHOd0ORm0bBDvr34fgK41ujKl7RT8vPxu+J6zGzeyOjKSxBMnsFitVHv2WWq9/LJO1c5FVGpERMSjXEm9Qvd53Vm8dzEAw5sP563mb91wQrAzLY1tEyey84svwDAIuOsuQqOiKNqgQU7GliygUiMiIh7jyOUjhMWEse3sNnztvkxtO5WuNbvecHzc/v2siojg0q5dAJRv354GERF45cuXU5ElC5k646ls2bJYLJZrfvr06WNmLBERj5GWmMa7we/ybvC7pCWmmR0nW606torGnzdm29ltFM9XnBU9V9yw0BguF3u++YYfunTh0q5d+BQowL3jx9N01CgVmlzM1CM169evx+l0Zjzfvn07Dz/8MJ07dzYxlYiIZ0k6n2R2hGz3zZ/f8OyiZ0lzplG3eF1iw2MpGVjyumOTzpxhzdChnF61CoAS99xD01Gj8Au++RWFxf2ZWmqC//YBio6OpkKFCjRv3tykRCIikpu4DBdv/vImY34fA0D7qu35uv3XBHgHXHf8kR9+YP2IEaTFx2Pz9aXewIFU6tbtlu7ILe7PbebUpKWl8c033zBgwIAbfrhSU1NJTU3NeB4fH59T8URExM0kpiXy9IKnmbdrHgCR90Qy6sFRWC3XzqxIu3KFDaNHczg2FoBCNWrQbOxYAsuVy9HMkr3cptQsWLCAy5cv07NnzxuOiYqKYsSIETkXSkRE3NKJ+BO0mdGGTac24W3z5rOwz3i6ztPXHXtm/XpWR0aSdOoUFquVGs8/T80XX8Tq5ZXDqSW7uU2p+eKLL3jssccICQm54ZjIyEgGDBiQ8Tw+Pp5SpUrlRDwREXETG05uoE1MG04lnCLYP5j5Xedzd+m7rxnnTEvjzw8/ZNfUqWAY5CtVitDoaILr1s3xzJIz3KLUHDlyhGXLljFv3rybjvPx8cHH5+Z3UhUREc81e8dsnl7wNCnpKdQIrsHi7ospW6DsNeMu793LqkGDuLx3LwAVOnak/qBBeAVcf66NeAa3KDVTpkyhaNGitGrVyuwoIiIexWK1ENIwJONxbmUYBqN+G8Vby98C4PFKjxPTMYZAn8Crx7lc7P7qK7aOH4/L4cCnUCGajBhByQcfNCO25DDTS43L5WLKlCn06NEDu930OCIiHsXLz4ve63ubHeOOJDuSeXbRs8RsjwHg1aav8u7D72Kz2q4al3jqFGsGD+bMunUAhDRvTpO338avSJEczyzmML1FLFu2jKNHj/LMM8+YHUVERNzM6YTTtJvRjrUn1mK32vn48Y/p3eDaknZ48WLWjxqF48oVbH5+NHjjDSp07qxTtfMY00vNI488gmEYZscQERE3s/X0VsJiwjgWf4yCvgWZ22UuD5R74KoxaXFxrB85kiNLlgBQuHZtQqOjCSxTxozIYjLTS42IiGQfR5KDidUnAtBnZx+8/HPHacyL9iyi+9zuJDoSqVy4MovDF1OpcKWrxpxes4Y1Q4aQdPo0FpuNmi++SI3nn8eqqQx5lv7Li4h4MMMwiDsSl/HY3RmGwbur3iViWQQGBi3Kt2BWp1kU9CuYMcaZmsqW8ePZ89VXAOQvU4bQ6GiK1K5tVmxxEyo1IiLiFlLTU3nxuxeZumUqAC81fIkJj07Ay/a/o0uXdu9m1aBBxO3fD0DFrl2pP3Agdn9/MyKLm1GpERER051POk+HmR1YeXQlVouVCY9OoG/jvhl/7nI62T11Kn9++CGu9HR8CxemyciR3KV7Bcr/o1IjIiKm2nluJ62nt+bQ5UME+gQyq9MsWlZsmfHniSdPsjoykrMbNgBQ8sEHaTxiBL6FCpkVWdyUSo2IiJjmh/0/0HVOV+JT4ylfsDyx4bFUD64O/DW/5nBsLBtGj8aRkIDdz48GkZGU79BBp2rLdanUiIhIjjMMg3+v+zevLn0Vl+HivjL3MbfLXIr4/3WhvNTLl1n/9tscXboUgCJ16xIaFUX+0qXNjC1uTqVGRMSDWSwWgqsHZzx2Bw6ng1eWvMKkjZMA6FW3F5NaT8Lb5g3AqT/+YM2QISSfO4fFbqfWyy9T/dlndaq2/CN9QkREPJiXvxcv73jZ7BgZLiVfovPszvx86GcsWHjn4Xd4LfQ1LBYL6SkpbBk3jr3ffgtAYLlyhEZHU7hmTZNTS26hUiMiIjli34V9tI5pzd4LewnwCmB6x+m0qdIGgIs7d7Jq0CDiDx4EoFJ4OPVeew27n5+ZkSWXUakREZFs98uhX+g0qxOXUi5ROqg0seGx1C5WG5fTya4vv+TPjz7CSE/Ht0gRmo4aRci995odWXIhlRoREQ/mSHLwWaPPAOi9vrcpt0mYvHEyfb7vQ7ornaYlm7Kg6wKK5StGwvHjrI6I4NzmzQCUatGCRsOH41uw4D8sUeT6VGpERDyYYRic23ku43FOcrqcvPbja0xYOwGA7rW680WbL/Cx+XBg/nw2jhlDelIS9oAAGg4eTLm2bd1mMrPkTio1IiKS5eJT4+k2pxtL9v919+xRD4xi8L2DSb18mZXDB3F82TIAguvXJzQqinwlS5oZVzyESo2IiGSpQ5cO0TqmNTvP7cTP7sdX7b+iU/VOnFy5kjVDh5Jy/jxWu51afftS7ZlnsNpsZkcWD6FSIyIiWeb3o7/TfmZ7ziedJyR/CAu7LaRuwRqsHzmSfTNmABBUoQKhY8dSqFo1k9OKp1GpERGRLDFtyzR6x/bG4XLQoEQDFnZbiO+Riyzp3Ykrhw8DUOXJJ6nz6qvYfX3NDSseSaVGRETuiMtwMfjnwYz9YywAHat1ZGrYlxya+i3bJ03CSE/Hr2hRmo4eTYlmzUxOK55MpUZExINZLBaCygRlPM5qCWkJPDX/KRbsXgDA0HuHMrDCc/zx7Itc2LoVgNKPPkqjN9/Ep0CBLF+/yP+nUiMi4sG8/L3of7h/tiz7WNwxwmLC2HpmKz42H74I+5wm+3xZ+kZH0pOT8cqfn4ZDh1K2VSudqi05QqVGREQybd2JdbSd0ZbTCacpGlCUuS2/wvlpLOt+/RWAoo0aETpmDAEhISYnlbxEpUZERDJlxvYZ9FrYi5T0FGoVrcW00kM51Gc0KRcuYPXyok6/flTt0QOL1Wp2VMljVGpERDyYI9nB1PumAtDzt554+d3+bRIMw2D48uG8/dvbALQr+ziv7a/Lro9GABBUqRLNxo6lYJUqd5xb5Hao1IiIeDDDZXByw8mMx7cr2ZFMz4U9mbVjFgDDQp6j/ozjHD26AICqPXpQp18/bD4+d5xZ5Hap1IiIyE2dunKKtjPasv7kenzx4lPH03j9ey0JTif+xYvTdPRoijdtanZMEZUaERG5sc2nNhMWE8aJKyeokhbMyJ0NcexdjQGUadWKRkOH4h0YaHZMEUClRkREbmD+rvk8Of9JktKS6H6mKmHr/HCkHMErMJBGb75J2ccfNzuiyFVUakRE5CqGYRD9ezSDfxlMULKN6J31KXUgBRepFGvShNAxY/AvXtzsmCLXUKkREZEMqempPL/4eb7a+hUNTuTnX1vK45WYgtXbm7qvvkqVJ5/UqdritlRqREQ8nH8R/1sadzbxLB1mdmDjwdU8vyWE5ocKAA4KVKlCs7FjKVCpUrbmFLlTKjUiIh7MO8Cb18+9/o/jtp/dTlhMGF77zxC9riLBCXawWKjWqxe1//UvbN7eOZBW5M6o1IiI5HHf7f2O7rO70XKzH212l8VqWAgICSE0KoqiDRuaHU/klqnUiIjkUYZhMH7NeN6fO5jX15Sg/CU/AMq2aUPDwYPxzp/f5IQimaNSIyLiwRzJDr597FsAnljyRMZtEtKcafT9rg+HZ81l1J9l8XZa8Q4KovGwYZRu2dLMyCK3TaVGRMSDGS6DIyuOZDwGuJB0gae/bE/N+Ue4/3QJAIo3a0bT0aPxL1rUtKwid8r08/JOnDjBk08+SeHChfHz86NWrVps2LDB7FgiIh5p9/nd9BoSSpsvzlDndD7wttNg8GAe+PRTFRrJ9Uw9UnPp0iXuvvtuHnjgAZYsWUJwcDD79u2jYMGCZsYSEfFIP+9Ywvcj+9HtYAAAvpXK8tD7HxJUoYLJyUSyhqmlZuzYsZQqVYopU6ZkvFauXDkTE4mIeK49L0YQmhqAywLlezxBk34Ddaq2eBRTS82iRYto2bIlnTt3ZsWKFdx11128/PLL9O7dO1PLSUtMI82Wds3rVpsVu6/9qnE3YrFaMibQZXasI8mBYRjXH2ux4OV/m2OTHRnfgV+Pd4D3bY1NT0nH5XRlyVgvfy8sFstfY1PTcaVn0Vg/LyzWv8Y605w4Hc4sGWv3tWO1WTM/1uHEmXaTsT52rPbMj3Wlu0hPTb/hWJu3DZuXLfNjnS7SU24y1suGzTvzYw2XgSPZkSVjrXYrdp+//n0ahoEjKYvGZuLffV7YR6SmJmc8L5ToS0IhG4+MnUBIg0Y4HeB0pGWM/S/tI7SPyIl9RHawGDf615MDfH19ARgwYACdO3dm/fr19OvXj0mTJtGjR49rxqemppKamprxPD4+nlKlShFBBL74XjO+0uOV6P5d94znYwLG3HBnWKZ5GXou75nx/N3gd0k6n3TdsSENQ+i9/n/Fa3zZ8cQdibvu2ODqwby84+WM5x/X+JhzO89dd2xQmSD6H+6f8fyzRp9xcsPJ6471L+J/1QW1pt4/NWMy4N95+XsxOHFwxvPpraaz7/t91x0LMMwYlvF4dufZ7Jyz84ZjIxMiM3ZwC3ouYOu0rTccO/DsQAKC/zrs/V2f79jw8Y3nTvU71I8CZQsA8OPrP7L6vdU3HPvS9pcoWuOvuQDLhy9nxYgVNxz73LrnuKvRXQD88e4fLHtj2Q3H9vi1B2XvLwvAuonrWNJ3yQ3Hhi8Op3KrygBsmbqFhb0W3nBsp1mdqNG5BgA7Zu9gTpc5Nxzbdkpb6vasC8De7/YS0zrmhmMf++gxGvdpDMDh5YeZ9sC0G45t8U4L7n79bgBOrD/B540/v+HY5sOac//w+wE4u+Msn9T85IZjQweG8si7jwBw+fBlJpSbcMOxDV9uSKuJrQBIPJfIe0Xfu+HYOj3q0G5qO+CvMhGVL+qGY6t3qk7n2Z0zno+wjLjhWE/fR3Rb/hDzX+7Npd/aXHfMf2kf8T/aR/wlu/cR8fHxBAUFERcXR2AW3uXd1CM1LpeLhg0bMmbMGADq1avH9u3bb1hqoqKiGDHixjsoERH5iyMlmV/Dn6SQw85lazqGYQfT/hdWJGeYeqSmTJkyPPzww3z++f8a4CeffMKoUaM4ceLENeNvdKTm3Mlz1216eeHQ8u2M1aFlHVrW10+ZH5tb9hFJZ87yU2R/Uv/ci9XqZHdJB2HjPqVu+Xu0j7iFsdpH/GdsNu8jPPJIzd13382ePXuuem3v3r2UKVPmuuN9fHzw8fG55nXvAO+r/pHdyK2MuZ2x/38nk6Vj/bJn7P/fiWfpWB87XPuf547H2rxv/TvYbBvr9b+dQVaOtdqteNtv7bOWqbE26y1/hjMz1mK1ZM9YS/aMhez7d++O+4gjS5bw+7ChWBJTcHi5+OM+X94e8xMhgSG3vEzQPuK2xmofAWTu3312MLXUvPrqqzRr1owxY8bQpUsX1q1bx+TJk5k8ebKZsUREcpW0+HjWjxrFke++wwIcKJjM4Sdr89GzMfh5+ZkdTyTHmPr1E8DixYuJjIxk3759lCtXjgEDBtzy2U/ZdfhKRCS3OLNuHX9ERpBy+gwui8GCauep+eKLDHtwBBaLhfSUdGZ1nAVAl7ldMnVkRSS7eOTXTwCtW7emdevWZscQEclVnGlp/Pnhh+yaOhUMg9P50vii2TnefHYS3Wp2yxjncroyzmS62dwXEU9geqkREZHMubx3L6sGDeLy3r0A/FL+Ej82czH7qaU0KdnE5HQi5lGpERHJJQyXi91ffcXW8eNxORzE+zj5rOEJnPUr8Ed4LKWCSpkdUcRUKjUiIrlA4smTrBkyhDPr1gGwqcQVPmt0kgfrhvF1+6/J553P5IQi5lOpERFxc4cXL2b9qFE4rlwh3cvC1Non+LX8ZQbdM4gxD43BarGaHVHELajUiIi4qbS4ONaPHMmRJX9dfv9UcQvv1tvLxSCYGjaVHnWvvfK6SF6mUiMi4oZOr1nD6sGDST5zBqxWltZN5JvyhyiUrwi/dJ3PPaXvMTuiiNtRqRERcSPO1FS2fPABe77++q8XShRmdK0/2VkgjhrBNYgNj6VcwXK3vDzvAO+rbkAp4slUakRE3MSlXbtYFRFB3P79AMTdV5FXi8SSajd4rOJjzOg0g0AfXWhU5EZUakRETOZyOtk9dSp/fvghrvR0fAoXZuXjQUxIXQRA/yb9efeRd7FbtcsWuRn9CxERMVHCiROsGTyYsxs2ABDc/G7GVNvK8ouLsVvtfPTYR7zQ8IXbXn56Sjrzn5oPQPuv2+s2CeLR9OkWETGBYRgcWrSIDaNHk56YiN3fn2J9nuSpi+9z7OIxCvoWZE6XOTxY7sE7Wo/L6WLnnJ0AtJ3aNiuii7gtlRoRkRyWevky60aM4NiPPwJQpG5dkp9/mFZ/9CXRkUjlwpWJDY+lcuHKJicVyV1UakREctDJ339n7dChJJ87h8Vup9bLL/NDjTje+PkZDAweKvcQszvPpqBfQbOjiuQ6KjUiIjkgPSWFLePGsffbbwEILF+eRmNGEnnwQ6b8PAWAFxu8yIePfYiXzcvMqCK5lkqNiEg2u7hzJ6sGDSL+4EEAKnfvTqmXetBl0RP8duQ3rBYr41uOp2/jvlgsFpPTiuReKjUiItnE5XSy64sv+HPiRIz0dPyCg2kyahSXqxSi2TfNOXjpIIE+gczsNJNHKz5qdlyRXE+lRkQkGyQcO8bqyEjObd4MQKlHHqHxsGEsP7+WLl+0Ij41nnIFyrG4+2KqB1c3Oa2IZ1CpERHJQoZhcHDBAjaOGUN6UhL2gAAaDhlC2bAwJq6fSP+l/XEZLu4tfS/zus6jiH+RbM3j5e9FZEJkxmMRT6ZSIyKSRVIuXmTdiBEcX7YMgOAGDQiNisKneFH6LunLJxs+AaBn3Z5MajUJH7tPtmeyWCx4B3hn+3pE3IFKjYhIFjjx22+sHTqUlAsXsNrt1P7Xv6jaqxdxafG0n/44yw4uw4KFsS3GMrDZQE0IFskGKjUiIncgPSmJze+9x76ZMwEIqlCB0LFjKVStGvsu7CMsJow9F/YQ4BXA9I7TaVOlTc7mS01n8QuLAWj9aWvsPtrti+fSp1tE5DZd2LaNVRERXDl8GIAqTz1F3Vdfxebjw6+HfqXjrI5cSrlEqcBSxIbHUqd4nRzP6Ep3sXXaVgAen/g4ZP83XiKmUakREckkV3o6OyZPZvukSRhOJ37FihE6ejTFQ0MB+GzjZ7z8/cuku9JpclcTFnRbQPF8xU1OLeL5VGpERDLhypEjrIqM5MLWv45+lH7sMRoNHYpPgQI4XU5e/+l1PljzAQDhNcP5su2X+Np9zYwskmeo1IiI3ALDMDgwZw6bxo4lPTkZr/z5aTR0KGVatcJisRCfGk/43HC+3/c9AG/f/zZD7xuqCcEiOUilRkTkH6RcuMDaYcM48euvABRt1IjQMWMICAkB4NClQ4TFhLHj3A787H5MazeNzjU6mxlZJE9SqRERuYnjv/7K2rfeIvXiRaxeXtTp35+qTz+NxWoF4I+jf9BuZjvOJ52nRL4SLApfRMOQhianFsmbVGpERK7DkZjIpnfe4cCcOQAUqFyZ0OhoClapkjHm661f81zsc6Q506hfoj6Lui3irsC7zIoskuep1IiI/M35rVtZFRFBwtGjYLFQtUcP6rzyCjafv86Hdhkuhv4ylKjfowDoUK0DX7X7igDvADNjX5eXvxcDzw7MeCziyVRqRET+w+VwsP3TT9kxeTKG04l/8eKEjhlDsSZNMsYkpiXy1PynmL97PgCD7xnMyAdHYrVYzYp9UxaLhYBg9ytbItlBpUZEBIg/fJhVgwZxcft2AMq2bk3DIUPwDgzMGHM8/jhhMWFsOb0Fb5s3X7T5gidrP2lWZBH5G5UaEcnTDMNg/8yZbHr3XZwpKXgFBtL4rbco89hjV41bd2IdbWe05XTCaYoGFGVB1wWElgo1KfWtS09NZ+mApQC0HNdSt0kQj6ZPt4jkWcnnzrH2rbc4+dtvABRr2pTQ0aPxL3711X9nbp9Jz4U9SUlPoWbRmiwOX0yZAmXMiJxprnQXGz7eAMDD7zys2ySIR1OpEZE86djPP7Nu2DBSL13C6u1N3QEDqPLEExmnasNfR3HeXvE2w1cMB6B15dZM7zCd/D75TUotIjejUiMieYojMZGNUVEcnP/XRN+CVasSOnYsBSpWvGpcsiOZXgt7MXPHX3fffi30Nca2GIvNasvxzCJya1RqRCTPOLdpE6siI0k8fhwsFqo/8wy1+vbF5u191bhTV07RbmY71p1Yh91qZ1KrSTxb/1mTUovIrTL1HMThw4djsViu+qlataqZkUTEAznT0tg6YQLLevQg8fhxAkJCaDF1KnUHDLim0Gw+tZnGnzdm3Yl1FPIrxLKnlqnQiOQSph+pqVGjBsuWLct4brebHklEPEjcgQOsiojg0s6dAJRr25YGkZF45792XsyC3Qt4Yt4TJDmSqFqkKrHhsVQsVPGacSLinkxvEHa7neJ/O9NAROROGYbB3unT2fL++zhTU/EOCqLx8OGUfuSR64595493iPw5EgODRyo8wsxOMyngWyDng4vIbTO91Ozbt4+QkBB8fX0JDQ0lKiqK0qVLX3dsamoqqampGc/j4+NzKqaI5CJJZ8+yZsgQTq9aBUCJu++myahR+Bctes3Y1PRUnl/8PF9t/QqAPo36MP7R8ditpu8es4SXnxf9DvXLeCziySyGYRhmrXzJkiUkJCRQpUoVTp06xYgRIzhx4gTbt28n/3UODQ8fPpwRI0Zc83pcXByB/++qnyKSdx1dupR1I0aQFheHzceHuq+9RuXu3bFYLNeMPZd4jvYz2/PHsT+wWWxMeHQCfRr3MSG1SN4SHx9PUFBQlv/+NrXU/N3ly5cpU6YM48aN49lnr52Yd70jNaVKlVKpERHSrlxh45gxHFq0CICC1avTLDqaoAoVrjt++9nthMWEcfjyYYJ8gpjVeRaPVLj2qykRyXrZVWrc6vhqgQIFqFy5Mvv377/un/v4+ODjo8thisjVzm7YwOrISBJPnsRitVL9ueeo+dJL15zZ9F/f7/uebnO6cSXtChUKVmBx98VULeKZZ14605z8PORnAB4a/RA2b11nRzyXW91WNiEhgQMHDlCiRAmzo4hILuBMS2Pz+++zrGdPEk+eJKBkSVpMm0adfv2uW2gMw2D8mvGExYRxJe0Kzcs0Z+1zaz220AA4HU5Wv7ea1e+txulwmh1HJFuZeqRm4MCBhIWFUaZMGU6ePMmwYcOw2WyEh4ebGUtEcoHL+/ez6o03uLxnDwDlO3SgQUQEXgEB1x3vcDro+31fJm+aDMBz9Z5jYquJeNuufzRHRHIfU0vN8ePHCQ8P58KFCwQHB3PPPfewZs0agoODzYwlIm7McLnY8803bPngA1xpafgULEjjESMo9dBDN3zPxeSLdJrViV8P/4oFC+898h6vNn31upOHRST3MrXUzJgxw8zVi0guk3T6NKuHDOHMmjUAhNx3H03efhu/m/yP0J7zewiLCWPfxX3k885HTMcYWldunVORRSQHudVEYRGRGzmyZAnr3n4bR3w8Nl9f6r/xBhW7dLnp0ZZlB5fReXZnLqdcpkxQGWLDY6lVrFYOphaRnKRSIyJuLS0+nvWjRnHku+8AKFSzJs3GjiWwbNmbvm/Shkn0/b4vTsNJaMlQFnRbQNGAay++JyKeQ6VGRNzWmbVrWT14MEmnT2Ox2ajx/PPUfOEFrF43vjJuuiud15a+xofrPgTgydpP8lnYZ/jafXMqtoiYRKVGRNzOf++qvXvaNDAM8pUqRbOxYylSp85N3xeXEkfXOV1ZemApAKMfHE3kPZF5ekKwl58XL21/KeOxiCdTqRERt3Jpzx5WR0Rwee9eACp27ky911+/4ana/3Xw0kFaT2/NrvO78Pfy5+v2X9OhWoeciOzWLFYLRWvoazfJG1RqRMQtGC4Xu6dNY+uECbgcDnwKFaLJ229T8oEH/vG9vx35jQ4zO3Ah+QJ35b+LReGLqF+ifg6kFhF3olIjIqZLPHmS1YMHc3b9egDuuv9+mrz9Nr6FC//je6dsnsILi1/A4XLQMKQhC7stJCR/SHZHzjWcaU5WjlkJwL2D79VtEsSjqdSIiGkMw+Dwd9+xYdQoHFeuYPfzo/6gQVTo1Okf58E4XU4if47k3VXvAtC5ememtpuKv5d/TkTPNZwOJytGrACg2evNVGrEo6nUiIgp0uLiWDdyJEeXLAGgcJ06NIuKIn+ZMv/43oS0BJ6Y9wSL9vx1R+637nuLYfcPw2pxq9vZiUgOU6kRkRx3evVqVg8ZQvKZM1hsNmq+9BI1evfGav/nXdLRuKOExYTx55k/8bH5MKXtFMJr6X5xIqJSIyI5KD0lha0ffMCeb74BIH/ZsjSLjqZwrVu7yu+a42toN6MdZxLPUCygGAu7LaRJySbZGVlEchGVGhHJEZd27WLVoEHEHTgAQKWuXak3cCB2/1ubAzN923SeWfgMqc5U6hSrw6LwRZQOKp2dkUUkl1GpEZFs5XI62T1lCn/++9+40tPxLVKEpqNGEXLvvbf2fsPF8OXDGfnbSADaVGnDtx2+JZ93vuyMLSK5kEqNiGSbhBMnWB0ZybmNGwEo2aIFjYcPx7dgwVt6f5IjiZ4LejJ752wA3mj2BlEtojQhWESuS6VGRLKcYRgcWriQDWPGkJ6YiN3fnwaDB1O+XbtbvmXBySsnaTujLRtObsDL6sXksMn0rNsze4N7ILuvnefWPZfxWMST6RMuIlkq9fJl1g0fzrGffgIguF49QqOiyFeq1C0vY+PJjbSZ0YaTV05S2K8w87vO594yt/Z1lVzNarNyV6O7zI4hkiNUakQky5z8/XfWDh1K8rlzWOx2avfpQ7Vnn8Vqu/ULvs3dOZen5j9Fcnoy1YOrExseS/mC5bMxtYh4CpUaEblj6cnJbBk3jr3TpwMQWL48zcaOpVD16re8DMMwGLNyDEN/HQrAoxUfZUbHGQT5BmVL5rzCmeZkzYQ1ADTt11RXFBaPplIjInfk4o4drIqIIP7gQQAqP/EEdQcMwO7re8vLSElP4blFz/Httm8B6NekH+898h52q3ZRd8rpcLLsjWUANHq5kUqNeDTtMUTktricTnZ+/jnbPv4YIz0dv+BgmowaRcg992RqOWcSztB+ZntWH1+NzWJj4uMTeaHhC9mUWkQ8mUqNiGRawrFjrIqI4PyWLQCUeuQRGg8bhk+BAplazrYz22gd05qjcUcp4FuAOZ3n8FD5h7I+sIjkCSo1InLLDMPg4Pz5bIyKIj0pCa98+Wg4ZAhlw8Ju+VTt/1q8dzHhc8NJSEugUqFKLO6+mMqFK2dTchHJC1RqROSWpFy8yLrhwzn+888AFG3YkNCoKAJCQjK1HMMwGLd6HK//9DoGBg+We5DZnWdTyK9QdsQWkTxEpUZE/tGJ335j7dChpFy4gNVup/Yrr1C1Z89MnaoNkOZM4+XvXuaLzV8A8Hz95/no8Y/wsnllR2wRyWNUakTkhtKTktj03nvsnzkTgKCKFWkWHU3BatUyvawLSRfoOKsjK46swGqxMu6RcbzS5JVMf20lInIjKjUicl3n//yT1RERXDlyBIAqTz1F3Vdfxebjk+ll7T6/m9bTW3Pg0gHye+dnZqeZPFbpsayOLNdh97XT49ceGY9FPJk+4SJyFVd6OjsmT2b7pEkYTid+xYoROmYMxZs2va3l/XjgR7rM7kJcahzlCpQjNjyWGkVrZHFquRGrzUrZ+8uaHUMkR6jUiEiG+CNHWB0RwYU//wSgzGOP0ejNN/EOur2r+n607iP6/9Afp+HkntL3MK/LPIIDgrMysohIBpUaEcEwDA7Mns3Gd97BmZyMV/78NBo6lLKtW9/W8tJd6fRb0o+PN3wMQI86Pfi09af42DP/1ZXcGafDycbJGwFo8HwDbF66orB4LpUakTwu+fx51g0bxonlywEo1rgxTUePzvSp2v91OeUyXWZ34aeDP2HBQnSLaF5v9romBJvEmeZkSd8lANTtWVelRjyaSo1IHnb8119Z+9ZbpF68iNXLizr9+1P16aexWK23tbz9F/fTenpr9lzYg7+XP992+JZ2VdtlbWgRkRtQqRHJgxyJiWx65x0OzJkDQIHKlWk2diwFKt/+FX2XH15Ox1kduZh8kZKBJYkNj6Vu8bpZlFhE5J+p1IjkMee2bGF1RAQJx46BxUK1nj2p/cor2Ly9b3uZn2/6nJe+e4l0VzqN72rMwm4LKZ6veBamFhH5Zyo1InmEy+Fg+6RJ7Jg8GcPlwr94cUKjoijWuPFtL9PpcvLGT28wbs04ALrV7MaXbb7Ez8svq2KLiNwylRqRPCD+0CFWRURwcft2AMq2bk3DIUPwDgy8/WWmxtN9bne+2/cdACPuH8Gb972pCcEiYprbmw2YDaKjo7FYLPTv39/sKCIewzAM9s2YwZJOnbi4fTvegYHc/d57NBs79o4KzeHLh7n7y7v5bt93+Np9mdlpJm81f0uFRkRMlekjNT169ODZZ5/lvvvuy7IQ69ev59NPP6V27dpZtkyRvC753DnWvPkmp1auBKB4aChNR4/Gv1ixO1ruqmOraDejHeeSzlEiXwkWdltIo7saZUVkyQZ2Hzvhi8MzHot4skwfqYmLi6NFixZUqlSJMWPGcOLEiTsKkJCQwBNPPMFnn31GwYIF72hZIvKXY8uW8X27dpxauRKrtzf1IyJ4YPLkOy40X2/9mgemPcC5pHPUK16Pdb3XqdC4OavdSuVWlancqjJWu9scnBfJFpn+hC9YsIATJ07w0ksvMXPmTMqWLctjjz3GnDlzcDgcmQ7Qp08fWrVqRYsWLf5xbGpqKvHx8Vf9iMj/OBITWTN0KCv79SP18mUKVq3Ko7NnU/Wpp2772jMALsPF4J8H8/SCp0lzptG+antW9lpJycCSWZheROTO3NZeLjg4mAEDBrB161bWrl1LxYoVeeqppwgJCeHVV19l3759t7ScGTNmsGnTJqKiom5pfFRUFEFBQRk/pUqVup34Ih7p3KZNfN+hAwfnzweLherPPccjMTEUqFjxjpabmJZI59mdifr9r3+ng+8ZzJwucwjwDsiK2JLNnA4nW6ZuYcvULTgdTrPjiGSrOzoWeerUKX766Sd++uknbDYbjz/+ONu2baN69ep88MEHN33vsWPH6NevH99++y2+vr63tL7IyEji4uIyfo4dO3Yn8UU8gjMtja0TJrCsRw8Sjx8nICSEFtOmUffVV+/o2jMAx+OPc++Ue5m3ax7eNm++avcVox8ajdWirzFyC2eak4W9FrKw10KcaSo14tkyPWvM4XCwaNEipkyZwo8//kjt2rXp378/3bt3J/A/Z1PMnz+fZ555hldfffWGy9m4cSNnz56lfv36Ga85nU5+++03PvroI1JTU7HZrr5HiY+PDz4+uiGeyH/FHTjAqogILu3cCUC5tm1pOHgwXvny3fGy159YT9sZbTmVcIpg/2Dmd53P3aXvvuPliohkl0yXmhIlSuByuQgPD2fdunXUrVv3mjEPPPAABQoUuOlyHnroIbZt23bVa7169aJq1aoMGjTomkIjIv9juFzsjYlhy/vv40xNxTsoiMbDh1P6kUeyZPmzdsyix4IepKSnULNoTWLDYylboGyWLFtEJLtkutR88MEHdO7c+aZfGRUoUIBDhw7ddDn58+enZs2aV70WEBBA4cKFr3ldRP4n6exZ1gwZwulVqwAocc89NB01Cr/g4DtetmEYjPxtJMOWDwOgVaVWTO84nUCf27+mjYhITsl0qXnqqaeyI4eI3IKjS5eybsQI0uLisPn4UG/gQCqFh2fJRe+SHck8u+hZYrbHADCg6QDeefgdbFYdNRWR3MGtrsS0fPlysyOIuKW0K1fYMGYMhxctAqBQjRqERkcTVL58liz/dMJp2s1ox9oTa7Fb7XzS6hOeq/9clixbRCSnuFWpEZFrnd2wgdWRkSSePInFaqX6c89R86WX7vjMpv/aenorYTFhHIs/RiG/QsztMpf7y96fJcsWEclJKjUibsqZlsaf//43u6ZMAcMgX6lShEZFEVyvXpatY+HuhTwx7wkSHYlUKVyFxd0XU7HQnV3XRtyL3cdOp1mdMh6LeDJ9wkXc0OX9+1n1xhtc3rMHgAodO1J/0CC8ArLmgneGYfDuqneJWBaBgUGL8i2Y1WkWBf10qxJPY7VbqdG5htkxRHKESo2IGzFcLvZ88w1bPvgAV1oaPgUL0njECEo99FCWrSM1PZUXv3uRqVumAvByw5cZ/+h4vGxeWbYOEREzqNSIuImk06dZPWQIZ9asASCkeXOajBiRJadq/9e5xHN0nNWRlUdXYrPYmPDoBPo07pNlyxf340p3sWv+LgCqta+mm1qKR1OpEXEDh7//nvUjR+KIj8fm50f9N96gYufOWXKq9n/tOLuDsJgwDl0+RKBPILM7z+aRCllzsT5xX+mp6czpMgeAyIRIvO1ZM8FcxB2p1IiYKC0+nvWjRnHku+8AKFyrFqHR0QSWLZul61mybwnd5nYjPjWe8gXLszh8MdWCq2XpOkREzKZSI2KS02vWsGbIEJJOn8Zis1HjhReo+fzzWL2ybm6LYRh8uPZDBvw4AJfh4r4y9zGvyzwK+xfOsnWIiLgLlRqRHOZMTWXrhAnsnjYNgHylS9Ns7FiK1K6dpetxOB38a8m/+HTjpwA8U/cZPmn9Cd42ff0gIp5JpUYkB13as4fVERFc3rsXgIpdulD/9dex+/tn6XouJl+k8+zO/HLoFyxYePfhdxkQOiBL5+iIiLgblRqRHGC4XOyeNo2tEybgcjjwLVyYxiNGUPKBB7J8XXsv7KX19Nbsu7iPfN75mN5hOmFVwrJ8PSIi7kalRiSbJZ48yerBgzm7fj0Adz3wAE1GjMC3cNbPa/nl0C90mtWJSymXKB1UmtjwWGoXy9qvtURE3JVKjUg2MQyDw4sXs2HUKBwJCdj9/KgfEUGFjh2z5WugTzd8St8lfUl3pRNaMpT5XedTLF+xLF+P5C42bxttp7TNeCziyVRqRLJB6uXLrB85kqM//ABA4Tp1aBYVRf4yZbJ8XemudAb+OJAJaycA8EStJ/i8zef42n2zfF2S+9i8bNTtWdfsGCI5QqVGJIudXr2a1UOGkHzmDBabjVovv0z1557Das/6f25xKXGEzw1nyf4lAIx6YBSD7x2sCcEikiep1IhkkfSUFLZ+8AF7vvkGgMBy5QiNiqJwrVrZsr6Dlw4SFhPGznM78bP78XX7r+lYvWO2rEtyL1e6i/1L9wNQsWVF3SZBPJpKjUgWuLhrF6sHDSLuwAEAKnXrRr2BA7H7+WXL+lYeWUmHWR04n3SekPwhLOq2iAYhDbJlXZK7paemE9M6BtBtEsTzqdSI3AGX08muL79k20cf4UpPx7dIEZqOGkXIvfdm2zqnbpnK87HP43A5aFCiAYvCFxGSPyTb1icikluo1IjcpoQTJ1gdGcm5jRsBKNWiBY2GD8e3YMFsWZ/LcBG5LJJ3Vr0DQKfqnZjWbhr+Xll74T4RkdxKpUYkkwzD4NDChWwYM4b0xETsAQE0jIykXLt22TZBNyEtgSfnPcnCPQsBePO+Nxl+/3CsFs2PEBH5L5UakUxIuXSJ9SNGcOynnwAIrleP0Oho8pUsmW3rPBp3lDYxbdh6Zis+Nh++bPsl3Wt1z7b1iYjkVio1Irfo5MqVrBk6lJTz57HY7dTu25dqzzyD1ZZ9FzRbe3wtbWe05UziGYoGFGVB1wWElgrNtvWJiORmKjUi/yA9OZnN77/Pvpi/ziAJLF+eZmPHUqh69Wxdb8y2GHot7EWqM5XaxWqzqNsiyhTI+ov3iYh4CpUakZu4uGMHqwYNIv7QIQAqP/kkdV99Fbtv9l2t12W4GLF8BG//9jYAYZXD+LbDt+T3yZ9t6xTPZfO28dhHj2U8FvFkKjUi1+FKT2fnF1+w7eOPMdLT8StalKajRlHi7ruzdb1JjiR6LezFrB2zAHi92etEPRSFzapfRnJ7bF42GvdpbHYMkRyhUiPyN1eOHmV1ZCTnt2wBoHTLljR66y18ChTI1vWevHKSdjPasf7kerysXnza+lN61euVresUEfEkKjUi/2EYBgfnzWNjdDTpSUl45ctHwyFDKBsWlu33Utp0ahNtYtpw4soJCvsVZl7XedxX5r5sXafkDS6ni6MrjwJQ+t7SWG26DIB4LpUaESDl4kXWDR/O8Z9/BqBoo0aEjhlDQEj2X6l33q55PDX/KZIcSVQrUo3Y8FgqFKqQ7euVvCE9JZ1pD0wD/nObhADdJkE8l0qN5HknVqxg7ZtvknLhAla7ndr9+lG1R49sPVUb/joyFP17NIN/GQxAywotmdlpJkG+Qdm6XhERT6VSI3lWelISm957j/0zZwIQVLEizcaOpWDVqtm+7tT0VHrH9ubrP78G4F+N/8W4luOwW/VPUkTkdmkPKnnS+T//ZHVEBFeOHAGgao8e1OnXD5uPT7av+2ziWdrPbM+qY6uwWWz8+7F/81Kjl7J9vSIink6lRvIUV3o6OyZPZvukSRhOJ/7Fi9N09GiKN22aI+vfdmYbYTFhHIk7QgHfAszuPJsW5VvkyLpFRDydSo3kGfFHjrB60CAubNsGQJnHH6fR0KF4B+XMHJbv9n5Ht7ndSEhLoFKhSsSGx1KlSJUcWbeISF6gUiMezzAMDsyezcZ33sGZnIxX/vw0evNNyrZqlWPr/2DNBwz8cSAGBg+UfYA5XeZQyK9QjqxfRCSvUKkRj5Z8/jxr33qLkytWAFCscWOajhlDQIkSObL+NGcafb7rw+ebPwegd/3eTHx8Il42rxxZv4jNy0aLd1pkPBbxZKZehemTTz6hdu3aBAYGEhgYSGhoKEuWLDEzkniQ47/8wvft23NyxQqs3t7Uf+MNHvziixwrNBeSLtDym5Z8vvlzrBYrH7T8gE9bf6pCIznK5m3j7tfv5u7X79a9n8TjmXqkpmTJkkRHR1OpUiUMw2DatGm0bduWzZs3U6NGDTOjSS7mSExk09ixHJg7F4AClSvT7J13KFCpUo5l2H1+N2ExYey/uJ/83vmZ0WkGj1d6PMfWLyKSF1kMwzDMDvH/FSpUiHfffZdnn332H8fGx8cTFBREXFwcgYGBOZBO3N25LVtYHRFBwrFjYLFQrWdPar/yCjbvnLuK6k8HfqLz7M7EpcZRtkBZYsNjqVm0Zo6tX+T/czldnNp0CoAS9UvoNgniFrLr97fbzKlxOp3Mnj2bxMREQkNDrzsmNTWV1NTUjOfx8fE5FU/cnMvhYNsnn7Dzs88wXC78S5QgNCqKYo0a5WiOj9d/zCtLXsFpOLm71N3M7zqf4IDgHM0g8v+lp6TzeeO/5nTpNgni6UwvNdu2bSM0NJSUlBTy5cvH/PnzqV69+nXHRkVFMWLEiBxOKO4u/tAhVg0axMUdOwAo26YNDQcPxjt//hzLkO5Kp/8P/Zm4fiIAT9d5msmtJ+Njz/6L+YmIyF9MPw5ZpUoVtmzZwtq1a3nppZfo0aMHO3fuvO7YyMhI4uLiMn6OHTuWw2nFnRiGwd6YGJZ06sTFHTvwDgzknnHjaBYVlaOF5nLKZVpNb8XE9ROxYCH6oWimtp2qQiMiksNMP1Lj7e1NxYoVAWjQoAHr169nwoQJfPrpp9eM9fHxwScHLmMv7i/53DnWDB3Kqd9/B6B4s2Y0HTUK/2LFcjTH/ov7CYsJY/f53fh7+fNth29pV7VdjmYQEZG/mF5q/s7lcl01b0bk74799BPrhg8n9fJlbD4+1B0wgMrdu2Ox5uyBxxWHV9BhVgcuJl+kZGBJFnVbRL0S9XI0g4iI/I+ppSYyMpLHHnuM0qVLc+XKFaZPn87y5ctZunSpmbHETTkSEtgYFcXBBQsAKFi1Ks3GjiXoP0f6ctKXm7/kxcUv4nA5aHxXYxZ0XUCJ/Dlz/RsREbk+U0vN2bNnefrppzl16hRBQUHUrl2bpUuX8vDDD5sZS9zQ2Y0bWR0ZSeKJE2CxUP2556j18ss5eqo2gNPlJGJZBO+tfg+ArjW6MqXtFPy8/HI0h4iIXMvUUvPFF1+YuXrJBZxpaWz7+GN2ffEFhstFwF13ERoVRdEGDXI8y5XUK3Sf153FexcDMLz5cN5q/hYWiyXHs4jcKpuXjebDmmc8FvFkbjenRuS/4vbvZ1VEBJd27QKgfLt2NIiMxCtfvhzPcuTyEcJiwth2dhu+dl+mtp1K15pdczyHSGbZvG3cP/x+s2OI5AiVGnE7hsvF3unT2TJuHM7UVHwKFKDx8OGUMulryVXHVtF+ZnvOJp6leL7iLOy2kMZ3NTYli4iI3JhKjbiVpDNnWDN0KKdXrQKgxD330HTUKPyCzbkq77d/fsszi54hzZlG3eJ1WdRtEaWCSpmSReR2GC6Dc7vOARBcLRiLVV+XiudSqRG3cXTpUtYNH05afDw2X1/qDRxIpW7dTJmz4jJcvPXrW4xeORqAdlXb8U37bwjwDsjxLCJ3wpHs4JOanwC6TYJ4PpUaMV3alStsGD2aw7GxABSqUYPQ6GiCypc3JU9iWiI9FvRg7q6/7vIdcXcEox8ajdVi+gW4RUTkJlRqxFRn1q9ndWQkSadOYbFaqd67N7Veegmrl5cpeU7En6DNjDZsOrUJb5s3n4V9xtN1njYli4iIZI5KjZjCmZbGnx9+yK6pU8EwyFeqFKFRUQTXM++KvBtObqBNTBtOJZyiiH8R5nedzz2l7zEtj4iIZI5KjeS4y/v2sWrQIC7v2QNAhY4dqT9oEF4B5s1XmbNzDk/Pf5rk9GRqBNcgNjyWcgXLmZZHREQyT6VGcozhcrH7q6/YOn48LocDn4IFafL225R88EHzMhkGo1eO5s1f3wTg8UqPE9MxhkCfQNMyiYjI7VGpkRyReOoUa4YM4czatQCENG9Ok7ffxq9IEdMypaSn8OyiZ5m+bToA/Zv0571H3sNm1VVXRURyI5UayXaHv/uO9aNG4YiPx+bnR/033qBi586m3l7gdMJp2s1ox9oTa7Fb7Ux8fCLPN3jetDwi2cXmZSN0YGjGYxFPplIj2SYtLo71o0Zx5PvvAShcqxah0dEEli1raq6tp7cSFhPGsfhjFPQtyNwuc3mg3AOmZhLJLjZvG4+8+4jZMURyhEqNZIvTa9awZsgQkk6fxmKzUfPFF6nx/PNY7eZ+5BbtWUT3ud1JdCRSuXBlFocvplLhSqZmEhGRrKFSI1nKmZrKlvHj2fPVVwDkL1OG0OhoitSubWouwzB4b9V7DFo2CAODh8o9xOzOsynoV9DUXCLZzXAZxB2NAyCodJBukyAeTaVGssyl3btZFRFB3L59AFTs0oX6r7+O3d/f1FxpzjReXPwiU7ZMAeClhi8x4dEJeNnMucCfSE5yJDuYUG4CoNskiOdTqZE75nI62T1tGn9++CEuhwPfwoVpMnIkdzVvbnY0ziedp8PMDqw8uhKrxcqERyfQp1EfUycpi4hI9lCpkTuSePIkqwcP5uz69QCUfPBBGo8YgW+hQiYng53ndhIWE8bBSwcJ9AlkVqdZtKzY0uxYIiKSTVRq5LYYhsHh2Fg2jB6NIyEBu58fDSIjKd+hg1scBflh/w90ndOV+NR4yhcsT2x4LNWDq5sdS0REspFKjWRa6uXLrH/7bY4uXQpAkbp1CY2KIn/p0iYn+6tsfbTuI/ov7Y/LcHFv6XuZ13UeRfzNu8ifiIjkDJUayZRTq1axZsgQks+exWK3U+ull6j+3HOmn6oN4HA66PdDPz7Z8AkAver2YlLrSXjbNDFSRCQvMP83keQK6SkpbPngA/Z+8w0AgeXKERodTeGaNU1O9pdLyZfoPLszPx/6GQsWxrYYy8BmA93iqzAREckZKjXyjy7u2sXqQYOIO3AAgErh4dR77TXsfn4mJ/vLvgv7aB3Tmr0X9hLgFcD0jtNpU6WN2bFE3ILVbqXhyw0zHot4MpUauSGX08muL79k20cf4UpPx7dIEZqOGkXIvfeaHS3Dr4d+peOsjlxKuUSpwFLEhsdSp3gds2OJuA27j51WE1uZHUMkR6jUyHUlHD/O6shIzm3aBECpFi1oNHw4vgXd5wq8kzdOps/3fUh3pdO0ZFPmd51P8XzFzY4lIiImUamRqxiGwaEFC9gQFUV6YiL2gAAaDh5MubZt3WZ+itPlZOCPAxm/djwA3Wt154s2X+Br9zU3mIgbMgyDpPNJAPgX8Xebf8ci2UGlRjKkXLrE+uHDObZsGQDB9esTGhVFvpIlTU72P/Gp8XSb040l+5cAMPKBkQy5d4h21CI34Ehy8F7R9wDdJkE8n0qNAHBy5UrWDB1KyvnzWO12avXtS7VnnsFqs5kdLcOhS4cIiwljx7kd+Nn9+Kr9V3Sq3snsWCIi4iZUavK49ORkNr/3HvtmzAAgsHx5mr3zDoWqVTM52dV+P/o77We253zSeULyh7Cw20IahjQ0O5aIiLgRlZo87ML27ayOiCD+0CEAKj/5JHVffRW7r3vNTflq61f0ju1NmjON+iXqs6jbIu4KvMvsWCIi4mZUavIgV3o6Oz//nG2ffIKRno5f0aI0HT2aEs2amR3tKi7DxZCfhxD9RzQAHat15Kv2X+Hv5W9yMhERcUcqNXnMlaNHWR0ZyfktWwAo/eijNHrzTXwKFDA1198lpCXw1PynWLB7AQBD7h3C2w+8jdWii4eJiMj1qdTkEYZhcGDuXDZFR5OenIxXvnw0HDqUsq1bu92ZQ8fijtFmRhu2nN6Cj82HL9p8wRO1nzA7loiIuDmVmjwg5cIF1g0fzvFffgGgaKNGhI4ZQ0BIiMnJrrXuxDrazmjL6YTTFA0oyoKuCwgtFWp2LJFcy2q3UqdHnYzHIp5MpcbDnVixgrVvvknKhQtYvbyo068fVZ5+2q1O1f6vGdtn0GthL1LSU6hVtBax4bGUKVDG7FgiuZrdx067qe3MjiGSI1RqPFR6UhKb3n2X/bNmARBUqRLNxo6lYJUqJie7lmEYjFgxghErRgDQunJrpneYTn6f/CYnExGR3MTUY5FRUVE0atSI/PnzU7RoUdq1a8eePXvMjOQRzv/5J9937JhRaKr26MGjM2e6ZaFJdiQTPjc8o9AMDB3Igq4LVGhEsohhGKQlppGWmIZhGGbHEclWph6pWbFiBX369KFRo0akp6czePBgHnnkEXbu3ElAQICZ0XIll8PB9smT2fHppxhOJ/7Fi9N09GiKN21qdrTrOnXlFG1ntGX9yfV4Wb2Y1HoSz9R7xuxYIh7FkeQgKl8UoNskiOcztdT88MMPVz2fOnUqRYsWZePGjdx3330mpcqd4o8cYfWgQVzYtg2AMq1a0WjIELyDgkxOdn2bT22mzYw2HI8/TiG/QszrMo/mZZubHUtERHIxt5pTExcXB0ChQoWu++epqamkpqZmPI+Pj8+RXO7MMAz2z57NpnfewZmcjFdgII3efJOyjz9udrQbmr9rPk/Of5IkRxJVi1RlcfhiKhSqYHYsERHJ5dym1LhcLvr378/dd99NzZo1rzsmKiqKESNG5HAy95V8/jxr33qLkytWAFCsSROajh5NQIkSJie7PsMwGPvHWCJ/jgTgkQqPMLPTTAr4FjA3mIiIeAS3KTV9+vRh+/bt/P777zccExkZyYABAzKex8fHU6pUqZyI53aO//ILa996i9RLl7B6e1O3f3+qPPUUFqt7XociNT2V5xc/z1dbvwKgb6O+fPDoB9itbvMRFBGRXM4tfqP07duXxYsX89tvv1GyZMkbjvPx8cHHxycHk7kfR2Iim8aO5cDcuQAUqFyZZu+8Q4FKlUxOdmPnEs/RfmZ7/jj2BzaLjQ8f+5CXG71sdiwREfEwppYawzD417/+xfz581m+fDnlypUzM47bO7dlC6sjIkg4dgwsFqr16kXtf/0Lm7f7ns2w/ex2wmLCOHz5MEE+QczuPJuHKzxsdiwREfFAppaaPn36MH36dBYuXEj+/Pk5ffo0AEFBQfj5+ZkZza24HA62ffIJOz/7DMPlwr9ECUKjoijWqJHZ0W7q+33f021ON66kXaFCwQos7r6YqkWqmh1LJE+x2qxU71Q947GIJ7MYJl6N6UY3UpwyZQo9e/b8x/fHx8cTFBREXFwcgYGBWZzOPcQdPMjqiAgu7tgBQNk2bWg4eDDe+d334nSGYTB+zXgG/jQQl+Hi/rL3M6fzHAr7FzY7moiIuIHs+v1t+tdPcn2GYbAvJobN77+PMyUF78BAGg8fTumWLc2OdlNpzjT6ft+XzzZ9BsBz9Z5jYquJeNvc9ysyERHxDG4xUViulnzuHGuGDuXUf84EK96sGU1HjcK/WDGTk93cxeSLdJzVkeWHl2PBwvuPvE//pv1veEROREQkK6nUuJljP/3EuuHDSb18GZuPD3Vfe43K4eFue6r2f+05v4fWMa3Zf3E/+bzzMaPjDFpVbmV2LJE8Ly0xTbdJkDxDpcZNOBIS2DBmDIcWLgSgYLVqNIuOJqhiRZOT/bNlB5fReXZnLqdcpkxQGRZ3X0zNote/gKKIiEh2UalxA2c3bmR1ZCSJJ05gsVqp9uyz1Hr5Zbc+Vfu/Pln/Cf9a8i+chpNmpZoxv+t8igYUNTuWiIjkQSo1JnKmpbFt4kR2fvEFGAYBd91FaFQURRs0MDvaP0p3pTNg6QD+ve7fADxV+ykmh03G1+5rcjIREcmrVGpMErd/P6siIri0axcA5du3p0FEBF758pmc7J/FpcTRdU5Xlh5YCsCYB8cQcU+EJgSLiIipVGpymOFysXf6dLaMG4czNRWfAgVoPHw4pR7OHVfZPXDxAGExYew6vwt/L3++bv81Hap1MDuWiIiISk1OSjpzhjVDh3J61SoAStx7L01HjsQvONjkZLfmtyO/0WFmBy4kX+Cu/HcRGx5LvRL1zI4lIiICqNTkmCM//MD6ESNIi4/H5utLvYEDqdStW675ymbK5im8sPgFHC4HDUMasrDbQkLyh5gdS0T+gdVmpdLjlTIei3gylZpslnblChtGj+ZwbCwAhWrUoNnYsQTmkpt3Ol1OIpZF8N7q9wDoUqMLU9pOwd/L3+RkInIr7L52un/X3ewYIjlCpSYbnVm/ntWRkSSdOoXFaqXG889T88UXsXp5mR3tllxJvcKT859k0Z5FAAxrPoxhzYflmqNLIiKSt6jUZANnWhp/fvghu6ZOBcMgX6lShEZHE1y3rtnRbtmRy0doM6MNf575Ex+bD1PbTaVbzW5mxxIREbkhlZosdnnvXlYNGsTlvXsBqNCxI/UHDcIrIMDkZLduzfE1tJ3RlrOJZykWUIyF3RbSpGQTs2OJyG1IS0zjvaJ/fX088OxA3SZBPJpKTRYxXC52f/UVW8ePx+Vw4FOoEE1GjKDkgw+aHS1Tpm+bzjMLnyHVmUqdYnWIDY+lVFAps2OJyB1wJDnMjiCSI1RqskDiqVOsGTKEM2vXAhDSvDlN3n4bvyJFTE5261yGi2G/DmPUylEAtK3Slm86fEM+b/e/GKCIiAio1Nyxw999x/pRo3DEx2Pz86PBG29QoXPnXDWZNsmRRI8FPZizcw4Ag+4exJiHxmC16PRPERHJPVRqblNaXBzrR47kyJIlABSuXZvQ6GgCy5QxOVnmnIg/QdsZbdl4aiNeVi8mh02mZ92eZscSERHJNJWa23B6zRrWDBlC0unTWGw2ar74IjWefx6rPXdtzo0nN9JmRhtOXjlJEf8izO86n3tK32N2LBERkduSu34Lm8yZmsqW8ePZ89VXAOQvU4bQ6GiK1K5tcrLMm7NzDk/Pf5rk9GSqB1cnNjyW8gXLmx1LRETktqnU3KJLu3ezatAg4vbvB6Bi167UHzgQu3/uurKuYRiMWTmGob8OBeCxio8xo9MMAn0CTU4mItnBYrVQpnmZjMcinkyl5h+4nE52T53Knx9+iCs9Hd/ChWkyciR3NW9udrRMS0lP4blFz/Httm8B6N+kP+8+8i52qz4GIp7Ky8+Lnst7mh1DJEfot9lNJJ48yerISM5u2ABAyQcfpPGIEfgWKmRyssw7k3CG9jPbs/r4auxWOx899hEvNHzB7FgiIiJZRqXmOgzD4HBsLBtGj8aRkIDd358GkZGUb98+V52q/V9/nvmTsJgwjsYdpaBvQeZ0mcOD5XLXRQFFRET+iUrN36Revsz6t9/m6NKlABSpW5dm0dHkK5U7r6obuyeW7vO6k5CWQOXClYkNj6Vy4cpmxxKRHJKWmMaEshMA6He4n26TIB5Npeb/ObVqFWuGDCH57Fksdju1Xn6Z6s8+m+tO1Ya/jja9v/p93vjpDQwMHiz3IHM6z6GgX0Gzo4lIDks6n2R2BJEckft+W2eD9JQUtowbx95v/5pAG1iuHKHR0RSuWdPkZLcnzZnGS4tf4sstXwLwQoMX+Pdj/8bL5mVyMhERkeyT50vNxZ07WTVoEPEHDwJQKTyceq+9ht3Pz+Rkt+d80nk6zurIb0d+w2qx8kHLD/hX43/lyrlAIiIimZFnS43L6WTXl1/y50cfYaSn4xccTJORIwm5916zo922Xed20TqmNQcvHSTQJ5CZnWbyaMVHzY4lIiKSI/JkqUk4fpzVERGc27wZgFIPP0zj4cPxKVDA3GB34McDP9JldhfiUuMoV6Aci7svpnpwdbNjiYiI5Jg8VWoMw+DgggVsHDOG9KQk7AEBNBw8mHJt2+bqr2c+WvcR/X/oj9Nwck/pe5jXZR7BAcFmxxIREclReabUpFy6xLrhwzm+bBkAwfXrExodTb677jI52e1zOB30/6E/H2/4GICedXsyqdUkfOw+JicTEXdhsVoIaRiS8VjEk+WJUnNy5UrWDB1KyvnzWO12av/rX1Tt1QurzWZ2tNt2KfkSXeZ0YdnBZViwMLbFWAY2G5irjziJSNbz8vOi9/reZscQyREeXWrSk5PZ/N577JsxA4CgChUIHTuWQtWqmZzszuy7sI+wmDD2XNhDgFcA33b4lrZV25odS0RExFQeW2oubNvGqogIrhw+DECVp56iTv/+2H19zQ12h5YfXk6HmR24lHKJUoGlWBS+iLrF65odS0RExHQeV2pc6ens+Owztk+a9Nep2sWKETp6NMVDQ82Odsc+3/Q5L333EumudJrc1YQF3RZQPF9xs2OJiBtzJDmYWH0iAH129sHLXxfhFM9lNXPlv/32G2FhYYSEhGCxWFiwYMEdLe/K0aMs69GDbf+59kzpRx/l8Xnzcn2hcbqcDFg6gN6xvUl3pRNeM5xfe/yqQiMi/8gwDOKOxBF3JA7DMMyOI5KtTD1Sk5iYSJ06dXjmmWfo0KHDbS/HMAz2z5nDpuho0pOT8cqfn4ZDh1K2VatcP3E2PjWe7nO7892+7wB4+/63GXrf0Fz/9xIREclqppaaxx57jMcee+yOl/PH669z+Y8/ACjaqBGhY8YQEBJyx8s12+HLhwmLCWP72e342n2Z1m4aXWp0MTuWiIiIW/KIOTUnf/uNfL6+1OnXj6o9emCxmvqtWpb44+gftJ/ZnnNJ5yiRrwQLuy2k0V2NzI4lIiLitnJVqUlNTSU1NTXjeXx8PPDXqdotxo2jYJUqZkXLUl9v/ZrnYp8jzZlGveL1WBS+iJKBJc2OJSIi4tZy1SGNqKgogoKCMn5KlSoFQIupUz2i0LgMF4N/HszTC54mzZlGh2odWNlrpQqNiIjILchVpSYyMpK4uLiMn2PHjgFg88n9twVITEuk06xORP0eBcDgewYzu/NsArwDTE4mIrmZxWIhuHowwdWDdYKBeLxc9fWTj48PPh5QYP7uePxx2sS0YfPpzXjbvPmizRc8WftJs2OJiAfw8vfi5R0vmx1DJEeYWmoSEhLYv39/xvNDhw6xZcsWChUqROnSpU1MlnPWn1hPmxltOJ1wmmD/YBZ0W0CzUs3MjiUiIpLrmFpqNmzYwAMPPJDxfMCAAQD06NGDqVOnmpQq58zaMYseC3qQkp5CzaI1WRy+mDIFypgdS0REJFcytdTcf//9efIKl4ZhMPK3kQxbPgyAVpVaEdMxhvw++U1OJiKexpHk4LNGnwHQe31v3SZBPFqumlPjCZIdyTyz6BlmbP/rzuGvhb7G2BZjsVltJicTEU9kGAbndp7LeCziyVRqctCpK6doN7Md606sw261M6nVJJ6t/6zZsURERDyCSk0O2XJ6C2ExYRyPP04hv0LM7TKX+8veb3YsERERj6FSkwMW7l7IE/OeINGRSNUiVYkNj6VioYpmxxIREfEouerie7mNYRiM/X0s7We2J9GRyMPlH2b1s6tVaERERLKBjtRkk9T0VF5Y/ALTtk4DoE+jPox/dDx2qza5iIhIdtBv2GxwLvEcHWZ14Pejv2Oz2Jjw6AT6NO5jdiwRyYMsFgtBZYIyHot4MpWaLLbj7A5ax7Tm8OXDBPkEMavzLB6p8IjZsUQkj/Ly96L/4f5mxxDJESo1WWjJviV0ndOVK2lXqFCwArHhsVQLrmZ2LBERkTxBE4WzgGEYTFgzgdYxrbmSdoXmZZqz9rm1KjQiIiI5SEdq7pDD6aDv932ZvGkyAM/We5aPW32Mt83b5GQiIuBIdjD1vqkA9PytJ15+uk2CeC6VmjtwMfkinWd35pdDv2DBwnuPvMerTV/VZDwRcRuGy+DkhpMZj0U8mUrNbdp7YS+tp7dm38V95PPOR0zHGFpXbm12LBERkTxLpeY2/HzwZzrN7sTllMuUCSpDbHgstYrVMjuWiIhInqaJwpn06YZPaflNSy6nXCa0ZCjreq9ToREREXEDKjW3KN2VTr8l/XjxuxdxGk6eqPUEv/T4haIBRc2OJiIiIujrp1sSlxJHt7nd+GH/DwCMfnA0kfdEakKwiIiIG1Gp+QcHLx0kLCaMned24mf34+v2X9OxekezY4mI3DL/Iv5mRxDJESo1N7HyyEraz2zPheQL3JX/LhaFL6J+ifpmxxIRuWXeAd68fu51s2OI5AiVmhuYumUqz8c+j8PloGFIQxZ2W0hI/hCzY4mIiMgNaKLw3zhdTgb9NIheC3vhcDnoXL0zK3quUKERERFxczpS8/8kpCXwxLwnWLRnEQBv3fcWw+4fhtWi7iciuZMj2cG3j30LwBNLntBtEsSjqdT8x9G4o7SJacPWM1vxsfkwpe0UwmuFmx1LROSOGC6DIyuOZDwW8WQqNcCa42toN6MdZxLPUCygGAu6LaBpyaZmxxIREZFMyPOlJmZbDL0W9iLVmUqdYnVYFL6I0kGlzY4lIiIimZRnJ4u4DBfDfh1G93ndSXWm0qZKG35/5ncVGhERkVwqTx6pSXIk0XNBT2bvnA3AG83eYMxDY7BZbSYnExERkduV50rNySsnaTujLRtObsDL6sXksMn0rNvT7FgiIiJyh/JUqdl0ahNtYtpw4soJCvsVZn7X+dxb5l6zY4mIZCsvf53GLXlDnik183bN48l5T5Kcnkz14OrEhsdSvmB5s2OJiGQr7wBvBicONjuGSI7w+InChmEwZuUYOs7qSHJ6Mo9WfJRVz6xSoREREfEwHn2kJiU9hd6xvfnmz28AeKXxK7zf8n3sVo/+a4uIiORJHvvb/WziWdrPbM+qY6uwWWx89PhHvNjwRbNjiYjkqPSUdGZ1nAVAl7ldsPt67G5fxDNLzbYz2wiLCeNI3BEK+BZgTuc5PFT+IbNjiYjkOJfTxb7v92U8FvFkHldqFu9dTPjccBLSEqhUqBKx4bFUKVLF7FgiIiKSzTxmorBhGIxbPY42MW1ISEvgwXIPsua5NSo0IiIieYRblJqJEydStmxZfH19adKkCevWrcvU+9OcaTwf+zyv/fgaBgbP13+eH574gUJ+hbIpsYiIiLgb00vNzJkzGTBgAMOGDWPTpk3UqVOHli1bcvbs2VteRrsZ7fh88+dYLVbGtxzPpNaT8LLpYlMiIiJ5iemlZty4cfTu3ZtevXpRvXp1Jk2ahL+/P19++eUtL+OPo3+Q3zs/i8MX069pPywWSzYmFhEREXdk6kThtLQ0Nm7cSGRkZMZrVquVFi1asHr16mvGp6amkpqamvE8Li4OgJK+JZnTZQ7VgqsRHx+f/cFFRHKJtMQ0UkgBID4+Hm+nt8mJRMj4XW0YRpYu19RSc/78eZxOJ8WKFbvq9WLFirF79+5rxkdFRTFixIhrXj8+/DhNhzfNtpwiIp4gOiTa7AgiV7lw4QJBQUFZtrxcdUp3ZGQkAwYMyHh++fJlypQpw9GjR7N0o+RF8fHxlCpVimPHjhEYGGh2nFxL2zHraFtmHW3LrKHtmHXi4uIoXbo0hQpl7Qk9ppaaIkWKYLPZOHPmzFWvnzlzhuLFi18z3sfHBx8fn2teDwoK0gcsiwQGBmpbZgFtx6yjbZl1tC2zhrZj1rFas3Zqr6kThb29vWnQoAE///xzxmsul4uff/6Z0NBQE5OJiIhIbmP6108DBgygR48eNGzYkMaNGzN+/HgSExPp1auX2dFEREQkFzG91HTt2pVz587x1ltvcfr0aerWrcsPP/xwzeTh6/Hx8WHYsGHX/UpKMkfbMmtoO2Ydbcuso22ZNbQds052bUuLkdXnU4mIiIiYwPSL74mIiIhkBZUaERER8QgqNSIiIuIRVGpERETEI7h9qZk4cSJly5bF19eXJk2asG7dupuOnz17NlWrVsXX15datWrx/fff51BS95eZbTl16lQsFstVP76+vjmY1j399ttvhIWFERISgsViYcGCBf/4nuXLl1O/fn18fHyoWLEiU6dOzfacuUFmt+Xy5cuv+UxaLBZOnz6dM4HdVFRUFI0aNSJ//vwULVqUdu3asWfPnn98n/aV17qdbal95bU++eQTateunXGRwtDQUJYsWXLT92TV59GtS83MmTMZMGAAw4YNY9OmTdSpU4eWLVty9uzZ645ftWoV4eHhPPvss2zevJl27drRrl07tm/fnsPJ3U9mtyX8ddXMU6dOZfwcOXIkBxO7p8TEROrUqcPEiRNvafyhQ4do1aoVDzzwAFu2bKF///4899xzLF26NJuTur/Mbsv/2rNnz1Wfy6JFi2ZTwtxhxYoV9OnThzVr1vDTTz/hcDh45JFHSExMvOF7tK+8vtvZlqB95d+VLFmS6OhoNm7cyIYNG3jwwQdp27YtO3bsuO74LP08Gm6scePGRp8+fTKeO51OIyQkxIiKirru+C5duhitWrW66rUmTZoYL7zwQrbmzA0yuy2nTJliBAUF5VC63Akw5s+ff9Mxb7zxhlGjRo2rXuvatavRsmXLbEyW+9zKtvz1118NwLh06VKOZMqtzp49awDGihUrbjhG+8pbcyvbUvvKW1OwYEHj888/v+6fZeXn0W2P1KSlpbFx40ZatGiR8ZrVaqVFixasXr36uu9ZvXr1VeMBWrZsecPxecXtbEuAhIQEypQpQ6lSpW7asuXG9JnMenXr1qVEiRI8/PDD/PHHH2bHcTtxcXEAN71RoD6Xt+ZWtiVoX3kzTqeTGTNmkJiYeMPbH2Xl59FtS8358+dxOp3XXFm4WLFiN/wO/fTp05kan1fczrasUqUKX375JQsXLuSbb77B5XLRrFkzjh8/nhORPcaNPpPx8fEkJyeblCp3KlGiBJMmTWLu3LnMnTuXUqVKcf/997Np0yazo7kNl8tF//79ufvuu6lZs+YNx2lf+c9udVtqX3l927ZtI1++fPj4+PDiiy8yf/58qlevft2xWfl5NP02CeKeQkNDr2rVzZo1o1q1anz66aeMHDnSxGSSV1WpUoUqVapkPG/WrBkHDhzggw8+4OuvvzYxmfvo06cP27dv5/fffzc7Sq53q9tS+8rrq1KlClu2bCEuLo45c+bQo0cPVqxYccNik1Xc9khNkSJFsNlsnDlz5qrXz5w5Q/Hixa/7nuLFi2dqfF5xO9vy77y8vKhXrx779+/Pjoge60afycDAQPz8/ExK5TkaN26sz+R/9O3bl8WLF/Prr79SsmTJm47VvvLmMrMt/077yr94e3tTsWJFGjRoQFRUFHXq1GHChAnXHZuVn0e3LTXe3t40aNCAn3/+OeM1l8vFzz//fMPv5UJDQ68aD/DTTz/dcHxecTvb8u+cTifbtm2jRIkS2RXTI+kzmb22bNmS5z+ThmHQt29f5s+fzy+//EK5cuX+8T36XF7f7WzLv9O+8vpcLhepqanX/bMs/TzexiTmHDNjxgzDx8fHmDp1qrFz507j+eefNwoUKGCcPn3aMAzDeOqpp4yIiIiM8X/88Ydht9uN9957z9i1a5cxbNgww8vLy9i2bZtZfwW3kdltOWLECGPp0qXGgQMHjI0bNxrdunUzfH19jR07dpj1V3ALV65cMTZv3mxs3rzZAIxx48YZmzdvNo4cOWIYhmFEREQYTz31VMb4gwcPGv7+/sbrr79u7Nq1y5g4caJhs9mMH374way/gtvI7Lb84IMPjAULFhj79u0ztm3bZvTr18+wWq3GsmXLzPoruIWXXnrJCAoKMpYvX26cOnUq4ycpKSljjPaVt+Z2tqX2ldeKiIgwVqxYYRw6dMj4888/jYiICMNisRg//vijYRjZ+3l061JjGIbx73//2yhdurTh7e1tNG7c2FizZk3GnzVv3tzo0aPHVeNnzZplVK5c2fD29jZq1KhhfPfddzmc2H1lZlv2798/Y2yxYsWMxx9/3Ni0aZMJqd3Lf08r/vvPf7ddjx49jObNm1/znrp16xre3t5G+fLljSlTpuR4bneU2W05duxYo0KFCoavr69RqFAh4/777zd++eUXc8K7kettQ+Cqz5n2lbfmdral9pXXeuaZZ4wyZcoY3t7eRnBwsPHQQw9lFBrDyN7Po8UwDCPzx3dERERE3IvbzqkRERERyQyVGhEREfEIKjUiIiLiEVRqRERExCOo1IiIiIhHUKkRERERj6BSIyIiIh5BpUZEREQ8gkqNiIiIeASVGhEREfEIKjUi4jbOnTtH8eLFGTNmTMZrq1atwtvb+5q7+IqI/J3u/SQibuX777+nXbt2rFq1iipVqlC3bl3atm3LuHHjzI4mIm5OpUZE3E6fPn1YtmwZDRs2ZNu2baxfvx4fHx+zY4mIm1OpERG3k5ycTM2aNTl27BgbN26kVq1aZkcSkVxAc2pExO0cOHCAkydP4nK5OHz4sNlxRCSX0JEaEXEraWlpNG7cmLp161KlShXGjx/Ptm3bKFq0qNnRRMTNqdSIiFt5/fXXmTNnDlu3biVfvnw0b96coKAgFi9ebHY0EXFz+vpJRNzG8uXLGT9+PF9//TWBgYFYrVa+/vprVq5cySeffGJ2PBFxczpSIyIiIh5BR2pERETEI6jUiIiIiEdQqRERERGPoFIjIiIiHkGlRkRERDyCSo2IiIh4BJUaERER8QgqNSIiIuIRVGpERETEI6jUiIiIiEdQqRERERGPoFIjIiIiHuH/AAwc75Pw2URlAAAAAElFTkSuQmCC\n" - }, - "metadata": {} - } - ] - }, - { - "cell_type": "markdown", - "source": [ - "**Return to slides here.**" - ], - "metadata": { - "id": "vACSnJgP87xA" - } - }, - { - "cell_type": "markdown", - "metadata": { - "id": "F3X4p66XVFXH" - }, - "source": [ - "In the **Elimination** example, the two equations in the system are:\n", - "$$ 2x - 3y = 15 $$\n", - "$$ 4x + 10y = 14 $$\n", - "\n", - "Both equations can be rearranged to isolate $y$. Starting with the first equation:\n", - "$$ -3y = 15 - 2x $$\n", - "$$ y = \\frac{15 - 2x}{-3} = -5 + \\frac{2x}{3} $$\n", - "\n", - "Then for the second equation:\n", - "$$ 4x + 10y = 14 $$\n", - "$$ 2x + 5y = 7 $$\n", - "$$ 5y = 7 - 2x $$\n", - "$$ y = \\frac{7 - 2x}{5} $$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "5x3kfV_WWhlR" - }, - "source": [ - "y1 = -5 + (2*x)/3" - ], - "execution_count": 116, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "dqA1lS0CWu5z" - }, - "source": [ - "y2 = (7-2*x)/5" - ], - "execution_count": 117, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "6CfRNs1DWzx5", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 455 - }, - "outputId": "c32b733e-c5b9-4654-d2f9-4d4f6cdfa93e" - }, - "source": [ - "fig, ax = plt.subplots()\n", - "plt.xlabel('x')\n", - "plt.ylabel('y')\n", - "\n", - "# Add x and y axes:\n", - "plt.axvline(x=0, color='lightgray')\n", - "plt.axhline(y=0, color='lightgray')\n", - "\n", - "ax.set_xlim([-2, 10])\n", - "ax.set_ylim([-6, 4])\n", - "ax.plot(x, y1, c='green')\n", - "ax.plot(x, y2, c='brown')\n", - "plt.axvline(x=6, color='purple', linestyle='--')\n", - "_ = plt.axhline(y=-1, color='purple', linestyle='--')" - ], - "execution_count": 118, - "outputs": [ - { - "output_type": "display_data", - "data": { - "text/plain": [ - "
" - ], - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAj4AAAG2CAYAAAB/OYyEAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABhhklEQVR4nO3dd3wUdeL/8dduekIILQECafTem9JrCElOvbPcnfoTC35VghRREA5FUbqCEM52d+jpneX0PE2jNxGQjiC9JIQkQGgJJCFt5/dH7vb0BIUUJrv7fj4eeTx2Z2Y3b5Zk952Zz3zGYhiGgYiIiIgLsJodQERERORWUfERERERl6HiIyIiIi5DxUdERERchoqPiIiIuAwVHxEREXEZKj4iIiLiMlR8RERExGWo+IiIiIjLUPERERERl+GwxWf27NlYLBbGjRtndhQRERFxEA5ZfLZt28bbb79Nhw4dzI4iIiIiDsThis+VK1e4//77effdd6ldu7bZcURERMSBuJsd4GaNHj2a6OhohgwZwiuvvPKz2xYWFlJYWGi/b7PZuHDhAnXr1sVisVR1VBEREakEhmFw+fJlgoODsVorts/GoYrPxx9/zM6dO9m2bdsNbT9r1ixeeumlKk4lIiIit0J6ejqNGzeu0HM4TPFJT09n7NixrFy5Em9v7xt6zPPPP8+ECRPs93NycggNDSU9PZ2aNWtWVVSnZrPZOHjwIACtWrWqcPMWkeqjKK+I14JfA+CZzGfw9PM0OZFImdzcXEJCQvD396/wczlM8dmxYwdnz56lS5cu9mWlpaVs2LCB+Ph4CgsLcXNz+9FjvLy88PLy+slz1axZU8WnnGw2GzVq1ADKXkcVHxHnUeRWhDdlf1jWrFlTxUeqncoYpuIwxWfw4MHs3bv3R8sefvhhWrVqxaRJk35SekRERET+l8MUH39/f9q1a/ejZX5+ftStW/cny0VERESuxWGKj4iIVC2ru5WOD3W03xZxRg5dfNatW2d2BBERp+Hu5c6d791pdgyRKqVKLyIiIi7Doff4iIhI5TEMg+L8YgA8fD000as4Je3xERERAIrzi5lVYxazasyyFyARZ6PiIyIiIi5DxUdERERchoqPiIiIuAwVHxEREXEZKj4iIiLiMlR8RERExGVoHh8REQHA6malzd1t7LdFnJGKj4iIAODu7c49/7jH7BgiVUqVXkRERFyGio+IiIi4DB3qEhERAIryiphVYxYAz195Hk8/T5MTiVQ+7fERERERl6HiIyIiIi5DxUdERERchoqPiIiIuAwVHxEREXEZKj4iIiLiMnQ6u4iIAGWXqWg+orn9togzUvERERGg7JIVv0/6vdkxRKqUKr2IiIi4DBUfERERcRk61CUiIkDZJSvmB80HYOLZibpkhTglFR8REbErzi82O4JIldKhLhEREXEZKj4iIiLiMlR8RERExGWo+IiIiIjLUPERERERl6GzukREBACL1UJY/zD7bRFnpOIjIiIAePh4MHLdSLNjiFQpHeoSERERl6HiIyIiIi5Dh7pERAQou2TFG+FvADA2dawuWSFOScVHRETs8s/lmx1BpErpUJeIiIi4DBUfERERcRkqPiIiIuIyVHxERETEZaj4iIiIiMvQWV0iIgKUXaYiuFuw/baIM1LxERERoOySFaO2jTI7hkiV0qEuERERcRkqPiIiIuIyVHxERASA4vxiFoYvZGH4Qorzi82OI1IlNMZHREQAMAyDnLQc+20RZ6Q9PiIiIuIyVHxERETEZaj4iIiIiMtQ8RERERGXoeIjIiIiLkNndYmICAAWi4XANoH22yLOSMVHREQA8PD14KnvnzI7hkiV0qEuERERcRkqPiIiIuIyVHxERAQou2TFH9v+kT+2/aMuWSFOyyXH+GgqdhGRnzIMg+z92fbbIs7IJff4LL/vPva99RZXTp0yO4qIiIjcQi5ZfHJTU/lu8WK+ioxkxf33c+Tjjym8dMnsWCIiIlLFXLL4dH/hBRrcdhsWq5Vzu3ezbcYM/tm/P+tHjyZt2TJKrl41O6KIiIhUAZcc4xMRE0PH3/+e/LNnSUtOJjUxkYsHDpCxbh0Z69bh7udH6NChhMfEENSjB1Y3N7Mji4iISCVwmD0+s2bNonv37vj7+xMUFMSdd97JoUOHKvScvkFBtB45kqjPPiP6yy9p+/jj+AUHU5KXx/F//Ys1jz3Gl4MHs3PuXC4cOKDBfiIiIg7OYYrP+vXrGT16NFu2bGHlypUUFxczbNgw8vLyKuX5A5o1o+PYsfxq+XKG/PWvNLvvPjxr1qQgO5uD77/PsrvvJvmOO/j+nXe4kpFRKd9TRKQ6sVgsBIQFEBAWoEtWiNOyGA66GyM7O5ugoCDWr19Pv379bugxubm5BAQEkJOTQ82aNX9x+9KiIrI2biQ1MZFTa9diKyqyrwvs0oXw2FhChw3Dq1at8v4zHI7NZmP//v0AtGnTBqvVYbqziIg4qJv9/P45DjvGJycnB4A6depcd5vCwkIKCwvt93Nzc2/qe7h5etJ40CAaDxpE0eXLpK9aRWpCAme2biV7506yd+5kx6uvEtyvH+ExMTQaMAA3L6/y/YNERESkyjnkHh+bzcavfvUrLl26xMaNG6+73fTp03nppZd+sryijTH/zJn/Doo+eNC+3KNGDUKGDSM8Opqg7t2dclC09viIiMitVpl7fByy+Dz55JOkpKSwceNGGjdufN3trrXHJyQkpFJeuP+4dOQIqYmJpCYlkZ+VZV/uU78+4SNGEB4TQ62WLZ3meLmKj4jzKi4o5r1+7wEwcsNIPHw8zA0k8m8uXXzi4uL48ssv2bBhAxERETf12Mp84f6XYbORvXMnqYmJpC1fTvEPDqsFNG9OeHQ04dHR+AUHV+r3vdVUfEScV1FeEbNqzALg+SvP4+nnaXIikTIuOcbHMAzGjBnDF198wbp162669FQ1i9VKULduBHXrRtcpU8j8+mtSExLIWLeOnCNH2LNwIXsWLiSoWzfCY2IIHTYMz4AAs2OLiIi4FIcpPqNHj+bvf/87X375Jf7+/pw+fRqAgIAAfHx8TE73Y26enoQMHkzI4MEU5eaSvnIlJxISOLt9u/1r+6uvEty/PxExMQT366dB0SIiIreAwxzqut4YmaVLlzJy5Mgbeo6qPNR1I/KysuyDoi8dPmxf7uHvT+iwYWUzRXfrhqUaHz7SoS4R56VDXVJdueyhLkfn17AhbR59lDaPPsqlw4f/Oyj69GmOff45xz7/HN8GDQgbMYKI2FhqtWhhdmQRERGnoj/XTVKrRQs6TZjAHStXMnjpUpr+5jd4+PuTf/o0B/7yF5Lvuovku+5i/5//TN4PzhYTERGR8nOYQ12VwexDXb+ktLCQzA0bOJGYSOb69diKi8tWWCwEdetGRGwsIUOH4mlidh3qEnFeRXlFvBH+BgBjU8fqUJdUGy59OntFVPfi80NFOTmcXLGC1KQkzm7bZl9u9fSkUf/+hMfGEty3L26et/aNScVHRERuNRWfcnKk4vNDeZmZpCYlkZqYSM7Ro/blHjVrEhYZSXhMDIFdutySQdEqPiIicqup+JSToxaf/zAMg0uHDpUNik5OpuDMGfs634YNyyZJjI2lVrNmVZZBxUdERG6lExdPEPdFHMmPJqv43CxHLz4/ZCst5ey2baQmJZG+YgXFV67Y19Vq2ZKI2FjCRozAt379yv2+Kj4iTqu4oJi/Rf0NgPtT7tclK8RUBcUFzPlmDnO+mcPVK1dhdsWvtQkqPk6h5OpVMtevJzUxkcwNG7CVlJStsFio36MH4bGxhAwZgqe/f4W/l4qPiPPSPD5SHRiGQcLhBMYtG8eJSycA6NegHxue2OBa8/jI9bl7exMaGUloZCSFly6VDYpOTCR7xw7OfPstZ779lm0vv0zjgQMJj4mhYZ8+t3xQtIiIyC85cv4IY5eNJeVoCgCNazbm9WGvM6zxMGo9UatSvoeKj5PxqlWL5vfeS/N77+VKRgZpSUmcSEgg9/hxTi5fzsnly/EMCCD0P4OiO3eu1jNFi4iI88srymPm1zOZv3k+RaVFeFg9eOa2Z5jabyo1PGuQ+4MLf1eUio8Tq9GoEW0ff5w2o0Zx8eDBsivHJyVRkJ3N0U8/5einn+LXqFHZoOiYGAKaNjU7soiIuBDDMPjngX8yfvl40nPTAYhsGsmiqEW0qFs1Vy9Q8XEBFouFOq1bU6d1azpNmFA2KDohgZMrV5KXkcH377zD9++8Q+3WrQmPiSkbFB0UZHZsERFxYgfPHeTplKdZeXwlAGEBYSwcvpA7Wt5x3etzVgYVHxdjdXOjQa9eNOjVi27TppGxbh2pCQlkbtzIxQMHuHjgALtfe436PXsSHhNDyJAheNSoYXZsERFxEpcLLzNjwwwWbFlAia0ELzcvJvWexKQ+k/D18K3y76/i48Lcvb0JGz6csOHDuXrxIunLl5cNit61i9ObN3N682a2vfwyjQYNIjw6moa9e2Nx14+MiDPz8NUp7FI1DMPgk+8/4ZkVz5B5OROAmBYxLIxcSNM6t26ohU5nl5+4cuqUfabo3OPH7cu9atUiJDISo21bfFu0oG3btjqdXUREftG+s/sYkzKGdanrAGhSuwlvDH+DmBYxN/R4zdxcTio+N8cwDC7u38+JxETSkpO5eu6cfZ1n/fo0u+MOImJjCWjSxMSUIiJSXeVczeGl9S+x6NtFlBqleLt7M6XPFJ7t/Sze7t43/DwqPuWk4lN+ttJSznz7LScSEji5YgW2q1ft6+q0bVs2KDoqCp/AQBNTiohIdWAYBh9+9yHPrnyWM3lll1e6q9VdvB75OuG1wm/6+VR8yknFp+JsNhv7du0id/t2SnbvJmvTJox/zxRtsVqp36sXEbGxNB48GA8/P5PTisjNKLlawqe/+RSAez+/F3dvjemTm7fn9B5GJ4/mm/RvAGhepzmLoxYT2Syy3M9ZmZ/f+qmWm2b18qJW7960GTWKokuXOLlsGalJSZzbvZvTmzZxetMm3Ly9aTxoUNlM0bffjtVDAyZFqjtbqY0jyUfst0VuxsWCi7yw9gX+uP2P2Awbvh6+TOs3jfG9xuPl7mV2PDsVH6kQ7zp1aPH739Pi97/n8smT9kHRl1NTSUtOJi05Ga/atQkdPpyI2FjqduhQpfMziIjIrWUzbLy3+z0mr5pMdn42APe2vZf5Q+cTEhBicrqfUvGRSuMfGkr7J5+k3RNPcGHfvrKZolNSuHr+PEc++ogjH31EjZAQwmNiCI+JoWZ4uNmRRUSkArZnbicuOY5vM74FoHW91iyOWszgJoNNTnZ9Kj5S6SwWC3Xbt6du+/Z0fvZZTm/ZQmpiIqdWreJKejr73nyTfW++SZ127YiIjSV0+HB86tUzO7aIiNyg8/nnmbpmKu/seAcDgxqeNZjefzpP93waD7fqPbRBxUeqlNXdneA+fQju04eS/HxOrVlDalISWd98w4V9+7iwbx87586lwW23ER4TQ+NBgzQoWkSkmiq1lfKnnX9iypopXCi4AMD97e9n7tC5BPsHm5zuxqj4yC3j7utrP8x19fx50pYtIzUxkfPffUfWxo1kbdyIm48PIYMHEx4TQ4PbbsOqmaJFRKqFLae2EJccx46sHQC0D2pP/Ih4+oX1MznZzdGnipjCu25dWt5/Py3vv5/ctDTSkpI4kZDAlZMnSU1MJDUxEe+6dQkdPpzwmBjqtm+vQdEiIibIzstm8qrJ/GX3XwCo6VWTGQNn8FT3p3C3Ol6N0Dw+clNsNhv79+8HoE2bNpV6yQrDMDi/d699UHThhQv2dTVCQ4mIjSU8Ohr/sLBK+54iInJtJbYS3tr+FtPWTuPS1UsAjOw0ktmDZ1O/Rv1bmkUTGJaTik/FVWXx+dH3KS7m9JYtnEhI4NSaNZQWFNjX1e3QwT5TtHedOlXy/UVEXNnGkxuJS45jz5k9AHRu0Jn4EfHcHnK7KXlUfMpJxafiblXx+aHivLyyQdGJiZzetAnDVjaxmsXNjYa9exMeHU3jQYNw9/Wt8iwiIs4s63IWk1ZN4oPvPgCgtndtXh30Ko93fRw3q5tpuVR8yknFp+LMKD4/VJCdbR8UfWHfPvtydx8fGg8ZUjYoulcvDYoWKYeSqyV88eAXANz1wV26ZIULKS4tJn5rPC+ue5HLRZexYOGxLo8xc/BM6vmaP92Iik85qfhUnNnF54dyU1PtA6GvpKfbl3vXrUtYVBThsbHUadtWg6JFblBRXhGzaswC4Pkrz+Pp52lyIrkV1qWuIy45ju+zvwege3B3loxYQvdG3U1O9l+6VpcIUDM8nA5xcbQfPZrz331XduX4Zcu4ev48hz78kEMffoh/eDjhMTFExMRQI6T6TZ0uImKWjNwMJq6cyMf7Pgagrk9dZg+ZzSOdH8FqMe+P2qqm4iMOz2KxUK9jR+p17EjXSZPI2rSJ1IQETq1dy+XUVPbGx7M3Pp56nToRHh1N6PDhGhQtIi6rqLSIhVsW8vL6l8krzsNqsfJE1yeYMWgGdXyc/71RxUecitXDg0b9+9Oof3+K8/JIX7WK1MREzmzZwrnduzm3ezc75swpGxQdE0PjgQNx9/ExO7aIyC2x8thKxqSM4dD5QwDc1vg2loxYQueGnU1Oduuo+IjT8vDzo8kdd9DkjjvKBkWnpJQNiv7+ezLXrydz/XrcfX0JGTKE8NhY6vfsidXNvLMWRESqysmck0xYPoHPD3wOQJBfEHOHzOXBjg869WGta1HxEZfgExhIq//3/2j1//4fOcePlw2KTkoi79QpTnz1FSe++gqfwEBCo6KIiI2lduvWGhQtIg6vsKSQ+Zvm8+rXr1JQUoCbxY24HnFMHzCdWt61zI5nChUfcTkBTZrQ8emn6TBmDOd27yY1IYGTy5dTkJ3Nob/+lUN//Ss1mzQpu65YdDQ1Gjc2O7KIyE1LPpLM2GVjOXrhKAD9wvqxOGoxHep3MDmZuXQ6u9yU6nQ6e2UqLSoi65tvSE1KImPNGkoLC+3rAjt3JjwmhtDhw/GqVcu8kCJVzDAMivOLAfDw9dBeTwd1/OJxxi8fz1eHvgKgYY2GzB82n9+1+53D/p9qHp9yUvGpOGctPj9UfOUK6StXkpqUxOktW+DfvyJWd3ca9u1LeEwMjQYMwN3b2+SkIiL/VVBcwJxv5jB742wKSwtxt7oztudYXuj/AjW9HPszT8WnnFR8Ks4Vis8P5Z89S1pyMqmJiVw8cMC+3N3Pj9ChQwmPjSWoe3cNihYR0xiGwVeHvmLc8nGkXkoFYFDEIBZHLaZNYBtzw1USFZ9yUvGpOFcrPj+Uc/QoqUlJpCYmkpeZaV/uExRE2IgRhMfEULtVK4fdlSxSUlhC4v8lAhDzdgzuXhoGWt0dOX+EscvGknI0BYDGNRvz+rDXubvN3U71XqTiU04qPhXnysXnPwybjexdu0hNTOTksmUU5eba1wU0bUp4bCzh0dH4BQebmFLk5umSFY4jryiPmV/PZP7m+RSVFuFh9WDi7ROZ0ncKNTxrmB2v0umSFSImslitBHXtSlDXrnR9/nmyNm4kNTGRU2vXknPsGHsWLmTPwoUEdu1aNih62DANihaRSmEYBv888E/GLx9Pem7ZNQojm0ayKGoRLeq2MDmdY1DxEakAN09PGg8aRONBgyi6fLlspuiEBM5s3Ur2jh1k79jBjldfJbhfP/ugaDcvL7Nji4gDOnjuIGNSxrDq+CoAwgLCWDh8IXe0vMOpDmtVNRUfkUri6e9P07vuouldd5F/+jRpKSmcSEjg0qFDnFqzhlNr1uBRowYhw4YRERNDUPfuWFzwUKGI3JzLhZeZsWEGC7YsoMRWgpebF5N6T2JSn0n4eviaHc/hqPiIVAHfBg1o/fDDtH74YS4dOWKfKTo/K4vj//wnx//5T3zq1yd8xAjCY2Op3bKl2ZFFpJoxDIOP933MxJUTybxcdkJFbItYFkQuoGmdpianc1wqPiJVrFbz5nQaP56OY8eSvXMnqYmJpC1fTsGZMxxYupQDS5cS0Lw5ETExhI0YoUHRIsK+s/sYkzKGdanrAGhSuwmLhi8iukW0ucGcgIqPyC1isVoJ6taNoG7d6DplCpkbNpCamEjGunXkHDnC7gUL2L1gAUHduxMeHU3osGF4BgSYHVtEbqGcqzlMXzedxVsXU2qU4u3uzZQ+U3i297N4u2vS1Mqg09nlpuh09spXlJtL+sqVnEhI4Oy2bfblVg8Pgvv3JyImhuB+/TQoWqqcYRjkn8sHwLeerwbM3kKGYfDhdx/y7MpnOZN3BoC7Wt3F65GvE14r3Nxw1YDm8SknFZ+KU/GpWnlZWfaZoi8dPmxf7uHvT+iwYWUzRXftqkHRIk5k9+ndxCXH8U36NwC0qNuCRcMXEdks0uRk1YeKTzmp+FScis+tc/HQobLxQMnJ5J8+bV/u26AB4dHRhMfEUKuF5u0QcVQXCy4ybe003tz+JjbDhp+HH9P6TWNcr3F4uWsP7w+p+JSTik/FqfjceobNxtnt28tmil6xguLLl+3rarVoQXhMDOHR0fg2aGBiSnEGJYUlLJ+wHIDI1yN1yYoqYjNsvLf7PSavmkx2fjYA97a9l9eGvUbjmo1NTlc9qfiUk4pPxan4mKu0sJCM9etJTUoic/16bMXFZSssFup37054TAwhQ4fiqZ9vKQddsqLqbc/cTlxyHN9mfAtAm8A2LI5azKCIQSYnq950yQoRF+Xm5UXosGGEDhtGUU4OJ1esIDUxkbPbt3Nm61bObN3KtldeoVH//oTHxhLcty9unvrwEjHb+fzzTFk9hXd3vouBgb+nP9MHTGdMjzF4uHmYHc+lqPiIOCjPgACa3XMPze65h7zMTPuV43OOHiV95UrSV67Es2ZNQiMjCY+JIbBLFw2KFrnFSm2lvLvzXaaumcqFggsA3N/+fuYNnUdD/4Ymp3NNKj4iTsAvOJi2o0bR5rHHuPTvQdGpSUkUnD3L0X/8g6P/+Ae+DRuWjQeKiaFWs2ZmRxZxeltObSEuOY4dWTsAaB/UnvgR8fQL62dyMtem4iPiRCwWC7VbtaJ2q1Z0HD+es9u2kZqURPqKFeRnZbH/3XfZ/+671G7VivB/zxTtW7++2bFFnMrZvLM8v+p5/rL7LwDU9KrJjIEzeKr7U7hb9bFrNv0PiDgpq5sbDXr1okGvXnSbOpXM9etJTUwkc8MGLh48yMWDB9n12mvU79mzbFD0kCF4+vubHVvEYZXYSnhr+1tMWzuNS1cvATCy00hmD55N/Rr6A6O6UPERcQHu3t6ERkYSGhlJ4aVLZYOiExLI3rmTM1u2cGbLFrbPmEGjAQMIj4mhYZ8+GhQtchM2ntxIXHIce87sAaBzg84sGbGE20JuMzmZ/C+dzi43RaezO5crGRmkJSVxIiGB3OPH7cs9AwIIHT6c8OhoAjt31qBoF2HYDHJO5gAQEBqAxapLVvySrMtZPLfqOT787kMAanvXZubgmYzqMgo3q5vJ6ZyH5vEpJxWfilPxcU6GYXDx4EFSExJIS06mIDvbvs6vUSP7TNEBTZuamFKk+iguLSZ+azwvrnuRy0WXsWDhsS6PMXPwTOr51jM7ntNR8SknFZ+KU/FxfrbSUs5u3Vo2U/TKlZTk5dnX1W7TpqwERUfjExhoYkoR86xLXUdcchzfZ38PQI9GPYiPiqd7o+4mJ3NeKj7lpOJTcSo+rqXk6lUy1q0jNSGBzI0bMUpKALBYrT8aFO1Ro4bJSaUylBaVsnrqagAGvzoYN08dqvmhU7mneHbls3y872MA6vrUZfaQ2TzS+RGsFr0XViUVn3JS8ak4FR/XdfXiRdKXL+dEQgLndu+2L3fz8qLRoEGER0fTsHdvDYp2YLpkxbUVlRaxcMtCXl7/MnnFeVgtVp7o+gQzBs2gjk8ds+O5BF2yQkRuOe/atWn+29/S/Le/5Up6un2m6NwTJziZksLJlBS8atUqGxQdE0O9Tp2wWDQ4VhzbymMrGZMyhkPnDwFwW+PbWDJiCZ0bdjY5mZSXw/25vmTJEsLDw/H29qZnz55s3brV7EgiLqdGSAjtnniC6IQEhn/6KS3/3//Du149Ci9d4sjHH7PygQdIiIpiz6JF5PzgbDERR3Ey5yR3f3o3wz4cxqHzhwjyC+K9O95j4yMbVXocnEPt8fnkk0+YMGECb731Fj179mThwoVERkZy6NAhgoKCzI4n4nIsFgt12ralTtu2dH7mGc5s3UpqQgLpq1ZxJT2d799+m+/ffps6bduWzRQdFaVB0VKtFZYUMn/TfF79+lUKSgpws7gR1yOOlwa8RIB3gNnxpBI41Bifnj170r17d+Lj44Gy8SYhISGMGTOGyZMn/+Lj/3OM8OLFixrjU042m42DBw8C0KpVK43xkWsqKSggY9060hITydq06ceDonv1IiwmhsaDB+Ph62tyUvmhorwi5tScA8Ck3EkuN8Yn+Ugy41eM5+iFowD0C+3HouGLaF+/vcnJJDc3l9q1a7vW4OaioiJ8fX357LPPuPPOO+3LH3roIS5dusSXX375k8cUFhZSWFhov5+bm0tISAibN2+mhs5CEbklSnJzubR5M5e+/pr8I0fsyy1eXgR060atvn3xb98ei7tD7YB2SiX5JXzZs+y99I5v78Dd1zX+T9KvpDP3u7msy1oHQKB3IM+0f4YRISM0Tq2auHLlCrfddptrDW4+d+4cpaWl1P+fCyrWr1/fvgfif82aNYuXXnrpVsQTketwr1mTepGR1IuMpPD0aS5t3MjFjRspysri0jffcOmbb3Dz96fW7bdTq08ffJs314eN3BJXS6/yl0N/4c+H/kyRrQh3izsPNHuAJ1o/gZ+Hn9nxpIo4TPEpj+eff54JEybY7/9nj0+rVq10qKucdKhLKqRNGxg0CMMwuPD996QlJZGWnEzhhQucX76c88uXUyMkhLDoaMKio6kZHm52Ypdi2AwafNcAgMDWgU57yQrDMPjq8FdMWD2B1EupAAwKH8Qbw9+gTWAbc8PJNeXm5lbaczlM8alXrx5ubm6cOXPmR8vPnDlDgwYNrvkYLy8vvLy8frLcarXqA7sS6HWUigjs0IHADh3o8uyznN68mdSkJE79Z1D0W2/x/VtvUaddOyJiYwkdPhyferoMQJWzQoP2134/dRZHzh9h7LKxpBxNAaBxzca8Pux17m5zt/Y0VmOV+VnjMMXH09OTrl27snr1avsYH5vNxurVq4mLizM3nIiUm9XdneC+fQnu25eS/HxOrVnDicRETm/axIV9+7iwbx87586lwW23ER4bS8igQbhrULTcpLyiPGZ+PZP5m+dTVFqEh9WDibdPZGrfqfh56rCWK3GY4gMwYcIEHnroIbp160aPHj1YuHAheXl5PPzww2ZHE5FK4O7rS3hMDOExMVw9f560ZctITUjg/N69ZG3cSNbGjWz18SFk8GDCY2JocNttWDUoutKUFpXy9cyvAeg7pa9TXLLCMAw+P/A5E5ZPID03HYDIppEsilpEi7otTE4nZnCYs7r+Iz4+nnnz5nH69Gk6derEokWL6Nmz5w09VpesqDhdskLMkJuWRmpiIqmJiVw5edK+3LtuXftM0XXbt9ehigpytktWHMg+wNPLnmbV8VUAhAWEsXD4Qu5oeYd+VhyMrtVVTio+FafiI2YyDIPze/eSmphIWkoKhRcu2Nf5h4WV7S2KjsY/LMzElI7LWYrP5cLLzNgwgwVbFlBiK8HLzYtJvScxqc8kfD10mNQR6VpdIuKSLBYL9Tp0oN5/BkVv2cKJhAROrVnD5bQ09i5Zwt4lS6jboYN9pmjvOrqIpKswDIOP933MxJUTybycCUBsi1gWRC6gaZ2mJqeT6kLFR0QcktXDwz4oujgvj1Nr1pCakMDpzZs5/913nP/uO3bOmUPD3r0Jj4mh8cCBGhTtxPad3Udcchzr09YD0LR2U94Y/gbRLaJNTibVjYqPiDg8Dz8/ImJjiYiNpSA7u2xQdGIiF/btI3PDBjI3bMDdx4fGQ4cSHh1Ng169NCjaSeRczWH6uuks3rqYUqMUH3cfpvSdwsTbJ+Lt7m12PKmG9JsvIk7FJzCQVg8+SKsHHyT3xImyQdFJSVxJTyf1q69I/eorvOvWJSwqivDYWOq0bauBrg7IMAw++O4Dnlv5HGfyyuZ3u6vVXSyIXEBYLY3xkutT8RERp1UzIoIOY8bQPi6Oc3v2kJqYyMmUFK6eP8+hDz/k0IcfUjMigrDoaCJiYqgREmJ2ZLkBu0/vJi45jm/SvwGgRd0WLBq+iMhmkSYnE0egs7rkpuisLnF0tuJisjZtIjUhgVNr11J69ap9Xb1OnQiPiSF0+HC8a9c2MaU5bKU2snZmAdCwS0OsbtXr9/tiwUWmrZ3Gm9vfxGbY8PPwY1q/aYy/bTyebo55BprcGJ3OXk4qPhWn4iPOpDgvj/RVq0hNTOTMli0YNhsAFnf3Hw+K9vExOalrsxk2lu5ayuTVkzmXfw6A+9rex/xh82lcs7HJ6eRWUPEpJxWfilPxEWdVkJ1NWnIyqUlJXPj+e/tyd19fQoYOJTwmhvo9e2J1c/zZjB3J9sztjE4ezdaMrQC0CWzD4qjFDIoYZHIyuZVUfMpJxafiVHzEFeQcP26fKTovI8O+3CcwkNCoKCJiY6ndurXTDYouLSplyxtbAOg1tpepl6w4n3+eKaun8O7OdzEw8Pf0Z/qA6YzpMQYPNw/Tcok5VHzKScWn4lR8xJUYhsG53btJTUggbdkyinJy7OtqNmlinym6RmPnONxSHWZuLrWV8u7Od5m6ZioXCspm5n6gwwPMHTKXhv4Nb3keqR40c7OIyC1gsVgI7NyZwM6d6TJ5MlnffENqYiIZa9eSe/w43y1axHeLFhHYuTPhsbGERkbiVauW2bEd1pZTWxidPJqdWTsBaB/UniUjltA3rK/JycSZqPiIiNwAN09PGg8cSOOBAym+coX0lStJTUri9JYtZO/aRfauXeyYOZOGffsSERtLcP/+uHtrAr0bcTbvLJNXTWbp7qUABHgFMGPgDJ7s/iTuVn1MSeXST5SIyE3yqFGDJnfdRZO77iL/zBnSUlJITUzk4oEDZKxdS8batbj7+RE6bBjhMTEEde+uQdHXUGIr4a3tbzFt7TQuXb0EwMhOI5k9eDb1a9Q3N5w4LRUfEZEK8K1fn9YjR9J65Ehyjh4lNSmpbFB0ZibHv/iC4198gU9QEGEjRhARE0OtVq2cblB0eWw8uZHRyaP57sx3AHRp2IX4qHhuC7nN5GTi7FR8REQqSUCzZnQcO5YOY8aQvWtX2UzRy5ZRcPYsB997j4PvvUdA06aEx8YSHh2NX3Cw2ZFvuazLWTy36jk+/O5DAGp712bm4JmM6jIKN6v2iknVU/EREalkFquVoK5dCerala7PP0/Wxo2cSEggY906co4dY8/ChexZuJDArl2JiIkhZNgwpx8UXVxazOKti5m+bjqXiy5jwcKoLqN4dfCr1POtZ3Y8cSE6nV1uik5nFym/osuXywZFJyZyZutW+Pfbr9XdneD+/QmPjqbRgAG4eXmZks9WauPk1ycBCO0bWmmXrFh7Yi1jUsbwfXbZxJA9GvUgPiqe7o26V8rzi/PTPD7lpOJTcSo+IpUj//RpUpOTSU1M5NKhQ/blHv7+hAwdSsS/B0VbHPh37FTuKSaumMgn338CQF2fusweMptHOj+C1eK4/y659VR8yknFp+JUfEQq36UjR+wzReefPm1f7tugAWFRUYTHxlK7ZUsTE96cotIiFm5ZyMvrXyavOA+rxcoTXZ9gxqAZ1PGpY3Y8cUAqPuWk4lNxKj4iVcew2cjeuZMTCQmcXLGC4txc+7qA5s2JiIkhLDoav4ZVM4NxaXEpO97ZAUDXx7vi5nHzg41XHlvJmJQxHDpfthfr9pDbiY+Kp3PDzpWaVVyLik85qfhUnIqPyK1RWlRE5oYNpCYkkLF+PbbiYvu6oO7dCY+JIXTYMDwr8b2sIpesOJlzkvHLx/PPA/8EoL5ffeYOncsDHR7QYS2pMF2yQkTEybl5ehIyZAghQ4ZQlJPDyX8Pij67bZv9a/srr9Cof3/CY2II7t8fN89bf22tqyVXeW3Ta7z69asUlBTgZnEjrkccLw14iQDvgFueR+SXqPiIiFRzngEBNLv7bprdfTd5WVmk/WdQ9OHDpK9aRfqqVXjUrEno0KGEx8YS1LXrLRkUnXwkmadTnubYxWMA9AvrR3xUPO3rt6/y7y1SXio+IiIOxK9hQ9o8+ihtHn2Ui4cOkZqYSFpyMvmnT3Ps88859vnn+DZoQHh0NOGxsdRq3rzSMxy/eJxxy8aRcDgBgIY1GvLasNf4bbvfalZqqfZUfEREHFTtli2p3bIlncaP5+z27WUzRa9YQf7p0+z/85/Z/+c/U6tFi7KZokeMwLdBgwp9v4LiAmZvnM2cb+ZQWFqIu9Wd8b3GM63fNPy9/CvpXyVStTS4WW6KBjeLVG+lhYVkrF9PamIimevXYyspKVthsVD/34OiQ4YOveag6OsNbjYMg68OfcW45eNIvZQKwOCIwSyOWkzrwNa35N8lrk2Dm0VE5JrcvLwIHTaM0GHDKLx0ifQVK0hNSuLs9u2c2bqVM1u3su2VV2g0YEDZoOi+fX92UPSR80cYu2wsKUdTAGhcszGvD3udu9vcrcNa4pBueo/PQw89xKOPPkq/fv2qKlOV0R6fitMeHxHHlJeZab9yfM7Ro/blnjVrEhoZSXhsLHXbd+TYyuMANBzYkNmbZzN/83yKSovwsHow8faJTO07FT9PP7P+GeKiTJ3H58477yQ5OZmwsDAefvhhHnroIRo1alShELfKf1647Mzsa75wVjcr7t7/3QlWlFd03eeyWC14+HiUa9vi/GKu97JbLBY8fMu5bUExhu36/50/nJPjZrYtuVqCrdQGlBWfgwcOAtCqdSusVut1t70WD18P+1+JJYUl2EoqaVsfDyzWsm1Li0opLS6tlG3dvd3t1yu6qW2LSykt+pltvdyxut/8trYSGyWFJdfd1s3TzT7p3E1tW2qj5OrPbOvhhpvnzW9r2AyKC4orZVuruxV3r7LfT8MwKM6vpG1v4vfeGd4jbKU2cg4fIW1ZStmV47Oz7etrNAoiPDqaw228mLR7Hhm5GQAMbTKUeUPn0axuM/u2N/N7r/eIa2yr9wjgxt8jTJ/AMDs7mw8++ID333+f/fv3M2TIEB599FHuuOMOPDw8fvkJTPKfF24yk/HG+yfrm49ozu+Tfm+/P9Nv5nXfMMP6hzFy3Uj7/XmB88g/l3/NbYO7BTNq2yj7/YXhC8lJy7nmtoFtAnnq+6fs9//Y9o9k78++5rYBYQGMSx1nv/9u93fJ3J55zW196/nybPaz9vvvDXiPtPVp19zWw9eDKXlT7Pf/Hv13jiQfuea2AC8aL9pv/+Oef7D/s/3X3faH4wb+NfJf7Hl/z3W3nXh2In6BZX9ZJo1OYvsft19327EnxlIrvBYAK55dweb5m6+77ZP7niSobRAA66avY/1L66+77WNbH6NR97Ji/828b1j13KrrbvvQ2ocIHxAOwNYlW0mJS7nutr9L/B0tolsAsPu93Xz58JfX3fbuT++m7T1tAfj+H9/z2b2fXXfbO5beQaeRnQA4nHSYj2I+uu62UfFR9BjdA4DUdam8P/D96247ZO4Qej/bG4CMbRn8qcefrrtt/xf7M2D6AADOfn+WN9u9ed1tb5t4G8PmDQPgUuol3oh447rbdnuqG9FLogHIy85jftD8627b8aGO3PnencCPx61cS5u723DPP+6x33/J8tJ1t3Xm9wg390KatPjvz+HRU/0wcq89GFrvEf+l94gyVf0eUZnFp1zHKQIDA5kwYQJ79uzh22+/pVmzZjz44IMEBwczfvx4jhy5/i+AiIhUPyW+HizpdZo91OdiThheJTqMLc6pQmd1ZWVl8de//pWlS5dy6tQpfvOb35CRkcH69euZO3cu48ePr8ysFaZDXTrUdbPbajf2v7fVoa6b3tYR3iMMw+Cz7z9jypopZF3JAmDqzKkA/PqdZmSsWc653bvt27t5ehHcvx+hkcMJHfrfmaJ1qEvvEU59qKu4uJivvvqKpUuXsmLFCjp06MBjjz3G73//e3uYL774gkceeYSLFy9WKFxl0+DmitPgZhHnsO/sPuKS41ifVnYYp2ntprze73V2dd4F/PeQ05WMDNKSkjiRkEDu8eP2x3sGBBA6fDgRMTHU69xZZ3hJlTL1dPaGDRtis9n43e9+x9atW+nUqdNPthk4cCC1atWqUDAREal8OVdzmL5uOou3LqbUKMXH3Ycpfacw8faJWAut7GLXj7av0agRbR9/nDajRnHx4EFSExJIS06mIDubo598wtFPPsGvUaOymaJjYgho2tSkf5nIjbnpPT4ffPAB99xzD97ePx0cXN1pj0/FaY+PiGMyDIMPvvuA51Y+x5m8MwD8uvWveX3Y64TVCgNu/OrsttJSzm7dyomEBNJXraIkL8++rnabNkTExBA2YgQ+gYFV/K8SV2H6WV2OSsWn4lR8RBzP7tO7iUuO45v0bwBoUbcFi6MWM6zpsB9td6PF54dKCgrIWLeubKbojRsx/j1TtMVqpX6vXoRHRxMydCgefpr7R8pPMzeLiMgvulhwkWlrp/Hm9jexGTb8PPyY1m8a428bj6fbL5eaG+Hu40NYVBRhUVFcvXiRk8uWkZqYyLnduzm9aROnN21i24wZNBo4kIiYGBr27o21Gk97Is5PxUdExMnYDBtLdy1l8urJnMs/B8B9be9j/rD5NK7ZuMq+r3ft2rT43e9o8bvfcSU93T5TdO6JE5xMSeFkSgpetWoROnw44bGx1OvYUYOi5ZbToS65KTrUJVK9bc/czujk0WzN2ApAm8A2LI5azKCIQb/4WFuJjQNfHACg9V2t7adIV4RhGFzcv58T/x4UffX8efu6GiEh9kHRNSMiKvy9xHlpjE85qfhUnIqPSPV0Lv8cU1dP5d2d72Jg4O/pz/QB0xnTYwwebtXj0JKtpIQz337LicRETq1cSUlBgX1dnbZtCY+JISwqSoOi5SdUfMpJxafiVHxEqpdSWynv7nyXqWumcqHgAgAPdHiAuUPm0tC/ocnprq8kP59Ta9eSmpRE1saNGKVlE/T9Z1B0RGwsjQcP1qBoAVR8yk3Fp+JUfESqjy2ntjA6eTQ7s3YC0KF+B+Kj4ukb1rdcz1cVh7puxNULFzi5bBknEhM5v+e/1+Zy8/Gh8cCBhMfG0vC22zQo2oWp+JSTik/FqfiImO9s3lkmr5rM0t1LAQjwCmDGwBk82f1J3K3lP2elPKezV7bLaWllg6KTkricmmpf7lWnDmH/HhRdt317DYp2MTqdXUTEBZXYSnhz25tMWzuNnMKyq7c/3OlhZg2eRf0a9U1OVzn8w8Jo/9RTtHvySS7s28eJxEROpqRw9fx5Dv/97xz++9/LBkXHxpYNig4LMzuyOBgVHxERB/B12tfEpcTx3ZnvAOjSsAvxUfHcFnKbycmqhsVioW779tRt354uzz7L6c2bSU1KIn3VKq6kp7Pvj39k3x//SN327e2Dor3r1jU7tjgAFR8RkWos63IWz616jg+/+xCA2t61mTl4JqO6jMLN6mZyulvD6u5OcN++BPftS3FeXtmg6MRETm/axPm9ezm/dy87586lwe23Ex4TQ8igQbj7+podW6opFR8RkWqouLSYxVsXM33ddC4XXcaChVFdRvHq4Fep51vP7Him8fDzIyImhoiYGK6eP0/asmWkJiRwfu9esr7+mqyvv2abjw+NBw8mPCaGBrfdhtVdH3XyX/ppEBGpZtaeWEtcShz7s8tOJOjRqAfxUfF0b9Td5GTVi3fdurS8/35a3n8/uWlppCYmkpqQUDZrdGIiqYmJeNetS2hUFOHR0RoULYCKj4hItXEq9xQTV0zkk+8/AaCebz1mD57Nw50fxmrRGZQ/p2ZYGB1Gj6b9U09xfu9eUhMSSFu2rGxQ9IcfcvjDD/EPCyM8JobwmBj8Q0PNjiwm0ensclN0OrtI5SsqLWLB5gXM2DCDvOI8rBYrT3Z7kpcHvkwdnzq3LEdpcSl7/7YXgPb3t8fNw7HHENmKi8navJnUxEROrV5N6dWr9nV1O3YkPDq6bFB0nVv3Gkv5aB6fclLxqTgVH5HKteLYCsakjOHw+cMA3B5yO/FR8XRu2NnkZM6lOC+PU2vWkJqQwOnNmzFsNgAsbm407N2b8JgYGg8ahLuPj8lJ5VpUfMpJxafiVHxEKkfapTQmrJjAPw/8E4D6fvWZO3QuD3Z4UONQqlhBdrZ9UPSF77+3L3f38aHx0KFExMRQv2dPDYquRlR8yknFp+JUfEQq5mrJVeZvms/Mr2dSUFKAm8WNMT3GMH3AdAK8A0zNZiuxcXT5UQCaRTa7ZZesMFPuiRNlA6GTkriSnm5f7l23LmEjRhAeE0Odtm1VRk2m4lNOKj4Vp+IjUn5Jh5MYu2wsxy4eA6BfWD/io+JpX7+9ycnKVIdLVpjFMAzO7dlD6r9nii68dMm+rmZERNmg6OhoaoSEmBfShemSFSIiDuT4xeOMWzaOhMMJAAT7BzN/6Hx+2+632pNQTVgsFgI7dSKwUye6TppE1jfflA2KXrOG3BMn+G7xYr5bvJh6nToRHhND6PDheNeubXZsKQcVHxGRKlJQXMDsjbOZ880cCksLcbe6M77XeKb1m4a/l7/Z8eQ6rB4eNBowgEYDBlCcl0f6qlWkJiZyZssWzu3ezbndu9kxezYNe/cmIjaWRgMGaFC0A1HxERGpZIZh8NWhrxi3fBypl1IBGBwxmMVRi2kd2NrccHJTPPz8aHLHHTS5446yQdHJyZxITOTi/v1krl9P5vr1uPv6EjJ0KBGxsQT16IHVzbGnAXB2Kj4iIpXoyPkjPL3saZYdXQZASM0QXo98nd+0/o0Oazk4n8BAWj30EK0eeoicY8fsg6LzMjI48eWXnPjyS3wCA+2Domu3bq3/82pIxUdEpBLkFeUx8+uZzN88n6LSIjysHky8fSJT+07Fz9PP7HhSyQKaNqXj2LF0ePppzu3aRWpiImnLllGQnc3B99/n4PvvU7NJk/8Oim7c2OzI8m8qPiIiFWAYBp8f+JwJyyeQnlt2OvTwZsN5Y/gbtKjbwuR0UtUsFguBXboQ2KULXSZPJmvjRlKTkshYu5bc48f5btEivlu0iMDOnQmPjSU0MhKvWrXMju3SdDq73BSdzi7yXweyD/D0sqdZdXwVAOG1wlkYuZBftfyVQx7iKC0uZcc7OwDo+nhXh79khZmKr1whfeVKTiQmcubbb+HfH7VWd3ca9u1LRGwswf374+7tbXJSx6B5fMpJxafiVHxE4HLhZV5e/zILv11Iia0ELzcvJveZzKTek/Dx0Nk98mP5Z86QlpJCamIiFw8csC/3qFGDkKFDCY+JIah7dw2K/hkqPuWk4lNxKj7iygzD4ON9HzNx5UQyL2cCENsiloXDF9KkdhOT04kjuHT0aNl4oKQk8jIz7ct9goIIj44mPDqaWq1aOeQew6rkcsUnNTWVGTNmsGbNGk6fPk1wcDAPPPAAU6dOxdPzxmcWVfGpOBUfcVX7zu4jLjmO9WnrAWhauylvDH+D6BbRJierPLZSGye/PglAaN9QrG76/a4qhs1G9r8HRZ9ctoyi3Fz7uoBmzeyDov2Cg01MWX243MzNBw8exGaz8fbbb9OsWTP27dvHqFGjyMvLY/78+WbHExEnlnM1hxfXvUj81nhKjVJ83H2Y2ncqz9z+DN7uzjU+o+RqCe8PfB9wvUtW3GoWq5Wgrl0J6tqVrs8/T9bGjZxISCBj3Tpyjh5lz8KF7Fm4kKBu3QiPjiY0MhLPAHOv5eYsHGKPz7XMmzePN998k+PHj9/wY7THp+K0x0dchc2w8eF3H/Lcyuc4k3cGgF+3/jWvD3udsFphJqerGq58ra7qoujyZdJXriQ1IYEz27b9d1C0hwfB/foRHhNDo/79cfPyMjnpreVye3yuJScnhzp16vzsNoWFhRQWFtrv5/5gV6KIyPXsPr2b0cmj2ZS+CYCWdVuyKGoRw5oOMzmZODtPf3+a/vrXNP31r8k/fZrU5GRSExK4dPgwp1av5tTq1Xj4+xM6bBjh0dEEde+ORX+A3hSHLD5Hjx5l8eLFv3iYa9asWbz00ku3KJWIOLqLBReZtnYab25/E5thw8/Dj2n9pjH+tvF4umnvh9xavg0a0OaRR2jzyCNcOnzYPlN0/unTHPv8c459/jm+DRr8d6boli3NjuwQTD3UNXnyZObMmfOz2xw4cIBWrVrZ72dkZNC/f38GDBjAn/70p5997LX2+ISEhOhQVwXoUJc4I5thY+mupUxePZlz+ecAuK/tfcwfNp/GNV1nxl0d6qr+DJuNszt2lA2KXrGC4h8cyajVogXh0dGERUfj17ChiSkrn9Oc1ZWdnc358+d/dpsmTZrYz9zKzMxkwIAB9OrVi/fee++mP3Q1xqfiVHzE2WzP3M7o5NFszdgKQJvANsRHxTMwYqDJyW49FR/HUlpUROaGDaQmJJCxfj224uKyFRZL2aDomBhChw3D0wk+75xmjE9gYCCBgYE3tG1GRgYDBw6ka9euLF26VB+4IlIh5/LPMWX1FP60808YGPh7+vPSgJeI6xGHh5uH2fFEfpGbpychQ4YQMmQIRTk5nFy5ktTERM5u22b/2v7KKzTq35/wmBiC+/fH7SamgHFWDnFWV0ZGBgMGDCAsLIz3338ftx/MbtmgQYMbfh7t8ak47fERR1dqK+Xdne8ydc1ULhRcAOCBDg8wd8hcGvo71+GBm1VaVMqWN7YA0GtsL9w8NZOwI8rLzCQtOZkTiYnkHDliX+5Rs2bZoOiYGIK6dnWoQdFOc6jrRr333ns8/PDD11x3M/FVfCpOxUcc2eb0zcSlxLEzaycAHep3ID4qnr5hfU1OJlI1Lh46ZB8UXXDmjH25b4MGZTNFx8ZSq3lzExPeGJcrPpVFxafiVHzEEZ3NO8vkVZNZunspAAFeAbwy6BWe6PYE7laHPLlV5KYYNhtnt28vGxS9fDnFV67Y19Vq2bJspugRI/C9iaMot5KKTzmp+FScio84khJbCW9ue5Npa6eRU5gDwMOdHmb2kNkE+QWZnK76sZXayNqZBUDDLg11yQonVVpYSMb69aQmJpK5fj22kpKyFRYL9bt3Jzw2lpChQ/H09zc36A+o+JSTik/FqfiIo/g67WviUuL47sx3AHRp2IUlI5bQq3Evk5NVXzqry/UUXrpE+ooVnEhMJHvHDvtyq6cnjQYMICI2loZ9+pg+KNppzuoSEalsWZezeG7Vc3z43YcA1PauzczBMxnVZRRuVg3WFfkhr1q1aHbvvTS7917yMjNJTUoiNSGBnGPHSF+xgvQVK/CsWZPQ4cMJj4khsHNnhxoUfS0qPiLiFIpLi1m8dTHT103nctFlLFgY1WUUrw5+lXq+9cyOJ1Lt+QUH03bUKNo89hiXDh4sK0FJSRScPcvRTz/l6Kef4hccTFh0NBExMQQ0a2Z25HJR8RERh7f2xFriUuLYn112GLZHox4sGbGEbsHdTE4m4ngsFgu1W7emduvWdBw/nrPbttlnis7LzGT/u++y/913qd2qFeGxsYSNGIFvkOOMmVPxERGHdSr3FBNXTOST7z8BoJ5vPWYPns3DnR/GanHs3fEi1YHVzY0GvXrRoFcvuv3hD2SuX8+JhASyvv6aiwcPcvHgQXbNn0/9nj2JiIkhZOhQPGrUMDv2z1LxERGHU1RaxILNC5ixYQZ5xXlYLVae7PYkMwbOoLZPbbPjiTgld29vQiMjCY2MpPDSJU4uX05qYiLZO3dyZssWzmzZwrYZM2g0cCDh0dHVYlD0taj4iIhDWXFsBWNSxnD4/GEAbg+5nSUjltCpQSdzg4m4EK9atWh+3300v+8+rmRkkJaUxImEBHKPH+fksmWcXLYMz4AAQocPJyImhnqdO2OxWMyODeh0drlJOp1dzJJ2KY0JKybwzwP/BKC+X33mDp3Lgx0erDZvqI6utKiUr2d+DUDfKX11yQq5KYZhcPHAAftM0VfPnbOv82vcmPARIwiPjSWgSZObfm7N41NOKj4Vp+Ijt9rVkqvM3zSfmV/PpKCkADeLG2N6jGH6gOkEeAeYHU9ErsFWWsrZrVs5kZBA+sqVlOTn29fVbtOGiJgYwkaMwOcGL1Su4lNOKj4Vp+Ijt1LS4STGLhvLsYvHAOgf1p/FUYtpX7+9yclE5EaVFBSQsW5d2aDob77B+PdM0Rarlfq9ehEeE0PIkCF4+Pld9zlUfMpJxafiVHzkVjh+8Tjjlo0j4XACAMH+wcwfOp/ftvutDmtVIcNmkH0gG4DA1oFYrHqtpXJdvXiRk8uWkZqYyLndu+3L3by9aTxwIOExMTTs3Rurh8ePHqfiU04qPhWn4iNVqaC4gNkbZzPnmzkUlhbibnVnfK/xTOs3DX+v6nPdIGelS1bIrXT55ElSk5JIS0oi98QJ+3Kv2rUJjYwkPDaWeh07YrFYdMkKEXEuhmHw5aEvGb98PKmXUgEY0mQIi6MW06peK3PDiUiV8A8Npf2TT9LuiSe48P33pCYmkpaczNXz5zny8ccc+fhjaoSEEB4dTZ3+/Svt+6r4iIipjpw/wtPLnmbZ0WUAhNQM4fXI1/lN69/osJaIC7BYLNRt14667drReeJEznz7LScSEzm1ciVX0tPZ99Zb5C9ZUmnfT8VHREyRV5THq1+/ymubX6OotAhPN08m3jaRKX2n4Od5/UGOIuK8rO7uNOzdm4a9e1MybRqn1q4lNTGRYxs2VNr3UPERkVvKMAw+2/8ZE1ZM4FTuKQCGNxvOouGLaF63ucnpRKS6cPf1JTw6mvDoaNqkpkJEROU8b6U8i4jIDTiQfYAxKWNYfWI1AOG1wlkYuZBftfyVDmuJyHV516lTac+l4iMiVe5y4WVeXv8yC79dSImtBC83Lyb3mcyk3pPw8fAxO56IuBAVHxGpMoZh8NG+j5i4YiJZV7IA+FXLX7EgcgFNat/8tPVStdw83Lht4m322yLOSMVHRKrEvrP7iEuOY33aegCa1m7KoqhFjGg+wuRkcj1unm4MmzfM7BgiVUrFR0QqVc7VHF5c9yLxW+MpNUrxcfdhat+pPHP7M3i7e5sdT0RcnIqPiFQKm2Hjgz0f8Nyq5zibdxaA37T+Da8Ne42wWmEmp5MbYdgMck7mABAQGqBLVohTUvERkQrbfXo3o5NHsyl9EwAt67ZkUdQihjXVYRNHUlxQzBsRbwC6ZIU4LxUfESm3iwUX+cOaP/DWjrewGTb8PPx4of8LjOs1Dk83fWiKSPWj4iMiN81m2PjLrr/w/OrnOZd/DoD72t7H/GHzaVyzscnpRESuT8VHRG7K9sztjE4ezdaMrQC0CWxDfFQ8AyMGmpxMROSXqfiIyA05l3+OKaun8Kedf8LAwN/Tn5cGvERcjzg83DzMjicickNUfETkZ5XaSnl357tMXTOVCwUXAHiww4PMGTKHhv4NTU4nInJzVHxE5Lo2p28mLiWOnVk7AehQvwNLRiyhT2gfk5OJiJSPio+I/MTZvLNMXjWZpbuXAhDgFcArg17hiW5P4G7V24azsrpb6fZUN/ttEWekdzARsSuxlfDmtjeZtnYaOYVlE9k90ukRZg2ZRZBfkMnppKq5e7kTvSTa7BgiVUrFR0QA+Drta+JS4vjuzHcAdGnYhSUjltCrcS+Tk4mIVB4VHxEXl3U5i2dXPsvf9v4NgDo+dZg5aCaPdXkMN6uu0O1KDMMg/1w+AL71fLFYdMkKcT4qPiIuqri0mMVbFzN93XQuF13GgoVRXUYxc/BM6vrWNTuemKA4v5j5QfMBXbJCnJeKj4gLWntiLXEpcezP3g9Az0Y9iR8RT7fgbiYnExGpWio+Ii7kVO4pnlnxDJ9+/ykA9XzrMWfIHEZ2GonVorN4RMT5qfiIuICi0iIWbF7AjA0zyCvOw2qx8lS3p3h54MvU9qltdjwRkVtGxUfEya04toIxKWM4fP4wAL1DehM/Ip5ODTqZG0xExAQqPiJOKu1SGhNWTOCfB/4JQH2/+swdOpcHOzyos3VExGWp+Ig4maslV5m/aT4zv55JQUkBbhY3xvQYw/QB0wnwDjA7noiIqVR8RJxI0uEkxi4by7GLxwDoH9af+BHxtAtqZ3IycQRWdysdH+povy3ijFR8RJzA8YvHGbtsLImHEwEI9g/mtWGvcV/b+3RYS26Yu5c7d753p9kxRKqUio+IAysoLmD2xtnM+WYOhaWFuFvdGd9rPNP6TcPfy9/seCIi1Y6Kj4gDMgyDLw99yfjl40m9lArAkCZDWBy1mFb1WpkbThyWYRgU5xcD4OHrob2F4pRUfEQczOHzhxm7bCzLji4DIKRmCAsiF/Dr1r/WB5VUSHF+MbNqzAJ0yQpxXio+Ig4iryiPV79+ldc2v0ZRaRGebp5MvG0iU/pOwc/Tz+x4IiIOQcVHpJozDIPP9n/GhBUTOJV7CoCoZlG8MfwNmtdtbnI6ERHHouIjUo0dyD7AmJQxrD6xGoDwWuEsjFzIr1r+Soe1RETKQcVHpBq6XHiZl9e/zMJvF1JiK8HLzYvJfSYzqfckfDx8zI4nIuKwVHxEqhHDMPho30dMXDGRrCtZAPyq5a9YELmAJrWbmJxORMTxqfiIVBN7z+wlLiWODWkbAGhauymLohYxovkIk5OJiDgPFR8Rk+VczeHFdS8SvzWeUqMUH3cfpvadyjO3P4O3u7fZ8cSFWN2stLm7jf22iDNS8RExic2w8cGeD3hu1XOczTsLwG9a/4bXI18nNCDU5HTiity93bnnH/eYHUOkSqn4iJhgV9Yu4lLi2JS+CYCWdVuyOGoxQ5sONTmZiIhzU/ERuYUuFFxg2pppvLXjLWyGDT8PP17o/wLjeo3D002z5IqIVDUVH5FbwGbY+Muuv/D86uc5l38OgN+2+y3zhs6jcc3GJqcTKVOUV6RLVojTU/ERqWLbM7czOnk0WzO2AtAmsA3xUfEMjBhocjIREdej4iNSRc7ln2PK6in8aeefMDDw9/TnpQEvEdcjDg83D7PjiYi4JBUfkUpWaivlnR3vMHXNVC5evQjAgx0eZO7QuTSo0cDkdCIirk3FR6QSbU7fTFxKHDuzdgLQoX4HloxYQp/QPiYnExERUPERqRRn884yadUk3tv9HgABXgG8MugVnuj2BO5W/ZqJiFQXekcWqYASWwl/3PZHXlj7AjmFOQA80ukRZg2ZRZBfkMnpRETkfzlc8SksLKRnz57s2bOHXbt20alTJ7MjiYvakLaBuOQ49p7dC0CXhl1YMmIJvRr3MjmZSPlY3aw0H9HcflvEGTlc8XnuuecIDg5mz549ZkcRF5V1OYtnVz7L3/b+DYA6PnWYOWgmj3V5DDerm8npRMrP3dud3yf93uwYIlXKoYpPSkoKK1as4PPPPyclJcXsOOJiikuLWfTtIqavn86VoitYsPB418d5ddCr1PWta3Y8ERG5AQ5TfM6cOcOoUaP417/+ha+v7w09prCwkMLCQvv93NzcqoonTm7tibXEpcSxP3s/AD0b9SR+RDzdgruZnExERG6GQxzENQyDkSNH8sQTT9Ct241/0MyaNYuAgAD7V0hISBWmFGd0KvcU9312H4P+Ooj92fup51uPP//qz2x6dJNKjzidorwiZvrNZKbfTIryisyOI1IlTC0+kydPxmKx/OzXwYMHWbx4MZcvX+b555+/qed//vnnycnJsX+lp6dX0b9EnE1RaRGzN86mZXxLPv3+U6wWK3Hd4zgcd5hHOj+C1eIQfzOI3LTi/GKK84vNjiFSZUw91PXMM88wcuTIn92mSZMmrFmzhs2bN+Pl5fWjdd26deP+++/n/fffv+Zjvby8fvIYkV+y4tgKxqSM4fD5wwD0DulN/Ih4OjXoZG4wERGpMFOLT2BgIIGBgb+43aJFi3jllVfs9zMzM4mMjOSTTz6hZ8+eVRlRXEjapTTGLx/PFwe/AKC+X33mDZ3HAx0ewGKxmJxOREQqg0MMbg4NDf3R/Ro1agDQtGlTGjdubEYkcSJXS64y75t5zNo4i4KSAtwsbozpMYbpA6YT4B1gdjwREalEDlF8RKpK0uEkxi4by7GLxwDoH9af+BHxtAtqZ3IyERGpCg5ZfMLDwzEMw+wY4sCOXzzO2GVjSTycCECwfzCvDXuN+9rep8NaIiJOzCGLj0h55RfnM2fjHOZ8M4fC0kLcre5M6DWBP/T7A/5e/mbHEzGVxWohrH+Y/baIM1LxEZdgGAZfHvqSccvGkZaTBsCQJkNYHLWYVvVamZxOpHrw8PFg5LqRZscQqVIqPuL0Dp8/zNhlY1l2dBkAITVDWBC5gF+3/rUOa4mIuBgVH3FaeUV5vLLhFV7b/BrFtmI83Tx59vZneb7P8/h5+pkdT0RETKDiI07HMAw+2/8ZE1ZM4FTuKQCimkXxxvA3aF63ucnpRKqvorwi3gh/A4CxqWPx9PM0OZFI5VPxEadyIPsAY1LGsPrEagDCa4XzxvA3iG0Rq8NaIjcg/1y+2RFEqpSKjziFy4WXeXn9yyz8diElthK83LyY3Gcyk3pPwsfDx+x4IiJSTaj4iEMzDIOP9n3ExBUTybqSBcCvWv6KBZELaFK7icnpRESkulHxEYe198xe4lLi2JC2AYBmdZrxxvA3GNF8hMnJRESkulLxEYdz6eolpq+bTvzWeEqNUnzcffhDvz8w4bYJeLt7mx1PRESqMRUfcRg2w8YHez7guVXPcTbvLAC/af0bXo98ndCA0F94tIiIiIqPOIhdWbuIS4ljU/omAFrWbcniqMUMbTrU5GQizsNitRDcLdh+W8QZqfhItXah4ALT1kzjrR1vYTNs+Hn48UL/FxjXaxyebppjRKQyefh4MGrbKLNjiFQpFR+plmyGjb/s+gvPr36ec/nnAPhtu98yf+h8GtVsZHI6ERFxVCo+Uu1sy9jG6OTRbMvcBkDbwLbEj4hnQPgAc4OJiIjDU/GRauNc/jmmrJ7Cn3b+CQMDf09/XhrwEnE94vBw8zA7nojTK84vZkmbJQCM3j8aD1/93onzUfER05XaSnlnxztMXTOVi1cvAvBghweZO3QuDWo0MDmdiOswDIOctBz7bRFnpOIjptqcvpnRyaPZdXoXAB3rdyR+RDx9QvuYnExERJyRio+Y4syVM0xePZn3dr8HQC3vWrwy8BX+r9v/4W7Vj6WIiFQNfcLILVViK+GP2/7IC2tfIKewbJf6I50eYdaQWQT5BZmcTkREnJ2Kj9wyG9I2EJccx96zewHo0rALS0YsoVfjXiYnExERV6HiI1Uu83Imz618jr/t/RsAdXzqMHPQTB7r8hhuVjeT04mIiCtR8ZEqU1xazKJvFzF9/XSuFF3BgoXHuz7Oq4Nepa5vXbPjicj/sFgsBLYJtN8WcUYqPlIl1pxYQ1xyHAfOHQCgZ6OexI+Ip1twN5OTicj1ePh68NT3T5kdQ6RKqfhIpTqVe4pnVjzDp99/CkA933rMGTKHkZ1GYrVYTU4nIiKuTsVHKkVhSSELtixgxoYZ5BfnY7VYearbU7w88GVq+9Q2O56IiAig4iOVYPnR5Ty97GkOnz8MQJ/QPsRHxdOxQUeTk4nIzSjOL+bd7u8CMGrbKF2yQpySio+UW+qlVCaunMgXB78AoL5ffeYNnccDHR7QwEgRB2QYBtn7s+23RZyRio/ctMLSQpYeXsqfv/wzV0uu4mZx4+meT/Ni/xcJ8A4wO56IiMh1qfjITUk6ksTolaNJz0sHoH9Yf+JHxNMuqJ3JyURERH6Zio/ckGMXjjFu+TgSDycCEOQdxIKoBfyu/e90WEtERByGio/8rPzifGZvnM3cb+ZSWFqIu9WdB5s9yBOtn6Bbu24qPSIi4lBUfOSaDMPgy0NfMm7ZONJy0gAY0mQIb0S+ge2szeR0IiIi5aPiIz9x+Pxhnk55muXHlgMQUjOEBZEL+HXrX2MYBvvP7jc5oYhUBYvFQkBYgP22iDNS8RG7vKI8XtnwCq9tfo1iWzGebp48e/uzPN/nefw8/QCd4irizDx8PRiXOs7sGCJVSsVHMAyDf+z/B8+seIZTuacAiGoWxRvD36B53eYmpxMREak8Kj4u7kD2AcakjGH1idUAhNcK543hbxDbIla7ukVExOmo+Lioy4WXeXn9yyz8diElthK83b2Z3Hsyz/V+Dh8PH7PjiYgJiguKea/fewCM3DASDx9dskKcj4qPizEMg4/2fcTEFRPJupIFwK9a/oqFkQuJqB1hcjoRMZNhM8jcnmm/LeKMVHxcyN4ze4lLiWND2gYAmtVpxhvD32BE8xEmJxMREbk1VHxcwKWrl3hx7Yss2baEUqMUH3cf/tDvDzxz2zN4uXuZHU9EROSWUfFxYjbDxgd7PuC5Vc9xNu8sAHe3uZvXhr1GaECoyelERERuPRUfJ7Uraxejk0ez+dRmAFrWbcniqMUMbTrU5GQiIiLmUfFxMhcKLvCHNX/g7R1vYzNs+Hn48WL/Fxnbayyebp5mxxMRETGVio+TsBk2/rzzzzy/+nnOF5wH4Hftfse8ofNoVLORyelExFH41vM1O4JIlVLxcQLbMrYxOnk02zK3AdA2sC3xI+IZED7A3GAi4lA8/Tx5NvtZs2OIVCkVHwd2Lv8cU1ZP4U87/4SBQU2vmrw04CVGdx+Nh5smHhMREflfKj4OqNRWyjs73mHqmqlcvHoRgAc7PMjcoXNpUKOByelERESqLxUfB7M5fTOjk0ez6/QuADrW70j8iHj6hPYxOZmIOLrigmL+FvU3AO5PuV+XrBCnpOLjIM5cOcOkVZN4f8/7ANTyrsUrA1/h/7r9H+5W/TeKSMUZNoO09Wn22yLOSJ+Y1VyJrYQ/bvsjL6x9gZzCHAAe7fwoMwfPJMgvyOR0IiIijkXFpxrbkLaBuOQ49p7dC0DXhl1ZMmIJPRv3NDmZiIiIY1LxqYYyL2fy3Mrn+NvesmPtdXzqMHPQTB7r8hhuVjeT04mIiDguFZ9qpLi0mEXfLmL6+ulcKbqCBQuPd32cVwe9Sl3fumbHExERcXgqPtXEmhNriEuO48C5AwD0bNSTJSOW0DW4q8nJREREnIeKj8nSc9KZuHIin37/KQCBvoHMGTKHhzo9hNViNTmdiLgaD1+dwi7OTcXHJIUlhSzYsoAZG2aQX5yP1WLlqW5P8fLAl6ntU9vseCLigjz9PJmSN8XsGCJVSsXHBMuPLufpZU9z+PxhAPqE9iE+Kp6ODTqanExERMS5qfjcQqmXUhm/fDz/OvgvABrUaMC8ofO4v/39WCwWc8OJiIi4ABWfW+BqyVXmfTOPmRtncrXkKm4WN57u+TTTB0ynpldNs+OJiABQcrWET39TNt7w3s/vxd1bHxHifPRTXcUSDycydtlYjl88DsCA8AHER8XTNqityclERH7MVmrjSPIR+20RZ6TiU0WOXTjGuOXjSDycCECwfzCvDXuN+9rep8NaIiIiJlHxqWT5xfnM3jibud/MpbC0EA+rB+N7jWda/2nU8KxhdjwRERGX5lATxSQlJdGzZ098fHyoXbs2d955p9mR7AzD4IsDX9BmSRtmbJhBYWkhQ5sM5bsnv2PO0DkqPSIiItWAw+zx+fzzzxk1ahQzZ85k0KBBlJSUsG/fPrNjAXD4/GGeTnma5ceWAxAaEMqCyAXc1eouHdYSERGpRhyi+JSUlDB27FjmzZvHo48+al/epk0bE1NBXlEer2x4hdc2v0axrRhPN0+evf1ZpvSdgq+Hr6nZRERE5Kccovjs3LmTjIwMrFYrnTt35vTp03Tq1Il58+bRrl276z6usLCQwsJC+/2cnBwAcnNzK5THMAy+OPgFU9dMJTM3E4AhTYYwd+hcmtZpSklBCbkFFfse1ZXNZuPKlStA2etotTrU0VIR+RlFeUVc5SpQ9vvtWeppciKRMv/53DYMo+JPZjiAjz76yACM0NBQ47PPPjO2b99u/O53vzPq1q1rnD9//rqPe/HFFw1AX/rSl770pS99OcHXsWPHKtwpLIZRGfWpfCZPnsycOXN+dpsDBw6wc+dO7r//ft5++20ef/xxoGxvTuPGjXnllVf4v//7v2s+9n/3+Fy6dImwsDBOnjxJQEBA5f1DXExubi4hISGkp6dTs6YmYKwIvZaVR69l5dDrWHn0WlaenJwcQkNDuXjxIrVq1arQc5l6qOuZZ55h5MiRP7tNkyZNyMrKAn48psfLy4smTZpw8uTJ6z7Wy8sLLy+vnywPCAjQD2ElqFmzpl7HSqLXsvLotawceh0rj17LylMZwytMLT6BgYEEBgb+4nZdu3bFy8uLQ4cO0adPHwCKi4tJTU0lLCysqmOKiIiIk3CIwc01a9bkiSee4MUXXyQkJISwsDDmzZsHwD333GNyOhEREXEUDlF8AObNm4e7uzsPPvggBQUF9OzZkzVr1lC7du0bfg4vLy9efPHFax7+khun17Hy6LWsPHotK4dex8qj17LyVOZraergZhEREZFbSZOwiIiIiMtQ8RERERGXoeIjIiIiLkPFR0RERFyGSxaf1NRUHn30USIiIvDx8aFp06a8+OKLFBUVmR3NISxZsoTw8HC8vb3p2bMnW7duNTuSw5k1axbdu3fH39+foKAg7rzzTg4dOmR2LIc3e/ZsLBYL48aNMzuKQ8rIyOCBBx6gbt26+Pj40L59e7Zv3252LIdTWlrKtGnTfvQZM2PGjMq5zpST27BhA7GxsQQHB2OxWPjXv/71o/WGYfDCCy/QsGFDfHx8GDJkCEeOHLmp7+GSxefgwYPYbDbefvttvv/+exYsWMBbb73FlClTzI5W7X3yySdMmDCBF198kZ07d9KxY0ciIyM5e/as2dEcyvr16xk9ejRbtmxh5cqVFBcXM2zYMPLy8syO5rC2bdvG22+/TYcOHcyO4pAuXrxI79698fDwICUlhf379/Paa6/d1JQhUmbOnDm8+eabxMfHc+DAAebMmcPcuXNZvHix2dGqvby8PDp27MiSJUuuuX7u3LksWrSIt956i2+//RY/Pz8iIyO5evXqjX+TCl/ty0nMnTvXiIiIMDtGtdejRw9j9OjR9vulpaVGcHCwMWvWLBNTOb6zZ88agLF+/Xqzoziky5cvG82bNzdWrlxp9O/f3xg7dqzZkRzOpEmTjD59+pgdwylER0cbjzzyyI+W/frXvzbuv/9+kxI5JsD44osv7PdtNpvRoEEDY968efZlly5dMry8vIyPPvrohp/XJff4XEtOTg516tQxO0a1VlRUxI4dOxgyZIh9mdVqZciQIWzevNnEZI4vJycHQD+D5TR69Giio6N/9LMpN+err76iW7du3HPPPQQFBdG5c2feffdds2M5pNtvv53Vq1dz+PBhAPbs2cPGjRuJiooyOZljO3HiBKdPn/7R73lAQAA9e/a8qc8gh5m5uSodPXqUxYsXM3/+fLOjVGvnzp2jtLSU+vXr/2h5/fr1OXjwoEmpHJ/NZmPcuHH07t2bdu3amR3H4Xz88cfs3LmTbdu2mR3FoR0/fpw333yTCRMmMGXKFLZt28bTTz+Np6cnDz30kNnxHMrkyZPJzc2lVatWuLm5UVpayquvvsr9999vdjSHdvr0aYBrfgb9Z92NcKo9PpMnT8Zisfzs1/9+QGdkZDB8+HDuueceRo0aZVJycWWjR49m3759fPzxx2ZHcTjp6emMHTuWv/3tb3h7e5sdx6HZbDa6dOnCzJkz6dy5M48//jijRo3irbfeMjuaw/n000/529/+xt///nd27tzJ+++/z/z583n//ffNjiY42R6fZ555hpEjR/7sNk2aNLHfzszMZODAgdx+++288847VZzO8dWrVw83NzfOnDnzo+VnzpyhQYMGJqVybHFxcSQmJrJhwwYaN25sdhyHs2PHDs6ePUuXLl3sy0pLS9mwYQPx8fEUFhbi5uZmYkLH0bBhQ9q0afOjZa1bt+bzzz83KZHjevbZZ5k8eTK//e1vAWjfvj1paWnMmjVLe88q4D+fM2fOnKFhw4b25WfOnKFTp043/DxOVXwCAwMJDAy8oW0zMjIYOHAgXbt2ZenSpVitTrXzq0p4enrStWtXVq9ezZ133gmU/ZW4evVq4uLizA3nYAzDYMyYMXzxxResW7eOiIgIsyM5pMGDB7N3794fLXv44Ydp1aoVkyZNUum5Cb179/7JlAqHDx8mLCzMpESOKz8//yefKW5ubthsNpMSOYeIiAgaNGjA6tWr7UUnNzeXb7/9lieffPKGn8epis+NysjIYMCAAYSFhTF//nyys7Pt67Tn4udNmDCBhx56iG7dutGjRw8WLlxIXl4eDz/8sNnRHMro0aP5+9//zpdffom/v7/9+HRAQAA+Pj4mp3Mc/v7+PxkX5efnR926dTVe6iaNHz+e22+/nZkzZ3LvvfeydetW3nnnHe0NL4fY2FheffVVQkNDadu2Lbt27eL111/nkUceMTtatXflyhWOHj1qv3/ixAl2795NnTp1CA0NZdy4cbzyyis0b96ciIgIpk2bRnBwsP2P8RtSiWeeOYylS5cawDW/5JctXrzYCA0NNTw9PY0ePXoYW7ZsMTuSw7nez9/SpUvNjubwdDp7+SUkJBjt2rUzvLy8jFatWhnvvPOO2ZEcUm5urjF27FgjNDTU8Pb2Npo0aWJMnTrVKCwsNDtatbd27dprvjc+9NBDhmGUndI+bdo0o379+oaXl5cxePBg49ChQzf1PSyGoakkRURExDVoYIuIiIi4DBUfERERcRkqPiIiIuIyVHxERETEZaj4iIiIiMtQ8RERERGXoeIjIiIiLkPFR0RERFyGio+IiIi4DBUfERERcRkqPiLisLKzs2nQoAEzZ860L9u0aROenp6sXr3axGQiUl3pWl0i4tCSk5O588472bRpEy1btqRTp07ccccdvP7662ZHE5FqSMVHRBze6NGjWbVqFd26dWPv3r1s27YNLy8vs2OJSDWk4iMiDq+goIB27dqRnp7Ojh07aN++vdmRRKSa0hgfEXF4x44dIzMzE5vNRmpqqtlxRKQa0x4fEXFoRUVF9OjRg06dOtGyZUsWLlzI3r17CQoKMjuaiFRDKj4i4tCeffZZPvvsM/bs2UONGjXo378/AQEBJCYmmh1NRKohHeoSEYe1bt06Fi5cyAcffEDNmjWxWq188MEHfP3117z55ptmxxORakh7fERERMRlaI+PiIiIuAwVHxEREXEZKj4iIiLiMlR8RERExGWo+IiIiIjLUPERERERl6HiIyIiIi5DxUdERERchoqPiIiIuAwVHxEREXEZKj4iIiLiMlR8RERExGX8fxq0rq6KoWGGAAAAAElFTkSuQmCC\n" - }, - "metadata": {} - } - ] - }, - { - "cell_type": "markdown", - "source": [ - "**Return to slides here.**" - ], - "metadata": { - "id": "CezB87Tb-QGh" - } - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bYDhomCP6SMj" - }, - "source": [ - "## Segment 3: Matrix Properties" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "-HGU_an66SMk" - }, - "source": [ - "### Frobenius Norm" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "pNQHvAqN6SMk", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "916cf013-483f-48a2-e66a-dcb8be2be58a" - }, - "source": [ - "X = np.array([[1, 2], [3, 4]])\n", - "X" - ], - "execution_count": 119, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[1, 2],\n", - " [3, 4]])" - ] - }, - "metadata": {}, - "execution_count": 119 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "T-q-Tzn26SMm", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "f13d60e4-6d8c-4628-acda-01a5bf85876f" - }, - "source": [ - "(1**2 + 2**2 + 3**2 + 4**2)**(1/2)" - ], - "execution_count": 120, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "5.477225575051661" - ] - }, - "metadata": {}, - "execution_count": 120 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "YVG8qiFw6SMn", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c0b5b228-8724-4bb2-b1ce-531b359e36c8" - }, - "source": [ - "np.linalg.norm(X) # same function as for vector L2 norm" - ], - "execution_count": 121, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "5.477225575051661" - ] - }, - "metadata": {}, - "execution_count": 121 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "FPnBflKVxyik" - }, - "source": [ - "X_pt = torch.tensor([[1, 2], [3, 4.]]) # torch.norm() supports floats only" - ], - "execution_count": 122, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "NCdTShVyx8z0", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b574d8aa-4700-4f75-959d-2b0501f44bc8" - }, - "source": [ - "torch.norm(X_pt)" - ], - "execution_count": 123, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(5.4772)" - ] - }, - "metadata": {}, - "execution_count": 123 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "blezf9fLx_nD" - }, - "source": [ - "X_tf = tf.Variable([[1, 2], [3, 4.]]) # tf.norm() also supports floats only" - ], - "execution_count": 124, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "LiCQzyf6ySCZ", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "560c2751-1cde-4622-d8f6-4f6661710b89" - }, - "source": [ - "tf.norm(X_tf)" - ], - "execution_count": 125, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 125 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "4c6rjVAf6SMo" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "OLN-MMIe6SMo" - }, - "source": [ - "### Matrix Multiplication (with a Vector)" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "XJw0j8cr6SMo", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "287c7d56-a31a-4990-e7ef-069d8ae3da65" - }, - "source": [ - "A = np.array([[3, 4], [5, 6], [7, 8]])\n", - "A" - ], - "execution_count": 126, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[3, 4],\n", - " [5, 6],\n", - " [7, 8]])" - ] - }, - "metadata": {}, - "execution_count": 126 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "zZQ1Aupc6SMq", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ef242b77-62eb-4ce2-f468-5c9bb93b20cf" - }, - "source": [ - "b = np.array([1, 2])\n", - "b" - ], - "execution_count": 127, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([1, 2])" - ] - }, - "metadata": {}, - "execution_count": 127 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ZbeVtNyW6SMq", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d0e9832a-e2a8-4819-c852-5ff9a427a6a6" - }, - "source": [ - "np.dot(A, b) # even though technically dot products are between vectors only" - ], - "execution_count": 128, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([11, 17, 23])" - ] - }, - "metadata": {}, - "execution_count": 128 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "srVI55X96SMu", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "881f9985-25e3-4589-942c-42afb9ecf7cb" - }, - "source": [ - "A_pt = torch.tensor([[3, 4], [5, 6], [7, 8]])\n", - "A_pt" - ], - "execution_count": 129, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[3, 4],\n", - " [5, 6],\n", - " [7, 8]])" - ] - }, - "metadata": {}, - "execution_count": 129 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "5SDn71Xc6SMv", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "2af7f92d-7799-446c-bad1-e6368df39940" - }, - "source": [ - "b_pt = torch.tensor([1, 2])\n", - "b_pt" - ], - "execution_count": 130, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([1, 2])" - ] - }, - "metadata": {}, - "execution_count": 130 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "OIeoJlsh6SMx", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "0b1008a9-b5e7-4d40-b88e-2bb0c0319f91" - }, - "source": [ - "torch.matmul(A_pt, b_pt) # like np.dot(), automatically infers dims in order to perform dot product, matvec, or matrix multiplication" - ], - "execution_count": 131, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([11, 17, 23])" - ] - }, - "metadata": {}, - "execution_count": 131 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "pnob9GkB6SMs", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "07b3a129-35f1-41b8-b1f9-7c208a4b9277" - }, - "source": [ - "A_tf = tf.Variable([[3, 4], [5, 6], [7, 8]])\n", - "A_tf" - ], - "execution_count": 132, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 132 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "vYtWxf8K6SMt", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "a83bf927-bb8e-4c60-b6da-3cb6cb5edd57" - }, - "source": [ - "b_tf = tf.Variable([1, 2])\n", - "b_tf" - ], - "execution_count": 133, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 133 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "NGBImWRH6SMt", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3d2f4fcd-04d8-4dbb-bee5-f9c2151fdea3" - }, - "source": [ - "tf.linalg.matvec(A_tf, b_tf)" - ], - "execution_count": 134, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 134 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "kzjZmdRR6SMy" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "21ySqay36SM5" - }, - "source": [ - "### Matrix Multiplication (with Two Matrices)" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "0YRG1Ig2cgvo", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "02ab25c2-ca74-4d3a-d845-6716906305a3" - }, - "source": [ - "A" - ], - "execution_count": 135, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[3, 4],\n", - " [5, 6],\n", - " [7, 8]])" - ] - }, - "metadata": {}, - "execution_count": 135 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "DyOEZk_c6SM5", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "bb6d846b-4a97-4b90-a6ca-d4010269a6c9" - }, - "source": [ - "B = np.array([[1, 9], [2, 0]])\n", - "B" - ], - "execution_count": 136, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[1, 9],\n", - " [2, 0]])" - ] - }, - "metadata": {}, - "execution_count": 136 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "SfKuNxH-6SM6", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3c6271ee-c89e-4707-ba0d-96e8a585ebf7" - }, - "source": [ - "np.dot(A, B)" - ], - "execution_count": 137, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[11, 27],\n", - " [17, 45],\n", - " [23, 63]])" - ] - }, - "metadata": {}, - "execution_count": 137 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "WcnQMF0s6SNB" - }, - "source": [ - "Note that matrix multiplication is not \"commutative\" (i.e., $AB \\neq BA$) so uncommenting the following line will throw a size mismatch error:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "_mwBGOXO6SNB" - }, - "source": [ - "# np.dot(B, A)" - ], - "execution_count": 138, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "JrrvPoNE6SM9", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "8e9b7b0e-c56e-4a28-f858-93886de0968d" - }, - "source": [ - "B_pt = torch.from_numpy(B) # much cleaner than TF conversion\n", - "B_pt" - ], - "execution_count": 139, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[1, 9],\n", - " [2, 0]])" - ] - }, - "metadata": {}, - "execution_count": 139 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Z6PfwCvX6SM-", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "37454147-f8c3-41f4-f8ce-6fb3ef8f4524" - }, - "source": [ - "# another neat way to create the same tensor with transposition:\n", - "B_pt = torch.tensor([[1, 2], [9, 0]]).T\n", - "B_pt" - ], - "execution_count": 140, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[1, 9],\n", - " [2, 0]])" - ] - }, - "metadata": {}, - "execution_count": 140 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "16ZNRaVe6SM_", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "c32eb80d-f84c-4670-dead-0f34b0f9498f" - }, - "source": [ - "torch.matmul(A_pt, B_pt) # no need to change functions, unlike in TF" - ], - "execution_count": 141, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[11, 27],\n", - " [17, 45],\n", - " [23, 63]])" - ] - }, - "metadata": {}, - "execution_count": 141 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "rkymNjE46SM8", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "712a076c-160d-4e9e-d57f-29ef716eff08" - }, - "source": [ - "B_tf = tf.convert_to_tensor(B, dtype=tf.int32)\n", - "B_tf" - ], - "execution_count": 142, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 142 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "rslTzFRk6SM8", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d424a4ee-e161-4ddf-e825-e533507d66f5" - }, - "source": [ - "tf.matmul(A_tf, B_tf)" - ], - "execution_count": 143, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 143 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "0eBiTmPp6SNC" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "L2H9F-DQ6SMz" - }, - "source": [ - "### Symmetric Matrices" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "5YsPoWo76SMz", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d41b1efa-ec4e-4fde-82fb-c3a98ddc3d49" - }, - "source": [ - "X_sym = np.array([[0, 1, 2], [1, 7, 8], [2, 8, 9]])\n", - "X_sym" - ], - "execution_count": 144, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[0, 1, 2],\n", - " [1, 7, 8],\n", - " [2, 8, 9]])" - ] - }, - "metadata": {}, - "execution_count": 144 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Skg1wSQVcgv2", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "dde3fe41-cc5b-4647-ffaf-5a207a82b644" - }, - "source": [ - "X_sym.T" - ], - "execution_count": 145, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[0, 1, 2],\n", - " [1, 7, 8],\n", - " [2, 8, 9]])" - ] - }, - "metadata": {}, - "execution_count": 145 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Jv40-i9H6SM1", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b23c92fa-c452-4a36-ca27-92fd1dcd64bc" - }, - "source": [ - "X_sym.T == X_sym" - ], - "execution_count": 146, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[ True, True, True],\n", - " [ True, True, True],\n", - " [ True, True, True]])" - ] - }, - "metadata": {}, - "execution_count": 146 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "QZFoUFkq6SM2" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Mq_c3ftZ6SM2" - }, - "source": [ - "### Identity Matrices" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "KVSNbH-Z6SM2", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "46ce4da4-9435-4744-e097-d7129142bf0b" - }, - "source": [ - "I = torch.tensor([[1, 0, 0], [0, 1, 0], [0, 0, 1]])\n", - "I" - ], - "execution_count": 147, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[1, 0, 0],\n", - " [0, 1, 0],\n", - " [0, 0, 1]])" - ] - }, - "metadata": {}, - "execution_count": 147 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "wcoPDhvR6SM3", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "2fb868dc-a57d-4c88-9fa9-d6009ec28a76" - }, - "source": [ - "x_pt = torch.tensor([25, 2, 5])\n", - "x_pt" - ], - "execution_count": 148, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 148 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "tuA4RsMv6SM4", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "d8983050-aafa-4d61-9fa2-8814a245bc5a" - }, - "source": [ - "torch.matmul(I, x_pt)" - ], - "execution_count": 149, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([25, 2, 5])" - ] - }, - "metadata": {}, - "execution_count": 149 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "bgDiOYLk6SM5" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "3S_6Yfdkcgv7" - }, - "source": [ - "### Answers to Matrix Multiplication Qs" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "pINsKNxH6SNC", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "749e8b80-4ece-4e71-d253-071ee8231e73" - }, - "source": [ - "M_q = torch.tensor([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", - "M_q" - ], - "execution_count": 150, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[0, 1, 2],\n", - " [3, 4, 5],\n", - " [6, 7, 8]])" - ] - }, - "metadata": {}, - "execution_count": 150 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "gfjWd8OO6SNE", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "e05d8691-6939-4356-ef6e-d4cb66311d79" - }, - "source": [ - "V_q = torch.tensor([[-1, 1, -2], [0, 1, 2]]).T\n", - "V_q" - ], - "execution_count": 151, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[-1, 0],\n", - " [ 1, 1],\n", - " [-2, 2]])" - ] - }, - "metadata": {}, - "execution_count": 151 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "boSkaV2M6SNF", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "412a3565-f9fd-4e6f-f3b0-c85ec84b1a9f" - }, - "source": [ - "torch.matmul(M_q, V_q)" - ], - "execution_count": 152, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[ -3, 5],\n", - " [ -9, 14],\n", - " [-15, 23]])" - ] - }, - "metadata": {}, - "execution_count": 152 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "slSNKUcN6SNG" - }, - "source": [ - "### Matrix Inversion" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "EW0i5ZRk6SNG", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "157f1989-5346-4066-98ed-f1c807610754" - }, - "source": [ - "X = np.array([[4, 2], [-5, -3]])\n", - "X" - ], - "execution_count": 153, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[ 4, 2],\n", - " [-5, -3]])" - ] - }, - "metadata": {}, - "execution_count": 153 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "hTYpxaWR6SNI", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "dee8a0e1-801f-48e7-ea6a-64d6822a6ba9" - }, - "source": [ - "Xinv = np.linalg.inv(X)\n", - "Xinv" - ], - "execution_count": 154, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[ 1.5, 1. ],\n", - " [-2.5, -2. ]])" - ] - }, - "metadata": {}, - "execution_count": 154 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "XFDBBdYOc-7E" - }, - "source": [ - "As a quick aside, let's prove that $X^{-1}X = I_n$ as per the slides:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "tyPhf-uZcvdB", - "outputId": "1b0f5f09-b0da-4f36-d248-f68056b0f4e1", - "colab": { - "base_uri": "https://localhost:8080/" - } - }, - "source": [ - "np.dot(Xinv, X)" - ], - "execution_count": 155, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[1.00000000e+00, 3.33066907e-16],\n", - " [0.00000000e+00, 1.00000000e+00]])" - ] - }, - "metadata": {}, - "execution_count": 155 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "SVwvbBvclul1" - }, - "source": [ - "...and now back to solving for the unknowns in $w$:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "Q5sQqFaz6SNK", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "92c47ee8-9375-4c79-c8ea-63c56ce5e396" - }, - "source": [ - "y = np.array([4, -7])\n", - "y" - ], - "execution_count": 156, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([ 4, -7])" - ] - }, - "metadata": {}, - "execution_count": 156 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "PK7m6F1I6SNL", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ac9bda7d-a937-4c90-e8d5-ed177a1f8dc0" - }, - "source": [ - "w = np.dot(Xinv, y)\n", - "w" - ], - "execution_count": 157, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([-1., 4.])" - ] - }, - "metadata": {}, - "execution_count": 157 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "fyBOHgdccgwD" - }, - "source": [ - "Show that $y = Xw$:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "SVBojjwacgwD", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "805a0f2c-8ae0-41d8-c121-c270bd67c479" - }, - "source": [ - "np.dot(X, w)" - ], - "execution_count": 158, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([ 4., -7.])" - ] - }, - "metadata": {}, - "execution_count": 158 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "rKELsi0PZCr0" - }, - "source": [ - "**Geometric Visualization**\n", - "\n", - "Recalling from the slides that the two equations in the system are:\n", - "$$ 4b + 2c = 4 $$\n", - "$$ -5b - 3c = -7 $$\n", - "\n", - "Both equations can be rearranged to isolate a variable, say $c$. Starting with the first equation:\n", - "$$ 4b + 2c = 4 $$\n", - "$$ 2b + c = 2 $$\n", - "$$ c = 2 - 2b $$\n", - "\n", - "Then for the second equation:\n", - "$$ -5b - 3c = -7 $$\n", - "$$ -3c = -7 + 5b $$\n", - "$$ c = \\frac{-7 + 5b}{-3} = \\frac{7 - 5b}{3} $$" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "ZASIBqroap7k" - }, - "source": [ - "b = np.linspace(-10, 10, 1000) # start, finish, n points" - ], - "execution_count": 159, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "9XsUdVsmaqTr" - }, - "source": [ - "c1 = 2 - 2*b" - ], - "execution_count": 160, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "8Mwn1tAca1Cl" - }, - "source": [ - "c2 = (7-5*b)/3" - ], - "execution_count": 161, - "outputs": [] - }, - { - "cell_type": "code", - "metadata": { - "id": "X5ozP9jZauQa", - "colab": { - "base_uri": "https://localhost:8080/", - "height": 455 - }, - "outputId": "d6d69af4-657c-4a70-9020-b028c43c20fa" - }, - "source": [ - "fig, ax = plt.subplots()\n", - "plt.xlabel('b', c='darkorange')\n", - "plt.ylabel('c', c='brown')\n", - "\n", - "plt.axvline(x=0, color='lightgray')\n", - "plt.axhline(y=0, color='lightgray')\n", - "\n", - "ax.set_xlim([-2, 3])\n", - "ax.set_ylim([-1, 5])\n", - "ax.plot(b, c1, c='purple')\n", - "ax.plot(b, c2, c='purple')\n", - "plt.axvline(x=-1, color='green', linestyle='--')\n", - "_ = plt.axhline(y=4, color='green', linestyle='--')" - ], - "execution_count": 162, - "outputs": [ - { - "output_type": "display_data", - "data": { - "text/plain": [ - "
" - ], - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjoAAAG2CAYAAAB20iz+AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeN0lEQVR4nO3dd3hUZcL+8e+ZmfSQhJLQO1KCNGkiipHeCZBCS8Cyuyru6rq6r6y/Fd3XXVz1XcvK2nZfJLQ0SOi9K9JFKYIgvYeWkJ7MzO+PvMsuC0iAzJxMcn+ua67LSZ5zzn05YXLnnDPPYzidTiciIiIiFZDF7AAiIiIirqKiIyIiIhWWio6IiIhUWCo6IiIiUmGp6IiIiEiFpaIjIiIiFZaKjoiIiFRYKjoiIiJSYanoiIiISIWloiMiIiIVlscUnddffx3DMK57tGzZ0uxYIiIiUo7ZzA5wJ1q3bs2qVauuPbfZPCq+iIiIuJlHNQWbzUatWrXMjiEiIiIewqOKzsGDB6lTpw6+vr5069aNKVOm0KBBg1uOLygooKCg4Npzh8PBpUuXqF69OoZhuCOyiIiI3COn08nVq1epU6cOFsud3XVjOJ1Op4tylamlS5eSnZ1NixYtOHPmDG+88QanTp1iz549VKlS5abbvP7667zxxhtuTioiIiKucOLECerVq3dH23hM0flPV65coWHDhvzlL3/hySefvOmY/zyjk5mZSYMGDThx4gRBQUHuiloqDruDhT9byL65+7B6WYlKjqJJzyY3jCvMKWRG3xmc33OeWu1rMW7ZOLz8vExIfPdyCnOo8z91ADj9m9MEeAeYnKjycjgc7N+/H4CWLVve8V9KIiLukJWVRf369bly5QrBwcF3tK1HXbr6dyEhITRv3pxDhw7dcoyPjw8+Pj43fD0oKKjcFR2AMXPGkBqbyv60/Swes5ixS8fS6NFG1w8KgscXPs5nnT7jyq4rbPztRoZ9McyjLsVZC63gW/LfQUFBKjomcjgcBAYGAiWvhYqOiJRnd/O7zmPf1bKzs/nxxx+pXbu22VHKjNXLSlRiFPcNvI/ivGJmD5rNia9P3DAupFEIUUlRGFaDbxO+ZcuHW0xIKyIiUv55TNF56aWXWL9+PUePHmXTpk0MHz4cq9XK6NGjzY5WpqzeVmLmxtC4V2OKcoqY1X8Wp3ecvmFck15N6PtuXwBW/GYFR9YccXdUERGRcs9jis7JkycZPXo0LVq0ICYmhurVq7N582ZCQ0PNjlbmbL42Rs0fRYNHGlCQVcDMvjM59925G8Z1fb4rbePa4rQ7SYlJ4crRK+4PexdsFhvj241nfLvx2Cwee/VUREQ8gMfejHw3srKyCA4OJjMzs1zeo/OfCq4WMKPPDE5tOYV/qD8T1k8gtNX1xa4or4hpj0zjzI4z1GxXkyc3PYmXv2fdnCzmcTgc7Nu3D4Dw8HDdoyMi5dK9/P7Wu1o55lPFh3HLxlGrQy1yM3JJ6JXApUOXrhvj5edFbFosAWEBnPv2HPOfmE8l6q4iIiI/SUWnnPMN8SVuRRxh94eRfSabhF4JXDl25boxwfWDiU6NxmKzsDdpL5ve2WRO2FJyOp3kFOaQU5ijUiYiIi6louMB/Gv4E7cqjuotqpN5PJOEnglkncq6bkzDRxrS/8P+AKx6ZRWHlt36Y/dmyy3KJXBKIIFTAsktyjU7joiIVGAqOh4isGYg8avjqdqkKpcPXyahVwLZ57KvG9Pp6U50eKoDOGHu6Lk3XOYSERGpbFR0PEhQ3SDi18QT3CCYiwcuMqP3DHIv/OuMiGEYDPxoIPUerEf+lXwSIxMpuFrwE3sUERGp2FR0PExIwxDiV8cTWDuQ83vOM6PvDPIu5137vs3HRszcGAJrB5KxN4P08ek4HboPRkREKicVHQ9UrVk14lfH4x/qz9lvzjJrwKzrztxUqVOF2HmxWL2t7E/bz8Y/bTQxrYiIiHlUdDxUaKtQ4lfF41fNj1NbTjF70GwKcwqvfb/eg/UYOHUgAGtfW8uBhQfMiioiImIaFR0PVrNtTcatGIdPsA/HNx4ncVgiRXlF177/wFMP0OnZTuCEeWPncWH/BRPTioiIuJ+Kjoer07EOY5eOxTvQmyOrj5ASlUJxQfG17/d/vz8NezSk8GohicMSyc/MNzFtCavFSlR4FFHhUVgtVrPjiIhIBaaiUwHU71afMYvHYPOzcXDJQeaOmou9yA6UrIgenRJNUL0gLv5wkXlj55l+c7KvzZeU6BRSolPwtfmamkVERCo2FZ0KomGPhoyaPwqrj5X96ftJj0/HYXcAEBAWQGx6LDZfGwcXH2Tt5LUmpxUREXEPFZ0KpGmfpsSkxmDxsrAncQ8Lnlxw7exNnY51GPzZYAA2vrmRfXP3mRlVRETELVR0Kpjmg5sTlRiFYTX4dvq3LH528bX1pNrFtePBXz8IQPr4dM7vOW9KxpzCHIw3DIw3DHIKc0zJICIilYOKTgXUakQrhicMBwN2fLqD5b9efq3s9Hm7D417NqYop4jEYYnkXcq7zd5EREQ8l4pOBdVmTBuG/mMoAFs+2MLqSatxOp1YbBaikqIIaRTC5cOXmTt67rV7eURERCoaFZ0KrMPjHRj4t5JJA7/681es/8N6oGQ19Nj0WLz8vfhxxY+snrTazJgiIiIuo6JTwXV+pjN9/9IXgPWvr+ert78CoFa7WgybNgyATe9sYvec3aZlFBERcRUVnUqg26+70fNPPQFY9V+r2PLhFgBax7Sm+yvdAVjw5ALOfHPGtIwiIiKuoKJTSTwy6RF6/L4HAMueX8aOz3YA0PPNnjTr34zivGKSIpPIydCnoEREpOJQ0alEIt6IoNtL3QBY9PQivk34FovVwojZI6jWrBqZxzNJjU29Nquyq1gtVgbeN5CB9w3UEhAiIuJSKjqViGEY9Hm7D52f6wxOmP/4fPYk7cGvqh+x6bF4B3pzdO1RVr680qU5fG2+LB6zmMVjFmsJCBERcSkVnUrGMAwGfDCADk91wOlwMm/sPPan7yesdRiRCZFAycfRd03fZWpOERGRsqCiUwkZFoPBnwymbVxbnHYnKTEpHFx6kFbDW9HjtZL7eBb9YhGntp4yOamIiMi9UdGppCxWC8P+dxjh0eE4ihwkj0jm8OrDREyOoMXQFtgL7CSNSCL7bHaZHzunMIeAPwUQ8KcALQEhIiIupaJTiVlsFkbMGkGLoS0ozi8mcWgix786zvAZw6nRsgZXT10lOSoZe2HZ35ycW5RLblFume9XRETk36noVHJWLytRyVE069+MotwiZg+aTcb3GYyaPwqfIB9OfHWCpc8vNTumiIjIXVHREWw+NmLmxdDosUYUXi1kVv9ZFOYUMmL2iJKFQT/ZcW3eHREREU+ioiMAePl5MXrBaOp3r0/+lXxm9JlBSMMQer5ZMqPykueWcGLTCZNTioiI3BkVHbnGO9CbMYvHUKdzHfIu5pHQO4GWI1oSHvV/NyyPTCbrVJbZMUVEREpNRUeu4xvsy7hl46jZriY553KY0XsGPV7rQdj9YWSfzSZ5RDLF+cVmxxQRESkVFR25gV81P+JWxhEaHsrVU1eZM2QOA/82EN+qvpzaeorFzy7G6XTe9f4thoVHGz7Kow0fxWLoR1BERFxHv2XkpgJCA4hbFUe1+6qReSyTBU8sYODfBmJYDHZN28W2qdvuet9+Xn6sm7COdRPW4eflV4apRURErqeiI7dUpXYV4lfHE9IohEuHLrHhjQ3/WgH9hWUcXXfU3IAiIiK3oaIjPym4fjDxa+IJqhfEhf0X2J+2n1YjW5UsHRGdQubxTLMjioiI3JKKjtxW1cZViV8dT2CtQM59d47Lhy8T1jaM3Au5JEYmUpRbdEf7yynMIfSdUELfCdUSECIi4lIqOlIq1ZtXJ351PP41/Dn7zVmsNit+1fw4+81ZFv584R3fnHwh9wIXci+4KK2IiEgJFR0ptdDwUOJWxeFb1ZczO88QVD8ILLB71m42v7fZ7HgiIiI3UNGRO1KrXS3GLR+HT5AP5749R/Xm1QFY+fJKDq86bHI6ERGR66noyB2r27kuY5aMwSvAi4v7LxJULwinw0lqbCqXD182O56IiMg1KjpyVxp0b8DohaOx+drIOpmFb1Vf8i7lkRiZSGF2odnxREREABUduQeNH2tMbHosVm8r+ZfzsfpaOb/7PPMfn39PMyeLiIiUFRUduSfN+jUjOiUai82CPd8OBuxL3ceXb315y20shoVOdTrRqU4nLQEhIiIupd8ycs9aDG3BiNkjMCwG/N+JnDW/W8PBJQdvOt7Py49tP9vGtp9t0xIQIiLiUio6UiZaR7cmcnokGP/6WuroVC7+cNG0TCIiIio6UmbajmvLkM+GXHtemFVIYmQiBVkFJqYSEZHKTEVHytQDTz3AgL8OuPb8wvcXSItLw+n4183JuUW5NHq/EY3eb0RuUa4ZMUVEpJJQ0ZEy1+W5LvR5p8+15wcWHGD9H9Zfe+50OjmWeYxjmcf06SwREXEpFR1xiYdeeoiIP0Rce77+jfXsT99vXiAREamUVHTEZXr8vx48/LuHrz1PHZVKxr4MExOJiEhlo6IjLmMYBj3f7EnX57sCYC+wM73ndPIz801OJiIilYXHFp233noLwzB44YUXzI4iP8EwDPq91492E9oBkHMuh1kDZpmcSkREKguPLDrbtm3j008/pW3btmZHkVIwDINh/xhGi2EtADj33TmTE4mISGVhMzvAncrOzmbs2LF8/vnnvPnmm3e1j5zCHKyF1hu+brVY8bX5XjfuViyG5bpZfe9kbG5R7i0/bWQYBv5e/nc1Nq8oD4fTccscAd4BdzU2vzgfu8N+z2MHzRlETu8cDm87TOj5UPxq+P3kx8v9vfwxjJIZCAuKCyh2FN9yrJ+X37XlJArthRTZi8pkrK/NF6vFesdji+xFFNpvvbipj80Hm8V2x2OLHcUUFN96XiJvqzdeVq9Sj7UaJXntTjs5hTlYLDf/28fL6oW31btkrMNOfvGtLz/++1iH00FeUV6ZjLVZbPjYfICST+791M/OnYy9k3/3eo+4+diyeo+AO/t3r/eIG8eW9XvEP8feyb97V71H3C2PKzoTJ05k0KBB9O7d+7ZFp6CggIKCf72IWVlZANT5nzrge+P4gfcNZPGYxdeeh70bdss3yEcbPsq6CeuuPW/0QSMu5F646dhOdTqx7Wfbrj0PnxrOscxjNx0bHhrO3mf3Xnve+fPO7MvYd9OxDYMbcvSFo9ee9/iiB9tPb7/p2Br+Nch4+V83Ag+YNYD1x9bfdKy/lz85v/vXm/LI5JEsObjkpmMBnJP/9SYblxZH6r7UW47NWpnFF+2/YOLfJpIWmUbYu2G3HHv+pfOEBoQC8OLyF/nb9r/dcuyR54/QKKQRAK+ufpV3v373lmP3PLOH1mGtAfjTxj/xxvo3bjl261Nb6Vy3MwAfbP6A36767S3Hrh2/lohGEQB8tuMznlv63C3HLhq9iEHNBwEwa/csHp//+C3HJkclE906GoC079OISY255dhpw6Yxof0EAJYfWs7gOYNvOfajAR/xTKdnANh5YSft57W/5di3e7/Ny91fLhl7Zidd/t7llmMnPzqZ1yNeB+D7jO+5/+P7bzn2pW4v8U7fdwA4nnmcxh80vuXYZzs9y9RBUwG4kHvhJ392xrcbzxeRXwAlRSBwSuAtx0aFR5ESnXLt+U+N1XtECVe+R2RPyr5WjH6x6BdM/3b6LcfqPaKEK98jJnaZCMDG4xt5bPpjtxzrjveIu+VRl64SExPZuXMnU6ZMKdX4KVOmEBwcfO1Rv359FyeU27HYLDz97dP41/C//WAREZF7ZDg9ZMa2EydO0KlTJ1auXHnt3pyIiAjat2/P+++/f9NtbnZGp379+pzOOE1QUNAN43Va+uZjXXFa+urpq/ylxV8oLCjEsBrEr4ynbpe6Nx0LOi3tyktX+/btw+600+S+Jrp0hS5d3c1YXbqquO8R5eXSVVZWFsHBwWRmZt709/dP8Ziik56ezvDhw7Fa/3Vvjd1uxzAMLBYLBQUF133vZu7lf5SUndyiXDp/3pnivGKifxeNV6EXVh8rT215ilrtapkdr1JxOBzs21dy2SM8PPyWRUdExEz38vvbY97VevXqxe7du9m1a9e1R6dOnRg7diy7du26bcmR8sPpdLIvYx8/ZP/A4M9Lrg/bC+xMe2SaJhQUEZEy5TE3I1epUoX777/+hqWAgACqV69+w9fFc7QZ1Yas77L4+n++pvBqIdN6TOPJr5+k+n3VzY4mIiIVgMec0ZGKq/efe9OkTxMA8i7mMT1iOpePXDY5lYiIVAQeXXTWrVt3yxuRxXNYrBaik6Op2qQqAFdPXyWhVwJZJ7NMTiYiIp7Oo4uOVBy+Ib6MXjQar4CSO/yvHLlCQq8Ess9mm5xMREQ8mYqOlBuhrUIZOWfktecXf7hIQq8EcjJu/bFcERGRn6KiI25nGAYNgxvSMLjhtTkw/qnFkBZE/CHi2vOMfRnM7DuTvMu3nmdBRETkVlR0xO38vfw5+sJRjr5w9LrJzP6px6s9aDm8JQCGxeDsrrPM6j+LgqxbT2wlIiJyMyo6Uu4YFoPI6ZGEhofidDix2Cyc2nqKWQNnUZh969lBRURE/pOKjpRLPlV8GDV/FL4hvjiKHVi9rZz46gRzhs6hKO/WU6yLiIj8OxUdcbu8ojw6f96Zzp93/sk1Tqo1q1Zyc7IB9kI7Vh8rR9ceJXlEMsUFt17TRkRE5J9UdMTtHE4H209vZ/vp7T+5aCBAs/7N6DWlV8l2xQ5svjYOLTtEamwq9qJbLwwoIiICKjriAbr/tjutY1vjtDux+dqw+lg5MP8AaePScBT/dFESEZHKTUVHyj3DMBj6j6HUbFeT/Cv5BNUPwuJlYW/yXuY/MR+nw2l2RBERKadUdMQjeAd4Myp9FH7V/bh86DINujfAsBp8N+M7Fj29CKdTZUdERG6koiMeI6RRCFFJURhWg6PrjtJmbBsMi8HOz3ey7PllKjsiInIDFR3xKE16NaHvu30B2D1rNw+99BAAW/+6lVX/tUplR0RErqOiI6ao4V+DGv417mrbrs93pW1cW5x2Jzv/sZOef+wJwKZ3NrHu9XVlmFJERDydio64XYB3ABkvZ5DxcgYB3gF3vL1hGAz+dDC1O9Ym72Iee5P30vvt3gBs+MMGvnzry7KOLCIiHkpFRzySl58XsWmxBIQFcO7bc5zZcebafDurJ61m8/ubTU4oIiLlgYqOeKzg+sFEp0ZjsVnYm7QXw2Lw6ORHAVj+6+Vs/2S7yQlFRMRsKjridnlFeUR8EUHEFxE/uQREaTR8pCH9P+wPwKpXVlG3a126/1d3ABY/s5hdX+y617giIuLBVHTE7RxOB+uPrWf9sfW3XQKiNDo93YkOT3UAJ8wbM48OT3agy6+6ALDgyQXsnrP7no8hIiKeSUVHPJ5hGAz8aCD1HqxH/pV8koYn8dh/P8YDP38Ap8NJWlwa38/73uyYIiJiAhUdqRBsPjZi5sYQWDuQjL0ZzJ8wn0FTB9FufDucdiepo1I5uOSg2TFFRMTNVHSkwqhSpwqx82KxelvZn7afL9/6kqH/GErr2NY4ihwkjUji8KrDZscUERE3UtGRCqXeg/UYOHUgAGtfW8vBJQcZPmM4LSNbYi+wM2foHI5tOGZyShERcRcVHalwHnjqATo926nk5uSx87j842VGJo6k2YBmFOcVM3vQbE5uPml2TBERcQMVHTGFv5c//l7+Ltt///f707BHQwqvFpI4LJHi/GJi5sbQuGdjCrMLmdl/Jmd2nnHZ8UVEpHxQ0RG3C/AOIOd3OeT8LueuloAoDauXleiUaILqBXHxh4vMGzsPm4+NUQtG0eDhBhRkFjCjzwzO7T7nkuOLiEj5oKIjFVZAWACx6bHYfG0cXHyQtZPX4h3gzZjFY6jbpS55l/KY0XsGF/ZfMDuqiIi4iIqOVGh1OtZh8GeDAdj45kb2zd2HT5APY5eNpVb7WuSczyGhVwKXfrxkclIREXEFFR1xu/zifAbNHsSg2YPIL853+fHaxbXjwV8/CED6+HTO7zmPX1U/4lbGEdo6lKunr5LQM4Erx664PIuIiLiXio64nd1hZ8nBJSw5uAS7w+6WY/Z5uw+NezamKKeIxGGJ5F3Kw7+GP/Gr4qnevDqZxzNJ6JVA1qkst+QRERH3UNGRSsFisxCVFEVIoxAuH77M3NFzcdgdBNYKJH51PCGNQ7j842USeiWQfS7b7LgiIlJGVHSk0vCv4U9seixe/l78uOJHVk9aDUBQvSDGrxlPUP0gLh64yIw+M8i9mGtyWhERKQsqOlKp1GpXi2HThgGw6Z1N11Y2D2kUwvg14wmsHcj53eeZ2Xcm+Vdcf/+QiIi4loqOVDqtY1rT/ZXuACx4cgFnvimZOLBas2rEr47HP9SfMzvPMGvALAquFpgZVURE7pGKjlRKPd/sSbP+JUtCJEUmkZORA0Boq1DiV8XjV82Pk5tPMmfwHIpyi0xOKyIid0tFRyoli9XCiNkjqNasGpnHM0mNTcVeVPIJsJptazJuxTh8gnw4tuHYtSUkRETE86joiNsFeAfgnOzEOdnpsiUgSsOvqh+x6bF4B3pzdO1RVr688tr36nSsw9ilY/EK8OLwqsMkRyVjL3TPR+FFRKTsqOhIpRbWOozIhEgAtnywhV3Td137Xv2H6jNm8RhsfiVLSMwdPRdHscOcoCIicldUdKTSazW8FT1e6wHAol8s4tTWU9e+1+jRRoxKH4XV28r3874nLT4Nh11lR0TEU6joiNvlF+cTnRJNdEq0W5aAKI2IyRG0GNoCe4GdpBFJZJ/916SBTfs2JTo1GovNwp45e1j41EKcDqeJaUVEpLRUdMTt7A47qftSSd2X6rYlIG7HsBgMnzGcGi1rcPXU1RvuyWkxpAUjE0diWA12fbGLJc8twelU2RERKe9UdET+j0+QD6Pmj8InyIcTX51g6fNLr/t++MhwhicMBwO2f7yd5S8uV9kRESnnVHRE/k315tUZMXsEGLDjkx3s+GzHdd9vM6YNQ/8+FIAt729hzatrVHZERMoxFR2R/9B8UHN6vtkTgCXPLeHEphPXfb/DEx0YOHUgAF9O+ZINb25we0YRESkdFR2Rm3h40sOER4XjKHKQPDKZrFNZ132/87Od6fs/fQFY99o6vnrnKzNiiojIbajoiNyEYRgMmzaMsPvDyD6bTfKI5BtmR+72Yjd6/rHkzM+q365iy1+3mBFVRER+goqOyC14B3oTmx6Lb1VfTm09xeJnF99wP84jv3uER/7fIwAs+9Uydny+42a7EhERk6joiNv5e/mTPSmb7EnZ+Hv5mx3nJ1VrWo2opCgMi8GuabvYNnXbDWMe+8NjdPtNN6BkwsFvZ3zr7pgiInILKjridoZhEOAdQIB3AIZhmB3ntpr2aUrvt3sDsOyFZRxdd/S67xuGQZ93+tB5YmdwwvwJ89mbvNeEpCIi8p9UdERKoduL3Wgzpg1Ou5OU6BQyj2de933DMBjw4QA6PNUBp8PJvLHz2D9/v0lpRUTknzym6Hz88ce0bduWoKAggoKC6NatG0uXLr39hlLuFBQXMCF9AhPSJ1BQXGB2nFIxDIMhnw+hVoda5F7IJTEykaLcouvHWAwGfzKYNmPb4Ch2kBqTyqFlh0xKLCIi4EFFp169erz11lvs2LGD7du307NnT4YNG8bevbpE4GmKHcVM/3Y607+dTrGj+PYblBNe/l7EpsXiX8Ofs9+cZeHPF95wc7LFaiHyi0jCo8KxF9pJGp7EkTVHTEosIiIeU3SGDBnCwIEDue+++2jevDl//OMfCQwMZPPmzWZHk0okpGEI0SnRGFaD3bN2s/m9G3/+LDYLI2aPoPmQ5hTnFzNnyByOf3nchLQiIuIxReff2e12EhMTycnJoVu3brccV1BQQFZW1nUPkXvVKKIR/d7rB8DKl1dyeNXhG8ZYvaxEJ0fTtG9TinKLmDVwFqe2nnJ3VBGRSs+jis7u3bsJDAzEx8eHp59+mrS0NMLDw285fsqUKQQHB1971K9f341ppSLr8lwX2k9oj9PhJDU2lcuHL98wxuZrIzYtlkYRjSi8WsjMfjM5u+usCWlFRCovjyo6LVq0YNeuXWzZsoVnnnmG8ePHs2/fvluOnzRpEpmZmdceJ06cuOVYkTthGAaDPh5E3S51ybuUR2JkIoXZhTeM8/L3YvTC0dR/qD75V/KZ0WcG5/eeNyGxiEjl5FFFx9vbm2bNmtGxY0emTJlCu3bt+OCDD2453sfH59qntP75ECkrNl8bMfNiCKgZwPnd55n/+PybrmTuHejNmCVjqNOpDrkXcknolcDFHy6akFhEpPLxqKLznxwOBwUFnvHxZKmYguoGETM3BouXhX2p+/jyrS9vOs432Jdxy8dRs21Ncs7lkNArgctHbrzcJSIiZctjis6kSZPYsGEDR48eZffu3UyaNIl169YxduxYs6PJHfL38uf8S+c5/9L5cr8ERGk06N6AgR8NBGDNq2s4uOTgTcf5VfMjblUcNVrVIOtkFgk9E8g8kXnTsSIiUjY8puicP3+e+Ph4WrRoQa9evdi2bRvLly+nT58+ZkeTO2QYBqEBoYQGhHrEEhCl0fHnHen4i47ghLlj5t7y0lRAaADxq+Op1qwaV45eIaFnAlfPXHVzWhGRysNw3uymggoqKyuL4OBgMjMzdb+OlDl7oZ3pPadz4qsT1GhVg6c2P4VPkM9Nx2aeyOSLHl9w5egVarSqwYR1EwgIC3Bz4pLLv/+8oT88PByLxWP+9hGRSuRefn/rXU3crqC4gImLJzJx8USPWQKiNKzeVmJSY6hSpwoXvr9AWnwaTsfN/44Irh9M/Jp4qtQtGTujzwzyLuW5ObGISMWnoiNuV+wo5m/b/8bftv/No5aAKI3AWoHEpsVi9bFyYP4B1v9h/S3HVm1clfFrxhNQM4Bz351jZr+Z5GfmuzGtiEjFp6IjUsbqdqnL4E8GA7D+jfXsT7/1KubVm1cnfnU8/jX8Ob39NLMHzr7pfDwiInJ3VHREXKD9hPZ0+VUXANLi0sjYl3HLsWGtw4hbGYdviC8nNp1gzpA5N6yMLiIid0dFR8RF+r7bt2T5h+xCEoclkn/l1pelarWvxbjl4/Cu4s3RdUdJGp5EcUHFuqwnImIGFR0RF7F6WYlKjiK4QTCXDl1i7pi5OOyOW46v26UuY5eMxcvfix9X/EhKdAr2QrsbE4uIVDwqOiIuFBAaQGx6LDY/G4eWHmLt79f+5PgGDzdg9MLR2Hxt/LDwB+aNnYej+NblSEREfpqKjoiL1e5Qm6F/HwrAl1O+ZG/y3p8c37hn45JPbnlb2Ze6j/mPz//JM0EiInJrKjridn5efhx5/ghHnj+Cn5ef2XHcos2YNnR7qRsA8x+fz9lvz/7k+Gb9mxGVHIXFZuG7md+x6OlFt5yTR0REbk1FR9zOYlhoFNKIRiGNsBiV50ew91u9adq3KUW5RSRFJpF7Mfcnx7cc1pIRs0ZgWAy++fs3LP3V0puuji4iIrdWeX7LiJjMYrUwcs5IqjapypWjV0iNTb3t/TetY1oz7IthYMC2qdtY+duVKjsiIndARUfcrtBeyMsrXublFS9TaK9ck+P5VfMjNj0WrwAvjqw+wsr/WnnbbdrFtWPwpyUTEH797tesm7zOxSlFRCoOFR1xuyJ7Ee9+/S7vfv0uRfbKNzFezTY1iZweCcDmv2zmu5nf3Xabjj/rSP8P+wOw4b83sPFPG10ZUUSkwlDRETFB+MhwHnn1EQAW/mwhp3ecvu02XX/Zld5v9wZgzatr+Pq9r12aUUSkIlDRETFJxBsR3DfoPorzi0kankTO+ZzbbtP95e5EvBEBwIoXV7Dtb9tcmlFExNOp6IiYxGK1MGLmCKo3r07WiaySmZCLbj8Tco/f9+DhSQ8DsGTiEr75329cHVVExGOp6IiYyDfEl9j0WLyreHNswzGW/3r5bbcxDIOef+xJ1xe6ArDgqQXsnr3b1VFFRDySio6IyUJbhTJi1gig5CPkO/+x87bbGIZBv7/0o+PTHcEJafFp7Ju7z9VRRUQ8joqOSDnQYkgLIv4QAcCSZ5dwcvPJ225jGAaDpg6i/YT2OO1O5o6ayw+LfnBxUhERz6KiI27n5+XHnmf2sOeZPZVmCYjS6PFqD1oOb4m90E7SiCSunrl6220Mi8GQvw/h/tH34yh2kDwymR9X/OiGtCIinkFFR9zOYlhoHdaa1mGtK9USELdjWAwip0cSGh5K9plskkcmU1xQfNvtLFYLkdMjr5WkxMhEjq4/6vrAIiIeQL9lRMoRnyo+jJo/Ct8QX05+fZIlzy0p1ZIPVi8rUYlRJR9Xzytm9qDZnPj6hBsSi4iUbyo64naF9kJeX/c6r697vdItAVEa1ZpVY+SckWDAN3//hh2f7ijVdlZvKzGpMTTp3YSinCJm9Z/F6e23n4hQRKQiU9ERtyuyF/HG+jd4Y/0blXIJiNJo1r8Zvab0AmDpL5dybOOxUm1n87URmx5Lg0caUJBVwIy+Mzj33TlXRhURKddUdETKqe6/7U7r2NY4ih2kRKWQeSKzVNt5B3gzZvEY6j1Yj/zL+ST0TiDj+wwXpxURKZ9UdETKKcMwGPqPodRsV5Oc8zkkDU+iKK90Z8B8qvgwdulYaj9Qm9yMXBJ6JXDp0CUXJxYRKX9UdETKMe8Ab0alj8Kvuh9ndpxh8dOLS3VzMpTMujxuxTjC2oSRfSab6T2nc+XoFdcGFhEpZ1R0RMq5kEYhRCVFYVgNvk34li0fbin1tv7V/YlbGUeNljXIOpFFQq8Esk5luTCtiEj5oqIj4gGa9GpC33f7ArDiNys4suZIqbcNrBlI3Ko4qjapyuXDl0nomUD22WxXRRURKVdUdEQ8RNfnu9I2ri1Ou5OUmJQ7ugwVVDeI+DXxBDcI5uIPF0nonUDuhVzXhRURKSdUdMTtfG2+bH1qK1uf2oqvzdfsOB7DMAwGfzqY2h1rk3cxj8TIRIpyS//x/JCGIcSviadKnSpk7M1gRt8Z5F3Oc2FiERHzqeiI21ktVjrX7Uznup2xWqxmx/EoXn5exKbFEhAWwLlvzzH/ifmlvjkZoFrTasSvjicgLICz35xl9sDZFGVrLiMRqbhUdEQ8THD9YKJTo7HYLOxN2sumdzbd0fY1WtYgblUcftX8OL31NF9N/Iri3NuvqSUi4olUdMTtCu2FvPPVO7zz1TtaAuIuNXykIf0/7A/AqldWcWjZoTvavmabmsStjMMn2IeLOy+y6VebSj1Hj4iIJ1HREbcrshfx21W/5berfqslIO5Bp6c70eGpDuCEuaPn3vGEgLUfqM2YJWOw+dvI2JJBalRqqVZLFxHxJCo6Ih7KMAwGfjSwZKmHK/kkRiZScLXgjvZR78F6PDT1Iay+Vg4tO8TcUXOxF9ldlFhExP1UdEQ8mM3HRszcGAJrB5KxN4P08ek4HaW/ORkgtFMo3T7shtXHyv70/aTFpeGwO1yUWETEvVR0RDxclTpViJ0Xi9Xbyv60/Wz808Y73kfNbjWJTonG4lVyg/OCJxbccWESESmPVHREKoB6D9Zj4NSBAKx9bS0HFh64433cN+g+ohL/tdTE4mdLv66WiEh5paIjUkE88NQDdHq2Ezhh3th5XNh/4Y730WpEK4bPGA4G7Ph0B8t/vVxlR0Q8moqOSAXS//3+NOzRkMKrhSQOSyQ/M/+O99FmdBuG/e8wALZ8sIXVk1ar7IiIx1LREbfztfmydvxa1o5fqyUgypjVy0p0SjRB9YK4+MNF5o2dd1f32rSf0J5BHw8C4Ks/f8X6P6wv66giIm6hoiNuZ7VYiWgUQUSjCC0B4QIBYQHEpsdi87VxcPFB1k5ee1f76fR0J/q91w+A9a+v58s/f1mWMUVE3EJFR6QCqtOxDoM/GwzAxjc3sm/uvrvaz4MvPEivKb0AWP3KajZ/sLnMMoqIuIOKjrhdkb2IqVunMnXrVM2M7ELt4trx4K8fBCB9fDrn95y/q/08/MrD9HitBwDLX1jOjs92lFlGERFXU9ERtyu0F/Lc0ud4bulzWuvKxfq83YfGPRtTlFNE4rBE8i7l3dV+Il6P4KGXHwJg0dOL2DV9VxmmFBFxHRUdkQrMYrMQlRRFSKMQLh++zNzRc+9q1mPDMOj95950+WUXcMKCJxawJ2mPCxKLiJQtFR2RCs6/hj+x6bF4+Xvx44ofWT1p9V3txzAM+n/Qnwd+9gBOh5N5Y+fxfdr3ZZxWRKRsqeiIVAK12tVi2LSSuXE2vbOJ3XN239V+DMNg8CeDaRvXFqfdSWpsKgeXHCzLqCIiZUpFR6SSaB3Tmu6vdAdgwZMLOPPNmbvaj2ExGPa/w2gd0xpHkYOkEUkcXn24LKOKiJQZFR2RSqTnmz1p1r8ZxXnFJEUmkZORc1f7sdgsDJ85nBbDWmAvsJM4NJFjG4+VcVoRkXvnMUVnypQpdO7cmSpVqhAWFkZkZCQHDtz5woUilZnFamHE7BFUa1aNzOOZzBs9D0fRnd+cDCWzMEclRdGsfzOKcouYPXA2J7ecLOPEIiL3xmOKzvr165k4cSKbN29m5cqVFBUV0bdvX3Jy7u4vUjGPj82HRaMXsWj0InxsPmbHqXT8qvoRmx6Ld6A3R9ceZfdf7u5+HQCbj42YeTE0eqwRhdmFzOo/664viYmIuILhvIvV+vZ+/jm+1avTdMSI677+47x5FFy6RPhTT5VZwFvJyMggLCyM9evX06NHj1Jtk5WVRXBwMJmZmQQFBbk4oUj59n3a9ySPSAag05udGDBpABbL3f3tU5hTyMx+Mznx1Qn8qvsxYd0Ewu4PK8u4IlKJ3cvv77t6VzuUnExQ48Y3fD24WTMOJiffzS7vWGZmJgDVqlW75ZiCggKysrKue4hIiVbDW/HI7x8BYOcfdnJq66m73pd3gDdjl4ylbpe65F3MI6FXAhcOXCirqCIid+2uik7ehQv4hYbe8HXfqlXJy8i451C343A4eOGFF+jevTv333//LcdNmTKF4ODga4/69eu7PJvcXpG9iC92fcEXu77QEhAme/S1R6kdURtHoYOUqBSyz2bf9b58gnwYu2wstdrXIud8Dgm9Erh8+HIZphURuXN3VXT8a9Ui45tvbvh6xjff4Bfm+tPVEydOZM+ePSQmJv7kuEmTJpGZmXntceLECZdnk9srtBfy+PzHeXz+41oCwmSGxaDzlM5UaVyFq6eukhyVjL3Qftf786vqx7gV4wgND+XqqatM7zmdzOOZZZhYROTO3FXRaRYVxc633uLHtDRyTp8m5/Rpfpw3j51//jPNoqLKOuN1nnvuORYtWsTatWupV6/eT4718fEhKCjouoeIXM8r0ItuH3bDJ8iHE1+dYOnzS+9pfwGhAcSvjqd68+pkHstkes/pXD19tYzSiojcGdvdbNTqiScouHKF7f/93ziKSi49WHx8CH/iCVr/7GdlGvCfnE4nv/zlL0lLS2PdunU0vsk9QiJyd6o0qsLwmcNJHJbIjk92ULtDbTr+vONd7y+wViDxq+OZ1mMal3+8TEKvBCasn0BAWEAZphYRub27+tTVPxXl5JB1+DBWX1+qNGyI1du7LLNd59lnn2X27NnMnz+fFi1aXPt6cHAwfn5+pdqHPnVVPuQU5hA4JRCA7EnZBHjrl59ZHA4H+/btAyA8PJyv3vqKNa+uweJlYcK6CdR/6N7ua7ty9ArTHplG1skswtqEMX7tePyr+5dFdBGpRNz+qat/8goIoHqbNoTcd59LSw7Axx9/TGZmJhEREdSuXfvaIykpyaXHFalMHp70MOFR4TiKHCSPTCbr1L19UjGkUQjxa+IJrBXI+d3nmdlvJvlX8ssorYjI7XnMhIFOp/OmjwkTJpgdTaTCMAyDYdOGEXZ/GNlns0kekUxxfvE97bP6fdWJXx2Pfw1/zuw4w6yBsyi4WlBGiUVEfprHFB0RcQ/vQG9i02PxrerLqa2nWPzsYu7hCjcAoeGhxK2Kw7eqLye/PsmcIXMoytXUAiLieio64nY+Nh+So5JJjkrWEhDlVLWm1YhKisKwGOyatottU7fd8z5rtavFuOXj8Any4dj6YyRGJt7z2SIRkdtR0RG3s1lsRLeOJrp1NDbLXX3wT9ygaZ+m9H67NwDLXljG0XVH73mfdTvXZezSsXgFeHF45WFSolPuad4eEZHbUdERkVvq9mI32oxpg9PuJCU6pUwm/6v/UH3GLBqDzdfGD4t+YO6YuTiK724FdRGR21HREbcrdhSTsjeFlL0pFDt06aI8MwyDIZ8PoVaHWuReyCUxMrFM7q1pFNGI2PRYrN5Wvp/7Penj03HYVXZEpOyp6IjbFRQXEJMaQ0xqDAXF+vRNeefl70VsWiz+Nfw5+81ZFv584T3fnAzQrF8zolOjsdgs7J69u2S/jnvfr4jIv1PREZHbCmkYQnRKNIbVYPes3Wx+b3OZ7LfFkBaMmD2i5Kbn/93FkueWlEmJEhH5JxUdESmVRhGN6PdePwBWvrySw6sOl8l+W0e3JnJ6JBiw/ePtrHhphcqOiJQZFR0RKbUuz3Wh/YT2OB1OUmNTuXz4cpnst+24tgz5fAgAm/+ymbW/X1sm+xURUdERkVIzDINBHw+ibpe65F3KIzEykcLswjLZ9wNPPsCAjwYAsPGPG9nw5oYy2a+IVG4qOiJyR2y+NmLmxRBQM4Dzu88z//H5ZXapqcvELvR5tw8Aa3+/lk3/s6lM9isilZeKjojcsaC6QcTMjcHiZWFf6j6+fOvLMtv3Q795iMf++zEAVr60kq1Tt5bZvkWk8lHREbfztnozbdg0pg2bhrfVtavei+s06N6AgR8NBGDNq2s4uORgme27x//rwSOvPgLA0ueWsvPvO8ts3yJSuajoiNt5Wb2Y0H4CE9pPwMvqZXYcuQcdf96Rjr/oCE6YO2YuF3+4WGb7fuy/H+PBFx8EYOHPF/LdzO/KbN8iUnmo6IjIPRnw4QDqd69PQWYBiZGJFGSVzSSQhmHQ992+dHq2EzghfXw6e1P2lsm+RaTyUNERtyt2FLP4h8Us/mGxloCoAKzeVmJSY6hSpwoXvr9AWnxamc1wbBgGA/86kPZPlHykfd6YeRxYcKBM9i0ilYOKjrhdQXEBg+cMZvCcwVoCooIIrBVIbFosVh8rB+YfYP0f1pfZvg2LwZDPhtBmTBscxQ5SolM4tPxQme1fRCo2FR0RKRN1u9Rl8CeDAVj/xnr2p+8vs31brBYip0fSamQr7IV2kiKTOLruaJntX0QqLhUdESkz7Se0p8uvugCQFpdGxr6MMtu3xWZh5OyRNB/cnOL8YmYPns3xr46X2f5FpGJS0RGRMtX33b40imhEYXYhicMSyb+SX2b7tnpbiU6JpkmfJhTlFDF74GxObTtVZvsXkYpHRUdEypTVy0pUchTBDYK5dOgSc8fMxWF3lNn+bb42RqWPouGjDSnIKmBmv5mc/fZsme1fRCoWFR0RKXMBoQHEpsdi87NxaOmhMl+k08vfi9ELR1OvWz3yL+czo/eMMr1MJiIVh4qOiLhE7Q61Gfr3oQB8OeVL9iaX7Rw4PlV8GLt0LLU71ib3Qi4JvRK4eLDsJiwUkYpBRUfcztvqzUcDPuKjAR9pCYgKrs2YNnR7qRsA8x+fX+aXmHyDfRm3fBw129Yk+2w2CT0TuHzkcpkeQ0Q8m4qOuJ2X1YuJXSYysctELQFRCfR+qzdN+zalKLeIpMgkci/mlun+/av7E7cyjhqtapB1MouEXglkncwq02OIiOdS0RERl7JYLYycM5KqTapy5egVUmNTcRSX3c3JAAFhAcSviqdq06pcOXKF6T2nc/XM1TI9hoh4JhUdcTu7w866o+tYd3Qddofd7DjiBn7V/IhNj8UrwIsjq4+w8r9WlvkxqtSpwvg14wluGMylg5eY0XsGORk5ZX4cEfEsKjridvnF+Tw2/TEem/4Y+cVlN8eKlG8129QkcnokAJv/stklq5EHNwhm/JrxVKlbhYx9GczsO5O8y3llfhwR8RwqOiLiNuEjw3nk1UcAWPizhZzecbrMj1G1SVXiV8cTUDOAs7vOMrPfzDJbUV1EPI+Kjoi4VcQbEdw36D6K84tJGp5Ezvmyv7xUo0UN4lfF41fdj9PbTjNr4CwKswvL/DgiUv6p6IiIW1msFkbMHEH15tXJOpFFSnQK9qKyv1cr7P4w4lbG4Rviy4mvTjBn6ByK8orK/DgiUr6p6IiI2/mG+BKbHot3FW+ObTjG8l8vd8lxaneozdhlY/Gu4s3RtUdJHpFMcUGxS44lIuWTio6ImCK0VSgjZo0AYNvUbez8x06XHKde13qMWTwGL38vDi07RGpsqkvOIIlI+aSiIyKmaTGkBRF/iABgybNLOLn5pEuO0/CRhoxaMAqrj5UD8w8wb+y8Mp/LR0TKJxUdcTsvqxdv936bt3u/rZmRhR6v9qDl8JbYC+0kjUhy2UR/TXo1ITYtFouXhX0p+5j/xHycDqdLjiUi5YeKjridt9Wbl7u/zMvdX9ZaV4JhMYicHkloeCjZZ7JJHum6+2juG3Af0cnRGFaD72Z8x6KnF+F0quyIVGQqOiJiOp8qPoyaPwrfEF9Ofn2SJc8tcVkBaRnZkhGzRmBYDHZ+vpNlzy9T2RGpwFR0xO3sDjvbTm1j26ltWgJCrqnWrBoj54wEA775+zfs+HSHy451f+z9DJs2DAzY+tetrPqvVSo7IhWUio64XX5xPl3+3oUuf++iJSDkOs36N6PXlF4ALP3lUo5tPOayY7WLb8fgTwYDsOmdTax7fZ3LjiUi5lHREZFypftvu9M6tjWOYgcpUSlknsh02bE6/rwj/T/oD8CGP2xg45SNLjuWiJhDRUdEyhXDMBj6j6HUbFeTnPM5JA1PcumMxl1/1ZXef+4NwJrfrWHz+5tddiwRcT8VHREpd7wDvBmVPgq/6n6c2XGGxU8vduk9NN1/251HX38UgOW/Xs72T7a77Fgi4l4qOiJSLoU0CiEqKQrDavBtwrds+XCLS4/36GuP0v2/ugOw+JnFfDPtG5ceT0TcQ0VHRMqtJr2a0PfdvgCs+M0Kjqw54rJjGYZBrym96Pp8VwAWPLmA3XN2u+x4IuIeKjoiUq51fb4rbePa4rQ7SYlJ4crRKy47lmEY9HuvHx1/0RGckBaXxvfzvnfZ8UTE9VR0xO28rF5MfnQykx+drCUg5LYMw2Dwp4Op3bE2eRfzSIxMpCjXdTcnG4bBoL8Not34djjtTlJHpfLD4h9cdjwRcS0VHXE7b6s3r0e8zusRr2sJCCkVLz8vYtNiCQgL4Ny350rWqXLhzcmGpeSTX/ePuh9HkYPkkckcXnXYZccTEddR0RERjxBcP5jo1GgsNgt7k/ay6Z1NLj2exWohMiGyZMHRAjtzhs7h2AbXTWAoIq6hoiNu53A62Ht+L3vP78XhdJgdRzxIw0ca0v/Dkgn+Vr2yikPLDrn0eFYvKyPnjKTZgGYU5xUze9BsTm4+6dJjikjZUtERt8sryuP+j+/n/o/vJ68oz+w44mE6Pd2JDk91ACfMHT2XS4cuufR4Nh8bMXNjaNyrMYXZhczsP5MzO8+49JgiUnY8quhs2LCBIUOGUKdOHQzDID093exIIuJmhmEw8KOB1HuwHvlX8kmMTKTgaoFLj+nl58Wo+aNo8EgDCjILmNFnBud2n3PpMUWkbHhU0cnJyaFdu3ZMnTrV7CgiYqJ/nmUJrB1Ixt4M0sen43S4dvVx7wBvxiwaQ92udcm7lEdCrwQu7L/g0mOKyL3zqKIzYMAA3nzzTYYPH252FBExWZU6VYidF4vV28r+tP1s/JPrF+T0CfJh3LJx1OpQi9yMXBJ6JXDpR9deOhORe+NRRUdE5N/Ve7AeA6cOBGDta2s5sPCAy4/pG+JL3Io4wu4P4+rpqyT0TODKsSsuP66I3J0KXXQKCgrIysq67iEiFcsDTz1Ap2c7gRPmjZ3nlstJ/jX8iVsVR/UW1ck8nklCzwSyTun9RaQ8qtBFZ8qUKQQHB1971K9f3+xIIuIC/d/vT8MeDSm8WkjisETyM/NdfszAmoHEr46napOqXD58mYReCWSfy3b5cUXkzlToojNp0iQyMzOvPU6cOGF2JKFkCYiXur3ES91e0hIQUiasXlaiU6IJqhfExR8uMm/sPJffnAwQVDeI+DXxBNUP4uKBi8zoPYPcC7kuP66IlF6FLjo+Pj4EBQVd9xDzeVu9eafvO7zT9x0tASFlJiAsgNj0WGy+Ng4uPsjayWvdctyQhiGMXzOewNqBnN9znhl9Z5B/xfVnlESkdDyq6GRnZ7Nr1y527doFwJEjR9i1axfHjx83N5iIlAt1OtZh8GeDAdj45kb2zd3nluNWa1aN+NXx+If6c/abs8waMMvlc/uISOl4VNHZvn07HTp0oEOHDgC8+OKLdOjQgddee83kZHInHE4HR68c5eiVo1oCQspcu7h2PPjrBwFIH5/O+T3n3XLc0FahxK+Kx6+aHyc3n2T2oNkU5hS65dgicmseVXQiIiJwOp03PL744guzo8kdyCvKo/EHjWn8QWMtASEu0eftPjTu2ZiinCIShyWSd8k9P2c129Zk3Ipx+AT5cHzjcZIikyjOL3bLsUXk5jyq6IiIlIbFZiEqKYqQRiFcPnyZuaPn4rC75+xhnY51GLtsLF4BXhxedZjkqGTshXa3HFtEbqSiIyIVkn8Nf2LTY/Hy9+LHFT+yetJqtx27frf6jFk8BptfyY3RqaNSsRep7IiYQUVHRCqsWu1qMWzaMAA2vbOJ3XN2u+3YjR5txKj5o7D6lCxRkR6f7razSiLyLyo6IlKhtY5pTfdXugOw4MkFnPnmjNuO3bRPU2JSY7DYLOxJ3MPCpxa6ZX4fEfkXFR0RqfB6vtmTZv2bUZxXTFJkEjkZOW47dvPBzRmZOBLDarDri10snrgYp1NlR8RdVHREpMKzWC2MmD2Cas2qkXk8k9RY994zEz4ynOEJw8GAHZ/sYPmLy1V2RNxERUfczmax8WynZ3m207PYLDaz40gl4VfVj9j0WLwDvTm69igrX17p1uO3GdOGof8YCsCW97ew+nerVXZE3EBFR9zOx+bD1EFTmTpoKj42H7PjSCUS1jqMyIRIALZ8sIVd03e59fgdHu/AwL8NBOCrt75iw39vcOvxRSojFR0RqVRaDW9Fj9d6ALDoF4s4te2UW4/f+ZnO9P1LXwDWTV7HV+985dbji1Q2Kjridk6nk4ycDDJyMnTqXkwRMTmCFkNbYC+wkzQ8ieyz2W49frdfd6PnH3sCsOq3q9jy1y1uPb5IZaKiI26XW5RL2LthhL0bRm5RrtlxpBIyLAbDZwynRssaXD111ZTZix/53SP0+H3JmaVlv1rGjs92uPX4IpWFio6IVEo+QT6Mmj8KnyAfTnx1gqXPL3V7hog3Iuj2UjcAFj29iG9nfOv2DCIVnYqOiFRa1ZtXZ8TsEdc+9u3usyqGYdDn7T50fq4zOGH+hPnsTd7r1gwiFZ2KjohUas0HNafnmyX3yyx5bgknNp1w6/ENw2DABwPo8FQHnA4nc8fMZf/8/W7NIFKRqeiISKX38KSHCY8Kx1HkIHlkMlmnstx6fMNiMPiTwbQd1xan3UlqTCqHlh1yawaRikpFR0QqPcMwGDZtGGH3h5F9NpvkEckU5xe7NYPFamHYtGGER4djLyz5NNiRNUfcmkGkIlLREREBvAO9iU2PxbeqL6e2nmLxs+5fk8piszBi1ghaDG1BcX4xc4bM4fiXx92aQaSiUdERt7NZbIxvN57x7cZrCQgpV6o1rUZUUhSGxWDXtF1sm7rN7RmsXlaikqNo2q8pRblFzBo4i1Nb3TupoUhFoqIjbudj8+GLyC/4IvILLQEh5U7TPk3p/XZvAJa9sIyj6466PYPNx0bsvFgaPdaIwquFzOw3k7O7zro9h0hFoKIjIvIfur3YjTZj2uC0O0mJTiHzeKbbM3j5ezF6wWjqd69P/pV8EnoncH7vebfnEPF0Kjridk6nk5zCHHIKc7QEhJRLhmEw5PMh1OpQi9wLuSRGJlKUW+T2HN6B3oxZPIY6neqQdzGPhF4JXPzhottziHgyFR1xu9yiXAKnBBI4JVBLQEi55eXvRWxaLP41/Dn7zVkW/nyhKcXcN9iXccvHUbNdTXLO5TC953QuH77s9hwinkpFR0TkFkIahhCdEo1hNdg9azeb39tsSg6/an7ErYwjNDyUq6euktArgcwT7r+cJuKJVHRERH5Co4hG9HuvHwArX17J4VWHTckREBpA3Ko4qjWrxpWjV0jomcDVM1dNySLiSVR0RERuo8tzXWg/oT1Oh5PU2FTTLh1VqV2F+DXxhDQK4dKhSyT0SiDnfI4pWUQ8hYqOiMhtGIbBoI8HUbdLXfIu5ZEYmUhhdqEpWYLrBxO/Jp6gekFc+P4CM/rMIO9SnilZRDyBio6ISCnYfG3EzIshoGYA53efZ/7j80371GDVxlWJXx1PYK1Azn13jpn9ZpKfmW9KFpHyTkVHRKSUguoGETM3BouXhX2p+/jyrS9Ny1K9eXXiVsXhX8Of09tPM3vgbNPOMomUZyo64nZWi5Wo8CiiwqOwWqxmxxG5Iw26N2DgRwMBWPPqGg4uOWhalrDWYcStjMM3xJcTm04wZ8gcU+b7ESnPVHTE7XxtvqREp5ASnYKvzdfsOCJ3rOPPO9LxFx3BCXPHzDV1Er9a7WsxbsU4vKt4c3TdUZKGJ7l95XWR8kxFR0TkLgz4cAD1u9enILOAxMhECrIKTMtSt3Ndxi4di1eAFz+u+JGUmBTshXbT8oiUJyo6IiJ3weptJSY1hip1qnDh+wukxafhdJi3pEmD7g0YvXA0Nl8bPyz8gXlj5+EodpiWR6S8UNERt8spzMF4w8B4wyCnUHOAiOcKrBVIbFosVh8rB+YfYP0f1puap/FjjUvyeFvZl7qP9AnpOOwqO1K5qeiIiNyDul3qMviTwQCsf2M9+9P3m5qnWf9mRKdEY7FZ2D1rN4t+scjUM00iZlPRERG5R+0ntKfLr7oAkBaXRsa+DFPztBjaghGzR2BYDL75xzcs/dVS0+b8ETGbio6ISBno+25fGkU0ojC7kMRhieRfMXcCv9bRrYmcHgkGbJu6jZUvr1TZkUpJRUdEpAxYvaxEJUcR3CCYS4cuMXfMXNPvj2k7ri1DPhsCwNf/8zVrX1trah4RM6joiIiUkYDQAGLTY7H52Ti09BBrf29+sXjgqQcY8NcBAGx8cyMb/rjB5EQi7qWiIyJShmp3qM3Qvw8F4MspX7I3ea/JiUpWX+/zTh8A1v6/tXz9l69NTiTiPio64nZWi5WB9w1k4H0DtQSEVEhtxrSh20vdAJj/+HzOfnvW5ETw0EsPEfGHCABW/GYF2/62zdxAIm6ioiNu52vzZfGYxSwes1hLQEiF1fut3jTt25Si3CKSIpPIvZhrdiR6/L8ePDzpYQCWTFzCN//7jcmJRFxPRUdExAUsVgsj54ykapOqXDl6hdTYVNNnKjYMg55/7MmDv34QgAVPLeC7Wd+ZmknE1VR0RERcxK+aH7HpsXgFeHFk9RFW/tdKsyNhGAZ9/6cvnZ7pBE5IH5/Ovrn7zI4l4jIqOuJ2OYU5BPwpgIA/BWgJCKnwarapWTKfDbD5L5v5bqb5Z1AMw2DgRwNp/3h7nHYnc0fN5cDCA2bHEnEJFR0xRW5RLrlF5t+zIOIO4SPDeeTVRwBY+LOFnN5x2uREYFgMhnw+hPtH34+j2EFKVAo/rvjR7FgiZU5FR0TEDSLeiOC+QfdRnF9M0vAkcs6bfzbTYrUwPGE4rUa0wl5oJzEykaPrj5odS6RMqeiIiLiBxWphxMwRVG9enawTWaREp2AvspsdC4ut5Kbp+wbdR3FeMbMHzebEphNmxxIpMyo6IiJu4hviS2x6LN5VvDm24RjLf73c7EgAWL2txKTG0KRPE4pyipg1YBant5t/eU2kLKjoiIi4UWirUEbMGgGULLa58x87TU5UwuZrY1T6KBr2aEhBVgEz+s7g3HfnzI4lcs9UdERE3KzFkBbXZile8uwSTm4+aW6g/+Pl78XoRaOp92A98i/nk9A7gYzvM8yOJXJPVHTE7SyGhUcbPsqjDR/FYuhHUCqnHq/2oOXwltgL7SSNSOLqmatmRwLAp4oPY5eOpfYDtcnNyCWhVwKXDl0yO5bIXfO43zJTp06lUaNG+Pr60rVrV7Zu3Wp2JLlDfl5+rJuwjnUT1uHn5Wd2HBFTGBaDyOmRhIaHkn0mm+SRyRQXFJsdCyi5l2jcinGEtQkj+0w203tO58rRK2bHErkrHlV0kpKSePHFF5k8eTI7d+6kXbt29OvXj/Pnz5sdTUTkjvlU8WHU/FH4hvhy8uuTLHluCU6n0+xYAPhX9yduZRw1WtYg60QWCb0SyDqZZXYskTtmOMvLv6pS6Nq1K507d+ajjz4CwOFwUL9+fX75y1/yyiuv3Hb7rKwsgoODuXz5MkFBQa6OK1LuORwO9u/fD0DLli2xWDzqb58K49CyQ8wZPAecMHDqQDo+3dHsSNdcPX2V6RHTufzjZao1r8b4teMJrBVodiypZLKysqhatSqZmZl3/PvbY4pOYWEh/v7+pKamEhkZee3r48eP58qVK8yfP/+GbQoKCigoKLj2PCsri/r16/P1118TGKh/qGbJLc6l/9L+ACwbsAx/m7/JiUTMd+AfB9jz/h4Mm0GPv/egRscaZke6Jud0DhsmbCD3TC5BzYLo8b898KnqY3YsqUSys7Pp1q3bXRUdj/nz7cKFC9jtdmrWrHnd12vWrMnZs2dvus2UKVMIDg6+9qhfv747okopXC68zOXCy2bHECk3mj/RnHr96+EsdrL5xc3kni0/S6QE1AngkX88gm+YL1mHstj4840UZhaaHUukVGxmB3ClSZMm8eKLL157/s8zOi1bttSlKxPlFObA/52Aa9WyFQHeAeYGqsR06ap8uS/5Pr545AvOfXuOXa/sYvz68Xj5eZkdq0Q4NFnbhITHEsjcn8mOF3cwbvk4fIJ0ZkdcLyvr7u8P85iiU6NGDaxWK+fOXT+B1blz56hVq9ZNt/Hx8cHH58Z/hBaLRW/oJvr3//d6LcoPvRbm863iy6j0UXzW6TPO7DjD0meXMuyLYRiGYXY0AMLCw4hbFcf0iOmc3nqaxCGJjF02Fu8Ab7OjSQV3L+9NHvOu5u3tTceOHVm9evW1rzkcDlavXk23bt1MTCYiUnZCGoUQlRSFYTX4NuFbtny4xexI16nZpiZxK+PwCfbh+JfHSRyaSFFekdmxRG7JY4oOwIsvvsjnn3/O9OnT+f7773nmmWfIycnh8ccfNzuaiEiZadKrCX3f7QvAit+s4MiaIyYnul7tB2ozbtk4vAO9ObLmSLmaA0jkP3lU0YmNjeXdd9/ltddeo3379uzatYtly5bdcIOyiIin6/p8V9rGtcVpd5ISk1LuJuyr92A9xiwZg5e/F4eWHmLuqLnlYjV2kf/kUUUH4LnnnuPYsWMUFBSwZcsWunbtanYkuUMWw0KnOp3oVKeTloAQuQXDMBj86WBqd6xN3sU8EiMTKcotX5eIGj7SkFELRmH1sbI/fT9pcWk47A6zY4lcR79lxO38vPzY9rNtbPvZNi0BIfITvPy8iE2LxT/Un3PfnmPBkwvKzczJ/9SkVxNi58Vi8bKwN2kvC55YgNNRvjJK5aaiIyJSjgXXDyZmbgwWm4U9iXvY9M4msyPd4L6B9113A/WiZxaVu0ImlZeKjohIOdfwkYb0/7BkNvFVr6zi0LJDJie6UavhrRgxcwSGxWDnZztZ9sIylR0pF1R0xO1yi3Jp9H4jGr3fiNyi8jP7q0h51unpTnR4qgM4Ye7ouVw6dMnsSDe4f9T9DP3foQBs/XArqyetVtkR06noiNs5nU6OZR7jWOYxvQmKlJJhGAz8aCD1HqxH/pV8EiMTKbhacPsN3az9+PYM+mQQAF/9+SvW/2G9yYmkslPRERHxEDYfGzFzYwisHUjG3gzSx6eXyxt/O/2iE/3e7wfA+tfX8+WfvzQ5kVRmKjoiIh6kSp0qxM6LxeptZX/afjb+aaPZkW7qwecfpNeUXgCsfmU1mz/YbHIiqaxUdEREPEy9B+sxcOpAANa+tpYDCw+YnOjmHn7lYR6d/CgAy19YzvZPt5ucSCojFR0REQ/0wFMP0OnZTuCEeWPncWH/BbMj3dSjkx/lod8+BMDipxeza/oucwNJpaOiIyLiofq/35+GPRpSeLWQxGGJ5Gfmmx3pBoZh0Put3nT5VRcAFjyxgD2Je0xOJZWJio64nWEYhIeGEx4ajmEYZscR8VhWLyvRKdEE1Qvi4g8XmTd2Xrm8OdkwDPq/358Hfv4AToeTeePm8X3a92bHkkpCRUfczt/Ln73P7mXvs3vx9/I3O46IRwsICyA2PRabr42Diw+ydvJasyPdlGEYDP54MO3i2+G0O0mNTeXgkoNmx5JKQEVHRMTD1elYh8GfDQZg45sb2Td3n8mJbs6wGAz9x1Bax7bGUeQgaUQSh1cfNjuWVHAqOiIiFUC7uHY8+OsHAUgfn875PedNTnRzFpuF4TOG0zKyJfYCO3OGzOHYxmNmx5IKTEVH3C63KJfWf2tN67+11hIQImWoz9t9aNyzMUU5RSQOSyTvUp7ZkW7K6mVlZOJImvVvRnFeMbMHzubklpNmx5IKSkVH3M7pdLIvYx/7MvZpCQiRMmSxWYhKiiKkUQiXD19m7ui5OOwOs2PdlM3HRsy8GBr3bExhdiEz+83kzM4zZseSCkhFR0SkAvGv4U9seixe/l78uOJHVk9abXakW/Ly82LUglE0eLgBBZkFzOg7o9xechPPpaIjIlLB1GpXi2HThgGw6Z1N7J6z2+REt+Yd4M2YxWOo26UueRfzSOiVwIUD5XPyQ/FMKjoiIhVQ65jWdH+lOwALnlzAmW/K72UhnyAfxi4bS632tcg5n0NCzwQu/XjJ7FhSQajoiIhUUD3f7Hntht+kyCRyMnLMjnRLflX9iFsZR2jrUK6evkpCzwQyj2eaHUsqABUdEZEKymK1MGL2CKo1q0bm8UxSY1OxF9nNjnVL/jX8iV8VT/Xm1ck8nsn0ntO5evqq2bHEw6noiNsZhkHD4IY0DG6oJSBEXMyvqh+x6bF4B3pzdO1RVr680uxIPymwViDxq+MJaRzC5R8vk9ArgZzz5fdMlJR/Kjridv5e/hx94ShHXziqJSBE3CCsdRiRCZEAbPlgS7lfQTyoXhDj14wnqH4QF/ZfIKF3ArkXNeeW3B0VHRGRSqDV8Fb0eK0HAIt+sYhT206ZnOinhTQKIX51PIG1Azm/+zwz+80k/0r5W51dyj8VHRGRSiJicgQthrbAXmAnaXgS2WezzY70k6rfV534VfH4h/pzZscZZg2YRcHVArNjiYdR0RG3yyvKo/Pnnen8eWfyisrnFPUiFZFhMRg+Yzg1Wtbg6qmrJEclYy8svzcnA4SGhxK3Mg7fqr6c3HySOYPnUJRbZHYs8SAqOuJ2DqeD7ae3s/30dhzO8jk9vUhF5RPkw6j5o/AJ8uHEVydY+vxSsyPdVq12tYhbEYdPkA/HNhwjMTKR4vxis2OJh1DRERGpZKo3r86I2SPAgB2f7GDHZzvMjnRbdTrVYezSsXgFeHF45WGPOBsl5YOKjohIJdR8UHN6vtkTgCXPLeHEphMmJ7q9+g/VZ8yiMdh8bRxcfJC5Y+biKNZZYflpKjoiIpXUw5MeJjwqHEeRg+SRyWSdyjI70m01imjEqPmjsHpb+X7u96SPTy+3K7RL+aCiIyJSSRmGwbBpwwi7P4zss9kkj0j2iHtfmvZtSnRqNBabhd2zd7PwZwtxOpxmx5JySkVHRKQS8w70JjY9Ft+qvpzaeorFzy7G6Sz/paHFkBaMnDMSw2Kwa9ouljy3xCNyi/up6IgpavjXoIZ/DbNjiAhQrWk1opKirpWGbVO3mR2pVMKjwktmfDZg+8fbWfGbFSo7cgMVHXG7AO8AMl7OIOPlDAK8A8yOIyJA0z5N6f12bwCWvbCMo+uOmhuolNqObcvQvw8FYPN7m1nz/9aYnEjKGxUdEREBoNuL3Wgzpg1Ou5OU6BQyj2eaHalUOjzRgQEfDQDgyz99yYY3N5icSMoTFR0REQFKbk4e8vkQanWoRe6FXBIjEz1mFuIuE7vQ93/6ArD292vZ9O4mkxNJeaGiI26XV5RHxBcRRHwRoSUgRMoZL38vYtNi8a/hz9lvzrLw5ws95r6Xbi9247E3HwNg5csr2frRVpMTSXmgoiNu53A6WH9sPeuPrdcSECLlUEjDEKJTojGsBrtn7Wbze5vNjlRqPV7twSOvPgLA0l8uZeffd5qcSMymoiMiIjdoFNGIfu/1A0rOjhxeddjkRKX32H8/RrffdANg4c8X8t3M70xOJGZS0RERkZvq8lwX2k9oj9PhJDU2lcuHL5sdqVQMw6DPO33oPLEzOCF9fDp7U/aaHUtMoqIjIiI3ZRgGgz4eRN0udcm7lEdiZCKF2YVmxyoVwzAY8OEAOjzZAafDybwx8ziw4IDZscQEKjoiInJLNl8bMfNiCKgZwPnd55n/+HyPuTnZsBgM/nQwbca2wVHsICU6hUPLD5kdS9xMRUdERH5SUN0gYubGYPGysC91H1++9aXZkUrNYrUQ+UUk4VHh2AvtJEUmcWTtEbNjiRup6Igp/L388ffyNzuGiJRSg+4NGPjRQADWvLqGg0sOmpyo9Cw2CyNmjaD5kOYU5xczZ8gcjn913OxY4iYqOuJ2Ad4B5Pwuh5zf5WgJCBEP0vHnHen4i47ghLlj5nLxh4tmRyo1q7eV6ORomvZtSlFOEbMGzOLUtlNmxxI3UNEREZFSG/DhAOp3r09BZgGJkYkUZBWYHanUbL42YtNiaRTRiMKrhczsN5Oz3541O5a4mIqOiIiUmtXbSkxqDFXqVOHC9xdIi0/D6fCMm5OhZObn0QtHU/+h+uRfzmdG7xlk7MswO5a4kIqOuF1+cT6DZg9i0OxB5Bfnmx1HRO5QYK1AYtNisfpYOTD/AOv/sN7sSHfEO9CbMUvGUKdTHXIv5JLQK4GLBz3nMpzcGRUdcTu7w86Sg0tYcnAJdofd7DgichfqdqnL4E8GA7D+jfXsT99vcqI74xvsy7jl46jZtibZZ7NJ6JnA5SOeMSGi3BkVHRERuSvtJ7Sny6+6AJAWl+Zxl4D8qvkRtzKOGq1qkHUyi4ReCWSeyDQ7lpQxFR0REblrfd/tW3Jzb3YhicMSyb/iWZejA8ICiF8dT7Vm1bhy5AoJvRK4euaq2bGkDHlM0fnjH//IQw89hL+/PyEhIWbHERERwOplJSo5iuAGwVw6dIm5Y+bisDvMjnVHqtSuQvzqeIIbBnPp4CVm9J5BTkaO2bGkjHhM0SksLCQ6OppnnnnG7CgiIvJvAkIDiE2PxeZn49DSQ6z9/VqzI92x4AbBjF8znip1q5CxL4OZfWeSdynP7FhSBjym6Lzxxhv8+te/pk2bNmZHERGR/1C7Q22G/n0oAF9O+ZK9yZ63WnjVJlUZv2Y8ATUDOLvrLDP7zyQ/07MuxcmNbGYHcKWCggIKCv41mVVmZslNZllZWWZFEiCnMAf+770jKysLu7c+eWUWh8NBdnY2UPJaWCwe87ePlEMNBzek7S/bsvWvW0makERc3Thqtqlpdqw74lXLi+HzhzN74GwObzvMP/r9g9h5sXgHepsdrVL75+/tu1pQ1ulhpk2b5gwODi7V2MmTJzsBPfTQQw899NCjAjx+/PHHO+4Npp7ReeWVV/jzn//8k2O+//57WrZseVf7nzRpEi+++OK151euXKFhw4YcP36c4ODgu9qnlI2srCzq16/PiRMnCAoKMjtOpabXovzQa1G+6PUoPzIzM2nQoAHVqlW7421NLTq/+c1vmDBhwk+OadKkyV3v38fHBx8fnxu+HhwcrB/aciIoKEivRTmh16L80GtRvuj1KD/u5vK6qUUnNDSU0NBQMyOIiIhIBeYxNyMfP36cS5cucfz4cex2O7t27QKgWbNmBAYGmhtOREREyiWPKTqvvfYa06dPv/a8Q4cOAKxdu5aIiIhS7cPHx4fJkyff9HKWuJdei/JDr0X5odeifNHrUX7cy2thOJ1381ktERERkfJPk2aIiIhIhaWiIyIiIhWWio6IiIhUWCo6IiIiUmFVyqJz9OhRnnzySRo3boyfnx9NmzZl8uTJFBYWmh2tUvrjH//IQw89hL+/PyEhIWbHqXSmTp1Ko0aN8PX1pWvXrmzdutXsSJXShg0bGDJkCHXq1MEwDNLT082OVClNmTKFzp07U6VKFcLCwoiMjOTAgQNmx6qUPv74Y9q2bXttwsZu3bqxdOnSO95PpSw6+/fvx+Fw8Omnn7J3717ee+89PvnkE373u9+ZHa1SKiwsJDo6mmeeecbsKJVOUlISL774IpMnT2bnzp20a9eOfv36cf78ebOjVTo5OTm0a9eOqVOnmh2lUlu/fj0TJ05k8+bNrFy5kqKiIvr27UtOTo7Z0SqdevXq8dZbb7Fjxw62b99Oz549GTZsGHv37r2j/ejj5f/nnXfe4eOPP+bw4cNmR6m0vvjiC1544QWuXLlidpRKo2vXrnTu3JmPPvoIKFnNvH79+vzyl7/klVdeMTld5WUYBmlpaURGRpodpdLLyMggLCyM9evX06NHD7PjVHrVqlXjnXfe4cknnyz1NpXyjM7NZGZm3tViYSKeqrCwkB07dtC7d+9rX7NYLPTu3Zuvv/7axGQi5UdmZiaAfj+YzG63k5iYSE5ODt26dbujbT1mZmRXOnToEH/961959913zY4i4jYXLlzAbrdTs2bN675es2ZN9u/fb1IqkfLD4XDwwgsv0L17d+6//36z41RKu3fvplu3buTn5xMYGEhaWhrh4eF3tI8KdUbnlVdewTCMn3z85xv4qVOn6N+/P9HR0fzsZz8zKXnFczevhYhIeTJx4kT27NlDYmKi2VEqrRYtWrBr1y62bNnCM888w/jx49m3b98d7aNCndH5zW9+w4QJE35yTJMmTa799+nTp3nsscd46KGH+Oyzz1ycrnK509dC3K9GjRpYrVbOnTt33dfPnTtHrVq1TEolUj4899xzLFq0iA0bNlCvXj2z41Ra3t7eNGvWDICOHTuybds2PvjgAz799NNS76NCFZ3Q0FBCQ0NLNfbUqVM89thjdOzYkWnTpmGxVKiTW6a7k9dCzOHt7U3Hjh1ZvXr1tZteHQ4Hq1ev5rnnnjM3nIhJnE4nv/zlL0lLS2PdunU0btzY7EjybxwOBwUFBXe0TYUqOqV16tQpIiIiaNiwIe+++y4ZGRnXvqe/ZN3v+PHjXLp0iePHj2O329m1axcAzZo1IzAw0NxwFdyLL77I+PHj6dSpE126dOH9998nJyeHxx9/3OxolU52djaHDh269vzIkSPs2rWLatWq0aBBAxOTVS4TJ05k9uzZzJ8/nypVqnD27FkAgoOD8fPzMzld5TJp0iQGDBhAgwYNuHr1KrNnz2bdunUsX778znbkrISmTZvmBG76EPcbP378TV+LtWvXmh2tUvjrX//qbNCggdPb29vZpUsX5+bNm82OVCmtXbv2pv8Oxo8fb3a0SuVWvxumTZtmdrRK54knnnA2bNjQ6e3t7QwNDXX26tXLuWLFijvej+bRERERkQpLN6aIiIhIhaWiIyIiIhWWio6IiIhUWCo6IiIiUmGp6IiIiEiFpaIjIiIiFZaKjoiIiFRYKjoi4tmSImDtC2anEJFySkVHREREKiwVHREREamwVHRExPM5imH1c/DXYJhaA776PWh1GxFBRUdEKoK908Fig7FboecHsP0vsPvvZqcSkXLAZnYAEZF7VqU+RLwHhgHVWkDGbtjxHrT9mdnJRMRkOqMjIp6v9oMlJeef6nSDKwfBYTcvk4iUCyo6IiIiUmGp6IiI5zu75frnZzZDyH1gsZqTR0TKDRUdEfF8Wcdh3Ytw6QB8Pwe++Ss88LzZqUSkHNDNyCLi+cLjoTgPZnUpOYvzwPPQ9udmpxKRcsBwOjXZhIiIiFRMunQlIiIiFZaKjoiIiFRYKjoiIiJSYanoiIiISIWloiMiIiIVloqOiIiIVFgqOiIiIlJhqeiIiIhIhaWiIyIiIhWWio6IiIhUWCo6IiIiUmGp6IiIiEiF9f8BVB9Vgo/fYFEAAAAASUVORK5CYII=\n" - }, - "metadata": {} - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "14uhePTna7ZV" - }, - "source": [ - "In PyTorch and TensorFlow:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "7qo-SAvaansp", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "3bca970e-71dc-4072-dff7-3c59a0fb9b37" - }, - "source": [ - "torch.inverse(torch.tensor([[4, 2], [-5, -3.]])) # float type" - ], - "execution_count": 163, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[ 1.5000, 1.0000],\n", - " [-2.5000, -2.0000]])" - ] - }, - "metadata": {}, - "execution_count": 163 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "uqtyF3Jqaz4l", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "2960b1e8-66a2-40be-eb09-a86754ab596c" - }, - "source": [ - "tf.linalg.inv(tf.Variable([[4, 2], [-5, -3.]])) # also float" - ], - "execution_count": 164, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "" - ] - }, - "metadata": {}, - "execution_count": 164 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "hcP2S4AMAHT_" - }, - "source": [ - "**Exercises**:\n", - "\n", - "1. As done with NumPy above, use PyTorch to calculate $w$ from $X$ and $y$. Subsequently, confirm that $y = Xw$.\n", - "2. Repeat again, now using TensorFlow." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "AMxROg326SNN" - }, - "source": [ - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "N8ZxpgcN6SNO" - }, - "source": [ - "### Matrix Inversion Where No Solution" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "RYORHY4E6SNO", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "ac15d9a5-90e8-4003-cd2e-1b2bccc6381c" - }, - "source": [ - "X = np.array([[-4, 1], [-8, 2]])\n", - "X" - ], - "execution_count": 165, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[-4, 1],\n", - " [-8, 2]])" - ] - }, - "metadata": {}, - "execution_count": 165 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "o7GcISdI6SNP" - }, - "source": [ - "# Uncommenting the following line results in a \"singular matrix\" error\n", - "# Xinv = np.linalg.inv(X)" - ], - "execution_count": 166, - "outputs": [] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "uk485dwoI021" - }, - "source": [ - "Feel free to try inverting a non-square matrix; this will throw an error too.\n", - "\n", - "**Return to slides here.**" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "BXomEBoyDQyO" - }, - "source": [ - "### Orthogonal Matrices\n", - "\n", - "These are the solutions to Exercises 3 and 4 on **orthogonal matrices** from the slides.\n", - "\n", - "For Exercise 3, to demonstrate the matrix $I_3$ has mutually orthogonal columns, we show that the dot product of any pair of columns is zero:" - ] - }, - { - "cell_type": "code", - "metadata": { - "id": "-Un4Aq805HM1", - "colab": { - "base_uri": "https://localhost:8080/" - }, - "outputId": "b3370a08-816a-4d36-a123-149db7fb661e" - }, - "source": [ - "I = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])\n", - "I" - ], - "execution_count": 167, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([[1, 0, 0],\n", - " [0, 1, 0],\n", - " [0, 0, 1]])" - ] - }, - "metadata": {}, - "execution_count": 167 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "CUhMi2_IEyaI", - "outputId": "0b7c751a-0cd2-49bb-dde5-8e3895d7a12d" - }, - "source": [ - "column_1 = I[:,0]\n", - "column_1" - ], - "execution_count": 168, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([1, 0, 0])" - ] - }, - "metadata": {}, - "execution_count": 168 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "xsOtn-siE63K", - "outputId": "a334201a-5e98-405b-d430-24e74307ac6d" - }, - "source": [ - "column_2 = I[:,1]\n", - "column_2" - ], - "execution_count": 169, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([0, 1, 0])" - ] - }, - "metadata": {}, - "execution_count": 169 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Pv3b51ExE9ZX", - "outputId": "2cd0a54e-2440-4fef-ff71-c0c0401cc94f" - }, - "source": [ - "column_3 = I[:,2]\n", - "column_3" - ], - "execution_count": 170, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "array([0, 0, 1])" - ] - }, - "metadata": {}, - "execution_count": 170 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "VhnRlc8eElYc", - "outputId": "2473373b-936c-45e8-8ef2-a305351a9390" - }, - "source": [ - "np.dot(column_1, column_2)" - ], - "execution_count": 171, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "0" - ] - }, - "metadata": {}, - "execution_count": 171 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "mVq4m-LfFGjQ", - "outputId": "d1b8a9b5-8b20-44d0-ce59-1a6b7ba05c2c" - }, - "source": [ - "np.dot(column_1, column_3)" - ], - "execution_count": 172, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "0" - ] - }, - "metadata": {}, - "execution_count": 172 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "nJj-aihzFILx", - "outputId": "e11e422d-0268-4830-d840-22ebd21db873" - }, - "source": [ - "np.dot(column_2, column_3)" - ], - "execution_count": 173, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "0" - ] - }, - "metadata": {}, - "execution_count": 173 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "FIzpkdJmFYdY" - }, - "source": [ - "We can use the `np.linalg.norm()` method from earlier in the notebook to demonstrate that each column of $I_3$ has unit norm:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "xKbbeqegFJSc", - "outputId": "b496a162-a447-44b8-cad8-ce768afda24b" - }, - "source": [ - "np.linalg.norm(column_1)" - ], - "execution_count": 174, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "1.0" - ] - }, - "metadata": {}, - "execution_count": 174 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Ycv0L8mpGKRC", - "outputId": "1dad31ec-d76d-450c-ae53-4388241db990" - }, - "source": [ - "np.linalg.norm(column_2)" - ], - "execution_count": 175, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "1.0" - ] - }, - "metadata": {}, - "execution_count": 175 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "LsEuRezYGLgY", - "outputId": "44d2899e-fef2-499f-9051-002e0b9deb33" - }, - "source": [ - "np.linalg.norm(column_3)" - ], - "execution_count": 176, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "1.0" - ] - }, - "metadata": {}, - "execution_count": 176 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "NMrHdyrsGQ1X" - }, - "source": [ - "Since the matrix $I_3$ has mutually orthogonal columns and each column has unit norm, the column vectors of $I_3$ are *orthonormal*. Since $I_3^T = I_3$, this means that the *rows* of $I_3$ must also be orthonormal.\n", - "\n", - "Since the columns and rows of $I_3$ are orthonormal, $I_3$ is an *orthogonal matrix*." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "K7EykcPdIZhE" - }, - "source": [ - "For Exercise 4, let's repeat the steps of Exercise 3 with matrix *K* instead of $I_3$. We could use NumPy again, but for fun I'll use PyTorch instead. (You're welcome to try it with TensorFlow if you feel so inclined.)" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "MXVyu8tSGMlZ", - "outputId": "454c991b-1357-4a7e-f642-de51ad634095" - }, - "source": [ - "K = torch.tensor([[2/3, 1/3, 2/3], [-2/3, 2/3, 1/3], [1/3, 2/3, -2/3]])\n", - "K" - ], - "execution_count": 177, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[ 0.6667, 0.3333, 0.6667],\n", - " [-0.6667, 0.6667, 0.3333],\n", - " [ 0.3333, 0.6667, -0.6667]])" - ] - }, - "metadata": {}, - "execution_count": 177 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "xqkmSYNfJmTs", - "outputId": "5b314b21-1200-45f0-b9ee-02ff3a76319f" - }, - "source": [ - "Kcol_1 = K[:,0]\n", - "Kcol_1" - ], - "execution_count": 178, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([ 0.6667, -0.6667, 0.3333])" - ] - }, - "metadata": {}, - "execution_count": 178 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "ncoc3IxNJ0v2", - "outputId": "9c0de5c0-1311-4d2a-f2a8-c8b035799afe" - }, - "source": [ - "Kcol_2 = K[:,1]\n", - "Kcol_2" - ], - "execution_count": 179, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([0.3333, 0.6667, 0.6667])" - ] - }, - "metadata": {}, - "execution_count": 179 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "ISEKQ0AqJ2wf", - "outputId": "beaa5849-4d3c-4af7-cead-e4cd90faee99" - }, - "source": [ - "Kcol_3 = K[:,2]\n", - "Kcol_3" - ], - "execution_count": 180, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([ 0.6667, 0.3333, -0.6667])" - ] - }, - "metadata": {}, - "execution_count": 180 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "jvdXuMzbJ44d", - "outputId": "8c35ee3b-9dbd-4a2e-c11a-857fa688d535" - }, - "source": [ - "torch.dot(Kcol_1, Kcol_2)" - ], - "execution_count": 181, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(0.)" - ] - }, - "metadata": {}, - "execution_count": 181 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "BUR60eBuKEbU", - "outputId": "abd95aa8-5557-4b29-871a-e09409095f62" - }, - "source": [ - "torch.dot(Kcol_1, Kcol_3)" - ], - "execution_count": 182, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(0.)" - ] - }, - "metadata": {}, - "execution_count": 182 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Rg7QvQwuKGJZ", - "outputId": "bf55069c-186d-4a53-9b68-84b42e5e4454" - }, - "source": [ - "torch.dot(Kcol_2, Kcol_3)" - ], - "execution_count": 183, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(0.)" - ] - }, - "metadata": {}, - "execution_count": 183 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "wNdV36-QKNK-" - }, - "source": [ - "We've now determined that the columns of $K$ are orthogonal." - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "DDgyhj1AKHdS", - "outputId": "ae72ed8b-2617-4cd4-91ec-7c94a1ab94d7" - }, - "source": [ - "torch.norm(Kcol_1)" - ], - "execution_count": 184, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(1.)" - ] - }, - "metadata": {}, - "execution_count": 184 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "KvQZy_E2KorW", - "outputId": "10824173-467d-45c4-e931-4e65bec68229" - }, - "source": [ - "torch.norm(Kcol_2)" - ], - "execution_count": 185, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(1.)" - ] - }, - "metadata": {}, - "execution_count": 185 - } - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "7H7dx8a7Kp9f", - "outputId": "7de33d74-e385-4542-9dd7-375ae744a50a" - }, - "source": [ - "torch.norm(Kcol_3)" - ], - "execution_count": 186, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor(1.)" - ] - }, - "metadata": {}, - "execution_count": 186 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "VPPtOk5OKtJ_" - }, - "source": [ - "We've now determined that, in addition to being orthogonal, the columns of $K$ have unit norm, therefore they are orthonormal.\n", - "\n", - "To ensure that $K$ is an orthogonal matrix, we would need to show that not only does it have orthonormal columns but it has orthonormal rows are as well. Since $K^T \\neq K$, we can't prove this quite as straightforwardly as we did with $I_3$.\n", - "\n", - "One approach would be to repeat the steps we used to determine that $K$ has orthogonal columns with all of the matrix's rows (please feel free to do so). Alternatively, we can use an orthogonal matrix-specific equation from the slides, $A^TA = I$, to demonstrate that $K$ is orthogonal in a single line of code:" - ] - }, - { - "cell_type": "code", - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "W8Ao0KhVNHqF", - "outputId": "3a3ae6c8-91c3-456a-9060-5b6e0524cde6" - }, - "source": [ - "torch.matmul(K.T, K)" - ], - "execution_count": 187, - "outputs": [ - { - "output_type": "execute_result", - "data": { - "text/plain": [ - "tensor([[ 1.0000e+00, -3.3114e-09, 3.3114e-09],\n", - " [-3.3114e-09, 1.0000e+00, 6.6227e-09],\n", - " [ 3.3114e-09, 6.6227e-09, 1.0000e+00]])" - ] - }, - "metadata": {}, - "execution_count": 187 - } - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "scM6FCpjNgYA" - }, - "source": [ - "Notwithstanding rounding errors that we can safely ignore, this confirms that $K^TK = I$ and therefore $K$ is an orthogonal matrix." - ] - } - ] -} \ No newline at end of file + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "colab_type": "text", + "id": "view-in-github" + }, + "source": [ + "\"Open" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "aTOLgsbN69-P" + }, + "source": [ + "# Intro to Linear Algebra" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "yqUB9FTRAxd-" + }, + "source": [ + "This topic, *Intro to Linear Algebra*, is the first in the *Machine Learning Foundations* series.\n", + "\n", + "It is essential because linear algebra lies at the heart of most machine learning approaches and is especially predominant in deep learning, the branch of ML at the forefront of today’s artificial intelligence advances. Through the measured exposition of theory paired with interactive examples, you’ll develop an understanding of how linear algebra is used to solve for unknown values in high-dimensional spaces, thereby enabling machines to recognize patterns and make predictions.\n", + "\n", + "The content covered in *Intro to Linear Algebra* is itself foundational for all the other topics in the Machine Learning Foundations series and it is especially relevant to *Linear Algebra II*." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "d4tBvI88BheF" + }, + "source": [ + "Over the course of studying this topic, you'll:\n", + "\n", + "* Understand the fundamentals of linear algebra, a ubiquitous approach for solving for unknowns within high-dimensional spaces.\n", + "\n", + "* Develop a geometric intuition of what’s going on beneath the hood of machine learning algorithms, including those used for deep learning.\n", + "* Be able to more intimately grasp the details of machine learning papers as well as all of the other subjects that underlie ML, including calculus, statistics, and optimization algorithms." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Z68nQ0ekCYhF" + }, + "source": [ + "**Note that this Jupyter notebook is not intended to stand alone. It is the companion code to a lecture or to videos from Jon Krohn's [Machine Learning Foundations](https://github.com/jonkrohn/ML-foundations) series, which offer detail on the following:**\n", + "\n", + "*Segment 1: Data Structures for Algebra*\n", + "\n", + "* What Linear Algebra Is \n", + "* A Brief History of Algebra\n", + "* Tensors\n", + "* Scalars\n", + "* Vectors and Vector Transposition\n", + "* Norms and Unit Vectors\n", + "* Basis, Orthogonal, and Orthonormal Vectors\n", + "* Arrays in NumPy \n", + "* Matrices\n", + "* Tensors in TensorFlow and PyTorch\n", + "\n", + "*Segment 2: Common Tensor Operations*\n", + "\n", + "* Tensor Transposition\n", + "* Basic Tensor Arithmetic\n", + "* Reduction\n", + "* The Dot Product\n", + "* Solving Linear Systems\n", + "\n", + "*Segment 3: Matrix Properties*\n", + "\n", + "* The Frobenius Norm\n", + "* Matrix Multiplication\n", + "* Symmetric and Identity Matrices\n", + "* Matrix Inversion\n", + "* Diagonal Matrices\n", + "* Orthogonal Matrices\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "2khww76J5w9n" + }, + "source": [ + "## Segment 1: Data Structures for Algebra\n", + "\n", + "**Slides used to begin segment, with focus on introducing what linear algebra is, including hands-on paper and pencil exercises.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "niG_MgK-iV6K" + }, + "source": [ + "### What Linear Algebra Is" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "LApX90aliab_" + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "E4odh9Shic1S" + }, + "outputs": [], + "source": [ + "t = np.linspace(0, 40, 1000) # start, finish, n points" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "N-tYny12nIyO" + }, + "source": [ + "Distance travelled by robber: $d = 2.5t$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "e_zDOxgHiezz" + }, + "outputs": [], + "source": [ + "d_r = 2.5 * t" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "djVjXZy-nPaR" + }, + "source": [ + "Distance travelled by sheriff: $d = 3(t-5)$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "JtaNeYSCifrI" + }, + "outputs": [], + "source": [ + "d_s = 3 * (t-5)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 472 + }, + "id": "SaaIjJSEigic", + "outputId": "c354282a-095b-4318-de3a-f26e4c45e354" + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "plt.title('A Bank Robber Caught')\n", + "plt.xlabel('time (in minutes)')\n", + "plt.ylabel('distance (in km)')\n", + "ax.set_xlim([0, 40])\n", + "ax.set_ylim([0, 100])\n", + "ax.plot(t, d_r, c='green')\n", + "ax.plot(t, d_s, c='brown')\n", + "plt.axvline(x=30, color='purple', linestyle='--')\n", + "_ = plt.axhline(y=75, color='purple', linestyle='--')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kpwZw64EYfs6" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "NgGMhK4B51oe" + }, + "source": [ + "### Scalars (Rank 0 Tensors) in Base Python" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ZXnTHDn_EW6b", + "outputId": "02e2301e-5913-47d0-a31d-aaf344084770" + }, + "outputs": [], + "source": [ + "x = 25\n", + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "VF8Jam76R4KJ", + "outputId": "d5bb3326-7b99-4489-d7dc-386cb76ba951" + }, + "outputs": [], + "source": [ + "type(x) # if we'd like more specificity (e.g., int16, uint8), we need NumPy or another numeric library" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ZBzYlL0mRd-P" + }, + "outputs": [], + "source": [ + "y = 3" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1i-hW0bcReyy", + "outputId": "ae377616-ba20-4b4b-938b-178a10bf8ff4" + }, + "outputs": [], + "source": [ + "py_sum = x + y\n", + "py_sum" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CpyUxB6XRk6y", + "outputId": "f5fe891a-b958-4b2a-d87d-1e103ca2ca76" + }, + "outputs": [], + "source": [ + "type(py_sum)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "V2UiLj-JR8Ij", + "outputId": "8a1a49bb-88e3-47ba-a1c9-a03136330b6d" + }, + "outputs": [], + "source": [ + "x_float = 25.0\n", + "float_sum = x_float + y\n", + "float_sum" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ikOwjp6ASCaf", + "outputId": "691b7803-b978-4598-c804-3ef53193f0cd" + }, + "outputs": [], + "source": [ + "type(float_sum)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SgUvioyUz8T2" + }, + "source": [ + "### Scalars in PyTorch\n", + "\n", + "* PyTorch and TensorFlow are the two most popular *automatic differentiation* libraries (a focus of the [*Calculus I*](https://github.com/jonkrohn/ML-foundations/blob/master/notebooks/3-calculus-i.ipynb) and [*Calculus II*](https://github.com/jonkrohn/ML-foundations/blob/master/notebooks/4-calculus-ii.ipynb) subjects in the *ML Foundations* series) in Python, itself the most popular programming language in ML.\n", + "* PyTorch tensors are designed to be pythonic, i.e., to feel and behave like NumPy arrays.\n", + "* The advantage of PyTorch tensors relative to NumPy arrays is that they easily be used for operations on GPU (see [here](https://pytorch.org/tutorials/beginner/examples_tensor/two_layer_net_tensor.html) for example).\n", + "* Documentation on PyTorch tensors, including available data types, is [here](https://pytorch.org/docs/stable/tensors.html)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "A9Hhazt2zKeD" + }, + "outputs": [], + "source": [ + "import torch" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "a211IRW_0-iY", + "outputId": "18107a49-adb2-45c8-ba6f-1851beeb75af" + }, + "outputs": [], + "source": [ + "x_pt = torch.tensor(25) # type specification optional, e.g.: dtype=torch.float16\n", + "x_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LvxzMa_HhUNB", + "outputId": "9a3a10fe-7a4f-427c-cfb3-1311fe2ea511" + }, + "outputs": [], + "source": [ + "x_pt.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eUyuZXlWS8T9" + }, + "source": [ + "### Scalars in TensorFlow (version 2.0 or later)\n", + "\n", + "Tensors created with a wrapper, all of which [you can read about here](https://www.tensorflow.org/guide/tensor): \n", + "\n", + "* `tf.Variable`\n", + "* `tf.constant`\n", + "* `tf.placeholder`\n", + "* `tf.SparseTensor`\n", + "\n", + "Most widely-used is `tf.Variable`, which we'll use here.\n", + "\n", + "As with TF tensors, in PyTorch we can similarly perform operations, and we can easily convert to and from NumPy arrays.\n", + "\n", + "Also, a full list of tensor data types is available [here](https://www.tensorflow.org/api_docs/python/tf/dtypes/DType)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "CHBYse_MEqZM" + }, + "outputs": [], + "source": [ + "import tensorflow as tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "sDv92Nh-NSOU", + "outputId": "33521159-174d-4b14-89b3-495c8bf205da" + }, + "outputs": [], + "source": [ + "x_tf = tf.Variable(25, dtype=tf.int16) # dtype is optional\n", + "x_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "EmPMBIV9RQjS", + "outputId": "d104c2a4-e776-4b61-82d4-8b23c9ada5f7" + }, + "outputs": [], + "source": [ + "x_tf.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "mEILtO9pPctO" + }, + "outputs": [], + "source": [ + "y_tf = tf.Variable(3, dtype=tf.int16)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "dvvWuaw6Ph_D", + "outputId": "928a5ae6-d456-453f-a695-1704f9654b3d" + }, + "outputs": [], + "source": [ + "x_tf + y_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "JZVhRnX9RUGW", + "outputId": "7170b45f-858e-4497-848a-92bada682572" + }, + "outputs": [], + "source": [ + "tf_sum = tf.add(x_tf, y_tf)\n", + "tf_sum" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "sVbMxT1Ey6Y3", + "outputId": "63b9e20e-724f-4b10-960b-79635d034d4c" + }, + "outputs": [], + "source": [ + "tf_sum.numpy() # note that NumPy operations automatically convert tensors to NumPy arrays, and vice versa" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LXpv69t0y-f6", + "outputId": "d728326e-38d4-48ae-9224-925febcec085" + }, + "outputs": [], + "source": [ + "type(tf_sum.numpy())" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "VszuTUAg1uXk", + "outputId": "cf5b7af9-3f1c-4ed4-dff1-dc8459d90567" + }, + "outputs": [], + "source": [ + "tf_float = tf.Variable(25., dtype=tf.float16)\n", + "tf_float" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "B5VRGo1H6010" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4CURG9Er6aZI" + }, + "source": [ + "### Vectors (Rank 1 Tensors) in NumPy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "T9ME4kBr4wg0", + "outputId": "fe23648e-9fda-4e84-8c45-d05852642b1f" + }, + "outputs": [], + "source": [ + "x = np.array([25, 2, 5]) # type argument is optional, e.g.: dtype=np.float16\n", + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ZuotxmlZL2wp", + "outputId": "46573b45-8ff1-4d80-a0be-11322fc7a7e6" + }, + "outputs": [], + "source": [ + "len(x)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OlPYy6GOaIVy", + "outputId": "b9e7b1d6-c780-4fad-8737-0e7c7dfc3353" + }, + "outputs": [], + "source": [ + "x.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "sWbYGwObcgtK", + "outputId": "489b8c9a-0a29-40e2-f294-09216be93fac" + }, + "outputs": [], + "source": [ + "type(x)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ME_xuvD_oTPg", + "outputId": "36265cc4-43b2-4510-d296-a0b7d6846979" + }, + "outputs": [], + "source": [ + "x[0] # zero-indexed" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hXmBHZQ-nxFw", + "outputId": "ddb7f5f0-a04b-437d-fda8-5d9c8503ab2e" + }, + "outputs": [], + "source": [ + "type(x[0])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "NiEofCzYZBrQ" + }, + "source": [ + "### Vector Transposition" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hxGFNDx6V95l", + "outputId": "c76eb8db-2df5-41d3-ef83-87df72f7a4d7" + }, + "outputs": [], + "source": [ + "# Transposing a regular 1-D array has no effect...\n", + "x_t = x.T\n", + "x_t" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "_f8E9ExDWw4p", + "outputId": "024104e6-7d16-49f9-e886-e1c9a08c756f" + }, + "outputs": [], + "source": [ + "x_t.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "AEd8jB7YcgtT", + "outputId": "b5489d91-8f91-4582-e990-0d02ecdd4095" + }, + "outputs": [], + "source": [ + "# ...but it does we use nested \"matrix-style\" brackets:\n", + "y = np.array([[25, 2, 5]])\n", + "y" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "UHQd92oRcgtV", + "outputId": "19065b81-34fa-41ba-b65c-7f65efca1478" + }, + "outputs": [], + "source": [ + "y.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SPi1JqGEXXUc", + "outputId": "12f118dd-a5bd-491e-9e1f-73f4a265426c" + }, + "outputs": [], + "source": [ + "# ...but can transpose a matrix with a dimension of length 1, which is mathematically equivalent:\n", + "y_t = y.T\n", + "y_t" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "6rzUv762Yjis", + "outputId": "39c8296e-f074-4010-dc21-b79325a8ac3a" + }, + "outputs": [], + "source": [ + "y_t.shape # this is a column vector as it has 3 rows and 1 column" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "xVnQMLOrYtra", + "outputId": "2117f2c2-dc95-42c9-af92-507f5b1a97ec" + }, + "outputs": [], + "source": [ + "# Column vector can be transposed back to original row vector:\n", + "y_t.T" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "QIAA2NLRZIXC", + "outputId": "8f77c7c5-7fe1-49ae-d4cd-dc232ef944f4" + }, + "outputs": [], + "source": [ + "y_t.T.shape" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Voj26mSpZLuh" + }, + "source": [ + "### Zero Vectors\n", + "\n", + "Have no effect if added to another vector" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-46AbOdkZVn_", + "outputId": "48e0ffe6-6c0b-40b9-ca08-1a68178da82b" + }, + "outputs": [], + "source": [ + "z = np.zeros(3)\n", + "z" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "c6xyYiwSnSGC" + }, + "source": [ + "### Vectors in PyTorch and TensorFlow" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "s2TGDeqXnitZ", + "outputId": "dca2930e-9dc8-4db3-bff1-7a0771b0be76" + }, + "outputs": [], + "source": [ + "x_pt = torch.tensor([25, 2, 5])\n", + "x_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-0jbHgc5iijG", + "outputId": "6c7b8c17-b848-44d2-8ba0-da1296a2d0c4" + }, + "outputs": [], + "source": [ + "x_tf = tf.Variable([25, 2, 5])\n", + "x_tf" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rTDDta1Ro4Pf" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "8fU5qVTI6SLD" + }, + "source": [ + "### $L^2$ Norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "lLc2FbGG6SLD", + "outputId": "d2baf173-70e9-4f1f-f773-64803527483d" + }, + "outputs": [], + "source": [ + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "AN43hsl86SLG", + "outputId": "c821c7fd-5aa7-4c77-ccee-feeb90c1722f" + }, + "outputs": [], + "source": [ + "(25**2 + 2**2 + 5**2)**(1/2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "D9CyWo-l6SLI", + "outputId": "936d1404-2538-4511-9db8-c5b5806bc5e4" + }, + "outputs": [], + "source": [ + "np.linalg.norm(x)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "TNEMRi926SLK" + }, + "source": [ + "So, if units in this 3-dimensional vector space are meters, then the vector $x$ has a length of 25.6m" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ugQC6k4h6SLK" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PwiRlMuC6SLK" + }, + "source": [ + "### $L^1$ Norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "lcYKyc5H6SLL", + "outputId": "905fe7dc-6b8e-4e8b-a65d-839c3564e1c5" + }, + "outputs": [], + "source": [ + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "8jNb6nYl6SLM", + "outputId": "629afdb9-02c7-4f3b-ea4d-210269cffb00" + }, + "outputs": [], + "source": [ + "np.abs(25) + np.abs(2) + np.abs(5)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WTPz0EBSAVee" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lQP73B916SLP" + }, + "source": [ + "### Squared $L^2$ Norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Qv1ouJ8r6SLP", + "outputId": "6cb6eebf-e88a-4b41-aa9f-ac82a78f99e6" + }, + "outputs": [], + "source": [ + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "eG3WSB5R6SLT", + "outputId": "b1971235-6180-4f41-f4fb-d539583f5748" + }, + "outputs": [], + "source": [ + "(25**2 + 2**2 + 5**2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "bXwzSudS6SLV", + "outputId": "264e1226-1066-4b64-db16-0070ba65b245" + }, + "outputs": [], + "source": [ + "# we'll cover tensor multiplication more soon but to prove point quickly:\n", + "np.dot(x, x)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "q3CIH9ba6SLX" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BHWxVPFC6SLX" + }, + "source": [ + "### Max Norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vO-zfvDG6SLX", + "outputId": "e3c17461-b374-47ed-e95b-f7ef61f1bfca" + }, + "outputs": [], + "source": [ + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vXXLgbyW6SLZ", + "outputId": "eae3e34d-ae6f-46f0-bb0b-a518792c2cb7" + }, + "outputs": [], + "source": [ + "np.max([np.abs(25), np.abs(2), np.abs(5)])" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3MVTsXA8nNR0" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "JzKlIpYZcgt9" + }, + "source": [ + "### Orthogonal Vectors" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4jHg9La-cgt9", + "outputId": "ece7c34c-c185-44bf-c5cd-218ab3e80c87" + }, + "outputs": [], + "source": [ + "i = np.array([1, 0])\n", + "i" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "3FyLhPK3cguA", + "outputId": "580fea73-ce7a-406e-c9b5-70ebe1f26a79" + }, + "outputs": [], + "source": [ + "j = np.array([0, 1])\n", + "j" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7eQtKhaDcguC", + "outputId": "4422976d-b8de-4526-88ba-be0e7aa43754" + }, + "outputs": [], + "source": [ + "np.dot(i, j) # detail on the dot operation coming up..." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "C6eMVPu4nNR7" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mK3AZH53o8Br" + }, + "source": [ + "### Matrices (Rank 2 Tensors) in NumPy" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "stk57cmaESW1", + "outputId": "3c147c9a-7d8a-4b1b-aebd-9656a7c1f368" + }, + "outputs": [], + "source": [ + "# Use array() with nested brackets:\n", + "X = np.array([[25, 2], [5, 26], [3, 7]])\n", + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IhDL4L8S6SLc", + "outputId": "599b1ac9-bae5-4bd0-ca42-2e437b22dcbf" + }, + "outputs": [], + "source": [ + "X.shape" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "q3oyaAK36SLe", + "outputId": "b12026d0-5614-4261-cdaa-15e06d33289f" + }, + "outputs": [], + "source": [ + "X.size" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "YN9CHzja6SLg", + "outputId": "fd31c5a0-75ea-4274-9d5e-43e07464edbe" + }, + "outputs": [], + "source": [ + "# Select left column of matrix X (zero-indexed)\n", + "X[:,0]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ih7nh4qC6SLi", + "outputId": "30cd3a8b-0daa-4137-e6d6-0464eb263353" + }, + "outputs": [], + "source": [ + "# Select middle row of matrix X:\n", + "X[1,:]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pg7numxP6SLl", + "outputId": "83d7da30-ec2a-448a-f3d0-c6a87736b5df" + }, + "outputs": [], + "source": [ + "# Another slicing-by-index example:\n", + "X[0:2, 0:2]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "HGEfZiBb6SLt" + }, + "source": [ + "### Matrices in PyTorch" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-bibT9ye6SLt", + "outputId": "c0b079b1-6fa6-48c5-85cd-2713dc27d8d6" + }, + "outputs": [], + "source": [ + "X_pt = torch.tensor([[25, 2], [5, 26], [3, 7]])\n", + "X_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "TBPu1L7P6SLv", + "outputId": "7f0dcc6a-237e-409e-f6c2-e2bb267208be" + }, + "outputs": [], + "source": [ + "X_pt.shape # pythonic relative to TensorFlow" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4mTj56M16SLw", + "outputId": "1b03c6e8-10ce-40d5-97e0-357936b1c510" + }, + "outputs": [], + "source": [ + "X_pt[1,:] # N.B.: Python is zero-indexed; written algebra is one-indexed" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "E026fQlD6SLn" + }, + "source": [ + "### Matrices in TensorFlow" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1gtGH6oA6SLn", + "outputId": "c0ce4bd3-5e6d-48f4-d2f6-6a89fb2be5fa" + }, + "outputs": [], + "source": [ + "X_tf = tf.Variable([[25, 2], [5, 26], [3, 7]])\n", + "X_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "4CV_KiTP6SLp", + "outputId": "aa1c1260-ad22-47df-ade8-acb92b37a406" + }, + "outputs": [], + "source": [ + "tf.rank(X_tf)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vUsce8tC6SLq", + "outputId": "6a67de12-22c8-45be-f757-2abc7a07e56e" + }, + "outputs": [], + "source": [ + "tf.shape(X_tf)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "QNpfvNPj6SLr", + "outputId": "af5fc194-5874-4894-dfc9-0d8a11087bcf" + }, + "outputs": [], + "source": [ + "X_tf[1,:]" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CodS4evY6SLy" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cMpfujF_6SLy" + }, + "source": [ + "### Higher-Rank Tensors\n", + "\n", + "As an example, rank 4 tensors are common for images, where each dimension corresponds to:\n", + "\n", + "1. Number of images in training batch, e.g., 32\n", + "2. Image height in pixels, e.g., 28 for [MNIST digits](http://yann.lecun.com/exdb/mnist/)\n", + "3. Image width in pixels, e.g., 28\n", + "4. Number of color channels, e.g., 3 for full-color images (RGB)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "KSZlICRR6SL1" + }, + "outputs": [], + "source": [ + "images_pt = torch.zeros([32, 28, 28, 3])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "6Dqj0vmh6SL2" + }, + "outputs": [], + "source": [ + "# images_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "7TASTVD96SLy" + }, + "outputs": [], + "source": [ + "images_tf = tf.zeros([32, 28, 28, 3])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ftOliyru6SL0" + }, + "outputs": [], + "source": [ + "# images_tf" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "O3sgkdXZ6SL3" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lmG3LEZK6SL4" + }, + "source": [ + "## Segment 2: Common Tensor Operations" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iSHGMCxd6SL4" + }, + "source": [ + "### Tensor Transposition" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "1YN1narR6SL4", + "outputId": "9d2f2223-2e97-4066-e774-b7b2990a4387" + }, + "outputs": [], + "source": [ + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5hf3M_NL6SL5", + "outputId": "a1e37e11-2dd8-47aa-e6fc-d8f41fc9c94f" + }, + "outputs": [], + "source": [ + "X.T" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vyBFN_4g6SL9", + "outputId": "18d89c80-72b3-4d8f-a343-5319478b103b" + }, + "outputs": [], + "source": [ + "X_pt.T" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "K2DuDJc_6SL6", + "outputId": "fe0b51cb-89c0-47fd-eeef-c99ef54e8b72" + }, + "outputs": [], + "source": [ + "tf.transpose(X_tf) # less Pythonic" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Hp9P1jx76SL_" + }, + "source": [ + "### Basic Arithmetical Properties" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WxaImEUc6SMA" + }, + "source": [ + "Adding or multiplying with scalar applies operation to all elements and tensor shape is retained:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "yhXGETii6SMA", + "outputId": "3b84b86c-40e4-4a9d-ba98-29400df1e3b4" + }, + "outputs": [], + "source": [ + "X*2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KnPULtDO6SMC", + "outputId": "7799494b-fdd7-40a9-a25d-6fae2c3dafba" + }, + "outputs": [], + "source": [ + "X+2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "MkfC0Gsb6SMD", + "outputId": "729eef68-a48e-45d9-e91f-e07e976c7dc2" + }, + "outputs": [], + "source": [ + "X*2+2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "04bIDpGj6SMH", + "outputId": "4a021c90-7e73-4939-eac7-e22f7a52135e" + }, + "outputs": [], + "source": [ + "X_pt*2+2 # Python operators are overloaded; could alternatively use torch.mul() or torch.add()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "2oRBSmRL6SMI", + "outputId": "5626b6c5-f89b-400f-fb19-2b550056065f" + }, + "outputs": [], + "source": [ + "torch.add(torch.mul(X_pt, 2), 2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OMSb9Otd6SMF", + "outputId": "ffac929e-7e5c-44a6-f792-9add7acc3ddc" + }, + "outputs": [], + "source": [ + "X_tf*2+2 # Operators likewise overloaded; could equally use tf.multiply() tf.add()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5ya2xZ4u6SMG", + "outputId": "2b473b86-0a90-414d-9e5b-d19752d64cc9" + }, + "outputs": [], + "source": [ + "tf.add(tf.multiply(X_tf, 2), 2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wt8Ls4076SMK" + }, + "source": [ + "If two tensors have the same size, operations are often by default applied element-wise. This is **not matrix multiplication**, which we'll cover later, but is rather called the **Hadamard product** or simply the **element-wise product**.\n", + "\n", + "The mathematical notation is $A \\odot X$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KUMyU1t46SMK", + "outputId": "1ccfbb86-871e-4a8a-cc4c-d5f332418d94" + }, + "outputs": [], + "source": [ + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "UNIbp0P36SML", + "outputId": "ac9a00b6-fc5b-46ed-b58f-a194efe0a9e2" + }, + "outputs": [], + "source": [ + "A = X+2\n", + "A" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "HE9xPWPdcgu4", + "outputId": "0e7fb59b-eabb-4099-9997-6a0884aa1fde" + }, + "outputs": [], + "source": [ + "A + X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "xKyCwGia6SMP", + "outputId": "b26a192f-6edf-4c4e-d7e3-1010ab17a565" + }, + "outputs": [], + "source": [ + "A * X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "B5jXGIBp6SMT" + }, + "outputs": [], + "source": [ + "A_pt = X_pt + 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "A7k6yxu36SMU", + "outputId": "a5057174-9c44-4234-8db6-0cc296cc67dd" + }, + "outputs": [], + "source": [ + "A_pt + X_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "r8vOul0m6SMW", + "outputId": "828619be-624e-4924-aa95-c9e314f48fe8" + }, + "outputs": [], + "source": [ + "A_pt * X_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "rQcBMSb76SMQ" + }, + "outputs": [], + "source": [ + "A_tf = X_tf + 2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "x6s1wtNj6SMR", + "outputId": "8904d6bb-ba2f-4a8d-eed3-fa97ab5f3b48" + }, + "outputs": [], + "source": [ + "A_tf + X_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "J1D7--296SMS", + "outputId": "da919529-d302-4f0f-c4df-3379722d20c2" + }, + "outputs": [], + "source": [ + "A_tf * X_tf" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FE5f-FEq6SMY" + }, + "source": [ + "### Reduction" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WPJ9FVQF6SMY" + }, + "source": [ + "Calculating the sum across all elements of a tensor is a common operation. For example:\n", + "\n", + "* For vector ***x*** of length *n*, we calculate $\\sum_{i=1}^{n} x_i$\n", + "* For matrix ***X*** with *m* by *n* dimensions, we calculate $\\sum_{i=1}^{m} \\sum_{j=1}^{n} X_{i,j}$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rXi2stvz6SMZ", + "outputId": "7ea95647-da4e-4293-c668-13d3bad8e1d0" + }, + "outputs": [], + "source": [ + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "W9FKaJbf6SMZ", + "outputId": "01308bf7-b99e-44cd-cc93-76d504affaf7" + }, + "outputs": [], + "source": [ + "X.sum()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "3y9aw7t66SMc", + "outputId": "234c1149-b78d-4d2d-9724-b25434e20760" + }, + "outputs": [], + "source": [ + "torch.sum(X_pt)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wcjRtFml6SMb", + "outputId": "c1b1f84d-83b0-4c45-ccd2-b312f6851305" + }, + "outputs": [], + "source": [ + "tf.reduce_sum(X_tf)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "awjH9bOz6SMc", + "outputId": "12d3fb04-b017-413c-cb8a-72c9795b9fa4" + }, + "outputs": [], + "source": [ + "# Can also be done along one specific axis alone, e.g.:\n", + "X.sum(axis=0) # summing over all rows (i.e., along columns)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "n2SASjsn6SMd", + "outputId": "ae990ba3-eca0-4ed6-dcbf-bc929e7f0c20" + }, + "outputs": [], + "source": [ + "X.sum(axis=1) # summing over all columns (i.e., along rows)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "uVnSxvSJ6SMh", + "outputId": "396683c4-e825-4287-b5d2-e97a8cef5a9f" + }, + "outputs": [], + "source": [ + "torch.sum(X_pt, 0)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "IO8drxz36SMe", + "outputId": "0a0f18a6-d220-4ecb-94a1-68f1978920f7" + }, + "outputs": [], + "source": [ + "tf.reduce_sum(X_tf, 1)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "gdAe8S4A6SMj" + }, + "source": [ + "Many other operations can be applied with reduction along all or a selection of axes, e.g.:\n", + "\n", + "* maximum\n", + "* minimum\n", + "* mean\n", + "* product\n", + "\n", + "They're fairly straightforward and used less often than summation, so you're welcome to look them up in library docs if you ever need them." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "r2eW8S_46SMj" + }, + "source": [ + "### The Dot Product" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "LImETgD76SMj" + }, + "source": [ + "If we have two vectors (say, ***x*** and ***y***) with the same length *n*, we can calculate the dot product between them. This is annotated several different ways, including the following:\n", + "\n", + "* $x \\cdot y$\n", + "* $x^Ty$\n", + "* $\\langle x,y \\rangle$\n", + "\n", + "Regardless which notation you use (I prefer the first), the calculation is the same; we calculate products in an element-wise fashion and then sum reductively across the products to a scalar value. That is, $x \\cdot y = \\sum_{i=1}^{n} x_i y_i$\n", + "\n", + "The dot product is ubiquitous in deep learning: It is performed at every artificial neuron in a deep neural network, which may be made up of millions (or orders of magnitude more) of these neurons." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "HveIE3IDcgvP", + "outputId": "7e47ff8a-3d70-4480-b585-b0cac5bea84e" + }, + "outputs": [], + "source": [ + "x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "3ZjkZcvVcgvQ", + "outputId": "3d4f0992-086a-4789-912d-6899b31dab37" + }, + "outputs": [], + "source": [ + "y = np.array([0, 1, 2])\n", + "y" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Xu8z0QB0cgvR", + "outputId": "684539d6-0ebb-47f3-d2f4-6ec21a14e1f8" + }, + "outputs": [], + "source": [ + "25*0 + 2*1 + 5*2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ThehRrr8cgvS", + "outputId": "23b6fb0b-f5c8-414a-c09d-1552a44a5591" + }, + "outputs": [], + "source": [ + "np.dot(x, y)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "J5Zdua4xcgvT", + "outputId": "d59bc178-eccd-46f2-d889-32b266eee56f" + }, + "outputs": [], + "source": [ + "x_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "b3vEdroXcgvU", + "outputId": "63ef19a6-2bbf-4336-d216-2b941658e9fe" + }, + "outputs": [], + "source": [ + "y_pt = torch.tensor([0, 1, 2])\n", + "y_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "F741E5imcgvV", + "outputId": "754c34c3-548f-4674-8cbd-925cb0d78a6d" + }, + "outputs": [], + "source": [ + "np.dot(x_pt, y_pt)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-W5loHc8cgvX", + "outputId": "18e71dbb-163f-4291-8090-366a21c35642" + }, + "outputs": [], + "source": [ + "torch.dot(torch.tensor([25, 2, 5.]), torch.tensor([0, 1, 2.]))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "jUwKBiqzcgvY", + "outputId": "710897b4-cdc5-48f8-ba45-a536a3d1c686" + }, + "outputs": [], + "source": [ + "x_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Xqt3Rac7cgvZ", + "outputId": "50214346-8c20-42aa-d25d-1eb711f35330" + }, + "outputs": [], + "source": [ + "y_tf = tf.Variable([0, 1, 2])\n", + "y_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "x4pgc5JEcgvc", + "outputId": "a3e872f5-c429-4e0f-ea1a-7751d9e289d5" + }, + "outputs": [], + "source": [ + "tf.reduce_sum(tf.multiply(x_tf, y_tf))" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mSmvC1cc6SMj" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9dfTV7hQR6zn" + }, + "source": [ + "### Solving Linear Systems" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "pCoSpqtWSyFd" + }, + "source": [ + "In the **Substitution** example, the two equations in the system are:\n", + "$$ y = 3x $$\n", + "$$ -5x + 2y = 2 $$\n", + "\n", + "The second equation can be rearranged to isolate $y$:\n", + "$$ 2y = 2 + 5x $$\n", + "$$ y = \\frac{2 + 5x}{2} = 1 + \\frac{5x}{2} $$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "1T0_m11kTh82" + }, + "outputs": [], + "source": [ + "x = np.linspace(-10, 10, 1000) # start, finish, n points" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "_K6IyUWcTh85" + }, + "outputs": [], + "source": [ + "y1 = 3 * x" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "DYkQdUJ5Th86" + }, + "outputs": [], + "source": [ + "y2 = 1 + (5*x)/2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 455 + }, + "id": "aBL9NDoHTh86", + "outputId": "dada3255-d947-4809-95e9-dc2485cb7a34" + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "plt.xlabel('x')\n", + "plt.ylabel('y')\n", + "ax.set_xlim([0, 3])\n", + "ax.set_ylim([0, 8])\n", + "ax.plot(x, y1, c='green')\n", + "ax.plot(x, y2, c='brown')\n", + "plt.axvline(x=2, color='purple', linestyle='--')\n", + "_ = plt.axhline(y=6, color='purple', linestyle='--')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vACSnJgP87xA" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "F3X4p66XVFXH" + }, + "source": [ + "In the **Elimination** example, the two equations in the system are:\n", + "$$ 2x - 3y = 15 $$\n", + "$$ 4x + 10y = 14 $$\n", + "\n", + "Both equations can be rearranged to isolate $y$. Starting with the first equation:\n", + "$$ -3y = 15 - 2x $$\n", + "$$ y = \\frac{15 - 2x}{-3} = -5 + \\frac{2x}{3} $$\n", + "\n", + "Then for the second equation:\n", + "$$ 4x + 10y = 14 $$\n", + "$$ 2x + 5y = 7 $$\n", + "$$ 5y = 7 - 2x $$\n", + "$$ y = \\frac{7 - 2x}{5} $$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "5x3kfV_WWhlR" + }, + "outputs": [], + "source": [ + "y1 = -5 + (2*x)/3" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "dqA1lS0CWu5z" + }, + "outputs": [], + "source": [ + "y2 = (7-2*x)/5" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 455 + }, + "id": "6CfRNs1DWzx5", + "outputId": "c32b733e-c5b9-4654-d2f9-4d4f6cdfa93e" + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "plt.xlabel('x')\n", + "plt.ylabel('y')\n", + "\n", + "# Add x and y axes:\n", + "plt.axvline(x=0, color='lightgray')\n", + "plt.axhline(y=0, color='lightgray')\n", + "\n", + "ax.set_xlim([-2, 10])\n", + "ax.set_ylim([-6, 4])\n", + "ax.plot(x, y1, c='green')\n", + "ax.plot(x, y2, c='brown')\n", + "plt.axvline(x=6, color='purple', linestyle='--')\n", + "_ = plt.axhline(y=-1, color='purple', linestyle='--')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "CezB87Tb-QGh" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bYDhomCP6SMj" + }, + "source": [ + "## Segment 3: Matrix Properties" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-HGU_an66SMk" + }, + "source": [ + "### Frobenius Norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pNQHvAqN6SMk", + "outputId": "916cf013-483f-48a2-e66a-dcb8be2be58a" + }, + "outputs": [], + "source": [ + "X = np.array([[1, 2], [3, 4]])\n", + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "T-q-Tzn26SMm", + "outputId": "f13d60e4-6d8c-4628-acda-01a5bf85876f" + }, + "outputs": [], + "source": [ + "(1**2 + 2**2 + 3**2 + 4**2)**(1/2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "YVG8qiFw6SMn", + "outputId": "c0b5b228-8724-4bb2-b1ce-531b359e36c8" + }, + "outputs": [], + "source": [ + "np.linalg.norm(X) # same function as for vector L2 norm" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "FPnBflKVxyik" + }, + "outputs": [], + "source": [ + "X_pt = torch.tensor([[1, 2], [3, 4.]]) # torch.norm() supports floats only" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "NCdTShVyx8z0", + "outputId": "b574d8aa-4700-4f75-959d-2b0501f44bc8" + }, + "outputs": [], + "source": [ + "torch.norm(X_pt)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "blezf9fLx_nD" + }, + "outputs": [], + "source": [ + "X_tf = tf.Variable([[1, 2], [3, 4.]]) # tf.norm() also supports floats only" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LiCQzyf6ySCZ", + "outputId": "560c2751-1cde-4622-d8f6-4f6661710b89" + }, + "outputs": [], + "source": [ + "tf.norm(X_tf)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4c6rjVAf6SMo" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "OLN-MMIe6SMo" + }, + "source": [ + "### Matrix Multiplication (with a Vector)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "XJw0j8cr6SMo", + "outputId": "287c7d56-a31a-4990-e7ef-069d8ae3da65" + }, + "outputs": [], + "source": [ + "A = np.array([[3, 4], [5, 6], [7, 8]])\n", + "A" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "zZQ1Aupc6SMq", + "outputId": "ef242b77-62eb-4ce2-f468-5c9bb93b20cf" + }, + "outputs": [], + "source": [ + "b = np.array([1, 2])\n", + "b" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ZbeVtNyW6SMq", + "outputId": "d0e9832a-e2a8-4819-c852-5ff9a427a6a6" + }, + "outputs": [], + "source": [ + "np.dot(A, b) # even though technically dot products are between vectors only" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "srVI55X96SMu", + "outputId": "881f9985-25e3-4589-942c-42afb9ecf7cb" + }, + "outputs": [], + "source": [ + "A_pt = torch.tensor([[3, 4], [5, 6], [7, 8]])\n", + "A_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5SDn71Xc6SMv", + "outputId": "2af7f92d-7799-446c-bad1-e6368df39940" + }, + "outputs": [], + "source": [ + "b_pt = torch.tensor([1, 2])\n", + "b_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "OIeoJlsh6SMx", + "outputId": "0b1008a9-b5e7-4d40-b88e-2bb0c0319f91" + }, + "outputs": [], + "source": [ + "torch.matmul(A_pt, b_pt) # like np.dot(), automatically infers dims in order to perform dot product, matvec, or matrix multiplication" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pnob9GkB6SMs", + "outputId": "07b3a129-35f1-41b8-b1f9-7c208a4b9277" + }, + "outputs": [], + "source": [ + "A_tf = tf.Variable([[3, 4], [5, 6], [7, 8]])\n", + "A_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "vYtWxf8K6SMt", + "outputId": "a83bf927-bb8e-4c60-b6da-3cb6cb5edd57" + }, + "outputs": [], + "source": [ + "b_tf = tf.Variable([1, 2])\n", + "b_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "NGBImWRH6SMt", + "outputId": "3d2f4fcd-04d8-4dbb-bee5-f9c2151fdea3" + }, + "outputs": [], + "source": [ + "tf.linalg.matvec(A_tf, b_tf)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kzjZmdRR6SMy" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "21ySqay36SM5" + }, + "source": [ + "### Matrix Multiplication (with Two Matrices)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "0YRG1Ig2cgvo", + "outputId": "02ab25c2-ca74-4d3a-d845-6716906305a3" + }, + "outputs": [], + "source": [ + "A" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "DyOEZk_c6SM5", + "outputId": "bb6d846b-4a97-4b90-a6ca-d4010269a6c9" + }, + "outputs": [], + "source": [ + "B = np.array([[1, 9], [2, 0]])\n", + "B" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SfKuNxH-6SM6", + "outputId": "3c6271ee-c89e-4707-ba0d-96e8a585ebf7" + }, + "outputs": [], + "source": [ + "np.dot(A, B)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "WcnQMF0s6SNB" + }, + "source": [ + "Note that matrix multiplication is not \"commutative\" (i.e., $AB \\neq BA$) so uncommenting the following line will throw a size mismatch error:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "_mwBGOXO6SNB" + }, + "outputs": [], + "source": [ + "# np.dot(B, A)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "JrrvPoNE6SM9", + "outputId": "8e9b7b0e-c56e-4a28-f858-93886de0968d" + }, + "outputs": [], + "source": [ + "B_pt = torch.from_numpy(B) # much cleaner than TF conversion\n", + "B_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Z6PfwCvX6SM-", + "outputId": "37454147-f8c3-41f4-f8ce-6fb3ef8f4524" + }, + "outputs": [], + "source": [ + "# another neat way to create the same tensor with transposition:\n", + "B_pt = torch.tensor([[1, 2], [9, 0]]).T\n", + "B_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "16ZNRaVe6SM_", + "outputId": "c32eb80d-f84c-4670-dead-0f34b0f9498f" + }, + "outputs": [], + "source": [ + "torch.matmul(A_pt, B_pt) # no need to change functions, unlike in TF" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rkymNjE46SM8", + "outputId": "712a076c-160d-4e9e-d57f-29ef716eff08" + }, + "outputs": [], + "source": [ + "B_tf = tf.convert_to_tensor(B, dtype=tf.int32)\n", + "B_tf" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "rslTzFRk6SM8", + "outputId": "d424a4ee-e161-4ddf-e825-e533507d66f5" + }, + "outputs": [], + "source": [ + "tf.matmul(A_tf, B_tf)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0eBiTmPp6SNC" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "L2H9F-DQ6SMz" + }, + "source": [ + "### Symmetric Matrices" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "5YsPoWo76SMz", + "outputId": "d41b1efa-ec4e-4fde-82fb-c3a98ddc3d49" + }, + "outputs": [], + "source": [ + "X_sym = np.array([[0, 1, 2], [1, 7, 8], [2, 8, 9]])\n", + "X_sym" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Skg1wSQVcgv2", + "outputId": "dde3fe41-cc5b-4647-ffaf-5a207a82b644" + }, + "outputs": [], + "source": [ + "X_sym.T" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Jv40-i9H6SM1", + "outputId": "b23c92fa-c452-4a36-ca27-92fd1dcd64bc" + }, + "outputs": [], + "source": [ + "X_sym.T == X_sym" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QZFoUFkq6SM2" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Mq_c3ftZ6SM2" + }, + "source": [ + "### Identity Matrices" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KVSNbH-Z6SM2", + "outputId": "46ce4da4-9435-4744-e097-d7129142bf0b" + }, + "outputs": [], + "source": [ + "I = torch.tensor([[1, 0, 0], [0, 1, 0], [0, 0, 1]])\n", + "I" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "wcoPDhvR6SM3", + "outputId": "2fb868dc-a57d-4c88-9fa9-d6009ec28a76" + }, + "outputs": [], + "source": [ + "x_pt = torch.tensor([25, 2, 5])\n", + "x_pt" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "tuA4RsMv6SM4", + "outputId": "d8983050-aafa-4d61-9fa2-8814a245bc5a" + }, + "outputs": [], + "source": [ + "torch.matmul(I, x_pt)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "bgDiOYLk6SM5" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3S_6Yfdkcgv7" + }, + "source": [ + "### Answers to Matrix Multiplication Qs" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "pINsKNxH6SNC", + "outputId": "749e8b80-4ece-4e71-d253-071ee8231e73" + }, + "outputs": [], + "source": [ + "M_q = torch.tensor([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", + "M_q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "gfjWd8OO6SNE", + "outputId": "e05d8691-6939-4356-ef6e-d4cb66311d79" + }, + "outputs": [], + "source": [ + "V_q = torch.tensor([[-1, 1, -2], [0, 1, 2]]).T\n", + "V_q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "boSkaV2M6SNF", + "outputId": "412a3565-f9fd-4e6f-f3b0-c85ec84b1a9f" + }, + "outputs": [], + "source": [ + "torch.matmul(M_q, V_q)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "slSNKUcN6SNG" + }, + "source": [ + "### Matrix Inversion" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "EW0i5ZRk6SNG", + "outputId": "157f1989-5346-4066-98ed-f1c807610754" + }, + "outputs": [], + "source": [ + "X = np.array([[4, 2], [-5, -3]])\n", + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "hTYpxaWR6SNI", + "outputId": "dee8a0e1-801f-48e7-ea6a-64d6822a6ba9" + }, + "outputs": [], + "source": [ + "Xinv = np.linalg.inv(X)\n", + "Xinv" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XFDBBdYOc-7E" + }, + "source": [ + "As a quick aside, let's prove that $X^{-1}X = I_n$ as per the slides:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "tyPhf-uZcvdB", + "outputId": "1b0f5f09-b0da-4f36-d248-f68056b0f4e1" + }, + "outputs": [], + "source": [ + "np.dot(Xinv, X)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SVwvbBvclul1" + }, + "source": [ + "...and now back to solving for the unknowns in $w$:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Q5sQqFaz6SNK", + "outputId": "92c47ee8-9375-4c79-c8ea-63c56ce5e396" + }, + "outputs": [], + "source": [ + "y = np.array([4, -7])\n", + "y" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "PK7m6F1I6SNL", + "outputId": "ac9bda7d-a937-4c90-e8d5-ed177a1f8dc0" + }, + "outputs": [], + "source": [ + "w = np.dot(Xinv, y)\n", + "w" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "fyBOHgdccgwD" + }, + "source": [ + "Show that $y = Xw$:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "SVBojjwacgwD", + "outputId": "805a0f2c-8ae0-41d8-c121-c270bd67c479" + }, + "outputs": [], + "source": [ + "np.dot(X, w)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "rKELsi0PZCr0" + }, + "source": [ + "**Geometric Visualization**\n", + "\n", + "Recalling from the slides that the two equations in the system are:\n", + "$$ 4b + 2c = 4 $$\n", + "$$ -5b - 3c = -7 $$\n", + "\n", + "Both equations can be rearranged to isolate a variable, say $c$. Starting with the first equation:\n", + "$$ 4b + 2c = 4 $$\n", + "$$ 2b + c = 2 $$\n", + "$$ c = 2 - 2b $$\n", + "\n", + "Then for the second equation:\n", + "$$ -5b - 3c = -7 $$\n", + "$$ -3c = -7 + 5b $$\n", + "$$ c = \\frac{-7 + 5b}{-3} = \\frac{7 - 5b}{3} $$" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "ZASIBqroap7k" + }, + "outputs": [], + "source": [ + "b = np.linspace(-10, 10, 1000) # start, finish, n points" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "9XsUdVsmaqTr" + }, + "outputs": [], + "source": [ + "c1 = 2 - 2*b" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "8Mwn1tAca1Cl" + }, + "outputs": [], + "source": [ + "c2 = (7-5*b)/3" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 455 + }, + "id": "X5ozP9jZauQa", + "outputId": "d6d69af4-657c-4a70-9020-b028c43c20fa" + }, + "outputs": [], + "source": [ + "fig, ax = plt.subplots()\n", + "plt.xlabel('b', c='darkorange')\n", + "plt.ylabel('c', c='brown')\n", + "\n", + "plt.axvline(x=0, color='lightgray')\n", + "plt.axhline(y=0, color='lightgray')\n", + "\n", + "ax.set_xlim([-2, 3])\n", + "ax.set_ylim([-1, 5])\n", + "ax.plot(b, c1, c='purple')\n", + "ax.plot(b, c2, c='purple')\n", + "plt.axvline(x=-1, color='green', linestyle='--')\n", + "_ = plt.axhline(y=4, color='green', linestyle='--')" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "14uhePTna7ZV" + }, + "source": [ + "In PyTorch and TensorFlow:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7qo-SAvaansp", + "outputId": "3bca970e-71dc-4072-dff7-3c59a0fb9b37" + }, + "outputs": [], + "source": [ + "torch.inverse(torch.tensor([[4, 2], [-5, -3.]])) # float type" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "uqtyF3Jqaz4l", + "outputId": "2960b1e8-66a2-40be-eb09-a86754ab596c" + }, + "outputs": [], + "source": [ + "tf.linalg.inv(tf.Variable([[4, 2], [-5, -3.]])) # also float" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "hcP2S4AMAHT_" + }, + "source": [ + "**Exercises**:\n", + "\n", + "1. As done with NumPy above, use PyTorch to calculate $w$ from $X$ and $y$. Subsequently, confirm that $y = Xw$.\n", + "2. Repeat again, now using TensorFlow." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "AMxROg326SNN" + }, + "source": [ + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "N8ZxpgcN6SNO" + }, + "source": [ + "### Matrix Inversion Where No Solution" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "RYORHY4E6SNO", + "outputId": "ac15d9a5-90e8-4003-cd2e-1b2bccc6381c" + }, + "outputs": [], + "source": [ + "X = np.array([[-4, 1], [-8, 2]])\n", + "X" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "o7GcISdI6SNP" + }, + "outputs": [], + "source": [ + "# Uncommenting the following line results in a \"singular matrix\" error\n", + "# Xinv = np.linalg.inv(X)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "uk485dwoI021" + }, + "source": [ + "Feel free to try inverting a non-square matrix; this will throw an error too.\n", + "\n", + "**Return to slides here.**" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "BXomEBoyDQyO" + }, + "source": [ + "### Orthogonal Matrices\n", + "\n", + "These are the solutions to Exercises 3 and 4 on **orthogonal matrices** from the slides.\n", + "\n", + "For Exercise 3, to demonstrate the matrix $I_3$ has mutually orthogonal columns, we show that the dot product of any pair of columns is zero:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "-Un4Aq805HM1", + "outputId": "b3370a08-816a-4d36-a123-149db7fb661e" + }, + "outputs": [], + "source": [ + "I = np.array([[1, 0, 0], [0, 1, 0], [0, 0, 1]])\n", + "I" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CUhMi2_IEyaI", + "outputId": "0b7c751a-0cd2-49bb-dde5-8e3895d7a12d" + }, + "outputs": [], + "source": [ + "column_1 = I[:,0]\n", + "column_1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "xsOtn-siE63K", + "outputId": "a334201a-5e98-405b-d430-24e74307ac6d" + }, + "outputs": [], + "source": [ + "column_2 = I[:,1]\n", + "column_2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Pv3b51ExE9ZX", + "outputId": "2cd0a54e-2440-4fef-ff71-c0c0401cc94f" + }, + "outputs": [], + "source": [ + "column_3 = I[:,2]\n", + "column_3" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "VhnRlc8eElYc", + "outputId": "2473373b-936c-45e8-8ef2-a305351a9390" + }, + "outputs": [], + "source": [ + "np.dot(column_1, column_2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "mVq4m-LfFGjQ", + "outputId": "d1b8a9b5-8b20-44d0-ce59-1a6b7ba05c2c" + }, + "outputs": [], + "source": [ + "np.dot(column_1, column_3)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "nJj-aihzFILx", + "outputId": "e11e422d-0268-4830-d840-22ebd21db873" + }, + "outputs": [], + "source": [ + "np.dot(column_2, column_3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "FIzpkdJmFYdY" + }, + "source": [ + "We can use the `np.linalg.norm()` method from earlier in the notebook to demonstrate that each column of $I_3$ has unit norm:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "xKbbeqegFJSc", + "outputId": "b496a162-a447-44b8-cad8-ce768afda24b" + }, + "outputs": [], + "source": [ + "np.linalg.norm(column_1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Ycv0L8mpGKRC", + "outputId": "1dad31ec-d76d-450c-ae53-4388241db990" + }, + "outputs": [], + "source": [ + "np.linalg.norm(column_2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LsEuRezYGLgY", + "outputId": "44d2899e-fef2-499f-9051-002e0b9deb33" + }, + "outputs": [], + "source": [ + "np.linalg.norm(column_3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "NMrHdyrsGQ1X" + }, + "source": [ + "Since the matrix $I_3$ has mutually orthogonal columns and each column has unit norm, the column vectors of $I_3$ are *orthonormal*. Since $I_3^T = I_3$, this means that the *rows* of $I_3$ must also be orthonormal.\n", + "\n", + "Since the columns and rows of $I_3$ are orthonormal, $I_3$ is an *orthogonal matrix*." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "K7EykcPdIZhE" + }, + "source": [ + "For Exercise 4, let's repeat the steps of Exercise 3 with matrix *K* instead of $I_3$. We could use NumPy again, but for fun I'll use PyTorch instead. (You're welcome to try it with TensorFlow if you feel so inclined.)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "MXVyu8tSGMlZ", + "outputId": "454c991b-1357-4a7e-f642-de51ad634095" + }, + "outputs": [], + "source": [ + "K = torch.tensor([[2/3, 1/3, 2/3], [-2/3, 2/3, 1/3], [1/3, 2/3, -2/3]])\n", + "K" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "xqkmSYNfJmTs", + "outputId": "5b314b21-1200-45f0-b9ee-02ff3a76319f" + }, + "outputs": [], + "source": [ + "Kcol_1 = K[:,0]\n", + "Kcol_1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ncoc3IxNJ0v2", + "outputId": "9c0de5c0-1311-4d2a-f2a8-c8b035799afe" + }, + "outputs": [], + "source": [ + "Kcol_2 = K[:,1]\n", + "Kcol_2" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "ISEKQ0AqJ2wf", + "outputId": "beaa5849-4d3c-4af7-cead-e4cd90faee99" + }, + "outputs": [], + "source": [ + "Kcol_3 = K[:,2]\n", + "Kcol_3" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "jvdXuMzbJ44d", + "outputId": "8c35ee3b-9dbd-4a2e-c11a-857fa688d535" + }, + "outputs": [], + "source": [ + "torch.dot(Kcol_1, Kcol_2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "BUR60eBuKEbU", + "outputId": "abd95aa8-5557-4b29-871a-e09409095f62" + }, + "outputs": [], + "source": [ + "torch.dot(Kcol_1, Kcol_3)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Rg7QvQwuKGJZ", + "outputId": "bf55069c-186d-4a53-9b68-84b42e5e4454" + }, + "outputs": [], + "source": [ + "torch.dot(Kcol_2, Kcol_3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "wNdV36-QKNK-" + }, + "source": [ + "We've now determined that the columns of $K$ are orthogonal." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "DDgyhj1AKHdS", + "outputId": "ae72ed8b-2617-4cd4-91ec-7c94a1ab94d7" + }, + "outputs": [], + "source": [ + "torch.norm(Kcol_1)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "KvQZy_E2KorW", + "outputId": "10824173-467d-45c4-e931-4e65bec68229" + }, + "outputs": [], + "source": [ + "torch.norm(Kcol_2)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "7H7dx8a7Kp9f", + "outputId": "7de33d74-e385-4542-9dd7-375ae744a50a" + }, + "outputs": [], + "source": [ + "torch.norm(Kcol_3)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "VPPtOk5OKtJ_" + }, + "source": [ + "We've now determined that, in addition to being orthogonal, the columns of $K$ have unit norm, therefore they are orthonormal.\n", + "\n", + "To ensure that $K$ is an orthogonal matrix, we would need to show that not only does it have orthonormal columns but it has orthonormal rows are as well. Since $K^T \\neq K$, we can't prove this quite as straightforwardly as we did with $I_3$.\n", + "\n", + "One approach would be to repeat the steps we used to determine that $K$ has orthogonal columns with all of the matrix's rows (please feel free to do so). Alternatively, we can use an orthogonal matrix-specific equation from the slides, $A^TA = I$, to demonstrate that $K$ is orthogonal in a single line of code:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "W8Ao0KhVNHqF", + "outputId": "3a3ae6c8-91c3-456a-9060-5b6e0524cde6" + }, + "outputs": [], + "source": [ + "torch.matmul(K.T, K)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "scM6FCpjNgYA" + }, + "source": [ + "Notwithstanding rounding errors that we can safely ignore, this confirms that $K^TK = I$ and therefore $K$ is an orthogonal matrix." + ] + } + ], + "metadata": { + "colab": { + "include_colab_link": true, + "name": "1-intro-to-linear-algebra.ipynb", + "provenance": [] + }, + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.7.6" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} From c4c4d3973de065c77878f8635589e2dac1923c35 Mon Sep 17 00:00:00 2001 From: Vasily Griaznov <32549267+vasilygrz@users.noreply.github.com> Date: Tue, 18 Feb 2025 18:32:55 +0100 Subject: [PATCH 2/2] Added comments in code cells providing additional explanations as well as alternative functions/solutions --- notebooks/1-intro-to-linear-algebra.ipynb | 34 +++++++++++++++++------ 1 file changed, 26 insertions(+), 8 deletions(-) diff --git a/notebooks/1-intro-to-linear-algebra.ipynb b/notebooks/1-intro-to-linear-algebra.ipynb index 826a278..541f4ed 100644 --- a/notebooks/1-intro-to-linear-algebra.ipynb +++ b/notebooks/1-intro-to-linear-algebra.ipynb @@ -720,7 +720,10 @@ "source": [ "# ...but can transpose a matrix with a dimension of length 1, which is mathematically equivalent:\n", "y_t = y.T\n", - "y_t" + "y_t\n", + "\n", + "# Alternatively, the 1D array can be reshaped into a 2D array with a single column\n", + "# x_t.reshape(-1, 1)" ] }, { @@ -1339,7 +1342,7 @@ }, "outputs": [], "source": [ - "tf.rank(X_tf)" + "tf.rank(X_tf) # equivalent to np.ndim" ] }, { @@ -1853,7 +1856,7 @@ }, "outputs": [], "source": [ - "X.sum()" + "X.sum() # equivalent to np.sum(X)" ] }, { @@ -2108,7 +2111,11 @@ }, "outputs": [], "source": [ - "torch.dot(torch.tensor([25, 2, 5.]), torch.tensor([0, 1, 2.]))" + "# Integer tensors can be created but they are CPU-only; float types are needed for GPU processing and gradient calculation\n", + "torch.dot(torch.tensor([25, 2, 5.]), torch.tensor([0, 1, 2.]))\n", + "\n", + "# Alternatively, existing tensors can be converted into a float type\n", + "# torch.dot(x_pt.to(torch.float), y_pt.to(torch.float))" ] }, { @@ -2154,7 +2161,11 @@ }, "outputs": [], "source": [ - "tf.reduce_sum(tf.multiply(x_tf, y_tf))" + "# Manual dot product calculation\n", + "tf.reduce_sum(tf.multiply(x_tf, y_tf))\n", + "\n", + "# TensorFlow has a more efficient built-in function\n", + "# tf.tensordot(x_tf, y_tf, axes=1) # similar but not equivalent to np.tensordot" ] }, { @@ -2515,7 +2526,9 @@ }, "outputs": [], "source": [ - "np.dot(A, b) # even though technically dot products are between vectors only" + "np.dot(A, b) # even though technically dot products are between vectors only\n", + "\n", + "# Note that np.dot and np.matmul produce the same results for 1D and 2D arrays, but they behave differently in higher dimensions" ] }, { @@ -2673,7 +2686,7 @@ }, "outputs": [], "source": [ - "np.dot(A, B)" + "np.dot(A, B) # equivalent to np.matmul(A, B) as we have 2D arrays" ] }, { @@ -2966,7 +2979,12 @@ }, "outputs": [], "source": [ - "torch.matmul(M_q, V_q)" + "torch.matmul(M_q, V_q)\n", + "\n", + "# Alternatively, the Einstein summation can be used as follows:\n", + "# M_q = torch.tensor([[0, 1, 2], [3, 4, 5], [6, 7, 8]])\n", + "# V_q = torch.tensor([[-1, 1, -2], [0, 1, 2]])\n", + "# torch.einsum('ij,kj->ik', M_q, V_q)" ] }, {