nlp_funcamental_in_tensor_flow.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "authorship_tag": "ABX9TyN9FhwDmChZffczodHAscC+",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/miller00315/ia_studies/blob/main/nlp_funcamental_in_tensor_flow.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction to NLP Funamentals in TensorFlow\n",
        "\n",
        "NLP has the goal of derivin information out of natural language (could be text or speech).\n",
        "\n",
        "Another common term for NLP problems is sequence to sequence problem (seq2seq)"
      ],
      "metadata": {
        "id": "XmmpZXhcW35H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!nvidia-smi -L"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7zStyPPVXixV",
        "outputId": "cc0ab1a0-6d4a-4e5c-e851-38704a235eff"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "GPU 0: Tesla T4 (UUID: GPU-784f4b93-126f-b9c0-5c89-f1f6683e910d)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Get helper function"
      ],
      "metadata": {
        "id": "1RsCRpNRXiuJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/extras/helper_functions.py"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G2ohxMh0XiqP",
        "outputId": "86f4f924-c80f-45ab-e234-59872a2e80c6"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-05-28 22:47:03--  https://raw.githubusercontent.com/mrdbourke/tensorflow-deep-learning/main/extras/helper_functions.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.111.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 10246 (10K) [text/plain]\n",
            "Saving to: ‘helper_functions.py’\n",
            "\n",
            "helper_functions.py 100%[===================>]  10.01K  --.-KB/s    in 0s      \n",
            "\n",
            "2024-05-28 22:47:03 (71.0 MB/s) - ‘helper_functions.py’ saved [10246/10246]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Import a series of helper functions for teh notebook\n",
        "from helper_functions import unzip_data, create_tensorboard_callback, plot_loss_curves, compare_historys, walk_through_dir, calculate_results"
      ],
      "metadata": {
        "id": "Xua5hLh4XinQ"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Get a text dataset\n",
        "\n",
        "The dataset we're going to be using is Kaggle's introduction to NLP dataset (text samples of Tweets labelled as disaster or not disaster).\n",
        "\n",
        "Se the orgin source here."
      ],
      "metadata": {
        "id": "z7yfdMbyXiio"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget https://storage.googleapis.com/ztm_tf_course/nlp_getting_started.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AON1LV4RXiga",
        "outputId": "d9459ef2-afa3-4f1c-cc6b-97f4f5da2dbe"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-05-28 22:47:09--  https://storage.googleapis.com/ztm_tf_course/nlp_getting_started.zip\n",
            "Resolving storage.googleapis.com (storage.googleapis.com)... 108.177.120.207, 142.251.183.207, 142.251.180.207, ...\n",
            "Connecting to storage.googleapis.com (storage.googleapis.com)|108.177.120.207|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 607343 (593K) [application/zip]\n",
            "Saving to: ‘nlp_getting_started.zip’\n",
            "\n",
            "\rnlp_getting_started   0%[                    ]       0  --.-KB/s               \rnlp_getting_started 100%[===================>] 593.11K  --.-KB/s    in 0.005s  \n",
            "\n",
            "2024-05-28 22:47:09 (109 MB/s) - ‘nlp_getting_started.zip’ saved [607343/607343]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "unzip_data(\"nlp_getting_started.zip\")"
      ],
      "metadata": {
        "id": "OrtRijIGXid_"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Visualizing a text dataset\n",
        "\n",
        "To visualize our text samples, we first have to read them in, one way to do so would be to use Python Pandas."
      ],
      "metadata": {
        "id": "6yPNBqXJXib2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "train_df = pd.read_csv(\"train.csv\")\n",
        "test_df = pd.read_csv(\"test.csv\")\n",
        "train_df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "CaXrbcXYYIUW",
        "outputId": "7f376052-9dc6-4c8a-c3f0-6ddeba7c3d25"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id keyword location                                               text  \\\n",
              "0   1     NaN      NaN  Our Deeds are the Reason of this #earthquake M...   \n",
              "1   4     NaN      NaN             Forest fire near La Ronge Sask. Canada   \n",
              "2   5     NaN      NaN  All residents asked to 'shelter in place' are ...   \n",
              "3   6     NaN      NaN  13,000 people receive #wildfires evacuation or...   \n",
              "4   7     NaN      NaN  Just got sent this photo from Ruby #Alaska as ...   \n",
              "\n",
              "   target  \n",
              "0       1  \n",
              "1       1  \n",
              "2       1  \n",
              "3       1  \n",
              "4       1  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8b432912-dbe5-4a47-9289-f2ecf36aad72\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>keyword</th>\n",
              "      <th>location</th>\n",
              "      <th>text</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Our Deeds are the Reason of this #earthquake M...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>4</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Forest fire near La Ronge Sask. Canada</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>5</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>All residents asked to 'shelter in place' are ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>6</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>13,000 people receive #wildfires evacuation or...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>7</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Just got sent this photo from Ruby #Alaska as ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8b432912-dbe5-4a47-9289-f2ecf36aad72')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8b432912-dbe5-4a47-9289-f2ecf36aad72 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8b432912-dbe5-4a47-9289-f2ecf36aad72');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-4453c52f-7c62-4644-a7c7-3de1ec20fe1f\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4453c52f-7c62-4644-a7c7-3de1ec20fe1f')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-4453c52f-7c62-4644-a7c7-3de1ec20fe1f button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "train_df",
              "summary": "{\n  \"name\": \"train_df\",\n  \"rows\": 7613,\n  \"fields\": [\n    {\n      \"column\": \"id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3137,\n        \"min\": 1,\n        \"max\": 10873,\n        \"num_unique_values\": 7613,\n        \"samples\": [\n          3796,\n          3185,\n          7769\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"keyword\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 221,\n        \"samples\": [\n          \"injury\",\n          \"nuclear%20reactor\",\n          \"engulfed\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"location\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3341,\n        \"samples\": [\n          \"Oklahoma\",\n          \"Starling City\",\n          \"Trinidad and Tobago\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 7503,\n        \"samples\": [\n          \"Three Homes Demolished in Unrecognized Arab Village - International Middle East Media Center http://t.co/ik8m4Yi9T4\",\n          \"Reid Lake fire prompts campground evacuation order http://t.co/jBODKM6rBU\",\n          \"FAAN orders evacuation of abandoned aircraft at MMA http://t.co/dEvYbnVXGQ via @todayng\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"target\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_df['text'][0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 36
        },
        "id": "jRke6OW9YIQo",
        "outputId": "9c4e330d-93c5-429e-e6be-de6eefb68ee4"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Our Deeds are the Reason of this #earthquake May ALLAH Forgive us all'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "test_df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "JRe4ybDLYINj",
        "outputId": "33c20be4-7f3c-4596-a046-c56be2bf48b7"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   id keyword location                                               text\n",
              "0   0     NaN      NaN                 Just happened a terrible car crash\n",
              "1   2     NaN      NaN  Heard about #earthquake is different cities, s...\n",
              "2   3     NaN      NaN  there is a forest fire at spot pond, geese are...\n",
              "3   9     NaN      NaN           Apocalypse lighting. #Spokane #wildfires\n",
              "4  11     NaN      NaN      Typhoon Soudelor kills 28 in China and Taiwan"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0031dea7-b067-4e6d-aa63-1a74486160d5\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>keyword</th>\n",
              "      <th>location</th>\n",
              "      <th>text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Just happened a terrible car crash</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Heard about #earthquake is different cities, s...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>there is a forest fire at spot pond, geese are...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>9</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Apocalypse lighting. #Spokane #wildfires</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>11</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Typhoon Soudelor kills 28 in China and Taiwan</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0031dea7-b067-4e6d-aa63-1a74486160d5')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-0031dea7-b067-4e6d-aa63-1a74486160d5 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-0031dea7-b067-4e6d-aa63-1a74486160d5');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-27ff7304-ba79-4d04-a9b7-cca6ebc3e033\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-27ff7304-ba79-4d04-a9b7-cca6ebc3e033')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-27ff7304-ba79-4d04-a9b7-cca6ebc3e033 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "test_df",
              "summary": "{\n  \"name\": \"test_df\",\n  \"rows\": 3263,\n  \"fields\": [\n    {\n      \"column\": \"id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3146,\n        \"min\": 0,\n        \"max\": 10875,\n        \"num_unique_values\": 3263,\n        \"samples\": [\n          8051,\n          425,\n          1330\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"keyword\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 221,\n        \"samples\": [\n          \"injury\",\n          \"nuclear%20reactor\",\n          \"engulfed\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"location\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 1602,\n        \"samples\": [\n          \"UAE\",\n          \"Tokio / Tokyo\",\n          \"Texas\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 3243,\n        \"samples\": [\n          \"Latest: USA: Huge sinkhole swallows up Brooklyn intersection http://t.co/vspKHg3nZy\",\n          \"I liked a @YouTube video http://t.co/a5YTAw9Vih S.O.S. Rona Guide - The Red Whirlwind\",\n          \"HitchBot travels Europe and greeted with open arms. Gets destroyed after two weeks in america. There's a lesson to be learned here.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# ow many examples of each class?\n",
        "train_df.target.value_counts()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Cicw96ALYIG7",
        "outputId": "6af1bd48-2e4a-4cc1-dd34-def38c17df1a"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "target\n",
              "0    4342\n",
              "1    3271\n",
              "Name: count, dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# How many total samples?\n",
        "\n",
        "len(train_df), len(test_df)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SvnUiG7Ua81L",
        "outputId": "91445a41-cb32-46e0-ff73-4fc13b5c8733"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(7613, 3263)"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle training dataframe\n",
        "train_df_shuffled = train_df.sample(frac=1, random_state=42) # shuffle with random_state=42 for reproducibility\n",
        "train_df_shuffled.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "ShI_Ps7sfkUP",
        "outputId": "3efb3afb-0c2b-4e14-c843-9a8fb4ecbd92"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "        id      keyword               location  \\\n",
              "2644  3796  destruction                    NaN   \n",
              "2227  3185       deluge                    NaN   \n",
              "5448  7769       police                     UK   \n",
              "132    191   aftershock                    NaN   \n",
              "6845  9810       trauma  Montgomery County, MD   \n",
              "\n",
              "                                                   text  target  \n",
              "2644  So you have a new weapon that can cause un-ima...       1  \n",
              "2227  The f$&amp;@ing things I do for #GISHWHES Just...       0  \n",
              "5448  DT @georgegalloway: RT @Galloway4Mayor: ÛÏThe...       1  \n",
              "132   Aftershock back to school kick off was great. ...       0  \n",
              "6845  in response to trauma Children of Addicts deve...       0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a980646f-678a-427d-b9de-f541334a3572\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>keyword</th>\n",
              "      <th>location</th>\n",
              "      <th>text</th>\n",
              "      <th>target</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>2644</th>\n",
              "      <td>3796</td>\n",
              "      <td>destruction</td>\n",
              "      <td>NaN</td>\n",
              "      <td>So you have a new weapon that can cause un-ima...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2227</th>\n",
              "      <td>3185</td>\n",
              "      <td>deluge</td>\n",
              "      <td>NaN</td>\n",
              "      <td>The f$&amp;amp;@ing things I do for #GISHWHES Just...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5448</th>\n",
              "      <td>7769</td>\n",
              "      <td>police</td>\n",
              "      <td>UK</td>\n",
              "      <td>DT @georgegalloway: RT @Galloway4Mayor: ÛÏThe...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>132</th>\n",
              "      <td>191</td>\n",
              "      <td>aftershock</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Aftershock back to school kick off was great. ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6845</th>\n",
              "      <td>9810</td>\n",
              "      <td>trauma</td>\n",
              "      <td>Montgomery County, MD</td>\n",
              "      <td>in response to trauma Children of Addicts deve...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a980646f-678a-427d-b9de-f541334a3572')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-a980646f-678a-427d-b9de-f541334a3572 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-a980646f-678a-427d-b9de-f541334a3572');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-88c33fe0-d240-49fe-90ae-49687a391729\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-88c33fe0-d240-49fe-90ae-49687a391729')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-88c33fe0-d240-49fe-90ae-49687a391729 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "train_df_shuffled",
              "summary": "{\n  \"name\": \"train_df_shuffled\",\n  \"rows\": 7613,\n  \"fields\": [\n    {\n      \"column\": \"id\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3137,\n        \"min\": 1,\n        \"max\": 10873,\n        \"num_unique_values\": 7613,\n        \"samples\": [\n          7061,\n          843,\n          10603\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"keyword\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 221,\n        \"samples\": [\n          \"blazing\",\n          \"emergency\",\n          \"rescue\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"location\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 3341,\n        \"samples\": [\n          \"617-BTOWN-BEATDOWN\",\n          \"Newcastle Upon Tyne, England\",\n          \"Federal Capital Territory\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 7503,\n        \"samples\": [\n          \"This week first responders and DART members are participating in a four day intensive Technical Large Animal... http://t.co/tL93AOd3ER\",\n          \"My mom is watching a show about bridges breaking/falling and the people on them drowning in their cars aka one of my biggest fears ????\",\n          \"@NickLee8  i went to school in a bombed out East End of London3 families to one house no bathroom outside loo &amp; poor so whats yr point\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"target\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Let's viaulaize some random training examples\n",
        "\n",
        "import random\n",
        "random_index = random.randint(0, len(train_df)-5)\n",
        "for row in train_df[[\"text\", \"target\"]][random_index:random_index+5].itertuples():\n",
        "  _, text, target = row\n",
        "  print(f\"Target: {target}\", \"(real disaster)\" if target > 0 else \"(not real disaster)\")\n",
        "  print(f\"Text:\\n{text}\\n\")\n",
        "  print(\"example\\ temp\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BWn-CFMCa8y3",
        "outputId": "9d87fa43-587b-47a2-afd6-5bf19bb530c0"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Target: 0 (not real disaster)\n",
            "Text:\n",
            "@GorpuaZikinak  and tongue out as she collapsed in the cum puddle her whole body covered.\n",
            "\n",
            "example\\ temp\n",
            "Target: 0 (not real disaster)\n",
            "Text:\n",
            "@indiepopmom I CANT BREATHE MY LUNGS COLLAPSED\n",
            "\n",
            "example\\ temp\n",
            "Target: 0 (not real disaster)\n",
            "Text:\n",
            "Zimbabwe is a country with a collapsed government ruled by a dictator while many live below the poverty line.\n",
            "\n",
            "example\\ temp\n",
            "Target: 1 (real disaster)\n",
            "Text:\n",
            "@durrellb Prices here are insane. Our dollar has collapsed against the US and it's punishing us. Thanks for the info.\n",
            "\n",
            "example\\ temp\n",
            "Target: 0 (not real disaster)\n",
            "Text:\n",
            "I just collapsed in my bed ugh I'm exhausted\n",
            "\n",
            "example\\ temp\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Split dat into trainig and validation datasets"
      ],
      "metadata": {
        "id": "vq0qF1Eoa8vU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import train_test_split"
      ],
      "metadata": {
        "id": "fCObA2Y_a8sq"
      },
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Use train_test_split to split training dat into training and validation sets\n",
        "\n",
        "train_sentences, val_sentences, train_labels, val_labels = train_test_split(train_df_shuffled[\"text\"].to_numpy(),\n",
        "                                                                            train_df_shuffled[\"target\"].to_numpy(),\n",
        "                                                                            test_size=0.1,\n",
        "                                                                            random_state=42)"
      ],
      "metadata": {
        "id": "sxe6Z-Vda8qP"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the lengths\n",
        "\n",
        "len(train_sentences), len(val_sentences), len(train_labels), len(val_labels)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tQuFaG2na8m9",
        "outputId": "9a43610e-e459-4d77-dfa8-de1c079119c1"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6851, 762, 6851, 762)"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "len(train_df_shuffled)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nWBK1TtjYIEK",
        "outputId": "ca218536-5aad-4bd0-ea40-0ddb330689f1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "7613"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the firt 10 sentences\n",
        "\n",
        "train_sentences[:10], train_labels[:10]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DVw91EpRfu45",
        "outputId": "f575293a-e53d-428e-e9c3-582caa046abc"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array(['@mogacola @zamtriossu i screamed after hitting tweet',\n",
              "        'Imagine getting flattened by Kurt Zouma',\n",
              "        '@Gurmeetramrahim #MSGDoing111WelfareWorks Green S welfare force ke appx 65000 members har time disaster victim ki help ke liye tyar hai....',\n",
              "        \"@shakjn @C7 @Magnums im shaking in fear he's gonna hack the planet\",\n",
              "        'Somehow find you and I collide http://t.co/Ee8RpOahPk',\n",
              "        '@EvaHanderek @MarleyKnysh great times until the bus driver held us hostage in the mall parking lot lmfao',\n",
              "        'destroy the free fandom honestly',\n",
              "        'Weapons stolen from National Guard Armory in New Albany still missing #Gunsense http://t.co/lKNU8902JE',\n",
              "        '@wfaaweather Pete when will the heat wave pass? Is it really going to be mid month? Frisco Boy Scouts have a canoe trip in Okla.',\n",
              "        'Patient-reported outcomes in long-term survivors of metastatic colorectal cancer - British Journal of Surgery http://t.co/5Yl4DC1Tqt'],\n",
              "       dtype=object),\n",
              " array([0, 0, 1, 0, 0, 1, 1, 0, 1, 1]))"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Converting text into numbers\n",
        "\n",
        "When dealing with a text problem, one of the first things you'll have to do before you can build the model is to convert your txt to numbers.\n",
        "\n",
        "The a few ways to do this, namely:\n",
        "\n",
        "* Tokeninzation - direct mapping of token (a token could be a word or a character) to number\n",
        "\n",
        "* Embeding - create a matrix of feature vector for each token (the size of the feature vector can be defined and this embeding can be learned)\n"
      ],
      "metadata": {
        "id": "EblnNNaSfu0D"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Text vectorization (tokenization)"
      ],
      "metadata": {
        "id": "aaIzqTIOfuxa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "train_sentences[:5]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PsM_hXW6kWSI",
        "outputId": "99b7fb55-d3d3-430c-a150-726a8a17eb96"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['@mogacola @zamtriossu i screamed after hitting tweet',\n",
              "       'Imagine getting flattened by Kurt Zouma',\n",
              "       '@Gurmeetramrahim #MSGDoing111WelfareWorks Green S welfare force ke appx 65000 members har time disaster victim ki help ke liye tyar hai....',\n",
              "       \"@shakjn @C7 @Magnums im shaking in fear he's gonna hack the planet\",\n",
              "       'Somehow find you and I collide http://t.co/Ee8RpOahPk'],\n",
              "      dtype=object)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow.keras.layers.experimental.preprocessing import TextVectorization"
      ],
      "metadata": {
        "id": "UsF8pcF1fuui"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Use the default TextVectorization parameters\n",
        "\n",
        "text_vectorizer = TextVectorization(max_tokens=None, # how many words in the vocabulary (automatically add <oov>)\n",
        "                                    standardize=\"lower_and_strip_punctuation\", # how to handle punctuation\n",
        "                                    split=\"whitespace\", # how to split tokens\n",
        "                                    ngrams=None, # creat a group of n-words?\n",
        "                                    output_mode=\"int\", # how to map tokens to numbers\n",
        "                                    output_sequence_length=None, # how long do you want your sequences to be, if None, it will adjust automatically\n",
        "                                    pad_to_max_tokens=False) # pad sequences to the same length\n"
      ],
      "metadata": {
        "id": "KGgkfyxkfuqC"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the average number of tokens (words) in the trainig tweets\n",
        "\n",
        "round(sum([len(i.split()) for i in train_sentences])/len(train_sentences))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pWQH9eUDfuoX",
        "outputId": "a0bd81ce-2c1f-4f29-d3fa-b2b5ab676100"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "15"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Steup text vectorization variables\n",
        "\n",
        "max_vocab_length = 10000 # max number of words to have in our vocabulary\n",
        "max_length = 15 # max length our sequences will be (e.g. how many words from a tweeot does a model see?)\n",
        "\n",
        "text_vectorizer = TextVectorization(max_tokens=max_vocab_length,\n",
        "                                    output_sequence_length=max_length,\n",
        "                                    output_mode=\"int\"\n",
        "                                    )"
      ],
      "metadata": {
        "id": "LSuUNkMKfulu"
      },
      "execution_count": 21,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "text_vectorizer.adapt(train_sentences)"
      ],
      "metadata": {
        "id": "0vvZA45DfuiO"
      },
      "execution_count": 22,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "sample_sentence = \"There's a flood in my street!\"\n",
        "text_vectorizer([sample_sentence])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mBa40MlSfufv",
        "outputId": "4fb58453-48cf-4f79-b2e4-7dc9b33b9c22"
      },
      "execution_count": 23,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 15), dtype=int64, numpy=\n",
              "array([[264,   3, 232,   4,  13, 698,   0,   0,   0,   0,   0,   0,   0,\n",
              "          0,   0]])>"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "random_sentence = random.choice(train_sentences)\n",
        "print(f\"Original text:\\n{random_sentence} \\n \\nVectorized version:\")\n",
        "\n",
        "text_vectorizer([random_sentence])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NA0drm3Kq_vn",
        "outputId": "8e4c11da-f6b1-4086-b00e-4dc2875e6dc7"
      },
      "execution_count": 24,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original text:\n",
            "Sammy and todd always to the rescue may not be in uniform but still to the rescue lmao. Forever KFC fam. \n",
            " \n",
            "Vectorized version:\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 15), dtype=int64, numpy=\n",
              "array([[8841,    7, 2650,  262,    5,    2,  389,  133,   34,   21,    4,\n",
              "        3292,   30,   80,    5]])>"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the unique words in the vocabulary\n",
        "\n",
        "words_in_vocab = text_vectorizer.get_vocabulary()\n",
        "\n",
        "top_10_words = words_in_vocab[:10]\n",
        "\n",
        "bottom_10_words = words_in_vocab[-10:]\n",
        "\n",
        "print(f\"Number of words in vocab: {len(words_in_vocab)}\")\n",
        "print(top_10_words)\n",
        "print(bottom_10_words)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9f4NnToVq_sc",
        "outputId": "24811201-8eb2-474c-dd1f-0b4247cb99af"
      },
      "execution_count": 25,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of words in vocab: 10000\n",
            "['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is']\n",
            "['painthey', 'painful', 'paine', 'paging', 'pageshi', 'pages', 'paeds', 'pads', 'padres', 'paddytomlinson1']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating a Embedding using an Embedding layer\n",
        "\n",
        "To make our embedding, we're going to use TensorFlow's embeding layer: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding\n",
        "\n",
        "The parameters we care most about for our embedding layer:\n",
        "* `input_dim` = the size of our vocabulary\n",
        "* `output_dim` = the size of the output embedding vector, for exampl, a value of 100 would mean each token gets represented by a vector 100 long.\n",
        "* `input_length` = length of sequnces being passed to the embedding layer."
      ],
      "metadata": {
        "id": "RK0ImRnZq_qE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras.layers import Embedding"
      ],
      "metadata": {
        "id": "s3PYgWWSq_nx"
      },
      "execution_count": 26,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "embedding = Embedding(input_dim=max_vocab_length, # set input shape\n",
        "                      output_dim=128, # output shape\n",
        "                      embeddings_initializer=\"uniform\", # default, intialize randomly\n",
        "                      input_length=max_length # how long is each input\n",
        "                      )"
      ],
      "metadata": {
        "id": "d_EAjN0Rq_lM"
      },
      "execution_count": 27,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "embedding"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rOSnUmxVq_iz",
        "outputId": "bf1b64bb-39f8-43ed-aa70-ec163f188491"
      },
      "execution_count": 28,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<keras.src.layers.core.embedding.Embedding at 0x7d837ef15f30>"
            ]
          },
          "metadata": {},
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get a random sentence from teh tarining set\n",
        "\n",
        "random_sentence = random.choice(train_sentences)\n",
        "print(f\"Original text:\\n{random_sentence} \\\n",
        "    \\n\\nEmbedded version:\")\n",
        "\n",
        "# Embed the random sentence\n",
        "\n",
        "sample_embed = embedding(text_vectorizer([random_sentence]))\n",
        "sample_embed"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jY1eU-kOq_g4",
        "outputId": "cfac54f2-86af-47dd-eb91-f821ee5e5715"
      },
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original text:\n",
            "#USGS M 1.4 - 4km E of Interlaken California: Time2015-08-06 00:52:25 UTC2015-08-05 17:52:25 -07:00 at ep... http://t.co/zqrcptLrUM #SM     \n",
            "\n",
            "Embedded version:\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 15, 128), dtype=float32, numpy=\n",
              "array([[[-0.01636697, -0.01405703, -0.03756199, ...,  0.04349514,\n",
              "         -0.032907  , -0.02320576],\n",
              "        [-0.00693915,  0.03426461, -0.01833714, ...,  0.04437156,\n",
              "         -0.03948905,  0.01359539],\n",
              "        [-0.01537759, -0.01024324,  0.04404697, ...,  0.03492235,\n",
              "          0.04057812, -0.00229375],\n",
              "        ...,\n",
              "        [ 0.0241408 ,  0.03049104,  0.04406604, ..., -0.00717366,\n",
              "          0.02090086, -0.03383999],\n",
              "        [ 0.02437088,  0.02813056, -0.03858282, ..., -0.00280961,\n",
              "          0.04563762, -0.02292827],\n",
              "        [-0.03353995,  0.00135516,  0.01736769, ...,  0.04655555,\n",
              "         -0.01037078, -0.03737321]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check out a single sample token's embedding\n",
        "\n",
        "sample_embed[0][0], sample_embed[0][0].shape, random_sentence"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5Gfpv4NHq_dy",
        "outputId": "6a84ac04-6ea9-4ed8-9c14-0af1a750c1cb"
      },
      "execution_count": 30,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(128,), dtype=float32, numpy=\n",
              " array([-0.01636697, -0.01405703, -0.03756199, -0.00613047,  0.00766439,\n",
              "         0.02890761, -0.02431258, -0.03983684, -0.02251066,  0.01238359,\n",
              "         0.02446589, -0.00876174, -0.03945863,  0.03121896, -0.00320773,\n",
              "         0.04305438,  0.04737249, -0.01296883,  0.03789448,  0.03030237,\n",
              "        -0.01316673,  0.03208568, -0.04419701,  0.03276697,  0.03039583,\n",
              "         0.03631986, -0.01503464, -0.03131682, -0.00653889,  0.03243371,\n",
              "         0.04557706, -0.03877397,  0.03293839,  0.03241087, -0.02752004,\n",
              "         0.00851177, -0.01450994, -0.04094936, -0.04779334,  0.04736649,\n",
              "         0.02180037, -0.01186055, -0.02918702,  0.03868252,  0.04630287,\n",
              "         0.03662261, -0.0149883 , -0.02388766, -0.00785365,  0.03173846,\n",
              "         0.03198148, -0.03221078,  0.00409622, -0.03005198, -0.00104891,\n",
              "        -0.02624422,  0.02173886, -0.00105823,  0.03302529,  0.04465704,\n",
              "        -0.04401683,  0.00698687,  0.01760227,  0.02041339,  0.042399  ,\n",
              "        -0.01619388,  0.03322487,  0.04341209, -0.04064216,  0.04309018,\n",
              "         0.01979201,  0.02124978,  0.02397964,  0.00913155, -0.04307352,\n",
              "         0.02106774, -0.03429465,  0.03309781, -0.01893555, -0.01826344,\n",
              "        -0.03701447, -0.01926025,  0.03313604, -0.02780157, -0.0013765 ,\n",
              "        -0.00180827, -0.00293106,  0.02546537, -0.03982763,  0.00269519,\n",
              "         0.00732106, -0.04424591,  0.02658984, -0.04781187, -0.03863739,\n",
              "         0.01439637,  0.02213767, -0.01760121, -0.01603753,  0.01252537,\n",
              "         0.00014647,  0.01800941, -0.00271609,  0.04927796,  0.00973666,\n",
              "         0.00990646,  0.02423216, -0.01708288, -0.00750332,  0.04036521,\n",
              "        -0.0354414 ,  0.03407282, -0.03793013, -0.01448382, -0.04357746,\n",
              "        -0.02395087,  0.02173713, -0.02533127, -0.03853495, -0.03461175,\n",
              "        -0.04696868,  0.00671645, -0.04607486, -0.0182166 , -0.03638401,\n",
              "         0.04349514, -0.032907  , -0.02320576], dtype=float32)>,\n",
              " TensorShape([128]),\n",
              " '#USGS M 1.4 - 4km E of Interlaken California: Time2015-08-06 00:52:25 UTC2015-08-05 17:52:25 -07:00 at ep... http://t.co/zqrcptLrUM #SM')"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Modelling a text datset (running a series of experiments)\n",
        "\n",
        "Now we've got a way to turn our text sequnces into numbers it's time to strat building series of modelling experitments.\n",
        "\n",
        "[Machine learnign map](https://scikit-learn.org/0.18/tutorial/machine_learning_map/index.html#)"
      ],
      "metadata": {
        "id": "ftBNAonMq_a6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Model 0: Geting a baseline\n",
        "\n",
        "As with all machine learning models, it's important to create a baseline model so you've got a benchmark for future experiments to build upon.\n",
        "\n",
        "To create our baseline, we'll use Sklearn's Multinomial Naive Bayes using the TF-IDF formula to convert our words to numbers.\n",
        "\n",
        "> 🔑 **Note:** It's common practice to use non-DL algorithms as a baseline because of their speed and then later using DL to see if you can improve upon them.'"
      ],
      "metadata": {
        "id": "dTcIuExbumcW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "from sklearn.pipeline import Pipeline"
      ],
      "metadata": {
        "id": "8BvdrisaumYJ"
      },
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create tokenization and modeling ppeline"
      ],
      "metadata": {
        "id": "yKTuRhdrumVr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_0 = Pipeline([\n",
        "                    (\"tfidf\", TfidfVectorizer()), # Convert words to numbers using tfidf\n",
        "                    (\"clf\", MultinomialNB()) # mdoel the text\n",
        "                  ])"
      ],
      "metadata": {
        "id": "SVh0RglIumTB"
      },
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Fite the pipeline to the training data\n",
        "model_0.fit(train_sentences, train_labels)"
      ],
      "metadata": {
        "id": "tPvCKzIyumPu",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 198
        },
        "outputId": "933e0988-5aab-45a5-95a4-1390c0ab9e42"
      },
      "execution_count": 33,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(steps=[('tfidf', TfidfVectorizer()), ('clf', MultinomialNB())])"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;tfidf&#x27;, TfidfVectorizer()), (&#x27;clf&#x27;, MultinomialNB())])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" ><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;tfidf&#x27;, TfidfVectorizer()), (&#x27;clf&#x27;, MultinomialNB())])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">TfidfVectorizer</label><div class=\"sk-toggleable__content\"><pre>TfidfVectorizer()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate our baseline model\n",
        "\n",
        "baseline_score =  model_0.score(val_sentences, val_labels)"
      ],
      "metadata": {
        "id": "5dvPsD9rZC-K"
      },
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(f\"Our baseline model achieves an accuracy of: {baseline_score*100:.2f}%\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DI49LCHWZC7J",
        "outputId": "c65a1ccc-8107-436e-fb47-b09eac161316"
      },
      "execution_count": 36,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Our baseline model achieves an accuracy of: 79.27%\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Make predictions\n",
        "\n",
        "baseline_preds = model_0.predict(val_sentences)\n",
        "\n",
        "baseline_preds[:20]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-fpF4VWSZC4d",
        "outputId": "4c9b0f46-9822-423e-ef2e-1421b642e18e"
      },
      "execution_count": 37,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([1, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1])"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "train_labels[:20]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_yERxDUkZC2Z",
        "outputId": "8abaeddc-b7b3-40b6-fabe-29840bfb4949"
      },
      "execution_count": 38,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0])"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Cerating a evaluation funtcion for our model experiments\n",
        "\n",
        "We could evaluate all of our model's predictions with different metrics every time, however, this will be cumbersome and could easily be fixed with a function.\n",
        "\n",
        "Let's create a function to compare our model's predictions with the truth labels using the following metrics:\n",
        "* Accuracy\n",
        "* Precision\n",
        "* Recall\n",
        "* F1-score"
      ],
      "metadata": {
        "id": "i3k0HOVFZCz7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to evaluate accuracy, precision, recall, f1-score\n",
        "\n",
        "from sklearn.metrics import accuracy_score, precision_recall_fscore_support\n",
        "\n",
        "def calculate_results(y_true, y_pred):\n",
        "  \"\"\"\n",
        "  Calculates model accuracy, precision, recall and f1 score of a binary classification model.\n",
        "  \"\"\"\n",
        "\n",
        "  # Calculate model accuracy\n",
        "  model_accuracy = accuracy_score(y_true, y_pred) * 100\n",
        "  # Calculate model precision, recall and f1 score using \"weighted\" average\n",
        "  model_precision, model_recall, model_f1, _ = precision_recall_fscore_support(y_true, y_pred, average=\"weighted\")\n",
        "  model_results = {\"accuracy\": model_accuracy,\n",
        "                  \"precision\": model_precision,\n",
        "                  \"recall\": model_recall,\n",
        "                  \"f1\": model_f1}\n",
        "  return model_results"
      ],
      "metadata": {
        "id": "b-JcrmpDZCxc"
      },
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Get baseline results\n",
        "\n",
        "baseline_results = calculate_results(y_true=val_labels,\n",
        "                                     y_pred=baseline_preds)\n",
        "baseline_results"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YontWaKLfEdj",
        "outputId": "7a9684ed-4329-4423-d6b8-92eae3c8f3de"
      },
      "execution_count": 40,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'accuracy': 79.26509186351706,\n",
              " 'precision': 0.8111390004213173,\n",
              " 'recall': 0.7926509186351706,\n",
              " 'f1': 0.7862189758049549}"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from helper_functions import create_tensorboard_callback, plot_loss_curves, compare_historys"
      ],
      "metadata": {
        "id": "87YX34nhipY7"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Model 1: A simple Feed-forward neural network"
      ],
      "metadata": {
        "id": "y9_5RhMxipWP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensorboard callback\n",
        "\n",
        "SAVE_DIR = 'model_logs'"
      ],
      "metadata": {
        "id": "8vhpoV4sipTl"
      },
      "execution_count": 42,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from tensorflow.keras import layers\n",
        "inputs = layers.Input(shape=(1,), dtype=\"string\") # inputs are 1-dimensional strings\n",
        "x = text_vectorizer(inputs) # turn the input text into numbers\n",
        "x = embedding(x) # create an embedding of the numerized numbers\n",
        "x = layers.GlobalAveragePooling1D()(x) # condense the feature vector for each token to one vector\n",
        "outputs = layers.Dense(1, activation=\"sigmoid\")(x) # create the output layer, want binary outputs so use sigmoid activation\n",
        "model_1 = tf.keras.Model(inputs, outputs, name=\"model_1_dense\") # construct the model"
      ],
      "metadata": {
        "id": "gj-fXeH0ipP3"
      },
      "execution_count": 58,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_1.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VDIUIbaalzFn",
        "outputId": "9334748d-f69f-43da-dde1-ec47c8290125"
      },
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_1_dense\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_4 (InputLayer)        [(None, 1)]               0         \n",
            "                                                                 \n",
            " text_vectorization_1 (Text  (None, 15)                0         \n",
            " Vectorization)                                                  \n",
            "                                                                 \n",
            " embedding (Embedding)       (None, 15, 128)           1280000   \n",
            "                                                                 \n",
            " global_average_pooling1d (  (None, 128)               0         \n",
            " GlobalAveragePooling1D)                                         \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 1)                 129       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 1280129 (4.88 MB)\n",
            "Trainable params: 1280129 (4.88 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_1.compile(loss=\"binary_crossentropy\",\n",
        "                optimizer=tf.keras.optimizers.Adam(),\n",
        "                metrics=[\"accuracy\"])"
      ],
      "metadata": {
        "id": "ZTr3nZRripLL"
      },
      "execution_count": 60,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model_1_history = model_1.fit(\n",
        "    x=train_sentences,\n",
        "    y=train_labels,\n",
        "    epochs=5,\n",
        "    validation_data=(val_sentences, val_labels),\n",
        "    callbacks=[create_tensorboard_callback(SAVE_DIR, \"model_1_dense\")]\n",
        "    )"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cTdZOO-3fEam",
        "outputId": "d491989d-d06a-48c5-b393-96beb71d6c46"
      },
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving TensorBoard log files to: model_logs/model_1_dense/20240528-235227\n",
            "Epoch 1/5\n",
            "215/215 [==============================] - 4s 19ms/step - loss: 0.2007 - accuracy: 0.9294 - val_loss: 0.5003 - val_accuracy: 0.7835\n",
            "Epoch 2/5\n",
            "215/215 [==============================] - 4s 20ms/step - loss: 0.1721 - accuracy: 0.9415 - val_loss: 0.5306 - val_accuracy: 0.7808\n",
            "Epoch 3/5\n",
            "215/215 [==============================] - 2s 10ms/step - loss: 0.1497 - accuracy: 0.9476 - val_loss: 0.5474 - val_accuracy: 0.7769\n",
            "Epoch 4/5\n",
            "215/215 [==============================] - 2s 9ms/step - loss: 0.1316 - accuracy: 0.9568 - val_loss: 0.5790 - val_accuracy: 0.7822\n",
            "Epoch 5/5\n",
            "215/215 [==============================] - 2s 8ms/step - loss: 0.1163 - accuracy: 0.9615 - val_loss: 0.6110 - val_accuracy: 0.7782\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plot_loss_curves(model_1_history)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 927
        },
        "id": "lTmHGMUHnkPC",
        "outputId": "d7378ff4-7268-4307-f791-56361e60b828"
      },
      "execution_count": 63,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAHHCAYAAABtF1i4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABHDklEQVR4nO3deXxU9aH///dMkpnsISErEEH2TRbZCtatoIiWisWqlUrEqlUB5eZqhV+vLPqw0cpFeoULLkVurRa/9Yr1URTEtOgVsSKIImBAKnsmG5A9M0nm/P6YZMgkE8hAkpMJr+fjkUcyZz5n5vPhEObN53wWi2EYhgAAAExiNbsCAADg4kYYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAuCBr166VxWLRF198YXZVAAQpwggAADAVYQQAAJiKMAKgzX355ZeaMmWKYmNjFR0drYkTJ+qzzz7zKVNdXa0lS5aoX79+Cg8PV9euXfXDH/5Qmzdv9pZxOByaNWuWevToIbvdrrS0NN188806dOhQO7cIQGsKNbsCADq3PXv26Morr1RsbKx+/etfKywsTC+++KKuueYaffTRRxo3bpwkafHixcrKytK9996rsWPHqqSkRF988YV27typ6667TpI0ffp07dmzR3PnzlWvXr2Un5+vzZs368iRI+rVq5eJrQRwISyGYRhmVwJA8Fq7dq1mzZql7du3a/To0U2ev+WWW/Tee+9p37596t27tyQpNzdXAwYM0MiRI/XRRx9JkkaMGKEePXrob3/7m9/3OX36tOLj4/Xcc8/p0UcfbbsGAWh33KYB0GZqa2v1wQcfaNq0ad4gIklpaWm688479cknn6ikpESS1KVLF+3Zs0cHDhzw+1oRERGy2WzasmWLTp061S71B9A+CCMA2kxBQYEqKio0YMCAJs8NGjRIbrdbR48elSQ9+eSTOn36tPr376/LLrtMjz32mL7++mtvebvdrmeffVbvv/++UlJSdNVVV+l3v/udHA5Hu7UHQNsgjADoEK666iodPHhQa9as0dChQ/XKK6/o8ssv1yuvvOItM2/ePO3fv19ZWVkKDw/XE088oUGDBunLL780seYALhRhBECbSUpKUmRkpHJycpo89+2338pqtSo9Pd17LCEhQbNmzdKf//xnHT16VMOGDdPixYt9zuvTp4/+/d//XR988IG++eYbuVwu/ed//mdbNwVAGyKMAGgzISEhuv766/XXv/7VZ/ptXl6e3njjDf3whz9UbGysJKmoqMjn3OjoaPXt21dOp1OSVFFRoaqqKp8yffr0UUxMjLcMgODE1F4ArWLNmjXauHFjk+OLFy/W5s2b9cMf/lAPPfSQQkND9eKLL8rpdOp3v/udt9zgwYN1zTXXaNSoUUpISNAXX3yht956S3PmzJEk7d+/XxMnTtRtt92mwYMHKzQ0VOvXr1deXp7uuOOOdmsngNbH1F4AF6R+am9zjh49qoKCAi1YsEBbt26V2+3WuHHj9PTTT2v8+PHeck8//bTeffdd7d+/X06nUz179tRdd92lxx57TGFhYSoqKtKiRYuUnZ2to0ePKjQ0VAMHDtS///u/62c/+1l7NBVAGyGMAAAAUzFmBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVEGx6Jnb7daJEycUExMji8VidnUAAEALGIah0tJSdevWTVZr8/0fQRFGTpw44bN/BQAACB5Hjx5Vjx49mn0+KMJITEyMJE9j6vexAAAAHVtJSYnS09O9n+PNCYowUn9rJjY2ljACAECQOdcQCwawAgAAUxFGAACAqQgjAADAVEExZqQl3G63XC6X2dVAB2Sz2c46pQwAYK5OEUZcLpe+//57ud1us6uCDshqterSSy+VzWYzuyoAAD+CPowYhqHc3FyFhIQoPT2d/wHDR/2Cebm5ubrkkktYNA8AOqCgDyM1NTWqqKhQt27dFBkZaXZ10AElJSXpxIkTqqmpUVhYmNnVAQA0EvTdCLW1tZJEFzyaVf93o/7vCgCgYwn6MFKP7nc0h78bANCxdZowAgAAghNhpBPo1auXli9f3uLyW7ZskcVi0enTp9usTo1dc801mjdvXru9HwAgeAT9ANZgdc0112jEiBEBhYjmbN++XVFRUS0uP2HCBOXm5iouLu6C3xsAgAtFz0gHZRiGampqWlQ2KSkpoJlENptNqampjKUAAEjVVdLxnZLbvEH+hBET3H333froo4/0+9//XhaLRRaLRWvXrpXFYtH777+vUaNGyW6365NPPtHBgwd18803KyUlRdHR0RozZow+/PBDn9drfJvGYrHolVde0S233KLIyEj169dP7777rvf5xrdp1q5dqy5dumjTpk0aNGiQoqOjdcMNNyg3N9d7Tk1NjR5++GF16dJFXbt21eOPP66MjAxNmzbtvP4MTp06pZkzZyo+Pl6RkZGaMmWKDhw44H3+8OHDmjp1quLj4xUVFaUhQ4bovffe8547Y8YMJSUlKSIiQv369dOrr756XvUAgIuKs1Q6/Kn02Wpp/YPSqiukrO7Sy9dKRQdNq1anu01jGIYqq81JdxFhIS3qbfj973+v/fv3a+jQoXryySclSXv27JEkzZ8/X0uXLlXv3r0VHx+vo0eP6sYbb9TTTz8tu92uP/7xj5o6dapycnJ0ySWXNPseS5Ys0e9+9zs999xzeuGFFzRjxgwdPnxYCQkJfstXVFRo6dKleu2112S1WvWLX/xCjz76qF5//XVJ0rPPPqvXX39dr776qgYNGqTf//73euedd3TttdcG+sckyRPIDhw4oHfffVexsbF6/PHHdeONN2rv3r0KCwvT7Nmz5XK59PHHHysqKkp79+5VdHS0JOmJJ57Q3r179f777ysxMVHfffedKisrz6seANBplRdKuV9Jjq8933O/lk42Ezgiu0qlJ6Sk/u1bxzqdLoxUVtdq8MJNprz33icnK9J27j/SuLg42Ww2RUZGKjU1VZL07bffSpKefPJJXXfddd6yCQkJGj58uPfxU089pfXr1+vdd9/VnDlzmn2Pu+++Wz//+c8lSb/97W/1X//1X/r88891ww03+C1fXV2t1atXq0+fPpKkOXPmeIOSJL3wwgtasGCBbrnlFknSihUrvD0VgaoPIVu3btWECRMkSa+//rrS09P1zjvv6Gc/+5mOHDmi6dOn67LLLpMk9e7d23v+kSNHNHLkSI0ePVqSp2cIAC5ahiGVHD8TOOrDR8lx/+Vje0hpw6S04VLqMM/Psd0lE2/dd7owEuzqP2DrlZWVafHixdqwYYNyc3NVU1OjyspKHTly5KyvM2zYMO/PUVFRio2NVX5+frPlIyMjvUFEktLS0rzli4uLlZeXp7Fjx3qfDwkJ0ahRo85rP6B9+/YpNDRU48aN8x7r2rWrBgwYoH379kmSHn74YT344IP64IMPNGnSJE2fPt3bpgcffFDTp0/Xzp07df3112vatGneUAMAnZrbLZ38l+T46kz4yP1Kqjzpv3zXvmcCR9pwKXW4FNW1fevcAucVRlauXKnnnntODodDw4cP1wsvvODzQdXY6dOn9Zvf/EZvv/22Tp48qZ49e2r58uW68cYbz7vizYkIC9HeJye3+uu29L0vVONZMY8++qg2b96spUuXqm/fvoqIiNCtt956zh2KGy97brFYzhoc/JU3DCPA2reee++9V5MnT9aGDRv0wQcfKCsrS//5n/+puXPnasqUKTp8+LDee+89bd68WRMnTtTs2bO1dOlS0+oLAK2utloq+PZM4HB8LTl2S66ypmWtoVLSIE/oSK0PHkMle0z71/s8BBxG3nzzTWVmZmr16tUaN26cli9frsmTJysnJ0fJyclNyrtcLl133XVKTk7WW2+9pe7du+vw4cPq0qVLa9S/CYvF0qJbJWaz2WwtWp5869atuvvuu723R8rKynTo0KE2rp2vuLg4paSkaPv27brqqqskeZZW37lzp0aMGBHw6w0aNEg1NTX65z//6e3RKCoqUk5OjgYPHuwtl56ergceeEAPPPCAFixYoJdffllz586V5JlBlJGRoYyMDF155ZV67LHHCCMAgperQsrb49vjkb9XqvXzH8/QCClliCdw1Pd4JA2SwsLbv96tJOBP7WXLlum+++7TrFmzJEmrV6/Whg0btGbNGs2fP79J+TVr1ujkyZP69NNPvf/75h6/58/gn//8pw4dOqTo6Ohmey369eunt99+W1OnTpXFYtETTzxxXrdGLtTcuXOVlZWlvn37auDAgXrhhRd06tSp85oe3K9fP918882677779OKLLyomJkbz589X9+7ddfPNN0uS5s2bpylTpqh///46deqU/vGPf2jQoEGSpIULF2rUqFEaMmSInE6n/va3v3mfA4AOr/J03biOBj0ehfslw8+/7fa4puM7uvaTQjr+f7oDEVBrXC6XduzYoQULFniPWa1WTZo0Sdu2bfN7zrvvvqvx48dr9uzZ+utf/6qkpCTdeeedevzxxxUS4v+2htPplNPp9D4uKSkJpJpB4dFHH1VGRoYGDx6sysrKZqemLlu2TPfcc48mTJigxMREPf7446b8eTz++ONyOByaOXOmQkJCdP/992vy5MnNXsNzefXVV/XII4/oxz/+sVwul6666iq999573sBaW1ur2bNn69ixY4qNjdUNN9yg559/XpKnV2nBggU6dOiQIiIidOWVV2rdunWt1lYAaDWleXWBo8H4jtOH/ZeNSvbt7UgdJsX3MnVgaXuxGAEMDDhx4oS6d++uTz/9VOPHj/ce//Wvf62PPvpI//znP5ucM3DgQB06dEgzZszQQw89pO+++04PPfSQHn74YS1atMjv+yxevFhLlixpcry4uFixsbE+x6qqqvT999/r0ksvVXh48HZRBRu3261Bgwbptttu01NPPWV2dc6KvyMA2pxheEJGw96O3K+ksjz/5btccmZAaX0AiUlt3zq3g5KSEsXFxfn9/G6ozft53G63kpOT9dJLL3lnYBw/flzPPfdcs2FkwYIFyszM9D4uKSlRenp6W1cVZ3H48GF98MEHuvrqq+V0OrVixQp9//33uvPOO82uGgC0L3etVHigwfoddeGjqrhpWYvVc1vFp8fjMikivv3r3YEFFEYSExMVEhKivDzfpJeXl+ddL6OxtLQ0hYWF+XTnDxo0SA6HQy6XSzabrck5drtddrs9kKqhjVmtVq1du1aPPvqoDMPQ0KFD9eGHH2rQoEE6cuSIz8DTxvbu3XvWBdoAoMOqcXoGkjbs8cjbI1VXNC0bYpOSBzUY3zHcM9DU1vK9wy5WAYURm82mUaNGKTs727sMuNvtVnZ2drMLcF1xxRV644035Ha7ZbV6Vp/fv3+/0tLS/AYRdEzp6enaunWr3+e6deumXbt2NXtut27d2qhWANCKnKWS4xvfFUsL9kluP/uEhUV5ejgaju9IGiiF8rl2PgK+TZOZmamMjAyNHj1aY8eO1fLly1VeXu6dXTNz5kx1795dWVlZkjwLVK1YsUKPPPKI5s6dqwMHDui3v/2tHn744dZtCUwTGhqqvn37ml0NAGi58qIzg0rrw0fRQUl+hlFGxPv2dqQNlxJ6S9YLX1sKHgGHkdtvv10FBQVauHChHA6HRowYoY0bNyolJUWSZ6nu+h4QyfM/6k2bNunf/u3fNGzYMHXv3l2PPPKIHn/88dZrBQAA/hiGVHLCt7cj9yup5Jj/8jHdzozvqA8fcT0uihktZgpoNo1ZzjYal5kSOBf+jgAXCbdbOvW976DS3K+kiiL/5RN6+67fkTpcik5q3zp3ch1mNg0AAK2utloqyPHt8XDsllylTctaQjzjORqO70i9TApv/sMR7YswAgDo2KorPTNYGvZ25O2Vap1Ny4aGe2aweMd3DJOSB0thEe1fb7QYYQQA0HFUnvb0cDRcLr1wv2T42cvLHtvgFktd+Ejs3+mWSr8YcMWCWK9evTRv3jzNmzfvnGUtFovWr1/vnZINAKYry68LHLvO9HicOuS/bFSSb29H2nCpSy+pwYQJBC/CCACgbRmGdPpI0xktZQ7/5eMuabQ53HDPUunMaOm0CCMAgNbjrpWKvmvU4/G1VHXaT2GLlNjPt8cjdZgUmdDOlYbZ6N8yyUsvvaRu3brJ7fbdMvrmm2/WPffco4MHD+rmm29WSkqKoqOjNWbMGH344Yet9v67d+/Wj370I0VERKhr1666//77VVZW5n1+y5YtGjt2rKKiotSlSxddccUVOnzYs9PkV199pWuvvVYxMTGKjY3VqFGj9MUXX7Ra3QAEiRqndGKXtPOP0oZHpVeuk7J6SCvHSm/fK21bIX3/sSeIWMM8M1hG/kKa8px0zwfSgmPSnO3SrX+QrnhY6n0NQeQi1fl6RgzD/54B7SEsssXdiD/72c80d+5c/eMf/9DEiRMlSSdPntTGjRv13nvvqaysTDfeeKOefvpp2e12/fGPf9TUqVOVk5Nzwfu8lJeXa/LkyRo/fry2b9+u/Px83XvvvZozZ47Wrl2rmpoaTZs2Tffdd5/+/Oc/y+Vy6fPPP5elrm0zZszQyJEjtWrVKoWEhGjXrl0KCwu7oDoB6OCqiusGlu4+s2ppwbfNLJUeKaUM9R3fkTRQCmXPMfjX+cJIdYX0W5P2Qvn/TrR4Q6T4+HhNmTJFb7zxhjeMvPXWW0pMTNS1114rq9Wq4cOHe8s/9dRTWr9+vd59991m9wFqqTfeeENVVVX64x//qKgoT31XrFihqVOn6tlnn1VYWJiKi4v14x//WH369JHk2dyw3pEjR/TYY49p4MCBkqR+/fpdUH0AdCCGIZXmNggdX3l+bm5gaXiXBuM76sJH174slY6AdL4wEkRmzJih++67T//93/8tu92u119/XXfccYesVqvKysq0ePFibdiwQbm5uaqpqVFlZaWOHDlywe+7b98+DR8+3BtEJM+Ghm63Wzk5Obrqqqt09913a/Lkybruuus0adIk3XbbbUpLS5Pk2Z/o3nvv1WuvvaZJkybpZz/7mTe0AAgibrd08mDd+h0NptNWFPovH5d+ZsGw+vEdLJWOVtD5wkhYpKeHwqz3DsDUqVNlGIY2bNigMWPG6P/+7//0/PPPS5IeffRRbd68WUuXLlXfvn0VERGhW2+9VS6Xqy1q3sSrr76qhx9+WBs3btSbb76p//iP/9DmzZv1gx/8QIsXL9add96pDRs26P3339eiRYu0bt063XLLLe1SNwDnobrKswOtd2O4rz0LiVWXNy1rsUqJA3xDR+pljOdAm+l8YcRiafGtErOFh4frpz/9qV5//XV99913GjBggC6//HJJ0tatW3X33Xd7P+DLysp06NChVnnfQYMGae3atSovL/f2jmzdulVWq1UDBgzwlhs5cqRGjhypBQsWaPz48XrjjTf0gx/8QJLUv39/9e/fX//2b/+mn//853r11VcJI0BH0XDhsPrbLYU5/sd3hEZ4VixNqwscqcOlFFYsRfvqfGEkyMyYMUM//vGPtWfPHv3iF7/wHu/Xr5/efvttTZ06VRaLRU888USTmTcX8p6LFi1SRkaGFi9erIKCAs2dO1d33XWXUlJS9P333+ull17ST37yE3Xr1k05OTk6cOCAZs6cqcrKSj322GO69dZbdemll+rYsWPavn27pk+f3ip1AxAA7460u8+s4eH42rOmhz8R8b6bwjG+Ax0EYcRkP/rRj5SQkKCcnBzdeeed3uPLli3TPffcowkTJigxMVGPP/64SkpKWuU9IyMjtWnTJj3yyCMaM2aMIiMjNX36dC1btsz7/Lfffqv/+Z//UVFRkdLS0jR79mz96le/Uk1NjYqKijRz5kzl5eUpMTFRP/3pT7VkyZJWqRuAZrhrpaKDDUJHXQBpbkfa+oXDGo7xiO3O+A50SBbDMAyzK3EuZ9uCmO3hcS78HUHQqa6S8vf4TqPN2+N/2QJLiJQ0oNHA0ss8vSCAyc72+d0QPSMAYKbKU76hw7FbKsjxvzFcWGSDHWnrQgc70qITIIx0Aq+//rp+9atf+X2uZ8+e2rNnTzvXCEAThiGVHK8LHQ2m0RY3M74jsqvvjrSpw6SufRjfgU6JMNIJ/OQnP9G4ceP8PsfKqIAJ3LVS4YG60PHVmQBSedJ/+S49626xDD9zuyW2G+M7cNEgjHQCMTExiomJMbsawMWpulLK23tmpdL69TtqKpuWtYR4lkVvOLA09TIpoku7VxvoSAgjANBSFSd9b7E4vpYK90uGn2n3YVFS6tC6wFF3uyVpkBTGIGqgsU4TRoJgUhBMwt8NBMwwpOJjDUJHXQApPuq/fGRio2m0w6WE3ozvAFoo6MNIWFiYLBaLCgoKlJSU5N1ZFpA8QaSgoEAWi4XxM/CvtkYqqhvfUb9omGO3Z5aLP/G9zgworQ8gMamM7wAuQNCHkZCQEPXo0UPHjh1rteXS0blYLBb16NFDISH8L/Wi56qQ8vf6ho68PVJNVdOy1lDPbRWf/VmGSuFx7V9voJML+jAiSdHR0erXr5+qq6vNrgo6oLCwMILIxajiZNPdaIsO+B/fYYuWUoY22J9lmJQ8SAq1t3+9gYtQpwgjkqeHhA8c4CJkGJ6xHA13o3XslkqO+S8fldR0/Y6E3pLV2r71BuDVacIIgItAbY1n9op3N9q6no+q0/7Lx1/qGzrS6sZ3AOhQCCMAOiZX+Zn1O7z7s+yVap1Ny1rDpOSBnp1o68d4pAyVwpvfCwNAx0EYAWC+8iLflUodX0tF3zU/vqPh2h2pl3kGmoba2r/eAFoFYQRA23G7PUugl+ZKpQ7/30tOSGV5/s+PTmm0G+0wz60XxncAnQphBEDgDMOzDkfDYFHmaBQ06r7cLZzlltCnQeiou90Sk9K27QDQIRBGAJxhGJKztPlejIbf/Y3daE5UkmfgaExa0+/RKVJiP8nO/krAxYowAlwsXOV+ei78/Fxd3vLXjEhoFDBSmgaO6BQphNVvATSPMAIEu+oqP7dIGn93SM6Slr+mPa4uUDTTm1EfMtj0DUArIIwAHVWNyzOw86zjMnKb30PFn7CoRsHCX9hIlWxRbdcuAGiEMAK0t9oaqbzg7L0YpblSRWHLXzM0/Oy9GPXfGZcBoAMijACtxe32BAi/4aLB4/J8/+tn+GMNa9CDcZaQEd6FXWMBBC3CCHAu3mmsuc33YpQ6PLdU3DUte01LiGfMxVl7M1I9A0RZUwNAJ0cYwcXLMKSq4qa3Ssry/ExjdbXwRS1nn8Za/z0qUbKysSMASIQRdFbOspatlVFT2fLXjOzadOBndP1U1voZJslMYwWAABFGEJyqSqT8vVLeN9Kpw01vnbhKW/5a4XHnHvgZnSKF2tuuPQBwESOMoGNzu6XThyTHN1LeHk/4cOyWTh8+97m2mEaDP/2FjFTJFtnmzQAANI8wgo7DWSrl7/OEjbxvPAEkf6/kKvNfPqablDLEs5R4w1sl9SuBMo0VAIICYQTtzzA8PRuObzyhoz54nPref/kQm5Q00LNxWspQTwBJGSpFdW3fegMA2gRhBG3LVS7l7fUNHXl7mh/TEZ0qpdYHjss8P3fty6BQAOjECCNoHYYhnT7iO64jb4908l+SjKblQ2xS0oC6no6hdQFkqGfKKwDgokIYQeBcFZ6xHXm7Gwws3SM5i/2Xj0o+EzZSL6sb59Gf3g4AgCTCCM7GMKTiY41usXwjFR2U394Oa1iD3o4hZwJIdHK7Vx0AEDwII/Corqxbt2OP78DSquZ6O5IahI66gaWJ/aVQW/vWGwAQ9AgjFxvDkEqO14WOuim0eXukou/8b95mDZUSB/j2dKQM9UydBQCgFRBGOrPqKqlgn++CYXnfeDZ98yeyq++4jpShntsurDwKAGhDhJHOwDA8S6E3Xrej6DvJqG1a3hLiuaXSeAptdArb0AMA2h1hJNhUV0kF3zadQlt50n/5iATf2yupQz23XcLC27feAAA0gzDSURmGZyt7xzeeKbT1A0sL9zfT22GVuvZrGjxi0ujtAAB0aISRjqDGKRXknBlMWj+wtKLIf/nwLr5Lo6cO9SyXHhbRrtUGAKA1EEbaW2le03U7CvdL7pqmZS1Wz1Lo9YNJ6wNIbDd6OwAAnQZhpK3UuDwho+G4jrxvpPIC/+XD4xotjT5EShrE9vYAgE6PMNIaygoaLY3+jee2i7vaT2GL1LWP734sKUOluB70dgAALkrnFUZWrlyp5557Tg6HQ8OHD9cLL7ygsWPH+i27du1azZo1y+eY3W5XVVXV+by1uWqr63o7Gi0YVpbnv7w9ttHS6JdJyfR2AADQUMBh5M0331RmZqZWr16tcePGafny5Zo8ebJycnKUnOx/D5LY2Fjl5OR4H1uCoQegvNB3XEd9b0ety09hi5TQ23dp9JQhUpdL6O0AAOAcAg4jy5Yt03333eft7Vi9erU2bNigNWvWaP78+X7PsVgsSk1NvbCatpXaGqnoQNMFw8oc/svbYpoujZ48SLJHt2+9AQDoJAIKIy6XSzt27NCCBQu8x6xWqyZNmqRt27Y1e15ZWZl69uwpt9utyy+/XL/97W81ZMiQZss7nU45nU7v45KSkkCq2XKv/VQ69IlU6/T/fPylTdftiLtEslrbpj4AAFyEAgojhYWFqq2tVUqK7yZpKSkp+vbbb/2eM2DAAK1Zs0bDhg1TcXGxli5dqgkTJmjPnj3q0aOH33OysrK0ZMmSQKp2fgy3J4jYouumzzaYQps8SLLHtH0dAAC4yLX5bJrx48dr/Pjx3scTJkzQoEGD9OKLL+qpp57ye86CBQuUmZnpfVxSUqL09PTWr9zk33qWRe/Si94OAABMElAYSUxMVEhIiPLyfGeP5OXltXhMSFhYmEaOHKnvvvuu2TJ2u112ezvsFJsyuO3fAwAAnFVA3QE2m02jRo1Sdna295jb7VZ2drZP78fZ1NbWavfu3UpLSwuspgAAoFMK+DZNZmamMjIyNHr0aI0dO1bLly9XeXm5d3bNzJkz1b17d2VlZUmSnnzySf3gBz9Q3759dfr0aT333HM6fPiw7r333tZtCQAACEoBh5Hbb79dBQUFWrhwoRwOh0aMGKGNGzd6B7UeOXJE1gbjL06dOqX77rtPDodD8fHxGjVqlD799FMNHswtEgAAIFkMwzDMrsS5lJSUKC4uTsXFxYqNjTW7OgAAoAVa+vnNFBIAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYKrzCiMrV65Ur169FB4ernHjxunzzz9v0Xnr1q2TxWLRtGnTzudtAQBAJxRwGHnzzTeVmZmpRYsWaefOnRo+fLgmT56s/Pz8s5536NAhPfroo7ryyivPu7IAAKDzCTiMLFu2TPfdd59mzZqlwYMHa/Xq1YqMjNSaNWuaPae2tlYzZszQkiVL1Lt37wuqMAAA6FwCCiMul0s7duzQpEmTzryA1apJkyZp27ZtzZ735JNPKjk5Wb/85S9b9D5Op1MlJSU+XwAAoHMKKIwUFhaqtrZWKSkpPsdTUlLkcDj8nvPJJ5/oD3/4g15++eUWv09WVpbi4uK8X+np6YFUEwAABJE2nU1TWlqqu+66Sy+//LISExNbfN6CBQtUXFzs/Tp69Ggb1hIAAJgpNJDCiYmJCgkJUV5ens/xvLw8paamNil/8OBBHTp0SFOnTvUec7vdnjcODVVOTo769OnT5Dy73S673R5I1QAAQJAKqGfEZrNp1KhRys7O9h5zu93Kzs7W+PHjm5QfOHCgdu/erV27dnm/fvKTn+jaa6/Vrl27uP0CAAAC6xmRpMzMTGVkZGj06NEaO3asli9frvLycs2aNUuSNHPmTHXv3l1ZWVkKDw/X0KFDfc7v0qWLJDU5DgAALk4Bh5Hbb79dBQUFWrhwoRwOh0aMGKGNGzd6B7UeOXJEVisLuwIAgJaxGIZhmF2JcykpKVFcXJyKi4sVGxtrdnUAAEALtPTzmy4MAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGCq8wojK1euVK9evRQeHq5x48bp888/b7bs22+/rdGjR6tLly6KiorSiBEj9Nprr513hQEAQOcScBh58803lZmZqUWLFmnnzp0aPny4Jk+erPz8fL/lExIS9Jvf/Ebbtm3T119/rVmzZmnWrFnatGnTBVceAAAEP4thGEYgJ4wbN05jxozRihUrJElut1vp6emaO3eu5s+f36LXuPzyy3XTTTfpqaeealH5kpISxcXFqbi4WLGxsYFUFwAAmKSln98B9Yy4XC7t2LFDkyZNOvMCVqsmTZqkbdu2nfN8wzCUnZ2tnJwcXXXVVc2WczqdKikp8fkCAACdU0BhpLCwULW1tUpJSfE5npKSIofD0ex5xcXFio6Ols1m00033aQXXnhB1113XbPls7KyFBcX5/1KT08PpJoAACCItMtsmpiYGO3atUvbt2/X008/rczMTG3ZsqXZ8gsWLFBxcbH36+jRo+1RTQAAYILQQAonJiYqJCREeXl5Psfz8vKUmpra7HlWq1V9+/aVJI0YMUL79u1TVlaWrrnmGr/l7Xa77HZ7IFUDAABBKqCeEZvNplGjRik7O9t7zO12Kzs7W+PHj2/x67jdbjmdzkDeGgAAdFIB9YxIUmZmpjIyMjR69GiNHTtWy5cvV3l5uWbNmiVJmjlzprp3766srCxJnvEfo0ePVp8+feR0OvXee+/ptdde06pVq1q3JQAAICgFHEZuv/12FRQUaOHChXI4HBoxYoQ2btzoHdR65MgRWa1nOlzKy8v10EMP6dixY4qIiNDAgQP1pz/9SbfffnvrtQIAAAStgNcZMQPrjAAAEHzaZJ0RAACA1kYYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADAVYQQAAJiKMAIAAExFGAEAAKYijAAAAFMRRgAAgKkIIwAAwFSEEQAAYCrCCAAAMBVhBAAAmIowAgAATEUYAQAApiKMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACAABMdV5hZOXKlerVq5fCw8M1btw4ff75582Wffnll3XllVcqPj5e8fHxmjRp0lnLAwCAi0vAYeTNN99UZmamFi1apJ07d2r48OGaPHmy8vPz/ZbfsmWLfv7zn+sf//iHtm3bpvT0dF1//fU6fvz4BVceAAAEP4thGEYgJ4wbN05jxozRihUrJElut1vp6emaO3eu5s+ff87za2trFR8frxUrVmjmzJktes+SkhLFxcWpuLhYsbGxgVT3rP6667hKKqvVPyVGA1Jj1CXS1mqvDQDAxa6ln9+hgbyoy+XSjh07tGDBAu8xq9WqSZMmadu2bS16jYqKClVXVyshIaHZMk6nU06n0/u4pKQkkGq22P98ekg7j5z2Pk6OsWtAaownnKTEqH9qjPqnRCvSFtAfEwAACEBAn7KFhYWqra1VSkqKz/GUlBR9++23LXqNxx9/XN26ddOkSZOaLZOVlaUlS5YEUrXzcu2AZHWJtCnHUarjpyuVX+pUfqlT/3eg0KfcJQmRdb0n0d5elN6J0bKFMv4XAIAL1a7/5X/mmWe0bt06bdmyReHh4c2WW7BggTIzM72PS0pKlJ6e3ur1mTuxn/fn0qpqHcgv035HqXLySrU/r1Q5jjIVljl15GSFjpys0If78rzlQ60WXZoYpf6pdb0odSHlkoRIhVgtrV5XAAA6q4DCSGJiokJCQpSXl+dzPC8vT6mpqWc9d+nSpXrmmWf04YcfatiwYWcta7fbZbfbA6naBYsJD9Pll8Tr8kvifY4XlTm1P6/ME07ySr1hpbSqRgfyy3Qgv0wblHum7qFW9UuJ9rnVMyAlRmlx4bJYCCkAADQWUBix2WwaNWqUsrOzNW3aNEmeAazZ2dmaM2dOs+f97ne/09NPP61NmzZp9OjRF1Th9tY12q7x0XaN79PVe8wwDDlKqpTjONODsr+uN8VZ49Y3x0v0zXHfcS4x9tC6MSgxGpAS7Q0pXaPbN3QBANDRBHybJjMzUxkZGRo9erTGjh2r5cuXq7y8XLNmzZIkzZw5U927d1dWVpYk6dlnn9XChQv1xhtvqFevXnI4HJKk6OhoRUdHt2JT2o/FYlFaXITS4iJ0zYBk7/Fat6GjJyt8elD255XqXwXlKnXWaMfhU9px+JTPayVG29S/wW0ez8/RigkPa+9mAQBgioDDyO23366CggItXLhQDodDI0aM0MaNG72DWo8cOSKr9czAzlWrVsnlcunWW2/1eZ1FixZp8eLFF1b7DibEalGvxCj1SozS5CFnblu5atz6vrBcOXmlynGUeHtSjpysUGGZS4VlRfr0YJHPa3XvEqH+dT0oA+tCSp+kaIWHhbR3swAAaFMBrzNihrZaZ8RsFa4aHcgra9KTklfi9FveapF6JUb5DJjtnxKjXl0jFRrCzB4AQMfS0s9vwkgHdLrCpf2NQkqOo1TFldV+y9tCrOqTHO0zFqV/Soy6d4mQlZk9AACTEEY6GcMwVFDq9AYTz+yeMh3IK1WFq9bvOVG2EPVrNKunf2q0kqLtzOwBALQ5wshFwu02dPx0pXJ81kcp1cGCMlXX+r+08ZFhPrd5BqTGqH9yjOIiGTQLAGg9hJGLXHWtW4eLypXjOHO7Z39eqQ4VlcvdzBVPjQ2v60GJ1oDUWA1IiVHf5GhF2Bg0CwAIHGEEflVV1+q7/DJvD0p9UDlRXOW3vMUi9fQuh3+mJ+XSxCiFMWgWAHAWhBEEpKSqWgcaLOBWH1ROlrv8lg8Lsah3YrS3J6U+pKTHRzJoFgAgiTCCVlJY5my0X0+p9ueVqcxZ47d8RFiI3+XwU2IZNAsAFxvCCNqMYRg6UVx1JqTUfT+QXyZXjdvvObHhob4DZuvCSnyUrZ1rDwBoL4QRtLtat6HDReU++/Xk5JXq+8Jy1TYzajYpxt5gEbdo79L4UfZ23VAaANAGCCPoMJw1tfpXQXmD2zyekHL0ZGWz56QnRDRZabZ3UpTsoczsAYBgQRhBh1furNGB/LImY1LyS/0vhx9itehSn+XwPT0plySwHD4AdESEEQStU+Uub++JtyfFUaqSKv+DZkOtFvWIj1B6QqR6do3UJQmRuiQhyvO9a6SiueUDAKYgjKBTMQxDeSXOJpsK7s8rVVW1/0Gz9bpG2XRJXUjpmRBZF1o8YSU5xs5UZABoI4QRXBTcbkOOkiodOVmhI0UVOnKyQodPVtQ9LtepCv+bC9azh1rrelIifXpWenaNVI/4SIWHMUYFAM4XYQSQZzG3I0UVOuoTUjzfj5+ubHaWT73U2HDv7Z5LfG4DRSohysbaKQBwFoQR4Byqa93KPV2lwyfLvSHlcF1QOXKyotmF3epF20M9vSkNwkp9YOnWJYLl8gFc9Fr6+c3IPly0wkKsnhDRNbLJc4Zh6FRFtQ4XlTe5BXT0ZIVyi6tU5qzRvtwS7cstaXJ+iNWibl3CfQbTentVukYqNpwdkgGgHj0jwHmoqq7VsVOVOnKy3NOjUhdS6ntWnM2sRFuvS2RYg8G0DWYAdY1Uamy4QhhUC6AToGcEaEPhYSHqmxytvsnRTZ5zuw0VlDk9PSlFZwbT1t/+KSxz6XRFtU5XFOurY8VNzreFWNUjPsLn1s8ldTOA0hMiFGnj1xZA58K/akArs1otSokNV0psuMb0SmjyfLmzxhtMfGYAFZXr2KlKuWrd+ldhuf5VWO739ZNi7I2mKZ+5/ZMUzYaEAIIPt2mADqTWbejE6Uq/s38OF5U3u/BbvYiwkCbTlOt7WHrER7CcPoB2xW0aIAiFWC1KrwsTE/w8X1xR7Z39c7jId5xKbnGlKqtrPSvX5pU2OddikbrFRSg9IUI968anNJwBFBcRRq8KAFPQMwJ0Eq4at46frtThonKfkFL/VeGqPev5MeGhTZbTr3+cFhfO/j8AAkbPCHCRsYVadWlilC5NjGrynGEYKixz1QWTch0pqtThk2dCS36pU6VVNfrmeIm+Od50qnKo1aLu8RE+PSkNZwCx/w+AC8G/IMBFwGKxKCnGrqQYu0b1jG/yfKWrVkdPNZ39c/hkhY6d9AyqPVy3KJw/XaNsjaYpn5kBxP4/AM6FMAJAEbYQ9U+JUf+UmCbPtWT/n6Jyl4rKXdp19HST8+2hVqUnNA4pZ/YDYv8fAIwZAXBBLnT/n5RYu7p3iVBaXIRS48KVFhfe4HuEkmPsLK0PBCnGjABoF7HhYRraPU5Du8c1ec7f/j8NF4Mrc9Yor8SpvBKnpNN+X99ikZKi7Q1CSoPQEut5nBJnZ9oyEMQIIwDaTEv2/zlyskK5pyuVW1wlR0mV53ux53FeSZWqaw3llzqVX+r0u2Jtva5RtkY9KxF1YeXM4wgbgQXoiAgjAExhsViUEGVTQpRNI9K7+C3jdhsqKnfJUVyl3OLKBmGl7nGx57Gzxu0dt7LnRNPZQPW6RIY1CCgRPreE6o8xMwhof/zWAeiwrNYzs4Au69H0NpDk6WE5XVFd17NS2SCsnAktucVVqnDV1u0JVK1vHU0XhasXYw9VaqNxKz6hJTZCsRGhLBAHtCLCCICgZrFYFB9lU3yUTYO7+R8gZxiGSqpqmvSoOIqrlFty5rZQaVWNSp01Ks0v04H8smbfMyIsxBtQfEJL7JnHCVE2AgvQQoQRAJ2exWJRXESY4iLCNCC16fTlemVOT2DxCS0lDXtaKnWqolqV1bVn3cxQ8ixClxob7juOJdYTWrp18TxOjGINFkAijACAV7Q9VH2To9U3ObrZMlXVtWfCid/bQlUqLHPKVeP2LsXfnNC6HZ4bT2du+Dgp2s5S/Oj0CCMAEIDwsBD1SoxSLz/L7tdz1tQqv8TZZHZQw9CSX1qlGreh46crdfx0ZbOvZbVIyTHhftdgqZ/enBIbLlsogQXBizACAK3MHhri3X25OdW1bhWUOpvODqq7LeSom9pcU7cCrqOkSruONv+eiT5rsTT4Hnump4XVbtFREUYAwARhIVZ16xKhbl0imi1T6zZUVOYJLN4elhLfHhZHcZVctW4VljlVWObU7uPNr8USHxnmcxuoW6PbQqmx4YpiajNMwN86AOigQqwWJceGKzk2XMPT/ZcxDEMny11+ZwfVh5UTxZWqqnbrVEW1TlVUa19u82uxxIaHNrs0f/3jGDtTm9G6CCMAEMQsFou6RtvVNdrud0l+qW5qc2WNcpsMuPV9XOasUUlVjUqqSpWT1/xaLFG2EJ+l+VNjw5Uca1ditGdNmPrvUbYQQgtahDACAJ2cxWJRXGSY4iLDNDC1+c3KSquqm8wMajxjqLiyWuWuWh0sKNfBguanNkue9VgSY2xKij4TUBqGlcRou5LrvrNU/8WNMAIAkCTFhIcpJjxM/VKaX4ulwtVwLZb6/YQqVVjqUkHduJWCUqcqXLWqrK7V0ZOVOnqy+dlC9aLtoXUBxeYJLdFNe1qSYuzqGm1jU8ROiDACAGixSFuoeidFq3dS82uxSFK5s8YbTOq/F5S5PN99jnnWZClz1qjMWaPvz7KQXL24iDBvaPHX45JU971rlI01WoIEYQQA0Oqi7KGKsoeqZ9fm12ORPONZSp01noBSF07OfPf0ttSHl8Iyp6prDRVXVqu4svqct4ksFik+su42Ud3tosa3ieqDTHykTSGshmsawggAwDQWi0Wx4WGKDQ9Tn3P0thiGJ4jU966cCSou7+P6IFNU5pTbkE6Wu3Sy3KWcvLPXw2qRukb79q40DDANg0yXyDAG5rYywggAIChYLBZ1ibSpS6TtrONaJM8aLacqXD63hAobh5e6YycrXHIb8h7bl3v2eoSFWNQ1quHtIVuzPS5Mg24ZwggAoNMJsVqUWNfTcS41tW6dLHcpv9nbRFXeAFNcWa3q2jOr4p6LLdRa18tib9DL4n+8y8W84NzF23IAACSFhli9i8udi7OmVkVl5+hxqQs0pc4auWrc59x/qF6kLcTPbaJwv+NdOtvS/oQRAABayB4acs5l/OtVVdc2GtvSaCZRgwBTWV2rClftOXd6rhdTPxXaZzyLrcnMoq5R9qDYRJEwAgBAGwgPO/eGifXK62YU+d4mqg8yLp/Hrhq3Sp01KnXW6F8tmArdJTLM77otDcNLUt0qvmbNKCKMAABgsvqp0L0Szz0VuqSqplHviv/bRYVlTtW4DZ2uqNbpimodyC8762u/ce84Teib2JrNajHCCAAAQcJisSguIkxxEeeeCu12102F9ull8TMlutSpk+VOJcace7BvWyGMAADQCVmtFsVH2RQfZVP/FkyFNnMCMmEEAICLnNmrz3b8IbYAAKBTI4wAAABTEUYAAICpCCMAAMBU5xVGVq5cqV69eik8PFzjxo3T559/3mzZPXv2aPr06erVq5csFouWL19+vnUFAACdUMBh5M0331RmZqYWLVqknTt3avjw4Zo8ebLy8/P9lq+oqFDv3r31zDPPKDU19YIrDAAAOpeAw8iyZct03333adasWRo8eLBWr16tyMhIrVmzxm/5MWPG6LnnntMdd9whu928BVUAAEDHFFAYcblc2rFjhyZNmnTmBaxWTZo0Sdu2bWu1SjmdTpWUlPh8AQCAzimgMFJYWKja2lqlpKT4HE9JSZHD4Wi1SmVlZSkuLs77lZ6e3mqvDQAAOpYOOZtmwYIFKi4u9n4dPXrU7CoBAIA2EtBy8ImJiQoJCVFeXp7P8by8vFYdnGq32xlfAgDARSKgnhGbzaZRo0YpOzvbe8ztdis7O1vjx49v9coBAIDOL+CN8jIzM5WRkaHRo0dr7NixWr58ucrLyzVr1ixJ0syZM9W9e3dlZWVJ8gx63bt3r/fn48ePa9euXYqOjlbfvn1bsSkAACAYBRxGbr/9dhUUFGjhwoVyOBwaMWKENm7c6B3UeuTIEVmtZzpcTpw4oZEjR3ofL126VEuXLtXVV1+tLVu2tOg9DcOQJGbVAAAQROo/t+s/x5tjMc5VogM4duwYM2oAAAhSR48eVY8ePZp9PijCiNvt1okTJxQTEyOLxdJqr1tSUqL09HQdPXpUsbGxrfa6HUlnbyPtC36dvY20L/h19ja2ZfsMw1Bpaam6devmc9eksYBv05jBarWeNVFdqNjY2E75F6yhzt5G2hf8OnsbaV/w6+xtbKv2xcXFnbNMh1xnBAAAXDwIIwAAwFQXdRix2+1atGhRp15grbO3kfYFv87eRtoX/Dp7GztC+4JiACsAAOi8LuqeEQAAYD7CCAAAMBVhBAAAmIowAgAATNXpw8jKlSvVq1cvhYeHa9y4cfr888/PWv4vf/mLBg4cqPDwcF122WV677332qmm5y+QNq5du1YWi8XnKzw8vB1rG5iPP/5YU6dOVbdu3WSxWPTOO++c85wtW7bo8ssvl91uV9++fbV27do2r+f5CrR9W7ZsaXL9LBaLHA5H+1Q4QFlZWRozZoxiYmKUnJysadOmKScn55znBcvv4fm0L9h+B1etWqVhw4Z5F8QaP3683n///bOeEyzXTwq8fcF2/Rp75plnZLFYNG/evLOWa+9r2KnDyJtvvqnMzEwtWrRIO3fu1PDhwzV58mTl5+f7Lf/pp5/q5z//uX75y1/qyy+/1LRp0zRt2jR988037Vzzlgu0jZJnlb3c3Fzv1+HDh9uxxoEpLy/X8OHDtXLlyhaV//7773XTTTfp2muv1a5duzRv3jzde++92rRpUxvX9PwE2r56OTk5PtcwOTm5jWp4YT766CPNnj1bn332mTZv3qzq6mpdf/31Ki8vb/acYPo9PJ/2ScH1O9ijRw8988wz2rFjh7744gv96Ec/0s0336w9e/b4LR9M108KvH1ScF2/hrZv364XX3xRw4YNO2s5U66h0YmNHTvWmD17tvdxbW2t0a1bNyMrK8tv+dtuu8246aabfI6NGzfO+NWvftWm9bwQgbbx1VdfNeLi4tqpdq1LkrF+/fqzlvn1r39tDBkyxOfY7bffbkyePLkNa9Y6WtK+f/zjH4Yk49SpU+1Sp9aWn59vSDI++uijZssE4+9hvZa0L5h/B+vFx8cbr7zyit/ngvn61Ttb+4L1+pWWlhr9+vUzNm/ebFx99dXGI4880mxZM65hp+0Zcblc2rFjhyZNmuQ9ZrVaNWnSJG3bts3vOdu2bfMpL0mTJ09utrzZzqeNklRWVqaePXsqPT39nP8DCDbBdg3P14gRI5SWlqbrrrtOW7duNbs6LVZcXCxJSkhIaLZMMF/DlrRPCt7fwdraWq1bt07l5eUaP3683zLBfP1a0j4pOK/f7NmzddNNNzW5Nv6YcQ07bRgpLCxUbW2tUlJSfI6npKQ0e3/d4XAEVN5s59PGAQMGaM2aNfrrX/+qP/3pT3K73ZowYYKOHTvWHlVuc81dw5KSElVWVppUq9aTlpam1atX63//93/1v//7v0pPT9c111yjnTt3ml21c3K73Zo3b56uuOIKDR06tNlywfZ7WK+l7QvG38Hdu3crOjpadrtdDzzwgNavX6/Bgwf7LRuM1y+Q9gXj9Vu3bp127typrKysFpU34xoGxa69aD3jx4/3SfwTJkzQoEGD9OKLL+qpp54ysWZoiQEDBmjAgAHexxMmTNDBgwf1/PPP67XXXjOxZuc2e/ZsffPNN/rkk0/MrkqbaGn7gvF3cMCAAdq1a5eKi4v11ltvKSMjQx999FGzH9jBJpD2Bdv1O3r0qB555BFt3ry5Qw+07bRhJDExUSEhIcrLy/M5npeXp9TUVL/npKamBlTebOfTxsbCwsI0cuRIfffdd21RxXbX3DWMjY1VRESESbVqW2PHju3wH/Bz5szR3/72N3388cfq0aPHWcsG2++hFFj7GguG30Gbzaa+fftKkkaNGqXt27fr97//vV588cUmZYPx+gXSvsY6+vXbsWOH8vPzdfnll3uP1dbW6uOPP9aKFSvkdDoVEhLic44Z17DT3qax2WwaNWqUsrOzvcfcbreys7ObvRc4fvx4n/KStHnz5rPeOzTT+bSxsdraWu3evVtpaWltVc12FWzXsDXs2rWrw14/wzA0Z84crV+/Xn//+9916aWXnvOcYLqG59O+xoLxd9DtdsvpdPp9LpiuX3PO1r7GOvr1mzhxonbv3q1du3Z5v0aPHq0ZM2Zo165dTYKIZNI1bLOhsR3AunXrDLvdbqxdu9bYu3evcf/99xtdunQxHA6HYRiGcddddxnz58/3lt+6dasRGhpqLF261Ni3b5+xaNEiIywszNi9e7dZTTinQNu4ZMkSY9OmTcbBgweNHTt2GHfccYcRHh5u7Nmzx6wmnFVpaanx5ZdfGl9++aUhyVi2bJnx5ZdfGocPHzYMwzDmz59v3HXXXd7y//rXv4zIyEjjscceM/bt22esXLnSCAkJMTZu3GhWE84q0PY9//zzxjvvvGMcOHDA2L17t/HII48YVqvV+PDDD81qwlk9+OCDRlxcnLFlyxYjNzfX+1VRUeEtE8y/h+fTvmD7HZw/f77x0UcfGd9//73x9ddfG/PnzzcsFovxwQcfGIYR3NfPMAJvX7BdP38az6bpCNewU4cRwzCMF154wbjkkksMm81mjB071vjss8+8z1199dVGRkaGT/n/9//+n9G/f3/DZrMZQ4YMMTZs2NDONQ5cIG2cN2+et2xKSopx4403Gjt37jSh1i1TP5W18Vd9mzIyMoyrr766yTkjRowwbDab0bt3b+PVV19t93q3VKDte/bZZ40+ffoY4eHhRkJCgnHNNdcYf//7382pfAv4a5skn2sSzL+H59O+YPsdvOeee4yePXsaNpvNSEpKMiZOnOj9oDaM4L5+hhF4+4Lt+vnTOIx0hGtoMQzDaLt+FwAAgLPrtGNGAABAcCCMAAAAUxFGAACAqQgjAADAVIQRAABgKsIIAAAwFWEEAACYijACIChYLBa98847ZlcDQBsgjAA4p7vvvlsWi6XJ1w033GB21QB0Ap12114AreuGG27Qq6++6nPMbrebVBsAnQk9IwBaxG63KzU11ecrPj5ekucWyqpVqzRlyhRFRESod+/eeuutt3zO3717t370ox8pIiJCXbt21f3336+ysjKfMmvWrNGQIUNkt9uVlpamOXPm+DxfWFioW265RZGRkerXr5/effdd73OnTp3SjBkzlJSUpIiICPXr169JeALQMRFGALSKJ554QtOnT9dXX32lGTNm6I477tC+ffskSeXl5Zo8ebLi4+O1fft2/eUvf9GHH37oEzZWrVql2bNn6/7779fu3bv17rvvqm/fvj7vsWTJEt122236+uuvdeONN2rGjBk6efKk9/337t2r999/X/v27dOqVauUmJjYfn8AAM5fm27DB6BTyMjIMEJCQoyoqCifr6efftowDM/utQ888IDPOePGjTMefPBBwzAM46WXXjLi4+ONsrIy7/MbNmwwrFar4XA4DMMwjG7duhm/+c1vmq2DJOM//uM/vI/LysoMScb7779vGIZhTJ061Zg1a1brNBhAu2LMCIAWufbaa7Vq1SqfYwkJCd6fx48f7/Pc+PHjtWvXLknSvn37NHz4cEVFRXmfv+KKK+R2u5WTkyOLxaITJ05o4sSJZ63DsGHDvD9HRUUpNjZW+fn5kqQHH3xQ06dP186dO3X99ddr2rRpmjBhwnm1FUD7IowAaJGoqKgmt01aS0RERIvKhYWF+Ty2WCxyu92SpClTpujw4cN67733tHnzZk2cOFGzZ8/W0qVLW72+AFoXY0YAtIrPPvusyeNBgwZJkgYNGqSvvvpK5eXl3ue3bt0qq9WqAQMGKCYmRr169VJ2dvYF1SEpKUkZGRn605/+pOXLl+ull166oNcD0D7oGQHQIk6nUw6Hw+dYaGiod5DoX/7yF40ePVo//OEP9frrr+vzzz/XH/7wB0nSjBkztGjRImVkZGjx4sUqKCjQ3LlzdddddyklJUWStHjxYj3wwANKTk7WlClTVFpaqq1bt2ru3Lktqt/ChQs1atQoDRkyRE6nU3/729+8YQhAx0YYAdAiGzduVFpams+xAQMG6Ntvv5Xkmemybt06PfTQQ0pLS9Of//xnDR48WJIUGRmpTZs26ZFHHtGYMWMUGRmp6dOna9myZd7XysjIUFVVlZ5//nk9+uijSkxM1K233tri+tlsNi1YsECHDh1SRESErrzySq1bt64VWg6grVkMwzDMrgSA4GaxWLR+/XpNmzbN7KoACEKMGQEAAKYijAAAAFMxZgTABeNuL4ALQc8IAAAwFWEEAACYijACAABMRRgBAACmIowAAABTEUYAAICpCCMAAMBUhBEAAGAqwggAADDV/w/yGI/MsYz8iAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAHHCAYAAABHp6kXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABX5UlEQVR4nO3deVxU5eI/8M/MwCzsIDuiKBqaCxYqYaamFGXxS8UyNUVLzUKvyvUWXPf8GraRXtfqulRimrlkWZZS2tXcLkRqIjdxIZVVhYEBBpg5vz+AkYFhGQRxjp/363VezpzznHOeh4GZj8/znDMSQRAEEBEREVk4aVtXgIiIiKglMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQ0R1Zu3YtJBIJgoOD27oqRHSfk/C7n4joTjz66KO4fv06Ll++jD///BNdunRp6yoR0X2KPTVE1GyXLl3Cr7/+ivj4eLi5uSEhIaGtq2SSRqNp6yoQ0V3AUENEzZaQkABnZ2c888wzGD16tMlQk5+fjzlz5sDPzw8KhQLt27fHxIkTkZeXZyhTWlqKxYsX44EHHoBSqYSXlxdGjRqF9PR0AMChQ4cgkUhw6NAho2NfvnwZEokEmzdvNqybNGkS7OzskJ6ejuHDh8Pe3h7jx48HAPznP//B888/jw4dOkChUMDX1xdz5sxBSUlJnXqfP38eL7zwAtzc3KBSqRAQEIB58+YBAH7++WdIJBLs3r27zn5bt26FRCLBsWPHzP55EtGdsWrrChCR5UpISMCoUaMgl8sxduxYrFu3DqdOnUK/fv0AAEVFRXjssceQmpqKl19+GQ8//DDy8vKwd+9eXL16Fa6urtDpdHj22WeRmJiIF198EbNmzUJhYSEOHDiAs2fPwt/f3+x6VVRUICwsDAMHDsT7778PGxsbAMCOHTtQXFyM1157De3atcPJkyexatUqXL16FTt27DDsf/r0aTz22GOwtrbGtGnT4Ofnh/T0dHzzzTdYtmwZhgwZAl9fXyQkJGDkyJF1fib+/v4ICQm5g58sETWLQETUDP/9738FAMKBAwcEQRAEvV4vtG/fXpg1a5ahzMKFCwUAwq5du+rsr9frBUEQhI0bNwoAhPj4+HrL/PzzzwIA4eeffzbafunSJQGAsGnTJsO6yMhIAYAQExNT53jFxcV11sXFxQkSiUS4cuWKYd2gQYMEe3t7o3U16yMIghAbGysoFAohPz/fsC4nJ0ewsrISFi1aVOc8RNT6OPxERM2SkJAADw8PPP744wAAiUSCMWPGYNu2bdDpdACAnTt3IjAwsE5vRnX56jKurq6YOXNmvWWa47XXXquzTqVSGR5rNBrk5eVhwIABEAQBv/32GwAgNzcXv/zyC15++WV06NCh3vpMnDgRWq0WX331lWHd9u3bUVFRgZdeeqnZ9Sai5mOoISKz6XQ6bNu2DY8//jguXbqECxcu4MKFCwgODkZ2djYSExMBAOnp6ejZs2eDx0pPT0dAQACsrFpuNNzKygrt27evsz4jIwOTJk2Ci4sL7Ozs4ObmhsGDBwMACgoKAAAXL14EgEbr3a1bN/Tr189oHlFCQgIeeeQRXgFG1EY4p4aIzPbTTz8hMzMT27Ztw7Zt2+psT0hIwJNPPtli56uvx6a6R6g2hUIBqVRap+wTTzyBmzdv4s0330S3bt1ga2uLa9euYdKkSdDr9WbXa+LEiZg1axauXr0KrVaL48ePY/Xq1WYfh4haBkMNEZktISEB7u7uWLNmTZ1tu3btwu7du7F+/Xr4+/vj7NmzDR7L398fJ06cQHl5OaytrU2WcXZ2BlB5JVVNV65caXKdz5w5g//973/49NNPMXHiRMP6AwcOGJXr3LkzADRabwB48cUXER0djS+++AIlJSWwtrbGmDFjmlwnImpZHH4iIrOUlJRg165dePbZZzF69Og6y4wZM1BYWIi9e/ciIiICv//+u8lLn4Wq+35GREQgLy/PZA9HdZmOHTtCJpPhl19+Mdq+du3aJtdbJpMZHbP68cqVK43Kubm5YdCgQdi4cSMyMjJM1qeaq6srnn76aWzZsgUJCQl46qmn4Orq2uQ6EVHLYk8NEZll7969KCwsxP/7f//P5PZHHnnEcCO+rVu34quvvsLzzz+Pl19+GUFBQbh58yb27t2L9evXIzAwEBMnTsRnn32G6OhonDx5Eo899hg0Gg0OHjyI119/Hc899xwcHR3x/PPPY9WqVZBIJPD398e3336LnJycJte7W7du8Pf3x9y5c3Ht2jU4ODhg586duHXrVp2y//rXvzBw4EA8/PDDmDZtGjp16oTLly9j3759SElJMSo7ceJEjB49GgCwdOnSpv8giajlteWlV0RkecLDwwWlUiloNJp6y0yaNEmwtrYW8vLyhBs3bggzZswQfHx8BLlcLrRv316IjIwU8vLyDOWLi4uFefPmCZ06dRKsra0FT09PYfTo0UJ6erqhTG5urhARESHY2NgIzs7OwquvviqcPXvW5CXdtra2Jut17tw5ITQ0VLCzsxNcXV2FqVOnCr///nudYwiCIJw9e1YYOXKk4OTkJCiVSiEgIEBYsGBBnWNqtVrB2dlZcHR0FEpKSpr4UySi1sDvfiIiugMVFRXw9vZGeHg4NmzY0NbVIbqvcU4NEdEd2LNnD3Jzc40mHxNR22BPDRFRM5w4cQKnT5/G0qVL4erqiuTk5LauEtF9jz01RETNsG7dOrz22mtwd3fHZ5991tbVISKwp4aIiIhEgj01REREJAoMNURERCQK983N9/R6Pa5fvw57e/s7+uZfIiIiunsEQUBhYSG8vb3rfKdbbfdNqLl+/Tp8fX3buhpERETUDH/99Rfat2/fYJn7JtTY29sDqPyhODg4tHFtiIiIqCnUajV8fX0Nn+MNuW9CTfWQk4ODA0MNERGRhWnK1BFOFCYiIiJRYKghIiIiUWCoISIiIlFgqCEiIiJRYKghIiIiUWCoISIiIlFgqCEiIiJRYKghIiIiUWCoISIiIlFgqCEiIiJRYKghIiIiUWCoISIiIlG4b77QkoiIiFpHSZkOt4rLAADeTqo2qwdDDREREQEABEFAkbYCtzTluFVcdnvRlCO/uAw3i8twq7jqcdW6W8VlKC3XAwCeeNADn0zs22b1Z6ghIiISIZ1egLqkHDeLyyrDh6bG4+Jy3NJUh5bqx5UhpUIvNOt8VlIJBKF5+7YUhhoiIqJ7XFmFHvklZYYelOqekpqP84trhJTiMhSUlKO5GUNpLYWzjbxysbWGk40cLjZyONtUPbaVw8nGGs41HtsprCCRSFq24WZiqCEiIrqLquef3NSUIb+43GiIxxBSimuEFE05irQVzT6fvcIKTrbWcLGRw6kqmDjbVgWWGo+dbKzhUvVYaS1rwRbfPQw1REREzSAIAgq1FcivGta5VWOY51ax6aGdm5oyaCv0zTqfRAI4qayrek9M95qYCirWsvvnQmeGGiIiuu/p9AIKSpowtGMILHc2/8RaJrnda1JjmKf6cXWvSXUZF1s5HJTWkErbdnjnXsdQQ0REolJWob89Gba4zNBTUvNx9ZU8+S0w/0RlLTPda2JbI7TUemwrl7X5/BMxYqghIqJ7VkmZrnJop2r+yc16hnjya8xRuaP5J0or03NNbORwqgomhrkpVT0rljr/RIwYaoiI6K7T6wVkqktxKVeDS3lFuJinQY5aW2seSvPnn0glgJNN/XNNavasVD++3+afiBFDDRERtQpBEHCruLwytORqcCnPeGlqYLGWSRqca2Lqih7OP7k/MdQQEdEdKS6ruB1WqsLLxarnBSXl9e5nLZOgg4sNOrnaobObLbwclcYhhfNPyEwMNURE1KhynR5/3Sw2hJeLNQJMlrq0wX19nFTo5Gp7e3GzRWdXW/g4qWDF4R5qQQw1REQEoHK4KFutxcW8osrgUmPIKONmMXQNXL7sYis3Ci6dq8JLRxdbqOScSEt3B0MNEdF9pqC43BBcave6lJTr6t1PZS0z6mmpGWKcbOR3sQVEpjHUEBGJUGm5DpdvVIaVi7Um6N7UlNW7n5W0ep7L7aGiyp4XO3g4KDi3he5pDDVERBaqQqfHtfwSo56W6uVafkmD+3o6KE32uvi62PCyZrJYDDVERPcwQRCQW6g16m25WHVvl4ybxSjX1T/PxUFphc5udrdDS1Wvi187W9gq+PZP4sPfaiKie4C6tLzO5dCX8opwKVcDTVn981wUVlLjK4tcbdHZzRadXO3gbGPN4SK6rzQr1KxZswbvvfcesrKyEBgYiFWrVqF///4my5aXlyMuLg6ffvoprl27hoCAALzzzjt46qmnDGUWL16MJUuWGO0XEBCA8+fPG56Xlpbi73//O7Zt2watVouwsDCsXbsWHh4ezWkCEdFdV1quQ8bN4hpXFd2erJtXVP88F6kE8K0xz6Wy58UOndxs4eWg5E3miKqYHWq2b9+O6OhorF+/HsHBwVixYgXCwsKQlpYGd3f3OuXnz5+PLVu24JNPPkG3bt3www8/YOTIkfj111/x0EMPGcr16NEDBw8evF0xK+OqzZkzB/v27cOOHTvg6OiIGTNmYNSoUTh69Ki5TSAiajU6vYDrhnkuRUY9L9fySxr80kR3e0WNnpaq4OJqiw4uNpBbcZ4LUWMkgmDe95IGBwejX79+WL16NQBAr9fD19cXM2fORExMTJ3y3t7emDdvHqKiogzrIiIioFKpsGXLFgCVPTV79uxBSkqKyXMWFBTAzc0NW7duxejRowEA58+fR/fu3XHs2DE88sgjjdZbrVbD0dERBQUFcHBwMKfJRERGBEHADU1Z1fyWIqOJulduFKNMV//t/+0VVsahpWqirp+rLew4z4WoDnM+v836CyorK0NSUhJiY2MN66RSKUJDQ3Hs2DGT+2i1WiiVSqN1KpUKR44cMVr3559/wtvbG0qlEiEhIYiLi0OHDh0AAElJSSgvL0doaKihfLdu3dChQ4d6Q41Wq4VWqzU8V6vV5jSViAhF2gpcNrqPy+2el8LS+r8JWm4lhV87G0Nw6Vxjkm47WznnuRC1ErNCTV5eHnQ6XZ15LB4eHkbzX2oKCwtDfHw8Bg0aBH9/fyQmJmLXrl3Q6W5PfAsODsbmzZsREBCAzMxMLFmyBI899hjOnj0Le3t7ZGVlQS6Xw8nJqc55s7KyTJ43Li6uzjwdIqLayir0yDDc/t/4Tro5hdp695NIgPbOqtuhpcbi7aSCjPNciO66Vu/rXLlyJaZOnYpu3bpBIpHA398fkydPxsaNGw1lnn76acPj3r17Izg4GB07dsSXX36JV155pVnnjY2NRXR0tOG5Wq2Gr69v8xtCRBZLrxeQqS419LbUvDz6r5vFaODu/3C1U9S5JLpz1f1clNa8/T/RvcSsUOPq6gqZTIbs7Gyj9dnZ2fD09DS5j5ubG/bs2YPS0lLcuHED3t7eiImJQefOnes9j5OTEx544AFcuHABAODp6YmysjLk5+cb9dY0dF6FQgGFQmFO84jIggmCgFvF5ZWhpdaN6C7laaCtqH+ei61cVhVY7GpcXVQ5z8VRZX0XW0FEd8KsUCOXyxEUFITExESMGDECQOVE4cTERMyYMaPBfZVKJXx8fFBeXo6dO3fihRdeqLdsUVER0tPTMWHCBABAUFAQrK2tkZiYiIiICABAWloaMjIyEBISYk4TiMgClVXokVukRba6FDlqLXILS5Gt1iKnsBQ5hVpkq7W4nl+CgpLyeo9hLau+/b9djYm6lQHGzZ63/ycSA7OHn6KjoxEZGYm+ffuif//+WLFiBTQaDSZPngwAmDhxInx8fBAXFwcAOHHiBK5du4Y+ffrg2rVrWLx4MfR6Pd544w3DMefOnYvw8HB07NgR169fx6JFiyCTyTB27FgAgKOjI1555RVER0fDxcUFDg4OmDlzJkJCQpp05RMR3ZtKy3XILawMJ9lqLXLUt0NKTmEpcgsrg8yt4vrDSm0+Tirjm9FVXV3k46SCFW//TyRqZoeaMWPGIDc3FwsXLkRWVhb69OmD/fv3GyYPZ2RkQCq9/cZRWlqK+fPn4+LFi7Czs8Pw4cPx+eefGw0jXb16FWPHjsWNGzfg5uaGgQMH4vjx43BzczOU+fDDDyGVShEREWF08z0iuvcUl1UgR61FTs3AUliKXLUW2YWlhm0N9azUJpdJ4WavgLuDAu72Cng4KOFur4C7vRLuDgp4Oirh186W81yI7mNm36fGUvE+NUR3rkhbgRx1aZ2elJxCLXKqAkuuWotCbf2XO9emsJJWBRUlPKr+dasRWqr/deIt/4nuS612nxoiEh9BEFBYFVZyavWkVAeW6vBS3MB3ENWmspbdDikOCnhU9ajU7mVxUFkxrBBRi2CoIRIpQRBQUFJ+O5zUCCq5tYaFSsvrvzKoNjuFFdztFUa9Ke4OlY/d7G/3uNgpGFaI6O5iqCGyMHq9gFvFZVXzVWqEFLXxFUE5hVqUNXAZc232SqsaPSg1QoqDEh5V/7rbK2DLW/kT0T2K705E9wi9vvL7hEz1pFQOC2mRqy5FbpEW5bqmT4VzsrE2mlDrbl9jrkrVsJCbvQIqOSfYEpFlY6ghamUVOj1uaMoqg0n1pNqqwJJruM9KKfKKyqBr6Na2tbjYyquGfqpDyu3AUr3OzV7Bq4GI6L7BUEPUTOU6PfKKtEb3V7l9n5XbQ0A3irQN3oa/JokEaGeruD1PpcbkWneH24HFzU4BuRXvuUJEVBNDDVEt2orqG8LVDCvGQSVHXYqbxWVo6g0RpJLK7xCqObG25nBQdS+Lq52cN4gjImomhhq6L+n1Ai7f0ODMtQL8cV2N81mFyC4oRU6heXevtZJKqq74UcCtRjipvBro9uN2tgp+azMRUStjqCHRq9DpkZ6rwdlrBTh7vQB/XFPjj+sF0DRwzxVrmaRGT0qtG8PVGBZysZFDyrBCRHRPYKghUdFW6PBndpEhwJy9pkZqptrkNzQrraXo7uWAnt6OeNDbAT5OKsNwkDPvXktEZHEYashilZbrkJqpxtnravxRFWLSsgpNXu5sK5ehh7cjevo4oqePA3r6OKKzqy3nrxARiQhDDVmEIm0Fzl1XGw0hXcgtMnkJtKPKujK4eDuih48jeno7wK+dLYeJiIhEjqGG7jkFxeX44/rt4aOz1wtwKU9j8kojVzt5Ze+Ld2UPTA9vR7R3VnHoiIjoPsRQQ20qr0iLs1VXIFX3wvx1s8RkWS9HZdUQkkNViHGEh4OCAYaIiAAw1NBdIggCstVaowm8Z68VIEtdarJ8BxcbQ89LTx9H9PB2gKud4i7XmoiILAlDDbU4QRBw9VaJUYD543oB8orK6pSVSIBOrraG4aOe3o7o4e0IRxvrNqg5ERFZMoYauiPVN7E7Wz18VLWoSyvqlJVJJejqbnd7CMnHEd29HGDHb30mIqIWwE8TajJzbmJnLZMgwNPeMPelp48junna88sViYio1TDUkEllFXr8L7sQf1wvwJlrTb+JXfU8mAc87PmFi0REdFcx1BBvYkdERKLAUHOfKdJWIDVTjTNXeRM7IiISF4YaEeNN7IiI6H7CUCMSvIkdERHd7xhqLIypm9j9cb0AmQW8iR0REd3fGGruYXd0EzsfR/Tw4k3siIjo/sFQc4+oeRO7P2qEmIKS8jpleRM7IiKiuvgp2AZ4EzsiIqKWx1DTymrexK76CqTUTDVKy3kTOyIiopbEUNOCeBM7IiKitsNQc4dOX83Hp79ewdlrBbyJHRERURtiqLlD+cXl2Jl81fCcN7EjIiJqGww1d6h3e0fMDu3Km9gRERG1MYaaO+RkI8fs0AfauhpERET3Pc5KJSIiIlFoVqhZs2YN/Pz8oFQqERwcjJMnT9Zbtry8HG+99Rb8/f2hVCoRGBiI/fv3G5WJi4tDv379YG9vD3d3d4wYMQJpaWlGZYYMGQKJRGK0TJ8+vTnVJyIiIhEyO9Rs374d0dHRWLRoEZKTkxEYGIiwsDDk5OSYLD9//nx89NFHWLVqFc6dO4fp06dj5MiR+O233wxlDh8+jKioKBw/fhwHDhxAeXk5nnzySWg0GqNjTZ06FZmZmYbl3XffNbf6REREJFISQRDqXoPcgODgYPTr1w+rV68GAOj1evj6+mLmzJmIiYmpU97b2xvz5s1DVFSUYV1ERARUKhW2bNli8hy5ublwd3fH4cOHMWjQIACVPTV9+vTBihUrzKmugVqthqOjIwoKCuDg4NCsYxAREdHdZc7nt1k9NWVlZUhKSkJoaOjtA0ilCA0NxbFjx0zuo9VqoVQqjdapVCocOXKk3vMUFBQAAFxcXIzWJyQkwNXVFT179kRsbCyKi4vrPYZWq4VarTZaiIiISLzMuvopLy8POp0OHh4eRus9PDxw/vx5k/uEhYUhPj4egwYNgr+/PxITE7Fr1y7odHW/5wio7PmZPXs2Hn30UfTs2dOwfty4cejYsSO8vb1x+vRpvPnmm0hLS8OuXbtMHicuLg5Lliwxp3lERERkwVr9ku6VK1di6tSp6NatGyQSCfz9/TF58mRs3LjRZPmoqCicPXu2Tk/OtGnTDI979eoFLy8vDBs2DOnp6fD3969znNjYWERHRxueq9Vq+Pr6tlCriIiI6F5j1vCTq6srZDIZsrOzjdZnZ2fD09PT5D5ubm7Ys2cPNBoNrly5gvPnz8POzg6dO3euU3bGjBn49ttv8fPPP6N9+/YN1iU4OBgAcOHCBZPbFQoFHBwcjBYiIiISL7NCjVwuR1BQEBITEw3r9Ho9EhMTERIS0uC+SqUSPj4+qKiowM6dO/Hcc88ZtgmCgBkzZmD37t346aef0KlTp0brkpKSAgDw8vIypwlEREQkUmYPP0VHRyMyMhJ9+/ZF//79sWLFCmg0GkyePBkAMHHiRPj4+CAuLg4AcOLECVy7dg19+vTBtWvXsHjxYuj1erzxxhuGY0ZFRWHr1q34+uuvYW9vj6ysLACAo6MjVCoV0tPTsXXrVgwfPhzt2rXD6dOnMWfOHAwaNAi9e/duiZ8DERERWTizQ82YMWOQm5uLhQsXIisrC3369MH+/fsNk4czMjIgld7uACotLcX8+fNx8eJF2NnZYfjw4fj888/h5ORkKLNu3ToAlZdt17Rp0yZMmjQJcrkcBw8eNAQoX19fREREYP78+c1oMhEREYmR2fepsVS8Tw0REZHlabX71BARERHdqxhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBSaFWrWrFkDPz8/KJVKBAcH4+TJk/WWLS8vx1tvvQV/f38olUoEBgZi//79Zh+ztLQUUVFRaNeuHezs7BAREYHs7OzmVJ+IiIhEyOxQs337dkRHR2PRokVITk5GYGAgwsLCkJOTY7L8/Pnz8dFHH2HVqlU4d+4cpk+fjpEjR+K3334z65hz5szBN998gx07duDw4cO4fv06Ro0a1YwmExERkRhJBEEQzNkhODgY/fr1w+rVqwEAer0evr6+mDlzJmJiYuqU9/b2xrx58xAVFWVYFxERAZVKhS1btjTpmAUFBXBzc8PWrVsxevRoAMD58+fRvXt3HDt2DI888kij9Var1XB0dERBQQEcHBzMaTIRERG1EXM+v83qqSkrK0NSUhJCQ0NvH0AqRWhoKI4dO2ZyH61WC6VSabROpVLhyJEjTT5mUlISysvLjcp069YNHTp0aPC8arXaaCEiIiLxMivU5OXlQafTwcPDw2i9h4cHsrKyTO4TFhaG+Ph4/Pnnn9Dr9Thw4AB27dqFzMzMJh8zKysLcrkcTk5OTT5vXFwcHB0dDYuvr685TSUiIiIL0+pXP61cuRJdu3ZFt27dIJfLMWPGDEyePBlSaeueOjY2FgUFBYblr7/+atXzERERUdsyK1m4urpCJpPVueooOzsbnp6eJvdxc3PDnj17oNFocOXKFZw/fx52dnbo3Llzk4/p6emJsrIy5OfnN/m8CoUCDg4ORgsRERGJl1mhRi6XIygoCImJiYZ1er0eiYmJCAkJaXBfpVIJHx8fVFRUYOfOnXjuueeafMygoCBYW1sblUlLS0NGRkaj5yUiIqL7g5W5O0RHRyMyMhJ9+/ZF//79sWLFCmg0GkyePBkAMHHiRPj4+CAuLg4AcOLECVy7dg19+vTBtWvXsHjxYuj1erzxxhtNPqajoyNeeeUVREdHw8XFBQ4ODpg5cyZCQkKadOUTERERiZ/ZoWbMmDHIzc3FwoULkZWVhT59+mD//v2Gib4ZGRlG82VKS0sxf/58XLx4EXZ2dhg+fDg+//xzo0m/jR0TAD788ENIpVJERERAq9UiLCwMa9euvYOmExERkZiYfZ8aS8X71BAREVmeVrtPDREREdG9iqGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEwaqtK0BERK1Dp9OhvLy8ratB1CBra2vIZLIWORZDDRGRyAiCgKysLOTn57d1VYiaxMnJCZ6enpBIJHd0HIYaIiKRqQ407u7usLGxueMPCqLWIggCiouLkZOTAwDw8vK6o+Mx1BARiYhOpzMEmnbt2rV1dYgapVKpAAA5OTlwd3e/o6EoThQmIhKR6jk0NjY2bVwToqar/n290zlgDDVERCLEISeyJC31+8pQQ0RERKLAUENERKLj5+eHFStWNLn8oUOHIJFIeMWYheNEYSIiuicMGTIEffr0MSuM1OfUqVOwtbVtcvkBAwYgMzMTjo6Od3xuajvN6qlZs2YN/Pz8oFQqERwcjJMnTzZYfsWKFQgICIBKpYKvry/mzJmD0tJSw3Y/Pz9IJJI6S1RUlKHMkCFD6myfPn16c6pPREQWSBAEVFRUNKmsm5ubWZOl5XJ5i9wnxdJZ+s0azQ4127dvR3R0NBYtWoTk5GQEBgYiLCzMcI15bVu3bkVMTAwWLVqE1NRUbNiwAdu3b8c///lPQ5lTp04hMzPTsBw4cAAA8Pzzzxsda+rUqUbl3n33XXOrT0R03xEEAcVlFW2yCILQpDpOmjQJhw8fxsqVKw3/cd28eTMkEgm+//57BAUFQaFQ4MiRI0hPT8dzzz0HDw8P2NnZoV+/fjh48KDR8WoPP0kkEvz73//GyJEjYWNjg65du2Lv3r2G7bWHnzZv3gwnJyf88MMP6N69O+zs7PDUU08hMzPTsE9FRQX+9re/wcnJCe3atcObb76JyMhIjBgxoklt3r9/PwYOHGjY/9lnn0V6erpRmatXr2Ls2LFwcXGBra0t+vbtixMnThi2f/PNN+jXrx+USiVcXV0xcuRIozbv2bPH6HhOTk7YvHkzAODy5cuQSCTYvn07Bg8eDKVSiYSEBNy4cQNjx46Fj48PbGxs0KtXL3zxxRdGx9Hr9Xj33XfRpUsXKBQKdOjQAcuWLQMADB06FDNmzDAqn5ubC7lcjsTExCb9bJrL7OGn+Ph4TJ06FZMnTwYArF+/Hvv27cPGjRsRExNTp/yvv/6KRx99FOPGjQNQ+Ys2duxYoxfFzc3NaJ/ly5fD398fgwcPNlpvY2MDT09Pc6tMRHRfKynX4cGFP7TJuc+9FQYbeeMfNStXrsT//vc/9OzZE2+99RYA4I8//gAAxMTE4P3330fnzp3h7OyMv/76C8OHD8eyZcugUCjw2WefITw8HGlpaejQoUO951iyZAneffddvPfee1i1ahXGjx+PK1euwMXFxWT54uJivP/++/j8888hlUrx0ksvYe7cuUhISAAAvPPOO0hISMCmTZvQvXt3rFy5Env27MHjjz/epJ+NRqNBdHQ0evfujaKiIixcuBAjR45ESkoKpFIpioqKMHjwYPj4+GDv3r3w9PREcnIy9Ho9AGDfvn0YOXIk5s2bh88++wxlZWX47rvvmnTummJiYvDBBx/goYceglKpRGlpKYKCgvDmm2/CwcEB+/btw4QJE+Dv74/+/fsDAGJjY/HJJ5/gww8/xMCBA5GZmYnz588DAKZMmYIZM2bggw8+gEKhAABs2bIFPj4+GDp0qNn1M4dZoaasrAxJSUmIjY01rJNKpQgNDcWxY8dM7jNgwABs2bIFJ0+eRP/+/XHx4kV89913mDBhQr3n2LJlC6Kjo+t0AyYkJGDLli3w9PREeHg4FixYUG/3olarhVarNTxXq9XmNJWIiO4iR0dHyOVyo/+8Vn9IvvXWW3jiiScMZV1cXBAYGGh4vnTpUuzevRt79+6t00NQ06RJkzB27FgAwNtvv41//etfOHnyJJ566imT5cvLy7F+/Xr4+/sDAGbMmGEIXACwatUqxMbGGnpHVq9ebVaoiIiIMHq+ceNGuLm54dy5c+jZsye2bt2K3NxcnDp1yhC8unTpYii/bNkyvPjii1iyZIlhXc2fS1PNnj0bo0aNMlo3d+5cw+OZM2fihx9+wJdffon+/fujsLAQK1euxOrVqxEZGQkA8Pf3x8CBAwEAo0aNwowZM/D111/jhRdeAFDZ8zVp0qRWH94zK9Tk5eVBp9PBw8PDaL2Hh4fhl6+2cePGIS8vDwMHDjSMh06fPt1o+KmmPXv2ID8/H5MmTapznI4dO8Lb2xunT5/Gm2++ibS0NOzatcvkceLi4oxeaCKi+5XKWoZzb4W12bnvVN++fY2eFxUVYfHixdi3bx8yMzNRUVGBkpISZGRkNHic3r17Gx7b2trCwcGh3qkTQOXoQHWgASpv4V9dvqCgANnZ2YaeCwCQyWQICgoy9KQ05s8//8TChQtx4sQJ5OXlGfbLyMhAz549kZKSgoceeqjenqSUlBRMnTq1SedqSO2fr06nw9tvv40vv/wS165dQ1lZGbRaraETITU1FVqtFsOGDTN5PKVSiQkTJmDjxo144YUXkJycjLNnzxoN97WWVr/66dChQ3j77bexdu1aBAcH48KFC5g1axaWLl2KBQsW1Cm/YcMGPP300/D29jZaP23aNMPjXr16wcvLC8OGDUN6errRL1212NhYREdHG56r1Wr4+vq2YMuIiCyDRCJp0hDQvar2VUxz587FgQMH8P7776NLly5QqVQYPXo0ysrKGjyOtbW10XOJRNJgADFVvqlzhJoiPDwcHTt2xCeffAJvb2/o9Xr07NnT0I7qrw+oT2PbTdXX1ETg2j/f9957DytXrsSKFSvQq1cv2NraYvbs2U2uF1A5BNWnTx9cvXoVmzZtwtChQ9GxY8dG97tTZk0UdnV1hUwmQ3Z2ttH67Ozseue6LFiwABMmTMCUKVPQq1cvjBw5Em+//Tbi4uLq/DJduXIFBw8exJQpUxqtS3BwMADgwoULJrcrFAo4ODgYLUREdO+Sy+XQ6XSNljt69CgmTZqEkSNHolevXvD09MTly5dbv4I1ODo6wsPDA6dOnTKs0+l0SE5ObtL+N27cQFpaGubPn49hw4ahe/fuuHXrllGZ3r17IyUlBTdv3jR5jN69ezc48dbNzc1oYvOff/6J4uLiRut29OhRPPfcc3jppZcQGBiIzp0743//+59he9euXaFSqRo8d69evdC3b1988skn2Lp1K15++eVGz9sSzAo1crkcQUFBRg3R6/VITExESEiIyX2Ki4shlRqfpvrLqmonyE2bNsHd3R3PPPNMo3VJSUkBcOff6ElERPcGPz8/nDhxApcvXzYajqmta9eu2LVrF1JSUvD7779j3LhxTR7yaUkzZ85EXFwcvv76a6SlpWHWrFm4detWk+aNODs7o127dvj4449x4cIF/PTTT0ajCwAwduxYeHp6YsSIETh69CguXryInTt3GuawLlq0CF988YXh6uIzZ87gnXfeMew/dOhQrF69Gr/99hv++9//Yvr06XV6n0zp2rUrDhw4gF9//RWpqal49dVXjTozlEol3nzzTbzxxhv47LPPkJ6ejuPHj2PDhg1Gx5kyZQqWL18OQRCMrspqTWZf0h0dHY1PPvkEn376KVJTU/Haa69Bo9EYroaaOHGi0UTi8PBwrFu3Dtu2bcOlS5dw4MABLFiwAOHh4UbfxKnX67Fp0yZERkbCysq4mzQ9PR1Lly5FUlISLl++jL1792LixIkYNGiQ0RgpERFZrrlz50Imk+HBBx+Em5tbvXNk4uPj4ezsjAEDBiA8PBxhYWF4+OGH73JtgTfffBNjx47FxIkTERISAjs7O4SFhUGpVDa6r1QqxbZt25CUlISePXtizpw5eO+994zKyOVy/Pjjj3B3d8fw4cPRq1cvLF++3PDZOWTIEOzYsQN79+5Fnz59MHToUKP7xn3wwQfw9fXFY489hnHjxmHu3LlNunfP/Pnz8fDDDyMsLAxDhgwxBKuaFixYgL///e9YuHAhunfvjjFjxtSZnzR27FhYWVlh7NixTfqZtAihGVatWiV06NBBkMvlQv/+/YXjx48btg0ePFiIjIw0PC8vLxcWL14s+Pv7C0qlUvD19RVef/114datW0bH/OGHHwQAQlpaWp3zZWRkCIMGDRJcXFwEhUIhdOnSRfjHP/4hFBQUNLnOBQUFAgCz9iEisjQlJSXCuXPnhJKSkrauyn1Hp9MJDzzwgDB//vy2rso94dKlS4JUKhWSkpIaLdvQ7605n98SQWjBWU/3MLVaDUdHRxQUFHB+DRGJVmlpKS5duoROnTrdvf8d36euXLmCH3/8EYMHD4ZWq8Xq1auxadMm/P777+jevXtbV6/NlJeX48aNG5g7dy4uXbqEo0ePNrpPQ7+35nx+8wstiYiImkEqlWLz5s3o168fHn30UZw5cwYHDx5E9+7dkZGRATs7u3qXxi4/t2RHjx6Fl5cXTp06hfXr19/Vc1vuNX5ERERtyNfXt95eCG9vb8MFLfVtF6shQ4a06KXv5mCoISIiamFWVlZGd/+lu4PDT0RERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REQkCn5+flixYkVbV4PaEEMNERERiQJDDRERURvT6XRt8k3jYsNQQ0REbe7jjz+Gt7d3nQ/25557Di+//DLS09Px3HPPwcPDA3Z2dujXrx8OHjzY7PPFx8ejV69esLW1ha+vL15//XUUFRUZlTl69CiGDBkCGxsbODs7IywsDLdu3QIA6PV6vPvuu+jSpQsUCgU6dOiAZcuWAQAOHToEiUSC/Px8w7FSUlIgkUhw+fJlAMDmzZvh5OSEvXv34sEHH4RCoUBGRgZOnTqFJ554Aq6urnB0dMTgwYORnJxsVK/8/Hy8+uqr8PDwgFKpRM+ePfHtt99Co9HAwcEBX331lVH5PXv2wNbWFoWFhc3+eVkKhhoiIrETBKBM0zZLE2+X//zzz+PGjRv4+eefDetu3ryJ/fv3Y/z48SgqKsLw4cORmJiI3377DU899RTCw8Ob/R1KUqkU//rXv/DHH3/g008/xU8//YQ33njDsD0lJQXDhg3Dgw8+iGPHjuHIkSMIDw+HTqcDAMTGxmL58uVYsGABzp07h61bt8LDw8OsOhQXF+Odd97Bv//9b/zxxx9wd3dHYWEhIiMjceTIERw/fhxdu3bF8OHDDYFEr9fj6aefxtGjR7FlyxacO3cOy5cvh0wmg62tLV588UVs2rTJ6DybNm3C6NGjYW9v36yflSXh1yQQEYldeTHwdht919A/rwNy20aLOTs74+mnn8bWrVsxbNgwAMBXX30FV1dXPP7445BKpQgMDDSUX7p0KXbv3o29e/dixowZZldr9uzZhsd+fn74v//7P0yfPh1r164FALz77rvo27ev4TkA9OjRAwBQWFiIlStXYvXq1YiMjAQA+Pv7Y+DAgWbVoby8HGvXrjVq19ChQ43KfPzxx3BycsLhw4fx7LPP4uDBgzh58iRSU1PxwAMPAAA6d+5sKD9lyhQMGDAAmZmZ8PLyQk5ODr777rs76tWyJOypISKie8L48eOxc+dOaLVaAEBCQgJefPFFSKVSFBUVYe7cuejevTucnJxgZ2eH1NTUZvfUHDx4EMOGDYOPjw/s7e0xYcIE3LhxA8XFxQBu99SYkpqaCq1WW+/2ppLL5ejdu7fRuuzsbEydOhVdu3aFo6MjHBwcUFRUZGhnSkoK2rdvbwg0tfXv3x89evTAp59+CgDYsmULOnbsiEGDBt1RXS0Fe2qIiMTO2qayx6Stzt1E4eHhEAQB+/btQ79+/fCf//wHH374IQBg7ty5OHDgAN5//3106dIFKpUKo0ePRllZmdlVunz5Mp599lm89tprWLZsGVxcXHDkyBG88sorKCsrg42NDVQqVb37N7QNqBzaAmD0TdXl5eUmjyORSIzWRUZG4saNG1i5ciU6duwIhUKBkJAQQzsbOzdQ2VuzZs0axMTEYNOmTZg8eXKd84gVe2qIiMROIqkcAmqLxYwPU6VSiVGjRiEhIQFffPEFAgIC8PDDDwOonLQ7adIkjBw5Er169YKnp6dh0q25kpKSoNfr8cEHH+CRRx7BAw88gOvXjUNf7969kZiYaHL/rl27QqVS1bvdzc0NAJCZmWlYl5KS0qS6HT16FH/7298wfPhw9OjRAwqFAnl5eUb1unr1Kv73v//Ve4yXXnoJV65cwb/+9S+cO3fOMER2P2CoISKie8b48eOxb98+bNy4EePHjzes79q1K3bt2oWUlBT8/vvvGDduXLMvge7SpQvKy8uxatUqXLx4EZ9//jnWr19vVCY2NhanTp3C66+/jtOnT+P8+fNYt24d8vLyoFQq8eabb+KNN97AZ599hvT0dBw/fhwbNmwwHN/X1xeLFy/Gn3/+iX379uGDDz5oUt26du2Kzz//HKmpqThx4gTGjx9v1DszePBgDBo0CBEREThw4AAuXbqE77//Hvv37zeUcXZ2xqhRo/CPf/wDTz75JNq3b9+sn5MlYqghIqJ7xtChQ+Hi4oK0tDSMGzfOsD4+Ph7Ozs4YMGAAwsPDERYWZujFMVdgYCDi4+PxzjvvoGfPnkhISEBcXJxRmQceeAA//vgjfv/9d/Tv3x8hISH4+uuvYWVVOWtjwYIF+Pvf/46FCxeie/fuGDNmDHJycgAA1tbW+OKLL3D+/Hn07t0b77zzDv7v//6vSXXbsGEDbt26hYcffhgTJkzA3/72N7i7uxuV2blzJ/r164exY8fiwQcfxBtvvGG4Kqta9VDayy+/3KyfkaWSCEITr7ezcGq1Go6OjigoKICDg0NbV4eIqFWUlpbi0qVL6NSpE5RKZVtXh9rI559/jjlz5uD69euQy+VtXZ1GNfR7a87nNycKExERiURxcTEyMzOxfPlyvPrqqxYRaFoSh5+IiEhUEhISYGdnZ3KpvteMWL377rvo1q0bPD09ERsb29bVues4/EREJCIcfqq8OV52drbJbdbW1ujYseNdrhE1hsNPREREJtjb298XXwlAdXH4iYhIhO6TTngSiZb6fWWoISISEWtrawAw3O6fyBJU/75W//42F4efiIhERCaTwcnJyXDPFBsbm/vmFvlkeQRBQHFxMXJycuDk5ASZTHZHx2OoISISGU9PTwAwBBuie52Tk5Ph9/ZOMNQQEYmMRCKBl5cX3N3dTX6RItG9xNra+o57aKox1BARiZRMJmuxDwsiS8CJwkRERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCs0KNWvWrIGfnx+USiWCg4Nx8uTJBsuvWLECAQEBUKlU8PX1xZw5c1BaWmrYvnjxYkgkEqOlW7duRscoLS1FVFQU2rVrBzs7O0RERNT7hWVERER0/zE71Gzfvh3R0dFYtGgRkpOTERgYiLCwsHpv8rR161bExMRg0aJFSE1NxYYNG7B9+3b885//NCrXo0cPZGZmGpYjR44YbZ8zZw6++eYb7NixA4cPH8b169cxatQoc6tPREREImX2fWri4+MxdepUTJ48GQCwfv167Nu3Dxs3bkRMTEyd8r/++iseffRRjBs3DgDg5+eHsWPH4sSJE8YVsbKq926CBQUF2LBhA7Zu3YqhQ4cCADZt2oTu3bvj+PHjeOSRR8xtBhEREYmMWT01ZWVlSEpKQmho6O0DSKUIDQ3FsWPHTO4zYMAAJCUlGYaoLl68iO+++w7Dhw83Kvfnn3/C29sbnTt3xvjx45GRkWHYlpSUhPLycqPzduvWDR06dKj3vFqtFmq12mghIiIi8TKrpyYvLw86nQ4eHh5G6z08PHD+/HmT+4wbNw55eXkYOHAgBEFARUUFpk+fbjT8FBwcjM2bNyMgIACZmZlYsmQJHnvsMZw9exb29vbIysqCXC6Hk5NTnfNmZWWZPG9cXByWLFliTvOIiIjIgrX61U+HDh3C22+/jbVr1yI5ORm7du3Cvn37sHTpUkOZp59+Gs8//zx69+6NsLAwfPfdd8jPz8eXX37Z7PPGxsaioKDAsPz1118t0RwiIiK6R5nVU+Pq6gqZTFbnqqPs7Ox658MsWLAAEyZMwJQpUwAAvXr1gkajwbRp0zBv3jxIpXVzlZOTEx544AFcuHABQOU3zpaVlSE/P9+ot6ah8yoUCigUCnOaR0RERBbMrJ4auVyOoKAgJCYmGtbp9XokJiYiJCTE5D7FxcV1gkv1F6wJgmByn6KiIqSnp8PLywsAEBQUBGtra6PzpqWlISMjo97zEhER0f3F7KufoqOjERkZib59+6J///5YsWIFNBqN4WqoiRMnwsfHB3FxcQCA8PBwxMfH46GHHkJwcDAuXLiABQsWIDw83BBu5s6di/DwcHTs2BHXr1/HokWLIJPJMHbsWACAo6MjXnnlFURHR8PFxQUODg6YOXMmQkJCeOUTERERAWhGqBkzZgxyc3OxcOFCZGVloU+fPti/f79h8nBGRoZRz8z8+fMhkUgwf/58XLt2DW5ubggPD8eyZcsMZa5evYqxY8fixo0bcHNzw8CBA3H8+HG4ubkZynz44YeQSqWIiIiAVqtFWFgY1q5deydtJyIiIhGRCPWNAYmMWq2Go6MjCgoK4ODg0NbVISIioiYw5/Ob3/1EREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREosBQQ0RERKLAUENERESiwFBDREREotCsULNmzRr4+flBqVQiODgYJ0+ebLD8ihUrEBAQAJVKBV9fX8yZMwelpaWG7XFxcejXrx/s7e3h7u6OESNGIC0tzegYQ4YMgUQiMVqmT5/enOoTERGRCJkdarZv347o6GgsWrQIycnJCAwMRFhYGHJyckyW37p1K2JiYrBo0SKkpqZiw4YN2L59O/75z38ayhw+fBhRUVE4fvw4Dhw4gPLycjz55JPQaDRGx5o6dSoyMzMNy7vvvmtu9YmIiEikrMzdIT4+HlOnTsXkyZMBAOvXr8e+ffuwceNGxMTE1Cn/66+/4tFHH8W4ceMAAH5+fhg7dixOnDhhKLN//36jfTZv3gx3d3ckJSVh0KBBhvU2Njbw9PQ0t8pERER0HzCrp6asrAxJSUkIDQ29fQCpFKGhoTh27JjJfQYMGICkpCTDENXFixfx3XffYfjw4fWep6CgAADg4uJitD4hIQGurq7o2bMnYmNjUVxcXO8xtFot1Gq10UJERETiZVZPTV5eHnQ6HTw8PIzWe3h44Pz58yb3GTduHPLy8jBw4EAIgoCKigpMnz7daPipJr1ej9mzZ+PRRx9Fz549jY7TsWNHeHt74/Tp03jzzTeRlpaGXbt2mTxOXFwclixZYk7ziIiIyIKZPfxkrkOHDuHtt9/G2rVrERwcjAsXLmDWrFlYunQpFixYUKd8VFQUzp49iyNHjhitnzZtmuFxr1694OXlhWHDhiE9PR3+/v51jhMbG4vo6GjDc7VaDV9f3xZsGREREd1LzAo1rq6ukMlkyM7ONlqfnZ1d71yXBQsWYMKECZgyZQqAykCi0Wgwbdo0zJs3D1Lp7RGwGTNm4Ntvv8Uvv/yC9u3bN1iX4OBgAMCFCxdMhhqFQgGFQmFO84iIiMiCmTWnRi6XIygoCImJiYZ1er0eiYmJCAkJMblPcXGxUXABAJlMBgAQBMHw74wZM7B792789NNP6NSpU6N1SUlJAQB4eXmZ0wQiIiISKbOHn6KjoxEZGYm+ffuif//+WLFiBTQajeFqqIkTJ8LHxwdxcXEAgPDwcMTHx+Ohhx4yDD8tWLAA4eHhhnATFRWFrVu34uuvv4a9vT2ysrIAAI6OjlCpVEhPT8fWrVsxfPhwtGvXDqdPn8acOXMwaNAg9O7du6V+FkRERGTBzA41Y8aMQW5uLhYuXIisrCz06dMH+/fvN0wezsjIMOqZmT9/PiQSCebPn49r167Bzc0N4eHhWLZsmaHMunXrAFTeYK+mTZs2YdKkSZDL5Th48KAhQPn6+iIiIgLz589vTpuJiIhIhCRC9RiQyKnVajg6OqKgoAAODg5tXR0iIiJqAnM+v/ndT0RERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCgw1REREJAoMNURERCQKDDVEREQkCs0KNWvWrIGfnx+USiWCg4Nx8uTJBsuvWLECAQEBUKlU8PX1xZw5c1BaWmrWMUtLSxEVFYV27drBzs4OERERyM7Obk71iYiISITMDjXbt29HdHQ0Fi1ahOTkZAQGBiIsLAw5OTkmy2/duhUxMTFYtGgRUlNTsWHDBmzfvh3//Oc/zTrmnDlz8M0332DHjh04fPgwrl+/jlGjRjWjyURERCRGEkEQBHN2CA4ORr9+/bB69WoAgF6vh6+vL2bOnImYmJg65WfMmIHU1FQkJiYa1v3973/HiRMncOTIkSYds6CgAG5ubti6dStGjx4NADh//jy6d++OY8eO4ZFHHmm03mq1Go6OjigoKICDg4M5TSYiIqI2Ys7nt1k9NWVlZUhKSkJoaOjtA0ilCA0NxbFjx0zuM2DAACQlJRmGky5evIjvvvsOw4cPb/Ixk5KSUF5eblSmW7du6NChQ73n1Wq1UKvVRgsRERGJl5U5hfPy8qDT6eDh4WG03sPDA+fPnze5z7hx45CXl4eBAwdCEARUVFRg+vTphuGnphwzKysLcrkcTk5OdcpkZWWZPG9cXByWLFliTvOIiIjIgrX61U+HDh3C22+/jbVr1yI5ORm7du3Cvn37sHTp0lY9b2xsLAoKCgzLX3/91arnIyIiorZlVk+Nq6srZDJZnauOsrOz4enpaXKfBQsWYMKECZgyZQoAoFevXtBoNJg2bRrmzZvXpGN6enqirKwM+fn5Rr01DZ1XoVBAoVCY0zwiIiKyYGb11MjlcgQFBRlN+tXr9UhMTERISIjJfYqLiyGVGp9GJpMBAARBaNIxg4KCYG1tbVQmLS0NGRkZ9Z6XiIiI7i9m9dQAQHR0NCIjI9G3b1/0798fK1asgEajweTJkwEAEydOhI+PD+Li4gAA4eHhiI+Px0MPPYTg4GBcuHABCxYsQHh4uCHcNHZMR0dHvPLKK4iOjoaLiwscHBwwc+ZMhISENOnKJyIiIhI/s0PNmDFjkJubi4ULFyIrKwt9+vTB/v37DRN9MzIyjHpm5s+fD4lEgvnz5+PatWtwc3NDeHg4li1b1uRjAsCHH34IqVSKiIgIaLVahIWFYe3atXfSdiIiIhIRs+9TY6l4nxoiIiLL02r3qSEiIiK6VzHUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNQQERGRKDDUEBERkSgw1BAREZEoMNTcqQotoMkDyooBQWjr2hAREd23rNq6AhYv4zjw2f+reiIBrG0AaxUgtwGsbav+tQHktlXbbOquqy5bZ78a+1vbAFbyNm0qERHRvYyh5k5VlNZ4IgDlmsqluBXOJbWqFZRqBJ7aAUleFa5MBSt5VYCqHbak7LgjIiLLxVBzpx4IAxbeBMqLK4egyotrPNbcXlemAcpLTKxrwn6CrvJc+gpAW1C5tAYrVa2gZKq3SdVAiGpgP5kckEhap95EdO8TBKCsCNDkApobVf/mAsV5lUP4mtzKMionQOlYY6nxXOVU+VzhAMj48UV18beiJUhlgMK+cmlpggDoypsQhjTGoai8pJ5gVVx332oVJZULbrR8OySyBobe6glD9Qar6iG+Gj1PUlnL15mIGlZecjuQaPKqAkquieBS9dioZ/sOye2MA48h9JgIQ7W3ye3ZMy1SDDX3Oomkci6NlRxQObf88fX6yiBTbxjSmOhJqhmiqoNSSd1gVVYM6MsrzyPogLLCyqU1yBQNzE1qIAzVDE613wCtlOxdovuLrrxWOKnRi1InuORV9ryYy9oGsHUFbN0Am6p/bV0rF4kUKMkHSguqlpqPCyq3lWsqj1NWVLmor5pfB4m0sren3jDkVM+2qu3WKr433KMYau53Umnlh7vcFoBbyx9fV968MNRob1PVfqi64kynBUq0QMmtlqu7TN74//iMtjsZb5NZt1xdiJpDr6v8m6gZSqpDilFwqfq3NN/8c8jkVeGkOqDUCCmmgovc9s7apCsHStVVgSffOPA0FIaq1+vKAEF/e//mkFo38l7QSFDiRR+thqGGWpfMGpBV/SG3NEGo7M42KwyZCljFlf/j06pvvwkK+so3v+oPg+awtmn8f3z1bVM4snuc6hKEyg/imkM7NYd3ageXkpuVv8vmkEhrBJF2xkHFxkRwUTjc3V4LmXVVvdo1b//y0nrCUO1/awel6vcGXWUPdHFVr1VzWKmaPlRWe5vSkcPtDWCoIcslkVRNXFYBaOYbnCmCAGgLm/C/PhPbS/JvD7FVh6jC682ohKSR7vFG3gDltuwetwSCUBmuTc5JyUPd+Sp5t4d0zaFyqdGT4lq398QQVNwqf4fEHKitlYC1J2Dvaf6+1ZOdm/I+YGqbVl15nIoSoLAEKMxsXhtqvjeY+x8nuZ2o3xsYaohqk0gApUPlAl/z99dV1Oj1yTcvEJUWVE3WFm5f6daci90ksia+yTmZ3matbMZJCUBlT0Dtq3pq/1tzW3Mmzyocag3v1AwptYKLjQuHQluKRHL7ohDH9ubvr9fV7flp8D2i1rbqCzu06sql4K9mtEFW9f7m1EgYqme7tcr8c95FDDVELU1mVflBYuPSvP0rtDXmDNR4Y2vqnAF9RWUXecnNyqU5rJT1h6EGe46qHovpcltdea3hnXouR67e1pzJ8FaquqGkZg9K7eBipWj5dlLrk8pa6L2h+j0g34w5Rfk13htuNX/+oUzR8HuBWzcgcEzzjt0CRPTOQyQSVgrAzq1yMZcgVM4parQrvIHtqJqrVFQKFGU3rw1yO/PnChi6x1v5clvD5Nn6ruqpFVyaM5lUal1POGlnPNRTPWflTifP0v2hRd4bGhoqy2/8vUGnBTQ5lYsp/sMYaoiohUgklZeoy20AB2/z99frK3sazBkuq7mt+hJfw+W215rTiAa6x2v/W2Ob1KpplyMX32jm5Nl2xr0p9c1LsWlXWR8Rz1sgC2T03uBl/v5NfW9o16Wla26WZoWaNWvW4L333kNWVhYCAwOxatUq9O/f32TZIUOG4PDhw3XWDx8+HPv27QMASOr543/33Xfxj3/8AwDg5+eHK1euGG2Pi4tDTExMc5pARKZIpbfDQnMY5hPlN28iZUUpAKHG/wxbkcrZxNBOPZcjq5zFPXmWqDF3+t5wl5gdarZv347o6GisX78ewcHBWLFiBcLCwpCWlgZ3d/c65Xft2oWysjLD8xs3biAwMBDPP/+8YV1mpvEM8O+//x6vvPIKIiIijNa/9dZbmDp1quG5vX0r3MGXiJrvTucTlZdWhiJzusSrt+nKaw3v1HhcO7hw8iyRKJkdauLj4zF16lRMnjwZALB+/Xrs27cPGzduNNlr4uJi/Oa2bds22NjYGIUaT0/jS+u+/vprPP744+jcubPRent7+zpliUhErJWVi13d/yARETXGrP7UsrIyJCUlITQ09PYBpFKEhobi2LFjTTrGhg0b8OKLL8LW1vTEuOzsbOzbtw+vvPJKnW3Lly9Hu3bt8NBDD+G9995DRUVFvefRarVQq9VGCxEREYmXWT01eXl50Ol08PDwMFrv4eGB8+fPN7r/yZMncfbsWWzYsKHeMp9++ins7e0xatQoo/V/+9vf8PDDD8PFxQW//vorYmNjkZmZifj4eJPHiYuLw5IlS5rQKiIiIhKDu3r104YNG9CrV696JxUDwMaNGzF+/HgolcY3/4qOjjY87t27N+RyOV599VXExcVBoah7z4bY2FijfdRqNXx9m3EjNSIiIrIIZg0/ubq6QiaTITvb+N4V2dnZjc510Wg02LZtm8lhpWr/+c9/kJaWhilTpjRal+DgYFRUVODy5csmtysUCjg4OBgtREREJF5mhRq5XI6goCAkJiYa1un1eiQmJiIkJKTBfXfs2AGtVouXXnqp3jIbNmxAUFAQAgMDG61LSkoKpFKpySuuiIiI6P5j9vBTdHQ0IiMj0bdvX/Tv3x8rVqyARqMxXA01ceJE+Pj4IC4uzmi/DRs2YMSIEWjXzvQXD6rVauzYsQMffPBBnW3Hjh3DiRMn8Pjjj8Pe3h7Hjh3DnDlz8NJLL8HZ2dncJhAREZEImR1qxowZg9zcXCxcuBBZWVno06cP9u/fb5g8nJGRAWmtm1SlpaXhyJEj+PHHH+s97rZt2yAIAsaOHVtnm0KhwLZt27B48WJotVp06tQJc+bMMZozQ0RERPc3iSAIQltX4m5Qq9VwdHREQUEB59cQERFZCHM+v3nfbyIiIhIFhhoiIiISBYYaIiIiEgWGGiIiIhIFhhoiIiISBYYaIiIiEoW7+t1Pban6ynV+WzcREZHlqP7cbsodaO6bUFNYWAgA/FJLIiIiC1RYWAhHR8cGy9w3N9/T6/W4fv067O3tIZFIWvTY1d8A/tdff4nyxn5sn+UTexvF3j5A/G1k+yxfa7VREAQUFhbC29u7zjcW1Hbf9NRIpVK0b9++Vc8h9m8DZ/ssn9jbKPb2AeJvI9tn+VqjjY310FTjRGEiIiISBYYaIiIiEgWGmhagUCiwaNEiKBSKtq5Kq2D7LJ/Y2yj29gHibyPbZ/nuhTbeNxOFiYiISNzYU0NERESiwFBDREREosBQQ0RERKLAUENERESiwFDTRGvWrIGfnx+USiWCg4Nx8uTJBsvv2LED3bp1g1KpRK9evfDdd9/dpZo2jznt27x5MyQSidGiVCrvYm3N88svvyA8PBze3t6QSCTYs2dPo/scOnQIDz/8MBQKBbp06YLNmze3ej2by9z2HTp0qM7rJ5FIkJWVdXcqbKa4uDj069cP9vb2cHd3x4gRI5CWltbofpb0N9icNlrS3+G6devQu3dvw03ZQkJC8P333ze4jyW9fua2z5JeO1OWL18OiUSC2bNnN1iuLV5Dhpom2L59O6Kjo7Fo0SIkJycjMDAQYWFhyMnJMVn+119/xdixY/HKK6/gt99+w4gRIzBixAicPXv2Lte8acxtH1B5x8jMzEzDcuXKlbtYY/NoNBoEBgZizZo1TSp/6dIlPPPMM3j88ceRkpKC2bNnY8qUKfjhhx9auabNY277qqWlpRm9hu7u7q1Uwztz+PBhREVF4fjx4zhw4ADKy8vx5JNPQqPR1LuPpf0NNqeNgOX8HbZv3x7Lly9HUlIS/vvf/2Lo0KF47rnn8Mcff5gsb2mvn7ntAyzntavt1KlT+Oijj9C7d+8Gy7XZayhQo/r37y9ERUUZnut0OsHb21uIi4szWf6FF14QnnnmGaN1wcHBwquvvtqq9Wwuc9u3adMmwdHR8S7VrmUBEHbv3t1gmTfeeEPo0aOH0boxY8YIYWFhrVizltGU9v38888CAOHWrVt3pU4tLScnRwAgHD58uN4ylvY3WFtT2mjJf4eCIAjOzs7Cv//9b5PbLP31E4SG22epr11hYaHQtWtX4cCBA8LgwYOFWbNm1Vu2rV5D9tQ0oqysDElJSQgNDTWsk0qlCA0NxbFjx0zuc+zYMaPyABAWFlZv+bbUnPYBQFFRETp27AhfX99G/0diaSzp9bsTffr0gZeXF5544gkcPXq0ravTZAUFBQAAFxeXestY+mvYlDYClvl3qNPpsG3bNmg0GoSEhJgsY8mvX1PaB1jmaxcVFYVnnnmmzmtjSlu9hgw1jcjLy4NOp4OHh4fReg8Pj3rnIGRlZZlVvi01p30BAQHYuHEjvv76a2zZsgV6vR4DBgzA1atX70aVW119r59arUZJSUkb1arleHl5Yf369di5cyd27twJX19fDBkyBMnJyW1dtUbp9XrMnj0bjz76KHr27FlvOUv6G6ytqW20tL/DM2fOwM7ODgqFAtOnT8fu3bvx4IMPmixria+fOe2ztNcOALZt24bk5GTExcU1qXxbvYb3zbd0U8sJCQkx+h/IgAED0L17d3z00UdYunRpG9aMmiIgIAABAQGG5wMGDEB6ejo+/PBDfP75521Ys8ZFRUXh7NmzOHLkSFtXpdU0tY2W9ncYEBCAlJQUFBQU4KuvvkJkZCQOHz5c7we/pTGnfZb22v3111+YNWsWDhw4cM9PaGaoaYSrqytkMhmys7ON1mdnZ8PT09PkPp6enmaVb0vNaV9t1tbWeOihh3DhwoXWqOJdV9/r5+DgAJVK1Ua1al39+/e/54PCjBkz8O233+KXX35B+/btGyxrSX+DNZnTxtru9b9DuVyOLl26AACCgoJw6tQprFy5Eh999FGdspb4+pnTvtru9dcuKSkJOTk5ePjhhw3rdDodfvnlF6xevRparRYymcxon7Z6DTn81Ai5XI6goCAkJiYa1un1eiQmJtY7XhoSEmJUHgAOHDjQ4PhqW2lO+2rT6XQ4c+YMvLy8Wquad5UlvX4tJSUl5Z59/QRBwIwZM7B792789NNP6NSpU6P7WNpr2Jw21mZpf4d6vR5ardbkNkt7/UxpqH213euv3bBhw3DmzBmkpKQYlr59+2L8+PFISUmpE2iANnwNW3Uaskhs27ZNUCgUwubNm4Vz584J06ZNE5ycnISsrCxBEARhwoQJQkxMjKH80aNHBSsrK+H9998XUlNThUWLFgnW1tbCmTNn2qoJDTK3fUuWLBF++OEHIT09XUhKShJefPFFQalUCn/88UdbNaFBhYWFwm+//Sb89ttvAgAhPj5e+O2334QrV64IgiAIMTExwoQJEwzlL168KNjY2Aj/+Mc/hNTUVGHNmjWCTCYT9u/f31ZNaJC57fvwww+FPXv2CH/++adw5swZYdasWYJUKhUOHjzYVk1o0GuvvSY4OjoKhw4dEjIzMw1LcXGxoYyl/w02p42W9HcYExMjHD58WLh06ZJw+vRpISYmRpBIJMKPP/4oCILlv37mts+SXrv61L766V55DRlqmmjVqlVChw4dBLlcLvTv3184fvy4YdvgwYOFyMhIo/Jffvml8MADDwhyuVzo0aOHsG/fvrtcY/OY077Zs2cbynp4eAjDhw8XkpOT26DWTVN9CXPtpbpNkZGRwuDBg+vs06dPH0EulwudO3cWNm3adNfr3VTmtu+dd94R/P39BaVSKbi4uAhDhgwRfvrpp7apfBOYahsAo9fE0v8Gm9NGS/o7fPnll4WOHTsKcrlccHNzE4YNG2b4wBcEy3/9zG2fJb129akdau6V11AiCILQun1BRERERK2Pc2qIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiIiISBQYaoiIiEgUGGqIiIhIFBhqiOi+IpFIsGfPnrauBhG1AoYaIrprJk2aBIlEUmd56qmn2rpqRCQC/JZuIrqrnnrqKWzatMlonUKhaKPaEJGYsKeGiO4qhUIBT09Po8XZ2RlA5dDQunXr8PTTT0OlUqFz58746quvjPY/c+YMhg4dCpVKhXbt2mHatGkoKioyKrNx40b06NEDCoUCXl5emDFjhtH2vLw8jBw5EjY2NujatSv27t1r2Hbr1i2MHz8ebm5uUKlU6Nq1a50QRkT3JoYaIrqnLFiwABEREfj9998xfvx4vPjii0hNTQUAaDQahIWFwdnZGadOncKOHTtw8OBBo9Cybt06REVFYdq0aThz5gz27t2LLl26GJ1jyZIleOGFF3D69GkMHz4c48ePx82bNw3nP3fuHL7//nukpqZi3bp1cHV1vXs/ACJqvlb/ykwioiqRkZGCTCYTbG1tjZZly5YJglD5bdXTp0832ic4OFh47bXXBEEQhI8//lhwdnYWioqKDNv37dsnSKVSISsrSxAEQfD29hbmzZtXbx0ACPPnzzc8LyoqEgAI33//vSAIghAeHi5Mnjy5ZRpMRHcV59QQ0V31+OOPY926dUbrXFxcDI9DQkKMtoWEhCAlJQUAkJqaisDAQNja2hq2P/roo9Dr9UhLS4NEIsH169cxbNiwBuvQu3dvw2NbW1s4ODggJycHAPDaa68hIiICycnJePLJJzFixAgMGDCgWW0loruLoYaI7ipbW9s6w0EtRaVSNamctbW10XOJRAK9Xg8AePrpp3HlyhV89913OHDgAIYNG4aoqCi8//77LV5fImpZnFNDRPeU48eP13nevXt3AED37t3x+++/Q6PRGLYfPXoUUqkUAQEBsLe3h5+fHxITE++oDm5uboiMjMSWLVuwYsUKfPzxx3d0PCK6O9hTQ0R3lVarRVZWltE6Kysrw2TcHTt2oG/fvhg4cCASEhJw8uRJbNiwAQAwfvx4LFq0CJGRkVi8eDFyc3Mxc+ZMTJgwAR4eHgCAxYsXY/r06XB3d8fTTz+NwsJCHD16FDNnzmxS/RYuXIigoCD06NEDWq0W3377rSFUEdG9jaGGiO6q/fv3w8vLy2hdQEAAzp8/D6DyyqRt27bh9ddfh5eXF7744gs8+OCDAAAbGxv88MMPmDVrFvr16wcbGxtEREQgPj7ecKzIyEiUlpbiww8/xNy5c+Hq6orRo0c3uX5yuRyxsbG4fPkyVCoVHnvsMWzbtq0FWk5ErU0iCILQ1pUgIgIq57bs3r0bI0aMaOuqEJEF4pwaIiIiEgWGGiIiIhIFzqkhonsGR8OJ6E6wp4aIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiESBoYaIiIhEgaGGiIiIRIGhhoiIiETh/wNV9ZtLqiz6YwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# check the results\n",
        "\n",
        "model_1.evaluate(val_sentences, val_labels)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cZTaIK91nvoD",
        "outputId": "2d9996a4-045a-4f39-fe94-9fc4180c7fb5"
      },
      "execution_count": 67,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "24/24 [==============================] - 0s 3ms/step - loss: 0.6110 - accuracy: 0.7782\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[0.6110004782676697, 0.778215229511261]"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_1_pred_probs = model_1.predict(val_sentences)\n",
        "model_1_pred_probs[:20]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xFgqctAJn0yh",
        "outputId": "83314aed-13f9-4c4a-be14-64de03f8b82a"
      },
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "24/24 [==============================] - 1s 7ms/step\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[4.7922114e-01],\n",
              "       [7.4940151e-01],\n",
              "       [9.9941742e-01],\n",
              "       [7.8580379e-02],\n",
              "       [9.5861815e-03],\n",
              "       [9.8293936e-01],\n",
              "       [8.8667774e-01],\n",
              "       [9.9972516e-01],\n",
              "       [9.9853313e-01],\n",
              "       [5.1044136e-01],\n",
              "       [9.9857245e-03],\n",
              "       [8.4685808e-01],\n",
              "       [8.7385662e-03],\n",
              "       [2.1605450e-01],\n",
              "       [4.2058120e-04],\n",
              "       [1.0342668e-01],\n",
              "       [7.7702096e-03],\n",
              "       [7.2711851e-03],\n",
              "       [1.4526099e-01],\n",
              "       [8.2238358e-01]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_1_preds = tf.squeeze(tf.round(model_1_pred_probs))\n",
        "\n",
        "model_1_preds[:20]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JzXJosxHn4sc",
        "outputId": "59119676-5b5e-4beb-b387-73054ec699a5"
      },
      "execution_count": 66,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(20,), dtype=float32, numpy=\n",
              "array([0., 1., 1., 0., 0., 1., 1., 1., 1., 1., 0., 1., 0., 0., 0., 0., 0.,\n",
              "       0., 0., 1.], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert model prerdiciton to label format\n",
        "for pred in model_1_preds[:20]:\n",
        "  if pred > 0.5:\n",
        "    print(\"Positive\")\n",
        "  else:\n",
        "    print(\"Negative\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yZ2jFAnEq4Dq",
        "outputId": "1d50472f-ab05-4a46-cf3d-0bc81f3815b7"
      },
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Negative\n",
            "Positive\n",
            "Positive\n",
            "Negative\n",
            "Negative\n",
            "Positive\n",
            "Positive\n",
            "Positive\n",
            "Positive\n",
            "Positive\n",
            "Negative\n",
            "Positive\n",
            "Negative\n",
            "Negative\n",
            "Negative\n",
            "Negative\n",
            "Negative\n",
            "Negative\n",
            "Negative\n",
            "Positive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Visualising learned embedding"
      ],
      "metadata": {
        "id": "-sxnnRj_sfpb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get teh vocanulary from the text vectorization layer\n",
        "\n",
        "words_in_vocab = text_vectorizer.get_vocabulary()\n",
        "len(words_in_vocab), words_in_vocab[:10]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "erdziZSrsfnr",
        "outputId": "0e2d7312-bb4f-479e-f450-53b99982e57c"
      },
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(10000, ['', '[UNK]', 'the', 'a', 'in', 'to', 'of', 'and', 'i', 'is'])"
            ]
          },
          "metadata": {},
          "execution_count": 78
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "max_vocab_length"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9uY2_cx1sflI",
        "outputId": "a31f24d6-f3cd-420c-d481-010ae995a789"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "10000"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 1 summary\n",
        "\n",
        "model_1.summary()\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "akg4UFr8sfjJ",
        "outputId": "c89f7e9a-a7ea-4bb3-f8e7-8bd5ef71efef"
      },
      "execution_count": 80,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"model_1_dense\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " input_4 (InputLayer)        [(None, 1)]               0         \n",
            "                                                                 \n",
            " text_vectorization_1 (Text  (None, 15)                0         \n",
            " Vectorization)                                                  \n",
            "                                                                 \n",
            " embedding (Embedding)       (None, 15, 128)           1280000   \n",
            "                                                                 \n",
            " global_average_pooling1d (  (None, 128)               0         \n",
            " GlobalAveragePooling1D)                                         \n",
            "                                                                 \n",
            " dense_3 (Dense)             (None, 1)                 129       \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 1280129 (4.88 MB)\n",
            "Trainable params: 1280129 (4.88 MB)\n",
            "Non-trainable params: 0 (0.00 Byte)\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the weight matrix of embedding layer\n",
        "\n",
        "# (the se are teh numeriacal representation of each token in our training data)\n",
        "embed_weights = model_1.get_layer(\"embedding\").get_weights()[0]\n",
        "\n",
        "print(embed_weights.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LtnAyYTxsfft",
        "outputId": "d2e115d8-73f9-40ff-fc28-196966985a07"
      },
      "execution_count": 83,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(10000, 128)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we've got the embedding matrix our model has learned to represent our tokens, let's see how we can visualize it.\n",
        "\n",
        "To do so, TensorFlow has an incredible tool called projector: https://projector.tensorflow.org/\n",
        "\n",
        "And Tensorflow also has an incredible guide on word embeddings themselves: https://www.tensorflow.org/tutorials/text/word_embeddings"
      ],
      "metadata": {
        "id": "pelRF5aiuXyp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "embed_weights"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KSE8gwSVsfch",
        "outputId": "7026f852-6a5e-4a3b-82f0-3c0d08d2e971"
      },
      "execution_count": 85,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[-3.7644275e-02, -3.4612384e-02, -7.7179917e-03, ...,\n",
              "        -5.9109461e-03,  2.0801665e-02, -4.4287462e-03],\n",
              "       [-4.7637038e-02,  1.6154282e-02, -1.4106596e-03, ...,\n",
              "         3.7145514e-02, -3.9344616e-03, -3.7972338e-02],\n",
              "       [-3.4115650e-02, -3.6986899e-02, -7.3975879e-03, ...,\n",
              "        -1.5464366e-03,  3.3235960e-02,  9.7131664e-03],\n",
              "       ...,\n",
              "       [ 4.0986206e-02, -1.0221079e-04,  4.2291846e-02, ...,\n",
              "        -5.4398403e-03, -4.1737009e-02,  1.7700303e-02],\n",
              "       [ 4.3766532e-02,  7.0328275e-03,  2.0170307e-02, ...,\n",
              "        -3.1688925e-02, -1.5768036e-02, -2.3616396e-02],\n",
              "       [ 1.6244060e-01,  8.4819131e-02,  1.4552449e-01, ...,\n",
              "        -7.3525071e-02, -1.3301471e-01,  1.2717751e-01]], dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import io\n",
        "\n",
        "out_v = io.open('vectors.tsv', 'w', encoding='utf-8')\n",
        "out_m = io.open('metadata.tsv', 'w', encoding='utf-8')\n",
        "\n",
        "for index, word in enumerate(words_in_vocab):\n",
        "  if index == 0:\n",
        "    continue  # skip\n",
        "  vec = embed_weights[index]\n",
        "  out_v.write('\\t'.join([str(x) for x in vec]) + \"\\temp\")\n",
        "  out_m.write(word + \"\\temp\")\n",
        "out_v.close()\n",
        "out_m.close()\n"
      ],
      "metadata": {
        "id": "5ZRG0uLisfaA"
      },
      "execution_count": 86,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "57ZZngG8sfXT"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "6w1Ag4ResfUq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "jmKbqx0dsfSO"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}