{
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "Vectorizado con métodos clásicos\n",
        "================================\n",
        "\n",
        "## Introducción"
      ],
      "metadata": {
        "id": "64VtYMpfSyf-"
      },
      "id": "64VtYMpfSyf-"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Un modelos de aprendizaje automático es, a groso modo, una función parametrizada `f(x)` que toma como entrada\n",
        "un vector `x`, `n-dimensional`, y produce un vector de salida `m-dimensional`. Tal función puede ser simple (para un modelo lineal por ejemplo) o más compleja (como una red neuronal).\n",
        "\n",
        "Cuando trabajamos con lenguaje natural, la mayoría de nuestros datos de entrada representarán características discretas y categóricas, ya sean palabras, letras o incluso utterancias (partes del discurso). La pregunta que nos haremos entonces es ¿Cómo codificamos esos datos categóricos de una manera que sea práctica para ser utilizada por un modelo de aprendizaje automático?\n",
        "\n",
        "Discutiremos las opciones disponibles.\n",
        "\n",
        " - [One-hot encoding](#One-hot-encoding)\n",
        " - [Index-based encoding](#Index-based-encoding)\n",
        " - [Basados en frecuencias](#Basados-en-frecuencias)\n",
        "\n",
        " Utilizaremos el siguiente corpus para nuestros ejemplos:\n"
      ],
      "metadata": {
        "id": "dLRE8wzSSygF"
      },
      "id": "dLRE8wzSSygF"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "source": [
        "corpus = [\"El hielo es agua en estado sólido\",\n",
        "          \"El hielo es uno de los cuatro estados naturales del agua\",\n",
        "          \"El agua pura se congela a 0 grados\",\n",
        "          \"El hielo es el nombre común del agua en estado sólido\"]"
      ],
      "outputs": [],
      "metadata": {
        "id": "FFWRRU00SygG"
      },
      "id": "FFWRRU00SygG"
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Para ejecutar este notebook"
      ],
      "metadata": {
        "id": "zyuxR3UTSygJ"
      },
      "id": "zyuxR3UTSygJ"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Para ejecutar este notebook, instale las siguientes librerias:"
      ],
      "metadata": {
        "id": "U0BdtmHGSygK"
      },
      "id": "U0BdtmHGSygK"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "source": [
        "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/docs/nlp/vectorization/vectorization.txt --quiet --no-clobber\n",
        "!pip install -r vectorization.txt"
      ],
      "outputs": [],
      "metadata": {
        "id": "Z_KMq7GnSygL"
      },
      "id": "Z_KMq7GnSygL"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Vocabulario"
      ],
      "metadata": {
        "id": "ypFyXQwbMNEO"
      },
      "id": "ypFyXQwbMNEO"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Primero crearemos nuestro vocabulario:"
      ],
      "metadata": {
        "id": "HPE0QyMbEpd0"
      },
      "id": "HPE0QyMbEpd0"
    },
    {
      "cell_type": "code",
      "source": [
        "vocab = { i:w for w,i in enumerate(set(' '.join(corpus).split())) }"
      ],
      "metadata": {
        "id": "JxpYrYfmEp3q"
      },
      "id": "JxpYrYfmEp3q",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "`vocab` es un diccionario que tiene las palabras como \"claves\" y su posición como \"valor\"."
      ],
      "metadata": {
        "id": "u8rI3u3LMPFP"
      },
      "id": "u8rI3u3LMPFP"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## One-hot encoding"
      ],
      "metadata": {
        "id": "uuc_1Ex6SygN"
      },
      "id": "uuc_1Ex6SygN"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Dado que el texto representará características discretas y categóricas, hace sentido pensar en utilizar métodos clásicos para este tipo de dato. **One-hot encoding** es una técnica sensilla que cosiste en representar las palabras con vectores de longitud igual al tamaño del vocabulario donde todas las posiciones son zero salvo la posición que corresponde al indice de la palabra en cuestión. Eso significa que las dimensiones de los vectores de entrada dependerá del tamaño del vocabulario y no del tamaño del cuerpo de texto. Este tipo de representación tiene la propiedad de que todas las palabras son igualmente relevantes para el modelo."
      ],
      "metadata": {
        "id": "2zskVUwgSygN"
      },
      "id": "2zskVUwgSygN"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Si bien este método es sencillo de implementar, genera representaciones dispersas, que genera dificultades a la hora de procesarlos. Sin embargo esto no significa que esta forma de codificación no sea práctica. Veremos más adelante que esta técnica se puede utilizar para aprender representaciones más complejas como [embeddings](https://m72109.readthedocs.io/es/latest/nlp/vectorization/embeddings.html). En estas configuraciones, la entrada de la red neuronal está especificada como una colección de vectores one-hot."
      ],
      "metadata": {
        "id": "11FT9sOoS_V9"
      },
      "id": "11FT9sOoS_V9"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Inicializamos nuestros vectores:"
      ],
      "metadata": {
        "id": "iiEVvxaFKPyF"
      },
      "id": "iiEVvxaFKPyF"
    },
    {
      "cell_type": "code",
      "source": [
        "vectors = np.zeros((len(corpus), 11, len(vocab)))"
      ],
      "metadata": {
        "id": "CUUxT666KQJo"
      },
      "id": "CUUxT666KQJo",
      "execution_count": 87,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "vectors.shape"
      ],
      "metadata": {
        "id": "GVJmBxrAO9sR",
        "outputId": "1c38c380-b8fb-45fc-f72c-c99dfa47101a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "id": "GVJmBxrAO9sR",
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4, 11, 23)"
            ]
          },
          "metadata": {},
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "> ¿Que representa el 11?"
      ],
      "metadata": {
        "id": "gN2Bq_4PMv-y"
      },
      "id": "gN2Bq_4PMv-y"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Ponemos un 1 en la posicion de la palabra:"
      ],
      "metadata": {
        "id": "zVe4FMt8Keqy"
      },
      "id": "zVe4FMt8Keqy"
    },
    {
      "cell_type": "code",
      "source": [
        "for doc_idx, doc in enumerate(corpus):\n",
        "  for word_idx, word in enumerate(doc.split()):\n",
        "    vectors[doc_idx][word_idx][vocab[word]] = 1"
      ],
      "metadata": {
        "id": "QXwGwscUKYcp"
      },
      "id": "QXwGwscUKYcp",
      "execution_count": 94,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Index-based encoding"
      ],
      "metadata": {
        "id": "fKhZpjEWSygO"
      },
      "id": "fKhZpjEWSygO"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Es una técnica similar a `one-hot encoding` salvo que aqui los vectores están representados utilizando los valores numéricos correspondientes a los índices que ocupan cada palabra dentro del vocabulario. Es decir que cada palabra está codificada como un número entero. Esto hace que el tamaño del vector no dependa del tamaño del vocabulario.\n",
        "\n",
        "En general, esta técnica no ofrece ninguna ventaja, pero se utiliza como una representación intermedia para implementar otras formas de vectorización. Por este motivo no se la suele referenciar como una técnica de **vectorización** sino que como un **diccionario de palabras** (ya que mapea *palabras* con *IDs únicos*) Veamos como utilizarla:"
      ],
      "metadata": {
        "id": "vb5pweKcSygP"
      },
      "id": "vb5pweKcSygP"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Utilizando el vocabulario, transformamos nuestros documentos en vectores:"
      ],
      "metadata": {
        "id": "v1cr590mSygQ"
      },
      "id": "v1cr590mSygQ"
    },
    {
      "cell_type": "code",
      "execution_count": 84,
      "source": [
        "vectors = [[vocab[w] for w in s.split(' ')] for s in corpus]"
      ],
      "outputs": [],
      "metadata": {
        "id": "qx-DdW3MSygQ"
      },
      "id": "qx-DdW3MSygQ"
    },
    {
      "cell_type": "code",
      "execution_count": 85,
      "source": [
        "import pandas as pd\n",
        "\n",
        "pd.DataFrame(vectors)"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   0   1   2   3   4   5   6     7    8     9    10\n",
              "0   8  14  16   7   1  19   0   NaN  NaN   NaN  NaN\n",
              "1   8  14  16   9  21   3  22   5.0  4.0  20.0  7.0\n",
              "2   8   7  15   2   6  10  12  11.0  NaN   NaN  NaN\n",
              "3   8  14  16  13  18  17  20   7.0  1.0  19.0  0.0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b723a27a-57f6-4b68-8b8e-c900702333f4\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "      <th>1</th>\n",
              "      <th>2</th>\n",
              "      <th>3</th>\n",
              "      <th>4</th>\n",
              "      <th>5</th>\n",
              "      <th>6</th>\n",
              "      <th>7</th>\n",
              "      <th>8</th>\n",
              "      <th>9</th>\n",
              "      <th>10</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>8</td>\n",
              "      <td>14</td>\n",
              "      <td>16</td>\n",
              "      <td>7</td>\n",
              "      <td>1</td>\n",
              "      <td>19</td>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>8</td>\n",
              "      <td>14</td>\n",
              "      <td>16</td>\n",
              "      <td>9</td>\n",
              "      <td>21</td>\n",
              "      <td>3</td>\n",
              "      <td>22</td>\n",
              "      <td>5.0</td>\n",
              "      <td>4.0</td>\n",
              "      <td>20.0</td>\n",
              "      <td>7.0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>8</td>\n",
              "      <td>7</td>\n",
              "      <td>15</td>\n",
              "      <td>2</td>\n",
              "      <td>6</td>\n",
              "      <td>10</td>\n",
              "      <td>12</td>\n",
              "      <td>11.0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>8</td>\n",
              "      <td>14</td>\n",
              "      <td>16</td>\n",
              "      <td>13</td>\n",
              "      <td>18</td>\n",
              "      <td>17</td>\n",
              "      <td>20</td>\n",
              "      <td>7.0</td>\n",
              "      <td>1.0</td>\n",
              "      <td>19.0</td>\n",
              "      <td>0.0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b723a27a-57f6-4b68-8b8e-c900702333f4')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-b723a27a-57f6-4b68-8b8e-c900702333f4 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-b723a27a-57f6-4b68-8b8e-c900702333f4');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-ff3cb702-a952-4146-8cea-204de327b921\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-ff3cb702-a952-4146-8cea-204de327b921')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-ff3cb702-a952-4146-8cea-204de327b921 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 4,\n  \"fields\": [\n    {\n      \"column\": 0,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 8,\n        \"max\": 8,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          8\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 1,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3,\n        \"min\": 7,\n        \"max\": 14,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          7\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 2,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 15,\n        \"max\": 16,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          15\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 3,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4,\n        \"min\": 2,\n        \"max\": 13,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          9\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 4,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9,\n        \"min\": 1,\n        \"max\": 21,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          21\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 5,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 7,\n        \"min\": 3,\n        \"max\": 19,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          3\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 6,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 9,\n        \"min\": 0,\n        \"max\": 22,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          22\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 7,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 3.0550504633038935,\n        \"min\": 5.0,\n        \"max\": 11.0,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          5.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 8,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 2.1213203435596424,\n        \"min\": 1.0,\n        \"max\": 4.0,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 9,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.7071067811865476,\n        \"min\": 19.0,\n        \"max\": 20.0,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          19.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": 10,\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 4.949747468305833,\n        \"min\": 0.0,\n        \"max\": 7.0,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 85
        }
      ],
      "metadata": {
        "id": "seKDU01pSygQ",
        "outputId": "64b7e053-33bf-4ac4-8988-cb4abb368029",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        }
      },
      "id": "seKDU01pSygQ"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Basados en frecuencias"
      ],
      "metadata": {
        "id": "Mk67wK4USygR"
      },
      "id": "Mk67wK4USygR"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Se trata de una familia de métodos de los más ampliamente utilizado durante mucho tiempo para convertir texto en representaciones numéricas. En general, estos métodos representan el texto utilizando una lista de frecuencia de palabras, es decir, basandose de alguna forma en la frecuencia en la que aparecen."
      ],
      "metadata": {
        "id": "VhIZIN0ySygR"
      },
      "id": "VhIZIN0ySygR"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Los métodos basados en bag-of-words tienen las siguientes limitaciones:\n",
        "\n",
        " - El orden de las palabras es ignorado\n",
        " - La frecuencia de la plabra no necesariamente encapsula la importancia\n",
        " - Las frecuencias marginales juegan un papel importante (relación entre files y columnas)"
      ],
      "metadata": {
        "id": "fyRnJpBiSygR"
      },
      "id": "fyRnJpBiSygR"
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Term frecuency"
      ],
      "metadata": {
        "id": "RQJEfZHSSygR"
      },
      "id": "RQJEfZHSSygR"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Utiliza un vector de longitud igual al tamaño del vocabulario, pero donde los valores corresponden a la frecuencia de la palabra w en el documento D. Las palabras más frecuentes tienen más relevancia.\n",
        "\n",
        "$$TF = \\frac {freq(w_i)} {len(doc)} $$"
      ],
      "metadata": {
        "id": "O32ZdBqQSygR"
      },
      "id": "O32ZdBqQSygR"
    },
    {
      "cell_type": "code",
      "execution_count": 104,
      "source": [
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "\n",
        "vectorizer = CountVectorizer(min_df=0., max_df=1.)\n",
        "vectors = vectorizer.fit_transform(corpus)"
      ],
      "outputs": [],
      "metadata": {
        "id": "_yHWF2NySygS"
      },
      "id": "_yHWF2NySygS"
    },
    {
      "cell_type": "code",
      "execution_count": 105,
      "source": [
        "vectors = vectors.todense()"
      ],
      "outputs": [],
      "metadata": {
        "id": "_CuP_PYSSygS"
      },
      "id": "_CuP_PYSSygS"
    },
    {
      "cell_type": "code",
      "execution_count": 106,
      "source": [
        "vectors.shape"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4, 20)"
            ]
          },
          "metadata": {},
          "execution_count": 106
        }
      ],
      "metadata": {
        "id": "dZHgVaQPSygS",
        "outputId": "612f4560-26a3-45c3-cd09-da74ffdd38e1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "id": "dZHgVaQPSygS"
    },
    {
      "cell_type": "markdown",
      "source": [
        "> ¿Que representa 20 en las dimensiones del vector?"
      ],
      "metadata": {
        "id": "EPNzHBq9SygS"
      },
      "id": "EPNzHBq9SygS"
    },
    {
      "cell_type": "code",
      "execution_count": 107,
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "# Obtenemos todas las palabras del vocabulario\n",
        "vocab = vectorizer.get_feature_names_out()\n",
        "# Vectores de cada uno de los documentos\n",
        "pd.DataFrame(vectors, columns=vocab)"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   agua  común  congela  cuatro  de  del  el  en  es  estado  estados  grados  \\\n",
              "0     1      0        0       0   0    0   1   1   1       1        0       0   \n",
              "1     1      0        0       1   1    1   1   0   1       0        1       0   \n",
              "2     1      0        1       0   0    0   1   0   0       0        0       1   \n",
              "3     1      1        0       0   0    1   2   1   1       1        0       0   \n",
              "\n",
              "   hielo  los  naturales  nombre  pura  se  sólido  uno  \n",
              "0      1    0          0       0     0   0       1    0  \n",
              "1      1    1          1       0     0   0       0    1  \n",
              "2      0    0          0       0     1   1       0    0  \n",
              "3      1    0          0       1     0   0       1    0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-1b88306d-fe98-4523-95b1-0f31d09efeda\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>agua</th>\n",
              "      <th>común</th>\n",
              "      <th>congela</th>\n",
              "      <th>cuatro</th>\n",
              "      <th>de</th>\n",
              "      <th>del</th>\n",
              "      <th>el</th>\n",
              "      <th>en</th>\n",
              "      <th>es</th>\n",
              "      <th>estado</th>\n",
              "      <th>estados</th>\n",
              "      <th>grados</th>\n",
              "      <th>hielo</th>\n",
              "      <th>los</th>\n",
              "      <th>naturales</th>\n",
              "      <th>nombre</th>\n",
              "      <th>pura</th>\n",
              "      <th>se</th>\n",
              "      <th>sólido</th>\n",
              "      <th>uno</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-1b88306d-fe98-4523-95b1-0f31d09efeda')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-1b88306d-fe98-4523-95b1-0f31d09efeda button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-1b88306d-fe98-4523-95b1-0f31d09efeda');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-95da7354-babd-4ca3-a464-85d5dfe1dae8\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-95da7354-babd-4ca3-a464-85d5dfe1dae8')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-95da7354-babd-4ca3-a464-85d5dfe1dae8 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 4,\n  \"fields\": [\n    {\n      \"column\": \"agua\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 1,\n        \"num_unique_values\": 1,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"com\\u00fan\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"congela\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cuatro\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"de\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"del\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"el\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 2,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"en\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"es\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"estado\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"estados\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"grados\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hielo\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"los\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"naturales\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"nombre\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"pura\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"se\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"s\\u00f3lido\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"uno\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 107
        }
      ],
      "metadata": {
        "id": "jeAe-jbxSygT",
        "outputId": "5239ee1c-2f0e-4cfc-83e8-03c33181853f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        }
      },
      "id": "jeAe-jbxSygT"
    },
    {
      "cell_type": "markdown",
      "source": [
        "### TF-IDF"
      ],
      "metadata": {
        "id": "tqDCQpTHSygT"
      },
      "id": "tqDCQpTHSygT"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Se trata de un método similar a Term Frecuency, pero cuyo objetivo es tratar de ajustar la frecuencia de la palabra en cada documento considerando que tan frecuente es dentro del corpus (dispersión). Dispersión justamente se refiere a que tan equitativamente las palabras están distribuidas entre los diferentes documentos del texto.\n",
        "\n",
        "Este método considera que:\n",
        "\n",
        " - Cuanto más frecuente es una palabra en el corpus (más dispersa), más general es su significado.\n",
        " - Cuanto más centralizada está el uso de una palabra dentro de todo el corpus (baja dispersión), más probable es que la palabra represente un tópico puntual.\n",
        "\n",
        "Para calcular el peso de cada palabra debemos obtener:\n",
        "\n",
        " - **TF:** La frecuencia del termino en el corpus.\n",
        "\n",
        " $$TF = \\frac {freq(w_i)} {len(doc)} $$\n",
        "\n",
        " - **IDF:** La frecuencia (inversa) del termino en el documento.\n",
        "\n",
        "$$IDF = 1 + log(\\frac {len(corpus)} {freq(w_i, corpus)}) $$"
      ],
      "metadata": {
        "id": "rMp0s9bqSygT"
      },
      "id": "rMp0s9bqSygT"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finalmente, `tfidf` se computa como la multiplicación de los dos terminos. Adicionalmente, se normaliza usando `L2`, es decir, la norma euclidiana. `L2` reducirá el tamaño de todos los pesos pero los hará 0. Si bien es menos eficiente en terminos de memoria, puede ser útil si queremos/necesitamos retener todos los parámetros.\n",
        "\n",
        "$$\n",
        "    \\textit{TF-IDF}_{normalized} = \\frac{tf \\times idf}{\\sqrt{(tf\\times idf)^2}}\n",
        "$$"
      ],
      "metadata": {
        "id": "FKH5deeOSygT"
      },
      "id": "FKH5deeOSygT"
    },
    {
      "cell_type": "markdown",
      "source": [
        "Veamos como aplicarlo:"
      ],
      "metadata": {
        "id": "srjr6hnKSygT"
      },
      "id": "srjr6hnKSygT"
    },
    {
      "cell_type": "code",
      "execution_count": 108,
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "vectorizer = TfidfVectorizer()\n",
        "vectors = vectorizer.fit_transform(corpus)"
      ],
      "outputs": [],
      "metadata": {
        "id": "m12pg5HiSygT"
      },
      "id": "m12pg5HiSygT"
    },
    {
      "cell_type": "code",
      "execution_count": 110,
      "source": [
        "vectors = vectors.todense()\n",
        "vectors.shape"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(4, 20)"
            ]
          },
          "metadata": {},
          "execution_count": 110
        }
      ],
      "metadata": {
        "id": "yrICiomwSygU",
        "outputId": "53df8b11-2c41-4b4a-eaf6-da1e21cad54c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "id": "yrICiomwSygU"
    },
    {
      "cell_type": "markdown",
      "source": [
        "> Notemos que cambiar la forma de vectorización en estos casos no cambia la longitud de nuestros vectores (que siempre está dada por la dimensionalidad del vocabulario, en este caso 20). Solo cambia los valores numericos que se asignan en los vectores."
      ],
      "metadata": {
        "id": "j54VtxB0SygU"
      },
      "id": "j54VtxB0SygU"
    },
    {
      "cell_type": "code",
      "execution_count": 111,
      "source": [
        "vocab = vectorizer.get_feature_names_out()\n",
        "pd.DataFrame(np.round(vectors, 2), columns=vocab)"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   agua  común  congela  cuatro    de   del    el    en    es  estado  \\\n",
              "0  0.29   0.00     0.00    0.00  0.00  0.00  0.29  0.44  0.36    0.44   \n",
              "1  0.18   0.00     0.00    0.35  0.35  0.28  0.18  0.00  0.23    0.00   \n",
              "2  0.24   0.00     0.47    0.00  0.00  0.00  0.24  0.00  0.00    0.00   \n",
              "3  0.20   0.39     0.00    0.00  0.00  0.31  0.40  0.31  0.25    0.31   \n",
              "\n",
              "   estados  grados  hielo   los  naturales  nombre  pura    se  sólido   uno  \n",
              "0     0.00    0.00   0.36  0.00       0.00    0.00  0.00  0.00    0.44  0.00  \n",
              "1     0.35    0.00   0.23  0.35       0.35    0.00  0.00  0.00    0.00  0.35  \n",
              "2     0.00    0.47   0.00  0.00       0.00    0.00  0.47  0.47    0.00  0.00  \n",
              "3     0.00    0.00   0.25  0.00       0.00    0.39  0.00  0.00    0.31  0.00  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-9aad31e2-df92-496f-a6c6-a18aee33cc8b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>agua</th>\n",
              "      <th>común</th>\n",
              "      <th>congela</th>\n",
              "      <th>cuatro</th>\n",
              "      <th>de</th>\n",
              "      <th>del</th>\n",
              "      <th>el</th>\n",
              "      <th>en</th>\n",
              "      <th>es</th>\n",
              "      <th>estado</th>\n",
              "      <th>estados</th>\n",
              "      <th>grados</th>\n",
              "      <th>hielo</th>\n",
              "      <th>los</th>\n",
              "      <th>naturales</th>\n",
              "      <th>nombre</th>\n",
              "      <th>pura</th>\n",
              "      <th>se</th>\n",
              "      <th>sólido</th>\n",
              "      <th>uno</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.29</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.29</td>\n",
              "      <td>0.44</td>\n",
              "      <td>0.36</td>\n",
              "      <td>0.44</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.36</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.44</td>\n",
              "      <td>0.00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.18</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.35</td>\n",
              "      <td>0.35</td>\n",
              "      <td>0.28</td>\n",
              "      <td>0.18</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.23</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.35</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.23</td>\n",
              "      <td>0.35</td>\n",
              "      <td>0.35</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.24</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.47</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.24</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.47</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.47</td>\n",
              "      <td>0.47</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.20</td>\n",
              "      <td>0.39</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.31</td>\n",
              "      <td>0.40</td>\n",
              "      <td>0.31</td>\n",
              "      <td>0.25</td>\n",
              "      <td>0.31</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.25</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.39</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.00</td>\n",
              "      <td>0.31</td>\n",
              "      <td>0.00</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-9aad31e2-df92-496f-a6c6-a18aee33cc8b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-9aad31e2-df92-496f-a6c6-a18aee33cc8b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-9aad31e2-df92-496f-a6c6-a18aee33cc8b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-aee76d75-6c1e-4651-bbb4-f669ede4a754\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-aee76d75-6c1e-4651-bbb4-f669ede4a754')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-aee76d75-6c1e-4651-bbb4-f669ede4a754 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "summary": "{\n  \"name\": \"pd\",\n  \"rows\": 4,\n  \"fields\": [\n    {\n      \"column\": \"agua\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.048562674281111544,\n        \"min\": 0.18,\n        \"max\": 0.29,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          0.18,\n          0.2,\n          0.29\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"com\\u00fan\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.19499999999999998,\n        \"min\": 0.0,\n        \"max\": 0.39,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.39,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"congela\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.235,\n        \"min\": 0.0,\n        \"max\": 0.47,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.47,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"cuatro\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"de\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"del\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.1707581135212419,\n        \"min\": 0.0,\n        \"max\": 0.31,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          0.0,\n          0.28\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"el\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.09322910847298,\n        \"min\": 0.18,\n        \"max\": 0.4,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          0.18,\n          0.4\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"en\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.22291627725822685,\n        \"min\": 0.0,\n        \"max\": 0.44,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          0.44,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"es\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.15121728296285006,\n        \"min\": 0.0,\n        \"max\": 0.36,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          0.23,\n          0.25\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"estado\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.22291627725822685,\n        \"min\": 0.0,\n        \"max\": 0.44,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          0.44,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"estados\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"grados\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.235,\n        \"min\": 0.0,\n        \"max\": 0.47,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.47,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hielo\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.15121728296285006,\n        \"min\": 0.0,\n        \"max\": 0.36,\n        \"num_unique_values\": 4,\n        \"samples\": [\n          0.23,\n          0.25\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"los\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"naturales\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"nombre\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.19499999999999998,\n        \"min\": 0.0,\n        \"max\": 0.39,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.39,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"pura\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.235,\n        \"min\": 0.0,\n        \"max\": 0.47,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.47,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"se\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.235,\n        \"min\": 0.0,\n        \"max\": 0.47,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.47,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"s\\u00f3lido\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.22291627725822685,\n        \"min\": 0.0,\n        \"max\": 0.44,\n        \"num_unique_values\": 3,\n        \"samples\": [\n          0.44,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"uno\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0.175,\n        \"min\": 0.0,\n        \"max\": 0.35,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0.35,\n          0.0\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 111
        }
      ],
      "metadata": {
        "id": "N4dmJHarSygU",
        "outputId": "8bb32cb3-efbd-4f41-e7c9-ff20a0ea7feb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        }
      },
      "id": "N4dmJHarSygU"
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conclusión"
      ],
      "metadata": {
        "id": "Be_gwPEcSygV"
      },
      "id": "Be_gwPEcSygV"
    },
    {
      "cell_type": "markdown",
      "source": [
        "En estos métodos, cada caractéritica o palabra que forma parte de nuestra entrada está representada en su propia dimensión, y como resultado, tenemos representaciones que dependen del tamaño del vocabulario. Estas representaciones tienen la característica de que la representación de cada palabra es independiente de las restantes. Esto quiere decir que el vector que corresponde a la palabra *perro* es tan distinto al vector que corresponde a *gato* como lo es al que corresponde con *heladera*.\n",
        "\n",
        "En la siguiente sección veremos técnicas de vectorización de vectores de espacios continuos, donde estas características no se cumplen."
      ],
      "metadata": {
        "id": "Wty0OepXSygV"
      },
      "id": "Wty0OepXSygV"
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.11"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}