{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iKENeWEEwKIb"
      },
      "source": [
        "Salency maps para NLP utilizando AllenNLP\n",
        "========================================="
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "<div class=\"admonition warning\">PRECAUCIÓN 😱: El tema presentado en esta sección está clasificado como avanzado. El entendimiento de este contenido es totalmente opcional.</div>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JctlpI10jJ15"
      },
      "source": [
        "Introducción\n",
        "------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sT_t9OxYwKIc"
      },
      "source": [
        "AllenNLP es un framework general de aprendizaje profundo para NLP, establecido por el mundialmente famoso Allen Institute for AI Lab. Contiene modelos de referencia de última generación que se ejecutan sobre el `PyTorch`. AllenNLP es una librería que ademas busca implementar abstracciones que permitan el rápido desarrollo de modelos y reutilización de componentes al despegarse de detalles de implementación de cada modelo.\n",
        "\n",
        "En este ejemplo, veremos como utilizar esta librería para generar salency maps utilizando los gradientes de las prediciones. Esto nos permita interpretar las predicciones de nuestros modelos basados en `transformers`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dcyc_TQ6dis7"
      },
      "source": [
        "### Para ejecutar este notebook"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nJFcNrbmjJ17"
      },
      "source": [
        "Para ejecutar este notebook, instale las siguientes librerias:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GgK8b6e_jJ17",
        "outputId": "53134203-0221-4675-88d2-b87c67559404"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 3.1 MB 4.4 MB/s \n",
            "\u001b[K     |████████████████████████████████| 831.4 MB 2.6 kB/s \n",
            "\u001b[K     |████████████████████████████████| 719 kB 57.2 MB/s \n",
            "\u001b[K     |████████████████████████████████| 596 kB 53.4 MB/s \n",
            "\u001b[K     |████████████████████████████████| 880 kB 52.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 3.3 MB 39.5 MB/s \n",
            "\u001b[K     |████████████████████████████████| 86 kB 4.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 125 kB 73.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 1.8 MB 44.9 MB/s \n",
            "\u001b[K     |████████████████████████████████| 592 kB 57.0 MB/s \n",
            "\u001b[K     |████████████████████████████████| 248 kB 63.0 MB/s \n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Installing backend dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "    Preparing wheel metadata ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[K     |████████████████████████████████| 1.2 MB 47.7 MB/s \n",
            "\u001b[K     |████████████████████████████████| 132 kB 64.7 MB/s \n",
            "\u001b[K     |████████████████████████████████| 77 kB 6.5 MB/s \n",
            "\u001b[K     |████████████████████████████████| 8.9 MB 54.4 MB/s \n",
            "\u001b[K     |████████████████████████████████| 79 kB 9.1 MB/s \n",
            "\u001b[K     |████████████████████████████████| 138 kB 65.9 MB/s \n",
            "\u001b[K     |████████████████████████████████| 127 kB 67.1 MB/s \n",
            "\u001b[K     |████████████████████████████████| 21.0 MB 1.2 MB/s \n",
            "\u001b[K     |████████████████████████████████| 23.2 MB 1.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 23.3 MB 84.3 MB/s \n",
            "\u001b[K     |████████████████████████████████| 23.3 MB 1.2 MB/s \n",
            "\u001b[K     |████████████████████████████████| 22.1 MB 70.1 MB/s \n",
            "\u001b[K     |████████████████████████████████| 22.1 MB 80.7 MB/s \n",
            "\u001b[K     |████████████████████████████████| 181 kB 63.7 MB/s \n",
            "\u001b[K     |████████████████████████████████| 145 kB 67.0 MB/s \n",
            "\u001b[K     |████████████████████████████████| 63 kB 1.6 MB/s \n",
            "\u001b[?25h  Building wheel for fairscale (PEP 517) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for jsonnet (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for pathtools (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Building wheel for sacremoses (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "torchtext 0.12.0 requires torch==1.11.0, but you have torch 1.9.0 which is incompatible.\n",
            "torchaudio 0.11.0+cu113 requires torch==1.11.0, but you have torch 1.9.0 which is incompatible.\n",
            "datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/NLP/Datasets/mascorpus/tweets_marketing.csv \\\n",
        "    --quiet --no-clobber --directory-prefix ./Datasets/mascorpus/\n",
        "\n",
        "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/m72109/nlp/explanation.py \\\n",
        "    --quiet --no-clobber --directory-prefix ./m72109/nlp/\n",
        "    \n",
        "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/docs/nlp/neural/allennlp_interpret.txt \\\n",
        "    --quiet --no-clobber\n",
        "%pip install -r allennlp_interpret.txt --quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5YjjRwLK4leb"
      },
      "source": [
        "Si ejecuta en Google Colab, adicionalmente deberá cambiar la version de la libraria `google-cloud-storage`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LuxJI9Cd4lec",
        "outputId": "a0ac3ffb-2ab3-4fbd-ab7a-89bfb3457445"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 104 kB 2.6 MB/s \n",
            "\u001b[K     |████████████████████████████████| 75 kB 5.0 MB/s \n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "google-cloud-bigquery 1.21.0 requires google-resumable-media!=0.4.0,<0.5.0dev,>=0.3.1, but you have google-resumable-media 1.3.3 which is incompatible.\u001b[0m\n",
            "\u001b[?25h"
          ]
        }
      ],
      "source": [
        "%pip install -U google-cloud-storage==1.40.0 --quiet"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f1nl84gT4led"
      },
      "source": [
        "Descargaremos un modelo previamente entrenando el el problema de clasificación de Tweets:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "3yqczZze4lee"
      },
      "outputs": [],
      "source": [
        "!wget https://santiagxf.blob.core.windows.net/public/models/tweet_classification_bert.zip --no-clobber --quiet\n",
        "!unzip -qq tweet_classification_bert.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_gBXNzwYwKIu"
      },
      "source": [
        "Cargamos el set de datos"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "I8vqJD9JwKIv"
      },
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "\n",
        "tweets = pd.read_csv('Datasets/mascorpus/tweets_marketing.csv')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-0uIrAVo4leg"
      },
      "source": [
        "Cargando un modelo entreando con Transformers en AllenNLP\n",
        "---------------------------------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nXgQvCX2wKJi"
      },
      "source": [
        "`allennlp` es un framework compatible con la libraría `transformers` lo cual resulta atractivo a la hora de utilizar modelos que son entrenados en una para luego llevarlo a la otra. Veamos entonces como podemos hacer para cargar el modelo que tenemos previamente entrenado para la clasificación de tweets utilizando una arquitectura `BERT` dentro de este framework. En particular, nuestro modelo se persistió en el directorio \"tweet_classification\"."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "dxqTaKSc4leh"
      },
      "outputs": [],
      "source": [
        "model_name = \"tweet_classification_bert\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8nvYalyDem-4"
      },
      "source": [
        "El detalle de como utilizar AllenNLP esta fuera de este curso, pero utilizaremos el format JSON para cargar modelos de esta libreria. El siguiente codigo carga un modelo exactamente igual al que creamos utilizando la libraria HuggingFace anteriormente."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "6efVECkjehN0"
      },
      "outputs": [],
      "source": [
        "from allennlp.common import Params\n",
        "from allennlp.data.dataset_readers import DatasetReader\n",
        "\n",
        "params = Params({\n",
        "      \"type\": \"text_classification_json\",\n",
        "      \"tokenizer\": {\n",
        "          \"type\": \"pretrained_transformer\",\n",
        "          \"model_name\": model_name,\n",
        "      },\n",
        "      \"token_indexers\": {\n",
        "          \"tokens\": {\n",
        "              \"type\": \"pretrained_transformer\",\n",
        "              \"model_name\": model_name,\n",
        "          }\n",
        "      }\n",
        "})\n",
        "\n",
        "dataset_reader = DatasetReader.from_params(params)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "-Wl4WwxEehES"
      },
      "outputs": [],
      "source": [
        "from allennlp.common import Params\n",
        "from allennlp.models import Model\n",
        "from transformers import AutoModelForSequenceClassification\n",
        "\n",
        "params = Params({\n",
        "    \"type\": \"basic_classifier\",\n",
        "    \"vocab\": {\n",
        "        \"type\": \"from_pretrained_transformer\",\n",
        "        \"model_name\": model_name,\n",
        "    },\n",
        "    \"text_field_embedder\": {\n",
        "        \"type\": \"basic\",\n",
        "        \"token_embedders\": {\n",
        "            \"tokens\": {\n",
        "                \"type\": \"pretrained_transformer\",\n",
        "                \"model_name\": model_name\n",
        "            }\n",
        "        }\n",
        "    },\n",
        "    \"seq2vec_encoder\": {\n",
        "        \"type\": \"bert_pooler\",\n",
        "        \"pretrained_model\": model_name\n",
        "    },\n",
        "    \"dropout\": 0.1,\n",
        "    \"num_labels\": 5,\n",
        "});\n",
        "\n",
        "model = Model.from_params(params)\n",
        "model._classification_layer.weight = AutoModelForSequenceClassification.from_pretrained(model_name).classifier.weight\n",
        "model._classification_layer.bias = AutoModelForSequenceClassification.from_pretrained(model_name).classifier.bias\n",
        "_ = model.eval()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "8uAu57uDe1_5"
      },
      "outputs": [],
      "source": [
        "from allennlp.predictors import TextClassifierPredictor\n",
        "\n",
        "predictor = TextClassifierPredictor(model, dataset_reader)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B3TU4m2O4lem"
      },
      "source": [
        "Recordemos que en el conjunto de datos de entrenamiento, las etiquetas se distribuyen como sigue:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "MZDuP8n8fP89"
      },
      "outputs": [],
      "source": [
        "labels = [\n",
        "    'ALIMENTACION',\n",
        "    'AUTOMOCION',\n",
        "    'BANCA',\n",
        "    'BEBIDAS',\n",
        "    'DEPORTES',\n",
        "    'RETAIL',\n",
        "    'TELCO'\n",
        "]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3kB6qBzO4lem"
      },
      "source": [
        "Interpretando nuestras predicciones\n",
        "-----------------------------------"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FavSqB7C4lem"
      },
      "source": [
        "Una vez que tenemos nuestro modelo correctamente cargado, veamos como podemos interpretar una predicción computando el salency map a partir de los gradientes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "NTX6HgF6sXrs"
      },
      "outputs": [],
      "source": [
        "from allennlp.interpret.saliency_interpreters import SimpleGradient, IntegratedGradient, SmoothGradient\n",
        "\n",
        "interpreter = SmoothGradient(predictor)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l5vXo70d4lem"
      },
      "source": [
        "Busquemos un tweet para interpretar:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GMOYNNQB4lem",
        "outputId": "725b473c-b5e3-4522-fe36-08a50605ae5c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Texto: @HyundaiPeru con Grupo Primax realiza este verano servicios de Inspección Digital Gratuita a vehículos Hyundai en e… https://t.co/TZ4XFziOd3 \\Sector: AUTOMOCION\n"
          ]
        }
      ],
      "source": [
        "sample_text_idx = 1522\n",
        "sample_text = tweets['TEXTO'][sample_text_idx]\n",
        "sample_label = tweets['SECTOR'][sample_text_idx]\n",
        "\n",
        "print(\"Texto:\", sample_text, \"\\Sector:\", sample_label)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nCv3OzOq4len"
      },
      "source": [
        "Calculemos los gradientes para cada token:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "3IMgIKxytBr5"
      },
      "outputs": [],
      "source": [
        "import numpy as np"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7_00Ya8ZelBk",
        "outputId": "a76d2b47-2f56-4639-fca5-06207a49941c"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py:974: UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.\n",
            "  warnings.warn(\"Using a non-full backward hook when the forward contains multiple autograd Nodes \"\n"
          ]
        }
      ],
      "source": [
        "interpretation = interpreter.saliency_interpret_from_json({\"sentence\": sample_text })\n",
        "outputs = predictor.predict(sample_text)\n",
        "grads = np.array(interpretation['instance_1']['grad_input_1'])\n",
        "probs = np.array(outputs['probs'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eF1pX0-lh7QT",
        "outputId": "39d718e3-5705-4c26-a039-6f288dbc9156"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "dict_keys(['logits', 'probs', 'token_ids', 'label', 'tokens'])"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "outputs.keys()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9gF--Q8v4len"
      },
      "source": [
        "Podemos graficar los resultados utilizando un mapa de calor marcando con colores más intensos aquellos tokens que tienen mayor impacto en las predicciones:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "dLcrGXYnhYcz"
      },
      "outputs": [],
      "source": [
        "from IPython.display import HTML\n",
        "from eli5.formatters import format_as_html\n",
        "from m72109.nlp.explanation import get_explanation_from_grads"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "id": "JtxoMSnJ-Rm5"
      },
      "outputs": [],
      "source": [
        "expl = get_explanation_from_grads(estimator_name=\"transformer\",\n",
        "                                  estimator_description=\"NLP transformer explanation\",\n",
        "                                  text=sample_text,\n",
        "                                  tokens=outputs['tokens'],\n",
        "                                  grads=grads,\n",
        "                                  probas=probs,\n",
        "                                  labels=labels)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 170
        },
        "id": "xPXW3jqq5BP-",
        "outputId": "e2d8a4b4-3a3b-4077-d005-035256c9cbca"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "    <style>\n",
              "    table.eli5-weights tr:hover {\n",
              "        filter: brightness(85%);\n",
              "    }\n",
              "</style>\n",
              "\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "        \n",
              "        <pre>NLP transformer explanation</pre>\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "        \n",
              "\n",
              "    \n",
              "\n",
              "        \n",
              "            \n",
              "                \n",
              "                \n",
              "    \n",
              "        <p style=\"margin-bottom: 0.5em; margin-top: 0em\">\n",
              "            <b>\n",
              "    \n",
              "        y=AUTOMOCION\n",
              "    \n",
              "</b>\n",
              "\n",
              "    \n",
              "    (probability <b>0.900</b>)\n",
              "\n",
              "top features\n",
              "        </p>\n",
              "    \n",
              "    <table class=\"eli5-weights\"\n",
              "           style=\"border-collapse: collapse; border: none; margin-top: 0em; table-layout: auto; margin-bottom: 2em;\">\n",
              "        <thead>\n",
              "        <tr style=\"border: none;\">\n",
              "            \n",
              "                <th style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\" title=\"Feature contribution already accounts for the feature value (for linear models, contribution = weight * feature value), and the sum of feature contributions is equal to the score or, for some classifiers, to the probability. Feature values are shown if &quot;show_feature_values&quot; is True.\">\n",
              "                    Contribution<sup>?</sup>\n",
              "                </th>\n",
              "            \n",
              "            <th style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">Feature</th>\n",
              "            \n",
              "        </tr>\n",
              "        </thead>\n",
              "        <tbody>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 80.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +1.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        Highlighted in text\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "        \n",
              "\n",
              "        \n",
              "        \n",
              "\n",
              "        </tbody>\n",
              "    </table>\n",
              "\n",
              "            \n",
              "        \n",
              "\n",
              "        \n",
              "\n",
              "\n",
              "    <p style=\"margin-bottom: 2.5em; margin-top:-0.5em;\">\n",
              "        <span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">@</span><span style=\"background-color: hsl(120, 100.00%, 60.00%); opacity: 1.00\" title=\"0.099\">Hyu</span><span style=\"background-color: hsl(120, 100.00%, 63.17%); opacity: 0.98\" title=\"0.088\">n</span><span style=\"opacity: 0.80\">da</span><span style=\"background-color: hsl(120, 100.00%, 86.04%); opacity: 0.84\" title=\"0.022\">iPe</span><span style=\"opacity: 0.80\">ru </span><span style=\"background-color: hsl(120, 100.00%, 86.04%); opacity: 0.84\" title=\"0.022\">con</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">Grupo</span><span style=\"opacity: 0.80\"> Prima</span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">x</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.033\">realiza</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 94.71%); opacity: 0.81\" title=\"0.005\">este</span><span style=\"opacity: 0.80\"> verano </span><span style=\"background-color: hsl(120, 100.00%, 86.04%); opacity: 0.84\" title=\"0.022\">servicios</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">de</span><span style=\"opacity: 0.80\"> Inspección </span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.025\">Digital</span><span style=\"opacity: 0.80\"> Gratuita a </span><span style=\"background-color: hsl(120, 100.00%, 77.33%); opacity: 0.89\" title=\"0.044\">vehículos</span><span style=\"opacity: 0.80\"> Hyun</span><span style=\"background-color: hsl(120, 100.00%, 94.71%); opacity: 0.81\" title=\"0.005\">da</span><span style=\"background-color: hsl(120, 100.00%, 79.35%); opacity: 0.88\" title=\"0.038\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 90.67%); opacity: 0.83\" title=\"0.012\">en</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.04%); opacity: 0.84\" title=\"0.022\">e</span><span style=\"opacity: 0.80\">…</span><span style=\"background-color: hsl(120, 100.00%, 98.00%); opacity: 0.80\" title=\"0.001\"> http</span><span style=\"opacity: 0.80\">s</span><span style=\"background-color: hsl(120, 100.00%, 88.59%); opacity: 0.83\" title=\"0.016\">:</span><span style=\"opacity: 0.80\">//</span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.033\">t.</span><span style=\"opacity: 0.80\">c</span><span style=\"background-color: hsl(120, 100.00%, 69.88%); opacity: 0.93\" title=\"0.066\">o</span><span style=\"opacity: 0.80\">/</span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">T</span><span style=\"opacity: 0.80\">Z</span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">4</span><span style=\"opacity: 0.80\">X</span><span style=\"background-color: hsl(120, 100.00%, 87.29%); opacity: 0.84\" title=\"0.019\">F</span><span style=\"opacity: 0.80\">z</span><span style=\"background-color: hsl(120, 100.00%, 94.71%); opacity: 0.81\" title=\"0.005\">i</span><span style=\"opacity: 0.80\">O</span><span style=\"background-color: hsl(120, 100.00%, 91.41%); opacity: 0.82\" title=\"0.011\">d3</span>\n",
              "    </p>\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "HTML(format_as_html(expl))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "id": "CEyPEV0J4ZxH"
      },
      "outputs": [],
      "source": [
        "expl = get_explanation_from_grads(estimator_name=\"transformer\",\n",
        "                                  estimator_description=\"NLP transformer explanation\",\n",
        "                                  text=sample_text,\n",
        "                                  tokens=outputs['tokens'],\n",
        "                                  grads=grads,\n",
        "                                  probas=probs,\n",
        "                                  labels=labels,\n",
        "                                  force_weights=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 934
        },
        "id": "McfMTL9u5EC8",
        "outputId": "6c154eea-8e0f-42e0-8216-d9d4abe2d5ab"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "\n",
              "    <style>\n",
              "    table.eli5-weights tr:hover {\n",
              "        filter: brightness(85%);\n",
              "    }\n",
              "</style>\n",
              "\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "        \n",
              "        <pre>NLP transformer explanation</pre>\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "        \n",
              "\n",
              "    \n",
              "\n",
              "        \n",
              "            \n",
              "                \n",
              "                \n",
              "    \n",
              "        <p style=\"margin-bottom: 0.5em; margin-top: 0em\">\n",
              "            <b>\n",
              "    \n",
              "        y=AUTOMOCION\n",
              "    \n",
              "</b>\n",
              "\n",
              "    \n",
              "    (probability <b>0.900</b>, score <b>1.000</b>)\n",
              "\n",
              "top features\n",
              "        </p>\n",
              "    \n",
              "    <table class=\"eli5-weights\"\n",
              "           style=\"border-collapse: collapse; border: none; margin-top: 0em; table-layout: auto; margin-bottom: 2em;\">\n",
              "        <thead>\n",
              "        <tr style=\"border: none;\">\n",
              "            \n",
              "                <th style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\" title=\"Feature weights. Note that weights do not account for feature value scales, so if feature values have different scales, features with highest weights might not be the most important.\">\n",
              "                    Weight<sup>?</sup>\n",
              "                </th>\n",
              "            \n",
              "            <th style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">Feature</th>\n",
              "            \n",
              "        </tr>\n",
              "        </thead>\n",
              "        <tbody>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 96.19%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.011\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        [CLS]\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        @\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        hyun\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##da\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 87.15%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.062\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##ipe\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##ru\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        con\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 89.93%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.044\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        grupo\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        prima\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##x\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        realiza\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 93.80%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.022\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        este\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 95.33%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.015\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        verano\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 83.65%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.087\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        servicios\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        de\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        inspección\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 94.55%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.018\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        digital\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        gratuita\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 93.80%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.022\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        a\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 87.69%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.058\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        vehículos\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 80.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.116\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        hyun\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 89.93%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.044\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##da\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 89.93%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.044\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##i\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 96.88%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.008\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        en\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 93.80%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.022\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        e\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 98.55%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.003\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        [UNK]\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        h\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 93.80%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.022\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##tt\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##ps\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        :\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 95.33%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.015\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        /\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 97.13%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.007\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        /\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        t\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 96.19%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.011\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        .\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 95.33%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.015\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        co\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 97.65%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.005\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        /\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        t\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 98.91%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.002\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##z\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 89.93%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.044\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##4\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 95.33%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.015\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##x\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##f\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 92.42%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.029\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##zio\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##d\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(120, 100.00%, 87.69%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.058\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        ##3\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "            <tr style=\"background-color: hsl(0, 100.00%, 100.00%); border: none;\">\n",
              "    <td style=\"padding: 0 1em 0 0.5em; text-align: right; border: none;\">\n",
              "        +0.000\n",
              "    </td>\n",
              "    <td style=\"padding: 0 0.5em 0 0.5em; text-align: left; border: none;\">\n",
              "        [SEP]\n",
              "    </td>\n",
              "    \n",
              "</tr>\n",
              "        \n",
              "        \n",
              "\n",
              "        \n",
              "        \n",
              "\n",
              "        </tbody>\n",
              "    </table>\n",
              "\n",
              "            \n",
              "        \n",
              "\n",
              "        \n",
              "\n",
              "\n",
              "    <p style=\"margin-bottom: 2.5em; margin-top:-0.5em;\">\n",
              "        <span style=\"background-color: hsl(120, 100.00%, 90.67%); opacity: 0.83\" title=\"0.011\">@Hyu</span><span style=\"opacity: 0.80\">n</span><span style=\"background-color: hsl(120, 100.00%, 88.59%); opacity: 0.83\" title=\"0.015\">da</span><span style=\"background-color: hsl(120, 100.00%, 68.58%); opacity: 0.94\" title=\"0.062\">iPe</span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.029\">ru</span><span style=\"opacity: 0.80\"> con </span><span style=\"background-color: hsl(120, 100.00%, 75.38%); opacity: 0.90\" title=\"0.044\">Grupo</span><span style=\"opacity: 0.80\"> Primax </span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.029\">realiza</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.022\">este</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 88.59%); opacity: 0.83\" title=\"0.015\">verano</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 60.00%); opacity: 1.00\" title=\"0.087\">servicios</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.029\">de</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.029\">Inspección</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 86.66%); opacity: 0.84\" title=\"0.018\">Digital</span><span style=\"opacity: 0.80\"> Gratuita </span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.022\">a</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 69.88%); opacity: 0.93\" title=\"0.058\">vehículos</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 69.88%); opacity: 0.93\" title=\"0.058\">Hyun</span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.022\">da</span><span style=\"background-color: hsl(120, 100.00%, 75.38%); opacity: 0.90\" title=\"0.044\">i</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 92.37%); opacity: 0.82\" title=\"0.008\">en</span><span style=\"opacity: 0.80\"> </span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.022\">e</span><span style=\"opacity: 0.80\">…</span><span style=\"background-color: hsl(120, 100.00%, 96.46%); opacity: 0.81\" title=\"0.003\"> http</span><span style=\"opacity: 0.80\">s</span><span style=\"background-color: hsl(120, 100.00%, 81.46%); opacity: 0.87\" title=\"0.029\">:</span><span style=\"background-color: hsl(120, 100.00%, 84.84%); opacity: 0.85\" title=\"0.022\">//</span><span style=\"opacity: 0.80\">t.co/</span><span style=\"background-color: hsl(120, 100.00%, 94.71%); opacity: 0.81\" title=\"0.005\">T</span><span style=\"opacity: 0.80\">Z</span><span style=\"background-color: hsl(120, 100.00%, 96.74%); opacity: 0.81\" title=\"0.002\">4</span><span style=\"opacity: 0.80\">X</span><span style=\"background-color: hsl(120, 100.00%, 88.59%); opacity: 0.83\" title=\"0.015\">F</span><span style=\"opacity: 0.80\">z</span><span style=\"background-color: hsl(120, 100.00%, 90.67%); opacity: 0.83\" title=\"0.011\">i</span><span style=\"opacity: 0.80\">O</span><span style=\"background-color: hsl(120, 100.00%, 88.59%); opacity: 0.83\" title=\"0.015\">d3</span>\n",
              "    </p>\n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "    \n",
              "\n",
              "\n",
              "\n"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "execution_count": 61,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "HTML(format_as_html(expl, force_weights=True))"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "allennlp_interpret.ipynb",
      "provenance": []
    },
    "kernel_info": {
      "name": "nlp-py38"
    },
    "kernelspec": {
      "display_name": "Python 3.8 - PyTorch",
      "language": "python",
      "name": "azureml_py38_pytorch"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.1"
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}