{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "nQGxT74Sdis4" }, "source": [ "Ejemplo: Modelo Generator con Keras\n", "===============================" ] }, { "cell_type": "markdown", "source": [ "
PRECAUCIÓN 😱: El tema presentado en esta sección está clasificado como avanzado. El entendimiento de este contenido es totalmente opcional.
" ], "metadata": { "id": "NFlKQn0uaLZV" } }, { "cell_type": "markdown", "metadata": { "id": "wLAibjXDaJHi" }, "source": [ "Introducción\n", "------------" ] }, { "cell_type": "markdown", "metadata": { "id": "vjuHJnQOdis6" }, "source": [ "Una de las formás más sencillas de utilizar esta idea es simplemente utilizando el vector resultante de la red RNN. De esta forma al modelo observa el estado final al que ha arrivado la red y toma una decisión dependiendo del caso a resolver. Tipicamente este vector resultante es conectado a una capa densa (fully connected layer) para general la predicción. Los grandientes de los errores son propagados en la red para toda la secuencia y suele ser cualquier función típica como ser `cross entropy`, `hinge`, etc.\n", "\n", "Veremos como podemos aplicar esta idea para crear un generador de tweets." ] }, { "cell_type": "markdown", "metadata": { "id": "Dcyc_TQ6dis7" }, "source": [ "### Para ejecutar este notebook" ] }, { "cell_type": "markdown", "metadata": { "id": "vONt_Ba7aJHk" }, "source": [ "Para ejecutar este notebook, instale las siguientes librerias:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "FbpqV27iaJHk" }, "outputs": [], "source": [ "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/NLP/Datasets/mascorpus/tweets_marketing.csv \\\n", " --quiet --no-clobber --directory-prefix ./Datasets/mascorpus/\n", "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/NLP/Utils/TextNormalizer.py \\\n", " --quiet --no-clobber --directory-prefix ./Utils/\n", "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/NLP/Utils/PadSequenceTransformer.py \\\n", " --quiet --no-clobber --directory-prefix ./Utils/\n", "\n", "!wget https://raw.githubusercontent.com/santiagxf/M72109/master/docs/nlp/neural/sequences-generator.txt \\\n", " --quiet --no-clobber\n", "!pip install -r sequences-generator.txt --quiet" ] }, { "cell_type": "markdown", "metadata": { "id": "Gl18LLmkeqjx" }, "source": [ "Descargamos nuestros vectores de word2vec en español" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LJVkiH2lesN1" }, "outputs": [], "source": [ "!mkdir -p ./Models/Word2Vec\n", "!wget https://santiagxf.blob.core.windows.net/public/Word2Vec/model-es.bin \\\n", " --quiet --no-clobber --directory-prefix ./Models/Word2Vec" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Ntcs1AlpfckX" }, "outputs": [], "source": [ "import warnings\n", "warnings.filterwarnings('ignore')" ] }, { "cell_type": "markdown", "metadata": { "id": "0rRpBRahditI" }, "source": [ "Instalamos las librerias necesarias" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "UgVf9-V7ditI" }, "outputs": [], "source": [ "!python -m spacy download es_core_news_sm 1> /dev/null" ] }, { "cell_type": "markdown", "metadata": { "id": "CC7vGpjqditL" }, "source": [ "Cargamos el set de datos" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "pmJpenUkditM" }, "outputs": [], "source": [ "import pandas as pd\n", "\n", "tweets = pd.read_csv('Datasets/mascorpus/tweets_marketing.csv')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "RKrEadrGditO" }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(tweets['TEXTO'], tweets['SECTOR'],\n", " test_size=0.33,\n", " stratify=tweets['SECTOR'])" ] }, { "cell_type": "markdown", "metadata": { "id": "kwRwMqoSditT" }, "source": [ "## Preprocesamiento de texto" ] }, { "cell_type": "markdown", "metadata": { "id": "_qrYYZ0FditT" }, "source": [ "Al igual que con Topic Modeling, nuestro primer paso es preprocesar el texto. Para focalizarnos en Word2Vec en este modulo, les preparé un modulo TweetTextNormalizer que hará todo el preprocesamiento por nosotros. Pueden explorar los parametros que recibe el constructor de esta clase para ver que opciones podemos configurar como Stemmer, Lemmatization, etc.\n", "\n", "En lo particular, estamos creando un TweetTextNormalizer que:\n", " - Aplicará un tokenizer especifico para Twitter\n", " - Eliminará URLs\n", " - Eliminará acentos\n", " - Eliminará las mayusculas\n", "\n", "Adicionalmente, el parametro text_to_sequence=True indica que la salida de este proceso no serán oraciones sino que tokens." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Jnuzl8qbditU" }, "outputs": [], "source": [ "from Utils.TextNormalizer import TweetTextNormalizer" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "kFYzMdiCditX" }, "outputs": [], "source": [ "normalizer = TweetTextNormalizer(preserve_case=False,\n", " strip_stopwords=False,\n", " lemmatize = False,\n", " text_to_sequence=True,\n", " token_min_len=0)" ] }, { "cell_type": "markdown", "metadata": { "id": "6sD5jK6ndita" }, "source": [ "Podemos probar como funciona:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ldKc0udIditb" }, "outputs": [], "source": [ "normalized_tweets = list(normalizer.transform(tweets[\"TEXTO\"]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Fbg3wchQaJH1", "outputId": "5d563722-4916-4bc2-c002-3147e87d7d1f" }, "outputs": [ { "data": { "text/plain": [ "['el', 'jazz', 'de', 'carrefour', 'es', 'muy', 'bueno', '.']" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "normalized_tweets[50]" ] }, { "cell_type": "markdown", "metadata": { "id": "QdoLyHg9ditd" }, "source": [ "## Vectorización de las palabras" ] }, { "cell_type": "markdown", "metadata": { "id": "a9iuTrwXditd" }, "source": [ "En las actividades anteriores utilizamos siempre un TF-IDF vectorizer para generar los vectores. En esta oportunidad utilizaremos vectores densos. Aprenderemos estos vectores densos como parte de la red que estamos entrenando.\n", "\n", "Por este motivo, realizaremos una vectorización utilizando `index-based encoding`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "jE-SlBWddite" }, "outputs": [], "source": [ "from tensorflow.keras.preprocessing.text import Tokenizer" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "G1VmkFh-aJH3" }, "outputs": [], "source": [ "BOS = \"/bos\"\n", "EOS = \"/eos\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "vkWxVLPYditg" }, "outputs": [], "source": [ "tokenizer = Tokenizer(oov_token=\"[UNK]\", filters='!\"#$%&()*+,-.:;<=>?@[\\\\]^_`{|}~\\t\\n')\n", "tokenizer.fit_on_texts(normalized_tweets)\n", "tokenizer.fit_on_texts([BOS])\n", "tokenizer.fit_on_texts([EOS])" ] }, { "cell_type": "markdown", "metadata": { "id": "UtL8BkQPditk" }, "source": [ "Verifiquemos el vocabulario resultante:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "M4LPSO-saJH4", "outputId": "9d7940e3-4992-484f-c644-a53c021df651" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "El tamaño del vocabulario es: 9999\n" ] } ], "source": [ "vocab_size = len(tokenizer.word_index) + 1 # OOV\n", "print('El tamaño del vocabulario es: %d' % vocab_size)" ] }, { "cell_type": "markdown", "metadata": { "id": "yof5yc6taJH5" }, "source": [ "Veamos algunos de los vectores resultantes:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "04CUhyuzaJH6", "outputId": "85b14295-a47b-407f-9630-2d7891f12aca" }, "outputs": [ { "data": { "text/plain": [ "[('[UNK]', 1),\n", " ('de', 2),\n", " (',', 3),\n", " ('.', 4),\n", " ('la', 5),\n", " ('y', 6),\n", " ('en', 7),\n", " ('que', 8),\n", " ('', 9),\n", " ('el', 10),\n", " ('a', 11),\n", " ('...', 12),\n", " ('!', 13),\n", " ('?', 14),\n", " ('un', 15),\n", " ('no', 16),\n", " ('me', 17),\n", " ('con', 18),\n", " ('para', 19),\n", " ('una', 20)]" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "list(tokenizer.word_index.items())[:20]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "82_3yklnaJH7", "outputId": "da73e75d-535b-442d-d4af-9a83cfb56b5e" }, "outputs": [ { "data": { "text/plain": [ "[[9998]]" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tokenizer.texts_to_sequences([\"/eos\"])" ] }, { "cell_type": "markdown", "metadata": { "id": "UBikLZLIaJH7" }, "source": [ "Aplicamos la transformación para pasar de palabras a indices:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "QpixJPK7aJH8" }, "outputs": [], "source": [ "encoder_tweets = list()\n", "decoder_tweets = list()\n", "target_tweets = list()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "sJORskk-aJH8" }, "outputs": [], "source": [ "for tweet in normalized_tweets:\n", " tweet_enc = tweet.copy()\n", " tweet_dec = tweet.copy()\n", " tweet_tar = tweet.copy()\n", "\n", " tweet_enc.insert(0, BOS)\n", " tweet_dec.insert(0, BOS)\n", " tweet_enc.append(EOS)\n", " tweet_dec.append(EOS)\n", " tweet_tar.append(EOS)\n", "\n", " encoder_tweets.append(tweet_enc)\n", " decoder_tweets.append(tweet_dec)\n", " target_tweets.append(tweet_tar)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Bqtr3S1gaJH9", "outputId": "7c14ab0e-e50c-44ba-8844-3db3c93780b3" }, "outputs": [ { "data": { "text/plain": [ "['/bos', '#tablondeanuncios', 'funda', 'nordica', 'ikea', '#madrid', '/eos']" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "encoder_tweets[0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "KoZATklXaJH9", "outputId": "ae50f363-c0d5-4514-d2cb-32f4a114f788" }, "outputs": [ { "data": { "text/plain": [ "['/bos', '#tablondeanuncios', 'funda', 'nordica', 'ikea', '#madrid', '/eos']" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "decoder_tweets[0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "7DoFwCnSaJH-", "outputId": "b9692bc2-2164-48c5-a6b1-44a8d3339e77" }, "outputs": [ { "data": { "text/plain": [ "['#tablondeanuncios', 'funda', 'nordica', 'ikea', '#madrid', '/eos']" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "target_tweets[0]" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "66MU1L9PaJH-" }, "outputs": [], "source": [ "input_encoder = tokenizer.texts_to_sequences(encoder_tweets)\n", "input_decoder = tokenizer.texts_to_sequences(decoder_tweets)\n", "target_decoder = tokenizer.texts_to_sequences(target_tweets)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "b52dcduwaJH-" }, "outputs": [], "source": [ "from Utils.Word2VecVectorizer import Word2VecVectorizer\n", "\n", "w2v = Word2VecVectorizer(model_path='Models/Word2Vec/model-es.bin', sequence_to_idx=False)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "eSC-HJ_ZaJII" }, "outputs": [], "source": [ "topics = tweets[\"SECTOR\"].to_numpy().reshape(-1, 1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "uBN4yVMaaJIJ" }, "outputs": [], "source": [ "topic_decoder = list(w2v.transform(topics))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "-LeZ3qjTaJIJ" }, "outputs": [], "source": [ "import numpy as np\n", "topic_decoder_np = np.asarray(topic_decoder).reshape(-1, 100)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "JYmS81mcaJIK", "outputId": "ad21c8fe-8c43-4285-8960-d4e26ac05376" }, "outputs": [ { "data": { "text/plain": [ "(3763, 100)" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "topic_decoder_np.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "uBNY37iIditm" }, "source": [ "## Construirmos un modelo basado en secuencias" ] }, { "cell_type": "markdown", "metadata": { "id": "vhfhArdbdit5" }, "source": [ "### Ajustando la longitud de las secuencias" ] }, { "cell_type": "markdown", "metadata": { "id": "_WpivJzsdit6" }, "source": [ "Los modelos basados en secuencias pueden adaptarse a cualquier longitud de secuencia, sin embargo, los parametros de nuestras redes neuronales deberan ser fijos. Para esto definiermos una longitud máxima de la secuencia que vamos analizar. Para esto podemos utilizar un valor especifico o utilizar el valor máximo de tokens que hay en nuestro corpus." ] }, { "cell_type": "markdown", "metadata": { "id": "QhNcdbgFaJIL" }, "source": [ "Para saber que valor es el correcto, podemos graficar la distribución de cantidad de palabras en los tweets:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "h9a70-LSaJIL", "outputId": "1cafe8ab-3edf-454d-e98f-7dac6450aa51" }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAXbUlEQVR4nO3df6xfdZ3n8efLimBEV4gXKBQC69bNgFkquXRnh2TDICtdx1idDKRmx60JuzVZWHWdzAjuH9Y/mpCJvza7qztVidVR2Tpq6DguMwgyxsRQSgeQgsRmYKD23rYqjZJN2NC+9497Gr6D91dbzvl8e+/zkXzzPd/POZ9z3j2593VPP9/zI1WFJGl4r2hdgCQtVwawJDViAEtSIwawJDViAEtSI69sXcDJWLduXd11112ty5AkgBxvh1P6CPjnP/956xIk6YSd0gEsSacyA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjml7wesk3P5FVcyNT095/yV553Hw7sfGLAiaXkxgJexqelprtm8fc75926+YcBqpOXHIQhJasQAlqRGDGBJasQAlqRGeg/gJCuS/F2S73Sfz05yd5Kfdu9njSx7a5K9SZ5Icl3ftUlSS0McAX8QeHzk8y3APVW1Grin+0ySS4ENwGXAOuCzSVYMUJ8kNdFrACdZBfwe8IWR5vXAtm56G/CukfY7qur5qnoS2Aus7bM+SWqp7yPgzwB/AhwdaTu3qqYAuvdzuvYLgGdGltvXtf0jSTYl2ZVk16FDh3opWpKG0FsAJ3kHcLCqHlxsl1na6jcaqrZW1WRVTU5MTJxUjZLUUp9Xwl0FvDPJ24EzgNcl+XPgQJKVVTWVZCVwsFt+H3DhSP9VwP4e65Okpno7Aq6qW6tqVVVdzMyXa/dW1R8CO4CN3WIbgTu76R3AhiSnJ7kEWA3s7Ks+SWqtxb0gbgO2J7kReBq4HqCq9iTZDjwGvADcVFVHGtQnSYMYJICr6j7gvm76F8Bb51huC7BliJokqTWvhJOkRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWqktwBOckaSnUkeTrInyce79s1Jfpbkoe719pE+tybZm+SJJNf1VZskjYNX9rju54Frquq5JKcBP0zyf7p5n66qT4wunORSYANwGXA+8L0kb6qqIz3WKEnN9HYEXDOe6z6e1r1qni7rgTuq6vmqehLYC6ztqz5Jaq3XMeAkK5I8BBwE7q6q+7tZNyd5JMntSc7q2i4Anhnpvq9re+k6NyXZlWTXoUOH+ixfknrVawBX1ZGqWgOsAtYmeTPwOeCNwBpgCvhkt3hmW8Us69xaVZNVNTkxMdFL3ZI0hEHOgqiqw8B9wLqqOtAF81Hg87w4zLAPuHCk2ypg/xD1SVILfZ4FMZHk9d30q4FrgZ8kWTmy2LuBR7vpHcCGJKcnuQRYDezsqz5Jaq3PsyBWAtuSrGAm6LdX1XeSfCXJGmaGF54C3g9QVXuSbAceA14AbvIMCElLWW8BXFWPAG+Zpf298/TZAmzpqyZJGideCSdJjRjAktRIn2PA0oIuv+JKpqan55y/8rzzeHj3AwNWJA3HAFZTU9PTXLN5+5zz7918w4DVSMNyCEKSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJamR3gI4yRlJdiZ5OMmeJB/v2s9OcneSn3bvZ430uTXJ3iRPJLmur9okaRz0eQT8PHBNVV0OrAHWJflt4BbgnqpaDdzTfSbJpcAG4DJgHfDZJCt6rE+SmuotgGvGc93H07pXAeuBbV37NuBd3fR64I6qer6qngT2Amv7qk+SWut1DDjJiiQPAQeBu6vqfuDcqpoC6N7P6Ra/AHhmpPu+ru2l69yUZFeSXYcOHeqzfEnqVa8BXFVHqmoNsApYm+TN8yye2VYxyzq3VtVkVU1OTEy8TJVK0vAGOQuiqg4D9zEztnsgyUqA7v1gt9g+4MKRbquA/UPUJ0kt9HkWxESS13fTrwauBX4C7AA2dottBO7spncAG5KcnuQSYDWws6/6JKm1V/a47pXAtu5MhlcA26vqO0l+BGxPciPwNHA9QFXtSbIdeAx4Abipqo70WJ8kNdVbAFfVI8BbZmn/BfDWOfpsAbb0VZMkjROvhJOkRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRvq8Ibt0Srj8iiuZmp6ed5mV553Hw7sfGKgiLRcGsJa9qelprtm8fd5l7t18w0DVaDlxCEKSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGvFKuEYWuvzVS1+lpc8AbmShy1+99FVa+nobgkhyYZLvJ3k8yZ4kH+zaNyf5WZKHutfbR/rcmmRvkieSXNdXbZI0Dvo8An4B+KOq2p3ktcCDSe7u5n26qj4xunCSS4ENwGXA+cD3krypqo70WKMkNdPbEXBVTVXV7m7618DjwAXzdFkP3FFVz1fVk8BeYG1f9UlSa4OcBZHkYuAtwP1d081JHklye5KzurYLgGdGuu1j/sCWpFNa7wGc5Ezgm8CHqupXwOeANwJrgCngk8cWnaV7zbK+TUl2Jdl16NChfoqWpAH0GsBJTmMmfL9aVd8CqKoDVXWkqo4Cn+fFYYZ9wIUj3VcB+1+6zqraWlWTVTU5MTHRZ/mS1Ks+z4II8EXg8ar61Ej7ypHF3g082k3vADYkOT3JJcBqYGdf9UlSa32eBXEV8F7gx0ke6to+CrwnyRpmhheeAt4PUFV7kmwHHmPmDIqbPANC0lLWWwBX1Q+ZfVz3u/P02QJs6asmSRon3gtCkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhpZVAAnuWoxbZKkxVvsEfB/X2SbJGmR5r0SLsm/An4HmEjy4ZFZrwNW9FmYJC11C12K/CrgzG651460/wr4g76KkqTlYN4Arqq/Bf42yZeq6h8GqkmSloXF3ozn9CRbgYtH+1TVNX0UJUnLwWID+BvA/wK+AHiLSEl6GSw2gF+oqs/1WokkLTOLPQ3tL5P8pyQrk5x97NVrZZK0xC32CHhj9/7HI20F/NOXtxxJWj4WFcBVdUnfhUjScrOoAE7y72drr6ovv7zlSNLysdghiCtHps8A3grsBgxgSTpBix2C+M+jn5P8E+ArvVQkScvEid6O8v8Cq1/OQiRpuVnsGPBfMnPWA8zchOe3gO19FSVJy8Fix4A/MTL9AvAPVbWvh3okadlY1BBEd1OenzBzR7SzgP/XZ1GStBws9okYNwA7geuBG4D7k3g7Skk6CYsdgvivwJVVdRAgyQTwPeAv+ipMkpa6xZ4F8Ypj4dv5xXH0lSTNYrEheleSv07yviTvA/4K+O58HZJcmOT7SR5PsifJB7v2s5PcneSn3ftZI31uTbI3yRNJrjvRf5QknQrmDeAk/yzJVVX1x8CfAf8CuBz4EbB1gXW/APxRVf0W8NvATUkuBW4B7qmq1cA93We6eRuAy4B1wGeT+Nw5SUvWQkfAnwF+DVBV36qqD1fVf2Hm6Pcz83Wsqqmq2t1N/xp4HLgAWA9s6xbbBryrm14P3FFVz1fVk8BeYO1x/nsk6ZSxUABfXFWPvLSxqnYx83iiRUlyMfAW4H7g3Kqa6tYzBZzTLXYB8MxIt31dmyQtSQudBXHGPPNevZgNJDkT+Cbwoar6VZI5F52lrX5joWQTsAngoosuWkwJy9blV1zJ1PT0nPOfPXx43v7PHj7MOedfOO8yK887j4d3P3Ai5UnL3kIB/ECS/1hVnx9tTHIj8OBCK09yGjPh+9Wq+lbXfCDJyqqaSrISOHZ2xT5g9Ld9FbD/peusqq1048+Tk5O/EdB60dT0NNdsnvuK8W984Np5+x89enTe/gD3br7hhGqTtHAAfwj4dpJ/x4uBOwm8Cnj3fB0zc6j7ReDxqvrUyKwdzDxh47bu/c6R9q8l+RRwPjM3+9m56H+JJJ1i5g3gqjoA/E6S3wXe3DX/VVXdu4h1XwW8F/hxkoe6to8yE7zbu6Pop5m5uo6q2pNkO/AYM2dQ3FRVPoFZ0pK12PsBfx/4/vGsuKp+yOzjujBzQ/fZ+mwBthzPdiTpVOXVbJLUiAEsSY0YwJLUiAEsSY0s9naU0qwWuljDCzWkuRnAOikLXazhhRrS3ByCkKRGDGBJasQAlqRGDGBJasQAlqRGDGBJasTT0DTWvCm8ljIDWGPNm8JrKXMIQpIaMYAlqREDWJIaMYAlqRG/hNMpzzuy6VRlAOuU5x3ZdKoygE9Rl19xJVPT0/Mu8+zhw8MUI+mEGMCnqKnp6QXPj/3GB64dqJq5LTQ84B8JLWcGsHq10PDAEH8k/COgcWUAa8kbhz8C0mwM4DHlUZu09BnAY8qjNmnp80IMSWrEAJakRnoL4CS3JzmY5NGRts1Jfpbkoe719pF5tybZm+SJJNf1VZckjYs+j4C/BKybpf3TVbWme30XIMmlwAbgsq7PZ5Os6LE2SWqutwCuqh8Av1zk4uuBO6rq+ap6EtgLrO2rNkkaBy3GgG9O8kg3RHFW13YB8MzIMvu6NklasoYO4M8BbwTWAFPAJ7v2zLJszbaCJJuS7Eqy69ChQ70UKUlDGDSAq+pAVR2pqqPA53lxmGEfMHrVwSpg/xzr2FpVk1U1OTEx0W/BktSjQQM4ycqRj+8Gjp0hsQPYkOT0JJcAq4GdQ9YmSUPr7Uq4JF8HrgbekGQf8DHg6iRrmBleeAp4P0BV7UmyHXgMeAG4qaqO9FWbNG4Wc3tRbyy/9PQWwFX1nlmavzjP8luALX3VI42zxdxe1BvLLz1eCSdJjRjAktSIASxJjRjAktSIASxJjRjAktSIASxJjRjAktSIASxJjfhQzh4s5rJSn2osyQDuwWIuK/WpxpIcgpCkRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRjwPWFqEZw8f5pzzL5xzvs9r04kwgKVFOHr06LwX1/i8Np0IhyAkqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIa6S2Ak9ye5GCSR0fazk5yd5Kfdu9njcy7NcneJE8kua6vuiRpXPR5JdyXgP8BfHmk7Rbgnqq6Lckt3eePJLkU2ABcBpwPfC/Jm6rqSI/1SS8bL1XWiegtgKvqB0kufknzeuDqbnobcB/wka79jqp6HngyyV5gLfCjvuqTXk5eqqwTMfQY8LlVNQXQvZ/TtV8APDOy3L6uTZKWrHH5Ei6ztNWsCyabkuxKsuvQoUM9lyVJ/Rk6gA8kWQnQvR/s2vcBowNoq4D9s62gqrZW1WRVTU5MTPRarCT1aegA3gFs7KY3AneOtG9IcnqSS4DVwM6Ba5OkQfX2JVySrzPzhdsbkuwDPgbcBmxPciPwNHA9QFXtSbIdeAx4AbjJMyAkLXV9ngXxnjlmvXWO5bcAW/qqR5LGzbh8CSdJy44BLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1Eifj6Vfsi6/4kqmpqfnnP/s4cPDFSPplGUAn4Cp6el5H0H+jQ9cO2A1kk5VDkFIUiMGsCQ1YgBLUiMGsCQ1YgBLUiMGsCQ1YgBLUiOeBywN4NnDhznn/Avnna/lxwCehVe66eV29OhRL97RbzCAZ+GVbpKG4BiwJDViAEtSI02GIJI8BfwaOAK8UFWTSc4G/jdwMfAUcENVPduiPkkaQssj4N+tqjVVNdl9vgW4p6pWA/d0nyVpyRqnL+HWA1d309uA+4CPtCpGGjcLncq28rzzeHj3AwNWpJPVKoAL+JskBfxZVW0Fzq2qKYCqmkpyzmwdk2wCNgFcdNFFQ9UrNbfQqWz3br5hwGr0cmgVwFdV1f4uZO9O8pPFduzCeivA5ORkncjGPc9X0jhoEsBVtb97P5jk28Ba4ECSld3R70rgYF/b9zxfSeNg8C/hkrwmyWuPTQNvAx4FdgAbu8U2AncOXZskDanFEfC5wLeTHNv+16rqriQPANuT3Ag8DVzfoDZJGszgAVxVfw9cPkv7L4C3Dl2PJLXilXCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNjNPNeCSdBG/Wc+oxgKUlwpv1nHocgpCkRgxgSWrEAJakRhwDlpaJhb6kA3juuec488wz55zvF3kvLwNYWiYW+pIOZu6F/U6/yBuMQxCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNeDtKSYt2sg/+vPyKK5manp53G8vpnsNjF8BJ1gH/DVgBfKGqbmtckqTOyT74c2p6esF7En/zQ29bNk93HqsATrIC+J/AvwH2AQ8k2VFVj7WtTNJiLHSE/OzhwwuuYzk93XmsAhhYC+ytqr8HSHIHsB4wgKVTwELh+Y0PXHvS2zjZYRBYeChkqKPsVFXvG1msJH8ArKuq/9B9fi/wL6vq5pFlNgGbuo//HHhioPLeAPx8oG1Zw/jXAONRhzWMTw1nVNWbj6fDuB0BZ5a2f/QXoqq2AluHKedFSXZV1eTQ27WG8axhXOqwhvGq4Xj7jNtpaPuA0f9brAL2N6pFkno1bgH8ALA6ySVJXgVsAHY0rkmSejFWQxBV9UKSm4G/ZuY0tNurak/jso4ZfNhjFtYwYxxqgPGowxpmnJI1jNWXcJK0nIzbEIQkLRsGsCQ1YgAvIMlTSX6c5KETOc3kJLZ7e5KDSR4daTs7yd1Jftq9n9Wghs1Jftbtj4eSvL3nGi5M8v0kjyfZk+SDXftg+2KeGgbbF0nOSLIzycNdDR/v2ofcD3PVMOjPRLfNFUn+Lsl3us+D/m7MUcNx7wfHgBeQ5ClgsqoGPck7yb8GngO+fOzk7iR/Cvyyqm5LcgtwVlV9ZOAaNgPPVdUn+truS2pYCaysqt1JXgs8CLwLeB8D7Yt5ariBgfZFkgCvqarnkpwG/BD4IPD7DLcf5qphHQP+THS1fBiYBF5XVe8Y+ndjjho2c5z7wSPgMVVVPwB++ZLm9cC2bnobMyEwdA2DqqqpqtrdTf8aeBy4gAH3xTw1DKZmPNd9PK17FcPuh7lqGFSSVcDvAV8YaR70d2OOGo6bAbywAv4myYPdZdAtnVtVUzATCsA5jeq4Ockj3RBF7//VOybJxcBbgPtptC9eUgMMuC+6//I+BBwE7q6qwffDHDXAsD8TnwH+BDg60jb0z8NsNcBx7gcDeGFXVdUVwL8Fbur+W76cfQ54I7AGmAI+OcRGk5wJfBP4UFX9aohtLqKGQfdFVR2pqjXMXCG6Nslx3XegxxoG2w9J3gEcrKoH+9rGSdRw3PvBAF5AVe3v3g8C32bmjm2tHOjGI4+NSx4cuoCqOtD9Eh4FPs8A+6Mbb/wm8NWq+lbXPOi+mK2GFvui2+5h4D5mxl6b/EyM1jDwfrgKeGf33cwdwDVJ/pxh98OsNZzIfjCA55HkNd2XLiR5DfA24NH5e/VqB7Cxm94I3Dl0Acd+yDvvpuf90X3x80Xg8ar61MiswfbFXDUMuS+STCR5fTf9auBa4CcMux9mrWHI/VBVt1bVqqq6mJlbFdxbVX/IgPthrhpOZD+M1aXIY+hc4Nszv3+8EvhaVd01xIaTfB24GnhDkn3Ax4DbgO1JbgSeBq5vUMPVSdYwMzb+FPD+Pmtg5mjjvcCPu7FHgI8y7L6Yq4b3DLgvVgLbMvPQglcA26vqO0l+xHD7Ya4avjLwz8RsBv3dmMOfHu9+8DQ0SWrEIQhJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJauT/A6/A2u9YK+jOAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "tweets_lens = [len(n) for n in input_encoder]\n", "tweets_lens\n", "\n", "import seaborn as sns\n", "sns.displot(tweets_lens)" ] }, { "cell_type": "markdown", "metadata": { "id": "XJk2VTUkaJIM" }, "source": [ "Utilicemos entonces:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "u-2T5o8Rdit9" }, "outputs": [], "source": [ "max_seq_len = 50" ] }, { "cell_type": "markdown", "metadata": { "id": "5Q2zBOXUdit6" }, "source": [ "La siguiente clase PadSequenceTransformer es un modulo que les preparé para simplificar este procesamiento. El mismo se encarga de ajustar cualquier secuencia para que tenga exactamente max_seq_len. Cuando la lingitud es mejor, se completan con ceros." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "NeVilfLMdit7" }, "outputs": [], "source": [ "from Utils.PadSequenceTransformer import PadSequenceTransformer" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "BpfZiKM_dit_" }, "outputs": [], "source": [ "padder = PadSequenceTransformer(max_len=max_seq_len, padding='post')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3r1DOM6oaJIN" }, "outputs": [], "source": [ "input_encoder_padded = padder.transform(input_encoder)\n", "input_decoder_padded = padder.transform(input_decoder)\n", "target_decoder_padded = padder.transform(target_decoder)" ] }, { "cell_type": "markdown", "metadata": { "id": "EdWHktR6aJIN" }, "source": [ "Convertimos nuestra entrada a arreglos de tipo `numpy`" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Uxd_eGn2aJIO" }, "outputs": [], "source": [ "from sklearn.preprocessing import LabelEncoder\n", "\n", "label_encoder = LabelEncoder()\n", "label_encoder = label_encoder.fit(tweets[\"SECTOR\"])\n", "number_classes = len(label_encoder.classes_)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Z4Bnc5z-aJIO" }, "outputs": [], "source": [ "import numpy as np\n", "\n", "input_encoder_np = np.array(input_encoder_padded).astype('int32')\n", "input_decoder_np = np.array(input_decoder_padded).astype('int32')\n", "target_decoder_np = np.array(target_decoder_padded).astype('int32')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "uTJPDRELaJIP" }, "outputs": [], "source": [ "topic_decoder = label_encoder.transform(tweets[\"SECTOR\"])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "1aoce4Q1aJIR" }, "outputs": [], "source": [ "import tensorflow.keras as keras\n", "\n", "topic_decoder_np = keras.utils.to_categorical(topic_decoder).astype('int32')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "7zmIdO5faJIS", "outputId": "9457c19d-17af-486b-db48-0468bd0ac562" }, "outputs": [ { "data": { "text/plain": [ "(3763, 7)" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "topic_decoder_np.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "1t0qsd0aaJIS" }, "source": [ "### Creando conjuntos de entrenamiento y validación" ] }, { "cell_type": "markdown", "metadata": { "id": "Ez7Gb5IkaJIS" }, "source": [ "Crearemos 2 conjuntos de datos para entrenar y validar" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LDQAWg04aJIT" }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "\n", "idx_train, idx_test = train_test_split(range(len(input_encoder_np)),\n", " test_size=0.33,\n", " stratify=tweets['SECTOR'])" ] }, { "cell_type": "markdown", "metadata": { "id": "KCS2cM1KaJIT" }, "source": [ "Necesitamos la variable de salida codificada también en forma de indices:" ] }, { "cell_type": "markdown", "metadata": { "id": "JvPFTyIYaJIU" }, "source": [ "Creamos los conjuntos de datos:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "x_Wy56AEaJIU" }, "outputs": [], "source": [ "from tensorflow.keras.utils import to_categorical\n", "\n", "input_encoder_train = to_categorical(np.array(input_encoder_np[idx_train]), num_classes=vocab_size)\n", "input_decoder_train = to_categorical(np.array(input_decoder_np[idx_train]), num_classes=vocab_size)\n", "target_decoder_train = to_categorical(np.array(target_decoder_np[idx_train]), num_classes=vocab_size)\n", "#input_decoder_train = np.array(label_encoder.transform(tweets[\"SECTOR\"][idx_train]))\n", "\n", "input_encoder_test = to_categorical(np.array(input_encoder_np[idx_test]), num_classes=vocab_size)\n", "input_decoder_test = to_categorical(np.array(input_decoder_np[idx_test]), num_classes=vocab_size)\n", "target_decoder_test = np.array(input_decoder_np[idx_test]), num_classes=vocab_size)\n", "#input_decoder_test = np.array(label_encoder.transform(tweets[\"SECTOR\"][idx_test]))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "lh1LzyhJaJIU", "outputId": "b3b96ed7-8e90-4457-ed05-cc03643285fb" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "TRAIN\n", "input_encoder_train: (2521, 50, 9999)\n", "target_decoder_train: (2521, 50, 9999)\n", "input_decoder_train: (2521, 50, 9999)\n", "\n", "TEST\n", "input_encoder_test: (1242, 50, 9999)\n", "target_decoder_test: (1242, 50, 9999)\n", "input_decoder_test: (1242, 50, 9999)\n" ] } ], "source": [ "print(\"TRAIN\")\n", "print(\"input_encoder_train:\", input_encoder_train.shape)\n", "print(\"target_decoder_train:\", target_decoder_train.shape)\n", "print(\"input_decoder_train:\", input_decoder_train.shape)\n", "print(\"\\nTEST\")\n", "print(\"input_encoder_test:\", input_encoder_test.shape)\n", "print(\"target_decoder_test:\", target_decoder_test.shape)\n", "print(\"input_decoder_test:\", input_decoder_test.shape)" ] }, { "cell_type": "markdown", "metadata": { "id": "cq6UgdfqaJIV" }, "source": [ "### Construyendo el modelo" ] }, { "cell_type": "markdown", "metadata": { "id": "lnhHx7Pddito" }, "source": [ "Para construir nuestro modelo, utilizaremos TensorFlow. En particular utilizaremos la API de Keras que nos permite componer modelos de redes neuronales como una secuencia de pasos o capas que se conectan en una dirección.\n", "\n", "Utilizemos los siguientes tipos de capas:\n", " * **Embedding:** Esta capa transforma vectores que representan indices dentro de una matriz en representaciones vectoriales densas. Básicamente en este caso nos resolverá la busqueda de las representaciones vectoriales para nuestras palabras. **Intentaremos aprender embdeddings de tamaño 100**\n", " * **SpatialDropout1D:** Este tipo de capas ayudan a promover la independencia entre filtros (feature maps). Funciona en forma analoga a Dropout pero en lugar de desconectar elementos individuales, desconecta el filtro completo.\n", " * **LSTM:** Long Short-Term Memory layer - Hochreiter 1997\n", " * **Dense:** Una típica capa de una red neuronal completamente conectada (fully connected)\n", "\n", "Algunos detalles para notar:\n", " * *loss='sparse_categorical_crossentropy'*, este problema de clasificación (crossentropy) de más de una clase (categorical). Sin embargo, nuestro output produce probabilidades de cada una de las clases posibles (7) en forma one-hot encoding.\n", " * *metrics=['accuracy']*: Si bien nuestra metrica es accuracy, Keras hará un promedio ponderado del accuracy de cada clase. Este es el comportamiento por defecto." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "iZzrBjtoaJIV" }, "outputs": [], "source": [ "emdedding_size = 100" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LQ2C980jditp" }, "outputs": [], "source": [ "import tensorflow as tf\n", "import tensorflow.keras as keras\n", "from tensorflow.keras.models import Sequential, Model\n", "from tensorflow.keras.layers import Embedding, Dense, TimeDistributed, LSTM, Input, GRU" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Mt2jya90aJIW", "outputId": "c393060e-9007-4f60-9720-e30e06542c95" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2021-09-30 16:58:53.771372: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory\n", "2021-09-30 16:58:53.775480: E tensorflow/stream_executor/cuda/cuda_driver.cc:313] failed call to cuInit: UNKNOWN ERROR (303)\n", "2021-09-30 16:58:53.776271: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (laptop): /proc/driver/nvidia/version does not exist\n", "2021-09-30 16:58:53.794306: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA\n", "2021-09-30 16:58:54.054306: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 1497600000 Hz\n", "2021-09-30 16:58:54.057681: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f55b4000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices:\n", "2021-09-30 16:58:54.057701: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version\n", "2021-09-30 16:58:54.172786: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 15998400 exceeds 10% of free system memory.\n", "2021-09-30 16:58:54.190806: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 15998400 exceeds 10% of free system memory.\n", "2021-09-30 16:58:54.199669: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 15998400 exceeds 10% of free system memory.\n", "2021-09-30 16:58:54.946643: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 15998400 exceeds 10% of free system memory.\n", "2021-09-30 16:58:54.959402: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 15998400 exceeds 10% of free system memory.\n" ] } ], "source": [ "# Define an input sequence and process it.\n", "encoder_inputs = Input(shape=(max_seq_len, vocab_size), name=\"sequence_input\")\n", "#embedder = Embedding(input_dim=vocab_size ,input_length=max_seq_len, output_dim=emdedding_size, mask_zero=True)\n", "encoder = LSTM(emdedding_size, return_state=True, name=\"encoder\")\n", "\n", "#encoder_embeddings = embedder(encoder_inputs)\n", "encoder_outputs, state_h, state_c = encoder(encoder_inputs)\n", "\n", "encoder_states = [state_h, state_c]\n", "\n", "# Set up the decoder, using `encoder_states` as initial state.\n", "decoder_inputs = Input(shape=(max_seq_len, vocab_size))\n", "#decoder_embeddings = embedder(decoder_inputs)\n", "\n", "# We set up our decoder to return full output sequences,\n", "# and to return internal states as well. We don't use the\n", "# return states in the training model, but we will use them in inference.\n", "decoder_lstm = LSTM(emdedding_size, return_sequences=True, return_state=True, name=\"decoder\")\n", "decoder_outputs, _, _ = decoder_lstm(decoder_inputs,\n", " initial_state=encoder_states)\n", "decoder_dense = Dense(vocab_size, activation='softmax')\n", "decoder_outputs = decoder_dense(decoder_outputs)\n", "\n", "# Define the model that will turn\n", "# `encoder_input_data` & `decoder_input_data` into `decoder_target_data`\n", "model = Model([encoder_inputs, decoder_inputs], decoder_outputs)\n", "model.compile(optimizer='rmsprop', loss='categorical_crossentropy')" ] }, { "cell_type": "markdown", "metadata": { "id": "XWc2N0O-ditw" }, "source": [ "Podemos inspeccionar el modelo:" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "vmk-mnMzditw", "outputId": "6f6fc014-3de4-4228-ef82-d54396dff34f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Model: \"model\"\n", "__________________________________________________________________________________________________\n", "Layer (type) Output Shape Param # Connected to \n", "==================================================================================================\n", "sequence_input (InputLayer) [(None, 50, 9999)] 0 \n", "__________________________________________________________________________________________________\n", "input_1 (InputLayer) [(None, 50, 9999)] 0 \n", "__________________________________________________________________________________________________\n", "encoder (LSTM) [(None, 100), (None, 4040000 sequence_input[0][0] \n", "__________________________________________________________________________________________________\n", "decoder (LSTM) [(None, 50, 100), (N 4040000 input_1[0][0] \n", " encoder[0][1] \n", " encoder[0][2] \n", "__________________________________________________________________________________________________\n", "dense (Dense) (None, 50, 9999) 1009899 decoder[0][0] \n", "==================================================================================================\n", "Total params: 9,089,899\n", "Trainable params: 9,089,899\n", "Non-trainable params: 0\n", "__________________________________________________________________________________________________\n" ] } ], "source": [ "model.summary()" ] }, { "cell_type": "markdown", "metadata": { "id": "TfXitOtraJIX" }, "source": [ "sudo apt install graphviz" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "vhis2NQjaJIY", "outputId": "31ef8e45-0c8a-4f3b-f824-3a3dc7af5214" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.\n" ] } ], "source": [ "keras.utils.plot_model(model, show_shapes=True)" ] }, { "cell_type": "markdown", "metadata": { "id": "VpwiD4GGdiuB" }, "source": [ "### Entrenamiento" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "gb-Tk_nkaJIZ" }, "outputs": [], "source": [ "history = model.fit([input_encoder_train, input_decoder_train],\n", " target_decoder_train,\n", " batch_size=124,\n", " epochs=20,\n", " validation_split=0.2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "3vpoVj0iaJIa" }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "id": "webM_rFCdiuD" }, "source": [ "Entrenamos nuestro modelo" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 1000 }, "id": "sJYPslBwdiuE", "outputId": "3c5e3f32-ec0c-4432-d03a-fe9b45fc66dc" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch 1/20\n", "21/21 [==============================] - 3s 136ms/step - loss: 1.9097 - accuracy: 0.3610 - val_loss: 1.8371 - val_accuracy: 0.4130\n", "Epoch 2/20\n", "21/21 [==============================] - 2s 116ms/step - loss: 1.6095 - accuracy: 0.5422 - val_loss: 1.3369 - val_accuracy: 0.6498\n", "Epoch 3/20\n", "21/21 [==============================] - 3s 120ms/step - loss: 0.8766 - accuracy: 0.8183 - val_loss: 0.8414 - val_accuracy: 0.8229\n", "Epoch 4/20\n", "21/21 [==============================] - 2s 103ms/step - loss: 0.3740 - accuracy: 0.9576 - val_loss: 0.6677 - val_accuracy: 0.8567\n", "Epoch 5/20\n", "21/21 [==============================] - 2s 118ms/step - loss: 0.1883 - accuracy: 0.9829 - val_loss: 0.6239 - val_accuracy: 0.8663\n", "Epoch 6/20\n", "21/21 [==============================] - 3s 121ms/step - loss: 0.1007 - accuracy: 0.9929 - val_loss: 0.6850 - val_accuracy: 0.8680\n", "Epoch 7/20\n", "21/21 [==============================] - 3s 119ms/step - loss: 0.0683 - accuracy: 0.9933 - val_loss: 0.7685 - val_accuracy: 0.8583\n", "Epoch 8/20\n", "21/21 [==============================] - 2s 113ms/step - loss: 0.0538 - accuracy: 0.9948 - val_loss: 0.6088 - val_accuracy: 0.8784\n", "Epoch 9/20\n", "21/21 [==============================] - 2s 117ms/step - loss: 0.0409 - accuracy: 0.9976 - val_loss: 0.6699 - val_accuracy: 0.8760\n", "Epoch 10/20\n", "21/21 [==============================] - 3s 123ms/step - loss: 0.0331 - accuracy: 0.9984 - val_loss: 0.6897 - val_accuracy: 0.8680\n", "Epoch 11/20\n", "21/21 [==============================] - 2s 118ms/step - loss: 0.0270 - accuracy: 0.9988 - val_loss: 0.7222 - val_accuracy: 0.8688\n", "Epoch 12/20\n", "21/21 [==============================] - 3s 120ms/step - loss: 0.0230 - accuracy: 0.9988 - val_loss: 0.7491 - val_accuracy: 0.8663\n", "Epoch 13/20\n", "21/21 [==============================] - 3s 121ms/step - loss: 0.0206 - accuracy: 0.9988 - val_loss: 0.7233 - val_accuracy: 0.8688\n", "Epoch 14/20\n", "21/21 [==============================] - 2s 114ms/step - loss: 0.0173 - accuracy: 0.9992 - val_loss: 0.7296 - val_accuracy: 0.8712\n", "Epoch 15/20\n", "21/21 [==============================] - 3s 124ms/step - loss: 0.0155 - accuracy: 0.9988 - val_loss: 0.7691 - val_accuracy: 0.8680\n", "Epoch 16/20\n", "21/21 [==============================] - 2s 119ms/step - loss: 0.0158 - accuracy: 0.9988 - val_loss: 0.7888 - val_accuracy: 0.8663\n", "Epoch 17/20\n", "21/21 [==============================] - 2s 116ms/step - loss: 0.0147 - accuracy: 0.9984 - val_loss: 0.7678 - val_accuracy: 0.8655\n", "Epoch 18/20\n", "21/21 [==============================] - 2s 119ms/step - loss: 0.0145 - accuracy: 0.9988 - val_loss: 0.7902 - val_accuracy: 0.8623\n", "Epoch 19/20\n", "21/21 [==============================] - 3s 120ms/step - loss: 0.0158 - accuracy: 0.9980 - val_loss: 0.7454 - val_accuracy: 0.8704\n", "Epoch 20/20\n", "21/21 [==============================] - 3s 120ms/step - loss: 0.0131 - accuracy: 0.9988 - val_loss: 0.7306 - val_accuracy: 0.8736\n" ] } ], "source": [ "history = model.fit(X_train,\n", " y_train,\n", " batch_size=124,\n", " epochs=20,\n", " validation_data=(X_test, y_test))" ] }, { "cell_type": "markdown", "metadata": { "id": "D3DtoSNTdiuJ" }, "source": [ "## Evalución de los resultados" ] }, { "cell_type": "markdown", "metadata": { "id": "aFmZCv5KdiuK" }, "source": [ "Probamos su performance utilizando el test set" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "kqM4G2BVdiuL" }, "outputs": [], "source": [ "predictions = model.predict(X_test).argmax(axis=-1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 255 }, "id": "XSIlnu1zdiuN", "outputId": "1a80c583-9afc-434c-eedf-33f36c0846cf" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " precision recall f1-score support\n", "\n", "ALIMENTACION 0.93 0.89 0.91 110\n", " AUTOMOCION 0.84 0.89 0.86 148\n", " BANCA 0.92 0.84 0.88 198\n", " BEBIDAS 0.84 0.85 0.84 223\n", " DEPORTES 0.97 0.90 0.93 216\n", " RETAIL 0.85 0.90 0.88 268\n", " TELCO 0.72 0.82 0.77 79\n", "\n", " accuracy 0.87 1242\n", " macro avg 0.87 0.87 0.87 1242\n", "weighted avg 0.88 0.87 0.87 1242\n", "\n" ] } ], "source": [ "from sklearn.metrics import classification_report\n", "\n", "print(classification_report(y_test, predictions, target_names=label_encoder.classes_))" ] } ], "metadata": { "colab": { "name": "Word2Vec - Secuencias.ipynb", "provenance": [], "toc_visible": true }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.2" } }, "nbformat": 4, "nbformat_minor": 0 }