Week_5_Assignment.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.5"
    },
    "colab": {
      "name": "Copy of Week5_Project.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ofzy8xHIXdQF",
        "colab_type": "text"
      },
      "source": [
        "[Bag of Words Meets Bags of Popcorn](https://www.kaggle.com/c/word2vec-nlp-tutorial/data)\n",
        "======\n",
        "\n",
        "## Data Set\n",
        "\n",
        "The labeled data set consists of 50,000 IMDB movie reviews, specially selected for sentiment analysis. The sentiment of reviews is binary, meaning the IMDB rating < 5 results in a sentiment score of 0, and rating >=7 have a sentiment score of 1. No individual movie has more than 30 reviews. The 25,000 review labeled training set does not include any of the same movies as the 25,000 review test set. In addition, there are another 50,000 IMDB reviews provided without any rating labels.\n",
        "\n",
        "## File descriptions\n",
        "\n",
        "labeledTrainData - The labeled training set. The file is tab-delimited and has a header row followed by 25,000 rows containing an id, sentiment, and text for each review.\n",
        "## Data fields\n",
        "\n",
        "* id - Unique ID of each review\n",
        "* sentiment - Sentiment of the review; 1 for positive reviews and 0 for negative reviews\n",
        "* review - Text of the review\n",
        "\n",
        "## Objective\n",
        "Objective of this dataset is base on **review** we predict **sentiment** (positive or negative) so X is **review** column and y is **sentiment** column"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dub6uxUzXdQH",
        "colab_type": "text"
      },
      "source": [
        "## 1. Load Dataset\n",
        "\n",
        "Let's first of all have a look at the data. You can download the file `labeledTrainData.tsv` on the [Kaggle website of the competition](https://www.kaggle.com/c/word2vec-nlp-tutorial/data), or on our [Google Drive](https://drive.google.com/file/d/1a1Lyn7ihikk3klAX26fgO3YsGdWHWoK5/view?usp=sharing)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "y1WU9XkYXdQI",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pandas as pd\n",
        "import numpy as np"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_iH9rAw7XdQL",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 340
        },
        "outputId": "cfcb87cb-b4e8-4172-8913-2bc1a1bbdd87"
      },
      "source": [
        "train_data = pd.read_csv('labeledTrainData.tsv', sep=\"\\t\")\n",
        "test_data = pd.read_csv('testData.tsv', sep=\"\\t\")\n",
        "train_data.info()\n",
        "test_data.info()"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 25000 entries, 0 to 24999\n",
            "Data columns (total 3 columns):\n",
            " #   Column     Non-Null Count  Dtype \n",
            "---  ------     --------------  ----- \n",
            " 0   id         25000 non-null  object\n",
            " 1   sentiment  25000 non-null  int64 \n",
            " 2   review     25000 non-null  object\n",
            "dtypes: int64(1), object(2)\n",
            "memory usage: 586.1+ KB\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 25000 entries, 0 to 24999\n",
            "Data columns (total 2 columns):\n",
            " #   Column  Non-Null Count  Dtype \n",
            "---  ------  --------------  ----- \n",
            " 0   id      25000 non-null  object\n",
            " 1   review  25000 non-null  object\n",
            "dtypes: object(2)\n",
            "memory usage: 390.8+ KB\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bbuaIdhWXdQO",
        "colab_type": "text"
      },
      "source": [
        "## 2. Preprocessing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wJr6juwHXdQP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "cb0d4f69-f8dd-4939-d6d1-03a0f499f8ac"
      },
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "from nltk.corpus import stopwords\n",
        "stop_words = stopwords.words('english')"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3v220Tp_XdQS",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Removing special characters and \"trash\"\n",
        "import re\n",
        "\n",
        "def preprocessor(text):\n",
        "    # Remove HTML markup\n",
        "    text = re.sub('<[^>]*>', '', text)\n",
        "    # Save emoticons for later appending\n",
        "    emoticons = re.findall('(?::|;|=)(?:-)?(?:\\)|\\(|D|P)', text)\n",
        "    # Remove any non-word character and append the emoticons,\n",
        "    # removing the nose character for standarization. Convert to lower case\n",
        "    text = (re.sub('[\\W]+', ' ', text.lower()) + ' ' + ' '.join(emoticons).replace('-', ''))\n",
        "    \n",
        "    return text"
      ],
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-o5ED67LXdQV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "87af88ee-48d5-4d7c-ce65-fae1bbb0594a"
      },
      "source": [
        "from nltk.stem import PorterStemmer\n",
        "\n",
        "porter = PorterStemmer()\n",
        "\n",
        "# Split a text into list of words\n",
        "def tokenizer(text):\n",
        "    return text.split()\n",
        "\n",
        "# Split a text into list of words and apply stemming technic\n",
        "def tokenizer_porter(text):\n",
        "    return [porter.stem(word) for word in text.split()]\n",
        "\n",
        "# Testing\n",
        "print(tokenizer('Hi there, I am loving this, like with a lot of love'))\n",
        "print(tokenizer_porter('Hi there, I am loving this, like with a lot of love'))"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "['Hi', 'there,', 'I', 'am', 'loving', 'this,', 'like', 'with', 'a', 'lot', 'of', 'love']\n",
            "['Hi', 'there,', 'I', 'am', 'love', 'this,', 'like', 'with', 'a', 'lot', 'of', 'love']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zjzOxu4MXdQZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# split the dataset in train and test\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "X = train_data['review']\n",
        "y = train_data['sentiment']\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=102)"
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "COr1xR7PXdQc",
        "colab_type": "text"
      },
      "source": [
        "## 3. Create Model and Train \n",
        "\n",
        "Using **Pipeline** to concat **tfidf** step and **LogisticRegression** step"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tOpwINJmXdQd",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 462
        },
        "outputId": "a35b8670-582b-49ca-b205-ae11a8bc2a11"
      },
      "source": [
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "tfidf = TfidfVectorizer(stop_words=stop_words,\n",
        "                        tokenizer=tokenizer_porter,\n",
        "                        preprocessor=preprocessor)\n",
        "\n",
        "# A pipeline is what chains several steps together, once the initial exploration is done. \n",
        "# For example, some codes are meant to transform features — normalise numericals, or turn text into vectors, \n",
        "# or fill up missing data, they are transformers; other codes are meant to predict variables by fitting an algorithm,\n",
        "# they are estimators. Pipeline chains all these together which can then be applied to training data\n",
        "clf = Pipeline([('vect', tfidf),\n",
        "                ('clf', LogisticRegression(random_state=0))])\n",
        "clf.fit(X_train, y_train)"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/feature_extraction/text.py:385: UserWarning: Your stop_words may be inconsistent with your preprocessing. Tokenizing the stop words generated tokens ['abov', 'ani', 'becaus', 'befor', 'doe', 'dure', 'ha', 'hi', 'onc', 'onli', 'ourselv', 'themselv', 'thi', 'veri', 'wa', 'whi', 'yourselv'] not in stop_words.\n",
            "  'stop_words.' % sorted(inconsistent))\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(memory=None,\n",
              "         steps=[('vect',\n",
              "                 TfidfVectorizer(analyzer='word', binary=False,\n",
              "                                 decode_error='strict',\n",
              "                                 dtype=<class 'numpy.float64'>,\n",
              "                                 encoding='utf-8', input='content',\n",
              "                                 lowercase=True, max_df=1.0, max_features=None,\n",
              "                                 min_df=1, ngram_range=(1, 1), norm='l2',\n",
              "                                 preprocessor=<function preprocessor at 0x7f72ab5e3d90>,\n",
              "                                 smooth_idf=True,\n",
              "                                 stop_words=['i', 'me', 'my', 'myself', '...\n",
              "                                 token_pattern='(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
              "                                 tokenizer=<function tokenizer_porter at 0x7f72bcbc7e18>,\n",
              "                                 use_idf=True, vocabulary=None)),\n",
              "                ('clf',\n",
              "                 LogisticRegression(C=1.0, class_weight=None, dual=False,\n",
              "                                    fit_intercept=True, intercept_scaling=1,\n",
              "                                    l1_ratio=None, max_iter=100,\n",
              "                                    multi_class='auto', n_jobs=None,\n",
              "                                    penalty='l2', random_state=0,\n",
              "                                    solver='lbfgs', tol=0.0001, verbose=0,\n",
              "                                    warm_start=False))],\n",
              "         verbose=False)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KYdoVMx_XdQf",
        "colab_type": "text"
      },
      "source": [
        "## 4. Evaluate Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z2ZOzHoaXdQg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 255
        },
        "outputId": "5c3f1d38-c0da-492d-b818-18c9476d0b50"
      },
      "source": [
        "# Using Test dataset to evaluate model\n",
        "# classification_report\n",
        "# confusion matrix\n",
        "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n",
        "\n",
        "predictions = clf.predict(X_test)\n",
        "print('accuracy:',accuracy_score(y_test,predictions))\n",
        "print('confusion matrix:\\n',confusion_matrix(y_test,predictions))\n",
        "print('classification report:\\n',classification_report(y_test,predictions))\n"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "accuracy: 0.8852\n",
            "confusion matrix:\n",
            " [[2164  339]\n",
            " [ 235 2262]]\n",
            "classification report:\n",
            "               precision    recall  f1-score   support\n",
            "\n",
            "           0       0.90      0.86      0.88      2503\n",
            "           1       0.87      0.91      0.89      2497\n",
            "\n",
            "    accuracy                           0.89      5000\n",
            "   macro avg       0.89      0.89      0.89      5000\n",
            "weighted avg       0.89      0.89      0.89      5000\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NCAuLC0aXdQi",
        "colab_type": "text"
      },
      "source": [
        "## 5. Export Model "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sYHo_x67XdQj",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Using pickle to export our trained model\n",
        "x_real_test = test_data['review']\n",
        "y_real_test = clf.predict(x_real_test)\n",
        "test_data['sentiment'] = y_real_test\n",
        "test_data = test_data[['id', 'sentiment']]\n",
        "test_data.to_csv('result.csv', index=False)"
      ],
      "execution_count": 11,
      "outputs": []
    }
  ]
}