2_7_3_Abstractive_Text_Summarization_using_BART.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# 📚 **Project Overview: Text Summarization with BART and Flask Deployment**\n",
        "\n",
        "## 🎯 **Objective**\n",
        "This project focuses on building a text summarization model using the pre-trained **BART** model. The model is fine-tuned on the CNN/Daily Mail dataset for summarization tasks, and a Flask web API is developed to deploy the model for real-time text summarization.\n",
        "\n",
        "## 🧩 **Components**\n",
        "\n",
        "### 1. **Dataset**\n",
        "The **CNN/Daily Mail** dataset is used, consisting of articles and highlights (summaries). The dataset is preprocessed, tokenized, and split into inputs (articles) and labels (summaries) for training.\n",
        "\n",
        "- **Dataset Structure**:\n",
        "  - `article`: Textual content to be summarized.\n",
        "  - `highlights`: Summarized content.\n",
        "\n",
        "### 2. **Text Preprocessing**\n",
        "The **BART tokenizer** from the `transformers` library is used to tokenize the input articles and summaries. The preprocessing steps include:\n",
        "- Tokenizing the articles and summaries.\n",
        "- Truncating and padding the sequences to fit the model's maximum length.\n",
        "\n",
        "### 3. **Model Training**\n",
        "The **BART model** is fine-tuned for the task of **text summarization**:\n",
        "- **Model**: `facebook/bart-large-cnn`\n",
        "- **Optimizer**: AdamW with a learning rate of `5e-5`\n",
        "- **Batch Size**: 8\n",
        "- **Epochs**: 3\n",
        "\n",
        "The model is trained using **PyTorch**, with the option to run on **GPU** if available for faster training.\n",
        "\n",
        "### 4. **Model Evaluation**\n",
        "After training, the model's performance is evaluated by generating summaries for the first few articles in the dataset. The evaluation metric is the quality of generated summaries.\n",
        "\n",
        "- **Summarization Process**:\n",
        "  - Text is tokenized.\n",
        "  - The model generates summaries using beam search for optimal output.\n",
        "\n",
        "### 5. **Model Deployment with Flask**\n",
        "A **Flask web application** is developed to deploy the trained summarization model:\n",
        "- **Route**: `/summarize`\n",
        "- **Method**: `POST`\n",
        "- **Input**: Raw text to be summarized (in JSON format).\n",
        "- **Output**: Summarized text in JSON format.\n",
        "\n",
        "### 6. **Model Saving**\n",
        "The fine-tuned model and tokenizer are saved for later use:\n",
        "- **Saved Files**: Model weights and tokenizer files.\n",
        "- **Directory**: `summarization_model`\n",
        "\n",
        "## ⚙️ **Technologies Used**\n",
        "- **Libraries**:\n",
        "  - `transformers`: For the pre-trained BART model and tokenizer.\n",
        "  - `torch`: For model training with PyTorch.\n",
        "  - `flask`: For creating the web API.\n",
        "  - `datasets`: For easy access to datasets like CNN/Daily Mail.\n",
        "- **Tools**:\n",
        "  - **Google Colab**: For model training and experimentation.\n",
        "  - **GitHub**: For version control and project management.\n",
        "\n",
        "## 🚀 **Execution Steps**\n",
        "1. **Install necessary libraries** using `pip`.\n",
        "2. **Load and preprocess the dataset** for text tokenization.\n",
        "3. **Fine-tune the BART model** on the dataset.\n",
        "4. **Evaluate the model** by generating summaries.\n",
        "5. **Deploy the model** using Flask to create a web API.\n",
        "6. **Save the model** for future use.\n",
        "\n",
        "## 🎓 **Educational Objective**\n",
        "This project is designed to demonstrate the process of **text summarization** using **transformer-based models** like **BART**. Additionally, it introduces the deployment of machine learning models using **Flask**, allowing the model to be accessed via a simple web interface. This hands-on project is ideal for learners who wish to apply NLP techniques and model deployment in real-world scenarios.\n",
        "\n",
        "## 📜 **Future Improvements**\n",
        "- Experimenting with different models (e.g., T5, GPT-3).\n",
        "- Optimizing model performance using hyperparameter tuning.\n",
        "- Extending the Flask app to support batch processing or multiple summarization models."
      ],
      "metadata": {
        "id": "-lQ9J8xpEAzg"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "X-YhTA9ZEABy"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "3vL7yWehA5Jf"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Et5zdq_oEEr3"
      },
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "YBafr99LBG-2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ⚙️ **Install Necessary Libraries**\n",
        "To begin, you'll need to install the required libraries for text summarization and web deployment. You can install them using `pip`. The essential libraries include:\n",
        "\n",
        "- **Transformers**: For using pre-trained models like BART.\n",
        "- **Torch**: For model training and deep learning operations.\n",
        "- **Flask**: To create a simple web API for deployment.\n",
        "- **Datasets**: To load and preprocess datasets such as CNN/Daily Mail.\n",
        "\n",
        "```bash\n",
        "!pip install transformers torch flask datasets"
      ],
      "metadata": {
        "id": "jYxN6-6GE3Kj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Install required libraries"
      ],
      "metadata": {
        "id": "bGxbh1bdA5ML"
      },
      "execution_count": 38,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install datasets transformers torch flask  # Installing necessary libraries for datasets, transformers, torch, and flask\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DrEyFni2A5Om",
        "outputId": "8b6dc5d8-c3ea-45c0-872b-013139188cf4"
      },
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: datasets in /usr/local/lib/python3.10/dist-packages (3.1.0)\n",
            "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (4.44.2)\n",
            "Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.5.0+cu121)\n",
            "Requirement already satisfied: flask in /usr/local/lib/python3.10/dist-packages (2.2.5)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from datasets) (3.16.1)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.26.4)\n",
            "Requirement already satisfied: pyarrow>=15.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (17.0.0)\n",
            "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.3.8)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (2.2.2)\n",
            "Requirement already satisfied: requests>=2.32.2 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.32.3)\n",
            "Requirement already satisfied: tqdm>=4.66.3 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.6)\n",
            "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.5.0)\n",
            "Requirement already satisfied: multiprocess<0.70.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.70.16)\n",
            "Requirement already satisfied: fsspec<=2024.9.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from fsspec[http]<=2024.9.0,>=2023.1.0->datasets) (2024.9.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.10.10)\n",
            "Requirement already satisfied: huggingface-hub>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.24.7)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (24.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.2)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers) (2024.9.11)\n",
            "Requirement already satisfied: safetensors>=0.4.1 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.4.5)\n",
            "Requirement already satisfied: tokenizers<0.20,>=0.19 in /usr/local/lib/python3.10/dist-packages (from transformers) (0.19.1)\n",
            "Requirement already satisfied: typing-extensions>=4.8.0 in /usr/local/lib/python3.10/dist-packages (from torch) (4.12.2)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.4.2)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.4)\n",
            "Requirement already satisfied: sympy==1.13.1 in /usr/local/lib/python3.10/dist-packages (from torch) (1.13.1)\n",
            "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy==1.13.1->torch) (1.3.0)\n",
            "Requirement already satisfied: Werkzeug>=2.2.2 in /usr/local/lib/python3.10/dist-packages (from flask) (3.0.6)\n",
            "Requirement already satisfied: itsdangerous>=2.0 in /usr/local/lib/python3.10/dist-packages (from flask) (2.2.0)\n",
            "Requirement already satisfied: click>=8.0 in /usr/local/lib/python3.10/dist-packages (from flask) (8.1.7)\n",
            "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.4.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (24.2.0)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.5.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.1.0)\n",
            "Requirement already satisfied: yarl<2.0,>=1.12.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.17.0)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (3.0.2)\n",
            "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.4.0)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (3.10)\n",
            "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2.2.3)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.32.2->datasets) (2024.8.30)\n",
            "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
            "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2024.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n",
            "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.10/dist-packages (from yarl<2.0,>=1.12.0->aiohttp->datasets) (0.2.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "MQ-j7kmnBNy8"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "cjC_IzTkBOfz"
      },
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "6c44FiPSGG30"
      },
      "execution_count": 39,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import warnings  # Import warnings module to handle warning messages"
      ],
      "metadata": {
        "id": "3XwTo11bGG8q"
      },
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "warnings.filterwarnings('ignore')  # This will suppress all warnings during execution"
      ],
      "metadata": {
        "id": "pwvDAlJqGRFu"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "GHqxy6ypBOjA"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "sAhBMoa_BOl1"
      },
      "execution_count": 41,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "0bGXyO1jBPcs"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 🧑‍💻 **Step 2: Load and Preprocess the Dataset**\n",
        "\n",
        "```markdown\n",
        "## 📚 **Load and Preprocess the Dataset**\n",
        "The dataset used in this project is the **CNN/Daily Mail** dataset, which contains news articles along with their summaries (highlights). To work with the dataset, we will load it using the `datasets` library and preprocess it for tokenization.\n",
        "\n",
        "1. **Dataset Overview**:\n",
        "   - `article`: The full article text.\n",
        "   - `highlights`: The concise summary of the article.\n",
        "\n",
        "2. **Preprocessing Steps**:\n",
        "   - Load the dataset using the `datasets` library.\n",
        "   - Tokenize both the articles (inputs) and the summaries (labels) using the BART tokenizer.\n",
        "\n",
        "```python\n",
        "from datasets import load_dataset\n",
        "dataset = load_dataset('cnn_dailymail', '3.0.0')\n",
        "\n",
        "# Sample preprocessing function (you can adjust tokenization as needed)\n",
        "def preprocess_function(examples):\n",
        "    inputs = examples['article']\n",
        "    targets = examples['highlights']\n",
        "    model_inputs = tokenizer(inputs, max_length=1024, truncation=True, padding='max_length')\n",
        "    labels = tokenizer(targets, max_length=150, truncation=True, padding='max_length')\n",
        "    model_inputs['labels'] = labels['input_ids']\n",
        "    return model_inputs\n",
        "\n",
        "tokenized_datasets = dataset.map(preprocess_function, batched=True)"
      ],
      "metadata": {
        "id": "jHVvuRdrE7Bw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Load and preprocess the dataset"
      ],
      "metadata": {
        "id": "MrLulby4BQUz"
      },
      "execution_count": 42,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from datasets import load_dataset  # Importing load_dataset function from datasets library"
      ],
      "metadata": {
        "id": "D6Bg0AZ0BOq7"
      },
      "execution_count": 43,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd  # Importing pandas for data manipulation"
      ],
      "metadata": {
        "id": "y9R1_jKTBOtk"
      },
      "execution_count": 44,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "4pvxaefvA5RD"
      },
      "execution_count": 44,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the CNN/Daily Mail dataset (first 100 rows for faster processing)"
      ],
      "metadata": {
        "id": "p_hV1w_JA5Tc"
      },
      "execution_count": 45,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dataset = load_dataset(\"cnn_dailymail\", \"3.0.0\", split='train[:100]')  # Load the first 100 rows"
      ],
      "metadata": {
        "id": "moZFd_COA5cO"
      },
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "2aGNWh8ZA5e1"
      },
      "execution_count": 46,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert the dataset to a pandas dataframe for easier manipulation"
      ],
      "metadata": {
        "id": "_UndUunIA5hH"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.DataFrame(dataset)  # Converting the dataset to a pandas dataframe"
      ],
      "metadata": {
        "id": "BaCRJ_UuA5lH"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "zaLj1OR1Bl1d"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "5lYrF2iGBl4B"
      },
      "execution_count": 48,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Display first 5 rows"
      ],
      "metadata": {
        "id": "hZ2bI22OBl7H"
      },
      "execution_count": 49,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"First 5 rows of the dataset:\")  # Printing the first 5 rows of the dataset\n",
        "print(df.head())  # Displaying the first 5 rows of the dataframe"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NubEkzNABl-u",
        "outputId": "b5df9f97-3fec-4a41-ddbe-636e50db6bdd"
      },
      "execution_count": 50,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "First 5 rows of the dataset:\n",
            "                                             article  \\\n",
            "0  LONDON, England (Reuters) -- Harry Potter star...   \n",
            "1  Editor's note: In our Behind the Scenes series...   \n",
            "2  MINNEAPOLIS, Minnesota (CNN) -- Drivers who we...   \n",
            "3  WASHINGTON (CNN) -- Doctors removed five small...   \n",
            "4  (CNN)  -- The National Football League has ind...   \n",
            "\n",
            "                                          highlights  \\\n",
            "0  Harry Potter star Daniel Radcliffe gets £20M f...   \n",
            "1  Mentally ill inmates in Miami are housed on th...   \n",
            "2  NEW: \"I thought I was going to die,\" driver sa...   \n",
            "3  Five small polyps found during procedure; \"non...   \n",
            "4  NEW: NFL chief, Atlanta Falcons owner critical...   \n",
            "\n",
            "                                         id  \n",
            "0  42c027e4ff9730fbb3de84c1af0d2c506e41c3e4  \n",
            "1  ee8871b15c50d0db17b0179a6d2beab35065f1e9  \n",
            "2  06352019a19ae31e527f37f7571c6dd7f0c5da37  \n",
            "3  24521a2abb2e1f5e34e6824e0f9e56904a2b0e88  \n",
            "4  7fe70cc8b12fab2d0a258fababf7d9c6b5e1262a  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "yEx04Y7fBmLs"
      },
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "3q9xv48pBmT5"
      },
      "execution_count": 50,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Display dataset info"
      ],
      "metadata": {
        "id": "hCwBc3KgBmWi"
      },
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"\\nDataset Info:\")  # Printing dataset information\n",
        "print(df.info())  # Displaying dataset info (column types, non-null counts)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_oYkXJr-A5pk",
        "outputId": "2aaae899-16a5-425d-b718-b2d59ec8f9b7"
      },
      "execution_count": 52,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Dataset Info:\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 100 entries, 0 to 99\n",
            "Data columns (total 3 columns):\n",
            " #   Column      Non-Null Count  Dtype \n",
            "---  ------      --------------  ----- \n",
            " 0   article     100 non-null    object\n",
            " 1   highlights  100 non-null    object\n",
            " 2   id          100 non-null    object\n",
            "dtypes: object(3)\n",
            "memory usage: 2.5+ KB\n",
            "None\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "mO3EVS5QA5t8"
      },
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "QqfBV7c6A5yG"
      },
      "execution_count": 52,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Display summary statistics"
      ],
      "metadata": {
        "id": "afY_Z2roA50X"
      },
      "execution_count": 53,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"\\nSummary Statistics:\")  # Printing summary statistics\n",
        "print(df.describe())  # Displaying summary statistics for numerical columns"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "a1A5UKUt6j98",
        "outputId": "7b7670dc-a091-4ab2-cd1c-9f126a552f4f"
      },
      "execution_count": 54,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Summary Statistics:\n",
            "                                                  article  \\\n",
            "count                                                 100   \n",
            "unique                                                 99   \n",
            "top     AMMAN, Jordan (CNN) -- In the sunbathed school...   \n",
            "freq                                                    2   \n",
            "\n",
            "                                               highlights  \\\n",
            "count                                                 100   \n",
            "unique                                                 99   \n",
            "top     Jordan opens school doors to all Iraqi childre...   \n",
            "freq                                                    2   \n",
            "\n",
            "                                              id  \n",
            "count                                        100  \n",
            "unique                                       100  \n",
            "top     f16446db34e2861f0450dfa34d8cdda541ab7b19  \n",
            "freq                                           1  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "-JuFQsncBzAR"
      },
      "execution_count": 54,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "BJCylLVwBzDV"
      },
      "execution_count": 54,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Check for missing values"
      ],
      "metadata": {
        "id": "wmmpqb-OBzF6"
      },
      "execution_count": 55,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"\\nMissing Values:\")  # Printing missing value information\n",
        "print(df.isnull().sum())  # Summing the missing values in each column"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WxFgqrYxBzKU",
        "outputId": "f39c6106-4393-494a-dd99-e7b5422126d9"
      },
      "execution_count": 56,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Missing Values:\n",
            "article       0\n",
            "highlights    0\n",
            "id            0\n",
            "dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "HyY1dqlbBzNN"
      },
      "execution_count": 56,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "FYrs1LISBzQ0"
      },
      "execution_count": 56,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Check for duplicate rows"
      ],
      "metadata": {
        "id": "gZnf0ufEBzYc"
      },
      "execution_count": 57,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"\\nDuplicate Rows:\")  # Printing duplicate row information\n",
        "print(df.duplicated().sum())  # Summing the number of duplicate rows"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "coLqw5TY6Jdu",
        "outputId": "55a0279c-9f2c-43e1-e950-d3ed9f0a7f39"
      },
      "execution_count": 58,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Duplicate Rows:\n",
            "0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "w5RXd-J2B7ss"
      },
      "execution_count": 58,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "IyqG2fvoB7v1"
      },
      "execution_count": 58,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Describe dataset, including non-numerical columns"
      ],
      "metadata": {
        "id": "pG7Jm3c2B7yf"
      },
      "execution_count": 59,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(\"\\nDataset Description:\")  # Printing descriptive statistics for all columns\n",
        "print(df.describe(include='all'))  # Displaying descriptive statistics for both numerical and non-numerical columns\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DgHt9ivMB71I",
        "outputId": "317fa4f3-f1cb-4aea-eb31-38bf1872642f"
      },
      "execution_count": 60,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Dataset Description:\n",
            "                                                  article  \\\n",
            "count                                                 100   \n",
            "unique                                                 99   \n",
            "top     AMMAN, Jordan (CNN) -- In the sunbathed school...   \n",
            "freq                                                    2   \n",
            "\n",
            "                                               highlights  \\\n",
            "count                                                 100   \n",
            "unique                                                 99   \n",
            "top     Jordan opens school doors to all Iraqi childre...   \n",
            "freq                                                    2   \n",
            "\n",
            "                                              id  \n",
            "count                                        100  \n",
            "unique                                       100  \n",
            "top     f16446db34e2861f0450dfa34d8cdda541ab7b19  \n",
            "freq                                           1  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "1BMUZlzSCAik"
      },
      "execution_count": 60,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "VNK1xYfOCAlg"
      },
      "execution_count": 60,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Show unique values in 'category' column if available"
      ],
      "metadata": {
        "id": "RsZCWYngCAob"
      },
      "execution_count": 61,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "if 'category' in df.columns:  # Checking if 'category' column exists\n",
        "    print(\"\\nUnique Categories in the Dataset:\")  # Printing unique values in the 'category' column\n",
        "    print(df['category'].unique())  # Displaying unique categories in the dataset"
      ],
      "metadata": {
        "id": "dpLH7p0wCAq7"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "fk8dfv3nCGsf"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "_4XsnvveCHqY"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "aqs45bSVCHtd"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "rTNmp9qaCHwl"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "FnhwDFUJCINN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Text Preprocessing with BART Tokenizer"
      ],
      "metadata": {
        "id": "P9Qir5yOCJFr"
      },
      "execution_count": 63,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import BartTokenizer  # Importing BART tokenizer from transformers library"
      ],
      "metadata": {
        "id": "zRZYTwz1CGIy"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "s7OidlEECGL0"
      },
      "execution_count": 64,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load pre-trained BART tokenizer"
      ],
      "metadata": {
        "id": "ia4JV-QnCGOf"
      },
      "execution_count": 65,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = BartTokenizer.from_pretrained(\"facebook/bart-large-cnn\")  # Loading the BART tokenizer"
      ],
      "metadata": {
        "id": "kvPtmOC0CGQn"
      },
      "execution_count": 66,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "ZZUX817BCGTW"
      },
      "execution_count": 66,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Preprocess dataset (tokenize articles and summaries)"
      ],
      "metadata": {
        "id": "FixveuuSCAtm"
      },
      "execution_count": 67,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def preprocess_data(df):  # Defining a function to preprocess the data\n",
        "    inputs = tokenizer(df['article'].tolist(), return_tensors='pt', max_length=1024, truncation=True, padding=True)  # Tokenizing the articles\n",
        "    labels = tokenizer(df['highlights'].tolist(), return_tensors='pt', max_length=150, truncation=True, padding=True)  # Tokenizing the highlights (summaries)\n",
        "    return inputs, labels  # Returning tokenized inputs and labels"
      ],
      "metadata": {
        "id": "oYC888H2CAv9"
      },
      "execution_count": 68,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "IYYX0KRsCA5b"
      },
      "execution_count": 68,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "inputs, labels = preprocess_data(df)  # Preprocessing the dataset"
      ],
      "metadata": {
        "id": "-wSqJ3vfB73C"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "braC0VvUFFxm"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "-y4a5wR2Cb9l"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "c236ryreCcCX"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "DTrG8y2AFHPJ"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "HskEEUhaFHbh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 🏋️‍♂️ **Step 3: Fine-Tune the BART Model**\n",
        "\n",
        "```markdown\n",
        "## 🔧 **Fine-Tune the BART Model**\n",
        "Now that the dataset is preprocessed, it's time to fine-tune the **BART** model. BART is a transformer-based model designed for text generation tasks such as summarization.\n",
        "\n",
        "### Fine-Tuning Process:\n",
        "1. **Model Selection**: Use the pre-trained `facebook/bart-large-cnn` model.\n",
        "2. **Training Parameters**: Use the AdamW optimizer with a learning rate of `5e-5`, a batch size of `8`, and train for `3 epochs`.\n",
        "3. **Hardware**: Utilize GPU if available to speed up the training process.\n",
        "\n",
        "```python\n",
        "from transformers import BartForConditionalGeneration, BartTokenizer, Trainer, TrainingArguments\n",
        "\n",
        "# Load the pre-trained BART model and tokenizer\n",
        "model = BartForConditionalGeneration.from_pretrained('facebook/bart-large-cnn')\n",
        "tokenizer = BartTokenizer.from_pretrained('facebook/bart-large-cnn')\n",
        "\n",
        "# Define training arguments\n",
        "training_args = TrainingArguments(\n",
        "    output_dir='./results',\n",
        "    evaluation_strategy=\"epoch\",\n",
        "    learning_rate=5e-5,\n",
        "    per_device_train_batch_size=8,\n",
        "    num_train_epochs=3,\n",
        "    weight_decay=0.01,\n",
        ")\n",
        "\n",
        "trainer = Trainer(\n",
        "    model=model,\n",
        "    args=training_args,\n",
        "    train_dataset=tokenized_datasets['train'],\n",
        "    eval_dataset=tokenized_datasets['test'],\n",
        ")\n",
        "\n",
        "# Start fine-tuning\n",
        "trainer.train()"
      ],
      "metadata": {
        "id": "eKMTk5DKFIb5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Load BART Model for Fine-Tuning"
      ],
      "metadata": {
        "id": "NlPY8HytCcFZ"
      },
      "execution_count": 70,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import BartForConditionalGeneration, AdamW  # Importing BART model and AdamW optimizer\n"
      ],
      "metadata": {
        "id": "14evhsFiCcIZ"
      },
      "execution_count": 71,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import torch  # Importing torch for tensor manipulation"
      ],
      "metadata": {
        "id": "LIITwIDwCcM7"
      },
      "execution_count": 72,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "BnbS-tXGCcQM"
      },
      "execution_count": 72,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load pre-trained BART model for sequence-to-sequence tasks (summarization)"
      ],
      "metadata": {
        "id": "7firy8GYCcUd"
      },
      "execution_count": 73,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model = BartForConditionalGeneration.from_pretrained(\"facebook/bart-large-cnn\")  # Loading the BART model for summarization\n"
      ],
      "metadata": {
        "id": "2bLCI31hB75c"
      },
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "1i6qmjwtB78f"
      },
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Optimizer"
      ],
      "metadata": {
        "id": "EBIys3l_5ZqE"
      },
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "optimizer = AdamW(model.parameters(), lr=5e-5)  # Defining the optimizer with a learning rate of 5e-5\n"
      ],
      "metadata": {
        "id": "Kj4BLvbECpTv"
      },
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Uuf90VrjCpWd"
      },
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Move model to GPU if available"
      ],
      "metadata": {
        "id": "2Ee2QzUtCpZM"
      },
      "execution_count": 77,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')  # Selecting device (GPU if available, otherwise CPU)\n"
      ],
      "metadata": {
        "id": "UYWsYpXPCpb3"
      },
      "execution_count": 78,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.to(device)  # Moving the model to the selected device (GPU/CPU)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CKZ9M61nCpeZ",
        "outputId": "aabe7ac3-82b5-4dd9-b917-f469c2153a0b"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "BartForConditionalGeneration(\n",
              "  (model): BartModel(\n",
              "    (shared): BartScaledWordEmbedding(50264, 1024, padding_idx=1)\n",
              "    (encoder): BartEncoder(\n",
              "      (embed_tokens): BartScaledWordEmbedding(50264, 1024, padding_idx=1)\n",
              "      (embed_positions): BartLearnedPositionalEmbedding(1026, 1024)\n",
              "      (layers): ModuleList(\n",
              "        (0-11): 12 x BartEncoderLayer(\n",
              "          (self_attn): BartSdpaAttention(\n",
              "            (k_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (v_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (q_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (out_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "          )\n",
              "          (self_attn_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "          (activation_fn): GELUActivation()\n",
              "          (fc1): Linear(in_features=1024, out_features=4096, bias=True)\n",
              "          (fc2): Linear(in_features=4096, out_features=1024, bias=True)\n",
              "          (final_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "        )\n",
              "      )\n",
              "      (layernorm_embedding): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "    )\n",
              "    (decoder): BartDecoder(\n",
              "      (embed_tokens): BartScaledWordEmbedding(50264, 1024, padding_idx=1)\n",
              "      (embed_positions): BartLearnedPositionalEmbedding(1026, 1024)\n",
              "      (layers): ModuleList(\n",
              "        (0-11): 12 x BartDecoderLayer(\n",
              "          (self_attn): BartSdpaAttention(\n",
              "            (k_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (v_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (q_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (out_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "          )\n",
              "          (activation_fn): GELUActivation()\n",
              "          (self_attn_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "          (encoder_attn): BartSdpaAttention(\n",
              "            (k_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (v_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (q_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "            (out_proj): Linear(in_features=1024, out_features=1024, bias=True)\n",
              "          )\n",
              "          (encoder_attn_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "          (fc1): Linear(in_features=1024, out_features=4096, bias=True)\n",
              "          (fc2): Linear(in_features=4096, out_features=1024, bias=True)\n",
              "          (final_layer_norm): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "        )\n",
              "      )\n",
              "      (layernorm_embedding): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)\n",
              "    )\n",
              "  )\n",
              "  (lm_head): Linear(in_features=1024, out_features=50264, bias=False)\n",
              ")"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Wxgu_OBRCphc"
      },
      "execution_count": 79,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "n8GO0YWKCzn3"
      },
      "execution_count": 79,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Create DataLoader for training"
      ],
      "metadata": {
        "id": "d4LNY43_Czq-"
      },
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from torch.utils.data import DataLoader, TensorDataset  # Importing DataLoader and TensorDataset for batching data\n"
      ],
      "metadata": {
        "id": "VXNXLrOOCztg"
      },
      "execution_count": 81,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from tqdm import tqdm  # Importing tqdm for progress bar during training"
      ],
      "metadata": {
        "id": "SDiyxi7GCzvx"
      },
      "execution_count": 82,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "uoApoPVQCz06"
      },
      "execution_count": 82,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_data = TensorDataset(inputs['input_ids'], labels['input_ids'])  # Creating a TensorDataset from input_ids and label_ids\n"
      ],
      "metadata": {
        "id": "BWmUPDFYCz_8"
      },
      "execution_count": 83,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_loader = DataLoader(train_data, batch_size=8, shuffle=True)  # Creating a DataLoader with batch size 8, shuffling the data\n"
      ],
      "metadata": {
        "id": "2XLbUOvXCpqq"
      },
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "_O4cEhsXC-FM"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "vt3RUtDmC-9_"
      },
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "Qz4ZGYLrC_BT"
      },
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "-j9Ird3xC_ac"
      },
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "mL4MXBwHC_px"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Training loop"
      ],
      "metadata": {
        "id": "_STVxm-qDAh4"
      },
      "execution_count": 85,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "epochs = 3  # Number of training epochs"
      ],
      "metadata": {
        "id": "V_n9YKe0DAn5"
      },
      "execution_count": 86,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for epoch in range(epochs):  # Looping over epochs\n",
        "    model.train()  # Setting the model to training mode\n",
        "    total_loss = 0  # Initializing total loss for this epoch\n",
        "    for batch in tqdm(train_loader, desc=f\"Training Epoch {epoch+1}/{epochs}\"):  # Looping over batches with progress bar\n",
        "        input_ids, labels = [item.to(device) for item in batch]  # Moving input_ids and labels to device\n",
        "\n",
        "        # Forward pass\n",
        "        outputs = model(input_ids=input_ids, labels=labels)  # Performing forward pass to get model outputs\n",
        "        loss = outputs.loss  # Extracting the loss from model outputs\n",
        "        total_loss += loss.item()  # Accumulating the total loss\n",
        "\n",
        "        # Backward pass and optimization\n",
        "        optimizer.zero_grad()  # Zeroing the gradients\n",
        "        loss.backward()  # Backpropagating the loss\n",
        "        optimizer.step()  # Performing optimization step\n",
        "\n",
        "    print(f\"Epoch {epoch+1} | Loss: {total_loss / len(train_loader)}\")  # Printing the average loss for the epoch\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vxmqRqFcDArG",
        "outputId": "48058316-a962-4d76-ecaf-c27cb9641357"
      },
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Training Epoch 1/3: 100%|██████████| 13/13 [00:40<00:00,  3.12s/it]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1 | Loss: 3.5102916955947876\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Training Epoch 2/3: 100%|██████████| 13/13 [00:42<00:00,  3.24s/it]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2 | Loss: 0.9729984861153823\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Training Epoch 3/3: 100%|██████████| 13/13 [00:42<00:00,  3.24s/it]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3 | Loss: 0.45802743159807646\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "OOX_8bP6DJTa"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "iYo95NYeDKMI"
      },
      "execution_count": 87,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "T5Vg12MNDKPD"
      },
      "execution_count": 87,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "_ckpr9mEDKSQ"
      },
      "execution_count": 87,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "F3SfxJhgDKm-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 📊 **Step 4: Evaluate the Model**\n",
        "\n",
        "```markdown\n",
        "## 🧪 **Evaluate the Model**\n",
        "After fine-tuning, it’s crucial to evaluate the model’s performance. Evaluation is done by generating summaries for a subset of test articles and comparing them to the actual summaries.\n",
        "\n",
        "### Evaluation Process:\n",
        "1. **Generate Summaries**: Use the fine-tuned model to generate summaries for test articles.\n",
        "2. **Compare Summaries**: Evaluate the quality of generated summaries.\n",
        "\n",
        "```python\n",
        "# Generate summaries for the test set\n",
        "model.eval()  # Switch to evaluation mode\n",
        "generated_summaries = []\n",
        "\n",
        "for article in tokenized_datasets['test']:\n",
        "    inputs = tokenizer(article['article'], return_tensors=\"pt\", max_length=1024, truncation=True, padding='max_length')\n",
        "    summary_ids = model.generate(inputs['input_ids'])\n",
        "    summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)\n",
        "    generated_summaries.append(summary)\n",
        "\n",
        "# Display the first few generated summaries\n",
        "print(generated_summaries[:5])"
      ],
      "metadata": {
        "id": "3BlP_k0bFSM_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Model Evaluation"
      ],
      "metadata": {
        "id": "kRqhNim8DLhN"
      },
      "execution_count": 88,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_summary(input_texts):  # Defining a function to generate summaries\n",
        "    inputs = tokenizer(input_texts, return_tensors=\"pt\", max_length=1024, truncation=True, padding=True).to(device)  # Tokenizing the input texts\n",
        "    summary_ids = model.generate(inputs['input_ids'], max_length=150, num_beams=4, length_penalty=2.0, early_stopping=True)  # Generating summaries\n",
        "    return tokenizer.decode(summary_ids[0], skip_special_tokens=True)  # Decoding the summary back to text\n"
      ],
      "metadata": {
        "id": "t9e1pJ98DLmg"
      },
      "execution_count": 89,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "I0PWUcjDDLpV"
      },
      "execution_count": 89,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "U89tE8lHDLr-"
      },
      "execution_count": 89,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Test the model on the first 5 articles in the dataset"
      ],
      "metadata": {
        "id": "dGTfCz79DLvg"
      },
      "execution_count": 90,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "test_articles = df['article'].iloc[:5].tolist()  # Selecting the first 5 articles from the dataset"
      ],
      "metadata": {
        "id": "wPBIuNt5DLzg"
      },
      "execution_count": 91,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for article in test_articles:  # Looping through the test articles\n",
        "    print(\"Original Article: \", article)  # Printing the original article\n",
        "    print(\"Generated Summary: \", generate_summary([article]))  # Printing the generated summary\n",
        "    print(\"\\n\" + \"-\"*100 + \"\\n\")  # Printing a separator line for better readability"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SpGhA_tuDL2-",
        "outputId": "08a9884f-7bdc-4e10-bf29-648403558b21"
      },
      "execution_count": 92,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original Article:  LONDON, England (Reuters) -- Harry Potter star Daniel Radcliffe gains access to a reported £20 million ($41.1 million) fortune as he turns 18 on Monday, but he insists the money won't cast a spell on him. Daniel Radcliffe as Harry Potter in \"Harry Potter and the Order of the Phoenix\" To the disappointment of gossip columnists around the world, the young actor says he has no plans to fritter his cash away on fast cars, drink and celebrity parties. \"I don't plan to be one of those people who, as soon as they turn 18, suddenly buy themselves a massive sports car collection or something similar,\" he told an Australian interviewer earlier this month. \"I don't think I'll be particularly extravagant. \"The things I like buying are things that cost about 10 pounds -- books and CDs and DVDs.\" At 18, Radcliffe will be able to gamble in a casino, buy a drink in a pub or see the horror film \"Hostel: Part II,\" currently six places below his number one movie on the UK box office chart. Details of how he'll mark his landmark birthday are under wraps. His agent and publicist had no comment on his plans. \"I'll definitely have some sort of party,\" he said in an interview. \"Hopefully none of you will be reading about it.\" Radcliffe's earnings from the first five Potter films have been held in a trust fund which he has not been able to touch. Despite his growing fame and riches, the actor says he is keeping his feet firmly on the ground. \"People are always looking to say 'kid star goes off the rails,'\" he told reporters last month. \"But I try very hard not to go that way because it would be too easy for them.\" His latest outing as the boy wizard in \"Harry Potter and the Order of the Phoenix\" is breaking records on both sides of the Atlantic and he will reprise the role in the last two films.  Watch I-Reporter give her review of Potter's latest » . There is life beyond Potter, however. The Londoner has filmed a TV movie called \"My Boy Jack,\" about author Rudyard Kipling and his son, due for release later this year. He will also appear in \"December Boys,\" an Australian film about four boys who escape an orphanage. Earlier this year, he made his stage debut playing a tortured teenager in Peter Shaffer's \"Equus.\" Meanwhile, he is braced for even closer media scrutiny now that he's legally an adult: \"I just think I'm going to be more sort of fair game,\" he told Reuters. E-mail to a friend . Copyright 2007 Reuters. All rights reserved.This material may not be published, broadcast, rewritten, or redistributed.\n",
            "Generated Summary:  Harry Potter star Daniel Radcliffe gets £20 million fortune on Monday.\n",
            "Young actor says he has no plans to fritter his cash away.\n",
            "Radcliffe's earnings from first five Potter films have been held in trust fund.\n",
            "Details of how he'll mark his landmark birthday are under wraps.\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n",
            "Original Article:  Editor's note: In our Behind the Scenes series, CNN correspondents share their experiences in covering news and analyze the stories behind the events. Here, Soledad O'Brien takes users inside a jail where many of the inmates are mentally ill. An inmate housed on the \"forgotten floor,\" where many mentally ill inmates are housed in Miami before trial. MIAMI, Florida (CNN) -- The ninth floor of the Miami-Dade pretrial detention facility is dubbed the \"forgotten floor.\" Here, inmates with the most severe mental illnesses are incarcerated until they're ready to appear in court. Most often, they face drug charges or charges of assaulting an officer --charges that Judge Steven Leifman says are usually \"avoidable felonies.\" He says the arrests often result from confrontations with police. Mentally ill people often won't do what they're told when police arrive on the scene -- confrontation seems to exacerbate their illness and they become more paranoid, delusional, and less likely to follow directions, according to Leifman. So, they end up on the ninth floor severely mentally disturbed, but not getting any real help because they're in jail. We toured the jail with Leifman. He is well known in Miami as an advocate for justice and the mentally ill. Even though we were not exactly welcomed with open arms by the guards, we were given permission to shoot videotape and tour the floor.  Go inside the 'forgotten floor' » . At first, it's hard to determine where the people are. The prisoners are wearing sleeveless robes. Imagine cutting holes for arms and feet in a heavy wool sleeping bag -- that's kind of what they look like. They're designed to keep the mentally ill patients from injuring themselves. That's also why they have no shoes, laces or mattresses. Leifman says about one-third of all people in Miami-Dade county jails are mentally ill. So, he says, the sheer volume is overwhelming the system, and the result is what we see on the ninth floor. Of course, it is a jail, so it's not supposed to be warm and comforting, but the lights glare, the cells are tiny and it's loud. We see two, sometimes three men -- sometimes in the robes, sometimes naked, lying or sitting in their cells. \"I am the son of the president. You need to get me out of here!\" one man shouts at me. He is absolutely serious, convinced that help is on the way -- if only he could reach the White House. Leifman tells me that these prisoner-patients will often circulate through the system, occasionally stabilizing in a mental hospital, only to return to jail to face their charges. It's brutally unjust, in his mind, and he has become a strong advocate for changing things in Miami. Over a meal later, we talk about how things got this way for mental patients. Leifman says 200 years ago people were considered \"lunatics\" and they were locked up in jails even if they had no charges against them. They were just considered unfit to be in society. Over the years, he says, there was some public outcry, and the mentally ill were moved out of jails and into hospitals. But Leifman says many of these mental hospitals were so horrible they were shut down. Where did the patients go? Nowhere. The streets. They became, in many cases, the homeless, he says. They never got treatment. Leifman says in 1955 there were more than half a million people in state mental hospitals, and today that number has been reduced 90 percent, and 40,000 to 50,000 people are in mental hospitals. The judge says he's working to change this. Starting in 2008, many inmates who would otherwise have been brought to the \"forgotten floor\"  will instead be sent to a new mental health facility -- the first step on a journey toward long-term treatment, not just punishment. Leifman says it's not the complete answer, but it's a start. Leifman says the best part is that it's a win-win solution. The patients win, the families are relieved, and the state saves money by simply not cycling these prisoners through again and again. And, for Leifman, justice is served. E-mail to a friend .\n",
            "Generated Summary:  Mentally ill inmates in Miami are housed on the \"forgotten floor\"\n",
            "Judge Steven Leifman says most are there as a result of \"avoidable felonies\"\n",
            "He says the system is unjust and he's fighting to change it.\n",
            "Leifman: \"I am the son of the president\"\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n",
            "Original Article:  MINNEAPOLIS, Minnesota (CNN) -- Drivers who were on the Minneapolis bridge when it collapsed told harrowing tales of survival. \"The whole bridge from one side of the Mississippi to the other just completely gave way, fell all the way down,\" survivor Gary Babineau told CNN. \"I probably had a 30-, 35-foot free fall. And there's cars in the water, there's cars on fire. The whole bridge is down.\" He said his back was injured but he determined he could move around. \"I realized there was a school bus right next to me, and me and a couple of other guys went over and started lifting the kids off the bridge. They were yelling, screaming, bleeding. I think there were some broken bones.\"  Watch a driver describe his narrow escape » . At home when he heard about the disaster, Dr. John Hink, an emergency room physician, jumped into his car and rushed to the scene in 15 minutes. He arrived at the south side of the bridge, stood on the riverbank and saw dozens of people lying dazed on an expansive deck. They were in the middle of the Mississippi River, which was churning fast, and he had no way of getting to them. He went to the north side, where there was easier access to people. Ambulances were also having a hard time driving down to the river to get closer to the scene. Working feverishly, volunteers, EMTs and other officials managed to get 55 people into ambulances in less than two hours. Occasionally, a pickup truck with a medic inside would drive to get an injured person and bring him back up even ground, Hink told CNN. The rescue effort was controlled and organized, he said; the opposite of the lightning-quick collapse. \"I could see the whole bridge as it was going down, as it was falling,\" Babineau said. \"It just gave a rumble real quick, and it all just gave way, and it just fell completely, all the way to the ground. And there was dust everywhere and it was just like everyone has been saying: It was just like out of the movies.\" Babineau said the rear of his pickup truck was dangling over the edge of a broken-off section of the bridge. He said several vehicles slid past him into the water. \"I stayed in my car for one or two seconds. I saw a couple cars fall,\" he said. \"So I stayed in my car until the cars quit falling for a second, then I got out real quick, ran in front of my truck -- because behind my truck was just a hole -- and I helped a woman off of the bridge with me. \"I just wanted off the bridge, and then I ran over to the school bus. I started grabbing kids and handing them down. It was just complete chaos.\" He said most of the children were crying or screaming. He and other rescuers set them on the ground and told them to run to the river bank, but a few needed to be carried because of their injuries.  See rescuers clamber over rubble » . Babineau said he had no rescue training. \"I just knew what I had to do at the moment.\" Melissa Hughes, 32, of Minneapolis, told The Associated Press that she was driving home when the western edge of the bridge collapsed under her. \"You know that free-fall feeling? I felt that twice,\" Hughes said. A pickup landed on top of her car, but she was not hurt. \"I had no idea there was a vehicle on my car,\" she told AP. \"It's really very surreal.\" Babineau told the Minneapolis Star-Tribune: \"On the way down, I thought I was dead. I literally thought I was dead. \"My truck was completely face down, pointed toward the ground, and my truck got ripped in half. It was folded in half, and I can't believe I'm alive.\"  See and hear eyewitness accounts » . Bernie Toivonen told CNN's \"American Morning\" that his vehicle was on a part of the bridge that ended up tilted at a 45-degree angle. \"I knew the deck was going down, there was no question about it, and I thought I was going to die,\" he said. After the bridge settled and his car remained upright, \"I just put in park, turned the key off and said, 'Oh, I'm alive,' \" he said. E-mail to a friend .\n",
            "Generated Summary:  \"I thought I was going to die,\" driver says.\n",
            "Gary Babineau: \"I probably had a 30-, 35-foot free fall\"\n",
            "Driver Bernie Toivonen says car was on a part of bridge that ended up tilted 45 degrees.\n",
            "Minnesota bridge collapsed during rush hour Saturday.\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n",
            "Original Article:  WASHINGTON (CNN) -- Doctors removed five small polyps from President Bush's colon on Saturday, and \"none appeared worrisome,\" a White House spokesman said. The polyps were removed and sent to the National Naval Medical Center in Bethesda, Maryland, for routine microscopic examination, spokesman Scott Stanzel said. Results are expected in two to three days. All were small, less than a centimeter [half an inch] in diameter, he said. Bush is in good humor, Stanzel said, and will resume his activities at Camp David. During the procedure Vice President Dick Cheney assumed presidential power. Bush reclaimed presidential power at 9:21 a.m. after about two hours. Doctors used \"monitored anesthesia care,\" Stanzel said, so the president was asleep, but not as deeply unconscious as with a true general anesthetic. He spoke to first lady Laura Bush -- who is in Midland, Texas, celebrating her mother's birthday -- before and after the procedure, Stanzel said. Afterward, the president played with his Scottish terriers, Barney and Miss Beazley, Stanzel said. He planned to have lunch at Camp David and have briefings with National Security Adviser Stephen Hadley and White House Chief of Staff Josh Bolten, and planned to take a bicycle ride Saturday afternoon. Cheney, meanwhile, spent the morning at his home on Maryland's eastern shore, reading and playing with his dogs, Stanzel said. Nothing occurred that required him to take official action as president before Bush reclaimed presidential power. The procedure was supervised by Dr. Richard Tubb, Bush's physician, and conducted by a multidisciplinary team from the National Naval Medical Center in Bethesda, Maryland, the White House said. Bush's last colonoscopy was in June 2002, and no abnormalities were found, White House spokesman Tony Snow said. The president's doctor had recommended a repeat procedure in about five years. A colonoscopy is the most sensitive test for colon cancer, rectal cancer and polyps, small clumps of cells that can become cancerous, according to the Mayo Clinic. Small polyps may be removed during the procedure. Snow said on Friday that Bush had polyps removed during colonoscopies before becoming president. Snow himself is undergoing chemotherapy for cancer that began in his colon and spread to his liver.  Watch Snow talk about Bush's procedure and his own colon cancer » . \"The president wants to encourage everybody to use surveillance,\" Snow said. The American Cancer Society recommends that people without high risk factors or symptoms begin getting screened for signs of colorectal cancer at age 50. E-mail to a friend .\n",
            "Generated Summary:  Five small polyps found during procedure; \"none worrisome,\" spokesman says.\n",
            "President reclaims powers; Cheney takes over as president.\n",
            "Bush's last colonoscopy was in June 2002; no abnormalities found.\n",
            "White House spokesman: President in good humor, will resume activities at Camp David.\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n",
            "Original Article:  (CNN)  -- The National Football League has indefinitely suspended Atlanta Falcons quarterback Michael Vick without pay, officials with the league said Friday. NFL star Michael Vick is set to appear in court Monday. A judge will have the final say on a plea deal. Earlier, Vick admitted to participating in a dogfighting ring as part of a plea agreement with federal prosecutors in Virginia. \"Your admitted conduct was not only illegal, but also cruel and reprehensible. Your team, the NFL, and NFL fans have all been hurt by your actions,\" NFL Commissioner Roger Goodell said in a letter to Vick. Goodell said he would review the status of the suspension after the legal proceedings are over. In papers filed Friday with a federal court in Virginia, Vick also admitted that he and two co-conspirators killed dogs that did not fight well. Falcons owner Arthur Blank said Vick's admissions describe actions that are \"incomprehensible and unacceptable.\" The suspension makes \"a strong statement that conduct which tarnishes the good reputation of the NFL will not be tolerated,\" he said in a statement.  Watch what led to Vick's suspension » . Goodell said the Falcons could \"assert any claims or remedies\" to recover $22 million of Vick's signing bonus from the 10-year, $130 million contract he signed in 2004, according to The Associated Press. Vick said he would plead guilty to one count of \"Conspiracy to Travel in Interstate Commerce in Aid of Unlawful Activities and to Sponsor a Dog in an Animal Fighting Venture\" in a plea agreement filed at U.S. District Court in Richmond, Virginia. The charge is punishable by up to five years in prison, a $250,000 fine, \"full restitution, a special assessment and 3 years of supervised release,\" the plea deal said. Federal prosecutors agreed to ask for the low end of the sentencing guidelines. \"The defendant will plead guilty because the defendant is in fact guilty of the charged offense,\" the plea agreement said. In an additional summary of facts, signed by Vick and filed with the agreement, Vick admitted buying pit bulls and the property used for training and fighting the dogs, but the statement said he did not bet on the fights or receive any of the money won. \"Most of the 'Bad Newz Kennels' operations and gambling monies were provided by Vick,\" the official summary of facts said. Gambling wins were generally split among co-conspirators Tony Taylor, Quanis Phillips and sometimes Purnell Peace, it continued. \"Vick did not gamble by placing side bets on any of the fights. Vick did not receive any of the proceeds from the purses that were won by 'Bad Newz Kennels.' \" Vick also agreed that \"collective efforts\" by him and two others caused the deaths of at least six dogs. Around April, Vick, Peace and Phillips tested some dogs in fighting sessions at Vick's property in Virginia, the statement said. \"Peace, Phillips and Vick agreed to the killing of approximately 6-8 dogs that did not perform well in 'testing' sessions at 1915 Moonlight Road and all of those dogs were killed by various methods, including hanging and drowning. \"Vick agrees and stipulates that these dogs all died as a result of the collective efforts of Peace, Phillips and Vick,\" the summary said. Peace, 35, of Virginia Beach, Virginia; Phillips, 28, of Atlanta, Georgia; and Taylor, 34, of Hampton, Virginia, already have accepted agreements to plead guilty in exchange for reduced sentences. Vick, 27, is scheduled to appear Monday in court, where he is expected to plead guilty before a judge.  See a timeline of the case against Vick » . The judge in the case will have the final say over the plea agreement. The federal case against Vick focused on the interstate conspiracy, but Vick's admission that he was involved in the killing of dogs could lead to local charges, according to CNN legal analyst Jeffrey Toobin. \"It sometimes happens -- not often -- that the state will follow a federal prosecution by charging its own crimes for exactly the same behavior,\" Toobin said Friday. \"The risk for Vick is, if he makes admissions in his federal guilty plea, the state of Virginia could say, 'Hey, look, you admitted violating Virginia state law as well. We're going to introduce that against you and charge you in our court.' \" In the plea deal, Vick agreed to cooperate with investigators and provide all information he may have on any criminal activity and to testify if necessary. Vick also agreed to turn over any documents he has and to submit to polygraph tests. Vick agreed to \"make restitution for the full amount of the costs associated\" with the dogs that are being held by the government. \"Such costs may include, but are not limited to, all costs associated with the care of the dogs involved in that case, including if necessary, the long-term care and/or the humane euthanasia of some or all of those animals.\" Prosecutors, with the support of animal rights activists, have asked for permission to euthanize the dogs. But the dogs could serve as important evidence in the cases against Vick and his admitted co-conspirators. Judge Henry E. Hudson issued an order Thursday telling the U.S. Marshals Service to \"arrest and seize the defendant property, and use discretion and whatever means appropriate to protect and maintain said defendant property.\" Both the judge's order and Vick's filing refer to \"approximately\" 53 pit bull dogs. After Vick's indictment last month, Goodell ordered the quarterback not to report to the Falcons training camp, and the league is reviewing the case. Blank told the NFL Network on Monday he could not speculate on Vick's future as a Falcon, at least not until he had seen \"a statement of facts\" in the case.  E-mail to a friend . CNN's Mike Phelan contributed to this report.\n",
            "Generated Summary:  NEW: NFL commissioner, Atlanta Falcons owner critical of Michael Vick's conduct.\n",
            "NFL suspends Atlanta Falcons quarterback indefinitely without pay.\n",
            "Vick due in federal court Monday; future in NFL remains uncertain.\n",
            "Judge will have final say on whether to approve a plea deal.\n",
            "\n",
            "----------------------------------------------------------------------------------------------------\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "6cjqnwG3DYZa"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "oSsgL-B5DZfJ"
      },
      "execution_count": 92,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "7tNx7No8DZkr"
      },
      "execution_count": 92,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "LxnqaNHqDZrX"
      },
      "execution_count": 92,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "0u2XtZn0DZ-c"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 🚀 **Step 5: Deploy the Model with Flask**\n",
        "\n",
        "```markdown\n",
        "## 🌐 **Deploy the Model with Flask**\n",
        "The final step is to deploy the model using **Flask**. Flask will allow us to create a simple web application that accepts input text (article) and returns the summary generated by the model.\n",
        "\n",
        "### Flask Web API:\n",
        "1. **Route**: `/summarize`\n",
        "2. **Method**: `POST`\n",
        "3. **Input**: Raw article text (in JSON format).\n",
        "4. **Output**: Summarized text (in JSON format).\n",
        "\n",
        "```python\n",
        "from flask import Flask, request, jsonify\n",
        "\n",
        "app = Flask(__name__)\n",
        "\n",
        "@app.route('/summarize', methods=['POST'])\n",
        "def summarize():\n",
        "    article = request.json['article']\n",
        "    inputs = tokenizer(article, return_tensors=\"pt\", max_length=1024, truncation=True, padding='max_length')\n",
        "    summary_ids = model.generate(inputs['input_ids'])\n",
        "    summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)\n",
        "    return jsonify({\"summary\": summary})\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    app.run(debug=True)"
      ],
      "metadata": {
        "id": "Cc1ida3eFXDK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Model Deployment with Flask"
      ],
      "metadata": {
        "id": "IFRc8fVsDbKw"
      },
      "execution_count": 93,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from flask import Flask, request, jsonify  # Importing necessary Flask modules"
      ],
      "metadata": {
        "id": "weMtREXzDbPU"
      },
      "execution_count": 94,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "NRKGlmcADbSn"
      },
      "execution_count": 94,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize Flask app"
      ],
      "metadata": {
        "id": "Rgn5tyxsDbVC"
      },
      "execution_count": 95,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "app = Flask(__name__)  # Initializing the Flask app"
      ],
      "metadata": {
        "id": "QKP-UlpyDXaU"
      },
      "execution_count": 96,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "tb1EzxgCDXdW"
      },
      "execution_count": 96,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Route to summarize text"
      ],
      "metadata": {
        "id": "gAZVa_rMDXf8"
      },
      "execution_count": 97,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "@app.route('/summarize', methods=['POST'])  # Defining a POST route to accept text for summarization\n",
        "def summarize():  # Defining the summarize function\n",
        "    data = request.json  # Extracting the input text from the request body\n",
        "    input_text = data['text']  # Storing the input text\n",
        "\n",
        "    # Generate summary\n",
        "    summary = generate_summary([input_text])  # Generating the summary for the input text\n",
        "\n",
        "    return jsonify({'summary': summary})  # Returning the summary as a JSON response\n"
      ],
      "metadata": {
        "id": "Y9AYPRs7DXiH"
      },
      "execution_count": 98,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "ie2px1F1DXkh"
      },
      "execution_count": 98,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Run the Flask app"
      ],
      "metadata": {
        "id": "VggGGBSBDAug"
      },
      "execution_count": 99,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "if __name__ == '__main__':  # Checking if the script is being run directly\n",
        "    app.run(debug=True)  # Running the Flask app with debugging enabled"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GYkDMyU_DAx2",
        "outputId": "e3da2324-2028-4179-8977-d1453b6af002"
      },
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " * Serving Flask app '__main__'\n",
            " * Debug mode: on\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "INFO:werkzeug:\u001b[31m\u001b[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.\u001b[0m\n",
            " * Running on http://127.0.0.1:5000\n",
            "INFO:werkzeug:\u001b[33mPress CTRL+C to quit\u001b[0m\n",
            "INFO:werkzeug: * Restarting with stat\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "7WcMZWm1FbQR"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "nwgRPsjNFcDl"
      },
      "execution_count": 100,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "mN8_bwSoFcSI"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "w3vppDzRDA80"
      },
      "execution_count": 100,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "d4zVg6-DFdAh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 💾 **Step 6: Save the Model for Future Use**\n",
        "\n",
        "```markdown\n",
        "## 💡 **Save the Model for Future Use**\n",
        "Once the model is trained and evaluated, it's essential to save it along with its tokenizer for future use, such as re-deployment or inference.\n",
        "\n",
        "### Saving the Model:\n",
        "1. **Save the Model**: Store the model weights and configuration files.\n",
        "2. **Save the Tokenizer**: Store the tokenizer files for consistent text preprocessing during future inferences.\n",
        "\n",
        "```python\n",
        "model.save_pretrained('./summarization_model')\n",
        "tokenizer.save_pretrained('./summarization_model')"
      ],
      "metadata": {
        "id": "RG1miD6eFeF4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Save the fine-tuned model and tokenizer"
      ],
      "metadata": {
        "id": "rPExwvkeC9R_"
      },
      "execution_count": 101,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "model.save_pretrained(\"summarization_model\")  # Saving the fine-tuned model"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "H4hJdnqdCn5c",
        "outputId": "d60fde9f-32a8-4660-fe89-4080028b40aa"
      },
      "execution_count": 102,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some non-default generation parameters are set in the model config. These should go into a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model) instead. This warning will be raised to an exception in v4.41.\n",
            "Non-default generation parameters: {'max_length': 142, 'min_length': 56, 'early_stopping': True, 'num_beams': 4, 'length_penalty': 2.0, 'no_repeat_ngram_size': 3, 'forced_bos_token_id': 0, 'forced_eos_token_id': 2}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer.save_pretrained(\"summarization_model\")  # Saving the tokenizer"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OeDoWXQnCn8k",
        "outputId": "12065ff5-d906-4f7b-f49b-35d174437c1d"
      },
      "execution_count": 103,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('summarization_model/tokenizer_config.json',\n",
              " 'summarization_model/special_tokens_map.json',\n",
              " 'summarization_model/vocab.json',\n",
              " 'summarization_model/merges.txt',\n",
              " 'summarization_model/added_tokens.json')"
            ]
          },
          "metadata": {},
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---"
      ],
      "metadata": {
        "id": "taSnIfdeD1VQ"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "MY33bpZbCn_L"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "z51I25GnCoBa"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "RcJQ-3vlCoDq"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "XainXITKCoHx"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "---\n",
        "\n",
        "### 📑 **Step 7: Summary and Next Steps**\n",
        "\n",
        "```markdown\n",
        "## 📌 **Summary and Next Steps**\n",
        "In this project, we have:\n",
        "1. Loaded and preprocessed the CNN/Daily Mail dataset.\n",
        "2. Fine-tuned the **BART** model for text summarization.\n",
        "3. Evaluated the model’s performance.\n",
        "4. Deployed the model using **Flask** for real-time summarization.\n",
        "5. Saved the model for future use.\n",
        "\n",
        "### Next Steps:\n",
        "- Experiment with other summarization models like **T5** or **GPT**.\n",
        "- Improve model performance by tuning hyperparameters or using a larger dataset.\n",
        "- Extend the Flask app to support batch processing or multi-model summarization.\n",
        "\n",
        "This project provides a hands-on introduction to fine-tuning transformer-based models and deploying them in real-world applications."
      ],
      "metadata": {
        "id": "6j7tsOcKFiqz"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "BVrLcK1nCoKo"
      },
      "execution_count": 103,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "FW32PwUACoNl"
      },
      "execution_count": 103,
      "outputs": []
    }
  ]
}