19 BITS

Mar Baselios College of Engineering and Technology (Autonomous)

Single-Node-Finetuning-of-Tiny-LLama-using-Intel-Xeon-SPR

This repository contains the implementation of Single-Node Finetuning of the Tiny LLaMA language model utilizing Intel Xeon Scalable Processors (SPR). The project was undertaken as part of the Intel Unnati Industrial Training program for the year 2024. The primary objective of this project aligns with Problem Statement PS-04: Introduction to Generative Artificial Intelligence (GenAI) Basic Large Language Model (LLM) Inference on CPUs and subsequent LLM Model Finetuning for the development of a Custom Chatbot.

🛠️ Installation Steps:

1. Environment Setup(Execute them in terminal):

Set up a Python environment with the necessary dependencies.

1.1. Create and activate a Conda environment:

conda create -n itrex-1 python=3.10 -y
conda activate itrex-1

This creates a new Conda environment named itrex-1 with Python 3.10 and activates it.

1.2. Install required Python packages:

pip install intel-extension-for-transformers

2. Cloning the Repository

2.1. Clone the repository:

git clone https://github.com/eternalflame02/Single-Node-FInetuning-of-Tiny-LLama-using-Intel-Xeon-SPR.git

2.2. Navigate to the fine-tuning directory:

cd ./Single-Node-FInetuning-of-Tiny-LLama-using-Intel-Xeon-SPR/Fine Tuning/

3. Installing Additional Dependencies

Install additional dependencies required for fine-tuning.

3.1. Install dependencies from the requirements.txt file:

pip install -r requirements.txt

3.2. Install Jupyter and IPython kernel:

python3 -m pip install jupyter ipykernel
python3 -m ipykernel install --name neural-chat--user

4. Setting Up Hugging Face Authentication

Authenticate with Hugging Face to access and download models.

4.1. Login to Hugging Face:

huggingface-cli login

Create a token in https://huggingface.co/settings/tokens insert them in the huggingface login interface.

5. Downloading Data

Download the dataset required for fine-tuning.

5.1. Download the Alpaca dataset:

Curl -O https://github.com/tatsu-lab/stanford_alpaca/raw/main/alpaca_data.json

💻 Built with

Public Repositories:

• intel-extension-for-transformers
• TinyLlama/TinyLlama-1.1B-Chat-v1.0
• stanford_alpaca

🍰 Contributors:

@Rohith NS
@Joshua Sunny Ninan
@Avin
@Edwin K Mathew