Training models is hard. You have to collect a dataset, clean it, get it in the right format, select a model, write the training code and train it. And that's the best-case scenario.
The goal of this project is to explore an experimental new pipeline to train a high-performing task-specific model. We try to abstract away all the complexity, so it's as easy as possible to go from idea -> performant fully-trained model.
Simply input a description of your task, and the system will generate a dataset from scratch, parse it into the right format, and fine-tune a LLaMA 2 model for you.
-
Dataset Generation: Using GPT-4,
gpt-llm-trainerwill generate a variety of prompts and responses based on the provided use-case. -
System Message Generation:
gpt-llm-trainerwill generate an effective system prompt for your model. -
Fine-Tuning: After your dataset has been generated, the system will automatically split it into training and validation sets, fine-tune a model for you, and get it ready for inference.
-
Open the notebook in Google Colab or in a local Jupyter notebook.
-
If you're using Colab, switch to the best GPU available (go to Runtime -> change runtime type).
-
Add your OpenAI API key to the line
openai.api_key = "YOUR KEY HERE".
- Define your
prompt. Thepromptis a description of what you want the trained AI to do. The more descriptive and clear you can be, the better. Additionally, set the temperature we will use to generate your dataset (high=creative, low=precise), and the number of examples you want to generate (100 is a good starting point).
For example:
prompt = "A model that takes in a puzzle-like reasoning-heavy question in English, and responds with a well-reasoned, step-by-step thought out response in Spanish."
temperature = .4
number_of_examples = 100
- Run all the cells (stop at
Merge the model and store in Google Drive).
It'll take some time (from 10 minutes to a couple of hours, depending on how many examples you generate), but soon, you'll have your fine-tuned model!
- After your model is trained, you can use the
Run Inferencecell to test the model, and the cells below that allow you to save and load the model to and from Google Drive for later use.
- improve the example generation pipeline for efficiency/cost reduction (using n=)
- add additional example generation prompts to create more diverse examples
- add example pruning, removing very similar examples to improve performance
- use GPT-4 to automatically choose the training hyperparameters (and potentially, even the model to fine-tune!) based on a few examples + high-level dataset details (i.e. number of examples)
- train multiple model variants and choose the one with the lowest eval loss
This project is MIT licensed.
Matt Shumer - @mattshumer_
Project Link: https://github.com/mshumer/gpt-llm-trainer
Lastly, if you want to try something even cooler than this, sign up for Personal Assistant (most of my time is spent on this). It's basically an AI that can operate your web browser to complete tasks for you.