mini-GPT

This repository contains an implementation of the GPT (Generative Pre-trained Transformer) architecture, a repo following the Karpathy's makemore series.

Data

Located in the data directory, you'll find input data used for model training:

• input.txt: Contains Shakespearean text.
• names.txt: A dataset of English names.

GPT Karpathy

Within the gpt_karpathy directory, you'll find a from-scratch implementation of the GPT architecture based on Andrej Karpathy's educational video titled "Let's build GPT: from scratch, in code, spelled out".

• bigram.py: (Description, if needed)
• gpt.py: Contains the main GPT implementation.

Notebooks

The notebooks directory contains Jupyter notebooks primarily based on the "Make More Neural Networks!" series. Some of the key notebooks include:

• GPT_dev.ipynb: GPT development and experimentation.
• build_makemore_mlp.ipynb: A part of the "Make More Neural Networks!" series. (Continue with brief descriptions for each notebook if needed.)

Getting Started

1. Clone the repository:

git clone [email protected]:11carlesnavarro/mini-gpt.git

Contributing

While this repository is primarily an educational resource, contributions or suggestions are welcome. Feel free to open an issue or submit a pull request.