🏫🙋🏻Self Learn LLMs

This repository constitutes some of the resources which I will use to learn about Large Language Models. I will also try to come up with a roadmap as I go forward in this self-learning journey, since a clear roadmap with milestones will be one of the best ways to learn about LLMs in a proper manner.

For this, I will include a mix of theoretical and practical hands-on resources to learn.

PS: Need to make this more visual

Edit: 6th Nov 2024

• Need to majorly refactor the repository.
• Remove courses or resources that are not relevant.
• In the model race, we can't go on listing models in the repository as it's tough to keep track of models and updates will make the previous models useless. Need to think of a better way to organize model zoo.
• Maybe add 1-2 contributors or open the repository to contributions to help out.
• How can we make it a gret learning experience, hosting notes and tutorials, open discussions, webpage?

Courses

NLP Fundamentals

1. CS224N Natural Language Processing with Deep Learning, Stanford
2. Natural Language Processing Specialization, Coursera

Transformers

1. HuggingFace NLP + Transformers Course
2. CS25: Transformers United V2, Stanford CS25, Fall 2021 Version

Large Language Models

Industrial and Open-Source courses

1. Activeloop Learn, this initiative GenAI360 provides 3 free courses on RAGs, fine-tuning LLMs, LangChain and VectorDBs.
2. LLM Course by Maxime Labonne, Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.
3. Hands on LLM Course, Learn about LLMs, LLMOps, and Vector DBs for free by designing, training, and deploying a real-time financial advisor LLM system source code + video & reading materials.
4. Full Stack Deep Learning, started out as a deep learning bootcamp and evolved into LLM bootcamp around April 2023, now is free to take up.
5. LLM University by Cohere, this course consists of 8 modules taught by the famous Luis Serrano, who is known for teaching concepts in a easy and visually appealing manner. The course contains topics like fundamentals, deployment, semantic search and RAG.
6. Deeplearning.ai Short Courses, Short courses by DL.AI on various domains of LLMs and Generative AI. These short courses are really useful as they have the perfect blend of theoretical and practical sessions. The courses are usually made in collaborations with companies like Hugging Face, Mistral, OpenAI, Microsoft, Meta, Google etc.
7. LLM Zoomcamp by DataTalksClub, LLM Zoomcamp - a free online course about building a Q&A system.
8. Applied LLMs Mastery 2024 Course by Aishwarya N Reganti, free 10 weeks course with a definite roadmap ranging from LLM Fundamentals, Tools and techniques, Deployment and evaluation to Challenges and future trends.
9. Weights and Biases Courses, provides different courses on MLOps, LLM Powered Apps etc.
10. LLM Models course, DataBricks x ed, professional certification by DataBricks.
11. Deeplearning.ai offers various short courses on LLMs like LangChain for LLM App Development, Serverless LLMs with AWS Bedrock, Fine-tuning LLMs, LLMs with Semantic Search etc.
12. Introduction to Generative AI Learning Path, Google Cloud.
13. Arize University hosts courses like llm-evaluation, llm agents tools and chains, llm-observability etc.

University Courses

1. [CS 324, Stanford](https://stanford-cs324.github.io/winter2022/) 2. [COMP790-101: Large Language Models, UNC Chapel Hill](https://github.com/craffel/llm-seminar) 3. [Large Language Models S-23, ETH Zurich](https://rycolab.io/classes/llm-s23/) 4. [Foundations of Large Language Models, University of Waterloo](https://uwaterloo.ca/watspeed/programs-and-courses/foundations-large-language-models)

Books

1. Natural Language Processing with Transformers Book

Blogs and Guides

Langchain Blogs

1. AIMultiple's blog on Large Language Models: Complete Guide in 2023
2. Cohere Docs
3. FutureSmart AI Blog on Building Chatbots using LangChain and ChatGPT
4. Task-driven Autonomous Agent Utilizing GPT-4, Pinecone, and LangChain for Diverse Applications

Important Papers

1. A Survey of Large Language Models Also check out this Repo: https://github.com/RUCAIBox/LLMSurvey
2. Understanding Large Language Models -- A Transformative Reading List, Sebastian Raschka

Reading Groups

1. Wiki CLSP, NLP Reading Group, a list of reading groups related to NLP which is updated frequently.

Tutorials and Talks

TED Talks

1. The Inside Story of ChatGPT’s Astonishing Potential | Greg Brockman | TED
2. Why AI Is Incredibly Smart — and Shockingly Stupid | Yejin Choi | TED

Upcoming Talks

1. 25th April 2023, Arize: Observe
2. 27th April 2023, Fine-Tuning LLMs with PyTorch 2.0 and ChatGPT

Datasets, Notebooks and Spaces

Datasets

Notebooks

Spaces

1. H2O Organization, HuggingFaces
2. OpenAssistant Organization, HuggingFaces
3. DataBricks Organization, HuggingFaces
4. BigScience Organization, HuggingFaces
5. EleutherAI Organization, HuggingFaces
6. NomicAI Organization, HuggingFaces
7. Cerebras Organization, HuggingFaces

Libraries, Frameworks and Toolkits

1. LLMStudio, H2O AI
2. LLamaIndex
3. NeMo Guardrails, NVIDIA, to prevent hallucinations and add programmable guardrails
4. MLC LLM, Develop optimize and deploy LLMs natively on everyone's devices)
5. LaMini LLM

Language Models

1. ChatGPT, OpenAI, Released 30th November 2022
2. Google Bard, Released 21st March 2023
3. Tongyi Qianwen AI, Alibaba, Released 11th April 2023
4. StableLM, Stability AI, Released 20th April 2023
5. Amazon Titan
6. HuggingChat, HuggingFaces, Released 25th April 2023
7. H2OGPT
8. Bloom Model, Commercial Use Allowed with RAIL
9. GPT-J, EleutherAI, Apache 2.0
10. GPT-NeoX, EleutherAI, Apache 2.0
11. GPT4All, NomicAI, MIT License
12. GPT4All-J, NomicAI, MIT License
13. Pythia, EleutherAI, MIT License
14. GLM-130B
15. PaLM, Google
16. OPT, Meta
17. FLAN-T5
18. LLaMA, Meta
19. Alpaca, Stanford
20. Vicuna, lm-sys

Communities

Usecases

1. ShareGPT

Projects and Ideas

Vector Databases

1. Pinecone
2. Weaviate
3. Milvus
4. ChromaDB

Autonomous Agents

1. BabyAGI
2. AutoGPT

LLM Influencers

People you should definitely follow to keep updated about LLMs. Researchers/Founders/Developers/AI Content Creators involved in LLM production/research/development

1. Sebastian Raschka, he is a legend and will burst your hype-up LLM bubble with his amazing tweets, blogs and tutorials. Subscribe to his newsletter Ahead of AI
2. Andrej Karpathy, so this legend worked in Tesla, took a break, started his YouTube channel to teach the fundamentals and blew us all with his amazing video on implementing GPT from scratch and finally rejoined OpenAI. I guess you cannot lose a legend :D
3. Jay Alammar, yup if you don't know about his ELI blog on Transformers go read that out first and be sure to follow him for updates.
4. Tomaz Bratanic, he is the author of famous book Graph Algorithms for Data Science, and currently writes great blogs on Medium related to GPT, Langchain and stuff.