🔥 PyTorch Workshop

A comprehensive PyTorch workshop covering the fundamentals and advanced techniques of deep learning.

📖 Table of Contents

📖 Main Notebooks

1. Introduction to Tensors
   Learn about PyTorch tensors, the foundational data structure, and how to manipulate them.
2. Gradient and Autograd
   Understand gradients, automatic differentiation, and how PyTorch handles backpropagation with autograd.
3. Perceptron and AdaLiNe
   Explore the basics of the simplest neural network model (perceptron) and Adaptive Linear Neuron (Adaline).
4. Regression Models
   Implement linear and logistic regression using PyTorch, including model training and prediction.
5. Multi-Layer Perceptrons
   Implement and explore multi-layer perceptron (MLP) for more complex tasks.
6. Radial Basis Function Networks
   Implement and explore Radial Basis Function (RBF) networks and how they differ from traditional neural networks.
7. Convolutional Neural Networks
   Explore concepts around convolutional neural networks (CNNs).
8. Feature Extraction
   Learn how to extract features from pre-trained models for downstream tasks.
9. Transfer Learning
   Apply transfer learning by using pre-trained models for a new tasks.
10. Fine-Tuning Models
    Understand how to fine-tune models by updating specific layers while freezing others.
11. Recurrent Neural Networks
    Explore concepts around recurrent neural networks (RNNs).

📖 Utilities

A collection of concepts and tools utilized in the main notebooks for training models, ...

• Activation Functions
  Study different activation functions (ReLU, Sigmoid, Tanh, ...) and their roles in neural networks.
• Checkpoints
  Learn how to save and load model checkpoints to resume training or for inference.
• Custom Implementations
  Learn how to define custom models, dataset, loss function, ... using PyTorch's class-based approach.
• Working with Datasets
  Understand how to work with datasets in PyTorch using torch.utils.data.Dataset and torch.utils.data.DataLoader.
• Parameters vs. Hyperparameters
  Understand the difference between parameters and hyperparameters in neural networks.
• Loss Functions
  Dive into common loss functions used in neural networks, including MSE, Cross-Entropy, and others.
• Normalization Techniques
  Understand normalization techniques such as Batch Normalization and Layer Normalization.
• Vision Transforms
  Learn to apply transforms like data augmentation on datasets using torchvision.transforms.v2.
• Word Embeddings
  Explore different word embedding techniques and their applications in natural language processing.

📖 Models

• CNN Architectures
  1. LeNet-5 Architecture
  2. AlexNet Architecture
  3. VGGNet Architecture
  4. GoogLeNet Architecture
  5. ResNet Architecture
  6. DenseNet Architecture
  7. EfficientNet Architecture

📖 Projects

1. Multi-Layer Perceptrons
   A comprehensive example to building, training, and deploying a MLP, from data preprocessing to prediction.
2. Convolutional Neural Networks
   A comprehensive example to building, training, and deploying a CNN, from data preprocessing to prediction.
3. Word2Vec Word Embedding
   An in-depth guide to implementing the Word2Vec algorithm for word embeddings in natural language processing.

📋 Prerequisites

• 👨‍💻 Programming Fundamentals
  • Proficiency in Python (data types, control structures, functions, classes, etc.).
    • My Python Workshop: github.com/mr-pylin/python-workshop
  • Experience with libraries like NumPy, Pandas and Matplotlib.
    • My NumPy Workshop: github.com/mr-pylin/numpy-workshop
    • My Pandas Workshop: Coming Soon
    • My Data Visualization Workshop: github.com/mr-pylin/data-visualization-workshop
• 🔣 Mathematics for Machine Learning
  • 🔲 Linear Algebra: Vectors, matrices, matrix operations.
    • Linear Algebra Review and Reference written by Zico Kolter.
    • Notes on Linear Algebra written by Peter J. Cameron.
    • MATH 233 - Linear Algebra I Lecture Notes written by Cesar O. Aguilar.
  • 📈 Calculus: Derivatives, gradients, partial derivatives, chain rule (for backpropagation).
    • Lecture notes on advanced gradient descent written by Cl´ement W. Royer.
    • MATH 221 – CALCULUS LECTURE NOTES VERSION 2.0 written by Sigurd Angenent.
    • Calculus written by Gilbert Strang.
  • 🎲 Probability & Statistics: Probability distributions, mean/variance, etc.
    • MATH1024: Introduction to Probability and Statistics written by Sujit Sahu.

⚙️ Setup

This project was developed using Python v3.12.3. If you encounter issues running the specified version of dependencies, consider using this specific Python version.

📝 List of Dependencies

📦 Install Dependencies

You can install all dependencies listed in requirements.txt using pip:

pip install -r requirements.txt

🛠️ Usage Instructions

1. Open the root folder with VS Code:
   • Windows/Linux: Ctrl + K followed by Ctrl + O
   • macOS: Cmd + K followed by Cmd + O
2. Open .ipynb files using Jupyter extension integrated with VS Code.
3. Allow VS Code to install any recommended dependencies for working with Jupyter Notebooks.

✍️ Notes:

• The table of contents embedded in the notebooks is not fully functional on GitHub!
• To navigate the table of contents effectively, open the notebooks locally or view them via nbviewer for a better experience.

🔗 Useful Links

PyTorch

• Source Code:
  • Over 3500 contributors are currently working on PyTorch.
  • Link: github.com/pytorch/pytorch
• Website:
  • The official website for PyTorch, offering comprehensive documentation, tutorials, and resources for deep learning and machine learning with PyTorch.
  • Link: pytorch.org
• Pytorch Documentations:
  • Detailed and comprehensive documentation for all PyTorch features and functionalities, including tutorials and guides to help you get started and master PyTorch.
  • Link: pytorch.org/docs/stable/index.html
• TorchVision Documentations:
  • The torchvision package [part of the PyTorch] consists of popular datasets, model architectures, and common image transformations for computer vision.
  • Link: pytorch.org/vision/stable/index.html
• TorchAudio Documentation:
  • The torchaudio package [part of the PyTorch] consists of audio I/O and signal processing functionalities, enabling efficient loading, transforming, and manipulating audio.
  • Link: pytorch.org/audio/stable/index.html

NumPy

• A fundamental package for scientific computing in Python, providing support for arrays, matrices, and a large collection of mathematical functions.
• Official site: numpy.org

Pandas

• A powerful, open-source data analysis and manipulation library for Python.
• Pandas is built on top of NumPy.
• Official site: pandas.pydata.org

Data Visualization

• A comprehensive collection of Python libraries for creating static, animated, and interactive visualizations: Matplotlib, Seaborn, and Plotly.
• Official sites: matplotlib.org | seaborn.pydata.org | plotly.com

🔍 Find Me

Any mistakes, suggestions, or contributions? Feel free to reach out to me at:

• 📍linktr.ee/mr_pylin

I look forward to connecting with you! 🏃‍♂️

📄 License

This project is licensed under the Apache License 2.0.
You are free to use, modify, and distribute this code, but you must include copies of both the LICENSE and NOTICE files in any distribution of your work.

✍️ Additional Licensing Information

• Original Images:
  • The images located in the ./assets/images/original/ folder are licensed under the CC BY-ND 4.0.
  • Note: This license restricts derivative works, meaning you may share these images but cannot modify them.
• Third-Party Assets:
  • Additional images located in ./assets/images/third_party/ are used with permission or according to their original licenses.
  • Attributions and references to original sources are included in the code where these images are used.