A collection of various deep learning architectures, models, and tips for TensorFlow and PyTorch in Jupyter Notebooks, projects and my learnings
Title | Dataset | Description | Notebooks |
---|---|---|---|
Convolutional Neural Network | TBD | TBD | |
CNN with He Initialization | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Replacing Fully-Connnected by Equivalent Convolutional Layers | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
AlexNet Trained on CIFAR-10 | TBD | TBD | |
AlexNet with Grouped Convolutions Trained on CIFAR-10 | TBD | TBD |
Title | Description | Daset | Notebooks |
---|---|---|---|
DenseNet-121 Digit Classifier Trained on MNIST | TBD | TBD | |
DenseNet-121 Image Classifier Trained on CIFAR-10 | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
"All Convolutionl Net" -- A Fully Convolutional Neural Network | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
LeNet-5 on MNIST | TBD | TBD | |
LeNet-5 on CIFAR-10 | TBD | TBD | |
LeNet-5 on QuickDraw | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
MobileNet-v2 on Cifar-10 | TBD | TBD | |
MobileNet-v3 small on Cifar-10 | TBD | TBD | |
MobileNet-v3 large on Cifar-10 | TBD | TBD | |
MobileNet-v3 large on MNIST via Embetter | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Network in Network Trained on CIFAR-10 | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Convolutional Neural Network VGG-16 Trained on CIFAR-10 | TBD | TBD | |
VGG-16 Smile Classifier | CelebA | TBD | |
VGG-16 Dogs vs Cats Classifier | TBD | TBD | |
Convolutional Neural Network VGG-19 | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
ResNet and Residual Blocks | MNIST | TBD | |
ResNet-18 Digit Classifier | MNIST | TBD | |
ResNet-18 Gender Classifier | CelebA | TBD | |
ResNet-34 Digit Classifier | MNIST | TBD | |
ResNet-34 Object Classifier | QuickDraw | TBD | |
ResNet-34 Gender Classifier | CelebA | TBD | |
ResNet-50 Digit Classifier | MNIST | TBD | |
ResNet-50 Gender Classifier | CelebA | TBD | |
ResNet-101 Gender Classifier | CelebA | TBD | |
ResNet-101 | CIFAR-10 | TBD | |
ResNet-152 Gender Classifier | CelebA | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Multilabel DistilBERT | Jigsaw Toxic Comment Challenge | DistilBERT classifier fine-tuning | |
DistilBERT as feature extractor | IMDB movie review | DistilBERT classifier with sklearn random forest and logistic regression | |
DistilBERT as feature extractor using embetter |
IMDB movie review | DistilBERT classifier with sklearn random forest and logistic regression using the scikit-learn embetter library |
|
Fine-tune DistilBERT I | IMDB movie review | Fine-tune only the last 2 layers of DistilBERT classifier | |
Fine-tune DistilBERT II | IMDB movie review | Fine-tune the whole DistilBERT classifier |
Please note that the following notebooks below provide reference implementations to use the respective methods. They are not performance benchmarks.
Title | Dataset | Description | Notebooks |
---|---|---|---|
Baseline multilayer perceptron | Cement | A baseline multilayer perceptron for classification trained with the standard cross entropy loss | |
CORAL multilayer perceptron | Cement | Implementation of Rank Consistent Ordinal Regression for Neural Networks with Application to Age Estimation 2020 | |
CORN multilayer perceptron | Cement | Implementation of Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities 2022 | |
Binary extension multilayer perceptron | Cement | Implementation of Ordinal Regression with Multiple Output CNN for Age Estimation 2016 | |
Reformulated squared-error multilayer perceptron | Cement | Implementation of A simple squared-error reformulation for ordinal classification 2016 | |
Class distance weighted cross-entropy loss | Cement | Implementation of Class Distance Weighted Cross-Entropy Loss for Ulcerative Colitis Severity Estimation 2022 |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Siamese Network with Multilayer Perceptrons | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Autoencoder (MNIST) | TBD | TBD | |
Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Variational Autoencoder | TBD | TBD | |
Convolutional Variational Autoencoder | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
A simple character RNN to generate new text (Charles Dickens) | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
AutoAugment & TrivialAugment for Image Data | CIFAR-10 | Trains a ResNet-18 using AutoAugment and TrivialAugment |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Cyclical Learning Rate | TBD | TBD | |
Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet) | TBD | TBD | |
Gradient Clipping (w. MLP on MNIST) | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Transfer Learning Example (VGG16 pre-trained on ImageNet for Cifar-10) | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
MLP in Lightning with TensorBoard -- continue training the last model | TBD | TBD | |
MLP in Lightning with TensorBoard -- checkpointing best model | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Gradient Checkpointing Demo (Network-in-Network trained on CIFAR-10) | TBD | TBD |
Title | Description | Notebooks |
---|---|---|
Using Multiple GPUs with DataParallel -- VGG-16 Gender Classifier on CelebA | TBD | |
Distribute a Model Across Multiple GPUs with Pipeline Parallelism (VGG-16 Example) | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Getting Gradients of an Intermediate Variable in PyTorch | TBD | TBD |
Title | Dataset | Description | Notebooks |
---|---|---|---|
Saving and Loading Trained Models -- from TensorFlow Checkpoint Files and NumPy NPZ Archives | TBD | TBD |
Title | Description | Notebooks |
---|---|---|
TorchMetrics | How do we use it, and what's the difference between .update() and .forward()? |