Deep Learning with TensorFlow, Keras, and PyTorch

This repository is home to the code that accompanies Jon Krohn's Deep Learning with TensorFlow, Keras, and PyTorch series of video tutorials.

There are three sets of video tutorials in the series:

1. The eponymous Deep Learning with TensorFlow, Keras, and PyTorch (released in Feb 2020)
2. Deep Learning for Natural Language Processing, 2nd Ed. (Feb 2020)
3. Machine Vision, GANs, and Deep Reinforcement Learning (Mar 2020)

The above order is the recommended sequence in which to undertake these tutorials. That said, the first in the series provides a strong foundation for either of the other two.

Taken all together, the series -- over 18 total hours of instruction and hands-on demos -- parallels the entirety of the content in the book Deep Learning Illustrated. This means that the videos introduce all of deep learning:

• What deep neural networks are and how they work, both mathematically and using the most popular code libraries
• Machine vision, primarily with convolutional neural networks
• Natural language processing, including with recurrent neural networks
• Artistic creativity with generative adversarial networks (GANs)
• Complex, sequential decision-making with deep reinforcement learning

These video tutorials also includes some extra content that is not available in the book, such as:

• Detailed interactive examples involving training and testing deep learning models in PyTorch
• How to generate novel sequences of natural language in the style of your training data
• High-level discussion of transformer-based natural-language-processing models like BERT, ELMo, and GPT-3
• Detailed interactive examples of training advanced machine vision models (image segmentation, object detection)
• All hands-on code demos involving TensorFlow or Keras have been updated to TensorFlow 2

Installation

Installation instructions for running the code in this repository can be found in the installation directory.

Notebooks

There are dozens of meticulously crafted Jupyter notebooks of code associated with these videos. All of them can be found in this directory.

Below is a breakdown of the lessons covered across the videos, including their duration and associated notebooks.

Deep Learning with TensorFlow, Keras, and PyTorch

• Seven hours and 13 minutes total runtime
• Lesson 1: Introduction to Deep Learning and Artificial Intelligence (1 hour, 47 min)
  • Shallow Net in TensorFlow
• Lesson 2: How Deep Learning Works (2 hours, 16 min) -- free YouTube video here
  • Sigmoid Function
  • Softmax Demo
  • Quadratic Cost
  • Cross-Entropy Cost
  • Intermediate Net in TensorFlow
• Lesson 3: High-Performance Deep Learning Networks (1 hour, 16 min)
  • Weight Initialization
  • Measuring Speed of Learning
  • Deep Net in TensorFlow
  • Regression in TensorFlow
  • Regression with TensorBoard
• Lesson 4: Convolutional Neural Networks (47 min)
  • LeNet in TensorFlow
• Lesson 5: Moving Forward with Your Own Deep Learning Projects (1 hour, 4 min)
  • Shallow Net in PyTorch
  • Deep Net in PyTorch
  • LeNet in TensorFlow for Fashion MNIST

Deep Learning for Natural Language Processing

• Five hours total runtime
• Lesson 1: The Power and Elegance of Deep Learning for NLP (46 min)
• Lesson 2: Word Vectors (1 hour, 7 min)
  • Creating Word Vectors with word2vec
• Lesson 3: Modeling Natural Language Data (1 hour, 43 min) -- free YouTube video here
  • Natural Language Preprocessing
  • Document Classification with a Dense Neural Net
  • Classification with a Convolutional Neural Net
• Lesson 4: Recurrent Neural Networks (25 min)
  • RNN
  • LSTM
  • GRU
• Lesson 5: Advanced Models (54 min)
  • Bidirectional LSTM
  • Stacked Bi-LSTM
  • Convolutional-LSTM Stack
  • Sequence Generation
  • Keras Functional API

Machine Vision, GANs, and Deep Reinforcement Learning

• Six hours and six minutes total runtime
• Lesson 1: Orientation (35 min)
• Lesson 2: Convolutional Neural Networks for Machine Vision (2 hours, 2 min) -- free YouTube video here
  • LeNet
  • AlexNet
  • VGGNet
  • Object Detection
  • Transfer Learning
• Lesson 3: Generative Adversarial Networks for Creativity (1 hour, 22 min)
  • Cartoon-Drawing GAN
• Lesson 4: Deep Reinforcement Learning (38 min)
• Lesson 5: Deep Q-Learning and Beyond (1 hour, 25 min)
  • Cartpole Game-Playing DQN

You've reached the bottom of this page! As a reward, here's a myopic trilobite created by Aglaé Bassens, illustrator of the book Deep Learning Illustrated:

Email from Nicholas Cauldwell

Dear All, I tracked down the Deep Learning Illustrated author's github repository and he has updated them to be much more compatible with the current version of TensorFlow / Keras. These live at https://github.com/jonkrohn/DLTFpT/tree/master/notebooks the specific Jupyter notebooks are https://github.com/jonkrohn/DLTFpT/blob/master/notebooks/shallow_net_in_tensorflow.ipynb https://github.com/jonkrohn/DLTFpT/blob/master/notebooks/intermediate_net_in_tensorflow.ipynb https://github.com/jonkrohn/DLTFpT/blob/master/notebooks/deep_net_in_tensorflow.ipynb The last line of each script currently breaks (another deprecated method disappearing) so two code changes needed: in the load dependencies block, add import numpy as np and at the very end, replace model.predict_classes(valid_0) with np.argmax(model.predict(valid_0), axis=-1) and as of 23rd Novemeber 12:14, these modifications have made all three scripts behave.

Best wishes, Nicholas