CORE Skills Data Science Springboard - Day 9

Today's session will introduce the main concepts of Artificial Neural Networks (ANNs) and Deep Learning. We will also discuss in more details two types of ANNs which are widely used in deep learning applications: the Convolutional Neural Networks (CNNs) and the Recurrent Neural Networks (RNNs). They are specialised networks to deal with image data and sequence data, respectively. We will also discuss the training strategies in order to allow deep nets to be powerful learning algorithms capable of learning complex patterns in data.

You should aim to understand:

• The general concepts of Artificial Neural Networks and Deep Learning.
• The strategies which are used to train the deep nets in a efficient way.
• How CNNS and RNNs work and examples of applications.
• The whole of activation functions in ANNs.

We will use python notebooks exercises on different datasets to have a practical go on the discussed methods in practise.

Pre-session Reading & Resources

In this session, we will use Scikit-learn, Keras and TensorFlow. Keras is a high-level API in python for neural networks, it makes the implementation of neural nets much easier. You can have a reading about Keras here. TensorFlow is an optimised library in python for dealing with tensors (for instance, weigh matrices in a neural network). TensorFlow documentation can be found here. By now you are familiar with Scikit-learn, but if you need to have a look at the documentation, it can be found here.

We will continue to use the book "Hands-on Machine Learning with Scikit-learn and Tensorflow" as the main reference. The collection of python notebooks related to the book with all the python implementations: https://github.com/ageron/handson-ml. You can also purchase a copy of the book from O'Reilly Books here: http://shop.oreilly.com/product/0636920052289.do

Stack Overflow is a useful website to find solutions for when you get stuck with python and/or scikit-learn/keras/tensorflow: https://stackoverflow.com/questions/tagged/scikit-learn, https://stackoverflow.com/questions/tagged/keras, https://stackoverflow.com/questions/tagged/tensorflow.

Here are two very good (and free!) textbooks about Deep Learning:

1. Neural Networks and Deep Learning: http://neuralnetworksanddeeplearning.com Author: Michael A. Nielsen.

2. Deep Learning: http://www.deeplearningbook.org. Authors: Ian Goodfellow and Yoshua Bengio and Aaron Courville

And here are some other links you may find interesting:

1. Google platform for creating simple feedforward neural nets and inspect, in real-time, various effects of training: http://playground.tensorflow.org/

2. Towards Data Science is a website with several interesting and useful articles on machine learning and deep learning: https://towardsdatascience.com/machine-learning/home

3. Andrew Ng deep learning page: https://www.deeplearning.ai

To get inspired:

A couple of interesting articles on Deep Learning.

The Lessons Learned from Applying Deep Learning for NLP Without Big Data: https://towardsdatascience.com/lessons-learned-from-applying-deep-learning-for-nlp-without-big-data-d470db4f27bf

Deep Learning: With massive amounts of computational power, machines can now recognize objects and translate speech in real time. Artificial intelligence is finally getting smart: https://www.technologyreview.com/s/513696/deep-learning/

One of the main aspects in deep learning: the real advantage of deep learning is more prominent when there are sufficient computational resources to train very large datasets on sophisticated architectures.