MNIST-cnn

This repository contains a Python 3 naïve implementation of a neural network with convolutional and pooling layers, useful for educational purposes. It was tested with satisfactory results the on the well-known MNIST data set.

Alessandro and Francesco

Prerequisites

The code makes heavy use of NumPy. Install it using pip:

~ ➤ pip3 install --user numpy

Then, download the MNIST data set (four .gz archives) and decompress it:

~ ➤ cd Downloads
Downloads ➤ ls
t10k-images-idx3-ubyte.gz  t10k-labels-idx1-ubyte.gz  train-images-idx3-ubyte.gz train-labels-idx1-ubyte.gz
Downloads ➤ gzip -d *
Downloads ➤ ls
t10k-images-idx3-ubyte  t10k-labels-idx1-ubyte  train-images-idx3-ubyte train-labels-idx1-ubyte

Note: on Windows, as per our tests, the extracted files will have a different name of the ones showed above (e.g. train-images.idx3-ubyte instead of train-images-idx3-ubyte). You can either manually rename the extracted files or modify lines 10-13 of src/utils.py.

Usage

Convert the downloaded data set to the NPZ binary data format using the function build_mnist_npz() from src/utils.py:

MNIST-cnn ➤ cd src
src ➤ ls
examples.py  functions.py layers.py    network.py   utils.py
src ➤ python3 -q
>>> import utils
>>> utils.build_mnist_npz('/Users/fcagnin/Downloads')
>>> exit()
src ➤ file mnist.npz
mnist.npz: Zip archive data, at least v2.0 to extract

Finally, run any of the included examples:

src ➤ time python3 -OO examples.py mnist.npz fcl01
Loading 'mnist.npz'...
Loading 'fcl01'...
def fcl01():
    net = n.NeuralNetwork([
        l.InputLayer(height=28, width=28),
        l.FullyConnectedLayer(100, init_func=f.glorot_uniform, act_func=f.sigmoid),
        l.FullyConnectedLayer(10, init_func=f.glorot_uniform, act_func=f.sigmoid)
    ], f.quadratic)
    optimizer = o.SGD(3.0)
    num_epochs = 1
    batch_size = 10
    return net, optimizer, num_epochs, batch_size
Training network...
Epoch 01 [==========] [50000/50000] > Validation accuracy: 95.41%
Testing network...
Test accuracy: 95.07%
python3 -OO examples.py mnist.npz fcl01  24.10s user 4.07s system 130% cpu 21.517 total

src ➤ time python3 examples.py mnist.npz cnn01
Loading 'mnist.npz'...
Loading 'cnn01'...
def cnn01():
    net = n.NeuralNetwork([
        l.InputLayer(height=28, width=28),
        l.ConvolutionalLayer(2, kernel_size=5, init_func=f.glorot_uniform, act_func=f.sigmoid),
        l.MaxPoolingLayer(pool_size=2),
        l.FullyConnectedLayer(height=10, init_func=f.glorot_uniform, act_func=f.softmax)
    ], f.log_likelihood)
    optimizer = o.SGD(0.1)
    num_epochs = 3
    batch_size = 10
    return net, optimizer, num_epochs, batch_size
Training network...
Epoch 01 [==========] [50000/50000] > Validation accuracy: 88.86%
Epoch 02 [==========] [50000/50000] > Validation accuracy: 89.84%
Epoch 03 [==========] [50000/50000] > Validation accuracy: 89.11%
Testing network...
Test accuracy: 88.53%
python3 examples.py mnist.npz cnn01  2869.89s user 11.16s system 99% cpu 45:12.89 total