Handwritten Digits Recognition using KNN (k-nearest neighbors) classifier, Average model, ANN (artificial neural network)
MNIST handwritten digits dataset is used for this. It contains 60000 images for training and 10000 images for testing, each images size 28x28 pixel square Download dataset: http://yann.lecun.com/exdb/mnist/
Flatten each 2D image size 28x28 square to 1D size 784.
Downsampling image 28x28 to nxn pixel (n = 4 or 7).
KNN algorithm is one of the more simple techniques used in machine learning, which:
- Find K instances in the training set are nearest distance to a test data. Here we will use Euclidean distance.
- Get the class with highest frequency from the K - most nearest distance above
This algorithm;
- Calculate 10 models which represent for 10 classes 0 - 9, each models is the mean value of training dataset which has same label.
- Calculate distance between 10 models and test data.
- Get the class with minimum distance
I build an ANN using tensorflow.keras with 3 layers:
- Input-layer
- 1 hidden-layer with 256 neurals
- Output-layer with 10 neurals represent for 10 class 0 - 9
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape = (img_train.shape[1:]))) # input-layer
model.add(tf.keras.layers.Dense(256, activation = 'relu')) # hidden-layer with 256 neural
model.add(tf.keras.layers.Dropout(0.25))
model.add(tf.keras.layers.Dense(10, activation = 'softmax')) # output-layer
training
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy', metrics = ['accuracy'])
model.fit(img_train, y_train, epochs = 10, verbose = 2)
Flatten | Histogram | Downsample 4x4 | Downsample 7x7 | |
---|---|---|---|---|
KNN (k = 100) | 94.44% | 33.68% | 80.78% | 93.12% |
KNN (k = 200) | 92.91% | 33.31% | 79.48% | 91.94% |
KNN (k = 300) | 91.80% | 32.68% | 78.36% | 91.10% |
Average model | 82.03% | 26.12% | 64.02% | 78.10% |
ANN | 98.52% | 34.65% | 88.02% | 97.38% |
[Sorry for my bad English :'(]