Semester 7 Minor Project - Gesture Detection and Hand Written Character Recognition System
ABSTRACT
We are living in an era where machines are comprehending and learning the workings of the neural and cerebral functioning of human minds and facsimile them. In an attempt to endeavour the same, we as a group are working to fabricate a similar project on Gesture Detection and Hand Written Image Detection using Natural Language Processing. There are many applications for image recognition. One of the largest applications that people are most familiar with would be facial recognition, which is the art of matching faces in pictures to identities. Image recognition goes much further. Moreover, it can allow computers to translate written text on paper into digital text and can help the field of machine vision, where robots and other devices can recognize people and objects. Here, our goal is to begin using machine learning, in the form of pattern recognition, to teach our program what text looks like using supervised learning and deep learning. We also teach the program to interpret the meaning of the captured gesture done by the user in order to convey their message. Supervised learning as the name stipulates the presence of a supervisor as a teacher. Basically, supervised learning is a type of learning in which we teach or train the machine using data that is well labelled. This means some data is already tagged with the correct answer. After that, the machine is provided with a new set of examples (data) so that supervised learning algorithm analyses the training data(set of training examples) and produces a correct outcome from labelled data. Deep learning (also known as deep structured learning or hierarchical learning) is part of an extensive family of machine learning methods based on artificial neural networks. Learning can be supervised, semi-supervised or unsupervised. Deep learning architectures such as deep neural networks, deep belief networks, recurrent neural networks and convolutional neural networks have been applied to fields including computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioinformatics, drug design, medical image analysis, material inspection and board game programs, where they have produced results comparable to and in some cases superior to human experts. Using such a salient application we can burgeon an implementation which will provide means for communication for people with partial and complete paralysis. The paralytic people cannot use their hands for communication but their fingers can be used to draw letters. Although there might be huge differences in the making of letters every time they draw due to paralysis but with immense execution and empiricism, it can be achieved. Moreover, by inculcation of gesture detection , this can help as a communication aid for people with speaking and hearing disabilities who rely on sign language for communicating their messages.
INTRODUCTION
Gesture Detection and Hand Written Letter Recognition System
This a software tool that can be used to recognize the handwritten textual characters along with hand made gestures. The tool takes in input from the user via touchpad and image captures from the camera device for analysing and generating the output after interpreting the provided input characters and gestures.
A brief description of all the features are given below:
Hand Written Letter Recognition
An easily comprehensible way of taking input from the user via a trackpad, digital touchpad or other input hardware is provided. The input is then interpreted by using a Convoluted Neural Network and Supervised Learning techniques in order to generate a digital version of the input as the output.
Gesture Detection
A user friendly way of capturing the gestures done by the user through the camera module is provided. The gesture is captured in the form of a sequence of images which are then analysed by the CNN model for generating the output message.
Database Creation
The user is provided a feature to use his inputs as samples for the database, in order to improve the accuracy of the CNN model.