Multiple Output Vectors for a single Input in Keras

Question

I want to create a Neural Network in Keras for converting handwriting into computer letters.

My first step is to convert a sentence into an Array. My Array has the shape (1, number of letters,27) . Now I want to input it in my Deep Neural Network and train.

But how do I input it properly if the dimension doesn't fit those from my image? And how do I achieve that my predict function gives me an output array of (1, number of letters,27) ?

Answer 1

Seems like you are attempting to do Handwritten Recognition or similarly Optical Character Recognition or OCR. This is quite a broad field, and there are many ways to proceed. Even though, one approach I suggest is the following:

It is commonly known that Neural Networks have fixed size inputs , that is if you build it to take, say, inputs of shape (28,28,1) then the model will expect that shape as their inputs. Therefore, having a dimension in your samples that depends on the number of letters in a sentence (something variable) is not recommended , as you will not be able to train a model in such way with NNs.

Training such a model could be possible if you design it to predict one character at a time , instead a whole sentence that can have different lengths, and then group the predicted characters. The steps you could try to achieve this could be:

1. Obtain training samples for the characters you wish to recognize (like the MNIST database for example), and design and train your model to predict one character at a time.

2. Take the image with writing to classify and pass a Sliding Window over it that matches your expected input size (say a 28x28 window). Then, classify each of those windows to a character. Instead of Sliding Window, you could try isolating your desired features somehow and just classify those 28x28 segments instead.

3. Group the predicted characters somehow so you get words (probably grouping those separated by empty spaces) or do whatever you want with the predictions.

You can also try searching for tutorials or guides for Handwriting recognition like this one I have found quite useful. Hope this helps you get on track, good luck.