How to format training input and output data on Keras

Question

I am new to Deep Learning and I struggle with some data format on Keras. My CNN is based on the Stacked Hourglass Networks for Human Pose Estimation from A.Newell et al.

On this network the input is a 256x256 RGB image and the output should be a 64x64 heatmap highlighting body joints (shoulder, knee,...). I manage to build the network and I have all the data (images) with their annotations (pixel labels for body joints). I was wondering how should I format the Input and Output Data of the training set to train my model. Currently I use a numpy array (256,256,3) for an image and I don't know how to format my output. Should I create a table [n,64,64,7]? (n being the size of the training set and 7 is the number of filters I use to obtain a heatmap for 7 joints)

Thank you for your time.

Answer 1

The output can also be a numpy array. Consider this example: Training set: 50 images of size 256x256x3. This can be combined into a single numpy array of shape(50, 256, 256, 3). Similar approach to format the output data. Sample code below:

    #a, b and c are arrays of size 256x256x3
    import numpy as np

    temp = []
    temp.append(a)
    temp.append(b)
    temp.append(c)
    output_labels = []
    output_labels = np.stack(temp)

The output_labels array will be of shape(3x256x256x3).

Answer 2

Keras recommend to create data generator to feed training data and ground truth to network. Specific to stacked hourglass network case, you can refer to my implementation for details https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras/tree/master/src/data_gen