In Keras, how could I input two images to a Dense layer?

Question

I would like to use Keras and TensorFlow to do the folowing:

I have 10.000 sets of three images, each 8x8 bytes with a color value of 0 to 16 in my training data. I call them image A, B and C. (actually they are represented as three strings of 64 hexadecimal chars in a csv file)

C is a sort of combination of A and B.

What I would like to do is train a network so that it gets A and B as inputs and C as a label. Then feed two images (A,B) the network has not seen before and generate C.

What I have done to read the csv file is:

import pandas as pds
training_data_filename = 'training_data.csv'

dataframeX = pds.read_csv(training_data_filename, usecols=[2, 3])
dataframeY = pds.read_csv(training_data_filename, usecols=[4])

print(dataframeX.head())
print(dataframeY.head())

The output is:

                                                   A  \
0  0x7ff10d4d9fc927fc0fdfcd43e1715a34b7def9d77299...   
1  0xe6da2b370a3b5b42cf2bc0a082e2e0165f2f321bd10c...   
2  0xf179a821c24164fcacd6e75b15892fa8c8e42cb1571a...   
3  0x27eb1825923ef55e80491199ec68438b94857f02ed47...   
4  0x77bc45a7728b4578066639ef4525f0fe5d80d450595c...   

                                                   B  
0  0xd9b40a45b5df976433ebdfddcfe290a66ecf576d18ae...  
1  0xf765866850270f595d8239acee9d0d8634249b9d4ac6...  
2  0x61cc8870ca4690953184f3b680bcdc38039215d2174b...  
3  0x515f6afe8e6f9098abc0da8807ee1070947a4686edd0...  
4  0x8ffb621c0398e392d0be0000e3088649eced85dd0c45...  
                                                   C
0  0x35c1f6ebc3ef2d424ced351b65be8c396f8396d69411...
1  0x4421ce104498ad13b639d6f26b105b4be092f3786f3b...
2  0xcbab77b6c025fe66f967f46ff6138f034dc28fedcb50...
3  0x1f6c372b5d2dfecf29592f71a49dfda5fb930e4e90d5...
4  0x2618f3b170046de775727421ed122174f31ae15fa555...

But I don't understand how to feed (A,B) to a Dense layer and use C as the desired output. How do I build my model?

Answer 1

You can not feed Strings into a dense layer. The dense layer expects a vector of numbers as input. So the first step would be to convert your input images to a numpy array (I am not sure why your images are hexadecimal strings).

Eg:

import cv2
numpy_array = cv2.imread("img.jpg")

From there it is pretty easy to feed the numpy array to a dense layer and perform classification for example. For a code example see: Training a simple deep NN on the MNIST dataset

That said, if I understand you correctly, you want to genereate an image C from two images A and B. This is not a job for a deep neural network. If this is what you want to do, have a look and Convolutional Neural Networks and Generative Adversarial Networks.