Multiple Inputs with different sample size in Neural Networks

Question

I am working on a specific neural network which gets two different inputs:

• the MNIST data set, the train set is a [50000,784] tensor
• an auxiliary vector with the TensorShape([Dimension(28)])]

When I define and run the model it as to below

from tensorflow.examples.tutorials.mnist import input_data
from keras.layers import Input, Dense, Lambda
from keras.models import Model
from keras.objectives import binary_crossentropy
from keras.callbacks import LearningRateScheduler
import numpy as np
import keras
import matplotlib.pyplot as plt
import keras.backend as K
import tensorflow as tf
from keras.callbacks import LambdaCallback

def load_dataset(flatten=False):
    (X_train, y_train), (X_test, y_test) = keras.datasets.mnist.load_data()
    # normalize x
    X_train = X_train.astype(float) / 255.
    X_test = X_test.astype(float) / 255.
    # we reserve the last 10000 training examples for validation
    X_train, X_val = X_train[:-10000], X_train[-10000:]
    y_train, y_val = y_train[:-10000], y_train[-10000:]
    if flatten:
        X_train = X_train.reshape([X_train.shape[0], -1])
        X_val = X_val.reshape([X_val.shape[0], -1])
        X_test = X_test.reshape([X_test.shape[0], -1])
    return X_train, y_train, X_val, y_val, X_test, y_test
X_train, y_train, X_val, y_val, X_test, y_test = load_dataset(True)

original_dim=784
m = 100 #batchsize
n_z =8
n_epoch = 10
n_d =int(n_z*(n_z - 1 )/2) #or n_d=28

A_vec = K.random_normal(shape=(n_d,), mean=0., stddev=1.)
image_inputs = Input(shape=(784,))
A_inputs = Input(shape=(n_d,))
inputs = keras.layers.concatenate([image_inputs, A_inputs])

h_q1 = Dense(512, activation='relu')(inputs)
h_q2 = Dense(256, activation='relu')(h_q1)
h_q3 = Dense(128, activation='relu')(h_q2)
h_q4= Dense(64, activation='relu')(h_q3)
mu = Dense(n_z, activation='linear')(h_q4)
log_sigma = Dense(n_z, activation='linear')(h_q4)

 ............

After running the model,

vae.fit([X_train,A_vec], outputs,shuffle=True, batch_size=m, epochs=n_epoch)

I get this error:

ValueError: All input arrays (x) should have the same number of samples. Got array shapes: [(50000, 784), TensorShape([Dimension(28)])]

It means my inputs have different sizes. How can I use differetn inputs when they have different sizes (or shapes)?

Answer 1

The inputs have to have the same size, eg (50000, 748) and (50000, 28), ie one per sample. Try create a numpy array size (50000, 28) for A_vec : numpy.random.normal(0., 1.0, (50000, 28) .

Or if you want the same vector for all, create it and repeat 50000 times.