Keras Functional API Multiple Input Model

Question

I'm trying to create a multiple-input model with Functional API in Keras with this structure:

There are three inputs: Team_1_In , Team_2_In , Home_In . Where Team_1_In and Team_2_In go through an Embedding layer, then BatchNormalization and Flatten layers. The problem is when I'm trying to add Flatten layer after BatchNormalization I get this error:

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last) <ipython-input-46-8354b255cfd1> in <module>
     15 batch_normalization_2 = BatchNormalization()(team_2_strength)
     16 
---> 17 flatten_1 = Flatten()(batch_normalization_1)
     18 flatten_2 = Flatten()(batch_normalization_2)
     19 

~/conda/lib/python3.6/site-packages/keras/engine/topology.py in
__call__(self, inputs, **kwargs)
    573                 # Raise exceptions in case the input is not compatible
    574                 # with the input_spec specified in the layer constructor.
--> 575                 self.assert_input_compatibility(inputs)
    576 
    577                 # Collect input shapes to build layer.

~/conda/lib/python3.6/site-packages/keras/engine/topology.py in assert_input_compatibility(self, inputs)
    488                                      self.name + ': expected min_ndim=' +
    489                                      str(spec.min_ndim) + ', found ndim=' +
--> 490                                      str(K.ndim(x)))
    491             # Check dtype.
    492             if spec.dtype is not None:

ValueError: Input 0 is incompatible with layer flatten_10: expected min_ndim=3, found ndim=2 

I tried to play with axis parameter of BatchNormalization layer but it didn't help. Here is my code:

# create embedding layer
from keras.layers import Embedding
from keras.layers import BatchNormalization, Flatten, Dense
from numpy import unique

# Create an embedding layer
team_lookup = Embedding(input_dim=n_teams,
                        output_dim=1,
                        input_length=1,
                        name='Team-Strength')

# create model with embedding layer
from keras.layers import Input, Embedding, Flatten
from keras.models import Model

# Create an input layer for the team ID
teamid_in = Input(shape=(1,))

# Lookup the input in the team strength embedding layer
strength_lookup = team_lookup(teamid_in)

# Flatten the output
strength_lookup_flat = Flatten()(strength_lookup)

# Combine the operations into a single, re-usable model
team_strength_model = Model(teamid_in, strength_lookup_flat, name='Team-Strength-Model')



# Create an Input for each team
team_in_1 = Input(shape=(1,), name='Team-1-In')
team_in_2 = Input(shape=(1,), name='Team-2-In')

# Create an input for home vs away
home_in = Input(shape=(1,), name='Home-In')

# Lookup the team inputs in the team strength model
team_1_strength = team_strength_model(team_in_1)
team_2_strength = team_strength_model(team_in_2)

batch_normalization_1 = BatchNormalization()(team_1_strength)
batch_normalization_2 = BatchNormalization()(team_2_strength)

flatten_1 = Flatten()(batch_normalization_1)
flatten_2 = Flatten()(batch_normalization_2)

# Combine the team strengths with the home input using a Concatenate layer, then add a Dense layer
out = Concatenate()([flatten_1, flatten_2, home_in])
out = Dense(1)(out)

Answer 1

As shown in the error, you need a 3D tensor for Flatten layer:

ValueError: Input 0 is incompatible with layer flatten_10: expected min_ndim=3, found ndim=2

In the first part of the code, where you passed your input to the embedding layer, everything is fine and it compiles successfully:

team_lookup = Embedding(input_dim=1,
                        output_dim=1,
                        input_length=1,
                        name='Team-Strength')

strength_lookup = team_lookup(teamid_in)

batch_normalization_1 = BatchNormalization()(strength_lookup)

strength_lookup_flat = Flatten()(batch_normalization_1)

team_strength_model = Model(teamid_in, strength_lookup_flat, name='Team-Strength-Model')
team_strength_model.compile(optimizer='adam', loss='categorical_crossentropy')

But in the second part the code, where you pass inputs to the team_strength_model it flatten the tensors which their shape converted to (batch, flatten) . When you pass this 2D tensor to BatchNormalization it throws such an exception.

To fix the issue :

1) Pass the input to the Embedding layer

2) Do BatchNormalization

3) Flatten the BatchNormalization 's output

Answer 2

This is a question about an exercise on Datacamp: https://campus.datacamp.com/courses/advanced-deep-learning-with-keras-in-python/multiple-inputs-3-inputs-and-beyond?ex=11

I reproduce it here not to infringe upon their IP, but to explain it a little more clearly. One of the issues I have with sites like Datacamp (which I use heavily) is that if you don't understand what's going on, there's no real recourse, aside from posting here on SO.

The question is not clearly phrased in Datacamp. The exercise introduces ensemble models; here, one model makes a prediction about the score differential between two basketball teams based on regular-season data, and another model makes a prediction about score differential based on that earlier prediction along with new data.

Unfortunately, in the exercise, both models are just called model , they are presented out of order, and the actual model used in the exercise is never clearly shared with the student. Oops.

Here is the first model. Not that this differs from the OP's model, in that it uses an embedding layer , not an embedding model . Again, the Datacamp exercise is not clear on this, and the nature of the Embedding is changed without informing the student. It predicts a score difference between two teams based on an integer id (dumb) and which team is home or away.

# Create an Input for each team
team_in_1 = Input(shape=(1,), name='Team-1-In')
team_in_2 = Input(shape=(1,), name='Team-2-In')

# Create an input for home vs away
home_in = Input(shape=(1,), name='Home-In')

# Create an embedding layer
team_lookup = Embedding(input_dim=n_teams,
                        output_dim=1,
                        input_length=1,
                        name='Team-Strength')

team_1_strength = team_lookup(team_in_1)
team_2_strength = team_lookup(team_in_2)

batch_normalization_1 = BatchNormalization(axis=1)(team_1_strength)
batch_normalization_2 = BatchNormalization(axis=1)(team_2_strength)

# flatten batch_normalization_1
flatten_1 = Flatten()(batch_normalization_1)

# flatten batch_normalization_2
flatten_2 = Flatten()(batch_normalization_2)

out = Concatenate()([flatten_1, flatten_2, home_in])
out = Dense(1)(out)

model = Model([team_in_1, team_in_2, home_in], out)
model.compile(optimizer="adam", loss="mean_absolute_error")

This model is then fit on the regular season data:

model.fit([games_season["team_1"], games_season["team_2"], games_season["home"]],
          games_season["score_diff"],
          epochs=1,
          verbose=True,
          validation_split=0.1,
          batch_size=2048)

... and used to predict a score difference for a different data set:

# Predict
games_tourney['pred'] = model.predict(
    [games_tourney["team_1"], 
     games_tourney["team_2"], 
     games_tourney["home"]])

Now split this data set into train and test.

games_tourney_train = games_tourney.loc[games_tourney["season"] < 2010]
games_tourney_test = games_tourney.loc[games_tourney["season"] >= 2010]

Here is the second model, which I've called preds_model for no specific reason:

# Create an input layer with 3 columns
input_tensor = Input((3, ))

# Pass it to a Dense layer with 1 unit
output_tensor = Dense(1)(input_tensor)

# Create a model
preds_model = Model(input_tensor, output_tensor)

# Compile the model
preds_model.compile(optimizer="adam", loss="mean_absolute_error")

preds_model.fit(games_tourney_train[['home', 'seed_diff', 'pred']],
          games_tourney_train['score_diff'],
          epochs=1,
          verbose=True)

And finally, evaluate your (first) ensemble model:

# Evaluate the model on the games_tourney_test dataset
print(preds_model.evaluate(games_tourney_test[["home", "seed_diff", "pred"]],
               games_tourney_test["score_diff"], verbose=False))

And your output:

11.94

I suspect there is a more straightforward introduction to ensemble models out there.