Keras 功能 api 多输入：传递给 model 的输入列表是冗余的

Question

. Answers to this question are eligible for a +50 reputation bounty. chefhose wants to to this question.

I have a huge networt ( keras-bert ) which works fine for classification. Since my data has two different columns, I'd like to fine-tune a BERT model for each column and connect them in the final layer. But I get the following error:

---> 20 model = keras.models.Model(inputs=[inputs1, inputs2], outputs=outputs)

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/network.py in _validate_graph_inputs_and_outputs(self)

1620 """Validates the inputs and outputs of a Graph Network."""

1621 # Check for redundancy in inputs.

-> 1622 if len(set(self.inputs)).= len(self:inputs):

1623 raise ValueError('The list of inputs passed to the model '

1624 'is redundant. '

TypeError: unhashable type: 'list'

In my code I have two bert models, model1 and model2 . With just one model it worked fine. The only things I added were that 2 models instead of one are loaded from checkpoint and the second input-layer and the concatenation of dense1 and dense2:

#load_trained_model_from_checkpoint is defined here:
# https://github.com/CyberZHG/keras-bert/blob/master/keras_bert/loader.py
model1 = load_trained_model_from_checkpoint(
    config_path,
    checkpoint_path,
    training=True,
    trainable=True,
    seq_len=SEQ_LEN,
    )
model2 = load_trained_model_from_checkpoint(
    config_path,
    checkpoint_path,
    training=True,
    trainable=True,
    seq_len=SEQ_LEN,
)

inputs1 = model1.inputs[:2] #model 1 for titles
inputs2 = model2.inputs[:2] #model 2 for texts
dense1 = model1.get_layer('NSP-Dense').output
dense2 = model2.get_layer('NSP-Dense').output
outputs = keras.layers.Dense(len(test_title_y[0]), activation='sigmoid')(keras.layers.concatenate([dense1, dense2]))


model = keras.models.Model(inputs=[inputs1, inputs2], outputs=outputs)

What am I overseeing? Do I somehow have to wrap the input?

Edit: I suspect that the problem has something to do with my input being a list of lists: the inputs1 and inputs2 look like that:

[<tf.Tensor 'Input-Token:0' shape=(?, 256) dtype=float32>, <tf.Tensor 'Input-Segment:0' shape=(?, 256) dtype=float32>]
[<tf.Tensor 'Input-Token_1:0' shape=(?, 256) dtype=float32>, <tf.Tensor 'Input-Segment_1:0' shape=(?, 256) dtype=float32>]

Can I somehow reshape or concatenate my input to overcome this error?

Edit:

The summaries of model1 looks like that, model2 looks the same but with LAYER_2 for each layer name:

Layer (type) Output Shape Param # Connected to

---

Input-Token (InputLayer) (None, 256) 0

---

Input-Segment (InputLayer) (None, 256) 0

---

Embedding-Token (TokenEmbedding [(None, 256, 768), ( 23440896 Input-Token[0][0]

---

Embedding-Segment (Embedding) (None, 256, 768) 1536 Input-Segment[0][0]

---

... (lots of layers in between)

---

NSP-Dense (Dense) (None, 768) 590592 Extract[0][0]

Answer 1

It looks like there is an internal check which looks for equal length inputs to a model. In this case both inputs are [batchSize, 256] . One thing that you can do is try packing the two inputs into one with three dimensions (ie [batchSize, inputs, 256] ). You would need to handle some slicing in the model to make sure you are passing the correct input to the layers.

Here is an example:

#load_trained_model_from_checkpoint is defined here:
# https://github.com/CyberZHG/keras-bert/blob/master/keras_bert/loader.py

model1 = load_trained_model_from_checkpoint(
    config_path,
    checkpoint_path,
    training=True,
    trainable=True,
    seq_len=SEQ_LEN,
    )
model2 = load_trained_model_from_checkpoint(
    config_path,
    checkpoint_path,
    training=True,
    trainable=True,
    seq_len=SEQ_LEN,
)

inputs1 = model1.inputs[:2] #model 1 for titles
inputs2 = model2.inputs[:2] #model 2 for texts
# concatenate the inputs into one tensor
concatInputs = keras.backend.concatenate([inputs1, inputs2],axis=0)

dense1 = model1.get_layer('NSP-Dense').output
dense2 = model2.get_layer('NSP-Dense').output

def getModel():
    inputs=keras.layers.Input([2,256])
    # slice inputs into individual tensors
    in1 = keras.layers.Lambda(lambda x: tf.slice(inputs,[0,0],[1,inputs.get_shape()[1]])
    in2 = keras.layers.Lambda(lambda x: tf.slice(inputs,[1,0],[2,inputs.get_shape()[1]])
    d1 = dense1(in1)
    d2 = dense2(in2)
    outputs = keras.layers.Dense(len(test_title_y[0]), activation='sigmoid')(keras.layers.concatenate([d1, d2]))

    return inputs,outputs

ins,outs = getModel()

model = keras.models.Model(inputs=[ins], outputs=[outs])

I didn't get a chance to test so the syntax may not be exact but the idea should work.

Answer 2

This may sound a little copy+paste, but I think you solved your own question when you noticed the inputs are lists of lists.

Instead of using [inputs1, inputs2] , use inputs1 + inputs2 .

Test if they are lists with isinstance(inputs1, list) .

Answer 3

The error means that somewhere, the function is trying to use a list as a hash argument instead. ie using a list as a key in a dictionary.

The Model(inputs=[input1,input2]) cannot take in a list.

Try using

np.reshape

On input1 and input2 first. Additionally, cast the lists to tuples.