tf.keras: feeding bool type variable along the input in model.fit method

Question

I am building a model and I need to pass a bool type variable with True value with model.fit method and with False value with model.predict because I want need to run a condition(tf.condition or switch) function that works differently when the conditional bool variable is true or false. When I tried to feed it along the input, it gives me error with validation split as input shape is (89084,127) while bool is just a bool value. Can anyone suggest me how to pass this bool variable to the model?

My code is here:

Model

######### \no return sequences
import tensorflow.keras.backend as K
global flag
flag=True
input_size=127
dummy=np.zeros((1,19),dtype=np.float64)
dummy=tf.convert_to_tensor(dummy)
embedding_size=100
lstm_size=128
learn_rate=0.01
drop_out=0.1
output_size=19

condition = Input( shape=[], dtype=bool,name = "condition")
#condition=K.placeholder(shape=(1,),dtype=tf.bool)
#----------------------------Model for current utterance--------------------------------
current_input=Input(shape=(input_size,)) 
emb_current = Embedding(vocab_size, embedding_size,weights=[embedding_matrix], input_length=input_size, name='current_embed',trainable=False)(current_input)
out_current_b1=Bidirectional(LSTM(units=lstm_size, return_sequences=True))(emb_current )
attention_vector_current=attention(return_sequences=False)(out_current_b1)
out_current = Reshape((1,attention_vector_current.shape[1]))(attention_vector_current)
context1_input_da=Input(shape=(output_size,))
#-----------------------------Combined Model-------------------------------------------
def true_fn():
       
    prev_DA=Reshape((1,output_size))(context1_input_da)
    combined_train= Concatenate(axis=-1,name='true_train')([prev_DA, out_current])
    return combined_train
def false_fn():
  combined_without_previous = Concatenate(axis=-1,name='without_context')([out_current])
  hidden_without_previous =LSTM(units=lstm_size,name="lstm_test")(combined_without_previous)
  output_without_previous = Dense(units=19, activation='softmax')(hidden_without_previous)
  # Slice the output_without_previous 
  previous_scores = Concatenate(axis=-2, name='with_dummy_scores')([dummy, output_without_previous[:-1]])
  previous_scores= tf.expand_dims(previous_scores, 1)
  print('previous',previous_scores.shape)
  combined_test= Concatenate(axis=-1,name='False_test')([previous_scores, out_current])
  return combined_test

combined = K.switch(condition, true_fn, false_fn)  
hidden=LSTM(units=lstm_size)(combined)
output = Dense(units=len(unique_tag_set), activation='softmax')(hidden)

model = Model(inputs=[current_input,context1_input_da], outputs=output)
opt  = Adam(lr=learn_rate)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

Fit method:

is_freeze= K.placeholder(shape=None,dtype=tf.bool)
is_freeze=True

history=model.fit([utt_minus_one,utt,prev_da_train], y_train,
                  epochs=1,batch_size=256,
                  shuffle = True, verbose = 1,
                  validation_split=0.2,
                  class_weight=custom_weight_dict) 

How can I access this is_freeze bool variable in model or how can I pass it within fit method?

Answer 1

It's not really the actual answer to your question but a more suitable way to solve the problem.

The problem about the method you asked is that you already make the model by the time you train but you need to make a new model with different structure as you change the way you make the layer entirely.

The trick is you don't need to make one model to do everything. You can use the same layers to build 2 or 3 models and they'll all share the same weights.

train_hidden=LSTM(units=lstm_size)(true_fn())
train_output = Dense(units=len(unique_tag_set), activation='softmax')(hidden)

inference_hidden=LSTM(units=lstm_size)(false_fn())
inference_output = Dense(units=len(unique_tag_set), activation='softmax')(hidden)

train_model = Model(inputs=[current_input,context1_input_da], outputs=train_output )
inference_model = Model(inputs=[current_input,context1_input_da], outputs=inference_output )
opt  = Adam(lr=learn_rate)
train_model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

train_model.fit(x,y)
inference_model.predict(x)

The problem is your false_fn() has these layers

  combined_without_previous = Concatenate(axis=-1,name='without_context')([out_current])
  hidden_without_previous =LSTM(units=lstm_size,name="lstm_test")(combined_without_previous)
  output_without_previous = Dense(units=19, activation='softmax')(hidden_without_previous)

Which will not get trained as you don't use this function when fit so there's no way you can use this model build with false_fn() to do anything unless you train it first.