How do i conditionally add layers to a Keras model?

Question

Here is what I have done so far

import itertools

final_param_list = []

param_list_gen = [[8, 16, 32], ["Sigmoid", "ReLU", "Leaky ReLU"],  [10, 20, 50], [1,2]]
for element in itertools.product(*param_list_gen):
    final_param_list.append(element)

the output looks like

[(8, 'Sigmoid', 10, 1), (8, 'Sigmoid', 10, 2), ....]

For each list the values at each index are:

index0 = batch size
index1 = activation funtion
index2 = number of nodes
index3 = number of layers

So in the first list

batch_size = 8
activation='Sigmoid'
units=10
layers=1

What I want to be able to do is loop through the lists in final_param_list = [] and not only set each param but I want to add a hidden layer only when layers=2. I could go the easy route and just create two separate models, one with one hidden layer and the other with 2 hidden layers, and loop through them individually but I want to do something a little more elegant then that.

NOTE: I'm aware that some of this could probably be done with gridsearch AND I am aware that hidden layer 1 and 2 will have the same parameters. Ultimately, I will work my way toward being able to tune them separately but the solution as I have described it will suffice for now.

Answer 1

Ended up exploring a solution that @mkrieger1 made in the comments. Seems like it worked. Here is my code.

for param in final_param_list:
    # ------ model 1 - 1 hidden layer ------ #
    # Check to see if we are calling for one or two layers . If one layer then proceed
if param[3] == 1:
    # hidden layer 1
    q2model1.add(Dense(param[0]))
    if param[1] != 'LeakyReLU':
        q2model1.add(Activation(param[1]))
    else:
        q2model1.add(LeakyReLU(alpha=0.1))
    # output layer
    q2model1.add(Dense(class_num, activation='softmax'))

# ------ model 1 - 2 hidden layers ------ #
else:
    # hidden layer 1
    q2model1.add(Dense(param[0]))
    if param[1] != 'LeakyReLU':
        q2model1.add(Activation(param[1]))
    else:
        q2model1.add(LeakyReLU(alpha=0.1))
    # hidden layer 2
    q2model1.add(Dense(param[0]))
    if param[1] != 'LeakyReLU':
        q2model1.add(Activation(param[1]))
    else:
        q2model1.add(LeakyReLU(alpha=0.1))

    # output layer
    q2model1.add(Dense(class_num, activation='softmax'))

q2model1.compile(loss='sparse_categorical_crossentropy', optimizer='RMSProp', 
metrics=['accuracy'])

history = q2model1.fit(X1, y1, epochs=20)