how to tuning parameter entering the layer using for loop?

Question

here is my code

model = Sequential()
model.add(LSTM(128, input_shape=(None, 1),return_sequences=True))
model.add(Dropout(0.3))
#I want test 32,64,128,256,512,1024 number of entering the layer

model.add(LSTM(128))
model.add(Dropout(0.3))
#I want test 32,64,128,256,512,1024 number of entering the layer

model.add(Dense(128))
model.add(Dropout(0.3))
#I want test 32,64,128,256,512,1024 number of entering the layer


#and if possible, I want to add more layer using for loop like below
for i in [LSTM, Dense]
    model.add(i,(j))

model.add(Dense(1))

I want to tuning the numbers to LSTM and Dense.

I want to use the for loop to test for the numbers in the code in my comments.

I wonder how it can be implemented.

and I wonder if there is a tool that can tune the parameters like this.

Your valuable opinions and thoughts will be very much appreciated.

Answer 1

You can build a list with all possible configuration for each parameter in your model you want to tune. Something like this:

all_configurations = [
    (32, 64, 128, 256, 512, 1024), # Number of output for the 1st layer
    (32, 64, 128, 256, 512, 1024), # Outputs for the 2nd layer
    (32,64,128,256,512,1024) # Outputs for the 3th layer
]

Now you can do:

from itertools import product

def test_nn(a, b, c):
    # a is the number of outputs for 1st layer, b for the 2nd and c for 3th
    # Build network with those parameters and test it
    # TODO
    pass

for configuration in product(all_configurations):
    test_nn(*configuration)

For each possible configuration of your three hyperparameters, test_nn will be called. Build and test your network inside that function