如何加载多个训练和有效性数据来训练和验证 keras 模型

Question

I have both training and validity data stored in train_data and valid_data folders.In both the folders data stored in .npz file. And each .npz file contain target and label data with same shape =(1024,28)(for example target=tu.npz['data1'] and label=tu.npz['data2']. I want to load them to a Keras model by a manner similar to ImageDataGenerator and want to train and validate the model, so I wrote and tried different custom generators but it is not working, here is my code. I hope somebody will help me.Thanks.

def tf_train_generator(file_list, batch_size = 1):
    i = 0
    while True:
        if i*batch_size >= len(file_list):  
            i = 0
            np.random.shuffle(file_list)
        else:
            file_chunk = file_list[i*batch_size:(i+1)*batch_size]
            print(len(file_chunk))      


            for file in file_chunk:
                print(file)
                temp = np.load(file)

               
                X = temp['data1']
               
                Y= temp['data2']  


               
                i = i + 1
                yield X, Y

Answer 1

Move the (i = i + 1 and yield X, Y) out of the for loop.

    i= 0
    def tf_train_generator(file_list, batch_size = 1):
        global i
        while True:
            if i*batch_size >= len(file_list):  
                i = 0
                np.random.shuffle(file_list)
            else:
                file_chunk = file_list[i*batch_size:(i+1)*batch_size]
                print(len(file_chunk))      
    
    
                for file in file_chunk:
                    print(file)
                    temp = np.load(file)
    
                   
                    X = temp['data1']
                    Y= temp['data2'] 
                   
            i = i + 1
            yield X, Y