我如何使用 2 numpy arrays 作为去噪自动编码器的数据集，并将它们进一步拆分为训练集和测试集

Question

I have 2 numpy arrays, one with clean data [4000 x [1000][25]] (4000 1000x25 arrays) and one with noisy data (same size as clean) to be used for a de-noising auto-encoder problem.

I want to be able to either map them and then store them into a tensorflow data set, or any other way which allows me to do this

clean[i] -> De-noising Autoencoder -> noisy[i]

Also implement a train and test split in a way that mapping remains.

I'm sorry if this is too vague, I'm new to ML and python.

Answer 1

assume you have your clean data in an array clean_data and your noisy data in an array noisy_data. Then use train_test_split from sklearn to split the data into a training set and a test as follows

from sklearn.model_selection import train_test_split
train_size=.7  # set this to the percentage you want for training
clean_train, clean_test, noisy_train, noisy_test=train_test_split(clean_data, noisy_data,
                                                 train_size=train_size, randon_state=123)