[英]How to create Cifar-10 subset?
I would like to train a deep neural network using fewer training data samples to reduce the time for testing my code.我想用更少的训练数据样本来训练一个深度神经网络,以减少测试我的代码的时间。 II wanted to know how to subset the Cifar-10 dataset using Keras TensorFlow.I have the following code which is training for Cifar-10 complete dataset.
我想知道如何使用 Keras TensorFlow 对 Cifar-10 数据集进行子集化。我有以下代码,用于训练 Cifar-10 完整数据集。
#load and prepare data
if WhichDataSet == 'CIFAR10':
(x_train, y_train), (x_test, y_test) = tensorflow.keras.datasets.cifar10.load_data()
else:
(x_train, y_train), (x_test, y_test) = tensorflow.keras.datasets.cifar100.load_data()
num_classes = np.unique(y_train).shape[0]
K_train = x_train.shape[0]
input_shape = x_train.shape[1:]
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
y_train = tensorflow.keras.utils.to_categorical(y_train, num_classes)
y_test = tensorflow.keras.utils.to_categorical(y_test, num_classes)
Use the pandas module to create a data frame and sample it accordingly.使用 pandas 模块创建数据帧并相应地对其进行采样。
import pandas as pd
(train_images1, train_labels), (test_images1, test_labels) = datasets.cifar10.load_data()
# Normalize pixel values to be between 0 and 1
train_images, test_images = train_images1 / 255.0, test_images1 / 255.0
#creating the validation set from the training set
df = pd.DataFrame(list(zip(train_images, train_labels)), columns =['Image', 'label'])
val = df.sample(frac=0.2)
X_train = np.array([ i for i in list(val['Image'])])
y_train = np.array([ [i[0]] for i in list(val['label'])])
The line val = df.sample(frac=0.2)
samples out 0.20 percent of the total data.行
val = df.sample(frac=0.2)
从总数据的 0.20% 中采样。
You can use val = df.sample(n=5000)
if you want a specific number of data records, by setting the n
value accordingly.如果您想要特定数量的数据记录,您可以使用
val = df.sample(n=5000)
,方法是相应地设置n
值。
You can use random_state = 0
if you want the same results every time you run the code.如果您希望每次运行代码时都获得相同的结果,则可以使用
random_state = 0
。 eg:例如:
val = df.sample(n=5000,random_state = 0)
Create susbset based on labels根据标签创建子集
Create a subset of dataset excluding few labels.创建不包括少数标签的数据集子集。 For example, to create a new train dataset with only first five class labels you can use below code
例如,要创建一个只有前五个 class 标签的新火车数据集,您可以使用以下代码
subset_x_train = x_train[np.isin(y_train, [0,1,2,3,4]).flatten()]
subset_y_train = y_train[np.isin(y_train, [0,1,2,3,4]).flatten()]
Create subset irrespective of labels不考虑标签创建子集
To create a 10% subset of train data you can use below code要创建 10% 的训练数据子集,您可以使用以下代码
# Shuffle first (optional)
idx = np.arange(len(x_train))
np.random.shuffle(idx)
# get first 10% of data
subset_x_train = x_train[:int(.10*len(idx))]
subset_y_train = y_train[:int(.10*len(idx))]
Repeat the same for x_test
and y_test
to get a subset of test data.对
x_test
和y_test
重复相同的操作以获取测试数据的子集。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.