[英]irregular/varying batch size in tensorflow?
I have a tensorflow dataset and would like to batch it such that batches do not have the same size - something like examples being grouped in batches whose sizes are defined by a vector of values rather than a fixed value.我有一个tensorflow数据集,想批就这样批不具有相同的尺寸-类似的例子分批其大小由值的向量,而不是一个固定值的定义进行分组。
Is there a way to do it within tensorflow?有没有办法在张量流中做到这一点?
And for a network without fixed batch size, is feeding irregular batches going to be a problem?而对于一个没有固定批量大小的网络,喂食不规则的批次会成为一个问题吗?
Thanks in advance!提前致谢!
The answer is yes.答案是肯定的。 model.fit() method allows to pass to it a generator which will generate randomly-sized batches.
model.fit() 方法允许将生成器传递给它,该生成器将生成随机大小的批次。
x_train_batches = ... # some list of data batches of uneven length
y_train_batches = ... # some list of targets of SAME lengths as x_train_batches
def train_gen(x_train_batches, y_train_batches):
i = 0
num_batches = len(x_train_batches)
while True:
yield (x_train_batches[i%num_batches], y_train_batches[i%num_batches])
i += 1
model.fit(train_gen(x_train_batches, y_train_batches), epochs=epochs, steps_per_epoch=NUM_BATCHES)
Another, more elegant, way would be to subclass tf.keras.utils.Sequence
and feed it to the model:另一种更优雅的方法是将
tf.keras.utils.Sequence
子类tf.keras.utils.Sequence
并将其提供给模型:
class UnevenSequence(keras.utils.Sequence):
def __init__(self, x_batches, y_batches):
# x_batches, y_batches are lists of uneven batches
self.x, self.y = x_batches, y_batches
def __len__(self):
return len(self.x)
def __getitem__(self, idx):
batch_x = self.x[idx]
batch_y = self.y[idx]
return (batch_x, batch_y)
my_uneven_sequence = UnevenSequence(x_train_batches, y_train_batches)
model.fit(my_uneven_sequence, epochs=10)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.