[英]What is tensorflow.python.data.ops.dataset_ops._OptionsDataset?
I am using the Transformer code from tensorflow - https://www.tensorflow.org/beta/tutorials/text/transformer我正在使用来自 tensorflow 的 Transformer 代码 - https://www.tensorflow.org/beta/tutorials/text/transformer
In this code, the dataset used is loaded like this -在这段代码中,使用的数据集是这样加载的 -
examples, metadata = tfds.load('ted_hrlr_translate/pt_to_en', with_info=True,
as_supervised=True)
train_examples, val_examples = examples['train'], examples['validation']
When I check the type of train_examples using :当我使用以下命令检查 train_examples 的类型时:
type(train_examples)
I get the following as output -我得到以下作为输出 -
tensorflow.python.data.ops.dataset_ops._OptionsDataset
Now I just wanted to change some entries of the dataset that is the sentences, but I am not able to as I don't understand the type.现在我只想更改数据集的一些条目,即句子,但我无法理解,因为我不理解类型。
I am able to iterate over it using :我可以使用以下方法对其进行迭代:
for data in train_examples:
print(data,type(data))
And type of data is -数据类型是 -
<class 'tuple'>
Finally what I want is to replace some of these tuples with my own data.最后我想要的是用我自己的数据替换这些元组中的一些。 Can someone tell me how to do this or give me some details about this type
tensorflow.python.data.ops.dataset_ops._OptionsDataset
.有人能告诉我怎么做,或者给我一些关于这种类型的细节
tensorflow.python.data.ops.dataset_ops._OptionsDataset
。
tensorflow.python.data.ops.dataset_ops._OptionsDataset
is just another class extending the base class tf.compat.v2.data.Dataset
(DatasetV2) which holds tf.data.Options
along with the original tf.compat.v2.data.Dataset
dataset (The Portuguese-English tuples in your case). tensorflow.python.data.ops.dataset_ops._OptionsDataset
只是扩展基类tf.compat.v2.data.Dataset
(DatasetV2) 的另一个类,它包含tf.data.Options
以及原始的tf.compat.v2.data.Dataset
集tf.compat.v2.data.Dataset
集(您的案例中的葡萄牙语-英语元组)。
( tf.data.Options
operates when you are using streaming functions over your dataset tf.data.Dataset.map
or tf.data.Dataset.interleave
) (
tf.data.Options
在您对数据集tf.data.Dataset.map
或tf.data.Dataset.interleave
使用流函数时tf.data.Dataset.interleave
)
How to view the individual elements?如何查看单个元素?
I'm sure there are many ways, but one straight way would be to use the iterator in the base class:我确信有很多方法,但一种直接的方法是在基类中使用迭代器:
Since examples['train']
is a type of _OptionsDataset
here is iterating by calling a method from tf.compat.v2.data.Dataset
由于
examples['train']
是_OptionsDataset
一种类型,这里通过调用tf.compat.v2.data.Dataset
的方法进行tf.compat.v2.data.Dataset
iterator = examples['train'].__iter__()
next_element = iterator.get_next()
pt = next_element[0]
en = next_element[1]
print(pt.numpy())
print(en.numpy())
Here is the output:这是输出:
b'o problema \xc3\xa9 que nunca vivi l\xc3\xa1 um \xc3\xbanico dia .'
b"except , i 've never lived one day of my life there ."
Substituting with your own data:用您自己的数据替换:
Since you've not mentioned what you want to substitute the original dataset with, I'll assume you have a CSV/TSV file of your own specific translations.由于您没有提到要用什么来替换原始数据集,我假设您有一个包含自己特定翻译的 CSV/TSV 文件。 Then it should be useful to create a separate
tf.compat.v2.data.Dataset
object itself by calling the CSV API to read your CSV file into a dataset:然后通过调用 CSV API 将您的 CSV 文件读入数据集来创建一个单独的
tf.compat.v2.data.Dataset
对象本身应该很有用:
tf.data.experimental.make_csv_dataset
https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/load_data/csv.ipynb https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/load_data/csv.ipynb
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.