AttributeError: 'tensorflow.python.framework.ops.EagerTensor' 對象沒有屬性 'to_tensor'

Question

我正在使用 Hugging Face、Keras、Tensorflow 庫微調 BERT 模型。

從昨天開始，我在 Google Colab 中運行我的代碼時遇到此錯誤。 奇怪的是，以前運行的代碼沒有任何問題，突然開始拋出這個錯誤。 更令人懷疑的是，代碼在我的 Apple M1 tensorflow 配置中運行沒有問題。 同樣，我沒有對我的代碼進行任何更改，但現在代碼無法在 Google Colab 中運行，盡管它過去運行時沒有任何問題。

兩種環境都有 tensorflow 2.6.0

錯誤截圖

我創建了下面的代碼來重現錯誤。 我希望你能對此有所了解。

!pip install transformers
!pip install datasets

import pandas as pd
import numpy as np
import tensorflow as tf
from transformers import AutoTokenizer
from datasets import Dataset

# dummy sentences
sentences = ['the house is blue and big', 'this is fun stuff','what a horrible thing to say']

# create a pandas dataframe and converto to Hugging Face dataset
df = pd.DataFrame({'Text': sentences})
dataset = Dataset.from_pandas(df)

#download bert tokenizer
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')

# tokenize each sentence in dataset
dataset_tok = dataset.map(lambda x: tokenizer(x['Text'], truncation=True, padding=True, max_length=10), batched=True)

# remove original text column and set format
dataset_tok = dataset_tok.remove_columns(['Text']).with_format('tensorflow')

# extract features
features = {x: dataset_tok[x].to_tensor() for x in tokenizer.model_input_names}

Answer 1

刪除to_tensor()給定的代碼按照@Harold G 的建議工作。

!pip install transformers
!pip install datasets

import pandas as pd
import numpy as np
import tensorflow as tf
from transformers import AutoTokenizer
from datasets import Dataset

# dummy sentences
sentences = ['the house is blue and big', 'this is fun stuff','what a horrible thing to say']

# create a pandas dataframe and converto to Hugging Face dataset
df = pd.DataFrame({'Text': sentences})
dataset = Dataset.from_pandas(df)

#download bert tokenizer
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')

# tokenize each sentence in dataset
dataset_tok = dataset.map(lambda x: tokenizer(x['Text'], truncation=True, padding=True, max_length=10), batched=True)

# remove original text column and set format
dataset_tok = dataset_tok.remove_columns(['Text']).with_format('tensorflow')

# extract features
features = {x: dataset_tok[x] for x in tokenizer.model_input_names}