运行BERT model后如何解决属性错误

Question

运行 BERT model 后收到错误。 至此，代码运行成功。 我收到的错误是 AttributeError: 'str' object has no attribute 'shape'。 代码之前的上一步是创建自定义数据生成器。 使用它创建了 model。 为了提供上下文，我使用的 model 位于网站https://keras.io/examples/nlp/semantic_similarity_with_bert/上，我用它来解释我自己的数据。

from ipywidgets import IntProgress

strategy = tf.distribute.MirroredStrategy()

with strategy.scope():
    # Encoded token ids from BERT tokenizer.
    input_ids = tf.keras.layers.Input(
        shape=(max_length,), dtype=tf.int32, name="input_ids"
    )
    # Attention masks indicates to the model which tokens should be attended to.
    attention_masks = tf.keras.layers.Input(
        shape=(max_length,), dtype=tf.int32, name="attention_masks"
    )
    # Token type ids are binary masks identifying different sequences in the model.
    token_type_ids = tf.keras.layers.Input(
        shape=(max_length,), dtype=tf.int32, name="token_type_ids"
    )
    # Loading pretrained BERT model.
    bert_model = transformers.TFBertModel.from_pretrained("bert-base-uncased")
    # Freeze the BERT model to reuse the pretrained features without modifying them.
    bert_model.trainable = False

    sequence_output, pooled_output = bert_model(
        input_ids, attention_mask=attention_masks, token_type_ids=token_type_ids
    )
    # Add trainable layers on top of frozen layers to adapt the pretrained features on the new data.
    bi_lstm = tf.keras.layers.Bidirectional(
        tf.keras.layers.LSTM(64, return_sequences=True)
    )(sequence_output)
    # Applying hybrid pooling approach to bi_lstm sequence output.
    avg_pool = tf.keras.layers.GlobalAveragePooling1D()(bi_lstm)
    max_pool = tf.keras.layers.GlobalMaxPooling1D()(bi_lstm)
    concat = tf.keras.layers.concatenate([avg_pool, max_pool])
    dropout = tf.keras.layers.Dropout(0.3)(concat)
    output = tf.keras.layers.Dense(3, activation="softmax")(dropout)
    model = tf.keras.models.Model(
        inputs=[input_ids, attention_masks, token_type_ids], outputs=output
    )

    model.compile(
        optimizer=tf.keras.optimizers.Adam(),
        loss="categorical_crossentropy",
        metrics=["acc"],
    )


print(f"Strategy: {strategy}")
model.summary()





---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-35-e6d50369bfa4> in <module>
     25     )
     26     # Add trainable layers on top of frozen layers to adapt the pretrained features on the new data.
---> 27     bi_lstm = tf.keras.layers.Bidirectional(
     28         tf.keras.layers.LSTM(64, return_sequences=True)
     29     )(sequence_output)

~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\layers\wrappers.py in __call__(self, inputs, initial_state, constants, **kwargs)
    528 
    529     if initial_state is None and constants is None:
--> 530       return super(Bidirectional, self).__call__(inputs, **kwargs)
    531 
    532     # Applies the same workaround as in `RNN.__call__`

~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in __call__(self, *args, **kwargs)
    980       with ops.name_scope_v2(name_scope):
    981         if not self.built:
--> 982           self._maybe_build(inputs)
    983 
    984         with ops.enable_auto_cast_variables(self._compute_dtype_object):

~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\base_layer.py in _maybe_build(self, inputs)
   2615     # Check input assumptions set before layer building, e.g. input rank.
   2616     if not self.built:
-> 2617       input_spec.assert_input_compatibility(
   2618           self.input_spec, inputs, self.name)
   2619       input_list = nest.flatten(inputs)

~\anaconda3\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\keras\engine\input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
    164         spec.min_ndim is not None or
    165         spec.max_ndim is not None):
--> 166       if x.shape.ndims is None:
    167         raise ValueError('Input ' + str(input_index) + ' of layer ' +
    168                          layer_name + ' is incompatible with the layer: '

AttributeError: 'str' object has no attribute 'shape'

Answer 1

model 的输入不能是字符串。 它应该是一个 numpy 数组或张量。 您应该对字符串进行编码，将字符转换为 numpy 数组。

从https://keras.io/examples/nlp/semantic_similarity_with_bert/也有用于标记化的代码示例：

self.tokenizer = transformers.BertTokenizer.from_pretrained(
            "bert-base-uncased", do_lower_case=True
        )

        # With BERT tokenizer's batch_encode_plus batch of both the sentences are
        # encoded together and separated by [SEP] token.
        encoded = self.tokenizer.batch_encode_plus(
            sentence_pairs.tolist(),
            add_special_tokens=True,
            max_length=max_length,
            return_attention_mask=True,
            return_token_type_ids=True,
            pad_to_max_length=True,
            return_tensors="tf",
        )

        # Convert batch of encoded features to numpy array.
        input_ids = np.array(encoded["input_ids"], dtype="int32")
        attention_masks = np.array(encoded["attention_mask"], dtype="int32")
        token_type_ids = np.array(encoded["token_type_ids"], dtype="int32")