加載時運行 SageMaker Batch 轉換失敗 model

Question

我正在嘗試使用 HuggineFace class 和微調 model 和自定義推理文件運行批量轉換作業。 加載 model 時作業失敗，但我可以在本地加載它。 我需要制作自定義推理文件，因為我需要保持輸入文件原樣，所以我不得不更改輸入 json 文件中的輸入鍵。

以下是例外情況：

PredictionException(str(e), 400)
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - mms.service.PredictionException: Can't load config for '/.sagemaker/mms/models/model'. Make sure that:
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - - '/.sagemaker/mms/models/model' is a correct model identifier listed on 'https://huggingface.co/models'
2022-05-08 16:49:45,499 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - - or '/.sagemaker/mms/models/model' is the correct path to a directory containing a config.json file
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle - 
2022-05-08 16:49:45,500 [INFO ] W-model-1-stdout com.amazonaws.ml.mms.wlm.WorkerLifeCycle -  : 400

我在腳本模式下運行：

from sagemaker.huggingface.model import HuggingFaceModel

hub = {
   # 'HF_MODEL_ID':'cardiffnlp/twitter-roberta-base-sentiment',
    'HF_TASK':'text-classification',
    'INPUT_TEXTS': 'Description'
}

huggingface_model = HuggingFaceModel(model_data='../model/model.tar.gz',
                                     role=role,
                                     source_dir="../model/pytorch_model/code",
                                     transformers_version="4.6", 
                                     pytorch_version="1.7", 
                                     py_version="py36",
                                     entry_point="inference.py",
                                     env=hub
                                     )

batch_job = huggingface_model.transformer(
    instance_count=1,
    instance_type='ml.p3.2xlarge',
    output_path=output_s3_path, # we are using the same s3 path to save the output with the input
    strategy='SingleRecord',
    accept='application/json',
    assemble_with='Line'

)

batch_job.transform(
    data=s3_file_uri,
    content_type='application/json',
    split_type='Line',
    #input_filter='$[1:]',
    join_source='Input'

)

自定義推理.py

import json
import os
from transformers import pipeline
import torch
DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'


def model_fn(model_dir):
    model = pipeline(task=os.environ.get('HF_TASK', 'text-classification'), model=model_dir, tokenizer=model_dir)
    return model

def transform_fn(model, input_data, content_type, accept):
    input_data = json.loads(input_data)
    input_text = os.environ.get('INPUT_TEXTS', 'inputs')
    inputs = input_data.pop(input_text, None)
    parameters = input_data.pop("parameters", None)

    # pass inputs with all kwargs in data
    if parameters is not None:
        prediction = model(inputs, **parameters)
    else:
        prediction = model(inputs)
    return json.dumps(
        prediction,
        ensure_ascii=False,
        allow_nan=False,
        indent=None,
        separators=(",", ":"),
    )

Answer 1

我認為問題出在“model_data”參數上。 它應該指向一個 S3 對象 (model.tar.gz)。

然后轉換作業將從 S3 下載 model 文件並加載它。

Answer 2

解決方案是將管道中的“任務”更改為“情感分析”

hub = {
   # 'HF_MODEL_ID':'cardiffnlp/twitter-roberta-base-sentiment',
    'HF_TASK':'sentiment-analysis',
    'INPUT_TEXTS': 'Description'
}