如何使用pkl文件在Lambda中使用线性回归model？

Question

I have trained a linear regression model and saved the model in a pkl file, with the following code:

import pickle

# save the model
filename = 'linear_model.pkl'
pickle.dump(mod, open(filename, 'wb'))
  
# load the model
load_model = pickle.load(open(filename, 'rb'))
 

After that I tried to use the model in Lambda by importing the pkl file. I did a lot of research and many attempts, but could not figure out how to do this. My last attempt was this one:

from io import BytesIO
import pickle
import boto3
import base64
import json

s3_client = boto3.client('s3')


def lambda_handler(event, context):
    # getting bucket and object key from event object
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    data = BytesIO()
    s3_client.download_fileobj(source_bucket, key, data)
    data.seek(0)    # move back to the beginning after writing
    print("Data", data.read())

    
    load_model = pickle.load(open(data, 'rb'))
    print("load_model", load_model)
    
    y_pred = load_model.predict([[140000]])

[ERROR] TypeError: expected str, bytes or os.PathLike object, not _io.BytesIO
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 25, in lambda_handler
    load_model = pickle.load(open(data, 'rb'))

I tried to fix this error using data.read() , but if you do that no file is found in the directory.

Answer 1

• data is a BytesIO but you try to open it like a file. You can not do it on BytesIO .
• Don't read from stream.
• pickle.load accepts a file. But you have data stream. You need to use loads function.

Update version of lambda function.

from io import BytesIO
import pickle
import boto3
import base64
import json

s3_client = boto3.client('s3')


def lambda_handler(event, context):
    # getting bucket and object key from event object
    source_bucket = event['Records'][0]['s3']['bucket']['name']
    key = event['Records'][0]['s3']['object']['key']

    data = BytesIO()
    s3_client.download_fileobj(source_bucket, key, data)
    data.seek(0)    # move back to the beginning after writing
    
    load_model = pickle.loads(data.read())
    print("load_model", load_model)
    
    y_pred = load_model.predict([[140000]])