如何使用 aws lambda 将 json 文件转换为镶木地板

Question

s3_obj = s3.Object(s3_bucket, file_prefix)
df= pd.read_json(jsonlines_doc,lines=True)
location=s3_obj.put(Body=json.dumps(jsonlines_doc))
df.to_parquet(location, engine='auto', compression='snappy', index=None)

code not working but able to save it locally imported pyarrow and parquet

Answer 1

Have you tried saving the file first into the /tmp/ available in lambda and then copy it to the s3 bucket of your choice.

Just to mention Each Lambda execution container provides 512 MB of ephemeral disk space in the /tmp directory, so if your file is bigger than this size you will probably have to use AWS EFS.