简体   繁体   English

从 api 保存到 s3 存储桶

[英]saving from api to s3 bucket

I'm trying to get the below python code to save the csv from an api to an amazon s3 bucket using bot03 and python, but I can't see where I'm going wrong.我正在尝试获取以下 python 代码,以使用 bot03 和 python 将 csv 从 api 保存到 amazon s3 存储桶,但我看不出哪里出错了。 When I execute the code I don't get any error but the file never appear in the s3 bucket.当我执行代码时,我没有收到任何错误,但文件从未出现在 s3 存储桶中。

import boto3
from botocore.exceptions import ClientError

file_name = "test.csv"
bucket = "mybucket"

def main():
    url = "https://api0.solar.sheffield.ac.uk/pvlive/v3/pes/10?start=2021-01-01T00:00:00&end=2021-07-06T00:00:00&data_format=csv"
    x = requests.get(url,headers={'Content-Type': 'application/json', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate',})

    s3 = bot03.client("s3")
    with open("test.csv","rb") as file2:
     s3.upload_fileobj(x.content, bucket, "test.cvc")

any tips/advice would be appreciated.任何提示/建议将不胜感激。 I'm a python/aws newbie so apologies if a basic question我是一个 python/aws 新手,如果有一个基本问题,我深表歉意

I used this code to acheive what I need我用这段代码来实现我所需要的

file_name = "test.csv"
bucket = "my_bucket"

def main():
    url = "https://api0.solar.sheffield.ac.uk/pvlive/v3/pes/10?start=2021-01-01T00:00:00&end=2021-07-06T00:00:00&data_format=csv"
    x = requests.get(url,headers={'Content-Type': 'application/json', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate',})



    s3_resource = boto3.resource('s3')
    s3_resource.Object(bucket, 'snowflake/csv/df1.csv').put(Body=x.content)


if __name__ == "__main__":
    main()

per the OP(Original Post), did you try根据 OP(Original Post),你试过了吗

(line 11) s3 = boto3.client("s3") -- OP: bot03.client("s3") (第 11 行)s3 = boto3.client("s3") -- OP: bot03.client("s3")

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM