[英]Boto3 S3 client methods throw NoSuchBucket even it has
I am trying to select one existing bucket then upload file into the bucket using Boto3.我正在尝试 select 一个现有的存储桶,然后使用 Boto3 将文件上传到存储桶中。 But all methods related with buckets aren't working correctly for me.
但是与存储桶相关的所有方法对我来说都无法正常工作。
The problems I'm confronting are like this:我面临的问题是这样的:
boto3.Session.client('s3').list_buckets()
method, it results an empty list of Buckets.boto3.Session.client('s3').list_buckets()
方法列出存储桶时,它会生成一个空的存储桶列表。boto3.Session.resource('s3').Bucket('existing_bucket_name')
then run upload_fileobj()
method, it throws NoSuchBucket
exception(which is very ridiculous the Bucket object's attribute name
correctly returns the bucket's name though).boto3.Session.resource('s3').Bucket('existing_bucket_name')
然后运行upload_fileobj()
方法时,它抛出NoSuchBucket
异常(这非常荒谬 Bucket 对象的属性name
正确返回桶的名称尽管)。 What I've tried so far:到目前为止我已经尝试过:
Session
instance with keyword arguments indicating account and region configuration.Session
实例,关键字 arguments 指示帐户和区域配置。 I had some issues with Boto3 config setup and finally gave up using config file then directly gave the instance config arguments.resource
instead of client
.resource
而不是client
。 client
method didn't work correctly, so I used resource
but it also didn't go well. client
方法没有正常工作,所以我使用了resource
,但它也没有 go 很好。 here is my code:这是我的代码:
class S3Client:
def __init__(self):
region = 'ap-northeast-2'
self.session = boto3.Session(
aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'],
region_name=region
)
self.client = self.session.client(service_name='s3', region_name=region,
aws_access_key_id=os.environ['AWS_ACCESS_KEY_ID'],
aws_secret_access_key=os.environ['AWS_SECRET_ACCESS_KEY'])
self.resource = self.session.resource('s3')
self.bucket = self.resource.Bucket(settings.AWS_S3['BUCKET_NAME'])
@staticmethod
def open_test_file_as_byte():
file_name = os.path.join(os.path.dirname(__file__), 'test_file.pdf')
data = open(file_name, 'rb')
return data
def upload_file(self, file_name):
with self.open_test_file_as_byte() as file_byte_stream:
return self.bucket.upload_fileobj(
file_byte_stream,
'reports/' + file_name,
ExtraArgs={'ACL': 'public-read-write'}
)
# this method worked once with message saying that creation successfully completed, but I cannot find anything in my S3
def create_bucket(self):
return self.client.create_bucket(
ACL='public-read-write',
Bucket='public-reports'
# CreateBucketConfiguration={
# 'LocationConstraint': 'ap-northeast-2'
# }
)
@mock_s3
class S3ClientTestCases(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.s3_client = S3Client()
def test_create_bucket(self):
logger.debug(self.s3_client.create_bucket())
logger.debug(self.s3_client.client.list_buckets())
def test_upload_file(self):
logger.debug(self.s3_client.upload_file('test.pdf'))
This is full error log:这是完整的错误日志:
======================================================================
ERROR: test_upload_file (apps.reports.test.S3ClientTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/moto/core/models.py", line 111, in wrapper
result = func(*args, **kwargs)
File "/Users/psymon/Documents/Git/portfolio-manager/apps/reports/test.py", line 54, in test_upload_file
logger.debug(self.s3_client.upload_file('test.pdf'))
File "/Users/psymon/Documents/Git/portfolio-manager/apps/reports/services/report_upload_service.py", line 71, in upload_file
return self.bucket.upload_fileobj(
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/boto3/s3/inject.py", line 678, in bucket_upload_fileobj
return self.meta.client.upload_fileobj(
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/boto3/s3/inject.py", line 636, in upload_fileobj
return future.result()
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/s3transfer/futures.py", line 266, in result
raise self._exception
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 139, in __call__
return self._execute_main(kwargs)
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/s3transfer/upload.py", line 758, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/botocore/client.py", line 515, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/Users/psymon/Documents/Git/portfolio-manager/.venv/lib/python3.10/site-packages/botocore/client.py", line 934, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the PutObject operation: The specified bucket does not exist
Bucket Policy:存储桶策略:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": "arn:aws:s3:::some_bucket_name/*"
}
]
}
Granted Permission to User:授予用户权限:
AmazonS3FullAccess
I guess something is wrong with AWS account setting but I'm not sure what exactly cause this entire malfunctions...我猜 AWS 帐户设置有问题,但我不确定到底是什么导致了整个故障......
Please any advice!请任何建议!
The cause of my problem was the annotation on the test code, @mock_s3
.我的问题的原因是测试代码
@mock_s3
上的注释。
Actually, I cannot even call this as problem, but a mistake.实际上,我什至不能称之为问题,而是一个错误。
Anyway, if anybody get in trouble with that annotation, I hope my stupid record help you.不管怎样,如果有人遇到那个注释的麻烦,我希望我的愚蠢记录能帮助你。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.