[英]Overwrite S3 endpoint using Boto3 configuration file
I'm trying to overwrite certain variables in boto3
using configuration file ( ~/aws/confg
).我正在尝试使用配置文件(
~/aws/confg
)覆盖boto3
某些变量。 In my usecase I want to use fakes3
service and send S3 requests to the localhost.在我的用例中,我想使用
fakes3
服务并将 S3 请求发送到本地主机。
In boto
(not boto3
), I can create a config in ~/.boto
similar to this one:在
boto
(不是boto3
)中,我可以在~/.boto
创建一个类似于这个的配置:
[s3]
host = localhost
calling_format = boto.s3.connection.OrdinaryCallingFormat
[Boto]
is_secure = False
And client can successfully pick up desired changes and instead of sending traffic to real S3 service, it will send it to the localhost.客户端可以成功获取所需的更改,而不是将流量发送到真正的 S3 服务,而是将其发送到本地主机。
>>> import boto
>>> boto.connect_s3()
S3Connection:localhost
>>>
Im trying to achieve similar result using boto3
library.我试图使用
boto3
库实现类似的结果。 By looking at the source code I found that I can use ~/aws/config
location.通过查看源代码,我发现我可以使用
~/aws/config
位置。 I've also found an example config in unittests
folder of botocore
.我还在
botocore
unittests
文件夹中找到了一个示例配置。
I tried to modify config to achieve desired behaviour.我试图修改配置以实现所需的行为。 But unfortunately it doesn't work.
但不幸的是它不起作用。
Here is the config:这是配置:
[default]
aws_access_key_id = XXXXXXXXX
aws_secret_access_key = YYYYYYYYYYYYYY
region = us-east-1
is_secure = False
s3 =
host = localhost
clients
variables using config file?clients
变量?You cannot set host in config file, however you can override it from your code with boto3.您不能在配置文件中设置主机,但是您可以使用 boto3 从您的代码中覆盖它。
import boto3
session = boto3.session.Session()
s3_client = session.client(
service_name='s3',
aws_access_key_id='aaa',
aws_secret_access_key='bbb',
endpoint_url='http://localhost',
)
Then you can interact as usual.然后你就可以像往常一样互动了。
print(s3_client.list_buckets())
boto3
only reads the signature version for s3 from that config file. boto3
仅从该配置文件中读取 s3 的签名版本。 You may want to open a feature request, but for now here is how you can address a custom endpoint:您可能想要打开一个功能请求,但现在您可以通过以下方式处理自定义端点:
import boto3
from botocore.utils import fix_s3_host
resource = boto3.resource(service_name='s3', endpoint_url='http://localhost')
resource.meta.client.meta.events.unregister('before-sign.s3', fix_s3_host)
That bit about the meta is important because boto3
automatically changes the endpoint to your_bucket_name.s3.amazonaws.com
when it sees fit 1 .关于元这一点很重要,因为
boto3
自动改变端点your_bucket_name.s3.amazonaws.com
当它认为合适的1 。 If you'll be working with both your own host and s3, you may wish to override the functionality rather than removing it altogether.如果您将同时使用自己的主机和 s3,您可能希望覆盖该功能而不是完全删除它。
using boto3 resource:使用 boto3 资源:
import boto3
# use third party object storage
s3 = boto3.resource('s3', endpoint_url='https://URL:443',
aws_access_key_id = 'AccessKey',
aws_secret_access_key = 'SecertKey')
# Print out bucket names
for bucket in s3.buckets.all():
print(bucket.name)
Another way:其它的办法:
import boto3
s3client = boto3.client('s3', endpoint_url='http://X.X.x.X:8080/',
aws_access_key_id = 'XXXXXXX',
aws_secret_access_key = 'XXXXXXXX')
bucket_name = 'aaaaa'
s3client.create_bucket(Bucket=bucket_name)
I've just submitted a PR to boto3 to add an env var to allow you to override the endpoint_url if you need to use a dependent module which uses boto3 (in which case you may not be able to modify the call to boto3.client directly)我刚刚向 boto3 提交了一个 PR 以添加一个 env var 以允许您覆盖 endpoint_url 如果您需要使用使用 boto3 的依赖模块(在这种情况下您可能无法直接修改对 boto3.client 的调用)
https://github.com/boto/boto3/pull/2746 https://github.com/boto/boto3/pull/2746
https://github.com/rwillmer/boto3 https://github.com/rwillmer/boto3
s3_client = boto3.client(
"s3",
"us-east-1",
aws_access_key_id="test_id",
aws_secret_access_key="test_key",
endpoint_url="https://localhost:7000",
use_ssl=True,
verify=False,
)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.