[英]How to use python ElasticSearch package with ECS Instance Role instead of passing credentials?
I've built an app that queries ElasticSearch at various times.我构建了一个应用程序,可以在不同时间查询 ElasticSearch。 Now that I'm moving the app to be hosted on a ECS, I want to switch my queries to be based on the Instance Role permissions rather than credentials that I've been passing as environment variables.现在我正在将应用程序移动到托管在 ECS 上,我想将我的查询切换为基于实例角色权限而不是我作为环境变量传递的凭据。
Based on the docs , if I pass no credentials to Boto3, I ought to get use my Instance Role.根据文档,如果我没有将凭据传递给 Boto3,我应该使用我的实例角色。 But in order to send my query using the ElasticSearc h package in python, I have to pass credentials.但是为了使用 python 中的ElasticSearc h package 发送我的查询,我必须通过凭据。 Is there a way to do this using Instance Roles?有没有办法使用实例角色来做到这一点?
The example code here gave me hope that I would just pass in the `boto3.Session().get_credentials() function to retrieve the Instance Role credentials. 这里的示例代码给了我希望,我只需传入 `boto3.Session().get_credentials() function 来检索实例角色凭据。 But it turns out, as explained by my aws admin, that Instance Roles do not actually have credentials (aws_key, aws_secret_key).但事实证明,正如我的 aws 管理员所解释的那样,实例角色实际上没有凭据(aws_key、aws_secret_key)。 As a result my code seems to generate an AWS key which is not valid on the cloud and returns only null values for my queries.结果,我的代码似乎生成了一个在云上无效的 AWS 密钥,并且只为我的查询返回 null 值。
I've been told that the only alternative is to use the ES Boto3 client and not pass credentials...but using ES with this client is not nearly as friendly as using the ElasticSearch python package interface.有人告诉我,唯一的替代方法是使用ES Boto3 客户端而不传递凭据……但是在此客户端上使用 ES 不如使用 ElasticSearch python ZEFE90A8E604A7C8470E88D03A687B6 接口友好。
Is there a way to make it work where the ElasticSearch package could query the ES without passing credentials and only relying on the Instance Role permissions?有没有办法让它在 ElasticSearch package 可以在不传递凭据且仅依赖实例角色权限的情况下查询 ES 的情况下工作?
Here is my code so far:到目前为止,这是我的代码:
if os.environ.get('LOCAL_DEV')==1:
AWS_KEY = os.environ['AWS_ACCESS_KEY']
AWS_SECRET_KEY = os.environ['AWS_SECRET_KEY']
else:
credentials = boto3.Session().get_credentials() #fails to use instance role
AWS_KEY=credentials.access_key
AWS_SECRET_KEY=credentials.secret_key
# It pulls my environment variables when running locally, and when running on the cluster, I need it to use the Instance Role
#### below is how I initialize and query with the ElasticSearch package
import elasticsearch as es
from requests_aws4auth import AWS4Auth
from constants import AWS_KEY, AWS_SECRET_KEY
awsauth = AWS4Auth(AWS_KEY, AWS_SECRET_KEY, 'us-west-2', 'es')
#requires a credential input
host ='search-xxx.xxx.xxxxxxx.xxxx.amazonaws.com'
e_s = es.Elasticsearch(
hosts=[{'host': host, 'port': 443}],
http_auth=awsauth,
use_ssl=True,
verify_certs=True,
connection_class=es.RequestsHttpConnection
)
res = e_s.search(index='foo', body='bar')
res = res['hits']['hits']
Any help on this would be very appreciated对此的任何帮助将不胜感激
What I ended up doing was pulling temporary credentials using boto3 that were based on the the InstanceRole permissions.我最终做的是使用基于 InstanceRole 权限的 boto3 提取临时凭证。 Since these are temporary credentials, they need to be rotated.由于这些是临时凭证,因此需要轮换。 However, it worked!然而,它奏效了! I was stumped until I discovered the need to use a temporary token in the AWS4Auth command我被难住了,直到我发现需要在 AWS4Auth 命令中使用临时令牌
temp_access=boto3.Session(region_name='foo-region').get_credentials().access_key
temp_secret=boto3.Session(region_name='foo-region').get_credentials().secret_key
temp_token=boto3.Session(region_name='foo-region').get_credentials().token
########
from requests_aws4auth import AWS4Auth
awsauth=AWS4Auth(temp_access, temp_secret, 'us-west-2', 'es', session_token=temp_token)
es.Elasticsearch(hosts=[{'host':host, 'port':443}],
http_auth=awsauth,
use_ssl=True,
verify_certs=True,
connection_class=es.RequestsHttpConnection)
This article was very helpful 这篇文章很有帮助
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.