简体   繁体   English

使用 Lambda 函数运行 AWS Athena 的查询

[英]Run AWS Athena’s queries with Lambda function

I created a table on AWS Athena on which I can run any query without any error:我在 AWS Athena 上创建了一个表,我可以在该表上运行任何查询而不会出现任何错误:

select * from mytestdb.test

The table has three columns, customer_Id, product_Id, price .该表包含三列, customer_Id, product_Id, price

I tried to create a lambda function that run the same query for me using boto3:我尝试创建一个使用 boto3 为我运行相同查询的 lambda 函数:

import time
import boto3

DATABASE = 'mytestdb'
TABLE = 'test'

output='s3://mybucketons3/'

COLUMN = 'Customer_Id'

def lambda_handler(event, context):

    keyword = 'xyz12345'

    query = "SELECT * FROM %s.%s where %s = '%s';" % (DATABASE, TABLE, COLUMN, keyword)

    client = boto3.client('athena')

    # Execution
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': DATABASE
        },
        ResultConfiguration={
            'OutputLocation': output,
        }
    )


    return

However I got the following error:但是我收到以下错误:

Response:
{
  "errorMessage": "An error occurred (AccessDeniedException) when calling the StartQueryExecution operation: User: arn:aws:sts::076088932150:assumed-role/Test/QueryTest is not authorized to perform: athena:StartQueryExecution on resource: arn:aws:athena:us-west-2:076088932150:workgroup/primary",
  "errorType": "ClientError",

It seems sort of access issue however I am not sure why because I have both lambda and athena db with the same account.这似乎是一种访问问题,但我不知道为什么,因为我同时拥有 lambda 和 athena db 具有相同的帐户。

As I've mentioned in the comment, your Lambda role should contain Allow policy to interact with Athena service.正如我在评论中提到的,您的 Lambda 角色应该包含允许与 Athena 服务交互的策略。 I've also added full permissions for your S3 bucket.我还为您的 S3 存储桶添加了完全权限。 Example:例子:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1547414166585",
      "Action": [
        "athena:StartQueryExecution"
      ],
      "Effect": "Allow",
      "Resource": "*"
    },
    {
      "Sid": "Stmt1547414166586",
      "Action": [
        "s3:*"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::your-bucket-name/*"
    } 
  ]
}

A.Khan's idea worked for me. A.Khan 的想法对我有用。

Use the AWS console to edit the Lambda's IAM role to have AmazonAthenaFullAccess and AmazonS3FullAccess policies.使用 AWS 控制台编辑 Lambda 的 IAM 角色以拥有 AmazonAthenaFullAccess 和 AmazonS3FullAccess 策略。

AWS 政策

Providing/adding full access for a service is not the best practice.为服务提供/添加完全访问权限不是最佳做法。 You can try restricting access to only the actions that your lambda needs to perform.您可以尝试仅访问 lambda 需要执行的操作。 Try to redeploy the IAM role with specific permissions and re-attach it to lambda function once deployed successfully.尝试重新部署具有特定权限的 IAM 角色,并在成功部署后将其重新附加到 lambda 函数。 Your lambda will surely work.你的 lambda 肯定会工作。 If it still gives access denied after adding required permissions then raise a aws support ticket from your account.如果在添加所需权限后仍然拒绝访问,则从您的帐户中提出 aws 支持票。

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM