簡體   English   中英

查詢 Athena 時 AWS Lambda 函數失敗

[英]AWS Lambda function fails while query Athena

我正在嘗試編寫一個簡單的 Lambda 函數來查詢 Athena 中的表。 但幾秒鍾后,我在 Cloudwatch 日志中看到“狀態:失敗”。 沒有關於失敗原因的描述性錯誤消息。

我的測試代碼如下:

import json
import time
import boto3

# athena constant
DATABASE = 'default'
TABLE = 'test'

# S3 constant
S3_OUTPUT = 's3://test-output/'

# number of retries
RETRY_COUNT = 1000

def lambda_handler(event, context):
    # created query
    query = "SELECT * FROM default.test limit 2"
    #  % (DATABASE, TABLE)
    # athena client
    client = boto3.client('athena')
    
    # Execution
    response = client.start_query_execution(
        QueryString=query,
        QueryExecutionContext={
            'Database': DATABASE
        },
        ResultConfiguration={
            'OutputLocation': S3_OUTPUT,
        }
    )
    
    # get query execution id
    query_execution_id = response['QueryExecutionId']
    print(query_execution_id)
    
    # get execution status
    for i in range(1, 1 + RETRY_COUNT):

        # get query execution
        query_status = client.get_query_execution(QueryExecutionId=query_execution_id)
        query_execution_status = query_status['QueryExecution']['Status']['State']

        if query_execution_status == 'SUCCEEDED':
            print("STATUS:" + query_execution_status)
            break

        if query_execution_status == 'FAILED':
            #raise Exception("STATUS:" + query_execution_status)
            print("STATUS:" + query_execution_status)

        else:
            print("STATUS:" + query_execution_status)
            time.sleep(i)
    else:
        # Did not encounter a break event. Need to kill the query
        client.stop_query_execution(QueryExecutionId=query_execution_id)
        raise Exception('TIME OVER')
        
    # get query results
    result = client.get_query_results(QueryExecutionId=query_execution_id)
    print(result)
    return

日志顯示以下內容:

2020-08-31T10:52:12.443-04:00
    
START RequestId: e5434651-d36e-48f0-8f27-0290 Version: $LATEST
    
2020-08-31T10:52:13.481-04:00
    
88162f38-bfcb-40ae-b4a3-0b5a21846e28
    
2020-08-31T10:52:13.500-04:00
    
STATUS:QUEUED
    
2020-08-31T10:52:14.519-04:00
    
STATUS:RUNNING
    
2020-08-31T10:52:16.540-04:00
    
STATUS:RUNNING
    
2020-08-31T10:52:19.556-04:00
    
STATUS:RUNNING
    
2020-08-31T10:52:23.574-04:00
    
STATUS:RUNNING
    
2020-08-31T10:52:28.594-04:00
    
STATUS:FAILED
    
2020-08-31T10:52:28.640-04:00
    
....more status: FAILED
....
    
END RequestId: e5434651-d36e-48f0-8f27-0290
    
REPORT RequestId: e5434651-d36e-48f0-8f27-0290 Duration: 30030.22 ms Billed Duration: 30000 ms Memory Size: 128 MB Max Memory Used: 72 MB Init Duration: 307.49 ms

2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds 

我認為我擁有授予該角色的 S3 存儲桶訪問權限(如果沒有,我會看到錯誤消息)。 存儲桶中也沒有創建文件。 我不確定這里出了什么問題。 我錯過了什么? 謝謝

日志中的最后一行顯示

2020-08-31T14:52:42.473Z e5434651-d36e-48f0-8f27-0290 Task timed out after 30.03 seconds 

在我看來,Lambda 函數的超時設置為 30 秒。 嘗試將其增加到超過 Athena 查詢所需的時間(最長為 15 分鍾)。

暫無
暫無

聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.

 
粵ICP備18138465號  © 2020-2024 STACKOOM.COM