AWS Athena CTAS 查詢失敗，建議清空空桶

Question

我正在運行“CREATE TABLE AS SELECT (CTAS) 查詢”（ https://docs.aws.amazon.com/athena/latest/ug/ctas.ZFC35FDC70D5FC69D25369883A8復制在底部C） 我收到以下錯誤消息：

HIVE_PATH_ALREADY_EXISTS: Target directory for table 'default.openaq_processed' already exists:
 s3://<processed-data-bucketname>/. You may need to manually clean the data at location 
's3://<athena-query-results-bucketname>/Unsaved/2021/04/29/tables/82025a35-8867-4865-8f42-f40adb6bee4c' 
before retrying. Athena will not delete data in your account.

This query ran against the "default" database, unless qualified by the query. Please post the
 error message on our forum or contact customer support with Query Id: 82025a35-8867-4865-8f42-f40adb6bee4c.

關於此錯誤的 AWS 知識中心頁面 ( https://aws.amazon.com/premiumsupport/knowledge-center/athena-hive-path-already-exists/ ) 與上面的錯誤消息一樣，表明修復是為了確保用於存儲查詢結果的位置必須為空。

但它已經是。 實際上s3://<athena-query-results-bucketname>/Unsaved/2021/04/29/中沒有tables/前綴/文件夾，而s3://<processed-data-bucketname>/存儲桶完全空的。

我已經在 AWS 論壇上發布了這個問題，但沒有回復，任何關於如何讓這個 CTAS 查詢成功的建議都將不勝感激。

更新：引發錯誤的查詢：

CREATE TABLE openaq_processed
WITH (format='PARQUET', 
parquet_compression='SNAPPY', 
partitioned_by=array['country', 'parameter'], 
external_location = '<processed-data-bucketname>') 
AS
SELECT date_utc as date_utc_str,
date_local as date_local_str,
CAST(from_iso8601_timestamp(date_utc) as timestamp) as timestamp_utc,
CAST(from_iso8601_timestamp(date_local) as timestamp) as timestamp_local,
"location",  -- location is a reserved word for Athena, needs quotes
value,
unit,
city,
attribution,
averagingperiod,
coordinates."latitude" as latitude,
coordinates."longitude" as longitude,
sourcename,
sourcetype,
mobile,
country,
parameter
FROM openaq_pq2_tables

Answer 1

因此，我尋求 AWS 開發人員支持並提出了這個問題。 我得到的確實解決了錯誤的響應是在我的external_location存儲桶中創建一個文件夾。 不知道為什么這是必要的，但顯然它是必要的。

因此，從 shell： $ aws s3 mb s3://<processed-data-bucketname>/processed_data/

（上面的mb代表“制作桶”）。

然后將上面查詢中的external_location = 's3://<processed-data-bucketname>'更新為external_location = 's3://<processed-data-bucketname>/processed_data/')