Bigquery 發生內部錯誤，無法完成請求。 錯誤：60909208

Question

我正在嘗試運行以下查詢，但它在運行大約 1 到 2 小時后一直拋出此錯誤

發生內部錯誤，無法完成請求。 錯誤：60909208

WITH
Liveramp AS(
 SELECT maid, segment_id, device_type, date
    FROM `mikesky.Segments.Segment_Liveramp_Raw`
    WHERE date BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 1 MONTH) AND CURRENT_DATE()
),

Neustar AS(
 SELECT maid, segment_id, device_type, date
    FROM `mikesky.Segments.Segment_Neustar_Raw`
    WHERE date BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 1 MONTH) AND CURRENT_DATE()
),

PT AS(
 SELECT maid, segment_id, device_type, date
    FROM `mikesky.Segments.Segment_PT_Raw`
    WHERE date BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 1 MONTH) AND CURRENT_DATE()
),

liveramp_split AS(
    SELECT maid, SPLIT(segment_id) AS segment_id, device_type, date
    FROM Liveramp
),

PT_split AS(
    SELECT maid, SPLIT(segment_id) AS segment_id, device_type, date
    FROM PT
),

Neustar_split AS(
    SELECT maid, SPLIT(segment_id) AS segment_id, device_type, date
    FROM Neustar
),
Liveramp_unnested AS(
    SELECT DISTINCT maid,segments AS segment_id, device_type, date 
    FROM liveramp_split , UNNEST(segment_id) AS segments
),
Neustar_unnested AS(
    SELECT DISTINCT maid, segments AS segment_id, device_type, date 
    FROM Neustar_split , UNNEST(segment_id) AS segments
),
PT_unnested AS(
    SELECT DISTINCT maid, segments AS segment_id, device_type, date 
    FROM PT_split , UNNEST(segment_id) AS segments
),

Taxonomy AS(
    SELECT segment_id, CAST(skydeo_segment_id AS STRING) AS skydeo_segment_id
    FROM `mikesky.Skydeo_Taxonomy.Segments_IDs_Mappings`
),

Liveramp_Taxonomy AS(
    SELECT maid, skydeo_segment_id, device_type, date
    FROM Liveramp_unnested  JOIN Taxonomy ON Liveramp_unnested.segment_id = Taxonomy.segment_id 
),

Neustar_Taxonomy AS(
    SELECT maid, skydeo_segment_id, device_type, date
    FROM Neustar_unnested JOIN Taxonomy ON Neustar_unnested.segment_id = Taxonomy.segment_id 
),

PT_Taxonomy AS(
    SELECT maid, skydeo_segment_id, device_type, date
    FROM PT_unnested JOIN Taxonomy ON PT_unnested.segment_id = Taxonomy.segment_id 
),

Liveramp_agg AS(
    SELECT maid, ARRAY_AGG(skydeo_segment_id) AS skydeo_segment_id, device_type, date 
    FROM Liveramp_Taxonomy
    GROUP BY maid, device_type, date
),

Neustar_agg AS(
    SELECT maid, ARRAY_AGG(skydeo_segment_id) AS skydeo_segment_id, device_type, date 
    FROM Neustar_Taxonomy
    GROUP BY maid, device_type, date
),

PT_agg AS(
    SELECT maid, ARRAY_AGG(skydeo_segment_id) AS skydeo_segment_id, device_type, date 
    FROM PT_Taxonomy
    GROUP BY maid, device_type, date
),

Super_set AS(
SELECT * FROM Liveramp_agg

UNION ALL

SELECT * FROM Neustar_agg

UNION ALL

SELECT * FROM PT_agg

)

SELECT *
FROM Super_set

該查詢應該結合來自 3 個表（Liveramp_Raw、PT_Raw、Neustar_Raw）的數據，然后將結果與 (Segments_IDs_Mappings) 表連接起來，然后聚合結果。

起初，我正在運行這個查詢的另一個版本，但它拋出了這個錯誤

查詢執行期間資源超出：您的項目或組織超出了可用於 shuffle 操作的最大磁盤和 memory 限制。 考慮配置更多槽、減少查詢並發或在此作業中使用更高效的邏輯。

所以我盡可能地優化它。

任何想法為什么我會收到此錯誤或它是什么意思？

Answer 1

當我有多個無效數據，多個數據報告錯誤（例如數據類型錯誤，日期錯誤）時，我得到了同樣的錯誤。

要查明問題，請嘗試在 SELECT 中一次注釋掉一個屬性。