BigQuery：执行创建表的脚本时出错

Question

I'm trying to run a SQL query via a python script to feed a table automatically.

However, I have this error message:

google.api_core.exceptions.BadRequest: 400 GET https://bigquery.googleapis.com/bigquery/v2/projects/****/queries/******************* too many subqueries or query is too complex.

The request works fine when I run it on Bigquery, but doesn't work when it is executed on the API via the python script.

Here is the part of the code that blocks my request:

 114:, SCOPE_FINAL AS
 115:(
 116:   
 117:  select
 118:    SCOPE_PARCOURS_2.DATE
 119:    , SCOPE_PARCOURS_2.SESSIONID
 120:   , CONCAT(NOM1.Name_CheckPoint
 121:                   , CASE WHEN NOM2.Name_CheckPoint IS NOT NULL THEN ' > ' ELSE '' END , CASE WHEN NOM2.Name_CheckPoint IS NOT NULL THEN NOM2.Name_CheckPoi
 122:                   , CASE WHEN NOM3.Name_CheckPoint IS NOT NULL THEN ' > ' ELSE '' END , CASE WHEN NOM3.Name_CheckPoint IS NOT NULL THEN NOM3.Name_CheckPoi
 123:                   , CASE WHEN NOM4.Name_CheckPoint IS NOT NULL THEN ' > ' ELSE '' END , CASE WHEN NOM4.Name_CheckPoint IS NOT NULL THEN NOM4.Name_CheckPoi
 124:                   , CASE WHEN NOM5.Name_CheckPoint IS NOT NULL THEN ' > ' ELSE '' END , CASE WHEN NOM5.Name_CheckPoint IS NOT NULL THEN NOM5.Name_CheckPoi
 125:                   ) PARCOURS
 126:    , SCOPE_PARCOURS_2.Name_CheckPoint
 127:    , SCOPE_PARCOURS_2.NUMSTEP
 128:    , SCOPE_PARCOURS_2.NB_ENTREE_TUNNEL
 129:    , SCOPE_PARCOURS_2.NB_LEAD
 130:  from SCOPE_PARCOURS_2
 131:  LEFT OUTER JOIN SCOPE_PARCOURS_2 NOM1
 132:   ON SCOPE_PARCOURS_2.DATE = NOM1.DATE
 133:   AND SCOPE_PARCOURS_2.SESSIONID = NOM1.SESSIONID
 134:   AND NOM1.NUMSTEP = 1
 135:  LEFT OUTER JOIN SCOPE_PARCOURS_2 NOM2
 136:   ON SCOPE_PARCOURS_2.DATE = NOM2.DATE
 137:   AND SCOPE_PARCOURS_2.SESSIONID = NOM2.SESSIONID
 138:   AND NOM2.NUMSTEP = 2
 139:   LEFT OUTER JOIN SCOPE_PARCOURS_2 NOM3
 140:   ON SCOPE_PARCOURS_2.DATE = NOM3.DATE
 141:   AND SCOPE_PARCOURS_2.SESSIONID = NOM3.SESSIONID
 142:   AND NOM3.NUMSTEP = 3
 143:   LEFT OUTER JOIN SCOPE_PARCOURS_2 NOM4
 144:   ON SCOPE_PARCOURS_2.DATE = NOM4.DATE
 145:   AND SCOPE_PARCOURS_2.SESSIONID = NOM4.SESSIONID
 146:   AND NOM4.NUMSTEP = 4
 147:   LEFT OUTER JOIN SCOPE_PARCOURS_2 NOM5
 148:   ON SCOPE_PARCOURS_2.DATE = NOM5.DATE
 149:   AND SCOPE_PARCOURS_2.SESSIONID = NOM5.SESSIONID
 150:   AND NOM5.NUMSTEP = 5
 151: UNION ALL
 152:   select
 153:    DATE
 154:    , SESSIONID
 155:   , 'VIDE' PARCOURS
 156:    , Name_CheckPoint
 157:    , NUMSTEP
 158:    , NB_ENTREE_TUNNEL
 159:    , NB_LEAD
 160:  from SCOPE_AUTRE```

Answer 1

This error occurs when you execute a query with many complex steps to get the results.

Therefore, to avoid this error message, you can try, if your use case allows it, to split the query into a simpler queries using temporary tables and try to avoid such amount of LEFT JOIN on the same table. Here you have a documentation with recommendations on how to avoid repeated joins [1].

[1] https://cloud.google.com/bigquery/docs/best-practices-performance-output#avoid_repeated_joins_and_subqueries