使用带有 _TABLE_SUFFIX 的通配符在 BigQuery 中查询表视图

Question

I try to query a large number (~140) different table views within Google BigQuery using _TABLE_SUFFIX . But this results in the following error massage:

"Views cannot be queried through prefix."

Currently I am using this code:

SELECT
  tableDate, 
  `TableA.20*`.ip AS IP, 
  `TableB.20*`.city AS city, 
....
  
  CAST(s.banner AS string) AS sourcecode,

FROM
  `TableA.20*`
  CROSS JOIN UNNEST(services) AS s
  FULL OUTER JOIN `TableB.20*` USING(ip)

WHERE
_TABLE_SUFFIX IN (SELECT table_date FROM `datasetX.dates_table` AS tableDate)

AND
REGEXP_MATCH(cast(s.banner AS string), r'(?i) .....

Structure of "dates_table":

table_date 
----------    
190305
190312
190319
190326
...
[weekly dates]

The original data set looks like this:

As I read in the BigQuery documentation , wildcards are only possible to use with legacy SQL and it is not possible to use wildcards to query >views<. My simple question is: What could be an alternative way to query data from different views? Is there another way to loop though views using wildcards?

Possible, but not working solutions:

The solutions suggested here are unfortunately not possible in my case. I cannot change the data set, as it is a set from a external provider. Trying to expose the _TABLE_SUFFIX column, as suggested here , does also not work in my case. Using UNION ALL for example, as suggested here , is not possible with 140 tables.

I would also be very happy to have a solution that uses BigQuery standard SQL so that I can use eg REGEXP_CONTAIN.

Any ideas? That would be great. Thanks a lot.

Frank

Answer 1

Only (suitable) Solution

As there is no way to iterate over table views (for whatever technical reason), it is necessary to hard-define table view names . Thus, I suggest to use UNION ALL and hard-code the table view names. It means much longer code and no automated/iterative process, but at least it works. ;-)

SELECT
  tableDate, 
  `TableA.20190305`.ip AS IP, 
  `TableB.20190305*`.city AS city, 
  CAST(s.banner AS string) AS sourcecode,

FROM
  `TableA.20*`
  CROSS JOIN UNNEST(services) AS s
  FULL OUTER JOIN `TableB.20190305` USING(ip)

UNION ALL

SELECT 
  tableDate, 
  `TableA.20190312`.ip AS IP, 
  `TableB.20190312`.city AS city, 
  CAST(s.banner AS string) AS sourcecode,

FROM
  `TableA.20*`
  CROSS JOIN UNNEST(services) AS s
  FULL OUTER JOIN `TableB.20190312` USING(ip)

UNION ALL 
<all further tables>


WHERE
REGEXP_MATCH(cast(s.banner AS string), r'(?i) .....