BigQuery 创建临时表

Question

I am trying to create a temporary table with a specific name so i can use it in other queries. The problem is that when i create a temp table like this:

BEGIN

  BEGIN TRANSACTION;
   --delete test_temp where 1=1;
   create temp table test_temp as SELECT * FROM `analytics.reports.analysis` ;

  COMMIT TRANSACTION;

END;

the table name is indeed test_temp but if i will try to select * from test_temp i get nothing. the table id always generates a new unique id. something like analytics-prod:_script3991beb3e9868774854ac09d407f1c397428a4f6.test_temp

Is there a way to make this temp table with a pre-defined name?

What I am trying to achieve basically is create a table from a select every 15min. I thought to create a scheduled query running this transaction that will delete + populate the temp table and then I can always reference to test_temp without changing the table id.

Answer 1

According to the docs :

You can refer to a temporary table by name for the duration of the current multi-statement query

So if you want to refer the table in other queries, an option could be use a permanent table in a separated dataset:

...
CREATE OR REPLACE TABLE `analytics.my_temp_dataset.test_temp` AS SELECT * FROM `analytics.reports.analysis` ;
...

Answer 2

In that case, you don't need a temp table but a permanent table you just replace on the next run using the CREATE OR REPLACE TABLE statement. If you are using the temp table as part of a scripting stage, then I suggest using running this instead:

BEGIN
      
  CREATE OR REPLACE TEMP TABLE _SESSION.test_temp AS
  SELECT * FROM `analytics.reports.analysis`;

  -- SELECT COUNT(*) CNT FROM _SESSION.test_temp;

END;

Since it is a temp table you don't need a DROP query at the end.

Answer 3

You can create a temporary table in a session in BigQuery.

You can create a session via the API and then create your temporary tables in the session. The tables will then be visible to all queries using that session.

There are limitations on sessions you should be aware of:

• They last up to 24 hours max
• No concurrent jobs running in a session

I use temporary tables with sessions to do complex transformations in an ETL pipeline. For example:

1. Create session
2. Create temporary tables
3. Do joins, and other transformations, write results to temporary tables
4. Do final transformations and output results to permanent tables
5. End session

I am writing a Python API for this and will put it on github at some point. I could probably post a gist if that is helpful.

In the meantime, see Google's documentation for sessions in BigQuery:

https://cloud.google.com/bigquery/docs/sessions-intro