简体   繁体   中英

insert values into temp tables very slow on bigquery

For some reason it takes BigQuery almost a minute to insert 6 rows into a temporary table. Just the null value insert took BQ 20 seconds.

Example:

create temporary table units (
  id string,
  Nof_units int,
  source_name string
);

insert into units values ('aFsd23j2', 45, 'a');
insert into units values ('aFsd23j2', 34, 'b');
insert into units values ('aFsd23j2', 12, 'c');
insert into units values ('r8cxn23n', 130, 'a');
insert into units values ('r8cxn23n', 139, 'b');
insert into units values ('r8cxn23n', null, 'c');

select *
from units

What would be the reason behind that?

Basically, OLAP DB like BigQuery is not optimized for mutating queries (ie INSERT, UPDATE, DELETE). If you reduce the number of insert statements, it will get faster than before.

create temporary table units (
  id string,
  Nof_units int,
  source_name string
);

insert into units values
('aFsd23j2', 45, 'a'),
('aFsd23j2', 34, 'b'),
('aFsd23j2', 12, 'c'),
('r8cxn23n', 130, 'a'),
('r8cxn23n', 139, 'b'),
('r8cxn23n', null, 'c');

select *
from units;

output: 16.9 sec --> 4.4 sec in my environment

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM