有什么好的方法可以快速将数据从 BigQuery 加载到 Clickhouse 中吗？

Question

I'm working on a tool which will load the results of a BigQuery query into Clickhouse for further analysis. Is there a common pattern to get data from a BigQuery table into Clickhouse fast?

So far it seems my best bet is a writing a custom tool to read the data out of the query results table using the google bigquery storage api, write it to disk as AVRO and then load it into clickhouse.

I just wanted to check if there are any good alternatives before I spend time on it. The most important factor for me is the time from query completion to data in clickhouse.

Answer 1

The easiest way to use airbyte or double.cloud . The last one have also managed clickhouse as service.