将Numpy矩阵从DataLab上传到BigQuery

Question

I have a numpy matrix as output in DataLab. How can I get it as a table in BigQuery?

so far it seems that I need to translate it into JSON, save in the Storage and upload from there. Is there a simpler way to get this done?

Thanks

Answer 1

Found a simple solution with pandas package.

Need to transform the data to dataframe and use then to_gbq approach.

Here is the documentation

The only downside is that that the records stay for hours if not days in BigQuery's streaming buffer and I'm not sure what is the impact of it.