合并GCP中的csv个文件

Question

The dataset on which I am working on GCP is in csv format and for each feature there is a separate csv file with no header. There is around 20 files and want to create a single file for all these variables with headers. However, I have access on the data bucket only when I try to open Vertex AI Workbench, it shows I don't have permission for that. Is there any way to combine all these files?

Answer 1

I wrote an article on how to use BigQuery to play with CSV files. I didn't mentioned how to merge to right. But typically I will do a join on row_number, something like that:

With table_left as (select *, ROW_NUMBER() OVER () as row_id from <tableLeft>),
table_right as (select *, ROW_NUMBER() OVER () as row_id from <tableRight>)
select tl.* except(row_id), tr.*  except(row_id) from table_left tl join table_right tr on tl.row_id = tr.row_id