How to load a shapefile in BigQuery? Is there a simpler way to upload polygon data in BigQuery?

Question

I have been trying to load polygon data in BigQuery, but it doesnt seem to work.

I have tried what Michael Entin explained in his " Loading large spatial features to BigQuery geography " article, but it didnt so well for me.

I also have tried what Lak Lakshmanan explained in his " How to load geographic data like shapefiles into BigQuery ", but didnt work either.

I found it rather confusing having to convert the shapefiles and/or geojsons into a csv to load into BigQuery.

Is there any simpler way to upload polygon data in BigQuery?

Answer 1

Managed to find a simple solution: :)

# First of all, I have a shapefile ready to go. The command below will return details about it:
ogrinfo -al -so data/pr_setores_censitarios/shapefile.shp

# Secondly, I convert the shapefile to a geojson:
ogr2ogr -f GeoJSON -t_srs crs:84 data/poligonos_setores_censitarios.geojson data/pr_setores_censitarios

# After, since BigQuery demands json and geojson files to be newline delimited, I modify it a bit:
cat data/poligonos_setores_censitarios.geojson | jq -c ".features[]" > data/poligonos_setores_censitarios_newlinedelimited.geojson

# Then, I upload it to Google Cloud Storage:
gsutil cp data/poligonos_setores_censitarios_newlinedelimited.geojson gs://cloud-storage-bucket01/

# And finally, load it in BigQuery:
bq load --source_format=NEWLINE_DELIMITED_JSON --json_extension=GEOJSON --autodetect mydataset.polygons gs://cloud-storage-bucket01/poligonos_setores_censitarios_newlinedelimited.geojson

That's it! We are ready to go!

The --autodetect flag should probably handle the schema, but you can also declare it.

I hope I have helped. This took me quite a while to figure out.