How to optimize BigQuery queries on a date field where most queries are at a year-month level

Question

A question about BigQuery query performance on date fields...

I have a very large data.table where each record has an 'event date' field. Most of the queries on the table are actually run at a calendar month level, eg January 2020. Is there any BigQuery performance gain to be had from having an extra field(s) that store either 'year-month' as one field or 'year' and 'month' as two separate extra fields?

Answer 1

Have you partitioned your table by month already, if not, doing so will allow the queries to scan much less data (only the specified month). The partition-by-month feature went to GA just weeks ago:

September 21, 2020

The following time-unit partitioning features are now Generally Available (GA):

Creating partitions using hourly, monthly, and yearly time-unit granularities.

https://cloud.google.com/bigquery/docs/creating-column-partitions#daily_partitioning_vs_hourly_partitioning