Bigquery - Data Consistency not guaranteed for external data sources

Question

The Google documentation for BigQuery mentions that the service does not guarantee data consistency for external data sources . Changes to the underlying data while a query is running can result in unexpected behavior.

Source: https://cloud.google.com/bigquery/external-data-sources

I am not exactly sure what it means. Can someone describe a situation where this may be problematic? (What happens vs what was the user expecting).

Answer 1

You can refer ACID properties ( Consistency ) of database:

Definition: The database must remain in a consistent state after any transaction. No transaction should have any adverse effect on the data residing in the database. If the database was in a consistent state before the execution of a transaction, it must remain consistent after the execution of the transaction as well.

Unexpected behavior: As the data source is heterogeneous (external) – Any changes to the underlying data in the data source while a query is running in BigQuery referring to external data source can result in unexpected behavior.