Stop a BigQuery script using a condition

Question

I am running a BigQuery script to generate a table. The script assumes the existence of another table, performs some transformations, and places the transformed data into an output table. However, I want the script to terminate its execution (and possibly post a message) if the input table does not comply with some conditions. What is the best way of terminating a BigQuery script using a condition?

Answer 1

To achieve this without any external app that call the BigQuery API and perform the requirements checks (which is a nicer way, easier to maintain and evolve), is to create a schedule query. In this case, it's well designed for recurring request. If not, code this in your preferred language.

So, with BigQuery scheduled queries, you can perform your query, define the destination table and define a notification channel

Set the PubSub topic that you want. However, this message isn't custom. You will have the status and the reason of the latest execution. Then you will need to dig into to understand exactly what happened during the query and perform complex code to read the log and find the root cause.

If your check wants to know the status OK/KO, this solution is suitable, if not, prefer your own code, you will have a better granularity on the error management.