BigQuery-不平等与平等联接-maximumBillingTier

Question

Here is the inequality condition that I have in my join (simple overlap conditions):

ON
(A.start <= B.End) AND (B.Start <= A.END)

It gives me the following error:

java.lang.RuntimeException: BigQueryError{reason=billingTierLimitExceeded, location=null, message=Query exceeded resource limits. 700920.3330645757 CPU seconds were used, and this query must use less than 529900.0 CPU seconds.

Surprisingly, this operation takes more than running the sequential algorithm (w/o any join) on a single instance (n1-highmem-16).

I have a couple of questions:

1) How can I calculate maximumBillingTier for my query?

2) Can someone explain how inequality joins work in BigQuery?

3) Why inequality joins are so expensive? 
Is it because of number of operations, or is it because of large number of outputs?

For the same query and input tables, inequality joins takes more than 13000 seconds and eventually gets canceled due to time-out, but if I change the condition to only cover equality, it would take only 70 secs.

Thanks!

Answer 1

1) How can I calculate maximumBillingTier for my query?

I think this goes down to the notion of Slots

A BigQuery slot is a unit of computational capacity required to execute SQL queries. BigQuery automatically calculates how many slots are required by each query, depending on query size and complexity.

The default number of slots for on-demand queries is shared among all queries in a single project. As a rule, if you're processing less than 100 GB of queries at once, you're unlikely to be using all 2,000 slots.

To check how many slots you're using, see Monitoring BigQuery Using Stackdriver .

See more details at Query Jobs Quotas

2) Can someone explain how inequality joins work in BigQuery?

This can really depends on data size and distribution
I would recommend Query Plan Explanation - it can help not only in understanding what is going on under-hood but also will help you to optimize your query