BigQuery：超出资源 - Google 表格服务超载

Question

I have a Google Spreadsheets with data connected to a Data Studio Panel. I'm using the following data flow to get the data:

Google SpreadSheets --> BigQuery External Table --> View To the External Table --> Data Studio (Updated every 10 minutes)

But for some reason that I don't know, sometimes, when executing a select on the BigQuery External Table I get the following error:

Resources exceeded during query execution: Google Sheets service overloaded for spreadsheet id:XXX

The Google SpreadSheet has only 1500x10 Columns, which I think is pretty small. Also, there are about 6 users.

What can cause that error? Any idea about how to solve this?

Thanks

Answer 1

The Google documentation has information about this error:

A BigQuery query can overload Sheets, resulting in an error like Resources exceeded during query execution: Google Sheets service overloaded. Consider simplifying your spreadsheet; for example, by minimizing the use of formulas.

It seems that along with size of the Sheet, the "complexity" also matters. We cannot know how complex is your spreadsheet without seeing it but consider reducing your formula usage. This article also mentions a max result size of 10MB and other pivot table limits. You could also try to divide the data or if the error rate is manageable you could also use some kind of retry strategy to query again until you get the results.

If this is not enough then you may have reached the limits of what you can do with Sheets. Digging deeper I found this Google issue tracker post which has a quote from their engineering team:

The BigQuery Engineering Team has stated that the current suggested approach is to simplify the spreadsheet. Sheets is designed for Web/Mobile use cases and not as a DB backend. Even a couple of thousand rows is large in this context, especially if there are formulas involved.

The post is a feature request to the Google engineering team to allow for more complexity, but these requests can take time and if they don't intend Sheets to be used that way it's also possible that they won't implement it. If you cannot reduce the spreadsheet's complexity enough to stop getting the error you may want to consider querying the data from a different source.