Google BigQuery 是否支持全文搜索

Question

I'm thinking about copying my text searchable content to Google's BigQuery and then perform full-text search using BigQuery API.

Does Google BigQuery support that scenario?

I could not find "search" command in Google BigQuery API: https://developers.google.com/bigquery/docs/reference/v2/

Answer 1

BigQuery support a collection of RegEx and String query functions, making it suitable for text search queries across STRING fields. However, there is a 64k per row (and field) limit for each BigQuery record, so it may not possible to support a totally unstructured, unlimited size, document text search case.

• https://developers.google.com/bigquery/docs/query-reference#stringfunctions
• https://developers.google.com/bigquery/docs/query-reference#regularexpressionfunctions

For a full text search capabilities in an App Engine application, I would suggest looking at the new Search API:

• https://developers.google.com/appengine/docs/python/search/overview

Answer 2

10 years late and here we are. Today (07/04/22) BigQuery launched It equivalent of Full Text Search. Here is the doc:

https://cloud.google.com/blog/products/data-analytics/pinpoint-unique-elements-with-bigquery-search-features/

Answer 3

The litecene library provides full-text search support for BigQuery using a "lucene light" syntax.

(smartphone OR "smart phone"~8 OR iphone OR "i phone" OR "apple phone" OR android OR "google phone" OR "windows phone") AND app*

It compiles the boolean query language down to regular expression matches. It also makes use of new BigQuery search features when possible, namely the SEARCH function and search indexes when possible, although at the time of this writing the searches supported by those features are fairly limited. Using litecene, full-text search can also be deployed against existing production datasets without any ETL changes or re-indexing using non-aggregate materialized views . The searches can target one or multiple columns.

Disclaimer: I am the author of the library.