在未授予 GCS 访问权限的情况下使用外部表

Question

We have 2 GCP projects project-a and project-b . And we want to give access to some external tables in the first project to users in second project using Authorized Views.

Here's what we've done so far:

• create a couple of BigQuery tables in project-a (private dataset) as external tables from GCS parquet files
• create a dataset (public dataset) in project project-b in which we created authorized views on the external tables from project project-a

However if we give access to users in the project project-b to query the public views they receive this error:

Access Denied: BigQuery BigQuery: Permission denied while globbing file pattern.

I know this means they should also have read permission on GCS buckets of project-a but we can't grant this permission in GCS.

Is there a way to achieve this? Or maybe another way of doing?

Answer 1

AFAIK, you must have the permission to access to the external data location to access the data (GCS, Google Sheet, or whatever are located the external data). There is no trick for that

Answer 2

This is now possible using BigLake tables . We simply need to create a connection resource in BigQuery then use it to define an external table. Users now only require access to BigQuery tables, no need to set permissions in data location (GCS here).

1. Create the connection using cloud shell bq command

bq mk --connection --location=REGION --project_id=PROJECT_ID \
    --connection_type=CLOUD_RESOURCE CONNECTION_ID

2. Show the connection and get the associated service account

bq show --connection PROJECT_ID.REGION.CONNECTION_ID

3. Grant the resource connection service account access to Cloud Storage bucket
4. Create a BigLake table using the resource connection:

CREATE EXTERNAL TABLE `PROJECT_ID.DATASET.EXTERNAL_TABLE_NAME`
WITH CONNECTION `PROJECT_ID.REGION.CONNECTION_ID`
OPTIONS (
    format ="TABLE_FORMAT",
    uris = ['FILE_PATH']
);