My service account is not able to access BigQuery Data even if it have sufficient permission

Question

I want to query data in BigQuery by using BigQuery API. These are what I've done.

1. Create service account from "API Services" menu screenshot
2. I gave "BigQuery Data Editor" and "BigQuery User" permission to it screenshot
3. Added key to this service account screenshot
4. Try to retrieve data from BigQuery table on that project with the service account I made. But it fails with the error message below.

Access Denied: Table [project ID]-[table name]: User does not have bigquery.tables.get permission for table [project ID]-[table name].

This is very weird because on BigQuery predefined IAM roles in Google document, "bigquery.tables.get" permission is included in "BigQuery Data Editor".
screenshot

I double checked that the service account have "BigQuery Data Editor" and "BigQuery User" permission from IAM & Admin menu.
screenshot
I try this process many times to confirm that I don't make a mistake, but the result won't changed. What can be the reason for this problem? Any comment will be appreciated.

ps I use Software named DataBricks, data analytics tool provided by microsoft, by following this step . FIrst I doubted the configuration on DataBricks, But as I succeeded to retrieve the data with another BigQuery account, I think the problem exist on this BigQuery account.

Answer 1

Possibly you're not running with the credentials you think? Can you run a query from your databricks tool to verify? Something like SELECT SESSION_USER() as whoami should confirm things.

The other most likely thing that could cause this error is that the tool is trying to use a different project ID than you're expecting. Does databricks have a configuration option for defining the default project ID to use?