BigQuery DataLab的query_params格式

Question

Hi there SO community,

So I've been trying to get this parameterized query to work on a DataLab notebook. Since I plan to execute the query multiple assignments of such parameters I cannot use the %%bq execute syntax. Instead I've been trying to use .execute(query_params=query_params) but I cannot figure out the appropriate format of the query_params data.

For completeness, this is a simplified version of what I have:

%%bq query -n myq
SELECT *
FROM `some_table`
WHERE some_field = @field

query_params = [{'name': 'field', 'type': 'STRING', 'value': 'POTATO'}]
myq.execute(query_params=query_params).result().to_dataframe()

I figured the query_params argument had to be a list by following this and figured that each element should be a dictionary with those keys because of this .

Nonetheless, when I execute this on my DataLab notebook I get an error indicating the parameter type is missing. See part of the error stacktrace below.

/usr/local/envs/py3env/lib/python3.5/site-packages/google/datalab/bigquery/_query.py in execute_async(self, output_options, sampling, context, query_params)
    268                                            allow_large_results=output_options.allow_large_results,
    269                                            table_definitions=self.data_sources,
--> 270                                            query_params=query_params)
    271     except Exception as e:
    272       raise e

/usr/local/envs/py3env/lib/python3.5/site-packages/google/datalab/bigquery/_api.py in jobs_insert_query(self, sql, table_name, append, overwrite, dry_run, use_cache, batch, allow_large_results, table_definitions, query_params)
    200       query_config['queryParameters'] = query_params
    201 
--> 202     return google.datalab.utils.Http.request(url, data=data, credentials=self.credentials)
    203 
    204   def jobs_query_results(self, job_id, project_id, page_size, timeout, start_index=0):

/usr/local/envs/py3env/lib/python3.5/site-packages/google/datalab/utils/_http.py in request(url, args, data, headers, method, credentials, raw_response, stats)
    156           return json.loads(str(content, encoding='UTF-8'))
    157       else:
--> 158         raise RequestException(response.status, content)
    159     except ValueError:
    160       raise Exception('Failed to process HTTP response.')

RequestException: HTTP request failed: Missing query parameter type

Any idea what am I missing here? I've search everywhere trying to find the appropriate format for this query_params thing and no luck so far.

Thanks for the help.

Answer 1

I finally got it to work based on this PR .

The expected format is

query_params = [
  {
    'name': 'field',
    'parameterType': {'type': 'STRING'},
    'parameterValue': {'value': field}
  }
]