![](/img/trans.png)
[英]AirFlow Python operator error: got an unexpected keyword argument 'conf'
[英]Airflow Invalis arguments and keyword argument
使用 GCSToBigQueryOperator 發生此錯誤
Broken DAG: [/opt/airflow/dags/injest_data.py] Traceback (most recent call last):
File "/opt/airflow/dags/injest_data.py", line 79, in <module>
> "sourceUris": [f"gs://{BUCKET_NAME}/*"],
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 397, in apply_defaults
raise AirflowException(f"missing keyword arguments {display}")
airflow.exceptions.AirflowException: missing keyword arguments 'bucket', 'destination_project_dataset_table','source_objects'****
當我嘗試更改為 BigQueryCreateExternalTableOperator 時,會發生其他錯誤
Broken DAG: [/opt/airflow/dags/injest_data.py] Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 411, in apply_defaults
result = func(self, **kwargs, default_args=default_args)
File "/home/airflow/.local/lib/python3.7/site-packages/airflow/models/baseoperator.py", line 760, in __init__
f"Invalid arguments were passed to {self.__class__.__name__} (task_id: {task_id}). "
airflow.exceptions.AirflowException: Invalid arguments were passed to BigQueryCreateExternalTableOperator (task_id: bq_external_table_task). Invalid arguments were:
**kwargs: {'tables_resouces': {'tableReferences': {'projectId': 'de-projects-373304', 'datasetId': 'stockmarket_dataset', 'tableId': 'stockmarket_ex
在此先感謝您的幫助...
我曾嘗試更改谷歌查詢運算符,甚至嘗試使用不同的方法將數據上傳到 bigquery,但說模式不存在,請幫助我了解哪里做錯了。 在此先感謝您的幫助,以下是導致錯誤的代碼
bq_external_table_task = BigQueryCreateExternalTableOperator(
task_id = "bq_external_table_task",
tables_resouces = {
"tableReferences": {
"projectId": PROJECT_ID,
"datasetId": BIGQUERY_DATASET,
"tableId":f"{DATASET}_external_table",
},
"externalDataConfiguration": {
"autodetect": True,
"sourceFormat": f"{INPUT_FILETYPE.upper()}",
"sourceUris": [f"gs://{BUCKET_NAME}/*"],
},
},
)
GCSToBigQueryOperator中沒有sourceUris
命名參數。 它應該有source_objects
。 請從以下官方文檔中檢查運算符的參數: GCSToBigQueryOperator
您的BigQueryCreateExternalTableOperator也有錯誤的參數名稱。 tables_resouces
應該有table_resource
。 您也可以從官方文檔查看此運算符的參數: BigQueryCreateExternalTableOperator
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.