简体   繁体   English

有没有办法监控 BigQuery (GCP) 中数据集下的所有表/视图更新?

[英]Is there a way to monitor all table/view updates under a dataset in BigQuery (GCP)?

I want to track all table/view updates in a separate tracking table ie table created timestamp, table modified timestamp etc using stored procedure or Cloud Function or Python?我想使用存储过程或 Cloud Function 或 Python 在单独的跟踪表中跟踪所有表/视图更新,即表创建时间戳、表修改时间戳等? I don't want to use Dataflow or Pub/Sub.我不想使用 Dataflow 或 Pub/Sub。 Any pointers?任何指针?

DATASET 1:数据集 1:

    |CREATED| MODIFIED |ROWS ADDED |ROWS DELETED|

It sounds like you are trying to create a table for logging.听起来您正在尝试创建一个用于记录的表。 You could do this through audit logs .您可以通过审计日志做到这一点。

Audit logs record the following:审核日志记录以下内容:

  • Administrative activity, such as creating or modifying resources.管理活动,例如创建或修改资源。
  • Data access, such as reading user-provided data from a resource.数据访问,例如从资源中读取用户提供的数据。
  • System events that are generated by Google systems, rather than by user actions.由 Google 系统而非用户操作生成的系统事件。

Once you have the logs, you can export them to BigQuery .获得日志后,您可以将它们导出到 BigQuery

声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 GCP Bigquery 未在错误流中提供所有错误记录 - GCP Bigquery not providing all bad records in the error stream 如何使用GCP Dataflow中的python管道代码读取BigQuery表 - How to read BigQuery table using python pipeline code in GCP Dataflow ValueError:必须使用beam.io.gcp.bigquery.ReadFromBigQuery 指定BigQuery 表或查询 - ValueError: A BigQuery table or a query must be specified with beam.io.gcp.bigquery.ReadFromBigQuery BigQuery GCP Python 集成 - BigQuery GCP Python Integration 如何通过API(补丁)授予对BigQuery数据集的新视图访问权限 - How give a new view access to a BigQuery dataset via API (patch) BigQuery | Python | 从BigQuery表VIEW导出数据 - BigQuery | Python | Export data from BigQuery table VIEW 将BigQuery数据集位置存储为变量的最佳方法-Python - Best way to store BigQuery dataset location as variable - python bigquery,extract_table AttributeError:“客户端”对象没有属性“数据集” - bigquery, extract_table AttributeError: 'Client' object has no attribute 'dataset' 有没有办法将策略标签附加到 Bigquery 表? - Is there a way to attach policy tags to a Bigquery table? 使用python处理来自bigquery的巨大数据集,并将其加载回bigquery表 - Processing huge dataset from bigquery using python, load it back to a bigquery table
 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM