在 GCP 中使用 Cloud Functions 抓取页面并将其加载到 BigQuery

Question

I have got question about GCP. Is it possible to scrape page on a daily basis using Cloud Functions and send it to BigQuery. I have already wrote working code in Python that scrapes page and save it to csv and now I would like to move it to cloud. How can I do that?

Answer 1

You can load your scraped ( csv ) data to Google Cloud Storage from python. Bigquery supports loading data as a new table or appending new data to a table. The latter requires you to do it in python, the former can be done using the UI in the cloud console.

More info Here And Here