How to read and modify a csv file on one bucket in cloud storage and save the results in another bucket using Cloud Functions

Question

I have CSV files coming to a folder on a Cloud Storage bucket and I want to create a Cloud Function that opens the CSV, adds a new column to it and then save the results to a new bucket as new.csv file.

Is there a way to do that using python Cloud Function???

Thanks in advance.

Answer 1

The idea that you're trying to implement is totally possible and can be achieved using Google Cloud Functions .

For that, you would need to create astorage-triggered Cloud Function . More specifically, you can create your function in a way that it will respond to change notifications emerging from your Google Cloud Storage.

These notifications can be configured to respond to various events inside a bucket: object creation , deletion , archiving and metadata updates .

For the situation described, you will need to use the trigger google.storage.object.finalize . This event is sent when a new object is created in the bucket or an existing object is overwritten , and a new generation of that object is created.

Here you can find a sample code of a storage-triggered Cloud Function written in Python, while this tutorial will give you a more detailed overview of the usage of storage-triggered functions.