Moving data from google cloud storage to Amazon s3 via cloud function in a serverless fashion

Question

I have data in Google Cloud Storage that I need to transfer to s3 bucket in a serverless fashion... one possible approach is to use cloud function and transmit data from cloud storage to s3 bucket using gsutil and boto3 for was credentials. I believe their is an extra fee from Google for outbound network request but this approach is possible.

Does anyone has a better approach or a suggestion?

Answer 1

At present there is no better than gsutil : https://stackoverflow.com/a/39333278/10801700

Did you consider the Always Free usage limits in your evaluation? There is "1 GB from North America to each GCP egress destination (Australia and China excluded)" each month. Once you've passed this limit, the general network usage pricing applies.

Note that for Cloud Storage : "Always Free is only available in us-east1 , us-west1 , and us-central1 regions. Usage calculations are combined across those regions."