GCP storage - read data from mongo db and write it into to file GCP bucket

Question

Background: Want to read data from mongo and write to the file in GCP bucket. I don't want to keep data retrieved from mongo into memory, instead after retrieving first chunk of data, write/flush it to the file in GCP bucket then fetch another chunk from mongo and again write to the same file.

Note: I don't want the data to be buffered in memory and write it to the file at the end/after retrieval of data from mongo is done.

How can this be achieved?

Answer 1

If you have deployed your mongodb in a VM on GCP, you need to use “Cloud Storage Fuse”. Cloud Storage Fuse is an plugin written in go language which helps you to mount your cloud storage bucket to your linux or macos file systems (Source: GCP docs ). You just have to mount your gcs storage and create a csv file in it and update it using file operations.