Is it possible to read files that are not in Google Cloud Platform from Cloud DataFlow

Question

I am trying to read a simple CSV that is outside of the Google Cloud Platform. Is it possible. ?

Answer 1

Try this snippet with the local CSV file:

PCollection<String> lines = p.apply(TextIO.Read.named("ReadMyFile").from("/tmp/myfile.csv"));

constructing-your-pipeline reading-data-into-your-pipeline

EDIT:

Based on the manual of TextIO.Read.from() it supports local files.

If you run the code on your local machine or non-GCP server, where the CSV file is, this snippet will be able to load the data into DataFlow. TextIO.Read.From documentation