Google Cloud - How to get data from remote PubSub as input for Apache Beam Pipeline executing locally?

Question

I've configured a local (java) environment for executing Apache Beam Pipeline following the official doc .

The sample project (WordCount) works perfectly, but now I'd like to change it and get input data from PubSub topic and put output to BigQuery.

I've already created my PubSub topic and my BigQuery dataset, but my question is: how can I configure REMOTE input (pubsub topic) and output (BigQuery) for a locally running Pipeline (for debug purpose)?

Answer 1

See classes PubsubIO / PubsubClient (or com.google.cloud.pubsub.v1 ).

For remote access you'd most likely need to register an external IP.