Google Cloud - Communicating between instances

Question

I am in the process of delivering a product.

What my product needs to do is spin up and down multiple instances on google cloud. Basically, I have a main instance which I plan to control the rest of them. A 'worker factory' if you will.

What would be a good way to communicate results from the workers back to the main instance.

The actually worker already inserts its results to a sql table also hosted by google, before spinning down.

However I am looking to additionally send off a log file to the main instance. What would be the best way of doing this?

ps - I am not extremely familiar with networking, however I am looking to learn

Answer 1

There are likely many ways to achieve your goal. One thought is to use Publish/Subscribe. In this concept, you create a resource called a topic and your main instance would register a subscription to the topic. Now, when your workers wish to transmit information they would executed a publish on the topic. There is no networking knowledge needed to make this all work. The topic becomes a centralized drop-off for information published.

Here are the docs on pub/sub: https://cloud.google.com/pubsub/docs/