Server streaming with Cloud Run

Question

On October 7 2020 and Januari 21 2021, Google introduced unidirectional server streaming and bidirectional web sockets respectively for Cloud Run. Here are the blog posts:

https://cloud.google.com/blog/products/serverless/cloud-run-now-supports-http-grpc-server-streaming https://cloud.google.com/blog/products/serverless/cloud-run-gets-websockets-http-2-and-grpc-bidirectional-streams

From the second link:

This means you can now build a chat app on top of Cloud Run using a protocol like WebSockets, or design streaming APIs using gRPC.

This raises some questions:

• How does it work with auto scaling?
• Say we build a chat app and we have ws connections distributes across multiple instances and need to push a message to all of them. How would we do?
• Is it okey for the instances to keep a state now(the web socket connection)? What are the consequences of this?

What I am trying to ask; How do we build a scaleable chat application with Cloud Run and other managed tools available in Google Cloud with features like private messages and public chat rooms?

Answer 1

How does it work with auto-scaling?

Each WebSocket connection will consume 1 connection out of 250 available connection capacity per container. (250 is subject to change in the future as it had been 80 but increased to 250 recently.) This limit info is available in the Google Cloud Run Limits doc. When container's all 250 connections are occupied, another container instance will start automatically.

Say we build a chat app and we have ws connections distributes across multiple instances and need to push a message to all of them. How would we do?

You would have to use some form of central datastore or pubsub to solve that problem. eg Google provides Google Cloud PubSub, or you can setup a Redis instance and use Redis' PubSub feature. There are many ways to tackle this problem.

Is it okay for the instances to keep a state now(the web socket connection)? What are the consequences of this?

It is always safe to keep a state in a container, but you just need to make sure that the container can be terminated at any time when there isn't an active connection. Also, according to the doc , Google Cloud Run will terminate all HTTP requests (including WebSockets) after request timeouts config, which has a default value of 5 min and can be increased to 15 min. Therefore, your WebSocket connections will likely be dropped after 15 min, and you should have a logic to handle auto reconnection. Google Cloud Run doc explicitly talks about this limit.