Increase EventGrid's event size, to prevent 413 Payload Too Large

Question

Some messages which are posted to an Azure EventGrid instance result in a 413 Payload Too Large error. Thedocs are quite clear on this :

When posting events to an event grid topic, the array can have a total size of up to 1 MB. Each event in the array is limited to 64 KB . If an event or the array is greater than the size limits, you receive the response 413 Payload Too Large.

Unfortunately, I ran into this issue too late and was unaware of this event size limit until I encountered the error in production. Reducing the body size is the way to go, as I'm aware the messages should be small in footprint size. And most of them are. But for the handful of messages that fail, it requires a lot of plumbing and architectural changes.

Until this is covered, is there a way in Azure EventGrid to increase the 64kb limit reasonably (for example to 128kb), albeit with a setting, a higher plan or a workaround? Just until this can be covered properly through code changes.

Answer 1

As per end of May, 2019 it was officially announced that Event Grid now supports events of up to 1MB in size. In order to use the announcement states:

... There are no configuration changes required, and this will work on existing event subscriptions. To try it out, just push larger events. Everything under 64 KB will be still be covered by our GA SLA.

This is backed by the docs which now formally mention that:

... Support for 64 KB in General Availability (GA). Support for 1 MB is currently in preview.

Although not GA just yet, this is very promising and already usable. This answers the original question for me.

Answer 2

64KB limit is a hard limit. You can't change that. You could use a claim check pattern to offload event data to a store and read it when a message is received.

Answer 3

I agree with Sean about using the ClaimCheck pattern. I've been working with largescale EDAs for 20 years and have always emphasised the need to keep event messages small and nimble (regardless of technology platform). Carrying a pointer to a blob (for example) in the event payload I believe is still good practice. However, I'm interested to understand why MS decided to up their limit to 1MB. This just makes the architect's job harder when trying to convince others of the 'small & nimble events' principle.