Split websocket message in multiple frames

Question

I am using native javascript websocket in browser and we have an application hosted on AWS where every request goes through API gateway. In some cases, request data is going upto 60kb, and then my websocket connection is closing automatically. In AWS documentation, I found out below explanation of this issue

https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-known-issues.html

API Gateway supports message payloads up to 128 KB with a maximum frame size of 32 KB. If a message exceeds 32 KB, you must split it into multiple frames, each 32 KB or smaller. If a larger message is received, the connection is closed with code 1009.

I tried to find how I can split a message in multiple frames using native javascript websocket but could not find any config related to frames in documentation or anywhere else

Although I find something related to message fragmentation but it seems like a custom solution that I need to implement at both frontend and backend https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_servers#message_fragmentation

Answer 1

As far as I know, you cannot do this using the JS AWS SDK "postToConnection" API. Best you can do is write your own poor's man fragmentation and send the chunks as independent messages.

const splitInChunks =
  (sizeInBytes: number) =>
  (buffer: Buffer): Buffer[] => {
    const size = Buffer.byteLength(buffer);

    let start = 0;
    let end = sizeInBytes;

    const chunks: Buffer[] = [];

    do {
      chunks.push(buffer.subarray(start, end));
      start += sizeInBytes;
      end += sizeInBytes;
    } while (start < size);

    return chunks;
  };

Where sizeInBytes must be smaller than 32KB. Then you iterate over the chunks:

await Promise.all(chunks.map(c => apiGatewayClient.postToConnection({ data: JSON.stringify(c), connectionId: myConnectionId })

Which may run into rate limits depending on the number of chunks, so consider sending the requests serially and not in parallel

---

Final remark: Buffer.prototype.subarray is very efficient because it does not reallocate memory: the new chunks point at the same memory space of the original buffer. Think pointer arithmetic in C.