一个优雅的异步解决方案，用于从 Java 中的 Vertx 工作程序 Verticle 将文件从 S3 发送到客户端

Question

I have a Vertx server with a worker vertical to asynchronously handle requests for S3 operations. We need a solution to transfer a file from S3 to the client through our server. A previous question Streaming S3 object to VertX Http Server Response is answered by tsegismont, but it appears that the recommendation would block the Vertx thread. The file transfer belongs in a separate vertical. This recommended solution would not work in a worker vertical since The RoutingContext is not sent across the bus as recommended by How can I send RoutingContext object from routing vertical to some other vertical using vertx.eventBus().send() method? . Note that here it seems the recommended solution, to create a special codec, would not work since a RoutingContext is required in the worker vertical.

Another solution would be to get the object from S3. Save it to file. Then use the fileSend method in WebClient API to send to the client. This... is not an elegant solution.

A third solution would be to abandon the WorkerVertical and use the blockingHandler method in the MainVertical. This is not an async call. The thread would not be released for potentially seconds and is worse than the previous solution.

Answer 1

I don't know if its an option for you, but you can utilize presigned url - https://docs.aws.amazon.com/AmazonS3/latest/dev/ShareObjectPreSignedURL.html

What I do most of the time is using the vertx server to handle coordination and generate presigned url's to get/upload files from s3.

The client get the presigned URL and can download / upload directly to s3 while protecting the actual credentials via the backend.

The presigned URL can have limited lifetime and you get all the scalability of AWS without dealing with file streaming / out-of-memory exceptions / etc.