如何使用 Webflux 异步将 object 放入 S3？

Question

An article, AWS S3 with Java – Reactive , describes how to use the AWS SDK 2.0 client with Webflux.

In the example , they use the following handler to upload to S3 then return a HTTP Created response:

@PostMapping
public Mono<ResponseEntity<UploadResult>> uploadHandler(@RequestHeader HttpHeaders headers,
  @RequestBody Flux<ByteBuffer> body) {

    long length = headers.getContentLength();
    String fileKey = UUID.randomUUID().toString();
    Map<String, String> metadata = new HashMap<String, String>();

    CompletableFuture future = s3client
      .putObject(PutObjectRequest.builder()
        .bucket(s3config.getBucket())
        .contentLength(length)
        .key(fileKey.toString())
        .contentType(MediaType.APPLICATION_OCTET_STREAM.toString())
        .metadata(metadata)
        .build(), 
      AsyncRequestBody.fromPublisher(body));

    return Mono.fromFuture(future)
      .map((response) -> {
        checkResult(response);
        return ResponseEntity
          .status(HttpStatus.CREATED)
          .body(new UploadResult(HttpStatus.CREATED, new String[] {fileKey}));
      });
}

This works as intended. Trying to learn WebFlux, I expected that the following would complete the HTTP upload to S3 asynchronously in the same thread that the subscribe method is called on:

@PostMapping
public Mono<ResponseEntity<UploadResult>> uploadHandler(@RequestHeader HttpHeaders headers, @RequestBody Flux<ByteBuffer> body) {

    long length = headers.getContentLength();
    String fileKey = UUID.randomUUID().toString();
    Map<String, String> metadata = new HashMap<String, String>();

    Mono<PutObjectResponse> putObjectResponseMono = Mono.fromFuture(s3client
        .putObject(PutObjectRequest.builder()
            .bucket(s3config.getBucket())
            .contentLength(length)
            .key(fileKey.toString())
            .contentType(MediaType.APPLICATION_OCTET_STREAM.toString())
            .metadata(metadata)
            .build(),
        AsyncRequestBody.fromPublisher(body)));

    putObjectResponseMono
        .doOnError((e) -> {
            log.error("Error putting object to S3 " + Thread.currentThread().getName(), e);
        })
        .subscribe((response) -> {
            log.info("Response from S3: " + response.toString() + "on " + Thread.currentThread().getName());
        });

    return Mono.just(ResponseEntity
        .status(HttpStatus.CREATED)
        .body(new UploadResult(HttpStatus.CREATED, new String[]{fileKey})));
}

The HTTP POST completes as expected, but the S3 put request fails with this log message:

2020-06-10 12:31:22.275 ERROR 800 --- [tyEventLoop-0-4] c.b.aws.reactive.s3.UploadResource       : Error happened on aws-java-sdk-NettyEventLoop-0-4

software.amazon.awssdk.core.exception.SdkClientException: 400 BAD_REQUEST "Request body is missing: public reactor.core.publisher.Mono<org.springframework.http.ResponseEntity<com.baeldung.aws.reactive.s3.UploadResult>> com.baeldung.aws.reactive.s3.UploadResource.uploadHandler(org.springframework.http.HttpHeaders,reactor.core.publisher.Flux<java.nio.ByteBuffer>)"
    at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:97) ~[sdk-core-2.10.27.jar:na]
    at software.amazon.awssdk.core.internal.util.ThrowableUtils.asSdkException(ThrowableUtils.java:98) ~[sdk-core-2.10.27.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.retryIfNeeded(AsyncRetryableStage.java:125) ~[sdk-core-2.10.27.jar:na]
    at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryExecutor.lambda$execute$0(AsyncRetryableStage.java:107) ~[sdk-core-2.10.27.jar:na]
    ........

I suspect the explanation involves the request to S3 being run on its own thread, but I'm stumped working out what is going wrong, can you shed any light on it?

Answer 1

try this

@RequestBody Flux<ByteBuffer> body
>>> replace @RequestBody byte[]

and

AsyncRequestBody.fromPublisher(body)
>>> replace .fromBytes(body)

and if you want to subscribe from another thread, use: .subscribeOn({Schedulers})