Java.net.SocketException：从 S3 读取文件时连接重置

Question

I'm trying to read some files from S3 and do some processing on each file. I can get through some files, but I keep getting Java.net.SocketException: Connection reset at the same line in a particular file during processing. The file in question should be ok though because I can process it locally using the same class and method (conversionUtils.convert()).

Service class:

public class FileService {

  @Inject
  private S3Utils s3utils;

  private ConversionUtils conversionUtils = new ConversionUtils();

  public void processFile() {
    List<S3ObjectSummary> files = s3utils.getAllFiles();
    List<S3Object> unprocessedFiles = s3utils.getUnprocessedFiles(files);

    for(S3Object file: unprocessedFiles) {
      InputStream content = file.getObjectContent();
      List<Record> records = conversionUtils.convert(content); //Exception thrown here
    }
  }
}

S3Utils class:

@Component
public class S3utils {
  @Inject AmazonS3 amazonS3;

  public List<S3ObjectSummary> getAllFiles() {
    ListObjectsV2Request request = new ListObjectsV2Request().withBucketName('something').withPrefix('some_prefix');
    ListObjectsV2Result result = amazonS3.listObjectsV2(request);;

    return result.getObjectSummaries();
  }

  public List<S3Object> getUnprocessedFiles(List<S3ObjectSummary> files) {
    //do some filtering here

    List<S3Object> unprocessedFiles = new ArrayList<>();
    for (S3ObjectSummary summary : filteredSummaries) {
      S3Object s3Object = amazonS3.getObject(new GetObjectRequest(summary.getBucketName(), summary.getKey()));
      unprocessedFiles.add(S3Object);
    }
    return unprocessedFiles;
  }
}

Config class:

@Configuration
public class Config {
  @Bean
  public AmazonS3 amazonS3() {
    return AmazonS3ClientBuilder.standard().withCredentials(new DefaultAWSCredentialsProviderChain()).build();
  }
}

I've read some other threads with similar errors where the issue was that the AmazonS3 client was being garbage collected, and therefore closing the stream, but I'm wondering if that's the case here. Any ideas on what the issue is? Thanks.

Answer 1

I was able to overcome this by setting the ClientConfig to keep the connection alive

AmazonS3ClientBuilder
   .standard()
   .withCredentials(new DefaultAWSCredentialsProviderChain())
   .withClientConfiguration(new ClientConfiguration())
   .withTcpKeepAlive(true)
   .build();

Answer 2

I ran into same issue with aws sdk 1.X .

Strange thing here is, it's working fine in local machine and Connection reset while application runs on server.

Tried with withTcpKeepAlive(true) , but result was same ie,

Java.net.SocketException: Connection reset

A workaround

I tried withRange(startPosition,endPosition) until I read all data and it worked.

Answer 3

在我将 s3 代理升级到 1.6.0 并将 aws-java-sdk-s3 升级到 1.1.285 后，连接重置问题得到解决。