简体   繁体   中英

Reduce verbosity/frequency of Amazon S3 AWS upload ProgressListener?

I am doing a multi-part file upload to S3. I added a ProgressListener to print out progress and it works great. Too great. The file I am uploading is several GB large and the messages I am getting logged are way too granular. I am getting several thousand of these:

2021-11-04 06:56:07 INFO  GenericSftpLoader:190 - transfered bytes: 8192

Is there any way to reduce this from printing out every 8192 bytes and instead something more reasonable like 1048576 bytes?

My actual code is in Scala but I followed this example closely from the AWS SDK documentation:

TransferManager tm = new TransferManager(new ProfileCredentialsProvider());        

    PutObjectRequest request = new PutObjectRequest(
            existingBucketName, keyName, new File(filePath));
    
    request.setGeneralProgressListener(new ProgressListener() {
        @Override
        public void progressChanged(ProgressEvent progressEvent) {
            System.out.println("Transferred bytes: " + 
                    progressEvent.getBytesTransferred());
        }
    });       

    Upload upload = tm.upload(request);

So I was able to get a solution. Here's what I did:

TransferManager tm = new TransferManager(new ProfileCredentialsProvider());

PutObjectRequest request = new PutObjectRequest(
        existingBucketName, keyName, new File(filePath));

request.setGeneralProgressListener(new ProgressListener() {
    private Long myBytes = 0
    @Override
    public void progressChanged(ProgressEvent progressEvent) {
        myBytes += progressEvent.getBytesTransferred();
        if (bytesTransferred > 1048576) {
            System.out.println("Transferred bytes: " + 
                    myBytes);
            myBytes = 0;
        }
    }
});       

Upload upload = tm.upload(request);

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM