AWS S3 上传文件名不匹配

Question

I am able to upload multiple files to s3 bucket at once. However there is a mismatch in the file name the one I provided and uploaded file. I am interested in file name as I need to generate cloud front signed url based on that.

File generation code

final String fileName = System.currentTimeMillis() + pictureData.getFileName();
final File file = new File(fileName); //fileName is -> 1594125913522_image1.png
writeByteArrayToFile(img, file);

AWS file upload code

public void uploadMultipleFiles(final List<File> files) {

        final TransferManager transferManager = TransferManagerBuilder.standard().withS3Client(amazonS3).build();
        try {
            final MultipleFileUpload xfer = transferManager.uploadFileList(bucketName, null, new File("."), files);
            xfer.waitForCompletion();
        } catch (InterruptedException exception) {
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("InterruptedException occurred=>" + exception);
            }
        } catch (AmazonServiceException exception) {
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("AmazonServiceException occurred =>" + exception);
            }
            throw exception;
        }
    }

Uploaded file name is 94125913522_image1.png . As you can see first two characters disappeared. What am I missing here. I am not able to figure out. Kindly advice.

private static void writeByteArrayToFile(final byte[] byteArray, final File file) {
        try (OutputStream outputStream = new BufferedOutputStream(Files.newOutputStream(Paths.get(file.getName())))) {
            outputStream.write(byteArray);
        } catch (IOException exception) {
            throw new FileIllegalStateException("Error while writing image to file", exception);
        }
    }

Answer 1

The reason of the problem

You lose the first two charecters of the file names because of the third argument of this method:

transferManager.uploadFileList(bucketName, null, new File("."), files);

What happens in this case

So, what is the third argument:

/**
...
* @param directory
* The common parent directory of files to upload. The keys
* of the files in the list of files are constructed relative to
* this directory and the virtualDirectoryKeyPrefix.
...
*/
public MultipleFileUpload uploadFileList(... , File directory, ...){...}

And how will it be used:

...
int startingPosition = directory.getAbsolutePath().length();
            if (!(directory.getAbsolutePath().endsWith(File.separator)))
                startingPosition++;
...
String key = f.getAbsolutePath().substring(startingPosition)...

Thus, the directory variable is used to define a starting index to trim file paths to get file keys.

When you pass new File(".") as a directory, the parent directory for your files will be {your_path}.

But this is a directory, and you need to work with files inside it. So the common part, retrieved from your directory file, is {your_path}./

That is 2 symbols more than you actually need. And for this reason this method trims the 2 extra characters - an extra shift of two characters when trimming the file path.

The solution

If you only need to work with the current directory, you can pass the current directory as follows:

MultipleFileUpload upload = transferManager.uploadFileList(bucketName, "", 
                            System.getProperty("user.dir"), files);

But if you start working with external sources, it won't work. So you can use this code, which creates one MultipleFileUpload per group of files from one directory.

private final String PATH_SEPARATOR = File.separator;
private String bucketName;
private TransferManager transferManager;

public void uploadMultipleFiles(String prefix, List<File> filesToUpload){
    Map<File, List<File>> multipleUploadArguments = 
                  getMultipleUploadArguments(filesToUpload);
    for (Map.Entry<File, List<File>> multipleUploadArgument:
                                      multipleUploadArguments.entrySet()){
        try{
            MultipleFileUpload upload = transferManager.uploadFileList(
                    bucketName, prefix,
                    multipleUploadArgument.getKey(),
                    multipleUploadArgument.getValue()
            );
            upload.waitForCompletion();
        } catch (InterruptedException ex) {
            throw new RuntimeException(ex);
        }
    }
}

private Map<File, List<File>> getMultipleUploadArguments(List<File> filesToUpload){
    return filesToUpload.stream()
            .collect(Collectors.groupingBy(this::getDirectoryPathForFile));
}

private File getDirectoryPathForFile(File file){
    String filePath = file.getAbsolutePath();
    String directoryPath = filePath.substring(0, filePath.lastIndexOf(PATH_SEPARATOR));
    return new File(directoryPath);
}