[英]Read AWS s3 File to Java code
我試圖從 AWS s3 讀取文件到我的 java 代碼:
File file = new File("s3n://mybucket/myfile.txt");
FileInputStream fileInput = new FileInputStream(file);
然后我得到一個錯誤:
java.io.FileNotFoundException: s3n:/mybucket/myfile.txt (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.(FileInputStream.java:146)
有沒有辦法從 AWS s3 打開/讀取文件? 非常感謝!
Java 中的“文件”類不了解 S3 的存在。 以下是從 AWS 文檔中讀取文件的示例:
AmazonS3 s3Client = new AmazonS3Client(new ProfileCredentialsProvider());
S3Object object = s3Client.getObject(new GetObjectRequest(bucketName, key));
InputStream objectData = object.getObjectContent();
// Process the objectData stream.
objectData.close();
在 2019 年,有一種更好的方法可以從 S3 讀取文件:
private final AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard().build();
private Collection<String> loadFileFromS3() {
try (final S3Object s3Object = amazonS3Client.getObject(BUCKET_NAME,
FILE_NAME);
final InputStreamReader streamReader = new InputStreamReader(s3Object.getObjectContent(), StandardCharsets.UTF_8);
final BufferedReader reader = new BufferedReader(streamReader)) {
return reader.lines().collect(Collectors.toSet());
} catch (final IOException e) {
log.error(e.getMessage(), e)
return Collections.emptySet();
}
}
在java中讀取S3文件的步驟可以是:
1 >>>
BasicAWSCredentials awsCreds = new BasicAWSCredentials("accessKey", "secretKey");
AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion("region_name_here").build();
2 >>>
S3Object object = s3Client.getObject(new GetObjectRequest("bucketName", "key"));
3 >>>
BufferedReader reader = new BufferedReader(new InputStreamReader(object.getObjectContent()));
String s = null;
while ((s = reader.readLine()) != null)
{
System.out.println(s);
//your business logic here
}
謝謝。
如果文件的內容是字符串,則可以使用getObjectAsString 。 否則,您可以在getObjectContent()
上使用IOUtils.toByteArray將文件內容讀入字節數組。
顯然,這些最適合用於易於放入內存的小型 S3 對象。
private final AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard().build();
private String loadStringFromS3() {
try {
return amazonS3Client.getObjectAsString(BUCKET_NAME, FILE_NAME);
} catch (final IOException e) {
log.error(e.getMessage(), e)
return null;
}
}
private byte[] loadDataFromS3() {
try (final S3Object s3Object = amazonS3Client.getObject(BUCKET_NAME, FILE_NAME)) {
return IOUtils.toByteArray(s3Object.getObjectContent());
} catch (final IOException e) {
log.error(e.getMessage(), e)
return null;
} finally {
IOUtils.closeQuietly(object, log);
}
}
我們也可以使用software.amazon.awssdk:s3
//Assuming the credentials are read from Environment Variables, so no hardcoding here
S3Client client = S3Client.builder()
.region(regionSelected)
.build();
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(bucketName)
.key(fileName)
.build();
ResponseInputStream<GetObjectResponse> responseInputStream = client.getObject(getObjectRequest);
InputStream stream = new ByteArrayInputStream(responseInputStream.readAllBytes());
System.out.println("Content :"+ new String(responseInputStream.readAllBytes(), StandardCharsets.UTF_8));
這是我的解決方案。 我正在使用彈簧靴 2.4.3
創建亞馬遜 s3 客戶端
AmazonS3 amazonS3Client = AmazonS3ClientBuilder
.standard()
.withRegion("your-region")
.withCredentials(
new AWSStaticCredentialsProvider(
new BasicAWSCredentials("your-access-key", "your-secret-access-key")))
.build();
創建一個亞馬遜轉賬客戶端。
TransferManager transferManagerClient = TransferManagerBuilder.standard()
.withS3Client(amazonS3Client)
.build();
在/tmp/{your-s3-key} 中創建一個臨時文件,以便我們可以將我們下載的文件放在這個文件中。
File file = new File(System.getProperty("java.io.tmpdir"), "your-s3-key");
try {
file.createNewFile(); // Create temporary file
} catch (IOException e) {
e.printStackTrace();
}
file.mkdirs(); // Create the directory of the temporary file
然后,我們使用傳輸管理器客戶端從 s3 下載文件
// Note that in this line the s3 file downloaded has been transferred in to the temporary file that we created
Download download = transferManagerClient.download(
new GetObjectRequest("your-s3-bucket-name", "your-s3-key"), file);
// This line blocks the thread until the download is finished
download.waitForCompletion();
現在 s3 文件已成功傳輸到我們創建的臨時文件中。 我們可以得到臨時文件的InputStream。
InputStream input = new DataInputStream(new FileInputStream(file));
因為不再需要臨時文件,我們就刪除它。
file.delete();
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.