[英]How to read a file from minIO in apache beam java sdk
I just started with minio and apache beam.我刚开始使用 minio 和 apache 光束。 I have created a bucket on play.min.io and added few files (let suppose files stored are one.txt and two.txt).
我在 play.min.io 上创建了一个存储桶并添加了一些文件(假设存储的文件是 one.txt 和 two.txt)。 I want to access the files stored on that bucket with Apache beam java sdk.
我想使用 Apache 光束 java sdk 访问存储在该存储桶上的文件。 When i deal with local files i just pass the path of file like C://new//.. but i don't know how to get files from minio.
当我处理本地文件时,我只是传递文件的路径,如 C://new//.. 但我不知道如何从 minio 获取文件。 Can anyone help me with the code.
谁能帮我写代码。
I managed to have it work with some configurations on top of the standard AWS configuration:我设法让它与标准 AWS 配置之上的一些配置一起工作:
PipelineOptions options = PipelineOptionsFactory.fromArgs(args).create();
...
options.as(AwsOptions.class).setAwsServiceEndpoint("http://localhost:9000");
This can be done by extending DefaultS3ClientBuilderFactory with this kind of MinioS3ClientBuilderFactory:这可以通过使用这种 MinioS3ClientBuilderFactory 扩展 DefaultS3ClientBuilderFactory 来完成:
public class MinioS3ClientBuilderFactory extends DefaultS3ClientBuilderFactory {
@Override
public AmazonS3ClientBuilder createBuilder(S3Options s3Options) {
AmazonS3ClientBuilder builder = super.createBuilder(s3Options);
builder.withPathStyleAccessEnabled(true);
return builder;
}
}
and inject it in the options like this:并将其注入如下选项中:
Class<? extends S3ClientBuilderFactory> builderFactory = MinioS3ClientBuilderFactory.class;
options.as(S3Options.class).setS3ClientFactoryClass(builderFactory);
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.