[英]how to read s3 files from apache beam python?
I am using Apache Beam python SDK to read s3 file data.我正在使用 Apache Beam python SDK 读取 s3 文件数据。
code I am using我正在使用的代码
ip = (pipe
| beam.io.ReadFromText("s3://bucket_name/file_path")
| beam.Map(lambda x: x.split(","))
| beam.Map(print)
)
but it gives a error但它给出了一个错误
S3ClientError('Unable to locate credentials', None)}")}
Anybody have any idea how to give credentials to s3 client thorugh apache beam python file任何人都知道如何通过 apache beam python 文件向 s3 客户端提供凭据
You need to ensure the credentials are available on your worker machines.您需要确保凭据在您的工作计算机上可用。 One way to do this is installing them in a custom container .
一种方法是将它们安装在自定义容器中。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.