[英]AWS: reading Kinesis Stream data using Kinesis Firehose in a different account
We have a team writing data into into Kinesis Stream on their account (Acc A).我们有一个团队将数据写入他们账户 (Acc A) 的 Kinesis Stream。 I want to consume this data in my account (Acc B) ideally using Kinesis Firehose and just moving that data into S3.
我想在我的账户 (Acc B) 中使用这些数据,理想情况下使用 Kinesis Firehose 并将该数据移动到 S3 中。
I went through pages and pages of AWS documentation but I haven't found an answer.我浏览了一页又一页的 AWS 文档,但没有找到答案。 Is it possible to consume data from Kinesis Stream using Firehose which is in a different AWS Account.
是否可以使用不同 AWS 账户中的 Firehose 使用来自 Kinesis Stream 的数据。 And if yes, then how and if not, what would be the most straight forward solution to get this data and move it to S3?
如果是,那么如何以及如果不是,获取这些数据并将其移动到 S3 的最直接的解决方案是什么?
Bear in mind that the owners of the Stream refuse to do any changes in their architecture.请记住,Stream 的所有者拒绝对其架构进行任何更改。
Thank you for any help.感谢您的任何帮助。
Having a Kinesis Data Stream connecting directly to Kinesis Firehose in a different account is right now not possible.现在无法将 Kinesis Data Stream 直接连接到不同账户中的 Kinesis Firehose。 But, you could for example use a lambda function to transfer data between cross account - This is a setup I am using and it works fine.
但是,您可以例如使用 lambda 函数在跨帐户之间传输数据 - 这是我正在使用的设置,并且工作正常。 This could be quite expensive depending on the amount of data.
根据数据量,这可能非常昂贵。
There is a tutorial available here - only difference to your requirement is that this lambda is based in account A.有可用的教程这里-只有你的要求不同的是,这拉姆达是基于帐户A.
I would recommend to convice the source account owner to create an kinesis delivery stream on their side to push data into your S3 bucket.我建议说服源帐户所有者在他们这边创建一个 kinesis 传输流,以将数据推送到您的 S3 存储桶中。 This is much easier.
这要容易得多。
Never the less, if this is not an option you can create a Kinesis Data Analytics Stream to consume data from another account which you then can pump into S3 using a Kinesis Delivery Stream.无论如何,如果这不是一个选项,您可以创建 Kinesis Data Analytics Stream 来使用来自另一个账户的数据,然后您可以使用 Kinesis Delivery Stream 将这些数据泵入 S3。
A similiar approach is explained here: https://docs.aws.amazon.com/kinesisanalytics/latest/java/examples-cross.html这里解释了类似的方法: https : //docs.aws.amazon.com/kinesisanalytics/latest/java/examples-cross.html
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.