[英]AWS DynamoDB Stream into Redshift
We would like to move data from DynamoDB NoSQL into Redshift Database continously as a stream. 我们希望将数据作为流从DynamoDB NoSQL连续移至Redshift Database。 I am having hard time understand all the new terms/technologies in AWS. 我很难理解AWS中的所有新术语/技术。 There is 有
1) DynamoDB Streams 1) DynamoDB流
2) AWS Lambda 2) AWS Lambda
3) AWS Kinesis Firehose 3) AWS Kinesis Firehose
Can someone provide a brief summary of each. 有人可以提供每个摘要吗? What are DynamoDB streams? 什么是DynamoDB流? How does this differ from AmazonKinesis? 这与AmazonKinesis有何不同? After reading all the resources, this is my hypothesis understanding, please verify below. 阅读所有资源后,这是我对假设的理解,请在下面进行验证。
(a) I assume DynamoDB Streams, create the streaming data of NoSQL, and start sending it out. (a)我假设使用DynamoDB流,创建NoSQL的流数据,然后开始将其发送出去。 It is the Sender. 是发件人。
(b) Lambda allows people for only time consumed, it is the time for rent Server which handles the DynamoDB Stream. (b)Lambda只允许人们消耗时间,这是处理DynamoDB流的Rent Server时间。
(c) Kinesis FireHose Converts the DynamoDB Stream, and places into Redshift. (c)Kinesis FireHose转换DynamoDB流,并放入Redshift。
(d) AmazonQuickSight is their business intelligence tool, (d)AmazonQuickSight是他们的商业智能工具,
Is that the correct understanding of the glossary terms? 这是对词汇表术语的正确理解吗? Reviewing Stack link , wanted more thorough information. 正在查看Stack链接 ,想要更全面的信息。
Amazon Kinesis can collect, process, and analyze video and data streams in real time. Amazon Kinesis可以实时收集,处理和分析视频和数据流。
DynamoDB streams are effective the same as a Kinesis Data Stream, but it is automatically generated by new/changed data in DynamoDB. DynamoDB流与Kinesis Data Stream一样有效,但是它由DynamoDB中的新数据/更改后的数据自动生成。 This allows applications to be notified when new data is added to a DynamoDB table, or when data is changed. 这样可以在将新数据添加到DynamoDB表或更改数据时通知应用程序。
A Kinesis Data Firehose can automatically output a stream into Redshift (amongst other destinations). Kinesis Data Firehose可以自动将流输出到Redshift(以及其他目的地)。
AWS Lambda can run code without provisioning or managing servers. AWS Lambda无需配置或管理服务器即可运行代码。 You pay only for the compute time you consume — there's no charge when your code isn't running. 您只需为您消耗的计算时间付费-代码未运行时无需付费。 You can run code for virtually any type of application or backend service — all with zero administration. 您几乎可以为任何类型的应用程序或后端服务运行代码-只需进行零管理即可。
Lambda is useful for inspecting data coming through a stream. Lambda对于检查流中的数据很有用。 For example, it could be used to manipulate the data format or skip-over data that is not required. 例如,它可以用于处理不需要的数据格式或跳过数据。
Putting it all together, you could have data added/modified in DynamoDB . 放在一起,您可以在DynamoDB中添加/修改数据。 This would cause a DynamoDB Stream to be sent that contains information about the change. 这将导致发送包含有关更改信息的DynamoDB流 。 An AWS Lambda function could inspect the data and manipulate/drop the message. AWS Lambda函数可以检查数据并处理/删除消息。 If could then forward the data to Kinesis Data Firehose to automatically insert the data into Amazon Redshift . 如果可以,则将数据转发到Kinesis Data Firehose以自动将数据插入Amazon Redshift 。
Here's an example: 这是一个例子:
The benefit of using these systems together is that they can provide rich application functionality with minimal coding. 一起使用这些系统的好处是它们可以用最少的编码提供丰富的应用程序功能。 In this example, only the Lambda function needed coding -- the rest worked via linking together various components. 在此示例中,仅Lambda函数需要编码-其余功能通过将各种组件链接在一起而起作用。 Also, it was totally serverless — that is, there was no need to run an application on an Amazon EC2 instance. 而且,它完全是无服务器的-也就是说,无需在Amazon EC2实例上运行应用程序。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.