简体繁体 English

如何为 Appflow on Demand 做增量加载

[英]How to do incremental load for Appflow on Demand

原文 2021-12-08 17:45:07 6 1 amazon-s3/ salesforce/ ondemand/ amazon-appflow

I am updating record from s3 to salesforce using Appflow on demand.What property i can define in the CFT to make my appflow on demand incremental based on the s3 last modified timestamp.我正在使用 Appflow on demand 将记录从 s3 更新到 salesforce。我可以在 CFT 中定义什么属性来使我的 appflow on demand 基于 s3 最后修改的时间戳进行增量。

1 个解决方案

There is no way to perform incremental data load for on-demand app flows.无法为按需应用程序流执行增量数据加载。

Ideally, this use-case might be fulfilled by “Run Flow on Event” settings under “Flow Trigger” option under Appflow, but this option is not available for S3 Bucket as a Source.理想情况下，此用例可能通过 Appflow 下“Flow Trigger”选项下的“Run Flow on Event”设置来实现，但此选项不适用于作为源的 S3 Bucket。

I found some steps that should address this requirement or similar requirement of reading only the latest file/data from the S3 bucket.我发现一些步骤应该可以解决这个要求或类似的要求，即只从 S3 存储桶读取最新的文件/数据。

My understanding is that you would like to configure the Appflow which runs every time when there is an S3 Bucket Change Event.我的理解是，您想配置每次发生 S3 存储桶更改事件时运行的 Appflow。 ( when a new file is added to the S3 bucket ) （当新文件添加到 S3 存储桶时）

However, as a workaround you can use the Lambda function to run the flow with “S3 Event notification” as a Trigger to the lambda.但是，作为一种变通方法，您可以使用 Lambda function 运行流程，将“S3 事件通知”作为 lambda 的触发器。

The Architecture will be as below.架构如下。

S3 Bucket (with new events) > Lambda > Appflow S3 Bucket（有新事件）> Lambda > Appflow

Create a Lambda python program that runs the “flow” every time there is a new event in the S3 Bucket创建一个 Lambda python 程序，每当 S3 存储桶中有新事件时运行“流”