Kinesis：获取未处理记录的列表

Question

What's the best way to get a list of unprocessed records inside a stream?

Kinesis seems to offer a number of high level metrics ( GetRecords.Records, GetRecords.Success, IncomingRecords, PutRecords.FailedRecords, PutRecords.Records, PutRecords.Success, PutRecords.SuccessfulRecords, PutRecords.TotalRecords ) but

a)those have been hard to correlate with each other and
b)they don't point to specific records.

Any advice would help.

Answer 1

Well, Kinesis is a messaging queue which uses producer/consumer to write/read data. Most of the logic requires to be implemented using the Kinesis client.

It is not so straight forward to capture the failed or unprocessed messages directly in kinesis stream.

One case can be - If you want to detect the failedRecords in Kinesis, you can configure your Kinesis Producer to capture the error_codes and the message which fails from put_records() operation

Same applies to processRecords() operation, you need to handle the exceptions gracefully.