在无服务器应用程序架构中处理大负载大小

Question

I have a serverless web app that is hosted on AWS.

Backend:

1- Codebuild that fetch data from mysql DB and put it into dynamoDB

2- API Gateway to send request to the Lambda that fetch all items in the dynamoDB

Front end:

1- Angular JS code that send end flag(LastEvaluatedKey ) via API request to lambda

2- Angular UI component primeng that show response in a table, the page takes large time to load.

The data in dynamo DB is pretty HUGE , and I'm doing a pagination on Lambda side to bring 100 record each time until it is completed.

issue : on UI side, it takes large time to bring the full data, how can I enhance this to provide more user friendly experience?

Answer 1

So your table is 6.7MB and you are doing a full table Scan which will require you to read the entire dataset, resulting in 7 Scan requests, each holding 1MB of data before filters are applied.

As you've come to know, this approach is not scalable, as the performance of your query will degrade over time as data grows (the exact issue NoSQL was created to combat).

My advice to you is to create a schema that allows you to use more efficient queries such as GetItem or Query where you only target the specific data tou require rather than the entire table.