Better Way to Pull Data Through API?

Question

So I'm trying to design a web app that requires a good amount of data (10,000 rows). This amount, however, will grow exponentially over time. Maybe into hundreds of thousands of rows. I want to describe two methodologies of how to pull in and present the data and get recommendations on which method would be faster overall. Also, the people using the webapplication are local to the servers (very low, if any latency)

The first method:

• Make one call to an API which returns all 10,000 rows of data on the initial application load (pulls all the required data using 1 large stored procedure call)
• Count through this data using JavaScript looping and sort it into 20 categories based on certain criteria (Also JS)
• When a user clicks on one of the counts displayed, they get a page that lists all the data rows pulled that matched that particular category. (This step would be extremely fast because all rows were preloaded)

Second Method:

• Make an initial API call that returns only the counts for each category based on certain criteria (Counts data on sql server instead of JS) When user clicks on count, makes another API call to return only the rows and data associated with that particular category (this means a separate SP for each category)
• This means, every time they click a category, a new API call will execute but the dataload will be much smaller

My hangup on this - The first method requires only 1 API call, but counts all the data in JavaScript and all at once, meaning even if you don't need some of the data, it is all loaded.

The second method requires multiple API calls, but counts all the data on the SQL Server and only returns the data that you need on demand. Will making a lot of separate API calls instead of 1 large API call slow things down?

Answer 1

Imho you shold go with the second option. i would not see a problem wrt to performance. Counting based on category will be fast and the next calls will be smaller as amount returned and processed by the server.

Answer 2

It's important to consider how many times you would need to retrieve this data in a day. If its a handful--like 500 times max--option 1 is fine. If the number of calls is significantly higher than that, option 2 would be better. You would just retrieve a subset as needed.