How can I chunk results (lazy load?) without rewriting my whole application (Laravel + jQuery, SPA style)

Question

I've developed a web application with the concept of Single Page Application but none of the modern techs and frameworks. So I have a jQuery page that dynamically requests data to localhost - a Laravel instance that compiles the entries in the DB (within a given time interval).

So the client wants to see all the entries for last week, the app works fine. But if he wants to see the results for the whole last month... well, they're so many that the default execution time of the php ins't enough to process all the data (30 seconds). I can easily override this, of course, but then the jQuery client will loop through these arrays of objects and do stuff with them (sort, find, sum...). So I'm not even sure jQuery can handle this many data.

So my question can be broken in two:

1. Can laravel ->paginate() be used so the ajax request of jQuery can also chunk the data? How does this work (hopefully in a manner that doesn't force me to rewrite all the code).

2. How could I store large amounts of information on the client? It's only temporary but the users will hang around for a considerable amount of time on my webpage, and I don't want them to wait 5 minutes every time they press a button

Thanks.

Answer 1

If you want to provide an interface to a large amount of data stored in a backed, you should paginate the data . This is a standard approach, so I'm sure your client will be ok with that.

Using pagination is pretty simple - see the docs for Laravel 5.0 here: http://laravel.com/docs/5.0/pagination

In order to paginate results in the backend, you need to call paginate($perPage) on your query instead of get() in your controller, like that:

$users = User::whereIsActive(true)->paginate(15);

This will return paginated result with 15 records per page. Page number will be taken from page parameter of the request. In order to get 3rd page of users, you'll need your frontend jQuery app to send a request to URL like:

/users?page=3

I don't recommend caching data in the frontend application. The data can be changed by some other user and you won't even know about it. And with pagination, your requests should be lightweight enough to stop worrying about a request sent to fetch every page of results.

Answer 2

Not sure if you're subscribed to laracasts but Jeffery Way is amazing in explaining features of Laravel and I highly recommend his videos.

In short you can paginate the results, then on the view when you call the foreach on your items you can array_chunk() the results to display them how you need to. But the paginated results are going to be fetched using a query in the URL, and i'm not sure that is what you want if you're already using a lot of jQuery to keep everything on the same page.

https://laracasts.com/lessons/crazy-simple-pagination

But assuming you're already paginating the results with whatever jQuery you've already written for the json data...

You could also use a query scope to get the data you need to for the amount of time to scope to create a simple api to use with ajax. I think that's probably what you're looking for.

So here's what I would do assuming you're already doing some pagination manually with your javascript.

1. Create a few query scopes to filter the data for different lengths of time
2. Create simple routes to fetch results from URI using the query scopes
3. Get the json data from the route preforming an ajax requests to the URIs created

More information on Query Scopes: http://laravel.com/docs/5.1/eloquent#query-scopes