How to implement server side rendering datatable, Using node and mongo db?

Question

So i have one user collection(mongo DB) which consists millions of user.

I m using nodejs as backend, angular js as frontend and datatable for displaying those users.

But datatable Load all users in one api call which load more then 1 million user.

This makes my API response two slow.

I want only first 50 users then next 50 then so on....

Server stack = node js + angular js + mongo DB

Thanks

Answer 1

If you are using datatable with huge amount of data you should consider using server side processing functionnality.

Server side processing for datatable is described here : https://datatables.net/manual/server-side

But if you feel lazy to implement this on your server you could use third parties like :

• https://github.com/vinicius0026/datatables-query
• https://github.com/eherve/mongoose-datatable

Hope this helps.

Answer 2

The way to solve you client trying to fetch users from your server(and DB) and then rendering them to a datatable is done using pagination. There a few ways of solving pagination which i have seen, let's assume you are using REST.

One way of doing this is having your API ending with:

/api/users?skip=100&limit=50

Meaning, the client will ask your server for users(using default sorting) and skipping the first 100 results it finds and retrieving the next 50 users.

Another way is to have your API like this(I don't really like this approach):

/api/users?page=5&pageSize=50

Meaning, the client will pass which page and how many results per page it wants to fetch. This will result in a server side calculation becuase you would need to fetch users from 250-300.

You can read on pagination a lot more on the web.

Having said that, your next issue is to fetch the desired users from the database. MongoDB has two functions for using skip and limit , which is why I like the first API better. You can do the query as follows:

users.find().skip(50).limit(50)

You can read more about the limit function here and the skip function here

Answer 3

First Thing you need in to add skip and limit to you mongo query like this Model.find().skip(offset).limit(limit)

then the next thing you have to do is enable server side processing in datatables

If you are using javascript data-table then this fiddle will work for you http://jsfiddle.net/bababalcksheep/ntcwust8/

For angular-datatables

http://l-lin.github.io/angular-datatables/archives/#/serverSideProcessing

One other way if you want to send own parameters

  $scope.dtOptions = DTOptionsBuilder.newOptions()
        .withOption('serverSide', true)
        .withOption('processing', true)

        .withOption('ajax', function (data, callback, settings) {
            // make an ajax request using data.start and data.length

            $http.post(url, {
                draw: draw,
                limit: data.length,
                offset: data.start,
                contains: data.search.value
            }).success(function (res) {
                // map your server's response to the DataTables format and pass it to
                // DataTables' callback
                draw = res.draw;

                callback({
                    recordsTotal: res.meta,
                    recordsFiltered: res.meta,
                    draw: res.draw,
                    data: res.data
                });
            });
        })

you will get the length per page and offset as start variable in data object in the .withOption('ajax' , fun...) section and from there you can pass this in get request as params eg /route?offset=data.start&limit?data.length or using the post request in above example

On hitting next button in table this function will automatically trigger with limit and start and many other datatable related value

Answer 4

@mahesh when loading page create 2 variables lets say skipVar=0 and limit when user clicks on next send *skipVar value key skip

var skipVar =0
on page load skip=skipVar&limit=limit
on next button 
skipVar=skipVar*limit
and send Query String as 
skip=skipVar&limit=limit