How to sort null values with pivot.js

Question

I use the Pivot.js library to create pivot views on my data. The library uses jquery.dataTables to display the final pivot table. Unfortunately, the Pivot.js has problems with null-values when sorting.

The error can easily be seen on the github-site when one uses the following settings:

Filter Fields: Englewood

Row Label Fields: last_name

Column Label Fields: invoice_yyyy

Summary Fields: billed_amount

Then, sort by the column "2010". The result will look like the following.

It seems that the column gets sorted like a string because some rows contain null values (logically) and are filled with ' '. Sorting works for complete columns that contain a value in each row. The column 2010 is not complete and $90.10 !< $850.00

How can this error be avoided without changing code within the Pivot.js library? Thank you.

Answer 1

You'll want to provide a new set of sort functions to datatables and then add an aTypes function to automatically assign the corresponding data type (eg formatted number) - http://datatables.net/release-datatables/examples/plug-ins/sorting_plugin.html

The tricky bit here is that jquery_pivot.js (as shipped with pivot.js) uses non-breaking space point (   ) for null values ( https://github.com/rjackson/pivot.js/blob/master/jquery_pivot.js#L347 ). I would advise that jquery_pivot.js is just a suggested UI implementation and you should be free to modify it for your app; however, if you don't want to modify it, you're left with two potentially undesirable behaviors:

1. dataTables' automatic type detection trips over the   and, as you've observed, treats the column as alphanumeric for sorting.
2. Even when you get it to detect numeric sorting, the nulls generally get treated as either the smallest or largest values of the set. In my application I prefer the NULLS LAST approach (ie, they always appear at the end of the table regardless of ascending/descending sort order).

My solution is far from the most elegant, but here is my code:

Sort functions:

jQuery.extend( jQuery.fn.dataTableExt.oSort, {
    "formatted-num-pre": function ( a ) {
        a = (a===' ') ? NaN : a.replace( /[^\d\-\.]/g, "" );
        return parseFloat( a );
    },
    "formatted-num-asc": function ( a, b ) {
        if(isNaN(a)) return 1;
        if(isNaN(b)) return -1;
        return a-b;
    },
    "formatted-num-desc": function ( a, b ) {
        if(isNaN(a)) return 1;
        if(isNaN(b)) return -1;
        return b-a;
    }});

Type detection:

jQuery.fn.dataTableExt.aTypes.unshift(
  function (data) { 
    return (data.charAt(0)==='$'||data===' ')?'formatted-num':null; 
  });

Hope it helps!