Updating D3.js (NVD3.js) Data by Merge

Question

I'm working on some realtime graphs built with NVD3.js. I currently refresh each chart with the following:

function reDraw(c) {
    d3.json(c.qs, function(data) {
        d3.select(c.svg)
        .datum(data)
        .transition().duration(500)
        .call(c.chart);
    });
}

c looks like:

function Chart(svg, qs, chart) {
    this.qs = qs;
    this.svg = svg;
    this.ylabel;
    this.chart;
}

This works fairly well, but with each refresh I am fetching the whole time series again. It would be more efficient to only grab recent elements and update each graph. There are examples of doing this by appending elements (This answer NVD3 line chart with realtime data and this tutorial for example) , but this isn't ideal for me since some recent elements might be updated that are not the most recent element.

So what I'm looking to do is grab say the most recent minute (by setting query string ( .qs ) to only get the most recent minute, then take that result and do the following:

• Overwrite any existing elements that have the same x value for each series with the most recent data
• Append and elements when there are new x values from the update in each series
• Expire elements past a certain age
• Update the NVD3.js script with the new data. Maybe still use datum with the new merged object?

Can anyone suggest an elegant way to perform the above Merge operation? The existing data object looks like the following:

> d3.select(perf.svg).data()[0]
[
Object
key: "TrAvg"
values: Array[181]
__proto__: Object
, 
Object
key: "RedisDurationMsAvg"
values: Array[181]
__proto__: Object
, 
Object
key: "SqlDurationMsAvg"
values: Array[181]
__proto__: Object
]
> d3.select(perf.svg).data()[0][0]['values'][0]
Object {x: 1373979220000, y: 22, series: 0}
> d3.select(perf.svg).data()[0][1]['values'][0]
Object {x: 1373979650000, y: 2, series: 1}

The object returned would look something like the following (Except will only be maybe 6 elements or so for each object):

> d3.json(perf.qs, function(data) { foo = data })
Object {header: function, mimeType: function, response: function, get: function, post: function…}
> foo
[
Object
, 
Object
, 
Object
]
> foo[0]
Object {key: "TrAvg", values: Array[181]}
> foo[0]['values'][0]
Object {x: 1373980220000, y: 49}

In this newer object the series value is missing - maybe that needs to get added or perhaps D3 can do it?

Answer 1

For the time being I used linq.js to perform this operation, and then use .datum() to bind a new dataset each time. The solution isn't very elegant but it does seem to function:

function reDraw2(c, partial) {
    if (partial) {
        qs = c.qs.replace(/minago=[0-9]+/, "minago=1")
    } else {
        qs = c.qs
    }
    d3.json(qs, function(new_data) {
        chart = d3.select(c.svg)
        if (c.ctype == "ts" && partial) {
            thirtyminago = new Date().getTime() - (60*30*1000);
            old_data = chart.data()[0]
            var union = new Array();
            for (var i = 0; i < old_data.length; i++) {
                var existing = Enumerable.From(old_data[i]['values']);
                var update = Enumerable.From(new_data[i]['values']);
                oldfoo = existing
                newfoo = update
                var temp = {}
                temp['key'] = old_data[i]['key']
                temp['values'] = update.Union(existing, "$.x").Where("$.x >" + thirtyminago).OrderBy("$.x").Select().ToArray();
                union[i] = temp

            }
            chart.datum(union)
        } else {
            chart.datum(new_data)
        }
        chart.call(c.chart)
    });