如何使用Crossfilter和DC.js处理丢失的数据？

Question

Let the following dataset:

var dataset = [
    {
        "user": "u1",
        "question1": "answer1",
        "question2": "answer2",
        ...
    },
    ...
];

Assume that this dataset is not complete: some users might have answered one question, but not another. Thus this dataset has some blanks where a value "questionX" does not appear.

Assume that we create for each question, the associated pie chart like this:

var questions = ["question1", "question2", ...];
var cf = crossfilter(dataset);

for (var i = 0; i < questions.length; i++) {

    var questionDim = cf.dimension(function(d) { return d[questions[i]]});
    var questionGrp = questionDim.group().reduceCount();

    plotPieChart("#dc-" + questions[i], questionDim, questionGrp); // helper function to plot standard DC pie chart based on a dimension and group.
}

It seems that the group does not correctly handle the missing values, and still categorizes the missing points into the first possible category.

1. Is it a bug?

2. If not, one possible solution is to preprocess the dataset and add the missing questions with a dummy answer (for instance 'NA'). However, by doing so, the 'NA' answer will appear as a pie. Thus how can you remove this dummy pie from the displayed results?

3. is there a better way to deal with such issue?

Thanks!

Answer 1

No, it's not a bug. Crossfilter dimensions must be naturally ordered or weird stuff happens.

You should define your dimensions to handle undefined values. You can do this like so:

var questionDim = cf.dimension(function(d) { return d[questions[i]] ? d[questions[i]] : "No answer"});

Then when you define your dc.js chart, you can filter out data you don't want, if you don't want the non-answers to display in your pie chart (though you probably should display them so that viewers understand the proportion of people who answered said question):

dc.pieChart('#pie-chart')
  .group(questionGrp)
  .data(function(group) {
     return group.all()
                 .filter(function(d) { return d.key !== "No answer"; }); 
  })