Node.js loops and JSON building

Question

Respected ppl ....

This is my node.js code ... https://gist.github.com/SkyKOG/99d47dbe5a2cec97426b

Im trying to parse the data of our exam results ...example ... http://www.vtualerts.com/results/get_res.php?usn=1MV09IS002&sem=7

Im getting the results ...and i am traversing back for previous seems too ... All works but traversing back happens in random ... prolly something wrong with the loops ...

            json.results = [];

            var output = '';

            var k = response.query.results.body.div.div[0].table[1].tr.length;

            for (var j = 1; j < k; j++) {

                for (var i = 0; i <= 5; i++) {
                    var result_obj = {};
                    result_obj.subjects = [];

                    for (key in response.query.results.body.div.div[0].table[1].tr[j].td[i]) {
                        if (typeof response.query.results.body.div.div[0].table[1].tr[j].td[i].em === "undefined") {
                            continue;
                        }

                        var subject_obj = {};

                        output += "Subject : " + response.query.results.body.div.div[0].table[1].tr[j].td[i].em + " " + "\n";

                        var subtext = response.query.results.body.div.div[0].table[1].tr[j].td[i].em + " " + "\n";
                        subject_obj.subjectname = subtext.replace(/[(].*[)]/, "").trim();

                        result_obj.subjects.push(subject_obj);
                        console.log(subject_obj);
                        break;

                    }

                    console.log(result_obj.subjects);

I presume there is something like async concepts which need to implemented correctly to make the reordering of sems in correct order ...

And to get the JSON in this format ... https://gist.github.com/SkyKOG/3845d6a94cea3b744296 I dont think im pushing the created objects at the right scope ...

Kindly help in this regard .... thank you ...

Answer 1

(I'll answer the ordering part. Suggest making the JSON issue a separate question to fit in with the Q&A format.)

When you make the HTTP request in your code (see the line below) you're bringing a varying delay into the order that responses are executed

new YQL.exec(queryname, function (response) {

You need to track the order of the requests yourself, or use a library to do it for you.

Code it yourself

In order to get around that you need something that keeps track of the original order of the requests. Because of the way closures work you can't just increment a simple counter because it'll be changed in the global scope as your loop progresses. The idiomatic way to solve this is by passing the counter into an immediately executed function (as a value type)

eg

var responseData = [];
for ( var i = 0; i < 100; i++ ){
    (function(){
        ...
        // http call goes in here somewhere
            responseData[i] = data_from_this_response
        ...
    })(i)
}

Use a library

Check out the async.parallel() call in Caolan's excellent library. You pass it an array of functions and it'll return to your callback with an array of the results.

https://github.com/caolan/async/#parallel

You'll need to create a loop that populates the array with curried versions of your function containing the appropriated variables.