Recursively add property to every node in a tree-like structure and return modified tree

Question

I have the following data structure for a tree of comments in a thread. This structure is contained inside a single object.

comment {
    id: 1,
    text: 'foo',
    children: [
        comment {
            id: 2,
            text: 'foo-child',
            children: []
        },
        comment {
            id: 3,
            text: 'foo-child-2',
            children: []
        }
    ]
},
comment {
    id: 4,
    text: 'bar',
    children: []
}

This is provided by a back-end API, there's no problem in that. What I want to do is recursively explore this tree and for each node (either root or child node) I want to perform an API call and get some extra data for every single node , slap in some extra properties, and return the entire tree with the new keys added to each node.

function expandVoteData(comments) {
    return new Promise((resolve, reject) => {
        let isAuth = Auth.isUserAuthenticated();
        // 'this' is the vote collection
        async.each(comments, (root, callback) => {
            // First get the vote data
            async.parallel({
                votedata: function(callback) {
                    axios.get('/api/comment/'+root.id+'/votes').then(votedata => {
                        callback(null, votedata.data);
                    });
                },
                uservote: function(callback) {
                    if(!isAuth) {
                        callback(null, undefined);
                    } else {
                        axios.get('/api/votes/comment/'+root.id+'/'+Auth.getToken(), { headers: Auth.getApiAuthHeader() }).then(uservote => {
                            callback(null, uservote.data); // Continue
                        });
                    }
                }
            }, function(error, data) {
                if(error) {
                    console.log('Error! ', error);
                } else {
                    // We got the uservote and the votedata for this root comment, now expand the object
                    root.canVote = isAuth;
                    root.totalVotes = data.votedata.total;
                    root.instance = 'comment';

                    if(data.uservote !== undefined) {
                        root.userVote = data.uservote;
                    }

                    if(root.children && root.children.length > 0) {
                        // Call this function again on this set of children
                        // How to "wrap up" this result into the current tree?
                        expandVoteData(root.children);
                    }
                    callback(); // Mark this iteration as complete
                }
            });
        }, () => {
            // Done iterating
            console.log(comments);
            resolve();
        });
    })
}

What it does is: accept a 'comments' parameter (which is the entire tree object), create a promise, iterate through each leaf node, and perform the respective API calls in asynchronous requests. If the leaf node has any children, repeat the function with each child node.

This theoretically would work perfectly in a synchronous world, but what I do need to do is to get the new tree after every node has been processed for further processing, as a single object, just like it was as an input. In fact, I get multiple console prints for each individual node in the tree, evidencing that the code works as it's written... I don't want individual prints though, I want to wrap up the entire set of results in a single object. Ideally, the function should be called like this:

expandVoteData(comments).then(expanded => {
    // yay!
});

Any tips on how to perform this? Thank you beforehand.

Answer 1

It gets more easy if you seperate the code into multiple functions and use the cool async / await syntax. Furst define an async function that updates one node without caring for the children:

async function updateNode(node) {
 const [votedata, uservote] = await Promise.all([
   axios.get('/api/comment/'+root.id+'/votes'),
    axios.get('/api/votes/comment/'+root.id+'/'+Auth.getToken(), { headers: Auth.getApiAuthHeader() })
 ]);

 node.totalVotes = votedata.total;
 node.instance = 'comment';

 if(uservote)
   node.userVote = uservote;
}

To update all nodes recursively its then as easy as:

async function updateNodeRecursively(node) {
  await updateNode(node);
  await Promise.all(node.children.map(updateNodeRecursively));
}

Answer 2

requests in serial

Below, addExtra accepts an input comment and asynchronously adds additional fields to the comment, and all of the comment's children recursively.

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...comment
   , children: await Promise.all (children.map (addExtra))
   , extra: await axios.get (...)
  })

To show this works, we first introduce a fake database. We can query a comment's extra fields by comment id

 const DB = { 1: { a: "one" } , 2: { a: "two", b: "dos" } , 4: [ "anything" ] } const fetchExtra = async (id) => DB [id] fetchExtra (2) .then (console.log, console.error) // { "a": "two" // , "b": "dos" // } 

Now instead of axios.get we use our fetchExtra . We can see addExtra works as intended, given the first comment as input

const comments =
  [ /* your data */ ]

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...comment
  , children: await Promise.all (children.map (addExtra))
  , extra: await fetchExtra (comment.id)
  })

addExtra (comments [0])
  .then (console.log, console.error)

// { id: 1
// , text: "foo"
// , children:
//   [ {id: 2
//     , text: "foo-child"
//     , children:[]
//     , extra: { a: "two", b: "dos" } // <-- added field
//     }
//   , { id: 3
//     , text: "foo-child-2"
//     , children:[]
//     }
//   ]
// , extra: { a: "one" } // <-- added field
// }

Since you have an array of comments, we can use map to addExtra to each

Promise.all (comments .map (addExtra))
  .then (console.log, console.error)

// [ { id: 1
//   , text: "foo"
//   , children:
//     [ {id: 2
//       , text: "foo-child"
//       , children:[]
//       , extra: { a: "two", b: "dos" } // <--
//       }
//     , { id: 3
//       , text: "foo-child-2"
//       , children:[]
//       }
//     ]
//   , extra: { a: "one" } // <--
//   }
// , { id: 4
//   , text: "bar"
//   , children:[]
//   , extra: [ 'anything' ] // <--
//   }
// ]

Using Promise.all is a burden on the user though, so it'd be nice to have something like addExtraAll

const addExtraAll = async (comments) =>
  Promise.all (comments .map (addExtra))

addExtraAll (comments)
  .then (console.log, console.error)

// same output as above

refactor and enlighten

Did you notice a code duplication? Hello, mutual recursion ...

const addExtraAll = async (comments) =>
  Promise.all (comments .map (addExtra))

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...comment
  , children: await Promise.all (children .map (addExtra))
  , children: await addExtraAll (children)
  , extra: await fetchExtra (comment.id)
  })

addExtra (singleComment) // => Promise

addExtraAll (manyComments) // => Promise

Verify the results in your own browser below

 const addExtraAll = async (comments) => Promise.all (comments .map (addExtra)) const addExtra = async ({ children = [], ...comment }) => ({ ...comment , children: await addExtraAll (children) , extra: await fetchExtra (comment.id) }) const DB = { 1: { a: "one" } , 2: { a: "two", b: "dos" } , 4: [ "anything" ] } const fetchExtra = async (id) => DB [id] const comments = [ { id: 1 , text: "foo" , children: [ {id: 2 , text: "foo-child" , children:[] } , { id: 3 , text: "foo-child-2" , children:[] } ] } , { id: 4 , text: "bar" , children:[] } ] addExtra (comments [0]) .then (console.log, console.error) // { id: 1 // , text: "foo" // , children: // [ {id: 2 // , text: "foo-child" // , children:[] // , extra: { a: "two", b: "dos" } // <-- added field // } // , { id: 3 // , text: "foo-child-2" // , children:[] // } // ] // , extra: { a: "one" } // <-- added field // } addExtraAll (comments) .then (console.log, console.error) // [ { id: 1 // , text: "foo" // , children: // [ {id: 2 // , text: "foo-child" // , children:[] // , extra: { a: "two", b: "dos" } // <-- // } // , { id: 3 // , text: "foo-child-2" // , children:[] // } // ] // , extra: { a: "one" } // <-- // } // , { id: 4 // , text: "bar" // , children:[] // , extra: [ 'anything' ] // <-- // } // ] 

add multiple fields

Above, addExtra is simple and only adds one extra field to your comment. We could add any number of fields

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...comment
  , children: await addExtraAll (children)
  , extra: await axios.get (...)
  , other: await axios.get (...)
  , more: await axios.get (...)
  })

merging results

Instead of adding fields to the comment , it's also possible to merge the fetched data in. However, you should take some precautions here ...

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...await fetchExtra (comment.id)
   , ...comment
   , children: await addExtraAll (children)
  })

addExtra (comments [0])
  .then (console.log, console.error)

// { 
// , a: 1 // <-- extra fields are merged in with the comment
// , id: 1
// , text: "foo"
// , children: [ ... ]
// }

Note the order of the calls above. Because we call ...await first, it's impossible for fetched data to overwrite fields in your comment. For example, if fetchExtra(1) returned { a: 1, id: null } , we would still end up with comment { id: 1 ... } . If you wish to have the possibility for added fields to overwrite existing fields in your comment, then you can change the ordering

And lastly, you can make multiple merges if you wanted

const addExtra = async ({ children = [], ...comment }) =>
  ({ ...await fetchExtra (comment.id)
   , ...await fetchMore (comment.id)
   , ...await fetchOther (comment.id)
   , ...comment
   , children: await addExtraAll (children)
  })

requests in parallel

One disadvantage of the approach above is that requests for the extra fields are done in serial order.

It would be nice if we could specify aa function that takes our comment as input and returns an object of the fields we wish to add. This time we skip the await keywords so that our function can parallelize the subrequests for us automatically

addFieldsAll
  ( c => ({ extra: fetchExtra (c.id), other: fetchOther (c.id) })
  , comments
  )
  .then (console.log, console.error)

// [ { id: 1
//   , children: [ ... ] // <-- fields added to children recursively
//   , extra:  ... // <-- added extra field
//   , other: ... // <-- added other field
//   }
// , ...
// ]

Here's one way to implement addFieldsAll . Also note, because of the ordering of the arguments to Object.assign it is possible for the descriptor to specify fields that would overwrite fields on the input comment – eg c => ({ id: regenerateId (c.id), ... }) . As described above, this behavior can be changed by reordering the arguments as desired

const addFieldsAll = async (desc = () => {} , comments = []) =>
  Promise.all (comments .map (c => addFields (desc, c)))

const addFields = async (desc = () => {}, { children = [], ...comment}) =>
  Object.assign
    ( comment
    , { children: await addFieldsAll (desc, children) }
    , ... await Promise.all
        ( Object .entries (desc (comment))
            .map (([ field, p ]) =>
              p.then (res => ({ [field]: res })))
        )
    )