Trying to merge two collections together in meteor

Question

I have two collections in my application that are parsed from two separate json files. I have inserted data from the two files into separate collections. The collections have corresponding numerical ID's and I want to match them up in a new collection. For example: the postmeta collection has a post_id value and the posts collection has a corresponding ID .

To explain this further here is a simple collections example. One thing to note is that there are over 730 collection posts and although there are matching ID's they are not sorted so when I view them they don't match each other.

The posts collection example:

{
  "_id": "kTeQxenYZcQfPiaYv",
  "ID": "44",
  "post_content": "Today we talked about the letter Hh..."
} 

The postsmeta collection example:

{
  "_id": "otEGQYxvv6MkCABST",
  "post_id": "44",
  "meta_value": "http://www.mrskitson.ca/wp-content/uploads/2010/11/snackTime.jpg"
}

What I would like to do is parse through the collections and take for example posts collection where the ID matches the postsmeta collection. Once I find a match I want to insert the collections content ( post_content & meta_value ) into a new collection.

Here is all my code so far.

lib/collections/posts.js

Postsmeta = new Mongo.Collection('postsmeta');
Posts = new Mongo.Collection('posts');

server/publications.js

Meteor.publish('postsmeta', function() {
  return Postsmeta.find();
});

Meteor.publish('posts', function() {
  return Posts.find();
});

server/main.js

Meteor.startup(() => {

  var postsmeta = JSON.parse(Assets.getText('postsmeta.json'));
  var posts = JSON.parse(Assets.getText('posts.json'));
  var length =  postsmeta.length;

  for(x=0; x < length; x++){

    Posts.insert({
      ID: posts[x].ID,
      post_content: posts[x].post_content
    });

    Postsmeta.insert({
      post_id: postsmeta[x].post_id,
      meta_value: postsmeta[x].meta_value
    });   

  }

});

Answer 1

Let's refactor your code a bit. We'll build the Postsmeta collection first and then jointly create the Posts and PostsCombined collections. Since Postsmeta will already exist we can just search inside it to find matching documents.

Meteor.startup(() => {
  const postsmeta = JSON.parse(Assets.getText('postsmeta.json'));
  postsmeta.forEach(doc => {
    Postsmeta.insert({ post_id: doc.post_id, meta_value: doc.meta_value }); 
  });

  const posts = JSON.parse(Assets.getText('posts.json'));
  posts.forEach(doc => {
   const post = { ID: doc.ID, post_content: doc.post_content}
    Posts.insert(post); // omit if you don't need the uncombined collection
    const metadoc = Postsmeta.findOne({post_id: doc.ID}); // essentially a JOIN
    if (metadoc) post.meta_value = metadoc.meta_value; // guard against no matching meta
    PostsCombined.insert(post);
  });   
});

The following IDs are not present in your postsmeta data :

   ["56", "322", "521", "563", "583", "608", "625", "671", "707", "708",
    "711", "713", "754", "758", "930", "1068", "1126", "1235", "1237", "1238",
    "1239", "1246", "1249", "1256", "1263", "1355", "1375", "1678", "1680", "1763",
    "1956", "2107", "2121", "2148", "2197", "2249"]

Answer 2

Do you want to put the collections together for consultation? because the insertion is correct for two different collections.

Tip one

If it is for query use the " find().map() ", if you are using mongodb, within the function it will return the values ​​of each row of the first collection and soon you can call the other collection and check the id of the collection and return a JSON or Array of what you need. I do not pretend to do it that way, but it's a way of putting the two collections together.

Best solution

The correct way is not thinking as if noSql was a relational database like the other postgres, mysql and etc ... think that it is a dynamic bank, where in the same collection you can have everything you need at that moment, so I think You create a new collection that would be the junction of the two, when save saves the data in this other collection, which would be the query collection, and in that it would weigh less the query and until it would return the data faster, but suppose a 5x more faster than the above example ...

I hope I have helped, any questions or doubts I will be here. Hugs!