Insert documents in collection from array of objects in Meteor

Question

I have an array dataRows consisting of multiple objects, eg

{
  'The Name': 'John',
  'My Age': 44,
  'My Address': 'Some street',
  [...]
}

I want to insert these objects into a collection.

Since I could just use MyCollection.insert( object ) , I guess I can just insert the objects directly into a collection; however, the problem is that the keys in my objects are differently named than the field names in my collection and I don't want to insert all of the fields in my collection.

I could just do this:

dataRows.forEach( function ( row ) {
  MyCollection.insert( {
    name: row['The Name'],
    age: row['My Age'],
    address: row['My Address'],
  } );
} )

but I think it would be better if I could manipulate the array dataRows directly.

Besides, I don't want to skip fields with empty values.

Edit

I have an array

[
  {
    'The Name': 'John',
    'My Age': 44,
    'My Address': 'Some street',
    [...]
  },
  {
    'The Name': 'Michael',
    'My Age': 57,
    'My Address': '',
    [...]
  },
  {
    'The Name': 'Anne',
    'My Age': 31,
    'My Address': 'Some other street',
    [...]
  },
  [...]
]

I want to manipulate the array, so the keys can be renamed (eg The Name should be name , My Age should be age , etc.), fields with no value should be removed (eg the second object in the array has a key My Address with an empty value. The key should be removed from this object), and I want which keys should be kept and the rest of the keys should be remove (eg all objects in the array should have fields name , age , address and all other fields should be removed from each object.

I guess I should use array.map() , array.filter() , etc.

I hope it clarifies my question.

Answer 1

You can still manipulate the array using the forEach() method and within the loop take advantage of using a write commands Bulk API that allows for the execution of bulk insert operations which are simply abstractions on top of the server to make it easy to build bulk operations, thus streamlining your inserts . These bulk operations come mainly in two flavours:

• Ordered bulk operations . These operations execute all the operation in order and error out on the first write error.
• Unordered bulk operations . These operations execute all the operations in parallel and aggregates up all the errors. Unordered bulk operations do not guarantee order of execution.

You can get raw access to the collection and database objects in the npm MongoDB driver through rawCollection and rawDatabase methods on Mongo.Collection . For example, manipulate the array using forEach() , construct the object to be inserted within the loop, send the insert operations in batches for improved write perfomances as follows:

MyCollection = new Meteor.Collection("mycollection");

if (Meteor.isServer) {
    Meteor.startup(function () {
        Meteor.methods({
            insertData: function() {
                var bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp(),
                    counter = 0,
                    dataRows = [...]; // the raw array

                dataRows.forEach(function(row) {
                    var data = {
                        name: row['The Name'],
                        age: row['My Age'],
                        address: row['My Address'],
                    };
                    bulkOp.insert(data);

                    counter++;
                    if (counter % 1000 == 0) {
                        // Execute per 1000 operations and re-initialize every 1000 update statements
                        bulkOp.execute(function(e, rresult) {
                            // do something with result
                        });
                        bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp();
                    }
                }); 

                // Clean up queues
                if (counter % 1000 != 0){
                    bulkOp.execute(function(e, result) {
                        // do something with result
                    });
                }
            }
        }); 
    });
}