How to import json into MongoDB using Mongoose

Question

I have a few problems with this, which is whats making it tricky, so...

I am using Mongoose and MongoLab, I can store data and retrieve it just fine, but I want a system that allows me to do a base seed of the database.

I have the schemas created for the collections, but none are ran because there is no data, so I can't seem to run a normal mongoimport as the collection isn't yet created.

I want to add something to my node server so that if the collection doesn't exist or is empty, it loads a schema for a collection and then inserts the json for the seed data.

so I have this...

var Club = require('./schemas/Club');

I normally use Club.find, or Club.save etc, thats working fine.

I want to just run a save against an array of Objects to the Club collection which it needs to create.

I did look into mongoose-fixture but its not been updated in years, and there is probably a way of doing this without needing so much extra code, as I have the schema defined, and the array of json ready.

This is the success event I listed for when I guess I want to do the check and import.

mongoose.connection.on('open', function () {
  console.log('mongoose.connection.opened');
});

Also, to consider, if I wanted to create two collections, and when it generates the ObjectId() for the items in the first collection, I can imagine wanting to use those in the second collection as a ref.

Just assume Club objects only have one string property for now.

// contents of data/club.json
[
  { 'name' : 'Barcelona' },
  { 'name' : 'Real Madrid' },
  { 'name' : 'Valencia' }
]

Any help much appreciated

Answer 1

If I understand it well, all you need is to upload a JSON document to your MongoDB collection from Mongoose. Given that your model is named Club , you can access raw driver methods through Club.collection . And use insertMany to achieve what you want.

Here is a stand alone example (the interesting stuff is at the end):

> var mongoose = require('mongoose')
> var assert = require('assert')

> mongoose.connect('mongodb://localhost/test');

> var Schema = mongoose.Schema
> var clubSchema = new Schema({
...   name: String,
... })

> var Club = mongoose.model('Club', clubSchema)

// Now, the interesting part:
> data = [
...   { 'name' : 'Barcelona' },
...   { 'name' : 'Real Madrid' },
...   { 'name' : 'Valencia' }
... ]
> Club.collection.insertMany(data, function(err,r) {
...       assert.equal(null, err);
...       assert.equal(3, r.insertedCount);
... 
...       db.close();
... })

And check from the Mongo Shell:

> db.clubs.find()
{ "_id" : ObjectId("5574b464b680174d79e37601"), "name" : "Barcelona" }
{ "_id" : ObjectId("5574b464b680174d79e37602"), "name" : "Real Madrid" }
{ "_id" : ObjectId("5574b464b680174d79e37603"), "name" : "Valencia" }

Answer 2

Sometimes shell scripts may help:

for filename in *; do mongoimport -d mydb -c $filename --jsonArray done

See also: MongoDB Bulk import using mongoimport from Windows folder