How can I save entire MongoDB collection to json/bson file using C#?

Question

I have process, which first, generates lots of data which is save into mongoDB collection, then data is analyzed, and last - I want to save the whole collection to file on disk, and erase the collection. I know I could do it easily with MongoDump.exe, but I was wondering is there any way to do it directly from c#? - I mean not running console precess with - but using some functionality that is inside MOngo C# driver.

And, if it can be done - how would I do the reverse operation in c# ? - namely: loading .bson file into collection?

Answer 1

You can use C# BinaryFormatter to serialize object graph to disk. Also you can deserialize back to object graph.

Serialize: https://msdn.microsoft.com/en-us/library/c5sbs8z9%28v=VS.110%29.aspx

Deserialize: https://msdn.microsoft.com/en-us/library/b85344hz%28v=vs.110%29.aspx

However that is not mongodb or C# driver feature.

After serializing you can use the driver to drop the collection. And after deserializing you can use the driver to insert objects into a new collection.

Based on your rules, you may want to do some locking on that collection at the time you are doing the export process before you drop it.

Answer 2

Here's two methods that you can use to accomplish this:

public static async Task WriteCollectionToFile(IMongoDatabase database, string collectionName, string fileName)
{
    var collection = database.GetCollection<RawBsonDocument>(collectionName);

    // Make sure the file is empty before we start writing to it
    File.WriteAllText(fileName, string.Empty);

    using (var cursor = await collection.FindAsync(new BsonDocument()))
    {
        while (await cursor.MoveNextAsync())
        {
            var batch = cursor.Current;
            foreach (var document in batch)
            {
                File.AppendAllLines(fileName, new[] { document.ToString() });
            }
        }
    }
}

public static async Task LoadCollectionFromFile(IMongoDatabase database, string collectionName, string fileName)
{
    using (FileStream fs = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    using (BufferedStream bs = new BufferedStream(fs))
    using (StreamReader sr = new StreamReader(bs))
    {
        var collection = database.GetCollection<BsonDocument>(collectionName);

        string line;
        while ((line = sr.ReadLine()) != null)
        {
            await collection.InsertOneAsync(BsonDocument.Parse(line));
        }
    }
}

And here's an example of how you would use them:

// Obviously you'll need to change all these values to your environment
var connectionString = "mongodb://localhost:27017";
var database = new MongoClient(connectionString).GetDatabase("database");
var fileName = @"C:\mongo_output.txt";
var collectionName = "collection name";

// This will save all of the documents in the file you specified
WriteCollectionToFile(database, collectionName, fileName).Wait();

// This will drop all of the documents in the collection
Task.Factory.StartNew(() => database.GetCollection(collectionName).DeleteManyAsync(new BsonDocument())).Wait();

// This will restore all the documents from the file you specified
LoadCollectionFromFile(database, collectionName, fileName).Wait();

Note that this code was written using version 2.0 of the MongoDB C# driver, which you can obtain via Nuget . Also, the file reading code in the LoadCollectionFromFile method was obtained from this answer .