MongoDB Java驱动程序性能降低

Question

I'm working on a project which implement as front : VUE.JS and back : Java servlet (tomcat). My servlet perform request on a mongoDB database with thousands products. ( I am new with mongodb query ).

The problem for me is that my servlet take many time to respond with a data array.

I need advice to speed up my query and speed up my servlet process to send data to the front.

1. First I would like to know if in mongodb, is it efficient to use query like this (this is an example of query with many lookup, because I have to fetch data from many collections) :

    db.getCollection('collection_name').aggregate([ { "$match":{ "_id": ObjectId("xxxxxxxxxxxxxxxxx") } }, { "$lookup":{ "from":"collection_name", "localField":"id", "foreignField":"id", "as":"trans" } }, { "$unwind":"$trans" }, { "$lookup":{ "from":"collection_name", "localField":"trans.id", "foreignField":"trans_id", "as":"orders" } }, { "$unwind":"$orders" }, { "$match":{ "$and":[ { "orders.status":"status_state" } ] } }, { "$unwind":"$orders.products" }, { "$lookup":{ "from":"collection_name", "localField":"orders.products.id", "foreignField":"id", "as":"detailsProducts" } }, { "$unwind":"$detailsProducts" }, { "$unwind":"$detailsProducts.products" }, { "$project":{ "_id":1, "orders":1, "detailsProducts": 1 } } ]) 

2. In second as I say, my java servlet take many time to perform datas treatment, after definding my aggregate i perform a loop operation to push data into a json array like this :

    // Define json array JSONArray array = new JSONArray(); // Get collection AggregateIterable<Document> myCollection = MongoDB.getCollection("coll_name").aggregate(...); // Going through all documents (thousands of document) for (Document orderDocument : orderCollection) { // Get all products and they references arrays.put(new JSONObject(orderDocument.toJson())); } // AFTER THE END OF LOOP, MY SERVLET SEND THE ARRAY TO THE FRONT 

Note : I work with mongodb java driver 3.5 (i try both : core driver and async drive)

I would like to get some advise to speed up my treatment and optimize my request.

Answer 1

Building a JSONArray with thousands of elements is not going to scale very well. You're better off streaming your JSON directly back to the client. So, assuming this returns in reasonable time:

// Get collection
AggregateIterable<Document> myCollection =
    MongoDB.getCollection("coll_name").aggregate(...);

then something like the following should improve things considerably:

import java.io.IOException;
import javax.servlet.http.HttpServletResponse;
import org.bson.Document;
import org.bson.codecs.DocumentCodec;
import org.bson.codecs.Encoder;
import org.bson.codecs.EncoderContext;
import org.bson.json.JsonWriter;
...
public void writeAsJsonTo(AggregateIterable<Document> orderCollection,
        HttpServletResponse httpServletResponse) throws IOException {
    final Encoder<Document> encoder = new DocumentCodec();
    final JsonWriter jsonWriter =
        new JsonWriter(httpServletResponse.getWriter());
    final EncoderContext encoderContext = EncoderContext.builder()
        .isEncodingCollectibleDocument(true).build();

    jsonWriter.writeStartDocument();
    jsonWriter.writeStartArray("orders");
    // Going through all documents (thousands of document)
    for (Document orderDocument : orderCollection) {
        encoder.encode(jsonWriter, orderDocument, encoderContext);
    }
    jsonWriter.writeEndArray();
    jsonWriter.writeEndDocument();
}