MongoDB上的C＃查询未返回正确结果

Question

Im currently running into an issue when querying MongoDb using c#. The problem is that I am not returned the correct results or the correct number of results. I do not know the exact number of results but it should be less than 100; instead, I am receiving around 350k-500k results (many of which are null). The other problem is that the program takes upwards of 10 minutes to finish processing.

You can see the problematic portion of code in the following:

public List<BsonDocument> find_All_Documents_With_pIDs()
    {            
        List<string> databases = new List<string>();
        List<BsonDocument> pDocs = new List<BsonDocument>(); 
        databases.AddRange(mongo_Server.GetDatabaseNames());

        //iterate through each db in mongo
        foreach (string dbName in databases)
        {
            List<string> collections = new List<string>();
            var database = mongo_Server.GetDatabase(dbName);
            collections.AddRange(database.GetCollectionNames());

            //iterate through each collection
            foreach (string colName in collections)
            {
                var collection = database.GetCollection(colName);

                //Iterate through each document
                foreach (var document in collection.FindAllAs<BsonDocument>())
                {   
                    //Get all documents that have a pID in either the main document or its sub document                     
                    IMongoQuery query = Query.Exists(document.GetElement("_id").ToString().Remove(0,4) + ".pID");
                    IMongoQuery subQuery = Query.Exists(document.GetElement("_id").ToString() + ".SubDocument.pID");
                    pDocs.AddRange(collection.Find(query));
                    pDocs.AddRange(collection.Find(subQuery));
                }
            }
        }

        //Theres a collection used earlier in the program to backup the documents before processing. Not removing the documents from the list found in this location will result in duplicates. 
        return remove_Backup_Documents_From_List(pIDs);
    }

Any help is appreciated!

EDIT:

The following is a screen capture of the data received. Not all the data is null like the following but a very large amount is:

Answer 1

Your script is first bringing all your documents from the database

collection.FindAllAs<BsonDocument>()

and then assembling a query for each one. That's probably the reason the query is so slow.

As an alternative you could do the following:

    foreach (string colName in collections)
{
    var collection = database.GetCollection(colName);

    //Query for all documents that have pID
    IMongoQuery query = Query.And([Query.Exists("pID"), // The field exists
      Query.NE("pID", BsonNull.Value), //It is not "null"
      Query.NE("pID", BsonString.Null)]); //It is not empty i.e. = ""

    //Query for all documents that have Subdocument.pID
    IMongoQuery subQuery = Query.And([Query.Exists("SubDocument.pID"), // The field exists
      Query.NE("SubDocument.pID", BsonNull.Value), //It is not "null"
      Query.NE("SubDocument.pID", BsonString.Null)]); //It is not empty i.e. = ""


    IMongoQuery totalQuery = Query.Or([query, subQuery]);


    List<BsonDocument> results = collection.Find(totalQuery);
    if (results.Count > 0) {
      pDocs.AddRange(results); //Only add to pDocs if query returned at least one result
    }
}

That way you assemble a query that returns only the documents that have either pID or Subdocument.pID fields set.