编写表格单元格实时python

Question

I would like to loop trough a database, find the appropriate values and insert them in the appropriate cell in a separate file. It maybe a csv, or any other human-readable format. In pseudo-code:

for item in huge_db:
   for list_of_objects_to_match:
      if itemmatch():
         if there_arent_three_matches_yet_in_list():
            matches++
            result=performoperationonitem()
            write_in_file(result, row=object_to_match_id, col=matches)
         if matches is 3:
            remove_this_object_from_object_to_match_list()

can you think of any way other than going every time through all the outputfile line by line? I don't even know what to search for... even better, there are better ways to find three matching objects in a db and have the results in real-time? (the operation will take a while, but I'd like to see the results popping out RT)

Answer 1

Assuming itemmatch() is a reasonably simple function, this will do what I think you want better than your pseudocode:

for match_obj in list_of_objects_to_match:
  db_objects = query_db_for_matches(match_obj)
  if len(db_objects) >= 3:
      result=performoperationonitem()
      write_in_file(result, row=match_obj.id, col=matches)
  else:
      write_blank_line(row=match_obj.id)  # if you want

Then the trick becomes writing the query_db_for_matches() function. Without detail, I'll assume you're looking for objects that match in one particular field, call it type . In pymongo such a query would look like:

def query_db_for_matches(match_obj):
    return pymongo_collection.find({"type":match_obj.type})

To get this to run efficiently, make sure your database has an index on the field(s) you're querying on by first calling:

pymongo_collection.ensure_index({"type":1})

The first time you call ensure_index it could take a long time for a huge collection. But each time after that it will be fast -- fast enough that you could even put it into query_db_for_matches before your find and it would be fine.