使用生成器函数python

Question

I was practicing using generator functions in python, so I defined a functions as follows:

def MySQL_product():
   #Establish connection to database
   try:
       connection = msql.connect(host = 'localhost', user = 'max', passwd = 'password', db = 'schools')
   except:
       pass

   #Iterate through each product and insert them in database
   with connection:
       cursor = connection.cursor()
       cursor.execute("SELECT name, age, gender, school
                    WHERE GroupId = 'student' AND Exchange = 'foreign'")
       for product in cursor.fetchall():
           yield product

def main():
    for column in range (0, number_of_schools):
        for product in MySQL_product():
            print product

However, when I run this code, all I see as an output is generator object at ... I am trying to print the contents that are found in the database. Also, none of the print statements in MySQL_product() are executed. The point of the generator is that instead of return a list of all of the rows in the database, it should return it one by one. Then I wanted to access/print those items. What can I do to fix this code?

Answer 1

If you're doing a cursor.fetchall(), that means you're copying every result available from the SQL server into python's memory. So in this context - a generator affords you nothing.

If you instead used cursor.fetchmany() or cursor.fetchone(), the only benefit you would see is memory consumption on Python's side since you would only be processing 'some' or 'one' result at a time. On the SQL side, the server would still have that result set cached (burning up valuable resources on the SQL server)

But, ultimately - if you did process results in chunks - since you'd be in a loop of:

while there_are_more_results:
    results = cursor.fetchmany(10)
    for result in results:
        do_something(result)

Having a generator would provide you no real advantage since you would have to block while you get more results from mysql.

However, to answer your question

What you want to do to make the code you have work is:

def main():
    for column in range (0, number_of_schools):
        for student in MySQL_product():
            print student

generators are really useful when you're doing things asynchronously - basically if a generator isn't ready yet - you just skip over it and let other things work.

Answer 2

Yes, that's how a generator behaves - it always returns an iterator which you can then use with eg the for statement. You probably want to change your main() function to something like this:

def main():
    for column in range (0, number_of_schools):
        for student in MySQL_product():
            print student

It's also possible to get the next result yielded by an iterator with the next() function, but in general you should prefer to iterate directly with for item in iterator: # ... (because your code will be easier to read, and less fragile in the event that you do something like switch back from a generator function to a regular one that returns a list).

Answer 3

My first answer was incorrect and others have already given the best solution, so I'll state the alternative and a potential use case. If you need to iterate over a generator two items at a time, or in any other custom way, the next method can be useful.

def gen():
    for i in range(20):
        yield i

for i in gen():
    print "Center", str(i).center(10)

a = gen()    
for i in range(10):
    left = str(a.next()).ljust(10)
    right = str(a.next()).rjust(10)