Python smtplib模块泄漏内存

Question

I have written a daemon of sorts to monitor files in a directory and send out an email whenever there is a change in any of the files. I used bb-freeze to compile it to a Windows .exe. I noticed after letting it run for a couple of days that it was taking more and more space in memory.

I used Heapy to monitor the memory usage in the .py file (not the compiled .exe) and found out that for each call to the function, the number of objects was increasing by 3 with a corresponding memory usage increase of 484 Bytes. It uses the smtplib module and I cannot understand where the leak is happening.

from guppy import hpy
import time
import gc

import os
import smtplib
import mimetypes
from email.MIMEMultipart import MIMEMultipart
from email.MIMEBase import MIMEBase
from email.MIMEText import MIMEText
from email.MIMEAudio import MIMEAudio
from email.MIMEImage import MIMEImage
from email.Encoders import encode_base64

def sendMail(subject, text, to='blah@gmail.com', username='more.blah@gmail.com', password='blah', smtpServer='smtp.gmail.com', smtpPort=587):
   gmailUser = username
   gmailPass = password
   recipient = to

   msg = MIMEMultipart()
   msg['From'] = gmailUser
   msg['To'] = recipient
   msg['Subject'] = subject
   msg.attach(MIMEText(text))

   mailServer = smtplib.SMTP(smtpServer, smtpPort)
   mailServer.ehlo()
   mailServer.starttls()
   mailServer.ehlo()
   mailServer.login(gmailUser, gmailPass)
   mailServer.sendmail(gmailUser, recipient, msg.as_string())
   mailServer.quit()

   print('Sent email to "%s"' % recipient)

if __name__=='__main__':
   while True:
      sendMail("Function", "Blah!")
      gc.collect()
      print hpy().heap()
      time.sleep(10)

I just saw this code somewhere on the internet and copied it. It works but leaks memory. Can someone help me find out where the memory leak is happening?? :(

EDIT: Seems like using msg.as_string() is what is causing the memory leak. Using a plain text such as msg="Blah" in place of msg.as_string() fixes the issue. But that does not allow me to add a subject line.

Answer 1

I would bet that you have a reference cycle.

Edit: I altered your code slightly to:

import time
import gc
import smtplib
import mimetypes
from email.MIMEText import MIMEText

def sendMail(subject, text):
   gmailUser = 'myemail@gmail.com'
   gmailPass = 'mypassword'
   recipient = gmailUser

   msg = MIMEMultipart()
   msg['From'] = gmailUser
   msg['To'] = recipient
   msg['Subject'] = subject
   msg.attach(MIMEText(text))

   mailServer = smtplib.SMTP('smtp.gmail.com', 587)
   mailServer.starttls()
   mailServer.login(gmailUser, gmailPass)
   mailServer.sendmail(gmailUser, recipient, msg.as_string())
   mailServer.quit()

   print('Sent email to "%s"' % recipient)

if __name__=='__main__':
   gc.set_debug(gc.DEBUG_LEAK)
   for item in range(1000):
      sendMail("Function", "Blah!")
      gc.collect()
      time.sleep(2)

Guppy does not work for the version of python and c++ compiler I have, so I couldn't test that output (maybe it doesn't work for you either?). What I can tell you is that I watched the garbage collection output and some memory statistics in process explorer and found no significant variance or issues with a leak in that code. Key changes: removed calls to SMTP.ehlo() (unnecessary), removed default function parameters (I suspected these might stay around as objects that could be referred to as long as long as the function was in scope, which might somehow keep the SMTP objects around). So you might want to try one then the other and see which fixes your problem.

Check out this post for assistance and some tools.

Answer 2

Do you have access to Valgrind ? That is a great tool for finding memory leaks. You could also try using the Python Debugger .

EDIT:

Sorry, just noticed you said you were on Windows, which valgrind is not available for :(.

Anyway, you might want to look into whatever library Python for Windows uses for crypto. On my Mac, the script you provided shows a memory leak around the mailServer.starttls() call, and the culprit looks to be the python wrapper around libssl/libcrypto. Sorry if this is no help at all (being non-windows...).

Answer 3

1) You could try to do the same code with simple string text as message.
2) You could remove the calls to ehlo and see if the memory leak is fixed by this. (they are unnecessary)

import smtplib


def main() :
    fromaddr = 'ph111@gmail.com'
    toaddrs  = 'ph222@gmail.com'
    msg = 'my simple message'
    username = 'ph111'
    password = 'mypassword'
    server = smtplib.SMTP('smtp.gmail.com:587')
    server.starttls()
    server.login(username,password)
    server.sendmail(fromaddr, toaddrs, msg)
    server.quit()

if __name__ == '__main__':
    main()