Twisted memory leak when connect attempt failed

Question

i was being tortured twisted memory leaks.

Here is my code:

# Python:  Python 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)] on win32
# Twisted: Twisted-12.2.0.win32-py2.6
# OS:      Windows 7 (64bit)
from twisted.internet.protocol import Protocol, ClientFactory
from twisted.internet import reactor


class Echo(Protocol):
    def __del__(self):
        print 'Echo.__del__'

    def connectionMade(self):
        print 'Echo.connectionMade'
        self.transport.write('Hello world!')


class EchoClientFactory(ClientFactory):
    def __del__(self):
        print 'EchoClientFactory.__del__'

    def startedConnecting(self, connector):
        print 'Started connecting ...'

    def buildProtocol(self, addr):
        print 'connected. %r' % addr
        return Echo()

    def clientConnectionFailed(self, connector, reason):
        print 'Connection failed.'


def connect(ip, port):
    factory = EchoClientFactory()
    reactor.connectTCP(ip, port, factory)


##Start
import gc
gc.enable()
gc.set_debug(gc.DEBUG_LEAK)

# Trying to connect to a port that does not exist
connect('127.0.0.1', 7777)

reactor.callLater(5, reactor.stop)
reactor.run()

# Show garbages
print "gc.collect()"
gc.collect()
print 'gc.garbage:', len(gc.garbage)

for i, item in enumerate(gc.garbage):
    print '%d) %r' % (i + 1, item)

after run, gc shows memory leak is happen:

after run, gc shows memory leak is happen:

after run, gc shows memory leak is happen:

Started connecting ...
Connection failed.
EchoClientFactory.__del__
gc.collect()
gc: collectable <Client 02CEA7B0>
gc: collectable <dict 02CEFC00>
gc: collectable <tuple 02B13FD0>
gc: collectable <list 02CD2558>
gc: collectable <instancemethod 02818E40>
gc: collectable <instancemethod 02CF04B8>
gc: collectable <tuple 02B205F8>
gc.garbage: 7
1) <<class 'twisted.internet.tcp.Client'> to ('127.0.0.1', 7777) at 2cea7b0>
2) {'_tempDataBuffer': [], 'protocol': None, '_tempDataLen': 0, 'realAddress': ('127.0.0.1', 7777), 'doRead': <bound method Client.doConnect of <<class 'twisted.internet.tcp.Client'> to ('127.0.0.1', 7777) at 2cea7b0>>, 'doWrite': <bound method Client.doConnect of <<class 'twisted.internet.tcp.Client'> to ('127.0.0.1', 7777) at 2cea7b0>>, 'reactor': <twisted.internet.selectreactor.SelectReactor object at 0x02699D50>, 'addr': ('127.0.0.1', 7777)}
3) ('127.0.0.1', 7777)
4) []
5) <bound method Client.doConnect of <<class 'twisted.internet.tcp.Client'> to ('127.0.0.1', 7777) at 2cea7b0>>
6) <bound method Client.doConnect of <<class 'twisted.internet.tcp.Client'> to ('127.0.0.1', 7777) at 2cea7b0>>
7) ('127.0.0.1', 7777)

Only happened when the connect failed.

Any ideas on why?

Answer 1

Adding __del__ to a program most likely adds object leaks to it. The presence of __del__ on an object in a cycle prevents the entire cycle from being collected.

Try debugging your actual program without using __del__ anywhere. Or if your actual program uses __del__ , try getting rid of it.

Answer 2

Maybe I found a solution.

In the twisted source code, open this file: Python26\\site-packages\\twisted\\internet\\base.py

class BaseConnector(styles.Ephemeral):
    .
    .
    .

    def connectionFailed(self, reason):
        self.cancelTimeout()
        if self.transport.doWrite == self.transport.doConnect:    # 1
            del self.transport.doRead                             # 2
            del self.transport.doWrite                            # 3
        self.transport = None
        self.state = "disconnected"
        self.factory.clientConnectionFailed(self, reason)
        if self.state == "disconnected":
            # factory hasn't called our connect() method
            self.factory.doStop()
            self.factoryStarted = 0
    .
    .
    .

Lines marked with 1,2,3 are my newly added.