我的PyV8上下文泄漏内存

Question

Whatever I try with PyV8, I always get massive memory leaks. Even when executing evals with empty strings, it still seems to leak memory somehow. In the example posted below, 10 Million executions of eval with an empty string generate 320MB of memory, of which only 20MB gets collected when explicitly calling the garbage collector afterwards. Whether the context is reachable or not reachable anymore doesn't seem to make a difference for me. I've tested my python itself with a similar test and it doesn't leak memory. Am I doing something wrong?

Versions

PyV8 Revision 557, built on same machine using PyV8's setup.py

V8 Revision 19632, built on same machine using PyV8's setup.py

OS: Ubuntu 12.04

Test Code

import unittest, gc, os, logging as python_logging
from PyV8 import JSEngine, JSContext

_proc_status = '/proc/%d/status' % os.getpid()
_scale = {'kB': 1024.0, 'mB': 1024.0*1024.0,
          'KB': 1024.0, 'MB': 1024.0*1024.0}

def log_memory_usage(intro_text=""):
    python_logging.info(
        (
            '%s process %d now uses %.1f MB resident'
                %(intro_text, os.getpid(), resident()/(1024*1024))
        ).strip()
    )

def _VmB(VmKey):
    '''Private.
    '''
    global _proc_status, _scale
     # get pseudo file  /proc/<pid>/status
    try:
        t = open(_proc_status)
        v = t.read()
        t.close()
    except:
        return 0.0  # non-Linux?
     # get VmKey line e.g. 'VmRSS:  9999  kB\n ...'
    i = v.index(VmKey)
    v = v[i:].split(None, 3)  # whitespace
    if len(v) < 3:
        return 0.0  # invalid format?
     # convert Vm value to bytes
    return float(v[1]) * _scale[v[2]]

def resident(since=0.0):
    '''Return resident memory usage in bytes.
    '''
    return _VmB('VmRSS:') - since   

class TestMemoryWithJSContext(unittest.TestCase):
  def test_python_memory_management(self):
    def inner():
        with JSContext() as ctx:
            log_memory_usage("before empty evals")
            for index1 in range(1000):
                for index2 in range(10000):
                    ctx.eval("")
            log_memory_usage("after empty evals")
            JSEngine.collect()
    log_memory_usage("before JSContext memory tests")
    inner()
    JSEngine.collect()
    gc.collect()
    JSEngine.collect()
    gc.collect()
    log_memory_usage("after JSContext memory tests and gc")
    print "Py gc.garbage:", gc.garbage

class CardEngineTestSuite(unittest.TestSuite):
    def __init__(self):
        super(CardEngineTestSuite, self).__init__()
        self.addTests(unittest.TestLoader().loadTestsFromTestCase(TestPython))
        self.addTests(unittest.TestLoader().loadTestsFromTestCase(TestMemoryWithJSContext))

if __name__ == '__main__':
    python_logging.basicConfig(level=python_logging.INFO, format='%(asctime)s %(message)s')
    unittest.TextTestRunner().run(CardEngineTestSuite())
    unittest.TextTestRunner().run(CardEngineTestSuite())
    python_logging.info(str(gc.garbage))

Output

.2014-03-28 21:41:34,198 before JSContext memory tests process 110 now uses 14.1 MB resident
2014-03-28 21:41:34,199 before empty evals process 110 now uses 14.4 MB resident
2014-03-28 21:41:55,513 after empty evals process 110 now uses 348.8 MB resident
2014-03-28 21:41:56,926 after JSContext memory tests and gc process 110 now uses 322.3 MB resident
Py gc.garbage: []
.
----------------------------------------------------------------------
Ran 2 tests in 26.838s

OK
.2014-03-28 21:42:01,103 before JSContext memory tests process 110 now uses 322.5 MB resident
2014-03-28 21:42:01,104 before empty evals process 110 now uses 322.5 MB resident
2014-03-28 21:42:25,714 after empty evals process 110 now uses 636.5 MB resident
2014-03-28 21:42:28,459 after JSContext memory tests and gc process 110 now uses 629.3 MB resident
Py gc.garbage: []
.
----------------------------------------------------------------------
Ran 2 tests in 31.532s

OK

Answer 1

The answer described in the following PyV8 ticket solved it for me - albeit the garbage collection now takes up a lot of CPU time. To get around this I moved my PyV8 code into a celery worker and let it do the garbage collection in the background. Ping me if you want to see the code for this, right now I'm really out of time to prepare sample code.

https://code.google.com/p/pyv8/issues/detail?id=229&sort=-id