Python extremely slow to import

Question

I've got a webapp built on python/tornado. Sometimes (but not always!) it's extremely slow to start. Here's the top 10 slowest calls:

       358926 function calls (350348 primitive calls) in 7.933 seconds

 Ordered by: cumulative time

 ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      1    0.000    0.000    7.941    7.941 /Users/ysimonson/Desktop/dailymuse/themuse/venv/src/oz/oz/__init__.py:133(initialize)
  54/12    0.238    0.004    7.940    0.662 {__import__}
      1    0.164    0.164    1.853    1.853 /Users/ysimonson/Desktop/dailymuse/themuse/venv/src/oz/oz/plugins/sqlalchemy/__init__.py:1(<module>)
      1    0.001    0.001    1.842    1.842 ./themuse/common/__init__.py:1(<module>)
      1    0.002    0.002    1.811    1.811 ./themuse/common/actions/__init__.py:1(<module>)
      1    0.112    0.112    1.696    1.696 /Users/ysimonson/Desktop/dailymuse/themuse/venv/src/oz/oz/plugins/aws_cdn/__init__.py:1(<module>)
      1    0.001    0.001    1.294    1.294 ./themuse/common/actions/analytics.py:1(<module>)
      1    0.005    0.005    0.956    0.956 /Users/ysimonson/Desktop/dailymuse/themuse/venv/lib/python2.7/site-packages/sqlalchemy/__init__.py:9(<module>)
      1    0.000    0.000    0.951    0.951 ./themuse/api_v1/__init__.py:1(<module>)
      1    0.084    0.084    0.951    0.951 ./themuse/api_v1/api_v1_routes.py:1(<module>)

The process took 8 seconds to start. All of these slow calls are imports, but most of them are pretty simple - eg there's 3 instances in which it's an import from __future__ :

from __future__ import absolute_import, division, print_function, with_statement, unicode_literals

What's going on here? Some additional info:

• The problem coincided with the upgrade to OS X Yosemite. I'm not sure if that's a coincidence, as we're constantly changing code.
• I do have circular imports, although that's never been a problem in the past. The profiler also doesn't seem to be indicating that's the issue, as some simple non-circular imports are going very slowly.
• When this happened, the process was reloaded via tornado's autoreload . I'd guess that's unrelated since it's not showing up in the profiler, but maybe it's somehow related to autoreload 's use of execv ?

Answer 1

Try invalidating your module lookup cache by doing importlib.invalidate_caches() . You could also consider re-installing the module to see if that makes a difference.