Python import cycle when importing modules only

Question

I know this question has been asked countless times here, but I've been stuck with this problem for a long time and have not been able to find a solution online.

I have an import cycle, and here is the stack trace:

Traceback (most recent call last):
  File "openldap_lookup.py", line 2, in <module>
    import pure.directory_service.ds_lookup as dsl
  File "/home/myname/purity/tools/pure/directory_service/ds_lookup.py", line 8, in <module>
    import pure.authorization.auth as auth
  File "/home/myname/purity/tools/pure/authorization/auth.py", line 16, in <module>
    import auth_unix as auth_impl
  File "/home/myname/purity/tools/pure/authorization/auth_unix.py", line 17, in <module>
    import pure.directory_service.ad_lookup as ad_lookup
  File "/home/myname/purity/tools/pure/directory_service/ad_lookup.py", line 1, in <module>
    import pure.authorization.auth as auth
AttributeError: 'module' object has no attribute 'auth'

I import modules only; I avoid the from <module> import <class> and from <module> import <method>

I tried to reproduce the error locally, but python has no complaints. These are the local test files:

openldap_lookup.py

import ds_lookup
def openldap_foo():
    print ds_lookup.ds_foo
print 'openldap_lookup importing ds_lookup'

ds_lookup.py

import auth as au
def ds_foo():
    print au.auth_foo
print 'ds_lookup importing auth'

auth.py

import auth_unix
def auth_foo():
    print auth_unix.auth_unix_foo
print 'auth importing auth_unix'

auth_unix.py

import ad_lookup
def auth_unix_foo():
    print ad_lookup.ad_foo
print 'auth_unix importing ad_lookup'

ad_lookup.py

import auth as au
def ad_foo():
    print au.auth_foo
print 'ad_lookup importing auth'

But python doesn't complain:

myname@myname-mbp:~/cycletest$ python openldap_lookup.py
ad_lookup importing auth
auth_unix importing ad_lookup
auth importing auth_unix
ds_lookup importing auth
openldap_lookup importing ds_lookup
myname@myname-mbp:~/cycletest$

I am not a python expert, but I understand that an import cycle is causing the error. But why doesn't the same error occur with the small test files? When is an import cycle legal in python and when is it not? What can I do to resolve this?

I would greatly appreciate any help from the python experts out there.

---

Since many are bound to ask, why do I have this cycle in the first place?

Well, openldap_lookup and ad_lookup both contain subclasses of a base class in ds_lookup. ds_lookup requires constants from auth. auth requires auth_unix as an implementation, and auth_unix in turn calls the implementations openldap_lookup and ad_lookup.

I would love to move the constants out from auth and remove the cycle, but this code is part of a large git repo where hundreds of files depend on the constants and methods in auth, and I would like to avoid having to refactor all of them if possible.

Answer 1

Actually, you're not just importing modules -- you're importing modules from packages, and your test case doesn't actually reflect this.

I think the problem is that in your first import of pure.authorization.auth, the interpreter is still building the pure.authorization module (it hasn't yet bound auth into pure.authorization, because it hasn't finished importing auth), so the second time it encounters this, it finds the pure.authorization module, but there is no global variable auth in it yet.

As far as breaking the cycle goes, does auth really need to import auth_unix right away, or could that be deferred until you really need an auth implementation?