python import circular dependency (and perhaps function declaration)

Question

Hello I did got into circular dependency what is not refactori-zable other than doubling code.

I have something like this (only much more complex):

myParser.py:

import sys
import main                      #comment this to make it runnable
def parseEvnt():
    sys.stdout.write("evnt:")
    main.parseCmd(1)             #comment this to make it runnable

tbl.py:

import myParser
tblx = {
       1:("cmd",),
       2:("evnt",myParser.parseEvnt),
       }

main.py:

import tbl
def parseCmd(d):
    print(tbl.tblx[d][0])
data=[1,2]
for d in data:
    if(d<2):
        parseCmd(d)
    else:
        fce = tbl.tblx[d][1]
        fce()    

Obvious error I'm getting is:

File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 1, in <module>
    import tbl
  File "D:\Data\vbe\workspace\marsPython\testCircular2\tbl.py", line 1, in <module>
    import myParser
  File "D:\Data\vbe\workspace\marsPython\testCircular2\myParser.py", line 2, in <module>
    import main
  File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 7, in <module>
    parseCmd(d)
  File "D:\Data\vbe\workspace\marsPython\testCircular2\main.py", line 3, in parseCmd
    print(tbl.tblx[d][0])
AttributeError: module 'tbl' has no attribute 'tblx'

In CI think I would just tell by declaration in tbl.py hey there is function parseEvnt() . I would not need to include myParser and there would be no circular include.

In python I do not know how to do it.

I read few threads and there is always some wise guy recommending refactorizing. But in this case parseCmd() needs to see tblx which needs to see parseEvnt() (unless function declaration) and parseEvnt() need to call parseCmd() (cos evnt contains triggering cmd and I do not want to double the decoding cmd code).

Is there way how to make it working in python?

Answer 1

Circular imports should be avoided. Refactoring is required, any workaround that still requires a circular import is not a good solution.

That being said, the refactoring doesn't have to be extensive. There are at least a couple of fairly simple solutions.

Solution 1: move shared functions to a shared module

Since you want to use parseCmd from more than one place, move it to a separate file. That way both main.py and myParser.py can import the function.

Solution 2: have main pass parseCmd to parseEvnt .

First, make parseEvnt accept an argument to tell it which function to run:

# myParser.py
import sys
def parseEvnt(parseCmd):
    sys.stdout.write("evnt:")
    parseCmd(1) 

Next, when you call myParser.parseEvnt , pass in a reference to main.parseCmd :

# main.py:

...
else:
    fce = tbl.tblx[d][1]
    fce(parseCmd) 

There are other ways to accomplish the same thing. For example, you could add a "configure" method in myParser , and then have main.py call the configure method and pass in a reference to its parseCmd . The configure method can then store this reference in a global variable.

Answer 2

Another choice is to import main in the function that uses it:

main.py

import sys

def parseEvnt():
    import main
    sys.stdout.write("evnt:")
    main.parseCmd(1)     

Answer 3

If you're insistent on not refactoring (which is the real solution to this - not being a wise guy), you could move your problematic import into your function in myParser.py

import sys

def parseEvnt():
    import main  ## import moved into function
    sys.stdout.write("evnt:")
    main.parseCmd(1)

Again, see if you can redesign your code so such interdependencies are avoided.

The above solution is sort of a hack and won't solve future problems you might run into due to this dependency.

Answer 4

You can frequently get away with circular dependencies as long as the modules don't try to use each other's data until all importing is complete - as a practical matter, that means referencing with namespace ( from module import something is forbidden) and only using the other modules inside functions and methods (no mystuff = module.mystuff in the global space). That's because when importing starts, python puts the module name in sys.modules and won't try to import that module again.

You ran into trouble because when you run main.py , python adds __main__ to sys.modules . When the code finally came around to import main , there was no "main" in the module list and so main.py was imported again... and its top level code tried to run.

Lets rearrange your test case and throw in a few print statements to tell when import happens.

myParser.py

print('  + importing myParser')
import sys

print('import parsecmd')
import parsecmd
def parseEvnt():
    sys.stdout.write("evnt:")
    parsecmd.parseCmd(1)

tbl.py

print('  + importing tbl')
print('import myParser')
import myParser
tblx = {
       1:("cmd",),
       2:("evnt",myParser.parseEvnt),
       }

Parsecmd.py (new)

print('  + importing parsecmd')
print('import tbl')
import tbl
def parseCmd(d):
    print(tbl.tblx[d][0])

main.py

print('running main.py')

print('import parsecmd')
import parsecmd

if __name__ == "__main__":
    data=[1,2]
    for d in data:
        if(d<2):
            parsecmd.parseCmd(d)
        else:
            fce = parsecmd.tbl.tblx[d][1]
            fce()    

When I run it I get

running main.py
import parsecmd
  + importing parsecmd
import tbl
  + importing tbl
import myParser
  + importing myParser
import parsecmd                     <-- didn't reimport parsecmd
cmd
evnt:cmd