根据参数导入Python模块

Question

I'm trying to dynamically import a python-based SQL query module from a sub-folder, and that folder is obtained by using the argparse module.

My project structure :

main_file.py
Data_Projects/
   ITA_data/
      __init__.py
      sqlfile.py
   UK_data/
      __init__.py
      sqlfile.py

Within main_file.py is the argparse module with the argument 'dir' containing the location of the specified directory ie

parser.add_argument('--dir', default='Data_Projects/ITA_data/', type=str,
                        help="set the data directory")

My understanding thus far is that modules should be imported at the top and to import just one sql query I would use: from Data_Project.ITA_data import sqlfile

I know I can't set the import statement after the args have been defined, so how can I keep the format correct with the imports at the top, and yet retrospectively update this with the arguments that get defined afterwards?

Many thanks.

UPDATE

Thanks to the below answer. I've now tried to assign :

sqlfile = __import__(in_arg.dir + 'sqlfile.py')

However I'm getting the following error:

*** ModuleNotFoundError: No module named 'Data_Projects/ITA_data/sqlfile'

I've tried using things like

os.path.join(Path(__file__).resolve().parents[0], in_arg.dir + 'sqlfile')

If it helps, when I try just :

__import__('Data_Projects') - works fine
__import__('Data_Projects/ITA_data') - doesn't work - ModuleNotFound

And as a check to verify I'm not crazy:

os.path.exists('Data_Projects/ITA_Data/sqlfile.py') >>> True
os.path.exists(in_arg.dir + 'sqlfile.py') >>> True

Answer 1

You can use __import__(filename: str) function instead of import statement. It does the same:

# option 1
import foo as bar
# option 2
bar = __import__('foo')

If you need to import from aside, you need to add your directory to module search paths . There are several ways to achieve that, depending on your version of Python. You can find them all in great post:

How to import a module given the full path?

Answer 2

I don't see anything wrong with

import argparse
parser = ...
parser.add_argument('data', choices=['UK', 'ITA'])
args = parser.parse_args()
if args.dir == 'UK':
    import UK_data as data
elif args.dir == 'ITA':
    import ITA_data as data
else ...

You could refine this with functions and __name__ etc. But a conditional import is ok, just so long as it occurs before the data module is used.

Answer 3

The issue was resolved by using :

import sys
sys.path.insert(0, os.getcwd() + "/" + in_arg.dir)

This sets the PYTHONPATH variable to include the directory I want (which changes depending on the argument) to use to search for the file.

From there using grapes help it was a case of doing:

sqlfile = __import__('sqlfile')

And from there I could use the variable to perform the relevant sql query.