What is the argument for Python to seemingly frown on importing from different directories?

Question

This might be a more broad question, and more related to understanding Python's nature and probably good programming practices in general.

I have a file, called util.py . It has a lot of different small functions I've collected over the past few months that are useful when doing various machine learning tasks.

My thinking is this: I'd like to continue adding important functions to this script as I go. As so, I will want to use import util.py often, now and in the future, in many unrelated projects.

But Python seems to feel like I should only be able to access the code in this file if it lives in my current directly, even if the functions in this file are useful for scripts in different directories. I sense some reason behind the way that works that I don't fully grasp; to me, it seems like I'll be forced to make unnecessary copies often.

If I should have to create a new copy of util.py every time I'm working from within a new directory, on a different project, it won't be long until I have many different version / iterations of this file, scattered all over my hard drive, in various states. I don't desire this degree of modularity in my programming -- for the sake of simplicity, repeatability, and clarity, I want only one file in only one location, accessible to many projects.

The question in a nutshell: What is the argument for Python to seemingly frown on importing from different directories?

Answer 1

If your util.py file contains functions you're using in a lot of different projects, then it's actually a library, and you should package it as such so you can install it in any Python environment with a single line ( python setup.py install ), and update it if required (Python's packaging ecosystem has several features to track and update library versions).

An added benefit is that right now, if you're doing what the other answers suggested, you have to remember to manually have put util.py in your PYTHONPATH (the "dirty" way). If you try to run one of your programs and you haven't done that, you'll get a cryptic ImportError that doesn't explain much: is it a missing dependency? A typo in the program?

Now think about what happens if someone other than you tries to run the program(s) and gets those error messages.

If you have a library, on the other hand, trying to set up your program will either complain in clear, understandable language that the library is missing or out of date, or (if you've taken the appropriate steps) automatically download and install it so things are ready to roll.

On a related topic, having a file/module/namespace called "util" is a sign of bad design. What are these utilities for? It's the programming equivalent of a "miscellaneous" folder: eventually, everything will end up in it and you'll have no way to know what it contains other than opening it and reading it all.

Answer 2

Another way, is adding the directory/you/want/to/import/from to the path from within the scripts that need it. You should have a file __init__.py in the same folder where utils.py lives, to tell python to treat the folder as a package. The file __init__.py may be empty, or not, you can define other things in there.

Example:

/home/marcos/python/proj1/
__init__.py
utils.py
/home/marcos/school_projects/final_assignment/
my_scrpyt.py

And then inside my_script.py

import sys
sys.path.append('/home/marcos/python/')
from proj1 import utils

MAX_HEIGHT = utils.SOME_CONSTANT
a_value = utils.some_function()

Answer 3

First, define an environment variable. If you are using bash , for example, then put the following in the appropriate startup file:

export PYTHONPATH=/path/to/my/python/utilities

Now, put your util.py and any of your other common modules or packages in that directory. Now you can import util from anywhere and python will find it.