Need proper directory structure for python packages and guidelines on good API design

Question

I am in the midst of releasing a python package, and am confused about every aspect of packaging.

To start, my directory structure is as follows:

SamplePackage/
 - setup.py
 - README.rst
 - LICENSE.rst
 - sampledir/
    -__init__.py
    -sample.py
    -utils.py

currently __init__ and setup are unpopulated. sample.py is the file that any user of the package would want to import. It contains the api in the form of different functions: foo1 , foo2 .

utils.py contains helper fictions for smaple.py . The latter contains a statement import utils

Any scripts placed under the sampledir directory can easily import sample and use the fictions as sample.foo1() . Stepping out of this directory, I can call import sampledir , but not import sample , which is expected. So I need to do from sampledir import sample . This results in an error on the import utils line in sample.py

ImportError: No module named 'utils'

In some places I have seen import .utils for files in the same directory. But when I try that, it results in a syntax error.

Why can I not import sample.py from outside sampledir?

Also, what directory structure would allow users who have installed the package to simply be able to call import sample followed by sample.foo1() , and not have to do from sampledir import sample or import sampeldir.sample . For example, in using the HTTP library requests , one simply has to import it and call requests.get(url) . requests.requests.get('url') is not required, like it is in urllib . What is the correct directory naming and arrangement to achieve that, if I want the package to be named sample ?

Answer 1

First of all, I thing you've misunderstood what is difference between package and project. You directory structure should be something like this

arbitrary_name_of_project/
 - setup.py
 - README.rst
 - LICENSE.rst
 - package_name/
    -__init__.py
    -sample.py
    -utils.py

Python package is "marked" by a presence of file __init__.py . Python search on his PYTHONPATH and seeks for all directories with file __init__.py (simply put, it's actually much more complicated). Those are packages that you can import during run-time.

In our case, only package_name directory will be installed somewhere to PYTHONPATH . Therefore, once installed (via setup.py and disutils or setuptools ), you can type

import package_name

Now, you can call arbitrary class or function or whatever from any module in this package like

package_name.sample.func1()

If you want to avoid typing package_name.sample in you call, you can place this line in __init__.py

from .sample import func1

The function func1 will be then put in your global name space during "import time". This happens when import package_name is executed in your program. This mystery can be resolved by the fact that __init__.py file are executed every time package is imported.

It is sometimes necessary to spit the structure even more. You can have "nested" packages (ie packages in package that you need to import). There are also namespace packages, where you can "spit" one package among more directories. The world is diverse. :)

Disclaimer

I wrote that without testing the code I provided. There might be small typos but I hope the story is more or less correct. Please, correct me if I'm wrong.