Cython modules can be imported along with python package

Question

I made a python package with Cython components. When I import the package, I am able to import the Cython component directly. Is this normal?

For example,

import pkg1
import util  # Works for some reason
import foo  # Does not work
from pkg1 import foo  # Works as expected

works. How can I make it so you need the following to use util :

from pkg1 import util

Project Tree:

pkg1/
    setup.py
    pkg1/
        __init__.py
        util.pyx
        foo.py
        setup.py

I think there might be something wrong with my pkg1/setup.py

import setuptools
from distutils.core import setup
from distutils.extension import Extension

extensions = [
    Extension('pgk1.util', ['pkg1/util.c'])
]

setuptools.setup(
    # ...
    ext_modules=extensions,
    # ...
)

pkg1/pkg1/setup.py

from distutils.core import setup
from setuptools.extension import Extension
from Cython.Build import cythonize

extensions = [
    Extension(name="util",
              sources=["util.pyx"])
]

setup(
    ext_modules=cythonize(extensions, language_level="3")
)

Also, what is difference between Extension from setuptools and distutils ?

This is my first Python package, so I am not exactly sure what everything does.

Answer 1

I think you've been led into a mess by a typo which was meaning things weren't ending up where they should be:

Extension('pgk1.util', ...

should be pkg1.util - note that I've swapped two letters!

You only want one setup.py filo. It should be at the outermost level. In principle it can be responsible for building multiple packages (yours isn't). It takes care of all both the .pyx -> .c step and the .c -> .so step.

The following works:

import setuptools
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

extensions = [
    Extension('pkg1.util', ['pkg1/util.pyx'])
]

setuptools.setup(
    packages = setuptools.find_packages(),
    ext_modules=cythonize(extensions, language_level=3),
    # ...
)

The changes are: I've fixed the pkg1/pgk1 type, I've added cythonize to the extensions, I've changed the source file to .pyx, and I've added packages = setuptools.find_packages() . This last change isn't needed if you're doing build_ext --inplace but does ensure that the .py files end up included if you use setup.py to install the package.

Delete your pkg1/pkg1/setup.py. It is not needed. Before you run this new one it's worth deleting the build directory and deleting the generated .c file(s) to ensure that everything gets rebuilt.