Python error: attempted relative import beyond top-level package

Question

I have a file: lib/deep_restore.py that has:

from ..utils.model_loader import load_masking_model, load_reconstruction_model

And in utils/model_loader.py , I have:

import glob
import importlib
import json
import sys
import torch
import os.path as path


def load_masking_model(wandb_id, device, make_4d=False):
    wandb_dir = list(glob.iglob(
        path.join('wandb', '*' + wandb_id), recursive=False))[0]
    model_path = path.join(wandb_dir, 'best-model.pt')

    (head, tail) = path.split(model_path)
    ...

From evaluate.py , I have:

from lib.deep_restore import DeepRestore

but I get the error: ValueError: attempted relative import beyond top-level package

What am I doing wrong?

Answer 1

The problem here is that Python does not see your "utils" and "lib" packages as part of another, larger, package. Instead, as far as Python knows, there is one package named "utils" and another named "lib" - and relative imports - which is, prefixing a module name with "." so that Python searches from the current folder inside a package, will only work inside the same package.

Changing this to an absolute import is easy - just remove the "."s:

from utils.model_loader import load_masking_model, load_reconstruction_model

And your code will "work". If it is a fine design? No!

The correct thing there is to have a parent package that encompasses everything you are doing, that is above the "lib" and "utils" directories, so that both directories are viewed as subpackages.

Let's say you call it "myproject" - Then you have "myproject.lib" and "myproject.utils" - and you can import either by fully specifying these names, or, doing ..lib.deep_restore from inside the "utils" subpackage.

For that to work all you need is: add an empty __init__.py file to the "myproject" directory, and either work from the super-directory for that (ie the directory containing the my_project directory), or properly install my_project as an "editable package" - for that you need a minimal "setup.py" file, in that parent directory, and run pip install -e. .