[英]"ModuleNotFoundError" after installing a python package
I am very new to python package development.我是 python package 开发的新手。 I developed a package and published it at TestPyPI .
我开发了一个 package 并将其发布在 TestPyPI 上。 I install this package trough
pip
with no errors.我安装这个 package 槽
pip
没有错误。 However, python is giving me a "ModuleNotFoundError" when I try to import it, and I have no idea why.但是,当我尝试导入时,python 给出了“ModuleNotFoundError”,我不知道为什么。 Can someone help me?
有人能帮我吗?
First, I install the package with:首先,我安装 package:
pip install -i https://test.pypi.org/simple/ spark-map==0.2.76
Then, I open a new terminal, start the python interpreter, and try to import this package, but python gives me a ModuleNotFoundError
:然后,我打开一个新终端,启动 python 解释器,并尝试导入这个 package,但是 python 给我一个
ModuleNotFoundError
:
>>> import spark_map
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'spark_map'
When I cd
to the root folder of the package, and open the python interpreter, and run import spark_map
, it works fine with no errors;当我
cd
到 package 的根文件夹,打开 python 解释器,并运行import spark_map
时,它工作正常,没有错误;
That pip
did not installed the package succesfully;那个
pip
没有安装package成功; However I checked this.但是我检查了这个。 I got no error messages when I install the package, and when I run
pip list
after the pip install
command, I see spark_map
on the list of installed packages.当我安装 package 时,我没有收到任何错误消息,当我在
pip install
命令后运行pip list
时,我在已安装包的列表中看到spark_map
。
> pip list
... many packages
spark-map 0.2.76
... more packages
spark_map
was installed can be out of the module search path of Python;spark_map
的文件夹可以在模块搜索路径Python之外; I checked this as well.pip
is installing the package on a folder called Python310\lib\site-packages
, and this folder is included inside the sys.path
variable: pip
正在将 package 安装在名为Python310\lib\site-packages
的文件夹中,该文件夹包含在sys.path
变量中:>>> import sys
>>> for path in sys.path:
... print(path)
C:\Users\pedro\AppData\Local\Programs\Python\Python310\python310.zip
C:\Users\pedro\AppData\Local\Programs\Python\Python310\DLLs
C:\Users\pedro\AppData\Local\Programs\Python\Python310\lib
C:\Users\pedro\AppData\Local\Programs\Python\Python310
C:\Users\pedro\AppData\Local\Programs\Python\Python310\lib\site-packages
C:\Users\pedro\AppData\Local\Programs\Python\Python310\lib\site-packages\win32
C:\Users\pedro\AppData\Local\Programs\Python\Python310\lib\site-packages\win32\lib
C:\Users\pedro\AppData\Local\Programs\Python\Python310\lib\site-packages\Pythonwin
I am on Windows 10, Python 3.10.9, trying to install and import the spark_map
package, version 0.2.76.( https://test.pypi.org/project/spark-map/ ).我在 Windows 10、Python 3.10.9 上,尝试安装和导入
spark_map
package,版本 0.2.76.( https://test.pypi.org/project/spark-map/ )。
The package source code is hosted at GitHub , and the folder structure of this package is essentially this: package源代码托管在GitHub ,这个package的文件夹结构本质上是这样的:
root
│
├───spark_map
│ ├───__init__.py
│ ├───functions.py
│ └───mapping.py
│
├───tests
│ ├───functions
│ └───mapping
│
├───.gitignore
├───LICENSE
├───pyproject.toml
├───README.md
└───README.rst
The pyproject.toml
file of the package: package的
pyproject.toml
文件:
[build-system]
requires = ["setuptools>=61.0", "toml"]
build-backend = "setuptools.build_meta"
[project]
name = "spark_map"
version = "0.2.76"
authors = [
{ name="Pedro Faria", email="pedropark99@gmail.com" }
]
description = "Pyspark implementation of `map()` function for spark DataFrames"
readme = "README.md"
requires-python = ">=3.7"
license = { file = "LICENSE.txt" }
classifiers = [
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
]
dependencies = [
"pyspark",
"setuptools",
"toml"
]
[project.urls]
Homepage = "https://pedropark99.github.io/spark_map/"
Repo = "https://github.com/pedropark99/spark_map"
Issues = "https://github.com/pedropark99/spark_map/issues"
[tool.pytest.ini_options]
pythonpath = [
"."
]
[tool.setuptools]
py-modules = []
As @Dorian Turba suggested, I moved the source code into a src
folder.正如@Dorian Turba 建议的那样,我将源代码移到了
src
文件夹中。 Now, the structure of the package is this:现在,package 的结构是这样的:
root
├───src
│ └───spark_map
│ ├───__init__.py
│ ├───functions.py
│ └───mapping.py
│
├───tests
├───.gitignore
├───LICENSE
├───pyproject.toml
├───README.md
└───README.rst
After that, I executed python -m pip install -e.
之后,我执行
python -m pip install -e.
(the log of this command is on the image below). (此命令的日志在下图中)。 The package was compiled and installed succesfully.
package编译安装成功。 However, when I open a new terminal, in a different location, and try to run
python -c "import spark_map"
, I still get the same error.但是,当我在不同的位置打开一个新终端并尝试运行
python -c "import spark_map"
时,我仍然遇到同样的错误。
I also tried to start a virtual environment (with python -m venv env
), and install the package inside this virtual environment (with pip install -e.
).我还尝试启动一个虚拟环境(使用
python -m venv env
),并在该虚拟环境中安装 package (使用pip install -e.
)。 Then, I executed python -c "import spark_map"
.然后,我执行
python -c "import spark_map"
。 But the problem still remains.但问题仍然存在。 I executed
pip list
too, to check if the package was installed.我也执行
pip list
,以检查是否安装了 package。 The full log of commands is on the image below:命令的完整日志如下图所示:
The standard file layout is the following ( Python doc: Packaging Python Projects ):标准文件布局如下( Python 文档:打包 Python 项目):
packaging_tutorial/ ├── LICENSE ├── pyproject.toml ├── README.md ├── src/ │ └── example_package_YOUR_USERNAME_HERE/ │ ├── __init__.py │ └── example.py └── tests/
You need to put your package inside a src
folder.您需要将 package 放在
src
文件夹中。
For your project, it should look like this:对于您的项目,它应该如下所示:
root
│
├───src/
│ └───spark_map/
│ ├───__init__.py
│ ├───functions.py
│ └───mapping.py
├───tests/
│ ├───functions
│ └───mapping
├───.gitignore
├───LICENSE
├───pyproject.toml
├───README.md
└───README.rst
You don't need to build a wheel distribution to test that, you can install the package directly to your env with pip install.
您不需要构建一个 wheel 发行版来测试它,您可以使用 pip install 将 package 直接安装到您的
pip install.
(or in edit mode if you want your update to be directly accessible without having to reinstall pip install -e.
). (或者如果您希望可以直接访问更新而无需重新安装
pip install -e.
,则在编辑模式下)。
The source of the problem is at the "build process" of the package. In other words, pip install
was installing a "not valid package".问题的根源在于 package 的“构建过程”。换句话说,
pip install
正在安装“无效包”。
Basically, I use setuptools to build the package. When I compiled (or "build" the package with python -m build
, the source code of the package (that is, all contents of the src
directory), was not included in the compiled TAR archive.基本上,我使用 setuptools 构建 package。当我编译(或使用
python -m build
“构建”package 时,package 的源代码(即src
目录的所有内容)未包含在编译的焦油档案。
The documentation for setuptools talks about this issue of finding the source code for your project . setuptools 的文档讨论了查找项目源代码的问题。 In essence, setuptools was not finding the source code of the package. So I needed to help him find these files, by adding these two options to my
pyproject.toml
file:本质上,setuptools 没有找到 package 的源代码。所以我需要通过将这两个选项添加到我的
pyproject.toml
文件来帮助他找到这些文件:
[tool.setuptools]
packages = ["spark_map"]
package-dir = {"" = "src"}
If you are having a similar problem at installing and importing your package, you might have this same problem, as I did.如果您在安装和导入 package 时遇到类似问题,您可能会遇到与我相同的问题。 To check if that is your case, build your project with
python -m build
.要检查这是否是您的情况,请使用
python -m build
构建您的项目。 Then, open the source distribution of your package (that is, the TAR archive), and check if the source code is there, inside this TAR file.然后,打开您的 package 的源代码分发(即 TAR 存档),并检查源代码是否在这个 TAR 文件中。 If not, than, you have this exact problem.
如果不是,那么,您就有这个确切的问题。
Change the build system to hatchling将构建系统更改为孵化
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
You also need to change the layout of your project like explained in this answer: src folder is required .您还需要像这个答案中解释的那样更改项目的布局: src folder is required 。
pip list
).pip list
)。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.