[英]How to include package data with setuptools/distutils?
When using setuptools, I can not get the installer to pull in any package_data
files.使用 setuptools 时,我无法让安装程序拉入任何
package_data
文件。 Everything I've read says that the following is the correct way to do it.我读过的所有内容都表明以下是正确的方法。 Can someone please advise?
有人可以建议吗?
setup(
name='myapp',
packages=find_packages(),
package_data={
'myapp': ['data/*.txt'],
},
include_package_data=True,
zip_safe=False,
install_requires=['distribute'],
)
where myapp/data/
is the location of the data files.其中
myapp/data/
是数据文件的位置。
I realize that this is an old question, but for people finding their way here via Google: package_data
is a low-down, dirty lie .我意识到这是一个老问题,但对于通过谷歌找到他们的方式的人来说:
package_data
是一个低调的、 肮脏的谎言。 It is only used when building binary packages ( python setup.py bdist ...
) but not when building source packages ( python setup.py sdist ...
).它构建二进制包(时才使用
python setup.py bdist ...
),但不建立源代码软件包(当python setup.py sdist ...
)。 This is, of course, ridiculous -- one would expect that building a source distribution would result in a collection of files that could be sent to someone else to built the binary distribution.这当然是荒谬的——人们会期望构建一个源代码发行版会产生一组文件,这些文件可以发送给其他人来构建二进制发行版。
In any case, using MANIFEST.in
will work both for binary and for source distributions.在任何情况下,使用
MANIFEST.in
将工作既为二进制和源分布。
I just had this same issue.我刚刚遇到了同样的问题。 The solution, was simply to remove
include_package_data=True
.解决方案只是删除
include_package_data=True
。
After reading here , I realized that include_package_data
aims to include files from version control , as opposed to merely "include package data" as the name implies. 阅读此处后,我意识到
include_package_data
旨在包含来自版本控制的文件,而不仅仅是顾名思义的“包含包数据”。 From the docs:从文档:
The data files [of include_package_data] must be under CVS or Subversion control
数据文件 [of include_package_data] 必须在 CVS 或 Subversion 控制之下
...
...
If you want finer-grained control over what files are included (for example, if you have documentation files in your package directories and want to exclude them from installation), then you can also use the
package_data
keyword.如果您想对包含的文件进行更细粒度的控制(例如,如果您的包目录中有文档文件并希望将它们从安装中排除),那么您还可以使用
package_data
关键字。
Taking that argument out fixed it, which is coincidentally why it also worked when you switched to distutils, since it doesn't take that argument.去掉那个论点就解决了它,这也是为什么当你切换到 distutils 时它也能工作的原因,因为它不接受那个论点。
Following @Joe 's recommendation to remove the include_package_data=True
line also worked for me.按照@Joe 的建议删除
include_package_data=True
行也对我有用。
To elaborate a bit more, I have no MANIFEST.in
file.详细说明一下,我没有
MANIFEST.in
文件。 I use Git and not CVS.我使用 Git 而不是 CVS。
Repository takes this kind of shape:存储库采用这种形状:
/myrepo
- .git/
- setup.py
- myproject
- __init__.py
- some_mod
- __init__.py
- animals.py
- rocks.py
- config
- __init__.py
- settings.py
- other_settings.special
- cool.huh
- other_settings.xml
- words
- __init__.py
word_set.txt
setup.py
: setup.py
:
from setuptools import setup, find_packages
import os.path
setup (
name='myproject',
version = "4.19",
packages = find_packages(),
# package_dir={'mypkg': 'src/mypkg'}, # didnt use this.
package_data = {
# If any package contains *.txt or *.rst files, include them:
'': ['*.txt', '*.xml', '*.special', '*.huh'],
},
#
# Oddly enough, include_package_data=True prevented package_data from working.
# include_package_data=True, # Commented out.
data_files=[
# ('bitmaps', ['bm/b1.gif', 'bm/b2.gif']),
('/opt/local/myproject/etc', ['myproject/config/settings.py', 'myproject/config/other_settings.special']),
('/opt/local/myproject/etc', [os.path.join('myproject/config', 'cool.huh')]),
#
('/opt/local/myproject/etc', [os.path.join('myproject/config', 'other_settings.xml')]),
('/opt/local/myproject/data', [os.path.join('myproject/words', 'word_set.txt')]),
],
install_requires=[ 'jsonschema',
'logging', ],
entry_points = {
'console_scripts': [
# Blah...
], },
)
I run python setup.py sdist
for a source distrib (haven't tried binary).我为源发行版运行
python setup.py sdist
(还没有尝试过二进制)。
And when inside of a brand new virtual environment, I have a myproject-4.19.tar.gz
, file, and I use当在一个全新的虚拟环境中时,我有一个
myproject-4.19.tar.gz
文件,我使用
(venv) pip install ~/myproject-4.19.tar.gz
...
And other than everything getting installed to my virtual environment's site-packages
, those special data files get installed to /opt/local/myproject/data
and /opt/local/myproject/etc
.除了将所有内容安装到我的虚拟环境的
site-packages
,这些特殊数据文件还安装到/opt/local/myproject/data
和/opt/local/myproject/etc
。
include_package_data=True
worked for me. include_package_data=True
为我工作。
If you use git, remember to include setuptools-git
in install_requires
.如果您使用 git,请记住在
install_requires
包含setuptools-git
。 Far less boring than having a Manifest
or including all path in package_data
( in my case it's a django app with all kind of statics )远没有一个
Manifest
或者在package_data
包含所有路径那么无聊(在我的例子中它是一个带有各种静态的 django 应用程序)
( pasted the comment I made, as k3-rnc mentioned it's actually helpful as is ) (粘贴我所做的评论,因为k3-rnc提到它实际上很有帮助)
I had the same problem for a couple of days but even this thread wasn't able to help me as everything was confusing.几天来我遇到了同样的问题,但即使是这个线程也无法帮助我,因为一切都令人困惑。 So I did my research and found the following solution:
所以我做了我的研究并找到了以下解决方案:
Basically in this case, you should do:
基本上在这种情况下,你应该这样做:
from setuptools import setup setup( name='myapp', packages=['myapp'], package_dir={'myapp':'myapp'}, # the one line where all the magic happens package_data={ 'myapp': ['data/*.txt'], }, )
The full other stackoverflow answer here 完整的其他stackoverflow答案在这里
Update : This answer is old and the information is no longer valid.更新:这个答案是旧的,信息不再有效。 All setup.py configs should use
import setuptools
.所有 setup.py 配置都应该使用
import setuptools
。 I've added a more complete answer at https://stackoverflow.com/a/49501350/64313我在https://stackoverflow.com/a/49501350/64313添加了更完整的答案
I solved this by switching to distutils.我通过切换到 distutils 解决了这个问题。 Looks like distribute is deprecated and/or broken.
看起来分发已被弃用和/或损坏。
from distutils.core import setup
setup(
name='myapp',
packages=['myapp'],
package_data={
'myapp': ['data/*.txt'],
},
)
Ancient question and yet... package management of python really leaves a lot to be desired.古老的问题,然而……python 的包管理确实有很多不足之处。 So I had the use case of installing using pip locally to a specified directory and was surprised both package_data and data_files paths did not work out.
所以我有使用 pip 本地安装到指定目录的用例,并且很惊讶 package_data 和 data_files 路径都没有工作。 I was not keen on adding yet another file to the repo so I ended up leveraging data_files and setup.py option --install-data;
我并不热衷于向 repo 添加另一个文件,所以我最终利用了 data_files 和 setup.py 选项 --install-data; something like this
像这样的东西
pip install . --install-option="--install-data=$PWD/package" -t package
Just remove the line:只需删除该行:
include_package_data=True,
from your setup script, and it will work fine.从您的安装脚本,它会正常工作。 (Tested just now with latest setuptools.)
(刚刚用最新的 setuptools 测试过。)
I found this post while stuck on the same problem.我在遇到同样的问题时发现了这篇文章。
My experience contradicts the experiences in the other answers.我的经历与其他答案中的经历相矛盾。
include_package_data=True
does include the data in the bdist! include_package_data=True
确实包含 bdist 中的数据! The explanation in the setuptools
documentation lacks context and troubleshooting tips, but include_package_data
works as advertised. setuptools
文档中的解释缺少上下文和故障排除提示,但include_package_data
工作原理与宣传的一样。
My setup:我的设置:
setuptools
v47.3.1 setuptools
v47.3.1check-manifest
v0.42 check-manifest
v0.42 Here is my how-to guide.这是我的操作指南。
Here is the file structure for a project I published on PyPI.这是我在 PyPI 上发布的项目的文件结构。 (It installs the application in
__main__.py
). (它将应用程序安装在
__main__.py
)。
├── LICENSE.md
├── MANIFEST.in
├── my_package
│ ├── __init__.py
│ ├── __main__.py
│ └── _my_data <---- folder with data
│ ├── consola.ttf <---- data file
│ └── icon.png <---- data file
├── README.md
└── setup.py
Here is a generic starting point for the setuptools.setup()
in setup.py
.这是
setup.py
setuptools.setup()
的通用起点。
setuptools.setup(
...
packages=setuptools.find_packages(),
...
)
setuptools.find_packages()
includes all of my packages in the distribution. setuptools.find_packages()
包括我在发行版中的所有包。 My only package is my_package
.我唯一的包是
my_package
。
The sub-folder with my data, _my_data
, is not considered a package by Python because it does not contain an __init__.py
, and so find_packages()
does not find it.包含我的数据的子文件夹
_my_data
不被 Python 视为包,因为它不包含__init__.py
,因此find_packages()
找不到它。
A solution often-cited, but incorrect , is to put an empty __init__.py
file in the _my_data
folder.一个经常被引用但不正确的解决方案是在
_my_data
文件夹中放置一个空的__init__.py
文件。
This does make it a package, so it does include the folder _my_data
in the distribution.这确实使它成为一个包,因此它确实在分发中包含文件夹
_my_data
。 But the data files inside _my_data
are not included .但不包括
_my_data
中的数据文件。
So making _my_data
into a package does not help .因此,将
_my_data
成一个包并没有帮助。
The solution is:解决办法是:
sdist
already contains the data files sdist
已经包含数据文件include_package_data=True
to include the data files in the bdist
as wellinclude_package_data=True
以将数据文件也包含在bdist
中 There are three steps to make this a repeatable experiment:要使这个实验成为可重复的实验,需要执行三个步骤:
$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python setup.py sdist bdist_wheel
I will break these down step-by-step:我将逐步分解这些:
$ rm -fr build/ dist/ my_package.egg-info/
check-manifest
to be sure MANIFEST.in
matches the Git index of files under version control:check-manifest
以确保MANIFEST.in
与版本控制下文件的 Git 索引匹配:$ check-manifest
If MANIFEST.in
does not exist yet, create it from the Git index of files under version control:如果
MANIFEST.in
尚不存在,请从受版本控制的文件的 Git 索引创建它:
$ check-manifest --create
Here is the MANIFEST.in
that is created:这是创建的
MANIFEST.in
:
include *.md
recursive-include my_package *.png
recursive-include my_package *.ttf
There is no reason to manually edit this file.没有理由手动编辑此文件。
As long as everything that should be under version control is under version control (ie, is part of the Git index), check-manifest --create
does the right thing.只要应该在版本控制下的所有内容都在版本控制下(即,是 Git 索引的一部分),
check-manifest --create
做正确的事情。
Note: files are not part of the Git index if they are either:注意:文件不是Git 索引的一部分,如果它们是:
.gitignore
.gitignore
被忽略.git/info/exclude
.git/info/exclude
And if any files are under version control that should not be under version control, check-manifest
issues a warning and specifies which files it recommends removing from the Git index.如果任何文件受版本控制而不应受版本控制,
check-manifest
发出警告并指定它建议从 Git 索引中删除哪些文件。
$ python setup.py sdist bdist_wheel
Now inspect the sdist
(source distribution) and bdist_wheel
(build distribution) to see if they include the data files.现在检查
sdist
(源代码分发)和bdist_wheel
(构建分发)以查看它们是否包含数据文件。
Look at the contents of the sdist
(only the relevant lines are shown below):查看
sdist
的内容(下面只展示了相关的sdist
行):
$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png <-- yay!
...
So the sdist
already includes the data files because they are listed in MANIFEST.in
.所以
sdist
已经包含了数据文件,因为它们列在MANIFEST.in
。 There is nothing extra to do to include the data files in the sdist
.在
sdist
包含数据文件没有什么额外的事情要做。
Look at the contents of the bdist
(it is a .zip file, parsed with zipfile.ZipFile
):查看
bdist
的内容(它是一个 .zip 文件,用zipfile.ZipFile
解析):
$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD
Note: you need to create your own check-whl.py
script to produce the above output.注意:您需要创建自己的
check-whl.py
脚本来生成上述输出。 It is just three lines:它只有三行:
from zipfile import ZipFile
path = "dist/my_package-0.0.1a6-py3-none-any.whl" # <-- CHANGE
print('\n'.join(ZipFile(path).namelist()))
As expected, the bdist
is missing the data files.正如预期的那样,
bdist
缺少数据文件。
The _my_data
folder is completely missing. _my_data
文件夹完全丢失。
What if I create a _my_data/__init__.py
?如果我创建一个
_my_data/__init__.py
怎么_my_data/__init__.py
? I repeat the experiment and I find the data files are still not there!我重复了这个实验,发现数据文件还是没有! The
_my_data/
folder is included but it does not contain the data files! _my_data/
文件夹包含在内,但不包含数据文件!
Contrary to the experience of others, this does work:与其他人的经验相反,这确实有效:
setuptools.setup(
...
packages=setuptools.find_packages(),
include_package_data=True, # <-- adds data files to bdist
...
)
With the fix in place, redo the experiment:修复到位后,重做实验:
$ rm -fr build/ dist/ my_package.egg-info/
$ check-manifest
$ python.exe setup.py sdist bdist_wheel
Make sure the sdist
still has the data files:确保
sdist
仍然有数据文件:
$ tar --list -f dist/my_package-0.0.1a6.tar.gz
my_package-0.0.1a6/
...
my_package-0.0.1a6/my_package/__init__.py
my_package-0.0.1a6/my_package/__main__.py
my_package-0.0.1a6/my_package/_my_data/
my_package-0.0.1a6/my_package/_my_data/consola.ttf <-- yay!
my_package-0.0.1a6/my_package/_my_data/icon.png <-- yay!
...
Look at the contents of the bdist
:查看
bdist
的内容:
$ python check-whl.py
my_package/__init__.py
my_package/__main__.py
my_package/_my_data/consola.ttf <--- yay!
my_package/_my_data/icon.png <--- yay!
my_package-0.0.1a6.dist-info/LICENSE.md
my_package-0.0.1a6.dist-info/METADATA
my_package-0.0.1a6.dist-info/WHEEL
my_package-0.0.1a6.dist-info/entry_points.txt
my_package-0.0.1a6.dist-info/top_level.txt
my_package-0.0.1a6.dist-info/RECORD
I recommend troubleshooting/testing using the approach outlined above to inspect the sdist
and bdist
.我建议使用上述方法进行故障排除/测试以检查
sdist
和bdist
。
Note: pip install -e .
注意:
pip install -e .
does not show if data files are included in the bdist
.不显示数据文件是否包含在
bdist
。
The symbolic link causes the installation to behave as if the data files are included (because they already exist locally on the developer's computer).符号链接使安装的行为就像包含数据文件一样(因为它们已经存在于开发人员的本地计算机上)。
After pip install my_package
, the data files are in the virtual environment's lib/site-packages/my_package/
folder, using the exact same file structure shown above in the list of the whl
contents.在
pip install my_package
,数据文件位于虚拟环境的lib/site-packages/my_package/
文件夹中,使用与上面whl
内容列表中显示的完全相同的文件结构。
Publishing to TestPyPI and then installing and looking in lib/site-packages/my_packages
is a valid test, but it is too time-consuming.发布到 TestPyPI 然后在
lib/site-packages/my_packages
安装和查找是一个有效的测试,但它太耗时了。
Moving the folder containing the package data into to module folder solved the problem for me.将包含包数据的文件夹移动到模块文件夹中为我解决了这个问题。
See this question: MANIFEST.in ignored on "python setup.py install" - no data files installed?看到这个问题: MANIFEST.in 在“python setup.py install”上被忽略——没有安装数据文件?
Starting with setuptools 30.3.0 (released 2016-12-08), you can keep your setup.py
very small and move the configuration to a setup.cfg
file.从 setuptools 30.3.0(2016 年 12 月 8 日发布)开始,您可以将
setup.py
保持得非常小,并将配置移动到setup.cfg
文件中。 With this approach, you could put your package data in an [options.package_data]
section:使用这种方法,您可以将包数据放在
[options.package_data]
部分:
[options.package_data]
* = *.txt, *.rst
hello = *.msg
In this case, your setup.py
can be as short as:在这种情况下,您的
setup.py
可以很短:
from setuptools import setup
setup()
For more information, see configuring setup using setup.cfg files .有关详细信息,请参阅使用 setup.cfg 文件配置设置。
There is some talk of deprecating setup.cfg
in favour of pyproject.toml
as proposed in PEP 518 , but this is still provisional as of 2020-02-21.有一些关于弃用
setup.cfg
以支持PEP 518 中提议的pyproject.toml
讨论, setup.cfg
在pyproject.toml
日仍然是临时的。
Starting with Setuptools 62.3.0 , you can now use recursive wildcards ( "**"
) to include a (sub)directory recursively.从Setuptools 62.3.0开始,您现在可以使用递归通配符(
"**"
) 以递归方式包含(子)目录。 This way you can include whole folders with all their folders and files in it.这样,您可以包含整个文件夹及其所有文件夹和文件。
For example, when using a pyproject.toml
file, this is how you include two folders recursively:例如,当使用
pyproject.toml
文件时,这是递归包含两个文件夹的方式:
[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**"]
"ema_workbench.examples.models" = ["**"]
But you can also only include certain file-types, in a folder and all subfolders.但您也只能在文件夹和所有子文件夹中包含某些文件类型。 If you want to include all markdown (
.md
) files for example:如果您想包含所有 markdown (
.md
) 文件,例如:
[tool.setuptools.package-data]
"ema_workbench.examples.data" = ["**/*.md"]
It should also work when using setup.py
or setup.cfg
.使用
setup.py
或setup.cfg
时它也应该工作。
See https://github.com/pypa/setuptools/pull/3309 for the details.有关详细信息,请参阅https://github.com/pypa/setuptools/pull/3309 。
For a directory structure like:对于目录结构,如:
foo/
├── foo
│ ├── __init__.py
│ ├── a.py
│ └── data.txt
└── setup.py
and setup.py
和
setup.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from setuptools import setup
NAME = 'foo'
DESCRIPTION = 'Test library to check how setuptools works'
URL = 'https://none.com'
EMAIL = 'gzorp@bzorp.com'
AUTHOR = 'KT'
REQUIRES_PYTHON = '>=3.6.0'
setup(
name=NAME,
version='0.0.0',
description=DESCRIPTION,
author=AUTHOR,
author_email=EMAIL,
python_requires=REQUIRES_PYTHON,
url=URL,
license='MIT',
classifiers=[
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.6',
],
packages=['foo'],
package_data={'foo': ['data.txt']},
include_package_data=True,
install_requires=[],
extras_require={},
cmdclass={},
)
python setup.py bdist_wheel
works. python setup.py bdist_wheel
有效。
Like others in this thread, I'm more than a little surprised at the combination of longevity and still a lack of clarity, BUT the best answer for me was using check-manifest
as recommended in the answer from @mike-gazes像这个线程中的其他人一样,我对长寿和仍然缺乏清晰度的结合感到有点惊讶,但对我来说最好的答案是使用@mike-gazes 的答案中推荐的
check-manifest
So, using just a setup.cfg
and no setup.py
and additional text and python files required in the package, what worked for me was keeping this in setup.cfg:因此,仅使用
setup.cfg
而无需setup.py
以及包中所需的其他文本和 python 文件,对我有用的是将其保留在 setup.cfg 中:
[options]
packages = find:
include_package_data = true
and updating the MANIFEST.in
based on the check-manifest
output:并根据
check-manifest
输出更新MANIFEST.in
:
include *.in
include *.txt
include *.yml
include LICENSE
include tox.ini
recursive-include mypkg *.py
recursive-include mypkg *.txt
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.