pip prefix
简单记录一下 Python 包 pip 安装过程中遇到的一个坑。

当使用如下命令从本地安装 python 包时,setup.py 中定义的 data_files 命令行等文件可以正确的(这是用户指定 --prefix 选项所期望的)被安装到 /usr/bin 下:

$ pip3 --version
pip 8.1.1 from /usr/lib/python3/dist-packages (python 3.5)

$ sudo pip3 install --prefix /usr xxx

但是,当直接使用 root 用户时却安装到了 site-packages/xxx/usr/bin 下。

观察两种方式安装过程的行为如下:

$ sudo pip3 install -v --prefix /usr 21.1-19-gbad84ad4-0ubuntu1_16.04.4.zip
The directory '/home/runsisi/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Installing collected packages: cloud-init
  Running setup.py install for cloud-init ...
Successfully installed cloud-init-21.1
Cleaning up...
$ sudo su
# pip3 install -v --prefix /usr 21.1-19-gbad84ad4-0ubuntu1_16.04.4.zip
Building wheels for collected packages: cloud-init
  Running setup.py bdist_wheel for cloud-init ...
Successfully built cloud-init
Installing collected packages: cloud-init
Successfully installed cloud-init-21.1
Cleaning up...

注意两者的差异,前者直接执行 setup.py install 进行安装,后者会先构建 wheel 包 setup.py bdist_wheel,然后再安装,而前者如果使用 sudo -H 那么两者的行为就一致了。

显然这两种行为的差异是 pip cache 引入的差异,在 root 用户执行 pip install 时可以通过 --no-cache-dir 选项来跳过 wheel 的构建过程(注意这是老版本 pip 的行为,新版本并不会跳过 wheel 的构建),从而实现使用 root 用户保持与非 root 用户一样的行为(即安装到 --prefix 选项指定的绝对路径)。

Passing –no-cache-dir should not affect whether to use a wheel
https://github.com/pypa/pip/issues/5749

Whether to build wheels is no longer affected by –no-cache-dir
https://github.com/pypa/pip/pull/6303

Address #6197: use the ephemeral cache if we need to when autobuilding
https://github.com/pypa/pip/pull/6219

或者 --no-binary :all: 选项也可以达到同样的目的:

How to install data_files to absolute path?
https://stackoverflow.com/questions/40588634/how-to-install-data-files-to-absolute-path

bdist_wheel makes absolute data_files relative to site-packages
https://github.com/pypa/wheel/issues/92

install procedure and python package creation is not aligned with pip and setuptools
https://github.com/kytos/kytos/issues/471

Since 7.0.0 sdist are converted to wheels before being installed. As a result, a package is never setup.py install’ed but always bdist_wheel’ed first.

All packages that contain non-package data are now likely installed in a broken way since 7.0.0
https://github.com/pypa/pip/issues/2874

当然,如果 pip 是比较新的版本(如 19+),这两种方案都不可行,setup.py 中定义的 data_files 仍然会被以相对路径安装到 site-packages 目录,这也是 cloud-init 等工具需要用发行版的安装包进行安装的原因。

The –no-binary option is interpreted to mean that we won’t download wheels, but not that we won’t build a wheel as part of installing the project. This is required because for projects using PEP 517, there is no other way to install them except by building a wheel.

This is not going to be possible for pyproject.toml-based projects (i.e. what PEP 517 started with standardising). We’re going to remove setup.py install based installations from pip in a future release.

--no-binary :all: option not taken into account when building from source
https://github.com/pypa/pip/issues/7248


最后修改于 2023-11-27