runsisi's

technical notes

ceph-selinux policydb module version 19 does not match my version range 4-17

2019-01-14 runsisi#debug#selinux

最近发现在安装 ceph 时 ceph-selinux 包会报如下的错误(不影响安装过程,安装的错误码仍然返回 0),但 google 找不到任何有意义的信息:

libsemanage.semanage_pipe_data: Child process /usr/libexec/selinux/hll/pp failed with code: 255. (No such file or directory).
ceph: libsepol.policydb_read: policydb module version 19 does not match my version range 4-17
ceph: libsepol.sepol_module_package_read: invalid module in module package (at section 0)
ceph: Failed to read policy package
libsemanage.semanage_direct_commit: Failed to compile hll files into cil files.
(No such file or directory).
/usr/sbin/semodule: Failed!

查看 ceph-selinux 安装时到底在做什么:

~$ rpm -qp --scripts ceph-selinux-12.2.8-2.4.gc1f1dce.el7.x86_64.rpm
...
# Install the policy
/usr/sbin/semodule -i /usr/share/selinux/packages/ceph.pp
...

将 rpm 包解压出来,手工执行看看:

~$ rpm2cpio ceph-selinux-12.2.8-2.4.gc1f1dce.el7.x86_64.rpm | cpio -div
./usr/share/man/man8/ceph_selinux.8.gz
./usr/share/selinux/devel/include/contrib/ceph.if
./usr/share/selinux/packages/ceph.pp
203 blocks
~# semodule -i ceph.pp
libsepol.policydb_read: policydb module version 19 does not match my version range 4-17 (No such file or directory).
libsepol.sepol_module_package_read: invalid module in module package (at section 0) (No such file or directory).
libsemanage.semanage_load_module: Error while reading from module file /etc/selinux/targeted/modules/tmp/modules/ceph.pp. (No such file or directory).
semodule: Failed!

显然是 ceph.pp 要求的 policydb 版本过高导致的。

既然知道是 ceph.pp 导致的,显然不能通过反复制作 rpm 包来复现该问题,我们首先要尝试以简单的方式制作 ceph.pp[1]:

~$ cmake .. -DWITH_SELINUX=ON
~$ make help | grep ceph.pp
... ceph.pp
~$ make ceph.pp
Scanning dependencies of target ceph.pp
Compiling targeted ceph module
/usr/bin/checkmodule: loading policy configuration from tmp/ceph.tmp
/usr/bin/checkmodule: policy configuration loaded
/usr/bin/checkmodule: writing binary representation (version 19) to tmp/ceph.mod
Creating targeted ceph.pp policy package
Built target ceph.pp

在执行过程中,竟然发现了 version 19 这个非常重要的版本号,因此从 checkmodule 入手:

~$ rpm -qf /usr/bin/checkmodule
checkpolicy-2.5-6.el7.x86_64
~$ rpm -qi checkpolicy-2.5-6.el7.x86_64
Name : checkpolicy
Version : 2.5
Release : 6.el7
Architecture: x86_64
...
Source RPM : checkpolicy-2.5-6.el7.src.rpm
...

从 vault 网站找到对应的源代码:

http://vault.centos.org/7.5.1804/os/Source/SPackages/checkpolicy-2.5-6.el7.src.rpm

然后从代码中得到版本号 19 的由来:

checkmodule.c
...
#include <sepol/policydb/policydb.h>
...
unsigned int policyvers = MOD_POLICYDB_VERSION_MAX;
...
~# vi /usr/include/sepol/policydb/policydb.h
...
#define MOD_POLICYDB_VERSION_INFINIBAND   19
...
#define MOD_POLICYDB_VERSION_MAX MOD_POLICYDB_VERSION_INFINIBAND
...

看来是 sepol 的 devel 包版本太高导致的:

~$ rpm -qf /usr/include/sepol/policydb/policydb.h
libsepol-devel-2.5-6.el7.x86_64

从 vault 网站上找到了定义 MODPOLICYDBVERSION_MAX 为 17 的低版本 libsepol-devel:

http://vault.centos.org/7.2.1511/os/x86_64/Packages/libsepol-devel-2.1.9-3.el7.x86_64.rpm

因此要解决该问题,我们要做的就是把相关的版本包降级(通过添加 vault 中特定版本 CentOS 的 yum 源):

~# yum downgrade libsepol libsepol-devel libselinux libselinux-utils setools-libs libselinux-devel setools-console libselinux-python checkpolicy
~# yum versionlock add libsepol libsepol-devel libselinux libselinux-utils setools-libs libselinux-devel setools-console libselinux-python checkpolicy

最后再尝试编译:

~$ make ceph.pp
Compiling targeted ceph module
/usr/bin/checkmodule: loading policy configuration from tmp/ceph.tmp
/usr/bin/checkmodule: policy configuration loaded
/usr/bin/checkmodule: writing binary representation (version 17) to tmp/ceph.mod
Creating targeted ceph.pp policy package
Built target ceph.pp

可以看到,版本号已经变成 17 了。

上面的分析过程是通过观察 checkmodule 的输出得到的,实际上,直接分析 ceph.pp 的 Makefile 也可以得到类似的分析过程:

~$ vi /usr/share/selinux/devel/include/Makefile
...
#
# Build module packages
#
tmp/%.mod: $(m4support) tmp/all_interfaces.conf %.te
@$(EINFO) "Compiling $(NAME) $(basename $(@F)) module"
@test -d $(@D) || mkdir -p $(@D)
$(verbose) $(M4) $(M4PARAM) -s $^ > $(@:.mod=.tmp)
$(verbose) $(CHECKMODULE) -m $(@:.mod=.tmp) -o $@
...

[1] You Are Solving The Wrong Problem

https://uxmag.com/articles/you-are-solving-the-wrong-problem