问题现象
构建 proxmox-backup deb 包时总是出现如下的 lintian 错误:
1 2
| E: proxmox-backup-server: package-contains-ancient-file etc/apt/sources.list.d/pbs-enterprise.list 1973-11-29 E: proxmox-backup-server: package-contains-ancient-file lib/systemd/system/proxmox-backup-daily-update.timer 1973-11-29
|
问题分析
解压 deb 之后确认问题真实存在:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
| $ stat etc/apt/sources.list.d/pbs-enterprise.list File: etc/apt/sources.list.d/pbs-enterprise.list Size: 70 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 310438 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-05 18:34:50.100000000 +0800 Modify: 1973-11-30 05:33:09.000000000 +0800 Change: 2021-07-05 18:34:50.100000000 +0800 Birth: - $ stat lib/systemd/system/proxmox-backup-daily-update.timer File: lib/systemd/system/proxmox-backup-daily-update.timer Size: 184 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 310444 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-05 18:34:50.100000000 +0800 Modify: 1973-11-30 05:33:09.000000000 +0800 Change: 2021-07-05 18:34:50.100000000 +0800 Birth: -
|
回头看一下原始文件的时间戳:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
| $ ll etc/ total 44 -rw-r--r-- 1 runsisi runsisi 480 Nov 30 1973 Makefile -rw-r--r-- 1 runsisi runsisi 70 Nov 30 1973 pbs-enterprise.list -rw-r--r-- 1 runsisi runsisi 401 Jul 5 18:06 proxmox-backup-banner.service -rw-r--r-- 1 runsisi runsisi 375 Nov 30 1973 proxmox-backup-banner.service.in -rw-r--r-- 1 runsisi runsisi 233 Jul 5 18:06 proxmox-backup-daily-update.service -rw-r--r-- 1 runsisi runsisi 220 Nov 30 1973 proxmox-backup-daily-update.service.in -rw-r--r-- 1 runsisi runsisi 184 Nov 30 1973 proxmox-backup-daily-update.timer -rw-r--r-- 1 runsisi runsisi 408 Jul 5 18:06 proxmox-backup-proxy.service -rw-r--r-- 1 runsisi runsisi 407 Nov 30 1973 proxmox-backup-proxy.service.in -rw-r--r-- 1 runsisi runsisi 315 Jul 5 18:06 proxmox-backup.service -rw-r--r-- 1 runsisi runsisi 302 Nov 30 1973 proxmox-backup.service.in $ stat etc/pbs-enterprise.list File: etc/pbs-enterprise.list Size: 70 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 306444 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-05 18:06:29.900000000 +0800 Modify: 1973-11-30 05:33:09.000000000 +0800 Change: 2021-07-05 18:06:29.830000000 +0800 Birth: -
|
显然,原始文件的时间戳就是有问题的,之所以 lintian 只报错其中的两个文件,是因为只有这两个文件是原样拷贝的。
proxmox-backup 的源代码包是使用 debcargo 打包的,底层实际上也是调用的 cargo 的接口,因此,尝试创建一个最简的 cargo 工程用 cargo package 打包,看看是不是也存在类似的问题。tar xf 解压 cargo package 创建的 crate 文件(需要先重命名为 tar.gz 文件),然后查看时间戳:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
| $ stat Cargo.lock File: Cargo.lock Size: 150 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 310545 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-06 08:28:36.570000000 +0800 Modify: 1970-01-01 08:00:01.000000000 +0800 Change: 2021-07-06 08:28:36.570000000 +0800 Birth: - $ stat Cargo.toml File: Cargo.toml Size: 717 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 310546 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-06 08:28:36.570000000 +0800 Modify: 1970-01-01 08:00:01.000000000 +0800 Change: 2021-07-06 08:28:36.570000000 +0800 Birth: - $ stat Cargo.toml.orig File: Cargo.toml.orig Size: 295 Blocks: 8 IO Block: 4096 regular file Device: 810h/2064d Inode: 310547 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-06 08:28:36.570000000 +0800 Modify: 1973-11-30 05:33:09.000000000 +0800 Change: 2021-07-06 08:28:36.570000000 +0800 Birth: -
|
显然,cargo 自身是有问题的,而且从 cargo package 就得到两个特殊的时间戳,其中一个与 debcargo 得到的一致(回过头去看 debcargo 打包的源代码,其实所有文件的时间戳都是有问题的,而且这两个错误的时间戳都):
1 2
| Modify: 1973-11-30 05:33:09.000000000 +0800 Modify: 1970-01-01 08:00:01.000000000 +0800
|
我们回到 cargo 的源代码,从源代码分析问题所在。
源码分析
源码打包的代码是在 src/cargo/ops/cargo_package.rs 中的 tar
函数实现的,首先从 build_ar_list
可以看到 Cargo.toml, Cargo.lock, Cargo.toml.orig 以及其它文件是如何归类的:
1 2 3 4
| Cargo.toml -> FileContents::Generated Cargo.toml.orig -> FileContents::OnDisk Cargo.lock -> FileContents::Generated others -> FileContents::OnDisk
|
然后从 tar
函数来看看各种分类的文件是如何被处理的:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
| match contents { FileContents::OnDisk(disk_path) => { let mut file = File::open(&disk_path).with_context(|| { format!("failed to open for archiving: `{}`", disk_path.display()) })?; let metadata = file.metadata().with_context(|| { format!("could not learn metadata for: `{}`", disk_path.display()) })?; header.set_metadata_in_mode(&metadata, HeaderMode::Deterministic); header.set_cksum(); ar.append_data(&mut header, &ar_path, &mut file) .with_context(|| { format!("could not archive source file `{}`", disk_path.display()) })?; } FileContents::Generated(generated_kind) => { let contents = match generated_kind { GeneratedFile::Manifest => pkg.to_registry_toml(ws)?, GeneratedFile::Lockfile => build_lock(ws)?, GeneratedFile::VcsInfo(s) => s, }; header.set_entry_type(EntryType::file()); header.set_mode(0o644); header.set_size(contents.len() as u64); header.set_cksum(); ar.append_data(&mut header, &ar_path, contents.as_bytes()) .with_context(|| format!("could not archive source file `{}`", rel_str))?; } }
|
对于 FileContents::OnDisk
类型,header.set_metadata_in_mode
根据第二个参数来决定其行为(具体实现可参考 fill_platform_from
),比如在这里的 HeaderMode::Deterministic
参数下,最终会调用 self.set_mtime(123456789);
,看看这个时间戳是什么时候:
1 2
| $ date -d @123456789 Fri Nov 30 05:33:09 CST 1973
|
是不是很熟悉?
而对于 FileContents::Generated
类型,可以看到并没有显式设置时间戳,没有显式设置就是 0,那我们上面看到的时间戳为什么是:
1
| Modify: 1970-01-01 08:00:01.000000000 +0800
|
而不是:
1
| Modify: 1970-01-01 08:00:00.000000000 +0800
|
还记不记得之前给 cargo 打过补丁:
https://github.com/rust-lang/cargo/pull/9517
显然,这个 1s 的差异就是这里来的,那如果回过去看 debcargo 生成的文件:
1 2 3 4 5 6 7 8 9
| $ stat Cargo.lock File: Cargo.lock Size: 57136 Blocks: 112 IO Block: 4096 regular file Device: 810h/2064d Inode: 310551 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ runsisi) Gid: ( 1000/ runsisi) Access: 2021-07-06 09:07:10.660000000 +0800 Modify: 1970-01-01 08:00:00.000000000 +0800 Change: 2021-07-06 09:07:10.660000000 +0800 Birth: -
|
由于 debcargo 依赖的 cargo 是没有打过补丁的,所以就没有那个多出来的 1s。
解决思路
给 cargo 中的 tar
函数打补丁,其中 FileContents::OnDisk
类型直接设置成原始文件的时间戳(由于各平台差异,需要参考 fill_platform_from
的实现,这里简单起见默认用的 unix
平台实现),而 FileContents::Generated
类型使用当前时间戳。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
|
@@ -509,6 +509,8 @@ format!("could not learn metadata for: `{}`", disk_path.display()) })?; header.set_metadata_in_mode(&metadata, HeaderMode::Deterministic); + use std::os::unix::fs::MetadataExt; + header.set_mtime(metadata.mtime() as u64); header.set_cksum(); ar.append_data(&mut header, &ar_path, &mut file) .with_context(|| { @@ -524,6 +526,8 @@ header.set_entry_type(EntryType::file()); header.set_mode(0o644); header.set_size(contents.len() as u64); + use std::time::{SystemTime, UNIX_EPOCH}; + header.set_mtime(SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs()); header.set_cksum(); ar.append_data(&mut header, &ar_path, contents.as_bytes()) .with_context(|| format!("could not archive source file `{}`", rel_str))?;
|
需要注意的是 debcargo 引用的 cargo 要在 vendor 目录下打补丁,打补丁之前要在 Cargo.toml 中添加 patch
配置,否则下一次 cargo vendor 会覆盖补丁版本:
1 2
| [patch.crates-io] cargo = { path = "vendor/cargo" }
|
参考资料
cfg!(linux) should work
https://github.com/rust-lang/rust/issues/26726
Conditional compilation
https://doc.rust-lang.org/reference/conditional-compilation.html
Need a way to easily modify vendored crates for local testing and try pushes
https://bugzilla.mozilla.org/show_bug.cgi?id=1323557