标签 关于 RSS
- 目录 -
OpenBMC cross compile toolchain
如果只是简单的 C/C++ 代码需要验证，那么写一个 recipe 显得稍微有点麻烦，但是 OpenBMC 并没有提供简单的方法提供一套完整的交叉编译工具链。
folder 实现原理   label openbmc   bmc   gcc   schedule 5 min 33 s.

recipe sysroot

bitbake.conf 定义的 PATH 环境变量结合了应用构建目录下的 recipe-sysroot（主要是 host 机器的头文件和库）和 recipe-sysroot-native（主要是 build 机器的 GCC 工具链）两个文件夹（具体的值可以参考 temp 目录下的 run.do_configure.PID 文件），如：

export PATH="
/home/runsisi/working/bmc/openbmc/scripts:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot-native/usr/bin/arm-openbmc-linux-gnueabi:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot/usr/bin/crossscripts:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot-native/usr/sbin:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot-native/usr/bin:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot-native/sbin:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot-native/bin:
/home/runsisi/working/bmc/openbmc/poky/bitbake/bin:
/home/runsisi/working/bmc/openbmc/build/romulus/tmp/hosttools
"

这两个文件夹的内容由 do_prepare_recipe_sysroot 任务生成，每个应用构建目录下都有这两个文件夹（根据应用定义的依赖进行动态生成，具体的依赖可以参考各自文件夹下的 sysroot-providers 目录）：

// meta/classes-global/staging.bbclass

python do_prepare_recipe_sysroot () {
    bb.build.exec_func("extend_recipe_sysroot", d)
}

python extend_recipe_sysroot() {
    for dep in sorted(configuredeps):
        // 从 tmp/sstate-control 目录找到 manifest-ARCH-XXX.populate_sysroot 清单文件
        manifest, d2 = oe.sstatesig.find_sstate_manifest(c, setscenedeps[dep][2], "populate_sysroot", d, multilibs)
        
        newmanifest = collections.OrderedDict()
        with open(manifest, "r") as f:
                manifests[dep] = manifest
                for l in f:
                    // 简单的字符串替换，举个例子：
                    // /home/runsisi/working/bmc/openbmc/build/romulus/tmp/sysroots-components/arm1176jzs/bzip2/usr/lib/libbz2.so.1.0.8
                    // ->
                    // /home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/arm1176jzs-openbmc-linux-gnueabi/bmcweb/1.0+git/recipe-sysroot/usr/lib/libbz2.so.1.0.8
                    dest = l.replace(stagingdir, "")
                    dest = "/" + "/".join(dest.split("/")[3:])
                    // 字典的 key 指向 tmp/sysroots/components 下的文件路径
                    // value 指向各应用 recipe-sysroot 或 recipe-sysroot-native
                    // 文件夹下的文件路径（此时还不存在）
                    newmanifest[l] = targetdir + dest

        // 该路径对应 tmp/sysroots-components/manifests/XXX.HASH 文件
        sharedm = sharedmanifests + "/" + os.path.basename(taskmanifest)
        with open(sharedm, 'w') as m:
            for l in newmanifest:
                dest = newmanifest[l]
                // 去掉了 WORKDIR 前缀，举个例子：
                // /home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/arm1176jzs-openbmc-linux-gnueabi/bmcweb/1.0+git/recipe-sysroot/usr/lib/libbz2.so.1.0.8
                // ->
                // recipe-sysroot/usr/lib/libbz2.so.1.0.8
                m.write(dest.replace(workdir + "/", "") + "\n")

        // 建立 tmp/sysroots-components/manifests/XXX.HASH ->
        // recipe-sysroot-native/installeddeps/XXX.HASH 的硬链接
        os.link(sharedm, taskmanifest)

        # Finally actually install the files
        for l in newmanifest:
                dest = newmanifest[l]
                if l.endswith("/"):
                    staging_copydir(l, targetdir, dest, seendirs)
                    continue
                if "/bin/" in l or "/sbin/" in l:
                    # defer /*bin/* files until last in case they need libs
                    binfiles[l] = (targetdir, dest)
                else:
                    // l 对应 tmp/sysroots/components 下的源文件
                    // dest 对应应用 recipe-sysroot 或 recipe-sysroot-native 文件夹下的目标文件
                    // 当前 targetdir 参数没用到，可以忽略
                    staging_copyfile(l, targetdir, dest, postinsts, seendirs)

    # Handle deferred binfiles
    for l in binfiles:
        (targetdir, dest) = binfiles[l]
        staging_copyfile(l, targetdir, dest, postinsts, seendirs)

def staging_copyfile(c, target, dest, postinsts, seendirs):
    if os.path.islink(c):
        // 源文件是符号链接，新建一个符号链接
        linkto = os.readlink(c)
        os.symlink(linkto, dest)
    else:
        // 建立硬链接
        os.link(c, dest)
    return dest

def staging_copydir(c, target, dest, seendirs):
    bb.utils.mkdirhier(dest)

cross compile toolchain

通过参考 temp 目录下的 run.do_configure.PID 文件（不带 PID 后缀的文件是符号链接）可以得到使用 gcc/g++ 进行交叉编译的命令行：

export CC="arm-openbmc-linux-gnueabi-gcc  -marm -mcpu=arm1176jz-s -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security -D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 --sysroot=/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot"

export CXX="arm-openbmc-linux-gnueabi-g++  -marm -mcpu=arm1176jz-s -fstack-protector-strong  -O2 -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security -D_TIME_BITS=64 -D_FILE_OFFSET_BITS=64 --sysroot=/home/runsisi/working/bmc/openbmc/build/romulus/tmp/work/romulus-openbmc-linux-gnueabi/linux-aspeed/6.6.6+git/recipe-sysroot"

但是，这里有一个比较麻烦的问题是，recipe-sysroot 和 recipe-sysroot-native 的生成是动态的，每个应用都不尽相同，因此对于简单的代码，选择一个依赖比较多的应用的构建目录即可，复杂的话要么新增一个 recipe 并使用 bitbake 进行管理，要么模仿 extend_recipe_sysroot 的实现，从 tmp/sysroots-components 目录构建一个属于自己的 sysroot 文件夹（遍历 tmp/sstate-control 目录下的 manifiest-ARCH-XXX.populate_sysroot 文件，读取每一行并将每一行内容指向的文件拷贝到 sysroot 目录下）：

https://github.com/runsisi/obmc-utils/blob/master/obmc-sysroot.py

#!/usr/bin/python3
# coding: utf-8

import argparse
import glob
import os.path
import shutil
import sys


def parse_args():
    parser = argparse.ArgumentParser('obmc-sysroot')
    parser.add_argument(
        '-b', '--build',
        dest='build',
        required=True,
        help='openbmc build dir'
    )
    parser.add_argument(
        '-r', '--root',
        dest='root',
        required=True,
        help='root dir to create'
    )
    parser.add_argument(
        '-v', '--verbose',
        dest='verbose',
        action='store_true',
        default=False,
        help='more verbose'
    )
    return parser.parse_args()


def setup_root(build, root, verbose):
    if not os.path.exists(build):
        print("build dir does not exist", file=sys.stderr)
        sys.exit(1)
    if os.path.exists(root) and os.listdir(root):
        print("root dir already exists and is not empty", file=sys.stderr)
        sys.exit(1)

    sstate = os.path.join(build, 'tmp/sstate-control')
    if not os.path.exists(sstate):
        print("tmp/sstate-control dir does not exist", file=sys.stderr)
        sys.exit(1)

    os.makedirs(root, exist_ok=True)

    manifests = os.path.join(sstate, 'manifest-*.populate_sysroot')
    for i in glob.glob(manifests):
        n = os.path.basename(i)
        n = n.removeprefix('manifest-')
        n = n.removesuffix('.populate_sysroot')

        # conflicts with libgcc
        if n.endswith('libgcc-initial'):
            continue

        native = False
        if n.endswith("-native") or "-cross-" in n or "-crosssdk" in n:
            native = True

        sysroot = os.path.join(root, 'sysroot')
        if native:
            sysroot = os.path.join(root, 'sysroot-native')

        with open(i, 'r') as f:
            for s in f:
                s = s.strip()
                if s.endswith('/fixmepath'):
                    continue
                if s.endswith('/fixmepath.cmd'):
                    continue

                d = s.replace(build, '')
                d = '/'.join(d.split('/')[5:])
                d = os.path.join(sysroot, d)
                if s.endswith('/'):
                    if verbose:
                        print(f'mkdirs {d}')
                    os.makedirs(d, exist_ok=True)
                    continue
                ddir = os.path.dirname(d)
                if not os.path.exists(ddir):
                    if verbose:
                        print(f'mkdirs {ddir}')
                    os.makedirs(ddir)
                if os.path.isdir(s):
                    if verbose:
                        print(f'copytree {s} -> {d}')
                    shutil.copytree(s, d)
                    continue
                if os.path.islink(s):
                    to = os.readlink(s)
                    if os.path.lexists(d):
                        if os.readlink(d) == to:
                            continue
                if verbose:
                    print(f'copy {s} -> {d}')
                # use copy to preserve file permission bits
                shutil.copy(s, d, follow_symlinks=False)


def print_usage(root):
    export = f'''export PATH=\\
{root}/sysroot-native/usr/bin/arm-openbmc-linux-gnueabi:\\
{root}/sysroot/usr/bin/crossscripts:\\
{root}/sysroot-native/usr/sbin:\\
{root}/sysroot-native/usr/bin:\\
{root}/sysroot-native/sbin:\\
{root}/sysroot-native/bin\\
$PATH
'''

    sysroot = f'''GCC --sysroot={root}/sysroot, e.g.,
arm-openbmc-linux-gnueabi-gcc --sysroot=/home/runsisi/working/test/bmcroot/sysroot -o x x.c
arm-openbmc-linux-gnueabi-gdb ./x
'''

    print('\n*** setup bmc sysroot succeeded! ***\n')
    print(f'{export}\n{sysroot}')


if __name__ == '__main__':
    args = parse_args()
    build = os.path.abspath(os.path.expanduser(args.build))
    root = os.path.abspath(os.path.expanduser(args.root))
    setup_root(build, root, args.verbose)
    print_usage(root)

执行上述脚本以构建 sysroot 目录：

❯ rm -rf ~/working/test/bmcroot
❯ python ~/working/py/obmc-utils/obmc-sysroot.py -b ~/working/bmc/openbmc/build/romulus -r ~/working/test/bmcroot

*** setup bmc sysroot succeeded! ***

export PATH=\
/home/runsisi/working/test/bmcroot/sysroot-native/usr/bin/arm-openbmc-linux-gnueabi:\
/home/runsisi/working/test/bmcroot/sysroot/usr/bin/crossscripts:\
/home/runsisi/working/test/bmcroot/sysroot-native/usr/sbin:\
/home/runsisi/working/test/bmcroot/sysroot-native/usr/bin:\
/home/runsisi/working/test/bmcroot/sysroot-native/sbin:\
/home/runsisi/working/test/bmcroot/sysroot-native/bin\
$PATH

GCC --sysroot=/home/runsisi/working/test/bmcroot/sysroot, e.g.,
arm-openbmc-linux-gnueabi-gcc --sysroot=/home/runsisi/working/test/bmcroot/sysroot -o x x.c
arm-openbmc-linux-gnueabi-gdb ./x

设置 PATH 环境变量（也可以直接拷贝脚本给出的示例）：

❯ export ROOT=~/working/test/bmcroot
❯ export PATH=\
$ROOT/sysroot-native/usr/bin/arm-openbmc-linux-gnueabi:\
$ROOT/sysroot/usr/bin/crossscripts:\
$ROOT/sysroot-native/usr/sbin:\
$ROOT/sysroot-native/usr/bin:\
$ROOT/sysroot-native/sbin:\
$ROOT/sysroot-native/bin\
$PATH

使用 gcc 进行交叉编译：

❯ arm-openbmc-linux-gnueabi-gcc --sysroot=$ROOT/sysroot -o x x.c

❯ file x
x: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/lib/ld-linux.so.3, BuildID[sha1]=7f142f377e6d0e01cecbda99a3e9eaad875d59e3, for GNU/Linux 5.15.0, with debug_info, not stripped

如果有需要，可以修改动态库的查找路径：

❯ patchelf --set-interpreter ./glibc/ld-linux.so.3 --set-rpath '$ORIGIN:$ORIGIN/glibc:$ORIGIN/gcc-runtime:$ORIGIN/libgcc:$ORIGIN/boost' --force-rpath ./x

最后修改于 2024-01-02

不念过去 不畏将来
Runsisi's Blog
标签 关于 RSS
Hugo Theme Diary by Rise
移植自 Makito's Journal.

© 2024 不念过去 不畏将来
- 目录 -