2022-02-27
Suppose you have a PinePhone running Mobian ("bookworm") and a desktop or laptop running Debian ("sid"). How do you write a C program on the latter, that can run on the former?
The PinePhone is powerful enough that you could just install the build tools and everything you need directly on it and develop directly on the device. However, you may have reasons to want to avoid this approach, so let's look at the classic approach, where we build on the desktop and run on the mobile.
Cross-compiling
We can't run on the mobile a binary built for the desktop, because the two have different architectures. The mobile is, in our case, aarch64 a.k.a. arm64, while the desktop is x86-64.
A regular compiler would produce code for the same architecture it was run on.
Considering a C program in a main.c file,
if you call:
gcc -o example-x64 main.c
and then:
file example-x64
then the output will be something like:
example-x64: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=..., for GNU/Linux 3.2.0, not stripped
Attempting to execute that file on the mobile device would result in the following error:
-bash: ./example-x64: cannot execute binary file: Exec format error
The solution to this problem is a cross-compiler. That is, a compiler that can run on one architecture and produce binaries for another.
Let's install a cross-compiler on the desktop computer:
sudo apt install gcc-aarch64-linux-gnu
Now let's compile our program using the cross-compiler:
aarch64-linux-gnu-gcc -o example-arm main.c
This time, the output is a different kind of file and can be executed on the target device:
file example-arm
example-arm: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, BuildID[sha1]=..., for GNU/Linux 3.7.0, not stripped
Adding dependencies
We can build a simple program with no dependecies, but what happens when we need to link against a 3rd party library?
Let's take GTK as an example library,
and let's try the "Hello, World!" program from
the GTK homepage.
The following command assume the source code is in a file named
gtk-example.c.
Assuming we're not only interested in arm64 and that we also want to run the program on x86-64, we first need to install the development package for the desktop:
sudo apt install libgtk-4-dev
Then we build the program, using
pkg-config
for the library's compiler and the linker specific flags.
gcc \
$(pkg-config --cflags gtk4) \
-o gtk-example-x64 \
gtk-example.c \
$(pkg-config --libs gtk4)
If we run ldd on the resulting binary, we'll see
that it's linked agains libraries from
/lib/x86_64-linux-gnu, which are libaries specific
to the architecture where we built the program and where we'll
also run it:
$ ldd gtk-example-x64 | grep gtk
libgtk-4.so.1 => /lib/x86_64-linux-gnu/libgtk-4.so.1 (0x00007fed32150000)
But we'll need to build with the aarch64 toolchain and link against aarch64 libraries. Where do we get those libraries?
Sysroot
First make sure that your kernel can run binaries
of the arm64 architecture, by running arch-test.
If arm64 is in the list, then it will work.
Otherwise, it can be made available by installing the qemu-user-static
package.
sudo apt install qemu-user-static
Since we're using Debian on both systems, we can use
Debootstrap
to install a base Debian system with the same version (bookworm)
and for the same architecture (aarch64) as the mobile device
right into a subdirectory
(for example /aarch64-bookworm-sysroot)
of the desktop.
For the last parameter of the debootstrap command,
replace http://deb.debian.org/debian
with the URL of
a mirror next to you
.
sudo apt install debootstrap
sudo mkdir /aarch64-bookworm-sysroot
sudo debootstrap \
--arch=arm64 \
bookworm \
/aarch64-bookworm-sysroot \
http://deb.debian.org/debian/
Once the new system is set up, enter a chroot:
sudo chroot /aarch64-bookworm-sysroot
Inside the chroot, install the libraries you need for development, then exit.
apt update
apt install libgtk-4-dev
Now you should have the libraries in the new sysroot,
for example in /aarch64-bookworm-sysroot/usr/lib/aarch64-linux-gnu/libgtk-4.so.
We need to make pkg-config aware of the new sysroot
and make it look there when cross-compiling.
Create a wrapper script for pkg-config
with the content below,
name it aarch64-linux-gnu-pkg-config,
make it executable
and place it somewhere in your $PATH
(for example in $HOME/.local/bin).
The name of the wrapper script doesn't matter much right now
as we'll invoke it manually, but it will be important
in the next steps, when we'll use Autoconf. Autoconf will look
for a pkg-config executable in the $PATH
with a specific prefix,
based on the architecture we're building for.
This will still use the local pkg-config but
it will make it look for libraries in other places
(the new sysroot that we just created).
The Autotools Mythbuster
has more details about this wrapper script approach.
#!/bin/sh
SYSROOT=/aarch64-bookworm-sysroot
export PKG_CONFIG_PATH=
export PKG_CONFIG_LIBDIR=${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/lib/aarch64-linux-gnu/pkgconfig:${SYSROOT}/usr/share/pkgconfig
export PKG_CONFIG_SYSROOT_DIR=${SYSROOT}
exec pkg-config "$@"
Now we're all set for cross-compiling. We can just use
aarch64-linux-gnu-gcc instead of plain gcc
and our new aarch64-linux-gnu-pkg-config instead of
pkg-config. Additionally, we also need to make
gcc aware of the sysroot and prevent it from linking
against the libraries in the standard directories (which are for
x86-64, and not for arm64).
aarch64-linux-gnu-gcc \
$(aarch64-linux-gnu-pkg-config --cflags gtk4) \
-o gtk-example-arm \
gtk-example.c \
$(aarch64-linux-gnu-pkg-config --libs gtk4) \
--sysroot=/aarch64-bookworm-sysroot
Autotools
Calling aarch64-linux-gnu-gcc explicitly with all
those flags is fine once, for an example, but for a real project
we may want to use GNU Autotools. See the previous article, How to Set Up a Project With Autotools
for how to do that.
When building the same project for multiple platforms
we may want to use VPATH builds (with one build
directory for each platform). For this reason, the command below
shows a call to ../configure instead of the
usual ./configure.
To compile for another platform we need to specify the
--host flag (the host is the architecture where
the resulting binary will run). We also need to make
the linker aware of the sysroot (as before) by setting the
LDFLAGS variable to contain the --sysroot
flag.
../configure \
--host=aarch64-linux-gnu \
LDFLAGS="--sysroot=/aarch64-bookworm-sysroot"