Dylan T Bespalko

Jul 59 min read

Cross-Compiling UNIX Applications for QNX using the Buildroot Internal Build-flow

The goal of this tutorial is to demonstrate a minimum viable solution for cross-compiling UNIX applications to run on QNX. A better version of this document is on the gitlab repo: https://gitlab.com/dylan_bespalko_work/buildroot-qnx

Linux vs. QNX

Feature	Linux	QNX	Decision
Kernel Architecture	Monolithic	Micro	Should the filesystem, device driver, and inter-process communication belong to the Kernel or the the User?
Runtime	General-Purpose	Realtime	Do I need precise control of process scheduling?
Source Code	Open	Proprietary	Should the developer have the ability to view/modify the kernel source code?
User Land	Mostly POSIX-compliant	Formerly POSIX-certified	POSIX Compliance ensures that end-user applications can be ported between different operating systems

The Linux approach favors computers running SW that is connected to the internet, while QNX skews towards "mission-critical" embedded systems with custom hardware drivers.

Regardless of which kernel you choose, the POSIX standard ensures that your application code will be portable, but you will need to cross-compile it first. In this tutorial, I will cross-compile from a Ubuntu 22.04 host machine to a QNX 8.0.0 target machine.

Cross-Compiling Tools

The process for building an embedded system is as follows:

Build the toolchain (eg. A gcc compiler that builds on a host platform, but deploys on a target platform).
Build the bootloader (eg. U-Boot vs. Grub)
Build the kernel (eg. Linux vs. QNX)
Build the root filesystem (ie. Populate the applications and files to be stored on the target device).

The following cross-compiling tools do different things for different people and are mostly associated with Embedded Linux.

Yocto Project: One-Click build flow with the most software packages that can be combined with a board-support-package (BSP) (bootloader, kernel, and filesystem). Best suited for thin-clients.
Buildroot: Separation of bootloader, kernel, and filesystem with emphasis on the filesystem. Best suited for embedded-devices.
Crosstool-NG: Builds the cross-compile toolchain. The resulting toolchain is used by Yocto and Buildroot as the "cross-compiler".

Everything embedded developers do is insanely inefficient compared to desktop computing. In order to "maintain employment", work within you current abilities.

Buy a build server and port a software project from the Linux Desktop to an embedded device (make sure there is a BSP for your chosen board).
Move between embedded devices by using Buildroot. Some of your packages will not have build recipes and you will need to copy some from Yocto. You are now responsible for the BSP, but it is easier to write custom device drivers.
Try to change the processor architecture or the operating system. Now you may need to build your own toolchain using Crosstool-NG.In this tutorial, I am working at Step 2. by trying to load a pre-compiled QNX Toolchain into Buildroot .

Import QNX Toolchain into Buildroot

Clone (my fork of) the Buildroot Repository and access the Buildroot > Toolchain Menu Config:

The commented code was needed to fork the original Buildroot repo. This is a requirement of the Buildroot Internal Tree build-flow.

# Using Bash Shell for this Tutorial
$ /bin/bash
# Source the QNX Software Development Platform 
$ source ~/qnx800/qnxsdp-env.sh
# Create a separate Buildroot repo for QNX study
$ git clone git@gitlab.com:dylan_bespalko_work/buildroot-qnx.git
# This rest of this tutorial will assume this path
$ cd buildroot-qnx
# Fork Buildroot repo to apply QNX specific code
# $ git remote add upstream git@gitlab.com:buildroot.org/buildroot.git 
# Pull from the Buildroot repo
# $ git pull upstream
# Use a version that is close to QNX 8.0.0
# $ git checkout 2023.02
# Create a branch to add QNX stuff
$ git switch -c 2023.02-qnx
# Search for a simple defconfig file
$ ls configs/qemu_aarch64_virt_defconfig   
# Target defconfig a QEMU aarch64 machine
$ make qemu_aarch64_virt_defconfig     
# Test that Buildroot works before applying changes
$ make

Disable the Bootloader, Kernel, and Init System

Skip this step using:

git apply buildroot_patches/0001-disable-the-bootloader-kernel-and-init-system.patch
make qemu_aarch64_virt_defconfig

In Buildroot there are two types of config files:

defconfig only lists the config settings that are different from the default.
config contains an exhaustive list of all settings and their values.

Here we loaded a defconfig that configures Buildroot to build a Linux Virtual Machine that runs inside the QEMU emulation tool.After exiting the menuconfig there will be an updated .config file that contains all of the settings.

You can use the menuconfig in different ways:

Interactively navigate the settings using the arrow keys, enter and esc keys.
Search for an exact setting by pressing "/", then select the (#) entry that you want navigate to.

Press "?" on any "Buildroot Location" to see the "Buildroot Variable" in the config file.

We are only using Buildroot to build the root filesystem, so disable the bootloader, kernel, and busybox build as follows.

$ make menuconfig

Buildroot Location	Buildroot Variable	QNX Value	Description
Toolchain type > Toolchain type (External toolchain)	BR2_TOOLCHAIN_EXTERNAL	y	QNX is providing the cross-compile toolchain
Bootloaders > U-Boot	BR2_TARGET_UBOOT	n	Build "Das U-Boot" Boot Monitor
Kernel > Linux Kernel	BR2_LINUX_KERNEL	n	Build a Linux kernel your embedded device
System configuration > Init system (None)	BR2_INIT_NONE	y	INIT system (busybox)
Target Packages > Busybox	BR2_PACKAGE_BUSYBOX	n	The Swiss Army Knife of embedded Linux
System configuration > /bin/sh (none)	BR2_SYSTEM_BIN_SH_NONE	y	Select which shell will provide /bin/sh
System configuration > Purge unwanted locales	BR2_ENABLE_LOCALE_PURGE	n	If N, then all locales supported by packages are installed.
Filesystem images > ext2/3/ 4 root filesystem	BR2_TARGET_ROOTFS_EXT2	y	Build an ext2/3/ 4 root filesystem
Filesystem images > ext2/3/ 4 variant (ext2 (rev1))	BR2_TARGET_ROOTFS_EXT2_2r1	y	Build an ext2 v.1 root filesystem

Verify that the build runs and produces no target packages:

$ make clean
$ make
$ ls output/build | grep -v '^host' | grep -v '^toolchain' | grep -v '^buildroot' | grep -v '^skeleton' | grep -v '^packages-file-list' | grep -v '^ifupdown'
build-time.log

Configure the Toolchain using menuconfig

Skip this step using:

git apply buildroot_patches/0002-configure-the-toolchain-using-menuconfig.patch
make qemu_aarch64_virt_defconfig

Specify the following toolchain settings:

make menuconfig

Buildroot Location	Buildroot Variable	QNX Value	Description
Toolchain type > Toolchain (Custom Toolchain)	BR2_TOOLCHAIN_EXTERNAL_CUSTOM	y	Custom Toolchain
Toolchain type > Toolchain origin (Pre-installed toolchain)	BR2_TOOLCHAIN_EXTERNAL_PREINSTALLED	Pre-installed toolchain	The toolchain is on the local filesystem
Toolchain type > Toolchain path ($(QNX_HOST)/usr)	BR2_TOOLCHAIN_EXTERNAL_PATH	$(QNX_HOST)/usr	Path to where the external toolchain is installed.
Toolchain type > Toolchain prefix ($(ARCH)-unknown-nto-qnx8.0.0)	BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX	$(ARCH)-unknown-nto-qnx8.0.0	Cross-compile toolchain filename prefix formated as <arch>-<vendor>-<os>-<abi>
Toolchain type > External toolchain gcc version (12.x)	BR2_TOOLCHAIN_EXTERNAL_GCC_12	y	Version of the <arch>-<vendor>-<os>-<abi>-gcc compiler
Toolchain type > External toolchain kernel headers series (4.10.x)	BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_10	y	N/A: Not used
Toolchain type > External toolchain C library (glibc)	BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC	y	glibc should have the widest software support, but creates the largest binaries
Toolchain type > Toolchain has SSP support?	BR2_TOOLCHAIN_EXTERNAL_HAS_SSP	y	Stack-smashing support, Buildroot will complain if you set this wrong
Toolchain type > Toolchain has SSP string support?	BR2_TOOLCHAIN_EXTERNAL_HAS_SSP_STRONG	y	Stack-smashing string support, Buildroot will complain if you set this wrong
Toolchain type > Toolchain has C++ support?	BR2_TOOLCHAIN_EXTERNAL_CXX	y	Has C++ support. Note Buildroot compiles C++ code using gcc, not g++

All remaining toolchain settings are left blank. The build will independently verify some settings such as SSP and RPC support.

Verify that build fails on the toolchain-external-custom Configuring step:

$ make clean
$ make
toolchain-external-custom  Configuring
Unable to detect the toolchain sysroot, Buildroot cannot use this toolchain.

Help Buildroot Find libc.a

Skip this step using:

git apply buildroot_patches/0003-help-buildroot-find-libc.patch

There are 2 different issues that were solved in the `toolchain/helpers.mk` file:

You must specify the `--sysroot` when using `aarch64-unknown-nto-qnx8.0.0-gcc` to search for `libc.a` or `libstdc++.a`

sed -i 's&-print-file-name&--sysroot=${QNX_TARGET}/aarch64le -print-file-name&g' toolchain/helpers.mk

2. We need to print the arguments passed to `copy_toolchain_sysroot()` as it is important to the next step

echo "SYSROOT_DIR: $(strip $1)"; \
echo "ARCH_SYSROOT_DIR: $(strip $2)"; \
echo "ARCH_SUBDIR: $(strip $3)"; \
echo "ARCH_LIB_DIR: $(strip $4)"; \
echo "SUPPORT_LIB_DIR: $(strip $5)"; \

Verify that build fails on the toolchain-external-custom Configuring step:

$ make clean
$ make
<command-line>: fatal error: /home/dylan_bespalko/repos/buildroot-qnx/libc.a/usr/include/linux/version.h: No such file or directory
compilation terminated.

Copy the QNX_TARGET directory into the Buildroot STAGING_DIR Sysroot

Skip this step using:

git apply buildroot_patches/0004-copy-the-QNX_TARGET-directory-into-the-Buildroot-STAGING_DIR-sysroot.patch

There are 3 different issues that were solved in the `toolchain/toolchain-external/pkg-toolchain-external.mk` file:

As before, you must specify the `--sysroot` when using `aarch64-unknown-nto-qnx8.0.0-gcc` to search for `libc.a` or `libstdc++.a`

sed -i 's&-print-file-name&--sysroot=${QNX_TARGET}/aarch64le -print-file-name&g' toolchain/toolchain-external/pkg-toolchain-external.mk

2. The QNX sysroot supports multiple architectures (aarch64le, x86_64). Additional care was needed to set `ARCH_SYSROOT_DIR`

sed -i 's/ARCH_SYSROOT_DIR="$(call toolchain_find_sysroot/ARCH_SYSROOT_DIR="$(call toolchain_find_arch_sysroot/g' toolchain/toolchain-external/pkg-toolchain-external.mk

Then, replace the `toolchain_find_sysroot` function with the following code:

define toolchain_find_sysroot
$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:(aarch64le/)?(usr/)?lib(32|64)?([^/]*)?/([^/]*/)?libc\.a::')
endef

define toolchain_find_arch_sysroot
$$(printf $(call toolchain_find_libc_a,$(1)) | sed -r -e 's:(usr/)?lib(32|64)?([^/]*)?/([^/]*/)?libc\.a::')
endef

3. Comment out `check_kernel_headers_version` as it will search for the `linux/version.h` file, which is specific to Linux.

# $$(call check_kernel_headers_version,\
#       $$(BUILD_DIR),\
#       $$(call toolchain_find_sysroot,$$(TOOLCHAIN_EXTERNAL_CC)),\
#       $$(call qstrip,$$(BR2_TOOLCHAIN_HEADERS_AT_LEAST)),\
#       $$(if $$(BR2_TOOLCHAIN_EXTERNAL_CUSTOM),loose,strict)); \

Verify that build fails on the toolchain-external-custom Configuring step:

$ make clean
$ make
$ make toolchain-external-custom-reconfigure

If this patch is applied correctly, the arguments passed to `copy_toolchain_sysroot()` are:

1. SYSROOT_DIR: ${QNX_TARGET}
2. ARCH_SYSROOT_DIR: ${QNX_TARGET}/aarch64le/
3. ARCH_SUBDIR: aarch64le
4. ARCH_LIB_DIR: lib
5. SUPPORT_LIB_DIR:

Modify the Buildroot External Custom Toolchain Recipe

Skip this step using:

git apply buildroot_patches/0005-modify-the-buildroot-external-custom-toolchain-recipe.patch

Let's define the Custom Toolchain Buildroot recipe `toolchain/toolchain-external/toolchain-external-custom/toolchain-external-custom.mk`as follows:

################################################################################
#
# toolchain-external-custom
#
################################################################################

TOOLCHAIN_EXTERNAL_CUSTOM_SITE = $(patsubst %/,%,$(dir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL))))
TOOLCHAIN_EXTERNAL_CUSTOM_SOURCE = $(notdir $(call qstrip,$(BR2_TOOLCHAIN_EXTERNAL_URL)))

ifeq ($(BR2_TOOLCHAIN_EXTERNAL_CUSTOM),y)
# We can't check hashes for custom downloaded toolchains
BR_NO_CHECK_HASH_FOR += $(TOOLCHAIN_EXTERNAL_SOURCE)
ifeq ($(BR_BUILDING)$(BR2_TOOLCHAIN_EXTERNAL_DOWNLOAD),yy)
ifeq ($(TOOLCHAIN_EXTERNAL_CUSTOM_SOURCE),)
$(error No external toolchain url set, check your BR2_TOOLCHAIN_EXTERNAL_URL setting)
endif
endif
TOOLCHAIN_EXTERNAL_CUSTOM_TOOLCHAIN_WRAPPER_ARGS += '-DBR_ARCH="armv8-a"'
TOOLCHAIN_EXTERNAL_CUSTOM_TOOLCHAIN_WRAPPER_ARGS += '-DBR_QNX_HOST="$(QNX_HOST)"'
TOOLCHAIN_EXTERNAL_CUSTOM_TOOLCHAIN_WRAPPER_ARGS += '-DBR_QNX_STAGING="$(STAGING_DIR)"'
TOOLCHAIN_EXTERNAL_CUSTOM_TOOLCHAIN_WRAPPER_ARGS += '-DBR_QNX_TARGET="$(TARGET_DIR)"'
TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR = "qnx_patches"
endif

$(eval $(toolchain-external-package))

This code does the following:

add `#define BR_ARCH="armv8-a"` to `toolchain/toolchain-wrapper.c`.
add `#define BR_QNX_HOST="$(QNX_HOST)"` to `toolchain/toolchain-wrapper.c`.
add `#define BR_QNX_STAGING="$(STAGING_DIR)"` to `toolchain/toolchain-wrapper.c`.
add `#define BR_QNX_TARGET="$(TARGET_DIR)"` to `toolchain/toolchain-wrapper.c`.
Define a Buildroot Variable `TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR` that stores the QNX-specific patch files for each package.

Let's review what these folders mean:

TOOLCHAIN VAR	BUILDROOT VAR	QNX VAR	Description
BR_QNX_HOST	HOST_DIR	<QNX_HOST>	Source / Binaries that build and run on the Host Machine
QNX_STAGING	STAGING_DIR	<copy of QNX_TARGET>	Build-time Source / Binaries of the Target machine
QNX_TARGET	TARGET_DIR	<empty>	Run-time Binaries of the Target machine

Since QNX does not define a `${QNX_STAGING_DIR}`, build-time and run-time dependencies are copied into the Buildroot `${STAGING_DIR}`. This should be ok.

Prepend Arguments to the Toolchain Wrapper

Skip this step using:

git apply buildroot_patches/0006-prepend-agurments-to-the-toolchain-wrapper.patch

The `toolchain/toolchain-wrapper.mk` Makefile will build `toolchain/toolchain-wrapper.c` and install a gcc toolchain wrapperlocated at `output/build/toolchain-external-custom/toolchain-wrapper`. The toolchain-wrapper simply calls `gcc` with apredefined list of arguments. It is important to note that Buildroot always calls `gcc` when:

Compiling a C program (using `gcc -c`)
Compiling a C++ program (using `gcc -c -xc++`, not `g++ -c`)
Compiling Assembly code (using `gcc -Wa`, not `as`)
Linking Object code (using `gcc -Wl`, not `ld`)
Archiving binaries (using `gcc -Wl`, not `ar`)

Internally, `gcc` is forwarding linker arguments (`Wl`) and assembly arguments (`Wa`) to the correct toolchain binary, but Buildroot only calls `gcc`. The QNX `gcc` build arguments can be extracted from the`${QNX_HOST}/etc/qcc/gcc/12.2.0/gcc_ntoaarch64le.conf`file. We will use the GCC Nutrino release build settings for the64-bit ARM Little Endian processor (`gcc_ntoaarch64le`) located at the top of this file. The remaining settings in thisfile are for the debug, profile, code coverage, memory testing and would contain incremental changes that we could addlater.

--- a/toolchain/toolchain-wrapper.c
+++ b/toolchain/toolchain-wrapper.c
@@ -98,6 +98,52 @@ static char *predef_args[] = {
 #if defined(BR_MIPS_TARGET_BIG_ENDIAN) || defined(BR_ARC_TARGET_BIG_ENDIAN)
        "-EB",
 #endif
+#if defined(BR_QNX_HOST) && defined(BR_QNX_STAGING)
+// C++_COMPILER
+//"-nostdinc++",  // Cannot pass C++ argmuents when building C package
+//"-std=gnu++17", // Cannot pass C++ argmuents when building C package 
+"-isystem" BR_QNX_STAGING "/usr/include",
+"-isystem" BR_QNX_HOST "/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0/include",
+// C_COMPILER
+"-nostdinc",
+"-D__LANGUAGE_C",
+"-D_LANGUAGE_C",
+"-D__QNX__=800",
+"-D__QNXNTO__",
+"-D__GNUC__=12",
+"-D__GNUC_MINOR__=2",
+"-D__GNUC_PATCHLEVEL__=0",
+"-D__unix__",
+"-D__unix",
+"-D__ELF__",
+"-D__LITTLEENDIAN__",
+"-Asystem=unix",
+"-iplugindir=" BR_QNX_HOST "/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0/plugin",
+// LINKER  // Order of Object Files DOES matter. It should be ld -o output_file crti.o crtbegin.o … -lgcc crtend.o crtn.o
+"-Wl,--dynamic-linker=" BR_QNX_STAGING "/aarch64le/usr/lib/ldqnx-64.so.2",
+"-Wl,--hash-style=gnu",
+"-Wl,--warn-shared-textrel",
+//"-Wl,--eh-frame-hdr",  // Does not work (emulation specific options)
+//BR_QNX_STAGING "/aarch64le/lib/crt1.o",  // Order of Object Files DOES matter.
+//BR_QNX_STAGING "/aarch64le/lib/crti.o",  // Order of Object Files DOES matter.
+//BR_QNX_HOST "/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0//crtbegin.o",
+"-Wl,-rpath-link=" BR_QNX_STAGING "/aarch64le/lib:" BR_QNX_STAGING "/aarch64le/usr/lib:" BR_QNX_STAGING "/aarch64le/lib/gcc/12.2.0:" BR_QNX_STAGING "/aarch64le/opt/lib",
+"-Wl,-Y" BR_QNX_STAGING "aarch64le/lib:" BR_QNX_STAGING "aarch64le/usr/lib:" BR_QNX_STAGING "aarch64le/opt/lib",
+"-L" BR_QNX_STAGING "/usr/lib",
+"-L" BR_QNX_HOST "/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0",
+"-L" BR_QNX_STAGING "/aarch64le/lib",
+"-L" BR_QNX_STAGING "/aarch64le/lib/gcc/12.2.0",
+"-L" BR_QNX_STAGING "/usr/aarch64-unknown-nto-qnx8.0.0/lib",
+"-L" BR_QNX_STAGING "/aarch64le/usr/lib",
+"-L" BR_QNX_STAGING "/aarch64le/opt/lib",
+//"%(Bstatic:-Wl,-lgcc)%(!Bstatic:-Wl,-lgcc_s)",  // Buildroot packages should specify the libraries they need to link, not the toolchain
+//"-Wl,-lc", // Buildroot packages should specify the libraries they need to link, not the toolchain
+//"-Wl,-lgcc_eh",  // Buildroot packages should specify the libraries they need to link, not the toolchain
+//BR_QNX_HOST "/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0//crtend.o",  // Order of Object Files DOES matter.
+//BR_QNX_STAGING "/aarch64le/lib/crtn.o",  // Order of Object Files DOES matter.
+// ASSEMBLER
+"-Wa,-EL",
+#endif //BR_QNX_HOST && BR_QNX_STAGING
 #ifdef BR_ADDITIONAL_CFLAGS
        BR_ADDITIONAL_CFLAGS
 #endif
@@ -245,7 +291,7 @@ int main(int argc, char **argv)
        char *progpath = argv[0];
        char *basename;
        char *env_debug;
-       int ret, i, count = 0, debug = 0, found_shared = 0;
+       int ret, i, count = 0, debug = 2, found_shared = 0;
 
        /* Debug the wrapper to see arguments it was called with.
         * If environment variable BR2_DEBUG_WRAPPER is:

Some of the arguments have been commented out for the following reasons:

Cannot pass C++ arguments when building C package. This means that we must add the C++ Compiler arguments using per-package patches.
Order of Object Files DOES matter. It should be `ld -o output_file crti.o crtbegin.o <your-package-objects.o> -lgcc crtend.o crtn.o`.
Buildroot packages should specify the libraries they need to link, not the toolchain .

I had problems with the `--eh-frame-hdr` argument and I don't think it is required. If you set `debug = 2`, each time you invoke the toolchain-wrapper, the input argumentswill be passed on separate lines for improved readability.

Verify the toolchain-wrapper by building and calling the toolchain-wrapper.

$ make toolchain-external-custom-reconfigure
$ /home/dylan_bespalko/repos/buildroot/output/build/toolchain-external-custom/toolchain-wrapper
Toolchain wrapper was called with:
    '/home/dylan_bespalko/repos/buildroot/output/build/toolchain-external-custom/toolchain-wrapper'
Toolchain wrapper executing:
    '/home/dylan_bespalko/qnx800/host/linux/x86_64/usr/bin/toolchain-wrapper'
    '--sysroot'
    '/home/dylan_bespalko/repos/buildroot/output/build/aarch64-buildroot-linux-gnu/sysroot'
    '-mabi=lp64'
    '-isystem/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/include'
    '-isystem/home/dylan_bespalko/qnx800/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0/include'
    '-nostdinc'
    '-D__LANGUAGE_C'
    '-D_LANGUAGE_C'
    '-D__QNX__=800'
    '-D__QNXNTO__'
    '-D__GNUC__=12'
    '-D__GNUC_MINOR__=2'
    '-D__GNUC_PATCHLEVEL__=0'
    '-D__unix__'
    '-D__unix'
    '-D__ELF__'
    '-D__LITTLEENDIAN__'
    '-Asystem=unix'
    '-iplugindir=/home/dylan_bespalko/qnx800/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0/plugin'
    '-Wl,--dynamic-linker=/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/usr/lib/ldqnx-64.so.2'
    '-Wl,--hash-style=gnu'
    '-Wl,--warn-shared-textrel'
    '-Wl,-rpath-link=/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/lib:/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/usr/lib:/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/lib/gcc/12.2.0:/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/opt/lib'
    '-Wl,-Y/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysrootaarch64le/lib:/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysrootaarch64le/usr/lib:/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysrootaarch64le/opt/lib'
    '-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/lib'
    '-L/home/dylan_bespalko/qnx800/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx8.0.0/12.2.0'
    '-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/lib'
    '-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/lib/gcc/12.2.0'
    '-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/usr/aarch64-unknown-nto-qnx8.0.0/lib-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/usr/lib-L/home/dylan_bespalko/repos/buildroot/output/host/aarch64-buildroot-linux-gnu/sysroot/aarch64le/opt/lib'
    '-Wa,-EL'
    '-Wl,-z,max-page-size=4096'
    '-Wl,-z,common-page-size=4096'
    '-fstack-protector-strong'
    '-march=armv8-a'
    '-mcpu=cortex-a53'
    '-fPIE'
    '-pie'
    '-Wl,-z,now'
    '-Wl,-z,relro'

Congratulations you have implemented the QNX Buildroot Toolchain.

Build Buildroot Packages

Make sure buildroot can run to completion without building any packages for the target machine. Also make sure you havea folder called `qnx_pathes` that contains package-specific patch files.

make clean
make
ls -l qnx_patches
drwxrwxr-x 2 dylan_bespalko dylan_bespalko 4096 Jul  2 19:06 boost
drwxrwxr-x 2 dylan_bespalko dylan_bespalko 4096 Jul  2 19:05 ed
drwxrwxr-x 2 dylan_bespalko dylan_bespalko 4096 Jul  2 19:05 qt6base
drwxrwxr-x 2 dylan_bespalko dylan_bespalko 4096 Jul  2 19:06 zsh

Build ed (generic-package written in C)

Skip this step using:

git apply buildroot_patches/0007-add-ed-post-patches.patch

Use `make menuconfig` to select the following options:

Buildroot Location	Buildroot Variable	QNX Value	Description
Target Packages > Text editors and viewers > ed	BR2_PACKAGE_ED	y	A line-oriented text editor

Add the post-patch hook script to `package/ed/ed.mk` as follows:

define ED_POST_QNX_PATCH
        $(APPLY_PATCHES) $(@D) $(TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR)/ed \*.patch
endef
ED_POST_PATCH_HOOKS += ED_POST_QNX_PATCH

Then build the package:

# make ed
make ed-source
make ed-extract
make ed-patch  # QNX-specific patches are found in qnx-patches/ed
make ed-configure
make ed-build
make ed-install

Build zsh (autotools-package written in C)

Skip this step using:

git apply buildroot_patches/0008-add-zsh-post-patches.patch

Use `make menuconfig` to select the following options:

Buildroot Location	Buildroot Variable	QNX Value	Description
Target Packages > Shell and Utilities > zsh	BR2_PACKAGE_ZSH	y	zsh is a shell designed for interactive use, although it is also a powerful scripting language

Add the post-patch hook script to `package/zsh/zsh.mk` as follows:

define ZSH_POST_QNX_PATCH
        $(APPLY_PATCHES) $(@D) $(TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR)/zsh \*.patch
endef
ZSH_POST_PATCH_HOOKS += ZSH_POST_QNX_PATCH

Then build the package:

# make zsh
make zsh-source
make zsh-extract
make zsh-patch  # QNX-specific patches are found in qnx-patches/zsh
make zsh-configure
make zsh-build
make zsh-install

Build boost (generic-package written in C++)

Skip this step using:

git apply buildroot_patches/0009-add-boost-post-patches.patch

Use `make menuconfig` to select the following options:

Buildroot Location	Buildroot Variable	QNX Value	Description
Target Packages > Libraries > Other > boost	BR2_PACKAGE_BOOST	y	A general purpose C++ library
Target Packages > Libraries > Other > boost-system	BR2_PACKAGE_BOOST_SYSTEM	y	Operating system support, including the diagnostics support that will be part of the C++0x standard library

Add the post-patch hook script to `package/boost/boost.mk` as follows:

BOOST_OPTS += target-os=qnx
define BOOST_POST_QNX_PATCH
        $(APPLY_PATCHES) $(@D) $(TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR)/boost \*.patch
endef
BOOST_POST_PATCH_HOOKS += BOOST_POST_QNX_PATCH

Then build the package:

# make boost
make boost-source
make boost-extract
make boost-patch  # QNX-specific patches are found in qnx-patches/boost
make boost-configure
make boost-build
make boost-install

Build qt6base (cmake-package written in C++)

Skip this step using:

git apply buildroot_patches/0010-add-qt6base-post-patches.patch

Use `make menuconfig` to select the following options:

Buildroot Location	Buildroot Variable	QNX Value	Description
Target Packages > Graphic libraries and applications (graphic/text) > Qt6 > qt6base	BR2_PACKAGE_QT6BASE	y	Qt is a cross-platform application and UI framework for developers using C++

Add the post-patch hook script to `package/qt6/qt6base/qt6base.mk` as follows:

define QT6BASE_POST_QNX_PATCH
        $(APPLY_PATCHES) $(@D) $(TOOLCHAIN_EXTERNAL_CUSTOM_PATCH_DIR)/qt6base \*.patch
endef
QT6BASE_POST_PATCH_HOOKS += QT6BASE_POST_QNX_PATCH

Then build the package:

# make qt6base
make qt6base-source
make qt6base-extract
make qt6base-patch  # QNX-specific patches are found in qnx-patches/qt6base
make qt6base-configure
make qt6base-build
make qt6base-install

Why are Patches Needed to Fix the Source Code?

Since the Buildroot toolchain-wrapper does not support C++ arguments, I would expect all C++ dependencies to require patches. The following table summarizes different reasons for the patches:

Issue	QNX Version	Packages
add-missing-link-time-dependencies.patch	8.0.0	ed
add-cpp-flags.patch	8.0.0	qt6base
replace-ldl-with-lc.patch	8.0.0	qt6base

Having this list of standardized patches helps focus the effort on the remaining non-standardized patches.

Conclusions

The In-Tree Buildroot build-flow was used to prove that UNIX applications could be ported to QNX. The following files have been modified:

$ git status
On branch 2023.02-qnx
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   configs/qemu_aarch64_virt_defconfig
        modified:   package/boost/boost.mk
        modified:   package/ed/ed.mk
        modified:   package/qt6/qt6base/qt6base.mk
        modified:   package/zsh/zsh.mk
        modified:   toolchain/helpers.mk
        modified:   toolchain/toolchain-external/pkg-toolchain-external.mk
        modified:   toolchain/toolchain-external/toolchain-external-custom/toolchain-external-custom.mk
        modified:   toolchain/toolchain-wrapper.c

Application / BSP Developers

In the Buildroot External Tree build-flow, the application/BSP developer only defines application packages and board configs.

Toolchain Changes Required to Support External Buildroot Build-flow

This leaves the contents of the toolchain directory, which must be implemented by the toolchain developer, Blackberry.Blackberry could use Crosstool-NG to modify their toolchain `aarch64-unknown-nto-qnx8.0.0-*`to produce `aarch64-buildroot-nto-qnx8.0.0-*` a toolchain variant that integrates with Buildroot.

With the Buildroot External Tree build-flow, the following design-flow is enabled:

Cross-Compiling UNIX Applications for QNX using the Buildroot Internal Build-flow

Linux vs. QNX

Cross-Compiling Tools

Import QNX Toolchain into Buildroot

Disable the Bootloader, Kernel, and Init System

Configure the Toolchain using menuconfig

Help Buildroot Find libc.a

Copy the QNX_TARGET directory into the Buildroot STAGING_DIR Sysroot

Modify the Buildroot External Custom Toolchain Recipe

Prepend Arguments to the Toolchain Wrapper

Build Buildroot Packages

Build ed (generic-package written in C)

Build zsh (autotools-package written in C)

Build boost (generic-package written in C++)

Build qt6base (cmake-package written in C++)

Why are Patches Needed to Fix the Source Code?

Conclusions

Application / BSP Developers

Toolchain Changes Required to Support External Buildroot Build-flow

Recent Posts

Comments