A Technical Odyssey

In the second part we will take a look and see how to apply the RT-Preempt patches to the 3.18 kernel for BeagleBone Black. This is a continuation of a previous post where we built the 3.18 version and tested it on the BeagleBone Black.

RT Preempt Information

The instructions to apply the patches are given in the Real-Time Linux Wiki. It would be advisable to first go through the FAQs section and get a general understanding on RT Linux.

To apply the patches we go through the RT Preempt Howto which describes the general procedure. In the document the patches are applied on top of the vanilla kernel sources taken from http://kernel.org/. However we will have to apply the patches on top of Robert Nelson’s bb-kernel sources for version 3.18 which we used in the previous post.

Locating The Correct Patches

After going through the RT Preempt Howto, it becomes clear that there are different patches for different versions of the kernel source at:

https://www.kernel.org/pub/linux/kernel/projects/rt/

Figure 1. The patches for each kernel version

We cannot randomly apply any of the patches to our kernel source tree. We have to find the RT-Preempt patch version which fits the kernel verison we are using.

The exact version of our kernel can be determined from the Makefile in the kernel tree. As shown below our version is 3.18.5.

conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/KERNEL$ head Makefile
VERSION = 3
PATCHLEVEL = 18
SUBLEVEL = 5
EXTRAVERSION =
NAME = Diseased Newt

# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
# More info can be located in ./README
# Comments in this file are targeted only to the developer, do not
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/KERNEL$

When we check the patches available in the 3.18 folder of the rt-preempt patches location we see the latest version is at 3.18.11.

Figure 2. The latest patches for 3.18 kernel

There is a mismatch, we cannot use the 3.18.11 patches as our source code is of version 3.18.5. We take a look at the older patches in the older directory. The oldest version available is 3.18.7.

Figure 3. The latest patches for 3.18 kernel

Searching For A Solution

At this point we have to search for a solution as it looks like the rt-preempt patches are not compatible with the 3.18.5 kernel. After a bit of online browsing an answer was found on the wiki maintained by the aplication engineers at Digi-Key Corporation in the comments page for BeagleBone Black:

https://eewiki.net/display/linuxonarm/BeagleBone+Black+Comments

I got the 3.18.5-bone1 kernel working with hard real time! I simply applied the nearest available RT patch (patch-3.18.7-rt1.patch.xz), and by golly the patch had no rejects and only 4 "offset x lines" messages! And the updated kernel compiled cleanly and working right away! I really really didn’t expect that.

I didn’t bother to investigate the "offset" cases, since 3 of them were for other architectures, and the other was for raid5.c, which is a module that I don’t need.

— John S. Rhoades

Downloading And Applying The Patches

We will not download the patch used by John i.e. patch-3.18.7-rt1.patch.xz instead we will download the latest real-time patch for 3.18.7 i.e. patch-3.18.7-rt2.patch.gz.

conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Wget$ wget https://www.kernel.org/pub/linux/kernel/projects/rt/3.18/older/patch-3.18.7-rt2.patch.gz                (1)
--2015-04-15 12:52:34--  https://www.kernel.org/pub/linux/kernel/projects/rt/3.18/older/patch-3.18.7-rt2.patch.gz
Resolving www.kernel.org (www.kernel.org)... 149.20.4.69, 198.145.20.140, 199.204.44.194, ...
Connecting to www.kernel.org (www.kernel.org)|149.20.4.69|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 183993 (180K) [application/x-gzip]
Saving to: ‘patch-3.18.7-rt2.patch.gz’

100%[=============================================================================================================================================================================================================================>] 183,993     88.2KB/s   in 2.0s

2015-04-15 12:52:37 (88.2 KB/s) - ‘patch-3.18.7-rt2.patch.gz’ saved [183993/183993]

1	Downloading the patch with wget

Now we will apply the patch on top of our KERNEL as follows:

conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Wget$ cd ~/Git/bb-kernel/KERNEL/                (1)
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/KERNEL$ zcat ~/Wget/patch-3.18.7-rt2.patch.gz | patch -p1        (2)
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/KERNEL$ zcat ~/Wget/patch-3.18.7-rt2.patch.gz | patch -p1
patching file Documentation/hwlat_detector.txt
patching file Documentation/sysrq.txt
.
.
patching file arch/s390/mm/fault.c
Hunk #1 succeeded at 429 (offset -6 lines).        (3)
.
.
patching file arch/sh/mm/fault.c
Hunk #1 succeeded at 438 (offset -2 lines).        (4)
.
.
patching file arch/x86/mm/fault.c
Hunk #1 succeeded at 1133 (offset 5 lines).        (5)
.
.
patching file drivers/md/raid5.c
Hunk #3 succeeded at 5704 (offset -5 lines).        (6)
.
.
patching file net/sched/sch_generic.c
patching file scripts/mkcompile_h
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/KERNEL$

1	Change to the 3.18 kernel tree which we built in the previous part
2	Apply the patch without uncompressing before applying it
3	Ignoring as for s390
4	Ignoring as for sh
5	Ignoring as for x86
6	Raid not required

Rebuilding After Patching

So now that the patches have been applied we will see if it has broken our build by rebuilding the kernel as follows:

conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel$ ./tools/rebuild.sh
+ Detected build host [Ubuntu 14.04.2 LTS]
+ host: [i686]
+ git HEAD commit: [496f173e9148f4662257a6023435b429cfa5e69e]
-----------------------------
scripts/gcc: Using: arm-linux-gnueabi-gcc (Ubuntu/Linaro 4.7.3-12ubuntu1) 4.7.3
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-----------------------------
CROSS_COMPILE=/usr/bin/arm-linux-gnueabi-
scripts/kconfig/mconf Kconfig
.
.
.

Again we’re shown the menuconfig screen. Just exit and save the configuration and let the build proceed. The build completes successfully outputing the built binaries and archives on the console as was seen in the previous part when we built the vanilla kernel.

.
.
.
‘arch/arm/boot/zImage’ -> ‘/home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1.zImage’        (1)
‘.config’ -> ‘/home/conrad/Git/bb-kernel/deploy/config-3.18.5-rt2-bone1’
-rwxrwxr-x 1 conrad conrad 6.5M Apr 15 18:33 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1.zImage
-----------------------------
Building modules archive...
Compressing 3.18.5-rt2-bone1-modules.tar.gz...
-rw-rw-r-- 1 conrad conrad 18M Apr 15 18:33 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-modules.tar.gz
-----------------------------
Building firmware archive...
Compressing 3.18.5-rt2-bone1-firmware.tar.gz...
-rw-rw-r-- 1 conrad conrad 1.2M Apr 15 18:33 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-firmware.tar.gz
-----------------------------
Building dtbs archive...
Compressing 3.18.5-rt2-bone1-dtbs.tar.gz...
-rw-rw-r-- 1 conrad conrad 952K Apr 15 18:33 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-dtbs.tar.gz        (2)
-----------------------------
Script Complete
eewiki.net: [user@localhost:~$ export kernel_version=3.18.5-rt2-bone1]        (3)
-----------------------------
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel$

1	The zImage built
2	The device tree binaries archive
3	Version of the kernel that was built which is 3.18.5-rt2-bone1

Testing The Patched Kernel

We will follow the same steps as the part 1 section, "Testing The Kernel" and copy the 3.18.5-rt2-bone1.zImage binary and the device tree binary to the TFTP server folder to be downloaded and booted through U-Boot.

conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy$ sudo cp 3.18.5-rt2-bone1.zImage /var/lib/tftpboot/
[sudo] password for conrad:
no talloc stackframe at ../source3/param/loadparm.c:4864, leaking memory
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy$ mkdir temp        (1)
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy$ cd temp/
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy/temp$ tar xvzf ../3.18.5-rt2-bone1-dtbs.tar.gz         (2)
am335x-base0033.dtb
am335x-bone-4dcape-43.dtb
.
.
omap3-zoom3.dtb
omap3430-sdp.dtb
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy/temp$
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy/temp$ sudo cp -a am335x-boneblack.dtb /var/lib/tftpboot/am335x-boneblack.3.18.5-rt2-bone1.dtb        (3)
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy/temp$ cd ..
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy$ rm -Rf temp/                (4)
conrad@conrad-HP-Pavilion-dm3-Notebook-PC:~/Git/bb-kernel/deploy$

1	Make a temporary directory
2	Extract the archive file in the temporary directory
3	Copy the am335x-boneblack.dtb device tree binary to the TFTP download folder as am335x-boneblack.3.18.5-rt2-bone1.dtb
4	Remove the temporary directory

Booting The Patched Kernel

We now boot the kernel 3.18.5-rt2-bone1.zImage along with the device tree binary to see if it loads properly. We also print the version information to make sure it gives the right string.

.
.
U-Boot# tftpboot 0x81000000 3.18.5-rt2-bone1.zImage                        (1)
link up on port 0, speed 100, full duplex
Using cpsw device
TFTP from server 192.168.0.1; our IP address is 192.168.0.100
Filename '3.18.5-rt2-bone1.zImage'.
Load address: 0x81000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ########
         1.3 MiB/s
done
Bytes transferred = 6783096 (678078 hex)
U-Boot# tftpboot 0x82000000 am335x-boneblack.3.18.5-rt2-bone1.dtb        (2)
link up on port 0, speed 100, full duplex
Using cpsw device
TFTP from server 192.168.0.1; our IP address is 192.168.0.100
Filename 'am335x-boneblack.3.18.5-rt2-bone1.dtb'.
Load address: 0x82000000
Loading: #####
         1.2 MiB/s
done
Bytes transferred = 65155 (fe83 hex)
U-Boot# bootz 0x81000000 - 0x82000000                                        (3)
Kernel image @ 0x81000000 [ 0x000000 - 0x678078 ]
## Flattened Device Tree blob at 82000000
   Booting using the fdt blob at 0x82000000
   Using Device Tree in place at 82000000, end 82012e82

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset
.
.
Welcome to Buildroot
buildroot login: root
#
#
# uname -a
Linux buildroot 3.18.5-rt2-bone1 #2 Wed Apr 15 18:30:15 IST 2015 armv7l GNU/Linux        (4)
# [   36.134860] random: nonblocking pool is initialized
# halt 1                                (5)
The system is going down NOW!
Sent SIGTERM to all processes
Sent SIGKILL to all processes
Requesting system halt
[  773.768416] musb-hdrc musb-hdrc.1.auto: remove, state 4
[  773.773932] usb usb1: USB disconnect, device number 1
[  773.779789] musb-hdrc musb-hdrc.1.auto: USB bus 1 deregistered
[  773.786629] reboot: System halted

1	Downloading 3.18.5-rt2-bone1.zImage
2	Downloading am335x-boneblack.3.18.5-rt2-bone1.dtb
3	Booting the zImage
4	The string 3.18.5-rt2-bone1 indicates the kernel version
5	Properly halting the system

The patched kernel boots well so now we have the confidence to enable full real-time support.

Configuring Full Real-Time Support

We will have to run the ./tools/rebuild.sh script again to allow us to configure the full real-time support for the kernel. There are two options we will have to enable. We refer to the section, Configuring/compiling CONFIG_PREEMPT_RT and enable CONFIG_PREEMPT_RT_FULL (=y) and CONFIG_HIGH_RES_TIMERS (=y).

On executing the ./tools/rebuild.sh script we will again get the menuconfig screen. This time we have to enable the above values. First we search for CONFIG_PREEMPT_RT_FULL (Hit the '/' key on the menuconfig screen).

rt preempt menuconfig search config preempt rt full

Figure 4. Searching for CONFIG_PREEMPT_RT_FULL

rt preempt menuconfig search config preempt rt full 1

Figure 5. Search results for CONFIG_PREEMPT_RT_FULL. It is located at → Kernel Features → Preemptible Kernel

rt preempt menuconfig search config preempt rt full 4

Figure 6. Enabling Fully Preemptible Kernel(RT)

Secondly we search for CONFIG_HIGH_RES_TIMERS.

rt preempt menuconfig search config high res timers

Figure 7. Searching for CONFIG_HIGH_RES_TIMERS

rt preempt menuconfig search config high res timers 1

Figure 8. Search results for CONFIG_HIGH_RES_TIMERS. It is located at → General setup → Timers subsystem

rt preempt menuconfig search config high res timers 4

Figure 9. The CONFIG_HIGH_RES_TIMERS is enabled

After saving and exiting the menuconfig screen the build will proceed. The build is successful and we get similar results as the previous build in terms of output logs from the build script.

.
.
.
  LD [M]  sound/usb/misc/snd-ua101.ko
  LD [M]  sound/usb/snd-usb-audio.ko
  LD [M]  sound/usb/snd-usbmidi-lib.ko
-----------------------------
make -j2 ARCH=arm LOCALVERSION=-bone1 CROSS_COMPILE=/usr/bin/arm-linux-gnueabi- dtbs
-----------------------------
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
make[1]: `include/generated/mach-types.h' is up to date.
  CALL    scripts/checksyscalls.sh
‘arch/arm/boot/zImage’ -> ‘/home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1.zImage’
‘.config’ -> ‘/home/conrad/Git/bb-kernel/deploy/config-3.18.5-rt2-bone1’
-rwxrwxr-x 1 conrad conrad 6.7M Apr 15 21:57 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1.zImage        (1)
-----------------------------
Building modules archive...
Compressing 3.18.5-rt2-bone1-modules.tar.gz...
-rw-rw-r-- 1 conrad conrad 18M Apr 15 21:58 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-modules.tar.gz
-----------------------------
Building firmware archive...
Compressing 3.18.5-rt2-bone1-firmware.tar.gz...
-rw-rw-r-- 1 conrad conrad 1.2M Apr 15 21:58 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-firmware.tar.gz
-----------------------------
Building dtbs archive...
Compressing 3.18.5-rt2-bone1-dtbs.tar.gz...
-rw-rw-r-- 1 conrad conrad 952K Apr 15 21:58 /home/conrad/Git/bb-kernel/deploy/3.18.5-rt2-bone1-dtbs.tar.gz        (2)
-----------------------------
Script Complete
eewiki.net: [user@localhost:~$ export kernel_version=3.18.5-rt2-bone1]                (3)
-----------------------------

1	The full real-time zImage is built
2	The device tree binaries archive. This should ideally be the same everytime unless we modify the device tree source files.
3	Version of the kernel that was built which is 3.18.5-rt2-bone1

Booting The RT-Preempt Kernel

To test the kernel we copy the 3.18.5-rt2-bone1.zImage to the TFTP server folder with a slightly different name i.e. 3.18.5-rt2-bone1.rt.zImage in order to differentiate it from our previous image built without RT-Preempt support. The rest of the steps are similar to our previous kernel load.

.
.
.

U-Boot# tftpboot 0x81000000 3.18.5-rt2-bone1.rt.zImage                        (1)
link up on port 0, speed 100, full duplex
Using cpsw device
TFTP from server 192.168.0.1; our IP address is 192.168.0.100
Filename '3.18.5-rt2-bone1.rt.zImage'.
Load address: 0x81000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ######################
         1.3 MiB/s
done
Bytes transferred = 6991248 (6aad90 hex)
U-Boot# tftpboot 0x82000000 am335x-boneblack.3.18.5-rt2-bone1.dtb        (2)
link up on port 0, speed 100, full duplex
Using cpsw device
TFTP from server 192.168.0.1; our IP address is 192.168.0.100
Filename 'am335x-boneblack.3.18.5-rt2-bone1.dtb'.
Load address: 0x82000000
Loading: #####
         1.1 MiB/s
done
Bytes transferred = 65155 (fe83 hex)
U-Boot# bootz 0x81000000 - 0x82000000                                        (3)
Kernel image @ 0x81000000 [ 0x000000 - 0x6aad90 ]
## Flattened Device Tree blob at 82000000
   Booting using the fdt blob at 0x82000000
   Using Device Tree in place at 82000000, end 82012e82

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.18.5-rt2-bone1 (conrad@conrad-HP-Pavilion-dm3-Notebook-PC) (gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-12ubuntu1) ) #3 PREEMPT RT Wed Apr 15 21:54:59 IST 2015
[    0.000000] CPU: ARMv7 Processor [413fc082] revision 2 (ARMv7), cr=50c5387d
.
.
.
[    7.370106] Freeing unused kernel memory: 452K (c0977000 - c09e8000)
Starting logging: OK
Initializing random number generator... [    7.890589] random: dd urandom read with 67 bits of entropy available
done.
Starting network...
ip: RTNETLINK answers: File exists
Starting dropbear sshd: OK

Welcome to Buildroot
buildroot login: root
#
# [  206.655634] random: nonblocking pool is initialized
# dmesg | grep -ri "preempt"
[    0.000000] Linux version 3.18.5-rt2-bone1 (conrad@conrad-HP-Pavilion-dm3-Notebook-PC) (gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-12ubuntu1) ) #3 PREEMPT RT Wed Apr 15 21:54:59 IST 2015
[    0.000000] Preemptible hierarchical RCU implementation.
# uname -a
Linux buildroot 3.18.5-rt2-bone1 #3 PREEMPT RT Wed Apr 15 21:54:59 IST 2015 armv7l GNU/Linux        (4)

1	Downloading 3.18.5-rt2-bone1.rt.zImage
2	Downloading am335x-boneblack.3.18.5-rt2-bone1.dtb
3	Booting the zImage
4	The PREEMPT RT differentiates this kernel string version from the previous one.

Conclusion

We were successfully able to port RT-Preempt to the 3.18 kernel from Robert Nelson’s bb-kernel branch and boot the fully real-time kernel. We can now carry out tests with this kernel and benchmark its performance.

Porting RT Preempt To Kernel 3.18 For BeagleBone Black - Part 2

by Conrad Gomes on April 16, 2015