Comfortable kernel workflow on Beagleboard XM with nfsroot

Overview 

For the last couple of days, I wanted to concentrate on reducing time required to boot our robot. The first action I did was to find out minimum configuration for kernel. After couple of rounds of getting SD card between computer and Beagleboard XM, which is the base for our robot, I decided to check how other people are developing kernel on embedded systems. The standard approach is to boot kernel from network.

Fortunately, the hard part was already done: u-boot support for network chip is already implemented. Here I wanted to document steps, required to get it working. Most of the information is available somewhere on the Internet, here is just place to get it all together.

Unfortunately, I could not configure DHCP server to get required parameters for TFTP boot, so...

The mission: u-boot loads kernel from nfs and kernel mounts nfs root.

Assumptions 

Network configuration: I will be using static IP addresses to minimize influence of "bad" DHPC server. It will be good for many additional reasons. The beagleboard connected with ethernet cable to switch :)

beagleboard 192.168.0.9
server           192.168.0.2
router           192.168.0.1

The configuration is done on Debian stable (6.0), but should work on Ubuntu as well without major changes.

Preparation

Create user

In order to isolate work on this project, I created new user on the system and granted corresponding rights:

$ sudo adduser maker
$ sudo vigr

with the last step, add user maker to sudo group, as we need to use sudo pretty often.

Setup networking

as mentioned above, we are using static addressing, so first add beagleboard to /etc/hosts

$ sudo vi /etc/hosts

and add following line
192.168.0.9 beagleboard

in order to minimize hassle with regenerated rsa_key for ssh, add following to your ssh_config

$ sudo vi /etc/ssh/ssh_config

and add following
Host 192.168.0.9 beagleboard
  CheckHostIP no
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

then we need to grant permission for nfs clients, for easy configuration, just allow everyone. Obviously, if the server is used in some sort of public environment, this should correspondingly changed.

$ sudo vi /etc/hosts.allow

and add following
ALL: LOCAL

now we are ready to install nfs server, if it's not installed yet

$ sudo apt-get install nfs-kernel-server

Create local folder for remote root file system and export this with nfs server

$ mkdir ~/rootfs
$ sudo vi /etc/exports

add following line
/home/maker/rootfs  beagleboard(rw,fsid=0,no_root_squash,no_subtree_check)

and restart nfs-server
$ sudo /etc/init.d/nfs-kernel-server restart

check if export is ok
$ sudo exportfs

Setup toolchain

While, obviously, the toolchain could be compiled from sources, there is no need, as Angstrom distribution provides well tested and configured toolchain. There are plenty of versions, but I tested with Angstrom next toolchain

As I'm on x86_64, the file is angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2

download the file, extract and remove unnecessary structure

$ wget http://www.angstrom-distribution.org/toolchains/next/angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2
$ tar xjf angstrom-v20110117-x86_64-linux-armv7a-linux-gnueabi-toolchain.tar.bz2
$ mv usr/local/angstrom/arm .
$ rm -rf var usr

The installation of Angstrom toolchain, theoretically, should be done in root folder (/), but I don't like to have something in the root, which belongs to one user only. Thus, installation in the HOME folder.
Now, let's add some useful aliases and setup PATH correspondingly

$ vi ~/.bashrc

and add following to the end
export PATH=$PATH:${HOME}/arm/bin
alias cross='ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- CFLAGS=" -march=armv7-a -fno-tree-vectorize -mthumb-interwork -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 " LOCALVERSION="" '
alias sudo_cross='sudo ARCH=arm CROSS_COMPILE=arm-angstrom-linux-gnueabi- LOCALVERSION="" INSTALL_MOD_PATH=${HOME}/rootfs'

Pick the changes with . ~/.bashrc or logout and loging again.

Check, that cross compiler is available "arm-angstrom-linux-gnueabi-gcc -v".

Extract root filesystem

In order to boot from nfs root, we need a root file system. The easiest is to build required file system with Angstrom Narcissus or download demo root system from Angstrom beagleboard demo images . Required MLO and u-boot could be taken from there as well. The u-boot version already contains required code to boot from nfs. So, downloading following files:

$ wget http://www.angstrom-distribution.org/demo/beagleboard/Angstrom-Beagleboard-demo-image-glibc-ipk-2011.1-beagleboard.rootfs.tar.bz2
$ wget http://www.angstrom-distribution.org/demo/beagleboard/MLO
$ wget http://www.angstrom-distribution.org/demo/beagleboard/u-boot.img

note: we don't need uImage as we will be building our own kernel

now, extract root file system in ~/rootfs

$ sudo tar -C ~/rootfs/ -xjf Angstrom-Beagleboard-demo-image-glibc-ipk-2011.1-beagleboard.rootfs.tar.bz2

Prepare kernel sources 

I don't put the link to kernel sources, as everyone knows where they are.  We will be compiling 2.6.38.8 as this is the last version which is supported with latest Xenomai. Additionally, as example, we will be patching source with xenomai 2.6.0 patch and we need working defconfig, which could be taken from our source code at Veter project on Github. Download following files:

$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.38.8.tar.bz2
$ wget https://github.com/veter-team/veterlayer/raw/master/recipes/linux/linux-xenomai-arm-2.6.38.8/defconfig
$ wget https://github.com/veter-team/veterlayer/raw/master/recipes/linux/linux-xenomai-arm-2.6.38.8/linux-xenomai-arm-2.6.38.8.patch

now, let's unpack sources, setup privileges and patch files, with optional git in between

$ cd ~/rootfs/usr
$ sudo mkdir src
$ sudo chown maker src
$ cd src
$ tar xjf ~/linux-2.6.38.8.tar.bz2
$ ln -s linux-2.6.38.8 linux
$ cd linux
$ git init 
$ git add .
$ git commit -m "vanilla linux 2.6.38.8"
$ patch -p1 <~/linux-xenomai-arm-2.6.38.8.patch
$ cp ~/defconfig ./.config
$ git add .
$ git commit -m "xenomai 2.6.0 for arm"

Compiling initial kernel

now we have everything ready to cross-compile kernel for arm:

$ cross make -j16 uImage modules
$ sudo_cross make -j16 modules_install

Time for coffee !

let's fix some links to get it right from the beagleboard perspective

$ cd ~/rootfs/lib/modules/2.6.38.8
$ sudo ln -s ../../../usr/src/linux build
$ sudo ln -s ../../../usr/src/linux source
$ cd ~/rootfs/boot
$ sudo rm uImage
$ sudo ln -s ../usr/src/linux/arch/arm/boot/uImage .

For the above-mentioned image, which is still based on 2.6.32 kernel, we need to change console

$ sudo vi ~/rootfs/etc/inittab

change console line from ttyS2 to
S:2345:respawn:/sbin/getty 115200 ttyO2

and remove NetworkManager. Its "smart" logic is not smart enough to deal with nfsroot

$ sudo rm ~/rootfs/etc/rc5.d/S28NetworkManager

Prepare card

The micro SD card need to have only 1 partition. The exercise of creating partitions and formatting with fat is left for reader. Then copy MLO and u-boot.img to the root of the card. And the most important create uEnv.txt file with instructions as to where to look for a kernel and nfs file system. Assuming card is mounted on /media/boot/

$ cp ~/MLO /media/boot/
$ sync
$ cp ~/u-boot.img /media/boot/
$ vi /media/boot/uEnv.txt

and copy this content

serverip=192.168.0.2
ipaddr=192.168.0.9
netmask=255.255.255.0
gatewayip=192.168.0.1
hostname=beagleboard
console=ttyO2,115200n8
optargs=mem=80M@0x80000000 mem=384M@0x88000000 xeno_hal.cpufreq=800000000
mpurate=800
root=/dev/nfs rw
rootpath=/home/maker/rootfs
nfsdevice=usb0:off
nfsrootargs=vers=3
usbethaddr=6E:58:95:1B:82:65
nfsargs=setenv bootargs console=${console} mpurate=${mpurate} ${optargs} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}:${nfsdevice} nfsroot=${serverip}:${rootpath},${nfsrootargs} root=${root} 
uenvcmd=echo Initializing net ...; dcache off; usb start; echo Loading kernel ...; nfs ${loadaddr} ${serverip}:${rootpath}/boot/uImage ; echo Booting from nfs ...; run nfsargs; bootm ${loadaddr}

The content is self-explanatory. Now we are ready for a big moment !

Booting

now, unmount card, insert it into beagle and enjoy you kernel from nfsroot !!!

After this moment, all files are on the server, there is almost nothing on the beagleboard itself and you could just press the reset on board at any moment, the content is save on nfs.

Rebuilding kernel on server

Obviously, you want to change the kernel configuration and rebuild kernel couple of time, otherwise, all this effort is for nothing. Having set the links above, it's super easy, almost as it would be on the native system

$ cd ~/rootfs/usr/src/linux
$ cross make mrproper
$ cross make clean
$ cross make menuconfig
$ cross make -j16 uImage modules
$ sudo_cross make -j16 modules_install

And press reset on the beagleboard.

Building modules on beagleboard

After you made you perfect kernel, you may want to develop some kernel modules. Building them with cross-compiler is possible, but not very effective. On the beagleboard you have already tree compiled exactly for your version of the kernel. Why don't use them and compile modules natively ?

Sure you can do this. One thing you should just take care of is scripts in the kernel source. They were compiled on the server with x86_64 architecture and they don't work on the beagleboard. You just need to rebuild scripts.

Caution: now as a root on beagleboard !

root@beagleboard $ cd /usr/src/linux
root@beagleboard $ make scripts
root@beagleboard $ mkdir ../test
root@beagleboard $ cd ../test
root@beagleboard $ vi Makefile

and put this content in the Makefile
obj-m += hello.o
 
all:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) modules
 
clean:
 make -C /lib/modules/$(shell uname -r)/build M=$(PWD) clean

root@beagleboard $ vi hello.c

and create simple module
#include 
#include 
 
int init_module(void)
{
 printk(KERN_INFO "init_module() called\n");
 return 0;
}
 
void cleanup_module(void)
{
 printk(KERN_INFO "cleanup_module() called\n");
}

root@beagleboard $ make
root@beagleboard $ modinfo hello.ko
filename:       hello.ko
srcversion:     4F856ABA1F3290D5F81D961
depends:        
vermagic:       2.6.38.8+ preempt mod_unload modversions ARMv7 
root@beagleboard $ insmod hello.ko
root@beagleboard $ rmmod hello

Now all combination of compiling on server and beagleboard is possible, you could press reset at any time and reboot beagle, network is configured and working.

Congratulations: MISSION COMPLETED !!!

Content on this site is licensed under a Creative Commons Attribution 3.0 License .