Dev/Embedded/Rootfs/BusyBox: Difference between revisions

From Embeded Linux (and more) Wiki by Nathael
< Dev‎ | Embedded‎ | Rootfs
Jump to navigation Jump to search
Line 76: Line 76:


Most of the time when you start with busybox as the heart of your system you do not plan for a multi-user desktop box, but rather aim for a small, single purpose embedded system, so you won't need too many stuff to get it running.<br />
Most of the time when you start with busybox as the heart of your system you do not plan for a multi-user desktop box, but rather aim for a small, single purpose embedded system, so you won't need too many stuff to get it running.<br />
This example targets such a very simple system.


==== Minimal configuration ====
This example targets such a very simple system : [[Dev/Embedded/Rootfs/BusyBox/Example1|Example 1 : simple system]]
You can start with this '''[http://data.nathael.net/Dev/Base/BusyBox/config_mini_example_2025-01-10 example minimal config]'''.<br />
Some information about my choices :
* This configuration is based upon the "defaultconfig", but with many modifications (not all listed here)
* No "/usr" support (all binaries under /bin and /sbin)
* Archival utilities limited to tar with "xz" and "bz2" support
* Removed console utilities
* Removed most users handling utilities : users should be known when building the system (or added by direct editting of the files)
* Removed disk partitionning and formatting utilities save for UBIFS
* Removed runit support
* Removed all commands I never used on a full blown system
 
==== Compile using uClibc ====
The options selected for a minimal configuration should not impact the compilation process, unless you want a really small system, which is common when using BusyBox due to storage size constraints.<br />
In this case using uClibc instead of the GNU libc is a good idea.<br />
Most information about using uClibc in a cross-compilation toolchain can be found [https://www.uclibc.org/ on uClibc website].<br />
You will then need to use the cross-compilation prefix of your toolchain to build BusyBox.
 
With a stripped down BusyBox, you may well end up with :
Final link with: <none>
But note that in this case you'll still need the selected libc (uClibc) !
 
==== Directories and links ====
Here is a very simple set of directories for a configuration without "/usr", according to the minimal configuration given above :
mkdir -p dev etc lib/{modules,firmware} proc root run sys var/{log,spool/cron}
Then add some links :
ln -s run/tmp tmp
ln -s ../run/tmp var/tmp
 
==== Libraries ====
As this is a very simple system the set of libraries should be very limited. You may need additional ones for your application binaries so I cannot give you a full list, but if you chose to have dynamically linked binaries (which is usually a good idea if your system has more than one or two binaries) you will always need these two :
* The dynamic linker/loader (name depending on the target architecture/triplet) : ld-linux-*.so.*
* the libc
In order to check for additional library requirements, have a look at this part in the second example below.
 
Once you know which libraries you need, you'll need to figure which files to put in which directory for each library.
 
Unless you specified an absolute or relative pathname during the linking stage of the compilation of your binaries, there is no "fixed" location for each library file, save maybe the dynamic linker/loader (ld-linux-*.so.*), which usually goes under "/lib", and the dynamic linker will end up looking for libraries in some "default" path which usually includes "/lib".
 
So the easy way is to create a single "/lib" directory (remember that we chose to desactivate the use of "/usr" in busybox configuration).
 
==== Bootup script ====
* inittab
* rc script(s) and /etc/init.d/rcS
''' TO BE COMPLETED '''
==== Configuration files ====
''' TO BE COMPLETED '''
* /etc/nologin
* /etc/securetty
* /etc/shells
* /etc/{passwd,shadow,groups}
 
==== Application binaries ====
''' TO BE COMPLETED '''
==== Application related files ====
''' TO BE COMPLETED '''


=== Example 2 : toward a complete system ===
=== Example 2 : toward a complete system ===
==== Complex system configuration ====
''' FIXME''' : Add link to example complex config
''' TO BE COMPLETED '''
''' TO BE COMPLETED '''


More notes and warnings :
[[Dev/Embedded/Rootfs/BusyBox/Example2|Example 2 : toward a complete system]]
* Keep in mind that some tools may depend on specific kernel features (and includes) and that you may need to install your own kernel hearders and reference them instead of those comming with the libc.<br /> This is the case for the "tc" network applet which is included in the "defconfig".
* Adding headers or additionnal external libraries is done in the "Settings" part of the configuration, under "Additional LDFLAGS" and "Additional LDLIBS".
* As of 2025-01-03 selinux has deprecated "security_context_t" and "matchpathcon()" (at least) and you'll need to update the selinux related code if you need it.
 
 
==== Directories ====
You will not need a full set of directories from the [https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard FHS] but this can be a good place to get information if you do not know what you need.<br />
* The default busybox installation process has created four "*bin" directories for you, which is a good starting point (though you can merge them to get two if you want to follow the usermerge way, or even go further and get a single one bin directory). : /bin - /sbin - /usr/bin - /usr/sbin
* Then you will usually want somewhere to mount proc, sysfs ans devtmpfs : /proc - /sys - /dev
* You should also stick to the standard and place your text configuration under /etc
* Then you will need a place for your libraries. This can be a single place, or many, it's up to you (it's actually a real mess on some distributions, with multiarch support, 32 and 64 bits support, usrmerge, and maybe other funny stuff getting in the way). Maybe you should consider keeping it as simple as possible (refer to the next section to make your choices about this part). On a Devuan or Debian distribution you usually get those three at least : /lib - /usr/lib - /usr/lib/*arch-triplet* - ...
* A place to put "shared ressources" (architecture-independent data, but you're building an embedded system, so this may not have much sense put this way, simply consider it as a place for data used by the tools you'll need) : /usr/share
* A place for "variable files". As stated on the FHS page : "files whose content is expected to continually change during normal operation of the system, such as logs and spool files". These are files which you want to keep accross reboots : /var/log - /var/spool - ...
* A place for "run-time" files, which you do not want to keep accross reboots : /run
* A place for temporary files. Could be either one of the following : /tmp - /run/tmp
* A user home directory (or many if you're not building a single user, single task system) : /root - /home/....
* Somewhere you can mount removable stuff ? : /mnt
* If you need them, a place for modules and firmwares : /lib/modules - /lib/firmware
* And last but not least, some place specific for your application if you do not want it merged to the base system. Can be any or all of these : /opt - /srv - /usr/local ...
 
Of course, this may seem to be a lot, and it can be reduced to something simpler, depending on your needs !<br />
 
And another with many more directories (see below for the "triplet" you'll need) :
triplet=
mkdir -p dev etc mnt proc root run sys tmp usr/{lib/{${triplet},firmware,modules},share} var/{cache,lib,log,spool}
ln -s usr/lib lib
ln -s run/tmp tmp
ln -s ../run/tmp var/tmp
 
As some in the second example, many can also be links to other ones ... once again it's up to you and your needs !
 
==== Libraries ====
If you chose to use shared libraries (shared object files) when you compiled your binaries then you need to include them in your final system image.
 
This step is not very difficult in itself, but not as simple as busybox compilation and installation.
 
===== First, the answer about the "triplet" =====
I mentioned this "triplet" twice in the above section.
 
An inclomplete way to answer is that it is composed of three fields which identify the target for which you're building your system, though it is somehow much more complicated (it can be only two fields, or four), and there is not "one" answer. It is more of a convention which is not shared by all of the interested parties. You can get some information on [https://wiki.osdev.org/Target_Triplet osdev.org Wiki], on [https://wiki.debian.org/Multiarch/Tuples debian.org Wiki], or in [https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/System-Type.html#System-Type GNU autoconf manual] ([https://www.gnu.org/software/autoconf/manual/autoconf-2.65/html_node/Specifying-Target-Triplets.html also here]).
 
Most of the time it looks like '''"arch"-"os"-"abi"''', though the names are not these ones.<br />
It's usually what "-dumpmachine" or "-print-multiarch" option to the gcc from your toolchain will print, which is also the cross-compilation toolchain prefix.<br />
 
As I'm using Devuan/Debian's cross-toolchains I stick to the convention adopted by the Debian project.
 
===== Which are the required libraries ? =====
In order to know which libraries are required on your system you get a few option.
* Makefiles and build process output
* Use ldd
* Use "readelf -d"
 
The first option is some kind of theoretical approach. Have a look at the makefiles or the build output for the libraries used for the linking stage of the build process, for all your binaries. This could be a pretty hard task, and is not my choice.
 
The second option, which I'm using most of the time (because I never remember the last one, which is the best one) is to use ldd on a target sharing the architecture (the triplet mentionned above) of your target system.<br />
ldd is a command (script) which lists what shared libraries are used by given dynamically-linked executables (read comments in /bin/ldd for more information).<br />
The main limitation is that you need to have a target running a full-blown distribution at hand.<br />
On the other hand, it lists more than the libraries linked, and tells you whether it found them on the system, and which file is used (save for "linux-vdso.so.1" which is not a file but a "virtual dynamic shared object" provided by the Linux kernel : look at the [https://man7.org/linux/man-pages/man7/vdso.7.html vdso manpage] : man 7 vdso).<br />
Of course, you'll need to run this for all your binaries, but that's something you can perform with a succession of commands and filters (or a script), and get a list with single entries for the required libraries.<br />
Here is an example output of ldd :
$ ldd /bin/bash
    linux-vdso.so.1 (0x00007fbb70314000)
    libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6 (0x00007fbb70170000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbb6ff7a000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fbb70316000)
 
The last option, directly available on your host, is to use the readelf tool from the binutils part of your cross-toolchain.<br />
The main limitation for this third option is that it does not tell you about the dynamic linker/loader (ld-linux-*.so)<br />
Once again, you'll need to run it on all your binaries.<br />
Here is the interesting part of the output of "readelf -d" for the bash binary on my host system :
$ readelf -d /bin/bash
    Tag              Type                Name/Value
  0x0000000000000001 (NEEDED)    Shared library: [libtinfo.so.6]
  0x0000000000000001 (NEEDED)    Shared library: [libc.so.6]
 
===== The minimal list =====
I cannot give you a full list, but if you chose to have dynamically linked binaries (which is usually a good idea if your system has more than one or two binaries) you will always need these two :
* The dynamic linker/loader (name depending on the target architecture/triplet) : ld-linux-*.so.*
* the libc
 
===== Where do they go ? =====
Now that you know which libraries you need, you'll need to figure which files to put in which directory for each library.
 
Unless you specified an absolute or relative pathname during the linking stage of the compilation of your binaries, there is no "fixed" location for each library file, save maybe the dynamic linker/loader (ld-linux-*.so.*), which usually goes under "/lib", and the dynamic linker will end up looking for libraries in some "default" path which usually includes "/lib".
 
So the easy way is to create a single "/lib" directory especially if you chose to desactivate the use of "/usr" in busybox configuration.
 
If you want to build a base system closer to what you find in common distributions and use the "/usr", here are some information you should be aware of :
: The dynamic linker will end up looking for libraries in some "default" path, which varies depending on it's configuration and the architecture (usually "/lib" and then "/usr/lib", or '/lib64" and then "/usr/lib64").<br />
Even if the linker/loader will first look at the "cache" file ("/etc/ld.so.cache") which is generated using "ldconfig" it is not of much use in our case because this "ldconfig" is a binary which knows only of one architecture, so (as for ldd) this command must be run on a target of the same architecture with a complete running system.
 
(but is under "/usr/lib/**triplet**/" in a Debian, whith a link from "/usr/lib64/")
''' UNDER REDACTION '''
 
libc.so


==== bootup script ====
* inittab
* rc script(s)
''' TO BE COMPLETED '''
==== configuration files ====
''' TO BE COMPLETED '''
==== application binaries ====
''' TO BE COMPLETED '''
==== application related files ====
''' TO BE COMPLETED '''
== Create an initramfs image for the rootfs ==
== Create an initramfs image for the rootfs ==
Creating an initramfs ([[Dev/Embedded/Rootfs/InitrdVSInitramfs|a real one, not an initrd]]) is pretty simple and can be done in two different ways.
Creating an initramfs ([[Dev/Embedded/Rootfs/InitrdVSInitramfs|a real one, not an initrd]]) is pretty simple and can be done in two different ways.

Revision as of 21:49, 13 January 2025

Introduction

BusyBox is "The Swiss Army Knife of Embedded Linux".
BusyBox is a "multi-call" binary which combines tiny versions of many common UNIX utilities into a single small executable.

A simple description and a full list of embeddable commands and available options can be found here.
Most information can be found in the FAQ and all around the Busybow website, but here is a short "How-To".

The information you'll find here about BusyBox will be split among three part : a common one about the basics, and two different configuration targeting two completely different kind of systems : a very simple "minimal" system, and a second more complex one.

THIS PAGE IS A WORK IN PROGRESS

Common steps

Get the sources

Either use git (below) or download a version from here : Download BusyBox sources

git clone git://git.busybox.net/busybox

You can then extract a specific version if required.

Configure

BusyBox uses a menu equivalent to the Linux kernel's one which is accessible using :

make menuconfig

There's also a "defconfig" and an "allnoconfig" target (among others), and as for the Linux kernel, the configuration is saved in a ".config" file, which can easily be shared (see bellow for example configurations for the two different example systems).
Tune to your needs and move on to the next steps :)

Some notes and warnings :

  • The cross-compiling toolchain prefix can be set under "Settings"
  • Note that some configuration options will greatly impact your system architecture, and some may need specific development libraries to be installed.

Compile

Still in the kernel path, cross-compilation is made using "make", with possibly more than one compilation thread.
Note that when cross-compiling you can either set the "CROSS_COMPILE=" option on the commande line or in the configuration menu.

cross_toolchain_prefix=
nb_threads=
make CROSS_COMPILE=${cross_toolchain_prefix} -j ${nb_threads}

An the end of the compilation the build system displays the "additionnal" required libraries (of course, the libc is always required), for example, with an almost default configuration (make defconfig) :

Final link with: m resolv

In this case, you'll need to include the libm and libresolv.
Note that there are other ways to know which libs are required in your final system exposed under the second example below.

Install BusyBox

As stated previously Busybox is a "multi-call" binary.
This means that there is a single binary called "busybox", and, in order to use other commands the usual way, one must create symlinks to this binary with the name of each command included in the busybox binary.
Hopefully, this task is automated by the BusyBox build system :).
Run :

make install

and you will get a directory called "_install" (can be changed in configuration) containing the base "bin" directories with all the links for the commands you configured.
Note that you must remove the old "_install" directory before running the new "make install".
Of course this does not make for a full system, but you already have all the binaries (or almost all depending on your needs).

Populate the base system

A Linux based system is usually not limited to a single binary. You will need some more files so you can use your system as you would use any other Linux based system.
Some of these files are mandatory, while others depend on your application and the kind of system you want to build. Usually, what you need can be split in among the following categories :

  • directories
  • system binaries
  • libraries
  • bootup script
  • configuration files
  • application binaries
  • application related files

BusyBox installation process which installs the "busybox" binary and the symbolic links for all the selected utilities (commands), which may be most (or all) of the "system binaries".

The details for the other parts will be discussed within the examples below.

Before moving on to the next parts, let's make a short point on how to copy this "_install" directory somewhere usefull.
If you stick to a simple "cp" with no arguments, then you will get many full copies of the busybox binary instead of all the symbolic links, which is obviously not what we want.
We have three options to copy this "the right way" :

  • Use "cp -a" (--archive) : The easiest way
  • Use rsync (with the right set of options)
  • Use tar (make an archive with "tar -c", and then extract somewhere else with "tar -x").

I'm using the "cp -a" solution most of the time, unless I want to keep a backup somewhere (and use the tar solution).

Let's now move to the next steps required to build a the base system with two examples.

Example 1 : simple system

Most of the time when you start with busybox as the heart of your system you do not plan for a multi-user desktop box, but rather aim for a small, single purpose embedded system, so you won't need too many stuff to get it running.

This example targets such a very simple system : Example 1 : simple system

Example 2 : toward a complete system

TO BE COMPLETED

Example 2 : toward a complete system

Create an initramfs image for the rootfs

Creating an initramfs (a real one, not an initrd) is pretty simple and can be done in two different ways.

  • From the command line, on your own :
cd test && find . > ../liste_cpio
cat ../liste_cpio | cpio -H newc -o > ../initramfs.cpio
gzip -9 initramfs.cpio
mv initramfs.cpio.gz initramfs.img
  • Using the kernel build system.
TO BE CONTINUED