Starting a rootfs from boot

Now that we’ve played with rootfs in chroot a bit, let’s see if we can actually boot something. When I install Alpine 3.16, I see three partitions; A boot partition /dev/sda1, a swap partition /dev/sda2, and the rootfs /dev/sda3. When we start Alpine, the boot partition is mounted on /boot. Here we find a couple of files, including vmlinuz-lts. This file is our Linux kernel. Yup, if you never knew where that things was to be found, there it is! This can be a file or it can be a symbolic link to the actual kernel. On my Kubuntu for example, the kernel is called /boot/vmlinuz-5.15.0-52-generic, but linked to from /boot/vmlinuz. Note that on Kubuntu the file is vmlinuz while on Alpine it’s vmlinuz-lts. It seems the ‘vmlinuz’ name grew historically. Originally, Unix kernels were typically called ‘unix’, later it was possible to add an option called ‘Virtual Memory’ and the name of kernels compiled with that option were prefixed with ‘vm’. Later it got possible to compress the kernel file to save storage and the ‘x’ was changed to ‘z’ to show it’s a compressed kernel. Linux distributions just took over the same conventions. Anyhow, what we want to do now, is play with the rootfs and boot something up!

For the following we need an extra partition. You can use an extra drive, or add an extra partition with filesystem on the current drive. I don’t want to go into much detail on how you can do this stuff, you should find info online if needed. If you want to add a new formatted partition to the current drive, it’s probably easiest to do it from a live distro with a graphical DE (e.g. Trisquel) and do it with a graphical tool like gparted. In my case I have /dev/sda3 which holds my Alpine installation and /dev/sda4 which is the new partition.

When the kernel boots, it does some things and eventually starts /sbin/init. This is very similar to what we did with chroot. The big difference is that there we said what to start, here Linux decides to start /sbin/init. I’m unsure if this is hard coded or can be changed with config files somewhere, but from what I see online, it at least seems to be the norm to start this.

What we’ll try, is to use Alpine’s boot partition and boot something simple that we create on our own rootfs.

Just to see things working, I first booted into a live iso and copied all files from sda3 to sda4.

mkdir /mnt/sda1 /mnt/sda3 /mnt/sda4
mount /dev/sda1 /mnt/sda1
mount /dev/sda3 /mnt/sda3
mount /dev/sda4 /mnt/sda4

cp -r /mnt/sda3 /mnt/sda4

In the boot folder (/mnt/sda1) I notice a file extlinux.conf. This is from the part of our boot system where I don’t want to delve too much in yet, but one important line is APPEND root=UUID=91c0e3ea-fca1-4c7c-96ed-ebc551066204 modules=sd-mod,usb-storage,ext4 quiet rootfstype=ext4. This tells us what partition should be considered as the one with the rootfs. I take the guess that this will probably also work with the name instead of UUID, so I changed this to APPEND root=/dev/sda4 modules=sd-mod,usb-storage,ext4 quiet rootfstype=ext4. Just to be sure you can add a file or something to sda3 and sda4 just so you can be sure which of the two is used as rootfs, and then reboot the machine (without the live iso). The result was that we are now booted from /dev/sda4. That’s pretty sweet! Not only did we boot from another partition, the partition was filled up by simply copying files over. That implies to me that we don’t need certain bits to be set at a certain place on the drive or anything. Linux understands what’s on this drive and can execute things.

Let’s see if we can boot something up ourselves. First I change extlinux.conf back so it boots from /dev/sda3. First we make sure we can access the filesystem on our partitions, then we remove everything from sda3 and create the folders Linux needs. If something goes wrong, we can always boot again from a live iso, mount the drives where we think something went wrong, and fix things.

mkdir /mnt/sda3 /mnt/sda4
mount /dev/sda3 /mnt/sda3
mount /dev/sda4 /mnt/sda4
cd /mnt/sda3

rm -r ./
mkdir sys dev proc

If we reboot the machine now, we’ll get a kernel panic. Hooray! That means the kernel is booting, it just can’t continue which is normal since the partition it tries to boot from is empty, and therefor there’s no /sbin/init to start. Let’s try to start a simple shell on boot. This is very similar to how we set things up with chroot. To set things up, we’ll boot from a live iso again.

Alpine uses something called Busybox. This is a single binary that contains a whole set of tools. For example, if you have Busybox installed, instead of doing ls, you can also do busybox ls. So what we’ll do is, we’ll copy busybox and the needed libraries to the rootfs we’re creating and then call it from an init script.

mkdir /mnt/sda3 /mnt/sda4
mount /dev/sda3 /mnt/sda3
mount /dev/sda4 /mnt/sda4

# We copy the binaries and libraries just like we had to do when playing with chroot
mkdir /mnt/sda3/bin
cp /mnt/sda4/bin/busybox /mnt/sda3/bin/busybox
mkdir /mnt/sda3/lib
cp /mnt/sda4/lib/ld-musl-x86_64.so.1 /mnt/sda3/lib/ld-musl-x86_64.so.1

Then we’ll create our init script and make it executable

cd /mnt/sda3
mkdir sbin
touch sbin/init
chmod +x sbin/init

echo '#!/bin/busybox sh' >> sbin/init
echo "echo 'Welcome to your very own Linux Distro!'" >> sbin/init
echo '/bin/busybox sh' >> sbin/init

If you made sure we’re using /dev/sda3 as root partition again on boot, you can now reboot and should see something like

Welcome to your very own Linux Distro!
sh: can't access tty: job control turned off
/ #

Awesome! Since busybox has a whole bunch of tools, we can already use these. If ls doesn’t work, try busybox ls. You can also try to add some folders or files (e.g. busybox touch catgirl_titties), but you’ll quickly notice the filesystem is read-only. We can fix that with busybox mount -o remount,rw /. There’s also the error “sh: can’t access tty: job control turned off”, but I conveniently decided to consider this one out of scope for this article 😏

When we want to exit our shell, we can do exit, which will now give us a kernel panic. If you don’t want a kernel panic, you can call busybox poweroff -f, which will immediately poweroff the machine.

When you call a program, you can give it a bunch of parameters. In the case of busybox ls, we provide “ls” as a parameter. But one parameter that’s always provided is how the command was called. If we call busybox, then busybox will know that this is how it was called. If we link a file called “ls” to busybox and then call that “ls” file, busybox will know that it was called with the command “ls”. Busybox is programmed in such a way that it will use this information to conclude what command you wanted to run and then run that. If you want a list of the commands busybox has, you can run busybox --list. To see what location busybox expect these tools to be placed, you can do busybox --list-full.

Knowing these things, there’s some optimalisations we can add.

First we’ll make it easier to call the commands by making system links to busybox in the correct locations.

busybox mount -o remount,rw /

# Maybe there's a built in way to do this, idk, this works :)
for link in $(busybox --list-full)
do
folder=$(busybox echo "$link" | busybox sed -E 's@(.*/)[^/]*$@\1@')
busybox mkdir -p "$folder"
busybox ln -s /bin/busybox "$link"
done

Another thing we can do, is change the init file to have the following content. That way the file system will be mounted as read-write, and when we do exit, the machine will power off instead of throwing a kernel panic.

#!/bin/busybox sh
/bin/busybox mount -o remount,rw /
echo 'Welcome to your very own Linux Distro!'
/bin/busybox sh
/bin/busybox poweroff -f

Now we can reboot and we have our own very very minimal distro :D

Distro’s typically do more than just start a shell. They also have an init system to start up several services for example. Afaict busybox can do all that, but it requires more looking into. I’m not gonna do that now. For now we already have a neat thing to brag to our friends about, so let’s enjoy that first ;)