April 8, 2014

FreeBSD Experiment

Every once in awhile I try FreeBSD. I like using it, but eventually I miss something from Linux and abandon my FreeBSD experiment, let’s see what happens this time.

I am interested in FreeBSD for a few reasons:

1. “Performance” is a stated goal (although I have yet to see a real performance benefits over Linux).

2. Non-GNU userspace

3. Non-restrictive licensing

4. Better mascot

5. Ports beat binary packages any day

That being said, FreeBSD is quite similar to Linux in many ways since they have common ancestry. Also, FreeBSD includes Linux syscall emulation which allows it to run native Linux binaries.

This is cool and all, but I want to test out FreeBSD, not Linux. If I want Linux I can boot into my Lubuntu or BetterLinux systems. So, I want to avoid “Linuxisms” as much as possible to get the most BSD out of the experience and really see where BSD excels.

I also wanted the fullest exposure to the base system as possible, so as to learn the most about FreeBSD while I am at it.

In addition, I want to push FreeBSD to the limits of its performance to see what it can really do (like I do with BetterLinux).

To acheive these goal better, I set some rules for myself:

1. No using Linux binary compatibility

2. No 32 bit compatibility

3. No GNU C Library

4. Direct booting (no chainloading)

5. No “linuxisms” where BSD tools can work instead (e.g. iw instead of ifconfig)

I started out installing FreeBSD 11 by booting the memstick image in qemu on my Linux machine and using /dev/sdb as the second hard disk.

First of all, FreeBSD is still using its asinine “slice” partitioning. I see no reason for it and it make it more difficult to access my FreeBSD files in Linux. So I hacked around with the installer until I got it to install in a regular msdos partition. I did this by manually mounting the system in the installer.

Once the system was installed, I shut down the VM and started editing my grub.cfg file. I know you aren’t supposed to edit grub.cfg directly, but this is free software, so I don’t let people tell me what to do, as long as I don’t run grub-update, I don’t have to rebuild my grub.cfg.

I decided against using chainloading because I don’t like the FreeBSD bootloader either and I have had trouble with it in the past (it overwrote my MBR on all connected devices, including USBs). So, I set up grub to directly boot the FreeBSD kernel. I am pretty happy about this; my grub entry looks like this:

menuentry “FreeBSD” {

       insmod ufs2

       set root='(hd1,msdos1)’

       kfreebsd /boot/kernel/kernel

       kfreebsd_loadenv /boot/device.hints

       kfreebsd_module /boot/splash.bmp type=splash_image_data

       kfreebsd_module_elf /boot/modules/nvidia.ko

       set FreeBSD.vfs.root.mountfrom=ufs:ada1s1

       set FreeBSD.vfs.root.mountfrom.options=rw

       set kFreeBSD.vfs.root.mountfrom=ufs:ada1s1

       set kFreeBSD.vfs.root.mountfrom.options=rw

}

Then I rebooted successfully into FreeBSD. To install ports, I needed network, which meant wifi. My Atheros based card was supported by default, which is nice, but since I ran the installer with qemu instead of my real hardware, the installer didn’t set up my real network hardware.

To get my wifi card up, all I had to do was say “ifconfig ath0 up”, no problem. I knew that FreeBSD uses ifconfig to scan and connect to APs, but I didn’t remember exactly how, so I had to do some research in the man pages. I tried just “ifconfig ath0 ssid router”, but that always failed. Turns out in FreeBSD you have to create a “wlandev” from the atheros device and use that to scan and associate with an AP. So I did that and used dhclient to DHCP and then I was connected.

That’s all fine and good, but I didn’t want to do that every time I booted. Fortunately, it’s easy to set up in FreeBSD’s /etc/rc.conf file.

Next I used portsnap to get the ports tree. I know I am more productive if using a GUI interface, so my first goal was to install an X server.

This was a lot more trouble than it should have been, and took a long time. Part of the problem was my trying to disable absolutely everything I wouldn’t use. Seems some ports require specific options of other ports it depends on, and the ports system has no good way of managing this, which is OK with me, but it does take more work to get it all working.

Once I got X up and running, I needed a window manager. For some reason I chose enlightenment (e17). My original plan was to install something based on XCB instead, but there aren’t any like that in ports right now. Enlightenment isn’t the lightest weight window manager, but it’s not bad and it looks pretty.

Enlightenment built with very few problems. Running it was at first a problem though, it displayed the logo for what I thought was forever. After waiting a long time it does eventually get past that point. It’s very strange because it doesn’t seem to be loading anything during that time and my CPU stays mostly idle, so I don’t know what the deal is.

Enlightenment runs well for the most part, but I did experience a few crashes.

I installed xterm (and later st) and began building other ports I thought would be useful or neat.

The next thing I wanted to do was build a FreeBSD kernel. I figured this is a good way to get to know a system. Unfortunately, I didn’t install the FreeBSD sources from the installer, so I had to get them from the internet.

To do this first I needed a web browser. I am quite partial to Firefox, so I tried that, but all the builds failed no matter what I tried, that port seems broken to me. I then gave up and installed Midori since that is my second choice and seems to be easier to compile.

Midori installed and runs well, but the build took forever. I looked up how to get the sources, and there isn’t really good documentation on it, but I eventually figured it out and got the sources through SVN.

I created a new kernel config file for my custom kernel, removing a ton of extraneous modules. Anything I didn’t have or wouldn’t use got removed from the kernel.

I saved my config and started the build, but it failed with some error in the aesni module. I didn’t recall seeing this module in the config file, and I checked the config file again. Turns out that FreeBSD still tries to build all the modules, even if they won’t be part of the kernel. This might make sense, the config file just decides what gets actually built into the kernel and not loaded as a module.

However, this aesni module just wouldn’t compile. I looked up the problem online and found nothing on it. So I searched the file in general and found a proposed patch affecting the lines of code that generated the error, but I applied the patch and the error still remained in the patched code.

So, I tried plan B: I built GCC 4.9 from ports and tried to compile the kernel with that. This also failed, but due to unrecognized command line arguments. I looked this up online and apparently this is something you are not supposed to do since FreeBSD is supposed to be built with Clang now.

However, after some searching I found somebody that said he (or she) successfully built a working kernel with GCC (even though it’s not supported). The trick is in the /etc/make.conf file where you can set that stuff up. FreeBSD document’s this file quite well, so it wasn’t difficult to find what I needed to set there. Also, the working config file was posted and I copied most of that.

I restarted the build, but it failed in the same aesni module. Turns out theres some broken code in there with both GCC and Clang. I tried hacking away at the code, but didn’t solve the problem. Eventually, I figured that I can ask FreeBSD to no build that module via the make.conf file. So I built without this module. I used kldstat to see my loaded modules (none) and judged that it wasn’t in use, so probably not necessary anyway.

My final /etc/make.conf file looked like this:

CC=/usr/local/bin/gcc49

CXX=/usr/local/bin/g++49

CPP=/usr/local/bin/cpp49

CWARNFLAGS= -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes \

-Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual \

-Wno-pointer-sign -D__FreeBSD_cc_version=0 \

-Wno-unused-function -Wno-inline -Wno-format \

-Wno-uninitialized -Wno-array-bounds -Wno-missing-prototypes

NO_WERROR=true

NO_KERNELCLEAN=true

WERROR=-Wno-error

WITHOUT_MODULES=aesni

The kernel built successfully this time. I wanted to save my old working kernel in case booting the new one caused a problem, but the documentation I was reading at the time said that “make installkernel” would do this for me. So, I stopped worrying and installed the kernel. For some reason though, either I read incorrectly or the documentation was wrong, but the old kernel was not backed up, just overwritten.

Fortunately however, the kernel booted fine. There were a few problems initially with the new kernel, first of all, I failed to compile in the USB mouse module, called ums0. For some reason the module is compiled into the kernel that comes with the FreeBSD installer, but it is not named in the generic kernel config. I recompiled the kernel to fix the problem.

Next I wanted to get hardware graphics accelleration working. Unfortunately, my machine has an nvidia card. In Linux, Nuoveau is great, but that doesn’t seem to be an option in FreeBSD. I did a quick search and found that I can use the nvidia proprietary driver and install it through the ports system. I did that and modified my X config file to use it and it worked fine.

However, the driver includes a kernel module. Until then I was running with zero loaded modules, everything was built into the kernel. Loading it manually is easy, but of course I wanted to load the module on boot. FreeBSD typically uses the file /boot/loader.conf to list modules that should be loaded on boot. So I added the module there are rebooted, but when the system came back up the module was not loaded. After a few minutes I realized that /boot/loader.conf is never read by the kernel but by the bootloader. Since I was not using the FreeBSD bootloader, the modules weren’t getting loaded. Fortunately, grub can easily load FreeBSD modules, so I modified my grub.cfg to load the module.

Then I had graphics running well, and I wasted some time by installing some games, which was useful for killing time while waiting for ports to finish compiling, but the game often don’t run as well while compiling things.

I installed a few applications, Ted, abiword, geany, gnumeric, dia, xfe, mplayer2, gnome-mplayer, and scythia. These all seem to run well.

However, I noticed some things in the ports system, ports seem to pull in a lot more dependencies than I would expect. There was even a case where I disabled GTK3 support in a port, but it tried to build GTK3 anyway. I gave up on that port and installed an alternative.

I frequently use my computer as an alarm clock, so I needed an application that would function as an alarm clock (I am going to write my own soon, none of the available ones are good enough). On Linux I use one called alarm-clock-applet. Ports has that one but it seems to be broken. I downloaded the source from the official website manually and tried to compile it. It compiled successfully, but crashes everytime it should trigger an alarm, which obviously defeats the purpose.

Fotunately, enlightenment has an alarm clock module that works, though it’s less featureful and harder to set up, it is good enough for me.

At the time it was quite late, and I was worried that my neighbors might be bothered by my alarm clock testing, so I wanted to change the volume. In FreeBSD you can do this with the mixer command. It’s very easy to use and you can quickly change the volume to anything you need it to be. However, it has to be done on the command line and I haven’t found any good frontends for it, similar to wifi setup. The command line tools in FreeBSD are very nice and often superior to Linux equivalents, but FreeBSD is sorely lacking in graphical frontends for it’s command line tools.

Finally (the last thing I have done at the time of this writing), I decided to do some benchmarking to see how the C library and compiler are performing. This is still an ongoing project and I have not yet collected the data in Linux to compare those.

However, my benchmarking did reveal something. I ran each test 265 times with both GCC and Clang/LLVM. Then I averaged out the results and compared them. Turns out Clang/LLVM actually produced faster code on all but 3 of the tests. On average, Clang/LLVM produced code was about 1.8% faster than GCC 4.9 produced code.

However, this was not utilizing the highest levels of compiler optimization, this will be my next task, and I suspect GCC will win there. Then I will take the time to get the results from GNU/Linux and BetterLinux as well.

CONCLUSION:

FreeBSD is fun to play with, and it is capable of all the things a typical user will do with his/her system. It does not rely on GNU code and has a much more permissive license. It also matches the requirements of the BetterOS project for a usable OS.

The ports system gets mixed reviews. On one hand, it allows a good deal of compile time customization and allows for you to apply compiler optimizations that fit your machine very well, producing better executables for your system. However, it tends to pull in more dependencies than needed sometimes, and you can easily get into dependency hell using ports. In addition, there are more than a few ports which are just plain broken. So ports is a great idea, and I think it’s implementation is even better than Gentoo’s portage, but it needs more active maintainance.

As for OS code, I will give it a little break because I am testing the current branch and not the stable one, but I was very disappointed by the kernel compilation failure and the fact that there was a broken module in the source tree. Now, Linux is no better in this respect, there are many broken things in the Linux kernel source, but I really wanted FreeBSD to do better here. That being said, I am happy that I got it compiled and that I could do so with both Clang and GCC, even though compiling with GCC isn’t supposed to work anymore.

The base system is great, but has a few faults. First, the FreeBSD “slice” partition is just dumb. Also, the bootloader is fine I guess, but I don’t like it much, to be fair here, I haven’t tested it since FreeBSD 8. However, booting from grub works well but it not documented nearly at all. This isn’t FreeBSD’s fault though. The command line applications (e.g. for wifi and sound volume control) are great, but there is a lack of good graphical frontends for them. Seems FreeBSD users like the command line even more than Linux people. This is fine, but to be honest, changing volume levels with the mouse wheel is easier and faster than using the command line.

I ran into a few software bugs, specifically in Enlightenment which locked up every once in a while. gftp doesn’t work due to a mutex bug. These aren’t FreeBSD’s fault though. However, my whole system rebooted once when I plugged in a strange USB device. Not sure who to blame for that.

The benchmarking is still a work in progress, but I am excited about what the results will show. The results of these tests could set the stage for the next chapter of the BetterOS project. However, I am not expecting FreeBSD to beat Linux in terms of performance, but the data will explain some things about C libraries, compilers, and kernels.

That's all for now, I am going to keep using FreeBSD as much as possible for now and see how long this experiment will last. More updates should follow.

Peace.