‘Functional, free and secure by default’, OpenBSD remains a crucial yet largely unacknowledged player in the open-source field.
This series aims to highlight the project’s signature security features and development practices — razor-sharp focus on correct and secure code coupled with continuing code audit — as well as the project’s role as a source of innovation in security practices and an ‘upstream’ source for numerous widely-used components such as OpenSSH, PF, LibreSSL and others. Part 1 focused on the history, this post will focus on usage and user experience, and Part 3 will look at that packet filter.
So why use OpenBSD? What is it like?
What is OpenBSD like for a user or developer, and why is it better?
I’d say the short version is that it’s a real Unix. Unlike the Linuxes of the world that spent years muddling through an evolutionary succession of init systems and have ended up more or less settling on the ever-expanding systemd, which seems to have tentacles in everything and is on a clear course to replacing most of what we have traditionally thought of as the base system, OpenBSD has stayed with and refined the traditional BSD init. So you can have both uncluttered services management and a base system that consists of programs that, for the most part, adhere to the classical Unix philosophy that every program should do exactly one thing, and do that thing well.
If you are a developer, you will also appreciate hearing that the base system of well-designed programs that all have a readable and useful man page already contains basic Unix developer tools along with a C and C++ compiler — Clang, where supported, and gcc, where necessary — plus Perl and a host of tools. Basically, everything that is needed to build the base system from a fresh checkout of the source code is contained in the base system on a default install.
Ported software goes under /usr/local
Once you have the thing installed on whatever hardware you have, keeping in mind that you can run a selection of 14 platforms ranging from fairly ancient kit to modern hardware, you will likely turn to install ported third-party software from packages, using pkg_add(8), which will suck in whatever you tell it to fetch from the same mirror you installed from or what appears to be the most local one.
There is more software available for other popular and ‘exotic’ platforms.
For the OpenBSD 6.9 release, the most mainstream platform amd64 came with 11310 prebuilt and installable packages, while mips64 had only 8182 and the mips64el platform is marked as (still building).
Installing prebuilt packages is almost always more convenient and is recommended in most cases, but if you for one reason or the other want to build your own from a cvs checkout of the ports tree, you are free to do so at the cost of your own time watching the process.
Whichever route you choose to go, you will see that installing packages does not land you with any files in the directories used by the base system outside of a few that drop their configuration files in subdirectories under /etc and add their combined startup/shutdown scripts to the collection in /etc/rc.d. Anything else ends up under /usr/local, and you can see why the installer, by default, sets up that file system on a separate and fairly roomy partition. My previous article You’ve installed it. Now what? Packages! is a few years old, but gives you the main motivations and some background along with some pointers on packages practice.
Note: If you are more of a slides person than a full-text person, you may be relieved to hear that you can find the slides for the talk this article is based on (and vice versa).
The installer that was always good got better
When I found OpenBSD more than 20 years ago, my main Unix exposure was from working with Linuxes and FreeBSD. What attracted me to OpenBSD, and finally had me buy an OpenBSD 2.5 CD set, was the strong focus on security and code correctness. When the CD set and the classic wireframe daemon t-shirt finally arrived in the mail, I set about at first to install it on whatever spare hardware I had lying around.
If I remember correctly, the first machine I tried installing OpenBSD on was an 80386/33MHz with 8MB RAM and, I think, a 100MB IDE hard disk, which I can report, sounded pretty crappy even then, but the thing did work.
The initial install was fairly straightforward, and when I started poking around I found two things about myself and the new system; everything made sense, and everything I could think of had a readable man page. So the first change I am aware of that made the world better with OpenBSD was the decision to enforce the ‘No commit without documentation’ rule, which came into being early in the project’s life — probably roughly at the same time, the OpenBSD developers gave us a real-time view of development via anonymous CVS. You can see things happening in almost real-time.
It is worth mentioning the installer has remained famously non-graphical, text-only. The reason the installer remains text-only is that this is a major advantage that enables the developers and the users to handle the fairly diverse collection of hardware platforms that OpenBSD runs on with the same portable, familiar and compact code everywhere.
The installer was always scriptable and extensible. Over the years the installer has added automatic, repeatable and scriptable installs (dubbed autoinstall(8), which appeared in OpenBSD 5.5 in 2014) and the sysupgrade(8) extension (first found in OpenBSD 6.6 in 2019) that automates snapshot-to-snapshot or release, to subsequent release upgrades for all not too hacked-up configurations. Each of these moments, or more specifically when the new code started appearing in snapshots, had me appreciate the OpenBSD system a bit more and made me feel quality of life had improved.
Now something for your laptop — hardware support
Fast forward some twenty-plus years and the last article I published (and even got into Norwegian mainstream IT news site Digi.no) centres on a few moments involving new OpenBSD developments. It took some interaction with OpenBSD developers, but those interactions led to my new laptop with an 11th generation Intel Core chipset working even better with OpenBSD. Yes, OpenBSD developers and a significant subset of their user base run OpenBSD on their laptops. I do use a Mac and a work-issued Thinkpad with Ubuntu Linux too, but life is not complete without an OpenBSD laptop.
Now to be honest, what I saw within the space of a few days was a development that had me going from ‘Oh, sh*t, the SSD isn’t recognized’ — the controller was set to a RAID-ish mode by default — through this kernel panic (Figure 3), to seeing it all fully supported.
The SSD problem turned out to be simple to fix: I found the ‘Advanced’ BIOS option that turned the pseudo-raid feature off which let the operating system speak directly to the storage device.
For the rest, there was a period of a couple of weeks where I had to run with not yet committed patches in a home-baked kernel built from checkouts from Jonathan Grey’s git repo. When the code was committed to -current, I could resume my normal sysupgrade(8) routine, going from one development snapshot to the next.
The process, even with the need to build custom kernels for a while, was quite pleasant, and when the support code went into the main development branch, that too was a moment when I felt my life had been improved by changes in OpenBSD. The hardware support is now in snapshots and will be in OpenBSD 7.0 which is set for release on 1 November 2021.
Why use OpenBSD? IPSEC
As I mentioned earlier, OpenBSD was the first free system to ship with IPSEC — the tools for enabling encrypted network traffic — in its base system. OpenBSD 2.1 (1997) was the first version to feature IPSEC.
The tools worked, of course. However, in the early days, the complaint was IPSEC was hard and near impossible to debug from an almost-working to a fully working setup.
Further development took a while, but the tools got a major usability upgrade in OpenBSD 3.8 with ipsecctl and its human-readable configuration file /etc/ipsec.conf to serve as a friendlier front end to the IPSEC tools.
A complete configuration for a minimal setup could look like this:
# Set up two flows:
# First between the machines 192.168.3.14 and 192.168.3.100
# Second between the networks 192.168.7.0/24 and 192.168.8.0/24
flow esp from 192.168.3.14 to 192.168.3.100
flow esp from 192.168.7.0/24 to 192.168.8.0/24 peer 192.168.3.12This was a major step compared to other platforms. One famous example is preserved in a presentation by Mathieu Sauve-Frankel at AsiaBSDCon 2007, where he demonstrated that setting up the equivalent of the config shown here with Microsoft tools took the user through a sequence of no less than 36 dialogue boxes, and still there was a distinct possibility that the configuration was not a working one.
The problem in both the Microsoft implementation and others was that the developers had not given any thought to the user experience. The majority of the options the user was required to set had sensible defaults and could have easily been hidden from view.
The standards documents the developers had worked from were fairly unclear, so it is not entirely unreasonable that ‘it was hard to write, so it should be hard to use’ was a factor in how the products ended up from a user experience perspective.
What defaults would be actually sensible would perhaps not be clear to the developer from the specification, and the only way to see what would be the sensible default would be to use it in the field to gain experience. The OpenBSD developers who wrote ipsecctl had extensive experience with IPSEC and decided that IPSEC did not need to be so hard. The defaults should make sense.
The next milestone in IPSEC development on OpenBSD came with Internet Key Exchange (IKE) protocol support in OpenIKED in OpenBSD 3.8 with iked and ikectl. For convenience, ikectl can generate configurations for Windows and macOS clients too. That might save you the agony of clicking through dozens of dialogue boxes at the client end.
The thing that lured me in
But I hear you ask, what made me turn into an almost all-in OpenBSD user?
In 2001, I was still only experimenting with OpenBSD, but my experience with Linux and iptables had made me long for a switch to a saner firewall. I had done some small experiments with the IPF firewall that was in OpenBSD until the 2.9 release. Then, as some of us will remember, IPF’s license was not, in fact, free, so it needed to be replaced.
There was a distinct rush, not quite a stampede, to replace IPF over the months that followed. Fortunately, the new code that replaced the previous packet filter proved to perform better. The OpenBSD Packet filter, dubbed PF for short, had been born and made its debut in OpenBSD 3.0 in December 2001. The release had originally been planned for November but was pushed out a month to hack the ‘working prototype’ packet filter into something usable.
This turn of events finally pushed me to take the final steps to replace the Linux gateways I had in place with OpenBSD ones. I was pleasantly surprised to find that not only did they perform well, but they also came with complete and reasonably well-documented tools so I could understand what was going on. That’s how I got started on the process that lead to, among other things, writing The Book of PF and taking that text through three editions so far. But more about that later.
It is worth noting that the IPFilter copyright episode spurred the OpenBSD developers to perform a license audit of the entire source tree and ports to avoid similar situations in the future. This activity ran for some months and uncovered several potential problems. Theo de Raadt summed up the effort in a message to the openbsd-misc mailing list on 20 February 2003.
What they found when they started looking was that there was a significant number of files that were not under a free license, much like the entire IPF subsystem had been. Those needed to be replaced. Other parts had either no license or no copyright stated. In some cases, the developers gave explicit permission to continuing use, but quite a few things needed to be rewritten with a free license so OpenBSD and other free software would be able to move forward without copyright problems.
I later heard in a rather informal setting that among the no copyright and/or no license cases, it was usually possible to track down the developers via version control system logs or mailing list archives. In a large number of those cases, the initial reaction was along the lines ‘Say what? Are people still using that?’.
SSH, open and better
PF was written from scratch to replace a subsystem that it turned out was illegal to use in an open-source context. But it was not the first time the OpenBSD project had performed a ‘nonlibreectomy’, that is, taken on the task of replacing code for license reasons.
A few years earlier, it had become clear that the original developer of the secure shell system, ssh, had commercial ambitions and the license for the software had changed in a proprietary direction. After deliberating how to resolve the situation, the OpenBSD developers started digging around for earlier versions of the code that had been published with an acceptable license. Then they forked their version from the last version they found that still had a free license. Next came an intensive period of reintroducing the features that were missing in the old code.
The result was introduced as OpenSSH in OpenBSD 2.6 in 1999. Over the next few years, OpenSSH grew a portable version that started grabbing market share rapidly. The last I heard OpenSSH’s market share is somewhere in the high nineties percent.
With a state-of-the-art secure shell subsystem in place and growing all sorts of useful features, the time finally ended for unencrypted shell login sessions on OpenBSD. OpenBSD’s telnetd moved to the CVS attic in time for OpenBSD 3.8, which was released in November 2005.
One other notable thing about OpenSSH is that it was the first daemon to be properly privilege separated, a model practice that debuted with the overhauled OpenSSH in OpenBSD 3.2 in March 2002. Since then, privilege separation has been put in place in all daemons where it made sense to do so, and it is now a signature part of the secure by default stance of all newer OpenBSD daemons.
That brings us to that packet filter, which I’ll cover in Part 3.
Peter N. M. Hansteen is a puffyist, daemon charmer and penguin wrangler.
Adapted from original post which appeared on BSDLY.
The views expressed by the authors of this blog are their own and do not necessarily reflect the views of APNIC. Please note a Code of Conduct applies to this blog.