Upgrading to FreeBSD 14 - how to fix a broken BIOS bootcode

Published on 2023-11-22. Modified on 2023-11-27.

A lot of people running ZFS zroot have managed to break their FreeBSD systems upgrading from 13.2 to the new 14.0 release because of a broken BIOS bootcode. In this tutorial I'll show you how you can fix that without having to reinstall.

In my humble opinion, the release notes for FreeBSD 14.0 is lacking relevant information to a lot of users and I do not like how these issues have been handled.

On the blog of Colin Percival, who is the new FreeBSD Release Engineering Lead, he writes that:

I assumed the role of FreeBSD Release Engineering Lead a few days ago, and one of my first duties in the role was to write and send out the FreeBSD 14.0-RELEASE announcement. (To be clear: Glen Barber did all of the work of getting the release ready; the final bits had already been copied out to mirrors at the point that I took over.) FreeBSD 14 is a great release, but there are a few last-minute issues which deserve to be documented — probably somewhere on the FreeBSD website, but I can post to my blog much faster and hopefully we'll get these onto the FreeBSD website later.

Again, in my humble opinion, and with all due respect, that is just not good enough.

A project should NEVER release anything until ALL issues have been fully and clearly documented on the project website, in the release notes, and mailing list posts, etc.

The most devastating issue has been people running FreeBSD 13.2 on a BIOS setup rather than EFI who managed to break their systems completely, making them unable to boot.

This issue could have been avoided very easily with the correct and relevant information, but instead the release notes only deal with issues related to EFI systems, not BIOS systems.

The release notes says the following:

Note for systems that boot via EFI, using either binary or source upgrades: There are one or more copies of the boot loader on the MS-DOS EFI System Partition (ESP), used by the firmware to boot the kernel. If the root file system is ZFS, the boot loader must be able to support reading from the ZFS boot file system. After a system upgrade, but before doing a zpool upgrade, the boot loader on the ESP must be updated, or the system may become unbootable.

After finishing up the upgrade using the freebsd-update tool, most ZFS users typically upgrade their ZFS pools too. When that is done on the zroot pool the following message is supplied:

# zpool upgrade zroot
This system supports ZFS pool feature flags.

Enabled the following features on 'zroot':
  edonr
  zilsaxattr
  head_errlog
  blake3
  block_cloning
  vdev_zaps_v2

Pool 'zroot' has the bootfs property set, you might need to update
the boot code. See gptzfsboot(8) and loader.efi(8) for details.

Now, I am sure most users ran man gptzfsboot and man loader.efi or perhaps looked at the FreeBSD online manual pages to get the relevant information, but for many users, dealing with "boot code" has just not been relevant before.

When you install FreeBSD using the installation tool, you do not have to deal with any bootcode related stuff, regardless of whether you use a BIOS or EFI setup. So you can basically have been running FreeBSD for many years without ever looking at anything bootcode related.

In any case, looking at the EXAMPLES section at the man page for gptzfsboot doesn't really help either unless you truly know what this is all about.

gptzfsboot is typically installed in combination with a "protective MBR" (see gpart(8)). To install gptzfsboot on the ada0 drive:

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0

gptzfsboot can also be installed without the PMBR:

gpart bootcode -p /boot/gptzfsboot -i 1 ada0

The man page for loader.efi is completely useless in this regard as that is only related to the UEFI kernel loader.

If you reboot the computer without installing a new gptzfsboot bootcode, you will see something like the following during boot:

ZFS: unsupported feature: com.delphix:head_errlog
ZFS: pool zroot is not supported
Can't find /boot/zfsloader
Can't find /boot/loader
Can't find /boot/kernel/kernel

And the machine will then halt.

The solution

The reason for the problem with BIOS based systems is that gptzfsboot is used to boot from a filesystem in a ZFS pool. gptzfsboot is installed in a freebsd-boot partition of a GPT-partitioned disk with gpart. When the computer boots, gptzfsboot tries to find all ZFS pools that are composed of BIOS visible hard disks or partitions. gptzfsboot then looks for ZFS device labels on all visible disks and in discovered supported partitions for all supported partition scheme types.

Because ZFS was upgraded to version 2.2 a new gptzfsboot bootcode needs to be installed on the freebsd-boot partition.

So what you can do to solve the issue is:

Boot from a FreeBSD 14 installation media
Choose <LiveCD>
Log in as root (no password is required)

Then list your disks with the gpart show command. The disk I run zroot on in this example is ada1 so I'll use this here. If you are running your zroot on a mirror of e.g. two disks, you need to locate both of them.

Be careful that you choose the correct disks.

# gpart show
=>       40  234441568  ada1                        GPT  (112G)
         40       1024                           1  freebsd-boot  (512K)
       1064        984                              - free -  (492K)
       2048    4194304                           2  freebsd-swap  (2.0G)
    4196352  230244352                           3  freebsd-zfs  (110G)
  234440704        904                              - free -  (452K)

Then you just install the new bootcode from the install media with the following command (change ada1 to match your disk, and also pay attention to the partition number which is addressed with the -i option - you need the "freebsd-boot" partition, which is the first partition in this example):

# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
partcode written to ada1p1
bootcode written to ada1

Then repeat the command for the mirror disks, if you have that.

That's it. You can now reboot the machine and your zpool will work again.

Email from Aldo Gonzalez (shortened)

I actually had an issue with booting up a VM using EFI and had to boot from a live cd and run:

# zpool import zroot
# mkdir -p /tmp/esp/
# mount -t msdosfs /dev/da0p1 /tmp/esp
# cp /tmp/esp/EFI/BOOT/bootx64.efi /tmp/esp/EFI/BOOT/bootx64.efi.bak
# cp /boot/loader.efi /tmp/esp/EFI/BOOT/bootx64.efi
# reboot