Netbooting Mac Mini over MAAS

Metal As A Service

Jan 28, 2023

Some time ago, I was setting up a home lab managed by Metal-As-A-Service. MAAS is yet another product from Canonical - the same company that makes Ubuntu. This tool is dedicated to very fast server provisioning for data centers. It can turn real servers into the bare metal cloud. With just a few clicks, you can provision a fresh server or create a new Virtual Machine. And that's exactly what I needed.

The process of setting up a home lab itself is quite a different story and I will write about it sometime later. And for now, I would like to focus on one bit that made my nights sleepless - booting Mac Mini from the network.

MAAS controller provides its own Dynamic Host Configuration Protocol (DHCP) service. Once it is enabled, it not only offers IP assignment but also provides Operating System images (for instance, Ubuntu). Whenever the device boots from the network, it obtains a proper image and performs the cloud-init process. Then the device is registered at the MAAS controller, so it can be properly provisioned.

NetBoot

Apple provided its own way of booting devices from a network called NetBoot. It is based on a protocol called Boot Server Discovery Protocol (BSDP) that was loosely developed from DHCP and BootP.

NetBoot involves the following steps:

The computer is instructed to boot using NetBoot. For example, pressing N-key will instruct the computer to boot from the first available NetBoot server.
The computer uses DHCP to request an IP address and related information for the NetBoot process (steps 1-4).
The computer broadcasts a BSDP: LIST request on the local subnet to locate a NetBoot server and available OS images (steps 5-6).
The computer informs the NetBoot server about the selected image (steps 7-8).
The computer uses TFTP to download the boot file and the Mac boot process initiates.

BSDP protocol is embedded in the DHCP frame. Option 43 is used to carry vendor-specific options.

NetBooting over MAAS

Now coming back to Mac Mini. In principle, booting Mac from the network should be quite easy. After tunning on the device, just hold the N key to perform a network boot. However, when following that path, it turns out that actually nothing happens. After a few seconds, Mac starts to boot from the disk instead of the network. It is due to the fact MAAS lacks a BSDP boot sequence.

Therefore, we need to add a few DHCP snippets to MAAS. It can be done in DHCP snippets in the Settings section. Please refer to the screen below.

First of all, we must make the DHCP server authoritative. It can be achieved by adding the following snippet to the subnet configuration of MAAS.

authoritative;

This will make MAAS respond to DHCP INFORM with DHCP ACK. However, we still lack the BSPD protocol part.

We have to add another snippet to the subnet configuration.

class "netboot" {
    match if substring (option vendor-class-identifier, 0, 9) = "AAPLBSDPC";
    option dhcp-parameter-request-list 1,3,17,43,60;

    if (option dhcp-message-type = 1) {
        option vendor-class-identifier "AAPLBSDPC";
        option vendor-encapsulated-options
            08:04:81:00:00:89;
    } elsif (option dhcp-message-type = 8) {
        option vendor-class-identifier "AAPLBSDPC";
        if (substring(option vendor-encapsulated-options, 0, 3) = 01:01:01) {
            option vendor-encapsulated-options
              01:01:01:04:02:80:00:07:04:81:00:00:89:09:0e:81:00:00:89:09:54:68:65:2d:49:6d:61:67:65;
        } else {
            option vendor-encapsulated-options
                01:01:02:08:04:81:00:00:89:82:09:54:68:65:2d:49:6d:61:67:65;

            if (substring(option vendor-class-identifier, 10, 4) = "i386") {
                next-server 192.168.111.2;
		filename "mactel64.efi";
            }
        }
    }
}

When adding a netboot snippet, we must provide the following properties:

next-server that is an IP address of MAAS (in my case it is 192.168.111.2), and
filename file used to boot the device (files are located in /var/lib/maas/boot-resources/current/ on MAAS).

Once we add it, we get the full NetBoot sequence. However, we still have one more challenge to solve. MacMini still won't boot as it gets stuck in the GRUB prompt.

Custom EFI for MacMini

It turns out that net_default_interface in GRUB is not set. Therefore, Mac doesn't know what to do next. To solve it, we have to provide our own EFI. Here comes the grub.cfg file I used to create a new EFI file.

insmod memdisk
insmod efi_gop
insmod efi_uga
insmod video_bochs
insmod video_cirrus
insmod all_video
set gfxpayload=keep
insmod gzio
insmod part_gpt
insmod ext2
insmod net
insmod efinet
insmod http
insmod linuxefi

net_bootp efinet0
set net_default_interface=efinet0_dhcp
export net_default_interface

set timeout=1
set default=0
set fallback=1
echo 'Loading Linux on Mac...'
echo ${net_default_mac}
set root=http,${net_efinet0_dhcp_next_server}:5248
configfile /grub/grub.cfg-${net_default_mac}
configfile /grub/grub.cfg-default-amd64

Now we can create EFI by issuing the following command:

grub-mkimage -v -d /usr/lib/grub/x86_64-efi/ -O x86_64-efi -p '(tftp)/boot/grub' -o ./mactel64.efi -c ./grub/grub.cfg-mactel.efi normal configfile net efinet tftp http efi_gop efi_uga all_video gzio part_gpt ext2 echo linux linuxefi memdisk eval sfs regexp

Once it's done, we get mactel64.efi and we need to copy it to the correct directory.

sudo cp mactel64.efi /var/lib/maas/boot-resources/current/

💡 I believe that there is some other way of registering custom-made EFI files to MAAS with the maas CLI tool. However, I haven't explored that option yet.

Now, we have all stuff in place to boot up MacMini correctly under MAAS control. Have fun!