Subject: [bug#79248] GCD006: Rewrite bootloader subsystem

From: Denis 'GNUtoo' Carikli <[email-masked]>
Date: Sun, 17 Aug 2025 21:58:22 +0200

Hi,

On Fri, 15 Aug 2025 15:13:49 -0500
Lilah Tascheter <[email-masked]> wrote:
> this is a draft; if anyone has suggestions at this stage, [...]

I've some.

I'm a bit confused in the state of support of non-x86 support in Guix,
and as I understand just reading the code is insufficient as we need to
take project decisions to clarify what we want to have at a given time
in the first place. I'm also confused with this proposal for the same
reasons.

The quick summary of my problem is that there seems to be very
important undocumented behavior, so this makes it hard to understand
how to fix or modify because we don't even know what we're supposed to
have right now, so because of that even discussing changes is hard.

All that is not hard to fix with a very fast discussion ("this was the
intended behavior" from people who contribute to u-boot-bootloader),
which then enables to send patches to clarify that in the code and/or
documentation, which later enables to discuss and fix things (because
we know what we have).

Background:
-----------

Here's a first level of abstraction of what we have:

    User configuration (bootloader-configuration / image type)
    and documentation (guix manual)
                      ^
                      |
                      |
                      v
    Guix code (grub-efi32-bootloader) and bootloaders(upstream GRUB,
    u-boot, arm-trusted-firmware, etc).
                      ^
                      |
                      |
                      v
    Hardware and software interfaces to the bootloaders (BIOS, UEFI
    32bit on x86_64, etc).

In this model and in the current code/documentation, there are some
missing things, like there is no "coreboot + GRUB" (users can still
make it work by writing a custom installer function) or "Bios Boot
partition with BIOS and GPT" image types but as I understand, this
could just be fixed by adding new image types and configuration, so
the current model doesn't break with these.

Though I'm also unsure how to properly add a GRUB entry for something
lime memtest86+ as I cannot put ungexp in the grub-menu definitions.
Since I don't know how to fix, I cannot tell if it breaks the current
model or not (the fix could just be to add something like (thunked)
somewhere, I didn't look into it enough to know if that can work).

Bootflow issue:
---------------

How u-boot boots is documented in doc/develop/distro.rst and to make
things simple, some time ago u-boot tried to load a boot.scr and if
not found then loaded a syslinux configuration that Guix
generated.

There were also additional ways of booting but they require to do
things to support them, so it was and it is still obvious that Guix
doesn't support them yet.

However u-boot then introduced the bootflow mechanism and things
happened in Guix and boot was then broken for some ARM computers.

The "fix" here is to document what the people who contributed to this
bootloader API wanted in the first place (syslinux or bootflow or
something else), so we can send patches to document it in the manual
or code and send patch to make things working again and/or decide what
we want to change and document that. The "Background" above, Guix
manual or code look insufficient to understand that.

The problem is also that I'm unsure if the way of booting of each
board is that unified. For instance some computers (like the TBS2910)
also have compatibility with the vendor u-boot configuration.

For instance that computer has that:
> CONFIG_BOOTCOMMAND="mmc rescan; if run bootcmd_up1; then run
> bootcmd_up2; else run bootcmd_mmc || run distro_bootcmd; fi"

So it will run this code at boot instead of using the bootflow, so if
we supported it in Guix, users won't see any issues while other board
could see issues with bootflow.

Another example is that I want to add the rock-4c-plus to Guix. And
right now it hangs at boot because bootflow doesn't know where to boot
from. Should I make sure it uses bootflow before using syslinux.cfg? Or
is just using syslinux.cfg like before okay? Are other supported boards
working?

Given all that mess, the solution here is to at least document what we
have, and document things if we intend to change them, so when things
change unexpectly, we could simply revert to the old behavior or
document the new one, making it easy to fix.

Bootrom questions:
------------------

On ARM, Bootroms often support multiple way of booting (like multiple
offsets from the start of the block device, MBR + fat32 partition, etc),
and sometime a given way is incompatible with what the user wants (GPT
instead of MBR). I'm unsure if Guix managed to not have situations that
shows this or not.

Your proposal seems to address that at first sight but it doesn't
mention it as such and I wonder how it would look like practically
speaking.

Having some way to report the constraints (like comments or records for
SOCs bootroms / BIOS / EFI might) might work too as we need
documentation somewhere in order to know the tradeoffs we are
supposed to be doing.

An example is that (reordered to fix 70 characters limit):
> (define install-beaglebone-black-u-boot
> ;;http://wiki.beyondlogic.org/index.php?title=BeagleBoneBlack_Upgrading_uBoot
> ;; This first stage bootloader called MLO (U-Boot SPL) is expected at
> ;; 0x20000 by BBB ROM code. The second stage bootloader will be
> ;; loaded by the MLO and is expected at 0x60000.  Write both first
> ;; stage ("MLO") and second stage ("u-boot.img") images, read in
> ;; BOOTLOADER directory, to the specified DEVICE. 
> (write-u-boot-image '(("MLO" 256 256) ("u-boot.img" 768 1024)) 512))

While this looks well documented, there is a lot of unknowns there. The
OMAP3/OMAP4/AM335x, etc SOCs support multiple offsets and MLO can also
be on a FAT partition, with special requirements for the partitioning
table. 

All that is typically documented in u-boot and/or in the hardware
documentation of the SOC (the name of the pdf often changes depending on
the vendor, here it could be "Technical reference manual", and sometimes
it's called "Datasheet" while other time "Datasheet" is just some basic
facts about the chip, and sometimes this is reverse engineered, so it
could even be in a wiki or in u-boot).

And an additional issue is that as far as I know, the OMAP3430 has
have less offsets available than an AM335x. So these could also be
supported by the same code if we chose to.

Now these are important to document I think, especially when the offset
can end up inside GPT or MBR partitions, especially if the u-boot image
becomes too big over time (if we don't already, we'd need a check for
that, and your proposal seems to enable to do that in a more elegant
way).

So where can we document that? Is a big comment in the same file the
right place to do that? Or can we somehow move that into some structure
and abstractions where we clearly define why we choose this method over
others?

This also brings the next question about /boot. In some cases that
looks 100% controlled by Guix and it is re-generated during
reconfiguration in certain cases. Is that on purpose? Is it documented
somewhere? Is this because on x86 there is update-grub that can find
the bootloader but that, while code exists for some ARM SOCs (like some
Allwinner chips), there is no tool to just do that (no update-u-boot)?

New features that can potentially require API changes:
------------------------------------------------------

If we change the user facing API, how could we handle chained
bootloaders?

For instance [u-boot+EFI emulation] -> GRUB is possible. TowBoot does
it. There are also other interfaces possible between u-boot and GRUB.

And this is relevant to users here as "I want GRUB on ARM with u-boot"
is not some implementation details, how "GRUB on ARM with u-boot" is
done could be though. 

The user could also already have UEFI (for arm64) on the computer, so
the user should be able to also select "I have UEFI, I just want GRUB
arm64 like it is done on x86".

Should we have lists of bootloaders where the first in the list loads
the next one? Should we have some u-boot-then-grub bootloader? Do we
even need to think about this now?

This also brings the question of where the bootloader is installed and
possible variations of booting due to that. For instance a bootloader
on an SPI flash is different from one on a microSD and they might even
require to be built differently due to different load addresses.

However we could simply choose to not handle that yet given the
challenge that we already have for the rest.

In addition projects like GNU Boot could in the long run take care of
things that go in the SPI flash, assuming that some people are
interested in doing that.

Denis.