U-Boot: how to load U-Boot from SD card, but have it load its Environment from QSPI Flash?

Question

I'm having some difficulty configuring U-Boot, in particular setting it up so that it can load from an SD card, but access its Environment from QSPI Flash.

First of all, is this even possible? Can the U-Boot environment be stored in a different location from where U-Boot was loaded?

Currently, I have a U-Boot image running from QSPI Flash on a Xilinx MPSoC ARM64 platform. I'm using Xilinx PetaLinux however I think I understand how this configures U-Boot "under the hood" so I'll try to refer to uboot.cfg​ as much as I can.

Along with all the defaults, I have set:

#define CONFIG_ENV_OFFSET 0x2E00000
#define CONFIG_ENV_OVERWRITE 1
#define CONFIG_CMD_ERASEENV 1

On this particular platform, there is a physical switch to switch the Xilinx first-stage bootloader (and ARM Trusted Firmware) between QSPI Flash and SD card modes. This platform also requires a so-called "BOOT.BIN", which is an aggregation of the u-boot binary, the Xilinx FSBL, ARM Trusted Firmware, an FPGA image and other things, to be written to QSPI Flash, which I have done. This is all working correctly, and the serial console looks like this at reset or power-on:

Xilinx Zynq MP First Stage Boot Loader
Release 2021.2   Oct 13 2021  -  07:15:53
NOTICE:  BL31: v2.4(release):xlnx_rebase_v2.4_2021.1_update1-23-g9188496b9
NOTICE:  BL31: Built : 07:41:24, Oct 13 2021


U-Boot 2021.01 (Oct 12 2021 - 09:28:42 +0000)

CPU:   ZynqMP
Silicon: v3
Model: ZynqMP ZCU208 RevA
Board: Xilinx ZynqMP
DRAM:  4 GiB
PMUFW:  v1.1
EL Level:       EL2
Chip ID:        zu48dr
NAND:  0 MiB
MMC:   mmc@ff170000: 0
Loading Environment from SPIFlash... SF: Detected mt25qu02g with page size 512 Bytes, erase size 128 KiB, total 512 MiB
...

As you can see, U-Boot loads from QSPI Flash and the Environment is also loaded from QSPI Flash. All good.

What I want to do is modify the system so that the First Stage Boot Loader loads U-Boot from the SD card. In my config I already have CONFIG_ENV_IS_IN_FAT set to 1. When I copy BOOT.BIN to the first, FAT partition on an SD card, and set my platform's boot mode to "SD Card" via the physical switch, the unit loads U-Boot from the SD card successfully, and the serial console shows output similar to the above but with:

Loading Environment from FAT... *** Warning - bad CRC, using default environment

It works, although this warning is expected because there is no Environment saved on the SD card yet. It does show that the SD card is being checked for the Environment though, so that's good.

Note that this is the same build that I used for the QSPI flash boot earlier, but it's looking for the config on FAT, so I'm currently under the impression that U-Boot only looks for its Environment based on where it was loaded from, even if other locations are enabled. I also tried in QEMU and it seemed to think that the config came from "nowhere", but that may be a distraction.

---

The issue is that I would like to configure U-Boot to be loaded from the SD card (for ease of upgrade / development), but to use QSPI for its Environment, so that I can store persistent U-Boot configuration on the board itself.

In particular, the MAC address for the board is not readable via software so must be manually configured in U-Boot, and I'd like that to follow around the board, not the SD card. There are a few other options I'd like to set per-board also.

According to the U-Boot documentation I've read, disabling FAT as the default should be achievable by unsetting the following in u-boot.cfg :

CONFIG_ENV_IS_IN_FAT
CONFIG_SYS_MMC_ENV_DEV
CONFIG_ENV_FAT_DEVICE_AND_PART
CONFIG_ENV_IS_IN_FAT
CONFIG_ENV_FAT_FILE
CONFIG_SYS_MMC_ENV_PART
CONFIG_ENV_FAT_INTERFACE

Performing the same installation steps as above, to have this new U-Boot loaded from the SD card, results in the following output:

Xilinx Zynq MP First Stage Boot Loader
Release 2021.2   Oct 13 2021  -  07:15:53
NOTICE:  BL31: v2.4(release):xlnx_rebase_v2.4_2021.1_update1-23-g9188496b9
NOTICE:  BL31: Built : 07:41:24, Oct 13 2021

And that's it! No more output. There is an LED on the board that goes from red to green, which I think indicates that the FPGA image within BOOT.BIN has been loaded into the gate array successfully, so I think the earlier boot loader has found BOOT.BIN. I have no reason therefore to think that U-Boot has not also been loaded, but if it has, it is generating no output. I tried increasing the debug level of U-Boot to 7 (DEBUG) in the hope of seeing something from U-Boot, but that makes no difference. It's also not booting Linux, which would normally happen next.

Can anyone shed any light on why making this change might result in this behaviour? Is there some other configuration option I need to set to ensure that U-Boot can run from the SD card when CONFIG_ENV_IS_IN_FAT and friends are not set? Or perhaps I can keep CONFIG_ENV_IS_IN_FAT etc. and there's some other way to instruct U-Boot to use the QSPI Flash for its environment, even though it was loaded from SD card?

Note: I've also enabled ATF debugging, which produces the following output, but it still hangs with no sign of U-Boot:

INFO:    PSCI Power Domain Map:
INFO:      Domain Node : Level 1, parent_node -1, State ON (0x0)
INFO:      Domain Node : Level 0, parent_node 0, State ON (0x0)
INFO:      Domain Node : Level 0, parent_node 0, State ON (0x0)
INFO:      Domain Node : Level 0, parent_node 0, State ON (0x0)
INFO:      CPU Node : MPID 0x0, parent_node 0, State ON (0x0)
INFO:      CPU Node : MPID 0x1, parent_node 0, State ON (0x0)
INFO:      CPU Node : MPID 0x2, parent_node 0, State ON (0x0)
INFO:      CPU Node : MPID 0x3, parent_node 0, State ON (0x0)
Xilinx Zynq MP First Stage Boot Loader
Release 2021.2   Oct 13 2021  -  07:15:53
WARNING: BL31: invalid exception level (3)
NOTICE:  BL31: v2.4(debug):xlnx_rebase_v2.4_2021.1_update1-23-g9188496b9
NOTICE:  BL31: Built : 07:41:24, Oct 13 2021
INFO:    ARM GICv2 driver initialized
INFO:    BL31: Initializing runtime services
INFO:    BL31: PM Service Init Complete: API v1.1
INFO:    BL31: cortex_a53: CPU workaround for 855873 was applied
WARNING: BL31: cortex_a53: CPU workaround for 1530924 was missing!
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x8000000
INFO:    SPSR = 0x3c9

Answer 1

Answering my own question:

I do not know why the console remains blank/dead in this case, but the related code is this function here:

https://github.com/u-boot/u-boot/blob/master/board/xilinx/zynqmp/zynqmp.c#L850

According to this switch statement, when the boot mode is SD_MODE , the case does not consider CONFIG_ENV_IS_IN_SPI_FLASH at all, and therefore the function returns ENVL_NOWHERE when booted from SD with CONFIG_ENV_IS_IN_FAT disabled.

Modifying this function to return ENVL_SPI_FLASH in this case enables U-Boot access to the Environment in the QSPI Flash:

    case SD_MODE1:
        if (IS_ENABLED(CONFIG_ENV_IS_IN_FAT))
            return ENVL_FAT;
        if (IS_ENABLED(CONFIG_ENV_IS_IN_EXT4))
            return ENVL_EXT4;
++      if (IS_ENABLED(CONFIG_ENV_IS_IN_SPI_FLASH))
++          return ENVL_SPI_FLASH;
        return ENVL_NOWHERE;