U-Boot with direct NVMe boot support for eMMC/SPI Flash

[pcm720]

Update (08/07/2020):
Now there's also an option to use mainline U-Boot (v2020.07) with reference ARM Trusted Firmware, no need to use Rockchip blobs anymore.
You can build it yourself using this repository and flash the image as usual (flash_erase + dd works for Manjaro):

https://github.com/pcm720/u-boot-build-scripts

The build script uses patches from dhivael, sigmaris and Manjaro.
v2020.07 binaries for SD/eMMC and SPI Flash are available as well, you can find them on Releases page of the repo.
The resulting SPI image works on my PBP and boots from NVMe just fine.
Boot priority is SD -> eMMC -> NVMe -> USB, and there's no need to destroy U-Boot on SD card anymore.

---------
While it's possible to place the boot partition on eMMC and use NVMe as rootfs, the lack of NVMe support in stock U-Boot kept bugging me, so I decided to do something about it.

So, by porting Rockchip PCIe patches from Radxa and backporting some NVMe patches from upstream U-boot, I was able to get NVMe support going on mrfixit2001's fork and now can successfully boot from my NVMe drive:



You can find binary images for SD/eMMC/SPI flash and source code here.
Note that the current SPI image has issues when booting pre-made images from SD card.

For some reason Rockchip's SPL tries (and fails) to initialize trust mode with the microSD, so you'll have to zero out the bootloader, U-Boot and ATF on your SD card with the following commands:

dd if=/dev/zero bs=32k seek=1 count=1 of=<microSD>
dd if=/dev/zero bs=64k seek=128 count=64 of=<microSD>
dd if=/dev/zero bs=64k seek=192 count=64 of=<microSD>

The first command corrupts Rockchip's SPL, and second and third zero out U-Boot and ATF, respectively.

In other words, the microSD must not contain any of the bootcode. The same most likely goes for the eMMC.

As usual, I am not responsible for any damage or data loss caused by using these images. They work for me.
Don't flash it if you don't know how to recover from damaged bootloader and can't be bothered to search for recovery options.

If you're flashing the SPI image, make sure that all 16 Mbytes are accessible and that it can be flashed properly before doing anything.
If something doesn't look right, try using Manjaro. That's what I used to flash it to my device.

[Arglebargle]

While it's possible to place the boot partition on eMMC and use NVMe as rootfs, the lack of NVMe support in stock U-Boot kept bugging me, so I decided to do something about it.

So, by porting Rockchip PCIe patches from Radxa and backporting some NVMe patches from upstream U-boot, I was able to get NVMe support going on ayufan's fork and now can successfully boot from my NVMe drive:


Of course, since SPI flash is a no-go for now, U-Boot lives on eMMC: you still have to keep it connected. But that's a start.

You can find binary images and source code here.
As usual, I am not responsible for any damage or data loss caused by using these images. They work for me.
Don't flash it if you don't know how to recover from damaged bootloader and can't be bothered to search for recovery options.

What order is U-Boot configured to check devices for extlinux.conf/boot scripts? If you haven't already could you set it so that it looks for a boot script/config from SD > NVMe > eMMC ?

I'm writing install scripts for the Ubuntu images to make the process less error-prone and it'd be hugely helpful if we could get as many of the PBP U-Boot boot orders aligned so that they do roughly what most users expect when they pop an SD card in to boot.

Thanks for working on this, apologies for immediately piling on and asking for more.

[pcm720]

What order is U-Boot configured to check devices for extlinux.conf/boot scripts? If you haven't already could you set it so that it looks for a boot script/config from SD > NVMe > eMMC ?

I'm writing install scripts for the Ubuntu images to make the process less error-prone and it'd be hugely helpful if we could get as many of the PBP U-Boot boot orders aligned so that they do roughly what most users expect when they pop an SD card in to boot.

Thanks for working on this, apologies for immediately piling on and asking for more.

Right now, it's set to boot from NVMe first, then SD, then eMMC.
I suppose prioritizing SD makes sense. I'll change that as soon as I have the time.

[Arglebargle]

What order is U-Boot configured to check devices for extlinux.conf/boot scripts? If you haven't already could you set it so that it looks for a boot script/config from SD > NVMe > eMMC ?

I'm writing install scripts for the Ubuntu images to make the process less error-prone and it'd be hugely helpful if we could get as many of the PBP U-Boot boot orders aligned so that they do roughly what most users expect when they pop an SD card in to boot.

Thanks for working on this, apologies for immediately piling on and asking for more.

Right now, it's set to boot from NVMe first, then SD, then eMMC.
I suppose prioritizing SD makes sense. I'll change that as soon as I have the time.

Thanks! That'll help keep users out of trouble when they manage to mangle something and need to rescue the system from an sd card.

[pcm720]

I rebased these patches onto mrfixit2001's fork and updated the original post.
Boot times are now significantly faster, power LED also turns on as soon as U-boot starts to run boot scripts.
Basically, now it's the same stock U-Boot experience, but with NVMe support

Thanks! That'll help keep users out of trouble when they manage to mangle something and need to rescue the system from an sd card.

I also implemented your suggestion.

[mamboman777]

Watching this thread and hoping that this can be implemented and supported for all users (including relatively novice users such as myself). I would love to see a video of emmc vs nvme boot times!

[xalius]

nice work just browsing your github repo... do you plan to upstream this to BSP or mainline u-boot?

[pcm720]

nice work just browsing your github repo... do you plan to upstream this to BSP or mainline u-boot?

Thanks! I did create a pull request in mrfixit2001's repo, but I don't plan on upstreaming it further.
I didn't write that Rockchip PCIe driver and have almost no experience with ARM SoCs and Rockchip in particular, so I feel that it's not up to me to submit this anywhere else.
Although seeing mainline U-Boot running on Pinebook Pro would be really nice...

[danielt]

Is there a build guide anywhere? I got as far as u-boot.bin but am not sure how to massage that into u-boot.img (nor whether there are any changes to idbloader.img and trust.img).

[tophneal]

Of course, since SPI flash is a no-go for now, U-Boot lives on eMMC: you still have to keep it connected. But that's a start.

Pardon my ignorance, but why is u-boot on SPI a no-go for us right now? I've noticed that other rk3399 hardware has achieved it (Khadas Edge has a script to flash u-boot to SPI in their wiki.) Would it not be possible to adapt these existing tools to flash SPI on the PBP?

Awesome work on this modification, btw! I'm looking forward to loading it onto my CrOs install today.