+- PINE64 (https://forum.pine64.org)
+-- Forum: Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=111)
+--- Forum: General Discussion on Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=112)
+--- Thread: How can I flash SPI so that I can boot from NVMe? (/showthread.php?tid=14629)
How can I flash SPI so that I can boot from NVMe? - codingpanic - 08-10-2021
I've done several searches and there is one SPI package that appears to enable NVMe boot, but the repo was archived by the developer.
Is there any way to set this up so NVMe is the default boot drive? Without using the emmc at all?
Thanks
RE: How can I flash SPI so that I can boot from NVMe? - calinb - 08-11-2021
(08-10-2021, 04:11 PM)codingpanic Wrote: I've done several searches and there is one SPI package that appears to enable NVMe boot, but the repo was archived by the developer.If you're talking about this thread:
Is there any way to set this up so NVMe is the default boot drive? Without using the emmc at all?
Thanks
https://forum.pine64.org/showthread.php?tid=8439
then you can still download the binary images here or build from source (I think this is PCM720's most recent code but older code is archived elsewhere too):
https://github.com/pcm720/u-boot-build-scripts/releases
I have not tried to flash my SPI yet, but I dowloaded the images and I also built them myself natively on my Pinebook Pro (compile takes several hours IIRC). My native build binaries were a little smaller than PCM720's cross compile. Even though I did my own build I think I'll use PCM720's binary images, if I ever flash my SPI, because other users have had success with them. My native build is untested and I don't really want to risk the compiler differences.
I'm running Manjaro, which can boot from NVMe but it requires the eMMC to be retained and installed in the PPB with U-boot code intact (U-boot code is written to the "tracks" before the boot partition) so, if you want to remove the eMMC entirely, you'll need to flash your SPI memory, as you've indicated. The Manjaro U-boot redirects to the NVMe boot partition when a valid /boot is not found on the eMMC. Both / and /boot are on the NVMe drive however and are not required to be present on eMMC at all (though it's been handy for me too keep / and /boot on my eMMC as a backup). If the eMMC is repartitioned, the first partition must begin after offset to avoid overwriting the U-boot loader area.
Unfortunately Manjaro NVMe boot broke when Manjaro updated the kernel to 5.13 and I have not solved the problem. I edited my pacman.conf file to ignore kernel updates and so I've "pinned" my kernel to 5.12.
Here's my long thread where I tried to isolate the problem and learned to build a test kernel but there's more info there about NVMe boot on Manjaro:
https://forum.manjaro.org/t/nvme-support-in-uboot-pinebookpro-2021-04-1/63774
There's no Manjaro Kernel build howto for Pinbook Pro that I could find and the process is a little different on the Pinebook Pro so my thread is good for learning though my stuggles with a custom kernel package build too.
RE: How can I flash SPI so that I can boot from NVMe? - codingpanic - 08-11-2021
(08-11-2021, 12:30 AM)calinb Wrote:(08-10-2021, 04:11 PM)codingpanic Wrote: I've done several searches and there is one SPI package that appears to enable NVMe boot, but the repo was archived by the developer.If you're talking about this thread:
Is there any way to set this up so NVMe is the default boot drive? Without using the emmc at all?
Thanks
https://forum.pine64.org/showthread.php?tid=8439
then you can still download the binary images here or build from source (I think this is PCM720's most recent code but older code is archived elsewhere too):
https://github.com/pcm720/u-boot-build-scripts/releases
I have not tried to flash my SIP yet, but I dowloaded the images and I also built them myself natively on my Pinebook Pro (compile takes several hours IIRC). My native build binaries were a little smaller than PCM720's cross compile. Even though I did my own build I think I'll use PCM720's binary images, if I ever flash my SIP, because other users have had success with them. My native build is untested and I don't really want to risk the compiler differences.
I'm running Manjaro, which can boot from NVMe but it requires the eMMC to be retained and installed in the PPB with U-boot code intact (U-boot code is written to the "tracks" before the boot partition) so, if you want to remove the eMMC entirely, you'll need to flash your SIP memory, as you've indicated. The Manjaro U-boot redirects to the NVMe boot partition when a valid /boot is not found on the eMMC. Both / and /boot are on the NVMe drive however and are not required to be present on eMMC at all (though it's been handy for me too keep / and /boot on my eMMC as a backup). If the eMMC is repartitioned, the first partition must begin after offset to avoid overwriting the U-boot loader area.
Unfortunately Manjaro NVMe boot broke when Manjaro updated the kernel to 5.13 and I have not solved the problem. I edited my pacman.conf file to ignore kernel updates and so I've "pinned" my kernel to 5.12.
Here's my long thread where I tried to isolate the problem and learned to build a test kernel but there's more info there about NVMe boot on Manjaro:
https://forum.manjaro.org/t/nvme-support-in-uboot-pinebookpro-2021-04-1/63774
There's no Manjaro Kernel build howto for Pinbook Pro that I could find and the process is a little different on the Pinebook Pro so my thread is good for learning though my stuggles with a custom kernel package build too.
I think I may try this firmware on eMMC, that way I can at least boot NVMe directly. This way I dont regrest flashing my SPI. If I do have problems, I can always disable the eMMC and boot off SD card.
Thanks!
RE: How can I flash SPI so that I can boot from NVMe? - calinb - 08-11-2021
(08-11-2021, 05:07 AM)codingpanic Wrote: I think I may try this firmware on eMMC, that way I can at least boot NVMe directly. This way I dont regrest flashing my SPI. If I do have problems, I can always disable the eMMC and boot off SD card.
Thanks!
Yes. Also, given that I'm finding my eMMC to be a good place to store /boot and / backup images from NVMe SSD while I tinker with my kernel on NVME, I decided to wait before attempting to flash my SPI. I have the same functionality and no risk by simply using the NVMe-enabled Manjaro boot loader on my eMMC.
As long as I preserve the Manjaro U-boot loader on eMMC (the 32.0MB before the first partition), I suspect the Manjaro U-boot loader will work with other distros installed to my NVMe SSD too, but I've not tried anything but Manjaro on my NVMe SSD so far.
Code:
$ sudo parted -l
...
Model: MMC DA4064 (sd/mmc)
Disk /dev/mmcblk2: 62.5GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 32.0MB 256MB 224MB primary fat16 lba
2 256MB 62.5GB 62.3GB primary ext4https://forum.manjaro.org/t/nvme-support-in-uboot-pinebookpro-2021-04-1/63774
My Manjaro U-boot seems to have the functionality of Nadia's patch and I don't recall if pcm720's code requires it (I read that long thread over a month ago).
Also, be sure to check to see if you're are really booting from NVMe (/boot and / are reported by lsblk as mounted on my Manjaro system, though I guess boot might not be mounted on other systems after boot so maybe boot logs or dmesg would reveal where they booted from):
Code:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
mmcblk2 179:0 0 58.2G 0 disk
├─mmcblk2p1 179:1 0 213.6M 0 part
└─mmcblk2p2 179:2 0 58G 0 part
mmcblk2boot0 179:32 0 4M 1 disk
mmcblk2boot1 179:64 0 4M 1 disk
zram0 252:0 0 5.7G 0 disk [SWAP]
nvme0n1 259:0 0 931.5G 0 disk
├─nvme0n1p1 259:1 0 213.6M 0 part /boot
└─nvme0n1p2 259:2 0 931.3G 0 part /
RE: How can I flash SPI so that I can boot from NVMe? - wdt - 08-12-2021
>the Manjaro U-boot loader on eMMC (the 32.0MB before the first partition)
Err, no.... baked into the cpu, specifically rk3399 family, the boot rom,,
the boot rom has only a small amount of memory, loads idbloader at sector 64,
as a raw read (based on byte signature),, this activates and enumerates mem on board
So now there is enough memory to load uboot, at 8M
If it is bsp uboot, there is trust.img at 12 M, if mainline trust is enfolded into u-boot.itb
None of this goes past 16M, perhaps the space 16M to 32M is for
a future ESP (efi system partition)
Please note that uboot on SPI is a bit different, idbloader 'expanded' and offset zero,
I vaguely remember uboot at 6M
But the SPI code is a complete image so you don't have to bother with this
RE: How can I flash SPI so that I can boot from NVMe? - codingpanic - 08-24-2021
(08-10-2021, 04:11 PM)codingpanic Wrote: I've done several searches and there is one SPI package that appears to enable NVMe boot, but the repo was archived by the developer.
Is there any way to set this up so NVMe is the default boot drive? Without using the emmc at all?
Thanks
So, to anyone else wondering about this, all I had to do was this:
1. Upgrade emmc install of Manjaro to latest.
2. Install NVMe SSD
3. Flash Manjaro latest to NVMe (only tested with Manjaro so far)
4. Edit /etc/fstab on the NVMe drive to not use UUID of the / and /boot drives, but the Linux Device /dev/nvme0n1pX
5. Reboot, and you are running on the NVMe as / and /boot
6. You definitely want to tune the drive for power consumption as well!
I realize there was lots of work that has led up to this point, but it was surprisingly painless, and the performance is great!