PinePhone won't boot, U-Boot seems to be broken

[Marcade]

Good day all,

I have a PinePhone Manjaro Community Edition that suddenly stopped working. In looking around on the forums, I think the cause of the failure was a fully discharged battery (which I confirmed by testing the battery using a multi-meter). Leaving it plugged over the weekend did not seem to fix the problem so I ordered an external battery charger (and a serial cable for good measure). Seems that was a wise decision because even with a fully charged battery, the phone does not boot; neither from eMMC nor from the SD card.

When I connect the serial cable and try to boot up the phone, I see the following output:

Code:

U-Boot SPL 2020.07-rc4-2 (Nov 11 2020 - 21:49:00 +0000)
DRAM: 3072 MiB
Trying to boot from MMC2
NOTICE:  BL31: v2.3(debug):9a34348
NOTICE:  BL31: Built : 21:49:00, Nov 11 2020
NOTICE:  BL31: Detected Allwinner A64/H64/R18 SoC (1689)
NOTICE:  BL31: Found U-Boot DTB at 0x4064908, model: Pine64 PinePhone (1.2)
INFO:    ARM GICv2 driver initialized
INFO:    Configuring SPC Controller
INFO:    PMIC: Probing AXP803 on RSB

This all seems normal except it never proceeds from there (I would expect more output). The output is the same whether I'm trying to boot from eMMC or the SD card.

Does this mean that the U-Boot on my device is corrupted? If that's the case, how would I go about flashing a new boot loader. The only guides I have found to do that so far seem to require that the device be booted already.

[Marcade]

I did some more digging, and I'm starting to think that the issue is that the power management block is dead. I found the data sheet for the AXP803 part. I'm now understanding that this is the part responsible for power management (I'm going to take a guess that PMIC stands for Power Management Integrated Circuit).

Therefore it appears that the problem is not with U-Boot but with the AXP803. U-Boot is trying to probe the AXP803 and it hangs there. Now I need to figure out how to kick the AXP803.

Any suggestions?

[dsimic]

Does your phone power on automatically upon plugging in a "dumb" USB charger, i.e. a charger that conforms to the USB Battery Charging 1.2 specification?  As a note, most USB chargers that do not have a Type-C port are of the "dumb" variety.

If the phone doesn't power on automatically as described above, most probably there's something wrong with the AXP803 (which is also commonly referred to as the power management IC or PMIC for short).  The PMIC is hardwired internally to power on the phone automatically upon detecting a valid power source.

[Marcade]

My device powers on when I connect it to a USB charger and I get the same information output to the UART as when I power on with the battery: the red LED comes on and stays lit while U-Boot is probing the AXP803.

NOTE: I have performed this test with the battery removed.

From this information is it safe to conclude that the AXP803 chip is still operational?

This would bring me back to my first hypothesis that there is something wrong with the U-Boot initialization. From what I can understand from the serial output, the system is starting with the Boot ROM, then successfully running the secondary program loader, then the ARM trusted firmware, but it's hanging on BL33 (U-Boot proper) before even attempting to load Linux.

Thanks,
M

[dsimic]

So far, it can be concluded only that the PMIC isn't completely dead.

Could it be that the U-Boot somehow got corrupted on your phone?  I'd suggest that you try booting from a freshly flashed microSD card, and see what happens.  Also, please put the battery back into the phone, it's required to power some parts of the phone, regardless of the charger being connected or not.

[Marcade]

I tried booting the device using the latest Jumpdrive image. It appears to hang while initializing the ARM trusted firmware, or immediately after:

Code:

Trying to boot from MMC1
NOTICE:  BL31: v2.2(release):v2.2-904-gf9ea3a629
NOTICE:  BL31: Built : 15:32:12, Apr  9 2020
NOTICE:  BL31: Detected Allwinner A64/H64/R18 SoC (1689)
NOTICE:  BL31: Found U-Boot DTB at 0x4064410, model: PinePhone

What's curious is that the LED does not turn on when I use the Jumpdrive image as it does when I tried to boot Manjaro. I can't remember if that's normal (i.e. the last time I used Jumpdrive to flash the eMMC).

This problem has be a bit stumped, if I can't get passed U-Boot, I may not be able to fix U-Boot (assuming the problem is with the boot loader and not the PMIC).

Thanks,
M

[dsimic]

Please, also try booting from a freshly flashed PostmarketOS or Manjaro ARM image. It's worth a try.

[Marcade]

I flashed a new version of Manjaro (the plasma mobile edition) and the phone successfully booted. However, it took two attempts to boot using the SD card. On the first boot it managed to complete the initialization of the PMIC, but it locked up when loading the Linux kernel.

On the second attempt, the OS booted and I was able to go through the setup. I then shut it down, removed the SD card and the phone successfully booted from the eMMC. I've restarted twice now. Things seem to be mostly back to normal. However, I see the following error on the serial console when I boot the OS on my eMMC:

Code:

U-Boot 2020.07-rc4-2 (Nov 11 2020 - 21:49:00 +0000) Manjaro Linux ARM

CPU:   Allwinner A64 (SUN50I)
Model: Pine64 PinePhone (1.2)
DRAM:  3 GiB
MMC:   Device 'mmc@1c11000': seq 1 is in use by 'mmc@1c10000'
mmc@1c0f000: 0, mmc@1c10000: 2, mmc@1c11000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

In:    serial@1c28000
Out:   serial@1c28000
Err:   serial@1c28000

Could this indicate that there is a problem with the image on the eMMC?

Then later the following error is displayed:

Code:

Starting kernel ...

[    0.732177] SMCCC: SOC_ID: ARCH_SOC_ID(0) returned error: ffffffffffffffff

I'm going to guess that this means that the system on chip ID is not being returned correctly.

The other thing that I find curious is how the particular version of Manjaro that I used kicked the device into life. Having tried a fresh Jumpdrive and PostmarketOS image without success previously.

In any case, the issue seems to be resolved for the moment.

Thank you very much for your advice.

[dsimic]

I flashed a new version of Manjaro (the plasma mobile edition) and the phone successfully booted. However, it took two attempts to boot using the SD card. On the first boot it managed to complete the initialization of the PMIC, but it locked up when loading the Linux kernel.

On the second attempt, the OS booted and I was able to go through the setup. I then shut it down, removed the SD card and the phone successfully booted from the eMMC. I've restarted twice now. Things seem to be mostly back to normal.

Great results!  The issues were obviously caused by the Jumpdrive image; in particular, the root cause was surely the who-knows-what version of U-Boot it is using.  One of the possible issues that come to mind is that the U-Boot used in Jumpdrive lacks proper support for the PinePhones with 3 GB of RAM.  Supporting the 3 GB variant requires some "trickery" to get the A64 SoC to detect and use different ranks in a single DRAM chip.

By the way, Manjaro ARM uses the latest available version of U-Boot, with a set of patches (check out the sources of uboot-pinephone and uboot-pinephone-crust packages), so it's always best to verify or test the things out using the latest Manjaro ARM image.

It is curious why it took two boots to get the phone working...  On the first attempt you've most probably encountered a strange bug that sometimes prevents the phone from booting, which requires the phone to be forcedly powered off (long press on the power button) and then turned back on.  I've also experienced that bug on my PinePhone a few times, but I haven't managed to debug that (yet).

However, I see the following error on the serial console when I boot the OS on my eMMC:

Code:

U-Boot 2020.07-rc4-2 (Nov 11 2020 - 21:49:00 +0000) Manjaro Linux ARM

CPU:   Allwinner A64 (SUN50I)
Model: Pine64 PinePhone (1.2)
DRAM:  3 GiB
MMC:   Device 'mmc@1c11000': seq 1 is in use by 'mmc@1c10000'
mmc@1c0f000: 0, mmc@1c10000: 2, mmc@1c11000: 1
Loading Environment from FAT... *** Warning - bad CRC, using default environment

Could this indicate that there is a problem with the image on the eMMC?

It seems that U-Boot is unable to find a valid environment file (which is OK not to be present) on a FAT-formatted partition (i.e. /boot) and ends up printing the "*** Warning - bad CRC, using default environment" error mesage in the env_set_default() function, which is called from the env_import() function, which, in turn, is called from the env_fat_load() function.

Could you, please, post a detailed list of files in the /boot directory on your PinePhone, upon booting it up with this error message present in the U-Boot output?  That will allow a better insight into the issue.  Anyway, this isn't something to worry too much about.

Then later the following error is displayed:

Code:

[    0.732177] SMCCC: SOC_ID: ARCH_SOC_ID(0) returned error: ffffffffffffffff

I'm going to guess that this means that the system on chip ID is not being returned correctly.

I'm really not an expert in that field, but this seems to be an error generated by a lacking feature in the phone firmware; see this article for more information.  However, I wouldn't worry too much about this error message for now.

The other thing that I find curious is how the particular version of Manjaro that I used kicked the device into life. Having tried a fresh Jumpdrive and PostmarketOS image without success previously.

As already noted, Manjaro ARM uses the latest available version of U-Boot, while Jumpdrive and PostmarketOS probably don't.  In fact, PostmarketOS should be using the latest version of U-Boot, but I don't know for sure; their build system is so convoluted that I gave up on trying to get myself familiar with it.

In any case, the issue seems to be resolved for the moment.
Thank you very much for your advice.

You're welcome.