PINE64
soft brick with ayufan - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=111)
+--- Forum: Linux on Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=114)
+--- Thread: soft brick with ayufan (/showthread.php?tid=14440)

soft brick with ayufan - DimHummingbird - 07-15-2021

Hi everyone -

I wanted to install the ayufan buster-containers release onto my PineBook Pro, but now my machine won't boot. The boot LED doesn't even come on.

Here's what I did. Connected to USB-C power the whole time.

1) Use etcher on another machine to flash buster-containers-rock64-0.11.0-1185-arm64.img from https://github.com/ayufan-rock64/linux-build/releases/tag/0.11.0 onto a microSD card. (I previously tried, and couldn't boot with, the "stable" version 0.9.14).

2) Successfully booted from the sdcard on my pinebook pro and logged in.

3) Connected my USB ethernet adapter, and scp'ed over the same image file. Used md5sum to validate the transfer was OK.

4) Checked which device was mounted as root (emmc 1).

5) `dd if=buster-containers-rock64-0.11.0-1185-arm64.img of=/dev/emmc2 bs=64M status=progress`. (I don't remember the exact device naming convention, but it was the tab completed root device of whichever "mmc" was NOT mounted).

6) `sudo reboot now`

7) The machine hung at a blank screen with a single blinking cursor in the top left, so after a couple of minutes I powered off with a long-press of the power key.

Now I can't turn it back on again! I get a charging LED when I plug in USBC power, but no boot LED when I press the power button.From reading other threads and the doco, I have tried:

* an "extra sure hard power off" by holding power for >30 secs (once even > 1min) before trying to power on again
* validating the sd card contents. `p3` does contain what looks like a normal `/boot`. `p2` is an empty vfat. `p1` contains nilfs, fdisk output:
Code:
Disk /dev/mmcblk0: 116,16 GiB, 124721823744 bytes, 243597312 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E96A8F3C-0431-4043-8BFE-2D57A1A4CE67

Device          Start      End  Sectors  Size Type
/dev/mmcblk0p1    64      8191      8128    4M Linux filesystem
/dev/mmcblk0p2  8192    32767    24576    12M Microsoft basic data
/dev/mmcblk0p3  32768    524287    491520  240M Linux filesystem
/dev/mmcblk0p4 524288 243597278 243072991 115,9G Linux filesystem
* flipping the hardware switch to disable the internal emmc

I can't get it to show any sign of life, with or without the sd card. I'm stuck! What can I do next?

RE: dd ayufan to emmc causes no boot, no leds - DimHummingbird - 07-15-2021

UPDATE:
  • I can get into Maskrom mode. Actually I didn’t do anything, I gather it falls back to maskrom because it can’t boot from anything else. The only reason I know it's in Maskrom is because rkdeveloptool says so.
  • Just to be sure, I disabled emmr with the hard switch and removed the sd card. Still no visible sign of life on the device, but rkdeveloptool sees it and says it's in Maskrom mode.
  • Any rkdeveloptool command other than `ld` fails after about 20 seconds, with "Creating Comm Object failed!".
Like this:
Code:
~
❯ rkdeveloptool ld                                                                        [1]
DevNo=1 Vid=0x2207,Pid=0x330c,LocationID=309    Maskrom

~
❯ rkdeveloptool db Downloads/rk3399_loader_spinor_v1.15.114.bin                             
Creating Comm Object failed!

~ took 23s
  • The initial USB connection works OK according to `journalctl -f`:
Code:
Jul 15 14:31:09 banquo kernel: usb 3-9: new high-speed USB device number 8 using xhci_hcd
Jul 15 14:31:09 banquo kernel: usb 3-9: New USB device found, idVendor=2207, idProduct=330c, bcdDevice= 1.00
Jul 15 14:31:09 banquo kernel: usb 3-9: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Jul 15 14:31:09 banquo mtp-probe[2944]: checking bus 3, device 8: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-9"
Jul 15 14:31:09 banquo mtp-probe[2944]: bus: 3, device: 8 was not an MTP device
Jul 15 14:31:09 banquo mtp-probe[2972]: checking bus 3, device 8: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-9"
Jul 15 14:31:09 banquo mtp-probe[2972]: bus: 3, device: 8 was not an MTP device

But after about 20 seconds of any other rkdeveloptool command, the device fails to setup.
Code:
15 14:31:36 banquo kernel: usb 3-9: reset high-speed USB device number 8 using xhci_hcd
Jul 15 14:31:41 banquo kernel: usb 3-9: device descriptor read/64, error -110
Jul 15 14:31:52 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:52 banquo kernel: usb 3-9: reset high-speed USB device number 8 using xhci_hcd
Jul 15 14:31:52 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:52 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:52 banquo kernel: usb 3-9: reset high-speed USB device number 8 using xhci_hcd
Jul 15 14:31:52 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:53 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:53 banquo kernel: usb 3-9: device not accepting address 8, error -71
Jul 15 14:31:53 banquo kernel: usb 3-9: reset high-speed USB device number 8 using xhci_hcd
Jul 15 14:31:53 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:53 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:53 banquo kernel: usb 3-9: device not accepting address 8, error -71
Jul 15 14:31:53 banquo kernel: usb 3-9: USB disconnect, device number 8
Jul 15 14:31:54 banquo kernel: usb 3-9: new high-speed USB device number 9 using xhci_hcd
Jul 15 14:31:54 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:54 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:54 banquo kernel: usb 3-9: new high-speed USB device number 10 using xhci_hcd
Jul 15 14:31:54 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:54 banquo kernel: usb 3-9: device descriptor read/64, error -71
Jul 15 14:31:55 banquo kernel: usb usb3-port9: attempt power cycle
Jul 15 14:31:55 banquo kernel: usb 3-9: new high-speed USB device number 11 using xhci_hcd
Jul 15 14:31:55 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:55 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:55 banquo kernel: usb 3-9: device not accepting address 11, error -71
Jul 15 14:31:56 banquo kernel: usb 3-9: new high-speed USB device number 12 using xhci_hcd
Jul 15 14:31:56 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:56 banquo kernel: usb 3-9: Device not responding to setup address.
Jul 15 14:31:56 banquo kernel: usb 3-9: device not accepting address 12, error -71
Jul 15 14:31:56 banquo kernel: usb usb3-port9: unable to enumerate USB device

I’ve tried:
  • validated udev rule is in place for the device.
  • connecting to the PBP’s USB-A port, but there’s nothing there.
  • running with sudo
  • rebooting the host machine
  • connecting USB directly to my laptop, and through a powered dock (dell TB16)
According to the kernel docs, error -110 is -ETIMEDOUT: "Synchronous USB message functions use this code to indicate timeout expired before the transfer completed, and no other error was reported by HC.". Error -71 is -EPROTO: "1. bitstuff error, 2. no response packet received within the prescribed bus turn-around time, 3. unknown USB error"


There's a note on -EPROTO: "errors like [-EPROTO] normally indicate hardware problems such as bad devices (including firmware) or cables.Error codes like -EPROTO, -EILSEQ and -EOVERFLOW normally indicate hardware problems such as bad devices (including firmware) or cables."

So timeout expired before data transfer completed, and then the device goes away completely.

RE: dd ayufan to emmc causes no boot, no leds - DimHummingbird - 07-16-2021

I connected to a different computer, and rkdeveloptool worked fine. TIL I have a problem with my USB ports. Smile

1) "rkdeveloptool td" said it failed.
2) "rkdeveloptool db rk3399_loader_spinor_v1.15.114.bin" worked.
3) "rkdeveloptool wl 0 zerospi" successfully zeroed the SPI
4) "rkdeveloptool td" says it succeeded!

One "rkdeveloptool rd" later and I tried to boot from the known-good SD again... and still no LEDs, no sign of life.

I notice that most of the other posts about being soft bricked are in the General forum, so i will try and cross post there.

RE: dd ayufan to emmc causes no boot, no leds - DimHummingbird - 07-17-2021

this has gotten weird. I flashed manjaro to the SD card, and it boots, but seems to hang on the manjaro splash screen. I gave it a ctrl+alt+del, and it rebooted... into the ayufan-rock64 install on the emmc. Confirmed that was the /boot device with `mount`.

TIL that pinebook will mix and match boot devices. I guess it gets u-boot off the Manjaro SDcard, then warm reboot has it restart the "boot order" search and it finds the boot partitions on emmc. And previously, when I thought I was booting from the sdcard, it was really getting uboot from emmc and then loading boot partitions from the sdcard (the reverse of what's happening now).

I tried using fdisk and dd to copy the /boot partition from the manjaro SD card to the ayurfan emmc, but that gets me back to "no leds". I also tried ayurfan's pinebookpro LXD image, and that got me uboot (LED lights green) but no further (can't find boot partition).

So the only image I have with working uboot seems to be manjaro. I'm going to try installing it to emmc, then using fdisk to manually overwrite root.

RE: soft brick with ayufan - DimHummingbird - 07-17-2021

Tried on the SD card before yanking the mmc. This on my host computer:


Code:
# start with a manjaro image. (which works for uboot but hangs on splash)
sudo dd if=Manjaro-ARM-kde-plasma-pbpro-20.12.img of=/dev/mmcblk0 bs=512 status=progress
# copy the root partition from ayufan buster-containers.
sudo dd if=buster-containers-rock64-0.11.0-1185-arm64.img of=/dev/mmcblk0p2 bs=512 skip=524288 status=progress

I had the same problem, hung at the manjaro splash screen.

So I pulled the emmc and plugged it into my USB stick. And the partition table does NOT look like ayufan buster:

Code:
Disk /dev/sda: 58,24 GiB, 62537072640 bytes, 122142720 sectors
Disk model: STORAGE DEVICE 
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E96A8F3C-0431-4043-8BFE-2D57A1A4CE67

Device      Start       End   Sectors   Size Type
/dev/sda1   62500    500000    437501 213,6M Linux filesystem
/dev/sda4  524288 122142686 121618399    58G Linux filesystem
That looks like the Manjaro partition scheme.

So WTF was I dd'ing to when I first tried to flash the emmc with ayufan?!? Was it all just cached because I never ran `sync` ?  That seems really unlikely...  But I checked the source device ID for the root partition, and dd'ed to the other device.

Is it possible that I booted off of some bizarre mix of mounts?  How does uboot decide which partition becomes / ?

RE: soft brick with ayufan - wdt - 07-17-2021

Whichever uboot you have on emmc..... (bootrom search order,,,SPI, emmc, uSD)

strings uboot |grep -i boot_target ,,,, (usually "boot_targets=mmc1 nvme0 usb0 mmc0 pxe dhcp",,, mmc1 is uSD)

each of these is searched (in order) for boot.scr OR extlinux.conf OR bootaa64.efi
Inside one of these, the 1st one found, will be a root= statement
For boot.scr it sometimes hard to find what root is, often a uEnv.txt for over rides
BTW
>But I checked the source device ID for the root partition, and dd'ed to the other device.
Nearly always, the emmc will have boot0 and boot1 sections on it,,, don't use them
ie /dev/mmcblk1boot0 for example,, mmcblk1 is emmc,, write to /dev/mmcblk1
I do have a recent 128GB emmc that does NOT have boot0 and boot1 sections

RE: soft brick with ayufan - DimHummingbird - 07-17-2021

I pulled the two first stage boot loader files out of the first partition of that eMMC, and wrote them back to the sd card with buster, and the eMMC with Manjaro.

Code:
# dd if=/boot/idbloader.img conv=notrunc seek=64    of=/dev/mmcblkX

# dd if=/boot/u-boot.itb    conv=notrunc seek=16384 of=/dev/mmcblkX

And it boots! From mmc alone I get the manjaro splash screen hang. From sd I can actually boot into buster.

Big thanks to the clear documentation at https://wiki.pine64.org/wiki/RK3399_boot_sequence for helping me figure this out!

To summarize: I propose a change in the documentation. The way to install a new OS on your SD card or emmc is:

1) copy your existing tpl/spl/u-boot files from mmcblk2boot1 to a safe place.
2) `dd` your desired OS image. to /dev/mmcblk2 .
3) ensure the TPL/SPL/u-boot files are present with the dd commands above.

I just ran that sequence on the emmc, and it worked perfectly.

RE: soft brick with ayufan - wdt - 07-17-2021

boot0 and boot1 are not used by rk3399 cpus
It may be wise to make a copy of 1st 16M of a working (booting OK) emmc
just in case new uboot on SD doesn't work since in most cases a SD boot uses uboot from emmc
If the first 16 M goes onto a different device, the mbr will not be right

RE: soft brick with ayufan - DimHummingbird - 07-18-2021

(07-17-2021, 05:31 PM)wdt Wrote: boot0 and boot1 are not used by rk3399 cpus
It may be wise to make a copy of 1st 16M of a working (booting OK) emmc
just in case new uboot on SD doesn't work since in most cases a SD boot uses uboot from emmc
If the first 16 M goes onto a different device, the mbr will not be right

Ah, I thought the first 16M *was* mapped to boot0 and boot1.

Thank you for the reply and explanation! Even after the fact, it helps. Smile  I'll write a blog post about this and submit to the wiki to help future adventurers.

RE: soft brick with ayufan - wdt - 07-18-2021

The whole block device and boot0, boot1 are all on the same nand, boot0 and boot1 are
an artifact of the controller, some arm cpus use this, but not rk3399, rk3328, a64
The boot sections are 4MB in size, also there is a character mode section, rpmb,, 12MB