Rock64 no longer boots after dist-upgrade

[KernelPanic]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Code:

Resetting CPU ... DDR version 1.08 20170628 In SRX LPDDR3 333MHz Bus Width=32 Col=11 Bank=8 Row=15/15 CS=2 Die Bus-Width=32 Size=4096MB ddrconfig:7 OUT U-Boot SPL 2017.09-ga0a2b48 (Apr 02 2018 - 16:01:39) setup_ddr_param  1 booted from eMMC Trying to boot from MMC1 NOTICE:  BL31: v1.3(debug):9d3f591 NOTICE:  BL31: Built : 14:39:02, Jan 17 2018 NOTICE:  BL31:Rockchip release version: v1.3 tialized INFO:    Using opteed sec cpu_context! INFO:    boot t: pd status 0xe INFO:    BL31: Initializing runtime services oting device without OPTEE initialization. SMC`s destined for OPe service opteed_fast INFO:    BL31: Preparing for EL3 exit to INFO:    SPSR = 0x3c9 U-Boot 2017.09-ga0a2b48 (Apr 02 2018 - 16:01:54 +0000), Build: jenkins-linux-build-rock-64-209 Model: Pine64 Rock64 DRAM:  4 GiB MMC:   rksdmmc@ff520000: 0, rksdmmc@ff500000: 1 Card did not respond to voltage select! mmc_init: -95, time 10 *** Warning - No block device, using default environment In:    serial@ff130000 Out:   serial@ff130000 Err:   serial@ff130000 Model: Pine64 Rock64 misc_init_r cpuid=55524b50303930303900000000120918 serial=fdb482d7dd1a1b6 normal boot Net:   eth0: ethernet@ff540000 Hit any key to stop autoboot:  0 switch to partitions #0, OK mmc0(part 0) is current device Scanning mmc 0:6... Found /extlinux/extlinux.conf Retrieving file: /extlinux/extlinux.conf reading /extlinux/extlinux.conf 688 bytes read in 4 ms (168 KiB/s) select kernel 1:      kernel-latest 2:      kernel-previous Enter choice: 1:        kernel-latest Retrieving file: /initrd.img reading /initrd.img 6677437 bytes read in 932 ms (6.8 MiB/s) Retrieving file: /Image reading /Image 18995208 bytes read in 2640 ms (6.9 MiB/s) append: rw root=LABEL=linux-root rootwait rootfstype=ext4 panic=10 init=/sbin/init coherent_pool=1M ethaddr=be:68:6c2:d9 eth1addr=be:68:6c:57:b2:f9 serial=fdb482d7dd1a1b6 cgroup_enRetrieving file: /dtb reading /dtb 56836 bytes read in 11 ms (4.9 MiB/s) ## Flattened Device Tree blob at 01f00000    Booting using the fdt blob at 0x1f00000    Loading Ramdisk to fc8a3000, end fcf013bd ... OK    Loading Device Tree to 00000000fc892000, end 00000000fc8a2e03 ... OK Starting kernel ...

[pfeerick]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Any idea what version it updated to? So we can backtrack and see what's going on.

What has just happened is you lost your previous kernel, as previous kernel is now the faulty one, due to the second update. To get around it, yes, you should be able to copy another kernel / initrd image on the fat filesystem on the SD card, and edit the [/boot/]efi/extlinux/extlinux.conf file to point to the new kernel/initrd. I don't see why you can't add it as a third option if you wish the leave the other entries alone. If you

[evilbunny]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Do you have a spare sdcard, you could flash that with a known working image and copy the files in /boot over the top of the bad ones...

[KernelPanic]

First of all thanks for trying to help!

@pfeerick
Problems began when updating to 0.6.53. After reboot it would hang on "Starting kernel".
I managed to get around this by selecting previous kernel.

Then I stupidly tried installing the kernel from 0.6.44: jenkins-linux-build-rock-64-239 instead of first installing the one I knew worked. And broke my previous kernel as well.
So now I'm stuck without a working kernel and I can't figure out how to acces to emmc to edit it conf file and upload the new kernel.

I'm a noob when it comes to uboot, but here's me trying to list the contents of the mmc: (trying 1 instead of 0 also doesn't work)

=> mmc dev 0
switch to partitions #0, OK
mmc0(part 0) is current device
=> ls mmc 0
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
=>


@evilbunny
I also made some attempts booting a spare sdcard with an image I think should be working, but I think for some reason it will still boot the non working kernel from the emmc.
Here's what I do:
- Turn off Rock64
- Insert sdcard
- Attach jumper cables to the 2 pins closest to the recovery button (just like I did when installing to emmc)
- Boot Rock64
- Break at the boot timer
- Remove jumper cables
- type boot
- Stuck at same kernel error

When I remove the emmc another weird thing is going on, it's like its trying to do a network boot.. I'm going to reflash the sdcard just to make sure I didn't mess that up. In the meantime any more help is appreciated!

[evilbunny]

When I remove the emmc another weird thing is going on, it's like its trying to do a network boot.. I'm going to reflash the sdcard just to make sure I didn't mess that up. In the meantime any more help is appreciated!

The only way the r64 boots from the network is if you flash SPI with u-boot, I think there are some pins you can short to disable it.

[KernelPanic]

I think the image was just crap, I did a format and started from scratch and now it's booting.  

So now I have a working image.
Before I do another stupid thing, could you validate if I know have to do the following?

First I mount the root of emmc to newly created folder
sudo mount /dev/mmcblk0p7 /mnt/bootmmc

Seems to be correct, this is the contents of emmc boot:

Code:

rock64@rock64:/mnt/bootmmc/boot$ ls -l total 117660 -rw-r--r-- 1 root root  4615568 Apr 29 20:28 System.map-4.4.120-rockchip-ayufan-213 -rw-r--r-- 1 root root  4660138 May 27 18:39 System.map-4.4.126-rockchip-ayufan-239 -rw-r--r-- 1 root root  4557526 Jun 12 08:49 System.map-4.4.126-rockchip-ayufan-260 -rw-r--r-- 1 root root  4492973 Oct 12  2017 System.map-4.4.77-rockchip-ayufan-136 -rw-r--r-- 1 root root   147469 Apr 29 20:28 config-4.4.120-rockchip-ayufan-213 -rw-r--r-- 1 root root   148020 May 27 18:39 config-4.4.126-rockchip-ayufan-239 -rw-r--r-- 1 root root   148184 Jun 12 08:49 config-4.4.126-rockchip-ayufan-260 -rw-r--r-- 1 root root   143541 Oct 12  2017 config-4.4.77-rockchip-ayufan-136 lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb -> dtbs/4.4.126-rockchip-ayufan-260/rockchip/rk3328-rock64.dtb lrwxrwxrwx 1 root root       59 May 30 07:43 dtb-4.4.120-rockchip-ayufan-213 -> dtbs/4.4.120-rockchip-ayufan-213/rockchip/rk3328-rock64.dtb lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb-4.4.126-rockchip-ayufan-239 -> dtbs/4.4.126-rockchip-ayufan-239/rockchip/rk3328-rock64.dtb lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb-4.4.126-rockchip-ayufan-260 -> dtbs/4.4.126-rockchip-ayufan-260/rockchip/rk3328-rock64.dtb lrwxrwxrwx 1 root root       58 May 30 07:41 dtb-4.4.77-rockchip-ayufan-136 -> dtbs/4.4.77-rockchip-ayufan-136/rockchip/rk3328-rock64.dtb drwxr-xr-x 8 root root     4096 Jun 15 23:41 dtbs drwxr-xr-x 2 root root     4096 Apr  2 16:14 efi -rw-r--r-- 1 root root     8649 Dec 12  2017 filesystem.packages -rw-r--r-- 1 root root        1 Dec 12  2017 filesystem.packages-remove -rw-r--r-- 1 root root  6129960 May 30 07:41 initrd.img-4.4.120-rockchip-ayufan-213 -rw-r--r-- 1 root root  6130061 Jun 15 23:41 initrd.img-4.4.126-rockchip-ayufan-239 -rw-r--r-- 1 root root  6677437 Jun 12 19:38 initrd.img-4.4.126-rockchip-ayufan-260 -rw-r--r-- 1 root root  5873530 May 30 07:41 initrd.img-4.4.77-rockchip-ayufan-136 -rwxr-xr-x 1 root root 19369992 Apr 29 20:28 vmlinuz-4.4.120-rockchip-ayufan-213 -rwxr-xr-x 1 root root 19726344 May 27 18:39 vmlinuz-4.4.126-rockchip-ayufan-239 -rwxr-xr-x 1 root root 18995208 Jun 12 08:49 vmlinuz-4.4.126-rockchip-ayufan-260 -rwxr-xr-x 1 root root 18606088 Oct 12  2017 vmlinuz-4.4.77-rockchip-ayufan-136

Contents of the sdcard boot:

Code:

rock64@rock64:/boot$ ls -l total 28264 -rw-r--r-- 1 root root  4615568 Apr  2 16:06 System.map-4.4.120-rockchip-ayufan-209 -rw-r--r-- 1 root root   147469 Apr  2 16:06 config-4.4.120-rockchip-ayufan-209 lrwxrwxrwx 1 root root       59 Apr  2 16:31 dtb -> dtbs/4.4.120-rockchip-ayufan-209/rockchip/rk3328-rock64.dtb lrwxrwxrwx 1 root root       59 Apr  2 16:31 dtb-4.4.120-rockchip-ayufan-209 -> dtbs/4.4.120-rockchip-ayufan-209/rockchip/rk3328-rock64.dtb drwxr-xr-x 3 root root     4096 Apr  2 16:31 dtbs drwxr-xr-x 3 root root    16384 Jan  1  1970 efi -rw-r--r-- 1 root root     8649 Dec 12  2017 filesystem.packages -rw-r--r-- 1 root root        1 Dec 12  2017 filesystem.packages-remove -rw-r--r-- 1 root root  4760374 Apr  2 16:31 initrd.img-4.4.120-rockchip-ayufan-209 -rwxr-xr-x 1 root root 19369992 Apr  2 16:06 vmlinuz-4.4.120-rockchip-ayufan-209

So now I just copy the whole contents of sdcard boot to emmc boot?
sudo cp -rp /boot/ /mnt/bootmmc/boot/

Thanks again!

[evilbunny]

So now I just copy the whole contents of sdcard boot to emmc boot?
sudo cp -rp /boot/ /mnt/bootmmc/boot/

The 239 files look promising... https://github.com/ayufan-rock64/linux-b...ses/0.6.44

Also you need to mount partition 6 to /boot/efi otherwise you will miss copying those files.

[KernelPanic]

You are right, forgot the efi parition

Something weird seems to be going on with the one that's mounted by default (or I just don't understand

Code:

rock64@rock64:/mnt/efimmc$ df -h Filesystem      Size  Used Avail Use% Mounted on udev            2.0G     0  2.0G   0% /dev tmpfs           393M  5.4M  388M   2% /run /dev/mmcblk1p7   15G  1.6G   13G  12% / tmpfs           2.0G     0  2.0G   0% /dev/shm tmpfs           5.0M  4.0K  5.0M   1% /run/lock tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup /dev/mmcblk0p6  100M   50M   51M  50% /boot/efi tmpfs           393M     0  393M   0% /run/user/1000 /dev/mmcblk0p7   29G  9.3G   18G  35% /mnt/bootmmc /dev/mmcblk1p6  100M   24M   77M  24% /mnt/efimmc

The following two especially:

Code:

/dev/mmcblk1p7   15G  1.6G   13G  12% / /dev/mmcblk0p6  100M   50M   51M  50% /boot/efi

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?
I didn't notice at first and mounted what I thought was the emmc to my own mount point (/mnt/efimmc) but ended up copying the same files back and forth   
After still not being able to boot I noticed, rebooted, remounted the sdcard efi to /boot/efi and mmc to /mnt/efimmc copied and BAM we're back in action!

Thanks a lot for pointing me in the right direction!

---

Could it be that copying the /boot/ and /boot/efi messed something up with my build?

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

I checked, and indeed there is no /dev/net folder

[evilbunny]

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?

That happens because images use the same partition titles, you could try changing the fstab to which ever device you want mounted.

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

try apt-get --reinstall install openvpn

If that doesn't work, it may be your openvpn config loading from the wrong partition

[KernelPanic]

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?

That happens because images use the same partition titles, you could try changing the fstab to which ever device you want mounted.

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

try apt-get --reinstall install openvpn

If that doesn't work, it may be your openvpn config loading from the wrong partition

I think I figured it out after some more digging.  I'll post what I did here in case it might help someone else.

I think because I copied a kernel from the SDcard that I wasn't using before on emmc I am still missing some files I would've gotten with a normal install of the kernel. (In this case the modules in /lib/moduiles). 

Code:

sudo modprobe tun modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.120-rockchip-ayufan-209/modules.dep.bin' modprobe: FATAL: Module tun not found in directory /lib/modules/4.4.120-rockchip-ayufan-209                                                                                                                                                                                                                                         -ayufan-209 cd /lib/modules/ /lib/modules$ ls -l total 16 drwxr-xr-x 3 root root 4096 May 13 13:54 4.4.120-rockchip-ayufan-213 drwxr-xr-x 3 root root 4096 Jun 16 01:41 4.4.126-rockchip-ayufan-239 drwxr-xr-x 3 root root 4096 Jun 12 21:37 4.4.126-rockchip-ayufan-260 drwxr-xr-x 3 root root 4096 Apr 11 14:27 4.4.77-rockchip-ayufan-136

So I purged all the kernels and reinstalled the 213 kernel. After a reboot modprobe now doesn't return any more errors and OpenVPN works again.

Thanks!