Rock64 no longer boots after dist-upgrade

[KernelPanic]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Code:

Resetting CPU ...

DDR version 1.08 20170628
In
SRX
LPDDR3
333MHz
Bus Width=32 Col=11 Bank=8 Row=15/15 CS=2 Die Bus-Width=32 Size=4096MB
ddrconfig:7
OUT

U-Boot SPL 2017.09-ga0a2b48 (Apr 02 2018 - 16:01:39)
setup_ddr_param  1
booted from eMMC
Trying to boot from MMC1
NOTICE:  BL31: v1.3(debug):9d3f591
NOTICE:  BL31: Built : 14:39:02, Jan 17 2018
NOTICE:  BL31:Rockchip release version: v1.3
tialized
INFO:    Using opteed sec cpu_context!
INFO:    boot t: pd status 0xe
INFO:    BL31: Initializing runtime services
oting device without OPTEE initialization. SMC`s destined for OPe service opteed_fast
INFO:    BL31: Preparing for EL3 exit to INFO:    SPSR = 0x3c9


U-Boot 2017.09-ga0a2b48 (Apr 02 2018 - 16:01:54 +0000), Build: jenkins-linux-build-rock-64-209

Model: Pine64 Rock64
DRAM:  4 GiB
MMC:   rksdmmc@ff520000: 0, rksdmmc@ff500000: 1
Card did not respond to voltage select!
mmc_init: -95, time 10
*** Warning - No block device, using default environment

In:    serial@ff130000
Out:   serial@ff130000
Err:   serial@ff130000
Model: Pine64 Rock64
misc_init_r
cpuid=55524b50303930303900000000120918
serial=fdb482d7dd1a1b6
normal boot
Net:   eth0: ethernet@ff540000
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0(part 0) is current device
Scanning mmc 0:6...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
reading /extlinux/extlinux.conf
688 bytes read in 4 ms (168 KiB/s)
select kernel
1:      kernel-latest
2:      kernel-previous
Enter choice: 1:        kernel-latest
Retrieving file: /initrd.img
reading /initrd.img
6677437 bytes read in 932 ms (6.8 MiB/s)
Retrieving file: /Image
reading /Image
18995208 bytes read in 2640 ms (6.9 MiB/s)
append: rw root=LABEL=linux-root rootwait rootfstype=ext4 panic=10 init=/sbin/init coherent_pool=1M ethaddr=be:68:6c2:d9 eth1addr=be:68:6c:57:b2:f9 serial=fdb482d7dd1a1b6 cgroup_enRetrieving file: /dtb
reading /dtb
56836 bytes read in 11 ms (4.9 MiB/s)
## Flattened Device Tree blob at 01f00000
   Booting using the fdt blob at 0x1f00000
   Loading Ramdisk to fc8a3000, end fcf013bd ... OK
   Loading Device Tree to 00000000fc892000, end 00000000fc8a2e03 ... OK

Starting kernel ...

[pfeerick]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Any idea what version it updated to? So we can backtrack and see what's going on.

What has just happened is you lost your previous kernel, as previous kernel is now the faulty one, due to the second update. To get around it, yes, you should be able to copy another kernel / initrd image on the fat filesystem on the SD card, and edit the [/boot/]efi/extlinux/extlinux.conf file to point to the new kernel/initrd. I don't see why you can't add it as a third option if you wish the leave the other entries alone. If you

[evilbunny]

After upgrading to the latest version of ubuntu from Ayufan Rock64 no longer boots.
I had a workaround by selection "previous kernel", but now I tried installing the latest kernel and previous kernel also no longer works.

Can anyone help me in the direction to use a different kernel from uboot? Could I just hook up a usb and tell it to boot a kernel I've downloaded to that?

Do you have a spare sdcard, you could flash that with a known working image and copy the files in /boot over the top of the bad ones...

[KernelPanic]

First of all thanks for trying to help!

@pfeerick
Problems began when updating to 0.6.53. After reboot it would hang on "Starting kernel".
I managed to get around this by selecting previous kernel.

Then I stupidly tried installing the kernel from 0.6.44: jenkins-linux-build-rock-64-239 instead of first installing the one I knew worked. And broke my previous kernel as well.
So now I'm stuck without a working kernel and I can't figure out how to acces to emmc to edit it conf file and upload the new kernel.

I'm a noob when it comes to uboot, but here's me trying to list the contents of the mmc: (trying 1 instead of 0 also doesn't work)

=> mmc dev 0
switch to partitions #0, OK
mmc0(part 0) is current device
=> ls mmc 0
Failed to mount ext2 filesystem...
** Unrecognized filesystem type **
=>


@evilbunny
I also made some attempts booting a spare sdcard with an image I think should be working, but I think for some reason it will still boot the non working kernel from the emmc.
Here's what I do:
- Turn off Rock64
- Insert sdcard
- Attach jumper cables to the 2 pins closest to the recovery button (just like I did when installing to emmc)
- Boot Rock64
- Break at the boot timer
- Remove jumper cables
- type boot
- Stuck at same kernel error

When I remove the emmc another weird thing is going on, it's like its trying to do a network boot.. I'm going to reflash the sdcard just to make sure I didn't mess that up. In the meantime any more help is appreciated!

[evilbunny]

When I remove the emmc another weird thing is going on, it's like its trying to do a network boot.. I'm going to reflash the sdcard just to make sure I didn't mess that up. In the meantime any more help is appreciated!

The only way the r64 boots from the network is if you flash SPI with u-boot, I think there are some pins you can short to disable it.

[KernelPanic]

I think the image was just crap, I did a format and started from scratch and now it's booting.  

So now I have a working image.
Before I do another stupid thing, could you validate if I know have to do the following?

First I mount the root of emmc to newly created folder
sudo mount /dev/mmcblk0p7 /mnt/bootmmc

Seems to be correct, this is the contents of emmc boot:

Code:

rock64@rock64:/mnt/bootmmc/boot$ ls -l
total 117660
-rw-r--r-- 1 root root  4615568 Apr 29 20:28 System.map-4.4.120-rockchip-ayufan-213
-rw-r--r-- 1 root root  4660138 May 27 18:39 System.map-4.4.126-rockchip-ayufan-239
-rw-r--r-- 1 root root  4557526 Jun 12 08:49 System.map-4.4.126-rockchip-ayufan-260
-rw-r--r-- 1 root root  4492973 Oct 12  2017 System.map-4.4.77-rockchip-ayufan-136
-rw-r--r-- 1 root root   147469 Apr 29 20:28 config-4.4.120-rockchip-ayufan-213
-rw-r--r-- 1 root root   148020 May 27 18:39 config-4.4.126-rockchip-ayufan-239
-rw-r--r-- 1 root root   148184 Jun 12 08:49 config-4.4.126-rockchip-ayufan-260
-rw-r--r-- 1 root root   143541 Oct 12  2017 config-4.4.77-rockchip-ayufan-136
lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb -> dtbs/4.4.126-rockchip-ayufan-260/rockchip/rk3328-rock64.dtb
lrwxrwxrwx 1 root root       59 May 30 07:43 dtb-4.4.120-rockchip-ayufan-213 -> dtbs/4.4.120-rockchip-ayufan-213/rockchip/rk3328-rock64.dtb
lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb-4.4.126-rockchip-ayufan-239 -> dtbs/4.4.126-rockchip-ayufan-239/rockchip/rk3328-rock64.dtb
lrwxrwxrwx 1 root root       59 Jun 15 23:41 dtb-4.4.126-rockchip-ayufan-260 -> dtbs/4.4.126-rockchip-ayufan-260/rockchip/rk3328-rock64.dtb
lrwxrwxrwx 1 root root       58 May 30 07:41 dtb-4.4.77-rockchip-ayufan-136 -> dtbs/4.4.77-rockchip-ayufan-136/rockchip/rk3328-rock64.dtb
drwxr-xr-x 8 root root     4096 Jun 15 23:41 dtbs
drwxr-xr-x 2 root root     4096 Apr  2 16:14 efi
-rw-r--r-- 1 root root     8649 Dec 12  2017 filesystem.packages
-rw-r--r-- 1 root root        1 Dec 12  2017 filesystem.packages-remove
-rw-r--r-- 1 root root  6129960 May 30 07:41 initrd.img-4.4.120-rockchip-ayufan-213
-rw-r--r-- 1 root root  6130061 Jun 15 23:41 initrd.img-4.4.126-rockchip-ayufan-239
-rw-r--r-- 1 root root  6677437 Jun 12 19:38 initrd.img-4.4.126-rockchip-ayufan-260
-rw-r--r-- 1 root root  5873530 May 30 07:41 initrd.img-4.4.77-rockchip-ayufan-136
-rwxr-xr-x 1 root root 19369992 Apr 29 20:28 vmlinuz-4.4.120-rockchip-ayufan-213
-rwxr-xr-x 1 root root 19726344 May 27 18:39 vmlinuz-4.4.126-rockchip-ayufan-239
-rwxr-xr-x 1 root root 18995208 Jun 12 08:49 vmlinuz-4.4.126-rockchip-ayufan-260
-rwxr-xr-x 1 root root 18606088 Oct 12  2017 vmlinuz-4.4.77-rockchip-ayufan-136

Contents of the sdcard boot:

Code:

rock64@rock64:/boot$ ls -l
total 28264
-rw-r--r-- 1 root root  4615568 Apr  2 16:06 System.map-4.4.120-rockchip-ayufan-209
-rw-r--r-- 1 root root   147469 Apr  2 16:06 config-4.4.120-rockchip-ayufan-209
lrwxrwxrwx 1 root root       59 Apr  2 16:31 dtb -> dtbs/4.4.120-rockchip-ayufan-209/rockchip/rk3328-rock64.dtb
lrwxrwxrwx 1 root root       59 Apr  2 16:31 dtb-4.4.120-rockchip-ayufan-209 -> dtbs/4.4.120-rockchip-ayufan-209/rockchip/rk3328-rock64.dtb
drwxr-xr-x 3 root root     4096 Apr  2 16:31 dtbs
drwxr-xr-x 3 root root    16384 Jan  1  1970 efi
-rw-r--r-- 1 root root     8649 Dec 12  2017 filesystem.packages
-rw-r--r-- 1 root root        1 Dec 12  2017 filesystem.packages-remove
-rw-r--r-- 1 root root  4760374 Apr  2 16:31 initrd.img-4.4.120-rockchip-ayufan-209
-rwxr-xr-x 1 root root 19369992 Apr  2 16:06 vmlinuz-4.4.120-rockchip-ayufan-209

So now I just copy the whole contents of sdcard boot to emmc boot?
sudo cp -rp /boot/ /mnt/bootmmc/boot/

Thanks again!

[evilbunny]

So now I just copy the whole contents of sdcard boot to emmc boot?
sudo cp -rp /boot/ /mnt/bootmmc/boot/

The 239 files look promising... https://github.com/ayufan-rock64/linux-b...ses/0.6.44

Also you need to mount partition 6 to /boot/efi otherwise you will miss copying those files.

[KernelPanic]

You are right, forgot the efi parition

Something weird seems to be going on with the one that's mounted by default (or I just don't understand

Code:

rock64@rock64:/mnt/efimmc$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            2.0G     0  2.0G   0% /dev
tmpfs           393M  5.4M  388M   2% /run
/dev/mmcblk1p7   15G  1.6G   13G  12% /
tmpfs           2.0G     0  2.0G   0% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/mmcblk0p6  100M   50M   51M  50% /boot/efi
tmpfs           393M     0  393M   0% /run/user/1000
/dev/mmcblk0p7   29G  9.3G   18G  35% /mnt/bootmmc
/dev/mmcblk1p6  100M   24M   77M  24% /mnt/efimmc

The following two especially:

Code:

/dev/mmcblk1p7   15G  1.6G   13G  12% /
/dev/mmcblk0p6  100M   50M   51M  50% /boot/efi

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?
I didn't notice at first and mounted what I thought was the emmc to my own mount point (/mnt/efimmc) but ended up copying the same files back and forth   
After still not being able to boot I noticed, rebooted, remounted the sdcard efi to /boot/efi and mmc to /mnt/efimmc copied and BAM we're back in action!

Thanks a lot for pointing me in the right direction!

---

Could it be that copying the /boot/ and /boot/efi messed something up with my build?

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

I checked, and indeed there is no /dev/net folder

[evilbunny]

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?

That happens because images use the same partition titles, you could try changing the fstab to which ever device you want mounted.

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

try apt-get --reinstall install openvpn

If that doesn't work, it may be your openvpn config loading from the wrong partition

[KernelPanic]

mmcblk1 = sdcard
mmcblk0 = emmc

So am I right in saying it mounts the wrong efi by default?

That happens because images use the same partition titles, you could try changing the fstab to which ever device you want mounted.

Everything is working as before except for openvpn which can't initiate a connection anymore
ERROR: Cannot open TUN/TAP dev /dev/net/tun: No such file or directory (errno=2)

try apt-get --reinstall install openvpn

If that doesn't work, it may be your openvpn config loading from the wrong partition

I think I figured it out after some more digging.  I'll post what I did here in case it might help someone else.

I think because I copied a kernel from the SDcard that I wasn't using before on emmc I am still missing some files I would've gotten with a normal install of the kernel. (In this case the modules in /lib/moduiles). 

Code:

sudo modprobe tun
modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.120-rockchip-ayufan-209/modules.dep.bin'
modprobe: FATAL: Module tun not found in directory /lib/modules/4.4.120-rockchip-ayufan-209
                                                                                                                                                                                                                                        -ayufan-209
cd /lib/modules/
/lib/modules$ ls -l
total 16
drwxr-xr-x 3 root root 4096 May 13 13:54 4.4.120-rockchip-ayufan-213
drwxr-xr-x 3 root root 4096 Jun 16 01:41 4.4.126-rockchip-ayufan-239
drwxr-xr-x 3 root root 4096 Jun 12 21:37 4.4.126-rockchip-ayufan-260
drwxr-xr-x 3 root root 4096 Apr 11 14:27 4.4.77-rockchip-ayufan-136

So I purged all the kernels and reinstalled the 213 kernel. After a reboot modprobe now doesn't return any more errors and OpenVPN works again.

Thanks!