PINE64
what is the magic dd incantation to copy the system from the sd card to the emmc? - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=111)
+--- Forum: Linux on Pinebook Pro (https://forum.pine64.org/forumdisplay.php?fid=114)
+--- Thread: what is the magic dd incantation to copy the system from the sd card to the emmc? (/showthread.php?tid=17084)

Pages: 1 2

what is the magic dd incantation to copy the system from the sd card to the emmc? - geokon - 08-01-2022

I am running the latest Armbian

I had trouble installing it the conventional way to the emmc - where you copy the compressed system image to the emmc. It would boot and hang/spin in the Armbian loading screen

However the system has been running no problem from the SD card. Is there some way I could copy my existing/running system to the emmc?

And could I then copy it back (for backup) to the SD card?

Ideally I'd have a 32GB system partition that I could backup periodically from the emmc

The issues is i don't really know the correct Unix commands to copy the system back and forth.

Hope someone with more Linux foo could help me out with the right dd lines (or maybe it's more complicated bc of partitioning?)

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - wdt - 08-01-2022

Do be aware that your SD boot is using the uboot ON THE emmc,
this is almost certainly NOT the same as what is on the SD card,
which because of the spin-boot failure is not quite correct
So..maybe better to do partition dd's and then adjust fstab and extlinux.conf
(on emmc, to point to emmc)
You can verify my 1st supposition by disabling/remove emmc and see if it still boots
Then, backup SD mbr and dd 1st 16M from emmc -> SD, restore SD mbr, disable emmc, does it boot?
Or, better order, 1st save 1st 16M (emmc) somewhere on SD (for later convenience)

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - dumetrulo - 08-01-2022

You're a bit short on relevant details, such as the size of the SD card, the size of your eMMC, the number of partitions involved etc. As of now, I can only say for sure that the boot partition begins at sector 32768.

Please boot from your Armbian SD, and provide the following outputs:

Code:
sudo fdisk -l /dev/mmcblk1
sudo fdisk -l /dev/mmcblk2


If these commands should not produce the partitoning details of the SD card and eMMC, run lsblk to find the proper device names, and try with them.

Then I can tell you which dd commands you will need.

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - wdt - 08-03-2022

use lsblk 1st,, boot0,boot1 is emmc (DO NOT use boot0, boot1)
(as root, sudo),,, if= input file,,, of= output file
either /dev/mmcblkX, X=1 or 2 (whole device) OR /dev/mmcblkXpY (partition Y, Y= 1 or 2))
Never use (dd's) default block size for flash (512B), always at least 256K
(me,,, bs=1M),,,, sync when done
For 1st sector, mbr,,,, dd if=/dev/SD of=savedSD.mbr bs=512 count=1
For 1st 16M (mbr+uboot) ,,,, dd if=/dev/emmc of=1st16m.emmc bs=1M count=16
replace emmc and SD with appropriate, of= can have paths if needed,
ie of=~/saved.mbr,,,,~=home

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - geokon - 08-06-2022

(08-01-2022, 09:40 AM)wdt Wrote: Do be aware that your SD boot is using the uboot ON THE emmc,
this is almost certainly NOT the same as what is on the SD card,
which because of the spin-boot failure is not quite correct
So..maybe better to do partition dd's and then adjust fstab and extlinux.conf
(on emmc, to point to emmc)
You can verify my 1st supposition by disabling/remove emmc and see if it still boots
Then, backup SD mbr and dd 1st 16M from emmc -> SD, restore SD mbr, disable emmc, does it boot?
Or, better order, 1st save 1st 16M (emmc) somewhere on SD (for later convenience)

Yeah, that's a good point. I can't just clobber what's on the emmc. The uboot details are stored on there


(08-01-2022, 03:13 PM)dumetrulo Wrote: You're a bit short on relevant details, such as the size of the SD card, the size of your eMMC, the number of partitions involved etc. As of now, I can only say for sure that the boot partition begins at sector 32768.

Please boot from your Armbian SD, and provide the following outputs:

Code:
sudo fdisk -l /dev/mmcblk1
sudo fdisk -l /dev/mmcblk2


If these commands should not produce the partitoning details of the SD card and eMMC, run lsblk to find the proper device names, and try with them.

Then I can tell you which dd commands you will need.

Sorry for the late reply..
Code:
$ sudo fdisk -l /dev/mmcblk1
Disk /dev/mmcblk1: 29.12 GiB, 31267487744 bytes, 61069312 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa30be9a7

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk1p1      32768 60456959 60424192 28.8G 83 Linux


$ sudo fdisk -l /dev/mmcblk2
Disk /dev/mmcblk2: 116.48 GiB, 125069950976 bytes, 244277248 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa30be9a7

Device         Boot Start      End  Sectors  Size Id Type
/dev/mmcblk2p1      32768 12902399 12869632  6.1G 83 Linux

I think I was a bit unclear, but I'd like to repartition the emmc to make this kind of copying possible. Now I'm realizing I need to get the partitions right first - so straight dd isn't really the right tool here. I'm not sure on the details of how to go about it though. So any tips would be appreciated.

I'd also like to drop to some console output during Armbian boot to actually see whats going on. Any idea how I can hide the spinning graphic and see some useful info?


(08-03-2022, 09:02 AM)wdt Wrote: use lsblk 1st,, boot0,boot1 is emmc (DO NOT use boot0, boot1)
(as root, sudo),,, if= input file,,, of= output file
either /dev/mmcblkX, X=1 or 2 (whole device) OR /dev/mmcblkXpY (partition Y, Y= 1 or 2))
Never use (dd's) default block size for flash (512B), always at least 256K
(me,,, bs=1M),,,,  sync when done
For 1st sector, mbr,,,, dd if=/dev/SD of=savedSD.mbr bs=512 count=1
For 1st 16M (mbr+uboot) ,,,, dd if=/dev/emmc of=1st16m.emmc  bs=1M count=16
replace emmc and SD with appropriate, of=  can have paths if needed,
ie of=~/saved.mbr,,,,~=home
Thanks for the detailed rundown on how to use dd. I'm guessing I need to understand how to partition the emmc first though.. If I'm not mistaken here

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - calinb - 08-07-2022

Quote:I'd also like to drop to some console output during Armbian boot to actually see whats going on. Any idea how I can hide the spinning graphic and see some useful info?
in /boot/armbianEnv.txt:
Code:
verbosity=1
bootlogo=false

I'm having no luck getting Armbian jammy current to run from anything but SD. It's too bad, because Armbian Jammy KDE Fusion the best running OS I've tried on my PBP since I got it over a year ago, but I'm not running my PBP from an SD card forever!

There's not enough documentation support on the Armbian website or forum that explains how the Armbian /boot file set works and I've spent far too much time trying to debug something that's been trivial on other OSes. (Armbian doesn't use /boot/extlinux.) It appears to me that it needs a new kernel build to run on eMMC or Nvme, but I'm not going to try to learn that Armbian process right now.

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - wdt - 08-07-2022

Again
>I'm having no luck getting Armbian jammy current to run from anything but SD.
If there is an emmc, with a distro, then it is using that uboot
So, armbians uboot is not quite right
You have a SD boot, pull (or disable emmc), you wont have a boot
And you can verify,,, save SD mbr (1 sector), write emmc uboot (1st 16M) to SD, restore SD mbr
The test with emmc removed/disabled
armbian uses boot.txt, run thru a simple compiler to make boot.scr
This is one of the things uboot looks for
While the kernel has to be configured to use nvme, 1st uboot has to find it (and be able to read it)
This feature is only 2.5 yrs old now,,, mrfixit the 1st IIUR

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - calinb - 08-08-2022

(08-07-2022, 07:17 PM)wdt Wrote:
Quote:If there is an emmc, with a distro, then it is using that uboot
I expected Armbian to use my eMMC uboot, because it's the first uboot in the boot order in the hard-coded ROM. (According to the PBP wiki, the order is SPI, eMMC, SD, USB OTG.) I have been using PCM720's uboot on emmc for over a year now. It boots Manjaro configured with / or /boot on either NVMe or eMMC. Although PCM720 provided a SPI binary and I also compiled his code myself, I've never been brave enough to flash it to SPI memory! The Manjaro uboot functions similar to PCM720's code, I think.

As an aside note, I've twice had boot failures after a Manjaro kernel update, but only when I had /boot and / on my nVME. After I twice recovered the system from backup, I found that the kernel update did not cause a problem when /boot is on eMMC so I've been running /boot on eMMC and / on NVMe since the second time the update killed my system. I also built the same kernels that were in the problematic updates myself, and the package that my build produced did not result in the failure. Manjaro forum mod Stritt and I never got to the bottom of it.

Quote:So, armbians uboot is not quite right
You have a SD boot, pull (or disable emmc), you wont have a boot
So you are saying that even though the SD card is in the boot order, the uboot on the Armbian SD image does not function quite right so Armbian will not boot if there is no upstream uboot code in SPI or eMMC? Again, I realize that I'm using my eMMC uboot.

Quote:And you can verify,,, save SD mbr (1 sector), write emmc uboot (1st 16M) to SD, restore SD mbr
The test with emmc removed/disabled
armbian uses boot.txt, run thru a simple compiler to make boot.scr
This is one of the things uboot looks for
While the kernel has to be configured to use nvme, 1st uboot has to find it (and be able to read it)
This feature is only 2.5 yrs old now,,, mrfixit the 1st IIUR

I would be happy to get Armbian / running on eMMC or NVMe with /boot on the SD card (like Slackware does and I could umount /boot and remove the SD card after booting) but I couldn't get that to work either. I edited the armbianENV.txt file and fstab but Armbian complained that it could not find a device with the specified UUID.

It's really a shame, because Armbian is incredibly stable but I don't understand its boot process or how to re-compile the necessary boot files. Manjaro has this nasty ALSA bug and it's going to be tough to debug too, whereas ALSA works correctly in Armbian. Do you know anything about ALSA, wdt?

https://forum.manjaro.org/t/alsa-segmentation-fault-core-dumped/118704/2


RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - wdt - 08-08-2022

ALSA,, no sorry
You could probably get Armbian (on emmc) to work
by replacing its uboot with PCM
Not sure, I think PCM is a BSP, separate trust.img???
If you have a usb->emmc adapter
OR,,, as the very last stage of an install
---
boot order: you are correct for uboot load
But uboot can then change the search order
If you search uboot.img for "boot_targets" ,,, for uboot emmc is always mmc0,,, SD mmc1
something like,,,, strings uboot.img|grep -i boot_

RE: what is the magic dd incantation to copy the system from the sd card to the emmc? - calinb - 08-08-2022

(08-08-2022, 08:49 AM)wdt Wrote: You could probably get Armbian (on emmc) to work
by replacing its uboot with PCM
Not sure, I think PCM is a BSP, separate trust.img???
If you have a usb->emmc adapter
OR,,, as the very last stage of an install
Thanks for your prompt reply, wdt!

My Manjaro system (/ on NVMe) relies on eMMC of course (both Manjaro /boot and uboot are on my eMMC) so I don't think I'll mess around with my eMMC until I get another eMMC module for experimenting. I have the usb > eMMC adapter accessory from the Pine64 store but I never ordered a spare eMMC. The Foresee eMMC module appears to be a non-standard/common design so it's probably best to order the module from Pine64 too.

I remember reading about trust.img, etc. over a year ago, but I don't recall how it works. I'll have to re-study! PCM720 provides the following two files for eMMC/SD installation and, to the best of my knowledge Manjaro has not over-written them since I installed them to my eMMC.

Notes from PCM720's release page: https://github.com/pcm720/u-boot-build-scripts/releases

Quote:Writing to SD/eMMC:
# dd if=idbloader.img of=/dev/<mmc> seek=64 conv=notrunc
# dd if=u-boot.itb of=/dev/<mmc> seek=16384 conv=notrunc

Writing to SPI Flash (Manjaro):
# flash_erase /dev/mtd0 0 0
# dd if=spiflash.bin of=/dev/mtd0

If dd doesn't work for you (see #2), try nandwrite:
# nandwrite -p /dev/mtd0 spiflash.bin

As I mentioned, I have only installed the two SD/eMMC files to my eMMC idbloader.img and u-boot.itb). I have not attempted to install spiflash.bin to SPI.