Re-partitioning; Adding swap partition; Using GPT partitions

[Arwen]

Here is a guide of how I repartitioned my eMMC to allow for a physical swap partition, and used GPT partition layout, (instead of DOS).
Basically you need an 8GB or larger SD card you can over-write.

Note that both Linux experience and command line familiarity is required. Any mistake and you could accidentally overwrite your working OS. Last, backups, (preferably multiple), is recommended.

This copies the low level boot information:

Code:

# dd if=/dev/mmcblk1 bs=512 count=262144 of=/dev/mmcblk0

This section prepares the SD card. I had a 32GB one, so that is used in the sample below.

I don't give the details of how to use "fdisk", as the program has builtin help and can be quite verbose in use. But, you need to create a new partition label, (GPT in this example), as the one copied from above would almost certainly be wrong for the size of the media. Not to mention having a duplicate UUID number.

Do not change the start of either the "/boot" or the first partition, (swap or OS). Nor the size of the "/boot" partition. There is u-Boot code in there somewhere we don't want to overwrite.

Code:

# fdisk /dev/mmcblk0

Disklabel type: gpt

Device           Start      End  Sectors  Size Type
/dev/mmcblk0p1   32768   163839   131072   64M Microsoft basic data
/dev/mmcblk0p2  262144  4456447  4194304    2G Linux swap
/dev/mmcblk0p3 4456448 61497310 57040863 27.2G Linux filesystem

Create the file systems:

Code:

# mkfs.fat -n sd_boot -v /dev/mmcblk0p1
# mkswap -L sd_swap /dev/mmcblk0p2
# mke2fs -b 4096 -j -L sd_root -m 1 -t ext4 -v /dev/mmcblk0p3

This copies the boot partition:

Code:

# mount -t vfat /dev/mmcblk0p1 /mnt
# rsync -aAHSXx --stats /boot/ /mnt/
# vi /mnt/extlinux/extlinux.conf
{change references to root FS from /dev/mmcblk1 to /dev/mmcblk0}
# umount /mnt

This copies the OS partition:

Code:

# mount -t ext4 /dev/mmcblk0p3 /mnt
# rsync -aAHSXx --stats / /mnt/
# vi /mnt/etc/fstab
Add /boot & swap lines:
        /dev/mmcblk0p1 /boot vfat defaults 0 0
        /dev/mmcblk0p2 swap swap pri=5 0 0
# umount /mnt

Then reboot. If you have the ability to boot SD cards, this will create a bootable SD card.

Whence booted to SD card, you can redo do the eMMC as desired. I choose to allow for additional root file systems, so I made my first OS partition only 27GBs. More than enough with the 32bit ARM default OS.

Code:

root@Debian-Desktop:~# df -h / /boot;swapon
Filesystem      Size  Used Avail Use% Mounted on
/dev/root        27G  4.0G   23G  16% /
/dev/mmcblk1p1   64M   50M   15M  78% /boot
NAME           TYPE      SIZE USED PRIO
/dev/mmcblk1p2 partition   6G   0B    5

Code:

root@Debian-Desktop:~# fdisk -l /dev/mmcblk1  
Disk /dev/mmcblk1: 116.5 GiB, 125069950976 bytes, 244277248 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 3F39B914-D8D7-470B-BF6E-E7A2F2B33ABB

Device            Start       End  Sectors  Size Type
/dev/mmcblk1p1    32768    163839   131072   64M Microsoft basic data
/dev/mmcblk1p2   262144  12845055 12582912    6G Linux swap
/dev/mmcblk1p3 12845056  69885918 57040863 27.2G Linux filesystem
/dev/mmcblk1p4 69885952 126926814 57040863 27.2G Linux filesystem

Edit: Clarified that the user must create a new partition table since the one copied from the source disk would have a duplicate UUID. And likely be wrong in size.

[Arwen]

I poked around at the location of the U-Boot code, and found I could increase the size of "/boot" as below. This gives me room for initial RAM disks and multiple kernels. Don't have a need for it, yet. But, had to learn what I could and could not do.

Here is what I currently have on my eMMC;

Code:

Device             Start       End   Sectors  Size Type
/dev/mmcblk1p1     32768   1081343   1048576  512M Linux filesystem
/dev/mmcblk1p2   1081344  13664255  12582912    6G Linux swap
/dev/mmcblk1p3  13664256  70705118  57040863 27.2G Linux filesystem
/dev/mmcblk1p4  70705152 127746014  57040863 27.2G Linux filesystem
/dev/mmcblk1p5 127746048 244277214 116531167 55.6G Linux filesystem

Edit: I also made "/boot" an EXT4 file system. It seems EXT4 on "/boot" is supported just fine. Don't know why FAT was used originally.

[Zaf9670]

I poked around at the location of the U-Boot code, and found I could increase the size of "/boot" as below. This gives me room for initial RAM disks and multiple kernels. Don't have a need for it, yet. But, had to learn what I could and could not do.

Here is what I currently have on my eMMC;

Code:

Device             Start       End   Sectors  Size Type
/dev/mmcblk1p1     32768   1081343   1048576  512M Linux filesystem
/dev/mmcblk1p2   1081344  13664255  12582912    6G Linux swap
/dev/mmcblk1p3  13664256  70705118  57040863 27.2G Linux filesystem
/dev/mmcblk1p4  70705152 127746014  57040863 27.2G Linux filesystem
/dev/mmcblk1p5 127746048 244277214 116531167 55.6G Linux filesystem

Edit: I also made "/boot" an EXT4 file system. It seems EXT4 on "/boot" is supported just fine. Don't know why FAT was used originally.

FAT was probably used in the event you have boot issues you can connect it to more than just a linux machine to do modifications. I believe the Raspberry Pi and other ARM distros do the same thing from my past experiences.

[ab1jx]

FAT is a wonderfully simple system.  And simple also means reliable, easy for lesser processes to figure out.  I once read most of the Peter Norton Programmer's Bible from cover to cover and the boot sector, root directory, FAT, locations of files are easy to see.  The boot sector (if I remember right) specifies where the root directory begins.  Directories are just files really that link the file name to the starting disk sector number.  You look up that sector number in the FAT and read the next sector number until you hit an EOF marker.  Bad sectors can be marked as such in the FAT and ignored.  There can only be 512 files/directories  in the root.

I don't like GPT after a bad experience trying to increase the size of a partition table with Gparted, had to reload the SD card from an image.  I just wanted to clone a 64 GB onto a blank 128 GB and expand the partition.

[danielt]

With a bit of trickery it is also possible to preconfigure the GPT so that the idbloader, uboot and trusted firmware images appear as separate partitions. Not only does this make it easier to hack or alter the bootloaders but it also makes it difficult to accidentally damage them (tooling to trim unallocated space for example).

I'm not sure if it possible to create a partition label with a first-lba set to 64 using fdisk. I generally use an sfdisk for this:

cat <<EOF | sudo sfdisk /dev/mmcblk1
unit: sectors
first-lba: 64
/dev/mmcblk1p1 : start= 64, size= 16320, type=8DA63339-0007-60C0-C436-083AC8230908, name="IDBLoader", attrs="RequiredPartition"
/dev/mmcblk1p2 : start= 16384, size= 8192, type=8DA63339-0007-60C0-C436-083AC8230908, name="U-Boot", attrs="RequiredPartition"
/dev/mmcblk1p3 : start= 24576, size= 8192, type=8DA63339-0007-60C0-C436-083AC8230908, name="TrustedFirmware-A", attrs="RequiredPartition"
EOF

Once you have created the disk label above can create other partitions using the normal tooling.

[Arwen]

With a bit of trickery it is also possible to preconfigure the GPT so that the idbloader, uboot and trusted firmware images appear as separate partitions. Not only does this make it easier to hack or alter the bootloaders but it also makes it difficult to accidentally damage them (tooling to trim unallocated space for example).

I'm not sure if it possible to create a partition label with a first-lba set to 64 using fdisk. I generally use an sfdisk for this:

cat <<EOF | sudo sfdisk /dev/mmcblk1
unit: sectors
first-lba: 64
/dev/mmcblk1p1 : start= 64, size= 16320, type=8DA63339-0007-60C0-C436-083AC8230908, name="IDBLoader", attrs="RequiredPartition"
/dev/mmcblk1p2 : start= 16384, size= 8192, type=8DA63339-0007-60C0-C436-083AC8230908, name="U-Boot", attrs="RequiredPartition"
/dev/mmcblk1p3 : start= 24576, size= 8192, type=8DA63339-0007-60C0-C436-083AC8230908, name="TrustedFirmware-A", attrs="RequiredPartition"
EOF

Once you have created the disk label above can create other partitions using the normal tooling.

Perfect. I may give that a try. It may make things cleaner & clearer, (at least for me).

@ab1jx, There may be another reason to use FAT for the "/boot" partition. When I ran the update from 1.5 to 1.6, it complained about the standard Unix "lost+found" directory and asked if I wanted to delete it. I said no, and thought about putting in a request to have that directory ignored.

But, another problem seems to have occured. Don't know if it's related to my GPT layout or using EXT4 for "/boot", but for whatever reason, the update to 1.5, and the update 1.5 to 1.6 did not update my kernel.

I'll experiment and see if either the GPT layout or EXT4 for "/boot" had any impact.

[z4v4l]

jeez. @danielt, GPT holds partition information in the Partition Array. that has its minimum limit, to be able to hold at least the minimum number of partitions, I can't check the particular numbers, because typing it from a tablet, but the partition array still will be at least something kilobytes. and it comes BEFORE any partition. and now i see, you managed to have the first partition at LBA 64. well, i don't know what all these sfdisks really have been creating, but it's not GPT! kind of not worth mentioning after this, that GPT recommends to align a partition at 2MB.

FAT might be used because this file system is required by UEFI and can be used by it for booting. apparently Rockchip had this intention in mind.

also. if you are dd-ing beginning of one GPT partitioned disk onto another one, you get bogus drive. the must be unique GUIDs (for the disk and every partition) are not unique and also the Back up GPT header and partition array are absent. these two are severe violations, that make such setups just a broken thing, working by accident. and, of course, sooner or later, it may "pay off" in a very unpleasant way. GPT partitioning CANNOT be made by dd-ing!

[danielt]

jeez. @danielt, GPT holds partition information in the Partition Array. that has its minimum limit, to be able to hold at least the minimum number of partitions, I can't check the particular numbers, because typing it from a tablet, but the partition array still will be at least something kilobytes. and it comes BEFORE any partition. and now i see, you managed to have the first partition at LBA 64.

A normal GPT with the recommended partition array (plus the protective MBR) runs from LBA0 through to LBA 33 (and is repeated at the end of the disk). It is not a problem to have the first partition at LBA 64.

well, i don't know what all these sfdisks really have been creating, but it's not GPT! kind of not worth mentioning after this, that GPT recommends to align a partition at 2MB.

I disagree. The GPT created from the above script is an entirely conforming (it even regenerates the UUIDs correctly). As mentioned only the first and last 34 sectors are reserved and the 2MB alignment is a recommendation, not a requirement.

Note also that the 2MB recommendation exists to make filesystems that write heavily to the first few blocks (e.g. FAT) more efficient on some media types. That simply isn't relevant here. IDBLoader is the only partition that is not 2MB aligned and that is OK because it is not a filesystem and is seldom changed. There is no problem here.

IMHO it is far better to have proper protective partitions to ensure an OS installer doesn't nuke the system firmware (RequirePartitions tells installers to leave the firmware alone) than to rely on dumb luck. Proper partition labels are also a useful service to help curious admins understand their system without learning through acidentally breaking it.

Frankly I'm not sure who your comment about dd is aimed at. The only use of dd in the thread so far is to copy the firmware from eMMC to SD and the GPT is created *after* the dd.

[Arwen]

...
also. if you are dd-ing beginning of one GPT partitioned disk onto another one, you get bogus drive. the must be unique GUIDs (for the disk and every partition) are not unique and also the Back up GPT header and partition array are absent. these two are severe violations, that make such setups just a broken thing, working by accident. and, of course, sooner or later, it may "pay off" in a very unpleasant way. GPT partitioning CANNOT be made by dd-ing!

In my example, "dd" was only used to copy the boot loader and other information that @danielt clarified with his partition enhancement. Litterally the next section of my instructions say to use the partitioning tool, ("fdisk" in this case), to create a new GPT label and empty partition table. I can probably make those instructions clearer.

...
IMHO it is far better to have proper protective partitions to ensure an OS installer doesn't nuke the system firmware (RequirePartitions tells installers to leave the firmware alone) than to rely on dumb luck. Proper partition labels are also a useful service to  help curious admins understand their system without learning through acidentally breaking it.

Frankly I'm not sure who your comment about dd is aimed at. The only use of dd in the thread so far is to copy the firmware from eMMC to SD and the GPT is created *after* the dd.

Exactly.

Yes, this is all clumsy

Ideally we would have been given a proper OS installer which has either auto-layout, or manual layout for partitioning. But, ARM is different in needing specific things for booting. And each SoC needs it's own disk configuration. Perhaps a few years from now, we will have a template for booting, that has each SoC's support details. In our case, 3 separate blocks of code, at specific offsets. Plus, something for "/boot". And last, what devices are supported, in which order, (SPI, eMMC, SD card).

That comment about a proper OS installer is not meant as a criticism of my Pinebook Pro. This is somewhat where Linux was originally for x86. What we end up with, is easy installations. Not yet with alternate architectures, like ARM & ARM64.

Last, it has been a recomendation for a few years on AMD64 / x764 to have something like this, in case the boot loader needs more space.

Code:

/dev/sda1       2048     10239      8192     4M BIOS boot

There is a reserved GPT label just for that purpose.

[z4v4l]

A normal GPT with the recommended partition array (plus the protective MBR) runs from LBA0 through to LBA 33 (and is repeated at the end of the disk). It is not a problem to have the first partition at LBA 64.

it's a bare minimum - 1 block goes to the Protective MBR (LBA 0), 1block for the GPT Header (LBA 1) and in case of 512 byte blocks - 32 blocks go for the Partition Array. normally, utilities initializing GPT disks allocate larger space for avoiding "no entries for number of partitions you wanted" situations.
the alignment recommendation (it's 1 MB, not 2, I was wrong) is for efficiency purposes, again "only" a recommendation, but every normal utility respects it. it's both relared to RAID and flash based storages efficient operations.

Frankly I'm not sure who your comment about dd is aimed at. The only use of dd in the thread so far is to copy the firmware from eMMC to SD and the GPT is created *after* the dd.

i mentioned it because in the Arwen's recipe, he dd-s first 128 KB (256 KBlocks) of eMMC onto an SD card. but that chunk contains GPT structures! doesn't it? as I know, all the distributions, except armbian, use the Rockchip scheme, that in turn makes use of the GPT scheme. I was talking about this case - it's wrong in many ways, it won't even create the Back up parts of GPT tables. in addition to duplicating unique GUIDs.

2 arwen, just saw you response, i am using android tablet and i hate it! the "keyboard" is made by sadists, it's unusable.
i understood, that you dded for copying uboot pieces. i just thought, that copying gpt present on emmc, will make (s)fdisk think about modifying the existing gpt rather than creating new one. but that existing is wrong
since i use armbian on my pine boards, i probably am wrong about the layout of non armbian distributions for pine.

and yes, about a proper installation, i wrote that too a couple of weeks ago, maybe someone noticed. i am not a linux guy, so I can't try myself in it. but that thing would be very good to have. i don't agree, that it's arm difference though. nothing special, x86 starts its execution at the predefined address aw well, that platform just managed to agree on always having flash chip from which bios runs, starting it all. this mess with uboot pieces being hardcoded at places in the user areas of not so "firmware" storages is mostly due to overlooking on the OS side, linux, in this case  lack of development force, other reasons. should it get enough attention, the only normal place for these pieces are either (in)famous SPI NOR chip, pretty fatty on pine products or, shoul rockchip be not this lazy, the boot areas of eMMC, specifically made for this purpose, leaving all the collisions with user data. then creating a modern GPT laid out disk on user disks would be free of this clumsy stuff.