Getting Started | Visit the WIKI | IRC Logs | Forum Rules/Policy


Firmware Recovery
#1
Does anybody know if the firmware recovery method works on the PADI modules? According to Realtek's application notes, after upgrading the firmware via OTA it should be possible to fall back to the earlier default firmware by pulling GC0 to ground during power-on.

I wrote simple OTA server for Linux (link) and tried this with two of my BT-00 modules, but they always start the updated image. Is the first stage bootloader different on the PADI?

Tido
Reply
#2
(11-02-2016, 01:03 PM)tidklaas Wrote: Does anybody know if the firmware recovery method works on the PADI modules? According to Realtek's application notes, after upgrading the firmware via OTA it should be possible to fall back to the earlier default firmware by pulling GC0 to ground during power-on.

I wrote simple OTA server for Linux (link) and tried this with two of my BT-00 modules, but they always start the updated image. Is the first stage bootloader different on the PADI?

Tido

The firmware recovery works well on PADI and using the pulling GC0 to ground during power-on as you mentioned. I don't think the firmware recovery on BT-00 module works correctly.

The PADI and BT-00 do not using same set of firmware and there are some minor different between both module. If you encounter any PADI firmware rcovery issue, please visit Pine64 IRC, http://pine64,xyz, and ping klliew.
Reply
#3
I've had mixed results with the firmware recovery after OTA images. Sometimes the GC0 trick restored the image - but I still have one 'dead' PADI which I am now unable to connect to. Waiting for my JWT Debugger to investigate further.
Reply
#4
(11-03-2016, 05:02 AM)ItsDave Wrote: I've had mixed results with the firmware recovery after OTA images. Sometimes the GC0 trick restored the image - but I still have one 'dead' PADI which I am now unable to connect to. Waiting for my JWT Debugger to investigate further.

We are curious how you brick the PADI, we need to make sure that maker can program and play around PADI without the fear of bricking it.
Reply
#5
Hey Tllim,

It was just flashing the default image I compiled using the example project in the SDK. It all compiles fine, creates the 2 .bin files, but as soon as I OTA it, my serial console goes dead and i'm unable to get any output from it, or on any other pin (I tried them all!)

Edit: I wouldn't worry too much for now, as my serial debugger seems to have given up and has stopped working. So it may be that was related to it - i'll wait for the official debugger to arrive and report back Smile

Dave.
Reply
#6
(11-03-2016, 06:04 PM)ItsDave Wrote: Hey Tllim,

It was just flashing the default image I compiled using the example project in the SDK. It all compiles fine, creates the 2 .bin files, but as soon as I OTA it, my serial console goes dead and i'm unable to get any output from it, or on any other pin (I tried them all!)

Edit: I wouldn't worry too much for now, as my serial debugger seems to have given up and has stopped working. So it may be that was related to it - i'll wait for the official debugger to arrive and report back Smile

Dave.

Noted, welcome on improvement suggestion.

... TL
Reply
#7
I finally received a handful of PADI modules last week and spent a little quite some time over the weekend analyzing the OTA update and firmware recovery methods. Generally speaking, the firmware on the PADI is an improvement over the one on the B&T modules, but there are still a few caveats involved. 

First, a few definitions:
Bootloader (BL) / Img1
First stage image loaded by ROM code, always located at flash address 0x0. No source available in any SDK.

Img2
Runtime firmware image, loaded by Bootloader.

Default Image (DI)
First Img2. Located at address given in Bootloader's header. All Bootloader images I have seen point to address 0xB000.

Upgraded Image (UI)
Optional second Img2. Address is recorded in the System Data Sector at offset 0x0. If no address is given there, it is assumed that no Upgraded Image exists.

System Data Sector (SDS)
Located at address 0x9000. Holds address of Upgraded Image and Recovery Pin configuration

Recovery Pin (RP)
GPIO pin that is sampled by the Bootloader. If RP is pulled low and there are two Img2 available, BL will execute the older one.

Image Signature
The Img2 header contains an 8 byte signature used by the BL to determine which one of the two images is the current one. The signature for the current image is the string "81958711", the one for the previous/old image is "01958711"

SDK
I only used the GCC based SDK and did not look at the IAR or Arduino based SDKs

Now on to the analysis...

Boot flow:
According to AN0033, the BL will first look for the address of the Upgraded Image in the System Data Sector. If it does not find a valid address there, it will try to load and execute the Default Image from address 0xB0000.
If a valid address is found, it will load both images' headers, compare their signatures and execute the current Img2.

OTA update flow according to AN0033:
The OTA update function is supposed to figure out which of the two Img2 is the older one, erase its partition and write the new Img2 to it. If the update was written to the second Img2 partition, it must also update the offset address in the SDS. Afterwards it should make sure that the other Img2's signature is updated to mark it as the older Img2. 

Problems with OTA update:
The first problem is that the application note does not specify which Img2 is to be loaded if both images have the same signature. From my tests I know that in this case the BL will always load the UI.

Another problem is the undocumented use of the recovery pin. The BL will read the pin configuration from SDS offset 0x8 and then check the state of the given pin. If it is pulled low, it will invert its search pattern for the image
signatures. I.e., it will try to load the Img2 marked as the older one. Again, if both images have identical signatures, the UI is loaded by default.

Contrary to AN0033, in the default configuration the OTA update function from the SDK will always write the new Img2 to the second firmware partition. This is generally a good idea, as it will keep the original DI, preserving a 
working firmware for recovery. Or rather it would be, if the DI's signature was updated. Which it is not. So if the DI is the current one and the OTA installs a new UI, both images will be marked as current. As mentioned before, this
breaks the recovery mechanism because from now on the BL will always load the UI. 

OTA update functionality is implemented in component/common/utilities/update.c. There is a #define SWAP_UPDATE that will alter the update process. If this is enabled, the update function will choose the partition to update based on the images' signatures. The partition containing the older image (or no image at all) is erased and the new image is written in its place. Afterwards, the other image's signature is updated to mark it as the older image. The upside here is that there will always be exactly one current image, but after two updates the original factory firmware will be lost.

This feature is enabled in the PADI firmware found on the modules. When  updating such a module with an OTA image generated from an unmodified SDK (SWAP_UPDATE disabled) this can break the module's recovery capability. For example, given the following sequence of OTA updates using stock SDK images, the module will end up without working recovery mode:

Initial state:
0x0B000: PADI image, signature "81958711" <- running
0x80000: PADI image, signature "01958711"  
  => start 1st OTA update with SDK image, reset

After 1st OTA update:
0x0B000: PADI image, signature "01958711"
0x80000: SDK image,  signature "81958711" <- running
  => no OTA update, reset with recovery pin pulled low

After 1st OTA update, recovery mode:
0x0B000: PADI image, signature "01958711" <- running
0x80000: SDK image,  signature "81958711" 
  => start 2nd OTA update with SDK image, reset

After 2nd OTA update:
0x0B000: SDK image,  signature "81958711" <- running
0x80000: SDK image,  signature "01958711"
  => start 3rd OTA update, reset
 
After 3rd OTA update:
0x0B000: SDK image,  signature "81958711"
0x80000: SDK image,  signature "81958711" <- running
  => both Img2 are marked as current. BL will always load UI, recovery is broken!

Recover Pin Configuration:
The GPIO pin to use for recovery mode is not determined by a compiled in default. The BL reads the byte at offset 0x8 from the SDS and checks the pin configured there. Format is as follows:

((gpio_port & 0xF) << 4) | (port_pin & 0xF)

where  gpio_port is (probably) in the range 0-4 for GPIO ports A-E and gpio_pin is in the range 0-6. I only tested some combination on GPIO ports A and C. The default value found on the PADI module is 0x20, which is GC_0. I also verified that values 0x21 (GC_1), 0x01 (GA_1) and 0x02(GA_2) to work.

On the one hand this is a neat feature, as it allows the same BL image to be used with different boards. On the other hand there is a serious risk of losing the recovery capability during OTA update. The updater will copy the SDS to a backup sector, then erase the SDS, write the upgraded image's address and copy back the rest of the SDS from the backup sector. If there is a power loss between erasing the SDS and restoring the old content from the backup sector, the recover pin configuration is lost.

Problems with SWD/JTAG flashing:
Flashing an image via SWD/JTAG removes the recovery pin configuration.
When a firmware image is flashed via SWD/JTAG, the SDS is erased. This removes the recovery pin configuration and disables the firmware recovery method permanently.

Workaround: after compiling and before flashing, use a hex editor on  ram_all.bin to change the byte at offset 0x9008 from 0xff to 0x20. This restores the default recovery pin to be GC_0.
To enable fall-back to DI after OTA update, change the byte at offset 0xB008 from 0x38 to 0x30.

Okay, this post became a little longer than planned, but I hope the information will be useful to other people.


Tido
Reply
#8
(11-22-2016, 12:27 PM)tidklaas Wrote: I finally received a handful of PADI modules last week and spent a little quite some time over the weekend analyzing the OTA update and firmware recovery methods. Generally speaking, the firmware on the PADI is an improvement over the one on the B&T modules, but there are still a few caveats involved. 

First, a few definitions:
Bootloader (BL) / Img1
First stage image loaded by ROM code, always located at flash address 0x0. No source available in any SDK.

Img2
Runtime firmware image, loaded by Bootloader.

Default Image (DI)
First Img2. Located at address given in Bootloader's header. All Bootloader images I have seen point to address 0xB000.

Upgraded Image (UI)
Optional second Img2. Address is recorded in the System Data Sector at offset 0x0. If no address is given there, it is assumed that no Upgraded Image exists.

System Data Sector (SDS)
Located at address 0x9000. Holds address of Upgraded Image and Recovery Pin configuration

Recovery Pin (RP)
GPIO pin that is sampled by the Bootloader. If RP is pulled low and there are two Img2 available, BL will execute the older one.

Image Signature
The Img2 header contains an 8 byte signature used by the BL to determine which one of the two images is the current one. The signature for the current image is the string "81958711", the one for the previous/old image is "01958711"

SDK
I only used the GCC based SDK and did not look at the IAR or Arduino based SDKs

Now on to the analysis...

Boot flow:
According to AN0033, the BL will first look for the address of the Upgraded Image in the System Data Sector. If it does not find a valid address there, it will try to load and execute the Default Image from address 0xB0000.
If a valid address is found, it will load both images' headers, compare their signatures and execute the current Img2.

OTA update flow according to AN0033:
The OTA update function is supposed to figure out which of the two Img2 is the older one, erase its partition and write the new Img2 to it. If the update was written to the second Img2 partition, it must also update the offset address in the SDS. Afterwards it should make sure that the other Img2's signature is updated to mark it as the older Img2. 

Problems with OTA update:
The first problem is that the application note does not specify which Img2 is to be loaded if both images have the same signature. From my tests I know that in this case the BL will always load the UI.

Another problem is the undocumented use of the recovery pin. The BL will read the pin configuration from SDS offset 0x8 and then check the state of the given pin. If it is pulled low, it will invert its search pattern for the image
signatures. I.e., it will try to load the Img2 marked as the older one. Again, if both images have identical signatures, the UI is loaded by default.

Contrary to AN0033, in the default configuration the OTA update function from the SDK will always write the new Img2 to the second firmware partition. This is generally a good idea, as it will keep the original DI, preserving a 
working firmware for recovery. Or rather it would be, if the DI's signature was updated. Which it is not. So if the DI is the current one and the OTA installs a new UI, both images will be marked as current. As mentioned before, this
breaks the recovery mechanism because from now on the BL will always load the UI. 

OTA update functionality is implemented in component/common/utilities/update.c. There is a #define SWAP_UPDATE that will alter the update process. If this is enabled, the update function will choose the partition to update based on the images' signatures. The partition containing the older image (or no image at all) is erased and the new image is written in its place. Afterwards, the other image's signature is updated to mark it as the older image. The upside here is that there will always be exactly one current image, but after two updates the original factory firmware will be lost.

This feature is enabled in the PADI firmware found on the modules. When  updating such a module with an OTA image generated from an unmodified SDK (SWAP_UPDATE disabled) this can break the module's recovery capability. For example, given the following sequence of OTA updates using stock SDK images, the module will end up without working recovery mode:

Initial state:
0x0B000: PADI image, signature "81958711" <- running
0x80000: PADI image, signature "01958711"  
  => start 1st OTA update with SDK image, reset

After 1st OTA update:
0x0B000: PADI image, signature "01958711"
0x80000: SDK image,  signature "81958711" <- running
  => no OTA update, reset with recovery pin pulled low

After 1st OTA update, recovery mode:
0x0B000: PADI image, signature "01958711" <- running
0x80000: SDK image,  signature "81958711" 
  => start 2nd OTA update with SDK image, reset

After 2nd OTA update:
0x0B000: SDK image,  signature "81958711" <- running
0x80000: SDK image,  signature "01958711"
  => start 3rd OTA update, reset
 
After 3rd OTA update:
0x0B000: SDK image,  signature "81958711"
0x80000: SDK image,  signature "81958711" <- running
  => both Img2 are marked as current. BL will always load UI, recovery is broken!

Recover Pin Configuration:
The GPIO pin to use for recovery mode is not determined by a compiled in default. The BL reads the byte at offset 0x8 from the SDS and checks the pin configured there. Format is as follows:

((gpio_port & 0xF) << 4) | (port_pin & 0xF)

where  gpio_port is (probably) in the range 0-4 for GPIO ports A-E and gpio_pin is in the range 0-6. I only tested some combination on GPIO ports A and C. The default value found on the PADI module is 0x20, which is GC_0. I also verified that values 0x21 (GC_1), 0x01 (GA_1) and 0x02(GA_2) to work.

On the one hand this is a neat feature, as it allows the same BL image to be used with different boards. On the other hand there is a serious risk of losing the recovery capability during OTA update. The updater will copy the SDS to a backup sector, then erase the SDS, write the upgraded image's address and copy back the rest of the SDS from the backup sector. If there is a power loss between erasing the SDS and restoring the old content from the backup sector, the recover pin configuration is lost.

Problems with SWD/JTAG flashing:
Flashing an image via SWD/JTAG removes the recovery pin configuration.
When a firmware image is flashed via SWD/JTAG, the SDS is erased. This removes the recovery pin configuration and disables the firmware recovery method permanently.

Workaround: after compiling and before flashing, use a hex editor on  ram_all.bin to change the byte at offset 0x9008 from 0xff to 0x20. This restores the default recovery pin to be GC_0.
To enable fall-back to DI after OTA update, change the byte at offset 0xB008 from 0x38 to 0x30.

Okay, this post became a little longer than planned, but I hope the information will be useful to other people.


Tido
Thanks on the detail write up, internally we have explore a lot of on the firmware flashing and recovery method. Your suggestion will help us to improve further.

... TL Lim
Reply


Possibly Related Threads...
Thread Author Replies Views Last Post
  May we hope about original firmware ? Grag38 0 289 02-09-2017, 07:02 AM
Last Post: Grag38

Forum Jump:


Users browsing this thread: 1 Guest(s)