Pinetime and P8 Integration testing framework
#1
Once upon a time, i found my self in a position where i had to develop a firmware for an IoT device with different hardware revisions, different cellular modems used in several different countries / cellular networks. There was continous delivery of new and custom fw versions pushed over the air. To debug and verify new firmware releases was.. interesting.. 

I tried to set up solutions with jenkins or gitlab runners to automatically or on request deploy code to the targets, but it was hard to get it "right".

To make my life easier (possible), I developed an integration testing framework this framework consisted of  "test nodes" connecting to a "test manager" and then a API that i implementad a commandline interface and a web gui for. 

The test node:
The test node consist of a raspberry pi / orange pi / pine board, a jtag/swd flasher (or gpio flashing), a Ina219 module for current measuring, a servo and a relay.

DUT = Device Under Test, in this case a PineTime

The test node can:
  • cause button press / release

  • cause motion / tilt

  • measure current consumtion / charge current

  • turn on / off charging

  • read debug output (RTT and / or UART)
[Image: 92359212-2fe8fb00-f0eb-11ea-925c-080bcd278b3e.png]

The test manager
the manager keeps track of nodes and stores test results 

The API / GUI
lets the user allocate / queue test executions on test nodes. fetch / view logs of previous tests, view test results

The command line tool can be triggered like this:
python3 runOnTestNode.py completeTest node1 test.hex

you can also get tests, nodes, and test results listed as well as real time RTT/Debug output from the DUT.

while the GUI currently looks like this:


[Image: YGOvdb3Aw3A2hZIl7gcgR_rYkCaPyzswrwUaS6-R...authuser=0]


old comparison view between two executions:

[Image: 92360137-a33f3c80-f0ec-11ea-9298-821b8fd165db.png]


Everything is written in Python, and some html/javascript for the web interface. 

To make things easy, tests often rely on parsing debug output, this can be an issue in some cases, but in general it makes life easier for both developer and test framwork.

Here is an example of a test script (extract from powerOnPowerOff.py):
Code:
    def run(self):
        self.timeSinceStart = 0
        self.flash()

        # verify that modem was detected
        self.addDelayedParse(20,"IAmNow",1)

        # run, then turn off'
        self.addDelayedEvent(20,self.buttonIn,())
        self.addDelayedEvent(5,self.buttonOut,())
        self.addDelayedEvent(30,self.parse,("WillNowPowerOff",1))

        # verify that we do not boot when off and charging starts
        self.addDelayedEvent(10,self.chargerOn,())
        self.addDelayedEvent(20,self.parse,("OFFcharging",1))

        # wait, then turn on again, verify that we remember our modem type
        self.addDelayedEvent(10,self.buttonIn,())
        self.addDelayedEvent(5,self.buttonOut,())
        self.addDelayedEvent(30,self.parse,("IKnowIAmA",1))
        self.addDelayedEvent(1,self.end,())
        self.start()


I have also used this type of setup in production, where each test node is a test fixture for PCBA testing, you will want to modify the tests and framework a bit for this usecase.

So i will do two things, 1. i will release the test framework as open source, 2. I will leave my own hosting of the framework open for public use (with a pinetime node and a p8 node running)  

But first i must get everything up and running, its working fine right now, but i do only have "mocked" hw interfaces, i am working on wiring everything up and beautifying the user interface.


Messages In This Thread
Pinetime and P8 Integration testing framework - by maiden - 10-02-2020, 02:52 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Develop a new firmware for PineTime belushi 2 1,282 09-25-2023, 12:32 PM
Last Post: ccchan234
  Bluetooth BLE-MIDI-controller app for PineTime / InfiniTime Luno 0 499 08-20-2023, 05:17 AM
Last Post: Luno
  Zephyr Backlight Examples for PineTime lcj 0 691 05-06-2023, 02:54 PM
Last Post: lcj
  Zephyr is ready for pinetime jandy 1 2,074 05-06-2023, 02:15 PM
Last Post: lcj
  Send a message from Android to pinetime via BLE razrosman 0 905 11-05-2022, 08:24 AM
Last Post: razrosman
  PineTime Stuck in DFU Mode Eesha Barua 1 2,000 07-25-2022, 09:17 PM
Last Post: heyhewmike
  Idle tracking with PineTime: how versatile it is? schaman 1 1,610 07-13-2022, 12:50 AM
Last Post: wibble
  PineTime implemented with partial of pebble API jandy 0 1,535 03-20-2022, 08:58 PM
Last Post: jandy
  "Pine64 USB JTAG Adapter + OpenOCD + PineTime" should it work? ITCactus 4 3,874 03-02-2022, 05:58 AM
Last Post: wibble
  Zephyr based Pinetime jandy 4 4,631 11-11-2021, 05:53 AM
Last Post: jandy

Forum Jump:


Users browsing this thread: 1 Guest(s)