Pinetime and P8 Integration testing framework
#1
Once upon a time, i found my self in a position where i had to develop a firmware for an IoT device with different hardware revisions, different cellular modems used in several different countries / cellular networks. There was continous delivery of new and custom fw versions pushed over the air. To debug and verify new firmware releases was.. interesting.. 

I tried to set up solutions with jenkins or gitlab runners to automatically or on request deploy code to the targets, but it was hard to get it "right".

To make my life easier (possible), I developed an integration testing framework this framework consisted of  "test nodes" connecting to a "test manager" and then a API that i implementad a commandline interface and a web gui for. 

The test node:
The test node consist of a raspberry pi / orange pi / pine board, a jtag/swd flasher (or gpio flashing), a Ina219 module for current measuring, a servo and a relay.

DUT = Device Under Test, in this case a PineTime

The test node can:
  • cause button press / release

  • cause motion / tilt

  • measure current consumtion / charge current

  • turn on / off charging

  • read debug output (RTT and / or UART)
[Image: 92359212-2fe8fb00-f0eb-11ea-925c-080bcd278b3e.png]

The test manager
the manager keeps track of nodes and stores test results 

The API / GUI
lets the user allocate / queue test executions on test nodes. fetch / view logs of previous tests, view test results

The command line tool can be triggered like this:
python3 runOnTestNode.py completeTest node1 test.hex

you can also get tests, nodes, and test results listed as well as real time RTT/Debug output from the DUT.

while the GUI currently looks like this:


[Image: YGOvdb3Aw3A2hZIl7gcgR_rYkCaPyzswrwUaS6-R...authuser=0]


old comparison view between two executions:

[Image: 92360137-a33f3c80-f0ec-11ea-9298-821b8fd165db.png]


Everything is written in Python, and some html/javascript for the web interface. 

To make things easy, tests often rely on parsing debug output, this can be an issue in some cases, but in general it makes life easier for both developer and test framwork.

Here is an example of a test script (extract from powerOnPowerOff.py):
Code:
    def run(self):
        self.timeSinceStart = 0
        self.flash()

        # verify that modem was detected
        self.addDelayedParse(20,"IAmNow",1)

        # run, then turn off'
        self.addDelayedEvent(20,self.buttonIn,())
        self.addDelayedEvent(5,self.buttonOut,())
        self.addDelayedEvent(30,self.parse,("WillNowPowerOff",1))

        # verify that we do not boot when off and charging starts
        self.addDelayedEvent(10,self.chargerOn,())
        self.addDelayedEvent(20,self.parse,("OFFcharging",1))

        # wait, then turn on again, verify that we remember our modem type
        self.addDelayedEvent(10,self.buttonIn,())
        self.addDelayedEvent(5,self.buttonOut,())
        self.addDelayedEvent(30,self.parse,("IKnowIAmA",1))
        self.addDelayedEvent(1,self.end,())
        self.start()


I have also used this type of setup in production, where each test node is a test fixture for PCBA testing, you will want to modify the tests and framework a bit for this usecase.

So i will do two things, 1. i will release the test framework as open source, 2. I will leave my own hosting of the framework open for public use (with a pinetime node and a p8 node running)  

But first i must get everything up and running, its working fine right now, but i do only have "mocked" hw interfaces, i am working on wiring everything up and beautifying the user interface.
  Reply


Messages In This Thread
Pinetime and P8 Integration testing framework - by maiden - 10-02-2020, 02:52 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
Heart wasp-os: MicroPython for PineTime! danielt 85 23,126 3 hours ago
Last Post: laureote
  Update bootlogo and pinetime-app with OTA/DFU lookin 3 208 11-25-2020, 04:51 PM
Last Post: lookin
  PineTime Simulation Feature gjoyet 3 237 11-15-2020, 11:42 AM
Last Post: gjoyet
  Star Trek computer themed wasp-os on PineTime graynada 11 1,853 10-30-2020, 01:00 PM
Last Post: graynada
  Article: Create Your Own PineTime Watch Face in Rust... And Publish on crates.io lupyuen 2 445 10-18-2020, 09:24 PM
Last Post: lupyuen
  Article: Bluetooth Time Sync and LVGL on PineTime Mynewt lupyuen 0 263 10-15-2020, 05:51 PM
Last Post: lupyuen
  Remote PineTime for flashing and testing firmware remotely lupyuen 5 662 10-06-2020, 12:03 AM
Last Post: lupyuen
  Rust on PineTime at RIOT Summit lupyuen 3 895 09-25-2020, 05:05 PM
Last Post: lupyuen
  Article: Preview PineTime Watch Faces in your Web Browser with WebAssembly lupyuen 1 591 09-13-2020, 02:09 PM
Last Post: VMMainFrame
  Article: Porting PineTime Watch Face from C to Rust On RIOT with LVGL lupyuen 0 318 09-12-2020, 10:13 PM
Last Post: lupyuen

Forum Jump:


Users browsing this thread: 1 Guest(s)