My Daily Manual QA

[plainenough]

For a few weeks now I have been running through some simple QA on my pinephone. Things to note:

I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver.  (SMS, Calls, general sound quality, wifi, clock accuracy, etc. etc.)

Here are my results for today:
Wifi: seems fairly stable after reboot.
Boot time: No noticeable difference compared to day before. (~1 minute)
SMS in: Works as expected
SMS out: Works as expected
Calls out: Call failed
Calls in: Connects. no audio either direction.  (iphone and android)
Camera front: Black screen
Camera back: Black screen
Loud speaker: distorted audio from web browser

Noticeable issues:
Lockscreen wakes up without notification or interaction periodically (3-5 times per hour)
Lockscreen clock lags behind until unlock then corrects itself (over the course of the day by over an hour)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED.
After hours of idle phone unable to send texts or receive calls until reboot. (warm reboot)


Results of manual QA: Fail.
Phone not ready to trial daily driver.


If you would like to see more simple manual tests, or if you test something different feel free to post your reports.

[bcnaz]

In the name of comparing results,
  ( If you have a spare micro SD card)

  Try the Mobian nightly build on an SD card and compare those results with your current results.

Running Mobian from the SD card will not have any affect your currently installed operating system.

  I would be very interested to see your results comparing those two  .....
                                                                      ( Mobian may be a bit slower running from SD card, but close enough for testing )

Thanks,  BC

[plainenough]

In the name of comparing results,
  ( If you have a spare micro SD card)

  Try the Mobian nightly build on an SD card and compare those results with your current results.

Running Mobian from the SD card will not have any affect your currently installed operating system.

  I would be very interested to see your results comparing those two  .....
                                                                      ( Mobian may be a bit slower running from SD card, but close enough for testing )

Thanks,  BC

I will prep an extra card and make adjustments today. Will I need to flash the new build daily?

---

I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver.  (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)

OTA update - 91

Here are my results for today:
Wifi: seems fairly stable after reboot. ------------------------------------------------------------------------------------------------------------------------------------------------------PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 2m.30s) ------------------------------------------------------------------------------------PASS
SMS in: Works as expected after warm reboot -----------------------------------------------------------------------------------------------------------------------------------SOFT FAIL
SMS out: Works as expected after warm reboot ---------------------------------------------------------------------------------------------------------------------------------SOFT FAIL
Calls out: Call failed ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android call was stuffy, iphone call was clear -----SOFT FAIL
Camera front: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------------  FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker:  distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) -------------------------------------PASS

Noticeable issues: ( Will likely update this over the course of the day)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Clock lags behind until unlock then corrects itself (over the course of a hour is about 5 minutes of drift) Issue-179

[bcnaz]

After the initial download I believe you can just do OTA updates,  unless @awai does a new 'release'.

One problem I have had for a few weeks now,       but no one else seems to have,  (on a fresh boot or reboot)
  I need an incoming text to open my sms/text app (?)     Then it works fine. open and close, reopen etc.

Calls work almost flawless,  only small problem, the receiving party hears an echo if I am using 'speaker phone'...

wifi works pretty dependable...      Hot spot works,  but I have not used it much.

Camera sort of works,     but it is a work in progress. 

I think the sleep feature may be causing a problem on the July 16 nightly,  I noticed when I woke the phone, my recent text messages seem to have disappeared, they did not on the July 15 nightly.

I am using a Brave Heart phone running the Mobian nightly builds using a new flash for each nightly build.

I am definitely not the most qualified for testing,  I do look forward to hearing your comments and comparisons.

Thanks,  BC

[plainenough]

NOTE: Sorry for my few days of absence - I was just lazy.



I am running on the development channel.

I only test after a successful OTA update.

I have limited my scope to what I consider requirements for a daily driver.  (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)



OTA update - 103



Here are my results for today:

Wifi: seems fairly stable after reboot. ----------------------------------------------------------------------------------------------------------------------------------------------------- PASS

Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 3s) ------------------------------------------------------------------------------------ PASS

SMS in: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------- PASS

SMS out: Works as expected after cold boot ------------------------------------------------------------------------------------------------------------------------------------------- PASS

Calls out: Call never connects ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL

Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy ------------------------ SOFT FAIL

Camera front: Black screen ------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL

Camera back: Black screen -------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL

Loud speaker:  distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) ---------------------------------- PASS



Noticeable issues: ( Will likely update this over the course of the day)

Splash screen and desktop flicker. Might be related to auto-brightness.

LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Conclusion, not ready to be a daily driver. Issue-179

[bcnaz]

Have you compared this to Mobian (from sd card) ?

[plainenough]

Have you compared this to Mobian  (from sd card)  ?

 I am setting up an sdcard today to run the tests. I'll probably post results starting tomorrow. I'll post them in the debian thread. https://forum.pine64.org/showthread.php?tid=10811

-------------------------------------------------------------------------------------------------------------

I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver.  (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)


OTA update - 105

Here are my results for today:
Wifi: seems fairly stable after reboot. --------------------------------------------------------------------------------------------------------------------------------------------------- PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 15s) ------------------------------------------------------------------------------------------ PASS
SMS in: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------- PASS
SMS out: Works as expected after cold boot ------------------------------------------------------------------------------------------------------------------------------------------- PASS
Calls out: Call never connects ------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy ----------------------------------------- SOFT FAIL
Camera front: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker:  distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) ------------------------------------------------- PASS

Noticeable issues: ( Will likely update this over the course of the day)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Phone unable to keep time. Issue-179

Results: Not ready for daily driver.

[plainenough]

I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver.  (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)


OTA update - 107

Here are my results for today:
Wifi: seems fairly stable after reboot. ------------------------------------------------------------------------------------------------------------------------------------------------------------ PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 08s) ------------------------------------------------------------------------------------------ PASS
SMS in: Works as expected after cold boot ----------------------------------------------------------------------------------------------------------------------------------------------------- PASS
SMS out: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------------- PASS
Calls out: Call never connects ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy -------------------------------- SOFT FAIL
Camera front: Black screen --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker:  No audio from web browser (seems related to failed call, maybe audio routing works after reboot) ----------------------------------------------- FAIL

Noticeable issues: ( Will likely update this over the course of the day)
Unable to unlock phone for a certain amount of time after pushing lock button multiple times rapidly. Issue-180
Phone unable to keep time. Issue-179

Results: Not ready for daily driver.

[LazLong]

@plainenough,  Just a word of appreciation for these reports.

As a newbie with a BH received in Feb/March (but just now finding the time to begin testing) would love to see a daily poll
(from all those PP testers willing to participate) on each of your chosen QA functional categories/features particularly in/out calling,
in/out sms, and wifi.

I don't know if the forum software has a polling mechanism (thinking something like Survey Monkey) that could easily provide
a snapshot of gross results for each functionality test  or how it exactly should be done but it seems like this kind of daily QA feedback on the
present state of UBPorts,  Mobian, etc would help greatly and potentially save a lot of people a lot of time and frustration.

-- If there is something already like this available here in the forum, wiki or elsewhere that I haven't run across my apologies in advance.

[natasha]

yeah, I love them too. Rating UP!