+- PINE64 (https://forum.pine64.org)
+-- Forum: PinePhone (https://forum.pine64.org/forumdisplay.php?fid=120)
+--- Forum: PinePhone Software (https://forum.pine64.org/forumdisplay.php?fid=121)
+---- Forum: UBPorts on PinePhone (https://forum.pine64.org/forumdisplay.php?fid=125)
+---- Thread: My Daily Manual QA (/showthread.php?tid=10685)
My Daily Manual QA - plainenough - 07-15-2020
For a few weeks now I have been running through some simple QA on my pinephone. Things to note:
I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver. (SMS, Calls, general sound quality, wifi, clock accuracy, etc. etc.)
Here are my results for today:
Wifi: seems fairly stable after reboot.
Boot time: No noticeable difference compared to day before. (~1 minute)
SMS in: Works as expected
SMS out: Works as expected
Calls out: Call failed
Calls in: Connects. no audio either direction. (iphone and android)
Camera front: Black screen
Camera back: Black screen
Loud speaker: distorted audio from web browser
Noticeable issues:
Lockscreen wakes up without notification or interaction periodically (3-5 times per hour)
Lockscreen clock lags behind until unlock then corrects itself (over the course of the day by over an hour)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED.
After hours of idle phone unable to send texts or receive calls until reboot. (warm reboot)
Results of manual QA: Fail.
Phone not ready to trial daily driver.
If you would like to see more simple manual tests, or if you test something different feel free to post your reports.
RE: My Daily Manual QA - bcnaz - 07-15-2020
In the name of comparing results,
( If you have a spare micro SD card)
Try the Mobian nightly build on an SD card and compare those results with your current results.
Running Mobian from the SD card will not have any affect your currently installed operating system.
I would be very interested to see your results comparing those two .....
( Mobian may be a bit slower running from SD card, but close enough for testing )
Thanks, BC
RE: My Daily Manual QA - plainenough - 07-16-2020
(07-15-2020, 06:19 PM)bcnaz Wrote: In the name of comparing results,I will prep an extra card and make adjustments today. Will I need to flash the new build daily?
( If you have a spare micro SD card)
Try the Mobian nightly build on an SD card and compare those results with your current results.
Running Mobian from the SD card will not have any affect your currently installed operating system.
I would be very interested to see your results comparing those two .....
( Mobian may be a bit slower running from SD card, but close enough for testing )
Thanks, BC
I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver. (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)
OTA update - 91
Here are my results for today:
Wifi: seems fairly stable after reboot. ------------------------------------------------------------------------------------------------------------------------------------------------------PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 2m.30s) ------------------------------------------------------------------------------------PASS
SMS in: Works as expected after warm reboot -----------------------------------------------------------------------------------------------------------------------------------SOFT FAIL
SMS out: Works as expected after warm reboot ---------------------------------------------------------------------------------------------------------------------------------SOFT FAIL
Calls out: Call failed ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android call was stuffy, iphone call was clear -----SOFT FAIL
Camera front: Black screen --------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker: distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) -------------------------------------PASS
Noticeable issues: ( Will likely update this over the course of the day)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Clock lags behind until unlock then corrects itself (over the course of a hour is about 5 minutes of drift) Issue-179
RE: My Daily Manual QA - bcnaz - 07-16-2020
After the initial download I believe you can just do OTA updates, unless @awai does a new 'release'.
One problem I have had for a few weeks now, but no one else seems to have, (on a fresh boot or reboot)
I need an incoming text to open my sms/text app (?) Then it works fine. open and close, reopen etc.
Calls work almost flawless, only small problem, the receiving party hears an echo if I am using 'speaker phone'...
wifi works pretty dependable... Hot spot works, but I have not used it much.
Camera sort of works, but it is a work in progress.
I think the sleep feature may be causing a problem on the July 16 nightly, I noticed when I woke the phone, my recent text messages seem to have disappeared, they did not on the July 15 nightly.
I am using a Brave Heart phone running the Mobian nightly builds using a new flash for each nightly build.
I am definitely not the most qualified for testing, I do look forward to hearing your comments and comparisons.
Thanks, BC
RE: My Daily Manual QA - plainenough - 07-22-2020
NOTE: Sorry for my few days of absence - I was just lazy.
I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver. (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)
OTA update - 103
Here are my results for today:
Wifi: seems fairly stable after reboot. ----------------------------------------------------------------------------------------------------------------------------------------------------- PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 3s) ------------------------------------------------------------------------------------ PASS
SMS in: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------- PASS
SMS out: Works as expected after cold boot ------------------------------------------------------------------------------------------------------------------------------------------- PASS
Calls out: Call never connects ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy ------------------------ SOFT FAIL
Camera front: Black screen ------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen -------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker: distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) ---------------------------------- PASS
Noticeable issues: ( Will likely update this over the course of the day)
Splash screen and desktop flicker. Might be related to auto-brightness.
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Conclusion, not ready to be a daily driver. Issue-179
RE: My Daily Manual QA - bcnaz - 07-22-2020
Have you compared this to Mobian (from sd card) ?
RE: My Daily Manual QA - plainenough - 07-23-2020
(07-22-2020, 03:32 PM)bcnaz Wrote: Have you compared this to Mobian (from sd card) ?I am setting up an sdcard today to run the tests. I'll probably post results starting tomorrow. I'll post them in the debian thread. https://forum.pine64.org/showthread.php?tid=10811
-------------------------------------------------------------------------------------------------------------
I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver. (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)
OTA update - 105
Here are my results for today:
Wifi: seems fairly stable after reboot. --------------------------------------------------------------------------------------------------------------------------------------------------- PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 15s) ------------------------------------------------------------------------------------------ PASS
SMS in: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------- PASS
SMS out: Works as expected after cold boot ------------------------------------------------------------------------------------------------------------------------------------------- PASS
Calls out: Call never connects ------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy ----------------------------------------- SOFT FAIL
Camera front: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker: distorted audio from web browser (audio still the same, I think I have decided this is just the speaker) ------------------------------------------------- PASS
Noticeable issues: ( Will likely update this over the course of the day)
LED stays engaged after hitting the lock button multiple times. Requires another click or a full unlock to disable LED. Issue-180
Phone unable to keep time. Issue-179
Results: Not ready for daily driver.
RE: My Daily Manual QA - plainenough - 07-24-2020
I am running on the development channel.
I only test after a successful OTA update.
I have limited my scope to what I consider requirements for a daily driver. (SMS, 4G Calls, general sound quality, wifi, clock accuracy, etc. etc.)
OTA update - 107
Here are my results for today:
Wifi: seems fairly stable after reboot. ------------------------------------------------------------------------------------------------------------------------------------------------------------ PASS
Boot time: Changed reboot from simple restart to battery out cold boot. (~ 1m 08s) ------------------------------------------------------------------------------------------ PASS
SMS in: Works as expected after cold boot ----------------------------------------------------------------------------------------------------------------------------------------------------- PASS
SMS out: Works as expected after cold boot --------------------------------------------------------------------------------------------------------------------------------------------------- PASS
Calls out: Call never connects ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ FAIL
Calls in: Connects. Call connects, full audio ( still no audio routing) (iphone and android) *android and iphone call was stuffy -------------------------------- SOFT FAIL
Camera front: Black screen --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Camera back: Black screen ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- FAIL
Loud speaker: No audio from web browser (seems related to failed call, maybe audio routing works after reboot) ----------------------------------------------- FAIL
Noticeable issues: ( Will likely update this over the course of the day)
Unable to unlock phone for a certain amount of time after pushing lock button multiple times rapidly. Issue-180
Phone unable to keep time. Issue-179
Results: Not ready for daily driver.
RE: My Daily Manual QA - LazLong - 07-24-2020
@plainenough, Just a word of appreciation for these reports.
As a newbie with a BH received in Feb/March (but just now finding the time to begin testing) would love to see a daily poll
(from all those PP testers willing to participate) on each of your chosen QA functional categories/features particularly in/out calling,
in/out sms, and wifi.
I don't know if the forum software has a polling mechanism (thinking something like Survey Monkey) that could easily provide
a snapshot of gross results for each functionality test or how it exactly should be done but it seems like this kind of daily QA feedback on the
present state of UBPorts, Mobian, etc would help greatly and potentially save a lot of people a lot of time and frustration.
-- If there is something already like this available here in the forum, wiki or elsewhere that I haven't run across my apologies in advance.
RE: My Daily Manual QA - natasha - 07-24-2020
yeah, I love them too. Rating UP!