My Daily Manual QA

[plainenough]

I will see what I can do to template this out a little better and begin drafting something a little more formal. glad that you guys are enjoying my little daily project.

[bcnaz]

I do appreciate your efforts, thank you,

Not sure how much time & effort you want to exert ?

But perhaps a side by side comparison of some of the operating systems ?

That could show us how they compare, 'call function' yes or no , -- 'call quality' maybe rate 1 thru 10 ?

Just suggestions

[daniel]

yes, I like them too. Can you make it more like a table? maybe keep the different updates in the same table, so we can see progress. And as bcnaz said add comparing OS?
thanks for doing this!!

[plainenough]

I have made a change to the way I will be posting these results sets. I will be uploading a report every time I execute testing and will post a link (Maybe a screenshot if preferred.) 

You can check https://tosstalk.com/pinephone/ for any OSes I test.

report-2020-07-27

Conclusion: Not ready to be a daily driver.

[natasha]

@ plainenough

I read some people have problems with the sound or when switching from speakers and headphones. Have you tested these?
The speakers are very low, I barely could follow a you tube video.
Not sure if you have played with bluetooth, gps, data plan from carrier...

haha sorry for asking too much. Not sure how much effort you can put on this. However, you are the only one doing this... It is nice to have it all in the same place!!

[plainenough]

@ plainenough

I read some people have problems with the sound or when switching from speakers and headphones. Have you tested these?
The speakers are very low, I barely could follow a you tube video.
Not sure if you have played with bluetooth, gps, data plan from carrier...

haha sorry for asking too much. Not sure how much effort you can put on this. However, you are the only one doing this... It is nice to have it all in the same place!!

Yes I have tested audio routing in a few ways on ubports. I found the speakers to be very muffled, and on certain builds they appear to not work at all. Although I think this is an audio driver issue not a hardware issue. (Although some of the sound quality issues I have observed from the device have indicated less than stellar loud speaker.)  As soon as I get something put together that isn't so manual (since I am really just manually testing all of this myself.) I will expand upon my test cases, work on automating them when possible and begin generating cross referencing reports to begin scoring different operating systems on the pinephone specifically.

Note: Audio on Dev-114 didn't work for me on initial boot.

Conclusion: This os isn't ready for the pinephone to be a daily driver.

ubports/2020-07-28

[daniel]

@ plainenough

I read some people have problems with the sound or when switching from speakers and headphones. Have you tested these?
The speakers are very low, I barely could follow a you tube video.
Not sure if you have played with bluetooth, gps, data plan from carrier...

haha sorry for asking too much. Not sure how much effort you can put on this. However, you are the only one doing this... It is nice to have it all in the same place!!

Yes I have tested audio routing in a few ways on ubports. I found the speakers to be very muffled, and on certain builds they appear to not work at all. Although I think this is an audio driver issue not a hardware issue. (Although some of the sound quality issues I have observed from the device have indicated less than stellar loud speaker.)  As soon as I get something put together that isn't so manual (since I am really just manually testing all of this myself.) I will expand upon my test cases, work on automating them when possible and begin generating cross referencing reports to begin scoring different operating systems on the pinephone specifically.

Note: Audio on Dev-114 didn't work for me on initial boot.

Conclusion: This os isn't ready for the pinephone to be a daily driver.

ubports/2020-07-28

oh no!! 114 audio now fails?

[daniel]

Look at this:
https://forum.pine64.org/showthread.php?tid=10883

Something else to add in the QA?

[plainenough]

Look at this:
https://forum.pine64.org/showthread.php?tid=10883

Something else to add in the QA?

Not yet, since there are more glaring issues that prevent this from being a daily driver. I'm on dev-116 and outbound calls are still non working. Certainly that will at some point be something that needs to have some focus on it, but likely not soon.

Conclusion: This os isn't ready for the pinephone to be a daily driver.
Note: Still no audio since dev-114 persists across reboots using OTA updates. Also, Issues keeping time on phone clock.

ubports/2020-07-29

[plainenough]

Sorry for the mixup yesterday, I uploaded the wrong report into ubports folder. I have started building out a management system for test results and have recruited a fellow linux engineer to begin testing with another pinephone ce device. We should begin seeing a rampup of testing speeds and a little better comparison between different operating systems.

Conclusion: This os isn't ready for the pinephone to be a daily driver.

Note: Still no audio since dev-114 persists across reboots using OTA updates. Phone still doesn't keep time.


For a list of the test cases and basic steps: https://github.com/plainenough/pinehphon...asets.yaml Expect very much rough draft for this moment, but I will try to keep a list of simple steps and test cases for your viewing pleasure/questions that might come up about my methods.

ubports/2020-07-30