PINE64
Beta support: MMS on chatty/mmsd on Modem Manager - help/testing wanted! - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: PinePhone (https://forum.pine64.org/forumdisplay.php?fid=120)
+--- Forum: PinePhone Software (https://forum.pine64.org/forumdisplay.php?fid=121)
+--- Thread: Beta support: MMS on chatty/mmsd on Modem Manager - help/testing wanted! (/showthread.php?tid=12491)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

RE: Experimental support MMS on chatty - help/testing wanted! - kent - 01-05-2021

Something is still not working quite right. Here's what mmsd is outputting at the end of the attachment download request:

Code:
.
.
bytes read 40
[body] length 40
bytes read 903
[body] length 903
bytes read 0
mmsd[14964]: src/service.c:web_get_cb() status: 200
mmsd[14964]: src/service.c:web_get_cb() data size = 833811
mmsd[14964]: src/service.c:web_get_cb() request->result_cb=0xaaaac94dc610 vs. retrieve_conf=0xaaaac94dc610/send_conf=0xaaaac94da6c0/notify_resp=0xaaaac94dd400
mmsd[14964]: src/store.c:generate_uuid_from_pdu() pdu 0xffffab720000 len 833811
mmsd[14964]: src/store.c:mms_store_file() pathname /home/manjaro/.mms/any/A0E4098838D985980D725864ED21EE07B8A92867
.
.
When a mms comes in, mmsd outputs pages of "bytes read/[body] length" entries and then pauses for a good 30 seconds before popping up the final "bytes read 0". After that, the process continues to completion pretty quickly. The output reports a bit over 8K stored, and the file in ~/.mms/any matches the reported size.

On the Chatty side, the sms comes in immediately. A separate conversation is created with the "a mms has arrived, trying to d/l" message. After the mmsd long pause, 7 "file already exists" error messages pop up in the sms conversation. In ~/mms, an empty directory coded by date-time-sender is created for the mms that matches the Chatty error messages.

I'm not sure where the breakdown is happening.  I spent some time tracing the debug output through the mmsd code and there is nothing obviously amiss on the mmsd side. It looks like mmsd is properly getting the download, but that long pause after the data chunks might be significant.

How does the handoff to purple-mms-sms work? Is purple creating the files in ~/mms from the raw files in ~/.mms/any?

Also, does mmsd output a logfile somewhere? Debug output for 8k of chunks has overwhelmed my ssh buffer.

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-05-2021

Quote:On the Chatty side, the sms comes in immediately. A separate conversation is created with the "a mms has arrived, trying to d/l" message. After the mmsd long pause, 7 "file already exists" error messages pop up in the sms conversation. In ~/mms, an empty directory coded by date-time-sender is created for the mms that matches the Chatty error messages.

I'm not sure where the breakdown is happening.  I spent some time tracing the debug output through the mmsd code and there is nothing obviously amiss on the mmsd side. It looks like mmsd is properly getting the download, but that long pause after the data chunks might be significant.

How does the handoff to purple-mms-sms work? Is purple creating the files in ~/mms from the raw files in ~/.mms/any?
From what you are describing, the breakdown is happening on the purple-mm-sms side. purple takes the raw files from mmsd (that are in the ~/.mms/any/ directory) and works on them in the function "pur_mmsd_receive_message()" (and puts them in ~/mms). The debug for purple-mm-sms (successful) looks similar to this:





Code:
pur_mmsd_receive_message: New MMS at /org/ofono/mms/any/9D7B4C23EB3691E2A4C6EEF3260589E7446E425E
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:13.887: {'Status': <'received'>, 'Date': <'2021-01-05T06:41:06-0500'>, 'Subject': <'NoSubject'>, 'Sender': <'1xxxxxxxxxx'>, 'Recipients': <['+1yyyyyyyyyy']>, 'Attachments': <[('<image000000.jpg>', 'image/jpeg', '/home/mobian/.mms/any/9D7B4C23EB3691E2A4C6EEF3260589E7446E425E', uint64 501, uint64 60173)]>, 'Smil': <'<smil><head><layout><root-layout width="100%" height="100%"/><region id="Image" width="100%" height="100%" top="0%" left="0%" fit="meet"/></layout></head><body><par dur="5000ms"><img src="image000000.jpg" region="Image"/></par></body></smil>'>}
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:13.889: MMS Message Status: received
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:13.889: Recieved an MMS from someone else! Processing it....
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.100: ['+1yyyyyyyyyy']
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.100: [('<image000000.jpg>', 'image/jpeg', '/home/mobian/.mms/any/9D7B4C23EB3691E2A4C6EEF3260589E7446E425E', 501, 60173)]
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.110: modem_number: +1yyyyyyyyyy
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.110: You have 1 Recipients!
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.110: You only have one recipient! Posting MMS as IM.
(sm.puri.Chatty:2399): chatty-xeps-DEBUG: 06:41:14.111: Receiving msg on (nil) from +1xxxxxxxxxx with flags 2050
(sm.puri.Chatty:2399): chatty-manager-DEBUG: 06:41:14.139: Posting message id:(null) flags:2050 type:1 from:+1xxxxxxxxxx
** (sm.puri.Chatty:2399): DEBUG: 06:41:14.157: ../src/chatty-utils.c:83 number SMS
** (sm.puri.Chatty:2399): DEBUG: 06:41:14.182: ../src/chatty-utils.c:83 number +1xxxxxxxxxx
(sm.puri.Chatty:2399): mm-sms-DEBUG: 06:41:14.345: MMS delete finished
(06:41:19) util: Writing file blist.xml to directory /home/mobian/.purple
(06:41:19) util: Writing file /home/mobian/.purple/blist.xmlk




You may need to compile chatty with the debug options on to see the output there. I did it with



Code:
meson build -Db_sanitize=address -Dbuildtype=debug
During the build process (stock chatty on Mobian didn't give me the debug output I needed). WHen it is done compiling, go in the "build folder and run:

Code:
G_MESSAGES_DEBUG=all ./run --debug --verbose




You can also run pidgin with the debug options on with

Code:
G_MESSAGES_DEBUG=all pidgin


That should help you with where the issue is on purple-mm-sms. Note that I have been able to run chatty over ssh (so itis easier to debug), and pidgin I cannot (so it's a bit more difficult to do.



Quote:Also, does mmsd output a logfile somewhere? Debug output for 8k of chunks has overwhelmed my ssh buffer.
I am honestly not sure. I only started looking at mmsd to build an MM plugin. But I think your issue is in purple-mm-sms.

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-05-2021

So it seems that in order for an mmsd plugin to talk to modem manager, all of the dbus interfaces from within that plugin have to be rewritten to target dbus-2.0 vs dbus 1.0. I made a "wip branch" with the hopes of updating the dbus interfaces from within simple.c to dbus-2.0:

https://source.puri.sm/kop316/mmsd/-/tree/wip

If there is anyone with experience in dbus, I would really appreciate help from them!

RE: Experimental support MMS on chatty - help/testing wanted! - Lousy Fisherman - 01-05-2021

(01-05-2021, 11:02 AM)kop316 Wrote: So it seems that in order for an mmsd plugin to talk to modem manager, all of the dbus interfaces from within that plugin have to be rewritten to target dbus-2.0 vs dbus 1.0. I made a "wip branch" with the hopes of updating the dbus interfaces from within simple.c to dbus-2.0:

https://source.puri.sm/kop316/mmsd/-/tree/wip

If there is anyone with experience in dbus, I would really appreciate help from them!
I've been hacking on DBus on the PPhone for a month or 2 (or 100 it seems) now. What do you need?
I'm using python and shell

LF

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-05-2021

(01-05-2021, 11:51 AM)Lousy Fisherman Wrote:
(01-05-2021, 11:02 AM)kop316 Wrote: So it seems that in order for an mmsd plugin to talk to modem manager, all of the dbus interfaces from within that plugin have to be rewritten to target dbus-2.0 vs dbus 1.0. I made a "wip branch" with the hopes of updating the dbus interfaces from within simple.c to dbus-2.0:

https://source.puri.sm/kop316/mmsd/-/tree/wip

If there is anyone with experience in dbus, I would really appreciate help from them!
I've been hacking on DBus on the PPhone for a month or 2 (or 100 it seems) now. What do you need?
I'm using python and shell

LF

You are amazing!

Here is the original plugin code: https://source.puri.sm/kop316/mmsd/-/blob/simple/plugins/simple.c

And the WIP code: https://source.puri.sm/kop316/mmsd/-/blob/wip/plugins/simple.c

All I have done so far is comment out the old code and add in the dbus2.0 libraries at the top. However, everything that is commented needs to be changed to support dbus-2.0 instead of dbus-1.0.

I am working from this code as a reference: https://gitlab.gnome.org/GNOME/glib/-/blob/master/gio/tests/gdbus-example-server.c

Right now I cannot get the reference code to compile as a stand alone, so I am working on that so I can understand how it is run.

EDIT: I am finally able to compile the example-server with
Code:
gcc -Wall -o gdbus-example-server gdbus-example-server.c $(pkg-config --cflags --libs glib-2.0 gio-2.0)
I am also able to port all of the example server onto mmsd and get it to run, but it requires manual tweaking. the loader requires gio-2.0 (and I am not familiar enough with automake to see how to add it, but I suppose that is lower on the to do list rather than getting it to run). The plugin will eventually also require mm-glib too (that way the mmsd plugin can talk to modem manager).

I added those guts to simple.c in the "wip" branch, but to get it to compile and run, you must manually add "-lgio-2.0 -lglib-2.0 -lgobject-2.0" to LDFLAGS in the makefile.

RE: Experimental support MMS on chatty - help/testing wanted! - kent - 01-05-2021

(01-05-2021, 05:44 AM)kop316 Wrote: I am honestly not sure. I only started looking at mmsd to build an MM plugin. But I think your issue is in purple-mm-sms.

Yeah. I figured ultimately this would take looking at purple. Right now I'm doing more research on the long pause at the end of requests in mmsd. I have a couple specific quick questions.

Do you experience a long pause at the end of requests through the apn you are using?
Does mmsd output pages of debugging output while an mms downloads?
On the purple side, am I correct in understanding purple processes the files in ~/.mms/any/ and creates output in ~/mms?

Also, I peeked at the initial work you are doing in simple.c. Are you working to expand plugin.simple to replace plugin.ofono?

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-05-2021

(01-05-2021, 02:04 PM)kent Wrote: Do you experience a long pause at the end of requests through the apn you are using?

No I do not. From chatty registering the SMS WAP to the MMS appearing on chatty is less than 5 seconds? and there is no noticable pause.

(01-05-2021, 02:04 PM)kent Wrote: Does mmsd output pages of debugging output while an mms downloads?

Yes, there is a fair amount of debugging output when an mms downloads.

(01-05-2021, 02:04 PM)kent Wrote: On the purple side, am I correct in understanding purple processes the files in ~/.mms/any/ and creates output in ~/mms?

Yes. that is correct.

(01-05-2021, 02:04 PM)ken Wrote: Also, I peeked at the initial work you are doing in simple.c. Are you working to expand plugin.simple to replace plugin.ofono?
No, I plan on leaving plugin.ofono alone. I intend on plugin.simple to evolve to plugin.modemmanager. Any help on that is welcome! Right now I am working on the "wip" branch to get simple.c to be dbus-2.0 (this is required to be compatible with mm-glib).

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-06-2021

So I seem to have fixed the dbus 2.0 vs. 1.0 issue in mmsd. Essentially, simple.c will only talk 2.0, while the rest of mmsd talks 1.0. So I effectively have two dbusses. While this may not be the cleanest solution, it is the solution that ensures I don't have to rewrite mmsd to talk dbus 2.0 .

If others have a different idea I welcome it.

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-06-2021

I have been able to update the simple.c plugin to now be compatible with Modem Manager (and hope to start work on further integration of the two soon!).

In the simple.c, I also added two method calls which I hope will be useful: ChangeSettings and PowerOnorOff. ChangeSettings allows you to change the mmsc, mmsproxy, and the interface via the dbus, and the plugin will a) overwrite the config file with the new settings (to keep them), and b) restart the plugin to let the settings take effect. I will note that right now, PowerOnorOff is commented out, as it causes some weird issues if you turn it off. I think it would be useful though your system goes ito suspend and you want to suspend that too (so the code to do it is there, just commented out).

EDIT:

I got some great help from craftyguy in how to get the autotools to work! It was a lot simplier than I thought it would be. You need to update to the latest purple-mm-sms and mmsd if you want to try the changes:

https://source.puri.sm/kop316/purple-mm-sms/-/tree/mmsd
https://source.puri.sm/kop316/mmsd/-/tree/simple

The simple.c plugin is now fully compatible with Modem Manager! This is good because now the two can be glued together to work.

RE: Experimental support MMS on chatty - help/testing wanted! - kop316 - 01-07-2021

So some good news and bad news:

I like good news first:
Thanks to Purisms work in purple-mm-sms, integrating a Modem Manager plugin was actually really straight forward. I have a beta version of it here:

https://source.puri.sm/kop316/mmsd/-/tree/ModemManager
https://source.puri.sm/kop316/purple-mm-sms/-/tree/ModemManager

I still need to look at ofono's plugin and see what else makes sense to integrate. The primary one I see if their bearer handler (which makes a lot more sense now).

And now for the bad news:
It is largely related to this issue:
https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/issues/227
As of now, the Pinephone can only handle one bearer at a time. What does this mean in plain english? If you have a carrier with an MMS APN AND a data APN, you can only have access to MMS or data, NOT both at once. If it is a hardware limitation, there's nothing that can be done. If it is a software limitation, that has to be changed in Modem Manager

EDIT:
So with some help from Mr. Sadiq at Purism, we found the following:
Like I said before, Modem Manager should be able to handle more than one bearer:
https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/blob/master/src/mm-bearer-list.c#L62

The default value is "1":
https://gitlab.freedesktop.org/mobile-broadband/ModemManager/-/blob/master/src/mm-bearer-list.c#L374
Unless the driver overrides it. So I suspect the issue is that the EC25 driver does not override this (but I need to look into the code).

If someone wants to look into it, that would be appreciated!

.