Android remote control using SSH/ADB/scrcpy
#1
Hello everyone, first time poster (long time lurker). Smile

I wanted to share a experiment I've been working on. Maybe someone can find useful, or help me improve on the ideas described below. I purchased my Pinephone BE some while ago and I've been slowly working on it with hopes that I'll be able to turn it into my main phone. I'm using Archlinux with SXMO (flashed to eMMC) and I'm really happy with the level of customization and tinkering that's possible (it feels I'm still just scratching the surface). SXMO feels light and snappy, gestures are easy to remember (once you get accustomed). Calls, SMSs, mobile data (and even the mobile hotspot functionality) work excellently. Battery drains fast while in use, but lasts about 24+ hours while in suspend mode (I think I can live with that).

Detaching from Android has been difficult due to some services (like my home alarm system) only being controllable through an Android or IOS app. This has been a huge bummer, as it has kept me needing to carry my Android phone with me whenever I leave home (to arm/unarm and check on the system). If only there was a way to control my Android phone remotely, I'd be able to leave it at home most (if not all) of the time.

I've thus been looking into ways to achieve this. Ideally, I wanted to be able to control the droid phone through SSH. I've used scrcpy before to control my phone with my PC by plugging it to an USB port. Once I realized scrcpy could be SSH-forwarded I knew I was on to something!

The scrcpy utility is not available as an ARM package on the Archlinux repositories, but I was able to build it on the Pinephone easily by following instructions here:
https://github.com/Genymobile/scrcpy/blo...r/BUILD.md
Most dependencies were already installed on Arch by default (I think only exceptions where wget and meson). I think the android utilities (android-tools package on Arch) where also needed on the Pinephone as well.

I have an old laptop running 24/7 as a home server, so I installed the android-tools package there as well. This allows launching an ADB server on the background that can run continuously. I created a simple systemd service to achieve this and started/enabled it on the home-server:

Code:
[Unit]
Description=ADB server service
Requires=network.target
After=network-online.target

[Service]
User=paul
ExecStart=/usr/bin/adb -a nodaemon server
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

I already had debugging enabled on my Android phone, which means I can just plug it into the home-server and it gets detected by the adb daemon, nice!

As a last step I needed to be able to dynamically create a SSH connection on the Pinephone that forwards the ports used by scrcpy to connect to the adb server (5037 and 27183). SXMO allows creating userscripts that then become easily accessible through the UI. I created a simple shell script that creates the SSH connection and launches scrcpy:

Code:
#!/bin/sh

SOCKET=/tmp/droid-tunnel-ctrl-socket
SCRLOG=/tmp/droid-tunnel-scrcpy-log

# Parameters
SSHCON=$1
VIDSIZE=$2
VIDFPS=$3

notify-send "DROID TUNNEL" "Launching tunnel through '$SSHCON', size:$VIDSIZE, fps:$VIDFPS"

on_exit() {
        kill -9 $SCRPID
        ssh -S $SOCKET -O exit $SSHCON
}

trap on_exit EXIT

ssh -M -S $SOCKET -fnNT -L5037:localhost:5037 -L27183:localhost:27183 $SSHCON
ssh -S $SOCKET -O check $SSHCON

killall -v scrcpy
scrcpy -V verbose --force-adb-forward --max-fps $VIDFPS --max-size $VIDSIZE > $SCRLOG 2>&1 &
SCRPID=$!

while sleep 0.1 ; do
        grep --quiet "X connection to :0 broken" $SCRLOG && break
done

On the shell script above, the first argument (SSHCON) represents the SSH connection to the home-server. This is pre-configured on my ~/.config/ssh. VIDSIZE is the resolution of the generated video that's captured live from the Android phone (long side). VIDFPS (as you may have guessed) are the captured frames per second. The script first generates the SSH connection (using a control socket) that forwards the needed ports and then launches the in-built scrcpy binary. I found out the on_exit hook and polling for the X connection message was needed to make sure scrcpy does not hang when I close the window in SXMO (ugly but it works for now).

Result: Android on my Pinephone, yay. Big Grin

Video --> https://pauloliver.dev/files/sxmo-android-rc.mp4

I can invoke the script above using 2 different SSH connections, first a local connection on my home network, and also a remote connection (that goes through a tunnel set on a VPS I currently rent). The one that goes through the tunnel is set to a lower resolution and frame-rate, but it's completely usable, which means I can monitor/control the Android phone from anywhere. The scrcpy key-bindings even allow me to easily lock/unlock the Android phone and turn off the screen when not in use (way safer to leave the phone locked at home).

I'd like to come up with more hacky ways to control the Android apps programatically in the future (IDK if that's possible) but this will work for now. I can finally set/unset my home alarm from my Pinephone. Tongue
  Reply
#2
(11-28-2021, 09:15 AM)pauloliver Wrote: Hello everyone, first time poster (long time lurker). Smile

I wanted to share a experiment I've been working on. Maybe someone can find useful, or help me improve on the ideas described below. I purchased my Pinephone BE some while ago and I've been slowly working on it with hopes that I'll be able to turn it into my main phone. I'm using Archlinux with SXMO (flashed to eMMC) and I'm really happy with the level of customization and tinkering that's possible (it feels I'm still just scratching the surface). SXMO feels light and snappy, gestures are easy to remember (once you get accustomed). Calls, SMSs, mobile data (and even the mobile hotspot functionality) work excellently. Battery drains fast while in use, but lasts about 24+ hours while in suspend mode (I think I can live with that).

Detaching from Android has been difficult due to some services (like my home alarm system) only being controllable through an Android or IOS app. This has been a huge bummer, as it has kept me needing to carry my Android phone with me whenever I leave home (to arm/unarm and check on the system). If only there was a way to control my Android phone remotely, I'd be able to leave it at home most (if not all) of the time.

I've thus been looking into ways to achieve this. Ideally, I wanted to be able to control the droid phone through SSH. I've used scrcpy before to control my phone with my PC by plugging it to an USB port. Once I realized scrcpy could be SSH-forwarded I knew I was on to something!

The scrcpy utility is not available as an ARM package on the Archlinux repositories, but I was able to build it on the Pinephone easily by following instructions here:
https://github.com/Genymobile/scrcpy/blo...r/BUILD.md
Most dependencies were already installed on Arch by default (I think only exceptions where wget and meson). I think the android utilities (android-tools package on Arch) where also needed on the Pinephone as well.

I have an old laptop running 24/7 as a home server, so I installed the android-tools package there as well. This allows launching an ADB server on the background that can run continuously. I created a simple systemd service to achieve this and started/enabled it on the home-server:

Code:
[Unit]
Description=ADB server service
Requires=network.target
After=network-online.target

[Service]
User=paul
ExecStart=/usr/bin/adb -a nodaemon server
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

I already had debugging enabled on my Android phone, which means I can just plug it into the home-server and it gets detected by the adb daemon, nice!

As a last step I needed to be able to dynamically create a SSH connection on the Pinephone that forwards the ports used by scrcpy to connect to the adb server (5037 and 27183). SXMO allows creating userscripts that then become easily accessible through the UI. I created a simple shell script that creates the SSH connection and launches scrcpy:

Code:
#!/bin/sh

SOCKET=/tmp/droid-tunnel-ctrl-socket
SCRLOG=/tmp/droid-tunnel-scrcpy-log

# Parameters
SSHCON=$1
VIDSIZE=$2
VIDFPS=$3

notify-send "DROID TUNNEL" "Launching tunnel through '$SSHCON', size:$VIDSIZE, fps:$VIDFPS"

on_exit() {
        kill -9 $SCRPID
        ssh -S $SOCKET -O exit $SSHCON
}

trap on_exit EXIT

ssh -M -S $SOCKET -fnNT -L5037:localhost:5037 -L27183:localhost:27183 $SSHCON
ssh -S $SOCKET -O check $SSHCON

killall -v scrcpy
scrcpy -V verbose --force-adb-forward --max-fps $VIDFPS --max-size $VIDSIZE > $SCRLOG 2>&1 &
SCRPID=$!

while sleep 0.1 ; do
        grep --quiet "X connection to :0 broken" $SCRLOG && break
done

On the shell script above, the first argument (SSHCON) represents the SSH connection to the home-server. This is pre-configured on my ~/.config/ssh. VIDSIZE is the resolution of the generated video that's captured live from the Android phone (long side). VIDFPS (as you may have guessed) are the captured frames per second. The script first generates the SSH connection (using a control socket) that forwards the needed ports and then launches the in-built scrcpy binary. I found out the on_exit hook and polling for the X connection message was needed to make sure scrcpy does not hang when I close the window in SXMO (ugly but it works for now).

Result: Android on my Pinephone, yay. Big Grin

Video --> https://pauloliver.dev/files/sxmo-android-rc.mp4

I can invoke the script above using 2 different SSH connections, first a local connection on my home network, and also a remote connection (that goes through a tunnel set on a VPS I currently rent). The one that goes through the tunnel is set to a lower resolution and frame-rate, but it's completely usable, which means I can monitor/control the Android phone from anywhere. The scrcpy key-bindings even allow me to easily lock/unlock the Android phone and turn off the screen when not in use (way safer to leave the phone locked at home).

I'd like to come up with more hacky ways to control the Android apps programatically in the future (IDK if that's possible) but this will work for now. I can finally set/unset my home alarm from my Pinephone. 
Hi, great work with scrcpy, I think, that adb shell input tap x y command is way to go for you. You can programm it to do anything on your android phone for youSmile. Link to simple vid how it can be done:
https://youtu.be/Du__JfXqsAs
  Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  control power management aular 0 835 12-16-2023, 12:16 PM
Last Post: aular
  Android bank apps ionmich 4 3,066 12-12-2023, 01:01 PM
Last Post: mannzrespouse
  Headphone media control rmblror 1 1,508 03-27-2022, 07:20 AM
Last Post: wibble
  Android apps on Pinephones Raniyah 1 4,702 11-23-2020, 10:39 PM
Last Post: evilbunny
  Trigger happy proximity sensor, broken volume control & other newbie questions grug 3 4,977 09-10-2020, 05:57 AM
Last Post: megous
  Android on PinePhone? vanja 21 42,122 07-09-2020, 12:29 PM
Last Post: rsglobal
  A64 Android 7.0 on Pine phone tgv34 3 5,234 03-31-2020, 02:05 AM
Last Post: tgv34
  First boot boots to "A64 android" p1trson 3 3,852 02-17-2020, 11:04 AM
Last Post: edge
  Android brent.thierens 1 3,024 12-28-2019, 03:19 PM
Last Post: dukla2000

Forum Jump:


Users browsing this thread: 1 Guest(s)