PINE64
apt-get update hanging twice (work-around included) - Printable Version

+- PINE64 (https://forum.pine64.org)
+-- Forum: PinePhone (https://forum.pine64.org/forumdisplay.php?fid=120)
+--- Forum: PinePhone Software (https://forum.pine64.org/forumdisplay.php?fid=121)
+---- Forum: Mobian on PinePhone (https://forum.pine64.org/forumdisplay.php?fid=139)
+---- Thread: apt-get update hanging twice (work-around included) (/showthread.php?tid=10965)

Pages: 1 2

apt-get update hanging twice (work-around included) - motox6 - 08-05-2020

Sorry, I accidentally deleted the content of my prior post, because I did not understand how the reply editor worked.
So I deleted the entire post as what was left was not helpful.  (Was there a way to go back to prior versions?)

So here is synopsis without all the nice details I had in the prior post. Sad

I was doing a fresh install of Mobian to the EMMC.
I did an apt update and apt upgrade

and it got stuck at 7% for 20 hours.  Reading elsewhere I found people running into a similar issue and identify a system control restart command related to fstrim as the problem.  I killed the systemctl restart process related to fstrim

The process continued a little bit and exited.

I did another

apt upgrade

It got to 88% and got stuck again.

I killed another systemctl restart process this time related to something else.

This time the apt upgrade continued to completion.

I re-ran
apt upgrade
and
dpkg --configure -a

just to make sure everything was okay, and everything was okay.

I think in both hang cases there was also a process related to something about dialog.

Perhaps the upgrade was asking something that did not appear on the screen?

I ran the entire process once on the phone via Kings Cross and once via SSH using a screen session.

RE: apt-get update hanging twice (work-around included) - livanti21 - 08-06-2020

(08-05-2020, 01:43 PM)motox6 Wrote: Sorry, I accidentally deleted the content of my prior post, because I did not understand how the reply editor worked.
So I deleted the entire post as what was left was not helpful.  (Was there a way to go back to prior versions?)

So here is synopsis without all the nice details I had in the prior post. Sad

I was doing a fresh install of Mobian to the EMMC.
I did an apt update and apt upgrade

and it got stuck at 7% for 20 hours.  Reading elsewhere I found people running into a similar issue and identify a system control restart command related to fstrim as the problem.  I killed the systemctl restart process related to fstrim

The process continued a little bit and exited.

I did another

apt upgrade

It got to 88% and got stuck again.

I killed another systemctl restart process this time related to something else.

This time the apt upgrade continued to completion.

I re-ran
apt upgrade
and
dpkg --configure -a

just to make sure everything was okay, and everything was okay.

I think in both hang cases there was also a process related to something about dialog.

Perhaps the upgrade was asking something that did not appear on the screen?

I ran the entire process once on the phone via Kings Cross and once via SSH using a screen session.
So I tried to update and upgrade this morning and it got stuck and I rebooted the phone. I then had to run dpkg --configure -a because it was interrupted but running it it gets stuck at 'Setting up apt (2.1.8) and will do nothing else.

Do you have any ideas that could possibly help me?

RE: apt-get update hanging twice (work-around included) - motox6 - 08-06-2020

Today my apt upgrade is again hung

Code:
Unpacking apt (2.1.8) over (2.1.7) ...
Setting up apt (2.1.8) ...

Progress: [ 19%] [############....................................................]

And there is the suspicious stuck process

Code:
  24528 pts/2    Ss+    0:00 /usr/bin/dpkg --status-fd 20 --no-triggers --configure ap
  24529 pts/2    S+     0:00 /bin/sh /var/lib/dpkg/info/apt.postinst configure 2.1.7
  24565 pts/2    S+     0:00 /bin/systemctl restart apt-daily-upgrade.timer apt-daily.
  24573 pts/2    S+     0:00 /bin/systemd-tty-ask-password-agent --watch

Interesting.  It looks like dpkg and apt-daily-upgrade.timer are locking each other out.

kill 24565

allowed the upgrade process to continue and complete.
>So I tried to update and upgrade this morning and it got stuck and I rebooted the phone. I then had to run dpkg --configure -a because it was >interrupted but running it it gets stuck at 'Setting up apt (2.1.8) and will do nothing else.

>Do you have any ideas that could possibly help me?

Check my reply in this thread, I ran into exactly the same issue as you.

Find this process

/bin/systemctl restart apt-daily-upgrade.timer apt-daily

from the 'ps ax' list
and kill it.

RE: apt-get update hanging twice (work-around included) - User 18618 - 08-07-2020

Interesting - I was wondering which process locked dpkg.

I've encountered the 'stuck while setting up' problem twice, and the easy way to fix it is:

Code:
sudo reboot

sudo dpkg --configure -a
sudo apt --fix-broken install

# Continue upgrading, if required
sudo apt upgrade


RE: apt-get update hanging twice (work-around included) - spaetz - 08-07-2020

I guess, we could be seeing:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940840

We are using chrony as time sync daemon instead of the systemd ntp one. And the Debian bug then experiences similar hangs with the sstemd-time-wait-sync service running (which does not work with chronyd):

>  systemd-time-wait-sync.service      start running
Hm, this service is not enabled by default and I'm guessing it prevents
time-sync.target to be reached, blocking all subsequent services.


The man page says:
Code:
>        systemd-time-wait-sync is a system service that delays the start of units that depend on time-sync.target
>        until the system time has been synchronized with an accurate time source by systemd-timesyncd.service.

So, any timer that waits until the time is synchronized is on hold. And the apt-daily-upgrade.timer is one of those, I guess.

UPDATE: I don't experience the hangs myself. Could somebody who does check with

```systemctl status systemd-time-wait-sync.service```

to see if this service is somehow enabled on their mobian device?

RE: apt-get update hanging twice (work-around included) - selrahal - 08-07-2020

I experienced this hang and did some digging, found pretty much what you are saying. The time-sync target is not being reached, any timer depending on this cannot be started. My active jobs list shows this (since boot):


Code:
root@mobian:/home/mobian# systemctl list-jobs

JOB UNIT                           TYPE  STATE 
65  timers.target                  start waiting
70  apt-daily-upgrade.timer        start waiting
68  apt-daily.timer                start waiting
72  e2scrub_all.timer              start waiting
67  man-db.timer                   start waiting
69  fwupd-refresh.timer            start waiting
71  fstrim.timer                   start waiting
25  time-sync.target               start waiting
24  systemd-time-wait-sync.service start running





Curious enough, timedatectl shows everything is ok:



Code:
root@mobian:/home/mobian# timedatectl status
              Local time: Fri 2020-08-07 12:47:27 EDT 
          Universal time: Fri 2020-08-07 16:47:27 UTC 
                RTC time: Fri 2020-08-07 16:47:28     
                Time zone: America/New_York (EDT, -0400)
System clock synchronized: yes                         
              NTP service: active                     
          RTC in local TZ: no



systemd-time-wait-sync start the command '/lib/systemd/systemd-time-wait-sync' and waits for it to exit. If i start that manually it, correctly, exits immediately. But for some reason the process started earlier isn't getting updated. By restart the systemd-time-wait-sync service everything works without a reboot: 'systemctl restart systemd-time-wait-sync'



I think this bug is the culprit: https://github.com/systemd/systemd/issues/15947



I am running Mobian which has switched to chrony, not sure if that is related.

RE: apt-get update hanging twice (work-around included) - DarkManiels - 08-07-2020

(08-07-2020, 09:01 AM)spaetz Wrote: I guess, we could be seeing:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=940840

We are using chrony as time sync daemon instead of the systemd ntp one. And the Debian bug then experiences similar hangs with the sstemd-time-wait-sync service running (which does not work with chronyd):

>  systemd-time-wait-sync.service      start running
Hm, this service is not enabled by default and I'm guessing it prevents
time-sync.target to be reached, blocking all subsequent services.


The man page says:
Code:
>        systemd-time-wait-sync is a system service that delays the start of units that depend on time-sync.target
>        until the system time has been synchronized with an accurate time source by systemd-timesyncd.service.

So, any timer that waits until the time is synchronized is on hold. And the apt-daily-upgrade.timer is one of those, I guess.

UPDATE: I don't experience the hangs myself. Could somebody who does check with

```systemctl status systemd-time-wait-sync.service```

to see if this service is somehow enabled on their mobian device?
I was having this problem as well. Ran the code you suggested and got the following output:

● systemd-time-wait-sync.service - Wait Until Kernel Time Synchronized
    Loaded: loaded (/lib/systemd/system/systemd-time-wait-sync.service; enabled; vendor preset: disabled)
    Active: activating (start) since Fri 2020-08-07 08:12:27 MST; 2h 0min ago
      Docs: manConfusedystemd-time-wait-sync.service(8)
  Main PID: 1488 (systemd-time-wa)
      Tasks: 1 (limit: 1990)
    Memory: 1.0M
    CGroup: /system.slice/systemd-time-wait-sync.service
            └─1488 /lib/systemd/systemd-time-wait-sync

Aug 07 08:12:28 mobian systemd-time-wait-sync[1488]: adjtime state 5 status 40 time Fri 2020-08-07 15:12:27.709948 UTC
Warning: journal has been rotated since unit was started, output may be incomplete.

Looks like it is enabled on my device.

RE: apt-get update hanging twice (work-around included) - spaetz - 08-07-2020

OK, I think we are getting there. THe bug that I found is the Debian bug. essentially describing the same thing. So try to
systemctl disable systemd-time-wait-sync as it conflicts with chrony anyway according to the man page. And report if that helps Smile

RE: apt-get update hanging twice (work-around included) - DarkManiels - 08-07-2020

(08-07-2020, 12:43 PM)spaetz Wrote: OK, I think we are getting there. THe bug that I found is the Debian bug. essentially describing the same thing. So try to
systemctl disable systemd-time-wait-sync as it conflicts with chrony anyway according to the man page. And report if that helps Smile

Good to hear. But I haven't had the problem again since I finally got apt updated to 2.1.8 (that was the package it kept hanging on). I'll try it if the problem returns, but in the meantime maybe someone who's still having the problem currently can try.

RE: apt-get update hanging twice (work-around included) - spaetz - 08-07-2020

Upstream bug is https://github.com/systemd/systemd/issues/14061 I guess.