patchpilot

PatchPilot · Guides

How to check that unattended-upgrades is actually working

It fails quietly. A timer that stopped, an origin that does not match, a package held back — and the only sign is a log file nobody opens. Five checks, in the order that finds the problem fastest, for Debian and Ubuntu.

1. Is it switched on?

apt-config dump APT::Periodic

You want to see both of these set to "1":

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

They normally live in /etc/apt/apt.conf.d/20auto-upgrades. If either is missing or "0", the package is installed and does nothing. sudo dpkg-reconfigure -plow unattended-upgrades writes the file for you.

2. Are the timers running?

systemctl list-timers 'apt-daily*'

Two timers do the work: apt-daily.timer refreshes the package lists, apt-daily-upgrade.timer installs. Both should show a time under NEXT and a recent one under LAST. A timer that was masked or disabled during some old debugging session is the most common reason a correctly configured machine never upgrades.

3. What did the last run actually do?

sudo tail -n 40 /var/log/unattended-upgrades/unattended-upgrades.log

The lines that matter:

  • Packages that will be upgraded: … — what it attempted. All upgrades installed follows if it succeeded.
  • No packages found that can be upgraded unattended — nothing to do from the origins it is allowed to use (check 4).
  • Packages that are kept back: … — updates it found and did not install: a package on hold or on its blacklist, or one it could not upgrade without touching packages it is not allowed to.

The dpkg output of each run sits next to it in unattended-upgrades-dpkg.log, which is where an install that failed halfway explains itself.

4. What would it do right now?

sudo unattended-upgrade --dry-run --debug

Near the top it prints Allowed origins are: …. Only updates from those origins are ever installed, and everything else stays pending for good. By default that is the security pocket on Ubuntu, and security plus point releases on Debian — which is why "unattended-upgrades is running" and "this server is up to date" are different statements. The list is set in /etc/apt/apt.conf.d/50unattended-upgrades.

5. Make it tell you

By default it tells nobody. In /etc/apt/apt.conf.d/50unattended-upgrades, set:

Unattended-Upgrade::Mail "[email protected]";
Unattended-Upgrade::MailReport "only-on-error";

MailReport takes "always", "only-on-error" or "on-change". Mail goes out through a program that provides mailx (bsd-mailx, for instance) and a working mail transfer agent behind it. When no report ever arrives, that is almost always the reason — test the path on its own first:

echo "test from $(hostname)" | mail -s "mail test" [email protected]

What a mail per server cannot tell you

Each server reports on itself, and only when it runs. A machine whose timer stopped sends nothing, and from your inbox nothing looks exactly like everything being fine. Nor does any of this notice the day a server's release stops receiving updates at all: the updater keeps running, keeps succeeding, and there is simply nothing left for it to install.

Where PatchPilot comes in

Keep unattended-upgrades. It installs security updates on the machine it runs on, and it is good at that. PatchPilot is the view across all of them: one table with what is installed on each server, what is pending and which of it is a security fix, which machine is waiting for a reboot, and which release has run out of support — flagged months before it does.

A mail, or a message in Slack, Discord or Teams, when something needs you, not a mail from every machine every night. More on the PatchPilot home page.

Which of your servers did it skip last night?

One command on the first one, and you will know. It is free during the beta.

Free during the beta, no card.