Configuring Ubuntu to Patch Itself

After a couple of years in the IT infrastructure world, you’ll be faced with the greatest task of all, updating servers every month or so. This has always been the most brain dead task imaginable, but now with virtualization, we also have 10 times more servers to update. And almost every single week there’s another critical bug in some core service that has to be patched in a panic.

So, instead of doing all that, why don’t we just do what we always do when we have a task that humans don’t like to do? We get a computer to do it for us!

Ubuntu systems almost come with this feature built in, and is quite capable of handling it’s own patching needs.

Step 1 – Installing Unattended-Upgrades

sudo su
apt-get install unattended-upgrades

This step installs all the packages needed for automatic upgrades.

Step 2 – Configuring Unattended-Upgrades

Next thing we’ll need to do is configure how unattended-upgrades work. This is by editing the configuration file

nano /etc/apt/apt.conf.d/50unattended-upgrades

This opens up a quite massive configuration file. I’ll not go over everything that can be done here, but rather what I use. The important things are:

Uncomment the following lines by removing the “//” at the start of the line. This sets which patches will be applied automatically. Personally I like to have normal patches, security patches as well as the odd Extended Security Maintenance patch applied automatically.

"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
"${distro_id}ESM:${distro_codename}";
"${distro_id}:${distro_codename}-updates";

Uncomment this line and change the e-mail address to where you want a confirmation email to be sent:

Unattended-Upgrade::Mail "myemail@domain.com";

Below is a complete copy of my “standard” configuration file:

// Automatically upgrade packages from these (origin:archive) pairs
Unattended-Upgrade::Allowed-Origins {
"${distro_id}:${distro_codename}";
"${distro_id}:${distro_codename}-security";
// Extended Security Maintenance; doesn't necessarily exist for
// every release and this system may not have it installed, but if
// available, the policy for updates is such that unattended-upgrades
// should also install from here by default.
"${distro_id}ESM:${distro_codename}";
"${distro_id}:${distro_codename}-updates";
// "${distro_id}:${distro_codename}-proposed";
// "${distro_id}:${distro_codename}-backports";
};

// List of packages to not update (regexp are supported)
Unattended-Upgrade::Package-Blacklist {
// "vim";
// "libc6";
// "libc6-dev";
// "libc6-i686";
};

// This option allows you to control if on a unclean dpkg exit
// unattended-upgrades will automatically run
// dpkg --force-confold --configure -a
// The default is true, to ensure updates keep getting installed
//Unattended-Upgrade::AutoFixInterruptedDpkg "false";

// Split the upgrade into the smallest possible chunks so that
// they can be interrupted with SIGUSR1. This makes the upgrade
// a bit slower but it has the benefit that shutdown while a upgrade
// is running is possible (with a small delay)
//Unattended-Upgrade::MinimalSteps "true";

// Install all unattended-upgrades when the machine is shuting down
// instead of doing it in the background while the machine is running
// This will (obviously) make shutdown slower
//Unattended-Upgrade::InstallOnShutdown "true";

// Send email to this address for problems or packages upgrades
// If empty or unset then no email is sent, make sure that you
// have a working mail setup on your system. A package that provides
// 'mailx' must be installed. E.g. "user@example.com"
Unattended-Upgrade::Mail "grumpy@grumpytechie.net";

// Set this value to "true" to get emails only on errors. Default
// is to always send a mail if Unattended-Upgrade::Mail is set
//Unattended-Upgrade::MailOnlyOnError "true";

// Do automatic removal of new unused dependencies after the upgrade
// (equivalent to apt-get autoremove)
//Unattended-Upgrade::Remove-Unused-Dependencies "false";

// Automatically reboot *WITHOUT CONFIRMATION*
// if the file /var/run/reboot-required is found after the upgrade
Unattended-Upgrade::Automatic-Reboot "true";

// If automatic reboot is enabled and needed, reboot at the specific
// time instead of immediately
// Default: "now"
//Unattended-Upgrade::Automatic-Reboot-Time "02:00";

// Use apt bandwidth limit feature, this example limits the download
// speed to 70kb/sec
//Acquire::http::Dl-Limit "70";

Next we need to configure unattended-upgrades to run periodically:

dpkg-reconfigure -plow unattended-upgrades

This creates the file /etc/apt/apt.conf.d/20auto-upgrades which actually activates unattended-upgrades.

Step 3 – Enabling E-Mail Sending

Next we need to install something that enables the system to send e-mail. I prefer Postfix even though it’s over-kill for just sending e-mail from a script, but it works and is regularly updated.

apt-get install postfix

This installs Postfix, choose “Internet Site” from the settings below (we don’t need to receive e-mail, just to send them, so just SMTP is fine for our needs).

postfix

Next it asks you for a domain, it doesn’t really matter in this case, but insert your own domain here if you have one.

postfix_domain

Next we need to edit the main configuration file for Postfix

nano /etc/postfix/main.cf

The important things here are to remove everything behind mydestination = and adding your SMTP-server under relayhost =

The reason we’re removing everything under mydestination is to make sure Postfix sends everything upstream to the SMTP-server of our choice.

Here’s a complete copy of my main.cf file. (and yes, even the Ubuntu config files for Postfix reference Debian).

# See /usr/share/postfix/main.cf.dist for a commented, more complete version

# Debian specific: Specifying a file name will cause the first
# line of that file to be used as the name. The Debian default
# is /etc/mailname.
#myorigin = /etc/mailname

smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
biff = no

# appending .domain is the MUA's job.
append_dot_mydomain = no

# Uncomment the next line to generate "delayed mail" warnings
#delay_warning_time = 4h

readme_directory = no

# TLS parameters
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

# See /usr/share/doc/postfix/TLS_README.gz in the postfix-doc package for
# information on enabling SSL in the smtp client.

smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myhostname = time.grumpytechie.net
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = /etc/mailname
mydestination =
relayhost = smtp.domain.com:port
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mailbox_command = procmail -a "$EXTENSION"
mailbox_size_limit = 0
recipient_delimiter = +
inet_interfaces = all

Now that Postfix is configured, go ahead and reload Postfix

/etc/init.d/postfix reload

Step 4 – Test Your Work

Finally, you can do a so called “dry-run” of unattended-upgrades to check that it does what you expect it to do by issuing the following command:

unattended-upgrades --verbose --dry-run

This makes unattended-upgrades run through all it’s steps, without actually patching the system.

After this, you should start getting e-mails every now and then from you server that it’s happily patched. It won’t reboot itself with this configuration, so occasionally you’ll need to log in and reboot the server. If you want, you can set it up to reboot itself also, but I’ll leave that as an exercise for the reader.

Also, as a side note, all this also works on Ubuntu (for quite obvious reasons)!

SMTP Servers

Finding a SMTP server to use for this might be the most challenging part, especially if you just want to run a server or two in the Cloud somewhere.

Personally, I use SMTP2Go for my update scripts and what not, they allow you 1000 outgoing e-mails per month, for FREE, and they can do IP authentication, which is really handy for stuff like this where you can’t be bothered to configure authentication for Postfix. Do note that I don’t get any money from them for saying this, their service is free and I just like it, easy to set up and just works.

Advertisements


Categories: Tech

Tags: , , , , , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: