wireguard PPA no longer valid

[wpietri]

Today I added somebody to the VPN, as I had a number of times before. To my surprise, the VPN stopped working for everybody. After a fair bit of digging, I discovered this in the logs:

Aug  3 18:03:41 ip-172-31-13-114 systemd[1]: Starting Execute cloud user/final scripts...
Aug  3 18:03:41 ip-172-31-13-114 cloud-init[1806]: #!/bin/bash -v
Aug  3 18:03:41 ip-172-31-13-114 cloud-init[1806]: add-apt-repository "ppa:wireguard/wireguard"
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]:  #033[1m
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: ==========================================================================
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: DEPRECATION NOTICE: These packages have now moved into Ubuntu 20.04,
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: 19.10, 18.04, and 16.04. Therefore this PPA only has packages for Ubuntu
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: 14.04. However, we are planning to sunset this PPA and 14.04 support with
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: it. For that reason, if you are in fact using this PPA with 14.04, and
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: would like us to maybe reconsider the deprecation of this PPA, please
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: email us at team {at} wireguard.com to explain why it's inconvenient for
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: you to upgrade to a newer Ubuntu release and the size of your deployment.
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: If you're reading this because you're following an online tutorial on
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: installing WireGuard, please contact the authors to request that they
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: simplify their instructions to simply:
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]:     $ sudo apt install wireguard
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: This PPA is no longer required for WireGuard. Press CTRL+C, and do not
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: proceed with adding it.
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: More information can be found at:
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: - https://lists.zx2c4.com/pipermail/wireguard/2020-August/005737.html
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: - https://lists.zx2c4.com/pipermail/wireguard/2020-July/005670.html
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: ==========================================================================
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: #033[0m
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: WireGuard is a novel VPN that runs inside the Linux Kernel. This is the
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: Ubuntu packaging for WireGuard. More info may be found at its website,
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: listed below.
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: More info: https://www.wireguard.com/
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: Packages: wireguard wireguard-tools wireguard-dkms
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]: Install with: $ sudo apt install wireguard
Aug  3 18:03:42 ip-172-31-13-114 cloud-init[1806]:  More info: https://launchpad.net/~wireguard/+archive/ubuntu/wireguard
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: keyring `/tmp/tmpl7t0j4rn/secring.gpg' created
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: keyring `/tmp/tmpl7t0j4rn/pubring.gpg' created
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: requesting key 504A1A25 from hkp server keyserver.ubuntu.com
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: /tmp/tmpl7t0j4rn/trustdb.gpg: trustdb created
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: key 504A1A25: public key "Launchpad PPA for wireguard-ppa" imported
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg: Total number processed: 1
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: gpg:               imported: 1  (RSA: 1)
Aug  3 18:03:43 ip-172-31-13-114 cloud-init[1806]: OK

Hand-installing wireguard from the official repo and then hand-running the rest of the userdata script seems to have fixed things. I'm not going to mess with our VPN right now while people are using it, but perhaps it's sufficient to drop the ppa line in user-data.txt?

[wpietri]

Reading the included links, it looks like user-data.txt can just install the wireguard package, ignoring the others:

Users should never need to manually install wireguard-tools or
wireguard-dkms. Rather, *only* install the "wireguard" package, and this
will automatically choose the correct additional packages to pull in.

[zx2c4]

I'm interested to learn why this broke. The user-data.txt file says:

add-apt-repository "ppa:wireguard/wireguard"
apt-get update -y
apt-get upgrade -y -o Dpkg::Options::="--force-confnew"
apt-get install -y wireguard-dkms wireguard-tools awscli

With ppa:wireguard/wireguard being empty, it seems like that will essentially be a no-op, and wireguard-dkms and wireguard-tools will be installed from the Ubuntu repos, which is fine. Can you post more of your log to see where the error actually happens? So far that all looks fine.

[wpietri]

That's a good question. Now that you mention it, it seems like it should have just sailed on through. It looks like it's getting hung up on Grub. Here's a fuller chunk of logs:

logs.txt

I'm not sure if the PPA change is coincidence or causal. @jmhale, do you have a theory?

[wpietri]

I had the chance to set up a separate environment, so I could try this from scratch without stopping my colleagues from working. Same deal, getting hung up on Grub:

Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: Setting up grub2-common (2.02~beta2-36ubuntu3.27) ...
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: Setting up grub-pc-bin (2.02~beta2-36ubuntu3.27) ...
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: Setting up grub-pc (2.02~beta2-36ubuntu3.27) ...
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: debconf: unable to initialize frontend: Dialog
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: debconf: (TERM is not set, so the dialog frontend is not usable.)
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: debconf: falling back to frontend: Readline
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: Configuring grub-pc
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: -------------------
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: #015
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: A new version (/tmp/grub.998Q9i9Yvo) of configuration file /etc/default/grub is
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: available, but the version installed currently has been locally modified.
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: #015
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   1. install the package maintainer's version
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   2. keep the local version currently installed
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   3. show the differences between the versions
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   4. show a side-by-side difference between the versions
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   5. show a 3-way difference between available versions
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   6. do a 3-way merge between available versions (experimental)
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]:   7. start a new shell to examine the situation
Aug  6 17:26:18 ip-172-31-1-146 cloud-init[1814]: #015

I'll dig further and see if I can figure out what the issue is.

[wpietri]

I think the PPA change was a red herring, @zx2c4. Sorry to bother you with this.

I have a pull request that fixes the issue for me: #19. I also removed the PPA from that, as it's no longer necessary.

[jmhale]

Looks like this is working. Thanks!