puppet: Fix standalone certbot configurations.

This addresses the problems mentioned in the previous commit, but for
existing installations which have `authenticator = standalone` in
their configurations.

This reconfigures all hostnames in certbot to use the webroot
authenticator, and attempts to force-renew their certificates.
Force-renewal is necessary because certbot contains no way to merely
update the configuration.  Let's Encrypt allows for multiple extra
renewals per week, so this is a reasonable cost.

Because the certbot configuration is `configobj`, and not
`configparser`, we have no way to easily parse to determine if webroot
is in use; additionally, `certbot certificates` does not provide this
information.  We use `grep`, on the assumption that this will catch
nearly all cases.

It is possible that this will find `authenticator = standalone`
certificates which are managed by Certbot, but not Zulip certificates.
These certificates would also fail to renew while Zulip is running, so
switching them to use the Zulip webroot would still be an improvement.

Fixes #20593.

puppet/zulip/manifests/profile/app_frontend.pp

@@ -60,6 +60,10 @@ class zulip::profile::app_frontend {
     source  => 'puppet:///modules/zulip/letsencrypt/nginx-deploy-hook.sh',
     require => Package[certbot],
   }
+  exec { 'fix-standalone-certbot':
+    onlyif  => 'test -d /etc/letsencrypt/renewal && grep -qx "authenticator = standalone" /etc/letsencrypt/renewal/*.conf',
+    command => "${::zulip_scripts_path}/lib/fix-standalone-certbot",
+  }
 
   # Restart the server regularly to avoid potential memory leak problems.
   file { '/etc/cron.d/restart-zulip':

scripts/lib/fix-standalone-certbot

@@ -0,0 +1,14 @@
+#!/usr/bin/env bash
+
+set -eu
+set -o pipefail
+
+hostnames=$(grep -l 'authenticator = standalone' /etc/letsencrypt/renewal/*.conf | sed 's/.*\///; s/\.conf$//')
+
+for hostname in $hostnames; do
+    # Force a cert renewal to force the config file to update
+    /usr/bin/certbot certonly --webroot --webroot-path=/var/lib/zulip/certbot-webroot/ --force-renewal -d "$hostname"
+done
+
+# Pick up any updated certs
+service nginx reload