Problem and solutions

I got a simple task to restart reverse proxy process after certificate was updated. It sounds simple, but there is a complication in my case, certificates are generated on different server and get presented to system where reverse proxy runs.

There are two approaches to it:

  1. Use inotifywait or similar tool to run restart when file is updated. Proxy process could be restarted almost instantly after certs are upgraded. But you need to install inotifywait separately and then create shell script plus systemd service to run it. I do not like this approach.
  2. Periodically check if certificate file is updated and restart reverse proxy when it happen. It may take long time to detect changes, but it can be done using standard utilities and single systemd timer. Usually there is no need to apply new certificate immediately, it can way for next re-scan. This also allow you to schedule restart for time when it going to be less impacting, like middle of the night. This guide based on this approach.

Setup

  1. ha-proxy started by haproxy.service
  2. Site certificate stored in /etc/haproxy/mysite.pem
  3. ha-proxy PID file in */var/run/haproxy.pid
  4. system is Debian

Steps

  1. Create .timer file ./etc/systemd/system/check-cert.timer I’m not sure that /etc/systemd/system is right place for that file 
     [Unit]
 Description=Check if certificate was updated and restart haproxy if it was

 [Timer]
 OnCalendar=*-*-* 3:00:00
 Persistent=false

 [Install]
 WantedBy=timers.target

    Timer will be triggered once a day at 3:00am. Adjust OnCalendar to match your needs

  2. Create .service file./etc/systemd/system/check-cert.service Name of service should match name of timer. (you can overwrite it if you wish)

     [Unit]
 Description=Check if certificate was updated and restart haproxy if it was

 [Service]
 Type=oneshot
 ExecStart=/usr/bin/find /etc/haproxy/mysite.pem -cnewer /var/run/haproxy.pid -print -exec /bin/systemctl restart haproxy \;

    How does it work:

    find will search for all files in /etc/haproxy/mysite.pem ( which means just that file), check if it created after creation of /var/run/haproxy.pid and if it does execute command after -exec restarting haproxy. /var/run/haproxy.pid is created every time haproxy process restarted. So if haproxy was started before /etc/haproxy/mysite.pem was created, then /etc/haproxy/mysite.pem will match, and find will restart haproxy. if /etc/haproxy/mysite.pem is older than /var/run/haproxy.pid nothing will happen.

  3. Signal systemd to reload config files 
     sudo systemctl daemon-reload
  4. Run your service manually to ensure it works 
     # touch /etc/haproxy/mysite.pem
 # systemctl start check-cert.service
 # systemctl status check-cert.service
 ● check-cert.service - Check if certificate was updated and restart haproxy if it was
    Loaded: loaded (/etc/systemd/system/check-cert.service; static; vendor preset: enabled)
    Active: inactive (dead) since Sat 2021-12-04 22:46:55 AEDT; 31s ago
   Process: 23145 ExecStart=/etc/haproxy/mysite.pem -cnewer /var/run/haproxy.pid -print -exec /bin/systemctl restart haproxy ; (code=exited, status=0/SUCCESS)
  Main PID: 23145 (code=exited, status=0/SUCCESS)

 Dec 04 22:46:54 jellyfin systemd[1]: Starting Check if certificate was updated and restart haproxy if it was...
 Dec 04 22:46:54 jellyfin find[23145]: /etc/haproxy/mysite.pem
 Dec 04 22:46:55 jellyfin systemd[1]: Check if certificate was updated and restart haproxy if it was.

    and then check that haproxy been restarted.

  5. If haproxy got restarted as expected, enable and start timer 
     # systemctl enable check-cert.timer
 Created symlink /etc/systemd/system/timers.target.wants/check-cert.timer → /etc/systemd/system/check-cert.timer.
 # systemctl start check-cert.timer
  6. Check that new timer is running 
     # systemctl list-timers --all
 NEXT                          LEFT          LAST                          PASSED    UNIT                         ACTIVATES
 Sun 2021-12-05 03:00:00 AEDT  4h 21min left n/a                           n/a       check-cert.timer             check-cert.service

It is all, folks. You are done.

Updated: