Installing Let’s Encrypt on standalone servers with cPanel, or plain Linux servers without any control panel is an easy task that can be done within minutes.
However, days ago our tech team faced a new interesting challenge. One of our customers required to install let’s encrypt on a website that is running across many Nginx servers using round robin dns (rrDNS) balancing.
Just in case, I will assume you already have Nginx and Let’s Encrypt working on at least one single Nginx server, if you haven’t, I will recommend you to read this post: Install Let’s Encrypt on Nginx + CentOS
Let’s move to the fun part. This is the best way we’ve found to have Let’s Encrypt SSL running over mulitple servers.
Update, September 13, 2017: after publishing this and running several tests, I’ve found this not to be the best solution, as ssl renewal may not work as expected due to the rrDNS setup and the Let’s Encrypt verification process. Let’s Encrypt over rrDNS + Nginx will only work perfectly if you are using a shared storage service across all your clustered / balanaced servers (like NFS for example on rrDNS setups) for your .well-known/ directory, otherwise it may fail to verify the .well-known/acme-challenge/RANDOM URL file if it hits another server who doesn’t contain that active URL structure, more information can be found on Let’s Encrypt forums.
Configure Let’s Encrypt on Multiple Servers
Add the .well-known location configuration to your Nginx vhost config file, as you se below:
server {
access_log off;
error_log logs/nixcp.com-error_log warn;
listen 80;
server_name nixcp.com www.nixcp.com;
root /var/www/nixcp.com;
index index.html index.php;
location /.well-known {
root /var/www/nixcp.com;
}Here is the full Nginx configuration for a Let’s Encrypt based website:
### nixcp.com
server {
access_log off;
error_log logs/nixcp.com-error_log warn;
listen 80;
server_name nixcp.com www.nixcp.com;
root /var/www/nixcp.com;
index index.html index.php;
# SSL configuration
listen 443 ssl;
ssl on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
ssl_prefer_server_ciphers On;
ssl_dhparam /etc/nginx/conf/dhparams.pem;
ssl_certificate /etc/letsencrypt/live/nixcp.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/nixcp.com/privkey.pem;
ssl_session_cache shared:SSL:20m;
ssl_session_timeout 10m;
# End of SSL Configuration
location /.well-known {
root /var/www/nixcp.com;
}
# Parseo de php con php-fpm
location ~ \.(php|shtml)$ {
try_files $uri =404;
fastcgi_pass unix:/tmp/php5-fpm.sock;
fastcgi_index index.shtml;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
include fastcgi_params;
fastcgi_buffer_size 128k;
fastcgi_read_timeout 150;
fastcgi_buffers 256 4k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
}
}Let’s Encrypt SSL replication across mulitple servers
Lsyncd is a great tool to synchronize file and directories over Linux servers. In this case I’m using lsyncd to sync the Nginx vhost configuration files located at /etc/nginx/conf.d/*, and also the /etc/letsencrypt directories, which contains the .pem, .key files and the rest of the configuration files required to have Let’s Encrypt SSL working.
Setting up lsyncd
yum install lsyncd -y
or using apt-get for Debian/Ubuntu distros:
apt-get install lsyncd
Generate log directory and files:
mkdir -p /var/log/lsyncd -p touch /var/log/lsyncd/lsyncd.log touch /var/log/lsyncd/lsyncd-status.log
Edit /etc/lsyncd.conf file
nano -w /etc/lsyncd.conf
Then try to setup your lsyncd.conf configuration as you see below.
Make sure you replace:
- 192.168.1.105 with your ssh remote servers IP, this is an example for only one remote servers, but you can add as many servers as you like.
- /etc/nginx/conf.d/ with your real nginx vhost configuration directory.
- -p 22 with the real SSH port you are using on the remote servers.
- nixcp.com with the real domin name you are using.
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
statusInterval = 2
}
sync {
default.rsync,
source="/etc/nginx/conf.d/",
target="192.168.1.105:/etc/nginx/conf.d/",
rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}
sync {
default.rsync,
source="/etc/letsencrypt/",
target="192.168.1.105:/etc/letsencrypt/",
rsync = {
compress = true,
acls = true,
verbose = true,
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
}
}
After that, restart lsyncd:
service lsyncd restart
or
systemctl restart lsyncd.service
Let’s Encrypt SSL renewal
The renewal let’s encrypt script must be run only on the “master” server, that is the one that syncs the let’s encrypt ssl certificate to the “slave” servers.
Important: this will only work if you are using a shared storage service across all your clustered / balanaced servers (like NFS for example on rrDNS setups) for your .well-known/ directory, otherwise it may fail to verify the .well-known/acme-challenge/RANDOM URL file if it hits another server who doesn’t contain that active URL structure, more information can be found on Let’s Encrypt forums.
Setup a weekly cron to make sure your ssls are always renewed:
crontab -e
The paste this at the end of the file:
01 1 * * 1 /usr/bin/certbot renew >> /var/log/le-ssl-renew.log 06 1 * * 1 /sbin/service nginx reload
That’s all, now you know how to setup let’s encrypt ssl over multiple Nginx servers.
Let me know if you face any issues during the setup process.