This post follows my path of understanding how much I don’t understand DNS and why it is the root of all evil.
But also it was the path to achieving something I was not able to just find a clean how-to for,
something which is said to be impossible without additional cost,
and something which is a dream come true for all the self-server-hosters, homelabbers et al.
It enables exploring a plethora of new endeavors without compromising on safety or convenience.

A wildcard certificate for your domain.

But one that renews automatically.

With a twist - using a method that works for any domain registrar.

Backstory, or stories from wild(card) west

So, I got my domain.
No big deal, especially since google opened up registrations for the new gTLDs - a 4 letter one would be pricy, but my humble one was not.

Like any other fresh domain and server with public IP owner, I opened the registrar’s control panel,
set the domain to point in the general direction of my server’s IP, then on the server’s side spun up nginx and certbot and poof, my website is served with HTTPS.

But it was not enough.
I needed wanted a wildcard certificate for my domain.
Why?
Because juggling DNS aliases for my domain is annoying (also, LE’s rate limits).
Because they’re all explicitly named in the certificate.
Because no one did it before (as far as I know).

“No one? But I’ve seen LE’s wildcard certs all over the place!” I hear you say.
Yes and no - it’s mostly easy… but only if your registrar provides you with an API, so that certbot can interact with it and ask for changes (why it’s required will be explained soon).
But my registrar, Google Domains (now bought out by Squarespace, but I don’t think this bit changed) does not provide an API, so the answer so far was “move to registrar X or Y who provides an API” - but most of them didn’t provide gTLDs or weren’t dealing with customers from my country or just were much more expensive.

Therefore, I needed to piece together my own solution, presented below.

Intermission

I worked out this method almost 2 years prior to publishing this post (in July 2022), but then sat on the materials until being forced to redo the server from scratch thus revisiting the whole saga.

Technical goal

Ok, I want a wildcard certificate from LetsEncrypt, because it’s cool and free and all that jazz.
But how do you get LE to send you one?

Well, as with the good ol’ non-wildcard certs, you need to pass a challenge.
(Oh, and also you basically have to have a domain just one level deeper than a Public Suffix, like yourdomain.com or you’re gonna have rate limit and other such problems).
For non-wildcard it’s the HTTP-01 challenge, in which case your http server (or proxy) has to respond with a given value to a request sent for /.well-known/acme-challenge/TOKEN.
This proves that you control the domain and everything hosted on it.

For wildcard you need to prove that you control the domain and all of its subdomains, which is more tricky.
LE’s servers ask DNS for a randomly generated (well, derived from some stuff - but might as well be) TXT record for _acme-challenge.DOMAIN, which has to contain a specified generated value.

Execution

Basic idea

The main concept is to simply host your own mini-DNS server responding to the challenge requests, but nothing else.

It also has to communicate with certbot in order to answer the prepared challenges with appropriate values.

Registrar DNS configuration

The easiest method would be to just register your server as your domain’s NS (nameserver) record.
But this would result in an infinite loop, so some registrars explicitly don’t even allow this.

The workaround is as follows:

• point your domain and all of its subdomains to your server (typical default config),
• register a subdomain as a nameserver for all subdomains of that subdomain,
• point that subdomain explicitly to your server (so it doesn’t do a recursive search),
• alias the _acme-challenge subdomain to within the prepared subdomain.

In my case, the record list is as follows:

Here I’ve got 3 A (address) records, pointing to my server’s IP;
a NS (nameserver) record, stating that acmesh is resolved by whatever’s on that subdomain;
a CNAME (canonical name) record, effectively aliasing _acme-challenge to something within acmesh. subdomain.

Micro DNS server

For the DNS server I’m using acme-dns.
Quoting it’s description from the github page, it’s a:

Limited DNS server with RESTful HTTP API to handle ACME DNS challenges easily and securely.

Preparation

acme-dns is not present in the Debian repository, so compilation from sources it is.
Note: you can also use Docker, but I wanted to limit the number of cross-dependencies in services running on my server.

sudo adduser --system --gecos "acme-dns Service" --disabled-password --group --home /var/lib/acme-dns acme-dns
sudo mkdir /opt/acme-dns
sudo chown acme-dns:acme-dns /opt/acme-dns
sudo -u acme-dns git clone https://github.com/joohoi/acme-dns /opt/acme-dns
cd /opt/acme-dns
export GOPATH=/tmp/acme-dns
sudo -u acme-dns go build
sudo cp acme-dns /usr/local/bin/

Configuration

Copy the config.cfg provided in acme-dns directory to /etc/acme-dns/config.cfg and adjust as necessary.

I’ve modified the API part like this:

[api]
ip = "127.0.0.1"
port = "53443"

Systemd service

sudo cp /opt/acme-dns/acme-dns.service /etc/systemd/system/acme-dns.service
sudo systemctl daemon-reload
sudo systemctl enable --now acme-dns.service

Firewall

I’m using ufw, so this is enough:

sudo ufw allow dns

Alternatively, for firewalld:

sudo firewall-cmd --zone=public --add-service=dns --permanent
sudo firewall-cmd --reload

Getting the certificate

The certificate will be retrieved by certbot, using acme-dns-client as the auth hook.

Download the auth hook (acme-dns API client) program

wget https://github.com/acme-dns/acme-dns-client/releases/download/v0.3/acme-dns-client_0.3_linux_amd64.tar.gz
tar zxvf acme-dns-client_0.3_linux_amd64.tar.gz acme-dns-client
sudo cp acme-dns-client /usr/local/bin/
rm acme-dns-client_0.3_linux_amd64.tar.gz acme-dns-client

Register an acme-dns account for the domain

The following command starts the registration:

sudo /usr/local/bin/acme-dns-client register -v -d mjbogusz.dev -s http://127.0.0.1:53443

It will generate a CNAME record - put that into your registrar’s records.
For me it was:

Get the certificate

Finally, we’re ready to generate the first certificate.
I’ve generated mine with the following command (note two -d, or domain, entries - one for the top-level and one for wildcard):

sudo certbot certonly --manual --preferred-challenges dns --manual-auth-hook '/usr/local/bin/acme-dns-client' -d mjbogusz.dev -d '*.mjbogusz.dev'

Note any errors that pop up.
If everything went well, the log should contain lines with paths to the new certificate, like these:

Certificate is saved at: /etc/letsencrypt/live/mjbogusz.dev/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/mjbogusz.dev/privkey.pem

Note that this command also registers a cronjob to automatically renew the certificate.
You can verify that with (the unit should be enabled):

sudo systemctl status certbot.timer

Serving the certificate

I’m serving the certificate with nginx.
I’ve simply added this to my HTTPS server configuration:

ssl on;
ssl_certificate /etc/letsencrypt/live/mjbogusz.dev/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mjbogusz.dev/privkey.pem;

Of course some more configuration is needed, but it’s out of scope here.
For some examples on hardening SSL security, see Strong SSL Security on nginx by Remy van Elst.

Automatically reloading certificate on renewal

For reloading the certificates a full nginx restart is NOT required.
Nginx supports signals, and among those, there is ‘reload’.
So to trigger it after a cert renewal, add this line to /etc/letsencrypt/cli.ini:

deploy-hook = nginx -s reload

Sources and resources

sources:

• This thread on Let’s Encrypt’s community and especially jvanasco’s entry
• howto by Bill Walker (PDF version)

resources

• DNS server: acme-dns
• ACME-DNS-client: acme-dns-client