Systemd Socket Activation Tutorial with Caddy and Quadlets

-Ben

I’ve spent the past couple of days learning about how systemd.socket files work and how they work with a little caddy server. It feels like a bit of a super power, so I wanted to share. I referenced this blog post a lot while writing my own test sockets, it’s another great resource.

I’ll be using podman quadlets in this tutorial to run the caddy server, but you could run the caddy.service directly if you’d like. So many of these commands need to run as root, I may assume you’re the root user, or prepend sudo. Our service should do something you can see, so it will returns good tidings.

Why Socket Activation

Get the computer ready

Ubuntu 26.04 came out a few days ago and has podman 5.7 in its package repos, so that’s a good place to start. With a new machine, run these commands to get podman.

apt update
apt install podman

Write our files

Socket Unit

I’m going to write my files in the home directory of my user, cadusr. First up is a socket file. Names are important, the socket and container file names must match. I’ll call this tidings.socket

[Unit]
Description=socket for good tidings
Before=tidings.service

[Socket]
ListenStream=localhost:80
FileDescriptorName=http

[Install]
WantedBy=sockets.target

This socket file describes to systemd what socket it should listen on. The “Before” key tells systemd that tidings.service relies on tidings.socket. Socket units will actually get this by default, but I’ve put it here to illustrate. The Socket table has the port to listen on and gives it a name. The name sets LISTEN_FDNAMES in the environment with a colon separated list of names given to the sockets. systemd passes these sockets as file descriptors (think stdin, stdout) starting at 3. LISTEN_FDS gives you the number of sockets passed through. The “ListenStream” keyword specifies the type of socket, in our case this will work for http/s communication. The “WantedBy”

Caddyfile

Before we write our container file, let’s write the Caddyfile that we will configure caddy with. It likes to be mounted in a directory, not the file directly, so we’ll create a caddy directory for it.

mkdir caddy
touch caddy/Caddyfile

http:// {
  bind fd/3
  respond "Good tidings"
}

That’s it for the Caddyfile. This tells caddy to listen on http protocol but doesn’t give it a port or hostname. We bind “fd/3”, or file descriptor 3, as the interface, which we know systemd will provide for us.

Container file

Last, let’s set up our caddy container. In a file named tidings.container, write the following:

[Unit]
Description=send good tidings
Requires=tidings.socket
After=network.target

[Container]
Image=docker.io/library/caddy
Volume=/home/cadusr/caddy:/etc/caddy # replace with your own Caddyfile directory
ReloadCmd=/usr/bin/caddy reload --config /etc/caddy/Caddyfile --force

[Service]
User=1000 # replace with output from `id -u` when you aren't root
Group=1000 # replace with output from `id -g` when you aren't root

A Container Unit can be used by systemd because podman has a generator for it to turn it into a systemd service, this is why we can’t use docker or another container runtime. Table by table, the Unit table just sets up a description and some dependencies for us. systemd is great at dependency management. The Container table is where we put all our podman related keys. These correspond very closely with the command line values (or docker compose values). Notice I’ve put a ReloadCmd so we can call systemctl reload tidings.service to reload our caddy file; systemd knows how to do it! We also don’t publish any ports, which is unusual for a web service. The socket unit is what listens on the port in this case, caddy just binds to a file descriptor. So the container doesn’t need any ports open, or really any network priviledges! Lastly, the Service table. This is forwarded through the podman generator verbatim. In our case, we’re going to run our service as the user id 1000 and group id 1000, so our container doesn’t run as root.

Installing our unit files

“Installing” the unit files is as simple as copying to the right place and reloading systemd. podman has a fancy quadlet install command that helps us, but I’m not sure it does anything but copy (and maybe verify?) to the right place. Note, both these commands must be run as root. If you run podman quadlet install as your user, it will become a user unit file instead of a system unit file. Then your socket won’t be able to find your service and neither of them will work. Even though our container will run as an unpriviledged user, it’s still installed as a system unit.

cp tidings.socket /etc/systemd/system
podman quadlet install tidings.container
systemctl daemon-reload

Once we install the file, you can run systemctl cat tidings.service to see the generated service unit.

If you accidently install a bad container unit, sometimes podman will refuse to install a new one or remove the old one. It doesn’t give you great feedback unless you run podman quadlet list. In that case, you can just remove it manually from where it was “installed” (copeid) to. rm /etc/conatiners/systemd/tidings.container

Enable our units

Either quadlets or systemd generators don’t really need to be enabled, but we definitely want to enable and start the socket. This next command enables the socket (so it starts on boot) and starts it now for us.

systemctl enable --now tidings.socket

Now systemd is listening to port 80 for us! We can hit port 80 on our machine with curl. The first time it will take a moment as the caddy server starts. Subsequent invocations of curl will work much faster!

curl localhost
# should return "Good tidings"

Now we have a socket activated service that will be supervised by systemd! Good work!

Further Usage

More than one port

Most web servers will listen to at least two ports, 80 and 443, so this socket activation wouldn’t be very useful if we couldn’t bind to more than one port. Fortunately, you can add ListenStream as many times as you want in the socket file. systemd guarantees file descriptor ordering of sockets listed within the same .socket file (it does not guarantee them across socket files, if you had more than one socket file for the same service). If we wrote this socket file:

[Socket]
ListenStream=80
ListenStream=443

then fd/3 would always be port 80 and fd/4 would always be port 443. This is nice because caddy doesn’t yet support the names passed through.

You can also listen on specific interfaces. When we set our ListenStream to localhost:80 before, systemd opened a socket that only listened to the localhost (loopback) interface. With just setting the value to 80 like I did above, systemd will listen on all interfaces and your caddy server will be reachable externally to your computer. This is nothing special for caddy, but just know that you aren’t losing that here; it’s really only that the port is passed to you instead of your service needing to bind it itself.

Running it yourself

We set up unit files and installed them so systemd would handle the lifecycle of our app and make it easy for us to control with typical systemd commands. But systemd also provides a shell command you can run to set up a socket activation.

systemd-socket-activate -l "localhost:80" podman run --volume /home/cadusr/caddy:/etc/caddy docker.io/library/caddy

That command is the equivalent of our unit files from above. This is just for a one off command and a great way to play with socke activation. While the process is running it behaves the exact same way as our unit files. A few options I find very interesting are “accept” and “inetd” which basically make systemd into a web server for cgi-like scripts. You programming language would still need to set up the environment for you, but it’s a very simple stdin to stdout interface to talk with the world.

Writing a web service for socket activation

What if you are writing your web app and have it containerized and want it to support socket activation? I don’t think I saved the fabulous tutorial I found online about eating lunch. But the basic premise is to check for the LISTEN_FDS environment variable. If it’s set, you know you can connect to fd/3, however your programming language expresses that. For node I think this might be as simple as

net.createServer().listen({fd: 3})

although I haven’t made a good test for it.