Systemd

This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the unix category.

Last Updated: 2022-05-26

Big Idea

systemd starts up and supervises the entire system (hence the name...)

Concept of units

It is based around the notion of units - which have names and types. Unit configurations configuration is usually loaded directly from the file system (they correspond to file names).

Example: a unit avahi.service is read from a configuration file by the same name, and (should be) a unit encapsulating the Avahi daemon.

There are several kinds of units:

  1. service: these are the most obvious kind of unit: daemons that can be started, stopped, restarted, reloaded
  2. socket: this unit encapsulates a socket in the file-system or on the Internet. Each socket unit has a matching service unit, that is started if the first connection comes in on the socket or FIFO. Example: nscd.socket starts nscd.service on an incoming connection.
  3. device: this unit encapsulates a device in the Linux device tree. If a device is marked for this via udev rules, it will be exposed as a device unit in systemd. Properties set with udev can be used as configuration source to set dependencies for device units.
  4. mount: this unit encapsulates a mount point in the file system hierarchy. systemd monitors all mount points how they come and go, and can also be used to mount or unmount mount-points.
  5. automount unit has a matching mount unit, which is started (i.e. mounted) as soon as the automount directory is accessed.
  6. target: this unit type is used for logical grouping of units: instead of actually doing anything by itself, it simply references other units, which thereby can be controlled together. Examples for this are: bluetooth.target which is requested as soon as a bluetooth dongle becomes available and which simply pulls in bluetooth related services that otherwise would not need to be started: bluetoothd and obexd and suchlike.
  7. snapshot: similar to target units snapshots do not actually do anything themselves and their only purpose is to reference other units. Snapshots can be used to save/rollback the state of all services and units of the init system. Primarily it has two intended use cases: to allow the user to temporarily enter a specific state such as "Emergency Shell", terminating current services, and provide an easy way to return to the state before, pulling up all services again that got temporarily pulled down.

Example unit file

[Unit]
Description=Apache Solr for Nextcloud's nextant app fulltext indexing

# systemd supports several kinds of dependencies between units. After/Before can be used to fix the ordering how units are activated.
After=syslog.target network.target remote-fs.target nss-lookup.target systemd-journald-dev-log.socket
Before=nginx.service

[Service]
Type=forking
User=solr
WorkingDirectory=/path/to/solr/server
# commands to start and stop
ExecStart=/path/to/solr/bin/solr start
ExecStop=/path/to/solr/bin/solr stop
Restart=on-failure
Environment=APP_ENV=production
Environment=DEBUG=false
PIDFile=/run/mydaemon/solr.pid

[Install]
WantedBy=multi-user.target

Explaining the unit file:

Here's a table of the targets and their run levels:

# source: https://unix.stackexchange.com/questions/404667/systemd-service-what-is-multi-user-target/404671
Run Lvl Target Units                        Description
0       runlevel0.target, poweroff.target   Shut down and power off
1       runlevel1.target, rescue.target     Set up a rescue shell
2,3,4   runlevel[234].target,               Set up a non-gfx multi-user shell
        multi-user.target
5       runlevel5.target, graphical.target  Set up a graphical multi-user shell
6       runlevel6.target, reboot.target     Shut down and reboot the system

Run levels can be used, for example, to run some services only when a graphical system is there.

Relation to init.d?

It's the successor. Previously one put service scripts in /etc/init.d/ e.g. /etc/init.d/apache2 and called them e.g. /etc/init.d/apache2 status. A convenience for interacting with init.d was sudo service apache2 status. Init.d tended to use many shell scripts. However these are slow and brittle, therefore C programs are preferred in systemd.

How it's faster than predecessors?

Speed Increase 1: Starts less at boot

Speed Increase 2: Starts more in parallel

Previous init achieved inter-process-communication using sockets from p1 to p2. The issue was that p1 had to full boot before its socket was ready to pass into p2. This caused massive wasteful serialization.

The solution is clever: create the listening sockets before we actually start the daemons, and then just pass the socket during exec() to it. That way, we can create all sockets for all daemons in one step in the init system, and then in a second step run all daemons at once.

e.g. If you start syslog and various syslog clients at the same time, what will happen in the scheme pointed out above is that the messages of the clients will be added to the /dev/log socket buffer. As long as that buffer doesn't run full, the clients will not have to wait in any way and can immediately proceed with their start-up. As soon as syslog itself finished start-up, it will dequeue all messages and process them.

Other pluses:

How to use?

See systemctl doc.

Vs. the service command?

service is a wrapper for either init.d or systemctl depending on the version.

How to install on a server?

Put a x.service file in /etc/systemd/system

If it needs a PID/socket file, create a dir related to that daemon under /run/mydaemon. Then chown of that sub-dir (NEVER /run itself though) to the user that will run your daemon

How to remove a service?

Remove the x.service file in /etc/systemd/system

Resources