3.5.1. systemd Units: Services, Timers, Mounts, and Targets

💡 First Principle: Every type of system resource maps to a unit type. Services (.service) are processes. Timers (.timer) replace cron. Mounts (.mount) manage filesystems. Targets (.target) are synchronization points (equivalent to runlevels) that group units together. Understanding unit types tells you exactly which file to look at when something isn't working.

Unit File Locations (priority order — lower overrides higher):

/etc/systemd/system/          ← Admin-managed (highest priority)
/run/systemd/system/          ← Runtime (temporary, cleared on reboot)
/usr/lib/systemd/system/      ← Package-installed (do not edit directly)

Service Unit Anatomy:

# /etc/systemd/system/myapp.service
[Unit]
Description=My Application Service
Documentation=https://myapp.example.com/docs
After=network.target postgresql.service     # Start after these units
Requires=postgresql.service                 # Hard dependency (fail if DB fails)
Wants=redis.service                         # Soft dependency (start but don't fail if absent)

[Service]
Type=simple                  # Process stays in foreground (systemd tracks it directly)
# Type=forking               # Process forks and parent exits (legacy daemon style)
# Type=oneshot               # Runs once and exits (good for setup scripts)
# Type=notify                # Service sends sd_notify() when ready
ExecStart=/usr/bin/myapp --config /etc/myapp/config.yaml
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/usr/bin/myapp --stop
Restart=on-failure           # Automatically restart if service exits non-zero
RestartSec=5s                # Wait 5 seconds before restart
User=myapp                   # Run as this user
WorkingDirectory=/opt/myapp
Environment=LOG_LEVEL=info

[Install]
WantedBy=multi-user.target   # Activated when multi-user.target is reached

Timer Units (systemd replacement for cron):

# /etc/systemd/system/backup.timer
[Unit]
Description=Daily Backup Timer

[Timer]
OnCalendar=daily             # Run at midnight daily
OnCalendar=*-*-* 02:30:00    # Run at 02:30 daily
Persistent=true              # Run missed jobs after downtime (like anacron)
Unit=backup.service          # Which service to activate

[Install]
WantedBy=timers.target

systemctl list-timers --all       # List all timers and next run time
systemctl enable --now backup.timer

Key Targets (replacing runlevels):

systemctl get-default                    # Show default target
systemctl set-default multi-user.target  # Change default (persists across reboots)
systemctl isolate rescue.target          # Switch to target now (drops to rescue)

⚠️ Exam Trap: Units in /usr/lib/systemd/system/ (installed by packages) should never be edited directly — package updates overwrite them. To customize a unit, use systemctl edit unit.service which creates an override in /etc/systemd/system/unit.service.d/override.conf. This file survives package updates.

Reflection Question: You need to add an environment variable DB_HOST=10.0.0.5 to a running service unit that was installed by a package. What is the correct way to do this that will survive future package updates?