Host ID: 284
systemd For Learner Drivers
A graphic to help out: http://hackerpublicradio.org/eps/hpr1672.svg
This is a subject that attracts controversy, but I am not today going to be controversial, I hope. Many Linux systems are moving away from SysV Init and adopting systemd instead; both Linuxes that I use, Fedora and Mint have adopted systemd, and I understand that Debian has now forked to allow both sides of the argument to have their way. I am not going to get into the debate here. My personal stance is that I see both sides of the argument and I will continue to perch on top of the fence until systemd either proves itself or fails to do so.
In this HPR I am going to try to fill a gap that I have seen in the systemd discussion; that is - how to operate it. I am not an expert on systemd, I have just tried to work it, and in doing so I have fished around in my file system and in the documentation. If you want to know what I found, then keep on listening. By way of opening I will remind myself, and you also, what systemd is replacing.
SysV initd works with runlevels, the most common being
- 5 for graphical multiuser networked
- 3 for cli multiuser networked
- 1 for single user
- 6 for reboot
- 0 for halt
In moving to a runlevel, unwanted services are shut down and wanted services are started up. For most users on most systems the most appropriate default runlevel is 5 giving multiuser, GUI & networking. Services can be started and stopped on demand by inetd.
systemd works differently. It has target units. For most users on most systems the most appropriate default target is the graphical.target, which does a similar thing to runlevel 5 . Units are configured by unit configuration files. These files may start other units and stop other units. They can impose sequence and dependancies. There is a lot of cascading going on, with unit launching unit launching unit. Units also can be started and stopped on demand by systemd.
The term Unit refers to a resource that systemd is taking under its control. There are 12 different types of Unit.
- that starts/stops daemons
- activates network connections
- activates kernel devices
- controls mount points
- provides on-demand mounting of file systems
- does for swap what systemd.mount does for filesystems
- starts/stops external processes
- groups of services akin to init level 3, init level 5
- saves/restores the momentary state of other units
- triggers units based on date/time
- trigger units based on changes in file system objects
- organises units in a hierarchical tree of cgroups, for resource management purposes
Units files called by systemd live in
/etc/systemd/system. But these are symbolic links to the real ones stored in
There is a parallel
/etc/systemd/userstructure which does not seem to do anything on my computers, so I work for now like its not there.
There is also a
/run/systemd/systemstructure which appears to contain runtime configuration files with names like
session-xxxx.scope. These are the unit type for external processes.
Table 1. Directory structure for systemd Path Description
Units of installed packages
The next thing we need is Directives.
The unit configuration files contain directives to start/stop a unit, and directives that cascade to other unit configuration files that start/stop dependant units. Directives may impose conditions on whether or when to call a unit. There are a whole bunch of different directives listed in man systemd.unit. These are a few.
Requires=list of units to start. If any required units fail then abort this one
Conflicts=list of units to stop
After=the order in which units will start
Before=the order in which units will start
Wants=list of units to start. If any fail just continue anyway
As well wanted units listed by the WANTS directive, there may also be a 'wants' directory below the unit directory. So the unit conf file
/etc/systemd/system/default.targetwill cause two further unit conf files to be read in from the
Each required unit and wanted unit from the directives, as well as those in the wants directory are added to a job queue. If directives cascade to other unit files containing more directives then all of these dependences are also added to the job queue. A directive may start or stop another unit, or that change the detail of a job already in the queue. All directives ultimately cascade down to starting or stopping one of the base units in
To get a feel for how this all pans out in practice I will walk us through the cascade of unit files from bootup.
First, the default.taget is activated, which on my system is just a link to graphical.target
graphical.target[Unit] Description=Graphical Interface Documentation=man:systemd.special(7) Requires=multi-user.target After=multi-user.target Conflicts=rescue.target Wants=display-manager.service AllowIsolate=yes
- start multi-user.target
- start display-manager.service
- stop rescue.target
Also we have a wants directory
- starts accounts-daemon.service (for logging)
- starts rtkit-daemon.service (for realtime scheduling)
graphical target cascaded to multi-user.target.[Unit] Description=Multi-User System Documentation=man:systemd.special(7) Requires=basic.target Conflicts=rescue.service rescue.target After=basic.target rescue.service rescue.target AllowIsolate=yes
- start basic.target
- stop rescue.service
- stop rescue.target (again)
Also we have a wants directory
/etc/systemd/system/multi-user.target.wants/that- abrt-ccpp.service - abrtd.service - abrt-oops.service - abrt-vmcore.service - abrt-xorg.service - atd.service - auditd.service - avahi-daemon.service - chronyd.service - crond.service - cups.path - irqbalance.service - libvirtd.service - mcelog.service - mdmonitor.service - NetworkManager.service - nfs.target - remote-fs.target - rngd.service - rpcbind.service - rsyslog.service - smartd.service - vmtoolsd.service
graphical.target also cascaded to display-manager.service which is not present on F20 so I guess we don't need it.
So multiuser.target cascaded to basic.target, which itself cascades to- sysinit.target - sockets.target - timers.target - paths.target - slices.target - firewalld.service
basic.target cascaded to sysinit.target which itself cascades to- local-fs.target - swap.target - dmraid-activation.service - iscsi.service - lvm2-monitor.service - multipathd.service ( which looks like all the file system daemons)
basic.target also cascaded to sockets.target which itself cascades to- avahi-daemon.socket - cups.socket - dm-event.socket - iscsid.socket - iscsiuio.socket - lvm2-lvmetad.socket - rpcbind.socket
Now we start reaching the end-points of this trail at- systemd.sockets - systemd.timer - systemd.path - systemd.slice - systemd-fstab-generator
By the time all of that has finished, if I type the command
# systemctl list-units --type service
I see that 58 services are listed as running
Running and Configuring Services
If we are going to work with systemd we will have to give it instructions. In systemd parlance
- active = running, currently in use
- loaded = enabled, available for use
These terms crop up in the output from commands
Many instructions are given to systemd by the systemctl command.
Now to compare line up some common SysV init tasks with their systemd equivalent
Table 2. SysV init commands and their systemd equivalents command SysV Init systemd Check status
# service bluetooth status
# systemctl status bluetooth
# service bluetooth start
# systemctl start bluetooth
# service bluetooth stop
# systemctl stop bluetooth
# chkconfig --level 35 ntpd on
# systemctl enable ntpd
# chkconfig --level 35 ntpd off
# systemctl disable ntpd
Much has been said about the desirability or otherwise of binary logs, but systemd gives us these so we had better know what to do with them.
Journal instructions are given to systemd by the journalctl command
- To view all log entries in one go. This is verbose, mine came out at ~9000 lines
- To view from a specific date
# journalctl --since="2014-05-07"
- To view kernel logs
# journalctl -k
- To follow a log in realtime ... and then to close
# journalctl -f
- To view log entries associated with a given PID
# journalctl _PID=1
- To view log entries associated with a given service
# journatlctl -u bluetooth
Interrogating the system
More systemd information
- Get/Set system information. Works like uname, but is more verbose
- Get/Set timezone & timedate info
Table 3. SysV init information and their systemd equivalents SysV Init Info SysV Init command systemd info systemd command What services are available for init.d to manage
# ls /etc/init.d
What service units are available for systemd to run
# systemctl list-units --type service --all
What services are configured to be run by init.d for each run level
# chkconfig --list
What service units are currently active
# systemctl list-units --type service
- man pages for systemd, systemd.unit, systemctl, journalctl.
Plus a bunch of other man pages that are SEE ALSO in these man pages.
- systemd homepage
- A useful systemd blog posting, by Adam Dean of Asterisk Telephones
- A useful youtube vid by George Magklaras, UNiversity of Oslo, Norway
EVA - The Rules for Extravehicular Activity
Here I dip into the NASA experience of and rules for Extravehicular Activity, prompted at first by watching a film called The Europa Report, directed by Sebastian Cordero (2013).
WARNING - THIS PODCAST CONTAINS SPOILERS
While I have some gripes about the film, I was impressed by its general failfulness to the science
- It thought to find life on Europa, a moon of Jupiter considered by real exobiologists and planetary scientists to be a good candidate
- Neil deGrasse Tyson made a cameo appearance
- The portrayal of Europa's geography and character
- Having to drill through the ice to get at the sea below
- The behaviour of the crew as scientists and engineers
Science consultant on the film was Kevin Hand, an astrobiologist and expert on Europa at NASA's Jet Propulsion Laboratory
To my mind, the scientists were behaving like scientists and the engineers behaved like engineers. To follow along it might help to recall their names
- Captain - Willam Xu
- Pilot - Rosa Dasque
- Chief scientist - Daniel Luxembourg
- Marine biologist - Katya Petrovna
- Junior engineer - James Corrigan
- Chief engineer - Andrei Blok
All was going scientifically until the director drove the plot forward with two EVA incidents
EVA-1 : Flash back episode, engineers James and Andre go out to fix a failed communications circuit
- Andre rips his suit
- James gets squirted with rocket fuel
- Only one astronaut survives
I have problems with this because it's just too clumsy for trained professional astronauts. Where are the decontamination procedures, the tethers, the special tools?
EVA-2 : Down on the surface, Marine biologist Katya decides to walk out alone
- Tourtured debate in the ship
- Of four able and expendable crew members, none go with her
- Katya does not come back alive
With this I am shouting at the screen "No Way! Where's the fracking operating manual? No one goes EVA on their own"
So, that is why I researched the NASA rules for Extravehicular Activity. And I found that none of these events would have happened the way they were shown, had the crew, who were so professional in every other way, followed the NASA procedures.
The two astronauts issue
- The most recent occasion where an astronaut went solo EVA was in 1971, when David Scott stuck his head out of the airlock of Apollo 15.
- Most recent before that was in 1966, when Buzz Aldrin went EVA from Gemini 12 (Gemini craft only had two crew).
- Since 1971, there have been 358 space walks and every single one has had two crew.
- I found no written regulation, but de-facto, nobody leaves the spacecraft alone.
NASA documents on the internet discuss in exhaustive detail all considerations for EVA. What I present is a cherry-picked handful. I could not cover all of it
- reasons for EVA
- hazard mitigation
- procedures for safe conduct
- fall-back procedures
- failure handling
- accident control
International Space Station (ISS) EVA Procedures Checklists
- Presuming that all the equipment maintenance checks, and readiness checks have alread been done
- 30 minutes of Airlock preparation and testing
- 30 minutes of changing components for the suit to fit the astronaut
- 170 minutes of EVA-Prep
- Then you are ready to depressurise and leave the airlock
- EVA might last 2 - 8 hours
- Post EVA
- 30 minute procedure to take the suit off
- 10 minute procedure to disconnect internal equipment
- Recharge & maintain the Extravehicular Mobility Unit (EMU)
- Clean & maintain the Suit
Although this podcast is about EVA, it does reference the science in a film that I enjoyed and respect very much, so here is a gem that I only came across while researching the landing site. In the scientific journal Nature, Volume 479, 16 November 2011, Britney Schmidt et al, of University of Texas, Austin, published a paper titled "Active formation of 'chaos terrain' over shallow subsurface water on Europa." In the paper these authors suggest that in the Conemara zone of the Chaos Terrain, an area on the surface of Europa, the ice may be as little as 3 km thick. Then in the film the Conemara Chaos was the targetted landing zone and the drill broke through the ice at a depth of 2800m.
Well there is one more thing that the podcast says, but it is the ultimate spoiler. So if you have not already listened to the podcast, I highly recommend that you watch the film first.
- Early Spacewalks: http://en.wikipedia.org/wiki/List_of_spacewalks_and_moonwalks_1965-1999
- Late Spacewalks: http://en.wikipedia.org/wiki/List_of_spacewalks_since_2000
- NASA Manual on Systems Integration Standards: http://msis.jsc.nasa.gov/sections/section14.htm
- NASA payload Safety Conference, Feb 2000: http://paso.esa.int/5_training_materials/training_22_materials.pdf
- EVA-22 Cassidy and Parmitano complete ISS spacewalk: http://www.nasaspaceflight.com/2013/07/eva-22-cassidy-parmitano-iss-spacewalk-eva-22/
- EVA-23 terminated due to Parmitano EMU issue: http://www.nasaspaceflight.com/2013/07/astronaut-duo-us-spacewalk-outside-iss/
- Extravehicular Activity Operations Overview: http://www.colorado.edu/ASEN/asen3036/EVAOverview.pdf