Design

Requirements

  • Have the init and PID 1 be the same program, with platform-specific code for doing the initialization.

    • In other words, the init should not have to exec() into PID 1. This is to prevent errors, if at all possible.

    • The initialization should also be controllable by the user, probably through Yao.

  • The supervision system should also be in the same binary as the init.

    • This is to prevent as many errors as possible by using as many of the same code mappings as possible.

  • There will be quite a bit of coupling, but it should all be in the form of having code in the same binary and using mechanisms that users have access to. And those couplings should be able to be disabled, if the user wishes.

    • Such coupling should actually be used as a form of implementing sane defaults rather than locking users in.

Constraints

  • The init should be statically-linked only. There should not even be a build option for otherwise.

    • This includes forcing glibc to statically link, if at all possible.

  • The init should not dynamically allocate memory once it fork()’s the supervision system.

    • This is to prevent init from crashing should memory allocation fail.

    • And if possible, it should not allocate at all.

  • The supervision system should allocate as little memory as possible, but it will have to allocate.

  • The supervisor must be able to run without the init.

    • I think this should be the “default,” in that it should require no command-line arguments.

    • This should be so the supervisor can be run in containers or jails without the init.

  • All system files (files used by the root tree or any tree running as root) must be writable by only root.

    • Error if not.

    • Otherwise, users could try to hijack the system by modifying executable code.

  • Supported environment variables:

    • NO_COLOR.

Yao

  • Yao should have an easy way of defining both longruns and oneshots.

    • Defining oneshots is easy with Yao in its current design.

    • Defining longruns should (probably) be done by running the service in the background with an ampersand &.

  • There should be a way for services to define a “stop” target for themselves.

    • This is so that the shutdown can be done in the right order.

    • It is also so that special shutdown procedures for certain services can be done.

    • Use dynamic dependencies to create the dependencies: if service A depends on service B being up, then service B’s stop target should dynamically depend on service A’s stop target.

  • To implement template services, like for tty’s, use functions that register targets.

    • Use variables to set names and such.

    • Return the name of the target(s) (stop target too) so that the target(s) can be used later.

Scripts

  • There are two kinds of scripts:

    • Service scripts, for launching services.

    • Supervisor scripts, for use by supervisors.

      • These will be analogous to Rig master scripts.

      • These are the scripts that implement the supervision loops.

      • They should be able to access events from outside the supervision tree like D-Bus.

      • They should also define the startup for the supervision tree on boot (for root) or for login (for users).

Init

  • The init system should have a SIGCHLD handler with a self-pipe.

  • After fork()’ing the supervision system, all it should do is reap until it receives signals.

  • If the supervisor crashes, or otherwise fails in a manner not prescribed, then the init system should restart it.

    • The supervisor should use a predefined set of “error” codes to tell the init what to do (whether to shutdown, reboot, etc.).

    • Anything outside of those predefined ones should trigger a restart of the supervision system.

  • Boot process:

    1. TODO: Add initramfs.

    2. Init process is started by kernel.

    3. Init process runs the absolutely necessary init code.

      • This should be Yao code that is default or that the user provides.

      • It should include mounting filesystems, /proc, /dev, etc.

    4. Init fork()’s the supervisor.

    5. Init settles into a loop that select()’s or poll()’s.

      • SIGCHLD should self-pipe into the loop.

  • The init should be able to help the user recover should an error occur during the boot process above.

    • To do so, it will run a command for the user.

    • It should be able to run $SHELL or $EDITOR.

    • It should be able to spawn a tty.

    • It should be able to use Yao as a shell language itself.

      • When doing this, if the user makes a mistake and the parser or VM error, they should be given the option to restart.

      • It might be possible to let them redo only part, but that would add a lot of complexity in a place where it should be dead simple.

      • Having command-line history might be a better way to solve that.

    • Once the user runs the command, it should ask the user if they want to:

      • Reboot.

      • Retry the boot process.

      • Run another command and repeat.

  • Shutdown/reboot process:

    1. Receive SIGCHLD that the supervisor shutdown.

    2. Check the return code and execute accordingly.

  • There should be an option to disable parallel startup.

    • This is for debugging.

Supervisor

  • The supervisor should ensure that the environment is clear, among other things, to ensure reproducibility.

  • The supervisor should use librig for ensuring the dependencies are handled correctly.

  • There should be a mechanism for the supervisor to grab data from services.

    • This should include code for what to do with the data.

    • This will be used for logging (see below), as well as for implementing events, like systemd.

  • The supervisor should use one thread per child service.

    • It should use one particular thread for SIGCHLD with a self-pipe.

      • This one thread should then be able to fire off a message through an fd to the correct thread that is supervising that child.

    • The supervising threads should listen on both the fd for SIGCHLD stuff from the main thread and the fd to communicate with the child (if present).

  • It should be possible to run the supervisor under itself.

    • This will be used for various things, the most important of which would be to implement user-specific supervision trees.

  • Be able to manage services inside Linux containers, FreeBSD jails, and Solaris Zones from the outside.

  • Implement timers inside

    • This is so people can avoid cron.

    • It is also so that the supervisor script can multiplex on the timers.

  • It should be possible to give user permissions, per user, whether that user’s supervision tree should be terminated when the user logs out.

  • The supervisor is what should receive shutdown/reboot requests.

    • This is to prevent the init from having to receive the requests, either through a fifo or signals.

    • This should be done through a fifo.

    • On receiving one, it should gracefully (or forcefully after a configurable timeout) shutdown its tree.

      • Supervisors lower in the tree should do the same, recursively.

    • Then it should exit with a particular return code for the init system to know what it needs to do.

  • Boot process:

    1. Execute a Yao script provided by the user that will define:

      • What targets to import.

      • What targets to start (this should be like giving multiple targets on the Rig command-line).

    2. Execute those targets as Rig would.

    3. Settle into the monitor loops in each thread.

  • Shutdown process:

    1. Receive the request.

    2. Start executing the stop targets for all services, like Rig would.

    3. Cleanup and exit, passing the correct return code to the init.

Events

  • Have events add targets and dependencies.

Logging

  • The init should log everything to a well-known socket file.

    • It should log when children are reaped, including the command-line of the children, if possible. This is so admins can know what programs escaped supervision.

  • The supervisor should set up a logger to be used by all children, including itself.

    • This logger should also be in the same binary.

    • It should have an fd for every log for every service being supervised.

    • It should be able to pass that info to the supervisor, if the user asks for it.

      • This would be for inspecting all of the logs in the system in a central place.

    • The logger should be disablable.

    • The logger definition should use the mechanisms for the supervisor to get info from children in order to pass logs.

  • If possible (read the apenwarr post about logging), make sure the kernel logs are correctly redirected to a log controlled by the logger as well.

  • The logger should also implement log rotation. (I need to study that.)

  • Should I support the syslog protocol?

Debugging

  • Debugging is super important and must be able to be done in a minimal environment, such as when ur is the only thing running after boot when errors happened.

    • In that case, ur should keep running and drop the user into a rescue environment.

  • Because Yao will factor big into both the init and the supervisor, both should include the Yao debugger.

    • This means that the Yao debugger must work at the command-line in a minimal environment.

    • This should also include the Rig debugger of figuring out why targets were “built”.

  • Have a mode for checking the syntax of files needed for boot before boot.

  • Have a mode for checking for cyclic dependencies in the boot sequence before boot.

  • Have a mode for running a supervision tree before boot.

    • This means that, for a lot of services, they would need to have separate config items for testing mode.

    • Those service scripts should also be able to tell if they are in testing mode.

Rig

  • When declaring dependencies on certain targets, the reverse should also be declared for the equivalent stop targets: the dependency depends on the dependents to stop before it can stop.

  • Have the ur keyword after for implementing something like OpenRC’s after keyword, making it so no reverse dependency with the stop target is registered.