I've written before that I want Ick to do builds in an environment isolated both from the host and, optionally, from the network.

  • Isolation from host makes it easier to install any build dependencies needed to build a project. They can be installed inside a container. Installing things directly on the build host (worker) is risky, and may result in a non-functional build host. Also, if each project can install whatever it wants on a build host, different projects may keep altering the build environment in ways that make other projects fail to build. They will certainly not built reproducibly, if there's no guarantee about what's in the build environment.

  • Isolation from the network means that builds can't depend on things outside the build environment. This is good for the safety and security of the build. (Some builds inherently need to access the network, of course, so that needs to be possible.)

I've decided to use containers, as provided by systemd-nspawn, for the isolation, at least for now. This is not set in stone, and can be re-evaluated after ALPHA-1 is out. nspawn acts more or less like the chroot command, except it provides better isolation. It's a very easy interface to running thing in isolation.

I will add the concept of systree (system tree) to Ick. A systree is a directory tree in which all the software (operating system, applications, libraries, etc) that are needed for a build are installed. This directory will form the root directory inside the container.

In addition, the container will bind-mount a workspace directory from the host system, as /workspace inside the container. The workspace starts out empty, but the pipeline may fill it with stuff. Typically it will contain the source code of a project, but it can be anything. In this case, it is where the systree gets built.

Ick will allow a pipeline action to be executed in one of three ways: on the build host directly (where: host), in a chroot using the workspace (where: chroot), in a container (where: container). In the latter case Ick will construct the systree, create an empty directory as the workspace, and execute the action inside a container using the systree as the root directory, and the workspace as /workspace.

The systree for a container is constructed by running debootstrap on the host, to fill the workspace, and then archiving the workspace. The archive will be stored somewhere else. I've added the blob service component for this. When it's needed, Ick (worker-manager) will get it from the blob service, and unpack it into the systree directory. Voila!

The archive can, of course, be re-used any number of times, on any number of worker hosts.

I will further make Ick allow the user to specify how the systrees get constructed. The user will specify a project to build a systree:

project: systree_stretch
parameters:
  debian_codename: stretch
  packages:
    - git
    - ikiwiki
    - rsync
    - python3
  systree_name: stretch-ikiwiki
pipelines:
  - build_systree

In the above example, the project defines three parameters (debian_codename, packages, and systree_name), and a pipeline. The pipeline looks like this:

name: build_systree
parameters:
  - debian_codename
  - packages
  - systree_name
actions:
  - debootstrap: auto
    where: host
  - python:
      import subprocess
      packages = params['packages']
      subprocess.check_call(['apt', 'install', '-y'] + packages)
    where: chroot
  - archive: /workspace
    where: host

This pipeline takes the same three parameters, and runs three actions. The debootstrap action is run on the host, and uses the debootstrap program to install Debian into a directory. The action will instruct the program to install into the workspace directory. The auto parameter means the action will get the name of the Debian release from the debian_codename parameter.

The python action then installs additional packages into the workspace. It runs in a chroot, not container, as there isn't a systree yet in which to run. Running in a chroot means apt will install packages in the chroot, not the build host.

The archive action creates a tarball of the workspace, and uploads it to the blob service. The action takes the name of the blob from the systree_name parameter.

After this, we have a systree blob in the blob service. We next use it to run a build.

project: ick.liw.fi
parameters:
  git_url: git://git.liw.fi/ick.liw.fi.git
  git_ref: master
  rsync_target: www-data@ick.liw.fi:/srv/http/ick.liw.fi
pipelines:
  - ikiwiki_website

This example again defines some parameters, and then specifies a pipeline. There's nothing about systrees or containers here yet. That goes into the pipeline.

name: ikiwiki_website
parameters:
  - git_url
  - git_ref
  - rsync_target
systree: stretch-ikiwiki
actions:
  - shell: |
      url="$(params | jq -r .git_url)"
      ref="$(params | jq -r .git_ref)"
      tgt="$(params | jq -r .rsync_target)"
      git clone "$url" -b "$ref" src
      mkdir html
      cd src
      ikiwiki --setup ikiwiki.setup --verbose
      cd ../html
      rsync -av --delete . "$tgt/."
    where: container

This specifies that the stretch-ikiwiki blob is to be retrieved from the blob service, and unpacked as a systree. Then the shell code is to be run inside a container using the unpacked systree.

Voila! We have a way to construct systrees and use them to run other things in containers.

Compared to my current CI system, this means I don't need to keep installing build dependecies of my various personal projects to the CI build hosts, which is a small relief.