Automated builds and tests
Download Tails 0.23 March 19, 2014
  1. Rationale
  2. What we have
  3. Roadmap
  4. What we need
    1. Automated builds
    2. Automated tests
    3. Infrastructure
  5. Resources
    1. Jenkins
    2. Testing tools and examples
    3. Building Debian packages automatically
    4. emulated boot from USB
    5. Buildbot

Rationale

An automated build and test environment would be pretty useful to ensure a few facts:

  • new code does not break anything
  • new build tools (live-build) and included software (live-config, live-boot) don't break anything
  • updates pushed to the APT repositories we don't control (e.g. Debian's) don't break anything
  • the quality of our releases is good enough
  • power users get to test early images built from feature branches, or from experimental
  • we can ask bug reporters to test an image built from a bugfix branch, and see if it fixes their problem

What we have

Roadmap

  1. Complete phase one (build and publish ISOs automatically): fix all child tickets of ticket #5324.
  2. See other tickets in the Continuous Integration category, and organize the next steps.

What we need

Automated builds

Ideally, every commit pushed to Git should result in an ISO being built, and made available online.

Automated tests

Ideally, every automatically built ISO should be automatically tested, and the results displayed online.

Infrastructure

Putting automatic builds and automatic tests on the same physical machine (while using virtualization to separate concerns) saves hosting and bandwidth costs: fresh images can be tested without being transferred.

In order to be able to build Tails in memory (which greatly speeds up builds) and to properly test HIGHMEM memory wipes, the test server should have at least 32 GB. Using KVM inside a KVM virtual machine is supported by the "nested KVM" feature available since Linux 3.1. VirtualBox (Vagrant) in KVM is not supported as of March 2013.

We could set a threshold of tests that have to be successful before publishing one daily build image. This would prevent too-broken images to spread out and potentially harm users.

The setup of the automated build and test server should be done with public Puppet modules.

Resources

Jenkins

See our wiki page dedicated to Jenkins.

Testing tools and examples

Building Debian packages automatically

emulated boot from USB

See the page that's dedicated to that.

Buildbot

We ?used to have a buildbot setup.

Buildbot can be seen as a framework to deploy continuous integration. It has no real configuration file, but what deserves this role is a file that can be thought programatically. Thus it provide a very flexible environment that can be customize for most projects needs.

Some interesting pages, might be worth reading for people willing to play with buildbot and understand its logic :