APT repository
Download Tails 0.23 March 19, 2014
  1. Overview
  2. Build system
  3. SSH access
  4. HTTP access
  5. Workflow
    1. Creating a new branch
    2. Importing a new package
    3. Merging a topic branch
    4. Merging a main branch
    5. Resetting a suite to the state of another one
    6. Freezing devel into testing
    7. Tagging a new Tails release
    8. After a new Tails release is out
    9. Giving access to a core developer
    10. Contributing without privileged access

To avoid cluttering our main Git repository with Debian source and binary packages, we have set an APT repository up.

Overview

We use one single APT repository hosting multiple suites:

  • We have a (read-only) suite for every past release: 0.9, 0.10.1, etc.
  • We have a suite for each main branch: stable, testing, devel
  • We have a suite for each topic branch: bugfix/*, feature/*. Important note: the APT suite corresponding to a given Git topic branch contains only the packages this branch adds to the tag or main branch it diverged from.
  • We also have a less formal unstable suite, that should not be used by any Tails git branch; it can be used as hosting space for other packaging work we might do, e.g. acting as upstream or Debian maintainers.
  • We also have a debomatic-squeeze-backports-mozilla suite, used (hopefully temporarily) import build-deps for our web browser, that are not available elsewhere (anymore). This suite is used by the squeeze-backports-mozilla chroot in our [[contribute/Debian package builder]].

The suite(s) to use as sources for APT, during the build and inside the resulting system, are determined at Tails build time (auto/config). See details in the Build system section bellow.

We manage our APT repository with reprepro.

The Puppet modules used to manage this part of our infrastructure are listed on our Git page.

Basically, a cronjob fetches and scans the Tails Git repository every few minutes, detects new branches, and accordingly:

  • generates conf/distributions
  • generates conf/incoming
  • create new suites in the APT repository

Build system

The build system adds the relevant APT sources:

  • if the version in debian/changelog was released already (i.e. a matching tag exists), then add the suite corresponding to this release (e.g. 0.10 or 0.10.1);
  • else, if building from the testing branch, add the testing suite
  • else, if building from the experimental branch, add the experimental suite
  • else, if building from the devel branch, add its own suite

Also, if we're building from a bugfix or feature branch, add its own suite.

SSH access

One must configure their SSH client to connect to the APT server:

Host incoming.deb.tails.boum.org
    Port 3003

HTTP access

This is the http:// public APT repository used at Tails build time. The tails::reprepro Puppet class sets nginx up to serve that.

Workflow

Creating a new branch

Push your branch to Git and wait a few minutes for the new APT suite to appear.

Importing a new package

Building a package

Make sure the Distribution: field in your .changes file matches the suite you want the package to land in (e.g. pass --changes-option=-DDistribution=feature-torbrowser to pdebuild's --debbuildopts).

Make sure to have the .changes file include the original source archive (.orig.tar.{gz,bz2,xz}) if it is not already in our APT repository; this can be done by passing -sa to pdebuild's --debbuildopts.

Configuring an upload tool

Configuring dupload

Add this configuration snippet to your dupload configuration:

$config::cfg{'tails'} = {
        fqdn => "incoming.deb.tails.boum.org",
        method => "scp",
        login => "reprepro",
        incoming => "/srv/reprepro/incoming/",
        dinstall_runs => 1,
};

Confuguring dput

Add this to .dput.cf:

[tails]
fqdn            = incoming.deb.tails.boum.org
method          = scp
login           = reprepro
incoming        = /srv/reprepro/incoming/
run_dinstall    = 0

Uploading and importing process

Carefully prepare and build your package. Usual precautions, (Lintian etc.) apply.

Carefully check the .changes file (especially the Distribution control field, and the included files list; the former can be fixed with the changestool(1) command, from reprepro).

Sign the .changes file with a key that is in the uploaders list:

$ debsign $CHANGES_FILE

Upload the files to the incoming queue:

$ dupload --to tails $CHANGES_FILE

reprepro will automatically notice the new files and import them into the suite specified in your .changes file.

Check the result:

$ ssh reprepro@incoming.deb.tails.boum.org reprepro list $SUITE $PACKAGENAME

Merging a topic branch

When a Git topic branch is merged into a main branch, the corresponding operation must be done on the APT suites.

Example:

$ git checkout devel
$ git merge feature/icedove
$ ssh reprepro@incoming.deb.tails.boum.org \
     tails-merge-suite feature-icedove devel
$ git push

(Note that unfortunately, contrary to what whoever with a Git background would guess, the reprepro operation called pull is not what we want: it pulls from all other suites into the ones specified on the command-line.)

Merging a main branch

When a Git main branch (devel, experimental, testing, stable) is merged into another main branch, the corresponding operation must be done on the APT suites.

  1. Save the list of packages currently present in the APT suite we want to merge into, e.g. reprepro list experimental.
  2. Make sure you are not going to overwrite newer packages with older ones.
  3. Merge the APT suites the same way as when we merge a topic branch.
  4. Make sure not to re-add, into the branch we merge into, any package that was removed from it, but still is in the branch we merge from: e.g. when merging devel into experimental, it may be that experimental had some packages removed (e.g. due to previously merging a topic branch into it, whose purpose is to remove custom packages). To this end, compare the resulting list of (package, version) in the experimental APT suite with the one saved before the merge (hint: use the tails-diff-suites script), check Git merges history if needed, apply common sense, and remove from experimental the packages that were removed from it a while ago, and were just erroneously re-added by the merge operation.

Resetting a suite to the state of another one

  1. . First, set some environment variables:

    # the suite to reset
OLD=testing
# the final state it should be in
NEW=devel

  2. . Then, empty the OLD suite:

    ssh reprepro@incoming.deb.tails.boum.org \
      reprepro removematched $OLD '\*'

  3. . Finally, merge NEW into OLD

    ssh reprepro@incoming.deb.tails.boum.org \
      tails-merge-suite $NEW $OLD

Freezing devel into testing

  1. Merge devel branch into testing in Git
  2. (Manually) hard reset the testing suite to the current state of the devel one.

Tagging a new Tails release

Once the new release's Git tag is pushed, a cronjob should create a new APT suite on the APT repository's side within a few minutes. This new APT suite is called the same as the new release version. One may check it has appeared in ~reprepro/conf/distributions.

Then, the APT suite corresponding to the branch that was used to prepare the release must be copied to the new empty APT suite that just appeared:

If this is a major release:

$ ssh reprepro@incoming.deb.tails.boum.org \
     tails-merge-suite testing $TAG

Else, if this is a point-release:

$ ssh reprepro@incoming.deb.tails.boum.org \
     tails-merge-suite stable $TAG

After a new Tails release is out

If you just put out a final release:

  • merge stable or testing into devel
  • increment the version number in devel's debian/changelog to match the next major release, so that next builds from the devel branch do not use the APT suite meant for the last release
  • increment the version number in devel's debian/changelog to match the next point release, so that next builds from the stable branch do not use the APT suite meant for the last release

If you just released a RC:

  • add a dummy changelog entry (for the upcoming, non-RC version) in the branch used for the release (stable or testing), so that the next builds from it do not use the APT suite meant for the RC
  • add a dummy changelog entry (for the release after the one you released a RC for) in the branch used for the release (stable or testing), so that the next builds from it do not use the APT suite meant for the RC

If the release was a major one, then reset the stable APT suite to the state of the testing one.

Giving access to a core developer

  1. Give SSH access to the reprepro user on the system that hosts reprepro (using the ssh_authorized_key Puppet resource).
  2. Import the developer's public GnuPG key into the reprepro user's GnuPG keyring -- should be doable using Puppet, some day
  3. Add the developer's OpenPGP key ID to $reprepro_uploaders in our tails::reprepro Puppet module. Deploy.

Contributing without privileged access

Non-core developers without access to the "private" APT infrastructure would add the .deb they want to their Git branch as we have been doing until now, push the result on repo.or.cz or whatever... and at merge time, we would rewrite their history to remove the .deb, and import it into our APT repo.