Releasing Iceweasel + Torbrowser patches
Download Tails 0.23 March 19, 2014
  1. 1. Prepare environment
  2. 2. Was Iceweasel updated?
  3. 3. New Firefox release
  4. 4. New Iceweasel release
    1. If Debian's iceweasel was pushed to Git already
    2. If Debian's iceweasel was not pushed to Git yet
  5. 5. Update Torbrowser patches
  6. 6. Build packages
    1. Update debian/changelog
    2. Tag the release
    3. Clean up the source tree
    4. Build for unstable
    5. Build for squeeze-backports
    6. Import bundled preferences
  7. 7. Potential problems (and solutions)
    1. Problems with ./configure

1. Prepare environment

  • Clone the Tor browser Git repository if you do not have it handy yet.

  • Add (and fetch from) a Git remote for the Debian iceweasel packaging repository:

    git remote add -f debian git://git.debian.org/git/pkg-mozilla/iceweasel.git

  • Export the new upstream release to the environment of the one shell or three that will be used:

    export VERSION=17.0.9esr

2. Was Iceweasel updated?

It might have been updated in one of these sources:

If it was updated, then skip to New Iceweasel release. Else, skip to New Firefox release.

3. New Firefox release

If Iceweasel was not updated to match the new Firefox release we want, a bit more work is needed.

Note that usually, we're doing these steps (usually on Sunday or Monday) before the new ESR was officially released (which usually happens on Tuesday). Mozilla make the source available on previous Friday or Saturday, so that downstreams (such as us!) can get their stuff ready in time for the security announce.

  • Download the Firefox tarball and detached signature from https://ftp.mozilla.org/pub/mozilla.org/firefox/releases/VERSION/source/ (VERSION is the version we want to build, that is something like 17.0.7esr).
  • Check the signature.
  • Put the tarball in the parent directory of your Iceweasel Git repository.
  • Extract the tarball.
  • cd into the extracted directory.
  • Copy the debian/ directory from our previous package into the new upstream source directory.

  • Add a debian/changelog entry matching the new upstream version. Use 0 for the Debian packaging version, e.g. 17.0.5esr-0, to leave room for the official packaging that we will want to merge when it's out:

    dch -v ${VERSION}-0 "New upstream release."

  • Download and repack the other tarballs:

    make -f debian/rules download

  • cd into our Iceweasel Git directory.

  • Checkout the tails/master branch.

  • Unapply all quilt patches and commit:

    quilt pop -a

  • Get yourself a new upstream branch:

    git branch -D upstream
git branch upstream tails/master

  • Trick the tarball importer to import the correct version:

    cp ../mozilla-esr24/browser/config/version.txt browser/config/ && \
cp ../mozilla-esr24/debian/changelog debian/

  • Import the new upstream release into the upstream branch:

    make -f debian/rules import

  • Merge the import commit into tails/master:

    git reset --hard && git merge upstream

  • Get the debian directory back:

    git checkout HEAD^ -- debian && \
git commit -m 'Get Debian packaging directory back.'

  • Cleanup quilt status:

    rm -rf .pc

  • Apply all quilt patches:

    quilt push -a

    It might be that the last patch (configure.patch) fails. Ignore it for now.

  • Commit:

    git add . && git reset HEAD .pc && git commit -a -m 'Apply all quilt patches.'

4. New Iceweasel release

Skip this entire stage if you imported a new Firefox release.

The way to proceed is different depending on whether Debian's iceweasel was pushed to it yet, or not.

If Debian's iceweasel was pushed to Git already

  • Retrieve the update from the iceweasel Git repository and verify the Git tag you want to import, e.g.

    git fetch debian && git tag -v debian/17.0.8esr-1

  • Checkout our tails/master branch.

  • Unapply all Torbrowser patches:

    • If quilt knows they are applied (quilt applied will tell you), then use quilt pop as many times as needed.

    • Else, some manual care is needed so that quilt internal state matches the actual state of the source tree. We need to manually unapply all quilt patches, then reapply them all:

      for p in $(tac debian/patches/series) ; do
   patch -p1 -R < "debian/patches/$p"
done && quilt push -a

      ... and then use quilt pop as many times as needed to unapply all Torbrowser patches.

  • git add the new files and the modified ones

  • git rm the deleted files

  • Commit:

    git commit -m 'Remove Torbrowser patches.'

  • Merge the tag, e.g.

    git merge debian/17.0.8esr-1

  • Verify with that tails/master is in the same state as Debian's iceweasel, e.g.

    git diff --stat debian/17.0.8esr-1..tails/master
    All expected differences should be:
    • files modified: .gitignore, debian/{changelog,rules,control,control.in}, debian/{browser.mozconfig,xulrunner.mozconfig.in}
    • files added: .mozconfig*, debian/tails.*.mozconfig, debian/patches/series, and the Torbrowser patches.

If Debian's iceweasel was not pushed to Git yet

Then, we have to import the source package into Git ourselves, and merge from Debian's Vcs-Git later.

  • Download, verify and extract the new iceweasel source package with dget.

  • Checkout our tails/master branch.

  • Unapply all quilt patches and commit:

    quilt pop -a

  • git rm the deleted files

  • git add the new files and the modified ones

  • Commit:

    git commit -m 'Remove all quilt patches.'

  • Overwrite the files in the Git checkout with the new ones. Assuming the new extracted iceweasel package is in iceweasel-17.0.2esr, and our iceweasel Git repository checkout is in git:

    rsync --stats -a --exclude=.git --delete iceweasel-17.0.2esr/ git/

  • git rm the deleted files

  • git add *

  • Add other added or modified files but .pc.

  • Commit:

    git commit -m "Import $(head -n 1 debian/changelog | sed -e 's,).*,),')"

  • Verify with diff that the current state of the tails/master is exactly the same as Debian's iceweasel source package one:

    diff -Naur --exclude=.git iceweasel-17.0.2esr/ git/

  • Bring our changes back:

    • files modified: .gitignore, debian/{changelog,rules,control,control.in}, debian/{browser.mozconfig,xulrunner.mozconfig.in}
    • files added: .mozconfig*, debian/tails.*.mozconfig, debian/patches/series, and the Torbrowser patches.

5. Update Torbrowser patches

First, check if the Torbrowser patches were updated since the last time we imported them (that's why we always record in debian/changelog the TorBrowser Git commit we are importing from).

If the Torbrowser patches were not updated, then just apply them and commit:

quilt push -a && git commit -a -m 'Apply Torbrowser patches.'

... then skip this entire stage.

Else, proceed with the following steps.

  • Make sure all quilt patches are applied.
  • Unapply all Torbrowser patches: use quilt pop as many times as needed.

  • Revert our changes (with --no-commit) to debian/patches/configure.patch if needed, and deapply it:

    cat debian/patches/configure.patch | patch -p1 --reverse

  • quilt delete the configure.patch if it exists.

  • Remove all Torbrowser patches from the series:

    quilt unapplied | grep --color=never '^torbrowser/' | xargs -n 1 quilt delete

  • Remove Torbrowser patches from Git:

    git rm -r debian/patches/torbrowser/

  • Commit:

    git commit -a -m 'Remove Torbrowser patches.'

  • Import the latest TorBrowser patches:

    • Ensure you have Mike Perry's latest stuff available:

        git remote add -f mikeperry https://git.torproject.org/user/mikeperry/tor-browser.git
  git remote add -f ttp https://git.torproject.org/tor-browser.git

    • Find the most recent commit in ttp/tor-browser-24.2.0esr-1 that is an import from Mozilla (see e.g. 5175d069); save its ID:

        export LAST_MOZILLA_COMMIT=XXX

    • Export the Torbrowser patches:

        TORBROWSER_PATCHES_DIR=$(mktemp -d)
  git format-patch -o "$TORBROWSER_PATCHES_DIR" \
      "$LAST_MOZILLA_COMMIT..ttp/tor-browser-24.2.0esr-1"

    • Remove from $TORBROWSER_PATCHES_DIR the patches we don't want. See debian/changelog for the - list of patches skipped last time, see the TorBrowser Git log to make your opinion about new or updated patches, use common sense. Take note of your decisions and its rationale, you will need it later.

    • Import the Torbrowser patches:

        for patch in $(\ls --reverse ${TORBROWSER_PATCHES_DIR}/*.patch) ; do
    p=$(basename "$patch")
    quilt import -P "torbrowser/$p" "$patch"
  done
  git add debian/patches/torbrowser debian/patches/series
  TORBROWSER_COMMIT=$(git rev-parse ttp/tor-browser-24.2.0esr-1)
  git commit -m "Import Torbrowser patches at commit ${TORBROWSER_COMMIT}."

  • Apply Torbrowser patches:

    quilt push -a && git add . && git reset HEAD .pc && git commit -a -m 'Apply Torbrowser patches.'

  • Update debian/tails.*.mozconfig:

    • copy all ac_add_options lines from .mozconfig into the Tor Browser's options section in debian/tails.common.mozconfig, but the ones that break the xulrunner build, that go into Tor Browser's options specific to the browser component section in debian/tails.browser.mozconfig instead
    • review the changes to these settings
    • if needed, update the debian/tails.common.mozconfig's Override Tor Browser's options section

  • Push to Git:

    git push origin tails/master && git push --tags

6. Build packages

Update debian/changelog

  • set a version such as 17.0.5esr-0+tails1, e.g.

    dch -v "${VERSION}-0+tails1"

  • list our changes, especially the TorBrowser commit at which the patches were imported, and the ones we skipped

  • set distribution to unstable

  • commit:

    git commit debian/changelog \
    -m "$(head -n 1 debian/changelog | sed -e 's,).*,),')"

Tag the release

DEB_VERSION=$(dpkg-parsechangelog -SVersion)
git tag -s -m "$(head -n 1 debian/changelog | sed -e 's,).*,),')" "debian/${DEB_VERSION}"

Clean up the source tree

git clean -fdx -e /.pc/

Build for unstable

If you have no available non-Tails setup to comfortably test these packages, then skip this step.

  • Build for unstable and the architecture you can test on (most likely amd64), e.g. using our Debian package builder.
    Note: if building locally in a ramdisk, it needs to be at least 14GB large.
  • Copy browser/app/profile/000-tor-browser.js into /etc/iceweasel/pref/ on the test system.
  • Install and test the resulting packages.

Build for squeeze-backports

  • Checkout the tails/squeeze branch and merge the tag:

    git checkout tails/squeeze && git merge "debian/${DEB_VERSION}"

  • Add a squeeze-backport changelog entry:

    dch --bpo

    Adjust as needed.

  • Commit:

    git commit debian/changelog \
    -m "$(head -n 1 debian/changelog | sed -e 's,).*,),')"

  • Tag the backport:

    git tag -s -m "$(head -n 1 debian/changelog | sed -e 's,).*,),')" \
    "debian/$(dpkg-parsechangelog -SVersion | sed -e 's,~,_,')"

  • Build for squeeze-backports and i386, e.g. using our Debian package builder.

  • Integrate these debs into your apt-cacher cache. That's one cp to /var/cache/apt-cacher-ng/_import/ away, + 1 click in the web interface.
  • Test the resulting packages in Tails.
  • Make sure the .orig.* tarballs are included in the .changes file. FIXME: check if that's needed, or done automatically by the above instructions.
  • Upload the resulting packages to the relevant suite of our APT repository.
  • Merge this APT suite where you need it: generally, that's devel, experimental, one of stable or testing, and maybe a release tag.

  • Push to Git:

    git push origin tails/squeeze && git push --tags

Import bundled preferences

  • Copy browser/app/profile/000-tor-browser.js from the tag the Squeeze backport was built from, into config/chroot_local-includes/etc/iceweasel/pref/.
  • Commit this to the branch that is being used to prepare the release: ideally, a topic branch that will be reviewed and merged; in practice, more likely this will be stable or testing.

7. Potential problems (and solutions)

Problems with ./configure

E.g. configure.patch does not apply, or the build fails since {js/src/,}configure.in was modified but {js/src/,}configure was not refreshed.

In a nutshell, the solution is to:

  1. Make sure the patches that modify {js/src/,}configure.in are applied (this is the case after a quilt push -a, that is during most of the steps documented above).

  2. Update {js/src/,}configure:

     sudo apt-get install autoconf2.13
 make -f client.mk configure

    The make command may fail due to missing dependencies. We don't care, as long as {js/src/,}configure have been refreshed.

  3. Replace configure.patch with the diff between the original and updated version of {js/src/,}configure:

     configure_diff=$(mktemp) && \
    git diff configure js/src/configure > "$configure_diff" && \
    git reset --hard && \
    git clean -fdx -e /.pc/ && \
    quilt import -f -P configure.patch "$configure_diff" && \
    git commit debian/patches -m "Refresh configure.patch." && \
    quilt push && \
    git commit -a -m 'Apply configure.patch.'

Note that configure.patch must always be the last patch in the quilt series file, after the TorBrowser ones.