This reports covers the activity of Tails in November 2015.

Everything in this report can be made public.

B. Improve our quality assurance process

B.1. Automatically build ISO images for all the branches of our source code that are under active development

In November, 611 ISOs images were automatically built on our Jenkins setup.

B.2. Continuously run our entire test suite on all those ISO images once they are built

In November, 314 ISOs were tested by our Jenkins instance.

  • B.2.4. Implement and deploy the best solution from this research

    With the 1.7 release, we were able to unify back the way the branches based on our maintenance branch and the ones based on our development branch were tested. (#10409)

    We have deployed the feature that prints out in the test suite logs in Jenkins a direct link to the collected artifacts of the failing step. (#10445)

    We found out that Sikuli was sometimes leaving PNG files in the Jenkins workspaces, and we worked on a fix. (#10476)

    We are being short again in disk space and need to anticipate that future infrastructure features will require more, so we're sorting out what's the real needs of our isotesters on this side, now that we have more data points. (#10396)

    We are also short on isotesters to really test every single automatically built ISO image. So we're discussing if we need new hardware to cope with the load, and if so, which kind. (#9264)

    Meanwhile, our automated test suite will still be a bit more efficient on testing branches that actually change Tails' code, while testing only some subset for the branches that only change our website or documentation. (#10492)

    We spent a lot of the CI time gathering statistics about the test suite runs in Jenkins in order to help the team writing the tests to identify which scenarios were fragile, going forward to the release of the notifications to contributors.

    We discovered that lately our isostesters were not always getting back online after being rebooted and are investigating why. (#10601)

B.3. Extend the coverage of our test suite

  • First of all, it should be noted that our automated test suite has been key to identify regressions introduced by porting Tails to Debian Jessie. A number of such bugs have been found early, that we would otherwise not have discovered before the first release candidate of Tails based or Jessie at best, and quite possibly after the first actual release of Tails based on Jessie.

  • B.3.11. Fix newly identified issues to make our test suite more robust and faster

    • Increased the number of Tor circuit retries in the test suite (#10375).

    • Work is in progress to make some test cases, that cause false positives, more robust: SSH (#10498), Git (#10444), Windows Camouflage (#10493), Seahorse (#9095, #10500, #10501).

    • More such fragile test cases were identified, and are momentarily disabled in our Jenkins environment. They will be worked on later: Electrum tests (#10475) due to us shipping a too old version of Electrum, connecting to the #i2p IRC channel (#10474), Synaptic (#10441), APT (#10496), ICMP filtering (#10499), whois (#10523), wait_until_tor_is_working helper (#10497), time synchronization (#10440, #10494, #10495), Pidgin connecting to OFTC (#10443), watching a WebM video over HTTPS (#10442).

C. Scale our infrastructure

C.2. Be able to detect within hours failures and malfunction on our services

  • C.2.1. Research and decide what monitoring solution to use what tools and abstraction layer to use for configuring it, and where to host it.

    We researched more deeply how the different candidates were respecting our specifications, especially about the trust that we will put into the monitoring machine.

C.4. Maintain our already existing services

We kept on answering the requests from the community as well as taking as well as taking care of security updates as covered by "C.4.4. Administer our services upto milestone IV" until the end of November.

We settled on a date for a sysadmin sprint in December.

D. Migration to Debian Jessie

D.1. Adjust to the change of desktop environment to GNOME Shell

We finished porting Tails to GNOME Shell by fixing the remaining identified regression: raw HTML was displayed in desktop notifications (#7989).

D.2. Take advantage of systemd to improve the internals of Tails

The Tor daemon is now managed with systemd, that completes supervision of critical services (#5750).

We also polished code that was previously ported to systemd:

  • Introduced a dedicated systemd target for "Tor has bootstrapped" state (#9393).
  • Call out to the shell less often in htpdate.service (#10612).
  • Converted the 21-gdm_unit_file hook to a drop-in override (#10606).

D.3. Update our test suite for Tails Jessie

This is about ticket #7563 and subtasks. Good progress was made on this front in November.

These were resolved:

  • Updated the memory erasure automated tests for Jessie (#9705, #8317); in passing, made it more robust and accurate.
  • Disabled screen blanking during the test suite, since it broke some tests on Jessie (#10403); and in passing, we have refactored how we re-use GNOME/X's environment from scripts in our codebase.
  • Fixed the "I open the address" step by pasting URLs instead of typing them char by char (#10467).
  • Updated the "Connect to server" tests for Jessie (#10325).
  • Adapted GNOME notification handling (#8782) and applications menu handling (#9706) for Jessie in the test suite.
  • Ported tests that used syslog to instead use the systemd Journal.
  • Updated Icedove tests for Jessie.

Additionally, we made progress on:

  • With respect to "iptables_parse is buggy for IPv6" (#9704): sadly, we could not find an iptables rules parser in Ruby (to nicely integrate into our test suite's codebase) that clearly states that it's compatible with IPv6. So we started looking for candidate solutions in Python and Perl.
  • The GnuPG keyserver interaction test cases are being updated for Jessie (#9791).
  • Updating tests for: USB installation and upgrade, Tor Browser tests, ICMP filtering.

D.5.1. Release an official version of Tails based on Jessie

Our two usual release managers also did a full code review of the changes introduced on our Jessie branch so far. No serious problem was identified, but lots of small polishing tasks have been added.

Based on these good results and the ones we see in our partly ported automated test suite, we decided a release schedule for Tails Jessie which will be released on January 26 and be numbered 2.0. We renamed accordingly the versions coming after 2.0. Here is the new version numbers of the releases mentioned in past reports and budgets:

  • Tails 1.9 has been renamed Tails 2.0
  • Tails 1.10 has been renamed Tails 2.2 (the second major version in a row)
  • Tails 1.11 has been renamed Tails 2.3
  • Tails 1.12 has been renamed Tails 2.4
  • Tails 1.13 has been renamed Tails 2.5

Various porting to Jessie

A lot of other porting to Jessie work was done, that does not really fit into any of the above categories:

  • Merged and forward-ported changes introduced up to Tails 1.7.
  • Ported our automatic (incremental) upgrade system to Jessie (#8083), and made its check for free memory smarter (#8263, #10540).
  • Fixed synchronizing OpenPGP keys in Seahorse (#9792).
  • Fixed Totem access to the network (#10593, #10603).
  • Ensured we use a current version of obfs4proxy (#10605).
  • Made AppArmor profile for cupsd work in read-only persistence mode (#10591).
  • Fixed string encoding regression that was introduced when porting WhisperBack to Python 3 (#10577).
  • Restored association of application/pgp-keys MIME type with Seahorse's "Import Key" application (#10571).
  • Restored AppArmor confinement of Tor on Jessie (#10528) that briefly disappeared with the upgrade to Tor 0.2.7.
  • Adjusted to the fact that .xsession-errors does not exist anymore on Jessie (#9966).
  • Restored unmuting and sanitizing ALSA mixer levels at boot time (#7591).
  • Wrote a test case to ensure that NetworkManager change of behaviour wrt. watching configuration files does not affect us (#7966).

And some code was cleaned up in passing:

  • Removed obsolete bits of APT pinning (#10607).
  • Removed obsolete workaround for the HashType enum in Tails Upgrader (#7042).
  • Dropped custom syslinux package with patches for Chromebooks, that we had included in a Debian Jessie update (#9292).

We researched some candidate changed and investigated a few potential problems:

  • Made sure the services we disable at boot time indeed don't start on Jessie (#8313).
  • Considered replacing some of our code with the GDM to GNOME session keyboard layout pass-through (#8713).
  • Clarified ttdnsd.service (#10608) and laptop-mode.service (#9967) situation on Jessie.
  • Looked for screen reader support regressions compared to Wheezy (#8371).
  • Made sure that we have no regression wrt. logging into a text console on Jessie (#9408).
  • Check if we need to keep memlockd running longer (#8260).
  • Made sure that GNOME 3.14's captive portal handling is disabled on Tails/Jessie (#10526).
  • Evaluated the Caribou virtual keyboard in Tails/Jessie (#8281).
  • Considered replacing sound-juicer with goobox (#8393).

E. Release management

  • Tails 1.7 was released on 2015-11-03:

    • Add an offline mode to disable all networking.
    • Add Icedove as a technology preview.
    • Improve the wording of the first screen of Tails Installer.
    • Upgrade Tor Browser to version 5.0.4 (based on Firefox 38.4.0 ESR).
    • Update Tor to 0.2.7.4.
    • Restart Tor automatically if connecting to the Tor network takes too long. (#9516)
    • Prevent wget from leaking the IP address when using the FTP protocol. (#10364)
    • Prevent symlink attack on ~/.xsession-errors via tails-debugging-info.
    • Force synchronization of data on the USB stick at the end of automatic upgrades.
    • Make the "I2P is ready" notification more reliable.