We want to allow our users to backup the data they have in their persistent volume.

See the thread on tails-dev:

https://mailman.boum.org/pipermail/tails-dev/2014-April/005388.html

Roadmap

  • Research which requirements are really needed by our users
  • Evaluate existing tools based on these requirements
  • Implement the missing features, integrate into Tails
  • Write end-user documentation
  • Write automated tests
  • Evaluate UX

Requirements

MUST

  • Allow backup of Persistent folder
  • Allow backup on Local storage
  • Encrypt backups
  • Preserve permissions and ownership
  • Understandable error messages
  • Require administrator password when hitting a file not readable by amnesia (eg. private key of onion services).

SHOULD

  • Allow backup on a remote storage
    • ownCloud
  • Feature a backup history
  • Backing up everything in persistence (eg APT)
  • Only send the delta from previous backup (i.e incremental update)
  • Automated reminder
  • Include, exclude lists

Backup techniques

We shouldn't maintain yet another shiny new piece of code to do that. Or we might hit tons of corner cases to handle, that one initially does not think of. And once we take it all into account, then we get a large piece of code to maintain all by ourselves, because it was meant only for Tails' needs.

So, we shouldn't try to reinvent this wheel, and use an existing, proven solution instead.

GNOME Backups (Deja Dup)

Pros:

  • Has a simple interface for configuring backups. The nicest we could find and with a clear mission statment.
  • Restoration of single files or folders is integrated in Files.
  • Notifications to start backups periodically.
  • Supports different kinds of remote: FTP, SSH, WebDAV, and Windows share are displayed in the graphical interface, duplicity supports many more.

Cons:

  • Uses duplicity as back end with the following drawbacks:
    • It forces full backups now and then, making them unexpectedly extremelly slower than regular incremental ones.
    • Restoring a single file is slow.
    • Impossible to navigate the file system, either of the last or past increments without doing a full restoration.
  • No feedback whatsoever on the progress of the backup when they get started automatically.
  • Doesn't know how to ask for root password for backing up files it cannot access.
  • If files are mounted or linked in a different place, you can only restore from Nautilus the original places backed up by Deja Dup. That's a problem for dotfiles which are single files symlinked to $HOME and you can't easily get both the Nautilus integration and the full restoration.
  • You can only add folders to be backed up or ignored by Deja-Dup (and not single files).
  • Fails to restore symlinks.

See #9888.

Tar + GPG

A simple approach might be to combine tar and GPG:

tar cjf - . | gpg --cipher-algo AES -c - > /home/amnesia/YYYY-MM-DD-backup.tbz2.gpg

Initial implementation might be easy but that will probably end up being quite a big piece of custom code.

Duplicity

http://duplicity.nongnu.org/

It supports something that's basically "tar | gpg" for the first iteration, and it also leaves room for future improvements, thanks to its support for incremental and remote backups. Also, it allows users to restore or inspect their backups outside of Tails, without having to manually decipher yet another backup file format.

Cons

  • Duplicity creates tons of messy files on the file system.
  • And requires users to do a full backup from time to time when using incremental backups

Loopback LUKS

Pros

  • One file per backup.

Cons

  • We still need to find another tool to create the device and copy the files.
  • Maybe backups done this way would be much bigger than duplicity backups.
  • Duplicity supports incremental backups even if they have some limitations.

Obnam

Obnam only supports encrypting to a GnuPG key, so this would require another layer of encryption (such as a LUKS container file, or something else) if we want to support encrypting with a passphrase (and I think we should).

grsync + encrypted device

grsync is a GUI for the renown and rock-solid rsync, already shipped in Tails.

Grsync is packaged for debian.

The documentation for the creation of the encrypted device is already written.

It has a rock-science interface, basically displaying the complexity of the command line options in a graphical way.

How to test?

  • create an encrypted device.
  • install the grsync package.
  • paste those lines in a .grsync file, then double-click on it. (grsync ask you first to confirm the profile you want to use. Just click on "Open")
  • define the destination (i.e your encrypted device)
  • test wildly! and please report your results

Pros

  • not that much things to code
  • grsync can be easily preconfigurated, eventually with multiple profiles
  • this solution separates backup and encryption work
  • allows remote backups

Features to request

  • grsync should check if enough space is available on the destination before running. Update: rsync 3.1.0 introduces a --preallocate option. (Tails actually ships rsync 3.0.9 Tails 2.6 ships rsync 3.1.1)
  • grsync should ask for confirmation when the option "Delete on the destination" is activated
  • when user double-click on a .grsync file, a window appears to confirm which file to open. This may be confusing.

Misc

  • some files are normally not readable by rsync (for example persistence.conf, apt/*) Grsync can bypass that with the option "Run as superuser", we should investigate the consequences of using such an option. We still have the possibility to ignore those files: we just have then to add --exclude="apt" in the preconfiguration file.
  • decide if we activate the option --delete by default.
  • test restoration (see File → Invert source and destination). Then, at least, check files permissions.
  • test backup + restoration with symlinks and hardlinks in the Persistent folder.
  • eventually test remote backups.
  • see the thread on tails-dev

rdup

https://github.com/miekg/rdup

rdup separates the logic of backing up from the actual copying. It supports filters to compress and encrypt individual files (typically using gpg or mcrypt) as well as path names and can do both full as well as incremental backups.

Pros

  • simple and small command line programs
  • more sophisticated than tar+gpg and probably addresses many of the corner cases that would otherwise have to be handled by increasingly complicated scripts
  • in Debian Squeeze / Wheezy / testing

Cons

  • still requires a script of some sort to drive it
  • probably requires a gui to make it simple to use

borgbackup

https://borgbackup.readthedocs.io/en/stable/index.html

Borg is the perfect backup back end. It supports increments, encryption, data deduplication, local and remote backups, and mounting backups as FUSE file systems. And it way faster than obnam which advertises similar properties. But it doesn't have a graphical user interface.

Packages for borgbackup are in Jessie Backports and in Strech

restic

https://restic.github.io/

Ristic looks very similar to borgbackup. It is a small CLI tool for incremental, authenticated, and confidential backups of files.

It is not clear where the tools differ and it would be nice to have a comparison of both tools.

Packages are for restic are in Strech.

Clone everything

We could add to Tails Installer an option to Clone everything (Tails partition and persistent volume). This would clone the original USB stick and also backup all the persistent data.

We could encourage people to submit patches for this even if it's not a perfect solution. The back end could be quite simple and use rsync between the two persistent volumes.

A possible improvement would be to skip the cloning of the Tails partition if the destination USB is already a Tails USB stick.

Pros

  • That's a very simple user interface: it's only possible to do full backups. If you loose your data, use the backup USB stick. We don't have to provide an option to restore.

Cons

  • No option to do incremental backups.
  • People need a dedicated USB stick.

Duplicati

  • Mono application (.NET).
  • Unofficial Debian package. I found no reference to "duplicati" on bugs.debian.org.
  • The client side has both:
    • a command-line interface
    • a web server interface running on localhost:8200: https://duplicati.readthedocs.io/en/latest/03-using-the-graphical-user-interface/. I could pierce a hole in our firewall to access this from Firefox ESR in Tails but then I couldn't make it connect to an SSH server. I'm not sure how this would integrate with Tails in general but it doesn't seem the right user interface for our users.
  • Backend supports for many types of remote storage: https://duplicati.readthedocs.io/en/latest/05-storage-providers/.
  • Backend supports incremental backups and deduplication.
  • I didn't manage to make the SSH backend work from the command line (tried torsocks) but while doing so I can see in Onion Circuits that Duplicati is calling back home in several ways: checking for update and reporting usage stats.

Other solutions

  • sbackup, Simple Backup: unmaintained since 2008.

  • Lucky Backup: seems very oldish and not really active.

  • Back In Time which has a GNOME frontend. It does snapshots with hardlinks to reduce space. Can do local and SSH as remote.

spideroak

SpiderOak is a commercial online encrypted backup service. Reading their website might be useful for the user research part of this project.

User experience

  • The user should be prompted about where to save the backup as this will most probably end up directly on a storage device and not go through RAM.

  • Ask the user which folders of the persistence setup to backup. But maybe that's not needed for a first prototype, as we might assume that the info in persistence is kept to the minimum.

  • The restore step would do the current delete partition steps (if there was already a persistence partition found), then most of the current steps to create a partition, but instead of asking the user what they want to use their new partition for, it would instead ask the user to pick a backup and would restore it.

  • There could be a single menu entry called "Backup and restore persistent volume". Combining both actions in a single menu entry makes it clear that they are directly related, and limits the number of menu entries.

  • There should be a graphical way to restore backups. It is important let non-techy users restore their backups if necessary.

Implementation

  • Avoid running the GUI as root and rather run the GUI as amnesia or a dedicated user and execute precise actions through policykit (execute a command with pkexec or implement a dbus service)

  • Write the code in GTK3 for Tails 1.1 which will be the next major release, based on Debian Wheezy.

Script by a2