Linux dependencies

The current Tails Installer version (https://git-tails.immerda.ch/liveusb-creator/) has considerable changes when compared with the upstream Fedora liveusb-creator (https://git.fedorahosted.org/cgit/liveusb-creator.git).

The current package dependencies for the Tails Installer in Linux are:

  • dosfstools
  • gdisk
  • genisoimage
  • gir1.2-glib-2.0
  • gir1.2-gtk-3.0
  • gir1.2-udisks-2.0
  • mtools
  • p7zip-full
  • policykit-1
  • python-configobj
  • python-gi
  • python-urlgrabber
  • syslinux

If we list the set of requirements for each important source file then we have:

__init__.py

import gettext

if sys.platform == 'win32':
    import gettext_windows
    gettext_windows.setup_env()

creator.py

if 'linux' in sys.platform:
    import gi
    gi.require_version('UDisks', '2.0')
    from gi.repository import UDisks, GLib

Commands:
* syslinux
* sgdisk
* dd
* dosfslabel
* e2label
* extlinux
* pkexec
* mkdiskimage
* sync

gui.py

from gi.repository import Gdk, GLib, Gtk
urlgrabber

In general GTK3

launcher.py

from gi.repository import Gtk

utils.py

if 'linux' in sys.platform:
    from gi.repository import GLib

Alternatives for Windows:

The Windows specific code for the Tails Installer uses mostly the Python win32 interfaces:

import win32file, win32api, pywintypes

and a set of third parties tools listed here:

https://git-tails.immerda.ch/liveusb-creator/tree/tools

There are other tools that would be possible to explore like #10984.

Analysis regarding operations on storage devices

According to https://tails.boum.org/contribute/design/installation/, the steps to create a Tails bootable device are:

  1. Partition the device as a GPT partition.
  2. Create a FAT32 (VFAT) partition with the Tails files and Tails label.
  3. (Optionally, not supported yet) Create a persistence partition with LUKS.

Next we analyze the code related to above steps. First we list the current Linux specific operations carried out in the target installation device and then we list the current and proposed Windows alternatives.

For Linux - LinuxTailsInstallerCreator/creator.py:

detect_supported_drives:

self._udisksclient.get_object_manager().get_objects()
partition = obj.props.partition
filesystem = obj.props.filesystem
self._udisksclient.get_drive_for_block(block)
self._udisksclient.get_partition_table(partition)

Add to that a bunch of drive.props.* and block.props.*.

mount_device:

self._get_object().props.filesystem
mount = filesystem.call_mount_sync(
  arg_options = GLib.Variant('a{sv}', None),
  cancellable = None)

unmount_device:

filesystem = self._get_object(udi).props.filesystem
filesystem.call_unmount_sync(
   arg_options=GLib.Variant('a{sv}', None),
   cancellable=None)

partition_device:

block.call_format_sync(
    'gpt',
    arg_options=GLib.Variant('a{sv}', None),
     cancellable=None)
partition_table.call_create_partition_sync(
                arg_offset=0,
                arg_size=self.system_partition_size * 2**20,
                arg_type=ESP_GUID,
                arg_name=self.label,
                arg_options=GLib.Variant('a{sv}', None),
                cancellable=None)
system_partition.call_set_type_sync(ESP_GUID, GLib.Variant('a{sv}', None))
system_partition.call_set_name_sync(self.label, GLib.Variant('a{sv}', None))
self._set_partition_flags(system_partition, SYSTEM_PARTITION_FLAGS)

update_system_partition_properties:

system_partition.call_set_type_sync(ESP_GUID, GLib.Variant('a{sv}', None))
system_partition.call_set_name_sync(self.label, GLib.Variant('a{sv}', None))
self._set_partition_flags(system_partition, SYSTEM_PARTITION_FLAGS)

verify_filesystem:

self.popen('/sbin/dosfslabel %s %s' % (
                               self.drive['device'], self.label))
self.popen('/sbin/e2label %s %s' % (self.drive['device'],
                                                    self.label))

install_bootloader:

self.popen('/usr/bin/pkexec /usr/bin/syslinux %s -d syslinux %s' % (
                ' '.join(self.syslinux_options()),
                self.drive['device']))

initialize_zip_geometry:

self.popen('/usr/lib/syslinux/mkdiskimage -4 %s 0 %d %d' % (
               self._drive[:-1], heads, cylinders))

format_device:

block.call_format_sync(
            'vfat',
            arg_options=GLib.Variant(
                'a{sv}',
                {'label': GLib.Variant('s', self.label),
                 'update-partition-type': GLib.Variant('s', 'FALSE')}))
self._get_object().props.block.call_rescan_sync(GLib.Variant('a{sv}', None))

For Windows - WindowsTailsInstallerCreator/creator.py:

detect_supported_drives:

win32file, win32api, pywintypes
win32file.GetDriveType(drive) == win32file.DRIVE_REMOVABLE
win32api.GetVolumeInformation(drive)

mount_device:

XXX: ?

self.dest = self.drive['mount']

unmount_device:

pass

We might use this: https://technet.microsoft.com/en-us/library/cc772586.aspx

partition_device:

Not implemented.

Here what we need is to perform a GPT partition on the target device. The only realistic option we could find was to use the gdisk tools (http://www.rodsbooks.com/gdisk/). This library provides two utilities to create GPT partitions. The first is gdisk.exe a text mode interactive application and the second is sgdisk.exe, which works by command line. If we want to use sgdisk.exe which looks like the best option, we should compile it by ourselves, because no binary is provided. The compilation requires gnuwin32 popt library (http://gnuwin32.sourceforge.net/packages/popt.htm) which looks old and might only work up to Windows XP. On the other hand, the only thing popt does is parsing command line parameters, so in theory it should not be so difficult to run it in other Windows versions, but I could not find more information. The only alternative would be to use directly the WinAPI but it could also require a good amount of work.

update_system_partition_properties:

cmd = (  [ '/sbin/sgdisk' ]
           + [ '--typecode=1:%s' % ESP_GUID ]
           + [ self.drive['parent'] ])
    self.popen(cmd, shell=False)

Currently it uses sgdisk but we need to decide on using it or not.

verify_filesystem:

win32api, win32file, pywintypes
win32file.SetVolumeLabel(self.drive['device'], self.label)

install_bootloader:

self.popen('syslinux %s -m -a -d %s %s' %  (
            ' '.join(self.syslinux_options()), 'syslinux', device))

Ref: https://www.kernel.org/pub/linux/utils/boot/syslinux/

initialize_zip_geometry:

self.popen('/usr/lib/syslinux/mkdiskimage -4 %s 0 %d %d' % (
               self._drive[:-1], heads, cylinders))

format_device:

self.popen('format /Q /X /y /V:Fedora /FS:FAT32 %s' % self.drive['device'])

Looks broken

Persistent partition:

This is managed by the persistence-setup https://git-tails.immerda.ch/persistence-setup so it is beyond the scope of the current research.

PyGI Windows executable

I managed to create a native windows executable for a test Python/GI program under Windows 8.1

Downloads

Prepare

You need a Windows 8 (virtual) machine, with the files downloaded above

  • in Windows double-click python-3.4.3 and accept default choices but select "Install entire feature"
  • in Windows double-click pygi-aio-3.14.0-rev18-setup and choose to install only
    • GNOME libraries: GI, GTK, Pango (needed?)
    • non-GNOME libraraies: none
    • developpment files: GIR (needed?)
  • in Windows double-click cx_freeze-4.3.4.win32-py34 and accept default choices

Test Program

main.py

import gi
from gi.repository import Gtk

dialog = Gtk.MessageDialog(None, Gtk.DialogFlags.DESTROY_WITH_PARENT, Gtk.MessageType.INFO, Gtk.ButtonsType.CLOSE, "Hello!")
dialog.run()
dialog.destroy()

Gtk.main()

setup.py

Result:

import os, site, sys
from cx_Freeze import setup, Executable

## Get the site-package folder, not everybody will install
## Python into C:\PythonXX
site_dir = site.getsitepackages()[1]
include_dll_path = os.path.join(site_dir, "gnome")

## Collect the list of missing dll when cx_freeze builds the app
## This list should be updated with the output of ListDLLs.exe
missing_dll = ['libffi-6.dll',
               'libgirepository-1.0-1.dll',
               'libglib-2.0-0.dll',
               'libgobject-2.0-0.dll',
               'libgio-2.0-0.dll',
               'libgmodule-2.0-0.dll',
               'libintl-8.dll',
               'libzzz.dll',
               'libwinpthread-1.dll',
               'libgtk-3-0.dll',
               'libgdk-3-0.dll',
               'libatk-1.0-0.dll',
               'libcairo-gobject-2.dll',
               'libgdk_pixbuf-2.0-0.dll',
               'libpango-1.0-0.dll',
               'libpangocairo-1.0-0.dll',
               'libpangowin32-1.0-0.dll',
               'libfontconfig-1.dll',
               'libfreetype-6.dll',
               'libpng16-16.dll',
               'libjasper-1.dll',
               'libjpeg-8.dll',
               'librsvg-2-2.dll',
               'libpangoft2-1.0-0.dll',
               'libwebp-5.dll',
               'libpangoft2-1.0-0.dll',
               'libxmlxpat.dll',
               'libharfbuzz-gobject-0.dll'
]

## We also need to add the glade folder, cx_freeze will walk
## into it and copy all the necessary files
glade_folder = 'glade'

## We need to add all the libraries too (for themes, etc..)
gtk_libs = ['etc', 'lib', 'share']

## Create the list of includes as cx_freeze likes
include_files = []
for dll in missing_dll:
    include_files.append((os.path.join(include_dll_path, dll), dll))

## Let's add glade folder and files
include_files.append((glade_folder, glade_folder))

## Let's add gtk libraries folders and files
for lib in gtk_libs:
    include_files.append((os.path.join(include_dll_path, lib), lib))

base = None

## Lets not open the console while running the app
if sys.platform == "win32":
    base = "Win32GUI" 

executables = [
    Executable("main.py",
               base=base
    )
]

buildOptions = dict(
    compressed = True,
    includes = ["gi"],
    packages = ["gi"],
    include_files = include_files
    )

setup(
    name = "test_gtk3_app",
    author = "Alan",
    version = "1.0",
    description = "GTK 3 test",
    options = dict(build_exe = buildOptions),
    executables = executables
)

Build

  • start "Command Prompt"
  • cd Documents/test/
  • python setup.py bdist. The resulting ZIP file lives in dist/. It is 37M big!
  • python setyp.py bdist_msi created a windows installer in dist/.

Test

In another fresh windows 8.1VM

Next steps

Other tools

Conclusion

We have outlined the main requirements we should face on fully porting the Tails Installer to Windows. The major differences in the current Tails version regarding the former upstream Fedora version, are the usage of the Python interface for GTK3 (PyGI) and the udisks2 library for disk operations.

We have found that there are alternative libraries we could use on Windows, in order to perform the Linux specific operations. Some of them seem currently maintained and updated, for instance, PyGI for Windows and extlinux. Other like sgdisk and its dependencies might need some custom maintenance, which could rise the amount of required effort. However, even with the extra required work, it seems totally likely that we can successfully port Tails installer to Windows.

If our objective is to ease Tails adoption for Windows users, especially people that never have used Linux, then this project may be worth the effort. The question then becomes: is this the cheapest and/or best way to ease Tails adoption for Windows users?