Dead on Arrival: Recovering from Fatal Flaws in Email Encryption Tools

Authors: 

Juan Ramón Ponce Mauriés, University College London; Kat Krol, University of Cambridge; Simon Parkin, Ruba Abu-Salma, and M. Angela Sasse, University College London

Abstract: 

Background. Since Whitten and Tygar’s seminal study of PGP 5.0 in 1999, there have been continuing efforts to produce email encryption tools for adoption by a wider user base, where these efforts vary in how well they consider the usability and utility needs of prospective users.

Aim. We conducted a study aiming to assess the user experience of two open-source encryption software tools—Enigmail and Mailvelope.

Method. We carried out a three-part user study (installation, home use, and debrief) with two groups of users using either Enigmail or Mailvelope. Users had access to help during installation (installation guide and experimenter with domain-specific knowledge), and were set a primary task of organising a mock flash mob using encrypted emails in the course of a week.

Results. Participants struggled to install the tools—they would not have been able to complete installation without help. Even with help, setup time was around 40 minutes. Participants using Mailvelope failed to encrypt their initial emails due to usability problems. Participants said they were unlikely to continue using the tools after the study, indicating that their creators must also consider utility.

Conclusions. Through our mixed study approach, we conclude that Mailvelope and Enigmail had too many software quality and usability issues to be adopted by mainstream users. Methodologically, the study made us rethink the role of the experimenter as that of a helper assisting novice users with setting up a demanding technology.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {209362,
author = {Juan Ram{\'o}n Ponce Mauri{\'e}s and Kat Krol and Simon Parkin and Ruba Abu-Salma and M. Angela Sasse},
title = {Dead on Arrival: Recovering from Fatal Flaws in Email Encryption Tools},
booktitle = {The {LASER} Workshop: Learning from Authoritative Security Experiment Results ({LASER} 2017)},
year = {2017},
isbn = {978-1-931971-41-6},
pages = {49--57},
url = {https://www.usenix.org/conference/laser2017/presentation/mauries},
publisher = {{USENIX} Association},
}