Checking, nudging or scoring? Evaluating e-mail user security tools


Sarah Y. Zheng and Ingolf Becker, UCL


Phishing e-mail threats are increasing in sophistication. Technical measures alone do not fully prevent users from falling for them and common e-mail interfaces provide little support for users to check an e-mail's legitimacy. We designed three e-mail user security tools to improve phishing detection within a common e-mail interface and provide a formative evaluation of the usability of these features: two psychological nudges to alert users of suspicious e-mails and a "check" button to enable users to verify an email's legitimacy. Professional e-mail users (N=27) found the "suspicion score" nudge and "check" button the most useful. These alerted users of suspicious e-mails, without harming their productivity, and helped users assert trust in legitimate ones. The other nudge was too easily ignored or too disruptive to be effective. We also found that users arrive at erroneous judgements due to differing interpretations of e-mail details, even though two-thirds of them completed cybersecurity training before. These findings show that usable and therefore effective e-mail user security tools can be developed by leveraging cues of legitimacy that augment existing user behaviour, instead of emphasising technical security training.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {289480,
author = {Sarah Y. Zheng and Ingolf Becker},
title = {Checking, nudging or scoring? Evaluating e-mail user security tools},
booktitle = {Nineteenth Symposium on Usable Privacy and Security (SOUPS 2023)},
year = {2023},
isbn = {978-1-939133-36-6},
address = {Anaheim, CA},
pages = {57--76},
url = {},
publisher = {USENIX Association},
month = aug