“To Do This Properly, You Need More Resources”: The Hidden Costs of Introducing Simulated Phishing Campaigns


Lina Brunken, Annalina Buckmann, Jonas Hielscher, and M. Angela Sasse, Ruhr University Bochum


Many organizations use phishing simulation campaigns to raise and measure their employees' security awareness. They can create their own campaigns, or buy phishing-as-a-service from commercial providers; however, the evaluations of the effectiveness in reducing the vulnerability to such attacks have produced mixed results. Recently, researchers have pointed out "hidden costs" - such as reduced productivity and employee trust. What has not been investigated is the cost involved in preparing an organization for a simulated phishing campaign. We present the first case study of an organization going through the process of selecting and purchasing a phishing simulation. We document and analyze the effort of different stakeholders involved, and present reflection from semi-structured interviews with 6 key actors at the end of the procurement process. Our data analysis shows that procuring such simulations can require significant effort from different stakeholders - in our case, at least 50,000€ in person hours - and many hidden intangible costs. Evaluating if a product or service meets training requirements, is acceptable to employees, and preparing the technical infrastructure and operational processes for running such a product all require significant time and effort. The prevailing perception that phishing simulation campaigns are a quick and low-cost solution to providing security training to employees thus needs to be challenged.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {291166,
author = {Lina Brunken and Annalina Buckmann and Jonas Hielscher and M. Angela Sasse},
title = {{{\textquotedblleft}To} Do This Properly, You Need More {Resources{\textquotedblright}}: The Hidden Costs of Introducing Simulated Phishing Campaigns},
booktitle = {32nd USENIX Security Symposium (USENIX Security 23)},
year = {2023},
isbn = {978-1-939133-37-3},
address = {Anaheim, CA},
pages = {4105--4122},
url = {https://www.usenix.org/conference/usenixsecurity23/presentation/brunken},
publisher = {USENIX Association},
month = aug

Presentation Video