Why Is Our Security Research Failing? Five Practices to Change!

Note: Presentation times are in Pacific Standard Time (PST).

Wednesday, January 25, 2023 - 2:40 pm3:10 pm

Marcus Botacin, Texas A&M University


Everybody complains about the current cybersecurity state. Regardless of the reason, nobody is fully satisfied. I don't think "security is broken". In fact, we have "more" security nowadays than at any time in the past. However, it is true that the field has a lot to progress. Therefore, in this talk, I assume the methodological position that security research is failing and I present my discoveries when trying to understand which aspects of security could have been failing and thus might be enhanced. I support my investigation with results from a published systematic literature review of 400+ papers from the last 20 years of published malware research in the most reputable venues. I identified more than 20 challenges and pitfalls in security research and categorized them into 5 high-level categories that will be discussed in this talk: (i) the lack of diversity in study types; (ii) researchers not looking to the market and industry when needed; (iii) researchers focus too much on the industry and market; (iv) the lack of guidelines in the field; and (v) the reproducibility crisis that cybersecurity and almost all fields face. I close the talk with suggestions that one might adopt to mitigate these problems. My recommendations are divided according to the multiple stakeholders in the field and they range from (i) researchers developing more longitudinal studies with representative populations; to (ii) the field establishing more guidelines for experiment development; and (iii) venues clearly welcoming diversified study types.

Marcus Botacin, UFPR/TAMU

Marcus is a Computer Science Assistant Professor at Texas A&M University (TAMU). Marcus holds a Computer Science PhD (Federal University of Paraná, Brazil, 2021), Master in Computer Science (University of Campinas, Brazil, 2017), and a Computer Engineering Bachelor (University of Campinas, Brazil, 2015). His main research interests are malware analysis, reverse engineering, and the science of security. Marcus’ master dissertation was awarded by the Brazilian Computer Society (SBC) the best security research work developed in Brazil in 2017. Marcus published papers in top-venues (ACM TOPS, DIMVA, IEEE TDSC, and others) and he is currently a PC member for USENIX Security 2022 and 2023. Marcus was also awarded a student grant to attend USENIX Enigma 2019 and he was a USENIX Enigma 2021 speaker.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@conference {285627,
author = {Marcus Botacin},
title = {Why Is Our Security Research Failing? Five Practices to Change!},
year = {2023},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = jan

Presentation Video