Why Is Our Security Research Failing? Five Practices to Change!

Everybody complains about the current cybersecurity state. Regardless of the reason, nobody is fully satisfied. I don't think "security is broken". In fact, we have "more" security nowadays than at any time in the past. However, it is true that the field has a lot to progress. Therefore, in this talk, I assume the methodological position that security research is failing and I present my discoveries when trying to understand which aspects of security could have been failing and thus might be enhanced. I support my investigation with results from a published systematic literature review of 400+ papers from the last 20 years of published malware research in the most reputable venues. I identified more than 20 challenges and pitfalls in security research and categorized them into 5 high-level categories that will be discussed in this talk: (i) the lack of diversity in study types; (ii) researchers not looking to the market and industry when needed; (iii) researchers focus too much on the industry and market; (iv) the lack of guidelines in the field; and (v) the reproducibility crisis that cybersecurity and almost all fields face. I close the talk with suggestions that one might adopt to mitigate these problems. My recommendations are divided according to the multiple stakeholders in the field and they range from (i) researchers developing more longitudinal studies with representative populations; to (ii) the field establishing more guidelines for experiment development; and (iii) venues clearly welcoming diversified study types.