CSET '20 Workshop Program

Papers are available for download below to registered attendees now and to everyone beginning Monday, August 10, 2020. Paper abstracts are available to everyone now. Copyright to the individual works is retained by the author[s].

All the times listed below are in Pacific Daylight Time (PDT).

Downloads for Registered Attendees
(Sign in to your USENIX account to download these files.)

Attendee Files 
CSET '20 Paper Archive (ZIP)
CSET '20 Attendee List (PDF)

Monday, August 10

7:00 am–7:15 am

Opening Remarks

Program Co-Chairs: Tamara Denning, University of Utah, and Tyler Moore, University of Tulsa

7:15 am–8:15 am

Industrial Control Systems

HAI 1.0: HIL-based Augmented ICS Security Dataset

Hyeok-Ki Shin, Woomyo Lee, Jeong-Han Yun, and HyoungChun Kim, The Affiliated Institute of ETRI

Short Extended Work Paper

Available Media

Datasets are paramount to the development of AI-based technologies. However, the available cyber-physical system (CPS) datasets are insufficient. In this paper, we introduce HAI dataset 1.0 (https://github.com/icsdataset/hai), the first CPS dataset collected using the HAI testbed. The HAI testbed comprises three physical control systems, namely a GE turbine, Emerson boiler, and FESTO water treatment systems, combined through a dSPACE hardware-in-the-loop (HIL) simulator. We built an environment to remotely and automatically manipulate all components of a feedback control loop. Using this environment, we collected HAI dataset 1.0 while repeatedly running a large number of benign and malicious scenarios for a long period with minimal human effort. We will continue to improve the HAI testbed and release new versions of the HAI dataset.

Expansion of ICS Testbed for Security Validation based on MITRE ATT&CK Techniques

Seungoh Choi, Jongwon Choi, Jeong-Han Yun, Byung-Gil Min, and HyoungChun Kim, The Affiliated Institute of ETRI

Long Preliminary Work Paper

Available Media

To respond to cyber threats, all systems in an industrial control system (ICS) should be comprehensively monitored and analyzed. However, there is no dataset to perform this integrated monitoring and analysis study. In previous research, the testbed and dataset represented only one specific area, such as the network or physical level. This imposes limitations upon the testing, validating, and user training of the integrated monitoring system. Therefore, we are developing datasets to test systems that integrate and monitor the ICS operated in a wide range of areas. In this paper, we introduce a method to expand the existing testbed so that information can be collected and monitored during an ICS attack based on the MITRE ATT&CK framework. In addition, to create a dataset for simulating large-scale and long-term attack scenarios, a security dataset enrichment tool is proposed.

ICS Testbed Tetris: Practical Building Blocks Towards a Cyber Security Resource

Benjamin Green, Richard Derbyshire, William Knowles, James Boorman, Pierre Ciholas, Daniel Prince, and David Hutchison, Lancaster University, UK

Long Preliminary Work Paper

Available Media

Cyber attacks on Critical National Infrastructure (CNI) can be hugely detrimental to society, notably via compromising Industrial Control Systems (ICS) that underpin core CNI functions. In order to explore in-depth ICS Cyber Security challenges, testbeds are an essential tool, avoiding the need to experiment exclusively on live systems. However, ICS testbed creation is a complex multidisciplinary challenge, with a plethora of conflicting requirements. This paper, based on over six years of ICS testbed research and development that spans multiple diverse applications, proposes a flexible high-level model that can be adopted to support ICS testbed development. This is complemented by a baseline set of practical implementation guidance incorporating related and emerging technologies. As a collective, the model and implementation guidance offers a go-to guide for a wide range of end-users. Furthermore, it provides a coherent foundational structure towards establishing an online "living" resource, which can be expanded over time through broader community engagement.

On Design and Enhancement of Smart Grid Honeypot System for Practical Collection of Threat Intelligence

Daisuke Mashima, Derek Kok, and Wei Lin, Illinois at Singapore; Muhammad Hazwan and Alvin Cheng, Custodio Technologies

Long Preliminary Work Paper

Available Media

The smart grid system is exposed to cyberattacks, as demonstrated by the number of real-world incidents in the last few years. The attack strategies keep evolving, and security mechanisms must identify novel attack vectors ideally before they actually hit the system. In this direction, honeypot systems for smart grid infrastructure are considered effective. While use of honeypot systems for general IT security has a history already, implementations for smart grid systems, and industrial control systems in general, are not mature yet. In this paper, we summarize our efforts for designing, implementing, and evaluating our smart grid honeypot system. We started with a prototype implementation of the virtual smart grid infrastructure using open-source tools, evaluate the realism of it from an attacker's perspective through collaboration with cybersecurity experts. We then refined the honeypot system to offer better realism as well as logging features for capture attackers' behaviours.

8:15 am–8:45 am


8:45 am–9:45 am

Vulnerabilities, Exploits, Attacks

Representativeness in the Benchmark for Vulnerability Analysis Tools (B-VAT)

Kayla Afanador and Cynthia Irvine, Naval Postgraduate School

Short Preliminary Work Paper

Available Media

A variety of tools are used to support software vulnerability analysis processes. However, when analysts want to select the optimal tool for a particular use case, or attempt to compare a new tool against others, no standard method is available to do so. In addition, we have determined that although vulnerabilities can be categorized into vulnerability types, these types are often disproportionately represented in current datasets. Hence, when comparative analyses of tools based upon these datasets are conducted, the results are distorted and unrealistic. To address this problem, we are building a Benchmark for Vulnerability Analysis Tools (B-VAT).

Representativeness is a key element of a good benchmark. In this paper, we use stratified sampling to identify a representative set of vulnerabilities from over 800 CWE’s and 75,000 CVE’s. This set becomes the foundation upon which we will build a purpose-based benchmark for vulnerability analysis tools. By using the correlation between current CWE and CVE data, the proposed B-VAT will assess tools using vulnerabilities in the proportions their types occur in the wild.

Historical Analysis of Exploit Availability Timelines

Allen D. Householder, Carnegie Mellon University; Jeff Chrabaszcz, Govini; Trent Novelly, Carnegie Mellon University; David Warren, SEI CERT; Jonathan M. Spring, Carnegie Mellon University

Long Research Paper

Available Media

Vulnerability management is an important cybersecurity function. Within vulnerability management, there are multiple points where knowing whether an exploit targeting a given vulnerability is publicly available would inform vulnerability mitigation priority. Despite the value of this question, there is no available historical baseline of when and how many vulnerabilities get associated public exploits. We analyze all vulnerabilities with CVE-IDs since two common repositories of public exploit data became available and find that 4.1+/-0.1% of CVE-IDs have public exploit code associated with them within 365 days. We analyze eight features of a CVE-ID for how they influence exploit publication. Some categories of vulnerability (CWE) are much more likely to have exploit code published than others. Vendor is a sporadic predictor of exploit publication likelihood. More vendors involved in a CVE-ID does not appear to affect exploit publication. CVSS score, commonness of the CWE, and how recently the CVE-ID was published all slightly increase the exploit publication likelihood; the confidence intervals for the size of these three effects overlap. Of 75,807 vulnerabilities studied, 3,164 had public exploits over the whole six year study; for those with exploits, the median time to publication is two days, though the mean time is 91 days.

Towards Adversarial Phishing Detection

Thomas Kobber Panum and Kaspar Hageman, Department of Electronic Systems, Aalborg University; René Rydhof Hansen, Department of Computer Science, Aalborg University; Jens Myrup Pedersen, Department of Electronic Systems, Aalborg University

Long Position Paper

Available Media

Over the recent decades, numerous evaluations of automated methods for detecting phishing attacks have been reporting stellar detection performances based on empirical evidence. These performances often neglect the adaptive behavior of an adversary seeking to evade detection, yielding uncertainty about their adversarial robustness. This work explores the adversarial robustness of highly influential and recent detection solutions, by assessing their common detection strategies. Following discussions of potential evasion techniques of these strategies, we present examples of techniques that enable evasion through imperceptible perturbations. In order to enable and improve future evaluations for adversarial robustness, a set of design guidelines is proposed.

APTGen: An Approach towards Generating Practical Dataset Labelled with Targeted Attack Sequences

Yusuke Takahashi and Shigeyoshi Shima, NEC Corporation; Rui Tanabe, Institute of Advanced Sciences, Yokohama National University; Katsunari Yoshioka, Graduate School of Environment and Information Sciences, Yokohama National University

Long Research Paper

Available Media

In incident response for targeted cyber attacks, the responders investigate the sequence of attacks (attack sequence) that intruders have followed by analyzing the remaining logs. Their goal is to grasp and understand the whole picture of the incident. For accelerating incident response, it is important to develop technologies to automate the investigation of the attack sequences. However, we see lack of open dataset that contains logs and corresponding attack sequence information in order to evaluate these technologies.

In this paper, we propose APTGen, an approach for generating targeted attack dataset. APTGen is top-down, that is, it first generates artificial attack sequence from existing security reports based on the attack model defined in MITRE’s ATT&CK. Then, in order to obtain logs from execution environments, it executes corresponding attack tools to realize the attack sequences. Thanks to the top-down approach, we can obtain the attack sequence information corresponding to the attack trace left in the logs. We generate 800 different attack sequences and logs based on reports of eight actual security incidents. We publish generated sequences and logs as a dataset for R&D of incident responses.

9:45 am–10:45 am


10:45 am–11:30 am

Network Infrastructure & Attacks

Toward Orchestration of Complex Networking Experiments

Alefiya Hussain, USC/Information Sciences Institute; Prateek Jaipuria, Hulu; Geoff Lawler, Stephen Schwab, and Terry Benzel, USC/Information Sciences Institute

Long Experience Paper

Available Media

Experimentation is an essential tool for developing networked and distributed systems. However, it is inherently complex due to the concurrent, asynchronous, heterogeneous, and prototype-based systems that must be integrated into representative scenarios to conduct valid evaluations. This paper offers a retrospective on the development and use of MAGI, an orchestration tool, that translates an experiment specification into an execution on an emulation-based testbed with high-level directives for message passing, remote process execution, and failure tracking, for conducting large and complex experiments. The MAGI tool has been used for more than seven years in a variety of experiments, including undergraduate education, anonymous communication, cyber-physical systems, and attacker-defender games on the DETER testbed. We hope the insights and takeaways learned from using our tool will aid in developing the next-generation experiment management tools.

UBCIS: Ultimate Benchmark for Container Image Scanning

Shay Berkovich, BlackBerry Limited; Jeffrey Kam, University of Waterloo; Glenn Wurster, BlackBerry Limited

Short Preliminary Work Paper

Available Media

Containers are regularly used in modern cloud-native deployment practices. They support agile and continuous integration/continuous deployment (CI/CD) paradigms, isolating services. As containers become more ubiquitous, container security becomes crucial as well. Scanning container images for known vulnerabilities caused by vulnerable software is a critical security activity of the CI/CD process. Both commercial and open-source tools exist for container image scanning. Results from these scanners, however, are inconsistent. Inconsistent results make it hard for developers to choose the best solution for their environment. In this paper, we present the Ultimate Benchmark for Container Image Scanning (UBCIS), a benchmark for evaluating image scanners. UBCIS contains a classification of known vulnerabilities in common base container images, as well as a framework for running container vulnerability scanning tools. UBCIS makes it possible to evaluate scanners. We discuss intricacies of classifying vulnerabilities, presenting a process that can be used when determining the relevance of vulnerability. Finally, we provide recommendations for choosing the best scanner for a specific environment.

Bridging Missing Gaps in Evaluating DDoS Research

Lumin Shi, Samuel Mergendahl, Devkishen Sisodia, and Jun Li, University of Oregon

Short Preliminary Work Paper

Available Media

While distributed denial-of-service (DDoS) attacks become stealthier and more disruptive, real-world network operators often ignore academic DDoS defense research and instead rely on basic defense techniques that cannot adequately defend them. In fact, prior to the deployment of a DDoS defense solution, a network operator must understand its impact specifically on their network. However, without a sound empirical analysis of the solution, which is often the case even for the most cited academic work, the network operator may fear its poor defense efficacy or its adverse effects on legitimate traffic. In this work, we elaborate on the critical missing gaps in DDoS defense evaluation and propose a new evaluation platform to help produce the missing defense analytics. To identify the impact of a defense solution in realistic network settings, our platform offers to emulate a mini Internet topology with realistic IP address space allocation and generate representational, closed-loop background traffic specific to particular networks. As such, our platform fulfills the prominent missing gaps in current DDoS research. In the end, we conduct some experiments to demonstrate the correctness and efficiency of our platform.

11:30 am–11:45 am


11:45 am–12:45 pm


Panel Discussion and Audience Dialogue: Sharing Artifacts and Data for Cybersecurity Experimentation

ResearchSOC Project Panelist: Inna Kouper, Indiana University Bloomington
SEARCCH Project Panelists: David Balenson and Laura S. Tinnel, SRI International

Available Media

Sharing repeatable, reproducible, and reusable artifacts and data in cybersecurity experimentation can greatly enhance one’s ability to build upon the work of others and compare solutions. Good data, in particular, can be difficult to acquire and use for a number of reasons. Panelists discuss two NSF-funded efforts to help fill this need in the research community.

12:45 pm–1:00 pm

Closing Remarks

Program Co-Chairs: Tamara Denning, University of Utah, and Tyler Moore, University of Tulsa

1:00 pm–2:00 pm

Optional Networking