Isha Pali, UQ-IITD Research Academy; Shubham Thakur and Vikas Maurya, Indian Institute of Technology Delhi; Naipeng Dong, The University of Queensland, Australia; Vireshwar Kumar, Indian Institute of Technology Delhi; Dan Dongseong Kim, The University of Queensland, Australia
Network intrusion detection systems (NIDS) proposed for the automotive CAN are routinely evaluated on public datasets under the implicit assumption that these datasets represent comparable notions of benign traffic. We challenge this assumption through a measurement study of six public CAN datasets: CH, OTIDS, CAN-MIRGU, ROAD, SynCAN, and X-CANIDS. Characterizing each along four structural dimensions ID space size, transition density, payload entropy, and inter-ID correlation, we find variation of up to 10× in ID space and 2× in transition density, and different structurally isolated clusters in terms of payload entropy and transition density. These findings show that public CAN datasets are not interchangeable benchmarks, and that evaluation results are strongly conditioned on dataset structure.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
