Skip to main content
USENIX
  • Conferences
  • Students
Sign in
  • Overview
  • Attend
    • Registration Information
    • Registration Discounts
    • Venue, Hotel, and Travel
    • Students and Grants
    • Co-located Workshops
  • Program
  • Participate
    • Instructions for Participants
    • Call for Papers
  • Sponsorship
  • About
    • Summit Organizers
    • Services
    • Questions
    • Help Promote!
    • Past Summits
  • Overview
  • Attend
    • Registration Information
    • Registration Discounts
    • Venue, Hotel, and Travel
    • Students and Grants
    • Co-located Workshops
  • Program
  • Participate
    • Instructions for Participants
    • Call for Papers
  • Sponsorship
  • About
    • Summit Organizers
    • Services
    • Questions
    • Help Promote!
    • Past Summits

sponsors

Platinum Sponsor

help promote

CSET '15 button

Get more
Help Promote graphics!

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @usenix

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » Summit Program
Tweet

connect with us

Summit Program

All sessions will be held in Lexington/Bunker Hill unless otherwise noted.

The summit papers are available for download in a ZIP archive for registered attendees. Individual papers are available to everyone and can be downloaded from each paper's presentation page. Copyright to the individual works is retained by the author[s].

Downloads for Registered Attendees

Attendee Files 

(Registered attendees: Sign in to your USENIX account to download these files.)

3GSE '15 Paper Archive (ZIP)
3GSE '15 Attendee List (PDF)

 

Tuesday, August 11, 2015

8:00 am–9:00 am Tuesday

Continental Breakfast

9:00 am–10:30 am Tuesday

World 1-1: Inside the CTF Mind

Session Chair: Mark Gondree, Naval Postgraduate School

An Offline Capture The Flag-Style Virtual Machine and an Assessment of Its Value for Cybersecurity Education

Tom Chothia, University of Birmingham; Chris Novakovic, Imperial College London

Online Capture The Flag (CTF) competitions are a popular means of engaging students with the world of cybersecurity. This paper reports on the use of a virtual machine (VM) framework that has been developed as part of cybersecurity courses offered to both second-year undergraduate and master’s degree students in the School of Computer Science at the University of Birmingham; the framework features CTF-style challenges that must be solved in order to complete the courses’ formative assessment. As well as acquiring flags from the framework, students must also provide traditional written answers to questions and sit an examination. We analyse how well students’ performance on the CTF-style challenges correlates with their achievement in the remaining formative assessment and examination, thus providing evidence to show whether CTFs are effective as an assessment tool in academic cybersecurity courses.

Available Media

Multidisciplinary Experiential Learning for Holistic Cybersecurity Education, Research and Evaluation

Aunshul Rege, Temple University

Experiential learning is defined as learning through action, experience, and discovery and exploration. Capture the Flags (CTFs) events have been offering experiential learning to computer science/engineering students for over 20 years. However, existing pedagogical technical research recognizes the need to address certain shortcomings of CTFs, such as novice encouragement, skewed experiences of CTF type (attack-defend vs. attack only vs. defend only), the difficulty and clarity of CTF challenges, and temporal constraints. This paper argues that CTFs can offer valuable experiential learning experiences for criminology/criminal justice students as well, by improving their hands-on research skills as well as their understanding of cyberattacks/defense. The paper also argues that multidisciplinary experiential learning is critical in improving CTFs for both technical and criminological domains. Specifically, it offers four benefits: breakdown of disciplinary silos in cybersecurity education; innovative research; enhanced learning experiences; and greater transparency in the evaluation of CTFs. The paper also offers some challenges of multidisciplinary experiential learning and offers possible implementation suggestions.

Available Media

Engaging Novices in Cybersecurity Competitions: A Vision and Lessons Learned at ACM Tapia 2015

Jelena Mirkovic, University of Southern California and Information Sciences Institute; Aimee Tabor, University of California, Berkeley; Simon Woo, University of Southern California and Information Sciences Institute; Portia Pusey, National CyberWatch Center

Cybersecurity competitions are popular tools for attracting students to cybersecurity field. Yet, many competitions require extensive preparation, strong coding skills and solid background knowledge, not just in security, but also in system administration, networking and operating systems. As such, competitions may discourage novices that lack in one of these required areas. In this paper we discuss our experience in using Class Capture-the-Flag Exercises (CCTFs) to bridge this gap in classes, and in 2015 ACM Richard Tapia Security workshop. We recount lessons learned and map a way forward, towards collaborative, more structured cybersecurity competitions that better support and engage novices, and offer a positive learning experience to all.

Available Media

An Examination of the Vocational and Psychological Characteristics of Cybersecurity Competition Participants

Masooda Bashir, April Lambert, Jian Ming Colin Wee, and Boyi Guo, University of Illinois at Urbana–Champaign

The demand for cybersecurity professionals grows each year, and so do efforts to attract students to cybersecurity. Competitions are a popular way to address the shortage of cybersecurity professionals, but are competitions actually effective at attracting talent into the cybersecurity workforce? To date there has been little empirical evidence of the effectiveness of cybersecurity competitions, but this paper presents the results of an extensive survey of cybersecurity competition participants. These results provide a profile of the demographic, psychological, cultural, and vocations characteristics of competi-tions participants and may inform efforts to develop effective competitions and tools for identifying promising cybersecurity students.

Available Media
10:30 am–11:00 am Tuesday

Break with Refreshments

11:00 am–11:45 am Tuesday

World 2-1: Gamifying the Curriculum

Session Chair: Zachary N J Peterson, California Polytechnic State University

Using CTFs for an Undergraduate Cyber Education

Martin Carlisle, Michael Chiaramonte, and David Caswell, United States Air Force Academy

Over the last five years, the United States Air Force Academy (USAFA) has participated in numerous Capture the Flag (CTF) and other cyber competitions. At first, this was simply an extracurricular club activity; however, as we have seen the impact on student motivation and learning, we have greatly increased student and faculty participation. Additionally, we have started to base entire for-credit courses on a CTF framework. In this paper we discuss our rationale for utilizing CTFs as part of our formal curriculum, as well as key lessons learned relating to student engagement and avoiding cribbing. 

Available Media

Leveraging Competitive Gamification for Sustainable Fun and Profit in Security Education

Adrian Dabrowski, SBA Research; Markus Kammerstetter, Eduard Thamm, Edgar Weippl, and Wolfgang Kastner; Vienna University of Technology

With the ongoing IT security arms race advancing at a fast pace, there is a continuously high requirement for well-educated security professionals to protect today’s IT infrastructure from malicious attacks. While the necessary IT security expertise can be gained through continuous learning and practical exercise, the approach quickly becomes tedious and tiring for students. At Vienna University of Technology, we offer a series of two consecutive security courses leveraging gameful design and competition to increase the motivation among students. The courses have been established for a decade with currently more than 400 participants each year and 1,219 educated students since 2012. In this paper, we present our game-like course setup and evaluate the unique approach through student surveys. Our results indicate that the well-established gaming-like competitive approach is not only highly appreciated by our students, but also raises their interest and motivation to put more effort and extra work into their security education.

Available Media
11:45 am–12:30 pm Tuesday

World 2-2: The Challenge of Challenges

Julian Cohen, Flatiron Health

A Scaffolded, Metamorphic CTF for Reverse Engineering

Wu-chang Feng, Portland State University

Hands-on Capture-the-Flag (CTF) challenges tap into and cultivate the intrinsic motivation within people to solve puzzles, much in the same way Sudoku and crossword puzzles do. While the format has been successful in security competitions, there have been a limited number of attempts to integrate them into a classroom environment. This paper describes MetaCTF, a metamorphic set of Jeopardy-style, CTF challenges for teaching reverse code engineering. MetaCTF is 1) scaffolded in a way that allows students to make incremental progress, 2) integrated with the course material so that students can immediately apply knowledge gained in class, 3) polymorphic and metamorphic so that individual students within a class and between multiple offerings of a class are given unique challenges, and 4) extensible in order to allow students to design their own CTF challenges that can be later integrated into future offerings of the course.

Available Media

Automatic Problem Generation for Capture-the-Flag Competitions

Jonathan Burket, Peter Chapman, Tim Becker, Christopher Ganas, and David Brumley, Carnegie Mellon University

Computer security games, especially capture-the-flag (CTF) competitions, are growing in popularity. A typical CTF contest presents users with a set of hacking challenges, where correct solutions reveal a text “flag” that can be submitted to a scoring server. In traditional CTF architectures, the problem and the flag are the same across the competition.

In this paper we discuss automatic problem generation (APG), where a given challenge is not fixed, but rather can have many different automatically generated problem instances. APG offers players a unique competition experience and can facilitate deliberate practice where problems vary just enough to make sure a user can replicate the solution idea. APG also allows competition administrators the ability to detect when users submit a copied flag from another user to the scoring server. In 2014 we ran a large-scale CTF competition called PicoCTF, where we measured the prevalence of flag sharing. Our results indicate that about 0.8% of flags submitted to AGP problems were copied, with 14% of teams submitting at least one shared flag. In 68% of flag sharing cases, teams went on to eventually solve the problem on their own.

Available Media
12:30 pm–2:00 pm Tuesday

Luncheon for Summit Attendees


2:00 pm–3:00 pm Tuesday

Keynote Address

Game-Based Learning, Collateral Learning, and Beyond

Lee Sheldon, Worcester Polytechnic Institute

Instructors today, whether academic, government or corporate, are leaping on to the bandwagon of gamification with hopes high and eyes closed. Joining them onboard are those who use the techniques of last century’s corporate videos with branching choices and overt exams instead of the organic, embedded assessments of games, then rename their simulations as games. Badges and other extrinsic rewards, simply the gold stars of the digital era, are added indiscriminately to learning experiences, but do not magically transform them into good games. Game-based learning also does not mean blind faith in technology such as video games or social media clumsily shoehorned into a curriculum like a stepsister’s foot vainly squeezed into Cinderella’s slipper.

Instructors today, whether academic, government or corporate, are leaping on to the bandwagon of gamification with hopes high and eyes closed. Joining them onboard are those who use the techniques of last century’s corporate videos with branching choices and overt exams instead of the organic, embedded assessments of games, then rename their simulations as games. Badges and other extrinsic rewards, simply the gold stars of the digital era, are added indiscriminately to learning experiences, but do not magically transform them into good games. Game-based learning also does not mean blind faith in technology such as video games or social media clumsily shoehorned into a curriculum like a stepsister’s foot vainly squeezed into Cinderella’s slipper.

We will begin our quest with a brief history of games that purport to teach; then move to the Multiplayer Classroom, an entire class designed as a real-time game played in the real world. Elements include grading by accretion (XP and leveling up); learning by failing (allowing learners to redo assignments); intrinsic rewards (such as dividing students into guilds and rewarding an entire guild for the achievement of one member); peer teaching and more. Initial anecdotal evidence of success is now supported by recent studies involving hundreds of teachers and thousands of students: good game design is a powerful tool for learning. 

Next, we will fuse game design with storytelling on a variety of game platforms from the computer to the real world, engaging the heart as well as the mind, to create “collateral learning” where true games using competition, collaboration, human emotion, mystery and surprise and more are as exciting to play as games designed solely for entertainment. Whatever the subject being taught, students, no matter their age, learn in spite of themselves, because the well-designed game inspires their curiosity and imagination even as it challenges and sharpens their knowledge and critical thinking.

Lee Sheldon is a professional game writer and designer currently working on his 40th game. He recently joined the Interactive Media and Game Development (IMGD) program at Worcester Polytechnic Institute. For five years previously he was an Associate Professor in the Games and Simulation Arts and Sciences program at Rensselaer Polytechnic Institute. There he was co-director of the GSAS program for three years and created the first full writing for games program in the United States. 

Lee wrote the bestselling book The Multiplayer Classroom: Designing Coursework as a Game (2011); his book Character Development and Storytelling for Games (Second Edition, 2013) is the standard text in the field. The Facebook page for his method of teaching classes as multiplayer games is now followed by more than 1200 people in more than 40 countries.

His recent applied game projects include two games at Rensselaer: The Lost Manuscript 2: The Summer Palace Cipher, a virtual reality game teaching Mandarin and Chinese culture; and These Far Hills, a video game teaching engineering and science for an NSF proposal. He wrote Crimson Dilemma, a business ethics video game for Indiana University that debuted Fall 2014; and wrote and designed Secrets: A Cyberculture Mystery Game, an online class designed as a game teaching culture and identity on the Internet for Excelsior College that goes live Fall 2015.

His most recent entertainment game is the AAA Kinect title Disney Fantasia: Music Evolved for Harmonix, released in October 2014. He is currently writing a reboot of a popular Facebook game.

Before his career in games, Lee was a television writer-producer with over 200 produced shows ranging from Charlie’s Angels to Star Trek: The Next Generation.

Available Media
  • Read more about Game-Based Learning, Collateral Learning, and Beyond
3:00 pm–3:30 pm Tuesday

World 3-1: Formal Verification Games

Andy Davis, MIT Lincoln Laboratory

Lessons Learned in Game Development for Crowdsourced Software Formal Verification

Drew Dean, SRI International; Sean Gaurino and Leonard Eusebi, Charles River Analytics; Andrew Keplinger, Left Brain Games; Tim Pavlik, University of Washington; Ronald Watro, Raytheon BBN; Aaron Cammarata, VoidALPHA; John Murray, SRI International; Kelly McLaughlin, XPD Analytics; John Cheng and Thomas Maddern, Veracient LLC

The history of formal methods and computer security research is long and intertwined. Program logics that were in theory capable of proving security properties of software were developed by the early 1970s. The development of the first security models gave rise to a desire to prove that the models did, in fact, enforce the properties that they claimed to, and that an actual implementation of the model was correct with respect to its specification. Optimism reached its peak in the early to mid-1980s, and the peak of formal methods for security was reached shortly before the publication of the Orange Book, where the certification of a system at class A1 required formal methods. Formal verification of software was considered the gold standard evidence that the software enforced a particular set of properties. Soon afterwards, the costs of formal methods, in both time and money, became all too apparent. Mainstream computer security research shifted focus to analysis of cryptographic protocols, policies around cryptographic key management, and clever fixes for security problems found in contemporary systems.

Available Media
3:30 pm–4:00 pm Tuesday

Break with Refreshments

4:00 pm–5:00 pm Tuesday

World 4-1: What's in a Name?

A Community Discussion on a Shared Terminology for Security Games

Session Chair: Portia Pusey, National CyberWatch Center

5:00 pm–5:30 pm Tuesday

Bonus Level: Short Talks

Session Chair: Peter Chapman, Carnegie Mellon University

Platinum Sponsors

© USENIX

  • Privacy Policy
  • Contact Us