USENIX Security '20 Call for Artifacts

Sponsored by USENIX, the Advanced Computing Systems Association.

Submit an Artifact

After your paper has been (conditionally) accepted, please check the Artifact Submission Guidelines. Submit your artifact via the submission form. Artifact registration deadlines vary depending on the submission deadline for which you submitted your paper.

Spring Quarter Deadline

  • Notification for paper authors of spring quarter: Monday, August 19, 2019
  • Artifact registration deadline: Wednesday, August 21, 2019 (AoE)
  • Artifact finalization deadline: Monday, August 26, 2019 (AoE)
  • Answering AE reviewer questions: August 28–September 10, 2019
  • Artifact decisions announced: Monday, September 16, 2019
  • USENIX Security final papers deadline: Thursday, September 19, 2019 (11:59 pm PDT)

Summer Quarter Deadline

  • Notification for paper authors of summer quarter: Friday, November 1, 2019
  • Artifact registration deadline: Tuesday, November 5, 2019 (AoE)
  • Artifact finalization deadline: Friday, November 8, 2019 (AoE)
  • Answering AE reviewer questions: November 11–November 22, 2019
  • Artifact decisions announced: Wednesday, November 27, 2019
  • USENIX Security final papers deadline: Monday, December 2, 2019 (11:59 pm PDT)

Fall Quarter Deadline

  • Notification for paper authors of fall quarter: Saturday, February 1, 2020
  • Artifact registration deadline: Tuesday, February 4, 2020 (AoE)
  • Artifact finalization deadline: Friday, February 7, 2020 (AoE)
  • Answering AE reviewer questions: February 10–February 21, 2020
  • Artifact decisions announced: Wednesday, February 26, 2020
  • USENIX Security final papers deadline: Monday, March 2, 2020 (11:59 pm PDT)

Winter Quarter Deadline

  • Notification for paper authors of winter quarter: Friday, May 1, 2020 Friday, May 15, 2020
  • Artifact registration deadline: Tuesday, May 5, 2020 (AoE) Tuesday, May 19, 2020 (AoE)
  • Artifact finalization deadline: Friday, May 8, 2020 (AoE) Friday, May 22, 2020 (AoE)
  • Answering AE reviewer questions: May 11–May 22, 2020 May 25–June 5, 2020
  • Artifact decisions announced: Wednesday, May 27, 2020 Wednesday, June 10, 2020
  • USENIX Security final papers deadline: Monday, June 1, 2020, 11:59 pm EDT Monday, June 22, 2020, 11:59 pm EDT

The artifact evaluation process will take about two weeks. Please make sure that at least one of the authors is reachable to answer questions in a timely manner. Given the camera-ready deadline, we have unfortunately only a short amount of time to perform the actual artifact evaluation process, but we expect that this time is enough to evaluate artifacts in a reasonable way.

Artifact Evaluation Committee Co-Chairs

Thorsten Holz, Ruhr-Universität Bochum
Brendan Dolan-Gavitt, New York University

Artifact Evaluation Committee

Babak Amin Azad, Stony Brook University
Elias Athanasopoulos, University of Cyprus
Kevin Borgolte, Princeton University
Marcus Botacin, Federal University of Paraná Marcus Brinkmann, Ruhr-Universität Bochum
Jason Carter, Oak Ridge National Laboratory
Sofía Celi, Dyne
Andrea Continella, University of California, Santa Barbara
Moritz Eckert, EURECOM
Rob Gillen, Oak Ridge National Laboratory
Carson Harmon, Trail of Bits
Zhenghao Hu, New York University
Georges-Axel Jaloyan, École Normale Supérieure
Sebastiano Mariani, University of California, Santa Barbara
Maryam Masoudian, Sharif University of Technology/The Hong Kong University of Science and Technology
Luca Massarelli, Sapienza University of Rome
Robert Merget, Ruhr-Universität Bochum
Feargus Pendlebury, King's College London
Fabio Pierazzi, King's College London
Blagovesta Pirelli, École Polytechnique Fédérale de Lausanne (EPFL)
Goran Piskachev, Fraunhofer IEM
Andrew R. Reiter, Veracode Applied Research Group
Eyal Ronen, Tel Aviv University/Katholieke Universiteit Leuven
Gururaj Saileshwar, Georgia Institute of Technology
Solmaz Salimi, Sharif University of Technology
Mark Santolucito, Yale University
Juraj Somorovsky, Ruhr-Universität Bochum
Sooel Son, Korea Advanced Institute of Science and Technology (KAIST)
Santiago Torres-Arias New York University
Anjo Vahldiek-Oberwagner, Intel Labs
Ryan Vrecenar, Sandia National Laboratories
Ding Wang, Nankai University
Shuai Wang, The Hong Kong University of Science and Technology
Tianhao Wang, Purdue University
Christian Wressnegger, Karlsruhe Institute of Technology
Fengwei Zhang, Southern University of Science and Technology

Artifact Submission Guidelines

Step 1. Registration

By the registration deadline, please submit the abstract and PDF of your accepted USENIX Security '20 paper, as well as topics, conflicts, and any "optional bidding instructions" for potential reviewers via the web submission form.

Step 2. Package and Submit Artifact

By the artifact finalization deadline, please provide a stable URL or (if that is not possible) upload an archive of your artifact. You will no longer be able to change these after the artifact finalization deadline. For your artifact to be considered you also need to check the "ready for review" box before the finalization deadline.

We recommend creating a single web page at a stable URL that contains the artifact and instructions for installing and using the artifact (in case of software). For other types of artifacts, please make sure that they can be evaluated in a meaningful way, please describe your artifact in detail to simplify this process. If something is unclear, you will be contacted to clarify any questions regarding your artifact.

The artifact evaluation committee will read your accepted paper before evaluating the artifact. But it is quite likely that the artifact needs more documentation to use. In particular, please make concrete what claims you are making of the artifact, especially if these differ from the expectations set up by the paper. This is a place where you can tell us about difficulties we might encounter in using the artifact, or its maturity relative to the content of the paper. We are still going to evaluate the artifact relative to the paper, but this helps set expectations up front, especially in cases that might frustrate the reviewers without prior notice.

Artifact Authors Are Not Anonymous

The artifact submissions are not anonymous. The reviewers will see the authors for each artifact from the start. So please do not waste your time trying to hide the artifact authors.

Reviewer Anonymity

We ask that, during the evaluation period, you do not embed any analytics or other tracking in the website for the artifact or, if you cannot control this, that you do not access this data. This is important for maintaining the confidentiality of reviewers. If for some reason you cannot comply with this, please notify the chairs immediately such that we can discuss options.

Packaging Artifacts

Authors should consider one of the following methods to package the software components of their artifacts (though the AEC is open to other reasonable formats as well):

  • Source code:If your artifact has very few dependencies and can be installed easily on several operating systems, you may submit source code and build scripts. However, if your artifact has a long list of dependencies, please use one of the other formats below.
  • Virtual machine/container: A virtual machine or Docker image containing the software application already set up with the right tool-chain and intended runtime environment. For example:
    • For raw data, the VM would contain the data and the scripts used to analyze it.
    • For a mobile phone application, the VM would have a phone emulator installed.
    • For mechanized proofs, the VM would have the right version of the theorem prover used.
    • We recommend using VirtualBox or Docker, since both are freely available on several platforms. An Amazon EC2 instance is also possible.
  • Binary Installer: Please indicate exactly which platform and other runtime dependencies your artifact requires.
  • Live instance on the Web: Ensure that it is available for the duration of the artifact evaluation process.
  • Hardware: In case your artifact requires special hardware (e.g., SGX or another type of trusted execution environment) or your artifact is actually a piece of hardware, please make sure that the reviewers can somehow access the device. Some kind of VPN access to the device might be an option, but due to anonymity we would need to discuss options—please contact the chairs in advance.
  • Screencast: A detailed screencast of the tool along with the results, especially if one of the following special cases applies:
    • The artifact needs proprietary/commercial software or proprietary data that is not easily available or cannot be distributed to the committee.
    • The artifact requires significant computation resources (e.g., more than 24 hours of execution time to produce the results) or requires huge data sets.
    • The artifact requires specific hard- or software that is not generally available in a typical lab and where no access can be provided in a reasonable way.

Remember that the Artifact Evaluation Committee (AEC) is attempting to determine whether the artifact meets the expectations set by the paper. If possible, package your artifact to help the committee easily evaluate this.

If you have any questions about how best to package your artifact, please do not hesitate to contact the AEC chairs, at sec20aec@usenix.org.