Transparency and Evaluation

The process by which a voting system is state and federally approved to be fit for use in a local jurisdiction is widely believed to be inadequate and dysfunctional and is highly opaque. Existing Federal voting system guidelines are weak and out-of-date.ACCURATE. Public comment on the 2005 voluntary voting system guidelines, 2005, available at: http://accurate-voting.org/accurate/docs/2005_vvsg_comment.pdf Federally certified voting systems have lost votes when used on election dayMore Than 4,500 North Carolina Votes Lost Because of Mistake in Voting Machine Capacity, Associated Press / USA Today, November 5, 2004, available at: http://tinyurl.com/3nhfw. and critical parts of voting systems have made it through federal certification without being examined.NASED letter, ``Voting System Memory Card Issues'', March 22, 2006, avaliable under ``certification'' at: http://www.nased.org/. The federal certification process relies on Independent Testing Authority (ITA) laboratories to test voting systems for compliance with the federal voting system standards and guidelines.The set of federal standards that are in effect at the time of writing are the FEC's 2002 Voting System Standards (2002 VSS). The EAC's 2005 Voluntary Voting System Guidelines (2005 VVSG) have been approved by the EAC but will not go into effect until January 2008. See: http://guidelines.kennesaw.edu/vvsg/intro.asp. The ITAs are paid by the vendors and all communications and subsequent output from the ITA testing is considered confidential and protected under non-disclosure agreements (NDA) by the vendors.Kim Zetter, E-Voting Tests Get Failing Grade, Wired News, November 1, 2004, (article notes that ITAs cannot discuss specific systems due to NDAs with vendors) available at: http://www.wired.com/news/evote/0,65535-2.html. Vendors have claimed that the disclosure of information by the ITAs would implicate their intellectual property rights and compromise the security of their systems.ITAA letter to Assemblymember Tom Umberg, ``OPPOSE: AB 2097'', March 22, 2006, on file with author. Similar sentiments were expressed in written testimony to a California State Senate Committee on Elections hearing in February of 2006; see: http://tinyurl.com/rsk5e. In part, the vendors object to sharing information from the ITA review process based on their desire to maintain ``security through obscurity,'' a principle from computer science that has long been discredited.One of the best discussions of the notion of ``security through obscurity'' is available on the Wikipedia page for the term. See: Security through obscurity: http://en.wikipedia.org/w/index.php?title=Security_through_obscurity&oldid=33624306 (last visited January 2, 2006). Full disclosure: the author is one of the many editors of this Wikipedia page. Code review by independent, dedicated evaluation teams improves system security; however, the circumstances of the evaluation and relationship between the parties involved should be carefully considered to maximize the utility of evaluation and minimize any undue influence.Lipner, S. B. Security and source code access: Issues and realities. In SP Õ00: Proceedings of the 2000 IEEE Symposium on Security and Privacy (Washington, DC, USA, 2000), IEEE Computer Society, p. 124.

In light of the failures of the federal standard-setting, testing and certification process, state-level officials have begun to aggressively supplement federal testing to ensure that voting systems meet federal and state requirements and operate appropriately on election day.

Most open source licenses grant or withhold the exclusive rights, granted to creators under copyright law, of copying, modifying and distributing. For detailed inspection of the source code, inspectors would need at least the rights to copy and make modifications. That is, to properly test and debug a program, inspectors will need all source code necessary to build the binary application in a machine-readable format. They would then need to be able to transfer this code to their own build environment, verify that the source code behaves as it purports to, properly build the application and verify that the executible built behaves appropriately and matches the binaries on the target voting systems in the field. Transferring of code, compilation and modification necessary to test source routines implicates the right of reproduction and the right to prepare derivative works or modifications granted by copyright. The right to distribute the source code is not necessarily essential from this perspective as long as the inspecting parties get full access to the code.

In addition to these licensing requirements, system evaluators need a certain level of access to the code. They need the source code and any other software utilitiesThis includes third-party software or software libraries for which the vendor may not have a license to redistribute. needed in order to compileCompiling a set of source code involves using another computer program to translate high-level code (human-readable) into low-level code (suitable for machines to understand). the exact binary (executable) program that corresponds to the version of the program used in the voting system, the ability to compile the source code with modifications,One example, evaluators would want to set what are called ``debugging flags'' before compilation of the program which allow an evaluator or a program running under the operation of an evaluator to follow the exact flow of program control, input/output, etc. from step to step. and access to system design materials and documentation.Without design materials and documentation, very obscure code that would be impossible to test could become the norm. (Ping commented here: could still be a problem regardless of docs!)

In intellectual property terms, this means that evaluators of source code need the right to make copies and derivative works of voting system software, but not other rights such as distribution. As design documentation and source code might contain trade secrets, those will have to be distributed to evaluators under a proper non-disclosure agreement, or trade secrets will have to be excluded altogether in the case of public dissemination.

In addition to manual source code review, there are many bug-finding software applications.For a partial list of bug-finding tools, see: List of tools for static code analysis, http://en.wikipedia.org/w/index.php?title=List_of_tools_for_static_code_analysis&oldid=34047272 (last visited January 6, 2006). These tools are developed to automatically find bugs in software by examining source code files or while the software is running. Evaluators point these tools at large bodies of source code, such as the Linux codebase, and are making much progress at finding common programming errors and vulnerabilities.Ashcraft, K., And Engler, D. Using programmer-written compiler extensions to catch security holes. In SP Õ02: Proceedings of the 2002 IEEE Symposium on Security and Privacy (Washington, DC, USA, 2002), IEEE Computer Society, p. 143. If voting system software were available to bug-finding researchers, they could examine and perfect their tools further while increasing the integrity of the software. Of course, bug finding is just one example of security-increasing research applications that source code availability could catalyze.

It is an understatement to say that the licensing options available in free and open source software are vast and complicated. As with the case of the California Secretary of State's AVVPAT regulations discussed in §, to enact legislation or rules involving open source without detailed direction on what licenses are appropriate or what principles of open source are required is confusing. There are approximately one hundred open source licenses and they all have different, often incompatible terms.For an appreciation of the variety in open source licensing regimes, browse the Open Source Initiative's (OSI) ``Approved License'' list http://www.opensource.org/licenses and the Free Software Foundation's web page ``Various Licenses and Comments about Them'' http://www.gnu.org/philosophy/license-list.html. Open source licensing covers many licenses, some of which are incompatible with each other. Licenses span a spectrum of very simple -- like the modified BSD licenseThe BSD License as published by the OSI: http://www.opensource.org/licenses/bsd-license.php. -- to the very intricate and complex -- like the GNU General Public License.The Free Software Foundation's GNU General Public License: http://www.gnu.org/copyleft/gpl.html.

From a systems perspective, evaluation of code is not enough. We must also include other techniques such as adversarial penetration testing,Penetration testing (sometimes called ``Red team'' or ``tiger team'' attacks) involve a simulated attack on a system where the attack team may know everything (``white box'' testing) or very little (``black box'' testing) about a system and attempt to compromise it in the same manner as would a malicious actor. These types of exercises are common in the testing and implementation of high-integrity systems. For more on penetration testing rationales and methodologies, see: Open Source Security Testing Methodology Manual, available at: http://www.isecom.org/osstmm/. parallel monitoring,Parallel monitoring, employed during each election now in the State of California, involves randomly quarantining a subset of voting machines on election day and voting on them with fake voters and scripted votes to detect bugs, procedural flaws and evidence of possible malicious activity. For more, see: Douglas W. Jones, Testing Voting Systems: Parallel testing during an election, The University of Iowa, Department of Computer Science, available at: http://www.cs.uiowa.edu/~jones/voting/testing.shtml. reliability testing and forms of feedback that we have in other areas of computing such as incident reporting and feedback.For example, Carnegie Mellon University's Computer Emergency Response Team (CERT) is a computer security incident tracking and response service, see: http://www.cert.org/. In response to a question asked by the author at the NIST Voting Systems Threats workshop, EAC commissioners Davisdson and DeGregorio expressed interest in setting up a similar service and process for computerized voting systems.