|
USENIX Technical Program - Abstract - Domain-Specific Languages 99
Using Production Grammars in Software Testing
Emin Gün Sirer and Brian N. Bershad, University of Washington
Abstract
Extensible typesafe systems, such as Java, rely critically on a large
and complex software base for their overall protection and integrity,
and are therefore difficult to test and verify. Traditional testing
techniques, such as manual test generation and formal verification,
are too time consuming, expensive, and imprecise, or work only on
abstract models of the implementation and are too
simplistic. Consequently, commercial virtual machines deployed so far
have exhibited numerous bugs and security holes.
In this paper, we discuss our experience with using production
grammars in testing large, complex and safety-critical software
systems. Specifically, we describe lava, a domain specific
language we have developed for specifying production grammars, and
relate our experience with using lava to generate effective
test suites for the Java virtual machine. We demonstrate the
effectiveness of production grammars in generating complex test cases
that can, when combined with comparative and variant testing
techniques, achieve high code and value coverage. We also describe an
extension to production grammars that enables concurrent generation of
certificates for test cases. A certificate is a behavioral description
that specifies the intended outcome of the generated test case, and
therefore acts as an oracle by which the correctness of the tested
system can be evaluated in isolation. We report the results of
applying these testing techniques to commercial Java
implementations. We conclude that the use of production grammars in
combination with other automated testing techniques is a powerful and
effective method for testing software systems, and is enabled by a
special purpose language for specifying extended production
grammars.
- View the full text of this paper in
HTML form and
PDF form.
- If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.
- To become a USENIX Member, please see our Membership Information.
|
Need help? Use our Contacts page.
Technical Program