|
USENIX Technical Program - Abstract - Windows NT Symposium 99
The Record-Breaking Terabyte Sort on a Compaq Cluster
Samuel A. Fineberg and Pankaj Mehra, Compaq Tandem Labs
Abstract
Sandia National Laboratories (U.S. Department of Energy) and Compaq
Computer Corporation built a 72-node Windows NT cluster, which Sandia
utilizes for production work contracted by the U.S. government.
Recently, Sandia and Compaq's Tandem Division collaborated on a
project to run a 1-terabyte commercial-quality scalable sort on this
cluster. The audited result was a new world record of 46.9 minutes,
three times faster than the previous record held by a 32-processor
shared-memory UNIX system. The external sort utilizes a unique,
scalable algorithm that allows near-linear cluster scalability. The
sort application exploits several key hardware and software
technologies; these include dense-racking Pentium II based servers,
Windows NT Workstation 4.0, the Virtual Interface Architecture, and
the ServerNet I System Area Network (SAN). The sort code was highly
CPU-efficient and stressed asynchronous and sequential I/O and IPC
performance. The I/O performance when combined with a high-performance
SAN, yielded supercomputer-class performance
- View the full text of this paper in
HTML form and
PDF form.
- If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.
- To become a USENIX Member, please see our Membership Information.
|
Need help? Use our Contacts page.
Technical Program