Check out the new USENIX Web site.

Home About USENIX Events Membership Publications Students
FAST 2002 Abstract

Exploiting Inter-File Access Patterns Using Multi-Collective I/O

Gokhan Memik, EE Dept. UCLA; Mahmut Kandemir, CSE Dept. Penn State; Alok Choudhary ECE Dept. Northwestern University

Abstract

This paper introduces a new concept called Multi-Collective I/O (MCIO) that extends conventional collective I/O to optimize I/O accesses to multiple arrays simultaneously. In this approach, as in collective I/O, multiple processors co-ordinate to perform I/O on behalf of each other if doing so improves overall I/O time. However, unlike collective I/O, MCIO considers multiple arrays simultaneously; that is, it has a more global view of the overall I/O behavior exhibited by application. This paper shows that determining optimal MCIO access pattern is an NP-complete problem, and proposes two different heuristics for the access pattern detection problem (also called the assignment problem).

Both of the heuristics have been implemented within a runtime library, and tested using a large-scale scientific application. Our preliminary results show that MCIO out-performs collective I/O by as much as 87%. Our runtime library-based implementation can be used by users as well as optimizing compilers. Based on our results, we recom-mend future library designers for I/O-intensive applications to include MCIO in their suite of optimizations.

  • View the full text of this paper in PDF. Until January 2003, you will need your USENIX membership identification in order to access the full papers.
    The Proceedings are published as a collective work, © 2002 by the USENIX Association. All Rights Reserved. Rights to individual papers remain with the author or the author's employer. Permission is granted for the noncommercial reproduction of the complete work for educational or research purposes. USENIX acknowledges all trademarks within this paper.

  • If you need the latest Adobe Acrobat Reader, you can download it from Adobe's site.

  • To become a USENIX Member, please see our Membership Information.

?Need help? Use our Contacts page.

Last changed: 27 Dec. 2001 ml
Technical Program
FAST 2002 Home
USENIX home