Skip to main content
USENIX
  • Conferences
  • Students
Sign in

connect with us


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube

twitter

Tweets by @LISAConference

usenix conference policies

  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy

You are here

Home » Application Buffer-Cache Management for Performance: Running the World’s Largest MRTG
Tweet

connect with us

Application Buffer-Cache Management for Performance: Running the World’s Largest MRTG

Abstract: 

An operating system's readahead and buffer-cache behaviors can significantly impact application performance; most often these better performance, but occasionally they worsen it. To avoid unintended I/O latencies, many database systems sidestep these OS features by minimizing or eliminating application file I/O. However, network traffic measurement applications are commonly built instead atop a high-performance file-based database: the Round Robin Database (RRD) Tool. While RRD is successful, experience has led the network operations community to believe that its scalability is limited to tens of thousands of, or perhaps one hundred thousand, RRD files on a single system, keeping it from being used to measure the largest managed networks today. We identify the bottleneck responsible for that experience and present two approaches to overcome it.

In this paper, we provide a method and tools to expose the readahead and buffer-cache behaviors that are otherwise hidden from the user. We apply our method to a very large network traffic measurement system that experiences scalability problems and determine the performance bottleneck to be unnecessary disk reads, and page faults, due to the default readahead behavior. We develop both a simulation and an analytical model of the performance-limiting page fault rate for RRD file updates. We develop and evaluate two approaches that alleviate this problem: application advice to disable readahead and application-level caching. We demonstrate their effectiveness by configuring and operating the world's largest Multi-Router Traffic Grapher (MRTG), with approximately 320,000 RRD files, and over half a million data points measured every five minutes. Conservatively, our techniques approximately triple the capacity of very large MRTG and other RRD-based measurement systems.

David Plonka, University of Wisconsin—Madison

Archit Gupta, University of Wisconsin—Madison

Dale Carder, University of Wisconsin—Madison

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {245554,
author = {David Plonka and Archit Gupta and Dale Carder},
title = {Application Buffer-Cache Management for Performance: Running the World{\textquoteright}s Largest {MRTG}},
booktitle = {21st Large Installation System Administration Conference ({LISA} 07)},
year = {2007},
address = {Dallas, TX},
url = {https://www.usenix.org/conference/lisa-07/application-buffer-cache-management-performance-running-world{\textquoteright}s-largest-mrtg},
publisher = {{USENIX} Association},
month = nov,
}
Download

Presentation Audio

MP3 Download

Download Audio

Links

Paper (HTML): 
http://usenix.org/event/lisa07/tech/full_papers/plonka/plonka_html/index.html
Paper: 
http://usenix.org/event/lisa07/tech/full_papers/plonka/plonka.pdf
Award: 
Best Paper
  • Log in or    Register to post comments

© USENIX

  • Privacy Policy
  • Conference Policies
  • Contact Us