Taming Operations in the Apache Hadoop Ecosystem
LISA: Where systems engineering and operations professionals share real-world knowledge about designing, building, and maintaining the critical systems of our interconnected world.
The LISA conference has long served as the annual vendor-neutral meeting place for the wider system administration community. The LISA14 program recognized the overlap and differences between traditional and modern IT operations and engineering, and developed a highly-curated program around 5 key topics: Systems Engineering, Security, Culture, DevOps, and Monitoring/Metrics. The program included 22 half- and full-day training sessions; 10 workshops; and a conference program consisting of 50 invited talks, panels, refereed paper presentations, and mini-tutorials.
Kathleen Ting and Jonathan Hsieh, Cloudera, Inc.
The Apache Hadoop stack includes many distributed storage and processing systems, running on clusters ranging from tens to thousands of nodes. At Cloudera, we’ve been supporting tens of thousands of nodes in hundreds of our customers’ production clusters with diverse use cases. For five years, we have been navigating paths for sys admins to manage, tune, and debug the systems. We'll describe a methodology for debugging and tuning between the different layers (app, hadoop, jvm, kernel, networking). We’ll also talk about new tools and subsystems included in our operational best practices to keep your clusters always up, running, and secure.
Kathleen Ting, Cloudera
Kathleen Ting (@kate_ting) is currently a technical account manager at Cloudera where she helps strategic customers deploy and use the Apache Hadoop ecosystem in production. She's a frequent conference speaker, has contributed to several projects in the open source community, and is a committer and PMC member on Apache Sqoop. Kathleen is also a co-author of O’Reilly’s Apache Sqoop Cookbook.
Jonathan Hsieh, Cloudera
Jonathan Hsieh is a Software Engineer and HBase Team Tech Lead at Cloudera. He is an Apache HBase committer and PMC member and a committer and founder of Apache Flume. He has spoken at many conferences including Hadoop World, Hadoop Summit, HBaseCon and the USENIX NSDI Conference. Jonathan has an M.S. in Computer Science from University of Washington, an M.S. and a B.S. in Electrical and Computer Engineering from Carnegie Mellon University.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Kathleen Ting and Jonathan Hsieh},
title = {Taming Operations in the Apache Hadoop Ecosystem},
year = {2014},
address = {Seattle, WA},
publisher = {USENIX Association},
month = nov
}
