You are here
Elastic Memory: Bring Elasticity Back to In-Memory Big Data Analytics
Joo Seong Jeong, Woo-Yeon Lee, Yunseong Lee, Youngseok Yang, Brian Cho, Byung-Gon Chun, Seoul National University
Recent big data processing systems provide quick answers to users by keeping data in memory across a cluster. As a simple way to manage data in memory, the systems are deployed as long-running workers on a static allocation of the cluster resources. This simplicity comes at a cost: elasticity is lost. Using today’s resource managers such as YARN and Mesos, this severely reduces the utilization of the shared cluster and limits the performance of such systems. In this paper, we propose Elastic Memory, an abstraction that can dynamically change the allocated memory resource to improve resource utilization and performance. With Elastic Memory, we outline how we enable elastic interactive query processing and machine learning.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.