NodeFinder: Scalable Search over Highly Dynamic Geo-distributed State


Azzam Alsudais, University of Colorado Boulder; Zhe Huang, Bharath Balasubramanian, and Shankaranarayanan Puzhavakath Narayanan, AT&T Labs Research; Eric Keller, University of Colorado Boulder; Kaustubh Joshi, AT&T Labs Research


Finding nodes with certain criteria is a critical need for many cloud services. For example, a cloud monitoring service needs to query thousands of hosts in a data-center to check for resource usage while a cloud homing service needs to find edge data centers across the world that satisfy certain complex constraints. This is a challenging problem, especially when confronted with highly dynamic state, scale on the order of hundreds or even thousands, geo-distribution and complex query constraints that traverse decentralized data sources. In this paper, we address this problem through the design of NodeFinder that is based on a novel pull-based approach in which we maintain decentralized (peer-to-peer) groups of nodes structured according to the node attribute values (i.e., their state). This allows queries to be sent to only a few representatives of the groups that have the potential of satisfying the constraints, and then the representatives gossip with their peers and return the latest set of nodes. This guarantees freshness of results, and ensures directed and thereby scalable querying. We show NodeFinder's use in production use-cases such as host monitoring in our OpenStack clouds and NFV homing on edge clouds. Our preliminary experiments on Amazon EC2 illustrate NodeFinder's scalability and efficiency as compared to today's approaches.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {216831,
author = {Azzam Alsudais and Zhe Huang and Bharath Balasubramanian and Shankaranarayanan Puzhavakath Narayanan and Eric Keller and Kaustubh Joshi},
title = {{NodeFinder}: Scalable Search over Highly Dynamic Geo-distributed State},
booktitle = {10th USENIX Workshop on Hot Topics in Cloud Computing (HotCloud 18)},
year = {2018},
address = {Boston, MA},
url = {},
publisher = {USENIX Association},
month = jul