Rise of the Machines: Removing the Human-in-the-Loop

Viral Gupta and Yunbo Ouyang, LinkedIn


Most large-scale online recommender systems like notifications recommendation, newsfeed ranking, people recommendations, job recommendations, etc. often have multiple utilities or metrics that need to be simultaneously optimized. The machine learning models that are trained to optimize a single utility are combined together through parameters to generate the final ranking function. These combination parameters drive business metrics. Finding the right choice of the parameters is often done through online A/B experimentation, which can be incredibly complex and time-consuming, especially considering the non-linear effects of these parameters on the metrics of interest. In this talk we will present how we build generic solution to solve the problem at scale.

Viral Gupta, LinkedIn

Viral Gupta works as a Relevance Tech Lead for near real-time Notifications recommendation on the LinkedIn platform. He works on several problems including onboarding new notification onto the Relevance platform efficiently, improving the overall quality of the recommendations by incorporating multi-objective optimization, etc. He is one of the key players who designed and implemented the scalable hyper-parameter optimization library at LinkedIn. He spends time consulting several teams at LinkedIn on how can the parameter search needs for the specific use-cases be formulated as a multi-objective optimization problem and solved using the hyper-parameter optimization library.

Yunbo Ouyang, LinkedIn

Dr. Yunbo Ouyang is a senior software engineer in LinkedIn AI Foundations team who has expertise and works extensively on automatic hyperparameter tuning. He is one of the key players building LinkedIn’s offline and online hyperpameter tuning libraries. He obtained his Ph.D. in Statistics from University of Illinois at Urbana-Champaign. He has published papers in top conferences such as SIAM International Conference on Data Mining. He has been serving as a reviewer for multiple top conferences such as NeurIPS, KDD and AAAI. He has taught and TAed multiple advanced statistics and machine learning courses in UIUC.

OpML '20 Open Access Sponsored by NetApp

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@conference {256656,
author = {Viral Gupta and Yunbo Ouyang},
title = {Rise of the Machines: Removing the Human-in-the-Loop},
year = {2020},
publisher = {{USENIX} Association},
month = jul,

Presentation Video