Responsible Design through Experimentation: Learnings from LinkedIn

Guillaume Saint-Jacques, LinkedIn Corporation


As technology advances, there is increasing concern about individuals being left behind. Businesses are striving to adopt responsible design practices and avoid any unintended consequences of their products. We propose a novel approach to fairness and inclusiveness based on experimentation. We use experimentation in order to assess not only the intrinsic properties of products and algorithms but also their impact on people. We do this by introducing an inequality approach to A/B testing. We show how to perform causal inference over this inequality measure. We provide real examples from LinkedIn, as well as an open-source, highly scalable implementation of the computation of the Atkinson index and its variance in Spark/Scala. We also provide over a year's worth of learnings - gathered by scaling our method and analyzing thousands of experiments - on which areas and which kinds of product innovations seem to foster fairness through inclusiveness.

@inproceedings {257967,
author = {Guillaume Saint-Jacques},
title = {Responsible Design through Experimentation: Learnings from {LinkedIn}},
booktitle = {2020 {USENIX} Conference on Privacy Engineering Practice and Respect ({PEPR} 20)},
year = {2020},
url = {},
publisher = {USENIX Association},
month = oct

Presentation Video