Unlocking the Power of Inline Floating-Point Operations on Programmable Switches

Authors: 

Yifan Yuan, UIUC; Omar Alama, KAUST; Jiawei Fei, KAUST & NUDT; Jacob Nelson and Dan R. K. Ports, Microsoft Research; Amedeo Sapio, Intel; Marco Canini, KAUST; Nam Sung Kim, UIUC

Abstract: 

The advent of switches with programmable dataplanes has enabled the rapid development of new network functionality, as well as providing a platform for acceleration of a broad range of application-level functionality. However, existing switch hardware was not designed with application acceleration in mind, and thus applications requiring operations or datatypes not used in traditional network protocols must resort to expensive workarounds. Applications involving floating point data, including distributed training for machine learning and distributed query processing, are key examples.

In this paper, we propose FPISA, a floating point representation designed to work efficiently in programmable switches. We first implement FPISA on an Intel Tofino switch, but find that it has limitations that impact throughput and accuracy. We then propose hardware changes to address these limitations based on the open-source Banzai switch architecture, and synthesize them in a 15-nm standard-cell library to demonstrate their feasibility. Finally, we use FPISA to implement accelerators for training for machine learning and for query processing, and evaluate their performance on a switch implementing our changes using emulation. We find that FPISA allows distributed training to use 25-75% fewer CPU cores and provide up to 85.9% better throughput in a CPU-constrained environment than SwitchML. For distributed query processing with floating point data, FPISA enables up to 2.7× better throughput than Spark.

NSDI '22 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {278324,
author = {Yifan Yuan and Omar Alama and Jiawei Fei and Jacob Nelson and Dan R. K. Ports and Amedeo Sapio and Marco Canini and Nam Sung Kim},
title = {Unlocking the Power of Inline {Floating-Point} Operations on Programmable Switches},
booktitle = {19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22)},
year = {2022},
isbn = {978-1-939133-27-4},
address = {Renton, WA},
pages = {683--700},
url = {https://www.usenix.org/conference/nsdi22/presentation/yuan},
publisher = {USENIX Association},
month = apr
}

Presentation Video