ChameleonAPI: Automatic and Efficient Customization of Neural Networks for ML Applications

Authors: 

Yuhan Liu, University of Chicago; Chengcheng Wan, East China Normal University; Kuntai Du, Henry Hoffmann, and Junchen Jiang, University of Chicago; Shan Lu, University of Chicago and Microsoft Research; Michael Maire, University of Chicago

Abstract: 

ML APIs have greatly relieved application developers of the burden to design and train their own neural network models—classifying objects in an image can now be as simple as one line of Python code to call an API. However, these APIs offer the same pre-trained models regardless of how their output is used by different applications. This can be suboptimal as not all ML inference errors can cause application failures, and the distinction between inference errors that can or cannot cause failures varies greatly across applications.

To tackle this problem, we first study 77 real-world applications, which collectively use six ML APIs from two providers, to reveal common patterns of how ML API output affects applications' decision processes. Inspired by the findings, we propose ChameleonAPI, an optimization framework for ML APIs, which takes effect without changing the application source code. ChameleonAPI provides application developers with a parser that automatically analyzes the application to produce an abstract of its decision process, which is then used to devise an application-specific loss function that only penalizes API output errors critical to the application. ChameleonAPI uses the loss function to efficiently train a neural network model customized for each application and deploys it to serve API invocations from the respective application via existing interface. Compared to a baseline that selects the best-of-all commercial ML API, we show that ChameleonAPI reduces incorrect application decisions by 43%.

OSDI '24 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {298705,
author = {Yuhan Liu and Chengcheng Wan and Kuntai Du and Henry Hoffmann and Junchen Jiang and Shan Lu and Michael Maire},
title = {{ChameleonAPI}: Automatic and Efficient Customization of Neural Networks for {ML} Applications},
booktitle = {18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24)},
year = {2024},
isbn = {978-1-939133-40-3},
address = {Santa Clara, CA},
pages = {365--386},
url = {https://www.usenix.org/conference/osdi24/presentation/liu},
publisher = {USENIX Association},
month = jul
}

Presentation Video