Edge Inference on Unknown Models at Adobe Target

Customer’s Data Scientist: “I know much better my business, just give me the data and I’ll create the model you should run.” Transforming this into reality, in our production systems, comes with a lot of challenges, which we will discuss in this talk.

In today’s world, increasingly many companies build their own data science/ML departments. When needing to run their custom models on different systems, the models need to be converted to other frameworks or to a format interpretable in representation standards for machine learning models (e.g., ONNX). In this talk we discuss challenges and approaches to using such models in real-time, low-latency systems. We discuss the limitations of existing frameworks, scoring runtimes, model representations, and the existing solutions to overcome them. We discuss how these methods can be used today to build a solution that provides real-time scoring for high throughput workload.