Brian Pawlowski, Drivescale Inc.
Composable Infrastructure in this talk is a method for the dynamic creation of secure application clusters from disaggregated compute, storage and networking. The problems facing such a solution are ones of availability, durability, scalability, performance and most importantly correctness.
The target applications are widely deployed data analytics and NoSQL database applications that can consist of 100's to 1,000's of compute nodes with 10,000's of disk for each application in a secure cluster instance.
The talk consists of five parts. We present a very brief description to the user view of creating virtual clusters on a composable infrastructure platform. We follow this with a short description of the problems and requirements for the platform. That motivates the bulk of the presentation describing the state machine design for a correct and durable orchestration platform that scales to 100,000's of managed elements. Select code and data structures are used to point out implementation details. The fourth part of the talk describes how standard Linux networking and storage subsystems are managed and used to create virtual clusters (including NVME over Fabric), and the open source components used by the platform to achieve scale, availability, and security. The final part of the talk details key failure scenarios and the recovery mechanisms that maintain correctness and availability.
Brian Pawlowski, Drivescale Inc.
Brian Pawlowski is currently CTO of Drivescale Inc. where he is involved in the design of software to support cluster computing and developing a platform for composable infrastructure.
As Vice President and Chief Architect at Pure Storage, he was focused on product and architecture development, with an eye toward simplifying the user experience. Earlier, Brian was a Senior Vice President and Chief Technology Officer at NetApp for more than six years, leading the design of high performance, highly reliable storage systems. Brian led the design for NetApp's first SAN product and holds several patents related to that work. He also worked on open protocols for storage since an earlier role at Sun Microsystems, was co-author of the NFS Version 3 specification and is currently co-chair of the NFS Version 4 working group at the Internet Engineering Task Force (IETF).
Brian has presented on several occasions at USENIX conferences, and previously at Vault 2015, and SANE 2004. He has presented technology and product architecture at many industry events.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Brian Pawlowski},
title = {Design of a Composable Infrastructure Platform},
year = {2019},
address = {Boston, MA},
publisher = {USENIX Association},
month = feb
}