Network Subsystems Reloaded: A High-Performance, Defensible Network Subsystem

Traditionally, operating systems have used monolithic network stack implementations: implementations where the whole network stack executes in the kernel or (in microkernels) in a single, trusted, user level server. Code maintenance issues, ease of debugging, need for simultaneous existence of multiple protocols, and security benefit have argued for removing the networking implementation from kernel and dividing it into multiple user level protection domains. Previous attempts to do so have failed to deliver adequate performance. Given the advances made in both hardware (CPU, Memory, NIC) and micro-kernel design over the last decade, it is now appropriate to re-evaluate how these re-factored implementations perform, and to examine the reasons for earlier failures in greater detail.

Building on the primitives of the EROS microkernel, we have implemented two network subsystems: one a conventional, user mode, monolithic design and the other a domain-factored user level networking stack that restructures the network subsystem into several protection domains. We show that the restructuring maintains performance very close to that of the monolithic design, and that both designs compare favorably to a conventional in-kernel implementation. We discuss the issues faced in engineering the domain-factored implementation to achieve high performance, and present the quantitative evaluation of the resulting network subsystems.