zcIO: Enabling Transparent Zero-copy for NVMe/TCP

NVMe-over-Fabrics via TCP (NVMe/TCP) has become a datacenter standard for disaggregated storage, yet it remains severely CPU-bound in high-speed environments. While zero-copy I/Os could alleviate the load, traditional zero-copy TCP stacks often fail to meet the strict page-alignment requirements necessary for efficient hardware-to-software handoffs.

We present zcIO, a novel Linux NVMe/TCP stack that achieves seamless zero-copy I/Os through record-aware networking. By aligning TCP payloads with the NVMe PDU formats and leveraging large MTUs with header-data split capabilities, zcIO ensures that incoming DMA transfers are perfectly aligned with memory page boundaries. This precise alignment allows zcIO to replace expensive memory copies with low-overhead virtual memory page remapping. zcIO provides a transparent zero-copy data path requiring no application modifications and ensures safe page reclamation. Our evaluation shows that zcIO improves remote storage performance by up to 2.3×, reaching throughput of up to 67.6 GB/s (540.8 Gbps) with 15 CPU cores.