MoonBright: A GPU Memory Allocator with Device-Side Page Table Materialization and Deferred TLB Coherence

Modern GPU workloads increasingly rely on dynamic and fine-grained memory allocation, yet GPU memory management remains CPU-centric. In current GPU runtimes, allocation metadata updates, page-table construction, translation-state propagation, and TLB shootdowns are largely serialized through the host control path, introducing substantial latency. We present MoonBright, a GPU memory allocator that enables device-side page-table materialization and deferred TLB coherence on commodity GPUs. MoonBright keeps validation and allocation metadata on the host, but moves bulk page-table construction to the GPU, turning translation updates into data-parallel device-memory operations. To avoid costly TLB shootdowns on the common path, MoonBright assigns fresh virtual addresses to new mappings, ensuring that no stale same-address TLB entries can exist. Our evaluation shows that MoonBright reduces allocation latency, improves LLM inference performance, and mitigates allocator-level external fragmentation across diverse workloads. Unlike application-specific memory managers, MoonBright requires no GPU hardware modifications and runs on commodity NVIDIA and AMD GPUs with software-only changes. MoonBright is publicly available at https://github.com/MoonBright-project.