Check out the new USENIX Web site. next up previous
Next: IRQ notification Up: Reducing Network Virtualization Overheads Previous: Handling I/O ports in

Send combining

The second optimization further reduces world switches by taking advantage of the fact that I/O intensive workloads have a high interrupt rate and the VMM must switch to the host whenever it takes a host IRQ. In VMware Workstation 2.0, each packet sent on the Lance adapter causes a world switch to the host to send the packet over the bridged network. Since part of the Lance data port emulation is now performed in the VMM, the VMM can delay the actual transmission until the next interrupt-induced switch to the host world. Specifically, send combining work as follows: the VMM detects whether the system is experiencing a high world switch rate. If the rate (as recalculated periodically with an exponentially decaying counter) is high enough when the guest transmits a packet, the VMM queues it in a ring buffer and resumes the virtual machine. The next time a real interrupt occurs and control returns to the VMApp it transmits any pending packets in the ring buffer. This effectively allows a packet transmission world switch to be combined with an interrupt-induced one. Queueing the packets can be done without copying by leaving them in the virtual Lance controller's transmit ring buffer. If too many packets are delayed (currently defaulting to 3), the VMM will force a world switch to transmit the packets in order to insure that the native NIC is kept busy. In addition, there is a guaranteed world switch on the next IRQ from the host system timer so no packet will ever be delayed more than one tick (at which point the VMApp will discontinue send combining if necessary). This optimization works well on I/O intensive workloads because interrupt rates are high enough that world switches are saved while I/O utilization is sustained. Send combining also benefits both guest and host IRQs. Since the guest continues executing as soon as the packets are queued, there is a high probability that the guest will transmit multiple packets before the next mandatory world switch. This allows the VMApp to process multiple transmit packets on a single world switch and deliver only a single virtual IRQ for the batch. As noted earlier, virtual IRQ delivery and the associated privileged virtualization are expensive operations. Furthermore, transmitting multiple packets at once increases the probability that native send-complete interrupts are taken while executing in the host world and hardware interrupts taken in the host world are serviced faster than those taken in the VMM world.
next up previous
Next: IRQ notification Up: Reducing Network Virtualization Overheads Previous: Handling I/O ports in
Beng-Hong Lim 2001-05-01