Check out the new USENIX Web site. next up previous
Next: Causes of Blocking Up: Blocking in Web Servers Previous: Observing Blocking in Flash

Inferring Blocking in Apache

Directly observing a similar problem in Apache is more difficult because any of its processes may block on disk activity, and its multiple-process design exploits the fact that the OS will schedule another process when the running process blocks. While conventional wisdom holds that such blocking is necessary and affects only the request being handled, excess blocking may hinder parallelism and cause high latency.

Since Apache does not have easily-testable invariants regarding blocking such as Flash does, we use another mechanism to infer it. We can use the observation that blocking in Flash increases the burstiness of system activity to find a similar behavior in Apache. In particular, we note that if resource contention occurs in Apache, it would block other processes requesting the same resource, and the release of a resource would involve several processes becoming runnable at the same time. We expect that as more processes are involved, burstiness increases as does run queue variability.

We instrument the OS scheduler to report the number of runnable Apache processes, and test in two configurations. We use 256 and 1024 maximum server processes, an infinite-demand workload, and 1024 clients. Both configurations show roughly the same throughput, due to the infinite-demand model and LAN clients. In Figure 2, we show what percentage of the Apache processes are runnable at any given time.

In both cases, the distribution is very bimodal - most of the time, either no Apache processes are runnable or most of them are. The burstiness, when many processes suddenly become runnable at once, is more evident in the 1024 process case - all processes are blocked roughly one-third of the time, and over 80% of the processes are in the runnable queue over 40% of the time. The 256 process case is only slightly less bursty, with the run queue generally containing 60-80% of the total processes. Note that all processes being blocked does not imply the entire system is idle - disk and interrupt-driven network activity is still being performed in the kernel's ``bottom-half.''

Figure 3: HTTP request processing steps
\centerline {\epsfig{,width=3.25in}}\vspace{-.05in}\vspace{-.05in}\end{figure}

next up previous
Next: Causes of Blocking Up: Blocking in Web Servers Previous: Observing Blocking in Flash
Yaoping Ruan