Check out the new USENIX Web site. next up previous
Next: Simulating catastrophes Up: Phoenix evaluation Previous: Phoenix evaluation

Prototype evaluation

We tested our prototype on 63 hosts across the Internet: 62 PlanetLab hosts and one UCSD host. To simulate the diversity we obtained in the study presented in Section 4, we selected 63 configurations at random from our set of 2,963 configurations of general-purpose hosts, and made each of these configurations an input to the Phoenix service on a host. In the population we have chosen randomly, out of the 63 configurations 38 have Windows as their operating system. Thus, in our setting roughly 60% of the hosts represent Windows hosts. From Equation 1, we obtain that the load limit must be at least three.

For the results we present in this section, we use an OS hint list while searching for cores. Varying L, we obtained the values in Table 5 for coverage, core size, and load variance for a representative run of our prototype. For comparison, we also present results from our simulations with the same set of configurations used for the PlanetLab experiment. From the results in the table, coverage is perfect in all cases, and the average core size is less than 3 (less than 2 replica copies).

The major difference in increasing the value of L is the respective increase in load variance. As $L$ increases, load balance worsens. We also counted the number of requests issued by each host in its search for a core. Different from our simulations, we set a large upper bound on the number of request messages (diff_OS + same_OS $= 100$) to verify the average number of requests necessary to build a core, and we had hosts searching for other hosts only outside their own zones (same_OS = 0). The averages for number of requests are 14.2, 5.2, and 4.1 for values of L of 3, 5, and 7, respectively. Hence, we can tradeoff load balance and message complexity.

We also ran experiments without using an OS hint list. The results are very good, although worse than the implementation that uses hint lists. We observed two main consequences in not using a hint list. First, the average number of requests is considerably higher (over 2x). Second, for small values of $L$ ($L=3,5$), some hosts did not obtain perfect coverage.

Table 5: Implementation results on PlanetLab ("Imp") with simulation results for comparison ("Sim").
Load limit (L) Core size Coverage Load var.
Imp. Sim. Imp. Sim. Imp. Sim.
3 2.12 2.23 1.0 1.0 1.65 1.88
5 2.10 2.25 1.0 1.0 2.88 3.31
7 2.10 2.12 1.0 1.0 4.44 3.56

next up previous
Next: Simulating catastrophes Up: Phoenix evaluation Previous: Phoenix evaluation
Flavio Junqueira 2005-02-17