Our experiments consist of simulations of modem pools with different numbers of modems and different disconnection policies than those in place during tracing. Nevertheless, the actual user activity might have been different if the actual simulated policies were in effect. For instance, one common user reaction to strict idle timeout policies is to use programs that simulate activity so that the connection appears to be always in use.
This interaction of policies and user decisions is something that no predictive study involving human users can compensate for. Nevertheless, although this issue can certainly affect the exact results of our experiments, it should not affect the overall picture and our conclusions. There are two reasons for this. First, the most important of our observations are made under conditions that guarantee a quality of service similar to that of the tracing environment: busy signals are extremely rare and disconnections of users who want to be active again soon are very few. The second reason is more subtle but very important. The most likely user reaction (if any) to inconvenient disconnections is to simulate constant activity so that no disconnection occurs. This penalizes all disconnection policies equally per user (the user simply cannot be disconnected, and occupies a modem as long as she wants). Thus, a policy that causes more user inconvenience can be expected to be penalized more than a policy that causes less inconvenience (because more users will attempt to avoid disconnections in the former case). Therefore, even though the absolute values of our metrics may change, their relative differences for different disconnections policies will only be accentuated by the user's conscious choices. Hence, we believe that our results will hold true when actually employed.
Apart from the case of a conscious user choice, however, there are cases when programmatically generated activity may interact with our simulations. This means that the actual behavior might not be identical to the simulated one if the studied workloads were indeed handled with the disconnection policy under simulation. The differences are expected to be insignificant, but we discuss them here so that our assumptions are in the open. There are two kinds of systematic activities that may be affected by a different disconnection policy and may appear in our traces.
The potential problem in case 1 is dual. First, the observed behavior in our trace may include automatic reconnections after disconnections. This is behavior that would not have occurred if a different policy had not caused the disconnection. This affects only our AT&T Labs trace, which was collected in an environment with a 15 minute idle timeout. Nevertheless, the activity at 16 minutes of idle time (see Figure 2) is not high enough to suggest that this may be a significant interference. The second possible problem with case 1 is that new activity will appear after every disconnection, but this activity is not evident from the original trace. Therefore, all our simulations should be viewed as accurate under the assumption that users do not automatically reconnect on disconnect. Even in the opposite case, however, this behavior is likely to affect all simulated policies equally.
The potential problem with case 2 is that during tracing we may have missed some behavior because of the disconnection policy that was in place in the tracing environment. That is, the user might have been active, had she not been previously disconnected. (The issue clearly is relevant to programmatic activity, not interactive user activity--the user would explicitly connect if she wanted to use the network interactively.) This should not be a problem--if the behavior was not important enough to occur in the traced system, it was probably not important enough to occur in any simulated systems. It should also be noted that our traces reflect all activity that originated from a user. We have no way of telling if some of this activity would not have occurred if the user had previously been disconnected. Hence, our simulations represent the behavior of a system where users do dial in automatically every time they are disconnected and an application tries to access the network (e.g., every time their email client needs to download messages).