The injection component is responsible for delivering test cases into the distributed application for execution. A complication in the implementation of an injection component is that the behavior of a distributed application may depend on who originated a request, or where in the network a request originated. The injection component must reproduce these attributes of the test cases in order to accurately test the deployed distributed application. In particular, this means that the injection component is itself distributed, as test requests may be injected at different nodes of the application.
A possible design for a simple injection component is to require every node to support proxy or relay functionality. This would allow remote developers to masquerade as any node in the system. For example, if a test case requires that a message be sent from node A, then the node running the test simply sends/receives messages to a proxy at node A, which forwards them to the distributed applications (and returns messages from the distributed application to the tester). Messages that need to be sent by a particular user are handled in a similar manner. The ability to impersonate a user or machine naturally raises security issues that must be addressed. In addition, while this allows a developer to route via another node, it may not perfectly recreate the tested node's view of the system. For example, the timing of messages might be affected which in turn may affect the behavior of the system. A richer scripting mechanism to control nodes may provide a more accurate remote view of the system but may be much more complicated to implement.
The validation component is responsible for facilitating the checking of the application on the injected test cases. Note that it does not perform the check itself - the responsibility for determining the correctness of application behavior necessarily requires application specific knowledge. The duty of the validation component is to provide the data that enables validation of application behavior. The data gathering must be lightweight in terms of communications cost. The component must also minimize the effort required by the application developer to specify the data to be captured. At the same time, however, it must provide flexibility in the kind of data to be captured.
An important tool to help validation is support for a rich logging system. A developer should be able to specify a set of triggers, a set of data types, and a callback for each data type. Incoming messages at a node can be matched against the set of triggers and the callbacks are invoked to capture data when a message matches a trigger. The triggers can be checked before or after the message is processed by the application. Some key open challenges are the scalable distribution of these triggers and callbacks to the application nodes and the scalable collection of the resulting logs for analysis. It is likely that not all nodes in the system will be involved in the validation, thereby simplifying the task of distribution and collection.