There are a number of different approaches to representing the distributed system component of the testbed. The three following approaches outline the main alternatives and argue their applicability for this study. All three have their own advantages, consequently there is no completely correct choice.
Theoretical models are excellent for analysing simple systems, and models already exist for static load balancing systems. Payne [Pay82] states that, ``a system that can be analysed mathematically, will generally provide more accurate results and more information with potentially less effort than simulation''. However the more complex and interesting dynamic models tend to be intractable [Pay82, WM85].
Systems involving nonlinear elements such as queues and random influences can often be analysed with queueing theory. Unfortunately these solutions are only available for limited dynamic cases, and when different algorithms and distribution architectures are being studied, a change in the parameters of the system often requires a new model.
The system to be modelled is an unrestricted distributed system: potentially heterogeneous in composition, containing resource queues, and fundamentally dynamic in nature. Any one of these make the system a poor candidate for mathematical modelling.
Experimenting on a physical system has the primary advantage of realism. There are no simplifying assumptions that could invalidate the results, other than in the workload (when synthetically generated), or in the measurement of the effect on the system. There are however, some properties in a real system that reduce its desirability.
These problems are minor when compared to the value of the results from a real system, however, the experiments required for this thesis need to be repeatable. Otherwise valid comparisons and therefore support for the hypotheses can be difficult to make. Additionally the cost in implementation time as well as experimental time is too great.
Simulation is a tool for system analysis that has been studied for a considerable length of time. In this respect it is a well understood discipline, with well defined areas of application. Shannon [Sha75] provides the following formal definition:
Simulation is the process of designing a model of a real system and conducting experiments with this model for the purpose either of understanding the system or of evaluating various strategies (within the limits imposed by a criterion or set of criteria) for the operation of the system. An alternative definition of simulation comes from Franta [Fra77]:
A system is a big black box
Of which we can't unlock the locks,
And all we can find out about
Is what comes in and what goes out
Perceiving input-output pairs,
Connected by parameters,
Permits us, sometimes, to relate
An input, output and a state.
If this relation's good and stable
Then predict we may be able,
But if this fails us - heaven forbid!
We'll be compelled to force the lid!
Kenneth Boulding
The last part of Shannon's definition describes perfectly the purpose of this thesis, without the costs inherent in building or modifying a real system. Simulation can also deal with more complex systems and interactions between systems than mathematical models. Other advantages are:
The major limitation of simulation is not in the implementation, but in the validity of the models used to construct the simulation itself. Flawed models can destroy any value that the simulation results may have. For this reason alone, the process of system development suggested by Payne [Pay82], recommends that simulation should be used as a stage before the use of a prototype, to verify a hypothesis prior to the commitment of costly resources. It is this support of the hypotheses that is desired from this thesis, and therefore simulation is indeed a suitable choice for the distributed system component of the testbed.