Stability of three-wave simplex estimates of reliability

In this paper two major problems of using the three-wave simplex model to obtain reliability estimates are illustrated. The first problem is that the sampling variance of the reliability estimates can be very large, specially if the stability through time is low. The second problem is that, in order that the reliability parameter be identified, the model assumes a particular change process, namely a Markov or first order autoregressive process. We show that minor violations of this assumption can lead to a large bias in the reliability estimates. The problems are evaluated using both real and Monte Carlo data. Two possible solutions are suggested and their problems and implementation discussed and illustrated.