A Monte Carlo study of the robustness of coefficient alpha.

Abstract

The biasedness and efficiency of coefficient alpha were studied given various error score distribution, number of examinees, population reliability values, and number of subtests. A Monte Carlo methodology was used in which an additive true score matrix was constructed and an error score was added to each true score. The true scores were uniformly distributed while the error score distributions used were the normal, mixed normal, exponential, and the negative exponential. In addition, 1000 replications were used and the resulting sampling distributions were examined with respect to their skewness and kurtosis. The results indicated that, when 50, 100, and 200 examinees and 5, 10, 20, and 30 subtests were used, coefficient alpha was an unbiased estimator of population reliability values of 0.4, 0.6, 0.8, and 0.9. The efficiency of the estimate increased as the number of examinees or the population reliability increased, regardless of the error score distribution used. Of the four error score distributions used the normal was the most efficient. In addition, the sampling distributions tended to become less skewed and more leptokurtic as the number of examinees and the population reliability increased. Surprisingly, an increase in the number of subtests did not appear to affect the biasedness and efficiency of coefficient alpha or the skewness and kurtosis of its sampling distribution. The results are related to the literature and suggestions for future study are given.