Improving Precision of CAT Measures

This paper was the author’s presidential address at the 2015 conference of the International Association for Computerized Adaptive Testing


  • John Barnard


dichotomously scored items, option probability theory, scoring methods, subjective proba-bility


The basic idea of adaptive testing is quite simple and has been implemented for over a century. Dichotomously scored multiple-choice questions are commonly used to obtain response vectors in computerized adaptive tests (CATs). A response is scored as either correct or as incorrect. However, a correct response does not necessarily mean that the examinee knew the answer. Although the standard error increasingly decreases as the provisional ability is estimated, the question is whether the process can be improved at the item response level. In other words, can more information be extracted from a response than a simple 0 or 1? Implementation of option probability theory holds promise to address this question.