A Comparison of Four Methods for Obtaining Information Functions for Scores From Computerized Adaptive Tests With Normally Distributed Item Difficulties and Discriminations

Abstract

A simulation study compared four methods of estimating information for maximum-likelihood estimates (MLE) from fixed-length computerized adaptive tests (CAT): those by (1) Lord (1980), (2) Segall, Moreno, and Hetter (1997), (3) Ito, Pommerich, and Segall (2009a), and (4) Ito and Segall (2010), referred to, respectively, as “local,” “quasi-local,” “global-slope,” and “conditional-averaging-on-q” (CA-theta ). A 900-item bank was constructed using operational three-parameter logistic item parameter estimates [i.e., a (discrimination), b (difficulty), and c (pseudo-guessing)] such that the a and b parameters were as normally distributed as possible. Test length and the number of simulees at each theta point were varied. Generally, information functions for the quasi-local and global-slope methods were very close to those for the reference local method. However, those for the CA-theta method were distinctly different, which might be explained conceptually. These findings were considerably different than those from previous studies that had used a skewed item bank, and illustrated how bank characteristics, including distributions of item difficulties, discriminations, and ability-bank (mis)matches, could influence which methods would yield similar score information functions.