Estimating Measurement Precision in Reduced-Length Multistage-Adaptive Testing
DOI:
https://doi.org/10.7333/jcat.v1i0.19Abstract
The extent to which reducing the number of items in a multi-stage adaptive test (MST) affected measurement precision was evaluated. Using the Massachusetts Adult Proficiency Test for Reading (MAPT), a low-stakes MST in adult education, reliability, decision consistency, and decision accuracy estimates were compared for the original and reduced-length tests (from 40 to 35 items). Four different approaches were used: (1) the Spearman-Brown formula, (2) eliminating one item of average discrimination from consecutive stages, (3) completely reassembling new panels, and (4) simulating item responses to the original and shortened MSTs and comparing the standard errors of measurement for simulated examinees. Overall, results suggested comparable levels of measurement precision, improved content representation, and reduced testing time were achievable using the reduced-length tests. The Spearman-Brown estimates were surprisingly close to the estimates based on assembling new panels. Methods for assembling an MST to maintain measurement precision and practical lessons that could generalize to MSTs in other contexts are discussed.
Keywords: multi-stage adaptive testing, reliability, decision consistency, decision accuracy, response time, test development, validity.
Published
Issue
Section
License
Authors who publish in JCAT agree to the following terms:
- Authors retain copyright and grant the journal right of first publication.
- Articles may be copied and reproduced for academic and research purposes with no cost.
- All other reproduction requires permission of the author(s).
- If the authors cannot be contacted, permission can be requested from IACAT.
- Authors may enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., publish it in a book), with an acknowledgement of its initial publication in JCAT.