A Comparison of Multi-Stage and Linear Test Designs for Medium-Size Licensure and Certification Examinations

Abstract

Many large-scale testing organizations currently use multi-stage testing (MST) for their examinations. The MST design implements the test in several stages, where one module is administered per stage. Successive stages in MST might vary in difficulty depending on the estimated ability level of the examinee. Although several studies have been conducted to compare the performance of MST to the traditional linear test design, all of the investigations known to date have incorporated simulation studies that capitalize on large sample size requirements in order to reduce the number of replicated datasets. As a result, although the statistics under investigation have been estimated with reasonable stability, these studies have been better suited to investigate the performance of MST for large-scale examinations as opposed to small- or medium-size examinations. The purpose of this research was to conduct a series of studies based on simulated datasets for medium-size medical certification examinations. The results confirmed that more accurate ability estimates and more accurate and consistent pass-fail decisions are obtained under the MST design compared to the traditional linear design for these examinations.