New Article in the Journal of Computerized Adaptive Testing (JCAT)

The Journal of Computerized Adaptive Testing has published its latest article, "Optimizing Test Completion in Time-Constrained Adaptive Tests: A Post-Hoc Simulation Study."

Abstract:

In operational computerized adaptive tests (CATs), time limits are often imposed to manage logistical constraints, maintain test security, or preserve classroom instructional time. However, time constraints in CATs can lead to inaccurate ability estimates as some examinees might struggle to complete enough items within the allotted time due to differences in working speed and time intensities of individual items. This study proposes a simple dynamic approach to incorporate item response time (RT) data into the item selection algorithm, with the goal of optimizing test completion in a time-constrained setting. Linear optimization techniques were employed to balance information and RT when selecting subsequent items when needed. A post-hoc simulation study was conducted using real data from a 244-item digital assessment. CATs were simulated for 21,356 students, comparing the proposed (optimized) item selection algorithm with the conventional maximum Fisher information algorithm. The results indicate that the optimized algorithm improved test completion rates while slightly enhancing the accuracy and precision of ability estimates. The benefits were most pronounced under the shortest time constraint condition. Practical implications for implementations of time-constrained CATs are discussed.

The complete article is available at https://jcatpub.net/index.php/jcat using the Current option.