A Score-Based Method for Detecting Item Compromise and Preknowledge in Computerized Adaptive Testing

Authors

Abstract

In recent years, several methods have been proposed to detect compromised items (CI) and examinees with preknowledge (EWP) simultaneously. However, most of these methods are limited in one of two ways: (1) The method was specifically designed for non-adaptive tests and might therefore not be suitable for adaptive tests, or (2) the method involves the analysis of more than just the item scores. In this paper, a new method is developed for computerized adaptive tests that requires only an analysis of the item scores. The perfor-mance of the method is evaluated using simulations in which several factors are manip-ulated, such as test length, the percentage of CI, the percentage of EWP, and EWP ability distribution. Across most conditions, the method is shown to produce reasonable false positive rates and favorable true positive rates for both items and examinees.

Downloads

Published

2025-06-20

Issue

Vol. 12 No. 2 (2025): A Score-Based Method for Detecting Item Compromise and Preknowledge in Computerized Adaptive Testing

Section

Articles

License

Authors who publish in JCAT agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication.
  2. Articles may be copied and reproduced for academic and research purposes with no cost.
  3. All other reproduction requires permission of the author(s).
  4. If the authors cannot be contacted, permission can be requested from IACAT.
  5. Authors may enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., publish it in a book), with an acknowledgement of its initial publication in JCAT.