Detecting Item Preknowledge in Computerized Adaptive Testing Using Information Theory and Combinatorial Optimization
DOI:
https://doi.org/10.7333/jcat.v2i0.36Abstract
Item preknowledge occurs when some examinees (called aberrant examinees) have had access to a subset of items (called a compromised subset) from an administered test prior to an exam. As a result, aberrant examinees might perform better on compromised items as compared to uncompromised items. When the number of aberrant examinees is large, the corresponding testing program and its users might be negatively affected because the aberrant examinees might be given invalid scores. There are numerous item preknowledge detection methods exploiting the difference in an examinee’s performance between compromised items and uncompromised items. These methods are based on an incorrect assumption that the compromised subset is known and that it does not vary across subgroups of aberrant examinees. Computer simulations demonstrated that when this assumption is slightly violated the detection rate drops dramatically. This paper introduces a new algorithm—the 3D algorithm—merging information theory and combinatorial optimization for detecting subgroups of aberrant examinees and their corresponding compromised subsets of items.
Published
Issue
Section
License
Authors who publish in JCAT agree to the following terms:
- Authors retain copyright and grant the journal right of first publication.
- Articles may be copied and reproduced for academic and research purposes with no cost.
- All other reproduction requires permission of the author(s).
- If the authors cannot be contacted, permission can be requested from IACAT.
- Authors may enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., publish it in a book), with an acknowledgement of its initial publication in JCAT.