Design Considerations and Reporting Solutions for a Multiple Administrations Adaptive Testing System

Authors

Abstract

In this paper we overview the policy, test development, psychometric, and reporting deliberations that stakeholders engaging in the design of multiple administration adaptive test, otherwise known as a through-year assessment, will have to consider. We use the development of a design prototype for simulations coupled with newly designed score reports to serve as exemplars of the work to be done by states and vendors to a develop a through-year assessment with the intended purpose of merging two different assessment systems into a singular system that supports teachers in better understanding where students are in their learning and states in meeting accountability requirements.

Author Biographies

  • M Christina Schneider, Cambium Assessment

    Sr. Director, Psychometrics at Cambium Assessment.

    Christina Schneider supports assessment stakeholders in transitioning assessment design and development systems to learning-theory-based approaches to improve the accuracy and utility of the information provided to teachers regarding student learning, with a fondness for adaptive designs. Her blog Actionable Assessments in the Classroom and her book Using Formative Assessment to Support Student Learning Objectives are designed to guide teachers in identifying evidence of student learning from a progression-based lens. She investigates learning-theory-based assessment design in her research, which she and her co-authors have published in a variety of journals and book chapters.

  • Seung W. Choi, The University of Texas at Austin

    Professor, Department of Educational Psychology
    Pearson Endowed Professorship in Psychometrics, College of Education

    Seung Choi is a psychometrician with expertise in applying modern psychometric theories and techniques in developing and validating educational, psychological, and patient-reported outcomes measures. He teaches graduate courses in psychometrics, including Psychometric Theory and Methods, Item Response Theory, Computer Based Testing, and Applied Psychometrics. His research interests are in the development, application, and dissemination of computerized adaptive testing programs.

  • Daniel Lewis, Creative Measurement Solutions

    Founder & Chief Scientist, Creative Measurement Solutions LLC

    Daniel Lewis  is actively engaged in confronting measurement and assessment challenges for partners in the industry. He specializes in test design, standard setting, reporting, and policy issues. Dan's latest research, Embedded Standard Setting, embeds standard setting within principled assessment design, resulting in cut scores that emerge organically from existing test development practices and changing the nature and significantly reducing the scope of, or eliminating, the standard setting workshop. This is expected to result in (a) additional support for the assessment validity argument, (b) improved item and test alignment, (c) defensible cut scores, and (d) significant cost savings for sponsoring agencies.

Downloads

Published

2025-03-04

Issue

Vol. 12 No. 1 (2025): Special Issue: Through-Year Assessments: Advancements and Implications for Hybrid Interim-Summative Testing

Section

Articles

License

Authors who publish in JCAT agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication.
  2. Articles may be copied and reproduced for academic and research purposes with no cost.
  3. All other reproduction requires permission of the author(s).
  4. If the authors cannot be contacted, permission can be requested from IACAT.
  5. Authors may enter into separate contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., publish it in a book), with an acknowledgement of its initial publication in JCAT.