Abstract
Multiple Administrations Adaptive Testing (MAAT) is an extension of the shadow-test approach to CAT for the assessment framework involving multiple tests administered periodically throughout the year. The maat package utilizes multiple item pools vertically scaled across grades and multiple phases (stages) within each test administration, allowing for transitioning from an item pool to another as deemed necessary to further enhance the quality of assessment.
Keywords: R package, CAT, multistage testing, shadow-test approach, multiple administrations
The maat package extends the shadow-test approach to CAT (van der Linden & Reese, 1998) implemented in the TestDesign package (Choi & Lim, 2021; Choi et al., 2021) for multiple tests administered throughout the school year at specified times (e.g., Fall, Winter, and Spring). Each test is administered in two phases with the opportunity to transition to a higher/lower item pool (and associated test specifications) for the second phase. Due to the time elapsed, the latent ability of examinees is not assumed to be constant across test administrations. The maat system thus initializes a subsequent test with the examinee’s previous ability estimate but adapts to the performance on the current administration and, if prompted, transitions to another item pool between phases and also between tests. For instance, an examinee whose ability estimate at the conclusion of a phase (or test) exceeds a threshold specified for the grade level will be routed to an item pool designed for the next higher grade level for the subsequent phase (or test).
The maat package provides facilities for exposure control, routing rules between phases and tests, intra-individual item overlap control across phases and/or tests, and interim and final ability estimation. The current version of the package supports two routing policies based on the confidence interval of ability estimates and the distribution of item difficulty values.
The maat package is freely available on CRAN (https://cran.r-project.org/package=maat/) and GitHub (https://github.com/choi-phd/maat/). The package includes detailed documentation, sample data files, and a vignette introducing the methodology.
Supplemental Material
Supplemental Material, sj-gz-1-apm-10.1177_01466216211049212 for maat: An R Package for Multiple Administrations Adaptive Testing by Seung W. Choi, Sangdon Lim, Luping Niu, Sooyong Lee, Christina M. Schneider, Jay Lee and Garron J. Gianopulos in Applied Psychological Measurement
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Development of the maat package was funded by NWEA through a contract to the University of Texas at Austin. The content is solely the responsibility of the authors and does not necessarily represent the official views of NWEA.
Supplementary Material: Supplementary material is available for this article online.
ORCID iDs
Seung W. Choi https://orcid.org/0000-0003-4777-5420
Sangdon Lim https://orcid.org/0000-0002-2988-014X
References
- Choi S. W., Lim S. (2021). TestDesign: Optimal test design approach to fixed and adaptive test construction. https://CRAN.R-project.org/package=TestDesign/ [Google Scholar]
- Choi S. W., Lim S., van der Linden W. J. (2021). TestDesign: An optimal test design approach to constructing fixed and adaptive test in R. Behaviormetrika. 10.1007/s41237-021-00145-9. Advance online publication [DOI] [Google Scholar]
- van der Linden W. J., Reese L. M. (1998). A model for optimal constrained adaptive testing. Advances in Psychosomatic Medicine, 22(3), 259-270. 10.1177/01466216980223006 [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplemental Material, sj-gz-1-apm-10.1177_01466216211049212 for maat: An R Package for Multiple Administrations Adaptive Testing by Seung W. Choi, Sangdon Lim, Luping Niu, Sooyong Lee, Christina M. Schneider, Jay Lee and Garron J. Gianopulos in Applied Psychological Measurement