Table 3. 1-Year and 2-Year Intervention Impacts on Academic Outcomes by Student Characteristics.
Subject and test | 1-y Intervention impacta | 2-y Intervention impact on SY 2017-2018a | |||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
SY 2016-2017 | SY 2017-2018 | ||||||||||||||
Mean difference, Score | Effect sizeb | P value | 95% CI | 99% CI | Mean difference, Score | Effect sizeb | P value | 95% CI | 99% CI | Mean difference, Score | Effect sizeb | P value | 95% CI | 99% CI | |
Sex | |||||||||||||||
Female | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 8.73 | 0.15c | <.001 | 0.06 to 0.24 | 0.03 to 0.27 | 5.44 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.61 to 16.80 | 6.39 | 0.10 | ≥.99 | −14.02 to 14.22 | −18.46 to 18.66 |
PARCC | 2.44 | 0.08 | ≥.99 | −11.22 to 11.38 | −14.77 to 14.93 | 1.02 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | −0.18 | −0.01 | ≥.99 | −1.42 to 1.40 | −1.87 to 1.85 |
Mathematics | |||||||||||||||
i-Ready | 1.03 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | 1.62 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 | 2.22 | 0.06 | ≥.99 | −8.41 to 8.53 | −11.08 to 11.20 |
PARCC | 0.55 | 0.02 | ≥.99 | −2.80 to 2.84 | −3.69 to 3.73 | 4.07 | 0.13 | ≥.99 | −18.23 to 18.49 | −24.00 to 24.26 | 3.30 | 0.11 | .96 | −3.80 to 4.02 | −5.03 to 5.25 |
Male | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 0.36 | 0.01 | .48 | −0.02 to 0.04 | −0.03 to 0.05 | 9.00 | 0.14 | ≥.99 | −19.63 to 19.91 | −25.85 to 26.13 | 2.31 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 |
PARCC | 0.76 | 0.02 | ≥.99 | −2.80 to 2.84 | −3.69 to 3.73 | 1.19 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 | −2.02 | −0.06 | ≥.99 | −8.53 to 8.41 | −11.20 to 11.08 |
Mathematics | |||||||||||||||
i-Ready | −1.52 | −0.05 | ≥.99 | −7.11 to 7.01 | −9.33 to 9.23 | 5.29 | 0.14 | ≥.99 | −19.63 to 19.91 | −25.85 to 26.13 | 3.70 | 0.10 | ≥.99 | −14.02 to 14.22 | −18.46 to 18.66 |
PARCC | −0.56 | −0.02 | ≥.99 | −2.84 to 2.80 | −3.73 to 3.69 | 1.66 | 0.05 | ≥.99 | −7.01 to 7.11 | −9.23 to 9.33 | −3.67 | −0.12 | ≥.99 | −17.07 to 16.83 | −22.40 to 22.16 |
Grade level | |||||||||||||||
Elementary grades | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 6.14 | 0.11 | .48 | −0.19 to 0.41 | −0.28 to 0.50 | 8.40 | 0.13 | ≥.99 | −18.23 to 18.49 | −24.00 to 24.26 | 6.75 | 0.11 | ≥.99 | −15.43 to 15.65 | −20.31 to 20.53 |
PARCC | 1.86 | 0.06 | ≥.99 | −8.41 to 8.53 | −11.08 to 11.20 | 1.01 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | 0.17 | 0.01 | ≥.99 | −1.40 to 1.42 | −1.85 to 1.87 |
Mathematics | |||||||||||||||
i-Ready | 1.11 | 0.03c | <.001 | 0.01 to 0.05 | 0.01 to 0.05 | 3.54 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.62 to 16.80 | 4.38 | 0.12 | ≥.99 | −16.83 to 17.07 | −22.16 to 22.40 |
PARCC | 0.17 | 0.01 | ≥.99 | −1.40 to 1.42 | −1.85 to 1.87 | 5.57 | 0.18 | .96 | −6.22 to 6.58 | −8.23 to 8.59 | 1.13 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 |
Middle grades | |||||||||||||||
Reading | |||||||||||||||
i-Ready | −0.59 | −0.01 | .48 | −0.04 to 0.02 | −0.05 to 0.03 | 4.88 | 0.08 | ≥.99 | −11.22 to 11.38 | −14.77 to 14.93 | 1.33 | 0.02 | ≥.99 | −2.80 to 2.84 | −3.69 to 3.73 |
PARCC | 0.97 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | 1.18 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | −2.51 | −0.08 | ≥.99 | −11.38 to 11.22 | −14.93 to 14.77 |
Mathematics | |||||||||||||||
i-Ready | −6.91 | −0.21 | ≥.99 | −29.87 to 29.45 | −39.19 to 38.77 | 2.97 | 0.08 | ≥.99 | −11.22 to 11.38 | −14.77 to 14.93 | 1.56 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 |
PARCC | −0.44 | −0.01 | ≥.99 | −1.42 to 1.40 | −1.87 to 1.85 | 1.01 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 | −0.76 | −0.02 | ≥.99 | −2.84 to 2.80 | −3.73 to 3.69 |
Special education status | |||||||||||||||
Yes | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 13.87 | 0.25c | <.001 | 0.10 to 0.40 | 0.06 to 0.44 | 15.41 | 0.24 | ≥.99 | −33.66 to 34.14 | −44.31 to 44.79 | 14.50 | 0.23 | ≥.99 | −32.25 to 32.71 | −42.46 to 42.92 |
PARCC | 2.79 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.62 to 16.80 | −1.52 | −0.04 | ≥.99 | −5.69 to 5.61 | −7.47 to 7.39 | −1.07 | −0.03 | ≥.99 | −4.27 to 4.21 | −5.60 to 5.54 |
Mathematics | |||||||||||||||
i-Ready | −0.79 | −0.02 | ≥.99 | −2.84 to 2.80 | −3.73 to 3.69 | 1.83 | 0.05 | ≥.99 | −7.01 to 7.11 | −9.23 to 9.33 | 4.81 | 0.13 | ≥.99 | −18.23 to 18.49 | −24.00 to 24.26 |
PARCC | 0.60 | 0.02 | ≥.99 | −2.80 to 2.84 | −3.69 to 3.73 | 4.62 | 0.15 | ≥.99 | −21.04 to 21.34 | −27.69 to 27.99 | 5.16 | 0.17 | ≥.99 | −23.84 to 24.18 | −31.39 to 31.73 |
No | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 3.32 | 0.06 | .48 | −0.10 to 0.22 | −0.15 to 0.27 | 5.17 | 0.08 | ≥.99 | −11.22 to 11.38 | −14.77 to 14.93 | 2.41 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 |
PARCC | 1.48 | 0.05 | ≥.99 | −7.01 to 7.11 | −9.23 to 9.33 | 1.62 | 0.05 | ≥.99 | −7.01 to 7.11 | −9.23 to 9.33 | −0.99 | −0.03 | ≥.99 | −4.27 to 4.21 | −5.60 to 5.54 |
Mathematics | |||||||||||||||
i-Ready | 0.06 | 0.00 | ≥.99 | −0.56 to 0.56 | −0.74 to 0.74 | 3.52 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.62 to 16.80 | 2.49 | 0.07 | ≥.99 | −9.82 to 9.96 | −12.92 to 13.06 |
PARCC | −0.05 | 0.00 | ≥.99 | −0.56 to 0.56 | −0.74 to 0.74 | 2.70 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.62 to 16.80 | −1.00 | −0.03 | ≥.99 | −4.27 to 4.21 | −5.60 to 5.54 |
Baseline achievement level | |||||||||||||||
Students in lowest quartile | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 15.54 | 0.28c | <.001 | 0.11 to 0.45 | 0.06 to 0.50 | 7.88 | 0.12 | ≥.99 | −16.83 to 17.07 | −22.16 to 22.40 | 10.27 | 0.17 | ≥.99 | −23.84 to 24.18 | −31.39 to 31.73 |
PARCC | 3.17 | 0.10 | ≥.99 | −14.02 to 14.22 | −18.46 to 18.66 | 0.25 | 0.01 | ≥.99 | −1.40 to 1.42 | −1.85 to 1.87 | 0.95 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 |
Mathematics | |||||||||||||||
i-Ready | 2.18 | 0.07 | ≥.99 | −9.82 to 9.96 | −12.92 to 13.06 | 2.46 | 0.06 | ≥.99 | −8.41 to 8.53 | −11.08 to 11.20 | 4.34 | 0.11 | ≥.99 | −15.43 to 15.65 | −20.31 to 20.53 |
PARCC | 0.09 | 0.00 | ≥.99 | −0.56 to 0.56 | −0.74 to 0.74 | 6.31 | 0.20 | ≥.99 | −28.05 to 28.45 | −36.93 to 37.33 | 1.05 | 0.03 | ≥.99 | −4.21 to 4.27 | −5.54 to 5.60 |
Students in 26th-100th percentile | |||||||||||||||
Reading | |||||||||||||||
i-Ready | 1.50 | 0.03 | .48 | −0.05 to 0.11 | −0.08 to 0.14 | 6.61 | 0.11 | ≥.99 | −15.43 to 15.65 | −20.31 to 20.53 | 2.44 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 |
PARCC | 1.18 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 | 1.38 | 0.04 | ≥.99 | −5.61 to 5.69 | −7.39 to 7.47 | −1.72 | −0.05 | ≥.99 | −7.11 to 7.01 | −9.33 to 9.23 |
Mathematics | |||||||||||||||
i-Ready | −0.86 | −0.03 | ≥.99 | −4.27 to 4.21 | −5.60 to 5.54 | 3.48 | 0.09 | ≥.99 | −12.62 to 12.80 | −16.62 to 16.80 | 2.41 | 0.06 | ≥.99 | −8.41 to 8.53 | −11.08 to 11.20 |
PARCC | 0.05 | 0.00 | ≥.99 | −0.56 to 0.56 | −0.74 to 0.74 | 1.93 | 0.06 | ≥.99 | −8.41 to 8.53 | −11.08 to 11.20 | −0.24 | −0.01 | ≥.99 | −1.42 to 1.40 | −1.87 to 1.85 |
Abbreviations: PARCC, Partnership for Assessment of Readiness for College and Careers; SY, school year.
The models adjusted for student grade level (3-7), prior achievement (testing scores), and blocking variables used in randomization (charter school status, school type, pilot study participation, school proportion of low-income and Black students, and whether the school served more than 25% Latinx students). Effect size estimates presented are model-adjusted estimates.
The difference in score on a particular academic test between the intervention and control groups measured in SD units.
Statistically significant after Bonferroni correction at 2-sided P < .001 level, calculated by conducting a test to see if intervention × interaction = 0 after running the relevant model. Multiple comparison corrections were applied to P values using the Bonferroni procedure. P < .05 was considered statistically significant.