Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Nov 1.
Published in final edited form as: Learn Disabil Res Pract. 2018 Jul 24;33(4):229–236. doi: 10.1111/ldrp.12171

Upside-Down Response to Intervention: A Quasi-Experimental Study

Bobette Bouton 1, Laura A Barquero 2, John R McConnell 1, Donald L Compton 3, Jennifer K Gilbert 2
PMCID: PMC6328259  NIHMSID: NIHMS960942  PMID: 30636844

Introduction

Response to intervention (RTI) as a framework for early identification and prevention of reading difficulties has been increasingly employed in recent years (Berkeley, Bender, Peaster, & Saunders, 2009). RTI prevention systems are typically conceptualized as a three-tier system in which children receive more intensive and individualized instruction as they move down the tiers. In the first tier, all students participate in generally effective reading instruction in the regular classroom, and each student’s rate of reading growth is monitored. Children whose level of performance or rate of improvement is dramatically below that of peers (based on classroom, school, district, state, or national norms) are designated as at-risk for poor reading outcomes and possibly reading disability (RD). According to Tran et al. (2011), low response to intervention is associated with significantly lower performance at pretest on measures such as word identification, word attack, rapid naming speed, phonological awareness, reading fluency, and reading comprehension. Such children move to the second tier in the RTI process. In this tier, children receive small-group instruction, and their progress is again monitored. Children whose level of performance or rate of improvement in Tier 2 is dramatically below that of peers are designated to receive Tier 3 instruction, which typically involves more intensive individualized instruction. Thus, failure to respond appropriately to Tier 2 and 3 instruction signals the need for special education evaluation (see Fuchs et al., 2003).

A central tenet of RTI is that children are accurately identified as at-risk by their progression through the tiers, minimizing both “false positives” and “false negatives” when determining at-risk status (Fuchs, Compton, Fuchs, Bryant, & Davis, 2008). Studies indicate that children who are low responders or non-responders to Tier 1 and Tier 2 instruction generally start out lower than children who are responsive to intervention (Compton et al., 2012; Toste et al., 2014; Tran, Sanchez, Arellano, & Swanson, 2011). A former study (Compton et al., 2012) addressed this question more directly by predicting Tier 2 nonresponse with the following types of data: screening, responsiveness to Tier 1, norm-referenced tests, and responsiveness to Tier 2. They found that students who failed to respond to Tier 2 could be predicted without assigning students to and measuring progress in Tier 2. Taken together, these studies suggest that children who perform exceptionally low prior to Tier 2 assignment could be identified to advance directly from Tier 1 to Tier 3 in an upside down RTI (udRTI) framework.

Those children who are most at-risk for reading failure are likely to be far behind their peers in Tier 1 and then are subjected to further failure in Tier 2 before their individual needs are finally addressed in Tier 3 (Compton et al., 2012). Perhaps students who are markedly behind in achievement would be better served (and resources better utilized) if they had the opportunity to progress directly from Tier 1 to Tier 3. This would provide individually targeted, intensified instruction, and more frequent one-on-one instruction much earlier and would allow these students to experience success sooner rather than failing in Tier 2 while watching others succeed. Though the students initially bypass Tier 2 in this framework, with adequate progress they could potentially later move to Tier 2, and with completed remediation back to Tier 1. This is the idea behind the creation of this study. The posited question that drove the study is as follows: When students who have been identified as the most at-risk for reading difficulties based on pre-assessment data, does fast-tracking these students from Tier 1 to Tier 3 within the RTI model increase student learning in word-level reading skills?

Literature Review

RTI is now widely implemented in our school systems. However, the current model has only limited research in Tier 3 of the RTI model (Al Otaiba et al., 2014; Gilbert et al., 2013), yet students who are unresponsive to Tier 2 and require Tier 3 intervention make up approximately 5% of students (Lam & McMaster, 2014). These students who are most at-risk for academic failure move through each stage of the current RTI model at the same pace as students who will never need Tier 3 instruction, setting up a scenario in which they fall further and further behind (Al Otaiba et al., 2014; Compton et al., 2012).

Al Otaiba, Kim, Wanzek, Petscher, and Wagner (2014) published research addressing the inability of at-risk students to promptly advance to the tier of need within the current RTI program. In this study a cohort of students was allowed to move directly to Tier 3 based on appropriate test scores and students in this study made statistical reading gains. The Al Otaiba et al. (2014) study used two cohorts of students. The first cohort used what the study referred to as the “typical RTI” format, and the second used a “dynamic RTI” concept in which students were fast tracked to Tier 2 or 3, depending on their reading skill profile. All other conditions were the same for both groups of students with the exception of when they were provided with the intervention. All participants were assessed in both fall and spring. Results found no significant differences in the two cohorts of students for the Fall Reading factor scores, but found statistically significant higher Spring Reading scores for the dynamic RTI group (Al Otaiba et al., 2014).

The primary difference between this study and the upside-down RTI study is that, in the current study, Tier 3 was one-on-one tutoring. In Al Otaiba et al. (2014), Tier 3 consisted of small groups of 1–3 students. According to Greulich et al. (2014), “Our interventionists expressed they wished they could have provided some one-on-one sessions to learn if…emotions could be changed through individualized attention and increased mastery” (p. 11). This primary difference between a one-on-one approach and a one-to-three approach is where we believe our study separates from previous studies.

In addition to academic stagnation, some students can also experience a loss of confidence. “The child who fails initially to achieve reading skills will soon develop a lack of confidence in his/her ability to succeed. S/he will begin to avoid potentially humiliating situations and will refuse to take risks for fear of failure” (Galbraith & Alexander, 2005, p. 29). A positive relationship between academic achievement and self-esteem has been found in several studies, a finding that, also supports this idea (El-Anzi, 2005; Kaniuka, 2010). In addition to esteem and confidence issues, anxiety-related concerns are also more prevalent among students with learning disabilities (Sideridis et al., 2006). Sideridis et al. (2006) found that “striving to outperform classmates appears to be a significant negative predictor of comprehension difficulties” (p. 174). This competitive nature can be present in both Tiers 1 and 2 of the RTI model, in whole-class and/or small-group instruction. However, with a one-on-one academic setting as designed in Tier 3 of the udRTI model, the humiliation and competitive factors are naturally removed, allowing students to potentially make greater strides in their academic learning.

Over three decades have passed since Carol Dweck (1986) found links to student motivation and students’ own perceived views of their intelligence. Dweck (2006) has since popularized the term “growth mindset” to help explain this phenomenon in learning in her book Mindset: The New Psychology of Success. Growth mindset is “based on the belief that your basic qualities are things you can cultivate through your efforts, your strategies, and help from others” (Dweck, 2006, p. 7). This theory of student motivation works within the udRTI model. Using the udRTI, model allows students to work one-on-one with a tutor more quickly than with the current RTI framework. The tutors are able to work with the student independently to foster these growth mindset ideas, instead of the student becoming more frustrated in a group setting where they still feel that they are behind their peers academically.

Purpose and Research Question

The purpose of this quasi-experimental study was to explore a RTI design in which Tiers 2 and 3 are inverted for the most at-risk first grade students in reading intervention. The study was conducted as an exploratory study in the final year of data collection of a larger RTI study (Compton et al., 2010; Compton et al., 2012; Gilbert et al., 2013). The final phase of this study addressed the following research question:

For students who have been identified as the most at-risk for reading difficulties based on pre-assessment data, does fast tracking these students from Tier 1 to Tier 3 within the RTI model increase student learning in word-level reading skills?

Method

Participants

Upside-down RTI Intervention Participants

First grade students (n=24) who were determined to be at-risk for reading difficulties and who exhibited low responsiveness to Tier 1 instruction participated in the 32-week study. The students were from seven classrooms in two urban, public schools in a major city in the southeastern United States. Both schools had 300–400 enrolled students with greater than an 85% free or reduced lunch enrollment. The schools had a traditional school calendar with students attending school from mid-August through late May during the 2009–2010 school year. The sample was composed of 11 females and 13 males; 91.7% African-American and 8.3% Caucasian; 91.7% free or reduced lunch; 4.2% IEP; and 0% retained. The demographics of the sample align with the demographics of the schools (School A had 99% African-American/Hispanic/Native American enrollment, School B had 86% African-American/Hispanic/Native American enrollment). Students were teacher-identified as performing poorly in reading. To identify students for screening, teachers were asked to designate the lower half of readers in the classroom; however, teachers were encouraged to suggest additional students for screening as appropriate (i.e., large numbers of students in the class were considered below grade level in reading as determined by the teacher). Students were eligible for the study if they scored above a 70 on the Wechsler Abbreviated Scale of Intelligence (WASI) measure, and had parental consent. All teacher-selected students, who also met the above criteria, were administered short-term progress monitoring for six weeks. From there, growth modeling was used and students were rank-ordered on the resulting 6-week WIF intercept and slope. The lowest 24 students were selected from these progress monitoring data.

Regular RTI Participants

First grade students (n=24) who were determined to be at-risk for reading difficulties, and who exhibited low responsiveness to Tier 1 instruction participated in the 32-week study in two different cohorts. The students were from 12 classrooms in nine urban, public schools in a major city in the southeastern United States. The schools had a traditional school calendar, with students attending school from mid-August through late May during the 2006–2009 school years. The students consisted of 10 females and 14 males; 50% African-American, 37.5% Caucasian, 4.2% Hispanic, and 8.3% Multiracial; 75% free or reduced lunch; 25% IEP; and 12.5% retained. These demographics align with the demographics of the schools. Students were eligible for the study if they scored above a 70 on the WASI measure, were teacher-recommended, had parental consent, and scored in the low range on a word reading factor score. These students were selected based on propensity matching using the Woodcock Reading Mastery Test – R/NU: Word identification (WRMT-R: WID), Woodcock Reading Mastery Test –R/NU: Word Attack (WRMT-R: WA), The Test of Sight Word Reading Efficiency (TOWRE: SWE) and The Test of Decoding Efficiency (TOWRE: PDE) pretest scores (see measures section for descriptions). The propensity matching procedure will be explained in the data analysis and results sections.

Classroom teachers

The classroom teachers were recruited in the fall of the corresponding school year in which they participated. The teachers provided the initial list of the lowest performing readers.

Intervention tutors

The intervention tutors were masters and doctoral students working part-time as research assistants. The tutors received 40 hours of training prior to intervention. In addition, all tutors underwent fidelity checks prior to all testing and tutoring, as well as fidelity and interrater reliability checks during the intervention process. All tutors had prior experience working with school-age students.

Description of Intervention

Prior to tutoring, a (researcher-designed) screener was administered to determine individual instruction level and starting position within the intervention. Participants in udRTI received tutoring five days a week for 30 minutes per lesson. Tutoring consisted of scripted instruction in letter sounds, sight words, decoding, and reading fluency. Students received individual tutoring for seven weeks, during which Word Identification Fluency (WIF) progress monitoring measures were administered on a weekly basis. Growth modeling was used to determine responsiveness to intervention and a median split designated placement for the second phase of tutoring. In the eighth week of intervention, those students in individual tutoring continued with intervention five days per week for 30 minutes per session. Those students who made gains in individual tutoring moved to small groups, consisting of 2–3 students per small group, with tutoring three days per week for 45 minutes per session. The tutoring program remained the same for small group tutoring except that it was scripted for the participants to take turns reading during the session. WIF progress monitoring continued through the seven weeks of the second phase of tutoring.

The udRTI tutoring model was identical to the regular RTI tutoring model used in previous years. The only difference was that Tiers 2 and 3 were flipped in regard to all students. The udRTI began with Tier 3 and moved to Tier 2 at the midpoint of the study for students who showed measured progress through the scripted intervention.

Measures

Progress monitoring: Word identification fluency (WIF)

Timed (1-min) reading of a single-page list of 100 high-frequency words sampled from the pre-primer, primer, and first-grade level Dolch list (multiple test forms; Fuchs, Fuchs, & Compton, 2004). Test-retest reliability exceeded .90 for the larger study (Compton et al., 2012).

Rapid Letter Naming

Timed naming of an array of the 26 letters presented on a single sheet of paper. The score is the number of letters correctly identified in one minute. Test-retest reliability of RLN exceeds 95. Scores were prorated if the participant named all items in less than one minute. (researcher designed)

Untimed word identification skill

Woodcock Reading Mastery Test – R/NU: Word Identification (WRMT-R: WID, Woodcock, 1998a). A norm-referenced test that requires reading individual words ordered in increasing difficulty. Test ceiling of six sequential incorrect responses. Split-half reliability exceeded .90 for the normative first-grade sample (Woodcock, 1998b).

Untimed decoding skill

Woodcock Reading Mastery Test – R/NU: Word Attack (WRMT-R: WA, Woodcock, 1998a). A norm-referenced test that requires pronunciation of decodable pseudowords presented in ordered difficulty. Test ceiling of six sequential incorrect responses. Split-half reliability exceeded .90 for the normative first-grade sample (Woodcock, 1998b).

Sight word reading efficiency

The Test of Sight Word Reading Efficiency (TOWRE: SWE), (Torgesen, Wagner, & Rashotte, 1999a). A norm-referenced measure of sight word reading accuracy and fluency that requires reading a list of single words of increasing difficulty for 45 sec. Split-half reliability exceeded .91 for the normative first-grade sample (Torgesen, Wagner, & Rashotte, 1999b).

Phonemic decoding efficiency

The Test of Phonemic Decoding Efficiency (TOWRE: PDE, Torgesen et al., 1999a). A norm-referenced measure of decoding accuracy and fluency that requires reading a list of decodable pseudowords of increasing difficulty for 45 sec. Split-half reliability exceeded .90 for the normative first-grade sample (Torgesen, Wagner, & Rashotte, 1999b).

IQ measure

The Vocabulary subtest on the Wechsler Abbreviated Scale of Intelligence (WASI); (Wechsler, 1999) assesses expressive vocabulary, verbal knowledge, and foundation of information with 42 items. The participant is asked to orally define orally presented, increasingly difficult words. Responses are awarded a score of 0, 1, or 2 depending on quality. Testing is discontinued after five consecutive scores of 0. Split-half reliability is .86 to .87 at ages 6 to 7; the correlation with the Wechsler Intelligence Scale for Children–III Full Scale IQ is .72 (Wechsler, 1999b).

Matrix Reasoning on the WASI (Wechsler, 1999a) measures nonverbal reasoning using 2-dimensional patterns of differing colors and shapes presented in increasing difficulty. Examinees look at a matrix from which a section is missing and complete the matrix by saying the number of, or pointing to, one of five response options. Testing is discontinued after four errors on five consecutive items or after four consecutive errors. The score is the number of correct responses. As reported by the test developer, reliability is .94 to .96 for 6- to 7-year-olds (Wechsler, 1999b).

udRTI Procedure

Teachers were recruited in August 2009, and all first-grade teachers at each school (3 classrooms at school A, 4 classrooms at school B) participated in the study. Teachers provided lists of the lowest performing readers, and, after obtaining written parental consent, these children were screened with Rapid Letter Naming (RLN) and Word Identification Fluency (WIF) measures. A total of 61 students were screened in September. Of these, the 48 students with the lowest scores were selected for further screening via progress monitoring.

The 48 selected students received WIF progress monitoring each week for 6 weeks. One student moved during progress monitoring. For the remaining students, growth modeling of the progress monitoring data was used to identify the 24 lowest performing students. This sample size was determined based on study logistics and research personnel needed to accommodate the tutoring portion of the study. In November, pretesting was administered as two batteries given on separate days. Pretest batteries included TOWRE SWE, TOWRE PDE, WRMT-R WID, and WRMT-R WA. Tutoring sessions began after the pretest phase was completed. Tutoring was administered during the school day. Following the 14 weeks of tutoring, post-test was administered in April 2010. Post-test measures included TOWRE SWE, TOWRE PDE, WRMT-R WID, and WRMT-R WA.

Fidelity and Reliability of Implementation

Prior to testing or tutoring all research assistants had to pass a fidelity of administration on all testing and tutoring instruments with 90% or greater accuracy. In addition, each tutoring session was audio-recorded. A checklist of 64 tutoring steps based on the tutoring script was used to evaluate fidelity of implementation. A random sample of 20% of all audio recorded sessions was evaluated, revealing that tutors implemented the tutoring steps with over 90% accuracy for both the udRTI group and the regular RTI group.

Testing sessions were also audio-recorded. Inter-rater reliability exceeded 80% for all outcome measures in both udRTI and regular RTI. Because the sample size for udRTI was small, inter-rater scoring was conducted on the entire udRTI group, whereas for regular RTI the sample for inter-rater reliability was performed on a random 10% of the whole group from the larger study (N=649) (Gilbert et al., 2013).

Data Analysis

Because many variables could be potential confounders in this exploratory study, an attempt was made to control for confounding influences on word-level reading skill. Regular RTI participants were matched with udRTI participants based on propensity scores, as is increasingly common in educational research for evaluating behavioral and instructional interventions (Hong & Raudenbush, 2005; Hughes, Chen, Thoemmes, & Kwok, 2010; Thoemmes & Kim, 2011). Student propensity scores - i.e., predicted probabilities of being at risk for reading difficulties and unresponsive to Tier 1 intervention - for all the students were obtained using a binary logistic regression with the dependent variable being group and the covariates being WIF and pretest scores for PDE, SWE, WA, and WID. Regular RTI participants were then selected using simple 1:1 near neighbor matching of their propensity scores with those who received udRTI. Examination of standardized differences and frequency distributions revealed no imbalances of the covariates across those receiving regular and udRTI, thus resulting in matched samples of first graders, evenly distributed in the two groups.

To address the question of whether students differed in word-level reading skill, a multivariate analysis of variance (MANOVA) was conducted, in accordance with the methodology set forth by Cramer and Bock (1966), to help protect against inflating the Type I error rate in the follow-up analyses of variance (ANOVAs). Whether students received regular RTI or udRTI constituted the independent variable in the MANOVA, while post-test scores in PDE, SWE, WA, and WID served as the dependent variables. Wilk’s lambda was used as the test statistic for the MANOVA and was calculated as

Λ=∏i=1q11+λi

with q dependent variables in the MANOVA and λi denoting the sum of squares ratio (i.e., eigenvalue) for the ith canonical variate (Hand & Taylor, 1987). Partial eta squared (ηp2) was used to determine effect sizes and was calculated as

ηp2=dfeffect×Feffect(dfeffect×Feffect)+dferror.

Because the error terms for each comparison in this study were the same, using ηp2 estimates was deemed appropriate and defined as either small (ηp2 ≤ .03), medium (.03 < ηp2 ≤ .06), or large (ηp2 > .06), in accordance with those suggested by Cohen (1988). It should be noted, however, that care should be taken when comparing these estimates with those from other studies, as the error terms must be comparable in order to do so (Fritz, Morris, & Richler, 2012).

Results

Overall Word Level Reading

To avoid problems with multicollinearity, a series of Pearson correlations was performed between all pairwise combinations of the dependent variables, to verify that they correlated with each other in the moderate range (i.e., .30 – .90; Brace et al., 2006). In addition to their descriptive statistics, Table 1 presents the meaningful pattern of correlations found among the dependent variables, suggesting the appropriateness of MANOVA with these variables. Moreover, the Cronbach’s alpha for these scales was .83, indicating high consistency (i.e., ≥ .60; Netemeyer, Bearden, & Sharma, 2003) among these four measures of reading proficiency.

Table 1.

Correlations, Means, and Standard Deviations Associated with the Dependent Variables

1. 2. 3. 4. M SD
1. PDE 1.00 3.52 3.39
2. SWE .60* 1.00 17.50 8.07
3. WA .63* .47* 1.00 5.50 4.46
4. WID .67* .89* .56* 1.00 20.44 10.35
Open in a new tab

Note. N = 48. PDE = Phonemic Decoding Efficiency (Test of Word Reading Efficiency), SWE = Sight Word Efficiency (Test of Word Reading Efficiency), WA = Word Attack (Woodcock-Johnson III), WID = Word Identification (Woodcock-Johnson III).

*

p < .01.

A one-way MANOVA was conducted to test the hypothesis that there would be one or more mean differences between student groups in terms of overall word level reading. A p < .10 criterion was established, given the preliminary nature of this study. Based on Huberty and Petoskey's (2000) guideline, the multivariate test for homogeneity of dispersion matrices was not statistically significant (Box's M = 11.520; F(10,10116) = 1.043; p = .404), thus showing that this assumption was met. A statistically significant MANOVA effect was obtained, Wilks’ Λ = .828; F(4,43) = 2.228; p = .082, with the multivariate effect size (ηp2) estimated at .172, a large effect. This implied that 17.2% of the variance in the canonically derived dependent variable was attributable to the difference in intervention. Examining the means between student groups (see Table 2) indicated that udRTI worked better than regular RTI in regard to reading proficiency, with a 10% probability that this difference was due to random sampling error.

Table 2.

Pretest and Post-test Raw (and Standard) Scores by RTI Framework Group

Regular RTI udRTI
Measure M SD M SD
Pretest
  PDE 2.21 (−.14) 2.96 (.92) 1.46 (−.38) 2.70 (.84)
  SWE 9.21 (−.63) 6.17 (.79) 12.25 (−.24) 5.09 (.65)
  WA 3.21 (−.29) 3.51 (.86) 3.33 (−.26) 3.32 (.81)
  WID 11.21 (−.56) 8.98 (.91) 15.04 (−.18) 6.44 (.65)
Post-test
  PDE 2.75 (.02) 3.03 (.94) 4.29 (.50) 3.62 (1.12)
  SWE 14.25 (.02) 7.61 (.98) 20.75 (.85) 7.29 (.94)
  WA 4.46 (.02) 4.32 (1.06) 6.54 (.53) 4.43 (1.08)
  WID 16.83 (.01) 10.81 (1.09) 24.04 (.73) 8.65 (.87)
Open in a new tab

Note. N = 24 for each cell. Raw scores were transformed to sample specific z scores. PDE = Phonemic Decoding Efficiency (Test of Word Reading Efficiency), SWE = Sight Word Efficiency (Test of Word Reading Efficiency), WA = Word Attack (Woodcock-Johnson III), WID = Word Identification (Woodcock-Johnson III).

Individual Measures of Word Level Reading Skill

Prior to conducting a series of follow-up ANOVAs, the homogeneity of variance assumption was tested for all four dependent variables. Based on a series of Levene’s F tests, the homogeneity of variance assumption was considered satisfied for all variables. For each dependent variable, a one-way ANOVA was conducted to test for differences between regular and udRTI groups in the respective individual word level reading skill.

As Table 3 shows, the ANOVA results for SWE and WID were statistically significant, with ηp2 estimates of .166 and .124, respectively. These indicated that the means between student groups were most notably different for these two measures of word level reading, with students receiving udRTI improving more in these skills than those receiving regular RTI.

Table 3.

One-way ANOVA Results

Levene’s ANOVAs Regular RTI udRTI
F(1,46) p F(1,46) p ηp2 CI90 M SD M SD d
PDE .96 .333 2.57 .116 .05 [.00; .18] 2.75 3.03 4.29 3.62 .46
SWE .28 .598 9.13 .004* .17 [.03; .32] 14.25 7.61 20.75 7.29 .87
WA .40 .530 2.72 .106 .06 [.00; .19] 4.46 4.32 6.54 4.43 .48
WID 3.74 .059 6.51 .014* .12 [.01; .27] 16.83 10.81 24.04 8.65 .74
Open in a new tab

Note. N = 48. PDE = Phonemic Decoding Efficiency (Test of Word Reading Efficiency), SWE =Sight Word Efficiency (Test of Word Reading Efficiency), WA = Word Attack (Woodcock-Johnson III), WID = Word Identification (Woodcock-Johnson III).

*

p < .025, using the Bonferroni adjustment for protection of an experiment-wise error rate (αe) of .10.

Discussion

This study examined the effects of an udRTI approach on the word-level reading skill of first graders at-risk for reading difficulties compared to those receiving the regular RTI approach. A quasi-experimental exploratory design was used in which students who participated in udRTI were matched via propensity scores with students from prior years of the same study that utilized a regular RTI framework. The same intervention was used in both udRTI and regular RTI, so that the two frameworks of implementation could be directly compared. The intervention largely emphasized word-level reading skills, with focus on letter-sound correspondence, sight-word recognition, and decoding, and also included spelling and fluency. Statistically significant effects were found for the intervention on word reading measures (SWE and WID); however, differences for decoding measures (PDE and WA) were not found to be statistically significant. The significant growth in word identification may indicate that, within the udRTI framework, the intervention facilitates development of word identification skill over that of decoding, presumably due to earlier and increased opportunities for individualized word reading instruction. The lack of significant growth in udRTI for decoding measures may suggest that udRTI provides no additional benefit over regular RTI for strengthening of decoding skill. However, given that the PDE and WA had effect sizes of .025 (small) and .037 (medium), respectively, a larger sample may demonstrate a significant positive impact of udRTI on these measures as well. The results are encouraging, and the importance of meeting the needs of our most struggling students warrants the continued study of an udRTI model with a larger sample. The limitations and implications of the study are detailed in the following sections.

Limitations

While propensity score matching, like the more traditional regression approach, attempts to limit the bias of confounding variables on effect estimates, this is only achievable insofar as the extent to which the covariate(s) accounting for the confounding variance in the effect are observed and included in the model. We performed propensity matching using baseline word identification fluency, PDE, SWE, WA, and WID. We did not match on other variables such as school, teacher, SES, race, or gender. In addition, regular RTI and udRTI were implemented in different calendar years. Variables that were not included in the propensity matching process could play a role in responsiveness to the intervention.

Limited interpretation of the results is also necessitated by the low statistical power of the analyses that derives from the small sample size used for the study. The current study was constrained by the logistics of implementing udRTI as an exploratory offshoot of a larger study that was nearing completion; nonetheless, the results provide justification for conducting studies that are designed to investigate udRTI specifically. Further investigation with larger sample sizes is encouraged, to detect any true effects that may exist with small differences between groups. Because this was a research study with a scripted intervention, the intervention could not be modified to meet individual needs. It is possible that greater effects could be achieved using interventions that allow for individualization.

The two schools that were selected for the udRTI sample were selected because they had been the schools with the largest number of students performing in the lowest 25% in previous years of the study. The research team determined that these schools would be the most efficient to focus on for recruiting students at-risk for reading difficulties. Not surprisingly, the two schools had a high proportion of students of low SES. The regular RTI sample drew from a larger number of schools, with broader demographics. Because of the focus in udRTI on schools with the greatest number of at-risk students, the demographics of the udRTI and the regular RTI samples are quite different. While this is a limitation of the study, in that it may limit generalizability, this decision in study design was intended to target at-risk participants most efficiently.

It is also important to note that using the udRTI model could potentially result in a higher rate of false positives with more limited testing and monitoring occurring when fast-tracking students to Tier 3. Because of the exploratory, small-scale nature of this study it is difficult to determine whether false positives occurred and students were fast-tracked unnecessarily. However, if false positives occur it is also possible that these students will be identified quickly in Tier 3 and will be able to be moved back to Tier 2. A false positive in this instance would not be disruptive to the students’ learning, but could play a role financially for school districts if students are placed in a one-on-one, Tier 3 model instead of a group, Tier 2 learning setting.

An additional limitation for this study was that no data were collected involving socio-emotional growth. In addition, the study did not specifically look at the impact of individual intervention as compared to small group intervention regarding anxiety or self-esteem issues. As the literature review indicates, individual intervention can benefit students who are at risk for reading difficulties, but this study did not specifically look at or monitor students’ emotional growth due to grant and funding obligations and resources.

Implications

In 2004 the reauthorization of the Individuals with Disabilities Education Improvement Act allowed for RTI to be used as an alternative approach for identifying students with learning disabilities. “The basic assumption is that RTI can differentiate between two explanations of low achievement: poor instruction versus disability” (Fuchs, Fuchs, & Compton, 2004, p. 216). A 2014 10-year update of RTI “provided support for current practices of including word identification, alphabetic principle, fluency, and phonemic awareness for Tier 1 screening batteries” in order to predict students’ advancement to Tier 2 (Lam & McMaster, 2014, p. 11). This same report also suggested that, with the use of the same batteries, students for whom Tier 2 most likely will not be enough might benefit educationally from fast-tracking to Tier 3 (Lam & McMaster, 2014). However, although most states utilize Tier 3 in the RTI model, differences within Tier 3 are the most varied among the tiers. This can create confusion and an inability to effectively determine what is the most beneficial to students (Berkeley, Bender, Peaster, & Saunders, 2009).

With the introduction of RTI into our school systems, administrators and teachers alike must make critical decisions on how to utilize RTI in their schools. If the most at-risk students are not academically and emotionally achieving to the best of their ability in the current RTI model, then it seems logical to look for other avenues for success. The udRTI model uses the most influential parts of the current RTI model, but alters it to benefit the students who are at the lowest academic end. Instead of allowing these students to stay at status quo for additional and valuable academic weeks the udRTI model allows these students to get the one-on-one individualized instruction that they need not only to help them academically, but also to give them the self-esteem and emotional support needed to learn. This model could vastly improve the state of academic growth for those students who are learning to hate learning at a very early age due to constant academic failure. We believe that this study is the starting point for further research in this area and has the potential to meet all students where they are.

Acknowledgments

The research reported here was supported by the Institute of Education Sciences, US Department of Education, Grant R324G060036, Vanderbilt Kennedy Center, NICHD Grant P30 HD15052, and Department of Special Education, Peabody College, Vanderbilt University. The opinions expressed are those of the authors and do not represent views of the above supporters.

References

  1. Al Otaiba S, Connor CM, Folsom JS, Wanzek J, Greulich L, Schatschneider C, Wagner RK. To wait in tier 1 or intervene immediately: A randomized experiment examining first grade response to intervention (rti) in reading. Exceptional Children. 2014;81(1):11–27. doi: 10.1177/0014402914532234. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Al Otaiba S, Kim YS, Wanzek J, Petscher Y, Wagner RK. Long-term effects of first-grade multitier intervention. Journal of Research on Educational Effectiveness. 2014;7(3):250–267. doi: 10.1080/19345747.2014.906692. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Berkeley S, Bender WN, Peaster LG, Saunders L. Implementation of response to intervention: A snapshot of progress. Journal of Learning Disabilities. 2009;42(1):85–95. doi: 10.1177/0022219408326214. [DOI] [PubMed] [Google Scholar]
  4. Brace N, Kemp R, Snelgar R. SPSS for psychologists. NY: Palgrave Macmillan; 2006. [Google Scholar]
  5. Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
  6. Compton DL, Fuchs L, Fuchs D, Bouton B, Gilbert JK, Barquero LA, Cho E, Crouch RC. Selecting at-risk first grade readers for early intervention: Eliminating false positives and exploring the promise of a two-stage gated screening process. Journal of Educational Psychology. 2010;102(2):327–340. doi: 10.1037/a0018448. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Compton DL, Gilbert JK, Jenkins JR, Fuchs D, Fuchs LS, Cho E, Barquero LA, Bouton BD. Accelerating chronically unresponsive children to tier 3 instruction: What level of data is necessary to ensure selection accuracy? Journal of Learning Disabilities. 2012;45:204–216. doi: 10.1177/0022219412442151. [DOI] [PubMed] [Google Scholar]
  8. Cramer EM, Bock RD. Multivariate analysis. Review of Educational Research. 1966;36:604–617. [Google Scholar]
  9. Dweck CS. Motivational processes affecting learning. American Psychologist. 1986;41(10):1040–1048. [Google Scholar]
  10. Dweck CS. Mindset: The new psycholgy of success. New York: Penguin Random House; 2006. [Google Scholar]
  11. El-Anzi FO. Academic achievement and its relationship with anxiety, self-esteem, optimism, and pessimism in Kuwaiti students. Social Behavior and Personality. 2005;33(1):95–104. [Google Scholar]
  12. Fritz CO, Morris PE, Richler JJ. Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General. 2012;141:2–18. doi: 10.1037/a0024338. [DOI] [PubMed] [Google Scholar]
  13. Fuchs D, Compton DL, Fuchs LS, Bryant J, Davis GN. Making “secondary intervention” work in a three-tier responsiveness-to-intervention model: Findings from the first grade longitudinal reading study of the National Research Center on Learning Disabilities. Reading and Writing. 2008;21(4):413–436. [Google Scholar]
  14. Fuchs D, Fuchs LS, Compton DL. Identifying reading disabilities by responsiveness-to-intervention: Specifying measures and criteria. Learning Disabilities Quarterly. 2004;27(4):216–227. [Google Scholar]
  15. Fuchs D, Mock D, Morgan PL, Young C. Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research & Practice. 2003;18(3):157–171. [Google Scholar]
  16. Galbraith A, Alexander J. Literacy, self-esteem and locus of control. Support for Learning. 2005;20(1):28–34. [Google Scholar]
  17. Gilbert JK, Compton DL, Fuchs D, Fuchs LS, Bouton B, Barquero LA, Cho E. Efficacy of first-grade responsiveness-to-intervention prevention model for struggling readers. Reading Research Quarterly. 2013;48(2):135–154. [Google Scholar]
  18. Greulich L, Al Otaiba S, Schatschneider C, Wanzek J, Ortiz M, Wagner RK. Understanding inadequate response to first-grade multi-tier intervention: Nomothetic and ideographic perspectives. Learning Disability Quarterly. 2014;37(4):204–217. doi: 10.1177/0731948714526999. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hong G, Raudenbush SW. Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis. 2005;27:205–224. [Google Scholar]
  20. Huberty CJ, Petoskey MD. Multivariate analysis of variance and covariance. In: Tinsley H, Brown S, editors. Handbook of applied multivariate statistics and mathematical modeling. NY: Academic Press; 2000. [Google Scholar]
  21. Hughes JN, Chen Q, Thoemmes F, Kwok O. An investigation of the relationship between retention in first grade and performance on high stakes tests in third grade. Educational Evaluation and Policy Analysis. 2010;32:166–182. doi: 10.3102/0162373710367682. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Kaniuka TS. Reading achievement, attitude toward reading, and reading self-esteem of historically low achieving students. Journal of Instructional Psychology. 2010;37(2):184–188. [Google Scholar]
  23. Lam EA, McMaster KL. Predictors of responsiveness to early literacy intervention: A 10-year update. Learning Disability Quarterly. 2014;37(3):134–147. [Google Scholar]
  24. Netemeyer RG, Bearden WO, Sharma S. Scaling procedures: Issues and applications. Thousand Oaks, CA: Sage Publications; 2003. [Google Scholar]
  25. Sideridis GD, Mouzaki A, Simos P, Protopapas A. Classification of students with reading comprehension difficulties: The roles of motivation, affect, and psychopathology. Learning Disabilities Quarterly. 2006;29:159–179. [Google Scholar]
  26. Thoemmes F, Kim ES. A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research. 2011;46:90–118. doi: 10.1080/00273171.2011.540475. [DOI] [PubMed] [Google Scholar]
  27. Torgesen JK, Wagner RK, Rashotte CA. Test of word reading efficiency. Austin, TX: Pro-Ed; 1999a. [Google Scholar]
  28. Torgesen JK, Wagner RK, Rashotte CA. TOWRE, Test of Word Reading Efficiency: Examiner’s manual. Austin, TX: PRO ED; 1999b. [Google Scholar]
  29. Toste JR, Compton DL, Fuchs D, Fuchs LS, Gilbert JK, Cho E, Barquero LA, Bouton B. Understanding unresponsiveness to tier 2 reading intervention: Exploring the classification and profiles of adequate and inadequate responders in first grade. Learning Disability Quarterly. 2014;37(4):192–203. [Google Scholar]
  30. Tran L, Sanchez T, Arellano B, Swanson HL. A meta-analysis of the RTI literature for children at risk for reading disabilities. Journal of Learning Disabilities. 2011;44(3):283–295. doi: 10.1177/0022219410378447. [DOI] [PubMed] [Google Scholar]
  31. Wechsler D. Wechsler abbreviated scale of intelligence. San Antonio: Psychological Corporation; 1999a. [Google Scholar]
  32. Wechsler D. Wecshler Abbreviated Scale of Intelligence manual. San Antonio: The Psychological Corporation; 1999b. [Google Scholar]
  33. Woodcock RW. Woodcock reading mastery tests-revised-normative update. Circle Press, MN: AGS; 1998a. [Google Scholar]
  34. Woodcock RW. Woodcock reading mastery test-revised: Examiner’s manual. Circle Pines, MN: American Guidance Service; 1998b. [Google Scholar]

RESOURCES