Response to Intervention: Prevention and Remediation, Perhaps. Diagnosis, No.

Abstract

Fletcher and Vaughn (this issue) describe recent changes to federal laws governing special education eligibility for specific learning disabilities focusing on what is commonly known as response to intervention (RTI). We are concerned about what appears to us as a selective review of empirical support for RTI and a consequently overly optimistic view of many practical issues surrounding the implementation of RTI models that neglects the potential negative long-term impact on the range of students with and without a learning disability. These include (1) the lack of a firm evidence base reflected in vagaries and ambiguity of the critical details of the model in practice; (2) the worrisome shortcomings of the RTI process as a means of diagnosis or determination of a disability; (3) the contextual, situation-dependent nature of who is identified; (4) the seeming lack of consideration of bright struggling readers in the RTI process; and (5) the apparent lack of student-based data to guide the most effective choice of approaches to, and specific components of, intervention.

Fletcher and Vaughn (this issue) describe recent changes to federal laws governing special education eligibility for specific learning disabilities (LDs) focusing on what is commonly known as response to intervention (RTI). They note a shift to RTI models of service delivery explicated in the Individuals with Disabilities Education Act (IDEA; U.S. Department of Education, 2004), and indicate what they consider to be a standard protocol model of RTI. Although the notion of RTI as a process of service delivery may have potential, we do not believe the extant evidence supports Fletcher and Vaughn’s enthusiasm for its readiness, and we believe they are overly optimistic and incomplete in their presentation of the lack of evidence and its implications for acceptance of RTI.

Fletcher and Vaughn state the primary goal of RTI is “improved academic and behavioral outcomes for all students” with a “secondary goal of RTI” to “provide data for identification of learning disabilities (LDs)” (p. X). In this commentary on Fletcher and Vaughn’s review, we focus on the existing evidence that supports or fails to support RTI’s readiness to achieve these important goals. After carefully reading their article and reviewing the evidence, we conclude that RTI remains an unproven thesis. RTI currently lacks a trustworthy evidence base to indicate that it meets either of these two laudatory goals and remains instead a series of assumptions without validation. Consequently, implementation of RTI as described is premature. What leads us to such a conclusion?

Our concerns focus on five major areas reflecting the overall concern that RTI is an untested model: (1) the lack of critical knowledge concerning virtually every assumption influencing the RTI implementation process, including assessment of response, choice of parameters, and selection of cut-points; (2) lack of knowledge of efficacy of the fully implemented RTI model; (3) uncertainty of the characteristics of the students identified by RTI methods; (4) inconsistencies and unsupported assumptions; and (5) approaches to the identification of students who have LDs, again with unsupported assumptions about the superiority of RTI approaches over traditional ones. Here, we believe that the views and approaches to LD that Fletcher and Vaughn promote are contrary to central concepts of LD and, importantly, will be of great detriment to bright struggling readers. Space does not permit a discussion of each point, so we will focus on points 1, 4 and 5.

(1) RTI effectiveness is highly dependent on its ability to identify those who do not respond to good teaching, and yet Fletcher and Vaughn state, “there are no widely accepted criteria for identification of inadequate responders” (p. X). Speece and Walker (2007, p. 294) questioned what the R in RTI means: “At this point, the definitions of ‘response’ and ‘nonresponse’ in the literature are quite variable… If RTI is to be used as an ingredient in the definition of LD … much more work is required on the thorny issue of responsiveness,” indicating that “the promise of RTI swamps the evidence” (p. 287). Changes in how one defines the R in RTI clearly result in different children being identified (e.g., see Fuchs, Fuchs, & Compton, 2004; Reynolds, 2008).

Lack of a reliable evidence base to guide critical implementation decisions has downstream effects on policy decisions. Without an evidence base, regulations can only be vague and nonspecific. Implementation is then left to the vagaries, inconsistencies, and non-evidence-based beliefs of individual teachers, principals, and administrators. Thus, the USDE Office of Special Education and Rehabilitation (OSERS) regulations for special education promote RTI but are unable to provide sufficient guidance to allow for reasonable consistencies in its development, application, and outcomes. Wide variations in the conceptualization of RTI and specifics of practice thus are inevitable in matters that are nontrivial and directly affect which students and how many receive special education.

Although Fletcher and Vaughn see a consensus on RTI as well founded, many in the field see RTI as immature, controversial, and even polemic (see, for example, discussions of RTI on the listserv of the National Association of School Psychologists). The frenzy surrounding RTI is reminiscent of that characterizing the whole language approach a decade or so ago. Some see questioning RTI as nearly heretical, taking the position that those who oppose immediate, in-depth application of the RTI model are simply uninformed (see detailed discussion of this issue by Fuchs & Deschler, 2007).

With the publication of the report of the National Reading Panel (2000), education took a giant step into the world of modern science’s standard of relying not on opinion and mass frenzy, but rather on evidence in making educational decisions. Within such a context, the calls for rapid implementation of RTI—which currently lacks a sufficient evidence base—are unwarranted.

(4) Still another concern reflects Fletcher and Vaughn’s inconsistent, seemingly contradictory, and, at times, misleading statements. For example, in one section they tell the reader “there are no widely accepted criteria for identification of inadequate responders” (p. X), and yet, in another, they state “these models link with special education because inadequate instructional response allows for determination of adequate and inadequate responders” (p. X). It is also difficult to reconcile the statement “despite the research base supporting the assessment … components” (p. X) with their own acknowledgment of the lack of accepted criteria for identification of poor responders that would be necessary to develop a credible research base.

While there are well-known problems associated with current methods, there is also little evidence that RTI methods are any better or have a proven advantage. Similarly, Fletcher and Vaughn imply that RTI is needed because current methods lead to “flat levels of growth” for students in special education and that there is “little evidence that typical interventions close the achievement gap” (p. X), but at the same time, they fail to note that there is little evidence that the alternative, RTI, provides an improvement.

Fletcher and Vaughn review studies of the effect sizes associated with RTI models and laud their impact and success, citing in particular work by Swanson, Hoskyn, and Lee (1999) as the most comprehensive meta-analyses of the literature. However, Swanson (2008) remarked on the tenuousness of RTI research regarding its instructional effectiveness and suggested that there is less of a relationship between instructional methods and reading success. In reviewing what are considered the best studies of the effectiveness of RTI, Swanson concluded that “our results indicated that ‘best evidence’ studies are influenced by a host of environmental and individual differences variables that make a direct translation to assessing children at risk for LD based on a RTI only model difficult. In addition, although RTI relies on evidence based studies in the various tiers of instruction, especially in the area of reading, it is important to note that even under the most optimal instructional conditions (direct instruction) for teaching reading, less than 15% of the variance in outcomes is related to instruction” (Swanson, 2008, p. 34). Where there is evidence, effect sizes vary greatly and the types of studies and populations on which this work has been done are rather limited. Consider the area of writing instruction as another example. “So, even though there is an impressive amount of research testing different approaches to writing instruction, the lack of information on effective writing instruction for low-income, urban, low-achieving adolescent writers remains a serious gap in the literature” (Graham & Perin, 2007, p. 25). The very populations for whom we have the least amount of evidence upon which to base our decisions are in fact the populations most likely to come before referral sources and to be subjected to RTI methods.

(5) Perhaps most critically, Fletcher and Vaughn’s approach to the identification of students as having LDs fails to consider bright students who struggle to read, and fails to adhere to the basic concept of learning disability. Instead, in a somewhat mischievous manner, they appear to distort the intent and definition of LD. Within the Fletcher and Vaughn paradigm, bright students who are deigned to respond adequately, and who may nevertheless not be achieving up to their potential, are totally overlooked. Turning to a consensus conference (which is not evidence) and away from the conceptual model of LD, they offer three criteria for identification as having LD: (1) RTI, (2) assessment of low achievement, and (3) application of exclusionary criteria. The determination of an LD would become contextual—made under an RTI model based on a student's failure to progress at the same rate as other children in the same classroom (school or town), once it had been ascertained that appropriate instructional methods had been applied (how this would be determined is unspecified). Intellectual level would be ignored if above levels generally regarded as reflecting mental retardation, and no disturbance or dysfunction of any form of psychological processing would be considered or demonstrated. This represents a fundamental alteration in the concept of disability and cuts out the very roots basic to the concept of an LD as an unexpected difficulty in learning intrinsic to the child Dating back over a century when the most common LD, dyslexia, was first described (Morgan, 1896), the core concept has been unexpected achievement levels in relation to ability—not to teaching methods. While it is important to provide good instruction, the unexpected achievement levels come from comparison within the individual and should not be distorted by the desire to claim RTI as providing evidence of unexpected levels in relation to teaching. Here, Fletcher and Vaughn undermine the central concept of unexplained academic difficulty within the individual and switch to imply unexpected achievement in relation to instruction. Instruction has always been considered an exclusionary criterion, not an inclusion criterion. Thus, while they indicate “definitions of LD have always relied upon the elimination of known causes of low achievement, which includes inadequate instruction” (p. X), they imply that somehow this is central rather than the unexpected failure to read in relation to one’s ability.

Unfortunately, the use of this model and the removal of cognitive testing eliminate from consideration as LD those bright dyslexic children whose data do not differ in important characteristics from those who demonstrate low achievement. Although Fletcher and Vaughn gloss over this omission, it is emblematic of the distortion RTI represents of the identification of students who have LDs. For scientific (Swanson, 2008) and practical (see below) reasons, consideration of IQ is relevant—particularly if bright but struggling readers are to be identified as having LDs and not overlooked. For example, consider the impact of so-called peer comparison if a child is highly intelligent or even gifted and is in a class of “peers” who are functioning at lower cognitive levels. Such a bright student might be functioning below capability but at an absolute level comparable to the class average of less able peers. That struggling student would be entirely invisible and overlooked in such an RTI process. Often, the only way to identify such struggling readers is via comprehensive assessment that evaluates cognitive skills and psychological processes. Within RTI, such students would not be detected, much less referred for a comprehensive evaluation. Most critically, such struggling readers would not receive helpful interventions or accommodations “despite the fact that their relative deficit in a particular domain could cause severe psychological distress as well as unexpected underachievement” (Boada, Riddle, & Pennington, 2008, p. 185) and could be remedied by interventions. It is important to recognize that the data show these bright students, although reading at higher levels, still are performing below ability and share many qualities (such as phonological deficits) with lower functioning, struggling readers (Steubing et al., 2002; Hoskyn & Swanson, 2000). It is no more fair to deny these bright struggling readers intervention than it is to deny their lower functioning classmates. We share the concern others have expressed (e.g., Boada et al., pp. 184–185): “the methods of identification would seem to result in the term SLD [specific learning disability] specifying simply a group of low-achieving children, who were not responding well to good instruction.” These investigators go on to state, “children with grade level performance on some discrete measures may in fact be very discrepant on other more integrative tasks relative to their peers and to skills in other domains. These children would also benefit from intervention” (p. 190).

Fletcher and Vaughn note that “those children who do not respond adequately may be referred for a comprehensive evaluation for eligibility for special education services” (p. X). Nowhere do they indicate what such a “comprehensive” evaluation comprises. In fact, the vagaries of the OSERS rules for implementation of IDEA 2004 leave open the determination of what constitutes comprehensive assessment. Many states are now issuing regulations that define the RTI process as a comprehensive evaluation, relieving the state from providing any additional diagnostic work prior to declaring a child to be a student with a disability. This interpretation runs counter to what we see as the intent of the Federal regulations, and indeed Posny (2007), while Director of OSERS, clearly stated that a comprehensive evaluation was required before a student can be declared to have a disability.

RTI as a diagnostic model not only lacks in diagnostic coverage and validity, it also provides little guidance on what to do for instruction after a child fails to respond. More of the same ineffective instruction is not likely to work. One of the major purposes of a comprehensive assessment is to derive hypotheses emerging from a student’s cognitive profile that would allow the derivation of different and more effective instruction. By eliminating an evaluation of cognitive abilities and psychological processes, we revert to a one-size-fits-all mentality where it is naïvely assumed that all children fail for the same reason. In the area of reading, a model suggesting that phonological deficits fully account for reading problems in virtually all children is now being amended (Snowling, 2008). Today, we are witnessing many children whose phonological skills have been remediated, and remediated well, and who continue to struggle to read fluently and with comprehension (Shaywitz, Morris, & Shaywitz, 2008). At the current stage of scientific knowledge, it is only through a comprehensive evaluation of a student’s cognitive and psychological abilities and processes that we can gain insights into the underlying proximal and varied root causes of reading difficulties and then provide specific interventions that are targeted to each student’s individual needs, a process long advocated (e.g., see Reynolds, 1988; Shaywitz, 2005). We should also note that it is not only specification of which components require intervention, but also critical elements of the process of effective implementation—for example, intensity (group size) and duration (minutes per day and length of intervention over time)—that are currently guesswork and not evidence based for RTI procedures.

In summary, there are many reasons that RTI fails as a reliable approach to the accurate diagnosis and effective intervention with LDs; these include (1) how is RTI best implemented, as intensity and duration of intervention are currently unknown; (2) what constitutes the R in RTI? (3) RTI fails to identify bright, albeit struggling readers who require and would benefit from intervention and accommodation; (4) RTI delineates neither which components of, for example, reading (phonological awareness, fluency, orthographic processing, or others), require intervention, nor which specific strengths can assist in bootstrapping weaker areas such as vocabulary, conceptual/reasoning ability, comprehension, or others; and (5) RTI fundamentally alters and erodes the concept of LD as a disability residing within the child.

Given the extraordinary uncertainty and lack of trustworthy empiric data about the role of RTI in the identification and remediation of LDs, the field is left with the question “is RTI the answer to the search for the most effective strategy for the early identification and accurate diagnosis of a reading disability and for providing effective reading instruction and timely intervention services? Or is RTI more of a Trojan horse, outwardly appealing but filled with risky, unproven, and in the end, potentially harmful practices, or is it somewhere in between?” (Shaywitz, 2008, p. xiii). The current rush to RTI sans an evidence base may mean that a “wait to fail model” (the catchy characterization of the severe discrepancy model) is now being replaced by a “watch them fail” model known as RTI. We believe the best interests of potential LD students will be served when identification and intervention are guided by the evidence and not anecdote, and that all children, including bright students, receive equal opportunity for identification, remediation, and accommodation.