Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2015 May 1.
Published in final edited form as: Learn Disabil Q. 2013 Oct 28;37(2):100–110. doi: 10.1177/0731948713507263

Computerized Silent Reading Rate and Strategy Instruction for Fourth Graders at Risk in Silent Reading Rate

Jasmin Niedo 1,#, Yen Ling Lee 1,#, Zvia Breznitz 2, Virginia Berninger 1
PMCID: PMC4047714  NIHMSID: NIHMS526584  PMID: 24914248

Abstract

Fourth graders whose silent word reading and/or sentence reading rate was, on average, two-thirds standard deviation below their oral reading of real and pseudowords and reading comprehension accuracy were randomly assigned to treatment (n=7) or wait-listed (n=7) control groups. Following nine sessions combining computerized rapid accelerated-reading program (RAP), which individually tailors rate of written text presentation to comprehension criterion (80%), and self-regulated strategies for attending and engaging, the treated group significantly outperformed the wait-listed group before treatment on (a) a grade-normed, silent sentence reading rate task requiring lexical- and syntactic level processing to decide which of three sentences makes sense; and (b) RAP presentation rates yoked to comprehension accuracy level. Each group improved significantly on these same outcomes from before to after instruction. Attention ratings and working memory for written words predicted post-treatment accuracy, which correlated significantly with the silent sentence reading rate score. Implications are discussed for (a) preventing silent reading disabilities during the transition to increasing emphasis on silent reading, (b) evidence-based approaches for making accommodation of extra time on timed tests requiring silent reading, and (c) combining computerized instruction with strategies for self-regulation during silent reading.

The current study addressed three timely issues for teaching at risk readers: transition to silent reading in the middle grades, treatment outcomes for intervention for silent reading comprehension rate yoked to criterion comprehension levels, and combining computerized instruction with self-regulation strategies for attending to and engaging with computerized instruction. The educational relevance of these issues is explained to provide the background for the research design and tested hypotheses.

First, at a time in the history of education when transitions are increasingly emphasized from preschool to early schooling (e.g., Pianta, 2007) and from high school to postsecondary schooling (e.g., U.S. Department of Education, Office of Vocational and Adult Education, 2010), the transition from early childhood to middle childhood in the middle elementary grades also deserves attention. On the one hand, this fourth grade transition has been investigated from the perspective of a shift from learning to read in the first three grades to reading to learn in the subsequent grades (e.g., Chall, 1983). On the other hand, this fourth grade transition is also associated with a shift from a major focus on oral reading to an increasing reliance on silent reading for instructional activities and assignments, at least in the United States. Although research has generated considerable evidence about effective oral reading instruction, especially during the early grades (e.g., National Reading Panel, 2000; Schreiber, 1980; U. S. Department of Education, National Center for Education Statistics, 1995), relatively less research has focused on effective silent reading instruction, especially during the fourth grade transition.

Second, at a time when much research has focused on preventing oral reading disabilities through early intervention, relatively less research has focused on preventing silent reading rate problems that, if not treated, are also likely to result in persisting reading disabilities. Early intervention research clearly provides evidence for the effectiveness of oral reading fluency instruction (e.g., Nathan & Stanovich, 1991) for reading achievement in general (L. S. Fuchs, D. Fuchs, Hosp, & Jenkins, 2001) and reading comprehension in particular (Petscher & Kim, 2011; Schwanenflugel et al., 2006; Wanzek, Roberts, Linan-Thompson, Woodruff, & Murray, 2010). However, some students may develop adequate oral reading of single real words, but be at risk for developing adequate silent word reading or reading comprehension rate skills during the fourth grade transition. For example, Hale, Skinner, Williams, Hawkins, Neddenriep, and Dizer (2007) found that oral reading accounted for only 27.66% of the variance in silent reading of elementary students and only 14.38% of the variance in silent reading in high school students. Denton et al. (2011) found a weaker relationship between reading comprehension and oral reading fluency for older students in grades 6 to 8 than younger students. Oral reading fluency and silent reading fluency may be highly related, yet distinct constructs (e.g., Kim, Wagner, & Foster, 2011), allowing for possible individual differences in the levels to which each is developed. Indeed, some older readers in grades 4 and above may benefit from continued training in silent reading fluency (Hiebert et al., 2012; Rasinski, Samuels, Hiebert, Petscher, & Feller, 2011) and comprehension (Buly & Valencia, 2002; Dewitz, & Dewitz, 2003; Nation, 2004). Thus, the current study focused on response to silent reading instruction at the fourth grade transition to prevent future silent reading disability.

Third, computerized instruction is ideally suited for training silent reading comprehension rate by manipulating rate of presentation for specific kinds of comprehension tasks yoked to criterion levels of comprehension. For example, the Rapid Accelerated Program (RAP) program (e.g., Breznitz, 1987; 1997a, 1997b; 1997c; Breznitz & Share, 1992) individually tailors rate of written text presentation to the individual's comprehension level and then accelerates that rate when criterion levels of comprehension are achieved and maintained. Reading rate, which is controlled by regulating the rate at which letters disappear from the written language on the monitor, can be accelerated by increasing the rate at which the letters disappear. Breznitz's programmatic line of research showed that accelerating participants’ reading to their fastest individual pace consistently increased word reading accuracy (as measured by decreased errors) and comprehension in typical readers, poor readers, and those with dyslexia. Fast-paced reading may facilitate synchronized processing of information in working memory (Breznitz, 2006).

However, because prior research showed that poor readers have weaknesses in paying attention to written words (Thomson et al., 2005), the current research combined computerized silent reading comprehension rate instruction with self-regulated strategy instruction for paying attention to and engaging with the computerized instruction. Just as early intervention has shown that some students are non-responders (Torgesen, 2000), some students may be non-responders to computerized silent reading instruction: But are they non-responders because they are not paying attention to or engaging with the instruction or because the instruction is not effective? Indeed the research team had observed in prior research that some students had difficulty paying attention to or engaging with computerized instruction, even though it is often assumed that computer programs ensure fidelity of treatment implementation by keeping instruction constant across students. To maximize the likelihood that inattention and/or lack of engagement were not interfering with response to the computerized instruction, the research team developed and taught a set of strategies for self-regulation of attention and engagement during the computerized silent reading comprehension instruction (see Appendix). These strategies drew on approaches for teaching strategies for self-regulating reading comprehension (e.g., Souvignier & Mokhlesgerami (2006) or writing (e.g., Wong, 2001; Wong, Butler, Ficzere, & Kuperis, 1996). Although research on the effectiveness of single instructional components can be instructive (e.g., Heistad, 2008), it can also be useful to know the effectiveness of instruction that combines components.

Operationalizing Constructs

Silent reading rates at multiple levels of language

For the research aims of the current study, measures were selected to assess different levels of language in silent reading comprehension with focus on word (lexical), sentence (syntax), and text (discourse) levels (see Berninger & Niedo, in press, for a levels-of-language framework for assessing and teaching reading). Recognizing that fluency is a multi-dimensional construct, the researchers focused on one dimension of fluency—rate—and operationalized it with normed measures at pretest and posttest that are scored on basis of number of accurate responses within a time limit or yoked to total time to assess (a) rate of word identification during silent reading when sentence context clues are not available, (b) rate of word identification during silent reading when sentence context clues are available, and (c) rate of rendering sentence sense judgments on tasks that required careful attention to both the lexical units and syntax structures. In addition, they used scores for computerized tasks at the word, sentence, and text levels. See Methods section for nationally normed measures used for each of these constructs.

The focus on silent reading rates for word identification and sentence comprehension is grounded in prior research. Biemiller (1977-78) found differences in reading times between those with good and poor reading skills for letters, words out of context, and text (words in context). With age, rate of reading words increased to approximately rate of reading letters for those with good reading skills. The gap between letter time and word time for readers with poor reading skills, on the other hand, persisted. In terms of comprehension, findings did not support a difference between the readers who were more or less able in their ability to effectively use context. Drawing on Biemiller's work, Lovett (1987) reported markedly different profiles for children with accuracy disability (concurrent with rate disability) and those with only rate disability (but grade-appropriate word accuracy skills). Yet, during silent reading, children with both accuracy and rate reading disabilities in Lovett's study showed decreased comprehension in comparison to controls without reading disability. Berninger (1994) found individual differences in levels of language in sentence comprehension (lexical- and syntax- levels) on a computerized silent reading comprehension task employing a paradigm developed by Potter (1984) for simultaneous and sequential presentation of words in a sentence comprehension task.

Verbal working memory and inattention influences on silent reading rate

Perfetti's (1977) verbal efficiency model proposed that comprehension is compromised when limited working memory capacity is reallocated to nonautomatic word level reading, and as a consequence, fewer resources are available for syntax- and text- level comprehension processes. This model is likely to hold for silent reading rate as well as oral reading rate (for review, see Chard, Vaughn & Tyler, 2002; Katzir et al., 2006; Rasinski, Samuels, Hiebert, Petscher, & Feller, 2011). According to Breznitz (2003), lack of temporal coordination, supported by the timing mechanisms of working memory, may lead to asynchrony among the different processes involved in reading, resulting in slow reading (Breznitz, 2003; Horowitz-Kraus & Breznitz, 2011). It follows that effects of silent reading instruction may be affected by working memory, and thus working memory was assessed too. In addition, inattention ratings validated by Thomson et al. (2005) for written language tasks were collected because inattention might also may affect response to the computerized instruction for developing silent reading rate skills at both the lexical and syntactic levels.

Research Aims of the Current Study

The current study used Breznitz's computerized RAP software, but used it in a new way. Children used the software to develop silent reading rate in English at different levels of language—word cloze, sentence logic, and paragraph understanding (see procedures in methods section). Children who at the end of fourth grade had a profile in which silent word reading rate and/or silent sentence reading rate was underdeveloped compared to their oral reading accuracy for single real words and pseudowords on a list and reading comprehension accuracy participated in a wait-listed control research study. The goal was not to evaluate the effectiveness of the combined RAP and strategy training for listing on the What Works website or to use in schools, but rather to provide initial evidence about whether combining computerized instruction with strategy instruction for optimizing response to the computerized instruction might improve rate for silent word reading and silent sentence reading. Such initial evidence could be used to design future studies of silent reading during the fourth grade transition in reading development to inform instructional practices and tools. The schedule of observations to achieve this goal included (a) standardized, normed measures of reading and working memory given before and after treatment to each group, but timed so that the posttest for the first group was given at the time of the pretest for the second group, (b) parental ratings of children's attention prior to treatment, and (c) RAP scores for rate of presentation on each of the three tasks at a level meeting comprehension criteria (80% correct), which were collected in each of nine sequential instructional sessions.

Three hypotheses were tested. The first was that the initial treatment group would at the end of their treatment significantly outperform the wait-listed control group just before they began treatment on both the standardized silent reading measures and RAP silent reading rate at criterion level of comprehension (80%). The second was that both groups would improve significantly on the same learning outcomes from before to after participating in the nine instructional sessions. The third was that attention ratings and working memory for written words before treatment would predict outcomes on silent reading rates after treatment that combined multi-leveled computerized silent reading rate training and strategy instruction for paying attention to and engaging in the computerized training.

Method

Participants

This research was approved by the Institutional Review Board (IRB) at the university where it was conducted. Flyers were distributed in spring to fourth grade teachers in local schools near the university that announced the opportunity to participate in the research during the summer at the university. Teachers were encouraged to share the flyers with parents of students that they thought would benefit from participation in research designed to improve silent reading rate. Specifically, they were asked to refer students whom they were concerned could not keep up with reading assignments in fifth grade because their silent reading rate, not necessarily their oral reading, was underdeveloped based on their teaching experience for fourth graders. The flyer provided necessary university contact information for interested parents.

Referred children whose parents granted informed consent and who granted assent were given a battery of normed tests and parents completed attention ratings used in Thomson et al. (2005). Children whose reading comprehension and oral word or pseudoword reading accuracy fell in or above the average range (standard score 90 to 109), but silent reading rate fell in the low average range (below 90) on at least two of the three measures qualified for participation. Because two of these involved rate of silent word reading (with or without context) and one involved rate of silent sentence reading, all participants had relative weaknesses on at least one silent word reading measure. All the referred and tested children, who ranged in age from 9-5 to 11-1, met these research inclusion criteria.

Parents of the participating children tended to be highly educated: Of mothers, one did not graduate from high school but earned a GED, four graduated from college, and nine were college graduates with post-undergraduate education. Of fathers, three were high school graduates with some postsecondary education, two were college graduates, and nine were college graduates with postsecondary education. The child participants, none of whom were on free and reduced lunch, were diverse in ethnic background: European American only (n=7), and mixed (n=7), African-American, Hispanic, and European-American (n=2), European-American and Hispanic (n=1), Asian-American and African-American (n=1), European-American and African-American (n=2), or Hispanic and Arabic (n=1). However, English was the first language for all participants. All participating children were right handed.

Pretest and Posttest Assessment Battery for Reading and Reading-Related Skills

All measures were individually administered at the university. In addition, parents completed ratings related to self-regulation of attention and behavior. The pretest measures were used to evaluate any differences between the initial treatment group and the wait-listed group prior to each group receiving the research-designed instruction. Some of the normed measures were repeated at posttest to evaluate change over time on normed measures following instruction combining computerized and strategies training.

Oral real word reading accuracy

The Wechsler Individual Achievement Test, 2nd Edition (WIAT II) Real Word Reading subtest was administered according to standardized directions in the test manual (Psychological Corporation, 2005). The stability coefficient is .97 for ages 10-12.

Oral pseudoword reading accuracy

The WIAT II Pseudoword Reading subtest (Psychological Corporation, 2005) was used to assess a child's ability to read a list of nonsense words that do not have a semantic meaning but adhere to the morphophonemic-orthographic structure of the English language. The stability coefficient is .97 for ages 10-12.

Text reading comprehension accuracy

The WIAT II Passage Comprehension subtest (Psychological Corporation, 2005) was given to assess reading comprehension. The child was asked to read passages before orally answering literal or inferential questions. The stability coefficient is .95 for ages 10-12.

Silent word reading rate

The Test of Silent Word Reading Fluency (TOSWRF) (Mather, Hammill, Allen & Roberts, 2004) was given to measure the speed and accuracy of silent word recognition within a three minute time limit. The child was given a series of words without any spaces between them (e.g. dimhowfigblue) and asked to use slash lines to mark word boundaries (e.g. dim/how/fig/blue). The stability coefficient is .89 for the elementary level (ages 7-10). This test is a silent reading counterpart of the oral reading of real words.

Silent word reading rate for single words embedded in meaningful context

The Test of Silent Contextual Reading Fluency (TOSCRF) (Hammill, Wiederholt & Allen, 2006) was administered to determine the speed at which words are recognized in context within a 3 minute time limit. The child was asked to use slash lines to mark word boundaries in a series of words without spaces between them that could be grouped into sentence units but did not have punctuation marks. Thus, words could be identified not only on the basis of their morphophonemic-orthographic knowledge of single words but also on basis of semantic and syntactic clues from the words in sentence context. The stability coefficient is .88 for children in the elementary level (ages 7-11).

Silent sentence sense rate score

The Sentence Sense subtest of the Process Assessment of the Learner, 2nd Edition (PAL 2) (Berninger, 2007) was given to measure ability to coordinate word recognition with syntactic processing during silent sentence comprehension under time limits. Presented with a set of three sentences, the child was asked to identify the one that was a meaningful sentence. Each of the sentences had only real words and each sentence differed from each of the others by only one word. So children had to pay close attention to both single words and the syntactic context in which they occurred. The fluency score is based on a norming metric that takes into account both accuracy and time; that is, for a given level of accuracy, the score reflects the time taken based on the variability in time observed in the standardization sample. Reliability for grade 4 is .66 for the rate metric in the national standardization sample that, like the RAP program, yoked accuracy to time. Please note that this measure was selected instead of others available for silent reading fluency, which assess timed judgments of whether sentences reflect factual knowledge of the world, because of the specific aims of the funded research related to training silent reading comprehension within the context of a levels-of-language model.

Working memory measures

PAL 2 Working Memory--Words (Berninger, 2007) was given. Each item is scored for accuracy of questions posed by the examiner. For Working Memory—Words, the examiner named a word and asked the child to spell it forwards with the examiner, then with eyes closed spell the word backwards without assistance, and finally while thinking about the word in forward direction name a letter or letters in the word position or positions indicated by examiner. This measure (stability coefficient for grade 4 is .92) assesses ability to create a precise word-specific spelling in working memory and then analyze letters in the written word stored in working memory.

Attention/hyperactivity ratings

Parents were asked to complete an 18- item questionnaire regarding observations of their child's attention and level-of-activity problems in school and other daily activities before treatment. For example, parents were asked to mark always, pretty often, sometimes, never, problem, or not a problem for items such as their child's failure to pay close attention to detail. Prior research had validated a four-factor structure underlying the 18 items and showed that the factor score for inattention explained unique variance in reading rate (Thomson et at., 2005); thus only the inattention factor score was used to predict silent reading rate for word identification and sentence comprehension.

Procedures

Random assignment to groups in wait-listed control design

Children who met research inclusion criteria for being at risk for silent reading rate problems at the end of fourth grade (oral reading accuracy and reading comprehension in average range or better, with at least two measures of silent reading rate in the low average range or lower) were randomly assigned to one of two groups (initial treatment at beginning of summer or wait-listed with midsummer treatment) until the design was completed. Based on the number of computer work stations available on which RAP software was installed, seven were assigned to the initial treatment group and seven to the wait-listed control group. The initial treatment group, which received the RAP intervention first, included three girls and four boys who met the research inclusion criteria. The wait listed control group, who received the treatment after the first group completed it, also included three girls and four boys who met the research inclusion criteria for silent reading rate weaknesses. The assessments confirmed the judgments of the teachers who referred the children for silent reading rate problems.

Computerized instruction and assessment procedures

Each participant received nine treatment sessions, each lasting about an hour, in which three computerized tasks in English, which are described in the next section, were completed. A three member research team (Ph.D. student, postdoc, faculty investigator), all with considerable teaching experience, supervised these sessions with children each working independently at their work stations; the research team provided children with ongoing feedback about their reading rate at completion of each unit within a session; this feedback is produced by the RAP software. Children were praised when their reading rates (based on speed and accuracy levels) improved.

Each training session began with a 13-item sentence comprehension pretest to determine a child's individual reading time for computerized items at a level where the child met the comprehension criterion of 80% or better accuracy. The child read a single sentence and answered a multiple choice comprehension question, for example, What did John get? This question was answered in reference to a sentence such as Last month John got a new dog from the pet shop. This initial reading rate was calculated from the onset of the sentence up to the time the child clicked on the choice to answer the comprehension question.

Following this initial baseline, six cloze items, eight logical-judgment sentences, and four paragraph-understanding tasks were given. Children completed 22 subsets of these three tasks before a 13-item sentence comprehension RTI assessment was given to evaluate possible improvement at the end of each session. Also, the computer program manipulated reading rate after response to every set of ten questions, based on the child's performance on the sentence logic and paragraph tasks.

If a child's accuracy (number of correct items) fell below 80%, then the pace (rate at which words disappeared) remained the same in the next trial. If the child's accuracy reached 80% for a minimum of 8 correct responses to the last 10 questions following sentences or paragraphs, then the reading pace increased in the next trial. The RAP rate for presentation is the number of letters per second at which the child can reach the criterion level of comprehension accuracy. A higher score is a better score.

Sentences within each subtest varied within one trial; no stimuli were repeated within a trial set. However, across trials, the same items were repeated in the sets of cloze, logical judgment, and paragraph understanding tasks, which are described next.

Computerized reading tasks across levels of language

RAP required children to complete three subtasks: a) Word Cloze, b) Sentence Logic, and c) Paragraph Understanding. During the Word Cloze task, children were presented a sentence with a blank, for example, The race car can go very ____. This sentence was followed by a list of multiple choice words from which to choose one to fill in the blank to best complete the sentence. This cloze task assessed the integration of predicting next word and identifying a word that fits the sentence context. The Sentence Logic task required children to decide whether or not a sentence was meaningful, for example, Animals ask many questions. They pressed logical or illogical, which flashed on the screen, to communicate their judgment. During the Paragraph Understanding task, children read a short paragraph and answered questions related to the content of the passage. For example, for this paragraph that follows the question was When should whole grains be eaten? Paragraph: Whole grains are an important part of a balanced diet. They should be eaten on a daily basis. Many of your favorite cereals contain whole grains.

The computerized program individually tailors rate of presentation of written materials according to a child's comprehension accuracy and reading rate on the prior learning trial. To adjust rate at which written stimuli are presented, RAP controls the rate at which letters disappear in written words in a left-to- right direction. Thus the reader has to hold the written words in working memory as words disappear as a word, sentence, or text is read.

Strategies for self-regulating attendion and engagement

In other research employing computer tools for instruction the research team had observed that students may not attend to or stay engaged with the computer program. Thus, even though computerized programs may seem the ideal way to ensure fidelity of treatment implementation, if a student does not respond to instruction, it is important to consider whether it is because of failure to pay attention or engage or because the instruction is not effective. Thus, during the first two sessions of the initial treatment group, the team observed student response to the RAP program and developed the strategies in the Appendix to optimize children's attention to and engagement with the computerized instruction. These strategies were reviewed with the children at the beginning of sessions three to nine for both the initial treatment group and the wait-listed control group.

Data Analyses

First, the initially treated group was compared to the wait-listed control group on standardized, normed measures of silent word reading rate and silent sentence comprehension rate completed at pretest before each group began treatment. Second, the RAP scores across the last three learning trials of the initially treated group were compared to the RAP scores across the first three learning trials when the wait-listed group began treatment. Third, each group was analyzed separately in repeated measures ANOVAs, in which each participant served as his or her own control, for statistical significance of any pretest to posttest change during RAP treatment on standardized normed tests. Fourth, separate repeated measures ANOVAs for each group were conducted for performance across three time points within RAP learning trials within each group. Fifth, correlations were computed between pretest measures of attention and working memory and posttest performance on measures on which both groups showed RAP-treatment effects. The goal of the fifth analyses performed on the total sample of combined groups was to evaluate whether pretreatment individual differences in attention or working memory may affect learning outcomes for combined RAP and self-regulation.

Results

Participants’ Relative Weakness in Silent Reading Rate

Before treatment the children in both the treatment and wait-listed control groups fell on average in the average range (at or near the population mean) in accuracy of oral reading of real words and pseudowords and reading comprehension. However, their silent reading rate skills, on average, fell below these scores (in the low average range or below). So, for this sample, reading rate weaknesses existed despite average or better oral reading accuracy and comprehension accuracy. The treatment and wait-listed control groups did not differ significantly before treatment on any of the measures in Table 1. See Table 1 for the means and standard deviations for each of the measures at pretest and for the reading measures given again at posttest.

Table 1.

Pretreatment Means and Standard Deviations for Treatment Group (N=7) and Wait-Listed Control Group (N=7) and Posttest Means and Standard Deviations for Reading Measures Repeated at Posttest

Treatment Group Wait List Control Group
M (SD) M (SD)
WIAT2 word reading accuracy
        Pretest 98.86(12.84) 102.20 (7.05)
    Posttest n.a.
WIAT2 nonword reading accuracy
        Pretest 97.43 (13.06) 101.20 (7.52)
    Posttest n.a.
WIAT2 reading comprehension
        Pretest 98.86 (12.21) 102.40 (6.19)
    Posttest n.a.
TOSWRF Silent Word Reading Rate
        Pretest 88.57 (9.32) 91.80 (9.04)
        Posttest 91.57 (14.81) 90.86 (13.66)
TOSCRF Silent Contextual Reading Rate
        Pretest 84.43 (10.45) 90.40 (10.64)
        Posttest 92.71 (71 (13.15) 90.86 (13.66)
PAL 2 Sentence Sense fluency
        Pretest 8.57 (2.57) 8.00 (2.24)
        Posttest 10.43 (1.98) 8.71 (1.50)
Open in a new tab

Notes. See methods for tests. Treatment group and wait-listed control groups did not differ significantly on any tests before treatment (pretests). See text for the measures on which each group made significant gains from pretest to posttest. Note that the WIAT II, TOWCRF, and TOSCRF scores are on a scale with M=100, SD=15, but PAL-2 is on a scale of M=10, SD=3.

Effects of Combined RAP and Strategy Instruction

Standardized, normed test outcomes

The treatment and wait-listed control group were compared when the initial treatment group completed all RAP training sessions before the wait-listed control group began the RAP training. Differences in degrees of freedom are related to missing data for one student in the wait-listed group at posttest due to an unexpected family issue. The only treatment effect observed on normed measures in this quasi-experimental comparison was on a measure of silent sentence reading rate that required both word-level and sentence-level processing to judge whether a sentence was meaningful: F (1, 12) = 5.97, p=.03. The Treatment Group (M=10.43, SD= 1.98) outperformed the Wait-Listed Control (M=8.00, SD=2.24). See Table 1.

For additional evidence that the RAP treatment improved silent reading comprehension rate, repeated ANOVAs were conducted for each standardized test given at pretest and posttest for each group separately. For the initially treated group, when each participant served as his or her own control, improvements were observed over time on Sentence Sense rate, F (1, 6) =8.59, p=.026, from pretest (M=8.57, SD=2.57) to posttest (M=10.43, SD=1.98) and on Silent Contextual Reading Rate, F(1, 6)=7.26, p=.036 from pretest (M=84.43, SD=10.45) to posttest (M=92.71, SD =13.15). That is, treatment effects were observed not only for silent word reading word rate in the context of sentences, but also for silent sentence reading rate when the task involved word-level identification of all words in a sentence and meaning of a sentence based on each word and the syntax in which they are embedded. When the wait-listed control did receive treatment, significant improvement was observed on Sentence Sense rate, F (1, 5)=5.99, p=.058 (significant directional hypothesis), from pretest (M= 8.00 SD= 2.24) to posttest (M= 8.71, SD=1.50), but not on the rate measures for word reading with and without sentence context.

RAP learning trials as outcome

The last RAP learning trial for the initially treated group was compared to the first RAP learning trial for the wait-listed group. The hypothesis that the combined instruction would improve RAP rates for criterion comprehension levels was supported in this quasi-experimental comparison of two groups of fourth graders at risk for silent reading comprehension rate. When the initial treatment group had completed treatment and the wait-listed control group had not yet received treatment, the first group performed higher than the second group on the RAP score based on rate and accuracy of reading comprehension, F (1, 13)=25.63, p <.001. The treated group's mean RAP score at end of intervention was 19.43 (SD=8.40), whereas the wait-listed control group's mean RAP score before beginning intervention was 3. 29 (SD=1.25). That is, the group who had received RAP training could meet comprehension accuracy criterion if letters disappeared at a rate of 19.43 letters per second, whereas the untreated group could maintain comprehension accuracy criterion only if the mean rate at which the letters disappeared was on average 3.29 letters per second. However, no differential treatment effects were observed related to rate or accuracy on the specific kinds of items (cloze, logic, or paragraph tasks), which varied in level of language, used during RAP training.

Repeated measures showed that over the course of the RAP training (first block of three lessons, second block of three lessons, and last block of three lessons), the initial Treatment Group showed a significant time effect, F(1, 6)=66.50, p <.001. So did the wait-listed control group when they received their RAP training, F(1,6)=297.52, p <.001.

Individual Differences Affecting Outcomes for Combined RAP and Strategy Instruction

For analyses based on the combined sample (N=14), the parental ratings of their child's inattention and test scores for working memory for storing and processing written words were significantly correlated with Sentence Sense accuracy at posttest after completion of the instructional sessions: r=.653 for inattention ratings, p=.011; and r= .635 for working memory-written words, p=.026. However, Sentence Sense rate (silent sentence reading rate), for which RAP training effects were already reported, was significantly correlated with Sentence Sense accuracy at posttest, r=.735, p=.003. The working memory scores did not, however, significantly change from pretest to posttest. Thus, working memory may affect treatment outcomes rather than being changed by the combined RAP and strategy instruction used in the current study.

Discussion

Validity of Research Findings

Shadish, Cook, and Campbell (2002) discussed four kinds of validity in research designs: internal, construct, statistical, and external. Two findings support the internal validity, that is, probable cause-effect relationships for the combined computerized reading comprehension and self-regulation strategy treatment. First, although the treatment and wait-listed controls did not differ in their pretest measures of silent reading rate for words with or without context or silent sentence reading rate prior to treatment, after the initial treatment group received treatment, the treatment group did differ significantly from the wait-listed control before treatment on a normed measure of silent sentence reading rate. Second, each of the groups did show statistically significant improvement from pretest before treatment to posttest after treatment on the silent sentence reading rate measure, and the first group improved on word reading rate within sentence context. This replication of the effect for silent sentence rate across two separate groups on the same normed measure contributes to the internal validity of the outcome measure.

The treatment effect was also detected on the RAP scores for which individually tailored presentation rates were yoked to comprehension criteria for each of the three tasks requiring different levels of language processing—lexical, syntactic, and text. That RAP treatment effects were observed on both the normed measures before and after treatment and the RAP scores during treatment provides converging evidence across different learning outcomes. This converging evidence across dependent measures contributes to the construct validity of measures used for assessing learning outcomes.

Although sample size is relatively small for randomized controlled designs, statistically significant effects were found, which were replicated across two independent samples.

Statisticians such as Mosteller and Tukey (1977) have argued that statistically significant findings in small samples are more robust than those requiring very large samples to have the power to detect small statistically significant differences. Thus, statistical validity was adequate for an initial study exploring the potential application of RAP accelerated reading training to the fourth grade transition to silent reading.

External validity is limited to generalizing findings only to English-speaking children with documented relative weaknesses in silent reading rate despite adequate oral reading accuracy and reading comprehension at the fourth grade to fifth transition (end of fourth grade) who do not qualify for free and reduced lunch and whose parents are well educated. The results can be generalized to girls and boys (both groups had roughly the same number of girls and boys) and to mixed racial/cultural backgrounds (only half were only European-American).

Contribution of New Knowledge with Educational Applications

Results provided support for the first and second hypotheses for silent sentence rate of sentences on a normed measure and for words, sentences, and text on RAP scores during computerized silent reading rate training. That is, this study yielded evidence that identifying children with silent reading rate problems and providing them with combined computerized instruction for reading rate and strategy instruction for attending and engaging may help overcome these silent reading rate problems. However, such evidence was found only on the reading rate measures that took into account comprehension across levels of language—words, sentences, and text (RAP) or words and sentence syntax (Sentence Sense)—not just rate of silent word identification with or without context. The third hypothesis that attention and working memory may affect learning outcomes for silent reading comprehension rate was confirmed, but the hypothesis that the combined RAP and strategy treatment would improve working memory outcomes following treatment was not confirmed.

The current study adds new knowledge regarding the three issues highlighted in the introduction: (a) intervention components that can facilitate improvements in silent reading comprehension rate during the transition in the middle grades to increasing silent reading in school curriculum; (b) proof of concept that intervention at this transition time may contribute to prevention of future silent reading comprehension rate because participants showed improved silent reading after just nine sessions; and (c) the RAP computerized tool that controls rate of written text presentation yoked to criterion level of silent reading comprehension, when combined with strategy instruction for paying attention to and engaging in the computerized instruction, can lead to significant improvement in silent reading rate. Moreover, individual differences in working memory for storing and processing written words may affect learning outcomes for silent reading rate instruction.

Of interest, treatment effects observed on normed measures were observed on a measure of silent sentence reading rate which required integration of levels of language processing—lexical and syntactic—as does reading comprehension. That is not to say that some students do not have weaknesses or disabilities in silent word reading rate—they do. However, the combined computerized and strategy instruction used in the current study may not be the most effective approach to remediating that word-level silent reading rate problems alone. Rather the combined computerized and strategy instruction in the current study led to significant improvement in a silent sentence rate on a normed measure that required syntax (sentence)- as well as lexical (word) comprehension and RAP scores yoked to training comprehension across levels of language. Other interventions and assessment tools might be used in future research on teaching and assessing learning outcomes for silent reading skills.

Currently recommendations for accommodations in the form of more time on tests are made for students on the basis of oral reading rate measures even though the timed tests for which accommodations are generally sought tend to require silent reading. Measures of silent word reading rate and silent reading comprehension rate are needed to make decisions about the need for accommodations in the form of extra time on tests that require silent reading. More research is needed on evidence-based assessment for determining need for additional time on silent reading tests

The goal of the current research was not to develop definitions for qualifying students with silent reading weaknesses for pull-out special education services. Rather, the goal was to bring attention to the unmet needs in identifying silent reading problems during the transition to increasing reliance on silent reading and implementing evidence-based instruction to prevent silent word reading rate and/or silent sentence reading rate problems from developing into persisting specific learning disabilities that may affect silent reading comprehension. Early identification and intervention for oral reading problems—both accuracy and fluency (e.g., Read Naturally, 1997-2008), while necessary, may not be sufficient. More attention should also be paid to screening for silent reading rate problems across levels of language during the fourth grade transition to silent reading to identify those who would probably benefit from instructional intervention to improve silent reading comprehension accuracy and rate. Future research should also investigate computerized silent reading comprehension rate training across the life span (e.g., Breznitz, Shaul, Horowitz-Kraus, Sela, Nevat, & Karni, 2013). Research evidence is also accumulating for the effectiveness of explicit non-computerized instruction in strategies for reading comprehension (e.g., Denton et al., 2011; Katzir, Leseaux, & Kim, 2009; McKeown, & Beck, in press).

Teaching Silent Reading Instruction with Working Memory in Mind

Both attention and working memory-words may influence learning outcomes for RAP training. Each of these individual differences measures was correlated with accuracy of silent sentence comprehension task at posttest, which in turn was correlated with a measure of silent sentence comprehension rate. The finding for attention points to the importance of teaching children to self-regulate their attention during written language learning with or without use of computer tools, which alone may not be sufficient (see Buly & Valencia, 2002; Dewitz, & Dewitz, 2003). The strategies in the Appendix, or ones developed by other researchers and practitioners, might be used along with RAP or other computerized instructional tools in future research to investigate how strategies for self-regulation of attention may facilitate RTI for silent reading.

There was no evidence that RAP had a treatment effect on Working Memory for storing and processing written words. Rather, the relationship appears to operate in the other direction: Working memory for storing and processing words influences learning outcomes for RAP training, at least on a task requiring coordination of word- and sentence-level processing. This finding for the role of word-level working memory processes in reading is consistent with prior research using computerized paradigms such as Potter's (1984) Simultaneous Sentence Presentation (SSSP) and Rapid Serial Visual Processing (RSVP) with children (Berninger, 1994). Future RAP or other computerized training for silent reading comprehension accuracy and rate might investigate the effects of controlling pace of reading by having words rather than letters disappearing at specific individually tailored rates.

Summary and Conclusions

Without intervention to improve silent reading comprehension accuracy and rate, fourth graders with relative weaknesses in silent reading may struggle in the later grades when the amount and nature of silent reading required increases. Typically children with slower reading rates may qualify for accommodations in the form of more time on tests, which require silent reading. Such accommodations should not be based only on oral reading rates, but also silent reading rates. Moreover, during middle childhood children with silent word reading and/or reading comprehension accuracy and/or rate problems should be given specialized instruction in silent reading skills not just accommodation. Such instruction might utilize instruction that combines components regulated by the computer teacher to ensure fidelity of treatment implementation with self-regulation strategies taught by the human teachers to develop the student's self-regulation skills that transfer to other learning contexts in which neither a computer nor human teacher may be available and the student has to manage his or her own silent reading comprehension.

Acknowledgments

Data collection funded by a Binational Science Foundation Grant BSF Grant No. 2009053 (Israel and United States) awarded to Zvia Breznitz and Virginia Berninger. Data analyses and manuscript preparation supported by HD P50HD071764 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) at the National Institutes of Health (NIH).

Appendix

Student Strategies for Paying Attention to and Engaging with Rapid Accelerated Reading Program (RAP)

  • Read fast and go back to reread the sentences before the words disappear.

  • Read ahead if you have time to preview what is coming.

  • When you're asked questions about a paragraph:
    • ○ Pay attention right away! Some questions ask you about what you read in the beginning or the first sentences of the paragraph.
    • ○ Think ahead about questions that the computer may ask. Think about who the paragraph is about, what happened, where, when, why, and how.
  • When you're asked to fill in the blank:
    • ○ Think about the sentence as a whole.
    • ○ Think about the possible answers.
    • ○ Think about which answer makes the best sense in completing the sentence.

  • Keep in mind: Logical sentences make sense and not logical sentences do not make sense.

  • Get your hands ready to click on the right answer.

  • Always keep your eyes on the screen. You want to be sure you don't miss any words before they disappear.

  • Pay attention as soon as the sentences appear on the screen.

  • Remember, it's great to be a fast reader (rate), but it's even more important that you understand what you're reading and can make the best choice in answering questions (accuracy).

Footnotes

Citation: Niedo, J., Lee, Y. L., Breznitz, Z., & Berninger, V. (in press). Computerized silent reading rate and strategy instruction for fourth graders at risk in silent reading rate. Learning Disability Quarterly. Submitted September 22, 2013 for posting on PubMed Central one year following on-line journal publication.

Disclosure:

V.W. Berninger is the author of the PAL Reading and Writing Diagnostic Test (Sentence Sense and Working Memory subtests).

References

  1. Berninger V. Intraindividual differences in levels of language in comprehension of written sentences. Learning and Individual Differences. 1994;6:433–457. [Google Scholar]
  2. Berninger V. Diagnostic for Reading and Writing (PAL-II RW) 2nd Edition The Psychological Corporation. Now Pearson; San Antonio, TX: 2007. Process Assessment of the Learner. [Google Scholar]
  3. Berninger V, Niedo J. Individualizing instruction for students with oral and written language difficulties. In: Mascolo J, Flanagan D, Alfonso V, editors. Essentials of planning, selecting and tailoring intervention: Addressing the needs of unique learners. Wiley; New York: (in press) [Google Scholar]
  4. Biemiller A. Relationships between oral reading rates for letters, words, and simple text in the development of reading achievement. Reading Research Quarterly. 1978;13:223–253. [Google Scholar]
  5. Breznitz Z. Increasing first graders’ reading accuracy and comprehension by accelerating their reading rates. Journal of Educational Psychology. 1987;79:236–242. [Google Scholar]
  6. Breznitz Z. Effects of accelerated reading rate on memory for text among dyslexic readers. Journal of Educational Psychology. 1997a;89:289–97. [Google Scholar]
  7. Breznitz Z. Enhancing the reading of dyslexic children by reading acceleration and auditory masking. Journal of Educational Psychology. 1997b;89:103–113. [Google Scholar]
  8. Breznitz Z. Reading rate acceleration: Developmental aspects. Journal of Genetic Psychology. 1997c;158:427–441. doi: 10.1080/00221329709596680. [DOI] [PubMed] [Google Scholar]
  9. Breznitz Z. Fluency in reading: Synchronization of processes. L. Erlbaum Associates; Mahwah, N.J: 2006. DOI 10.1007/978-94-007-2327-6_11. [Google Scholar]
  10. Breznitz Z, Share DL. Effects of accelerated reading rate on memory for text. Journal of Educational Psychology. 1992;84:193–99. [Google Scholar]
  11. Breznitz Z, Shaul S, Horowitz-Kraus T, Sela I, Nevat M, Karni A. Nature Communications, on-line. Nature Communications. 2013;4:2041–1723. doi: 10.1038/ncomms2488. Article number: 1486 doi:10.1038/ncomms2488. [DOI] [PubMed] [Google Scholar]
  12. Buly MR, Valencia SW. Below the bar: Profiles of students who fail state reading assessments. Educational Evaluation & Policy Analysis. 2002;24:219–239. [Google Scholar]
  13. Chall J. Stages of reading development. McGraw-Hill; New York: 1983. [Google Scholar]
  14. Chard DJ, Vaughn S, Tyler B-J. A synthesis of research on effective interventions for building reading fluency with elementary students with learning disabilities. Journal of Learning Disabilities. 2002;35:386–406. doi: 10.1177/00222194020350050101. [DOI] [PubMed] [Google Scholar]
  15. Denton CA, Barth AE, Fletcher JM, Wexler J, Vaughn S, Cirino PT, et al. The relations among oral and silent reading fluency and comprehension in middle school: Implications for identification and instruction of students with reading difficulties. Scientific Studies of Reading. 2011;15:109–135. doi: 10.1080/10888431003623546. doi: 10.1080/10888431003623546. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dewitz P, Dewitz PK. They can read the words, but they can't understand: Refining comprehension assessment. The Reading Teacher. 2003;56:422–435. doi: 10.1177/1534508408330082. [Google Scholar]
  17. Fuchs L, Fuchs D, Hosp M, Jenkins JR. Oral reading fluency as an indicator of reading competence: A theoretical, empirical, and historical analysis. Scientific Studies of Reading. 2001;5:239–256. [Google Scholar]
  18. Hale AD, Skinner CH, Williams J, Hawkins R, Neddenriep CE, Dizer J. Comparing comprehension following silent and aloud reading across elementary and secondary students: Implication for curriculum-based measurement. Behavior Analyst Today. 2007;8:9–23. [Google Scholar]
  19. Hammill DD, Wiederholt JL, Allen EA. TOSCRF: Test of silent contextual reading fluency: Examiner's manual. PRO-ED; Austin, TX: 2006. [Google Scholar]
  20. Heistad D. The effects of Read Naturally on fluency and reading comprehension: A supplemental service intervention (four-school study) 2008 (Unpublished manuscript that met requirements of What Works Clearinghouse) [Google Scholar]
  21. Hiebert EH, Samuels SJ, Rasinski T. Comprehension-based silent reading rates: What do we know? What do we need to know? Literacy Research and Instruction. 2012;51:110, 124. doi:10.1080/19388071.2010.531887. [Google Scholar]
  22. Horowitz-Kraus T, Breznitz Z. Reaction time and accuracy in erroneous vs correct responses among dyslexic and regular readers: from letters to sentences. Dyslexia. 2011;17:72–84. doi: 10.1002/dys.417. doi: 10.1002/dys.417. Epub 2010 Aug 23. [DOI] [PubMed] [Google Scholar]
  23. Katzir T, Kim Y, Wolf M, O'Brien B, Kennedy B, Lovett M, et al. Reading fluency: The whole is more than the parts. Annals of Dyslexia. 2006;56:51–82. doi: 10.1007/s11881-006-0003-5. PMID: 17849208 [PubMed - indexed for MEDLINE] [DOI] [PubMed] [Google Scholar]
  24. Katzir T, Leseaux N, Kim Y-S. Reading self concept and reading comprehension for middle grade elementary school children. Reading and Writing: An Interdisciplinary Journal. 2009;22:261–276. [Google Scholar]
  25. Kim Y-S, Wagner RK, Foster E. Relations among oral reading fluency, silent reading fluency, and reading comprehension: A latent variable study of first-grade readers. Scientific Studies of Reading. 2011;15:338–362. doi: 10.1080/10888438.2010.493964. DOI:10.1080/10888438.2010.493964. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lovett MW. A developmental approach to reading disability: Accuracy and speed criteria of normal and deficient reading skill. Child Development. 1987;58:234–260. [PubMed] [Google Scholar]
  27. Mather N, Hammill DD, Allen EA, Roberts R. TOSWRF: Test of silent word reading fluency: Examiner's manual. PRO-ED; Austin, TX: 2004. [Google Scholar]
  28. McKeown MG, Beck IL. The role of metacognition in understanding and supporting reading comprehension. In: Hacker DJ, Dunlosky J, Graesser AC, Sternberg Robert, editors. Handbook of Metacognition in Education. The Educational Psychology Series. Lawrence Erlbaum Associates; Mahwah, NJ: (in press) [Google Scholar]
  29. Mosteller F, Tukey J. Data analysis and regression: A second course in statistics. Addison-Wesley; Boston: 1977. [Google Scholar]
  30. Nathan RG, Stanovich KE. The causes and consequences of differences in reading fluency. Theory into Practice. 1991;30:176–184. [Google Scholar]
  31. Nation K. To read but not to read: Identifying and understanding the nature of poor reading comprehension in children. In: Newman T, Sternberg RJ, editors. Students with both gifts and learning disabilities. Kluwer; Dordecht: 2004. pp. 119–129. [Google Scholar]
  32. National Reading Panel Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. 2000 http://www.nichd.nih.gov/publications/nrp/smallbook.htm.
  33. Perfetti C. Language comprehension and fast decoding: Some psycholinguistic prerequisite for skilled reading comprehension. In: Guthrie J, editor. Cognition, curriculum, and comprehension. International Reading Association; Newark, DE: 1977. pp. 20–41. [Google Scholar]
  34. Petscher Y, Kim YS. The utility and accuracy of oral reading fluency score types in predicting reading comprehension. Journal of school psychology. 2011;49:107–29. doi: 10.1016/j.jsp.2010.09.004. [ISSN] 1873-3506. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Pianta RC. Early education in transition. In: Pianta RC, Cox MJ, Snow KL, editors. School readiness & the transition to kindergarten in the era of accountability. Paul H. Brookes; Baltimore: 2007. pp. 3–10. www.ccsso.org. [Google Scholar]
  36. Potter M. Rapid serial visual presentation (RSVP): A method for studying language processing. In: Kieras D, Just M, editors. New methods in reading comprehension research. Erlbaum; Hillsdale, NJ: 1984. pp. 91–118. [Google Scholar]
  37. Psychological Corporation . Wechsler individual achievement test. 2nd ed. Author; San Antonio, TX: 2002. [Google Scholar]
  38. Rasinski T, Samuels SJ, Hiebert E, Petscher Y, Feller K. The relationship between a silent reading fluency instructional protocol on students’ reading comprehension and achievement in an urban school setting. Reading Psychology. 2011;32:75–97. doi: 10.1080/02702710903346873. textproject.org. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Read Naturally . Read Naturally Masters Edition (ME) Read Naturally, Inc.; Saint Paul, Minnesota: 1997-2008. [Google Scholar]
  40. Schreiber PA. On the acquisition of reading fluency. Journal of Literacy Research. 1980;12:177–186. [Google Scholar]
  41. Schwanenflugel PJ, Meisinger EB, Wisenbaker JM, Kuhn MR, Strauss GP, Morris RD. Becoming a fluent and automatic reader in the early elementary school years. Reading research quarterly. 2006;41:496–522. doi: 10.1598/RRQ.41.4.4. PMID: 20072665[PubMed] PMCID: PMC2805254. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Shadish W, Cook T, Campbell D. Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Company; Boston: 2002. [Google Scholar]
  43. Souvignier E, Mokhlesgerami J. Using self-regulation as a framework for implementing strategy instruction to foster reading comprehension. Learning and Instruction. 2006;16:57–71. [Google Scholar]
  44. Thomson J, Chennault B, Abbott R, Raskind W, Richards T, Aylward E, et al. Converging evidence for attentional influences on the orthographic word form in child dyslexics. Journal of Neurolinguistics. 2005;18:93–126. doi:10.1016/j.jneuroling.2004.11.005. [Google Scholar]
  45. Torgesen JK. Individual differences in response to early interventions in reading: The lingering problem of treatment resisters. Learning Disabilities Research and Practice. 2000;15:55–64. [Google Scholar]
  46. Torgesen JK, Wagner RK, Rashotte CA. Test of word reading efficiency. PRO-ED; Austin, TX: 1999. [Google Scholar]
  47. U. S. Department of Education, National Center for Education Statistics . Listening to children read aloud. Author; Washington, DC: 1995. This report is available from Education Publications Center (ED Pubs) www.EDPubs.gov. [Google Scholar]
  48. U.S. Department of Education, Office of Vocational and Adult Education . Postsecondary education transition: A summary of the findings from two literature reviews. Author; Washington, D.C: 2010. This report is available on the Department's Web site at: http://www.ed.gov/about/offices/list/ovae/pi/cclo/index.html. [Google Scholar]
  49. Wanzek J, Roberts G, Linan-Thompson S, Woodruff AL, Murray CS. Differences in the relationship of oral reading fluency and high-stakes measures of reading comprehension. Assessment for Effective Intervention. 2010;35:67–77. doi: 10.1177/1534508409339917. doi:10.1177/1534508409339917. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wiederholt J, Bryant B. Gray oral reading test. 3rd ed. Psychological Assessment Resources; Odessa, FL: 1992. [Google Scholar]
  51. Wolf M, Katzir-Cohen T. Reading fluency and its intervention. Scientific Studies of Reading. 2001;5:211–239. [Google Scholar]
  52. Wong BYL. Pointers for literacy instruction from educational technology and research on writing instruction. The Elementary School Journal. 2001;101:359–369. [Google Scholar]
  53. Wong BYL, Butler DL, Ficzere SA, Kuperis S. Teaching adolescents with learning disabilities and low achievers to plan, write, and revise opinion essays. Journal of Learning Disabilities. 1996;29:197–212. doi: 10.1177/002221949602900209. [DOI] [PubMed] [Google Scholar]

RESOURCES