The Relation Between Text Reading and Reading Comprehension Varies as a Function of Developmental Phase, Orthographic Depth, and Measurement Characteristics: Evidence From a Meta-Analysis

Abstract

We examined the relation between text reading fluency and reading comprehension, and moderators of the relation, including grade level, orthographic depth, and assessment task type (for text reading: text reading efficiency, accuracy, rate, sentence verification, maze; for reading comprehension: e.g., multiple-choice, oral retell, cloze), using a meta-analysis. Results from 401 studies (1,253 effect sizes; 266,880 participants) showed that across different types of text reading and reading comprehension tasks, text reading was strongly related with reading comprehension (r = .61, or .70 when correcting for measurement error) while text reading efficiency had a stronger relation (.65) than text reading accuracy (.59) or text reading rate (.54). Furthermore, the correlation differed by grade level and orthographic depth: .73 in primary grades, .69 in upper elementary grades, .59 in middle school, .54 in high school, and .44 for adults in deep orthographies, compared to .69 in primary grades, .52 in upper elementary grades, .42 in middle school, and .29 in high school in shallow orthographies. The maze and sentence verification tasks were more strongly related to measures of text reading than to reading comprehension measures. The magnitude of relation differed by measurement approaches: text reading measured by text reading efficiency and maze tasks had the strongest relation with reading comprehension; text reading had stronger relations with reading comprehension measured by the multiple-choice, the cloze task, and oral open-ended tasks than the written open-ended and retell tasks. The patterns of relations were the same when correcting for measurement error, although magnitudes were generally stronger.

Text reading fluency, widely known as reading fluency, is commonly defined as accurate, fast, and expressive reading of connected texts (Fuchs et al., 2001; Hudson et al., 2005; Kuhn et al., 2010; Rasinski et al., 2009). Text reading fluency is widely included as part of screening and progress monitoring assessments particularly in the United States (Council of Chief State School Officers, 2021; Kim et al., 2021; Sabatini et al., 2019) because of its consistent relation with reading comprehension (e.g., Fuchs et al., 2001; Jenkins et al., 2003; Katzir et al., 2006; Kim et al., 2010, 2011, 2015, 2021; Kuhn et al., 2010; Psyridou et al., 2022). Despite its wide use in educational settings and a general perception of its strong relation with reading comprehension, however, a comprehensive understanding of the relation between text reading fluency and reading comprehension, along with the moderating factors influencing this relation, remains absent in current literature. This gap is significant as critical questions, such as the following, remain unanswered: What is the strength of the relation between text reading fluency and reading comprehension? Is the relation between text reading fluency and reading comprehension uniform or different across developmental phases of reading (e.g., in elementary grades versus secondary school)? Are there variations in this relation among individuals learning to read in languages characterized by different orthographic depths (e.g., English, characterized by a deep orthography, as opposed to Finnish or Italian, featuring shallow orthographies)? Additionally, does the relation hold consistent across different text reading fluency and reading comprehension tasks? In order to fill this gap in the literature, the present study aimed to examine the magnitude of the relation between text reading fluency and reading comprehension and potential moderators of the relation such as developmental phase (proxied by grade level), orthographic depth, and the nature of text reading fluency and reading comprehension tasks, using a meta-analysis.

Addressing these inquiries has both theoretical and potential practical implications. Theoretically, a recent theoretical model, the direct and indirect effects model of reading (DIER; Kim, 2020a, 2020b, 2023), proposes that relations among skills are not fixed or uniform, but instead they vary systematically as a function of several factors, such as developmental phase, text features (including orthographic depth), and assessment characteristics (e.g., task types). We investigated this hypothesis in the context of text reading fluency and reading comprehension. The current meta-analysis on the relation between text reading fluency and reading comprehension is relevant to the recent discussion on the science of reading. As noted above, text reading fluency is widely used as part of screening and progress-monitoring assessments, aligning with the multi-tiered system of supports (MTSS) framework to enhance reading instruction and prevent reading difficulties (e.g., Fien et al., 2021). Our findings can shed light on whether text reading fluency has a consistent or varying relations to reading comprehension across different grade levels and orthographic depths. For instance, text reading fluency and reading comprehension might not be strongly related in high school; if this is true, investing substantial resources and time in measuring and improving text reading fluency in high school as an effort to enhance reading comprehension, possibly at the expense of instructional attention to other skills (e.g., inference making, perspective taking), may not be the most effective use of resources.

As noted above, text reading fluency refers to the accurate, fast, and expressive reading of connected texts (National Institute of Child Health and Human Development, 2000). Therefore, we included studies that operationalized text reading fluency as connected text reading, not reading of words in isolation (i.e., word reading). Although it develops from word reading and is strongly related with word reading skill, text reading fluency is a dissociable construct from word reading fluency (Jenkins et al., 2003; Kim, 2015; Kim & Wagner, 2015; Wolf & Katzir-Cohen, 2001). In addition, although the definition of text reading fluency includes reading prosody or expression (rhythm, phrasing, and changes in pitch while reading), reading prosody was not included in the current work for two reasons: one, the vast majority of studies on text reading fluency and its use in school and clinical settings do not include reading prosody, and two, the relation of reading prosody to reading comprehension has been reported in a recent meta-analysis (Wolters et al., 2022). In practice, text reading fluency is widely operationalized as text reading efficiency, which accounts for accuracy and speed of text reading (e.g., the number of accurately read words per min or per specified time). However, in addition to text reading efficiency, studies have reported text reading accuracy (e.g., the number of words read accurately in a text without taking time into account) and rate (e.g., the number of words attempted within a specified time without considering accuracy; e.g., Kasperski et al., 2016). In the present study, we use the term “text reading” to refer to text reading efficiency, text reading accuracy, and text reading rate collectively and examined the relation of each of these three operationalizations of text reading to reading comprehension.

It should be noted that the present meta-analysis examined the correlation between text reading fluency and reading comprehension and its moderators, not the directionality of the relation between text reading fluency and reading comprehension. Regarding directionality, studies have investigated whether text reading fluency is a predictor or an outcome of reading comprehension (Baker et al., 2011; Jenkins et al., 2003; Kim et al., 2021) or investigated a potential bidirectional relation (e.g., Little et al., 2017). Although this is an important topic, it is beyond the scope of the present meta-analysis.

Theoretical Background

LaBerge and Samuels’ (1974) automaticity theory of reading states that readers have limited cognitive reserves and that a substantial proportion of attention to decoding words leaves little resources for comprehension or meaning-making processes. Once readers can read texts with automaticity, they are able to shift their attention to higher order comprehension processes. Furthermore, Perfetti’s (1985) verbal efficiency theory emphasizes the importance of “efficiency” of the underlying systems—orthographic, phonological, and semantic processes—in constructing high-quality mental representations necessary for reading comprehension. Although Perfetti’s hypothesis does not directly address text reading fluency, its emphasis on the efficiency of underlying processes for reading comprehension is relevant as text reading fluency is built on these processes (Kim, 2015; Wolf & Katzir-Cohen, 2001). The idea of proficiency in underlying skills was elaborated in the developmental model of text reading fluency (Kim, 2015; Wolf & Katzir-Cohen, 2001) and the direct and indirect effects model of reading (DIER; Kim, 2020a, 2020b, 2023). According to the developmental model of text reading fluency, text reading fluency is the product of the development of skills and proficiency in underlying lower-level component skills of reading (e.g., phonological awareness, orthographic knowledge, word reading) as well as proficiency in semantic processes and skills (e.g., vocabulary; Kim, 2015; Wolf & Katzir-Cohen, 2001). This conceptualization is further specified in DIER (Kim, 2020a, 2020b), according to which text reading fluency is built on proximal skills such as word reading and listening comprehension as well as their component skills: sublexical processes that support word reading (knowledge and awareness of phonology, orthography, and morphology) and lexical, syntactic, and semantic processes that undergird listening comprehension (e.g., vocabulary, syntactic knowledge). Importantly, it is posited that although reading comprehension also taps into similar processes and associated skills as text reading fluency, text reading fluency and reading comprehension are not the same constructs because the postlexical semantic processing in text reading fluency taps shallow comprehension whereas listening comprehension and reading comprehension tap into both shallow and deep comprehension that requires higher order inference and evaluation of text (Kim, 2015, 2020). As an empirical support, a study using longitudinal data (i.e., correlational data from between-person differences) showed that higher order cognitive skills such as perspective taking needed for establishing coherence were uniquely related to reading comprehension but not in text reading fluency (see Kim, 2015, for further theoretical elaboration). Therefore, according to DIER, text reading fluency acts as a mediator or bridge connecting word reading and listening comprehension to reading comprehension. However, the nature of mediation differs for word reading and listening comprehension and as a function of developmental phase: text reading fluency increasingly mediates the relations of word reading and listening comprehension with development of reading skills, and at some developmental point, it completely mediates the relation of word reading to reading comprehension whereas it never completely mediates the relation of listening comprehension to reading comprehension.

Moderators of the Relation Between Text Reading Fluency and Reading Comprehension

DIER (Kim, 2020a, 2020b, 2023) not only includes text reading fluency as a component skill of reading comprehension, but also has explicit hypotheses about the nature of relations among component skills and reading such as hierarchical, interactive, and dynamic relations. Directly pertinent to the current study is the dynamic relations hypothesis, which posits that the relations between component skills and reading are not uniform, but dynamic. They vary as a function of several factors, including reading development, text characteristics such as orthographic depth and text complexity, and measurement characteristics such as task types. In the present study, we examined whether the relation between text reading and reading comprehension differs as a function of developmental phase of reading, orthographic depth, and task types of text reading and reading comprehension.

Developmental Phases of Reading

According to DIER (Kim, 2020a, 2020b, 2023), text reading fluency is hypothesized to be more strongly related to reading comprehension in the beginning phase of reading development. In the beginning phase of reading development, children rapidly develop word reading skill and its precursor skills (e.g., letter sound knowledge), and text reading fluency and reading comprehension are largely constrained by word reading skill; therefore, the relation between text reading fluency and reading comprehension is expected to be especially strong during this period. As children develop their word reading skills, their text reading fluency development plateaus, and this, in turn, diminishes the role of text reading fluency in reading comprehension (Jenkins et al., 2003; Kim, 2020a, 2020b). To examine this hypothesis, grade level was used as a proxy for developmental phase in the present study.

Orthographic Depth

According to DIER, another potential moderator in the relation between text reading fluency and reading comprehension is the orthographic depth of the language. Orthographic depth refers to the predictability of grapheme-phoneme correspondences (Frost et al., 1987). Languages with shallow (or transparent) orthographies, such as Finnish, Spanish, and Italian, have consistent grapheme-phoneme correspondences (Frost et al., 1987; Ziegler & Goswami, 2005) whereas deep (or opaque) orthographies, such as English, French, unvowelized Hebrew and Arabic, and Chinese have inconsistent grapheme-phoneme correspondences (Frost et al., 1987; Ziegler & Goswami, 2005). Research has shown that beginning readers in shallow orthographies develop word reading skills at a faster rate than those learning to read in deep orthographies (e.g., Aro & Wimmer, 2003; Florit & Cain, 2011; Seymour et al., 2003; Ziegler & Goswami, 2005). Therefore, readers of a transparent orthography would reach proficiency in text reading fluency faster (i.e., at an earlier grade level or with fewer years of instruction) than children learning to read in a deep orthography. By extension of the same logic elaborated in the dynamic relations hypothesis as a function of reading development described above, the magnitude of the relation between text reading fluency and reading comprehension in shallow orthographies would become weaker after fewer years of reading instruction whereas the relation would remain stronger for a longer period in deep orthographies.

Text Reading Task Types

According to the dynamic relations hypothesis of DIER, the nature of relations among component skills of reading varies as a function of measurement characteristics of constructs (Kim, 2020a, 2020b, 2023). Theoretical models typically discuss constructs with the assumption of precise and error-free measurement of constructs (see Kim, 2020a, 2023). In reality, constructs are not directly observable and are assessed through tasks that differ in their characteristics and include measurement error. DIER explicitly acknowledges the distinction between constructs and their measurement, and posits that measurement impacts the nature of relations. This is especially true for complex constructs like text reading and reading comprehension (Kim, 2020a, 2023). In the present study, we examined whether the magnitude of the relation between text reading and reading comprehension differs as a function of different approaches to measuring text reading and reading comprehension. Please see Appendix Table A1 for categories and examples of text reading and reading comprehension task types used in the present study.

Text reading has been operationalized and measured in various ways, such as text reading efficiency, text reading accuracy, text reading rate/speed, sentence verification, and maze. The most widely used approach is text reading efficiency (widely known as oral reading fluency), such as the number of words read accurately in connected texts within a specified time (e.g., 1 min; e.g., Sabatini et al., 2019; White et al., 2021). Another widely used approach to measuring text reading fluency is sentence verification within a specified time. In this task, the individual is asked to read a short, simple sentence (e.g., 3 words, Apples are blue) and asked to rate the veracity of the read sentences; the score is the number of accurately identified items within a specified time (e.g., 3 min; Test of Silent Reading Efficiency and Comprehension [TOSREC; Wagner et al., 2010]; Psyridou et al., 2022). Sentence verification is considered a text reading fluency measure given its timed nature and the simple sentence structures that are meant to tap shallow comprehension (e.g., Denton et al., 2011; Kim et al., 2011; Kim & Wagner, 2015), although some studies included it as a measure of reading comprehension (e.g., Kang & Shin, 2019; Martín-Aragoneses et al., 2023).

Another widely used task in the field is the maze task. In the maze task, students are asked to read connected texts with systematically omitted words at every nth word. Unlike cloze tasks, blanks in maze tasks include choices for the missing word, reducing the cognitive strain of the task (Miura Wayman et al., 2007). Similar to sentence verification, maze tasks are typically timed and the score is the number of accurate responses within a specified time (e.g., Dynamic Indicators of Basic Early Literacy Skills [DIBELS]; AIMSweb Reading Maze; Shinn & Shinn, 2002). Maze tasks have been conceptualized as a measure of reading comprehension in some studies (Hale et al., 2012; McCane-Bowling et al., 2014). However, previous studies have shown that maze is limited in tapping into deep comprehension (e.g., Pollitt & Harrison, 2021; Silberglitt et al., 2006), and some studies used it as a text reading fluency measure (see Miura Wayman et al., 2007, for a review of the maze task in reading research).

Reading Comprehension Task Types

Reading comprehension is one of the most difficult domains to assess as it is a complex, multidimensional construct (Magliano et al., 2007; RAND Reading Study Group, 2002). Comprehension is influenced by person characteristics (e.g., language and cognitive skills); text characteristics such as content, text complexity, and demands on knowledge as well as the nature and goals of reading (e.g., reading for leisure versus studying); and task types and format (Kim, 2020a, 2020b) and their interactions. DIER posits that relations among component skills and reading vary depending on measurement characteristics of constructs, including reading comprehension (Kim, 2020a, 2020b, 2023). Widely used reading comprehension assessment types include multiple-choice, short open-response (oral or written responses), cloze, or retell (oral or written retell) tasks. Studies have shown that various reading comprehension tasks draw on skills and knowledge differentially (Cain & Oakhill, 2006; Cao & Kim, 2021; Francis et al., 2006; Kendou et al., 2014; van den Broek et al., 2011). For example, Keenan and colleagues (2008) found that reading tasks differed in the extent to which they draw on word reading and listening comprehension skill for children in primary grades and also found that cloze tasks tapped decoding or word reading related skills to a greater extent during the beginning phase of reading development. In addition, studies suggested that retell tasks tend to rely more on decoding or word reading related skills compared to comprehension skill (e.g., Kendeou et al., 2012) and tap into shallow comprehension, not deep comprehension of texts (McNamara et al., 1996). If reading comprehension tasks vary in the extent to which they tap into component skills, it is reasonable to speculate that the magnitude of the relation between text reading fluency and reading comprehension might vary as a function of the types of reading comprehension tasks.

The Current Study

The construct of text reading, text reading fluency in particular, has an important role in reading development, and a large body of studies have examined the relation between text reading fluency and reading comprehension. Theories and studies reviewed in the preceding section suggest that the relation is intricate and nuanced. In the present meta-analysis, we delve into this complexity by examining the magnitude of the relation and moderators of the relation through the following three research questions. First, what is the overall magnitude of the relation between text reading (efficiency, accuracy, and rate) and reading comprehension? Second, does the relation vary by developmental phase (proxied by grade level), orthographic depth, or the interaction of the two? Third, does the relation vary by the nature of text reading and reading comprehension tasks?

Based on a large body of previous studies, we hypothesized that there would be a moderate to strong relation between text reading and reading comprehension. We also hypothesized that the relation would vary as a function of grade level, a proxy for developmental phase of reading skills, such that the relation would be stronger in lower grade levels than in higher grade levels (e.g., Fuchs et al., 2001; Jenkins et al., 2003; Kim, 2015; Kim & Wagner, 2015). We further posited that the relation between text reading and reading comprehension would become weaker after fewer years of reading instruction in transparent orthographies and the relation would remain stronger for a longer period in deep orthographies (see the Orthographic Depth section above for rationale). That is, we hypothesized differential magnitudes of relations as a function of both grade level and orthographic depth. Lastly, we hypothesized that the magnitude of the relation would vary depending on text reading and reading comprehension task types (please see the “Text Reading Task Types” and “Reading Comprehension Task Types” subsections above for rationale). Specifically, we anticipated that compared to text reading accuracy or speed/rate, text reading efficiency would have a stronger relation with reading comprehension because text reading efficiency, not text reading accuracy or rate alone, taps automaticity that is hypothesized to be important to reading comprehension in theoretical models. In addition, we posited that the relation would be stronger when reading comprehension is measured by the cloze task especially in the beginning phase of reading development compared to the other types of tasks because previous studies found that cloze tasks tended to rely more on word reading skill particularly in primary grades (e.g., Keenan et al., 2008).

Method

Literature Search Strategy

We searched articles from relevant databases on ProQuest including APA PsycArticles^®, APA PsycInfo^®, APA PsycTests^®, ERIC, Linguistics and Language Behavior Abstracts, ProQuest Dissertations & Theses A&I, ProQuest Dissertations & Theses Global, and Sociological Abstracts. The dates of the search were limited to the range of January 1, 1980, to December 31, 2022. We focused on this period because empirical research on text reading fluency was limited before the 1980’s. The search included all publication types (peer-reviewed journal articles, theses, dissertations, book chapters, and reports).

The search terms used Boolean operators to allow for a broad search. For reading comprehension, the following terms were included: comprehen* OR “read* skill*” OR “read* comprehension” OR “understand* text” OR “read* text” OR “read* passage” OR “understand* passage” OR “read* paragraph” OR “understand* paragraph”. “Read* skill*” was included as reading comprehension is frequently used as a measure of reading achievement or skill. For text reading fluency, the following terms were included: “read* fluen*” OR “read* efficien*” OR “read* automatic*” OR “read* competen*” OR “read* rate” OR “read* speed” OR “read* accura*” OR “text fluen*” OR “text automatic*” OR “text competen*” OR “text rate” OR “text accura*” OR “paragraph fluen*” OR “paragraph automatic*” OR “paragraph competen*” OR “paragraph rate” OR “paragraph accura*” OR “passage fluen*” OR “passage automatic*” OR “passage competen*” OR “passage rate” OR “passage accura*” OR TOSREC. TOSREC was specifically included in the search terms due to its prevalence as a sentence verification task.

We also conducted citation chains of previous reviews (albeit not meta-analyses) that included text reading fluency and reading comprehension (i.e., Jenkins et al., 2003; Klauda & Guthrie, 2008; Kuhn & Schwanenflugel, 2017; Kuhn & Stahl, 2003; Rasinski, 2004; Wolf & Katzir-Cohen, 2001). Author contacting was not employed.

Inclusion and Exclusion Criteria

To be included in the present study, a study had to include at least one measure of text reading and one measure of reading comprehension, a Pearson’s correlation value between the text reading and reading comprehension measures, and four or more participants. (There are no established guidelines about the number of participants needed for reliable correlations; therefore, four was arbitrarily used.) Studies had to be reported in English regardless of the language in which they were conducted. If a study included a randomized control trial, intervention, or experimental design, correlations for the control group and/or pretest data collection had to be reported. Studies that did not meet these inclusion criteria were excluded. Exclusion criteria were studies that reported aggregated correlations of different skills (e.g., a combination of text reading and word reading) and studies that included only clinical populations (i.e., participants with schizophrenia, traumatic brain injuries, aphasia, down syndrome). Studies of participants with learning disabilities (e.g., those with dyslexia, language impairment, ADHD) were included. Longitudinal correlations, defined as measurement intervals longer than 3 months, were excluded. This is because correlations generally weaken over time, and also the longitudinal time intervals varied widely across the studies, which complicates the interpretation of results.

Screening

Figure 1 presents the PRISMA chart of the current meta-analysis. A total of 7,080 documents were found and uploaded to Rayyan, an online application for systematic reviews and meta-analyses (Ouzzani et al., 2016). The title and abstract screening was conducted using the Rayyan application. Data extraction, screening, coding, and reliability calculation processes were conducted by two doctoral students in education. Discrepancies were discussed among the authors and were resolved. A total of 3,144 duplicate articles were identified, leaving 3,936 possible articles for screening. After abstract and title screening, 2,348 articles were included for full-text screening and 1,588 were excluded from further consideration. The most common reasons for exclusion in this phase were not including reading comprehension (n = 375), being a review article such as a meta-analysis or systematic review (n = 267), being a methodological report without results on text reading and reading comprehension (n = 139), including word reading (decoding) as a measure of text reading (n = 126), and not including text reading (n = 94). In the abstract and title screening, interrater agreement (exact agreement) was 94% on a randomly selected sample of 7% of reviewed studies (n = 300).

Figure 1.

PRISMA Chart

Note. TRF = Text reading fluency; RC = Reading comprehension.

During the full-text screening, 1,542 articles were excluded, an additional 132 duplicates were found (primarily dissertation studies that had later been published), and 204 articles could not be located, leaving 467 articles to be coded. The most common reasons for exclusion during this round were not reporting a Pearson’s correlation value between a measure of reading comprehension and text reading (n = 729), not including a measure of either reading comprehension or text reading (n = 592), only including clinical populations (n = 78), not including separate pretest or control group correlations (n = 49), and having an insufficient sample size (n = 29). The remaining studies (n = 65) were excluded for the following reasons: unclear assessment criteria (e.g., having a measure labelled text reading but ambiguously describing it in a manner that suggests the task was a word list), secondary data analysis, including only qualitative analyses, or being a review article. Interrater reliability (exact agreement) was 93% on a randomly selected sample of 4% of the articles (n = 90).

During the coding process, additional studies were excluded for the following reasons: having a duplicate sample to another included study (n = 14), reporting longitudinal relations (n = 8), correlating averaged scores across different time points (n = 8), missing a correlation value including either reading comprehension or text reading (n = 8), grouping pretest and posttest scores or control and experimental participants (n = 7), correlating a composite score with a skill not being observed (n = 7), using tasks assessing decoding or word reading skill as a measure of text reading (n = 6), only reporting Spearman’s correlations (n = 5), text reading task being focused on measuring prosody (n = 2), and values being correlated across languages (n = 2). A total of 401 studies met inclusion criteria and were coded. Exact agreement was 94% on a randomly selected sample of 10% of included studies (n = 40).

Coding

The 401 included studies were coded for the following: Pearson’s correlation coefficient between text reading and reading comprehension, sample size, mean age of participants, grade level of participants, percentage of female participants, percentage of participants with a disability, nature of the disability, percentage of second language learner participants, percentage of participants from low socioeconomic families, reading comprehension measure, text reading measure, reliability coefficients for reading measures, language of assessment, and orthographic depth of the language (0: languages with shallow/transparent orthography such as Finnish, Greek, or German; 1: languages with deep/opaque orthography such as English, unvowelized Hebrew, Chinese, and French).

Text reading tasks in the included studies were classified and coded as follows: text reading efficiency, text reading accuracy, text reading rate, sentence verification, and maze. Text reading efficiency was defined either as correct words per minute or specified time or as errors per minute/specified time. Reading accuracy was defined as words read correctly or any scoring of errors or mistakes without taking reading time into consideration. If a study included reading errors as its accuracy or efficiency measure and did not reverse code it for its correlation with reading comprehension, the coefficient was reverse coded. Reading rate was defined as the number of words attempted within a specified time without considering accuracy. In total, there were 587 effect sizes for text reading efficiency, 210 effect sizes for text reading accuracy, 258 effect sizes for text reading rate, 144 effect sizes for sentence verification, and 52 effect sizes for maze. One effect size was not coded for text reading task type as the article did not include a definition of the task employed.

Reading comprehension measures were classified and coded as follows: multiple-choice, oral short open-ended, written short open-ended, cloze, and retell. In total, there were 663 effect sizes for the multiple-choice tasks, 272 effect sizes for the oral open-ended tasks, 19 effect sizes for the written open-ended tasks, 195 effect sizes for the cloze tasks, and 102 effect sizes for the oral retell tasks. Note that written retell was coded but was excluded from the analysis due to the small number of studies. One effect size was not coded for reading comprehension task type as the article did not include an explanation of the task used.

Data Analysis Procedures

The data collected from the included studies were manually entered into Microsoft Excel (Microsoft Corporation, 2018). Once data entry was complete, the final data set was uploaded into R using RStudio version 4.3.1 (R Core Team, 2013; RStudio Team, 2022) and analyzed using the robumeta package (Hedges et al., 2010). This package was used as it considers nested structure of effect sizes within each unique sample and weights studies based on sample size using a robust variation estimator (Tipton, 2015). Fisher’s Z scores and variance were used to create a weighted effect size (Hunter & Schmidt, 2000). If a study included different samples (e.g., a cohort of Grade 1 students and a separate cohort of Grade 2 students), separate identification numbers were applied to the samples so that the groups were not clustered together. The unique identification number is how robumeta identified each unique sample and nested effect sizes. The overall effect size, confidence intervals, and meta-regression using robust variation estimation analysis were all calculated in robumeta. All correlation values were transformed into a Fisher’s Z score to allow for comparative analysis using the following equation: Z_r = 0.5*(LN((1+r)/(1-r))) in which r represents the original correlation value (Field, 2001). The correlation Z scores were then weighted through robumeta by the study’s sample size to ensure large sample correlations were not over-weighted, small sample correlations were not under-weighted, and samples with multiple correlations were adjusted to have the same weight as studies with one correlation. Heterogeneity of effects were examined using I² and Q statistics (Higgins & Thompson, 2002; Viechtbauer, 2007).

The first research question on the overall relation between text reading and reading comprehension was addressed by calculating the average of all of the correlation values. The overall relation was also estimated in metafor to conduct a second robust variation estimation for sensitivity analysis (Viechtbauer, 2010). While robumeta nests effect sizes within the unique sample (k = 1,253, unique sample n = 676, τ² = .17), metafor adjusts every effect size individually, based on the associated sample size (k = 1,253, τ² = .08).

The second research question on the moderating roles of grade level and orthographic depth and the third research question on the moderating role of task types of text reading and reading comprehension were addressed by using random effects meta-regression. The meta-regression models were performed in separate models for grade level and orthographic depth (see Table 1), as well as together in a model that includes grade level and orthographic depth (see Tables 4 and 5). The grade levels were clustered by similar developmental phases of literacy development (primary grades from kindergarten to Grade 2; upper elementary grades from Grades 3 to 5; middle school grades; high school grades; university and older). This was done in order to maximize the number of studies included in the analysis, as many studies included participants across multiple grades in their correlational analyses and did not report correlations by grade level. Studies that included participants in grade levels that did not fit into the same developmental stages, such as correlating the performances of participants in Grades 3 through 8 without reporting correlations for each grade level, were excluded from this moderation analysis (n = 36).

Table 1.

Meta-Regression of Relation Between Text Reading and Reading Comprehension, Including Moderators of Grade Level and Orthographic Depth Without Measurement Error Correction

Variable	Β	SE	df	p	CI.LL	CI.UL
Relation with all measures of text reading	.61***	.01	669	.00	.59	.63
Relation with text reading efficiency	.65***	.02	329	.00	.62	.69
Relation with text reading accuracy	.59***	.02	150	.00	.55	.63
Relation with text reading rate	.54***	.02	185	.00	.50	.58
Grade level a
Intercept (Primary)	.72***	.03	127	.00	.66	.78
Upper elementary	−.07*	.03	255	.04	−.14	−.00
Middle school	−.17***	.04	194	.00	−.25	−.10
High school	−.22***	.04	75	.00	−.30	−.12
University and adults	−.27***	.04	167	.00	−.35	−.18
Orthographic depth
Intercept (Shallow)	.54***	.03	136	.00	.49	.58
Deep	.10***	.03	211	.00	.04	.15

Note. Grade level Τ² = .16. Orthographic depth Τ² = .17.

^aPrimary: Kindergarten to Grade 2, Upper Elementary: Grades 3–5, Middle: Grades 6–8, High: Grades 9–12.

^*p < .05.

^***p < .001.

Table 4.

Meta-Regression of Relation Between Text Reading and Reading Comprehension by Text Reading Task Type, Controlling for Grade Level and Orthographic Depth Without Measurement Error Correction

Variable	β	SE	df	p	CI.LL	CI.UL
Text reading efficiency
Intercept	.79***	.03	152	.00	.72	.86
Accuracy	−.04	.03	208	.18	−.10	.02
Rate	−.12***	.03	261	.00	−.17	−.06
Maze	−.00	.06	35	.99	−.11	.11
Sentence verification	−.08**	.03	114	.009	−.14	−.02
Upper elementary	−.09**	.03	250	.009	−.15	−.02
Middle school	−.17***	.04	195	.00	−.25	−.11
High school	−.23***	.04	76	.00	−.31	−.16
University and adults	−.29***	.04	173	.00	−.38	−.21
Shallow orthography	−.11***	.03	207	.00	−.17	−.05
Text reading accuracy a
Intercept	.75***	.04	140	.00	.68	.83
Efficiency	.04	.03	208	.18	−.02	.10
Rate	−.07*	.03	229	.03	−.14	.01
Maze	.04	.06	42	.50	−.08	.16
Sentence verification	−.04	.04	159	.27	−.11	.03
Text reading rate a
Intercept	.68***	.04	151	.00	.60	.75
Efficiency	.12***	.03	261	.00	.06	.17
Accuracy	.07*	.03	229	.03	.01	.14
Maze	.12	.06	40	.06	−.00	.23
Sentence verification	.04	.04	149	.32	−.04	.11
Maze a
Intercept	.79***	.06	40	.00	.67	.91
Efficiency	.00	.06	35	1.00	−.11	.11
Accuracy	−.04	.05	42	.50	−.16	.08
Rate	−.12	.06	40	.06	−.23	.00
Sentence verification	−.08	.06	48	.19	−.20	.04

Note. Τ² = .16. The reference category or intercept is primary grades in deep orthography.

^aGrade level and orthographic depth were controlled for in all the models, but are shown only in the first panel as they are the same values across the models.

^*p < .05.

^**p < .01.

^***p < .001.

Table 5.

Meta-Regression of Relation Between Text Reading and Reading Comprehension by Reading Comprehension Task Type, Controlling for Grade Level and Orthographic Depth Without Measurement Error Correction

Variable	Β	SE	df	p	CI.LL	CI.UL
Multiple-choice
Intercept	.80***	.04	141	.00	.72	.87
Cloze	.02	.03	90	.50	−.04	.09
Oral open-ended	−.05	.04	141	.23	−.13	.03
Written open-ended	−.17**	.05	10	.006	−.28	−.06
Retell	−.20***	.05	58	.00	−.31	−.09
Upper elementary	−.10**	.04	228	.007	−.17	−.03
Middle school	−.21***	.04	197	.00	−.29	−.13
High school	−.28***	.04	84	.00	−.36	−.19
University and adults	−.34***	.05	174	.00	−.43	−.25
Shallow orthography	−.14***	.03	215	.00	.−.20	−.08
Cloze a
Intercept	.82***	.04	89	.00	.74	.90
Multiple-choice	−.02	.03	90	.50	−.09	.04
Oral open-ended	−.07	.05	139	.12	−.16	.02
Written open-ended	−.20**	.06	14	.003	−.31	−.08
Retell	−.22***	.06	97	.00	−.34	−.11
Oral open-ended a
Intercept	.75***	.04	122	.00	.67	.83
Multiple-choice	.05	.04	141	.23	−.03	.13
Cloze	.07	.05	214	.12	−.02	.16
Written open-ended	−.13	.06	12	.05	−.25	.00
Retell	−.15*	.06	85	.01	−.27	−.03
Written open-ended a
Intercept	.63***	.05	11	.00	.37	.60
Multiple-choice	.17**	.06	10	.006	.06	.28
Cloze	.20**	.06	14	.00	.08	.31
Oral open-ended	.13	.06	12	.05	−.00	.25
Retell	−.03	.07	16	.70	−.17	.12

Note. Τ² = .16. The reference category or intercept is primary grades in deep orthography in the top panel. In the other panels, the reference category of intercept is primary grades.

^aDue to lack of effect sizes in shallow orthographies, in these models, shallow orthography is not included as a control variable. Grade levels were controlled for in all the models, but are shown only in the first panel as they are the same values across the models.

^*p < .05.

^**p < .01.

^***p < .001.

Orthographic depth was coded following previous work (e.g., Seymour et al., 2003). Specifically, English, French, unvowelized Hebrew, unvowelized Arabic, Chinese, and Danish were considered to be deep orthographies whereas Finnish, Korean, Spanish, Finnish, Portuguese, Italian, Japanese (Hiragana or Katakana), Greek, Icelandic, Croatian, Kiswahili, Kikamba, Lubukusu, Turkish, vowelized Hebrew, vowelized Arabic, German, Afrikaans, Setswana, Dutch, Norwegian, Maltese, Russian, Bulgarian, and Amharic were considered to be shallow orthographies. To examine the moderating role of task types, we examined correlations by task types of text reading and reading comprehension. We also fitted meta-regression models to evaluate whether differences in magnitudes are statistically different as a function of task types. Furthermore, we examined the relation of text reading and various reading comprehension tasks by grade levels, given our hypothesis of a particularly stronger relation for reading comprehension measured by the cloze task in primary grades. To address the false discovery rate given the multiple comparisons in the meta-regression models, we applied the Benjamini-Hochberg procedure (Benjamini & Hochberg, 1995).

In the sensitivity analysis, we used metafor as an alternative estimator. We also ran an alternative model using the continuous grade variable to compare the model results with the categorical grade model. With regard to orthographic depth, Portuguese is considered to have a relatively deep orthography, similar to French (Seymour et al., 2003), and therefore, Portuguese was considered as a deep orthography in the sensitivity analysis. We also examined whether the relation is different for Chinese and Japanese Kanji as they have a morphosyllabic writing system whereas the other languages have alphabetic writing systems. Moreover, we replicated the analysis correcting for measurement error as measurement error attenuates relations (Spearman, 1904). Given ongoing debate about its appropriateness (e.g., over correction) and issues on operational procedures (see Borenstein et al., 2011; Kepes et al., 2013), we present results using original correlation values reported in the included studies without disattenuating them in the text here, and results correcting for measurement error (${\stackrel{-}{r}}_{xy}=\stackrel{-}{\rho }\times \stackrel{-}{\sqrt{r_{xx\text{'}}}}\times \stackrel{-}{\sqrt{r_{yy\text{'}}}}$; Wiernik & Dahlke, 2020) are found in Appendix Tables A4 to A9. Range restriction also influences the magnitude of relations (Pearson, 1903). However, although we recognize the importance of range restriction, we chose not to implement corrections for range restriction following the guidance of Sackett and colleagues (2022, 2023). Correcting for range restriction requires information on the degree of range restriction and assumes that the subset of studies affected by range restriction is a random sample from the full set of studies included in the meta-analysis (Sackett et al., 2022). However, many of the included studies did not provide information on the amount or nature of range restriction, and insufficient information can lead to an overestimation of results (Sackett et al., 2022). Publication bias was examined using an Egger’s regression test and funnel plot asymmetry (Egger et al., 1997).

The present study was not preregistered. Data are available from the authors upon reasonable request. R codes for statistical analysis are available in online supplemental materials.

Results

Characteristics of Studies

A total of 401 studies with 1,253 effect sizes, 676 unique samples, and 266,880 participants were included. Due to the large number of studies, characteristics and forest plot of the studies included in the present meta-analysis are presented in Table S1 and Figure S1 in the online supplemental materials. There were 67 studies in a language with a shallow orthography and 333 studies in a language with a deep orthography. Approximately 23% of studies were conducted in the primary grades (n = 104), 31% in the upper elementary grades (n = 183), 10% in middle school (n = 36), 5% in high school (n = 17), and 17% in university or consisting of adult participants (n = 69). The remaining 14% of studies either did not include grade or age information, or grouped multiple grade categories together (i.e., middle and high school students together; n = 58). Regarding text reading task types, 183 studies included a text reading efficiency measure, 90 studies included a reading accuracy measure, 84 studies included a reading rate measure, 36 studies included a sentence verification measure, and 6 studies included a maze measure. With regard to reading comprehension task types, 227 studies included a multiple-choice measure, 53 studies included a cloze measure, 88 studies included an oral open-ended measure, 9 studies included a written open-ended measure, and 22 studies included a retell measure (including four studies with a written retell measure, which were excluded from the primary analysis). The number of effect sizes by variables of interest is presented in Tables A2 and A3 in the Appendix.

Question 1: What is the Overall Magnitude of the Relation Between Text Reading and Reading Comprehension?

The overall average correlation between text reading (including all measures: text reading efficiency, text reading accuracy, and text reading rate) and reading comprehension was .61, p < .001 (see the top panel of Table 1). When the measures of text reading were examined separately, the average correlations were .65, .59, and .54 (ps < .001) for text reading efficiency, text reading accuracy, and text reading rate, respectively. Heterogeneity as measured by the I² and Q statistics was substantial (robumeta: I² = 98.47%, p < .001; metafor: I² = 96.45%, Q(1201) = 57897.33, p < .001; Higgins & Thompson, 2002), indicating that over 96% of the total variance can be attributed to between-study differences. Tau² values for each model are found in the notes section of each table. Across all included studies, the within-sample variance was low (metafor: H²=28.20%).

Question 2: Does the Relation Vary as a Function of Developmental Phase, Orthographic Depth, or the Interaction of the Two?

Developmental Phase

Results of the moderation by grade level (as a proxy for developmental phase) are shown in the middle panel of Table 1. The relation between text reading and reading comprehension was strong, β = .72 (p < .001), in primary grades (kindergarten through Grade 2). The relation was weaker in upper elementary grades (β = −.07, p < .05), middle school (β = −.17, p < .001), high school (β = −.22, p < .001), and university and adults (β = −.27, p < .001) in comparison to primary grades. Interpretation of meta-regression is identical to that of multiple regression. In this case, the intercept of .72 refers to the correlation between text reading and reading comprehension for the reference group, those in the primary grades, and the βs for each grade category indicate the increment or decrement of the correlation coefficient compared to the reference group. Applying this to the current focal results, the correlation between text reading and reading comprehension for each grade level across deep and shallow orthographies was as follows: .72 in primary grades, .65 in upper elementary grades, .55 in middle school, .50 in high school, and .45 for those in university or adults.

Orthographic Depth

When examining the differences by orthographic depth (see the bottom panel of Table 1), the relation between text reading and reading comprehension was weaker in shallow orthographies (r = .54) than in deep orthographies (r = .64 [.54 plus .10]).

Developmental Phase and Orthographic Depth

We also examined correlations by grade level and orthographic depth. As shown in the “across tasks” column of Table 2, we found consistently stronger correlations in deep orthographies (.44 ≤ rs ≤ .73) than in shallow orthographies (.30 ≤ rs ≤ .69) across the primary grades through high school.

Table 2.

Correlation (df) Between Text Reading and Reading Comprehension by Grade Level and Orthographic Depth Without Measurement Error Correction

Grade level	Across tasksa		Text reading efficiency		Text reading accuracy		Text reading rate
	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep
Primary	.69 (36)	.73 (95)	.81 (14)	.79 (53)	.57 (7)	.65 (19)	.52 (13)	.60 (21)
Upper elementary	.52 (55)	.69 (201)	.43 (13)	.71 (118)	.40 (13)	.62 (43)	.39 (16)	.68 (41)
Middle school	.42 (23)	.59 (67)	--	.60 (38)	.45 (6)	.71 (13)	.46 (5)	.55 (12)
High school	.30 (5)	.54 (36)	--	.63 (17)	--	.67 (7)	--	.40 (9)
University and adults	--	.44 (76)	--	.48 (43)	--	.65 (6)	--	.41 (26)
	Across tasksa		Reading comp multiple-choice		Reading comp cloze		Reading comp oral open-ended		Reading comp retell
	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep
Primary	.69 (36)	.73 (95)	.68 (17)	.69 (41)	--	.85 (20)	.74 (18)	.69 (30)	--	.65 (14)
Upper elementary	.52 (55)	.69 (201)	.57 (43)	.72 (147)	--	.69 (23)	--	.62 (27)	.--	.46 (25)
Middle school	.42 (23)	.59 (67)	.44 (21)	.58 (50)	--	.64 (11)	--	.42 (10)	--	--
High school	.30 (5)	.54 (36)	.29 (5)	.55 (32)	--	--	--	--	--	--
University and adults	--	.44 (76)	--	.44 (59)	--	.38 (10)	--	.51 (8)	--	--

Note. Missing values are marked as “--” and are due to insufficient samples to run correlational analyses (df less than 4). Correlations with small degrees of freedom should be interpreted with caution. Written open-ended tasks are not included here as a reading comprehension measure because there were insufficient df when disaggregating by orthographic depth and grade bands. All correlations are statistically significant at p < .01. Comp = comprehension.

^aThe values for the “across tasks” column are identical for the top and bottom panels by design.

Question 3: Does the Relation Vary as a Function of the Nature of the Text Reading or Reading Comprehension Task?

Correlations between text reading and reading comprehension across different task types are shown in Tables 2 and 3. Prior to examining the relations between text reading and reading comprehension, we examined the relations of text reading efficiency, text reading accuracy, and text reading rate with maze and sentence verification (the last two rows in Table 3) because maze and sentence verification tasks have been operationalized as a proxy for reading comprehension in a few previous studies (Hale et al., 2011; Kang & Shin, 2019; Martín-Aragoneses et al., 2023; McCane-Bowling et al., 2014). Their relations were very strong. Notable are .80 between the maze task and text reading efficiency, .90 between sentence verification and text reading efficiency, .93 between the maze task and text reading rate, and .98 between the maze task and text reading accuracy. We also examined the relations of maze and sentence verification tasks with reading comprehension tasks (multiple-choice, cloze, oral open-ended), and they ranged from .40 to .66 (see the two last columns in Table 3).

Table 3.

Correlation Matrix Between Text Reading and Reading Comprehension by Task Type Without Measurement Error Correction

Reading comprehension measure	Text reading fluency measure
	Efficiency	Accuracy	Rate	Maze	Sentence verification
Multiple-choice	.66*** (236)	.58*** (79)	.51*** (97)	.66*** (31)	.55*** (55)
Cloze	.77*** (36)	.64*** (11)	.70*** (21)	--	.56*** (26)
Oral open-ended	.63*** (45)	.66*** (47)	.57*** (47)	--	.40*** (7)
Written open-ended	--	--	.46*** (9)	--	--
Retell	.56*** (27)	.46*** (9)	.37*** (9)	--	--
Mazea	.80*** (21)	.98*** (13)	.93*** (9)	--	--
Sentence verificationa	.90*** (18)	-- --	-- --	--	--

Note. Missing values are marked as “--” and are due to insufficient samples to run correlational analyses (df less than 4).

^aIn the last two rows, we examined the relations of maze and sentence verification with text reading efficiency, text reading accuracy, and text reading rate because maze and sentence verification tasks have been operationalized as a proxy for reading comprehension in a few previous studies (see literature review).

^***p < .001.

Reading Comprehension and Different Task Types of Text Reading

Given the differential relations by grade level and orthographic depth reported in Research Question 2, the relations between reading comprehension and different tasks of text reading (which incorporates text reading efficiency, accuracy, rate, and maze and sentence verification tasks) are reported by grade level and orthographic depth in the top panel of Table 2. When the results for orthographic depth across grade levels were disaggregated by text reading efficiency, text reading accuracy, and text reading rate, there were sufficient degrees of freedom to run correlational analyses for deep orthographies across grade level categories, whereas for shallow orthographies, there were insufficient degrees of freedom for middle school, high school, and higher levels. Results showed that not surprisingly, the patterns of results remained largely similar to those with overall text reading (see the “across tasks” column). Magnitudes were stronger in lower grade levels than higher grade levels across orthographic depths. Magnitudes were generally stronger in deep orthographies than in shallow orthographies across text reading efficiency, text reading accuracy, and text reading rate.

In order to examine whether the magnitudes of the relation are statistically significantly different, we fitted meta-regression models controlling for grade level and orthographic depth. Results between various task types of text reading fluency and reading comprehension are shown in Table 4 and are as follows: Compared to text reading efficiency (see the first panel), text reading rate (β = −.12, p < .001) and sentence verification (β = −.08, p = .01) had weaker relations with reading comprehension whereas text reading accuracy (β = −.04 , p = .18) and the maze task (β = −.00, p = .99) did not significantly differ from text reading efficiency. Compared to text reading accuracy (see the second panel in Table 4), text reading rate had a weaker relation with reading comprehension (β = −.07, p = .03). Text reading rate (see the third panel in Table 4) did not significantly differ from maze or sentence verification (ps ≥ .06) in terms of its relation with reading comprehension. The maze task (see the last panel in Table 4) did not significantly differ from sentence verification (p = .19) in its relation with reading comprehension.

Text Reading and Different Task Types of Reading Comprehension

Results for the relation between text reading and reading comprehension as a function of different tasks of reading comprehension by grade level and orthographic depth are shown in the bottom panel of Table 2. There were sufficient degrees of freedom to run the correlational analyses for reading comprehension measured by multiple-choice tasks across shallow and deep orthographies. However, for the cloze, oral open-ended, and retell tasks of reading comprehension, there were few studies in the shallow orthographies. The multiple-choice task had a fairly strong relation with text reading (measured by various tasks) with similar magnitudes in deep orthographies (r = .69) and shallow orthographies (r = .68) in primary grades. In contrast, magnitudes differed by orthographic depth in other grade levels: in deep orthographies, r = .72 in upper elementary grades, r = .58 in middle school, and r = .55 in high school; in shallow orthographies, r = .57 in upper elementary grades, r = .44 in middle school, and r = .29 in high school. For the other reading comprehension tasks in deep orthographies, the magnitudes tended to be stronger in lower grade levels compared to higher grade levels.

To test whether differences in magnitudes are statistically significant, meta-regressions were conducted and results are presented in Table 5. For the multiple-choice reading comprehension task, there were enough effect sizes across deep and shallow orthographies to include both grade level and orthographic depth as control variables. For the other reading comprehension tasks, there were enough effect sizes only in the deep orthographies, and therefore, only grade level was included as a control variable. Compared to the multiple-choice task (see the first panel in Table 5), the relation was weaker for the written open-ended task (β = −.17, p < .01) and retell task (β = −.20, p < .001) but the relation did not differ for the cloze task (β = .02, p = .50) and oral open-ended task (β = −.05, p = .23). Compared to the cloze task (see the second panel in Table 5), the magnitude of the relation did not differ for the oral open-ended task (β = −.07, p = .12) whereas the relation was weaker for the written open-ended task (β = −.20, p < .01) and retell task (β = −.22, p < .001). The relation with text reading was stronger for the oral open-ended task compared to the retell task (β = −.15, p = .01; see the third panel in Table 5).

Sensitivity Analysis

The overall correlation was estimated using a different estimator, metafor, and a comparable effect size was found (β = .60, p < .001). We ran all analyses with Portuguese coded as a deep orthography and did not find any significant differences compared to the results reported above. The relation between text reading and reading comprehension in Chinese and Japanese Kanji was r = .57 (n = 16, k = 21, p < .001). We also fitted models that included grade level as a continuous variable (kindergarten = 0, Grade 1 = 1, … university and adults = 13). The results showed a statistically significant negative relation by each grade (β = −.02, df = 159, p < .001; see Appendix Table A4).

To address measurement error, we also replicated the analyses reported in Tables 1 to 5 using effect sizes corrected for measurement reliability for reading comprehension and text reading fluency (see Appendix Tables A5 to A9). The overall average correlation between the two constructs increased to .70, compared to .61 reported above. In general, magnitudes of relations were stronger than those in Tables 1 to 5, but patterns of relations were essentially identical.

Publication Bias

Results from the Egger’s test for funnel plot asymmetry showed no signs of publication bias in our sample, z = −1.52, p = .13, b = .62, 95% CI [.58, .65]. The overall distribution of our studies from the Egger’s funnel plot is presented in Figure 2.

Figure 2.

Funnel Plot Results

Discussion

Text reading fluency has garnered tremendous attention not only in practice (schools and clinical settings), but also in terms of theory and empirical research for its consistent relation with reading comprehension. According to the automaticity theory (LaBerge & Samuels, 1974), automaticity in reading allows cognitive resources to be available for higher order comprehension processes, thereby supporting reading comprehension. The developmental model of text reading fluency further added that text reading fluency is built on proficiency and automaticity in lower order processes as well as semantic processes and associated skills (Kim, 2015; Wolf & Katzir-Cohen, 2001). DIER (Kim, 2020a, 2020b, 2023) further elaborated on the proximal skills and processes that influence text reading fluency and distal lower order skills and processes that support proximal skills. Importantly DIER hypothesizes differential relations among component skills and reading skills as a function of several factors, such as development, text characteristics (e.g., orthographic depth), and measurement of constructs (i.e., dynamic relations hypothesis). In this study, we examined the relation between text reading fluency and reading comprehension informed by theory and prior work in order to develop a deeper and more nuanced understanding of the nature of the relation. Specifically, we investigated the overall relation between text reading and reading comprehension, as well as moderation of the relation by developmental phase, orthographic depth, and task types of text reading and reading comprehension.

We found that text reading and reading comprehension were fairly strongly related at .61 when averaged across developmental phases, languages, and all different types of measures of text reading and reading comprehension. When text reading was examined for text reading efficiency (accuracy and speed), text reading accuracy, and text reading rate/speed, the relation was .65, .59, and .54, respectively. These fairly strong relations, text reading efficiency in particular, are overall in line with the automaticity theory (LaBerge & Samuels, 1974), the developmental and component-based models of text reading fluency (Kim, 2015; Wolf & Katzir-Cohen, 2001), and DIER (Kim, 2020a, 2020b, 2023) regarding the relation between text reading fluency and reading comprehension.

Beyond the overall average relation, an important central point in the present study was the nuanced nature of relations informed by the dynamic relations hypothesis of DIER (Kim, 2020a, 2020b, 2023). The findings underscore that the relation between text reading and reading comprehension is neither simple nor uniform. Instead, it varies as a function of developmental phase and orthographic depth, and their interaction. On average, the relation was stronger in primary grades (.72) and became slightly weaker in upper elementary school (.65) and substantially weaker in middle school (.55), high school (.50), and university and beyond (.45). In addition, the average relation across grade levels was stronger in deep orthographies (.64) than in shallow orthographies (.54), and the relation in Chinese was .57. As stated above, we hypothesized a stronger relation in lower grade levels between text reading and reading comprehension because text reading and reading comprehension are largely constrained by word reading skill, a critical skill for text reading and reading comprehension, especially in the beginning phase of reading development. By extension of the same logic, the relation was posited to remain stronger for a prolonged time in deep orthographies because word reading is more complex and takes a longer time to develop in deep orthographies compared to shallow orthographies. Given the significance of both grade level and orthographic depth, the strength of the relation between text reading and reading comprehension should vary depending on the combination of both grade level and orthographic depth. Indeed, we found a moderation between these factors such that the relation was similarly strong regardless of orthographic depth in primary grades (r = .69 in shallow orthographies & r = .73 in deep orthographies), but in later phases of development in middle and high school, the relation was substantially weaker in shallow orthographies than in deep orthographies¹ (i.e., rs = .42 and .30 in middle and high school in shallow orthographies & rs = .59 and .54 in middle and high school in deep orthographies; see Table 2). These results together are in line with the dynamic relations hypothesis of DIER (Kim, 2020a, 2020b, 2023), which states that the magnitude of relations with reading comprehension varies as a function of development and characteristics of words and texts that children learn to read (including orthographic depth).

Our findings also supported the hypothesis that measurement characteristics of constructs matter in the relation between text reading and reading comprehension (i.e., dynamic relations hypothesis as a function of measurement; Kim, 2020a, 2020b). There are several aspects to unpack regarding these results. First, our findings revealed that compared to text reading accuracy or speed/rate, text reading efficiency tended to have a stronger relation with reading comprehension. Second, text reading efficiency had the strongest relation with various measures of reading comprehension except for oral open-ended tasks (see Table 3). These results are in line with the theoretical conceptualization of text reading fluency and its relation with reading comprehension: Automaticity in reading connected texts includes both accurate and fast reading, and this efficiency—not accuracy or speed alone—is important to making cognitive resources available for higher order, meaning-making processes and constructing an accurate mental representation (Jenkins et al., 2003; Kim, 2015; LaBerge & Samuels, 1974). Accurate but slow and laborious reading would take substantial mental resources, which hampers higher order meaning-making processes, while rapid but inaccurate text reading would interfere with the construction of an accurate mental model for deep comprehension. In other words, accurate reading of texts without speed or speed reading without accuracy is not as strongly related to reading comprehension; thus, operationalization of text reading fluency (excluding reading prosody) should take into consideration both accuracy and speed, not either one alone. Third, the field has not been clear about the conceptualization of maze and sentence verification tasks, and studies have conceptualized them as both text reading fluency (e.g., Denton et al., 2011; Kim & Wagner, 2015) and reading comprehension (e.g., Kang & Shin, 2019; Martín-Aragoneses et al., 2023). Our findings showed that the correlations of the maze and sentence verification tasks were stronger with other measures of text reading than with reading comprehension measures (see Table 3).

Lastly, with respect to different tasks of text reading and their relations with reading comprehension (see Table 4) controlling for grade level and orthographic depth, text reading efficiency, text reading accuracy, and maze tasks had equally strong relations with reading comprehension whereas text reading rate/speed and sentence verification tasks had slightly weaker relations with reading comprehension. These results indicate that text reading efficiency, text reading accuracy, and the maze task have stronger predictive validity for reading comprehension compared to text reading rate/speed and sentence verification after accounting for grade level and orthographic depth.

We also examined the relation by different task types of reading comprehension. Reading comprehension is a multidimensional construct, and tasks vary in the extent to which they tap into reading comprehension. As such, we hypothesized that the strength of the relation would vary depending on reading comprehension task types, and this was supported after controlling for grade level and orthographic depth (see Table 5). Specifically, magnitudes of the relation did not differ when reading comprehension was measured by multiple-choice, cloze, and oral open-ended tasks. In contrast, the relation was weaker when reading comprehension was measured by written open-ended response and retell tasks. When considering the results by different measures of text reading and reading comprehension (Table 3), it is notable that the cloze task had the strongest relation with text reading efficiency (.77). As noted above, studies have shown that the cloze task taps into word reading to a greater extent than comprehension at least for children in primary grades (e.g., Keenan et al., 2008). Hence, as text reading fluency and reading comprehension rely on word reading to a large extent particularly during the primary grades (e.g., Hoover & Gough, 1990; Kim, 2020a, 2020b, 2023; Kim & Wagner, 2015), we posited that the relation between text reading fluency and reading comprehension would be stronger when reading comprehension is measured by the cloze task during the primary grade period.

The differential relations between text reading and various reading comprehension tasks are in line with previous work, which showed that reading comprehension tasks vary in the extent to which they tap into different aspects of reading component skills (Cao & Kim, 2021; Cutting & Scarborough, 2006; Keenan et al., 2008). Overall, these results support the dynamic relations hypothesis of DIER, specifically dynamic relations as a function of the measurement of constructs (Kim, 2020a, 2020b, 2023), and suggest the importance of considering measurement of constructs in understanding the nature of relations.

Text reading fluency (operationalized as efficiency) is widely used in schools and districts as part of assessment batteries, especially for screening and progress monitoring (and presumably associated instruction), due to its perceived close relation with reading comprehension. Although the present study supports that text reading fluency is related with reading comprehension, results indicate that we need to consider developmental phases, orthographic depths, and measurement characteristics. In deep orthographies, the relation was strongest in primary and upper elementary grades, followed by moderate relations in later grade levels. In shallow orthographies, the relation was strong in primary grades and moderate in upper elementary grade levels, followed by substantially weaker relations in middle and high school. These findings suggest that the utility of text reading in relation to reading comprehension differs by developmental phase and orthographic depth. Specifically, text reading appears to be particularly useful and informative for reading comprehension in elementary grades across orthographies; in middle and high school, it remains informative and useful in deep orthographies, but not as much in shallow orthographies. Furthermore, measurement of text reading and reading comprehension mattered for the relation. Text reading efficiency tended to be more strongly related to reading comprehension measures compared to text reading accuracy or rate; maze and sentence verification tasks were more strongly related to text reading than to reading comprehension; and reading comprehension as measured by written open-ended and retell tasks had weaker relations with text reading compared to when reading comprehension was measured by multiple-choice, cloze, and oral open-ended tasks.

It is important to note, however, that the results may differ for students who struggle with reading. If part of their struggle is associated with word reading skill, the relation between text reading and reading comprehension will likely remain strong even in middle or high school. Unfortunately, we were not able to examine this hypothesis of whether the relation varies by disability status because although studies included students with reading difficulties and disabilities, only a limited number of studies reported the correlation matrices by disability status (see below).

Note that the current findings indicate the average relation between text reading and reading comprehension such that those who read text more accurately and quickly tend to do better in reading comprehension, on average. However, the current results should not be taken to indicate that readers approach reading texts in the same way regardless of text characteristics or the nature of reading goals. Theoretical frameworks and models recognize the interactions among reader characteristics, text characteristics, and activity context (Kim, 2020a, 2020b, 2023; RAND Reading Study Group, 2002). Research has shown the impact of text characteristics (e.g., complexity) on text reading fluency and comprehension (Best et al., 2008; Francis et al., 2008; Petscher & Kim, 2011; Pickren et al., 2022). Importantly, readers adjust their reading strategies based on the text’s nature and the purpose of reading: When reading for in-depth understanding (e.g., studying for an assignment), individuals tend to slow down and employ more extensive strategies such as rereading, annotating, and question asking and answering compared to when reading for leisure (Magliano et al., 2007). However, the current results of the relation between text reading fluency and reading comprehension should not be interpreted as negating intra-individual differences in reading strategies and behaviors based on reading goals or text and activity characteristics because modulating text reading speed intentionally is distinct from slow and inaccurate text reading due to lack of reading skills (i.e., dysfluent reading).

Limitations and Future Directions

By the nature of meta-analysis, the results reflect individual studies that include samples from multiple and diverse participant groups and various other characteristics (e.g., different languages, orthographic depth, task characteristics). Some of the systematic variations in participant groups were modeled in the moderation analysis based on the DIER theoretical model. However, other characteristics were not modeled. Specifically, although we coded for learning and reading disabilities as well as second language learner status, we could not include these in moderation analyses because the vast majority of studies did not report correlations by these characteristics. The potential variation in the relation between text reading fluency and reading comprehension due to these factors is undoubtedly important to understand and warrants future research. Therefore, the current findings should be interpreted with these limitations in mind.

As mentioned above, the present study did not include reading prosody, one aspect of the text reading fluency construct, for the reasons stated above in the introduction. Therefore, the findings should be interpreted with this in mind. Furthermore, as noted above in the introduction, we examined correlations between text reading fluency and reading comprehension, and therefore, we cannot make claims regarding directionality of the relation. There has been notable work on the directionality between text reading fluency and reading comprehension (e.g., Jenkins et al., 2003; Kim et al., 2021; Little et al., 2017).

We had sufficiently large effect sizes for most of the moderator variables, but there were few studies with high school and adult samples particularly in shallow orthographies; thus, their results should be interpreted with caution. Furthermore, as Table 2 show, we were not able to estimate the relation more comprehensively by reading comprehension and text reading task types and orthographic depth because of a lack of studies for certain types of tasks particularly in shallow orthographies. Specifically, there were few studies that examined reading comprehension using cloze tasks, oral and written open-ended tasks, and retell tasks in shallow orthographies. Similarly, there were few studies in shallow orthographies reported by text reading efficiency, accuracy, and rate/speed beyond elementary grade levels. Future work addressing this gap in the literature is needed.

Although orthographic depth was coded as a binary variable, orthographic depth is a range that is not fully represented by a binary grouping. Future studies may consider systematically capturing orthographic depth in a more continuous manner. In addition, longitudinal correlations (defined as those spanning longer than 3 months) were not included in the current study because correlations become weaker over time and time span varied largely across studies, which makes interpretation of results complex. Nonetheless, longitudinal relations can provide insights into the relation between text reading and reading comprehension over time, and this is a direction for future research to explore.

Conclusion

The evidence presented in this study illustrates a fairly strong relation between text reading fluency and reading comprehension. Importantly, it expands our understanding of factors that moderate their relation, such as developmental phase, orthographic depth, and the nature of assessment tasks. By synthesizing the work on text reading fluency and reading comprehension in the past 40 years, this meta-analysis offers important insights into the relation between the two key skills in reading.

Educational Impact And Implications.

Text reading efficiency (widely known as reading fluency), which encompasses both accuracy and rate, is likely a more useful measure of text reading in relation to comprehension than either text reading accuracy or text reading rate alone. Stronger relations in elementary grades compared to later grades across shallow and deep orthographies indicate greater predictive validity of text reading in the beginning phase of reading development than in the later phase of reading development. The findings also suggest that on average, the utility of text reading fluency in relation to reading comprehension remains for a longer period in deep orthographies than in shallow orthographies. In middle and high school, the relation remained relatively strong in deep orthographies whereas it was substantially weaker in shallow orthographies, suggesting text reading fluency is not a strong indicator of reading comprehension in middle and high schools in shallow orthographies. It is also important to consider measurement (e.g., assessment tasks) of text reading and reading comprehension when evaluating and interpreting the relation between text reading and reading comprehension.

Appendix

Table A1.

Description of Codes and Examples of Items Included in Analysis

Measure	Categories and Details
Grade level	Primary: Grades K-2 Upper Elementary: Grades 3–5 Middle School: Grades 6–8 High School: Grades 9–12 Adult: University and adults
Orthographic depth	Shallow: Korean, Spanish, Finnish, Portuguese, Italian, Japanese using Hiragana or Katakana, Greek, Icelandic, Croatian, Kiswahili, Kikamba, Lubukusu, Turkish, vowelized Arabic, German, Afrikaans, Setswana, Dutch, Norwegian, Maltese, Russian, Bulgarian, Vietnamese, and Amharic Deep: English, French, unvowelized Hebrew, Mandarin, Cantonese, Danish, Polish, and unvowelized Arabic
Text reading fluency tasks	Efficiency (accuracy and speed): any measurement of CWPM (e.g., DIBELS) Rate: measures such as WPM, reading rate, reading speed without accounting for reading accuracy Accuracy: measures of total correct words, total incorrect words (reverse coded) without accounting for reading time Sentence verification: measures that gave participants a sentence and asked them to verify its accuracy with Yes or No (e.g., TOSREC) Maze: measures that asked participants to read a passage with blanks containing word choices for the appropriate missing word (e.g., AIMSweb)
Reading comprehension tasks	Multiple-choice: Reading passages and answering associated multiple-choice questions (e.g., GRADE, SAT-10, TOEFL, GMRT, AIMSweb, STAAR, GORT-3) Oral open-ended: Reading passages and answering orally open-ended questions (e.g., GORT-4, GORT-5, WIAT, QRI, NARA) Cloze: Reading sentences and passages and filling in missing words in the sentences and passages (e.g., Woodcock Johnson Passage Comprehension subtest) Written open-ended: Reading passages and answering open-ended questions in writing (e.g., PROLEC, SAT [open-ended response section only]) Retell: Reading passages and retelling the passages (e.g., DIBELS retell task, the retell task of QRI)

Note. CWPM = correct words per minute; DIBELS = Dynamic Indicators of Early Literacy Skills; WPM = words per minute; TOSREC = Test of Silent Reading Efficiency and Comprehension; GRADE = Group Reading Assessment and Diagnostic Evaluation; SAT-10 = Stanford Achievement Test, 10^th Edition; TOEFL = Test of English as a Foreign Language; GMRT = Gates-MacGinitie Reading Test; STAAR = State of Texas Assessments of Academic Readiness; GORT-3, 4, and 5 = Grey Oral Reading Test, Third Edition, Fourth Edition, and Fifth Edition; WIAT = Wechsler Individual Achievement Test; QRI = Qualitative Reading Inventory; NARA = Neale Analysis of Reading Ability; PROLEC = Batería de Evaluación de los Procesos Lectores.

Table A2.

Number of Effect Sizes by Grade Level and Orthographic Depth

	Shallow orthography	Deep orthography	Total
Primary	91	244	335
Upper elementary	87	400	487
Middle school	25	128	153
High school	8	44	52
University and adults	4	106	110
Total	215	922	1137

Note. The numbers presented reflect the total studies categorized by grade level and their corresponding orthographic depth. Consequently, studies encompassing grade spans that did not align with these categories were omitted. Overall, 1,023 studies focused on deep orthographies and 229 studies examined shallow orthographies.

Table A3.

Number of Effect Sizes by Measurement Type

	Text reading measures
Reading comprehension tasks	Efficiency	Accuracy	Rate	Maze	Sentence verification	Total
Multiple-choice	309	101	137	45	71	663
Cloze	100	16	25	5	49	195
Oral open-ended	101	76	73	1	21	272
Written open-ended	3	5	10	0	1	19
Retell	73	12	13	1	2	101
Total	586	210	258	52	144	1250

Table A4.

Alternative Model of Meta-Regression of Relation Between Text Reading and Reading Comprehension Where Grade Level^a Is Treated as a Continuous Variable

Variable	β	SE	df	p	CI.LL	CI.UL
Not correcting for measurement error
Intercept (Kindergarten)	.72***	.02	383	.00	.68	.77
Grade level	−.02***	.00	188	.00	−.03	−.02
Correcting for measurement error
Intercept (Kindergarten)	.84***	.02	383	.00	.79	.88
Grade level	−.03***	.00	188	.00	−.03	−.02

Note. Τ² = .17

^aGrade levels kindergarten through Grade 12 are included in the analysis, as well as university and adults coded as 13 (the next year after Grade 12).

^***p < .001.

Table A5.

Meta-Regression of Relation Between Text Reading and Reading Comprehension, Including Moderators of Grade Level and Orthographic Depth With Measurement Error Correction

Variable	β	SE	df	p	CI.LL	CI.UL
Relation with all measures of text reading	.70***	.01	668	.00	.67	.73
Relation with text reading efficiency	.75***	.02	329	.00	.71	.79
Relation with text reading accuracy	.67***	.02	149	.00	.62	.72
Relation with text reading rate	.63***	.03	184	.00	.58	.68
Grade level a
Intercept (Primary)	.84***	.04	127	.00	.77	.92
Upper elementary	−.10*	.04	255	.02	−.19	−.02
Middle school	−.22***	.05	195	.00	−.31	−.13
High school	−.26***	.05	75	.00	−.36	−.16
University and adults	−.31***	.05	167	.00	−.42	−.21
Orthographic depth
Intercept (Shallow)	.63***	.03	136	.00	.58	.69
Deep	.09**	.03	211	.007	.02	.15

Note. Grade level Τ² = .23. Orthographic depth Τ² = .23.

^aPrimary: Kindergarten to Grade 2, Upper Elementary: Grades 3–5, Middle: Grades 6–8, High: Grades 9–12.

^*p < .05.

^**p < .01.

^***p < .001.

Table A6.

Correlation (df) Between Text Reading and Reading Comprehension by Grade Level and Orthographic Depth With Measurement Error Correction

Grade level	Across tasksa		Text reading efficiency		Text reading accuracy		Text reading rate
	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep
Primary	.81 (36)	.85 (95)	.99 (14)	.92 (53)	.66 (7)	.75 (19)	.60 (13)	.69 (21)
Upper elementary	.62 (55)	.78 (201)	.53 (17)	.80 (119)	.49 (13)	.71 (43)	.47 (16)	.78 (41)
Middle school	.50 (23)	.67 (67)	--	.68 (38)	.50 (6)	.79 (13)	.56 (5)	.60 (12)
High school	.34 (5)	.62 (36)	--	.68 (17)	--	.78 (7)	--	.45 (9)
University and adults	--	.51 (76)	--	.57 (43)	--	.73 (6)	--	.46 (26)
	Across tasksa		Reading comp multiple-choice		Reading comp cloze		Reading comp oral open-ended		Reading comp retell
	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep	Shallow	Deep
Primary	.81 (36)	.85 (95)	.78 (17)	.85 (41)	--	.94 (20)	.90 (18)	.82 (30)	--	.73 (14)
Upper elementary	.62 (55)	.78 (201)	.67 (43)	.82 (147)	--	.77 (23)	.25+ (6)	.72 (26)	.--	.53 (25)
Middle school	.50 (23)	.67 (67)	.53 (21)	.65 (50)	--	.71 (11)	--	.47 (10)	--	--
High school	.34 (5)	.62 (36)	.34 (6)	.63 (32)	--	--	--	--	--	--
University and adults	--	.51 (76)	--	.52 (57)	--	.44 (10)	--	.59 (8)	--	--

Note. Missing values are marked as “--” and are due to insufficient samples to run correlational analyses (df less than 4). Correlations with small degrees of freedom should be interpreted with caution. Written open-ended tasks are not included here as a reading comprehension measure because there were insufficient df when disaggregating by orthographic depth and grade bands. All correlations are statistically significant at p < .01 except for that marked by +, which is nonsignificant. Comp = comprehension.

^aThe values for the “across tasks” column are identical for the top and bottom panels by design.

Table A7.

Correlation Matrix Between Text Reading and Reading Comprehension by Task Type With Measurement Error Correction

Reading comprehension measure	Text reading fluency measure
	Efficiency	Accuracy	Rate	Maze	Sentence verification
Multiple-choice	.75*** (242)	.66*** (84)	.61*** (103)	.70*** (35)	.67*** (57)
Cloze	.84*** (34)	.67*** (12)	.78*** (22)	--	.68*** (27)
Oral open-ended	.75*** (41)	.73*** (49)	.65*** (48)	--	.49*** (8)
Written open-ended	--	--	.52*** (9)	--	--
Retell	.62*** (29)	.50*** (12)	.43*** (8)	--	--

Note. Missing values are marked as “--” and are due to insufficient samples to run correlational analyses (df less than 4).

^***p < .001.

Table A8.

Meta-Regression of Relation Between Text Reading and Reading Comprehension by Text Reading Task Type, Controlling for Grade Level and Orthographic Depth With Measurement Error Correction

Variable	β	SE	df	p	CI.LL	CI.UL
Efficiency
Intercept	.92***	.05	152	.00	.83	1.01
Upper elementary	−.12**	.04	250	.006	−.20	−.03
Middle school	−.22***	.04	195	.00	−.31	−.13
High school	−.28***	.05	76	.00	−.37	−.19
University and adults	−.34***	.05	173	.00	−.44	−.23
Shallow orthography	−.11***	.03	207	.00	−.17	−.03
Accuracy	−.06	.04	209	.11	−.12	.01
Rate	−.14***	.04	261	.00	−.20	−.06
Maze	−.03	.06	35	.63	−.15	.09
Sentence verification	−.08*	.04	114	.03	−.15	−.01
Accuracy a
Intercept	.87***	.05	140	.00	.78	.96
Efficiency	.06	.04	209	.11	−.01	.12
Rate	−.08*	.04	230	.04	−.16	−.00
Maze	.03	.06	42	.66	−.10	.15
Sentence verification	−.02	.04	159	.58	−.11	.06
Rate a
Intercept	.79***	.05	151	.00	.70	.87
Efficiency	.14***	.04	261	.00	.07	.20
Accuracy	.08*	.04	230	.04	.00	.15
Maze	.11	.06	40	.10	−.02	.23
Sentence verification	.06	.04	149	.18	−.03	.14
Maze a
Intercept	.89***	.07	40	.00	.76	1.03
Efficiency	.03	.06	35	.63	−.09	.15
Accuracy	−.03	.06	42	.66	−.15	.10
Rate	−.11	.06	40	.09	−.23	.02
Sentence verification	−.05	.06	48	.43	−.18	.08

Note. Τ² = .23. The reference category or intercept is primary grades in deep orthography.

^aGrade level and orthographic depth were controlled for in all the models, but are shown only in the first panel as they are the same values across the models.

^*p < .05.

^**p < .01.

^***p < .001.

Table A9.

Meta-Regression of Relation Between Text Reading and Reading Comprehension by Reading Comprehension Task Type, Controlling for Grade Level and Orthographic Depth With Measurement Error Correction

Variable	β	SE	df	p	CI.LL	CI.UL
Multiple-choice
Intercept	.93***	.05	142	.00	.83	1.04
Upper elementary	−.14**	.05	228	.005	−.23	−.04
Middle school	−.27***	.05	197	.00	−.37	−.16
High school	−.33***	.06	84	.00	−.44	−.22
University and adults	−.39***	.06	175	.00	−.51	−.27
Shallow orthography	−.14***	.04	215	.00	−.21	−.08
Cloze	.00	.04	91	.99	−.08	.08
Oral open-ended	−.06	.05	141	.19	−.15	.03
Written open-ended	−.20*	.07	10	.01	−.35	−.06
Retell	−.25***	.06	58	.00	−.37	−.13
Cloze a
Intercept	.94***	.05	90	.00	.84	1.03
Multiple-choice	−.00	.04	91	.99	−.08	.08
Oral open-ended	−.06	.05	139	.23	−.16	.04
Written open-ended	−.20*	.07	13	.01	−.35	−.06
Retell	−.25***	.06	97	.00	−.37	−.12
Oral open-ended a
Intercept	.87***	.05	122	.00	.78	.97
Multiple-choice	.06	.05	141	.19	−.03	.15
Cloze	.06	.05	139	.22	−.04	.16
Written open-ended	−.14	.07	12	.07	−.30	.01
Retell	−.19**	.07	85	.005	−.32	−.05
Written open-ended a
Intercept	.73***	.07	10	.00	.58	.88
Multiple-choice	.20*	.07	10	.01	.06	.35
Cloze	.20*	.07	13	.01	.06	.35
Oral open-ended	.14	.07	12	.07	−.01	.30
Retell	−.04	.08	15	.58	−.21	.12

Note. Τ² = .23. The reference category or intercept is primary grades in deep orthography in the top panel. In the other panels, the reference category of intercept is primary grades.

^aDue to lack of effect sizes in shallow orthographies, in these models, shallow orthography is not included as a control variable. Grade levels were controlled for in all the models, but are shown only in the first panel as they are the same values across the models.

^*p < .05.

^**p < .01.

^***p < .001.