Does the Effectiveness of a Reading Intervention Differ Dependent on Students’ Spanish or English Proficiencies?

Abstract

This study investigated whether linguistic proficiencies in students’ first language (L1)— Spanish—and English (L2) moderated the response to intensive reading intervention for sixth- and seventh-grade multilingual learners (MLs) with reading difficulties. We used confirmatory factor analysis to estimate proficiency scores in English and Spanish using measures of expressive and receptive vocabulary, syntax, and grammar. We then used latent variable moderated structural equation modeling to evaluate how proficiency in English and Spanish moderated the effect of treatment on students’ reading outcomes in response to intervention. Two important findings occurred. First, the overall linguistic proficiencies of the sample were below average, suggesting the prevalence of low L1 and L2 may be high amongst the population of middle gradeMLs with reading difficulties. Second, we observed only one significant moderation effect: the effect of treatment on students’ letter and word recognition was statistically significantly higher for students with higher English proficiency.

1. Introduction

Many questions of how to best support reading achievement of multilingual learners (MLs) with reading difficulties remain unresolved. This situation is especially true for middle school students, whom intervention research has not historically targeted (Baker et al., 2014; Richards-Tutor et al., 2016). However, a growing number of studies have progressed toward needed answers. Two main strands in this body of research include: 1) identifying the linguistic and literacy factors associated with reading and language development and difficulties for MLs; and 2) developing and implementing instruction and intervention that leverages MLs’ strengths and addresses their linguistic and literacy needs. This paper uses advances from the latter line of research to contribute to the former. Specifically, we conducted a secondary data analysis using data collected as part of an extensive and intensive intervention for sixth- and seventh-grade MLs with reading difficulties that provided instruction in word reading, comprehension, fluency, and academic vocabulary over two years.

We address the research question: did linguistic proficiencies in both students’ first language (L1)—Spanish—and English (L2) moderate the response to intervention for Spanish-speaking sixth- and seventh-grade MLs with reading difficulties. This investigation adds nuance to the field’s understanding of the impact of linguistic proficiencies on reading comprehension for middle grade MLs with reading difficulties. Although there is much theoretical and empirical work describing the influence of language and one’s status as an ML on students’ reading proficiency, there is less work describing heterogeneity within reading achievement for the population of MLs, especially for MLs in middle grades. Greater insight into differential responses to intervention dependent on L1 and L2 proficiencies has implications for the instruction and intervention of this vulnerable population.

1.1. Factors Associated with Reading Proficiency in Middle Grade English Learners

Multilingual learners are not just acquiring oral English proficiency, they are also acquiring knowledge in reading, writing, speaking, and/or understanding English as a means of also accessing and understanding content knowledge. For many MLs in middle school, limited English proficiency impacts their ability to acquire content from text (Author). . Common theoretical models of reading comprehension including the Simple View of Reading (Hoover & Gough, 1990), the lexical quality hypothesis (Perfetti & Hart, 2002; Perfetti, 2017), and the Construction-Integration Model (Kintsch, 1988; 2018) can be useful in framing the role linguistic proficiencies play in reading development for MLs. These models identify constituent factors associated with the reading process, especially the role of linguistic comprehension (i.e., vocabulary and listening comprehension, Cho et al., 2019) and thereby provide a framework to understand both the role linguistic proficiencies play in the development of reading comprehension as well as areas where proficient reading processes falter, resulting in reading difficulties.

Proposed by Hoover and Gough (1990), the Simple View of Reading argues that proficient reading is the product of two processes: 1) top-down (e.g., efficient and accurate word decoding/ recognition) and 2) bottom-up (e.g., linguistic comprehension of both isolated words and connected speech). Although each process influences successful reading independently, they also interact and enhance one another, with, for example, superior linguistic comprehension improving skilled word reading. The lexical quality hypothesis also theorizes the importance of word reading and linguistic comprehension to reading proficiency but underscores the contribution of vocabulary. Specifically, when students possess a well-developed understanding of a word’s spelling and its contextually-based meaning(s), they can read words proficiently. This proficiency in turn improves reading comprehension (Perfetti, 2017; Richter et al., 2013). Language, including understandings at the word level (i.e., vocabulary) and at the level of discourse, also plays a fundamental role in reading comprehension according to the Construction-Integration Model. According to this model, linguistic proficiency affects the formation of the textbase as readers construct their understanding of texts first at the surface (i.e., word) level and then at the level of connected text. Readers next integrate this textbase with relevant background knowledge or schema to form a situation model understanding of text (Kintsch, 1988; 2018).

For multilingual students learning to read in their second language theoretical models of reading comprehension (i.e., simple view of reading) appear to hold (Verhoeven & Van Leeuwe, 2012) suggesting the components associated with reading proficiency are the same regardless of language. However, linguistic comprehension in both L1 and L2 may affect the ways in which components develop and interact. One hypothesis describing these relationships is the Interdependence Hypothesis (Cummins, 1979) which states that skills developed in one language can transfer to the other. There is evidence to suggest components of proficient reading including phonological (Gorman, 2012) and morphological awareness (Verhoeven, 2017), and orthographic knowledge, especially when L1 and L2 share orthography (Bialystok et al., 2005; Deacon et al., 2013) do transfer across languages. Furthermore, improvements in one language can affect reading processes in the other. For example, Dixon and colleagues (2012) found linguistic proficiency in L1 could improve students’ L2 phonological awareness while results from Wang and colleagues (2009) suggested morphological awareness in either L1 or L2 can improve word reading in either language for older readers.

Vocabulary and reading comprehension, on the other hand, appear to not transfer (Manis et al., 2003; Sparks et al., 2008), suggesting L2 linguistic comprehension is an important predictor of L2 reading comprehension. This idea was argued in Cummins’ (1976) threshold hypothesis, which states that students must develop sufficient understanding of a language before those linguistic resources benefit their learning. When applied to reading, this means that L2 reading comprehension requires a threshold of L2 linguistic comprehension. Research bears out this hypothesis. For example, in a meta-analysis of L2 reading comprehension and the components associated with it, Jeon and Yamashita (2014) found L2 language knowledge—specifically vocabulary and grammar (i.e., tense, word order, subject-verb agreement)—had high correlation with L2 reading comprehension (r = .85 and .79, respectively) while L1 reading comprehension had a correlation r = .50. Additionally, while linguistic comprehension has the strongest impact on reading performance of MLs of varied reading abilities, native English speakers (who do not lack English proficiency) are most impacted by word reading skills (Babayiğit, 2014; Grant et al., 2011; Mancilla-Martinez & Lesaux, 2010). English vocabulary has been demonstrated to directly influence the reading comprehension of multilingual students who have developed adequate decoding skills (Proctor et al., 2005) and indirectly through both listening comprehension and word reading for students with word reading difficulties (Cho et al., 2019). These findings suggest that while word reading difficulties are associated with poorer reading comprehension for older readers, regardless of English proficiency, linguistic comprehension has a stronger influence on reading comprehension for MLs versus their English proficient peers.

1.2. Within-Group Differences: The Influence of Linguistic Proficiencies within the Population of English Learners

Each of the previous studies used a between-groups approach to compare the roles of word reading and linguistic proficiency on reading skills, including comprehension, between the populations of English learners and their non-ML peers. Accordingly, these studies may overlook important differences within the population of MLs (Montecillo Leider et al., 2013). As a population, MLs possess a range of proficiencies in L1 and L2. To date, limited research has explored the variation within the population of MLs, especially for readers beyond the primary grades (i.e., K-2; Baker et al., 2014; Richards-Tutor et al., 2016) thus leaving unanswered important questions about how variations in L1 and L2 linguistic proficiencies influence reading comprehension development, particularly in the context of intensive reading interventions.

This small body of work does make key contributions, however. Findings from cross-sectional research suggest reading performance for upper elementary-aged students differs as a function of biliteracy (i.e., students with demonstrated reading proficiency in both Spanish and English, Aguilar et al., 2020) and bilingualism (Mancilla-Martinez et al., 2020), including breadth of Spanish vocabulary (Proctor et al., 2006). Montecillo Leider and colleagues (2013), in a study of third-, fourth-, and fifth-grade MLs found that biliterate students (i.e., students with demonstrated reading proficiency in both Spanish and English, n = 56 out of 123 students in the study) outperformed their monoliterate peers (i.e., MLs who did not demonstrate Spanish reading proficiency) in English word recognition, Spanish word recognition, vocabulary, and syntax. Monoliterate students demonstrated significantly higher English vocabulary than biliterate peers.

Additionally, longitudinal research reveals protective factors across the spectrum of both L1 and L2 proficiencies that impact MLs’ reading growth over time. These factors include strong Spanish reading (Relyea & Amendum, 2020) or, in the absence of strong L1 reading higher English proficiency (Kieffer, 2008; 2011; Relyea & Amendum, 2020). Students with these protective factors at kindergarten entry demonstrated greater reading growth through grade four (Relyea & Amendum, 2020), grade five (Kieffer, 2008), and through middle school (Kieffer, 2011). Conversely, students with weaker Spanish reading skills or English proficiency at school entry made slower reading progress through grade four (Relyea & Amendum, 2020). As students progress through school grades, their L2 (English)with below-average word reading displayed the lowest reading achievement in fifth grade, remained on this trajectory through seventh grade, and continued to display the lowest reading performance at the end of seventh grade (Mancilla-Martinez et al., 2011).

Findings from these studies offer preliminary but compelling evidence that variations in L1 and L2 proficiencies influence reading growth and comprehension over time. Applying theoretical models of reading to these findings allows us to theorize about the interaction between linguistic comprehension in both L1 and L2 and decoding skills as it relates to reading comprehension. First, if MLs display English proficiency quite early into their school careers, they demonstrate reading growth equal to their monolingual English peers (Kieffer, 2008; 2011). In this scenario, English proficiency implies students have crossed the threshold of linguistic comprehension in L2 needed to use that language to comprehend what they read (Cummins, 1976). Remaining reading difficulties would likely be due to word reading (Hoover & Gough, 1990). Empirical research suggests just that: the primary mechanism behind reading difficulty for English proficient primary grade students appears to be word reading skills (e.g., Babayiğit, 2014).

Second, if MLs do not demonstrate English proficiency early in their schooling but have a protective factor such as foundational Spanish reading skills (e.g., decoding or phonological awareness; Relyea & Amendum, 2020), English word reading would not affect reading comprehension as much as linguistic comprehension (e.g., Proctor et al., 2005). This idea may be evidence of cross-linguistic transfer (Cummins, 1979) where the word reading skills developed in L1 are useful for decoding in L2.

Third, older students who display limited English proficiency, have, by definition limited English linguistic comprehension that, according to theoretical models of reading, would affect their reading in L2. When these older MLs also display reading difficulties, both linguistic comprehension and word reading appear to influence reading comprehension (Cho et al., 2019) and slow reading growth (Mancilla-Martinez et al., 2011). Word reading difficulties may be the product of disabilities (e.g., dyslexia) that affect MLs and their native English-speaking peers similarly but limited L2 vocabulary poses an additional risk to MLs. Limited vocabulary impedes MLs ability to form visual representations of unknown words and store them for later recall (Verhoeven, 1990). Difficulties forming and storing visual representations in turn reduce the lexical quality of students’ L2 vocabulary and their ability to understand words in the context of connected text (Perfetti, 2017). This process suggests students will have difficulty forming a textbase which then impedes text understanding by limiting the formation of a situation model (Kintsch 1988; 2018). As students progress through school grades and more content is contained in text (Carnegie Council on Advancing Adolescent Literacy, 2010), difficulty understanding text and acquiring the knowledge within them, further erodes their ability to form situation models (Kintsch 1994; Stanovich, 2009). Instructional intervention to remediate these challenges is needed.

1.3. Instruction and Intervention for Middle Grade English Learners

A small but growing body of research has tackled the question of how to intervene with the population of MLs in the middle grades experiencing reading difficulties. Studies within this corpus have taken a variety of approaches and have been implemented in multiple settings including social studies (Author), science (e.g., August et al., 2009; Lara-Alecio et al., 2012), and English language arts (e.g., Hall et al., 2020; Lesaux et al., 2010). Results suggest focusing on developing academic vocabulary through morphology instruction (Crosson & Moore, 2017), multisyllabic word reading strategies (Author), instruction in high-utility academic words (e.g., evidence, method, integrate; Lesaux et al., 2011) as well as a multicomponent approach including both vocabulary and comprehension instruction (Denton et al., 2008; Hall et al., 2017; Author; Wanzek & Roberts, 2012; Author) are effective methods of intervening with this population.

Additionally, within the body of intervention research for middle grade MLs, studies have begun to consider potential variations in student-level characteristics that predict differential response to intervention. For example, Author found heterogeneity among treatment effects such that the intervention was more effective for students with higher vocabulary but lower TOSREC (Wagner et al., 2010) scores at pretest. Lawrence et al. (2012) investigated whether English proficiency within language minority middle school students moderated their response to a vocabulary intervention, Word Generation. They found that English proficient students (i.e., students who achieved English proficiency prior to school entry) exposed to another language at home outperformed both their limited English-proficient peers and English proficient students from English-speaking homes after receiving the intervention. Hwang et al. (2015) confirmed and extended these results in findingthat redesignated fluent English proficient students (i.e., students who had been identified as limited English proficiency but currently demonstrated adequate proficiency) displayed reading performance comparable to English-only peers and English proficient students had superior performance to the English-only students after intervention. Taken together, these studies suggest English proficiency can moderate MLs’ response to intervention. However, we have not located a study that examined the heterogeneity of treatment effects due to variations in linguistic proficiencies in both students’ L1 and L2.

1.4. Present Study

Prior research has operationalized theoretical models of reading to both understand components associated with reading proficiency and difficulties for multilingual learners as well as to evaluate the effectiveness of instruction designed to mitigate weaknesses in components and improve reading outcomes. Specifically, intervention research has sought to impact reading comprehension of middle grade MLs with reading difficulties by addressing component skills at the word level (e.g., vocabulary and decoding that can facilitate the formation of a textbase, Kintsch, 1998; 2018) and at the level of discourse or connected text. However, there are open questions about how linguistic proficiencies in L1 and L2 and these component skills interact and develop in the context of an intensive, multi-component reading intervention that targets both code-based (i.e., word reading) and meaning-based (i.e., reading comprehension) skills for older students displaying a range of L1 and L2 proficiencies. In a meta-analysis investigating cognitive and linguistic factors that predict individual response to reading interventions for students in grades K-3, Stuebing and colleagues (2015) found that individual differences in linguistic proficiency measured prior to intervention were associated with small, but statistically significant differences in posttest outcomes. To the best of our knowledge, these questions have not been investigated in a sample of older MLs, among whom L1 and L2 linguistic proficiencies may function differently than was observed in research with monolingual English speakers. The present study directly addresses this gap in research.

1.4.1. Context: RISE Intervention

As described by Author, the [Intervention] program was a multicomponent reading intervention that targeted word reading skills, academic vocabulary, reading comprehension and fluency. All instruction could be characterized as explicit (e.g., Archer et al., 2010) as it specified opportunities for teachers to provide direct instruction in word reading and comprehension strategies, to model instructional strategies in action, to support students through guided and then independent practice as they attempted these strategies on their own in connected text.

Word reading instruction targeted advanced phonics including common vowel patterns in single and multisyllabic words, affixes, and irregular words that did not follow predictable patterns. Fluency instruction included teacher modeling of fluent reading (i.e., reading text with accuracy, appropriate rate, and prosody/expression) followed by students reading the text in pairs. Students were trained to give their partner corrective feedback. Finally, students received instruction in strategies useful for summarizing, deciphering the main idea, and making inferences as well as in use of a self-monitoring document. This document encouraged students to set goals before reading, to monitor progress during reading, and to evaluate performance after reading. Students were exposed to informational and narrative grade- appropriate texts (“stretch texts”).

The intervention was implemented daily for two academic years and included summer reading activities (i.e., novels with accompanying reading guides that included comprehension and vocabulary support) between years one and two. Students received more instruction on code- based word reading skills and fluency during the first semester of year one in an effort to increase their ability to access more complex text in later semesters. As the intervention went on, more instructional time was dedicated to developing academic vocabulary and reading comprehension. However, in March of Year 2, the COVID-19 pandemic disrupted in-person learning. While schools attempted to continue supporting students through videoconference or digital assignments, participating schools reported that engagement was quite limited. Although they did not collect systematic information about engagement in remote instruction at the beginning of the pandemic, school officials reported that many students in the study had not engaged in any remote learning from March to June and engagement was sporadic for those who had any engagement. When we spoke with parents, many reported that they lacked resources (e.g., no computer or Wi-Fi needed to access instruction) to engage in remote instruction. For these reasons, the research team did not attempt to continue the intervention remotely. In total, however, students completed 85% of the intended intervention sessions across the two years.

Additional details on fidelity of implementation (e.g., adherence, dosage, instructional quality, and comparison with the counterfactual) are provided in the primary study report (Author). Results of the study suggested significant treatment effects on students’ word reading skills when measured by a researcher-created proximal measure. Results drawn from standardized measures of word reading, fluency, vocabulary, and reading comprehension showed no significant treatment effect (Author). The participants from the Author study were the focus of the research questions addressed in this study.

2. Material and Methods

2.1. Participants

As previously reported by Author, this study took place at six middle schools in the Southwest United States. Four of the schools were urban and contained at least 91% students of Hispanic ethnicity. Two schools were suburban with 46–47% of the student body identifying as Hispanic. All schools received Title I funding and large proportions of students displayed limited English proficiency. This study was approved by the Institutional Review Board at the University of Houston and student participants and their legal guardians/parents gave written informed consent for participation.

The sample for this study focuses on 155 sixth- and seventh-grade students who completed all study measures specified below (see Measures) at pre- and post-test. These 155 students were drawn from a larger population of sixth- and seventh-grade students who met several criteria: 1) had failed the previous year’s state reading test; 2) were either currently identified as limited English proficiency or had been reclassified as English proficient within the previous two school years based on performance on state assessments of English listening, speaking, reading, and writing; 3) had a guardian identify Spanish as the language spoken in the home; and 4) were of Mexican or Central American origin. These criteria were established to align with a genetic study that was part of the overall project. Forty-six percent of students were in sixth grade, 37% identified as female, and the median age was 12.32. Sixteen percent received special education services and 80% received free/reduced price lunch (information on free/reduced price lunch status was not available for 19% of the population).

2.2. Measures: Linguistic Proficiencies

Student receptive and expressive vocabulary, listening comprehension, and expressive syntax and semantics were assessed in Spanish and English both before the start of the intervention and in the Spring of Year 2. All assessment but one (Sentence Assembly subtest from the Clinical Evaluation of Language Fundamentals-Fourth Edition, CELF-4 [Semel et al., 2006]) included English subtests and corresponding Spanish versions. Prior to administering assessments, a senior research team member recruited and trained a team of bilingual testers. Assessment team members received training in test procedures and demonstrated 100% reliability in a mock testing session. Assessors were blind to study condition and individually administered all measures. To ensure reliability, all assessments were scored in adherence to the scoring procedure detailed by each standardized test manual; each assessment was double-scored and - entered.

2.2.1. Woodcock-Johnson-III Subtests

Three WJ-III subtests were administered: WJ-III Memory for Sentences, WJ-III Picture Vocabulary, WJ-III Understanding Directions (Woodcock et al., 2007); corresponding subtests from the Woodcock-Muñoz Batería III were: Woodcock-Muñoz Batería III Memory for Sentences, Woodcock-Muñoz Batería III Picture Vocabulary, Woodcock-Muñoz Batería III Understanding Directions (Muñoz-Sandoval et al., 2007). The Picture Vocabulary subtests require students to provide a single word or phrase that corresponds to a pictured stimulus. Based on published psychometric validation, median internal consistency reliability (Cronbach’s alpha) for these instruments is over .85. Memory for Sentences subtests ask students to hear, remember, and repeat isolated words, phrases, and increasingly grammatically complex sentences. This subtest assesses students’ short-term memory, auditory memory, and listening comprehension (Schrank & Wendling, 2018). Median internal consistency reliability for these subtests is .89. Understanding Directions subtests measure students’ listening comprehension by asking them to listen to a series of instructions and follow directions by pointing to various objects in sequence. Median split-half reliability for these subtests is good (.77).

2.2.2. Receptive One-word Picture Vocabulary Test

The Receptive One-word Picture Vocabulary Test (ROPVT-4; Martin & Brownell, 2011). Sentence Assembly English requires students to match a spoken word that is an object, action, or concept with its visual counterpart. For the purposes of this study, each item was administered in Spanish first for all students. Items were then administered in English if the student responded incorrectly in Spanish. This procedure deviated from the procedure specified in the manual which requires test administrators to present items in the student’s dominant language first, followed by presentation in L2 at points of error. However, scores derived from the present study’s procedure can be considered reliable as correlation between this procedure and scores obtained from standard administrator was strong at r = .94. According to the ROPVT-4 manual, internal consistency reliability for this measure is .91.

2.2.3. Clinical Evaluation of Language Fundamentals-Fourth Edition Subtests

Sentence Assembly subtest from the CELF-4 (Semel et al., 2006) was the only subtest administered exclusively in English. It measures students’ ability to formulate correct sentences (i.e., valid syntax and semantics) in response to visually and verbally presented words. Test- retest Reliability for this measure is reported by CELF-4 authors to be greater than .85.

2.3. Measures: Reading Achievement

Several subtests evaluated students’ basic reading skills including identifying words, reading fluency, and comprehension. Much like the tests of linguistic proficiencies, reading achievement subtests were administered by trained assessment team members who were blind to study conditions. All assessments were administered in English and were scored in adherence to the scoring procedure detailed by each standardized test manual. Each assessment was double-scored and -entered.

Administration of reading measures at posttest was completed in May - June 2020. However, with the onset of the COVID-19 pandemic in March 2020 and subsequent switch to virtual instruction, planned in-person assessment had to be conducted remotely. Accordingly, testing was completed via telephone. A bilingual member of the assessment team conducted the evaluation over the phone with the help of the student’s family. Assessment stimuli were packaged in sealed envelopes and mailed to the student with instructions to leave test materials sealed until test administration began. Although this administration procedure is not typical, it is appropriate. Recent research (e.g., Magimairaj et al., 2022) suggests this procedure is reliable. Additionally, we report Cronbach’s Alpha and correlations between subtests at pre- and posttest as measures of reliability. As reported in Table 1, all alpha values are around or above 0.7, suggesting adequate internal consistency across timepoints and all correlations are significant.

Table 1.

Correlations And Reliabilities Among Language Measures at Pre and Posttest

	1	2	3	4	5	6
1. Gates-MacGinitie Pretest	1
2. Gates-MacGinitie Posttest	.696**	1
3. KTEA3-WRF Pretest	.440**	.421**	1
4. KTEA3-WRF Posttest	.485**	.496**	.857**	1
5. KTEA3-LWR Pretest	.373**	.427**	.727**	. 703**	1
6. KTEA3-LWR Pretest	.434**	.517**	.685**	.761**	.728**	1
	Pretest	Posttest
Gates-MacGinitie	0.728	0.815
KTEA3-WRF	0.735	0.684
KTEA3-LWR	0.707	0.742

2.3.1. KTEA-3 Letter & Word Recognition (KTEA-3 LWR)

Students’ ability to accurately recognize letters and read words was assessed through the KTEA-3 LWR (Kaufman & Kaufman, 2014). Students are presented first with letters and then with words of increasing difficulty. Split-half reliabilities for ages 13–15, as reported in the test manual, are high (.96-.97).

2.3.2. KTEA-3 Word Recognition Fluency (KTEA-3 WRF)

The KTEA-3 WRF (Kaufman & Kaufman, 2014) assesses students’ ability to read single words quickly and accurately. Students are presented with a list of words and are given two 15- second intervals to accurately read as many words as possible. Alternate form reliability is high (.89), according to the publishers’ test manual.

2.3.3. Gates-MacGinitie Reading Test Reading Comprehension Subtest

The Gates-MacGinitie Reading test comprehension subtest (GMRT-RC; MacGinite et al., 2000) is a timed test of reading comprehension. Students are given 35 minutes to read increasingly difficult expository and narrative text passages that range in length from 3 to 15 sentences. Following each passage, students answer three to six multiple-choice questions. The GMRT-RC manual reports internal consistency reliability ranges from .91 to .93 and alternate form reliability is .80-.87.

2.4. Data Analysis

We examined if the effectiveness of the [Intervention] differs depending on students’ proficiency in English and Spanish using latent variable moderation analysis. We fit confirmatory factor models to estimate proficiency scores in English and Spanish, basing proficiency on English-language and Spanish-language measures of expressive vocabulary, receptive vocabulary, expressive syntax/grammar, and receptive syntax/grammar following Macdonald et al. (2022). We then applied latent variable moderated SEM (Klein & Moosbrugger, 2000) to evaluate how students’ proficiency in English and Spanish at pretest moderated treatment’s effect on students’ reading outcomes at posttest. We contrasted the model with latent interactions of treatment and proficiency in English and Spanish (Model 2) to the model with equality constraints on the latent interactions terms (Model 1) to test the hypothesis that the more restricted model (equality constraints on the regression parameters of interest) fits the data as well as (i.e., statistically non-different) the less restricted model (interaction parameters freely estimated). This was done using the Wald test. Figure 1 displays a conceptual diagram of the full structural model. The latent moderations between treatment and English proficiency and the latent moderations between treatment and Spanish proficiency are indicated as a filled circle. We used the Johnson- Neyman technique (J-N; Johnson and Neyman, 1936) to follow up significant interactions involving continuous moderators. The J-N technique identifies values along the moderator’s range at which treatment’s effect was statistically significant to non-significant (Lin, 2020).

Figure 1.

Conceptual Structural Model with Latent Variable Interaction

All analyses were performed using Mplus 8.7 software (Muthén & Muthén, 1998–2022). Latent moderated structural models were estimated using the XWITH command. We used a full information maximum likelihood estimator to address non-normality and to obtain robust of standard errors. We examined model fit using the chi-square statistic (good fit is indicated by a nonsignificant value) in combination with a set of global fit indices according to standard rules of thumb (Hu & Bentler, 1999; Kline, 2011). Acceptable fit was indicated by CFI values above .90 and RMSEA and SRMT values under .08. We used CFI > .95 and RMSEA and SMRT values of < .06 as evidence of good model-to-data fit.

2.5. Transparency and Openness

All analyses were performed using Mplus 8.7 software (Muthén & Muthén, 1998–2022); data and analysis code used in this study are available upon request to the fourth author. While the analysis for the present study was not pre-registered, the sampling and data collection procedures were registered as part of the larger intervention study at clinicaltrials.gov [Study ID: NCT03695068].

3. Results

3.1. Descriptive Statistics

Prior to addressing the main research questions, we report distributions for the selected variables. Ten outlying values were identified based on skewness and kurtosis indices. These students had standard scores ranging from 1 to 39 on the Bateria-III Memory for Sentences and Bateria-III Picture Vocabulary. After removing these cases, skewness (ranged from −0.850 to −0.316) and kurtosis (ranged from 0.398 to 1.63) were acceptable across all variables. The final sample included 155 English learners with reading difficulties. The sample was 43.2% (n=67) female. A small proportion of students received special education services (15.5%). Means, standard deviations, and intercorrelations for reading measures are presented in Table 2.

Table 2.

Correlations And Descriptive Statistics Among Language Measures

	1	2	3	4	5	6	7	8	9
1. WJIII: Memory for Sentences	1
2. CELF4: Sentence Assembly	.34**	1
3. WJIII: Picture Vocabulary	.31**	0.14	1
4. WJIII: Understanding Directions	.29**	.18*	.22**	1
5. RECEPTIVE ONE-WORD - English	27**	.20*	. 41**	28**	1
6. BATERIA III: Memory for Sentences	.38**	.23**	−.17*	.11	.05	1
7. BATERIA III: Picture Vocabulary	.09	.09	−.10	−.10	.03	.60**	1
8. BATERIA III: Understanding
Directions	.10	.16	−.09	.12	.01	.46**	.49**	1
9. RECEPTIVE ONE-WORD - SPANISH	.17*	.19*	.01	.06	.18*	.40**	.41**	.25**	1
M	75.85	5.21	76.96	84.62	81.52	73.55	73.19	78.94	79.52
SD	12.33	2.32	9.27	9.27	11.22	16.84	16.47	13.51	13.12
N	155	155	155	154	155	155	154	155	155

3.2. Latent Moderated Structural Equation Models

The two-factor measurement model for students’ proficiency with English language and with Spanish language fit the data (χ² (24) = 45.75, p = .001, CFI=.93, RMSEA = .08, SRMR = .08), and all indicators loaded significantly on their respective factors (standardized factor loading ranged from .37 to 81). We estimated structural models with the interactions (Model 2) and with equality constraints on the latent interaction terms (Model 1) for all the three outcome variables of interest (Gates MacGinitie Reading, KTEA letter and word recognition, and KTEA Word Recognition Fluency). The results of the Wald-test (χ²(1) = 5.11, p = 0.02)) complemented the results of the parameter estimates presented in Table 4 which indicate that including interactions of treatment by English and Spanish proficiency in the model did not result in statistically significant improvements in the fit of the models, except for one. The Wald test for the model predicting KTEA letter and word recognition outcomes was significant, indicating that the treatment by proficiency in English interaction effect was significant (β = 3.36, SE = 1.49, p = 0.02) and suggesting that treatment’s effectiveness differed depending on students’ English proficiency scores.

Table 4.

Results of Latent Moderation Analysis

	Gates MacGinitie Reading Comprehension			KTEA Letter and Word Recognition			KTEA Word Recognition Fluency
Parameters	Estimate	SE	p-value	Estimate	SE	p-value	Estimate	SE	p-value
Intercept	83.10	0.91	0.00	85.27	1.05	0.00	84.76	0.79	0.00
Pretest	0.63	0.06	0.00	0.60	0.07	0.00	0.86	0.05	0.00
Treatment	0.68	1.15	0.55	1.66	1.50	0.27	0.66	1.06	0.53
Proficiency in English	1.83	1.12	0.10	−0.04	1.14	0.97	2.39	1.11	0.03
Proficiency in Spanish	2.57	0.80	0.00	1.33	0.92	0.15	0.47	0.81	0.56
English * Treatment	0.02	1.26	0.99	3.36	1.49	0.02	−2.29	1.41	0.11
Spanish * Treatment	−1.97	1.13	0.08	0.78	1.70	0.65	1.40	1.10	0.21

LOOP Plots in Figure 2 represent moderation by students’ proficiency in English and Spanish at pretest. The straight blue line represents the estimated moderator function, and the black curves represent confidence intervals. As the graph shows, the treatment effect had a greater positive value as students’ proficiency in English increased, meaning that treatment’s effect was more pronounced among students with higher proficiency scores in English. The J-N technique revealed that treatment was effective for a subset of students: those who scored 0.7 SD above average on English proficiency score. For students with lower proficiency scores in English treatment was not effective when controlling for pre-intervention reading scores and Spanish language scores.

Figure 2.

Visualization Of Proficiency in English as A Moderator of Intervention Effect on KTEA Letter-Word Recognition Score at Posttest (top). 95% Confidence Bands of The Simple Slope for Treatment Effect (bottom)

On Gates MacGinitie reading comprehension subtest and KTEA word recognition fluency, the interaction terms were not significant indicating that the effectiveness of the treatment was similar regardless of students’ proficiency in English and Spanish.

4. Discussion

This study used secondary data from a previously administered intervention study to understand the relationship between differences in linguistic proficiencies—specifically students’ English and Spanish vocabulary and listening comprehension— and response to intervention for middle school English learners with reading difficulties. In asking this question, we wished to uncover potential heterogeneity of treatment effects within the ML population due to variations in L1 and L2 proficiencies. Understanding differential response to intervention can in turn illuminate for whom interventions may be effective and therefore can shape our understanding of how English reading develops for this population and the design of future interventions targeting this at-risk population.

Our results suggested one significant moderation effect. Namely, treatment significantly improved the KTEA letter and word recognition scores for MLs who displayed above sample average English proficiency. Otherwise, language proficiencies did not moderate response to intervention. We hypothesize the lack of moderation is driven by students’ low levels of language proficiency in both English and Spanish. First, descriptive statistics presented in Table 2 indicate students had well-below average Spanish and English expressive vocabulary, Spanish and English receptive language, and Spanish listening comprehension. Students’ English listening comprehension and receptive vocabulary appeared to be relative strengths but were still below average. The low levels of proficiencies likely suppressed students’ reading performance, even in response to intervention. Second, students with relatively stronger English proficiency were able to improve letter and word recognition, as measured by the KTEA-3 LWR subtest, providing further support for the idea that linguistic proficiencies were a gatekeeper to students’ acquisition of foundational reading skills.

The question then becomes what the significance of L2 (English) proficiency on reading outcomes when both L1 and L2 are below average. For students learning to read in L2, L1 vocabulary and comprehension appear not to transfer across languages (Manis et al., 2003; Sparks et al., 2008), forcing students to rely on skills developed in L2. English vocabulary has been shown to play a key role in proficient reading (e.g., Proctor et al., 2005). When students—such as those in the present sample—experience limited L2 proficiency and display word reading difficulties, their word reading and linguistic comprehension interact with vocabulary to influence reading comprehension (Cho et al., 2019). English vocabulary affects the formation of lexical quality (e.g., a well-developed visual representation of a word and understanding of its meaning in context; Perfetti, 2017; Verhoeven, 1990). Because lexical quality affects students’ ability to build the textbase, limited lexical quality impedes students’ ability to read with understanding (Kintsch, 1998; 2018). Furthermore, L1 resources may transfer to L2 reading (Cummins, 1979). For example, L1 proficiency, which been shown to support development of L2 phonological awareness (Dixon et al., 2012). However, for students, like those in the present sample, with limited L1 proficiency, they cannot access this resource to assist with word reading. Taken together, students were likely able to capitalize upon relative strengths in English proficiency to develop word reading skills in response to intervention.

Theoretical models of reading suggest how word reading interacts with linguistic comprehension to enable students’ construction of the textbase (Kintsch, 1988; 2018) and produce reading comprehension (Hoover & Gough, 1990). Word reading is essential to lexical quality, which in turn facilitates understanding of words in context and overall reading comprehension. In this manner, word reading can facilitate language proficiency (Lervåg et al., 2018), which can in then be leveraged to produce reading comprehension (Kieffer, 2008; 2011).

This is speculative at present. The study from which these data were extracted was a randomized trial. However, we did not randomize within levels of the moderators, so results of the analyses presented in this paper are less internally valid that the main effects reported by Author. Additional research is needed to verify the apparent pattern of findings described here. Nonetheless, our results represent the best available evidence to date for addressing important questions.

Identifying linguistic mechanisms behind response to intervention for middle school MLs with reading difficulties permits greater understanding of how to support the reading progress of this population. Multicomponent interventions that build students’ academic vocabulary, provide continued instruction in decoding increasingly complex words, and support students’ effective use of reading comprehension strategies have been shown to improve reading proficiency for middle grade MLs (Denton et al., 2008; Author; Wanzek & Roberts, 2012; Author). Additionally, interventions conducted in Tier I content area classes including science, social studies and English language arts can improve academic vocabulary (August et al., 2009; Hall et al., 2020; Lara-Alecio et al., 2012; Lesaux et al., 2011). However, the small body of work that explored heterogeneity of treatment effects suggests students’ differentially respond to intervention. Specifically, students with higher English proficiency broadly and English vocabulary specifically have greater response to intervention (Hwang et al., 2015; Lawrence et al., 2012; Author). Findings from the present study partially support this idea. Additionally, the fact that all students in this sample displayed below average English proficiency and well below average Spanish proficiency suggests many students may lack the vocabulary needed to access and respond to intervention (MacDonald et al., 2022).

Given the scope of [Intervention] and students’ limited response to this intervention, it appears necessary to take additional measures to support developing academic vocabulary of MLs with reading difficulties in the middle grades. We argue this support might need to take the form of school-wide implementation of evidence-based instruction, such as that described by Baker et al. (2014). According to Baker and colleagues, this instruction is designed to promote MLs’ language development, especially the English vocabulary Proctor et al. (2005) found so vital to students’ reading comprehension outcomes. Preliminary work such as that described by Stevens and colleagues (2020) suggests aligning the Tier II intervention provided struggling middle grade readers with the instruction they receive in Tier I settings can improve students’ content knowledge and vocabulary. We believe this alignment would allow MLs to receive multiple exposures to evidence-based instruction that could promote English proficiency and increase the response to intervention for this vulnerable group.

4.1. Limitations and Directions for Future Research

There are several important limitations to note. First, in randomized studies, subgroup analyses estimate the effect of treatment across levels of a secondary factor—language proficiencies in our case. However, unless the secondary factor is randomized (i.e., unless the design is multifactorial), causal claims involving the moderator can be misleading in the presence of confounding variables (Wang & Ware, 2013). We control for potential confounding by English language proficiency and Spanish language proficiency. However, to the extent that other pre-existing unmeasured factors may be differently distributed across the groups and to the extent that those factors correlate with outcomes, bias may be present.

These unmeasured factors can include the background and educational experiences of the sample of students under investigation, which we do not know. Specifically, we do not know whether these students were born in the United States or when they arrived in the country, if not present at birth. We also do not know how long students had been identified as limited English proficiency and how many years of support they received to develop proficiency. Finally, we do not know the educational histories of these students are and whether, for instance, they had interrupted schooling, were exposed to reading instruction in Spanish, or progressed through school grades in the United States. Answers to these questions would provide clarity on how ‘typical’ the present sample is and would therefore help us understand how generalizable the results of this study are. Specifically, we found all students in this sample had below average L1 and L2 proficiencies. Is this result pervasive amongst the broader population of middle grade MLs with reading difficulties who also display limited English proficiency or who have been recently reclassified? Understanding the answer to this question could provide invaluable insight into how best to support students’ language development and reading outcomes. If, for example, the present sample does represent the broader population, it is likely that even the most intensive and extensive interventions conducted in Tier II settings will not close reading achievement gaps between MLs and their peers. Valuable school resources may therefore be better allocated to aligning instruction across Tier I content area classes to support MLs’ development of English proficiency.

A second limitation to note is that we do not know what Tier I instruction looked like for the present sample of students. It is possible that the present sample received evidence-based practices aligned with Baker et al. (2014) within their content area classes. Of course, if this were the case, the present sample of students demonstrated limited response to intervention despite receiving this instruction. However, we suspect students did not receive evidence-based instruction across Tier I content area classes. We base this suspicion on the documented structural challenges faced by high-poverty schools, such as those in the present sample, including inequitable funding and resource allocation that results in limited presence of evidence-based reading instruction (Slavin, 2020) and the presence of outdated instructional materials and less experienced teachers (Ingersoll et al., 2018) who may be poorly prepared and equipped to provide needed instruction to MLs (Goldenberg, 2013).

Future research should address these limitations. To begin, the field would benefit from studies that describe L1 proficiency. Results of these studies would provide greater understanding of how pervasive limited L1 proficiency is in the population of middle grade MLs and would therefore guide intervention decisions. Next, observation studies should document the typical instruction provided to middle grade MLs across their school day. These studies could provide a needed baseline of what instruction MLs are commonly exposed to while also illuminating places for future intervention. Findings could in turn inform the design of schoolwide instructional models as well as the systems of teacher professional learning needed to implement these models. Finally, studies should explore the impact of schoolwide instructional models that align instruction and intervention across tiers. Do MLs who have limited L1 and L2 proficiencies respond when exposed to evidence-based instruction in multiple classrooms across their school day? What contextual factors such as teacher professional development opportunities and instructional materials supports hinder or facilitate this alignment? Until studies investigate some of these questions, it is likely that middle grade MLs with limited proficiencies in L1 and L2—a population that may be more pervasive than commonly assumed—will continue to have weak response to interventions and poor reading performance.

5. Conclusion

This study examined the extent to which English and Spanish proficiency influenced students’ response to a reading intervention for middle school MLs with reading difficulties. Findings reveal that English and Spanish proficiencies yielded minimal impact on students’ response to intervention—perhaps because the overall language proficiency in both Spanish and English for this sample was—like their reading comprehension—quite low. Impact on response to intervention was observed only for students with relatively stronger English-language skills. These findings highlight once again the importance of linguistic proficiency to students’ reading achievement and suggest that without linguistic proficiency even an intensive and extensive intervention may not meet students’ reading needs. We interpret this suggestion as a rationale for more intensive language and literacy supports beyond the context of a tier 2 intervention and into tier 1 content area classes. With these supports it is possible students develop the linguistic proficiency needed to increase reading proficiency.

Table 3.

Descriptive Statistics for the Outcome Measures

		Pretest			Posttest
		n	M	SD	n	M	SD
Gates MacGinitie Comprehension	Control	75	80.52	9.55	72	82.76	10.40
	Treatment	79	80.75	9.94	77	84.04	9.75
KT3 Word Recognition Fluency	Control	75	80.24	12.44	72	83.17	13.31
	Treatment	80	83.29	12.15	77	87.09	11.85
KT3 Letter Word Recognition	Control	75	83.72	14.54	72	83.96	13.23
	Treatment	80	87.00	14.51	77	88.23	14.20

Educational Impact and Implications.

Understanding how to improve reading outcomes for middle school multilingual learners with limited English proficiency and reading difficulties continues to be challenging. Although the role of language in reading proficiency is well established (e.g., Hoover & Gough, 1990; Kintsch, 1998), there is limited information regarding the interaction of multiple linguistic proficiencies (i.e., proficiency in a students’ first and second languages) on response to intensive reading intervention for this group of students. This study found that a nearly two-year intervention was more effective for students with higher English proficiency scores than students with lower proficiency scores in English. These results suggest improving the reading outcomes for middle school MLs may require instruction in both language and literacy skills throughout their school day—both within and beyond the intervention setting.

• Proficiencies in Spanish and English were low for 6^th & 7^th graders with reading difficulties

• Only students with relatively strong English proficiency responded to intensive intervention

• Supporting language and literacy development requires instruction beyond intervention settings