The Influences of Multiple Informants’ Ratings of Inattention on Preschoolers’ Emergent Literacy Skills Growth

Abstract

There is strong evidence that inattention is a correlate of reading-related skills; however, less research has examined the unique and longitudinal relations between multiple informants’ ratings of inattention and the development of early reading skills across the preschool year. This study used latent growth curve analysis to examine whether ratings of inattention, completed by multiple informants, were unique predictors of emergent literacy development in preschoolers. Participants included 284 preschool children. ADHD-rating scales were completed by three different informants (i.e., classroom teachers, project teachers, and examiners) and measures of emergent literacy skills, a measure of working memory, and a measure of non-verbal cognitive ability were completed by the preschoolers. Each informant’s rating of inattention uniquely predicted children’s initial emergent literacy skills, but only the ratings of inattention made by project teachers were uniquely associated with growth in emergent literacy skills over the course of the preschool year.

Emergent literacy skills provide a crucial foundation for the development of conventional reading skills (Whitehurst & Lonigan, 1998). Inattention is a potentially malleable behavioral factor that consistently relates to academic achievement across development (Aaron, Joshi, Palmer, Smith, & Kirby, 2002; Walcott, Scheemaker, & Bielski, 2010; Willcutt, Pennington, & DeFries, 2000). Although the causal linkage between inattention and learning difficulties is not completely understood, many inattentive behaviors have been posited to impact achievement negatively by impeding learning-supportive behaviors (Ogg, Volpe, & Rogers, 2016). The early identification of academically impairing attention problems has the potential to inform intervention efforts and minimize children’s long-term academic difficulties. An improved understanding of which informants provide inattention ratings that are most relevant to academic development will help streamline the early identification process and guide intervention decisions. In the present study, we examined the relations between multiple informants’ ratings of inattention and children’s emergent literacy skill development over the preschool year.

Relations between Inattention and Emergent Literacy Development

Emergent literacy skills in preschool are predictive of reading skills across the early elementary years (Lonigan, Schatschneider, & Westberg, 2008). Multiple potential sources of influence on these important skills have been identified, including instructional factors (Piasta & Wagner, 2010), interest in literacy (Hume, Allan, & Lonigan, 2016), cognitive abilities (Rabiner & Coie, 2000), and behavioral factors like self-regulation (Purvis & Tannock, 2000). Inattention is a specific component of self-regulation that may interfere with the development of these skills (e.g., Welsh, Nix, Blair, Bierman, & Nelson, 2010). Deficits in attention can be represented by a variety of behavior problems, such as not completing tasks, being easily distracted, making careless mistakes, and being forgetful (American Psychiatric Association, 2013). In young children, attention is thought to be part of a group of distinct but related executive function skills that enable the resolution of conflicting information (Rueda, Posner, & Rothbart, 2004). These self-regulatory skills have a period of rapid development between the ages of 2 and 6 years (Espy, 2004).

A large body of research supports the concurrent links between inattention and literacy skills (e.g., Aaron et al., 2002; Purvis & Tannock, 2000; Willcutt et al., 2000). Inattentive behaviors, such as those indicative of Attention Deficit/Hyperactivity Disorder (ADHD), are associated with reading difficulties during childhood (Willcutt et al., 2000) and adolescence (Willcutt & Pennington, 2000). With older children, attentional processes relate to both decoding and comprehension skills (Arrington, Kulesz, Francis, Fletcher, & Barnes, 2014). Although less research has examined the relation between inattention and literacy in the preschool years, there is both concurrent and longitudinal evidence that measures of inattention are negatively associated with preschoolers’ emergent literacy skills (e.g., Lonigan et al., 1999; Sims & Lonigan, 2013; Walcott et al., 2010; Willcutt et al., 2007).

There also is a longitudinal link between inattention and the development of reading-related skills. For example, inattention in preschool predicts emergent literacy skills in kindergarten, after controlling for emergent literacy skills in preschool (Walcott et al., 2010). Inattention also predicts reading achievement across the elementary school years even after controlling for IQ, prior reading abilities, and behavior problems (Rabiner & Coie, 2000). Welsh et al. (2010) reported that direct measures of both inattention and working memory (WM) predicted emergent literacy skills in kindergarten when controlling for emergent literacy skills in preschool. These longitudinal studies identify inattention as a long-term predictor of risk. Given the significant relation between inattention and reading skills across development, research is needed to examine the early emergence of this link and to explore how different manifestations of inattention across contexts relate to growth in emergent literacy skills.

Attention is an important moderating factor in the effectiveness of reading interventions in school-age children, suggesting that deficits in attention may limit the ability of children to benefit from high-quality reading instruction (Dion et al., 2011). Dally (2006) pointed to the numerous instances in which the ability to “attend to” aspects of written and spoken language such as phonemes, graphemes, and orthographic elements (Share, 1995; Tunmer & Hoover, 1993) is cited as necessary to becoming a successful reader. Further, the inattentive characteristics and behaviors captured by the items on behavioral rating scales also may impact learning negatively. Distractibility, failure to listen, and failure to attend to details make it difficult to process and internalize the information presented during instruction. Organizational difficulties such as keeping track of materials may impact academic development by detracting from the time spent on active engagement in learning (Langberg, Epstein, Urbanowicz, Simon, & Graham, 2008). In sum, inattention may have an exponential negative effect on early reading development because it impedes learning both within and across academic lessons.

Measurement of Inattention

One factor to consider when examining the relation between inattention and other constructs is the multiple types of informants who may be asked to provide ratings of a child’s inattentive behaviors. One of the most common means of assessing attention problems in both clinical practice and research involves rating scales completed by teachers, parents and other observers. Teachers are considered a valuable source of information regarding children’s behavior problems because they have regular contact with many children, giving them a good perspective for making decisions regarding how a particular child compares to other same-age children (e.g., Evans, Allen, Moore, & Strauss, 2005). Furthermore, teachers observe the child in activities that vary in structure, many of which require some degree of engagement and attention. Therefore, teachers can provide information about the child’s inattentive behaviors as they occur during learning activities that require engagement. This is also true of individuals, such as interventionists and specialists, who work with children primarily in small-group or individual academically-focused contexts. Examiners, who observe the child in a time-limited but highly structured performance-based setting, have also been informants of interest (Bauermeister et al., 2005; Kerr, Lunkenheimer, & Olson, 2007). These informants provide the opportunity to obtain information regarding a child’s behavior in a relatively short time period.

Although different informants are asked to respond to similar or identical items to assess a child’s behaviors, responses across informants are typically only weakly-to-moderately correlated (Collett, Ohan, & Myers, 2003; De Los Reyes & Kazdin, 2004; Phillips & Lonigan, 2010; Sims & Lonigan, 2012). For example, Phillips and Lonigan reported poor-to-moderate agreement between ratings of inattention made by teachers, parents, and trained observers (inter-rater intra-class coefficients = .13 to .46). Whereas the modest agreement between informants could be attributable to compromised reliability and potential bias in informant ratings (e.g., Hartung et al., 2010), the lack of strong agreement between raters may also be the result of true differences in children’s behavior across settings (Mares et al., 2007). Stronger agreement has been reported for individuals with similar roles who observe the child in a similar context (e.g., teachers; Loughran, 2003). Therefore, informants may provide valid information about a child that varies depending on the context in which the rating is made.

Evidence consistently demonstrates that the inclusion of reports from multiple informants improves the validity of ADHD diagnoses (Power, et al. 1998). However, there is no consensus regarding how to best integrate information provided by different informants (e.g., Shemmassian & Lee, 2012). Because different informants may provide unique and meaningful information about a child’s symptom presentation, symptom severity, and risk of associated deficits (De Los Reyes & Kazdin, 2004), there is an increased interest in leveraging the differences between informants to guide diagnostic and treatment decisions. Some individuals may be better informants of certain symptom clusters. For example, research with school-age children and adolescents suggests that teachers may be better informants of inattentive symptoms than are parents (Martel, Schimmack, Nikolas, & Nigg, 2015). Ratings made by different informants may also point to different co-occurring problems. For example, in studies of adolescents, caregivers’ reports of inattention were associated with callous-unemotional traits whereas adolescents’ reports of inattention were associated with internalizing symptoms (Hogue, Dauber, Lichvar, & Spiewak, 2014). Thus, inattention noted by different informants may suggest different risks for associated difficulties.

Identifying Children At-Risk for Learning Difficulties

In the context of education, there is a movement to provide children who are struggling academically with more timely additional support. Response to Intervention (RTI; U.S. Department of Education, 2004) is an approach to instruction in which children who are not progressing at expected rates are given increasingly intensive and individualized instruction based on their response to interventions. When children fail to meet benchmarks, a typical first step is to provide additional educational support in a small-group context. Small groups allow the student to have more personalized instruction and closer monitoring from the educator. If children do not respond sufficiently to this level of support, they may be provided more individualized support in a one-on-one context. Thus, young children who are demonstrating emerging academic difficulties may work with several different types of educators who observe their behaviors in different contexts. Although it is traditionally lead classroom teachers who are asked to provide behavioral ratings, individuals who observe the child in other, sometimes more focused, educational contexts may offer unique insights regarding children’s learning- and performance-related behaviors. Determining which informants’ ratings best predict slow rates of skill development may streamline the process of identifying children who require expedited advancement to more intensive and individualized instruction.

In the current study, children were receiving small-group intervention as a part of a larger study examining the effectiveness of small-group reading and code-based instruction interventions. Although the type of shared reading intervention and code-related instructions that children received were based on random assignment, all interventions were generally designed to improve emergent literacy skills in at-risk preschoolers. As a part of the children’s participation in the study, they interacted with and had their behavior observed by classroom teachers, who interacted with children in a general classroom setting, project teachers, who delivered an early-literacy intervention, and examiners, who conducted the assessments of children’s early-literacy skills that were used in this study.

Current Study

Despite the large amount of research documenting the relations between inattention and academic achievement (Aaron et al., 2002; Arrington et al., 2014; Willcutt et al., 2000), less research has examined the extent to which ratings of inattention are predictive of skill acquisition during the school year, particularly in samples of young children who have been identified as at-risk for academic difficulties. Further, no studies have examined this link using ratings from multiple school-based informants whose ratings may be differentially predictive of children’s risk for poor response to educational interventions. Research examining this link is particularly important for groups of children who have been identified as at-risk for educational difficulties and may need different intensities of instruction to make substantial progress. This study examined how inattention, as rated by three different informants, predicted the development of emergent literacy skills across the preschool year. Analyses were conducted using latent growth curve analysis, which models change based on individual rates of growth and may be a superior method for examining change compared to more commonly used autoregressive techniques (e.g., Curran & Muthen, 1999; Stoolmiller, Duncan, Bank, & Patterson, 1993). Analyses were also conducted controlling children’s WM and non-verbal cognitive abilities, other factors that relate to literacy skill development (Rabiner & Coie, 2000; Welsh et al., 2010).

To examine the degree to which different informants’ ratings of inattention were jointly or uniquely predictive of emergent literacy skills, analyses were conducted first with each informant’s rating of inattention entered as the only measure of inattention and then with all three informants’ ratings of inattention entered simultaneously. Given the relations between inattention and reading-related skills (Kibby & Cohen, 2008; Welsh, et al., 2010), it was hypothesized that inattention at the beginning of the preschool year would predict both initial levels of emergent literacy skills and the rates at which emergent literacy skills grew across the year. Specifically, it was expected that higher initial inattention would be associated with both lower initial emergent literacy skills and less growth over time. Given that different informants may provide unique information regarding the child’s ability to regulate attention in different settings (e.g., Phillips & Lonigan, 2010; Rommelse et al., 2015), it was expected that each informant’s rating of inattention would uniquely predict both initial skills and growth in skills during the preschool year.

Method

Participants

As part of a larger study, children were recruited from 13 preschools that served primarily children from low-income families in north Florida. Parents’ informed consent was obtained for 365 3- to 5-year-old children (mean age = 54.07 months, SD = 5.91) who completed at least some of the initial assessment. After informed consent was obtained, and pretesting was completed, children were randomly assigned within school to one of five intervention conditions (i.e., four variations of component early literacy skill instruction and a no-additional-instruction control; see Authors, 2013). All children, regardless of assignment, continued to receive the classroom instruction provided by their preschools. In this study, we were interested in comparing behavior ratings across classroom teachers, project teachers (who delivered the small-group early-literacy interventions), and examiners (who conducted the early literacy skills assessments). Therefore, we included only those children who had been assigned to one of the four active intervention groups in the intervention study. The final sample of 284 children (mean age = 53.73 months, SD = 5.97) included 128 girls and 156 boys. Most children in the sample were African American (81%); 15% were white; and the remaining 4% were of other races/ethnicities (i.e., Latino/Hispanic, Asian American). As a group, these children were at substantial risk for later academic difficulties. Mean scores on standardized measures of oral language and nonverbal cognitive abilities collected prior to the intervention were below the 16th percentile (e.g., average standard score on the Expressive One-Word Picture Vocabulary Test-Revised was 82.9 [SD = 12.78], and the average scaled score on three subtests of the Stanford-Binet was 42.6 [SD = 4.38]).

Measures

Vocabulary.

Two standardized tests were used to measure different aspects of vocabulary. Expressive Vocabulary was measured using the Expressive One-Word Picture Vocabulary Test-Revised (EOWPVT-R; Gardner, 1990). The EOWPVT-R is reported to have an internal consistency of .94. Receptive Vocabulary was measured using the Basic Concepts subscale of the Clinical Evaluation of Language Fundamentals-Preschool (CELF-P; Wiig, Secord, & Semel, 1992). The internal consistency of the CELF-P Basic Concepts subscale ranges from .70 to .81 in this population. Raw scores from both measures were used to create a composite Vocabulary scale.

Phonological Awareness.

Children completed eight measures of phonological awareness, involving either blending (three tasks), elision (three tasks), or rhyming (two tasks). Each of the measures included at least two practice trials that included corrective feedback. Some tasks were free-response and others were multiple choice (e.g., pointing to a picture that represented the resulting word) On the elision tasks, children were required to remove one word from a compound word (e.g., “say ‘sunshine’ without ‘shine’”) or remove a syllable or phoneme from a word (e.g., say ‘bat’ without the /b/ sound). Coefficient alpha estimates for the free-response word elision task were .93, .94, and .94 at Time 1, Time 2, and Time 3, respectively. Estimates for the free-response syllable/phoneme task were .86, .85, and .80 at Time 1, Time 2, and Time 3, respectively. Estimates for the multiple-choice elision task were .52, .59, and .52 at Time 1, Time 2, and Time 3, respectively.

On the word blending task, children were required to combine two simple words into a compound word or to blend syllables or phonemes orally to form a word. For example, children were asked to say the resulting word when the words “foot” and “ball” were combined. Coefficient alpha estimates for the free response word blending task were .93, .92, and .94 at Time 1, Time 2, and Time 3, respectively. Estimates for free-response syllable/phoneme blending task were .83, .84, and .81 at Time 1, Time 2, and Time 3, respectively. Estimates for the multiple-choice blending task were .63, .64, and .64 at Time 1, Time 2, and Time 3, respectively.

Two tasks were used to measure rhyming. On the rhyme oddity task, children were shown three pictures and told to select the one picture that did not rhyme with (or “sound like”) the other two pictures. Coefficient alpha estimates for this task were .45, .66, and .79 at Time 1, Time 2, and Time 3, respectively. On the rhyme matching task, children were shown a picture on a small card and were told to indicate which of two additional pictures it rhymed with. Coefficient alpha estimates for this task were .71, .75, and .85 at Time 1, Time 2, and Time 3, respectively.

Letter Knowledge.

One task, with two components, Letter Names and Letter Sounds, were designed to measure children’s letter knowledge. Letter Names required children to name the letter of the alphabet presented to them on a flashcard. The 3-month test-retest reliability correlation for this measure was .85. Letter Sounds required the child to give the sound associated with the letter. The 3-month test-retest reliability for this measure was .56. Raw scores from these two components were aggregated to form a Letter Knowledge scale.

Working memory.

The listening span task is a measure of WM adapted from the reading span task developed by Daneman and Carpenter (1980). During this task, children were presented with simple sentences and had to answer whether the statement was true or false (e.g., Do dogs bark? Can cats fly?). Then, children had to say the last word of the questions after answering “yes” or “no” to all questions in a trial. As trials progressed, children were required to answer an increasing number of questions (ranging from one question to six questions) before being asked to recall the last words of the sentences in that trial. Children received 1 point for each last word they correctly identified. In other large sample of young children, this task has been shown to have acceptable internal consistency (.60) and split-half reliability (.60) and has demonstrated good validity as evidenced by large correlations with other measures of WM used with preschool-age children (rs = .47 – .50).

Nonverbal cognitive abilities measure.

The Pattern Analysis, Bead Memory, and Copying subtests from the Stanford-Binet (4^th Ed.; Thorndike, Hagen, & Sattler, 1986) were administered at pretest. Scores on these measure were aggregated to form a nonverbal cognitive abilities composite variable. Reliability estimates for these subtests are reported as .92, .87, and .87, respectively (Thorndike et al.).

Conners’ Teacher Rating Scale-Revised (CTRS; Conners, 1997; Gerhardstein et al., 2003).

The CTRS used in this study was a composite measure that included items from the 28-item CTRS revised short form (Conners, 1997) and all the non-overlapping items from the 1989 CTRS long form. This resulted in a 44-item scale. Three items were dropped that were determined to not be relevant to a preschool population (i.e., problems in spelling, reading, and arithmetic). The CTRS is the most commonly used tool to evaluate childhood behavior disorders (Conners, 1997) and has been shown to have good reliabilities and validity in a wide range of children, including preschoolers (Fantuzzo et al., 2001). This study utilized the CTRS scoring outlined by Gerhardstein et al., as this sample was similar in demographics. The CTRS comprised three factors designed to measure inattention, hyperactivity and oppositional defiant behavior. Only the 11 items that measured inattention were used in this analysis (see Gerhardstein et al.). Cronbach’s alpha values for the 11 items were .91 for teacher report, .94 for project teacher report, and .80 for examiner report.

Procedure

Procedures were approved by the University’s Institutional Review Boards and parental informed consent was obtained prior to the beginning of the study. Initial assessments of children’s early literacy and cognitive skills were conducted in August and September. Children completed a second wave of assessments in January, and a final wave in May (i.e., end of the preschool year). Following initial assessments, children participated in one of two types of small-group shared reading interventions until the end of the preschool year (i.e., dialogic reading or simple shared reading). During the fall (i.e., September/October to January), the shared-reading groups were conducted for 20 minutes daily, five days per week, and in the spring (i.e., January to May), the shared-reading groups were conducted for 10 minutes per day, five days per week. Beginning in January through the end of the preschool year, children in three of the four intervention groups also participated in small-group code-focused interventions that focused on letter knowledge, phonological awareness, or both. These code-focused intervention groups were conducted for 10 minutes per day five days per week.

Children’s classroom teachers completed the CTRS on the children approximately four weeks after initial assessments were completed. These teachers were employees of the school system or Head Start agency in which the study was conducted, the majority of which required degreed and certified teachers. Teachers typically spent 3–6 hours each day with the children. Project teachers completed the CTRS on the child roughly four weeks after initial assessments were completed. These teachers interacted with children in small-group (i.e., 3 to 5 children) pull-out interventions that lasted 10 to 20 minutes per day, five days per week. Project teachers were employed by the research team, had masters or bachelors degrees in psychology, education, or speech-language pathology and prior experience working with young children. Examiners completed the CTRS on the children at the time of initial testing in August or September. Examiners were employed by the research team and spent approximately 15 to 30 minutes completing the emergent literacy and cognitive ability measures with each child they rated. Examiners either had a degree or were working towards a degree in psychology, communications disorders, special education, or a related field.

Data Analytic Procedure

Descriptive statistics were computed and examined using SPSS version 18. Latent growth curve modeling was conducted using Mplus version 6 (Muthén & Muthén, 2010) to determine the degree to which classroom teacher, project teacher, and examiner ratings of inattention were predictive of initial emergent literacy scores and growth of emergent literacy scores over the three time points. The nonverbal cognitive abilities measure, the measure of WM, and children’s ages were included as control variables. All predictor variables were mean-centered. Unconditional models for the emergent literacy scores were first fit to the data; then, models including the predictor variables were fit to the data.

Because we were interested in the independent as well as the unique predictive associations between informants’ ratings and emergent literacy scores, models were first fit separately for each informant, and the final model included all informants together. Full Information Maximum Likelihood was used to account for missing data, and the Yuan-Bentler scaled chi-square (Y-B χ²) was used to account for nonnormality and nonindependence, and to make adjustments to correct standard errors (Yuan & Bentler, 2000). The Y-B χ² and several fit indices were used to assess overall model fit. A nonsignificant Y-B χ² indicates that the overall test of model fit was acceptable. A comparative fit index (CFI) greater than or equal to .95, square root mean residual (SRMR) below .08, and root mean square error of approximation (RMSEA) below .06 also indicate acceptable overall model fit (e.g., Hu & Bentler, 1999).

Results

Descriptive Statistics and Preliminary Analyses

Means, standard deviations, and correlations among variables are shown in Table 1. Correlations among variables were in the expected directions. Data were examined for normality. Significant skew was present for Time 2 letter knowledge and vocabulary as well as for WM, and classroom teacher-, project teacher-, and examiner-rated inattention. Missing data for emergent literacy variables ranged from 0 to 13%. Missing data for the predictor variables ranged from 1 to 21%. However, because the Y-B χ² is robust to nonnormality and missing data, no corrections were made to the data.

Table 1.

Correlations between Time 1, Time 2, and Time 3 Emergent Literacy Scores and Predictor Variables

Measures	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
1. Time 1 PA	--	.43*	.61*	.76*	.46*	.59*	.73*	.43*	.51*	.34*	.18*	.34*	−.30*	−.39*	−.21*
2. Time 1 LK		--	.41*	.43*	.87*	.39*	.45*	.74*	.33*	.26*	.13*	.10	−.22*	−.27*	−.20*
3. Time 1 Voc			--	.61*	.44*	.80*	.63*	.46*	.77*	.45*	.20*	.36*	−.26*	−.40*	−.28*
4. Time 2 PA				--	.50*	.60*	.79*	.43*	.53*	.35*	.14*	.36*	−.22*	−.38*	−.24*
5. Time 2 LK					--	.47*	.52*	.85*	.41*	.26*	.16*	.14*	−.26*	−.35*	−.22*
6. Time 2 Voc						--	.64*	.45*	.77*	.41*	.20*	.34*	−.27*	−.42*	−.26*
7. Time 3 PA							--	.55*	.67*	.43*	.09	.32*	−.32*	−.50*	−.25*
8. Time 3 LK								--	.46*	.24*	.17*	.19*	−.21*	−.43*	−.18*
9. Time 3 Voc									--	.40*	.10	.26*	−.23*	−.45*	−.24*
10. Age										--	−.38*	.27*	−.08	−.30*	−.17*
11. Nonverbal											--	.08	−.19*	−.14*	−.08
12. WM												--	−.09	−.22*	−.12
13. T Inattention													--	.27*	.18*
14. Prj Inattention														--	.24*
15. Ex Inattention															--
Mean	28.65	7.63	33.33	37.21	10.63	40.13	45.17	15.38	45.55	53.73	43.79	6.81	.63#	.50#	.17#
SD	12.65	10.02	12.72	15.61	11.38	13.53	17.17	11.79	13.88	5.97	6.16	5.99	.64	.68	.29

Note. PA = Phonological Awareness. LK = Letter Knowledge. Voc = Vocabulary. Nonverbal = Composite of nonverbal abilities from the Stanford Binet. WM = WM. T Inattention = Teacher-rated inattention. Prj Inattention = Project teacher-rated inattention. Ex Inattention = Examiner-rated inattention.

^#Values are presented as average values.

^*p ≤ .05.

Unconditional Growth Curve Models

Unconditional models for phonological awareness, letter knowledge, and vocabulary were examined, with time centered at initial status, to determine whether these models fit the data well. A nonsignificant negative residual variance for Time 1 letter knowledge in the letter knowledge model was fixed to zero. Children were nested within forty classrooms, which could be problematic because of potential underestimation of the standard errors and the subsequent influence on Type I error (Hox, 1998). The design effect, which takes into account the intraclass correlation (ICC; percentage of variance accounted for at the classroom level) as well as the average number of children nested within schools can be used to determine whether it is necessary to model classroom effects. Design effects greater than 2.0 suggest potential nontrivial variance. All design effects were well below this threshold. As such, we did not model classroom-level variance in this study.

Goodness-of-fit statistics for the unconditional models are presented in Table 2. The phonological awareness and vocabulary models provided excellent fits to the data as demonstrated by nonsignificant Y-B χ² values and all fit indices within suggested cutoffs. The letter-knowledge model provided adequate fit to the data. The Y-B χ² value was significant; however, the CFI, SRMR, and RMSEA were within suggested cutoffs, indicating that the conditional models could be examined.

Table 2.

Model Fit Statistics for Unconditional and Conditional Models of Emergent Literacy

Model	Y-B χ2	df	p value	CFI	SRMR	RMSEA
Unconditional Models
Phonological Awareness	.17	1	.68	1.00	.01	.00
Letter Knowledge	9.21	2	.01	.98	.03	.11
Vocabulary	1.08	1	.30	1.00	.01	.02
Conditional Models
Teacher Models
Phonological Awareness	4.90	5	.43	1.00	.01	.00
Letter Knowledge	11.01	6	.09	.99	.02	.05
Vocabulary	4.67	5	.46	1.00	.01	.00
Project Teacher Models
Phonological Awareness	3.29	5	.66	1.00	.01	.00
Letter Knowledge	10.23	6	.12	.99	.02	.05
Vocabulary	4.67	5	.46	1.00	.01	.00
Examiner Models
Phonological Awareness	1.51	5	.91	1.00	.01	.00
Letter Knowledge	11.52	6	.07	.99	.02	.06
Vocabulary	4.56	5	.47	1.00	.01	.00
All Raters Models
Phonological Awareness	7.41	7	.39	1.00	.01	.01
Letter Knowledge	12.63	8	.13	.99	.02	.05
Vocabulary	5.00	7	.66	1.00	.01	.00

Note. N = 285 for Phonological Awareness and Letter knowledge and N = 283 for Vocabulary models. Y-B χ²= Satorra-Bentler Scaled Chi-Square. CFI = Comparative fit index. SRMR = Standardized root mean square residual. RMSEA = Root mean square error of approximation.

Conditional Growth Curve Models

Phonological awareness.

Conditional models for phonological awareness, letter knowledge, and vocabulary were examined for each rater independently as well as across all raters. Model fit indices revealed that all models demonstrated excellent fit to the data (see Table 2). Results of models predicting phonological awareness across classroom teacher, project teacher, and examiner as raters of inattention, independently (i.e., independent-rater models), and entered simultaneously in one model (i.e., simultaneous-raters model) are shown in Table 3. Controlling for age, WM, and nonverbal cognitive abilities, classroom teacher (B = −3.85, p ≤ .001) and project teacher (B = −4.07, p ≤ .001) ratings of inattention were negatively associated with initial phonological awareness skills in the independent-rater models. Similarly, classroom teacher (B = −3.00, p ≤ .01) and project teacher (B = −3.23, p ≤ .01) ratings of inattention were negatively associated with initial phonological awareness skills in the simultaneous-rater models. Project teacher ratings of inattention were negatively associated with phonological awareness skill growth in both the independent-rater (B = −2.03, p ≤ .001) and the simultaneous-raters (B = 1.73, p ≤ .01) models. This was the only significant predictor of growth in phonological awareness.

Table 3.

Model Parameter Estimates for the Effects of Inattention Ratings on Phonological Awareness

	Classroom Teacher		Project Teacher		Examiners		All Raters
	Parameters	SE	Parameters	SE	Parameters	SE	Parameters	SE
Intercept	28.56***	.63	28.56***	.63	28.57***	.64	28.57***	.62
VarianceI	96.11***	13.00	93.72***	13.06	95.35***	13.29	93.18***	12.62
Slope	7.89***	.36	7.89***	.35	7.92***	.36	7.87***	.35
VarianceS	21.95***	6.66	20.39**	6.57	18.34**	6.80	21.71***	6.42
Covariance	−3.44	6.66	−4.51	6.40	.49	6.66	−7.47	6.52
Intercept Predictors
Age	.79***	.11	.68***	.11	.81***	.11	.63***	.11
Nonverbal	.79***	.16	.76***	.15	.88***	.15	.66***	.15
WM	.42***	.13	.38**	.13	.42**	.13	.37**	.13
Teacher IA	−3.85***	1.03					−3.00**	1.00
Project IA			−4.07***	.93			−3.23**	.92
Examiner IA					−4.12	2.17	−2.00	2.07
Slope Predictors
Age	.24***	.07	.17**	.08	.23***	.07	.16*	.08
Nonverbal	−.02	.10	−.07	.09	−.01	.10	−.10	.09
WM	.03	.07	.01	.07	.03	.07	.01	.07
Teacher IA	−1.15	.59					−.71	.60
Project IA			−2.03***	.57			−1.73**	.62
Examiner IA					−2.69	1.45	−2.09	1.43

Note. Intercept is centered at Time 1. All predictor variables centered at the mean. Parameters = Unstandardized Parameters. Variance^I = Intercept Variance. Variance^S = Slope Variance. Nonverbal = Composite of nonverbal abilities from the Stanford Binet. WM = WM measure. IA = Inattention rating.

^*p ≤ .05,

^**p ≤ .01,

^***p ≤ .001.

Letter knowledge.

Results of independent-rater and simultaneous-raters growth models predicting letter knowledge across classroom teacher, project teacher, and examiner as raters of inattention are shown in Table 4. Controlling for age, WM, and nonverbal cognitive abilities, classroom teacher (B = −2.32, p ≤ .01), project teacher (B = −2.48, p ≤ .001), and examiner (B = −4.11, p ≤ .01) ratings of inattention were all negatively associated with initial letter knowledge in the independent-rater models. Similarly, classroom teacher (B = −1.73, p ≤ .05), project teacher (B = −1.86, p ≤ .01), and examiner (B = −3.01, p ≤ .05) ratings of inattention were all negatively associated with initial letter knowledge in the simultaneous-raters model. Project teacher ratings of inattention were negatively associated with growth in letter knowledge in the independent-rater model (B = −1.60, p ≤ .001) and the simultaneous-raters (B = −1.55, p ≤ .001) models.

Table 4.

Model Parameter Estimates for the Effects of Inattention Ratings on Letter Knowledge

	Classroom Teacher		Project Teacher		Examiners		All Raters
	Parameters	SE	Parameters	SE	Parameters	SE	Parameters	SE
Intercept	7.61***	.55	7.66***	.55	7.60***	.55	7.66***	.54
VarianceI	85.14***	7.27	84.70***	7.07	85.88***	7.22	82.75***	6.96
Slope	3.40***	.24	3.42***	.24	3.40***	.24	3.42***	.24
VarianceS	14.02***	1.94	13.14***	1.81	14.10***	1.94	13.14***	1.81
Covariance	−7.95***	2.16	−9.22***	2.27	−7.67***	2.13	−9.15***	2.25
Intercept Predictors
Age	.58***	.09	.51***	.10	.57***	.10	.47***	.10
Nonverbal	.53***	.16	.51**	.16	.57***	.16	.44**	.17
WM	−.05	.11	−.07	.11	−.05	.11	−.07	.11
Teacher IA	−2.32**	.89					−1.73*	.86
Project IA			−2.48***	.78			−1.86**	.78
Examiner IA					−4.11**	1.39	−3.01*	1.40
Slope Predictors
Age	.04	.05	−.02	.05	.05	.04	−.02	.05
Nonverbal	.06	.06	.001	.06	.07	.06	.003	.06
WM	.10*	.04	.08	.04	.10*	.05	.08	.04
Teacher IA	−.42	.42					−.08	.41
Project IA			−1.60***	.37			−1.55***	.40
Examiner IA					−.22	.82	.22	.88

Note. Intercept is centered at Time 1. All predictor variables centered at the mean. Parameters = Unstandardized Parameters. Variance^I = Intercept Variance. Variance^S = Slope Variance. Nonverbal = Composite of nonverbal abilities from the Stanford Binet. WM = WM measure. IA = Inattention rating.

^*p ≤ .05,

^**p ≤ .01,

^***p ≤ .001.

Vocabulary.

Results of independent-rater and simultaneous-raters growth models predicting vocabulary across classroom teacher, project teacher, and examiner as raters of inattention are shown in Table 5. Controlling for age, WM, and nonverbal cognitive abilities, classroom teachers (B = −3.06, p ≤ .01), project teachers (B = −3.39, p ≤ .001), and examiners (B = −6.33, p ≤ .001) ratings of inattention were all negatively associated with initial vocabulary across the independent-rater models. Similarly, classroom teachers (B = −2.20, p ≤ .05), project teachers (B = −2.49, p ≤ .05), and examiners (B = −4.85, p ≤ .01) ratings of inattention were all negatively associated with initial vocabulary across the simultaneous-rater models. Project teacher ratings of inattention were negatively associated with growth in vocabulary across the independent-rater (B = −1.11, p ≤ .01) and simultaneous-raters (B = −1.01, p ≤ .05) models.

Table 5.

Model Parameter Estimates for the Effects of Inattention Ratings on Vocabulary

	Classroom Teacher		Project Teacher		Examiners		All Raters
	Parameters	SE	Parameters	SE	Parameters	SE	Parameters	SE
Intercept	33.25***	.57	33.30***	.57	33.26***	.57	33.29***	.56
VarianceI	67.52***	10.41	66.95***	10.79	68.10***	10.48	62.97***	10.28
Slope	6.08***	.28	6.09***	.28	6.08***	.28	6.08***	.28
VarianceS	1.87	4.35	2.00	4.26	2.18	4.32	1.79	4.15
Covariance	4.89	4.86	3.47	4.83	5.01	4.82	3.42	4.74
Intercept Predictors
Age	1.15***	.10	1.05 ***	.11	1.13***	.09	1.00***	.11
Nonverbal	1.08***	.17	1.05***	.17	1.12***	.16	.95***	.17
WM	.36***	.11	.33**	.11	.36***	.11	.33***	.10
Teacher IA	−3.06**	1.06					−2.20*	1.00
Project IA			−3.39***	1.04			−2.49*	.99
Examiner IA					−6.33***	1.96	−4.85**	1.85
Slope Predictors
Age	−.02	.06	−.06	.06	−.01	.06	−.06	.06
Nonverbal	−.12	.07	−.15*	.07	−.11	.07	−.16*	.07
WM	−.05	.05	−.06	.05	−.05	.05	−.06	.05
Teacher IA	−.58	.44					−.34	.45
Project IA			−1.11**	.37			−1.01*	.40
Examiner IA					−.55	1.10	−.22	1.10

Note. Intercept is centered at Time 1. All predictor variables centered at the mean. Parameters = Unstandardized Parameters. Variance^I = Intercept Variance. Variance^S = Slope Variance. Nonverbal = Composite of nonverbal abilities from the Stanford Binet. WM = WM measure. IA = Inattention rating.

^*p ≤ .05,

^**p ≤ .01,

^***p ≤ .001.

Notably, in a multi-group analysis examining whether findings differed across boys and girls, results of a Wald test (Wald χ² value = 25.73, p = .012), indicated that the prediction of intercept and slope for vocabulary skills differed for boys and girls. In general, results suggested that project teachers’ ratings of inattention significantly predicted vocabulary development in girls, but not boys. For the models examining the development of phonological awareness (Wald χ² value = 7.73, p = .806) and letter knowledge (Wald χ² value = 16.12, p = .186), there were not overall differences in the prediction of intercept and slope for boys and girls.

Models Examining the Impact of Intervention Status

Because children had received one or more targeted interventions for vocabulary, phonological awareness, or letter knowledge, additional analyses were conducted to examine the potential impact of intervention condition on the results. Analyses in which intervention status variables (i.e., a dummy-coded variable for each intervention reflecting whether or not the child received the intervention) were included as control variables, did not substantially alter the results (see Table 6). Analyses also were conducted for each outcome domain that included intervention status (i.e., receipt or nonreceipt) for that domain and interaction terms representing children’s intervention status and the ratings of inattention by examiners, classroom teachers, and project teachers. In these analyses, the interaction terms were not significant predictors of the slopes for phonological awareness (ps > .81), letter knowledge (ps > .15), or vocabulary (ps > .11), and these interaction terms were not significant predictors of intercepts for letter knowledge (ps > .36) or vocabulary (ps > .40). The interaction between receipt of the phonological awareness intervention and examiners’ ratings of inattention was a significant predictor of the intercept for phonological awareness (b = 11.36; p = .05); however, inclusion of this term in the model did not alter the pattern or significance of the results for the other predictors of intercept or slope.

Table 6.

Model Parameter Estimates for the Effects of Inattention Ratings on Early Literacy Skills Including Intervention Status

	Phonological Awareness		Letter Knowledge		Vocabulary
	Parameters	SE	Parameters	SE	Parameters	SE
Intercept	24.47***	3.28	4.11	2.70	34.37***	2.85
VarianceI	94.07***	12.54	81.97***	7.05	61.46***	10.15
Slope	5.70*	1.98	1.13	1.29	6.55***	1.40
VarianceS	23.12***	6.31	12.83***	1.80	.51	4.16
Covariance	−9.47	6.46	−9.24***	2.20	4.10	4.80
Intercept Predictors
Lang Intv	1.35	1.80	.60	1.53	−1.10	1.56
PA Intv	3.43*	1.60	2.64	1.46	1.42	1.42
Letter Intv	.78	1.76	1.55	1.45	−1.63	1.59
Age	.60***	.12	.45***	.10	1.00***	.11
Nonverbal	.64***	.15	.42*	.16	.95***	.16
WM	.38*	.13	−.07	.11	.32**	.10
Teacher IA	−3.35*	1.03	−1.77*	.90	−2.44*	1.02
Project IA	−3.23***	.93	−1.92*	.79	−2.30*	.99
Examiner IA	−2.22	2.09	−3.21*	1.42	−4.83**	1.77
Slope Predictors
Lang Intv	.58	1.08	1.42*	.68	1.18	.79
PA Intv	1.56	1.01	.56	.68	−.59	.73
Letter Intv	.81	1.05	1.12	.70	−1.14	.75
Age	.15	.08	−.03	.05	−.06	.06
Nonverbal	−.11	.09	.003	.06	−.14*	.06
WM	.02	.07	.09	.04	−.06	.05
Teacher IA	−.70	.62	−.06	.42	−.16	.46
Project IA	−1.75*	.62	−1.63***	.40	−1.10*	.38
Examiner IA	−2.15	1.43	.02	.85	−.22	1.10

Note. Intercept is centered at Time 1. All predictor variables centered at the mean. Parameters = Unstandardized Parameters. Variance^I = Intercept Variance. Variance^S = Slope Variance. Nonverbal = Composite of nonverbal abilities from the Stanford Binet. Lang Intv = Receiving language intervention. PA Intv = Receiving phonological awareness intervention. Letter Intv = Receiving letter knowledge intervention. WM = Working memory measure. IA = Inattention rating.

^*p ≤ .05,

^**p ≤ .01,

^***p ≤ .001.

Discussion

The primary goal of this study was to examine the associations between inattention, as rated by three different informants, and the development of emergent literacy skills in preschool children. The results demonstrated that ratings of inattention made by all three informants were uniquely related to children’s initial skill levels. However, only ratings of inattention made by project teachers, who interacted with children in small-group instructional settings, were predictive of growth in emergent literacy skills. These results suggest that contextual factors, such as the attention demands and student-to-teacher ratios in a given setting, impacted the abilities of raters who interacted with children in each of these contexts to observe inattentive behaviors that were associated with the development of early reading skills. These results have implications for the identification and treatment of children who are at risk for later problems in reading.

Inattention and Emergent Literacy

The finding that inattention was uniquely associated with children’s initial skill levels when controlling for several other important factors including age, WM, and non-verbal cognitive ability implicates attention as a self-regulatory mechanism that may contribute to early reading skills. This was the first study to examine the unique relations between inattention and early reading skills using multiple informants’ ratings of inattention and using latent growth curve analysis, a technique that is better able to model individual developmental change over time than are the auto-regressive techniques used in the majority of research on this topic (Stoolmiller et al., 1993). The findings of this study are consistent with research showing that inattention is both concurrently (e.g., Aaron et al., 2002; Willcutt et al., 2000) and longitudinally (e.g., Dally, 2006; Rabiner & Coie, 2000) associated with reading-related skills.

Inattention as rated by small-group teachers appears to be associated both with performance on measures of emergent literacy skills and with the attainment of these skills over time. This underscores the importance of early intervention to prevent the compounding impact that attention difficulties may have on learning. For the children in this study, the relation between inattention and skill development was apparent even in the context of receiving small-group instruction to supplement typical classroom instruction. As such, children who are observed specifically by small-group instructors as having attentional difficulties may be good candidates for receiving individualized one-on-one interventions earlier in the school year. The results of this study suggest that without more intensive intervention, these children will continue to fall behind their peers in terms of skills development.

Inattention across Contexts

Ratings of inattention by individuals who observed children in different contexts were not equally associated with emergent literacy skill development, suggesting that the context in which attentional difficulties are observed is as important as if it is observed. The modest correlations between different informants’ ratings of inattention were consistent with prior research showing that raters typically do not have strong agreement when rating the same behavior across varying contexts (e.g., De Los Reyes, Henry, Tolan, & Wakschlag, 2009; Dirks et al., 2012; Loughran, 2003; Mares et al., 2007; Phillips & Lonigan, 2010). The finding that each of the informants’ ratings of children’s inattention was uniquely associated with children’s emergent literacy skills indicates that the relatively weak agreement between informants was not the result of error or poor reliability of these informants’ reports. Rather, this finding echoes the theory that discrepancies between informant ratings exist because children exhibit situation-specific behavior (Dirks et al., 2012; De Los Reyes, Thomas, Goodman, & Kundey, 2013). That is, the different circumstances in which the classroom teachers, project teachers, and examiners observed the children affected how the children behaved and the nature of the behaviors that were observed such that each informant provided unique and meaningful evaluations of children’s attention.

There are three primary contextual factors that may have influenced the ratings obtained from informants and impacted their relations to emergent literacy skills. These contextual factors include the demands for attention in a given situation, the ability to observe signs of inattention, and the ability to help children regulate their attention. Examiners, who had one-on-one interactions with the children, had the best opportunity both to observe and to help regulate children’s attention. Although the demands for attention in this context were high (i.e., children had to attend and respond to test items), the examiner had the ability to monitor and redirect the child continuously. Therefore, it is likely that children rated by examiners as high in inattention were those children who both displayed a high level of off-task and inattentive behaviors and who were less responsive to examiners’ redirections, and children rated as moderately inattentive were likely those children who either displayed a high level of off-task and inattentive behaviors but were responsive to redirection or who displayed only moderate levels of off-task and inattentive behaviors. Many of the children rated as low in inattention were likely those who displayed a high level of attention across contexts, both with and without close adult guidance. However, some children rated as low in inattention may have displayed a higher level of inattention in other contexts (e.g., in the classroom; in small-group activities) but displayed a low level of off-task and inattentive behaviors in the context of close monitoring by the examiner. Therefore, the attentive behaviors displayed by these children in the examination context may not have accurately reflected their typical behaviors.

Classroom teachers, who interacted with children primarily in the context of whole-group classroom activities, had a relatively poor opportunity to observe and help regulate individual children’s attention, given that they were simultaneously responsible for the entire classroom of children. Moreover, the demands of the situation were likely variable and primarily low, given that attention was only required for some activities (e.g., listening to instructions, group shared reading), and all classrooms in the study used traditional early childhood curricula that involved little explicit instruction. Because the ability of classroom teachers to redirect individual children was low (i.e., a low chance of providing consistent and frequent redirection), it is likely that children rated as high or moderate in inattention were both those children whose self-regulation was very poor across contexts and those children whose regulation of attention was poor in whole-group context but may have improved in the more structured settings in which examiners and project teachers observed children.

In contrast to the classroom teachers, project teachers interacted with children in a situation in which the demands for attention were high, and the ability both to observe and to regulate children’s attention was at least moderate. In the small-group instructional activity setting, the demand for attention was constant because the project teacher was focused on teaching children specific new skills. Children had to attend and respond to prompts directed toward them, and they had to follow the interactions with other children to be able to respond appropriately when prompted. In a group of three to five children, the ability to notice inattention was likely higher than it was in a classroom setting. However, project teachers had a moderately high ability to help children regulate their attention. Consequently, it is likely that children rated as high in inattention were those children who both displayed a high level of off-task and inattentive behaviors and who were less responsive to project teachers’ attempts to redirect and focus them, and children rated as moderately inattentive were likely those children who either displayed a high level of off-task and inattentive behaviors but were responsive to redirection or who displayed only moderate levels of off-task and inattentive behaviors. Project teachers also worked with children solely in an educationally-focused context, specifically focused on the development of reading-related skills. Therefore, projects teachers’ ratings of inattention may partially reflect their projections of the students’ abilities to succeed in acquiring skills.

It is possible that unmeasured rater-level characteristics contributed to the observed differences between the ratings provided by each of the informants. However, it is notable that there was substantial overlap in the background, education, and experiences of the classroom and project teachers. That is, the majority of sites for this study were staffed by degreed and certified teachers. The project teachers also were individuals with bachelors or masters degrees in education or related fields, with experiences working with preschool children in educational contexts. Therefore, it is unlikely that differences in education or experience contributed to the observed differences between project and classroom teachers’ ratings. Both classroom teachers and project teachers differed from examiners on these dimensions, because most examiners were undergraduate or graduate students in psychology or related fields.

The different pattern of associations between each informant’s ratings and emergent literacy skills has implications for both the identification and treatment of children experiencing difficulties in the classroom. Attention problems that occur in focused, small-group instructional settings may reflect the manifestation of attention problems that are most likely to have a long-term impact on a child’s academic development. Thus, it is likely that ratings from teachers who observe children in this context provide the most accurate appraisal of attention problems that may interfere with learning. These findings also suggest that interventions targeting general classroom behavior may not have a specific effect on the particular aspects of inattention that impede long-term academic development. Rather, interventions designed to improve academic difficulties associated with inattention should target children’s capacity to self-monitor and self-regulate during learning-focused activities.

Limitations and Future Directions

Although this study was novel in that it examined the link between different informants’ ratings of inattention and the development of emergent literacy skills using growth models and controlling for several important factors, there were a few limitations. Because this study focused on children at risk of academic difficulties and children from low-income backgrounds were overrepresented, the findings may not generalize to a sample of children with more representative levels of emergent literacy and self-regulation. We were also limited by the use of only informant reports to assess inattention. Additional research also should be conducted using other methods of assessing inattention (e.g. direct-assessment) to better understand how the different manifestations of inattention measured by specific assessments or informants relate to literacy development. In this study, we did not measure the specific instructional activities children received as part of their typical classroom experiences. It is possible that children’s attentional skills moderated the effectiveness of classroom instruction.

Summary

This study adds to the growing literature exploring linkages between components of self-regulation and the development of academic skills. This study is unique in that it examined the independent longitudinal associations between different informants’ ratings of inattention and emergent literacy development at three timepoints across the preschool year and because it did so using latent growth curve, rather than autoregressive, analytical techniques. The findings of this study underscore the importance of self-regulation, particularly attention, to the development of early reading-related skills in young children who are at risk for learning difficulties. This study points to inattention that is displayed and observed in specific types of contexts, such as small-group or focused instructional situations, as being particularly relevant to the development of early reading skills. Results suggest that children who demonstrate attentional difficulties in small-group contexts may require even more intensive (e.g., one-on-one) academic instruction to improve skill development trajectories. Further research exploring the impact and malleability of self-regulation and its subcomponents is needed to determine how best to identify and intervene with children whose academic development is being hindered by deficits in these processes.