An Evaluation of Two Emergent Literacy Screening Tools for Preschool Children

Abstract

Children’s reading success in early elementary school can be predicted from their emergent literacy skills. Consequently, there has been an increased focus on early childhood education as a means of identifying children at risk for later reading difficulty. Because diagnostic measures are impractical for this use, emergent literacy screening tools have been developed. In this study, 176 preschool children ranging in age from 42 to 55 months were administered the Revised Get Ready to Read! (GRTR-R), the Individual Growth and Development Indicators (IGDIs), and a diagnostic measure at two time points. Results indicated that GRTR-R either matched or outperformed IGDIs in terms of test-retest reliability and concurrent validity.

One of the most significant educational accomplishments in a literate society is learning to read and write. However, a significant percentage of children have difficulty accomplishing this task. In 2007, the National Assessment of Educational Progress (NAEP; National Center for Education Statistics, 2007) revealed that 33% of 4th graders in the United States were unable to read at a basic level. In addition, some research indicates that the effect of deficient reading worsens throughout children’s academic careers. For example, Chall, Jacobs, and Baldwin (1990) assessed the reading, writing, and language achievement of children in grades 2 through 7. They found that, overall, children made the greatest gain in literacy achievement from grades 2 to 3, a smaller amount of gain from grades 4 to 5, and the least amount of gain from grades 6 to 7. That is, children’s rate of literacy achievement declines as they progress through school. Literacy achievement in below-average readers, however, declines earlier and more steeply than literacy achievement in above-average readers (Chall et al., 1990). Thus, the discrepancy between below- and above-average readers, which is already present at the start of formal education, grows increasingly large over the late elementary and middle school years. Reading achievement, in particular, serves as a gateway through which children acquire knowledge in writing, math, and content-area classes (e.g., science, social studies, and history). Poor readers tend to struggle with writing, are exposed to less content knowledge, and have smaller vocabularies than good readers (Cunningham & Stanovich, 1998). It is essential for researchers and practitioners alike to work toward alleviating reading difficulties to ensure that all children have the same educational opportunities.

The acquisition of children’s reading skills was once thought to originate with the start of reading instruction in elementary school, but research now supports the idea that learning to read is a continuous developmental process that emerges early in life (Lonigan, 2006; Snow, Burns, & Griffin, 1998; Whitehurst & Lonigan, 1998). Researchers have termed the skills, knowledge, and attitudes that children have about reading and writing before they are formally taught to read and write “emergent literacy” (Sulzby & Teale, 1991; Teale & Sulzby, 1986; Whitehurst & Lonigan, 1998). Children’s reading success throughout elementary school can be predicted from their emergent literacy skills (Lonigan, Burgess, & Anthony, 2000; Lonigan, Schatschneider, & Westburg, 2008, Spira & Fischel, 2005; Storch & Whitehurst, 2002). Thus, researchers have increased their focus on emergent literacy in an attempt to identify children who may be at risk for later reading difficulty, potentially eliminating this risk before children begin elementary school (e.g., Scarborough, 1989; Whitehurst & Fischel, 2000; Whitehurst & Lonigan, 1998). Over the past three decades, many different skills have been proposed to explain how children learn to read; however, three skills consistently emerge as the strongest predictors of reading: phonological awareness, print knowledge, and oral language (Lonigan, 2006; Lonigan et al. 2008; Whitehurst & Lonigan, 1998).

Phonological awareness refers to the ability to detect and manipulate the sounds of language, independent of meaning (Lonigan, 2006; Wagner & Torgesen, 1987). Tasks tapping this ability include matching, segmenting, isolating, deleting, or counting the sounds making up a word (Phillips, Menchetti, & Lonigan, 2008; Wagner & Torgeson, 1987). Even after accounting for other factors affecting reading ability (e.g., intelligence, receptive vocabulary, memory skills, social class, etc.), phonological awareness is strongly related to the acquisition of reading (Perfetti, Beck, Bell, & Hughes, 1987; Wagner, Torgesen, & Rashotte, 1994). Children who are able to detect increasingly smaller units of sound are more capable of breaking the alphabetic code (i.e., that the letters in print reflect the specific sounds in spoken words; Adams, 1990; Whitehurst & Lonigan, 1998).

In addition to acquiring phonological awareness, children must comprehend how print is organized (print knowledge). First, children learn the conventions of print, such as knowing that text on a page progresses from left to right and top to bottom (e.g., in English), knowing which part of the book is the front, and understanding the purpose of punctuation. Second, children learn the alphabet, including letter names and letter sounds. Stevenson and Newman (1986) found that knowledge of letter names prior to kindergarten was predictive of reading ability in 5th and 10th grade.

Even if a child is capable of sounding out a word, this does not mean that he or she is “reading.” Sounding out a word does not spontaneously convey meaning; this meaning must be in place before the word will be understood. Understanding what is read is partly dependent upon a child’s oral language skills. Oral language refers to all of the words in a child’s vocabulary as well as his or her ability to use these words to understand and convey meaning successfully. One good measure of oral language skill is vocabulary. Children with larger vocabularies, relative to their same-age peers, become more proficient readers (e.g., Bishop & Adams, 1990; Scarborough, 1989).

Children who are at risk for later reading problems have weaker emergent literacy skills than children not at risk for later reading problems (Lonigan et al., 2000; Storch & Whitehurst, 2002). Several studies examining the predictive validity between emergent literacy skills later reading skills have found that emergent literacy skills are good indicators of whether a child will have trouble with reading in the early elementary grades (e.g., Bishop & Adams, 1990; Perfetti et al., 1987; Scarborough, 1989; Stevenson & Newman, 1986; Storch & Whitehurst, 2002; Wagner et al., 1994) Therefore it is helpful for teachers to be able to measure accurately these emergent literacy skills to determine who is most at risk for later reading problems and implement interventions geared toward improving emergent literacy skills with these at-risk children.

There are three ways teachers can evaluate their students’ emergent literacy skills: informal assessment, diagnostic assessment, and screening. Perhaps the most common and easiest method of assessment is informal assessment, which is accomplished by directly observing a child during routine activities and interactions. Although observation can be beneficial when engaging in teaching activities (e.g., asking a child informal questions about concepts to gauge whether further explanation is necessary), it is unknown whether informal assessment provides teachers with reliable data on how a child is performing in specific areas relative to his or her peers. Moreover, many emergent literacy skills are not casually observable in the day-to-day interactions of preschool (e.g., a child with age-appropriate phonological awareness may not display that skill in the course of typical preschool activities). Bailey and Drummond (2006) found that kindergarten and first-grade teachers, although accurate in identifying the children in their classes were most at risk for later reading difficulties, had difficulty identifying specific areas of weakness. For teachers to identify children who need additional instruction in key emergent literacy domains, they must be able to determine accurately which children have developed age-appropriate emergent literacy skills and which children are lagging behind..

One type of formal assessment is diagnostic assessment, which provides accurate, in-depth measurement of children’s emergent literacy skills relative to their peers. Diagnostic assessments are typically standardized and validated on large samples of children (Lonigan, 2006). For these reasons, diagnostic assessments are able to provide information regarding each child’s unique set of strengths and weaknesses in comparison to his or her same-age peers. Because details regarding exactly which emergent literacy skills are strong or weak for each child can be useful in planning effective instruction, diagnostic assessment is most likely the best way to measure children’s emergent literacy skills.

Diagnostic assessments, however, must be administered by trained personnel for scores to be valid, and, in general, teachers do not have the training to use diagnostic assessment tools. This can be problematic for several reasons, including that immediate testing of a child may not be possible and that shy or anxious children may not cooperate with an unfamiliar adult. In addition, diagnostic assessments can be expensive. Although having specific and detailed data for every child may be optimal, it is not feasible for preschools with limited funds to assess every preschooler using a diagnostic assessment. Thus, a type of assessment that can provide reliable and valid information regarding children’s emergent literacy skills and also meet financial and time constraints is necessary. This type of assessment is often referred to as a “screening tool,” which is a brief measure that allows a snapshot of a child’s current skills.

Currently, there are two screening tools available that measure children’s emergent literacy skills: the Get Ready to Read! (GRTR; Whitehurst & Lonigan, 2001) and the Individual Growth and Development Indicators (IGDIs; McConnell, 2002). Teachers can administer either of these screening tools easily and uniformly, and each usually takes less than 10 minutes to complete. The GRTR is a 20-item task that measures print knowledge and phonological awareness. The IGDIs contains a number of subtests designed to measure a diverse array of developmental domains from birth to approximately age eight. The subtests relevant to emergent literacy include Alliteration and Rhyming (measures of phonological awareness) as well as Picture Naming (a measure of oral language). All three of these subtests consist of a series of several flashcards randomly presented to a child, who is required to answer questions about rhyming, alliteration, and one-word picture vocabulary as quickly as possible. Children typically do not complete all items.

To date, there have been four studies examining the psychometric and predictive characteristics of the GRTR. Whitehurst (2001) validated the GRTR on a sample of 342 preschool children and determined that the GRTR was concurrently correlated with the Developing Skills Checklist (DSC), a diagnostic measure designed to assess emergent literacy skills, at.69. Molfese, Molfese, Modglin, Walker, and Neamon (2004) found that the correlations between the GRTR and measures of vocabulary, environmental print, phonological processing, and rhyming ranged from .12 to .51 (median r = .46) among a sample of 3-year-old children (N = 73) and .09 to .45 (median r = .41) among a sample of 4-year-old children (N = 79). Molfese et al. (2006) compared the GRTR with a measure of letter knowledge (Wide Range Achievement Test; WRAT), among low-income preschoolers and found that the correlation between one-year gains in WRAT scores and GRTR scores was .48. With regard to the predictive validity of the GRTR, Phillips, Lonigan, and Wyatt (2008) re-assessed children at approximately 20 months (n = 40), 28 months (n = 49), and 35 months (n = 50) after their initial assessment. Phillips et al. found that the GRTR was predictive of blending, elision, rhyming, letter knowledge, and word identification, with correlations ranging from .25 to .40 (median r = .32) for the individual criterion measures.

Currently available psychometric data for the IGDIs suggest that it is a good measure of emergent literacy skills. According to the IGDIs Technical Report #8 (Missall & McConnell, 2004), one-month test-retest reliability for the three tasks relevant to emergent literacy ranged from .44 to .89 (Alliteration: rs from .46 to .80; Rhyming: rs from .83 to .89; Picture Naming: rs from .44 to .78). With regard to concurrent validity (McConnell, Priest, Davis, & McEvoy, 2002; Missall, 2002; Missall & McConnell, 2004), correlations between all three IGDIs tasks and measures of emergent literacy, including print knowledge, phonological awareness, and vocabulary, ranged from .32 to .79 (Picture Naming: rs = .32 to .75; Rhyming: rs = .44 to .68; Alliteration: rs = .34 to .79). With regard to predictive validity (Missall et al. 2007), administration of the IGDIs in preschool was predictive of oral reading fluency at both the end of kindergarten (rs = .26 to .58; median r = .37) and the end of first grade (rs = .26 to .50; median r = .37). Further, Missall et al. found that the Rhyming and Alliteration subtests were more predictive of oral reading fluency for older children (i.e., end of the preschool year) than they were for younger children (i.e., beginning of the preschool year). Additionally, IGDIs scores in preschool were predictive of letter knowledge and phonological awareness scores in kindergarten, again the Rhyming and Alliteration subtests were more predictive of these variables for older children than they were for younger children. In general, the Picture Naming subtest was equally predictive for younger and older children.

Although there have been other measures proposed as screening measures of emergent literacy skills, the GRTR-R and IGDIs are the only two widely used screening tools validated for this age group (i.e., preschool) that are quick to administer. For example, one popular measure of preschool emergent literacy is the Phonological Awareness Literacy Screen-PreK (PALS-PreK; Invernizzi, Sullivan, & Meier, 2001). Data on the PALS-PreK suggest that it is comparable to the GRTR-R and IGDIs as a screening measure of emergent literacy skills (e.g., Invernizzi, Cook, & Gellar, 2002-2003). However, whereas previous studies investigating the psychometric properties of the GRTR-R and IGDIs have used unrelated criterion measures, studies investigating the PALS-PreK have used another version of the PALS (e.g., PALS-Kindergarten version) as the criterion. This, in conjunction with the fact that the 121-item PALS-PreK takes much longer than either the GRTR-R or IGDIs to administer, suggests that the GRTR-R and IGDIs are better candidates for screening tools of emergent literacy skills given the time and financial constraints of most preschools.

There is evidence of validity for both the IGDIs and GRTR; however, making a direct comparison of these two screeners based solely on the current body of research could lead to inaccurate conclusions. Previous studies have used different types of samples (e.g., ages and percentage of minority-status children), different criterion measures, and have tested children at different times of the year (e.g., prior to kindergarten vs. throughout the kindergarten year). No study has examined the psychometrics of these two screening measures using the same criterion measure in the same population. To this end, the purpose of this study was to compare the GRTR and IGDIs using the Test of Preschool Early Literacy (TOPEL; Lonigan, Wagner, Torgesen, & Rashotte, 2007), a recently published diagnostic measure of emergent literacy skills, as the criterion measure for all psychometric calculations. These data allowed direct comparison unspoiled by differences between samples or criterion measures. This study is important for researchers and practitioners because the GRTR-R and IGDIs are the two most practical screening tools of emergent literacy currently available. They are both quick to administer and can be and given and interpreted accurately by laypersons. Comparing these two screening tools provides further support for how these measures function relative to each other and relative to a good diagnostic measure of the three most important emergent literacy skills. From this study, information can be obtained about these screening tools and decisions made about whether to use the GRTR-R, IGDIs, or both to assess preschoolers’ emergent literacy skills.

Based on previous findings and the item content of the measures, it was predicted that the GRTR-R and IGDIs would demonstrate similar test-retest reliability over a three-month period of time. With regard to concurrent validity, it was predicted that the GRTR-R would outperform the IGDIs in the domain of print knowledge, the IGDIs would outperform the GRTR-R in the domain of oral language, and that these screeners would perform equally in the domain of phonological awareness.

Method

Participants

Twenty-one preschools in north Florida agreed to participate in this study. From these preschools, parents of 199 children signed consent forms allowing their children to participate in the study. These 199 children ranged in age from 42 to 55 months, with a mean age of 48.55 months (SD = 3.69). Child sex was divided equally among boys (50%) and girls (50%), and the majority of the children were Caucasian (70% Caucasian, 19% African American, 11% other ethnicity). Although 199 children were assessed at Time 1 (July), 23 children were unavailable for assessment at Time 2 (October). These 23 children were mostly from ethnic minority groups (52% African American and 9% other ethnicity), boys (61%), and they had obtained lower Time 1 scores on the GRTR-R, F(1, 198) = 4.32, p = .04; IGDIs total score, F(1, 198) = 5.28, p = .02; and TOPEL Early Literacy Index, F(1, 198) = 6.22, p = .01.¹ The 176 children remaining in the sample ranged in age from 42 to 55 months at Time 1, with a mean age of 48.49 months (SD = 3.68), and were comprised of 49% boys. Most of these children were Caucasian (74%); 15% were African American; and 11% were classified as other ethnicity.

Preschools and Child-care Centers

Informal observations of the preschools and child-care centers were made to identify the general structure of these environments. Although materials available to children and activity structure varied between locations, the majority of centers did not engage in formal literacy-related instruction; that is, children primarily spent their time in self-directed activities in and out of the classroom. However, 3 of the 21 preschools did informally teach children about letters, numbers, and storybooks.² For example, children recited the alphabet, colored in outlines of numbers, and sat in a group listening to an adult read stories aloud.

Measures

Get Ready to Read Screening Tool (GRTR Whitehurst & Lonigan, 2001)

The GRTR was revised recently (Revised Get Ready to Read!; GRTR-R). One of the original 20 items was excluded and five new items were added to the original GRTR with the goal of making it a more useful measure of the emergent literacy skills of children from middle and upper-middle socioeconomic backgrounds as well as older preschool children. The GRTR-R is a 25-item test that measures print knowledge and phonological awareness and takes less than 10 min to administer. For each item, the child is shown a page with four pictures. The test administrator reads the question at the top of each page aloud and the child answers by pointing to one of the four pictures. For example, for the first item, the child is presented with illustrations of a book positioned in four different ways, and the adult says, “These are pictures of a book. Find the one that shows the back of the book.” At the end of the GRTR, correct answers are summed into a single score encompassing both print knowledge and phonological awareness. Internal consistency reliability for the GRTR-R in the normative sample (N = 866 3-, 4-, and 5-year-old children) was .88 (Lonigan & Wilson, 2008).

Individual Growth and Development Indicators (IGDIs; McConnell, 2002)

The IGDIs is a compilation of tests designed to describe young children’s growth and development, including expressive communication, adaptive ability, motor control, social ability, and cognition. For this study, the three tasks related to emergent literacy skills were chosen: Alliteration and Rhyming (to evaluate phonological awareness), and Picture Naming (to evaluate oral language); note that there is no IGDIs test for print or letter knowledge. It takes less than 10 min to administer all three tasks, including transitioning between tasks. For each of these three tasks, a set of flashcards is available as an item pool. The set of cards is shuffled between task administrations, such that each child is given a different set and order of cards. For the Alliteration Task, the child is shown a page with one picture at the top and three additional pictures at the bottom. The task administrator points to the picture at the top, names it, and says, “Point to the one that starts with the same sound as ___.” For the Rhyming Task, the procedure is nearly identical, except the task administrator says, “Point to the one that sounds the same as ___.” For the Picture Naming Task, the child is shown pictures, one per flashcard, and told to name the pictures as fast as they can. Scores on each task are the number of items completed correctly within a two-minute (for Alliteration and Rhyming) or one-minute (for Picture Naming) administration period. For both Alliteration and Rhyming tasks, if the child does not provide a correct answer for at least two of the first four cards shown, the remainder of the task is not administered. According to the IGDIs Technical Report #8 (Missall & McConnell, 2004), one-month test-retest reliability for these three tasks ranged from .44 to .78 (Picture Naming), .83 to .89 (Rhyming), and .46 to .80 (Alliteration). Internal consistency cannot be calculated for the IGDIs because items are not consistent across test administration.

Test of Preschool Early Literacy (TOPEL; Lonigan et al., 2007)

The development and testing of the TOPEL was based on the last decade of research concerning the development of emergent literacy, and the final version was normed on a sample of 842 children representative of the national population on several domains, including gender, ethnicity, family income, and highest level of parent education, all of which remained relatively consistent when stratified by children’s ages (i.e., 3-, 4-, and 5-year-olds). The TOPEL is a diagnostic measure of emergent literacy skills that includes three subtests: Print Knowledge, Definitional Vocabulary, and Phonological Awareness. According to the test manual (Lonigan et al., 2007), internal consistency reliability for these subtests ranges from .86 to .96 for 3- to 5-year-olds, and test-retest reliability over a one- to two-week period ranges from .81 to .89. Concurrent validity for the subtests ranges from .59 to .77. In addition to these subtest scores, an Emergent Literacy Index can be generated. The internal consistency reliability of the TOPEL Early Literacy Index is .96.

The Print Knowledge subtest measures print concepts (e.g., “Find the picture that has a word in it”), letter discrimination (“Which one is M?”), word discrimination (“Which one can you read?”), letter-name identification (“What is the name of this letter?”) and letter-sound identification (“What sound does this letter make?”). The Definitional Vocabulary subtest measures children’s single word spoken vocabulary and their ability to formulate definitions for words. Children are shown pictures of objects and asked to name the object. Next, children are asked a follow-up question relevant to the picture, such as, “Where does it live?” when presented a picture of a pig and, “What sound do they make?” when presented with a picture of several sheep. The Phonological Awareness subtest includes both multiple-choice and free-response items along the developmental continuum of phonological awareness from word awareness to phonemic awareness. Children are required to perform both blending (putting sounds together to form a new word) and elision (removing sounds from a word to form a new word) by answering questions such as, “What word do these make? Bath – tub,” and, “Say ‘shoelace.’ Now say ‘shoelace’ without saying ‘lace.’”

The three TOPEL subtests and the TOPEL Early Literacy Index have shown good convergent validity with other measures of similar constructs. For example, the TOPEL is highly correlated with concurrent tests of print knowledge (TERA-3, r with TOPEL Print Knowledge = .77), expressive vocabulary (EOWPVT, r with TOPEL Definitional Vocabulary = .71), phonological awareness (CTOPP, rs with TOPEL Phonological Awareness = .59 to .65), and overall early reading ability (TERA-3, r with TOPEL Early Literacy Index = .67; Lonigan et al., 2007). Data also support that the TOPEL is predictive of later reading skills. For example, scores on the TOPEL Print Knowledge and Phonological Awareness subtests administered in preschool were found to be significant correlates both of measures of phonological awareness (median r = .40), which is a strong concurrent correlate of decoding skills, and of measures of reading skills (e.g., word identification, word attack; rs = .30 to .60) administered when children were in kindergarten and first grade (Sims & Lonigan, 2008; Lonigan & Farver, 2008).

Procedure

Written consent was obtained from each child’s parent before the first assessment battery, and verbal child assent was obtained before each interaction with individual children. The TOPEL, GRTR-R, and IGDIs were each administered twice: First in July then three months later in October. Only children remaining in preschool for that school year were included in the study. To ensure that order of screener administration did not affect performance, the order of administration of GRTR-R or IGDIs was counterbalanced across children, determined randomly; all children completed the TOPEL last. Each testing session lasted approximately 40 to 50 min and was conducted by trained examiners, all of whom were required to demonstrate proficiency with the assessment battery before testing at any preschools.

For preschool children, 40-50 min of testing could potentially cause fatigue. For this reason, examiners were instructed to discontinue the testing session if a child indicated that he or she no longer wished to participate. In these rare cases, testing was continued on a second day. However, children typically enjoyed these tasks, and very few examiners reported needing to discontinue a testing session due to a child’s fatigue.

Results

Preliminary Analyses and Descriptive Data

Prior to analyses, all variables were examined for accuracy of data entry, missing values, and fit between their distributions and the assumptions of multivariate analysis. The only variables containing missing values were the Alliteration and Rhyming subtests of the IGDIs. These missing values were due to children who were not able to answer at least two of the four practice items correctly. All of these missing values were replaced with zeros. Normality of these data was determined by evaluating kurtosis and skew. Two variables were kurtotic and four were skewed; specifically, there were floor effects on the IGDIs phonological awareness composite (see below) and IGDIs total score. However, analyses using untransformed variables are reported because analyses using transformed variables produced the same findings. For the IGDIs, Alliteration and Rhyming were summed into a Phonological Awareness composite score; this decision was made because the correlation between these measures was moderate (r = .32 at Time 1 and r = .47 at Time 2) and because they are both part of the construct of phonological awareness. In addition to calculating an IGDIs Phonological Awareness composite, a total IGDIs composite was calculated by adding all raw scores across the three IGDIs subtests. Across time, the correlation between IGDIs Phonological Awareness and IGDIs Picture Naming fluctuated from weak (r = .12) to moderate (r = .33). To get a true picture for how the IGDIs functions as an emergent literacy screener, the IGDIs composite score, IGDIs Phonological Awareness score, and IGDIs Picture Naming score were tested independently.²

Descriptive statistics for all measures at Time 1 and Time 2 are shown in Table 1. Scores on all measures increased significantly from Time 1 to Time 2 (all ps < .01; see Table 1). Order of test administration did not affect scores at Time 1 (GRTR-R score, F[1,175] = .77, p = .38; IGDIs score, F[1, 175] = .01, p = .91) or at Time 2 (GRTR-R score, F[1, 175] = 1.77, p = .19; IGDIs score, F[1, 175] = .57, p = .45).

Table 1.

Descriptive Statistics for Time 1 and Time 2 Emergent Literacy Skills

	Time of Assessment
	Time 1			Time 2
Measure	Mean	(SD)	Range	Mean	(SD)	Range	t for Time Contrast	Test-Retest Reliability
GRTR-R
Total Raw Score	12.95	(4.70)	3-25	15.54	(4.72)	4-25	10.02***	.73***
IGDIs Raw Scores
Composite Score	24.42	(9.52)	3-64	29.16	(11.46)	2-69	5.77***	.48***
Phonological Awareness	5.96	(6.27)	0-31	8.88	(8.16)	0-37	4.77***	.42***
Picture Naming	18.46	(6.48)	2-34	20.28	(5.81)	2-35	3.56***	.38***
TOPEL Standard Scores
Early Literacy Index	96.05	(13.58)	60-135	103.22	(14.10)	68-142	9.51***	.76***
Print Knowledge	102.08	(16.04)	78-144	106.20	(15.65)	77-142	5.49***	.81***
Definitional Vocabulary	98.14	(12.69)	62-126	104.52	(12.49)	51-127	7.44***	.59***
Phonological Awareness	91.47	(14.38)	55-135	97.71	(15.48)	57-134	5.34***	.48***

Note. N = 176. TOPEL = Test of Preschool Early Literacy; GRTR-R = Revised Get Ready to Read!; IGDIs = Individual Growth and Development Indicators. t for time contrast with 175 df.

^***p ≤ .001.

Test-Retest Reliability

Correlations between scores at Time 1 and scores at Time 2 for the GRTR-R, IGDIs, and TOPEL are shown in Table 1. Differences between test-retest coefficients were tested using the computation presented by Alf and Graf (1999; case 2, pp. 72), which evaluates significance depending on whether the 95% confidence interval around the squared difference in correlation coefficients overlaps with zero. The test-retest correlation coefficient for the GRTR-R was higher than the test-retest correlation for the IGDIs composite score (.20 < r 2 GRTR-R - r 2 IGDIs_score < .40), the IGDIs Phonological Awareness (PA) score (.25 < r 2 GRTR-R - r 2 IGDIs_PA < .46), and the IGDIs Picture Naming (PN) score (.28 < r 2 GRTR-R - r 2 IGDIs_PN < .50).

Concurrent Validity

Time 1

Correlations between scores on the GRTR-R and IGDIs with the TOPEL at Time 1 are shown in the upper panel of Table 2. Because these correlation coefficients were correlated, differences between correlations were evaluated using the computational procedure outlined by Meng, Rosenthal, and Rubin (1992). For the TOPEL Early Literacy Index, the GRTR-R correlation was higher than all three IGDIs correlations (IGDIs composite, Z = 4.94, p < .001; IGDIs PA, Z = 6.09, p < .001; IGDIs PN, Z = 5.84, p < .001). Results were similar for TOPEL Print Knowledge and TOPEL Phonological Awareness; the GRTR-R correlation was higher than all three IGDIs correlations (TOPEL Print Knowledge: IGDIs composite, Z = 6.29, p < .001; IGDIs PA, Z = 6.57, p < .001; IGDIs PN, Z = 7.07, p < .001; TOPEL Phonological Awareness: IGDIs composite, Z = 2.75, p = .003; IGDIs PA, Z = 3.19, p = .001; IGDIs PN, Z = 3.24, p = .001). With regard to TOPEL Definitional Vocabulary, the GRTR-R correlation was higher than the IGDIs PA correlation (Z = 2.55, p = .005); however, the GRTR-R correlation was statistically equivalent to both the IGDIs composite correlation (Z = .48, p = .32) and the IGDIs PN correlation (Z = 95, p = .17).

Table 2.

Concurrent Validity Correlations between the Revised Get Ready to Read and Individual Growth and Development Indicators with the Test of Preschool Early Literacy

Measure	TOPEL Print Knowledge	TOPEL Definitional Vocabulary	TOPEL Phonological Awareness	TOPEL Early Literacy Index
Time 1
GRTR-R Score	.73***	.43***	.43***	.72***
IGDIs Composite	.39***	.40***	.25**	.46***
IGDIs PA	.32***	.24**	.19**	.34***
IGDIs PN	.27***	.36***	.18*	.35***
Time 2
GRTR-R Score	.65***	.48***	.51***	.71***
IGDIs Composite	.35***	.49***	.44***	.55***
IGDIs PA	.39***	.40***	.47***	.54***
IGDIs PN	.15*	.41***	.20**	.31***

Note. N = 176. TOPEL = Test of Preschool Early Literacy; GRTR-R = Revised Get Ready to Read!; IGDIs = Individual Growth and Development Indicators; PA = Phonological Awareness; PN = Picture Naming.

^*p ≤ .05.

^**p ≤ .01.

^***p ≤ .001.

Time 2

Correlations between scores on the GRTR-R and IGDIs with the TOPEL at Time 2 are shown in the lower panel of Table 2. For the TOPEL Early Literacy Index, the GRTR-R coefficient was higher than all three IGDIs correlations (IGDIs composite, Z = 3.23, p = .001; IGDIs PA, Z = 3.38, p < .001; IGDIs PN, Z = 5.88, p < .001). Results were similar for TOPEL Print Knowledge: the GRTR-R coefficient was higher than all three IGDIs correlations (IGDIs composite, Z = 5.25, p < .001; IGDIs PA, Z = 4.58, p < .001; IGDIs PN, Z = 6.65, p < .001). With regard to TOPEL Phonological Awareness, the GRTR-R correlation was higher than the IGDIs PN correlation (Z = 3.93, p < .001); however, the GRTR-R correlation was statistically equivalent to both the IGDIs composite correlation (Z = 1.18, p = .12) and the IGDIs PA correlation (Z = .68, p = .25). With regard to TOPEL Definitional Vocabulary, the GRTR-R correlation was statistically equivalent to all three IGDIs correlations (IGDIs composite, Z = -.17, p = .57; IGDIs PA, Z = 1.31, p = .10; IGDIs PN, Z = .94, p = .17)

Comparison of Time 1 and Time 2

Differences in the correlations between scores on the GRTR-R and IGDIs with the TOPEL at Time 1 and Time 2 were compared using the computation outlined by Alf and Graf (1999); confidence intervals for all comparisons can be found in Table 3. Both the IGDIs composite correlation and IGDIs PA correlation increased across time for the TOPEL Early Literacy Index, Definitional Vocabulary subtest, and Phonological Awareness subtest. Additionally, the GRTR-R correlation decreased across time for TOPEL Print Knowledge and increased across time for TOPEL Phonological Awareness. All other scores were statistically equivalent across time.

Table 3.

95% Confidence Intervals for Comparing Concurrent Validity Correlations across Test Administrations

	Criterion Measure
Screening Measure	TOPEL Print Knowledge	TOPEL Definitional Vocabulary	TOPEL Phonological Awareness	TOPEL Early Literacy Index
GRTR-R Score	.04, .18a*	−.01, .10	.01, .14*	−.01, .04a
IGDIs Composite	−.02, .08a	.01, .15*	.04, .22*	.02, .16*
IGDIs PA	−.01, .11	.02, .18*	.08, .29*	.08, .27*
IGDIs PN	−.01, .11a	−.01, .09	−.02, .03	−.02, .07a

Note. N = 176. TOPEL = Test of Preschool Early Literacy; GRTR-R = Revised Get Ready to Read!; IGDIs = Individual Growth and Development Indicators; PA = Phonological Awareness; PN = Picture Naming; T1 = Time 1; T2 = Time 2.

^acorrelation was higher for Time 1 than Time 2; all other correlations were higher for Time 2 than Time 1.

^*p ≤ .05.

Discussion

The primary purpose of this study was to compare the GRTR-R and IGDIs using the same sample and same criterion measure. Overall, the results of this study indicate that the GRTR-R is a better screening tool of preschool children’s emergent literacy skills than the IGDIs. Whereas previous studies have examined the psychometrics of these two measures, the results of this study expand earlier findings by comparing the two measures in the same population, within the same time frame, and using the same criterion measure. In terms of reliability and concurrent validity, this study indicated that the GRTR-R consistently performed better than or equal to the IGDIs. A partial explanation for why the IGDIs did not fare as well as the GRTR-R, particularly relating to code-related emergent literacy skills (i.e., print knowledge and phonological awareness), concerns the breadth of skills assessed by each measure as well as administration rules. For many younger children and children with limited emergent literacy skills, IGDIs is an insensitive measure that fails to provide scores that differentiate between low levels of skill and no skill. Consequently, the GRTR-R is likely to have higher educational utility for teachers and other early childhood education professionals than are the IGDIs.

Research on the key components of emergent literacy indicates that, in the absence of intervention, children’s vocabulary, phonological awareness, and print knowledge skills are relatively stable over time (e.g., Baydar, Brooks-Gunn, & Furstenberg, 1993; Burgess & Lonigan, 1998; Lonigan et al., 2000; Stevenson & Newman, 1986). That is, although children’s emergent literacy skills are continually improving, children’s rakings relative to their peers remains constant; a child with emergent literacy skills higher than his or her peers will continue to have emergent literacy skills higher than his or her peers. Consequently, it would be expected that measures of these skills, like the GRTR-R and the IGDIs, would show moderate to strong test-retest correlations over a three-month interval (with correlations of .1 to .3 indicating a small correlation, .3 to .5 indicating a moderate correlation, and over .5 indicating a strong correlation; Cohen, 1992). In general, these data demonstrate that the GRTR-R is a more reliable measure than the IGDIs with preschool children. Basic psychometric data on the GRTR-R and IGDIs were comparable to data from previous studies. Moving beyond issues related to internal consistency and test-retest reliability, results concerning concurrent validity favored the GRTR-R over the IGDIs. At both assessment periods, the GRTR-R was generally more highly correlated with the concurrently administered subscales of the TOPEL, a comprehensive diagnostic measure of emergent literacy skills, than were IGDIs scores.

One explanation for why the IGDIs performed worse than the GRTR-R in terms of concurrent validity with the TOPEL concerns the formatting of these measures. Although the IGDIs and GRTR-R are similar in that they both provide answer choices for children and require only nonverbal responses, the IGDIs is a timed task and the GRTR-R is not. The TOPEL is like the GRTR-R in that it is not a timed task, which may have resulted in the GRTR-R’s higher correlations with the TOPEL because these two measures assesses emergent literacy skill only, but the IGDIs measures both emergent literacy and processing speed. Without also collecting a measure of processing speed, it is impossible to determine the extent of the influence of this cognitive ability on phonological awareness.

A second explanation for why the IGDIs performed worse than the GRTR-R in terms of both concurrent validity and reliability concerns the inability of many younger children to complete the IGDIs phonological awareness measures. The administration rules for both IGDIs Alliteration and IGDIs Rhyming subtests require that children answer at least two of the four practice items correctly to proceed to test administration. Alliteration and rhyming are aspects of phonological awareness that emerge later in the developmental continuum of phonological awareness abilities (Anthony, Lonigan, Driscoll, Phillips, & Burgess, 2003; Phillips, Menchetti, & Lonigan, 2008). For example, Anthony et al. reported that 2- to 6-year-old children’s phonological awareness develops gradually, with word-level skills developing first, followed by syllable-level skills, then onset/rime-level skills, and lastly, phoneme-level skills. Consequently, children in the early stages of acquiring phonological awareness or who have limited phonological awareness skills will not be administered IGDIs Rhyming or IGDIs Alliteration, as these are onset/rime-level skills.

In this study, only 40 percent of children completed the IGDIs Alliteration subtest and only 55 percent completed the IGDIs Rhyming subtest at Time 1, and only 62 percent of children completed the IGDIs Alliteration subtest and only 65 percent completed the IGDIs Rhyming subtest at Time 2. Due to the administration rules of the IGDIs phonological awareness tasks, between 35 and 60 percent of children received scores of zero, depending on the subtest and time of assessment. In contrast, less than 1 percent of children received a score of zero on the phonological awareness subtest of the criterion measure (i.e., the TOPEL) at either assessment point. These findings, as well as conceptual and empirical models of young children’s developing phonological awareness, suggest that measures of phonological awareness that rely solely on tasks involving rhyme or alliteration are insufficient to capture the range of skills that reflect preschool children’s developing phonological awareness abilities. For young children and children with limited phonological awareness abilities, the IGDIs Rhyming and Alliteration tasks are simply too difficult and too unreliable to provide a stable and valid estimate of phonological awareness. For a screening tool to assess the skills of a heterogeneous sample of children accurately, it is important that items cover the full range of the ability it purports to measure.

The increased percentage of children who were able to complete the IGDIs Rhyming and IGDIs Alliteration subtests at Time 2 was likely a result of children’s increasing emergent literacy skills. That is, over the course of three months, children were likely to become better equipped to complete these two IGDIs tasks due to increased development of phonological awareness. Findings of increased numbers of children who were able to complete the IGDIs phonological awareness tasks as well as the increases in the concurrent correlations between scores on these tasks and scores on the criterion measure of phonological awareness, are consistent with previously reported results for the IGDIs. For example, Missall et al. (2007) found that mean IGDIs Alliteration and IGDIs Rhyming scores increased steadily in a sample of children followed from the beginning of preschool to the end of kindergarten. Additionally, Gibbons et al. (2003) reported that that with a sample of children assessed at two-month intervals, the correlations between the later intervals were higher than the correlations between the earlier intervals (r = .57 for Rhyming and .26 for Alliteration versus .77 and .46 two months later).

Because the GRTR-R measures a range of phonological awareness abilities across the developmental continuum of this emergent literacy skill, it is likely a better measure of this skill than the IGDIs. In contrast with the IGDIs Rhyming and Alliteration tasks, the GRTR-R measures word-level, syllable-level, and onset/rime-level skills. Anthony et al. (2003) suggested that, given the developmental continuum of phonological awareness, the best measure for preschoolers would be one that spans across several levels of phonological awareness to allow children’s skills in phonological awareness to be measured accurately. The IGDIs assesses only one component along this developmental continuum, whereas measures like GRTR-R and the TOPEL include multiple components. Floor effects are less likely on the GRTR-R because phonological awareness skills along the developmental continuum are sampled. Moreover, children complete all 25 items on the GRTR-R regardless of their performance on earlier items. Additionally, the types of items sampled by the GRTR-R are likely more reliable than the types of items on the IGDIs. Lonigan, Burgess, Anthony, and Barker (1998) reported that although rhyming and alliteration tasks were not reliable with most 3- and 4-year-old children, blending and elision tasks were very reliable with children in this age range (e.g., α = .89 to .96). With regard to predictive validity, there is a significant literature noting that rhyme is not the best phonological awareness task in predicting early reading development (Lonigan et al., 2008; Muter, Hulme, Snowling, & Stevenson, 2004; Muter, Hulme, Snowling, & Taylor, 1997).

Whereas the GRTR-R is a better measure than the IGDIs for use with younger children and children with less well-developed emergent literacy skills, it might be the case that the psychometric properties of the IGDIs would be as strong as the GRTR in a sample of older children or children with more well-developed emergent literacy skills. Overall, in this study, the concurrent correlations between the IGDIs and the criterion measures increased from the first assessment to the second assessment. It would be useful for future studies to compare the performance of the IGDIs and the GRTR-R in a sample of children with a wide age range so that each measure’s psychometrics could be evaluated for older and younger children separately. Such analyses would allow a determination of the approximate age at which the IGDIs becomes a good indicator of children’s phonological awareness skills. For example, Missall et al. (2007) found that IGDIs Picture Naming and Rhyming were relatively consistent in predicting kindergarten skills, whether measured in the fall, winter, or spring of the preschool year. IGDIs Alliteration, on the other hand, demonstrated higher correlations with criterion measures when administered later in the preschool year as compared to earlier in the preschool year. Thus, it might be that whereas the IGDIs is equally as good at measuring vocabulary and rhyming skills in younger preschoolers, it is a better measure of alliteration in older preschoolers. Future studies comparing the performance of the IGDIs and the GRTR-R in a sample of children with a wider age range would also allow determination how these screening tools function with older children.

The educational use of screening measures such as the GRTR-R or IGDIs could help identify children who have weaker than expected emergent literacy skills and children who are experiencing satisfactory development of their emergent literacy skills. Children identified as having weaknesses in emergent literacy could then receive additional assessment to identify areas of strengths and weakness. These children could receive additional targeted instruction to promote the development of their emergent literacy skills, and teachers could use screening measures to assess children’s response to instruction. For the value of this educational use of screening measures to be realized, however, the measures need to accurately measure the skills they are intended to measure. The measures need to provide information concerning the likely outcomes for children over time in key skill areas, and they need to provide a stable basis for measuring children’s improvement or lack of improvement in these key skill areas. Based on these criteria, this study indicates that the GRTR-R is a better screening measure of key emergent literacy skills for preschool children.

Overall, the results of this study were clear. However, there are at least two issues deserving mention that limit the broad generalization of the results of this study. First, this study only examined the accuracy of these screening tools in describing children’s emergent literacy skills. Whereas this information is good for planning initial instruction, most early childhood educators also would be interested in predicting performance on reading measures in kindergarten at the beginning of the preschool year. Thus, studies examining the long-term predictive validity of these screening tools to more applied measures of reading are needed. Phillips et al. (2008) provided some data on the 1.5- to 3-year predictive validity of the GRTR, suggesting that the predictive value of the GRTR extends over this time frame. Likewise, Missall et al. (2007) provided data on the predictive validity of preschool-administered IGDIs to kindergarten and first grade criterion measures; this study demonstrated that the IGDIs is a valid predictor of skills up to at least first grade. However, having data on both the GRTR and the IGDIs predicting the same criterion measures would help inform which tool provides better or worse predictive power.

Another potential limitation of the study was that although the distribution of the demographics of the sample of children in this study was similar to the overall distribution of demographics found in Florida (U.S. Census Bureau, 2000), the percentage of children considered “at-risk” at Time 2 (determined by performance below the 25th percentile on the TOPEL) was 16 percent. Thus, the current sample over-represented children not at risk for later reading difficulties. The ethnic distribution of the 23 children not tested at Time 2 was mostly children from minority ethnic groups (39% Caucasian, 52% African American and 9% other ethnicity). Historically, children from ethnic minority groups in the United States perform less well on measures of literacy than persons from the majority group (e.g., Brooks-Gunn, Klebanov, & Duncan, 1996; National Assessment of Education Progress, 2007; Wolfle, 1985), and this was true in this study’s data. The children who were most likely to be at-risk differentially dropped out of the sample. It is important for future studies to focus more on this lower-performing population, as screening measures are designed to identify just such at-risk children. Perhaps future studies can sample a higher percentage of preschools serving children at risk and include a sufficiently large sample to allow group comparisons to be made with adequate statistical power. Results from such a study would allow more confident generalization to a group of children with whom these measures would most likely be used.

In summary, a growing body of research has increased our understanding of emergent literacy skills and their associations with later performance in reading. Researchers have uncovered several interventions capable of boosting emergent literacy skills in children identified as needing additional instruction (e.g., see Lonigan & Phillips, 2007; What Works Clearinghouse [http://ies.ed.gov/ncee/wwc/] for summaries). However, before children are assigned to receive additional instruction, they must be identified correctly as needing this instruction. The results of this study indicate that the GRTR-R is better than the IGDIs as a screener of the three emergent literacy skills measured by the TOPEL: print knowledge, oral language, and phonological awareness. Future studies should examine if the IGDIs is a better measure of emergent literacy skills in older children and whether either of these screeners maintains good psychometric properties in a more diverse and a more at-risk sample.