Consultation for Teachers and Children’s Language and Literacy Development during Pre-Kindergarten

Abstract

MyTeachingPartner (MTP) is a teacher professional development program designed to improve the quality of teacher-child interactions in pre-kindergarten classrooms and children’s language and literacy development. The program includes language/literacy activities and two Web-based resources—video exemplars of effective interactions and individualized consultation—designed to support teachers’ high quality implementation of these activities. This study examined the impacts of the MTP Web-based resources on the language and literacy development of 1,165 children during pre-kindergarten. Children whose teachers were randomly assigned to receive access to both the video exemplars and participated in consultation (MTP Consultancy n=65) made greater gains in receptive language skills during pre-kindergarten compared to children whose teachers were randomly assigned to receive access to the video exemplars only (MTP Video Library n=69). Further, among MTP Consultancy teachers, more hours of participating in the consultation process was positively associated with children’s receptive language development, and more hours implementing the language/literacy activities was positively associated with children’s language and literacy development. Implications for improving children’s school readiness and promoting teachers’ participation in professional development programs are discussed.

Based on contemporary estimates, public investments in early education programs for 3- and 4-year olds continue to rise annually, and in 2007–2008, state preschool programs served over one million children and cost state governments over 4.6 billion dollars (Barnett, Epstein, Friedman, Boyd, & Hustedt, 2008). In fact, across the full range of available preschool opportunities, 82% of 4-year-olds currently attend some type of center-based early education program prior to enrolling in kindergarten (Barnett et al., 2008). The ultimate goal for many, if not most, of these public programs is to improve the school readiness and educational outcomes of children who are growing up in disadvantaged circumstances. Evidence from rigorous experimental evaluations, as well as quasi-experimental analyses of programs implemented at scale, show rather convincingly that enrollment in high quality programs does indeed contribute to children’s readiness for school (e.g., Bryant, Burchinal, Lau, & Sparling, 1994; Howes, Phillips, & Whitebrook, 1992; Lamb & Ahnert, 2006; NICHD ECCRN, 1999; NICHD ECCRN, 2002; Peisner-Feinberg & Burchinal, 1997; Peisner-Feinberg et al., 2001) .

Recent studies now indicate that a key mechanism through which the benefits of attending preschool programs are transmitted to children is teachers’ effective implementation of instruction through their interactions with children (Hamre & Pianta, 2005; Howes et al., 2008; Mashburn et al., 2008; National Council on Teacher Quality [NCTQ], 2005; NICHD ECCRN, 2000). Because, on average, the quality of teacher-child interactions in many such programs is not high and effective curriculum implementation is inadequate (NICHD ECCRN, 2002; Peisner-Feinberg & Burchinal, 1997; Pianta et al., 2005), professional development and training of teachers in instructional/interaction skills has the potential to magnify the benefits of children’s participation in preschool (Zaslow & Martinez-Beck, 2005), particularly with respect to promoting early language and literacy skills. In the present study, we used data from an experimental evaluation of MyTeachingPartner (MTP; Pianta et al., 2008)—a suite of web-based professional development tools designed to improve the quality of teachers’ interactions with children and children’s language and literacy skills—to examine children’s growth in early language and literacy skills during pre-kindergarten (pre-k) as a function of these professional development resources.

One hundred thirty-four pre-k teachers were randomly assigned to one of two study conditions. Teachers in the MTP Consultancy condition (n=65) received access to MyTeachingPartner Language & Literacy activities and video exemplars of effective interactions, as well as individualized consultation to support their implementation of these language/literacy activities. Teachers in the MTP Video Library condition (n=69) received the MyTeachingPartner Language & Literacy activities and access to web-based video exemplars of effective teacher-child interactions. The video exemplars included dozens of brief (1–2 minute) video clips with annotations describing teachers’ behaviors and interactions. Intent-to-treat analyses compared the receptive language and emergent literacy development of children whose teachers were assigned to the MTP Consultancy and MTP Video Library conditions to estimate the effects of the teaching consultation on children’s outcomes. Further, among teachers who participated in the MTP Consultancy condition, treatment-on-treated analyses examined the extent to which variation in use of three intervention components—language/literacy activities, web-based resources, and teaching consultation—was associated with children’s language and literacy development during pre-k.

Teacher Professional Development

Results from recent studies of children’s development during pre-k confirm that the quality of the emotional and instructional interactions that children experience on a daily basis in pre-k classrooms are key mechanisms through which pre-k programs transmit academic, language, and social competencies to children (Hamre & Pianta, 2007; Howes et al., 2008; Mashburn et al., 2008; Pianta, 2003). One potentially effective method to improve the quality of interactions in pre-k classes is through in-service teacher professional development programs. A framework for effective teacher professional development is described in No Child Left Behind—to improve teaching and learning, professional development programs must be intensive, sustained, and classroom-focused (U.S. Department of Education, 2002). Emerging from this framework has been a recent shift away from static, knowledge-based professional development workshops to more active, collaborative coaching, mentoring, and consultation techniques that provide continuous, practice-focused support and guidance to teachers to improve their daily interactions with children (Abdal-Haqq, 1995; Darling-Hammond & McLaughlin, 1995; Ingersoll & Kralik, 2004; Lieberman, 1995; Pianta, 2005; Putnam & Borko, 1997; Richardson, 2003). There is considerable overlap among this class of professional development approaches, but common across them is teachers’ receiving feedback regarding their classroom practices.

There have been a number of recent efforts to develop, implement, and test these practice-focused approaches to professional development for teachers of young children. For example, the Quality Interventions for Early Care and Education (QUINCE) project involves the implementation of two individualized on-site consultation models that provide training and support for home-based childcare providers and teachers in childcare centers (Bryant, 2007). The Project Great Start Professional Development Initiative provides course-work and on-going consultation supports to teachers in center and home-based care settings to improve preschool teachers’ language and literacy instructional practices and child outcomes (Koh & Neuman, 2009). Powell, Diamond, Burchinal, and Koehler (2010) recently reported that video-based coaching and feedback was helpful for improving teachers’ practices in early education settings and Landry et al. (2006) have shown that coaching-type approaches with early educators can be effective for improving child outcomes. These and other professional development programs in which teachers receive consistent, non-evaluative feedback about providing high quality interactions and implementation of activities have the potential to improve children’s experiences in classrooms and learning outcomes, particularly when their focus is around the interactions and implementation of activities that most effectively support children’s language and literacy skills.

Children’s language and literacy skills are a natural focus for the professional development of preschool teachers, given their importance to children’s later school success (e.g., Storch & Whitehurst, 2002). Many children enrolled in publicly-funded preschool programs exhibit significant disadvantages on measures of early language and literacy development (e.g., vocabulary knowledge, print awareness) relative to more advantaged peers (e.g., Chaney, 1998; Justice, Bowles, & Skibbe, 2006). Because early language and literacy skills exhibit causal associations with later reading achievement in both word recognition and reading comprehension (e.g., Anthony, Lonigan, Driscoll, Phillips, & Burgess, 2003; Lonigan, Burgess, Anthony, & Barker, 1998; Storch & Whitehurst, 2002), interventions in early childhood education settings to improve these early skills can have long-term benefits for children and school systems (Torgesen, 1998). Despite evidence accruing on the effectiveness of specific curricula or instructional techniques in these learning domains (e.g., Byrne & Fielding-Barnsley, 1993, 1995; Girolametto, Pearce, & Weitzman, 1996; Girolametto, Weitzman, & Clements-Baartman, 1998; Justice & Ezell, 2002; Penno, Wilkinson, & Moore, 2002; Wasik & Bond, 2001; Whitehurst, Epstein, Angell, Crone, & Fischel, 1994) observational studies show that demonstrably effective literacy interventions have no effect on child outcomes when the quality and effectiveness of implementation is low (Dickinson & Brady, 2005; Howes et al., 2008).

As a result, evidence suggests that preschool teachers need support in identifying high-priority skill targets in language and literacy (Cunningham, Perry, Stanovich, & Stanovich, 2004; Justice & Ezell, 1999; Lonigan, 2004; Moats, 1994) and require ongoing supports in how to effectively implement specific instructional activities in early literacy and engage in interactions and conversations with children that promote language skills (Morrison & Connor, 2002; NICHD ECCRN, 2002). Preschool teachers are rarely exposed to multiple field-based examples of objectively-defined high quality practice, and have few if any opportunities to receive feedback about the extent to which their classroom interactions and instruction promote these skill domains; these represent important limitations of current approaches to the professional development of the early education workforce. In the next section, we describe MyTeachingPartner, a professional development program designed to improve pre-k teachers’ interactions during language and literacy instruction and children’s language and literacy development.

MyTeachingPartner Professional Development Program

The MyTeachingPartner (MTP) professional development program was developed to provide teachers with a package of integrated supports for delivering effective language and literacy instruction [see Kinzie et al. (2006) for a description of the program]. The MTP conceptualization of teacher performance and professional development explicitly links effective implementation of curricula to teachers’ skillful use of instructional interactions, and the MTP program provides professional development supports that are embedded in teachers’ everyday interactions with children (e.g., Burchinal et al., 2000; Howes et al., 2008; Hyson & Biggar, 2005; NICHD ECCRN, 2002). More specifically, MTP is a web-based professional development program for pre-k teachers comprising three types of resources designed to improve the quality of teacher-child interactions and children’s language and literacy development. The first component is a set of language and literacy activities, whereas the second and third components are resources developed specifically to support teachers’ effective implementation of these activities. Next, we review the literature that informed the development of each MTP component and provide a brief description of each professional development resource.

Language and Literacy Activities

An evidence-based set of language and literacy activities were designed to focus on “high-priority” instructional targets in preschool language and literacy and effective approaches to translating these instructional targets into high-quality, sustainable classroom instruction. The resulting product, MyTeachingPartner-Language & Literacy (MTP-LL; Justice, Pullen, Hall, & Pianta, 2003), includes explicit, targeted instructional activities encompassing six language and literacy domains. MTP-LL activities also include design features that reflect the current literature on how instructional targets can be embedded within high-quality, sustainable classroom instruction, including explicitness (e.g., Justice & Ezell, 2000, 2002), intensity and repetition (Elley, 1989; Justice, Chow, Capellini, Flanigan, & Colton, 2003; Nash & Donaldson, 2005; Penno et al., 2002; Robbins & Ehri, 1994; Sénéchal, 1997), and storybooks as a medium of instruction for making language and literacy instruction meaningful and naturalistic (e.g., Elley, 1989; Penno et al., 2002).

A high priority target for preschool language and literacy instruction is one that (a) is consistently and at least moderately linked to school-age reading and academic achievement, (b) is amenable to change through intervention, and (c) is likely to be under-developed among at-risk pupils (Lonigan, 2004). The six targets for MTP-LL were based on the conclusions of several meta-analyses (e.g., Hammill, 2004; National Early Literacy Panel [NELP], 2004) and longitudinal studies identifying specific early language and literacy skills predictive of later reading success (e.g., Bryant, MacLean, & Bradley, 1990; Catts, Fey, Zhang, & Tomblin, 2001; Chaney, 1998; Christensen, 1997; Gallagher, Frith, & Snowling, 2000; Schatschneider, Fletcher, Francis, Carlson, & Foorman, 2004; Storch & Whitehurst, 2002). The first three targets (phonological awareness, alphabet knowledge, print awareness) are “code-based” literacy skills that consistently predict school-age decoding (Storch & Whitehurst, 2002), are amenable to change via classroom-based interventions (e.g., Justice & Ezell, 2002; Ukrainetz, Cooney, Dyer, Kysar, & Harris, 2000; van Kleeck, Gillam, & McFadden, 1998; Whitehurst et al., 1994) and are comparatively underdeveloped in at-risk pupils (e.g., Bowey, 1995; Lonigan, Anthony, Bloomfield, Dyer, & Samwel, 1999; Snowling, Gallagher, & Frith, 2003). The latter three targets (vocabulary/linguistic concepts, narrative, social communication/pragmatics) are “meaning-based” skills also associated with later academic success, with particular associations to outcomes in reading comprehension for the former two (Pankratz, Plante, Vance, & Insalaco, 2007; Storch & Whitehurst, 2002) and social/behavioral competencies for the latter (e.g., Rimm-Kaufman, Pianta, & Cox, 2002). All three aspects of early development tend to be areas of relative disadvantage for children reared in poverty (e.g., Fazio, 1994; Justice, Meier, & Walpole, 2005) that can be readily improved through targeted interventions (Lonigan et al., 1999; Penno et al., 2002; Reese & Cox, 1999; Whitehurst et al., 1988). By addressing all six targets within a single curriculum, the MTP-LL offered a relatively more comprehensive approach to early language and literacy instruction than is typical (e.g., Aram & Biron, 2004; Byrne & Fielding-Barnsley, 1991; van Kleeck et al., 1998).

Professional Development Supports

Just as the language and literacy activities were designed with evidence-based child targets in mind, the professional development supports were designed based on evidence about the specific types of teacher-child interactions that are associated with positive developmental outcomes for young children (Howes et al., 2008; Mashburn et al., 2008). This focus on teacher-child interactions is consistent with new policy statements related to professional development and career development being suggested by the National Association for the Education of Young Children. These policy statements reflect a wide range of studies that have evaluated various features of program quality in relation to child outcome gains and found consistent evidence for the unique and significant role of high quality teacher-child interactions in producing children’s learning gains during preschool (Howes et al., 2008; Mashburn et al., 2008) and into kindergarten (Burchinal et al., 2008).

However, descriptive studies of classroom interactions in preschools demonstrate that the quality of instructionally supportive interactions present in most pre-k settings are quite low (LoCasale-Crouch et al., 2007; Justice, Mashburn, Hamre, & Pianta, 2008; Mashburn et al., 2008; Pianta et al., 2005), and it is the instructional domain of teachers’ classroom interactions (e.g., stimulation of conceptual development, provision of feedback) that is most consistently and strongly related to growth in children’s language, literacy, and math skills (Howes et al., 2008; Mashburn et al., 2008). Thus, the MyTeachingPartner professional development program explicitly targets improvement in teachers’ instructional, emotional, and management-focused interactions with children. More specifically, MyTeachingPartner provides extensive opportunities for teachers to engage in (a) observation of high quality instruction through analysis and viewing of multiple video examples; (b) skills training in identifying/appropriately responding to children’s cues, and how these contribute to language and literacy skills; and (c) repeated opportunities for individualized feedback and support for high-quality instruction, implementation, and interactions with children. There were two types of professional development supports focused on improving the quality of teacher-child interactions: access to a website and teaching consultation.

Teachers received access to the MTP website, which offers several resources to teachers to promote high quality interactions. First, the MTP website provided detailed descriptions of the following ten dimensions of high quality teacher-child interactions that are theoretically derived (Hamre & Pianta, 2007), empirically validated (Hamre, Pianta, Mashburn, & Downer, 2007), and predictive of children’s language, literacy, and social-emotional growth in the classroom (Howes et al., 2008; Mashburn et al., 2008): Positive Climate, Teacher Sensitivity, Regard for Students’ Perspectives, Behavior Management, Productivity, Instructional Learning Formats, Concept Development, Quality of Feedback, Language Modeling, and Literacy Focus. In addition to detailed descriptions, the MTP website also includes a video library with numerous annotated video examples of teachers demonstrating each dimension within their classrooms, which helps teachers become critical observers of classroom behavior and more attuned to the effects that teachers’ behavior have on children. Importantly, these videos portrayed high quality interactions during the implementation of the MTP-LL activities, so the videos provided real-time supports for teachers preparing to implement language and literacy lessons.

The second, and more intensive, form of professional development support was the MTP Consultancy. Consistent with the literature on adult learning (e.g., Abdal-Haqq, 1995; Darling-Hammond & McLaughlin, 1995; Putnam & Barko, 2000) as well as recent trends in teacher education (e.g., Darling-Hammond & McLaughlin, 1995; Lieberman, 1995) and professional development in early childhood education (Landry et al., 2006; Powell et al., 2010), the MTP Consultancy was designed to provide teachers with ongoing, practice-focused support and feedback regarding their interactions with children. Consultation has been used for some time in education settings, and rigorous evaluations show that to be effective, consultation has to (a) target a specific problem with a validated link to a desired outcome, (b) use procedures implemented in a standardized fashion in relation to a protocol, and (c) rely on a common understanding of the focus/target on the part of the consultant and client (Caplan & Caplan, 1993; Knotek & Sandoval, 2003; Landry et al., 2006; Powell et al., 2010; Rosenfield & Gravois, 1996). The MTP Consultancy was designed with these principles in mind.

The MTP Consultancy provides one-on-one teacher consultation, via the web, that focuses on observed implementation of instructional activities and feedback on interactions with children, as well as builds teachers’ skills to observe their own interactions and practices. The consultancy process occurs in the following four steps. First, a teacher uses a digital video camera to record video of her class for 30 min. Second, the teacher sends the digital video cassette to her consultant, who watches the video, selects clips for the teacher to review, and posts the clips on a secure website. Third, the teacher reviews the clips and answers reflective questions about her teaching practice in an on-line journal. Finally, the teacher and consultant participate in a meeting for 30 min via videoconference to discuss teaching practices and determine future goals. This four step cycle is repeated every two weeks. Importantly, the content of the consultation focused explicitly on teachers’ interactions with young children as they implemented language and literacy activities, using a validated observational framework (Pianta, La Paro, & Hamre, 2008).

Present Study

An experimental study was conducted within pre-k classrooms during 2004–2005 and 2005–2006 to evaluate the impacts of MyTeachingPartner on improving the quality of classroom interactions and children’s development. Teachers were randomly assigned to one of three study conditions: one group received the language and literacy activities only; the second group received the language and literacy activities and access to the video library (MTP Video Library condition); and the third group received the language and literacy activities, access to the video library, and participated in the consultancy (MTP Consultancy condition). An initial study of MTP examined the impacts of the teaching consultation on improving the quality of teachers’ interactions with children by comparing changes in the quality of classroom interactions between teachers who participated in the MTP Video Library condition and the MTP Consultancy condition (Pianta et al., 2008). Results indicated that teachers assigned to the MTP Consultancy condition had more positive growth on all dimensions of teacher-child interactions compared to teachers assigned to the MTP Video Library condition. For three dimensions of interaction quality—Teacher Sensitivity, Instructional Learning Formats, and Language Modeling—the rates of change were significantly more positive for MTP Consultancy teachers compared to MTP Video Library teachers.

In this study, we extend this test of MyTeachingPartner to examine its impacts on children’s language and literacy development during the pre-k year. Specifically, the purposes of the present study were to (a) compare the development of language and literacy skills for children whose teachers were randomly assigned to the MTP Consultancy condition and to the MTP Video Library condition, and (b) estimate the extent to which use of three program components—the MTP language and literacy activities, the MTP video library, and the MTP consultation—by teachers’ in the MTP Consultancy condition was positively associated with children’s development of language and literacy skills during pre-k. The first question is the “intent-to-treat“ analysis that examines the impacts of the MTP consultation on children’s language and literacy development during pre-k. The second question is the “treatment-on-treated” analysis that involves only teachers who were randomly assigned to participate in the MTP Consultancy (n=65) to identify the specific intervention components that are the “active ingredients” in producing changes in child outcomes. This latter analysis is quasi-experimental, because teachers were not randomly assigned to different levels of use of the three program components; thus, it is important to note that results from this analysis do not infer causal impacts of each program component on children’s development. The two central research questions that were addressed are:

1. Do children whose teachers were randomly assigned to participate in the MTP Consultancy condition achieve greater rates of development of language and literacy skills during pre-k compared to children whose teachers were randomly assigned to participate in the MTP Video Library condition?

2. To what extent is variation in use of three MyTeachingPartner components—language and literacy activities, web-based resources, and consultation—by teachers in the MTP Consultancy condition associated with children’s language and literacy development during pre-k?

We hypothesize that children whose teachers participated in the MTP Consultancy condition will achieve greater rates of language and literacy development during pre-k than children who teachers participated in the MTP Video Library condition. Further, we hypothesize that for children whose teachers participated in the MTP Consultancy condition, greater use of each of the three MTP intervention components is positively associated children’s development of language and literacy skills during the pre-k year.

METHOD

Participants

This study included 134 pre-k teachers who participated for two years in the MyTeachingPartner professional development program. All participating teachers and classrooms were a part of a state-funded pre-k program within a single state that serves children who meet “at-risk” status determined by the following criteria: a) poverty; b) homelessness; c) parents or guardians are school dropouts, have limited education, or are chronically ill; d) family stress as evidenced by poverty, episodes of violence, crime, underemployment, unemployment, homelessness, incarceration, or family instability; e) developmental problems; or f) limited English proficiency.

A total of 182 teachers were originally selected to participate in the study; however, 29 dropped out of the study during the first year and 19 dropped out of the study during the second year, resulting in an attrition rate of 16% during year 1 and 12% during year 2. To estimate potential attrition bias, analyses were conducted comparing teacher and classroom characteristics for the 134 teachers who fully participated and the 48 teachers who did not fully participate. Results indicate that there were no statistically significant differences between the two groups of teachers with regard to the percentage who had an advanced degree (χ²=0.35, p=.56), the percentage with training in the field of early childhood education (χ²=0.19, p=.67), or the number of years of experience teaching pre-k (t=−1.76, p=.08). In addition, there were no statistically significant differences in the classroom composition of teachers who did and did not fully participate with regard to the percentages of children who were poor (t=0.46, p=.65), mean maternal education levels of children in the classrooms (t=−0.42, p=.67), or children’s mean pretest scores for four language or literacy assessments: receptive vocabulary (t=−0.58, p=.57), print awareness (t=0.38, p=.70), the elision sub-test (t=−1.36, p=.18), and the emergent literacy composite (t=0.15, p=.88). There were, however, significant differences in classroom compositions for teachers who did and did not fully participate with regard to the percentage of children with limited English proficiency (t=−2.51, p=.01) and the mean pretest scores on the blending sounds sub-test (t=2.94, p=.00). Specifically, compared to classrooms of teachers who did not participate, teachers who did participate had a higher proportion of children enrolled in their classrooms with limited English proficiency and lower pretest scores on the blending sub-test. Characteristics of teachers and classrooms that participated in the study are presented in Table 1.

TABLE 1.

Teacher and Classroom Characteristics (n=134) and Use of Intervention Components by MTP Consultancy Teachers (n=65)

	n	%	Missing	Mean	Sd
Classroom characteristics
Mean pretest receptive vocabulary	133		1	30.40	1.98
Mean pretest blending sounds	132		2	6.31	1.47
Mean pretest elision	132		2	4.43	1.34
Mean pretest print awareness	132		2	16.70	3.86
Mean pretest emergent literacy composite	126		6	34.70	9.07
Proportion non-English	134		0	14.00	26.00
Proportion poor	134		0	69.00	23.00
Mean maternal education (years)	134		0	12.70	0.86
Teacher characteristics
Level of education	133		1
Bachelor’s degree	86	65
Advanced degree	47	35
Field of study	132		2
Early childhood education	52	39
Other	80	61
Years teaching pre-k	132		2	9.75	7.94
Study condition
MTP consultancy	65	49	0
MTP video library	69	51	0
Use of intervention components (n=65)
MTP-LL activities use (hr/week)	65		0	1.53	0.83
MTP video library (total hr)	65		0	15.50	21.10
MTP consultancy (total hr)	65		0	19.60	10.90

During fall of each of the two study years, four to five children whose parents provided consent to allow their child to participate, met the age criteria for kindergarten eligibility the following year, and spoke English well enough to understand simple instructions were randomly selected from each classroom for participation in ongoing measurements. In instances in which a selected child was no longer enrolled in the classroom at the end of the year, a new child from the classroom was randomly selected to participate. The resulting number of children who participated in the study was 1,165; an average of 8.7 children per teacher over the course of the two study years. Demographic characteristics of children who participated in the study are presented in Table 2.

TABLE 2.

Child and Family Characteristics and Child Fall and Spring Language and Literacy Skills (n=1165)

	n	%	Missing	Mean	Sd
Child and family characteristics
Gender			49
Boy	570	51
Girl	546	49
Language(s) spoken at home			45
English only	910	81
Other language	210	19
Maternal education (years)	1104		61	12.70	2.05
Family income			155
Poverty	695	69
Not poverty	315	31
Year of intervention			0
Year 1	600	52
Year 2	565	49
Language and literacy assessments
Fall receptive vocabulary	1066		99	30.40	3.99
Spring receptive vocabulary	1022		143	34.10	3.41
Fall blending sounds	1052		113	6.31	3.07
Spring blending sounds	1014		151	8.29	2.88
Fall elision	1039		126	4.40	3.21
Spring elision	1006		159	7.43	3.41
Fall print awareness	1056		109	16.70	9.03
Spring print awareness	1019		146	28.90	7.60
Fall emergent literacy composite	854		311	34.10	16.60
Spring emergent literacy composite	913		252	60.20	12.60

Procedures

Participants were recruited for this study through several steps. First, districts participating in the state-funded pre-k program were selected for recruitment based on having more than one pre-k classroom; many districts in the state supported only one pre-k classroom and therefore were not recruited. Invitation letters were then sent to selected district-level coordinators that described the study and the MTP professional development program and indicated that the study team was interested in recruiting teachers in each district’s program to enroll in the study. Following the initial mailing, a meeting was held with the subset of interested district coordinators to describe the study in more detail and the procedures for recruitment of teachers. Forty-one district coordinators agreed to facilitate recruitment of teachers; including three of the four largest districts in the state and more than one-third of all the districts in which the state-funded program was operating. District coordinators provided the study team with contact information for pre-k program teachers in their district, and they co-signed a letter of recruitment, indicating the district’s permission for teachers to enroll in the study. Teachers received an individual letter inviting them to participate in the study.

The experimental evaluation was carried out using a hierarchical design in which each of the 41 participating districts was randomly assigned to one of the three study conditions, and all teachers within a district participated in the same study condition. These assignments by district were first stratified by district size (large, medium, and small) and then randomly assigned by size to condition. For the purposes of the present study, teachers from 16 districts who were assigned to participate in the study condition that received only the language and literacy activities were excluded, because they did not receive access to either of the two intervention resources designed to promote effective implementation of the activities (website, consultation). As a result, the 134 teachers participating in this study came from 25 districts.

Random assignment was conducted at the district level for two reasons. First, there was concern about contamination of intervention effects across conditions if teachers within the same district (often in the same building) were enrolled in different study conditions. For example, teachers participating in the teaching consultation would be exposed to more detailed descriptions of effective practices and receive direct facilitation of their use of the video exemplars on the website, which they could share with teachers who did not participate in the consultation. This would potentially reduce the key distinction between these conditions. Second, in recruitment meetings with district coordinators, they expressed a preference that all teachers in their program receive the same professional development opportunities for the sake of perceived equity and equal opportunities for teachers to participate in the intervention. The decision to conduct random assignment at the district level precludes causal inferences at the teacher level.

After enrollment in the study (in May 2004), teachers and district coordinators received a series of letters to inform them about the study and to describe the activities in which they would be engaged during the subsequent academic year. In fall of 2004, prior to the start of the school year, teachers in each district attended a training and introductory workshop held at a convenient location. At the workshop, teachers were oriented to the purpose of the study, trained in the intervention to which they were assigned, and informed of a set of data-collection requirements. They also received a laptop computer to ensure equal access to the web-based resources. Over the course of the year, all teachers received a generic series of MTP newsletters, reminders, and updates.

Measures

Child and Family Characteristics

Parents of each child in the study completed a brief family questionnaire that assessed the following child and family characteristics: gender, language spoken in the home, and years of maternal education. In addition, household income and the number of children and adults living in the household were used to calculate an income-to-needs ratio. A family was defined as “in poverty” using 150% of the Federal poverty guidelines as the threshold for the income-to-needs ratio (see Table 2). Study year was also included to identify if the child participated in the first year or the second year of the intervention.

Language and Emergent Literacy Assessments

The study included implementation of subtests from two direct assessments of children’s language and literacy skills: the Phonological Awareness and Literacy Screening-PreKindergarten (PALS; Invernizzi, Sullivan, & Meier, 2002) and the Preschool Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP; Lonigan, Wagner, Torgesen, & Rashotte, 2002). The PALS-PreK is a criterion-referenced, broad-based assessment of children’s emergent literacy skills that includes measurement of phonological awareness (rhyme, nursery rhyme, and beginning sounds subtests), print knowledge (alphabet knowledge and print and word awareness), emergent writing (name writing). It is designed for use with children who are 3- to 5-years of age. Raw scores for each of the individual subtests are summed to create an Emergent Literacy Composite score, which was used for analyses in the present study. Psychometric qualities include acceptable levels of test-retest and inter-rater reliability, and concurrent validity (Invernizzi, 2000).

The Pre-CTOPPP (Lonigan et al., 2002) was designed for use with children from 3 to 5 years of age and is a precursor to the slightly revised and more recently published Test of Preschool Early Literacy (TOPEL; Lonigan, Wagner, Torgesen, & Rashotte, 2007). The Pre-CTOPPP provides an assessment of blending sounds, elision, print awareness, and receptive vocabulary. Standardized scores are not available for this test, as national norms for these versions of the subtests are unavailable; therefore, raw scores are reported and used in analyses. The Blending subtest includes items that measure whether children can blend initial phonemes into one-syllable words, initial syllables into two-syllable words, and ending phonemes into one-syllable words. The Elision subtest measures whether children can break apart initial and ending phonemes, as well as initial syllables, from one- and two-syllable words. Print Awareness items measure whether children recognize individual letters and letter-sound correspondences, and whether they differentiate words in print from pictures and other symbols. Receptive Vocabulary items measure children’s single-word vocabulary knowledge. The Pre-CTOPPP subtests have shown adequate internal consistency, test-retest reliability, and concurrent validity in past research by the test developers and in several large, federally funded studies, including the Head Start Impact Study, IES Even Start Classroom Literacy Interventions and Outcomes Study, IES Pre-school Curriculum Evaluation Research Study, and IES Early Reading First National Evaluation (Lonigan, McDowell, & Phillips, 2004).

Teachers in this study administered the PALS-PreK and Pre-CTOPPP assessments to their pupils enrolled in the study. Small-scale pilot tests have demonstrated adequate reliability and validity of data collected using assessments administered by teachers in Head Start classrooms. In addition, most teachers participating in the study had administered PALS-PreK to children in their classrooms in the past and were very familiar with standardized testing procedures. At the beginning of the project, all teachers completed training focused on administration of the language and literacy battery, and fidelity of administration was randomly checked via videotape for 20% of teachers in the fall of year 1. Teachers accurately administered standardized items over 90% of the time and reported that for 96% of the assessments children’s performances were “most typical” or “very typical” of their usual classroom functioning. Children’s mean scores on these measures are presented in Table 2.

Classroom and Teacher Characteristics

Within a single classroom, characteristics of children who attended were highly correlated for year 1 and year 2, because these classrooms served the same geographic and demographic populations across the two study years. Thus, classroom characteristics were computed as averages across the two study years. A class-level measure of mean pre-test scores was computed as the average score on each fall assessment for all children who participated from each classroom in either of the two study years. Within each classroom during each study year, parents of 75% of children completed the family questionnaire. The income-to-needs ratio for each child was computed, poverty status was assigned using 150% of the Federal poverty guidelines as the threshold, and a class-level measure of poverty was computed as the proportion of children (ranging from 0 to 1) within each class who met this criterion across the two study years. The percentage of children in each classroom with limited English proficiency (LEP) was determined from teachers’ reports of the number of children with LEP divided by the total number of children enrolled in the classroom. This proportion (ranging from 0 to 1) was averaged across the two study years as the class-level measure of limited English proficiency. From the family questionnaires, respondents reported the mother’s number of years of education, and a class-level measure of maternal education was computed as the average within each class across the two study years.

Characteristics of teachers participating in the study were collected from a teacher demographic questionnaire. Teachers reported their level of education (Bachelor’s degree or Advanced degree), the field of study in which they received their degree (Early Childhood Education or another field), and the number of years of experience teaching pre-k. Table 1 presents descriptive statistics for classrooms and teachers.

Teachers’ Use MyTeachingPartner Intervention Components

There are three components to the MTP intervention accessible to teachers in the MTP Consultancy condition—language and literacy activities, the MTP video library, and the MTP consultation—and the extent to teachers used each was assessed. Use of the MTP language and literacy activities was reported by teachers on a survey completed at the end of each study year. Specifically, teachers reported the number of days per week and the number of hours per day they used MTP activities, from which the number of hours per week of activity use was computed. Use of these activities by a teacher was highly correlated in year 1 and year 2; thus, usage was computed as the average hours per week across the two study years. On average, teachers used the MTP activities for 1.5 hr per week. The standard deviation was 0.83 hr, and the range was .25 hr to 5 hr per week.

Use of the MTP video library was measured using a web-tracking system that documented the amount of time each teacher spent on each web-page. To reduce the likelihood of over estimating web-use by teachers who stayed logged on to the website but were not actively using its resources, the maximum length of time for each page visit was truncated to 15 min. From this information, we assessed the total number of hours teachers spent on three portions of the website containing content designed to improve teachers’ interactions with children and implementation of language and literacy activities (language and literacy activity descriptions, quality teaching descriptions, and videos depicting high quality interactions), excluding time spent on pages without content (e.g., log-in, welcome, log-out). Although all teachers had access to these pages, there was wide variability in the extent to which teachers utilized these resources. The average amount of time teachers cumulatively spent on the website during the two study years was 15 hr and 31 min, the standard deviation was 21 hr and 7 min, and the range was 0 hr to 138 hr.

Participation in the MTP Consultancy was determined by computing the total number of consultancy hours teachers spent involved with two parts of the consultancy process. The first part was the amount of time teachers spent reviewing their video clips posted to the website by their consultant and answering reflective questions about the clips in an on-line journal. The web-tracking system documented the total amount of time teachers spent on these portions of the website, and the maximum length of time for each video page was truncated to 60 min. The second component was the amount of time teachers participated in videoconferences with their consultant to discuss teaching practices and determine future goals. The total number of min the teacher participated in video conferences was computed by multiplying the total number of conferences for each teacher by the average amount of time each conference lasted, which was documented by the consultant at the conclusion of each conference. The total number of hours teachers used the consultancy was computed by summing the number of hours over the course of the two year study that teachers viewed video clips and responded in their on-line journal, and participated in teleconferences. There was substantial variability in the number of hours that the two components of the consultancy process were utilized. Overall, the mean number of total hours of use for this group was 19.5 hr, with a standard deviation of 10.8 hr.

Analyses

The structure of the data resulting from the study’s design included four potential levels at which data can be analyzed: child, classroom, teacher, and district. There were two classrooms per teacher (one per year), and we removed classroom as a level of analysis and aggregated classroom data across the two study years to create teacher level variables. As a result, the data structure included both cohorts of study children nested within each teacher (approximately 8 children per teacher), and a “study year” variable was added as a child-level predictor to account for any differences in children’s language and literacy associated with whether the child participated in year 1 or year 2 of the study. We then explored the possibility of running three-level models that included child, teacher, and district levels; however, the results of unconditional multi-level models indicated non-significant amounts of between-district variance. As a result, we removed district as a level of analysis and examined impacts of the teaching consultation at the teacher level by comparing rates of development of children whose teachers were in the MTP Consultancy and MTP Video Library conditions. Because randomization to the study condition occurred at the district-level and this intent-to-treat analysis was conducted at the teacher level, results that compare rates of children’s development between teachers in the MTP Consultancy and MTP Video Library conditions may be confounded by between-district differences. This precludes causal interpretation of results that compare children’s development across the two study conditions. In the treatment-on-treated analysis, teachers’ use of each of the three intervention components varied across teachers, and it is appropriate to specify these variables as predictors at the teacher-level of analysis.

The resulting data structure for the analysis involved an average of 8.7 children in each of the 134 pre-k teacher’s classes, and we conducted 2-level hierarchical linear modeling (HLM; Raudenbush & Bryk, 2002) to account for the nesting of children within teachers. To address the first research question, we examined differences in children’s spring language and literacy skills between children who attended classrooms with teachers who participated in the MTP Consultancy condition and the MTP Video Library condition, controlling for fall language or literacy skills, child characteristics (level-1) and teacher and classroom characteristics (level-2). The level-1 model (Equation 1) specifies that children’s spring language or literacy assessment is a function of their fall pretest score, gender (boy=1, girl=0), language spoken in the home (non-English=1, English=0), years of maternal education, poverty status (yes=1, no=0), and the year the child participated in the study (year 2=1, year 1=0).

$${\text{Y}}_{\text{ij}}\text{=}{\beta }_{\text{0j}}\text{+}{\beta }_{\text{pj}}\text{(pretest)+}{\beta }_{\text{cj}}\text{(child demographics)+}{\beta }_{\text{yj}}{\text{(study year)+r}}_{\text{ij}}$$ [1]

In the level-2 model, classroom characteristics (mean pre-test scores, proportion in poverty, proportion non-English) and teacher characteristics (advanced degree, major in ECE, years teaching PK) were entered, along with the teachers’ study condition (MTP Consultancy=1, MTP Video Library=0). The magnitude and direction of the coefficient (γ_0c) indicates the associations between children’s development of language and literacy skills during pre-k and whether their teacher participated in the MTP Consultancy condition.

$${\beta }_{\text{0j}}={\gamma }_{00}+{\gamma }_{0c}(\text{classroom})+{\gamma }_{0t}(\text{teacher})+{\gamma }_{0c}(\text{MTP Consultancy})+{\text{u}}_{\text{0j}}$$ [2]

The second research question examined the extent to which use of MTP intervention components by teachers in the MTP Consultancy condition was uniquely associated with children’s development of language and literacy skills during pre-k, controlling for fall language or literacy skills, child characteristics (level-1), and teacher and classroom characteristics (level-2). In the level-2 model, a more parsimonious set of classroom and teacher characteristics (mean pre-test scores, proportion in poverty, proportion non-English, and years teaching PK) were entered as covariates, because fewer teachers (n=65) participated in the MTP Consultancy condition and were included in this analysis. In addition, the following measures of teachers’ use of the MTP components were included: number of hours per week teachers implemented the MTP-LL activities; number of cumulative hours teachers spent on the MTP website, and the number of cumulative hours teachers spent participating in the MTP consultation. The magnitude and direction of the coefficients (γ_0c) indicate associations between children’s spring language and literacy skills controlling for prior skills at pre-k entry, and teachers’ use of language and literacy activities, use of MTP website, and participation in MTP Consultation

$${\beta }_{\text{0j}}={\gamma }_{00}+{\gamma }_{0c}(\text{classroom})+{\gamma }_{0t}(\text{teacher})+{\gamma }_{0c}(\text{MTP Components})+{\text{u}}_{\text{0j}}$$ [3]

All analyses were conducted using the Mixed procedure in SAS (Singer, 1998), and missing data were estimated using multiple imputation procedures that created 20 complete data files (refer to Tables 1 and 2 for a summary of amount of missing data). The multi-level analyses were conducted for each of the 20 imputed data files, and coefficients and standard errors resulting from each analysis were aggregated.

RESULTS

Results of the two level hierarchical linear model that address research question 1 are presented in Table 3, which includes unstandardized coefficients (B) and standard errors (SE) that identify the magnitudes and directions of the associations between each level 1 and level 2 predictor and the five measures of language or literacy development. The pretest scores on each of the five assessments were positively associated with the corresponding assessment score in fall. For two of the literacy measures (elision and print awareness) boys’ spring scores were significantly lower than girls’, after accounting for pretest scores and other demographic characteristics. Maternal education was positively associated with spring receptive vocabulary, blending, and elision, and children who participated in the second year of the study achieved higher scores in spring on receptive vocabulary, blending, and emergent literacy composite scores.

TABLE 3.

MTP Consultancy and Children’s Development of Language and Literacy Skills During Pre-k

	Receptive Vocabulary		Blending Sounds		Elision		Print Awareness		Emergent Literacy Composite
	B	SE	B	SE	B	SE	B	SE	B	SE
Level-1
Child characteristics
Pre-test	0.46***	0.02	0.31***	0.03	0.54***	0.03	0.47***	0.02	0.49***	0.03
Boy	−0.05	0.17	−0.32	0.17	−0.54**	0.17	−1.20**	0.36	−1.20	0.68
Non-English	−0.66	0.36	−0.00	0.32	−0.16	0.36	1.06	0.72	1.05	1.35
Maternal education (years)	0.11*	0.05	0.11*	0.05	0.19**	0.06	0.14	0.10	0.07	0.18
Poverty	−0.13	0.22	−0.35	0.22	0.04	0.22	−0.09	0.43	−0.43	0.83
Second year of intervention	0.54***	0.16	0.33*	0.16	0.26	0.19	0.41	0.34	1.45**	0.56
Level-2
Classroom characteristics
Mean pre-test score	0.16*	0.06	0.16	0.08	0.01	0.09	0.14	0.08	−0.10	0.06
Proportion in poverty	0.19	0.60	0.04	0.68	0.58	0.61	1.22	1.66	2.12	2.77
Proportion non-English	−0.19	0.55	−0.54	0.59	−0.32	0.58	−3.53*	1.44	−9.25***	2.62
Mean mother’s education	−0.03	0.16	0.14	0.18	0.27	0.16	0.06	0.44	0.17	0.77
Teacher characteristics
Advanced degree	0.04	0.21	−0.17	0.24	−0.18	0.23	−0.37	0.59	−0.16	1.01
Major in ECE	0.37	0.21	0.30	0.23	0.25	0.21	0.80	0.60	2.74**	0.99
Years teaching pre-k	−0.00	0.01	0.01	0.01	0.01	0.01	0.02	0.04	0.09	0.06
Intervention condition
Consultancy (1)/video library (0)	0.50*	0.23	0.15	0.27	−0.02	0.24	−0.05	0.65	0.79	1.14

^*p ≤ .05;

^**p ≤ .01;

^***p ≤ .001.

There were few significant associations between the teacher characteristics included in the level-2 model and children’s spring language and literacy achievement. Teachers whose children, on average, achieved higher pretest scores on receptive vocabulary achieved significantly higher spring receptive vocabulary scores, after accounting for the associations with classroom and teacher characteristics and the MTP study condition. For the spring print awareness and emergent literacy composite scores, there were significantly higher scores in classrooms with a greater proportion of children who spoke English as a native language and in classrooms with teachers who received their degree in early childhood education. The level-2 model also included intervention condition, and there was a significant difference in receptive language scores between children whose teachers were in the MTP Consultancy condition and in the MTP Video Library condition. Specifically, controlling for their fall language assessment and child, teacher, and classroom characteristics, children with teachers in the MTP Consultancy condition scored .50 points higher on the spring receptive language assessment than children with teachers in the MTP Video Library. The size of this effect was computed by dividing the magnitude of the difference between the two groups (represented by the B) by the pooled between-classroom standard deviation for spring post-test scores on the measure of receptive language (sd=1.82). Results indicate that the magnitude of the difference in children’s spring language scores between MTP Consultancy and MTP Video Library teachers was d=0.27.

Table 4 presents results of analyses that address the second research question that examined associations between MTP Consultancy teachers’ use of the MTP-LL activities, use of the MTP website, and participation in the MTP consultation and children’s development of language and literacy skills during pre-k. Results indicated a significant, positive association between the number of hours teachers participated in the MTP consultation and children’s development of receptive language skills. Specifically, a one-standard deviation (10.9 hr) difference in the number of hours teachers participated in the MTP consultation was associated with a 0.55 difference in children’s spring receptive vocabulary; an effect size of d=.30.

TABLE 4.

Associations Between Teachers’ Use of the MyTeachingPartner Language and Literacy Activities, Website, and Consultancy, and Children’s Development of Language and Literacy Skills During Pre-k

	Receptive Vocabulary		Blending Sounds		Elision		Print Awareness		Emergent Literacy Composite
	B	SE	B	SE	B	SE	B	SE	B	SE
Level 1
Child characteristics
Pre-test	0.47***	0.03	0.29***	0.05	0.50***	0.04	0.51***	0.03	0.52***	0.03
Boy	0.01	0.23	−0.33	0.24	−0.33	0.25	−1.28*	0.50	−0.49	0.88
Non-English	−0.82*	0.39	−0.69	0.43	−0.38	0.45	1.12	0.92	1.79	1.69
Maternal education (years)	0.15**	0.05	0.13*	0.07	0.23***	0.07	0.02	0.13	0.08	0.27
Poverty	−0.39	0.30	−0.37	0.31	0.14	0.35	−0.23	0.71	−0.01	1.16
Second year of intervention	0.78***	0.22	0.38	0.23	0.78**	0.26	0.63	0.48	1.78*	0.87
Level 2
Classroom/teacher characteristics
Mean pre-test score	0.14	0.08	0.26*	0.12	0.05	0.12	0.30**	0.10	0.02	0.10
Proportion in poverty	1.22	0.70	−0.20	0.84	−1.09	0.79	0.02	1.96	4.04	3.71
Proportion non-English	−0.48	0.63	0.29	0.74	0.33	0.74	−3.70*	1.66	−9.97**	3.02
Years teaching PK	−0.02	0.02	−0.00	0.02	−0.01	0.02	−0.11*	0.05	−0.09	0.10
Intervention component
MTP-LL activities (hr/week)	0.37*	0.18	0.40*	0.21	0.31	0.20	0.65	0.50	1.84*	0.87
MTP website (total hr)	−0.01	0.01	−0.00	0.01	−0.01	0.01	−0.01	0.03	0.02	0.05
MTP consultancy (total hr)	0.05*	0.02	0.01	0.02	0.03	0.02	0.08	0.05	0.03	0.11

^*p ≤ .05;

^**p ≤ .01;

^***p ≤ .001.

In addition, implementation of the language and literacy activities for more hours was positively associated with children’s development of receptive vocabulary (B=.37), blending (B=.40) and emergent literacy skills (B=1.84). The pooled between-classroom standard deviation for spring scores on the measure of receptive vocabulary, blending and emergent literacy were 1.82, 1.59, and 7.59, respectively. A one-standard deviation (0.83 hr) difference in the number of hours teachers implemented MTP-LL activities each week was associated with a 0.30, 0.33, and 1.52 difference in spring for children’s spring receptive vocabulary, blending, and emergent literacy composite scores; an effect size of 0.16, 0.21, and 0.20, respectively.

DISCUSSION

Considering the proliferation of early childhood education programs in the United States and reports of less than ideal levels of quality in these classrooms (e.g., LoCasale-Crouch et al., 2007; Pianta et al., 2005), professional development supports for teachers are clearly needed to ensure that children’s experiences within pre-k classrooms are indeed promoting the language and literacy skills that they will need to be successful in later schooling. A previous study of the impacts of MyTeachingPartner—a suite of professional development resources designed to improve the quality of teacher-child interactions during language and literacy instruction—indicated that teachers who were randomly assigned to participate in individualized, web-mediated consultation had greater improvements in the quality of their interactions with children compared to teachers who did not participate in the teaching consultation (Pianta et al., 2008). Findings reported herein provide initial evidence suggesting that teachers who participated in the MTP Consultancy also had children who experienced greater rates of receptive vocabulary development during the pre-k year compared to children in classrooms with teachers who did not participate in the consultation. These initial reports for the positive impacts of MTP consultation on child outcomes are qualified both by the modest effect size and pertained to only one of five assessed outcomes.

Thus, these findings provide general support that early childhood professional development programs that expose teachers to resources that emphasize high quality interactions during implementation of language and literacy activities can have positive benefits for children’s early language skills, which are empirically linked to their later achievements in language and reading comprehension (NICHD Early Child Care Research Network [ECCRN], 2005; Storch & Whitehurst, 2002). The MTP teaching consultation approach is consistent with other professional development models that identify the following key principles of effective consultation (Caplan & Caplan, 1993; Knotek & Sandoval, 2003; Landry et al., 2006; Powell et al., 2010; Rosenfield & Gravois, 1996): provision of on-going support throughout the school year; a focus on the specific classroom practices and problems; implementation using a standardized protocol; and reliance on a common understanding of the focus/target on the part of the consultant and client, which in this case was a standardized observation of teacher-child interactions.

Results also indicated that among teachers who participated in the consultation, more hours of participating in the consultation activities was positively related to children’s development of receptive language skills, and not surprisingly, more hours teachers spent implementing the MTP language and literacy activities was positively related to children’s gains in language and early literacy skills. Thus, a key ingredient to the effectiveness of exposing teachers to professional development supports on children’s gains in literacy and/or language skills is promoting teachers use of the resources. At a broad level, this suggests the need to actively promote preschool teachers’ use of teaching resources, rather than passively making them available and expecting all teachers to make use of them equally. This finding is consistent with the other work on school-based prevention and curricular interventions showing the importance of attending to fidelity of implementation when initiating interventions in schools (e.g., Greenberg, Domitrovich, Graczyk, & Zins, 2005; O’Donnell, 2008). In the next section, we discuss the associations between variability in teachers’ use of each of the three professional development resources and children’s language and literacy development during the pre-k year.

Variability in Use of Professional Development Resources

For teachers participating in the MTP Consultancy condition of the MyTeachingPartner professional development program, three intervention components were available (language and literacy activities, video library resources, and individualized consultation), and there was considerable variability in teachers’ use of each of the resources. For example, teachers’ reports of how frequently they implemented MTP language and literacy activities over the course of two years ranged from an average of 15 min per week to 1 hr per day. The average number of hours teachers’ were logged on to the MTP website cumulatively over two years was 15.5 hr, but there was wide variability in its use. Six of the 65 participating teachers accessed the web resources for less than 1 hour total over the course of the two years that they were available, and two of these teachers never logged onto the website. In contrast, 13 teachers (20%) accessed the web resources for over 20 hr. In addition, the average amount of time that teachers participated in the MTP Consultancy was 19.6 hr, but the amount of time teachers participated widely varied. At one extreme, one teacher participated in a total of three meetings with her consultant that lasted an average of 13 min each, and she observed her videos for a total of 26 min. At the other extreme, one teacher participated in a total of 33 meetings with her consultant that lasted an average of 30 min each, and she observed her videos for a total of 47.6 hr. This variation in implementation was, as noted previously, a factor in the impacts of these supports on teacher and child outcomes. In further development of these resources presently being tested in controlled trials, modifications have been made to the video library to ease access and interpretation, and increased quantity and standardization of supports to consultants has been employed, including weekly conference calls and targeted plans for ensuring individual teachers’ participation. In addition, in the consultation condition, the web-based feedback to teachers on their own practice now regularly includes links to relevant pages on the video library.

Language and Literacy Activities

Teachers’ self-reported frequency of implementation of MTP language and literacy activities was significantly related to growth in children’s vocabulary, blending, and emergent literacy skills. In line with other recent research (Aram & Biron, 2004; Dickinson & Caswell, 2007; Justice & Ezell, 2002; Justice et al., 2005; Ukrainetz et al., 2000; van Kleeck et al., 1998), this suggests that intentional, explicit activities are an important element of preschool instruction when targeting specific domains of learning. It is unclear, however, why use of the MTP activities was not associated with children’s gains in elision and print awareness. Although the activities targeted all of these domains of literacy achievement, it is possible that teachers chose to implement activities that targeted vocabulary, blending, and emergent literacy skills or that these activities were offered with higher levels of quality than other types of activities. Greater detail about the amount and type of the specific language and literacy activities that teachers’ implemented would enable a more precise analysis of the effects of particular types of literacy and language activities on children’s development of these skills.

MTP Web Resources

The MTP Video Library was designed to provide teachers with support in their implementation of the MTP language and literacy activities through written descriptions and videos of high quality interactions between teachers and children in pre-k classrooms. These resources are intended to promote teachers’ knowledge of high quality interactions and abilities to observe high quality interactions; however, the previous study of the impacts of MTP did not find associations between greater use of the web resources and improvements in the quality of teacher-child interactions (Pianta et al., 2008). Results from this study indicated that greater use of these resources was not associated with children’s development of language and literacy skills either. There are a few considerations when interpreting the lack of significant associations between use of the MTP video library and changes in teacher interactions and children’s language and literacy development.

First, the video library was made available with no stipulations or requirements for how frequently it should be accessed or which sections to access, or methods to ensure that teachers were using the resources in the ways they were intended. Some consultants may have directed teachers to view particular web pages or videos and others may have not, and some teachers opted on their own to view different parts of the website, for different amounts of time, and with different levels of engagement. As a result, there was wide variability in the amount of time teachers spent on the website related to teacher interest and motivation and the degree to which these resources were emphasized by the consultant. Further work is need to examine the potential impacts of the web resources alone under conditions in which they were intended—frequent use, low variability of use, and high engagement—to examine the extent to which they effectively promote teachers’ knowledge, observation abilities, and interactions with children, and improvements in children’s language and literacy development.

It is also important to consider possible barriers to accessing the videos that led to the wide variability in teachers’ overall use of the MTP web-based resources. All teachers were provided a lap top computer to access the MTP website; however, computers were connected to the internet in different ways, ranging from dial-up service to high-speed cable connections, which impacted the speed in which the web-pages and videos were downloaded and accessible for viewing. A teacher with fast internet service may be exposed to more of the content of the MTP website in a shorter period of time; a teacher with slow internet service may spend more time on the MTP website but receive less exposure to its content. Thus, the amount of time teachers were logged on to the video library may not reflect the amount of exposure to the resources. More importantly, the amount of time teachers’ spent on the website provided no indication of the degree to which the teacher was engaged in its content. Further, slower downloading of the web content may lead to frustration by teachers, which potentially discouraged their subsequent use of these web resources. And, teachers’ level of comfort with using the internet and amount of experience with web-based learning activities may also pose barriers to the effectiveness of the website on changing teachers’ interactions and children’s development. Thus, further studies of the impacts of the web resources must consider factors such as the speed of downloading the content, the level at which teachers are actively engage with the content, and teachers’ comfort with the internet as a tool for learning and professional development.

MTP Consultancy

Greater number of hours participating in the MTP consultation process was found to be positively associated with children’s gains in receptive vocabulary. This is particularly interesting given that the previous study of the impacts of MTP on the quality of teacher-child interactions (Pianta et al., 2008) indicated positive change for teachers in the types of interactions with children that specifically scaffold the development of language (Teacher Sensitivity, Language Modeling) as a result of receiving consultation support. Together, these results suggest that consultation may be associated with children’s vocabulary development due to an increase in high-quality, language-rich teacher-child interactions (Girolametto & Weitzman, 2002; Landry et al., 2006; Powell et al., 2010), though this pattern of associations needs to be modeled in an experimental mediation analysis for confirmation of a causal link.

Effects of consultation on children’s academic performance in the early childhood field have been mixed, with some showing positive effects (e.g, Landry et al., 2006; Podhajski & Nathan, 2005; Powell et al., 2010) and others failing to show such effects (Jackson, Larzelere, & Clair, 2006). Evaluations of consultation interventions in mental health and school psychology show that consultation, to be effective, has to (a) target a specific problem with a validated link to a desired outcome, (b) use procedures implemented in a standardized fashion in relation to a protocol, and (c) rely on a common understanding of the focus/target on the part of the consultant and client (Caplan & Caplan, 1993; Knotek & Sandoval, 2003; Rosenfield & Gravois, 1996). The MTP Consultancy was quite explicit in each of these components. In particular, the MTP Consultancy model varies from other consultation models in that it explicitly targets changes in teacher-child interactions which have shown links to children’s academic development. Pianta et al. (2008) argued that focusing on specific teacher-child interactions/implementation as targets, anchoring these interaction targets in standardized, validated measurement systems such as the Classroom Assessment Scoring System (Pianta et al., 2008) or other structured observational tools (see Dickinson, St. Pierre, & Pettengill, 2004) and using standardized consultation procedures will lead to more systematic effects of consultation. This study provides initial support for that contention—at least as related to children’s vocabulary development.

The fact that more hours of participation in the consultation predicted positive changes in children’s language development may be a function of variation in the focus of the consultation during the bi-weekly review of video and feedback sessions. Although all consultation sessions were focused on teacher-child interactions, teachers and consultants co-determined what aspect of teacher-child interactions to work on throughout the year. It may be that language modeling interactions were more frequently addressed than interactions that might be more closely aligned with literacy outcomes. As we learn more about the specific types of teacher-child interactions that make a difference for young children, as well as how to help teachers change these practices, we can modify consultation interventions accordingly.

Limitations

Several limitations to these data and analyses must be considered when interpreting findings reported herein. First and foremost, these results are correlational and cannot be used to make inferences about causal mechanisms. For example, significant associations between use of the intervention resources and children’s development could be the result of selection, whereby better teachers chose to make more consistent use of the intervention components. Alternatively, classrooms comprising children who develop more rapidly may evoke teachers’ use of the available resources to better challenge their children. Despite these potential limitations, these analyses are crucial to developing our understanding of the specific intervention components that may be the most “active ingredients” and, therefore, worthy of further experimental attention or further development and refinement. Because the positive results of consultation were detected for only one of five directly-assessed child outcomes suggests the benefits of this approach to professional development should be interpreted cautiously and with a need for replication.

A second limitation worth noting is that teachers self-reported their rates of using the language and literacy activities retroactively, which could be influenced by social desirability and inaccurate recollections. Ideally, time spent implementing these language and literacy activities would be measured more objectively, through independent observation or through daily or weekly logs. Third, although these findings indicate that greater use of these intervention components may be beneficial to children, this study did not identify factors that may contribute to these usage rates. More research is needed to identify teacher, child, and institutional characteristics that may serve as supports and barriers to teachers’ efforts to make use of available teaching resources. Fourth, these findings are only applicable to the use of web-mediated remote consultation in which teachers and consultants were at different locations when they engaged in the consultation process, and they do not speak to the potential efficacy of other modes of conducting consultation. Although remote consultation offers a more flexible, lower cost, and potentially more scalable method of conducting consultation than traditional face-to-face approaches, questions remain about which consultation mode provides greater benefits for changing teachers’ interactions and children’s development.

A fifth limitation involves the generalizability of findings given that the current sample comprised relatively well-educated teachers in state-funded, center-based pre-k classrooms. Even though these classrooms were heavily populated with children from families below the poverty line, it is unclear whether use of these intervention components would have a similar effect on children’s language and literacy development for the full array of early childhood programs, including Head Start and child care, staffed by teachers with more diverse educational backgrounds and professional experiences. However, it is important to consider past findings that high quality implementation of language and literacy activities are rare in early childhood settings, regardless of teacher credentials and availability of curricula (Pianta et al., 2005). This would suggest that use of MTP components could be beneficial to a wider audience of pre-k teachers and classrooms, despite the limited sample in the current study. Finally, replication of these results in experimental studies, considerable refinement and strengthening of implementation supports, and effectiveness trials are needed before this approach should be considered for wider use.

CONCLUSION

Professional development resources for preschool teachers are a critical medium for providing supports that can ultimately translate into high quality learning experiences and development of foundational early language and literacy skills for young children. This study provides preliminary evidence for the efficacy of teacher consultation in improving children’s receptive language analysis. Follow-up “treatment-on-treated” analyses among teachers assigned to the consultation condition suggest that the impacts of two professional development resources—teaching consultation and language and literacy activities—on children’s development depended upon the frequency with which teachers used these resources. Thus, these findings suggest that exposing teachers to teaching consultation that is on-going, implemented using a standardized protocol, and focused on improving proving specific classroom interactions can improve children’s language outcomes. More importantly, these findings indicate that a critical ingredient for realizing the promise of professional development programs for improving teaching and learning is both increasing implementation fidelity and engaging and motivating teachers to make use of available professional development opportunities in the ways they are intended.