A Mixed-Methods Investigation Examining Site-level Variation in Reach Out and Read Implementation

Abstract

Objectives:

Reach Out and Read (ROR) is an evidence-based early childhood intervention that has been implemented at scale, yet description of ROR implementation is inconsistent. This study engages implementation science to examine ROR delivery and site-level variation.

Methods:

As part of an ongoing clinical trial, we conducted a mixed-methods study in 3 community health centers (CHCs) that serve low-income Latino families. We integrated quantitative parent survey data, qualitative data from monthly key informant interviews with ROR site leaders over 1 year, and in-depth interviews with 18 additional clinicians. At enrollment, parents reported whether they received a children’s book, guidance on reading, and modeling from clinicians. We analyzed quantitative data using descriptive statistics, and qualitative data iteratively engaging emergent and a priori codes drawn from the Template for Intervention Description and Replication Checklist (TIDieR).

Results:

Three hundred Latino parents (mean age: 31; 75% ≤HS education) completed surveys. The mean child age was 8 months. Overall, most parents reported receiving a book (84%) and guidance (73%), but fewer experienced modeling (23%). Components parents received varied across CHCs. Two themes emerged to explain the variation observed: (1) Differences in the perceived purpose of shared reading and book delivery aligned with variation in implementation, and (2) Site-level barriers affected what components were implemented.

Conclusion:

Because of substantive variation in ROR implementation across sites, systematic descriptions using established frameworks and corresponding measurement to characterize ROR implementation may enhance our understanding of mechanisms underlying ROR’s effects, which clinicians and policymakers can use to maximize ROR’s impact.

Introduction

Pervasive racial/ethnic and income-based inequities in developmental health are a major threat to long-term wellbeing at the individual and population levels.^1,2 The COVID-19 pandemic has intensified these inequities.³ Early childhood is the ideal time to intervene given the high degree of neuroplasticity that is unique to this developmental stage.⁴ However, in general and across sectors, early childhood interventions have faced major challenges including modest effect sizes on average, limited replication of promising models, and difficulty achieving impact at scale.⁵

Buried within average effect sizes are high degrees of heterogeneity where some children benefit disproportionately from early childhood interventions compared to others.⁶ There is a critical need to understand this heterogeneity and link early childhood interventions to theories of change in order to understand what works for whom to maximize impact.⁵ Sources of heterogeneity in treatment effects vary and under-specification of intervention models and fidelity measurement may be contributing factors. Complete published intervention descriptions are an imperative for researchers to understand why and how interventions work or do not. Expert consensus emphasizes that intervention description requires more than providing a list of component parts, but requires detailed information on the duration, intensity, mode of delivery, essential processes, and monitoring.⁷ Yet few studies on early childhood interventions achieve this degree of specification.⁸

Primary care is a unique setting to study early childhood interventions. Primary care is one of the few settings with near-universal contact with young children and frequent encounters that provide opportunities to build parent-clinician relationships that can be leveraged to support optimal child development with population-level reach.⁹ Reach Out and Read (ROR) is a model early childhood intervention implemented in primary care and stands out as one that has been implemented at scale.¹⁰ First developed in one clinic in the late 1980’s, approximately 6000 sites nationwide now implement ROR.¹⁰ ROR increases parent-child reading and enhances child language outcomes.^11,12 However, complex health interventions like ROR have multiple components and there can be major differences in the way clinicians implement them across settings, which can result in heterogeneity of treatment effects.^13,14 Descriptions of ROR in the existing literature are remarkably inconsistent and measurement of ROR implementation is even more limited.¹⁵ Few studies on ROR have engaged implementation science, which provides insight into the processes and factors associated with integrating evidence-based interventions into different settings.¹⁶

The present study responds to these gaps in the literature by engaging the Template for Intervention Description and Replication Checklist (TIDieR).⁷ Specifically, we demonstrate the application of this framework for reporting on the core components of ROR to enhance consistency in its description and measurement. The current research aims to demonstrate the value and imperative for systematic investigation into the implementation of literacy promotion and early childhood interventions to maximize their impact.

Methods

Setting

This study uses data from an ongoing effectiveness–implementation hybrid type I randomized clinical trial (NCT04609553) occurring at three urban community health centers (CHCs) in central NJ that serve a low-income Latino population (Supplemental Table 1). Data for this study were collected between October 2020 and June 2022. This study was approved by the Rutgers Institutional Review Board and all participants provided informed consent.

Data Collection

We employed a convergent mixed-methods design, collecting quantitative survey data from parents, longitudinal key informant interviews with site ROR champions, and one-time semi-structured interviews with champions and clinicians from the three CHCs. Accordingly, our convergent study design engaged both respondent and methodological triangulation to bring a holistic understanding of variation in ROR implementation.¹⁷

Quantitative survey

We recruited parent-child dyads from the three CHCs. Participants were enrolled on a rolling basis. At the time of this study, of the 521 eligible dyads referred to us thus far, approximately 58% were enrolled. Parents/caregivers were eligible to participate if they were ≥18 years old, identified as Latino, spoke English or Spanish as their primary language, had a child 6–12 months old, and received regular care at one of the CHCs. Parents were ineligible if their child had been diagnosed with multiple congenital anomalies or genetic disorders or had any previously identified developmental disabilities or if they planned to discontinue care at the CHC. A trained research assistant surveyed parents about receipt of ROR components at their last visit (Supplemental Table 2) – being given a book, receiving anticipatory guidance about reading with an additional question regarding modeling, and being exposed to a literacy-rich clinic environment. Due to physical distancing requirements and other COVID-19 precautions, all CHCs placed restrictions on literacy-rich activities, like having readers and books in waiting rooms, during the study period. As a result, this component is omitted from the current analyses.

Unstructured interviews

As parent participants were enrolled and surveyed, we conducted monthly unstructured individual interviews with one clinician ROR champion from each CHC to learn about the CHC culture and operations, stay informed about any happenings or changes at the CHC that may have influenced ROR delivery, and to inform our semi-structured interview guide. Thirty-four unstructured interviews were conducted across CHCs from 10/2021 to 12/2022. Each lasted approximately 30 minutes and were unrecorded; notes were written immediately following these interviews.

Semi-structured interviews

We designed semi-structured interviews informed by these unstructured interviews to learn more about variations in ROR delivery from clinicians. Two researchers conducted interviews over Zoom with the three champions and 18 other clinicians across CHCs, from 01/2022 to 05/2022. We conducted purposive sampling and extended invitations to nearly all clinicians (49 of 53 total) across the 3 CHCs. We achieved thematic saturation (when additional interviews do not yield new insights) after 21 interviews, and discontinued attempts at reaching the other clinicians at this point. We asked all participants about their ROR training and experience, and then asked them to walk us through their ROR delivery process. Interviews lasted between 20–40 minutes and were recorded and professionally transcribed. Clinicians were not compensated.

Analysis

Quantitative survey

We analyzed quantitative data from ROR surveys overall and by CHC using descriptive statistics (e.g., measures of central tendency and spread).

Qualitative interviews

Unstructured and semi-structured interviews were read and discussed by the team at biweekly meetings that spanned six months. Two experienced qualitative researchers (one with PhD training and over a decade of experience; one trained by this person and the other study team members) conducted the interviews and coded them using a template coding method, using the descriptions of the TIDieR checklist as the template for coding.¹⁸ TIDieR was developed by an international group of experts to enhance the completeness of reporting on interventions and has been widely used.⁷ We adapted TIDieR to suit ROR, to focus on why each CHC implements ROR (mission or goal for implementing ROR); what components each CHC implements as ROR; how much each CHC implements the different components (percent of parents receiving component) and how the CHC deliver these components. Analysts coded interviews by CHC, producing a matrix that the team further iterated until reaching consensus on descriptions of the TIDieR components by CHC. The group conducted analysis by CHC and then compared them to discover areas of variation and consistency, as well as the explanatory factors for variation observed. Analyses of qualitative and quantitative data were then summarized in a series of joint displays that arrayed quantitative and qualitative data according to each of the adapted TIDieR domains.

Results

Three hundred Latino parents (mean age: 31; 75% ≤HS education) completed surveys. The mean child age was 8 months (Range: 6–12 months). Ninety-one percent of parents reported limited English proficiency (Table 1). Twenty-one clinicians participated in the interviews. On average clinicians had practiced at their CHC for 9 years and 24% identified as Latino (Table 2).

Table 1.

Characteristics of Respondent (Caregiver) and Child and ROR Receipt at Last Clinic Visit^*

	All n=300	CHC 1 n=125	CHC 2 n=72	CHC 3 n=103
Caregiver’s mean age, years	31	31	31	31
Caregiver’s sex
Male	3 (1%)	2 (2%)	1 (1%)	0 (0%)
Female	297 (99%)	123 (98%)	71 (99%)	103 (100%)
Caregiver’s country of origin
Mexico	105 (35%)	51 (41%)	10 (14%)	44 (43%)
Dominican Republic	71 (24%)	22 (18%)	36 (50%)	13 (13%)
Honduras	54 (18%)	34 (27%)	3 (4%)	17 (17%)
Other1	70 (23%)	18 (14%)	23 (32%)	29 (28%)
Caregiver’s English proficiency
Speaks very well	26 (9%)	9 (7%)	6 (8%)	11 (11%)
Speaks well	44 (15%)	18 (14%)	10 (14%)	16 (16%)
Speaks not well	130 (43%)	50 (40%)	31 (43%)	49 (48%)
Speaks not at all	100 (33%)	48 (38%)	25 (35%)	27 (26%)
Household income
<$20,000/ year	74 (25%)	31 (25%)	18 (25%)	25 (24%)
$20,000–$29,999/ year	70 (23%)	33 (26%)	10 (14%)	27 (26%)
≥$30,000/ year	83 (28%)	35 (28%)	21 (29%)	27 (26%)
Missing	73 (24%)	26 (21%)	23 (32%)	24 (23%)
Caregiver’s education
Less than high school	51 (17%)	25 (20%)	6 (8%)	20 (19%)
Some high school	50 (17%)	24 (19%)	8 (11%)	18 (17%)
High school graduate	125 (42%)	52 (42%)	35 (49%)	38 (37%)
Some college	33 (11%)	7 (6%)	13 (18%)	13 (13%)
College graduate	37 (12%)	17 (14%)	7 (10%)	13 (13%)
Post-college degree	4 (1%)	0 (0%)	3 (4%)	1 (1%)
Child’s mean age, months	8	8	8	8
Child’s sex
Male	141 (47%)	58 (46%)	37 (51%)	46 (45%)
Female	159 (53%)	67 (54%)	35 (49%)	57 (55%)
ROR receipt at last clinic visit
Book delivery	252 (84%)	113 (90%)	65 (90%)	74 (72%)
Anticipatory guidance	219 (73%)	104 (83%)	44 (61%)	71 (69%)
Modeling	69 (23%)	54 (43%)	4 (6%)	11 (11%)

^*Data rounded to whole numbers

¹Argentina, Columbia, Costa Rica, Ecuador, El Salvador, Guatemala, Panama, Peru, Spain, Uruguay, US territory (Puerto Rico)

Table 2.

Demographic Characteristics of Respondents (Clinicians) (N =21)^*

Mean age, years	41
Sex
Male	5 (24%)
Female	16 (76%)
Race
Black of African American	3 (14%)
White	5 (24%)
Asian	9 (43%)
Other	1 (5%)
More than one race	3 (14%)
Ethnicity
Hispanic or Latino	5 (24%)
Non-Hispanic/Latino	16 (76%)
Spanish proficiency
Speaks very well	5 (24%)
Speaks well	8 (38%)
Speaks not well	6 (29%)
Speaks not at all	2 (10%)
Languages in which clinical care is delivered
English	6 (29%)
English and Spanish	11 (52%)
English, Spanish, Other1	3 (14%)
English, Other2	1 (5%)
Average years of employment at site (range)	9 (2–30)
Average years since terminal degree received (range)	14 (2–35)

^*Data rounded to whole numbers

¹Hebrew, Punjabi, Hindi, Vietnamese

²Arabic

Varied Implementation of ROR Components

The three CHCs differed in their implementation of the three ROR components studied. Parent reports corroborated clinician descriptions. Table 3 presents data according to TIDieR domains, adapted for ROR. Below, we describe ROR implementation by each CHC according to these TIDieR domains, then we present explanatory factors for the variation observed.

Table 3.

Joint Display of Clinician Interview and Parent Survey Data, adapting TIDieR domains to Reach Out and Read (ROR)

	Why CHC implements ROR	How many ROR components parents received at their last visit; Clinician approach to ROR component*
		Book delivery		Anticipatory Guidance		Modeling
		Parents received	Clinician Approach	Parents received	Clinician Approach	Parents received	Clinician Approach
CHC 1 (n=125)	Shared reading promotes Early Relational Health	90%	Gives bilingual book at the start of well-visit based on child’s age/developmental stage	83%	Observes baby with book; integrates points about reading into multiple topics throughout entire visit (i.e., developmental expectations; night-time routines; techniques for redirection)	43%	Shows parents how to perform interactive reading: position baby so baby can see book and caregiver’s face, use animated voice, ask questions, use pictures, tell stories
CHC 2 (n=72)	Help parents to give children a head start	90%	Gives bilingual or English book at start of well-visit based on age/developmental stage; also charts books given so parents do not receive the same book twice	61%	Explains ROR program and that reading helps babies’ development at visit beginning. Observes baby and parent interact with book. Often ends visit with comment to keep reading	6%	Observes parents reading with their babies; for children experiencing developmental delays, shows parents how to use animated voice, ask questions, use pictures, tell stories
CHC 3 (n=103)	Promote literacy to support child development	72%	Gives Spanish or English book at beginning or end of visit, based on parent language and child age	69%	Discusses reading during inquiry of child’s verbal skills (they have a set of questions in their EMR); instructs parents to make reading with baby a daily habit	11%	Displays pages and shows parents how to point to pictures and repeat words representing the pictures

^*Data rounded to whole numbers

CHC 1: Shared reading promotes Early Relational Health

CHC 1 identified high quality, bilingual books as critical for the population they serve, low-income, Spanish-speaking families, because shared reading can evoke warm parent-child interactions. Accordingly, 90% of the parents reported that they received a book at their last well visit. Clinicians also said that guidance on reading is an important part of their well visits; they described weaving this guidance throughout many topics, so that parents received multiple messages about reading. Eighty-three percent of parents reported receiving guidance. Clinicians described modeling as a complement to guidance, often occurring at the same time. The champion emphasized that interactive reading is not intuitive for all parents, especially positioning babies so they can see both the book and their parent’s face. Forty-three percent of parents reported experiencing modeling.

CHC 2: Help parents give their children a head start

CHC 2 described early shared reading as a pathway to children’s lifelong learning and success; the engagement between parents and children over books is essential to why ROR works as a literacy promotion program. Giving families books at the beginning of well visits is a routine that ensures families receive books; notably 90% of parents reported receiving a book at their last visit. Fewer parents reported receipt of anticipatory guidance (61%) and modeling (6%). Clinicians reported briefly discussing the relationship between reading and development, usually toward the beginning of the visit. While the clinicians find most parents do not need modeling, they prioritize modeling when children are experiencing developmental delays.

CHC 3: Promote literacy to support child development

CHC 3 implements ROR to improve literacy and help children achieve developmental milestones. Most parents reported receiving a book (72%) and guidance (69%). Clinicians reported giving books to parents either at the beginning or end of the visit, based on their preference. Their electronic medical record prompts questions about reading; these questions often provide a springboard for anticipatory guidance. For example, after reviewing language milestones, clinicians reported instructing parents to make at least 10 minutes for reading each day. Fewer parents (11%) reported receiving modeling, which corresponds with clinician data that suggests modeling is brief and is done to promote milestones when it occurs. For example, clinicians reported showing pictures and emphasizing that word repetition builds language.

Explanatory themes

Two themes emerged to help explain the variations in ROR implementation across the three CHCs (Table 4).

Table 4.

The influence of clinician perspectives on shared reading, book delivery and barriers on Reach Out and Read (ROR) implementation

	Why shared reading and book delivery are important	Barriers to ROR implementation	How much ROR components parents received at their last visit
CHC 1 Shared reading promotes Early Relational Health	Shared reading “I would say that the critical, most important component are those warm parent-child interactions that happen surrounding the books… I mean, exposure to print and books is good, but without the warm parent-child interactions that happen around that… it loses its magic.” Book delivery “[Books] might not be as important for other families… where parents are able to afford to buy books for their kids… high-quality… bilingual books, Spanish books, are really hard to come by… [F]or me, part of the goal is for [parents] to sort of also feel they can be a part of their kids’ learning and learning journeys.”	Language discordance between parents and clinicians “When the residents provide Reach Out and Read, language is a huge barrier, I think. Because they’re providing it through an interpreter. And so, it’s simply not the same, I feel like. And we talk about that a whole lot, and how we can still make it good, despite doing it with an interpreter.”	Parent reports Book delivery: 90% parents Anticipatory guidance: 83% parents Modeling: 43% parents Clinician description “We walk in the room with the book. And we sort of give it to the baby. We watch how the baby interacts with the book, while we’re talking to the mom… I make it a point to notice what the baby’s doing with the book… I kind of say, look, he wants the book. And then I’m making mental notes as to how’s the baby handling the book… I kind of tell them… it’s totally okay for him to put the book in his mouth… he’s not gonna pay attention for ever and ever. And then I’ll talk to the baby… I’ll model really quickly that interactive, high-energy kind of way to sort of read with the kid this age… [ROR] is very woven into the whole visit… [W]hen we’re talking about sleep, I’ll talk about… how the book can be part of the big bedtime routine, that shared reading.”
CHC 2 Help parents to give children a head start	Shared reading “I think if you can try to give a parent a book in their language, that’s gonna go a long way. Because I think there’s gonna be a level of confidence there that they can share something with their child, share some knowledge…” Book delivery “And also, just the ability to be able to have a book… if we could provide [a book] just as an introduction … I think that’s all people kind of need, sometimes.”	Funding “I think really funding is the main one… I mean sometimes it’s almost a little stressful because… I love introducing literacy to someone, because I just feel like that is the starting point for a better life…. So, I feel really driven [to find funding for books]…We gotta get more money. And then we just somehow find it.” Time “I think part of the barrier is time… I mean, in an ideal world, a physical probably should be an hour…. So, maybe if we had a little bit more time to kind of – we can flesh out [ROR] a little bit more.” Language discordance between parents and clinicians “Occasionally, I’ll get a parent who speaks a language I just don’t have any way to pull from… I don’t really know. But I just somehow try to figure that out…. But for the most part, we’re able to deliver, I think, pretty effectively.”	Parent reports Book delivery: 90% parents Anticipatory guidance: 61% parents Modeling: 6% parents Clinician description “.when I come into the room, my style is to lead with my books… I just say, ‘oh, look. I have a book for the baby.’ I say, ‘We have this program here called Reach Out and Read. We like to give these books out to help families start reading with their baby to help with their development.’ … So, the babies will take the book. And I’m setting myself up. And I’m observing… sometimes will see the mother kind of engaged with their child with the book… and then that’s it…Wrapping up with a parent at the end [of the visit]: ‘oh, your baby’s doing really great. Development is wonderful.’ I might put another spiel in to kind of ‘let’s keep reading.’ … I actually don’t need to model the reading because the parent does it. They do it so often… You don’t need a special skill to do that… ”
CHC 3 Promote literacy to support child development	Shared reading “I’m going to talk to them … about the child’s verbal skills… Are you able to do some reading with them every day? Have you been able to put aside time? … [T]hat marries into your verbal milestones… So you’re… trying to see if you can kind of relate the two of them together for the family.” Book delivery “[T]he ability to distribute books, I think, is probably one of the things that really makes it work. I mean, because you can talk to people about literacy, but having a book and showing somebody a book, that makes your discussion… a little more realistic… and they can then keep the book and take the book home.”	Funding “Sometimes it’s just a matter of making sure we have books. I think that’s always a barrier for us, too. It’s like, we don’t have our own fundraising arm and things like that.” Time “There’s no question, time is always a barrier. If I’m running behind in my visits, and I have families who need to be seen, and they want to be seen right away, and they need to get out because their other son is waiting to be picked up at the bus stop or whatever it is, I can guarantee you, mom doesn’t want to spend a whole lot of time talking about books.” Language discordance between parents and clinicians “Obviously, not all our clinicians speak Spanish…. And you know, doing an interpretation in a visit with a patient is going to make a visit much slower. It’s going to make it much longer. And so that’s obviously going to affect the amount of time that each clinician has to do Reach Out and Read…”	Parent reports Book delivery: 72% parents Anticipatory guidance: 69% parents Modeling: 11% parents Clinician descriptions “I’ll open the book, show the book, show the pictures of the book, and talk about how, even at this age, the child isn’t necessarily reading, but right now, we’re teaching them about that. We’re using them to see pictures, develop language skills. We read the story, but also, we focus at this age, we might focus on the pictures of the book. And we’re saying them, and we’re repeating them. In that way, through repetition, we develop language skills, and verbal language skills and receptive language skills…. I actually tend to do [ROR] more towards the latter stages of the visit…. I know there are certain things on a lot of mothers, fathers, grandmothers’ minds, and I need to get those off their minds.”

(1). Perceived purpose of shared reading and book delivery align with variation in implementation of ROR across sites

Clinicians differed in their perspectives on why shared reading and book delivery are important, which aligned with how the book was delivered and to what extent other ROR components were implemented. CHC 3 focused on how daily shared reading can help children meet developmental milestones that are important for educational success by “trying to… relate the two together for the family.” CHC 3’s champion reported giving the book at the end of the visit, to ensure attention to other important topics first, but uses the book to discuss language repetition in relation to pictures. CHC 3 described book delivery as the glue that made the literacy promotion stick for parents: “having a book and showing somebody a book, that makes your discussion about literacy, probably a little more realistic… it probably drives the point home.” This pattern of book delivery later in the visit is consistent with CHC 3’s description of more instruction than modeling and is corroborated by parent-report data indicating that most parents receive books and anticipatory guidance but few experience modeling.

Like CHC 3, CHCs 1 & 2 noted the connection with future educational success; but also emphasized shared reading as a form of parent-child interaction and engagement. For CHC 1, exposure to books is good for children, but “without the warm parent-child interactions that happen around that… it loses its magic.” CHC 2 described books as “gifts” that open new worlds, but what makes books special is the parent-child engagement: “it’s something a [parent] and a child do together… interacting over a book… Just you, the book, and your kid.” CHCs 1 and 2 described books as an opportunity for enabling parent-child bonding and parent teaching; CHC 1 described how bilingual books help parents “feel they can be part of their kids’ learning journeys” and CHC 2 notes that same-language books can give parents “confidence… that they can share something with their child, share some knowledge.”

Both CHC 1 and 2 reported giving the book early in the visit and 90% of parents at both CHCs reported receiving books. However, they differed in their approach to anticipatory guidance and modeling. CHC 2’s champion said their clinicians open a visit with a brief description about how critical reading is for development and a comment at the end to “keep it up.” Observing parents and children with the book, their champion said they usually “do not need to model… because the parent does it… so often”; parents do not need a “special skill” to read to or tell their child a story, but they need to engage with their child. CHC 1 held that the interactive shared reading was not intuitive: “parents just vary in their ability… to do this well… [Y]ou have to show people how to make it interactive… ” The champion described weaving anticipatory guidance throughout the visit and modeling how to use an excited voice and position the child to see both the parent’s face and the book. These differences are corroborated by parent-report data. Notably, CHC 1 had the highest number of parents report anticipatory guidance and modeling. Most of the parents from CHC 2 reported anticipatory guidance, but few reported modeling.

(2). Site-level barriers affect implementation of ROR components

Across the CHCs, implementation of ROR components was affected by the presence or absence of barriers. CHCs 2 and 3 experienced consistent strains around funding and purchasing books. CHC 2’s champion was responsible for securing funding for books. She applied to foundations and found other agencies that donated books. She ordered books from sellers that sold bundled packages and gave free books with orders. She said clinicians stretched books across age groups and languages to ensure they had enough. CHC 3’s office manager was responsible for ordering books. Their champion explained they did not have a “fundraising arm” for books. Clinicians explained there were times the CHC did not have books or had a limited selection. This description was corroborated by parent-report data from CHC 3; while most parents reported receiving a book, the percentage was lower compared to other sites. In contrast, CHC 1 did not have problems with book supply. Their clinicians had time allocated for academic responsibilities that facilitated grant-writing. They have received consistent grant funding for the past several years.

Visit time and busyness also influenced which components and how much of components were implemented. All clinicians said that visits are already too short for all they need to do. ROR implementation was most affected by time and busyness for CHC 2 and 3. CHC 2 described how walk-in appointments disrupted schedules. They often were overbooked. Visit time was regimented and there are other important topics to address. CHC 3 described how staff turn-over and need for interpreters (CHC 3 had the largest portion of non-Spanish speaking clinicians) affected visit schedules. This atmosphere created challenges for providing ROR guidance or modeling; the champion indicated that parents had time constraints, as well. Although the clinicians at CHC 1 all speak Spanish, they had many non-Spanish-speaking residents who rotated there and worried more that language discordance could affect the quality of ROR delivery and parent receptivity than visit time.

Discussion

We identified substantive variation in ROR implementation across three CHCs serving Latino families from under-resourced communities. Systematic description of implementation using TIDieR facilitated understanding what components were delivered and how they were delivered. We found that perceptions of why shared reading and book delivery are important and barriers including funding, time, and language discordance help explain differences in implementation. Given ROR’s impact and scale, systematic description using frameworks like TIDieR and corresponding measurement to characterize ROR implementation could open the door to a deeper understanding of how components impact outcomes, heterogeneity of treatment effects, and the mechanisms underlying ROR’s effects. In turn, clinicians and policymakers can use this knowledge to amplify ROR’s impact at the population level, which can serve as a model for other early childhood interventions.

ROR is notable among pediatric primary care innovations for the scale it has achieved and the evidence it has accumulated. ROR’s rapid uptake over its 30-year history reflects how it responds to a clear need to support child development that resonates with practitioners across the US.¹⁹ During this time, several studies have documented ROR’s effectiveness, building strong practice-based evidence for this innovation.^20–23 To date, fewer studies have focused on ROR implementation. One examined clinicians’ experiences implementing ROR.²⁴ The others suggest variation across sites,^25,26 which corresponds with our findings. In fact, a recent scoping review illustrates how the number of studies examining ROR has proliferated in recent years, reflecting its importance, yet description of implementation and measurement remain remarkably inconsistent.¹⁵ As ROR builds on its impact focusing on implementation is consistent with the prevention research cycle progressing from efficacy to implementation science and represents an opportunity to amplify its impact.²⁷ In this study, we demonstrate how frameworks like TIDieR can be used to generate systematic descriptions to guide such work.

We found that differences in clinician’s perceived importance of shared reading and book delivery and practice level barriers affect to what extent components are implemented. Across the CHCs, clinicians identified the children’s book as the foundational component of ROR, on which the other components build, and a high percentage of parents reported receiving a book at each CHC. In contrast, far fewer parents reported experiencing modeling, and the CHC with the highest percentage, was the one where clinicians identified ROR as an opportunity to promote early relational health, which is a stated priority in ROR’s Next Chapter initiative.²⁸ The flexibility of the ROR model has likely contributed to its rapid and wide uptake. Along with this flexibility comes modifications that occur locally and organically based on sociopolitical, organizational, and local community context, but little is known regarding how these modifications affect outcomes.

Currently, there is a paucity of measures that assess the fidelity of ROR implementation. More consistent descriptions of ROR implementation using frameworks like TIDieR could inform the development of such measures, and also systematically characterize modifications. Systematic measurement of implementation can provide the prerequisite information needed to address unanswered questions such as which components matter most for producing outcomes that matter at the population level and what mechanisms underlie these effects. In turn, this knowledge could inform training since previous work illustrates how training can influence both clinician perceptions of ROR and reported literacy promotion behaviors,^29,30 as well as guide planned adaptations. Given ROR’s population-level reach, such work could serve as a model for other early childhood intervention research.

This study has certain limitations. First, we relied on parent and clinician report to assess ROR implementation, which may be subject to social desirability and recall bias. However, triangulation across sources (i.e., clinicians, parents) and methods (i.e., quantitative and qualitative) helps guard against these biases and is a notable strength. While direct observation was not possible due to pandemic restrictions, it would strengthen future work. In this regard, work by Needlman et al. is notable in its use of observation to examine ROR implementation, in particular modeling, which we identified as a key area of variation.³¹ Second, the study focused on Latino families from under-resourced communities and CHCs that serve them, thus our findings may not transfer to all settings. Third, data on how much support each CHC receives from the state ROR chapter and specific information on training was unavailable. Such factors can function as important barriers and facilitators and will be important to examine in future work. Fourth, while we were able to characterize variation in implementation of ROR components using an established implementation science framework, we did not examine to what extent parent and clinician characteristics affect variation or link variation in components to parenting and child outcomes. This work is beyond the scope of this study but will be the subject of future research using the current study as a foundation.

Conclusion

Because of substantive variation in ROR implementation across sites, systematic descriptions using established frameworks like TIDieR and corresponding measurement to characterize ROR component implementation are needed. Detailed insights into implementation are prerequisites for understanding of heterogeneity of treatment effects, what components are most important for whom, and the mechanisms underlying the observed effects of ROR. In turn, this knowledge can open the door to work that amplifies ROR’s impact at the population level.

What’s new:

Reach Out and Read (ROR) is an evidence-based literacy promotion intervention that has been implemented at scale. This study leverages implementation science to describe site-level variation in what ROR components parents receive and understand why variation occurred.