Pragmatic Health Assessment in Early Childhood: The PROMIS^® of Developmentally Based Measurement for Pediatric Psychology

Abstract

Objective

To illustrate the integration of developmental considerations into person-reported outcome (PRO) measurement development for application in early childhood pediatric psychology.

Methods

Combining the state-of-the-science Patient-Reported Outcome Measurement Information System (PROMIS^®) mixed-methods instrument development approach with considerations from developmental measurement science, we developed 12 PROMIS early childhood (PROMIS EC) parent report measures to evaluate common mental, social, and physical health outcomes for ages 1–5. Through this interdisciplinary effort, we identified key considerations for early childhood PROs that enable reliable and valid assessment within the real-world constraints of clinical care settings.

Results

Four key considerations are highlighted as key to this process: (a) Engage diverse content experts to identify meaningful and relevant constructs; (b) Balance salient features for early childhood with lifespan coherence of constructs; (c) Emphasize observable features across the typical/atypical spectrum; and (d) Ensure feasibility and relevancy for clinical and research application. Each consideration is discussed using exemplars from the PROMIS EC measurement development process.

Conclusions

PROMIS EC provides an illustration of how well-established PRO measures for youth can be adapted for younger children by incorporating developmental considerations. This process and resulting key considerations provide clinicians and researchers in the field of pediatric psychology with guidance for adapting PROs to early childhood, enabling critical continuity in domains of high salience to pediatric psychologists.

Person-reported outcomes (PROs) offer the potential to provide reliable and valid measures that are feasible for clinical application in pediatric psychology (Bevans, Riley, Moon, & Forrest, 2010). However, standard PRO methods in pediatric psychology typically begin assessment in middle childhood, precluding capturing critical early expressions of child health. Capitalizing on recent advances in developmentally sensitive survey measurement (Wakschlag et al., 2014), we propose four considerations for developing early childhood PROs. To illustrate, we discuss the recent generation of 12 Patient-Reported Outcomes Measurement Information System (PROMIS^®) early childhood parent-report instruments. These measures align with four overarching components: (a) global health (overall evaluation of children’s general health); (b) mental health and well-being (i.e., anger/irritability; anxiety; depressive symptoms; positive affect; engagement—curiosity; engagement—persistence; self-control—adaptability; self-control—self-regulation); (c) social health (i.e., social relationships, including family/caregiver and peers); and (d) physical health (i.e., physical activity; sleep health, including disturbance and impairment; see Table I for domain frameworks).

Table I.

PROMIS Early Childhood Domains and Definitions

Domain	Definition
Global health	Overall evaluation of one’s physical, mental, and social health
Mental health
Anger/irritability	Angry mood (irritability, grouchiness) and behavior (frustration, tantrums, and management of angry behavior)
Anxiety	Fear (fearfulness, panic), anxious misery (worry/dread), hyperarousal (tension, nervousness), and social/separation anxiety (fear/distress when separating from caregivers, in unfamiliar situations)
Depressive symptoms	Sad/withdrawn, negative views of self (self-criticism, low self-esteem), and anhedonia (loss of interest, inability to engage in play, lack of enjoyment)
Positive affect	Feelings and moods associated with momentary positive affective experiences (contentment, happiness, and joy)
Engagement—curiosity	Emotional, behavioral, and cognitive curiosity and interest, initiative taking
Engagement—persistence	Sustained engagement and effort in problem solving and completing challenging activities, self-confidence
Self-control—adaptability	Ability to adapt in response to environmental demands, changes, and expectations (flexibility)
Self-control—self-regulation	Recognition and regulation of emotions and behaviors in service of one’s own goals (coping, frustration tolerance)
Social health
Social relationships	Family relationships reflecting positive interactions, experiences, and connectedness with the primary caregiver and broader family unit reflecting warmth and affection, trust, dependability, and support
	Peer relationships related to positive peer interactions, sociability (getting along well with others), and empathic behaviors
Physical health
Physical activity	General physical activity behaviors and associated intensity and physiological symptoms
Sleep health	Sleep disturbance pertaining to delayed sleep, sleep onset, sleep continuity, and sleep quality
	Sleep-related impairment related to the impact of impact of poor sleep on daytime functioning, routines, and mood

Note. Bolded words and phrases in the PROMIS Early Childhood definitions reflect developmentally sensitive modifications made from the existing PROMIS pediatric (5–17 years) domain definitions.

Foundational Paradigms

PROMIS provides brief, norm-referenced self- and proxy-reported outcome measures for lifespan health assessment (DeWalt et al., 2015; Lai, Jensen, Charrow, & Listernick, 2019). A distinguishing feature of PROMIS is its state-of-the-science mixed-methods approach (for overview, see: Cella et al., 2007, 2010; Forrest et al., 2012), which includes (a) in-depth literature and measurement reviews; (b) expert input to articulate domain frameworks and associated concepts; (c) semistructured concept elicitation interviews with target respondents to confirm frameworks and identify new concepts for potential inclusion; (d) item drafting based on such frameworks; (e) cognitive interviews to confirm or modify item wording; and (f) field testing items with a nationally representative sample of the target population for item calibration and score norming. PROMIS uses item response theory to maximize precision across the full range of measurement concepts, surpassing that accomplished by most instruments developed using classical test theory.

One limitation of PROMIS is the current lower bound for assessment at age 5. Increasing evidence suggests early expression of lifespan health and disease states can be detected and measured reliably and validly as young as 1 year of age (Biedzio & Wakschlag, 2019; Briggs-Gowan, Carter, Irwin, Wachtel, & Cicchetti, 2004; Edwards, Rapee, Kennedy, & Spence, 2010; Luby, 2010; Wakschlag et al., 2019). Wakschlag et al. (2014) introduced the developmental specification model as a method to harness developmental variation to ensure robust and valid measurement of early childhood health outcomes by defining atypicality as deviation from normative variation in young children (Wakschlag, Tolan, & Leventhal, 2010; Wakschlag et al., 2018).

Incorporating developmentally salient features within the lifespan approach of PROMIS addresses several important gaps in the field. First, such integration extends PROMIS instruments to children under 5 years old. Second, extant tools for early childhood typically emphasize developmental nuance over efficiency, rendering them impractical for clinical use and epidemiological studies. Finally, developmental measurement science has been applied most deeply in mental health and has not emphasized a holistic approach that includes social and physical health, particularly relevant in pediatric psychology.

Here, we describe four-key considerations that emerged during the process of integrating PROMIS with developmental measurement science to create early childhood parent report measures (PROMIS EC, 1–5 years). While this integration is salient across the lifespan, our focus on early childhood is motivated by highlighting a period where differentiating normative variation from clinically salient patterns is of highest impact (Wakschlag et al., 2010) and where measurement gaps are most pronounced.

Key Consideration 1: Engage Diverse Content Experts to Identify Meaningful and Relevant Constructs

To avoid the problematic qualities of traditional classification systems that often include developmentally improbable and impossible symptoms for young children (e.g., conduct disorder symptoms such as truancy and fire setting; depressive symptoms such as anhedonic reduced libido; Luby et al., 2003; Wakschlag et al., 2010), the PROMIS EC development process began with the identification of conceptually relevant and clinically meaningful domains pertinent to assess in early childhood. In addition to literature and measurement reviews, this iterative process included remote and in-person meetings with experts from diverse fields, including developmental psychology and psychopathology; child psychiatry; pediatrics; education; cross-cultural measurement science; health equity; and healthcare systems, services, and policies. These meetings included reviewing and refining existing PROMIS constructs to ensure appropriateness for younger children. Experts provided insight into condensing definitions due to age-inappropriateness (e.g., excluding sense of belonging from the social relationships domain) and expanding definitions to include age-appropriate facets (e.g., adding social/separation anxiety to the anxiety domain given its particular salience in early childhood [Buss, 2011]).

As recommended by Glasgow (2013), we engaged a range of stakeholders. In addition to researchers and clinicians, we conducted 38 semistructured concept elicitation interviews with parents of 1- to 5-year-olds from diverse socioeconomic strata to elicit their definitions of each domain to ensure parental perspectives were well captured. Interviews generally supported the existing researcher-developed domain frameworks. Two notable exceptions were social relationships and physical activity. For the former, whereas the existing peer relationships framework focuses on the quality and reciprocity of peer interactions (e.g., “My child was able to have fun with friends;” “Other children wanted to be my child’s friend”), parents of 1- to 5-year-olds focused more on the development of social skills, using phrases such as “interacting well with others.” For the latter, parents often conflated “being active” with “being physically active,” describing a range of activities from reading to bicycling. This input was used to refine domain definitions. Additionally, phrases parents used were adopted as potential items (e.g., the positive affect item “My child was playful” is a direct result of a parent’s comment).

Key Consideration 2: Balance Salient Features for Early Childhood with Lifespan Coherence of Constructs

The Society of Pediatric Psychology (SPP) Division 54 Task Force’s evaluation of extant pediatric assessments underscored that many instruments are modifications of adult measures without consideration of developmental sensitivity (Cohen et al., 2008). To avoid repeating this top-down approach, we combined the value of alignment of lifespan concepts with the developmental appropriateness of existing PROMIS pediatric instruments and principles from developmental measurement science. This necessitated complementing existing PROMIS principles with more in-depth attention to item content that distinguished extensive normative variation from markers of concern. Whereas at older ages the presence or absence of a behavior is a problem marker, in younger children many common problem markers occur in most children (e.g., crying frequently, anxiety in new places, and tantrums). Thus, we retained the existing timeframes (i.e., past 7 days or past 4 weeks) and past tense item wording to align with PROMIS, but also relied on developmental model features that were empirically validated and are relevant across domains of behavior to ensure sufficient coverage of salient features specific to young children. This additional dimension of measurement rigor introduced features of intensity and context toward efficient delineation of markers of health concern.

For example, to generate early childhood domains that conceptually align with the PROMIS domains of anger/irritability, anxiety, and depressive symptoms in a developmentally meaningful way, we applied the Multidimensional Assessment Profile (MAP; Biedzio & Wakschlag, 2019; Wakschlag et al., 2014)—a framework and set of survey tools that are designed to pinpoint features that differentiate normative variation from atypical patterns in early childhood. To achieve such differentiation, MAP integrates consideration of behavioral context as well as developmental capacity and varied form of expression in early childhood (Buss, 2011; Luby et al., 2003; Petitclerc et al., 2015). We drew on such practices by adding contextual items related to familiar/unfamiliar settings, at home/out in public, and with parents/other adults (e.g., “My child was inconsolable when separating from me or other parent in a familiar setting;” “My child lost his/her temper or had a temper tantrum when out in public”).

While we considered the potential for age-based (infant/toddler vs. preschool age) versions of each measure, we also acknowledged the strength of PROMIS whereby a single measure per domain covers a broad age period. Thus, we chose to develop one set of instruments for the full early childhood age range, with a few exceptions where age-based items for 3- to 5-year-olds only were required, particularly for items requiring verbal skills (e.g., “My child said negative things about him/herself”).

Another important consideration was determining developmental face validity, that is, can a construct be meaningfully applied in early childhood? For example, the existing PROMIS mental health component includes three domains covering hedonic (i.e., life satisfaction and positive affect) and eudemonic (meaning and purpose) well-being (Ravens-Sieberer et al., 2014). Experts agreed that positive affect (as currently construed) was developmentally appropriate and could be meaningfully measured via parent report based on the readily observable behavioral characteristics of this domain, such as smiling and laughing. While experts thought life satisfaction and meaning and purpose were inappropriate within the current conceptualization of higher order reflective thinking, they agreed that the latent constructs could be translated to meaningful terms for early childhood. Indeed, components of these domains (e.g., interest and engagement, goal-directedness) are meaningful markers of well-being in young children that have long-term implications for children’s health and development (Nigg, 2017). We modified these relevant facets with developmentally appropriate expressions identified in the literature, including the Positive Health Framework (Forrest, Blackwell, & Camargo, 2018), Model of Child Well-Being (Moore, Bethell, Murphy, Martin, & Beltz, 2017), positive indicators of child well-being framework (Lippman, Moore, & McIntosh, 2011), and head start early learning framework (Office of Head Start, 2015). See Table I for final domain frameworks.

Key Consideration 3: Emphasize Observable Features across the Typical/Atypical Spectrum

In order to distinguish typical variation and developmental change from atypical patterns in young children, particularly in the pre- and early verbal stages, assessments require precise discriminating features that can easily be observed rather than inferred (Edwards et al., 2010). This can be particularly challenging when core features of constructs reflect internal mental states, such that caregivers have difficulty providing reliable and valid responses. Guidance from the U.S. Food and Drug Administration (U.S. Department of Health and Human Services, 2009) for pediatric PROs discourages the use of proxy assessments, where caregivers infer the child’s subjective experience, and instead emphasizes the use of caregiver report measures that focus on the child’s observable behaviors. This is particularly crucial for assessing young children who may or may not have the capacity to express moods and feelings verbally.

Whenever possible, the wording of PROMIS items for older youth were maintained, with minor modifications as needed to shift to from parent proxy to parent report. In Table II, we provide exemplar item modifications for each domain, including items directly adapted from existing PROMIS items as well as new items. We prioritized consistency with the original PROMIS framework when possible, similar to the approach taken that has also been used in prior work such as the early childhood adaptation of the Diagnostic and Statistical Manual of Mental Disorders (DSM) framework (Scheeringa, 2003). At one level, this involved changing the phrase “My child felt” to use a more observable verb, such as “seemed,” “acted,” or “was” (e.g., “My child felt sleepy when he/she woke up” was modified to “My child seemed sleepy when he/she woke up”). Similarly, we modified causal attributions to avoid requiring parents to infer whether their child’s behavior was due to some unobservable characteristic. The existing PROMIS item, “My child got mad easily because he/she was sleepy,” for example, requires parents to interpret the child’s mood based on an unobservable state. While the core concept of this item is sound (2-year-olds can be quick to anger when tired), we revised the wording: “When my child didn’t sleep well, he/she got mad easily.” We also added tantrum items to the Anger/Irritability domain (e.g., “My child felt upset” was modified to “My child had a temper tantrum when upset or angry”), as tantrums are observable and amongst the most highly salient indicators of concern in early childhood (Wakschlag et al., 2018). All modifications and new items were corroborated through 41 cognitive interviews with parents of 1- to 5-year-olds.

Table II.

Exemplar Adaptations and Extensions of PROMIS Parent-Proxy (5–17years) Items for PROMIS Early Childhood Parent Report (1–5years) Measures

Domain	Original PROMIS parent-proxy item	Adaptations	Developmentally specific featuresa
Global health	In general, how would you rate your child’s mental health, including mood and ability to think?	In general, how would you rate your child’s mental health?	How well is your child meeting developmental milestones?
		How would you rate your child’s mood?
		How would you rate your child’s ability to think?
Anger/irritability	My child was so angry he/she felt like throwing something.	My child broke or destroyed things during a temper tantrum.	My child had a temper tantrum till exhausted.
Anxiety	It was hard for my child to relax.	My child seemed tense.	My child was inconsolable when separating from me or other parent in a familiar setting.
Depressive symptoms	My child didn’t care about anything.	My child wasn’t interested in doing things he/she usually likes.	My child acted withdrawn when in a group of children.
Positive affect	My child felt enthusiastic.	My child was enthusiastic.	My child was playful.
Engagementb—curiosity	My child’s life is filled with things that interest him/her.	My child actively explored the world around him/her.	My child asked “why,” “what,” and “how” questions.
Engagementb—persistence	N/A	N/A	My child didn't give up when something was hard.
Self controlc—adaptability	N/A	N/A	My child adjusted easily to changes in routines.
Self controlc—self regulation	N/A	N/A	My child bounced back quickly when things didn’t go his/her way.
Social relationships	My child knew our family was interested in what he/she was doing.	Our family was interested in what my child was doing.	My child was good at expressing his/her needs to me or other parent.
	Other kids wanted to talk to my child.	Other kids wanted to play with my child.	My child showed interest in other children.
Physical activity	How many days did your child exercise or play so hard that his/her body got tired?	How many days did your child play so hard that he/she got physically exhausted?	How many days did your child play so hard that he/she fell asleep early?
			How many days did your child play so hard that he/she needed an extra or longer nap?
Sleep healthd	My child had trouble controlling his/her feelings because he/she was sleepy.	When my child didn’t get enough sleep, he/she became frustrated easily.	When my child didn’t sleep well, he/she had more temper tantrums than usual.

Note. N/A = No parallel PROMIS Parent-Proxy items exist. Therefore, all items within these item pools represent “developmentally specific features” of these domains.

^aAll items in the “developmentally specific features” columns are new items and not modifications of existing PROMIS items.

^bThe Engagement domain is the PROMIS EC adaptation for the PROMIS well-being domains of life satisfaction and meaning and purpose. While the overall domains were deemed inappropriate for 1- to 5-year-olds, several items from these banks were modified as part of the PROMIS EC Engagement measure.

^cThere is no comparable PROMIS domain for self-regulation but experts identified this construct as a critical component of early childhood well-being.

^dNo modifications were made to the PROMIS sleep disturbance items for use in early childhood, so only example items from the sleep-related impairment facet are presented.

Key Consideration 4: Ensure Feasibility and Relevance for Clinical and Research Application

Core competencies in pediatric psychology include being well-versed in multiple assessment methodologies and effectively selecting and interpreting instruments that are developmentally appropriate for children (Palermo et al., 2014). With the integration of PROs into pediatric clinical environments gaining traction, there is an increased need for robust tools that meet both the scientific integrity of the field as well as the practical constraints of busy clinical settings.

As Cohen et al. (2008) described, issues of cost, time-efficiency, and ease of use are critical for pediatric psychology when considering whether and what assessment to use. At the same time, assessments must be clinically relevant above and beyond specific diagnostic criteria (Cohen et al., 2008). In a review of extant measures, the SPP Division 54 Task Force found assessments too long and complex for use in clinical environments. The present measurement development initiative aimed to fulfill these priorities by enabling domain-oriented evaluation of critical early childhood health and well-being constructs that can be easily implemented in clinical settings. All PROMIS EC measures will have short forms to enable feasible assessment of many core early childhood domains without increasing respondent burden. The ability to use computer adaptive tests (CATs) can also provide more precise and reliable individual assessments without the burden of lengthy questionnaires. Given the brevity of clinical visits, such options offer a feasible way to include robust PROs in pediatric psychology to facilitate early detections and further early interventions.

Furthermore, the PROMIS EC measures provide robust tools for researchers and clinicians that are applicable across diverse participant populations, complex study designs, and large research consortia. In addition to using advanced psychometric methods to ensure items function the same across demographic and clinical subgroups, every item underwent cognitive interviewing with at least 5 parents, including at least 1 parent with less than a 12th grade education and 2 non-white parents. Such interviews focused on item comprehensibility to ensure feasibility across diverse populations. All items were also subjected to readability testing with Lexile Analyzer^® to evaluate approximate reading level. Of the 138 final items, the majority (88%) were below 1st grade reading level, and all were at or below 6th grade reading level. Items also underwent translatability review to ensure conceptual equivalence across languages, and PROMIS EC measures will be translated into Universal Spanish using advanced forward and back-translation processes and cognitive interviews with native Spanish speakers (Devine et al., 2018). Concurrent and predictive validity testing are underway in both general pediatric populations and young children at risk for psychopathology.

Conclusion

Given the particular interest of pediatric psychology in evaluating health across developmental periods, there is need for assessments that are developmentally sensitive, lifespan coherent, and universally applicable to all children. Often, such needs are at odds with one another. The speed with which children undergo developmental changes makes it challenging to measure concepts common across developmental periods because the behavioral expression of certain concepts can shift over the lifespan (Carter, Gray, Baillargeon, & Wakschlag, 2013). At the same time, the demands of research and clinical practice require brief assessments applicable to diverse populations and that can flexibly be integrated into visits.

To address such competing needs, we developed a process for adapting existing PROs for use in early childhood by combining the rigorous methods of PROMIS with state-of-the-art developmental measurement science to achieve measures that balance developmental-sensitivity with lifespan coherence. This innovative process, guided by four-key considerations, was applied across a wide range of child health domains, with input from parents as well as academic and clinical experts. These four considerations helped us maintain a dual loyalty to developmental sensitivity and coherent lifespan measurement. The application of such considerations toward the goal of extending the reach of PROMIS down 1-year-olds required input from multiple stakeholders in order to ensure ultimate success.

Overall, these considerations provide researchers and clinicians in the field of pediatric psychology with guidance for adapting measures to early childhood as well as a set of brief, efficient, and robust new tools to assess young children’s physical, mental, and social health. Clinical users of these measures can test their utility in differentiating typical (normative) from atypical early development for capturing emergent indicators of poor health status. Such earlier identification can lead to opportunities for intervening earlier in the disease trajectory, which can have profound positive influences on long-term outcomes. Researchers can apply these considerations when assessing a range of lifespan-relevant health outcomes that are modified as needed to remain responsive to developmental change. The new measures, with their flexibility and brevity, also offer unique opportunities for epidemiological research, particularly in longitudinal and large-scale research consortia endeavors.

Funding

Research reported in this publication was supported by the Environmental influences on Child Health Outcomes (ECHO) program, Office of The Director, National Institutes of Health, under Award Number U24OD023319 with co-funding from the Office of Behavioral and Social Sciences Research (OBSSR; Person Reported Outcomes Core). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.