Measuring the impact of burn injury on the parent-reported health outcomes of children 1-to-5 years: Item pool development for the Preschool_1–5 Life Impact Burn Recovery Evaluation (LIBRE) Profile

Abstract

Modern, reliable, and valid outcome measures are essential to understanding the health needs of young children with burn injuries. Burn-specific and age-appropriate legacy assessment tools exist for this population but are hindered by the limitations of existing paper-based instruments. The purpose of this study was to develop item pools comprised of questions appropriate for children aged 1–5 with burn injuries. Item development was based on a framework provided by previous work to develop the Preschool Life Impact Burn Recovery Evaluation (LIBRE) Conceptual Model. The Preschool LIBRE Conceptual Model work established four sub-domains of functioning for children with burns aged 1–5. Item development involved a systematic literature review, a qualitative item review process with clinical experts, and parent cognitive interviews. Four item pools were established: (1) communication and language development; (2) physical functioning; (3) psychological functioning and (4) social functioning for preschool-aged children with burn injuries. We selected and refined candidate items, recall periods, survey instructions, and response option choices through clinical and parental feedback during the qualitative review and cognitive interview processes. Item pools are currently being field-tested as part of the process to calibrate and validate the Preschool_1–5 LIBRE Computer Adaptive Test (CAT) Profile.

1. Introduction

The first five years of life represent a time where children rapidly grow and achieve a multitude of developmental milestones [1]. Fledgling mobility skills – such as children learning to crawl and walk – as well as their natural curiosity to explore surroundings place young children at risk to sustain a burn injury [2]. Since the first five years of life are a formative time in young children’s growth, burn injuries at this age can have a significant impact on health and development [3–5]. The assessment of burn injury recovery in children is critical to improving pediatric health outcomes. Physical, psychological, and social effects of the burn injury can potentially alter the trajectory of a child’s life and these effects can persist into adolescence and adulthood [6].

The assessment of burn recovery through the use of patient-reported outcomes measures (PROMs) is essential to measuring long-term physical and psychosocial outcomes [7]. For young children, parent-reported PROMs can provide insights into the complex, multi-dimensional challenges that children may face after burn injury. PROMs provide an opportunity to obtain information directly from the patient/parent perspective, including functional status, symptom severity, and overall well-being following treatment and intervention, which allows patients and families to champion recovery efforts and actively engage in care [8–10]. While traditional burn-specific measures exist, such as the Shriners Hospitals for Children/American Burn Association Burn Outcomes Questionnaire for Children 0–4 (BOQ_0–4), the science of instrument development has advanced with the advent of contemporary approaches to measurement such as computer adaptive testing (CAT) and benchmarking of recovery trajectories [6,11–17]. These approaches can be applied to assessing children with burn injuries in order to improve upon traditional outcomes measures and to provide more contemporary, sophisticated assessment tools.

Shifting from static PROMs to CAT-based instruments has several advantages when measuring long-term outcomes: (1) precision is optimized because CAT-based measures only administer psychometrically relevant items to the respondent; (2) each assessment is customized to the respondent based on what items are endorsed; and (3) by filtering out irrelevant items, CAT-based instruments are efficient and less burdensome for the respondent [18]. However, these advantages are dependent on the development of comprehensive questions that comprise the item pools. Items pools are administered to large samples in field studies and items are then calibrated with advanced psychometric approaches using item response theory (IRT), which is the measurement paradigm underlying CAT-based assessments. Item response theory is the foundation for developing a computer adaptive test as it uses a graded response model which is unidimensional for each domain. These calibrated item banks are administered with a CAT-based approach. In order to create a CAT-based PROM, researchers begin with developing items, in partnership with clinical experts, patients, and families, to ensure that important activities relevant to patients and clinicians are assessed by these measures [9,18].

Burn outcomes research for young children has been previously hindered by limitations of traditional legacy assessments. CAT-based PROMs overcome these limitations by efficient administration of large comprehensive item banks in the home and clinical setting via CAT web-based platforms. Recent strides to develop CAT-based outcomes measures with wider research and clinical applications demonstrate a focus in improving recovery metrics tailored to the needs of individual burn survivors. CAT-based instruments use a computer algorithm, based on IRT to select items from calibrated item banks. The algorithm uses an individual’s response for an item to select the next item from the calibrated item banks based on two considerations: the level of item difficulty that best matches the individual’s ability and the item that best discriminates between individuals at a particular ability level. The CAT approach personalizes each assessment but since all assessments share a common metric score, results can be compared even though different items were administered. Conversely, legacy measures require the completion of all items of a domain to estimate an individual’s score [19,20]. With CAT-based technology, items selected for administration are based on real-time estimates of an individual’s ability. A precise estimate of ability is achieved with administration of a few (e.g., 5–10) selected items from the entire item bank for a given domain [20]. CAT features are optimized when item banks contain a wide range of items appropriate for different levels of ability on a unidimensional metric [21]. Thus, CAT-based instruments require the iterative development of item pools that cover the scope of the underlying construct and must contain items with consider-able conceptual breadth and depth.

When developing a parent-reported, burn-specific PROM, researchers can leverage clinical experts’ knowledge along with input from parents of children with burns using a collaborative, co-productive process to generate items that assess important long-term physical, psychosocial and emotional outcomes for children with burn injury [22]. The advantage of directly engaging with parents and burn care clinicians is the ability to gain unique parental and clinical perspectives when generating item content, revising item wording, and evaluating the range of item coverage. Clinical and parental input can aid researchers in identifying activities specific to burn recovery that are important to incorporate in a new instrument. Additionally, parents and clinicians can offer insights into the comprehensibility and clarity of each candidate item and these insights serve to create a more comprehensive and relevant measure for future end-users [22–24].

The primary focus of this study was to develop comprehensive item pools based on the four functioning sub-domains previously established by the Preschool Life Impact Burn Recovery Evaluation (LIBRE) Conceptual Model [25]. A secondary focus of this study was to seek clinician and parental feedback regarding their understanding of the item content. This work aims to form the foundation of a new parent-reported CAT-based outcomes instrument that measures recovery after burn in preschool-aged (1–5 years) children (Fig. 1).

Fig. 1 –

Preschool Life Impact Burn Recovery Evaluation (LIBRE) CAT Profile development process. Steps 1–3 are detailed elsewhere [22]. Steps 4–8 are described in methods below.

2. Methods

The process for developing item pools involved two phases: (1) Item pool development (January–April 2018) and (2) Item pool refinement (April–October 2018). Preliminary item pool development consisted of generating initial item pools from pre-existing measures and checklists such as the Centers for Disease Control and Prevention’s (CDC) Developmental Milestones [1]. This phase also involved a qualitative item review process with clinicians and researchers where candidate items were binned together and winnowed down. The second phase, item pool refinement, included clinical expert consensus meetings, parent cognitive interviews, and final revisions of candidate items (Fig. 2).

Fig. 2 –

Flow diagram of item pool development.

2.1. Preliminary item pool development

2.1.1. Generating initial item pools

Initial candidate items were generated from 95 generic and burn-specific outcome measures and developmental checklists identified from a literature review conducted by Brady et al. [25]. Items from each measure and checklist were then extracted and entered into a structured data collection spreadsheet. For each measure reviewed, the following additional information was extracted: intended domains and constructs, unique recall periods, item stems, and response options.

2.1.2. Qualitative item review

Items underwent an iterative review process characterized by binning and winnowing [15,26]. Goals of the binning process were to group items assessing similar content together to allow for the winnowing process where the best items in each group were selected and duplicate items were removed. Grouping items together based on the conceptual content allowed the research team to determine which relevant domains and items were necessary in the assessment of burn injury recovery for children in this age group. Identification of the underlying construct assessed by each item was performed based on a review of the original scale developers’ definition and the opinions of burn care experts. Item binning was informed by the Preschool LIBRE Conceptual Model and items were divided into four groups of functioning: (1) communication and language development, (2) physical functioning, (3) psychological functioning and (4) social functioning [25]. Based on a qualitative review of items in each of the four domains, the research team ordered items appropriate for assessing different ability levels (low to high) for each construct. For example, within the gross motor sub-domain of physical functioning, items assessing ‘crawling’ abilities were ordered as ‘lower’ ability level items, appropriate for younger children; whereas, items assessing ‘running’ abilities were ordered as ‘higher’ ability items, appropriate for older children. This review allowed for detailed consideration of a wide range of items that assess developmentally appropriate abilities for each domain.

Item pools were iteratively culled to ensure that the selected items assessed abilities and behaviors that were age-relevant and focused on health and developmental concepts that are important in the assessment of preschool-aged children’s burn recovery. Based on previous work by Brady et al., health and development concepts were related to physical functioning, psychological functioning, social functioning, and communication and language development for children aged 1–5 years old [25]. Candidate items were removed from item pools if they were poorly worded, conceptually too narrow, irrelevant to this age group, inconsistent with the domain definition, or represented a concept that could not be assessed consistently by parent-report. For example, in the psychological functioning domain, several items assessed pediatric sleep behaviors. The item, “My child had frightening dreams or nightmares” was selected over other items with similar content, including, “This child has more nightmares”, “[This child is] Alarmed by scary dream”, “[This child is] Often frightened by dreams or the nighttime”. Clinician focus groups and parent semi-structured interviews were conducted concurrently with the qualitative item review to further develop and refine item pool content. During the early phase of project development, semi-structured interviews included parents of children who were 1–2 years of age. These methods are described elsewhere [25].

2.2. Item pool refinement

2.2.1. Clinical expert consensus meetings

As a first step for item pool refinement, consensus meetings were conducted with clinical experts to modify and standardize items selected from the qualitative review. Clinical experts across four primary research sites (Shriners Hospitals for Children burn centers: Boston, Cincinnati, Northern California, and Galveston) worked collaboratively from April 2018 until August 2018. Clinical expert consensus meetings consisted of child life specialists, social workers, pediatric burn nurse practitioners, occupational and physical therapists as well as an elementary school teacher. Clinical experts used an iterative process to revise original items and reconcile the following inconsistencies: recall period solidification, item stem, and/or item body that required revision. The meetings served as an opportunity for experts to clarify item language, such as re-phrasing double-barreled or multi-barreled items to address a single construct. Clinical experts worked to select common English phrases, choose simple examples for certain items and avoided slang terms and/or regionally-understood jargon. Verb tense was standardized for each item within a domain. Literacy demands was also reviewed by clinical experts on the research team and items were simplified as needed.

An additional purpose of the expert meetings was to ensure comprehensive coverage of the four conceptual areas. For each item pool, 30–50 items were selected to reflect a representative sampling of the domain construct. Experts confirmed that selected items were clinically, developmentally, and culturally appropriate for assessment of parent-reported burn recovery among preschool-aged children. Final items underwent further rounds of expert consensus review to construct new items, and standardize recall periods, response options, verb tense, and literacy demands across all items for comprehension.

2.2.2. Parent cognitive interviews

Parents with a child aged 1–5 with a burn injury were recruited from Shriners Hospitals for Children — Boston to complete a one-time cognitive interview regardless of inpatient or outpatient status. Parents were additionally required to speak and understand English to meet eligibility criteria. Parents of children who sustained a burn injury prior to their first birthday but who were aged 1–5 at time of recruitment were invited to participate in the study. Potential parent participants were identified from electronic health records (EHR) of children within this age group who received inpatient and/or outpatient burn care at Shriners Hospitals for Children — Boston. Additional eligibility criteria included burn size with total body surface area (TBSA) equal to or greater than 5% and/or had burn injuries to critical areas such as face, hands, feet, or genitalia. Of 16 parents scheduled for cognitive interviews, only eight parents completed the cognitive interviews. Remaining eight parents did not answer at the scheduled time and study staff was unable to reach them at follow-up phone calls. Study staff attempted to re-contact parents by performing three follow-up calls at alternating morning, afternoon, and early evening intervals. If parents were unable to be reached, voicemails containing the study phone number and email were left. If all three follow-up calls were performed and a cognitive interview was unable to be rescheduled, the parent was deemed lost to follow-up.

One-on-one parent cognitive interviews were performed to evaluate candidate items. The focus of these interviews was to ensure item quality, relevance, and comprehensibility and each interview lasted approximately 45 min. By obtaining direct feedback from parents of pediatric burn survivors, improvements could be made to the four item pools to ensure that each item would be understood and interpreted as intended by the research team. The purpose of the cognitive interview was explained prior to the beginning of the interview along with completion of a brief demographic survey. An example was completed to ensure parents understood the process of the interviews (see Supplemental appendix). Cognitive interviews included potential survey items and background items (207 items). Parents were divided into two groups with each group reviewing 95 and 112 items, respectively.

Each candidate item was reviewed by at least four different parents to obtain feedback on language and clarity of the item pools. Items were presented one at a time to parents by Microsoft Power Point slides. Parents read each item aloud and a member of the research team queried each parent regarding item interpretation and response selection. Questions about item interpretation included: (1) What do you believe the item is asking?, (2) Is the item easy or hard to respond to?, (3) Should the item be reworded?, and (4) How do you think the item could be made clearer?. Additional follow-up questions were asked, tailored to individual parents’ responses. Candidate items were either refined and finalized or eliminated from item pools based on this parental input. Parent feedback was captured in written notes and synthesized. To synthesize parent cognitive interview data, the research team analyzed written notes using methods informed by grounded theory [27–29]. Grounded theory in the context of theoretical saturation used a constant comparative analysis process moving in and out of data collection and analysis [30]. Data were collected until saturation was reached. Recurring themes that occurred in the cognitive interviewing process indicated that a point of saturation was achieved.

2.2.3. Ethical approval

This study was approved by the Western Institutional Review Board (BOS1805E and BOS1807) and the Boston University Medical Campus Institutional Review Board (H-37821 and H-37814). Verbal informed consent of each participating parent was obtained for this study.

3. Results

3.1. Generating initial item pools

Identification of items from existing measures generated a total of 9509 initial candidate items.

3.2. Qualitative item review

Table 1 shows the corresponding sub-domains for the four item pools and measures used to derive candidate items. For example, similar items that addressed crawling, walking, and running were grouped together within the gross motor sub-domain of physical functioning. A total of 188 final items were included across the four item pools.

Table 1 –

Four item pool domains and subsequent sub-domain within each. Legacy measures listed represent example assessment tools and developmental checklists used to inform items.

Item pool domains and subdomains:	Example legacy measures used to inform item content for each domain:	Example items within each domain:
• Physical Functioning  ◦ Gross motor  ◦ Fine motor	• Survey of Well-being of Young Children Developmental Milestones [48] • PROMIS Parent Proxy Item Bank v2.0 — Upper Extremity [49] • Burn Outcomes Questionnaireo0–4 [14] • Child Development Inventory [50] • PROMIS Parent Proxy Mobility [51] • Ages and Stages Questionnaire-3 (ASQ) [52] • Battelle Developmental Inventory- Second Edition (BDI-2) [53] • CDC Milestones Checklist [1] • Bayley Scales of Infant Development [54]	• Kick a balla • Put on his/her socks without helpa
• Psychological Functioning  ◦ Internalizing   - Depression   - Withdrawal   - Anxiety   - Separation Distress  • Externalizing   - Aggression   - Defiance  • Dysregulation   - Negative emotionality   - Sleep behaviors   - Eating behaviors  • Trauma  • Toileting	• Burn Outcomes Questionnaire0–4 [14] • Infant and Toddler Social Emotional Assessment [55] • NIH Toolbox — Fear — Over Anxious (Ages 3—7) Parent Report [56] • Brief Infant and Toddler Social Emotional Assessment [57] • Survey ofWell-being ofYoung Children Pediatric Symptom Checklist [48] • Achenbach Child Behavior Checklist [58] • Ages and Stages Questionnaire: Social-Emotional (ASQ) [59] • PROMIS Parent Proxy Item Bank v2.0 — Anxiety [60] • PROMIS Parent Proxy Item Bank v2.0 — Depressive Symptoms [60] • PROMIS Pediatric Item Bank v1.0 — Positive Affect [61] • Temperament and Atypical Behavior Scale (TABS) [62] • KiddyKINDL parent questionnaire 4—6 years [63] • Pediatric Symptom Checklist (PSC) — 35 [64] • PROMIS Parent Proxy — Physical Stress Experiences [65]	• Seemed unresponsive to affectionb • Seemed withdrawnb • Needed to be told over and over things were okayb • Cried or hung on to me when I tried to leaveb • Was aggressiveb • Destroyed things belonging to othersb • Was whiny or fussy when he/she was not tiredb • Had trouble falling asleep at nightb • Refused to eatb • Avoided doing this that reminded him/her of the injuryb • Used the toilet on his/her ownb
• Social Functioning  ◦ Connecting with family/peers  ◦ Friendships  ◦ Play	• NIH Toolbox Fixed Form v2.0 — Companionship — Positive Peer Interactions (Ages 3—12) Parent Report [56] • Infant and Toddler Social Emotional Assessment [55] • Burn Outcomes Questionnaire0–4 [14] • Survey of Well-being of Young Children [48] • Brief Infant and Toddler Social Emotional Assessment [57] • Ages and Stages Questionnaire: Social-Emotional (ASQ) [59] • NIH Toolbox Fixed Form v2.0 — Companionship — Social Withdrawal (Ages 3—12) Parent Report [56] • NIH Toolbox Fixed Form v2.0 — Social Distress — Peer Rejection (Ages 3—12) Parent Report [56] • PROMIS Parent Proxy — Peer Relationships [66]	• Got along with other children of the same sexb • Had at least one favorite friend (a child)b • Initiated familiar play routinesb
• Communication and Language Development  ◦ Receiving language  ◦ Producing language	• American Speech-Language Hearing Developmental Guide [67] • Burn Outcomes Questionnaire0–4 [14] • Survey of Well-being of Young Children [48] CDC Milestones Checklist [1]	• Could follow two-part instructions- like “Get the spoon and put it on the table”a • Used 4—5 word sentencesb

^aItem aims to assess ability; response options: unable to do, with much difficulty, with some difficulty, with little difficulty, with no difficulty.

^bItem aims to assess frequency of behavior: response options: never, rarely, sometimes, often, always.

3.3. Clinical expert consensus meetings

The final review standardized recall period to “In the past 7 days . . . ”. During cognitive testing, parents confirmed they were able to remember various observations and behaviors in this time frame. Parents responded, “I wouldn’t go any further beyond 7 days in terms of recall period”, “[In the past 7 days . . . ] recall makes it easier to answer”, and “Time frame of 7 days is good for recall”. This observation was consistent with previous PROM research demonstrating that a shorter recall time frame reduces bias and response shift problems [31]. Final consensus meetings also assigned a 5-point Likert scale set of response options to each item. Two different response options were considered based on whether the item content assessed ability or frequency. For items that assess ability, response options included “Unable to do”, “With much difficulty”, “With some difficulty”, “With little difficulty”, and “With no difficulty”. Response options for items that assess frequency of behaviors included “Never”, “Rarely”, “Sometimes”, “Often”, and “Always”. Finally, clinical experts modified items so that certain concepts were both negatively and positively phrased. For example, within the psychological functioning domain, two items included are “My child seemed sad or unhappy” and “My child seemed happy”. Negatively phrased and positively phrased items were included to potentially reduce both acquiescent and extreme response biases. Including both positive and negative wording provides more expansive, comprehensive coverage of the metric for the domain at both higher and lower levels [32].

3.4. Parent cognitive interviews

3.4.1. Study demographics

The cognitive interview study population consisted of eight parents of pediatric burn patients aged 1–5. On average, the parent sample age was 34±5.5 years with a greater proportion (62.5%) being female. Parent cognitive interview participants identified as White (50%) and a majority (75%) had completed some form of higher education (Bachelor’s degree or greater). Of the eight parents, three had children in the 1–2 age group and five had children in the 3–5 age group. Children were mostly male (87.5%) and on average, 24 months had elapsed since the date of the children’s burn injuries (Table 2).

Table 2 –

Cognitive interview parent participants and child demographics and characteristics of the burn injury (n = 8).

Parent
Age (years), mean (sd)	34 (5.5)
Male, n (%)	3 (37.5%)
Race, n (%)
White	4 (50%)
Black or African American	2 (25%)
Other	2 (25%)
Education, n (%)
Completed high school or equivalent	1 (12.5%)
Completed some college	1 (12.5%)
Completed Bachelor’s degree	4 (50%)
Completed Master’s degree	2 (25%)
Living with spouse/partner, n (%)	7 (87.5%)
Children under 18 living at home, mean (sd) (n = 7)	2.6 (0.8)
Child
Age at time of burn, n (%)
1 to <3 years	7 (87.5%)
3—5 years	1 (12.5%)
Age at time of interview, n (%)
1 to <3 years	3 (37.5%)
3—5 years	5 (62.5%)
Male, n (%)	7 (87.5%)
Months elapsed since burn injury, mean (sd)	24.1 (7.8)

3.4.2. Parent cognitive interviews feedback

Parents were able to comprehend the majority of candidate items and had minimal critiques for recall period and response options. Eight cognitive interviews identified issues with comprehension and relevancy which led to alternative wording for select items. Parents identified language that was difficult or confusing for them to understand and offered suggestions on how to better re-phrase items. The following four broad themes of concerns about item content emerged from the cognitive interviews: (1) item age-appropriateness, (2) the use of qualifier language within items, (3) items addressing extreme behaviors, and (4) issues with semantic meaning. Of the 207 items presented to participants during cognitive interviews, 5 were removed, 36 items were reworded for the final item pool, 1 item was separated into two items, and 7 new items were added. Table 3 highlights examples of original items and how items were refined based on parent cognitive interview test findings.

Table 3 –

Examples of candidate items before and after cognitive interview-based refinement.

Original item	Notes from cognitive interview	Broad theme of concern	Final item
My child used a single word with meaning.	• Seems too young for child; my child is able to currently speak in paragraphs; sparse out questions for age	Item age-appropriateness	My child used a single word with meaning.
My child made sounds like ‘ga,’ ‘ma,’ or ‘ba’.	• Same age issue; child more articulate because ofage • Babbling and baby talk; is the child babbling/trying to communicate	Item age-appropriateness	My child made sounds like ‘ga,’ ‘ma,’ or ‘ba’.
My child used a lot of new words.	• Child is acquiringnewwords; Suggestion to take out “a lot” • My child is developing a vocabulary; Have I noticed my child using new words?; “A lot” makes it hard to respond to this. • “A lot” is subjective; Not sure if we should quantify; Depending on the child and development ofthe child, there could be a delay; Child has a growing vocabulary	Use of qualifier language	My child used new words.
My child worried a lot or was very serious.	• Not sure about “very serious”; Makes more sense for parent if it was my child worried a lot; Worried and serious do not go hand and hand; “Serious” could be interpreted as focused, or concentrating; Does not view “serious” as a bad thing • Anxious; Maybe more important to a parentwho has an anxious child	Use of qualifier language	My child was worried.
My child hurt animals on purpose.	• Pretty straightforward; Aggressive question; A bit startling; Even if (my) child did hurt animals, I would answer never	Extreme behaviors	Candidate item removed from psychological functioning item pool.
My child could say words like “feet” for more than one foot and “men” for more than one man.	• Suggestion to not use “feet” as an example • Child understands “feet” as an actual body part	Semantic meaning issues	My child could say words like “men” for more than one man.
My child copied the gestures of others.	• Confused by copying gestures; I don’t understand what you’re getting at; Gesture could be waving — maybe my child waved back at someone who waved to him/her • Is this talking about good or bad gestures?; This would change my response; Suggestion to put it more in the middle	Semantic meaning issues	My child copied the gestures of others- like waving back to someone.

Age-appropriateness of items in relation to a child’s current abilities and developmental progress was identified as an issue for a number of potential candidate items. Some parents had difficulties responding to questions that were outside of their child’s current developmental age range. To remedy this issue, 3 candidate items were rephrased, 2 were eliminated, and 2 new items were added. Additional age-related language was also included in the pre-survey instructions section: “Some questions may not seem age-appropriate for your child, but it is important you still respond to these items based on the response that best describes your child. Even if you are unsure, try your best to answer every question.” This pre-survey language was added not only in response to the parent cognitive interviews but also for the purpose of future field testing. As part of item pool field testing, parent participants are required to answer all of the items for psychometric analyses and our aim was to deter parents from skipping any questions they deemed age inappropriate.

Parents also had challenges with qualifier language within item text. They vocalized having difficulties conceptualizing terms such as “a lot” and “very.” Parents deemed qualifier language as too subjective and suggested that these terms be removed from item text. In response to this issue, the research team removed qualifier language from 4 candidate items, eliminated 1 candidate items, and added 1 new item to prevent parental misunderstanding in later field testing. Simplifying item language was a direct result of parent suggestions from the cognitive interviews and was applied by the research team during clinical expert consensus meetings.

Item content that addressed extreme behaviors generated important feedback from parents. For example, the research team eliminated the candidate item, “My child hurt animals on purpose” from the psychological item pool in response to parental input. During cognitive interviews, some parents commented that these items were “aggressive” and “startling.” In addition, one parent indicated that even if his/her child engaged in these behaviors, he/she would not endorse the item.

The final theme that emerged from parent cognitive interviews was concern about semantic meaning. Parents indicated that some items were too vague and recommended clarifying these ambiguous items with concrete examples. For example, parents reported being confused by the item, “My child copied the gestures of others.” They stated, “I don’t understand what you’re getting at” and “Is this talking about good or bad gestures?”. Twenty-eight items were revised to include clarifying language, 1 item was removed, and 4 new items were added. In addition, some item language garnered parental responses about containing double meaning and these items were also amended.

3.5. Final item pools

Four domains emerged as important when assessing recovery after a burn injury among preschool-aged children: (1) communication and language development (42 items) to assess the child’s ability to receive meaning and produce language; (2) physical functioning (53 items) to assess the child’s gross and fine motor abilities; (3) psychological functioning (56 items) to assess the child’s emotions and behavior (internalizing and externalizing behaviors, dysregulation, toileting, and response to trauma); and (4) social functioning (37 items) to assess the child’s social participation and abilities. Table 4 summarizes the item pool content for each domain.

Table 4 –

Mapping finalized item pools to the conceptual framework of parent-reported child health outcomes after burn injury for children 1-to-5 years.

Functioning
Physical Functioning (n = 53)	Psychological Functioning (n = 56)	Social Functioning (n = 37)	Communication and Language Development (n = 42)
Push his/her arms through a shirt	Had a good appetite	Played games- like “peek-a-boo” or “pat-a-cake”	Used a single word with meaning
Tie his/her shoelaces without help	Was impatient or easily frustrated	Tried to get my attention by saying “Look at me”	Made sounds like “ga,” “ma,” or “ba”
Button his/her shirt or pants	Cried a lot	Initiated familiar play routines	Called me “mama” or “dada” or a similar name
Draw a square	Got upset if things were not done a certain way	Showed awareness and interest in others	Copied sounds I made
Hold a cup	Had frightening dreams or nightmares	Imitated an adult’s previous action during a play activity	Looked around when I said things like “Where’s your bottle?” or “Where’s your blanket?”
Draw recognizable pictures	Got angry or pouted	Appearance interfered with his/her relationships	Pointed to pictures in a book when I named them
Help turn pages in a book, turning more than one page at a time	Had toileting accidents	Had at least one favorite friend (a child)	Used new words
Draw a line	Had trouble adjusting to changes	Pretended objects were something else (for example, used a banana as a phone)	Talked a lot
Hold up his/her arms to be picked up	Had trouble staying asleep	Imitated clapping or waving “bye-bye”	Pointed to something far away to show someone what he/she wanted
Draw a complete circle	Refused to eat	Liked to play near and be with family members and friends	Used words like “me” or “mine”
Cut paper with scissors	Was whiny or fussy when he/she was not tired	Played house with other children	Used words to ask for help
Open a jar by himself/herself	Got very upset	Played well with other children (not including brothers or sisters)	Used 4—5 word sentences
Put on his/her clothes without help	Had trouble falling asleep at night	Hugged our fed stuffed animals	People understood when my child spoke
Write or draw with a pen or pencil	Was hard to soothe when upset	Pretended to do grown-up things- like talking on the phone	Used words like “yesterday” and “tomorrow” correctly to talk about things in the past and future
Stand up on his/her tiptoes	Was restless and couldn’t sit still	Showed pleasure when he/ she succeeded (for example, clapped for self)	Asked “Why?”
Turn door handles without help	Was worried	Took turns when playing with others	Had a word for many things
Put on his/her socks without help	Destroyed things belonging to others	Fought with other children	Followed other people’s conversations
Zip up his/her clothes	Behaved well	Laughed or smiled when I played with him/her	Could follow directions- like “Come here” or “Give me the ball”
Pick up a cheerio with his/her hand	Hit, bit, or kicked me (or other parent)	Would ask for things nicely when playing with other children	Copied the gestures of others- like waving back to someone
Put on a T-shirt	Felt like something awful might happen to him/her	Had trouble playing with other children	Could say his/her first name when asked
Scoop with a spoon and bring it to his/her mouth	Was disobedient or defiant	Played alongside other children	Could point to at least three body parts- like nose, hand, or tummy
Stack three small blocks or toys on top of each other by himself/herself	Quieted down when I said “shh”	Avoided other children	Could follow three-part instructions- like “Point to the cat, dog, and monkey”
Stay in the lines when coloring	Used the toilet on his/her own	Got along with other children of the opposite sex	Could ask questions beginning with “why” or “how”- like “Why no cookie?”
String small items- like beads, macaroni, or pasta “wagon wheels” on to a string	Was able to show feelings	Was left out by other children	Put two or more words together-like “more water” or “go outside”
Use the pads of his/her fingertips to grasp a crayon or pencil	Laughed easily or a lot	Played with friends	Could name at least one color
Walk down stairs without holding on to anything	Did not feel much like doing anything	Got along with other children of the same sex	Could name at least three body parts- like nose, hand, or tummy
Get around the room without help	Hung on to me or wanted to be in my lap when with other people	Was not well liked by other children	Could connect two or more thoughts to tell a simple story
Climb up a ladder at a playground	Wanted to be left alone	Was picked on by other children	Could point to at least two familiar things- like cat or dog
Pedal a tricycle (3-wheel bike)	Seemed sad or unhappy	Kept to himself/herself	Could follow two-part instructions- like “Get the spoon and put it on the table”
Climb onto a sofa	Got very upset if reminded of the injury	Wanted to play with other children	Could use two words together with meaning
Pull a toy while walking	Was affectionate with loved one	Liked playing with other children	Could answer questions like “What do you do when you are cold? or “when you are sleepy?”
Crawl	Let me know when he/she needed to use the toilet	Shared when playing with other children	Could follow simple instructions like- “Give ball to daddy”
Jump off the groundwith two feet	Had sudden changes in mood or feelings	Liked being around children his/her age	Could compare things- using words like “bigger” or “shorter”
Walk up stairs with help	Seemed to have no energy	Played alone	Could say words like “men” for more than one man
Bend over to pick something up	Seemed unresponsive to affection	Was teased and made fun of by other children	Could tell me a story from a book or TV
Carry a backpack	Seemed withdrawn	Played with adults he/she knows well	Could name at least two familiar things- like cat or dog
Kick a ball	Was in a good mood	Played with an imaginary friend	Could name the days of the week in the correct order
Get onto a bed by himself/herself	Hit, shoved, kicked, or bit children (not including brother or sister)		Could take turns talking in conversations
Get up from a regular toilet	Was worried about what could happen to him/her		Could name most familiar things
Walk beside furniture while holding on with only one hand	Reenacted getting injured in his/her play (such as with toys or dolls)		Could sing a song or say a poem from memory such as his/her “A—B—C’s” or “Happy Birthday”
Turn pages in a book, one page at a time	Seemed nervous, tense, or fearful		Could name pictures in books
Walk up stairs without holding on to anything	Was aggressive		Could use simple gestures- like shaking his/her head “no”
Carry an object that could be held in one hand	Complained of feeling sick before separating from those he/she is close to
Pull himself/herself up to a standing position	Cried or hung on to me when I tried to leave
Ride a two-wheeled bicycle with training wheels	Smiled
Get off a bed by himself/herself	Worried about being separated from a loved one
Ride a two-wheeled bicycle without training wheels	Felt energetic
Squat without help	Avoided doing things that reminded him/her of the injury
Turn his/her neck to look over his/her shoulder	Seemed happy
Lift a cup to drink	Reported more physical complaints when reminded of the injury (headache, stomachache, nausea, difficulty breathing, etc.)
Walk holding on to people or furniture	Startled easily (for example, he/she jumped when hearing sudden or loud noises)
Run	Needed to be told over and over things were okay
Walk without help	Used the toilet on his/her own but needed help with wiping, redressing, washing hands, etc.
	Showed little interest in the things around him/her felt comfortable
	Hurt himself/herself on purpose (for example, banged head)

4. Discussion

This study developed comprehensive item pools based on the conceptual framework previously developed for the Preschool LIBRE CAT Profile. These item pools provide a solid foundation for continued research that will add to the growing field of outcomes research directed towards tracking and improving recovery and rehabilitation efforts for burn survivors using CAT-based PROMs [15,16]. Item pool development processes identified items from existing PROMs and optimized extant item content using an approach focused on co-production using parental feedback and clinical expert review.

This work aims to advance measurement beyond the limitations of traditional, legacy measures which require all respondents to complete all items in a survey, even if some of the items are less relevant for the individual respondent. Traditional legacy assessments require administration of large numbers of items to adequately assess a child’s functioning and recovery across multiple health domains. For example, the Shriners Hospitals for Children/American Burn Association Burn Outcomes Questionnaire for Children 0–4 (BOQ_0–4) is ones of a few burn-specific outcomes measures available to assess long-term outcomes in preschool-aged children. The BOQ_0–4 is a historical measure that was an important landmark PROM in the burn literature. The assessment is administered by paper and pencil or telephone and requires respondents to complete all 55-items, across ten domains [12–14,33–35,37,38]. While the large number of items ensures that the instrument assesses activities that represent the developmental continuum, the resulting assessment is cumbersome, burdensome for the respondent, and not practical for administration in the clinic setting [19,39,40].

By building upon on the foundations of existing PROMs, preliminary research has shown evidence of the feasibility of using electronic platforms to collect outcomes data [7]. In addition to being conceptually comprehensive and timesaving, a CAT potentially can ease data handling and monitoring. Use of CAT-based assessments offer the opportunity to examine recovery of the burn survivor by integrating the derived scores with an electronic health records (EHR) system [7,10]. This is not unlike the results of a clinical test monitored over time in the EHR. Connectivity between an EHR system and PROMs data could potentially provide clinicians with information that directly impacts a patient’s course of care. In addition, a fully integrated system has the potential to facilitate patient – clinician communication, patient engagement, and could lead to improved outcomes and increased patient care satisfaction [41].

It is important to state that the use of a CAT-based metric is not always feasible in routine clinical practice. CATs are computer-based platforms that are dependent on technology such as laptops, desktops, tablets, and cell phones for use. In order to address possibilities where computers are not available to administer a CAT, a fixed short-form format can be administered by mail or telephone. This will put the subject on the same common metric. Additionally, short forms can be administered by simple computer interface with a researcher recording responses during a direct in-person interview or telephone interview.

Another finding of this study supports previous literature emphasizing the importance of using qualitative interviews to inform item content when developing new PROMs [16,42–44]. Recent metric development guidelines recommend obtaining direct feedback from patients and families as early as the conceptual framework stage and through item generation [3]. Marrying clinical and parental input provides opportunities to frame the vision of a domains framework and inform subsequent item content. This process ensures that items in the outcome metric represent relevant health and age-appropriate domains as determined by clinicians and patients/parents. By partnering with future end-users, the research team can build a conceptual model to generate items that cover the full spectrum of the underlying construct and address the multidimensional nuances of the relationships across the content areas described. Stemming directly from domains represented in the conceptual model, high quality item pools include items of different levels of ability, laying the foundation for the CAT-based PROM. A qualitative review of item difficulty along the continuum of the metric can ensure that potential gaps in content are covered. Following field study data collection, IRT-based psychometric analyses will convert these item pools into hierarchically organized calibrated item banks which will exist behind the scenes of the CAT. Both the parents and clinicians can play an important role where synergies involve input from both parties early on in this process. [15,22].

Limitations of the study should be noted due to potential impact on the transferability of our findings. First, each item was reviewed by at least four different parents during cognitive interviewing. When designing new questionnaires, some authors recommend conducting 10–15 interviews to identify conceptually difficult domains and items [44]. We aimed to mitigate the limited number of cognitive interviews through extensive clinical expert review and decided to halt interviews once saturation was reached. Second, while some items may have been derived from pre-existing valid and reliable metrics, these items may function differently due to our study changes in item stem, body, and/or recall period. However, this approach was used with an eye towards IRT-based psychometric analyses, using a unidimensional approach for each of the conceptually-based domains. Third, the conceptual model for Preschool LIBRE_1–5 aimed to include a family domain in the instrument [25]. During expert consensus meetings, we recognized that family is an important domain with important subtleties and nuances. Another concern related to increasing respondent burden when field-testing the item pool. To give proper justice to family items, ancillary items were added with five of sixteen total items related to parental satisfaction and family functioning. Considering a family-focused assessment with other detail and subdomains will be a further focus for a future study.

While the content validity of the item pools were established by the conceptual grounding of the well-known and highly credible World Health Organization’s International Classification, Disability, and Health for Children and Youth and interviews with parents of pediatric burn survivors, there is currently no empirical grounding that each construct is separate and unique [25]. Content validity of a PROM is at risk if the measure insufficiently captures or fails to capture important health domains that are relevant to the specific population [3]. Failure to generate initial quality items can result in loss of precision and accuracy, thus limiting the instrument’s responsiveness and effectiveness during later psychometric testing and clinical use. Future studies will include psychometric analyses, including confirmatory factor analysis and IRT-based analysis to evaluate and validate the domains and items to be administered by the Preschool LIBRE CAT Profile [45–47].

Despite these limitations, these findings represent an important contribution to the field of burn care and outcomes research for assessing health and recovery after burn in children aged 1–5. The use of the Preschool LIBRE CAT has the potential to shift clinical practice with the use of a low burden parent-reported PROM that uses cutting edge psychometric approaches as part of the treatment process. Future work will also include the development of measures for older age groups, such as school-aged children and teenagers. Once these age-specific instruments are developed and psychometrically tested, research teams would be able to utilize age-specific metrics to track outcomes across a patient’s childhood by bridging assessments throughout their age span in these formative years. The development of these instruments allows for age-specific cross-comparison of different domains and outcomes, and the opportunity to conduct longitudinal studies. Additionally, future work will have greater focus of cultural representation across different populations. As a future evaluative measure for nurses, physicians, other interdisciplinary care providers, and parents, the Preschool LIBRE may be used to identify the needs for outpatient care in routine clinical practice. Clinical and research use of the Preschool LIBRE CAT has the potential to optimize interventions and personalize care for young children with burn injuries.

5. Conclusions

Continued expansion and improvement of the BOQ_0–4 is critical to understanding the impact of burn injuries in children 1–5 years of age. We applied the Preschool LIBRE Conceptual Model to guide the development of item pools that address four conceptual areas impacted by burn injuries in preschool-aged children. The resulting 188 items represent item coverage across the four functioning sub-domains of the framework, including: (1) communication and language development, (2) physical functioning, (3) psychological functioning, and (4) social functioning. Our item development methodologies are the result of an iterative qualitative group consensus process across four primary research sites. By building upon item content derived from existing PROMs, this work forms the foundation and will guide the development of calibrated item banks for a new parent-reported, CAT-based outcomes instrument that measures recovery after burn in preschool-aged children, the Preschool Life Impact Burn Recovery Evaluation (LIBRE) CAT Profile.