Considerations to Support Use of PROMIS Pediatric Measures in Ambulatory Clinics

Abstract

Objective:

To identify challenges to the use of Patient-Reported Outcomes Measurement Information System (PROMIS) Pediatric measures in the ambulatory pediatric setting and possible solutions to these challenges.

Study design:

18 semi-structured telephone interviews of health system leaders, measurement implementers, and ambulatory pediatric clinicians were conducted. Five coders used Applied Thematic Analysis to iteratively identify and refine themes in interview data.

Results:

Most interviewees had roles in leadership or the implementation of patient-centered outcomes; 39% were clinicians. Some had experience using PROMIS clinically (44%) and 6% were considering this use. Analyses yielded six themes: 1) selection of PROMIS measures, 2) method of administration, 3) use of PROMIS Parent Proxy measures, 4) privacy and confidentiality of PROMIS responses, 5) interpretation of PROMIS scores, and 6) using PROMIS scores clinically. Within the themes, interviewees illuminated specific unique considerations for using PROMIS with children, including care transitions and privacy.

Conclusions:

Real world challenges continue to hamper PROMIS use. Ongoing efforts to disseminate information about the integration of PROMIS measures in clinical care is critical to impacting the health of children.

Patient-reported outcome (PRO) measures quantify patient health and health-related experiences directly from the patient perspective,[1] which is important for patient-centered care.[2, 3] PROs can be used to monitor trends in patients’ symptoms, function, or well-being, to inform decision-making, and to prompt additional patient education or referrals.[4] Use of PRO scores in clinical practice has improved recall of patient concerns by clinicians,[5–11] increased shared decision-making,[5] and enhanced care processes and treatment planning.[10, 12–14] At the health system level, PROs can inform programmatic decisions and resource allocation by contributing data on the quality and value of clinical services from the perspective of patients.[15–18]

The National Institutes of Health (NIH) Patient-Reported Outcomes Measurement Information System (PROMIS) provides standardized PRO measures for use in clinical practice, with specific measures available for adults, children from 8–17 years of age, and parent proxies for children 5–17 years old.[19] PROMIS Pediatric measures produce T-scores with a mean of 50 in a reference population (typically the US general population or US general population supplemented with a clinical sample) and standard deviation of 10. Higher scores may indicate better health (eg, physical function) or in other cases suggest poorer health (e.g., fatigue). PROMIS measures are based on item banks calibrated to a common scale using item response theory.[20] For pediatric use, PROMIS measures are currently available for over 25 physical, mental, and social health concepts. Although item banks available for each health concept, measures are commonly administered as short forms (4–10 items per concept) or as computer adaptive tests (CATs) that tailor the items administered based on how the patient responds to each question. PROMIS measures are generally not condition-specific and can be used for children with any health condition. Each measure has undergone rigorous psychometric evaluation, providing evidence for reliability and validity in a broad range of pediatric populations with different conditions and diseases.[21, 22]

To realize the benefits of using PROMIS measures in pediatric clinical settings, health system and clinician leaders must attend to how the measures are implemented and used, as well as the support required to achieve this goal. Although there are several guidance documents about the clinical use of PROs broadly,[16, 23–25] there is a shortage of guidance specific to PROMIS in ambulatory pediatric settings. PROMIS use must account for nuances unique to pediatric care, such as the multiple longitudinal encounters occurring over a child’s development, often necessitating parent proxy reporting initially and subsequent transition to the child’s self-report.[5, 26–28] Existing work offers limited information to support health systems and clinics in considering potential challenges to PROMIS implementation and use in the context of ambulatory pediatric settings. We address this gap by using qualitative research methods to understand the real-world barriers to PROMIS Pediatric clinical use as identified by clinicians and health system leaders.

Methods

To understand the challenges, considerations, and potential solutions related to the use of PROMIS in the ambulatory pediatric setting, the project lead and two researchers conducted 18 semi-structured telephone interviews of health system leaders, PROs implementers, and ambulatory pediatric clinicians with expertise or interest in implementing PROs or PROMIS measures in pediatric settings. Interviewees were recruited through a combination of literature review, PROMIS expert knowledge of institutions that had either implemented or purchased a license for pediatric PROMIS, and snowball sampling from interviewees. The interview guide (Appendix; available at www.jpeds.com) was informed by a 7-member Steering Committee of experts in pediatric PROs or healthcare quality measurement, and by reviewing existing guidance on PRO implementation.[1, 5, 15, 24, 29–31] To introduce the topic and guide selection of questions from the guide, we began interviews by learning about the interviewee’s role and experiences with PROMIS. We probed about unique challenges to PROMIS use in pediatrics and specific populations.

We also queried about resources that were helpful during the implementation process. Depending on the background, experiences, and expertise of the interviewees, different sections of the interview guide were prioritized. The study was deemed exempt from ethics review. Interview length ranged from 38–51 minutes.

Transcribed interviews were coded in NVivo by five trained coders using Applied Thematic Analysis.[32] Coders independently read a subset of the interview transcripts and identified themes. The team then met regularly to discuss and refine the thematic coding, resolving differences in between coders through discussion. Through an iterative process of coding, analysis, and categorization, themes and subthemes were identified and coded in the interview data. A codebook was iteratively refined throughout coding. The purpose of this inductive, data driven analysis approach was to identify issues related to PROMIS clinical use and relevant to pediatric settings, as described by the interviewees.

Results

The majority of interviewees were employed at academic medical institutions (83%) located across all regions of the US (28% in West, 28% in Midwest, 17% in Northeast, and 28% in South) (Table 1). Many interviewees were health systems leaders (56%); 72% had roles in implementing PROs and 39% were clinicians. Some interviewees had used PROMIS in pediatric clinical settings (44%), and another 28% had used PROMIS, but not within a pediatric clinical setting. Still others (22%) had used PROs, but not specifically PROMIS, and another 6% expressed interest in implementing PROs or PROMIS for clinical use. Health system leaders included directors of clinical or academic programs and initiatives. Clinicians were generalists or subspecialists in areas like endocrinology, neuropsychology, or pediatric surgery. Interviewees had a wide range of years of experience, with a median of 14 years. The analysis of the interview content yielded six themes specific to PROMIS use in ambulatory pediatric settings: 1) selection of PROMIS measures, 2) method of administration, 3) use of PROMIS Parent Proxy measures, 4) privacy and confidentiality of PROMIS responses, 5) interpretation of PROMIS scores, and 6) using PROMIS scores clinically (Table II).

Table 1.

Organization affiliations and experience of interview participants, N=18^*

Organizational characteristics	% (n)
Academic medicine	83% (15)
Non-academic	17% (3)
US census region
West	28% (5)
Midwest	28% (5)
Northeast	17% (3)
South	28% (5)
Interviewee Role†		Definition
Health systems leader	56% (10)	Provided PROs approval or resources
PROs implementer	72% (13)	Implements PROs in clinics
Clinician	39% (7)	Conducts clinical care
Interviewee Experience		Definition
Years of experience (median (IQR))	14 (8, 25)
Using PROMIS clinically in pediatrics	44% (8)	Experience with PROMIS in pediatric clinical settings
Using PROMIS	28% (5)	Experience with PROMIS in non-clinical or non-pediatric settings
Using PROs	22% (4)	Experience with PROs (not specifically PROMIS) in any setting
Considering PROs for clinical use	6% (1)	Can include PROs or PROMIS

^*Values may not add to 100% due to rounding

^†Values will not add to 100% due to interviewees having overlapping roles and experiences

Table 2.

Six identified topics specific to PROMIS use in ambulatory pediatric settings from interview data

Selection of PROMIS measures

“We’re in very early stages of understanding […] what this is going to look like in a cerebral palsy clinic, […] in a really busy spine clinic, and back to the drawing board about consensus decisions about which domains are going to be chosen.”

“We got some feedback early on that families of patients who are in wheelchairs do not like answering questions about, ‘Can you stand? Can you walk?’ […] because they don’t know how to answer.”

“For the Spanish language, I’ve had some of our translators tell me, ‘That’s not really what it means.’ So I think there’s a problem with the different dialects.”

Method of administration

“Clinicians want to do the right thing, but make it [PROMIS] a part of what they do, build it into their order set, build it into their chart.”

“For a long time, and it’s still true for a heavy majority, you have to transfer the data from the paper-based into whatever system or visit note. So it was clunky.”

“We’re able to integrate our PROs directly into the EHR […] through this […] tablet platform, which is like a touch screen, which is kind of easier for teens to interface with.”

Use of PROMIS Parent Proxy measures

“The thing I hear the most about is the issue of self-report versus parent-report and how do you know which one to do, what you do when you’re transitioning between a proxy-report to a self-report because they are measuring slightly different things, and then what do you do at their other transition from adolescence to young adulthood?”

“[W]e can do a parent and a child score on the same encounter, but […] the problem was, if we were looking back at scores, which score do we use.”

Privacy and confidentiality of PROMIS responses

“I think our patients refuse to provide information when they have concerns about how is this— they don’t know how this information is being used, […] who has access to it, […] how it’s being stored—securely or not. And I think those are all very legitimate concerns.”

“So that gets into the privacy issue. […] In discussions with our legal department, what we have decided to do is […] the first time [families] come to clinic, they get an information page that says, ‘these are data within the medical record. Whoever has access to the chart has access to see this.’”

“What we found is that sometimes even when it says, ‘Please hand this to your child,’ if it’s an iPad or tablet or even a paper, the parent is still filling out the forms.”

“So, in the teenage population, I think that the best thing that can happen is if you give the parents a job of filling out a survey and kid a different job on a different tablet or paper and fill out there. So that everybody is occupied.”

“We have one screener that has a flag [in the EHR], but for some reason it’s not very visible to the clinicians. So that’s actually the next hurdle I need to figure out in clinic: If people are screening positively and asking for help, we need to actually do something.”

Interpretation of PROMIS Pediatric measure scores

“We had quite a debate about what threshold of standard deviation score we should use to identify who is higher risk. Should it be 2 standard deviations, or 1 or…”

“When [clinicians] say, ‘I want to use [PROs] to track how my patients are doing […], then it’s a question of ‘okay, well what information would be helpful to have,’ because […] we don’t have all of the interpretation information we would want.”

Using PROMIS scores clinically

“There’s a need to identify who to train, to have clear, consistent messaging about why we’re asking patients to do this […] how to introduce it, and then training on where you find scores, how you interpret scores, how you might use them.”

“It just gives me a nice way to, in just a few seconds, kind of eyeball where parents are at in terms of how many concerns they may have […] and it also gives me direction to follow up on the ones that they’ve endorsed, and I don’t have to waste time asking about the other ones.”

“So for that [depression] measure, with a given score, […] for moderate [scores], refer to social work. High [scores], refer to social work and behavioral health.”

Selection of PROMIS measures

Interviewees identified several challenges related to selecting PROMIS measures for clinical use. With the large number of PROMIS Pediatric (and other PRO) instruments available, health systems needed to make decisions on which measures to use, whether to use the same measures in all clinics and with all patients, and what process to use to guide those decisions. For example, one interviewee highlighted the different needs that may arise in different specialty clinics and attempts to gain consensus on this: “We’re in very early stages of understanding […] what this is going to look like in a cerebral palsy clinic, […] in a really busy spine clinic, and back to the drawing board about consensus decisions about which domains are going to be chosen.”

Additional topics around measure selection arose for specific clinic populations. One interviewee recounted concerns from patient stakeholders about the wording of certain items related to mobility: “We got some feedback early on that families of patients who are in wheelchairs do not like answering questions about, ‘Can you stand? Can you walk?’ [...] because they don’t know how to answer.” Another special population that interviewees mentioned are non-English-speaking children or parents. PROMIS Pediatric measures have been translated into many languages. Even so, one interviewee reported concerns about the understanding of items: “For the Spanish language, I’ve had some of our translators tell me, ‘That’s not really what it means.’ So I think there’s a problem with the different dialects.”

Method of administration

Interviewees had used PROMIS measures through a variety of administration systems. Some had experience using data capture systems such as REDCap, some had PROMIS measures integrated into the electronic health record (EHR) system, and some recounted experiences using pen-and-paper administration. The ease of using PROMIS and having the data available was highlighted as an important condition for acceptance by physicians: “Clinicians want to do the right thing, but make it [PROMIS] a part of what they do, build it into their order set, build it into their chart.” Similarly, interviewees highlighted that capturing PROMIS data electronically and integrating it into the EHR also provided benefits for child patients who completed the measure. One respondent described how in their system they “integrate our PROs directly into the EHR […] through this […] tablet platform, which is like a touch screen, which is kind of easier for teens to interface with.” Conversely, administration by pen-and-paper posed challenges in making the data available to providers, as paper forms first needed to be entered into an EHR, resulting in delay and additional staffing needs: “For a long time, and it’s still true for a heavy majority, you have to transfer the data from the paper-based into whatever system or visit note. So it was clunky.”

The reason why some health systems nonetheless relied on pen-and-paper administration were the logistics of integrating measures into EHR systems and the resources required for doing do. An interviewee who at the time of the interview scored PROMIS measures by hand and entered them into the EHR herself described how more integration would be desirable, but faced a long timeline for implementation: “Down the line, we’d love to put [PROMIS measures] in a flow sheet where you can just enter all the things, it will automatically score, and then it’s in the medical record. But I know the waiting list for [EHR software provider] build requests is like, two or three years long right now.” Paper-based surveys, then, were a way to bypass this long and resource-intensive process of integration into the EHR, as one interviewee described: “[F]or one screener, it took us a year to get approval with lots of presentations and preliminary data and things like that. But if I wanted to do a paper-based survey, they [health system leadership] don’t care.”

Use of PROMIS Parent Proxy measures

Several interviewees reported using Parent Proxy measures, which exist for all PROMIS Pediatric measures. Proxies provide an opportunity to collect data from younger children, children who are unable to complete the measures by themselves, or to get data from both the child’s and the parent’s perspective. However, interviewees highlighted two issues around the use of proxy measures. First, when child patients are seen over several years, providers are faced with transitions, first from proxy report to self-report when patients turn 8 years of age, and then another transition from PROMIS Pediatric measures to adult PROMIS measures. One interviewee described how these transitions require decisions about which version to use and also how to interpret the results from differing versions: “The thing I hear the most about is the issue of self-report versus parent-report and how do you know which one to do, what you do when you’re transitioning between a proxy report to a self-report because they are measuring slightly different things, and then what do you do at their other transition from adolescence to young adulthood?” A related concern arises when both the child and their parent complete measures. Although this provides additional information that providers can use, it also requires decision on how to use the information. This is especially salient when the results from child-report and parent-report differ or when comparing scores over time: “[W]e can do a parent and a child score on the same encounter, but […] the problem was, if we were looking back at scores, which score do we use?”

Privacy and confidentiality of PROMIS responses

Interviewees also talked about needs around ensuring privacy and confidentiality for patients completing PROMIS surveys. Developing the right approach to privacy and confidentiality is driven by what patients prefer, legal requirements, and the need to have information readily accessible for clinical use. One interviewee described how not having adequate privacy mechanisms set up and communicated can result in patients not completing measures or providing information: “I think our patients refuse to provide information when they have concerns about how is this—they don’t know how this information is being used, […] who has access to it, […] how it’s being stored—securely or not. And I think those are all very legitimate concerns. And those are all things that can be addressed.” In addition to patient concerns, legal requirements also needed to be taken into account when setting up data collection and storage system, as another respondent described: “So that gets into the privacy issue. […] In discussions with our legal department, what we have decided to do is […] the first time [families] come to clinic, they get an information page that says, ‘These are data within the medical record. Whoever has access to the chart has access to see this.’” Although interviewees raise privacy and confidentiality as a concern, as in this case, they often considered these issues resolvable.

Additional issues related to privacy and confidentiality surrounded understanding who may have completed a PROMIS measure and how PROMIS results were made available to clinicians. Several interviewees recounted experiences when they either had doubts about who completed a measure or had witnessed instances where a parent completed a child’s survey, a parent helped a child, or, conversely, a bilingual child helped their non-English-speaking parent complete a survey. “What we found is that sometimes even when it says, ‘Please hand this to your child,’ if it’s an iPad or tablet or even a paper, the parent is still filling out the forms.” One way that interviewees tried to minimize this issue was by keeping parents busy with other questionnaires while the child completed their own: “So, in the teenage population, I think that the best thing that can happen is if you give the parents a job of filling out a survey and kid a different job on a different tablet or paper and fill out there. So that everybody is occupied.” Interviewees also noted how attempts to ensure privacy of information could negatively affect the availability of PROMIS data for clinicians. One interviewee described this challenge as something they hadn’t been able to resolve yet for a sensitive measure related to mental health: “We have one screener that has a flag [in the EHR], but for some reason it’s not very visible to the clinicians. So that’s actually the next hurdle I need to figure out in clinic. If people are screening positively and asking for help, we need to actually do something.”

Interpretation of PROMIS Pediatric measure scores

Once providers have access to PROMIS scores, they need to interpret them. In our interviews, respondents described this as challenging. Despite clear and consistent guidelines for scoring PROMIS measures, interviewees pointed out that it was less clear to them how to interpret a patient’s score: “We had quite a debate about what threshold of standard deviation score we should use to identify who is higher risk. Should it be 2 standard deviations, or 1 or…” More broadly, interviewees pointed out that a PROMIS score is only a piece of information that clinicians use to inform care, and it needs to be put in the context of other clinical data. This again required guidance on how to interpret PROMIS scores contextually. “When [clinicians] say, ‘I want to use [PROs] to track how my patients are doing […], then it’s a question of ‘okay, well what information would be helpful to have,’ because […] we don’t have all of the interpretation information we would want.”

Using PROMIS scores clinically

Despite the challenges with using PROMIS clinically, interviewees provided many examples of how they overcame these and actually used the measures in practice. Engaging stakeholders, communicating the purpose of PROMIS measures to clinicians and patients, and training providers and other staff on how to administer, score, and interpret measures were some of the elements that interviewees recounted: “There’s a need to identify who to train, to have clear, consistent messaging about why we’re asking patients to do this […] how to introduce it, and then training on where you find scores, how you interpret scores, how you might use them.” Once these things were in place, PROMIS measures provided an efficient and effective way for clinicians to gather relevant information. One interviewee described their use of PROMIS as follows: “It just gives me a nice way to, in just a few seconds, kind of eyeball where parents are at in terms of how many concerns they may have […] and it also gives me direction to follow up on the ones that they’ve endorsed, and I don’t have to waste time asking about the other ones.” Notably, this interviewee did not just use the overall PROMIS score, but also used answers to individual items to shape the clinical encounter. Another interviewee described a more formalized approach, where PROMIS scores were used to make referral decisions: “So for that [depression] measure, with a given score, […] for moderate [scores]: refer to social work. High [scores]: refer to social work and behavioral health.”

Discussion

Our work identified six priority topics that clinicians and health systems may consider as they integrate PROMIS measures into ambulatory pediatric practice. The topics arose within both the early stages of implementation in clinical settings (e.g., which measures to use and how to collect the data) and also during use within the clinical encounter (e.g., how to interpret, discuss, and act upon PROMIS results). Most identified challenges were not specifically related to the PROMIS metrics themselves, but often to health system and organizational factors that influence the implementation and integration of PROMIS within the pediatric clinical work system.

Among our interviewees, decisions about which measures to administer presented challenges early on. Measures selected for administration should reflect the most clinically impactful and actionable health concepts. This prioritization requires input from clinicians, children, parents, and other pertinent stakeholders. Administered measures should also focus on internally-experienced emotions, symptoms, and perceptions (e.g., pain, stress experiences); functioning (e.g., mobility, sleep); or social participation (e.g., peer or family relationships) which are best answered by patient-report. Measures used for screening which typically assesses multiple health concepts (e.g., PROMIS Pediatric Profiles (groups of short forms) and the Global Health measure) are generally briefer, less precise, and rely on fewer items than measures used to monitor change in a patient’s condition over time.[33] Although the commonly administered PROMIS short forms (4–12 items) were designed to provide precise measurement across the full range of severity of the health concept, their precision may decrease at the extreme ends of scale scores. Consequently, knowing the purpose of the measure and specific population is critical. A small number of measure recommendations are available, but more are needed.[34]

Respondents also raised questions about the appropriateness of some PROMIS measures and items for specific populations. All PROMIS items have been evaluated to ensure they are unbiased across key demographic groups (e.g., age, sex, race/ethnicity) and strong evidence for the measurement equivalence of the PROMIS short form measures in ethnically, socio-demographically diverse groups is emerging.[35] PROMIS Pediatric measures have all undergone both cultural harmonization as the items were written, as well as subsequent rigorous translation into dozens of languages.[36] As raised in some of our interviews, translation of PROMIS items into Spanish uses a form of “Universal Spanish” that can result in wording that is unfamiliar or more/less formal. However, the impact of this issue on PROMIS scores is likely clinically insignificant.[37] In some instances, it may be possible to customize the short form to address concerns about language or differing abilities in some clinical populations, through the substitution of one item with another item from the PROMIS item bank. However, this kind of change must be done with caution. Institutions should carefully assess the tradeoffs between modifying an instrument to be more relevant or comprehensible to its patient population, and the psychometric implications that changing a PROMIS short form measure has. In many cases, the benefits of modifying a measure may not warrant the change.

Stakeholders described needing data collection tools that minimize clinician burden, automate scoring, and make results easily accessible at the time of a clinical encounter. However, decisions about how to administer PROMIS measures was often informed by available technology. Fortunately, electronic administration of PROMIS is increasingly integrated into EHR systems and allows collection and scoring directly within the EHR.[15, 16, 33] Although preferences between electronic and paper collection has not been widely studied across populations,[38] in 2018, at least 93% of US adolescents own or have access to a smartphone and 75% to a desktop computer or laptop, suggesting this is an accessible tool for most, but an alternative tool will be needed to include all patients.[39] PROMIS measures can be completed electronically from home by accessing a patient portal over the internet or via a tablet or kiosk at the clinic visit.[5, 15, 16] When data are collected with EHR software, results are electronically loaded for review by the clinician and patient at the visit, which facilitates meaningful use of the data.[5] Electronic administration also allows the use of CAT, which offers better measurement precision while requiring the patient or family to answer fewer items. Using CAT also supports adaptation of the measure’s items to meet the needs of specific populations such as those with differing abilities, like a child who uses a wheelchair, a challenge noted by interviewees.

In clinics and health systems without the capability to directly embed PROMIS measures in the EHR, two options are available—electronic collection and scoring through a data capture system such as REDCap (available in over 4,000 healthcare institutions)[40, 41] or pen-and-paper data collection. Electronic data capture outside the EHR requires significant administrative support to operationalize data collection and to facilitate real-time feedback of results for clinical use during patient encounters, sometimes placing additional burden on the clinician.[15] In addition, pen-and-paper administration was favored for some interviewees because organizational processes to secure approval and resources for electronic administration to support PROMIS use clinically were arduous and slow. Although paper and electronic methods of PROMIS administration produce similar results among adults,[42, 43] the impact of method of collection on rates of PROMIS measure completion and scores among pediatric patients is an active area of research.

Considerations around the use and interpretation of PROMIS Parent Proxy measures were also raised by interviewees. Although self-report of feelings and functioning is ideal, proxy data from parents is valuable when the child is too ill, young, or cognitively impaired to self-report.[44–47] Companion PROMIS Parent Proxy measure were developed in parallel for each PROMIS Pediatric measures and evidence for the proxies’ reliability and validity is available for parents of children between 5 and 17 years of age.[22, 48–54] Yet, clinicians must be cautious about assuming that parent-reported data are equivalent to what a child would self-report. A parent’s report will be impacted by their own health and life experiences as well as their perspectives on the child’s everyday experiences in the settings where they cohabitate.[44, 55] For example, as the parent’s own health worsens, they tend to over-report worse health for their child.[44, 56] Congruence between child self-report and parent-report is typically better for more observable domains such as physical functioning than for less observable domains such as anxiety or fatigue.[55–62] When feasible, maintaining respondent consistency (self-report or parent proxy), especially when anticipating the need to interpret scores over time for children ≥8 years of age, is recommended.

Our interviewees also raised issues and potential solutions related to privacy and confidentiality when using PROMIS in ambulatory pediatric settings. Assuring children that all data will be kept confidential is especially important because of power imbalances between children and adults and can limit the information families are willing to provide. Interviewees suggested this assurance be communicated to families verbally or in written instructions prior to any data completion. However, such assurances should acknowledge required disclosure in instances where a child is at risk for harm to themselves or others.[63] Respondents also noted that sometimes parents answer questions for a child or adolescent, or the parent may be watching as the measure is completed, both of which may affect responses.[64] The strategy of making sure that both parent and child were occupied with filling out surveys may be a viable solution to support privacy as well as the child’s independent completion of the measure.

Although it is critical to confidentiality that PROMIS responses be stored securely, our interviewees described balancing this with meaningful use, which requires the information be readily retrievable by clinicians and also by children and families when appropriate. For example, meaningful use requires avoiding situations where clinicians struggle to access information or only some clinicians may have access to certain sensitive information (e.g., mental health encounters and diagnoses).[65] Health care systems are also working to provide patients access to their own health information, including PROMIS scores and/or responses. Adults in other studies found being able to view PROMIS or other PRO scores useful and desirable.[66, 67] Some health systems are pursuing portal use as encouraged through the Meaningful Use Program.[68] Providing access in an efficient yet confidential manner can be challenging, but various portal designs (e.g., parent-only, a confidential portal for adolescents, or shared access) can facilitate confidentiality for adolescents.[69–74] To best align with guidelines from the Society for Adolescent Health,[75] confidential portals for adolescents should be strongly considered.

Our interviewees also raised a need for guidance around understanding which scores might warrant clinical intervention. PROMIS scores will not replace the human interaction of the patient encounter, but are intended to direct the clinician’s attention to the areas of greatest importance to the patient within a time-constrained encounter.[16, 23, 76–79] One strategy for interpreting a PROMIS score is to consider where it is located in standard deviation units from the mean of the reference population. For example, a sleep disturbance score of 62 is 1.2 standard deviations worse (i.e., 12 T-score points worse than 50) than the population used to construct the measure. A second approach for score interpretation is to identify cut-points and assign labels describing the level of severity, for example, as mild, moderate, or severe.[80] To interpret scores, clinicians may also want to compare a patient’s score not to the general population but to other, similar patients. However, only a few disease-specific reference values are currently available for children.[81] In the absence of disease-specific reference values, a growing body of research provides score averages for specific patient samples,[82–89] which clinicians can use to understand how a patient compares with others with the same condition. In addition, to address disease-specific scoring, investigators have explored cut-points using standard setting, or a method called bookmarking,[90] which is currently limited to two pediatric in juvenile idiopathic arthritis.[91, 92]

Interviewees were also interested in using PROMIS to track their patients’ well-being over time. Unfortunately, there is no simple guideline for identifying the magnitude of change in scores that is important to each patient in each context.[93] Published meaningful change values based on group-level methods or a half-standard deviation can set a lower bound estimate for meaningful change for an individual.[94] Yet, whether that change is important to a given patient is highly individualized[95] and should be evaluated through the triangulation of multiple sources of information (e.g., patient/parent’s perspective). New methods for understanding change in PROMIS Pediatric measure scores, such as the reliable change index, are under development.[91, 93] In addition, visualization of scores could support rapid recognition of change, as has been done with other questionnaires like the Vanderbilt ADHD Diagnostic Rating Scale.[96]

Despite challenges, our interviewees were able to identify key factors that had supported the successful use of PROMIS scores in clinical settings: commitment to discuss the PROMIS scores with the patient at the visit and training in the use of the scores during the clinic visit, including awareness of interventions to address concerning scores. For patients whose scores indicate one or more areas of concern, clinicians can open the encounter by acknowledging the PROMIS scores with prompts such as, “I notice that you are having more difficulty with getting around. Can you tell me what has changed?” For patients whose PROMIS scores do not indicate concern or are improving, clinicians are strongly encouraged to acknowledge the patient’s responses, as this is critical to patient willingness to continue to complete the measures in the future.[5] Developing clinical support tools and providing training can allow clinicians and staff to access appropriate interventions and can foster responsiveness to concerning scores.[76, 77] Ideal clinical support tools are grounded in data- or expert-derived, nationally-accepted guidelines for best practice, with specific local resources identified to operationalize the recommendation. For example, for clinicians who use PROMIS to assess anxiety or depressive symptoms, the clinical support tool could include mental health resources available to the community or specific patient population and the process for referrals. Clinical support tools should be accompanied by a plan to raise awareness of their availability and training on their use. Training to support use of PROMIS in the clinical encounter is available through organizations such as the International Society for Quality of Life Research and the PROMIS Health Organization.[97, 98]

Our work on the use of PROMIS measures in ambulatory pediatric settings may have limitations. Our interviewees, while representing a wide array of expertise and geographic location, were primarily employed in academic medical centers, so generalizing to community-based settings should be done with caution. Given the differences in healthcare delivery systems internationally, we chose to focus on the experiences of US health systems and clinicians. Our work focused on understanding the perspective of health systems and providers. Understanding the challenges and solutions from the perspective of patients and parents would also be valuable. Future work could also assess any gaps in the availability of PROMIS metrics that represent priority metrics for pediatric quality of care.

To foster patient- and family-centered care in ambulatory pediatric clinical practice, assessment of PROs is encouraged and clinicians should be confident in using PROMIS Pediatric instruments in the US pediatric population. The work of NIH’s PROMIS initiative has resulted in a broad set of measures with extensive evidence of their reliability and validity in a range of pediatric populations. The availability of these instruments requires institutions to prospectively decide how best to collect, store, protect, and use the information, given the unique needs of children, adolescents, and families. Fortunately, guidance on how to do this is growing.[99] Our work advances these efforts by enabling health systems to proactively consider the 6 priority topics drawn from the experiences of other US health systems and clinics. Health systems can consider each topic as it relates to their own context and its potential impact on PROMIS implementation. This prioritization will depend on factors such as the available data collection platform, the patient population of interest, and the goals of implementing PROMIS. As noted from our interviews, “Clinicians want to do the right thing, but make it [PROMIS] a part of what they do…” Ongoing efforts to disseminate experiences and best practices for the implementation and integration of PROMIS measures in clinical care is critical to having this initiative improve the health of children.

Abbreviations

CAT

computer adaptive test

HER

electronic health record

NIH

National Institutes of Health

PRO

patient-reported outcome

PROMIS

Patient-Reported Outcomes Measurement Information System