Assessment of pediatric asthma exacerbation with the use of new PROMIS measures

Abstract

Objective

Patient reported outcome measures, such as the Patient Reported Outcomes Measurement Information System (PROMIS) may be used to assess patient functioning for asthma and aid in understanding the impact of asthma exacerbation. These domains may be utilized as endpoints in clinical trials and to guide clinical care. The purpose of this study was to determine psychometric properties of the new PROMIS measures for children with asthma, at baseline and with exacerbation.

Methods

We conducted a cross-sectional analysis of children with acute asthma exacerbation or at baseline health. Psychometric properties of validity (using known groups and correlation) and reliability (using Cronbach’s alpha and IRT) for the new PROMIS measures were determined.

Results

Our study included 220 subjects, 102 were enrolled during an acute exacerbated state. Cronbach’s alpha and IRT reliability was greater or equal to 0.75. Our subjects experiencing an acute exacerbated state reported worse T-scores for pain related domains: pain behavior 45.7 vs 53.5 (p <0.001), pain quality sensory 44.4 vs 48.5 (p <0.005), pain quality affective 42.5 vs 51.3 (p <.001), and physical stress experience 60.5 vs 65.4 (p <0.001); and asthma impact 47.9 vs 61.0 (p <0.001), than subjects at baseline. Child and parent-proxy agreement ranged from 35% to 56%.

Conclusions

The new Pediatric PROMIS domains are valid and reliable for use in children with asthma, for both child-reported and parent-proxy reported outcomes. It was determined that children with acute asthma exacerbation have worse patient reported outcomes (PROs) for the new pain related domains and asthma impact.

Introduction

Asthma is a worldwide problem with a wide range of effects on patients’ health and well-being (1). To assess the impact of this disease on patients’ functioning, it is imperative to have valid and reliable tools. The Patient Reported Outcomes Measurement System (PROMIS), funded by the National Institutes of Health, is one of the tools that can be used to assess child functioning in asthma (2). Prior studies have supported the use of PROMIS domains, such as asthma impact and global health, in children with asthma (3–5). Additional research has identified impaired functioning in the targeted domains of fatigure, pain, and physical functioning (mobility) in children with asthma (6). Since these studies have been published, new PROMIS domains have been developed to assess a child’s physical functioning including physical stress, but their measurement properties are unknown for children with asthma. These new domains may be useful in asthma to better assess and understand the symptoms that asthma have on the functioning of children. In addition, it is unknown how well both the new and existing PROMIS measures perform psychometrically for children experiencing an asthma exacerbation. Once these new PROMIS domains are found to be valid and reliable for children with asthma, including in an exacerbative state, they may be used to measure patient response to therapy in clinical trials, to manage and improve the health of patient populations, and to provide significant and complementary information to providers about patients who attend the clinic.

Our primary objective of this work was to determine the psychometric properties of the new PROMIS measures for children with asthma, and the use of PROMIS measures for an acute asthma exacerbation. We hypothesized that PROMIS domains are valid and reliable for asthma exacerbation states and that the new pediatric PROMIS domains of physical activity, physical stress experiences, and strength impact would be valid and reliable for children with asthma.

Our secondary objective was to determine the relationship between parent-proxy and child self-report PROMIS measures. We hypothesized that parent-proxy and child self-report PROs would be highly correlated for observable domains and moderately correlated for non-observable domains.

Methods

Study Design and Study Populations:

For this cross-sectional study, we recruited a convenience sample of children with asthma ages 5 to 17 years old from a tertiary asthma clinic and from the emergency department (ED) at Children’s Wisconsin during May 2016 to June 2017. To be eligible for participation, all children had to be able to speak and understand English, provide assent/consent, have a healthcare provider diagnosis of asthma, and have a history of using asthma reliever medications in the prior year. Patients recruited from the ED were eligible if they were experiencing an acute asthma exacerbation. Asthma exacerbation was defined as a child presenting to the ED who had wheezing or respiratory distress, had a health care provider diagnosis of asthma, and had used a reliever medication in the prior year. Children ages 8 to 17 years of age were eligible to self-report the PROs. Parents of children 5 to 18 years of age provided parent-proxy information of the child’s PROs (7) . Subject demographic information was provided by parent-proxy, including: age, gender, ethnicity, and race. Subject asthma severity, was determined using the Asthma Control Test (ACT) Score (8) and National Heart, Lung, and Blood Institute (NHLBI) Asthma Type (9), which were abstracted from the patients’ electronic health records, if available.

Study Procedure:

PROMIS surveys were obtained for children with asthma in their baseline state of health (recruited in the clinic) and for those experiencing acute exacerbation (recruited in the ED). Baseline state of health was defined as a subject who attended a standard asthma follow-up appointment (non-sick visit) in clinic with their asthma healthcare provider. Study participants completed the PROMIS questionnaires either electronically via REDCap (Research Electronic Data Capture) or using paper forms (10). Parents also provided their child’s demographic information, including age, sex, race, ethnicity, school grade level, insurance, and information about their child’s medical history and disease management strategies.

Pediatric PROMIS measures:

The new pediatric PROMIS domains (8 item short forms) included in this study were pain behavior, pain quality sensory, pain quality affective, physical stress experience, physical activity, and strength impact. They were developed to understand distinct constructs related to pain and physical functioning, in the past 7 days.

The pain behavior domain was to assess behaviors such as asking for medicine for pain and moving slower, that communicates a child’s pain experience and expression. The pain quality sensory domain was to assess specific subjective pain sensations, such as burning and throbbing. The pain quality affective domain was to measure pain descriptions, such as unbearable and worrying. The physical stress domain was to measure sensations, such as trouble breathing, muscles feeling tight, etc. The physical activity domain was to assess frequency, duration, and intensity of physical activities, such as exercise or playing hard. The strength impact domain was to assess a child’s capacity to perform functional activities of daily living that require physical strength, such as reaching above their head to get heavy things or jumping up and down. All domains, excluding the pain quality affective domain, utilized a Likert scale for response. The pain quality affective domain utilized a “Yes” or “No” response.

In addition, to determine convergent validity, subjects completed the previously validated PROMIS Pediatric-25 profile (11). The PROMIS Pediatric-25 profile included the domains of pain interference, pain intensity, physical function mobility, anxiety, depressive symptoms, fatigue, and peer relationships. They also completed the PROMIS Asthma Impact Short Form questionnaire. It was designed to measure daily functioning specific to asthma symptoms over the past week. Some of the statements included were: “I felt scared that I might have trouble breathing because of my asthma” and “It was hard for me to play sports or exercise because of my asthma.”

Parent proxy and child self-report are available for all PROMIS domains administered in this study, except for the pain quality domain where only a child self-report is available.

PROMIS measures are scored on a T-score metric with a mean of 50 and standard deviation (SD) of 10. Fifty represents the mean of the child sample in which the item response theory (IRT) parameters for the measures were estimated (12). A higher score on PROMIS domains indicates more of the concept, for example, a higher T-score on PROMIS fatigue domain means more fatigue. All PROMIS domains are publicly available and can be accessed online. (http://healthmeasures.net/search-view-measures).

Statistical analysis:

Descriptive statistics were used to report the demographic and baseline clinical characteristics of the study cohort. Analyses done to obtain specific results are described as follows:

Construct and convergent validity:

We utilized a known groups method to determine construct validity. Specifically, we compared mean PROMIS T-scores between children experiencing an acute asthma exacerbation in the ED to children with asthma who were in their baseline health state and receiving routine care in the pediatric tertiary care clinic. T-scores (child self-report as well as parent proxy) were compared across ED vs clinic groups using a t-test. False discovery rate was used to adjust for multiple comparisons.

Convergent validity was assessed by determining the correlations of the new PROMIS measures with the previously validated PROMIS Asthma Impact questionnaire and the PROMIS Pediatric-25 profile. Pearson correlation coefficients were used. The correlation coefficients were designated as small (0.10), medium (0.30), and large (0.50) (13).

Floor/ceiling effects:

The proportion of scores at the floor and ceiling were determined for each PROMIS domain. Floor effects correspond to the proportion of minimum scores and ceiling effects correspond to the proportion of maximum scores. Domains with more than 15% of participants demonstrating floor or ceiling effects were considered to be less precise at extremes of the scales (14, 15).

Reliability:

Internal consistency reliability was determined using Cronbach’s alpha with 95% confidence intervals and using IRT reliability. The IRT reliability was calculated as the ratio of true variance between scores and the variance between measured scores. Reliability ≥ 0.70 was considered acceptable (16, 17).

Agreement between child self-report and parent-proxy report:

Agreement was calculated as the absolute value of differences in T-scores, as done in previous studies (18, 19). If the absolute value was less than or equal to half of the standard deviation of the child’s scores, then the child self-report and parent-proxy scores were considered to be in agreement. However, if the absolute value was greater than half of the standard deviation of the child’s scores, then the direction of mismatch between the child self-report and the parent-proxy T-scores was described.

Intraclass correlation coefficients (ICC) with 95% confidence intervals were calculated to determine the agreement between child self-report and parent-proxy scores for all PROMIS domains, excluding pain quality sensory and affective (parent-proxy’s did not complete these domains).The ICC values at or below 0.4 were determined to be a poor correlation, 0.4-0.6 were moderate, and 0.6 and above were strong correlations (20, 21).

Results

Our study included 220 participants for which PROMIS data were collected (Figure 1). Of these, 162 children were 8 years or older (91 from ED and 71 from clinic) and completed the child self-report PROMIS questionnaires. Table 1 shows the demographic and baseline clinical characteristics of our study population.

Figure 1.

Study population with self-report and parent-proxy PROMIS scores

^a2 subjects enrolled and then declined to fill out questionnaire

^b1 subject completed 56% of questionnaire

^c1 subject data missing asthma impact questionnaire response, all other data was obtained

Table 1.

Demographics and baseline clinical characteristics of children with asthma in the study

	N (%)
Patient characteristics	Total (N = 220)	Clinic (N = 102)	ED (N = 118)
Age at enrollment
<8 years  8 – 12 years  13 – 18 years	58 (26.4) 111 (50.4) 51 (23.2)	31 (30.4) 43 (42.2) 28 (27.4)	27 (22.9) 68 (57.6) 23 (19.5)
Age (years) at questionnaire completion, child self-report, mean (SD) (N)	11.6 (2.6) (162)	12.2 (2.7) (71)	11.1 (2.5) (91)
Gender, femalea	89 (40.4)	37 (36.2)	52 (44.1)
Ethnicity, Hispanic or Latinob	33 (15.0)	17 (16.7)	16 (13.6)
Racec
Black	120 (54.6)	49 (48.0)	71 (60.2)
White	57 (25.9)	32 (31.4)	25 (21.2)
Others/>1 Race	36 (16.4)	16 (15.7)	20 (17.0)
ACT Score, ≤19d	88 (40.0)	40 (39.2)	48 (40.7)
NHLBI Asthma Typee
Intermittent	31 (14.1)	9 (8.8)	22 (18.6)
Persistent Mild	46 (20.9)	30 (29.4)	16 (13.6)
Persistent Moderate	84 (38.2)	41 (40.2)	43 (36.4)
Persistent Severe	34 (15.4)	20 (19.6)	14 (11.8)

^aGender not reported for 1 clinic subject

^bEthnicity not reported for 27 subjects

^cRace not reported for 7 subjects

^dACT score was missing for 52 subjects

^eNHLBI was missing for 25 subjects

Construct and convergent validity:

Child self-report: For the new PROMIS domains, there was a statistically significant difference in pain behavior, pain quality sensory, pain quality affective, physical stress experience, and strength impact between patients with an acute asthma exacerbation and those who were in their baseline health state (Table 2). However, there was no significant difference in physical activity. There was a significant difference in pain interference, pain intensity, physical function mobility, fatigue, and asthma impact between patients with an acute asthma exacerbation and those who were in their baseline health state (Table 2). There were no significant differences found in the remaining PROMIS domains (anxiety, depressive symptoms, or peer relationships).

Table 2.

Known-groups T-score comparison for child self-report PROMIS measures by report acute asthma exacerbation vs. routine clinic care

	Mean (SD)
	Clinic (N = 71)	ED (N = 91)	P-value
New PROMIS Domains
Pain behavior	45.7 (10.5)	53.5 (8.8)	<0.001a
Pain quality sensory	44.4 (7.9)	48.5 (8.7)	0.005a
Pain quality affective	42.5 (7.0)	51.3 (7.9)	<0.001a
Physical stress experience	60.5 (8.5)	65.4 (8.2)	<0.001a
Physical activity	50.5 (6.4)	49.7 (7.7)	0.480
Strength impact	42.7 (9.0)	37.9 (6.7)	<0.001a
Pediatric-25 Profile
Pain interference	50.8 (10.6)	57.7 (10.8)	<0.001a
Pain intensityb	3.1 (2.3)	5.9 (2.8)	<0.001a
Physical function mobility	48.9 (8.8)	44.2 (9.0)	0.002a
Anxiety	48.8 (11.1)	51.8 (12.0)	0.150
Depressive symptoms	49.0 (10.1)	47.7 (9.4)	0.470
Fatigue	48.8 (10.1)	53.7 (11.6)	0.009a
Peer relationships	49.7 (9.5)	49.0 (9.6)	0.670
Asthma impact	47.900 (9.500)	61.000 (8.600)	<0.001a

^aSignificant at p < 0.05, T-test

^bData reported is not a T-score, question response reflects pain scale score 0-10 (no pain-worst pain)

Table 3 shows the correlations between the child self-report new PROMIS domains and the PROMIS Pediatric-25 domains. The pain behavior and pain quality affective domains were moderately to highly correlated with all Pediatric-25 domains, aside from depressive symptoms and peer relationships. The pain quality sensory domain was moderately to highly correlated to all Pediatric-25 domains, aside from peer relationships. The physical stress experience domain had the same correlations as seen with pain behavior and pain quality affective. The physical activity was not correlated to any of the Pediatric-25 domains. The strength impact domain was moderately correlated to the physical function mobility, fatigue, and pain interference domains. All new PROMIS domains, except physical activity and strength impact, were correlated with the Asthma Impact Short Form.

Table 3.

Correlation between child self-report of new PROMIS domains and Pediatric Profile-25 domains

New PROMIS Domain	Pediatric-25 Profile (Pearson’s Correlation Coefficient)
	Pain interference	Pain intensity	Physical function mobility	Anxiety	Depressive symptoms	Fatigue	Peer relationships	Asthma impact
Pain behavior	0.66	0.52	−0.49	0.39	0.23	0.49	−0.05	0.50
Pain quality sensory	0.58	0.43	−0.38	0.51	0.36	0.47	−0.08	0.44
Pain quality affective	0.57	0.55	−0.41	0.37	0.29	0.49	−0.10	0.58
Physical stress experience	0.53	0.36	−0.46	0.44	0.30	0.48	0.04	0.53
Physical activity	−0.11	−0.07	−0.24	0.02	−0.05	−0.23	0.24	−0.12
Strength impact	−0.33	−0.24	0.37	−0.19	−0.09	−0.41	0.14	−0.26

Parent Proxy Construct and Convergent Validity: All domains displayed significant differences between parent proxy-report T-scores for patients in their baseline health in the clinic and those in the ED (Supplemental table 1).

Supplemental table 2 displays the correlation between parent proxy-report PROMIS domains and the Pediatric-25 domains. The pain behavior domain and physical stress experience domains had a moderate to high correlation with all Pediatric-25 domains, aside from peer relationships. The physical activity domain was not correlated with any Pediatric-25 domains. The strength impact domain was moderately correlated with the pain inference, pain intensity, physical function mobility, and fatigue domains. All the new PROMIS domains, except for physical activity, were correlated to the Asthma Impact Short Form.

Floor and ceiling effects:

Child self-report: For the new PROMIS domains, only the pain quality affective domain displayed a floor effect, and the strength impact domain displayed a ceiling effect (Supplemental Table 3).

Parent proxy: For the new PROMIS domains, the pain behavior domain displayed a floor effect, and the strength impact domain displayed a ceiling effect (Supplemental Table 4).

Reliability:

Child self-report: All of the new child self-report PROMIS measures have high reliability (≥ 0.70) as determined via Cronbach’s alpha and marginal IRT reliability statistics (Table 4).

Table 4.

Intraclass correlation (ICC) between parent and child PROMIS scores

	ICC (95% CI)
New PROMIS measures
Pain behavior	0.43 (0.28 – 0.56)
Physical stress experience	0.35 (0.20 – 0.48)
Physical activity	0.20 (0.04 – 0.34)
Strength impact	0.38 (0.22 – 0.51)
Pediatric-25 Profile
Pain interference	0.40 (0.26 – 0.52)
Pain intensity	0.61 (0.47 – 0.72)
Physical function mobility	0.39 (0.25 – 0.52)
Anxiety	0.48 (0.35 – 0.59)
Depressive symptoms	0.40 (0.25 – 0.52)
Fatigue	0.51 (0.38 – 0.62)
Peer relationships	0.40 (0.26 – 0.53)
Asthma impact	0.63 (0.50 – 0.72)

Parent proxy: All of the new parent-proxy PROMIs measures have high reliability (≥ 0.70) as determined via Cronbach’s alpha and marginal IRT reliability statistics (Supplemental Table 4).

Agreement between child self-report and parent-proxy PROMIS response: Child and parent-proxy PROMIS domain agreement is shown in Figure 2. Agreement ranges from approximately 35% to 56%. The highest agreement was seen with the pain intensity and asthma impact domains. The lowest agreement was seen with the pain quality affective, and physical stress experience domains.

Figure 2.

Agreement between child and parent-proxy PROMIS scores. A: Domains for which higher scores indicate more impairment. B: Domains for which higher scores indicate better functioning. (DS: Depressive symptoms; FTG: Fatigue; PINT: Pain interference; Intensity: Pain intensity; PSE: Physical stress experience; PB: Pain behavior; AI: Asthma impact; FR: Family relationships; PFM: Physical function mobility; PA: Pain quality affective; PS: Pain quality sensory)

The intraclass correlations (ICC) report low-moderate agreement between parent-proxy and child self-report scores for all new domains (Table 4). This excludes physical activity, which displayed an ICC of 0.20. High agreement was also reported with pain intensity with an ICC of 0.61. Lower ICCs were found for the new domains: physical stress experience and strength impact, with an ICC of 0.35 and 0.38, respectively.

Discussion

Our study findings demonstrate the reliability and validity of the new pediatric PROMIS measures of pain behavior, pain quality (sensory and affective), physical stress experience, and strength impact for children with asthma, including during acute exacerbation. However, contrary to our hypothesis, the physical activity domain was not found to be valid in distinguishing between children with an asthma exacerbation compared to children who were in their baseline state of health. All of the new PROMIS domains had acceptable internal consistency and were found to be precise. In addition, the low-moderate agreement between parent-proxy and child self-report measures for multiple domains supports the need to use a child-self report where able and to use parent-proxy measures as complementary to the child self-report measures. The strength of these parent-proxy agreements may be improved with a larger cohort study.

PROMIS measures have not been previously studied in children experiencing acute asthma exacerbation. Our study found that multiple domains of PROMIS are valid and reliable for use in children with an acute asthma exacerbation, including pain behavior, pain quality (sensory and affective), physical stress experience, and strength impact. These new pediatric PROMIS domains are valuable tools to aid in determining PROs in children with asthma.

We identified moderate to high correlations between the new domains and the PROMIS Pediatric-25 profile. This supports the validity of the new domains, with the exception of physical activity, for children with asthma. In addition, all the new domains, with the exception of physical activity, differentiated children with an acute asthma exacerbation from those at their baseline. It is unclear as to why no differences were identified in the two patient populations’ physical activity. The questions asked in this domain focus primarily on the patient’s experience with intense aerobic exercise and may represent similar experiences in our two study groups.

There has not been much research done into pain experiences in children with asthma (6, 11). No studies have been identified that specifically look at pain experience during times of acute asthma exacerbation. Our participants experiencing acute asthma exacerbations reported a significantly worse pain experience than participants enrolled in the clinic across all pain domains. Howell et al. reported differences in PROMIS pain interference in children with poor asthma control compared to good control with baseline T-scores of 48.9 and 47.7, respectively. Both of these groups had less pain interference than either of our groups (acute exacerbation and clinic care) did, where there were scores of 57.7 and 50.8, respectively, with higher scores reporting more impairment. De Walt et al. reported a significant difference in the baseline poor and good asthma groups’ pain interference with scores of 55.0 and 47.0, respectively. Our results support Howell and De Walt’s groups that pain is experienced by children with asthma, specifically those experiencing an acute asthma exacerbation.

A substantial portion of our subjects, from both the ED and the clinic, reported no manifestations of any affective pain quality, as found by the high floor effects. This implies that while the pain quality affective domain may be able to determine differences at a high level of impairment, it may not be as well suited to determine differences at lower levels of affective pain. Alternatively, it may represent that these patients do not experience pain that is of the pain quality affective type.

It has been established that, in the case of childhood chronic disease, variances in parent-proxy report and child self-report of PROs are to be expected (6, 22-24). Our study identified a low-moderate agreement between parent-proxy and child self-report scores. Our highest agreement was with the asthma impact domain. This domain had a moderate agreement at 63%, which was comparable to the agreement of 53% previously reported, with a different correlation method (23). Contrasting from our hypothesis, our study found a moderate correlation for observable domains and a higher correlation for non-observable domains between child self-report and parent-proxy PROs. For example, the observable domain of physical activity only had a 20% agreement, but our non-observable domain of pain intensity had a 61% agreement. In general, our non-observable domains reported similar agreement between parent-proxy and child self-report to previously reported findings. This is with the exception of the physical function mobility domain of the Pediatric-25 Profile. Our study reported a 39% agreement, which was much lower than the previously reported 63% agreement (23). This is potentially because Varni et al. utilized a larger sample size and more measurements within the domain.

Our study had a few limitations, including the use of a convenience sample of children with asthma. In addition, our patient population was recruited within a tertiary care facility and likely represent more severe patients than those who receive their care at primary care facilities. ACT scores were not obtained in the ED and these scores were unavailable for such subjects recruited from the ED who did not receive routine care at Children’s Wisconsin. Our study population was majority African American with a majority of our overall population reporting moderate to severe asthma. Lastly, our study was conducted at a single site in Milwaukee, Wisconsin and included only English-speaking families. This may limit the generalizability of our study.

Conclusion

The new pediatric PROMIS domains of pain behavior, pain quality sensory, pain quality affective, physical stress experience, and strength impact are valid and reliable for use in children with asthma, including those experiencing acute asthma exacerbation. In addition, children experience pain at times of exacerbation. Further, parent-proxy report responses are valid and reliable to use in children with asthma. Our study supports the use of these new domains to help assess the health and functioning of children with asthma, including during an exacerbation event.