PROMIS Computer Adaptive Tests and Their Correlation With Disease Activity in Juvenile Idiopathic Arthritis

Abstract

Background/Objective:

The importance of patient-reported outcomes, like the Patient-Reported Outcomes Measurement Information System (PROMIS) measures, is increasingly recognized both in clinical care and in research. While “short forms” have been studied in juvenile idiopathic arthritis (JIA), study of PROMIS computer adaptive tests (CATs) in JIA is limited. This cross-sectional study evaluates whether PROMIS CATs correlate with disease activity in patients with JIA.

Methods:

A convenience sample of patients with JIA (N = 44) was recruited from a single center. Patients and parents completed pediatric and parent proxy PROMIS CATs. Disease activity was evaluated using the Juvenile Arthritis Disease Activity Score in 71 joints (JADAS-71) and the Childhood Health Assessment Questionnaire (CHAQ). Correlation of the CAT T scores with disease activity was assessed using Spearman correlation coefficients.

Results:

Forty-four of 80 eligible subjects (29 patients and 15 parents) completed all or some PROMIS CATs. Pain interference and mobility CATs correlated moderately with JADAS-71. Nearly all correlations with the JADAS-71 were weakened when the patient global was removed. Pain interference, mobility, and fatigue were strongly correlated with the CHAQ. Among parent proxy CATs, only mobility and depressive symptoms correlated strongly with the CHAQ.

Conclusions:

Only pain interference and mobility PROMIS CATs showed strong correlation with standard disease activity measures in JIA, and nearly all correlations were weakened when the patient global was removed. Correlations of the CATs with the CHAQ were stronger than correlations with the JADAS-71, indicating that although the CHAQ is no longer routinely used it may be a better measure of health-related quality of life in routine clinical care.

Juvenile idiopathic arthritis (JIA) is a chronic autoimmune disease with multiple subtypes, affecting approximately 300,000 children in the United States.¹ Diagnosis and therapeutic management depend on comprehensive history, physical examination, and relevant laboratory and radiological data. In addition, monitoring of disease activity is crucial, but this is especially challenging in children who may be unable to quantify or describe symptoms as easily as adults. To this end, instruments have been developed to promote standardization of assessment of disease activity in JIA. The criterion standard and most widely used instrument is the Juvenile Arthritis Disease Activity Score in 71 joints (JADAS-71).² The JADAS-71 is a composite disease-specific outcome measure that includes the physician global health assessment, the patient global health assessment, the swollen joint count, and a modified erythrocyte sedimentation rate. However, despite including both objective and subjective components, and being developed specifically for JIA, data suggest that the JADAS-71 is not a reliable measure of the global experience of children with this disease.³

Patient-reported outcomes (PROs) have been increasingly recognized as important in the assessment of health status in both clinical care and research.⁴ However, few PROs are currently utilized in the evaluation of patients with JIA. The Childhood Health Assessment Questionnaire (CHAQ) is a generic PRO that has been validated in JIA.⁵ However, it is mainly focused on physical function and is limited by a significant floor effect. The Patient-Reported Outcomes Measurement Information System (PROMIS) is a National Institutes of Health–directed collection of PRO measures that cover a wide range of domains and like the CHAQ are not disease specific.⁶ The most precise versions are the computer adaptive tests (CATs). These instruments utilize item response theory, in which the patient’s answer to one question influences which question is administered next. PROMIS CATs therefore provide increased precision with efficient administration.

To date, the evaluation of PROMIS CATs in JIA has been limited.^7,8 There have been some studies of the psychometric properties of PROMIS CATs in JIA, and the correlation of CATs with inactive disease has been assessed.⁹ However, the performance of PROMIS CATs in patients with JIA with varying degrees of activity remains unknown.

The objective of this study was to test the hypotheses that (1) relevant PROMIS CATs correlate with the JADAS-71, the criterion standard for assessing disease activity; (2) relevant PROMIS CATs also correlate with the generic CHAQ, the only validated PRO in JIA; and (3) PROMIS CATs have good test/retest reliability in children with JIA.

METHODS

Between October 13, 2016, and April 30, 2018, a consecutive sample of children and adolescents was recruited from the pediatric rheumatology clinic at a single academic center during a standard-of-care visit. Inclusion criteria included (1) patients aged 1 to 18 years, (2) English speaking, and (3) any JIA subtype, according to the International League of Associations for Rheumatology criteria.¹⁰ Patients who had an additional rheumatic disease diagnosis were excluded. This study was approved by the Hospital for Special Surgery Institutional Review Board.

Data collected at the study visit included the physical examination and 71-joint count, the physician global health assessment, the patient global health assessment, and erythrocyte sedimentation rate, all of which are components of the JADAS-71. The JADAS-71 is scored from 0 to 101, with higher scores indicating more active disease. Study participants also completed the CHAQ, which has a score of 0 to 3, with higher scores indicating more difficulty with physical function. In addition, participants also completed all available pediatric PROMIS CATs, including fatigue v1.0, pain interference v1.0, peer relations v1.0, anxiety v1.1, depressive symptoms v1.1, and mobility v1.0. PROMIS CATs results are reported as T scores, ranging from 1 to 100, with 50 being the US population mean and a SD of 10. Higher scores represent more of the domain being measured; for example, a higher anxiety score represents more anxiety.

The CHAQ and patient global health assessment were completed by children 10 years or older and by a parent for children aged 1 to 9 years; this was specified in order to standardize data collection based on expectations of children at those ages. Similarly, pediatric PROMIS CATs were completed by patients 10 years or older, and the parent proxy PROMIS CATs were completed by parents when patients were aged 1 to 9 years. Aside from the patient global health assessment, the JADAS-71 was completed by the physician. All surveys were administered and managed using REDCap electronic data capture tools hosted at Weill Cornell Medicine Clinical and Translational Science Center.¹¹

To demonstrate test-retest reliability, PROMIS CATs were administered a second time 7 days later, the standard time frame for test-retest reliability assessment. At the second administration, the anchor question “How do you feel now, compared with how you felt when you last completed these surveys?” was asked, with a 5-item Likert scale response. Patients who answered “about the same” should have had similar PROMIS CATs scores on both occasions and were therefore included in the test-retest reliability analysis.

PROMIS raw scores were converted to T scores using standard software. A change of 0.5 (at least ½ SD) is considered clinically meaningful.¹² Correlations between the JADAS-71, CHAQ, and PROMIS CATs were assessed using Spearman correlation coefficients. We report the correlation coefficient, or r value, which represents the magnitude of correlation, as well as the p value, which represents statistical significance. As a sensitivity analysis, correlations with the JADAS-71 were performed with and without the patient global health question. This allowed comparison of the JADAS-71 performance when it comprised physician-only responses versus when a PRO was included. Test/retest reliability was assessed using intraclass correlations.

RESULTS

PROMIS CATs were completed by 44 of 80 families approached, 29 children with JIA and 15 parents of younger children with JIA (Table 1). Approximately 80% completed these questionnaires remotely on the same day as their visit, rather than on-site during their visit. Nine subjects completed the retest within 1 week. The children were predominantly girls (73%) with a median (interquartile range [IQR]) age of 12.6 (7.3, 15.3) years. Eighty-four percent were white, 23% of subjects had polyarticular JIA, 23% had psoriatic arthritis, and 45% had spondyloarthropathy (see Table 1 for more details on subtypes). The median (IQR) JADAS-71 score was 5 (1.8, 10.4), indicating low disease activity, although a few patients had very active disease.

TABLE 1.

Patient Characteristics

Characteristic	All (N = 44)	Children ≥10 (n = 29)	Parents (n = 15)	Retest (n = 9)
Age, median (IQR), y	12.6 (7.3, 15.3)	14.9 (12.6, 16.0)	4.9 (3.4, 8.7)	14.1 (10.0, 17.4)
Sex
Female	32 (73%)	11 (55%)	8 (88.9%)	3 (75%)
Race
White	37 (84%)	23 (79%)	14 (93%)	5 (56%)
Black or African American	2 (5%)	1 (4%)	1 (7%)	2 (22%)
Asian	3 (6%)	3 (10%)	0 (0%)	2 (22%)
Other	2 (5%)	2 (7%)	0 (0%)	0 (0%)
Ethnicity
Non-Hispanic	39 (89%)	26 (90%)	13 (87%)	9 (100%)
Insurance
Medicaid	4 (9%)	2 (7%)	2 (13%)	1 (11%)
Private insurance	40 (91%)	27 (93%)	13 (87%)	8 (89%)
Device used to complete CATs
Smartphone	34 (77%)	24 (83%)	10 (67%)	8 (89%)
iPad	3 (7%)	1 (3%)	2 (13%)	0 (0%)
Computer	7 (16%)	4 (14%)	3 (20%)	1 (11%)
Location
In hospital	9 (20%)	8 (28%)	1 (7%)	2 (22%)
Remotely	35 (80%)	21 (72%)	14 (93%)	7 (78%)
JADAS-71 score, median (IQR)	5 (1.8, 10.4)	6 (3.2, 8)	4.8 (1, 19)	3 (1, 6.2)
JIA subtype
Polyarticular	10 (23%)	6 (21%)	4 (27%)	4 (45%)
Psoriatic	10 (23%)	2 (7%)	8 (53%)	1 (11%)
Spondyloarthropathy	20 (45%)	20 (69%)	0 (0%)	3 (33%)
Oligoarticular	4 (9%)	1 (3%)	3 (20%)	1 (11%)

Mean (SD) patient and parent proxy PROMIS CAT T scores are shown in Figures 1 and 2. PROMIS CATs were generally normally distributed. The mean CAT T scores, both for patients and for parent proxies, were generally slightly better than the population mean across all PROMIS domains. Similarly, JADAS-71 and CHAQ scores were relatively normally distributed. The CHAQ had large floor effects, with 16 (36%) of 44 subjects scoring 0, the lowest possible score indicating no problems with physical function. PROMIS CATs had less significant floor or ceiling effects, with no subject scoring the lowest or highest possible score in any domain.

FIGURE 1.

Mean T scores, pediatric PROMIS CATs.

FIGURE 2.

Mean T scores, parent proxy PROMIS CATs.

Correlations between pediatric PROMIS CATs and legacy instruments for children 10 years or older are shown in Table 2. The Childhood Health Assessment Questionnaire showed strong correlations with PROMIS CATs for fatigue, pain interference, and mobility (r = 0.65, 0.78, −0.84 respectively; p < 0.05 for all). The Childhood Health Assessment Questionnaire showed moderate correlations with anxiety and depression (r = 0.41, p < 0.05; and r = 0.37, p > 0.05, respectively) and poor correlations with peer relations (r = −0.18, p > 0.05). JADAS-71 showed weaker correlations with all PROMIS CATs compared with the CHAQ, the strongest correlations being with PROMIS pain interference and mobility CATs (r = 0.55 and −0.52, p < 0.05). When the patient global health assessment question was excluded from the JADAS-71, these correlations were weakened (r = −0.41, p > 0.05; and −0.45, p < 0.05).

TABLE 2.

Spearman Correlation Coefficients, Pediatric PROMIS CATs

PROMIS Domain	Sample Size	JADAS Score, r	JADAS Without PG, r	CHAQ Score, r
Patient scores
Fatigue T score	29	0.29	0.17	0.65*
Pain interference T score	29	0.55*	0.41	0.78*
Peer relations T score	28	−0.17	−0.19	−0.18
Anxiety T score	27	0.13	0.09	0.41*
Depressive symptoms T score	26	0.26	0.22	0.37
Mobility T score	26	−0.52*	−0.45*	−0.84*

^*p < 0.05.

PG indicates patient global health assessment.

Correlations between parent proxy PROMIS CATs and parental responses for children younger than 10 years on legacy instruments are shown in Table 3. PROMIS mobility and depressive symptoms CATs showed strong, statistically significant correlations with the CHAQ (r = −0.8 and 0.62, p < 0.05), moderate correlations for pain interference (r = 0.41, p > 0.05), and poor correlations with fatigue, anxiety, and peer relations (r = 0.17, 0.25, and −0.20; p > 0.05). Correlations with the JADAS-71 were moderate for mobility, with and without the proxy patient global (r = −0.42 and −0.41, p > 0.05), but poor for all other domains, both with and without the proxy patient global (r = 0.003 and −0.34, p > 0.05 for all).

TABLE 3.

Spearman Correlation Coefficients, Parent Proxy PROMIS CATs

PROMIS Domain	Sample Size	JADAS Score, r	JADAS Without PG, r	CHAQ Score, r
Parent proxy scores
Fatigue T score	15	0.30	0.14	0.17
Pain interference T score	14	0.34	0.28	0.41
Peer relations T score	13	−0.08	0.08	−0.20
Anxiety T score	13	0.31	0.34	0.25
Depressive symptoms T score	13	0.003	0.01	0.62*
Mobility T score	13	−0.42	−0.41	−0.80*

^*p < 0.05.

PG indicates patient global health assessment.

Nine of 44 subjects reported feeling “about the same” 7 days after initial PROMIS CATs completion, and these 9 study subjects completed all PROMIS CATs a second time to assess test-retest reliability. Intraclass correlations were generally moderate (Table 4); however, given the small number of respondents, none of these were statistically significant.

TABLE 4.

Intraclass Correlations

PROMIS Domain	ICC (95% CI)
Patient scores
Fatigue T score	0.51 (0.12–0.88)
Pain interference T score	0.66 (0.25–0.92)
Peer relations T score	0.35 (0.05–0.85)
Anxiety T score	Not estimable
Depressive symptoms T score	Not estimable
Mobility T score	0.35 (0.03–0.89)

CI indicates confidence interval; ICC, intraclass correlation coefficient.

DISCUSSION

While prior studies have investigated the construct validity of PROMIS CATs in children with JIA, ours is the first to assess the correlation of PROMIS CATs with both the most widely used JIA-specific measure of disease activity, the JADAS-71, and the most commonly used generic PRO measure, the CHAQ. Morgan et al.⁹ evaluated the psychometric properties of the PROMIS CATs in children with JIA, focusing mainly on minimally important differences in PROMIS CATs for use in prospective evaluation and management. Brandon et al.⁸ further evaluated the psychometric properties of the PROMIS CATs in children with JIA and studied the correlation of the PROMIS CATs with the JADAS-3 in patients with active versus inactive disease. Similar to our study, physical function domains correlated significantly with disease activity; however, domains measuring affective symptoms and relations with others correlated less with the JADAS-71 measure of disease activity.

We found that in general pediatric PROMIS CATs physical function domains, such as mobility and pain interference, showed moderate/strong correlations with legacy physical function measures. In contrast, peer relations, anxiety, and depression generally had moderate/weak correlations. This highlights that the standard disease activity measures may be inadequate to assess psychosocial patient-centered domains that are important in the care of children with chronic disease.¹³ In addition, all correlations between PROMIS CATs and the JADAS-71 score were weakened when the patient global health assessment was removed, suggesting that strong correlations were largely driven by the single PRO included in the JADAS-71. The generic CHAQ had much stronger correlations with the PROMIS CATs than the JADAS-71, even in mobility and pain domains. This suggests that the JADAS does not capture important aspects of health-related quality of life in children with JIA. In children, disease activity as measured by the JADAS-71 may be important for monitoring response to therapy, but a poor measure of overall health-related quality of life.

Evaluating PROs in young children is challenging, and so parent proxy PROMIS CATs are used in this setting. However, not all physicians have access to the technology needed to administer the online PROMIS CATs in the clinical setting. We found that parent proxy answers on the JADAS were extremely poorly correlated with PROMIS CATs, and we do not recommend the JADAS-71 be used in this way. The parent proxy CHAQ had stronger correlations with PROMIS CATs and would be the preferable PRO if administering standard legacy forms. Further work needs to be done validating pediatric proxy PROMIS short forms in the JIA populations.

Few patients completed the PROMIS CATs a second time in order to assess for test/retest reliability. The moderate test/retest reliability of PROMIS CATs in our patients should be interpreted cautiously because of this small sample size; this finding requires further evaluation.

Strengths of our study are that we compared a range of pediatric and parent proxy PROMIS CATs to both a JIA-specific disease activity measure and the most commonly used generic PRO. We also showed that PROMIS CATs can be successfully completed remotely by a large proportion of study participants.

Limitations of this study include a relatively small sample size at a single center, with a limited sociodemographic population and low disease activity. It is important for future studies of PROMIS CATs to assess larger, more diverse populations. Our findings should also be further explored in prospective studies.

PROMIS CATs are a more parsimonious method of accurately measuring overall health-related quality of life in children with JIA than legacy instruments such as the JADAS-71 or CHAQ. Ongoing study of the utility of PROMIS CATs in disease management in specific subgroups of JIA, especially surrounding response to biologic therapies, will be an important area of future research in children with JIA.