Validity and Reliability of Four Parent/Patient–Reported Outcome Measures for Juvenile Idiopathic Arthritis Remote Monitoring

Abstract

Objective

The aim of this work was to provide evidence of validity and reliability for 4 parent/child–reported outcome measures included in the Outcome Measures in Rheumatology juvenile idiopathic arthritis core domain set: the evaluation of the child's pain and level of disease activity, the assessment of morning stiffness duration, and an active joint count for proxy/self‐assessment.

Methods

Patients were included in the multinational study Epidemiology Treatment and Outcome of Childhood Arthritis. Criterion validity was assessed by examining the correlation of the 4 tested measures with physician measures and the clinical Juvenile Arthritis Disease Activity Score in 10 joints (cJADAS10) in the whole sample and after grouping patients by International League of Associations for Rheumatology (ILAR) category, geographic area, and education level. Reliability was assessed comparing 2 visits 7–14 days apart with intraclass correlation coefficients (ICCs).

Results

A total of 8,643 parents and 6,060 patients had all the evaluations available. Correlations of tested measures were moderate (0.4–0.7) with physician‐reported measures. The level of correlation with the cJADAS10 remained stable after grouping patients by ILAR category, geographic areas, and level of education of the parent filling the questionnaire. In 442 parents and 344 children, ICCs ranged between 0.79 and 0.87 for parents and 0.81 and 0.88 for children.

Conclusion

The 4 tested parent/child–reported outcomes showed good criterion validity and excellent reliability. These tools can be considered for remote patient assessment, when in‐person evaluation might not be possible.

INTRODUCTION

In recent years, the interest in the assessment of parent/child–reported outcomes in pediatric rheumatic diseases has gained increasing importance (1, 2, 3). These measures reflect the parent's and child's perception of the disease course and effectiveness of therapeutic interventions. The integration of these perspectives in clinical assessment may facilitate concordance with physicians’ choices and improve adherence to treatment and participation in a shared decision‐making strategy (4, 5, 6). In addition, the use of parent/child–reported outcomes may help the physician to identify with greater accuracy the salient issues for each patient and to focus the attention on relevant matters. Thus, information obtained from the parent or the child may contribute to the success of patient care (7). Moreover, the availability of reliable parent/child–reported outcomes could be crucial for remote monitoring of patients when in‐person clinical evaluation may be difficult or even impossible.

SIGNIFICANCE & INNOVATIONS.

• The integration of parent/child–reported outcomes in clinical assessment may facilitate concordance with physicians’ choices and improve adherence to treatment and participation in a shared decision‐making strategy in juvenile idiopathic arthritis.

• The selected measures of parent/patient assessment of pain, disease activity level, joints with active arthritis, and morning stiffness were valid and reliable tools for patient self‐monitoring.

• The selected measures are ideally suited for remote assessment of disease course and could potentially be included in a patient/parent–reported disease activity score for juvenile idiopathic arthritis.

The Outcome Measures in Rheumatology (OMERACT) Juvenile Idiopathic Arthritis (JIA) Working Group has recently provided a new core set of domains to be considered for the evaluation of children with JIA. JIA patients, their parents, and parents’ associations other than clinicians and researchers expert in pediatric rheumatology, contributed substantially to the identification and ranking of the most relevant disease domains (8, 9). Consensus methods and selection of domains procedure have been described in detail elsewhere (9). The domains may refer to physician‐reported measures, parent/child–reported outcomes, or laboratory examinations; some domains, such as the joint inflammatory signs, could be assessed by both a physician and a parent or patient. The aim of this work was to provide further evidence of validity and reliability for 4 parent/child–reported outcome measures, domains included in the OMERACT JIA core domain set. Among the domains that can be assessed by a parent/patient–reported measure, those that obtained the highest ranking after consensus voting were “pain” and “joint inflammatory signs/active joints.”

Pain is the most relevant symptom of children with JIA (10). Several studies have shown that pain is more prevalent in JIA than previously recognized and that a sizeable percentage of patients continue to report pain long after disease onset (11). High levels of pain limit physical activities, disrupt school attendance, and contribute to psychosocial distress. These issues make reduction of pain a key goal of treatment, and therefore the identification of a reliable tool to measure this domain is of major importance.

The evaluation of joint inflammatory signs and the count of joints with active disease is traditionally considered a physician‐reported domain. Joint count assessment by physicians through swollen and tender joints is considered the most conventional way of detecting clinical synovitis (12), and its importance in disease activity assessment is supported by the inclusion of joint counts in core data sets of disease activity indices such as the Juvenile Arthritis Disease Activity Score (JADAS) (13) and the American College of Rheumatology (ACR) pediatric response criteria (14) used in clinical trials, research, and clinical practice. Although only few data are available on self‐ or proxy‐reported joint count in JIA (15), a recent systematic literature review in adults with rheumatoid arthritis (RA) showed that patient‐reported joint counts have a potential role in the monitoring of disease activity, with satisfactory intraobserver and interobserver reliability (16).

Another domain that was highly ranked in the process leading to the development of the OMERACT JIA core domain set is the “patient's perception of disease/overall well‐being.” Surprisingly, physicians and other stakeholders considered this domain as more important than parents and patients. The domain of a patient's perception of disease activity is traditionally measured by the patient's global assessment or well‐being scale, such as in all the JADAS versions. Overall well‐being, or global health, and the patient's perception of disease activity, however, should probably be considered as different domains, with the former being broader and probably including the latter. Conceptually “global health” includes several aspects of health outcomes, that is, also those unrelated or not directly related to disease activity (17). The most widely adopted disease activity indices for RA include a patient self‐report measure. In the Simplified Disease Activity Index and the Clinical Disease Activity Index, this item is defined as “patient global assessment of disease activity,” whereas it is defined as “global health” in the Disease Activity Score (DAS) and in the 28‐joint DAS (18, 19). A measure of parent/patient perception of disease activity is available for JIA (20), but so far, that measure has never been incorporated in disease activity scores or in core measurement sets.

Finally, we decided to include in the study a fourth domain, “stiffness,” which was also highly ranked in the OMERACT core domain set consensus process. Morning stiffness is a major symptom of active disease in children with JIA and may have a profound impact on physical function and health‐related quality of life (21, 22). Assessment of morning stiffness was incorporated in the 2011 criteria for clinically inactive disease in JIA; patients can satisfy the definition of clinically inactive disease only if they have morning stiffness lasting ≤15 minutes (23). This cutoff was based on the belief that morning stiffness ≤15 minutes may represent damage from previous active disease or may be due to reasons other than active inflammation. Further analyses have shown that the presence of morning stiffness in JIA patients classified to be in clinically inactive disease by formal definitions is associated with worse parent perception of a child's health and disease status (24). Furthermore, morning stiffness was also a consistent predictor of worse outcome in various categories of JIA patients (25).

The aim of this study was to provide evidence of validity and reliability for 4 outcome measures assessing the parent/patient–reported domains of pain, joint inflammatory signs, patient's perception of disease, and morning stiffness. The selected tools are included in the Juvenile Arthritis Multidimensional Assessment Report (JAMAR), which was recently translated and cross‐culturally validated in the national language of 49 countries (26). These tools can be considered for inclusion in a parent/patient disease activity score.

PATIENTS AND METHODS

Subjects

Patients’ data were obtained from a large multinational data set of subjects enrolled in the Epidemiology Treatment and Outcome of Childhood Arthritis (EPOCA) study (27). Briefly, the EPOCA study is a survey conducted by the Pediatric Rheumatology International Trials Organization between 2011 and 2016, involving 9,081 JIA patients from 130 pediatric rheumatology centers in 49 countries, grouped into 8 geographical areas. Each participating center was asked to enroll 100 patients meeting the International League of Associations for Rheumatology (ILAR) criteria for JIA that were seen consecutively over 6 months or, if the center did not expect to see at least 100 patients within 6 months, to enroll all patients seen consecutively within the first 6 months after study start. Patients were included irrespective of their disease duration. For each visit, retrospective and physician‐reported data were collected, together with parent/child–reported outcomes included in the JAMAR, filled by a legal guardian and, when appropriate, by the patient. Ethical approval was obtained in all countries involved in the EPOCA study.

Outcome measures

In the EPOCA study, the questionnaire was proposed for completion by a caregiver (proxy‐reported measure) and by the patient when he/she was deemed by the caring physician able to understand and respond to the questions in the JAMAR (self‐reported measures). In some instances, the questionnaire was filled only by the patient.

The intensity of the child's pain was rated on a 21‐numbered circular scale corresponding to the traditional visual analog scale (VAS; 0 = no pain, 10 = extreme pain) (28), responding to the question “How much pain has your child had because of the illness over the past week?” The question was adapted for the patient's self assessment.

The level of the child's disease activity was also rated on a 21‐numbered circular scale (0 = no activity, 10 = maximum activity), responding to the question “Considering all the symptoms, such as pain, joint swelling, morning stiffness, fever (if due to arthritis), and skin rash (if due to arthritis), please evaluate the level of activity of your child's illness at the moment.” The question was adapted for the patient's self assessment.

The duration of morning stiffness was measured with a 5‐point Likert scale, with the following anchors: “less than 15 minutes,” “15–30 minutes,” 30 minutes to 1 hour,” “1–2 hours,” and “more than 2 hours.” The assessment of morning stiffness duration was preceded by a question asking whether morning stiffness was present or absent.

The proxy‐ and self‐assessment of joint inflammatory signs was obtained by asking the parent or the patient to rate the presence of pain or swelling in the following joints or joint groups, listed in a table: cervical spine, lumbo‐sacral spine, shoulders, elbows, wrists, small hand joints, hips, knees, ankles, and small foot joints. Patients or parents had to mark with an “X” by the affected joint/joints group. To each joint or joint group, 1 point was given in case of monolateral involvement and 2 points in case of bilateral involvement, if applicable. The sum obtained yielded the parent/patient joint count, with a score range of 0–18.

Validity

Criterion validity of tested measures was assessed by examining the correlation of the 4 tested measures with physician‐reported measures, an acute phase reactant (erythrocyte sedimentation rate [ESR]), and composite disease activity scores. Physician measures included the physician global assessment (PhGA) on a 0–10 scale, the number of joints with active arthritis, swollen joint count, tender joint count, and the number of joints with limitation on motion. Composite scores included the clinical JADAS in 10 joints (cJADAS10). The cJADAS10 is given by the sum of the PhGA, the parent/patient assessment of well‐being on a 0–10 VAS, and the number of joints with active arthritis cut at 10. For each analysis, the correlations of the well‐being VAS with physician‐reported measures and ESR were also presented, as a reference. Correlations of the well‐being VAS with the composite scores were not considered, the former being part of the latter.

To further assess the validity of the tools, correlations of the parents’ and patients’ measure with the cJADAS10 were also computed after grouping patients by ILAR category and by geographic area (northern Europe, western Europe, southern Europe, eastern Europe, North America, Latin America, Africa and Middle East, and southeast Asia). Correlations of parents’ measures were also analyzed grouped by family socioeconomic status (subjectively rated by the attending physician as low, average, or high), and by education level (elementary or lower, high school, or degree) of the parent completing the questionnaire. Finally, correlations of patients’ measures were analyzed after grouping subjects into 4 age groups: “6–10 years,” “11–13 years,” “14–18 years,” and “>18 years.”

Correlations were computed using Spearman's rank correlation method. Correlations were considered high if >0.7, moderate from 0.4–0.7, and low if <0.4 (28). We expected that correlations of tested tools would be higher with those measures more closely related to disease activity, such as the number of joints with active arthritis or the PhGA. Moreover, we expected that correlations would be higher with the composite score, because it includes a parent/child–reported outcome.

Reliability

When both parent's and patient's evaluations were available at the same visit, the Spearman's correlation (95% confidence interval) between the parent's and the child's rating of the 4 tested measures were calculated to demonstrate the interrater reliability of the tools. To assess test–retest reliability, a randomly selected subset of subjects was asked to complete the JAMAR again 7–14 days after the first time. In this subset of subjects, test–retest reliability of each measure was assessed with the intraclass correlation coefficient (ICC), using a 2‐way mixed‐effects model. The ICC was classified as follows: <0.2 = poor, 0.2–0.39 = fair, 0.4–0.59 = moderate, 0.6–0.79 = substantial, and ≥0.80 = almost perfect reproducibility (29). Test–retest reliability for individual measures was further examined by the Bland‐Altman approach (30) to test for random error of each variable. In this approach, the differences between the first and second measurement were plotted against their means. The mean difference ±1.96 × SD with its resulting interval represents 95% limits of agreement.

RESULTS

Descriptive characteristics of patients

A total of 8,643 parents and 6,060 patients had all the evaluations available for the tested tools in the EPOCA data set. In 5,947 instances, the questionnaire was filled by the patient and a parent at the same visit. Demographic figures, disease activity parameters, and parent/child–reported outcomes of patient samples are shown in Table 1.

Table 1.

Descriptive statistics of the EPOCA patient samples^*

	Parents (n = 8,643)	Patients (n = 6,060)
Female, no. (%)	5,756 (66.6)	3,968 (65.5)
Age at onset, years	5.4 (2.4–9.6)	7.3 (3.6–10.8)
Age at visit, years	11.3 (7.4–14.6)	13.1 (10.5–15.5)
ILAR category, no. (%)
Systemic arthritis	928 (10.7)	812 (13.4)
Persistent oligoarthritis	2,750 (31.8)	717 (11.8)
Extended oligoarthritis	931 (10.8)	1,573 (26.0)
RF‐negative polyarthritis	2,028 (23.5)	220 (3.6)
RF‐positive polyarthritis	355 (4.1)	1,474 (24.3)
Psoriatic arthritis	287 (3.3)	329 (5.4)
Enthesitis‐related arthritis	880 (10.2)	626 (10.3)
Undifferentiated arthritis	484 (5.6)	309 (5.1)
Socioeconomic status, no. (%)
Low	1,401 (19.6)	1,018 (20.6)
Average	4,954 (69.4)	3,399 (68.7)
High	786 (11.0)	533 (10.8)
Education, no. (%)
Elementary or lower	1,492 (24.4)	1,112 (26.6)
High school	2,823 (46.2)	1,956 (46.7)
Degree	1,790 (29.3)	1,120 (26.7)
Physician global assessment	1.0 (0.0–3.0)	1.0 (0.0–3.0)
Swollen joint count	0.0 (0.0–1.0)	0.0 (0.0–1.0)
Tender joint count	0.0 (0.0–1.0)	0.0 (0.0–2.0)
Joints with motion limitation	0.0 (0.0–2.0)	0.0 (0.0–2.0)
Joints with active arthritis	0.0 (0.0–2.0)	0.0 (0.0–2.0)
Erythrocyte sedimentation rate	10.0 (5.0–20.0)	10.0 (5.0–20.0)
JADAS10	3.5 (0.5–9.0)	3.5 (0.5–9.0)
cJADAS10	3.0 (0.5–8.0)	3.0 (0.5–8.0)
JADAS10 disease state, no. (%)†
Inactive disease	3,874 (44.8)	2,689 (44.4)
Minimal disease activity	1,442 (16.7)	1,009 (16.7)
Moderate disease activity	2,676 (31.0)	1,900 (31.4)
High disease activity	651 (7.5)	462 (7.6)
Pain VAS	1.0 (0.0–3.5)	1.0 (0.0–3.5)
Parent joint count	1.0 (0.0–2.0)	1.0 (0.0–2.0)
Morning stiffness	0.0 (0.0–1.0)	0.0 (0.0–1.0)
Disease activity VAS	1.0 (0.0–3.5)	0.5 (0.0–3.5)
Well‐being VAS	1.0 (0.0–3.5)	0.5 (0.0–3.5)

^*Values are the median (interquartile range) unless indicated otherwise. cJADAS10 = clinical Juvenile Arthritis Disease Activity Score in 10 joints; EPOCA = Epidemiology Treatment and Outcome of Childhood Arthritis; ILAR = International League of Associations for Rheumatology; RF = rheumatoid factor; VAS = visual analog scale.

^†According to the American College of Rheumatology 2021 JADAS10 cutoffs (ref. 31).

Validity correlations

In the EPOCA parents’ data set, correlations of all tested measures are in the moderate range with physician‐reported measures of disease activity, with the exception of morning stiffness (ρ = 0.17–0.24) and in the poor range with the limited joint count (ρ = 0.30–0.41) and with ESR (ρ = 0.32–0.43). Correlations of the parent/patient joint count, the disease activity scale, and the pain scale were strong with the cJADAS10 (Table 2). Correlations of patient‐reported measures were similar.

Table 2.

Spearman's correlations of the parent‐ and patient‐reported outcomes with physician's and laboratory measures in EPOCA data sets^*

	PhGA	NAJ	SJC	TJC	NLJ	ESR	cJADAS10
Parent‐reported outcomes
Parent joint count	0.59 (0.58, 0.61)	0.56 (0.54, 0.57)	0.5 (0.48, 0.52)	0.57 (0.55, 0.58)	0.41 (0.39, 0.43)	0.24 (0.22, 0.26)	0.69 (0.68, 0.71)
Morning stiffness	0.43 (0.41, 0.44)	0.35 (0.33, 0.37)	0.32 (0.3, 0.34)	0.43 (0.42, 0.45)	0.3 (0.28, 0.32)	0.17 (0.14, 0.19)	0.53 (0.51, 0.54)
Disease activity VAS	0.6 (0.58, 0.61)	0.48 (0.47, 0.5)	0.43 (0.41, 0.45)	0.53 (0.51, 0.54)	0.37 (0.35, 0.39)	0.24 (0.21, 0.26)	0.73 (0.72, 0.74)
Pain VAS	0.57 (0.55, 0.58)	0.45 (0.43, 0.46)	0.39 (0.37, 0.41)	0.55 (0.53, 0.56)	0.35 (0.33, 0.37)	0.22 (0.19, 0.24)	0.71 (0.7, 0.73)
Well‐being VAS	0.54 (0.53, 0.56)	0.43 (0.41, 0.45)	0.38 (0.36, 0.39)	0.5 (0.48, 0.52)	0.35 (0.33, 0.36)	0.22 (0.2, 0.24)	0.82 (0.82, 0.83)
Patient‐reported outcomes
Patient joint count	0.58 (0.56, 0.59)	0.52 (0.5, 0.54)	0.46 (0.44, 0.48)	0.58 (0.56, 0.6)	0.37 (0.35, 0.39)	0.21 (0.18, 0.24)	0.68 (0.66, 0.69)
Morning stiffness	0.42 (0.4, 0.45)	0.35 (0.33, 0.37)	0.31 (0.29, 0.33)	0.43 (0.41, 0.45)	0.27 (0.25, 0.3)	0.15 (0.13, 0.18)	0.52 (0.5, 0.54)
Disease activity VAS	0.59 (0.57, 0.6)	0.47 (0.45, 0.49)	0.41 (0.39, 0.44)	0.53 (0.51, 0.55)	0.34 (0.32, 0.36)	0.21 (0.19, 0.24)	0.71 (0.7, 0.72)
Pain VAS	0.56 (0.54, 0.58)	0.43 (0.41, 0.45)	0.37 (0.34, 0.39)	0.54 (0.52, 0.56)	0.33 (0.31, 0.35)	0.2 (0.17, 0.22)	0.7 (0.68, 0.71)
Well‐being VAS	0.53 (0.51, 0.55)	0.41 (0.39, 0.44)	0.36 (0.34, 0.39)	0.5 (0.48, 0.52)	0.32 (0.29, 0.34)	0.21 (0.18, 0.24)	0.71 (0.7, 0.73)

^*Values are the correlation (95% confidence interval). cJADAS10 = clinical Juvenile Arthritis Disease Activity Score in 10 joints; EPOCA = Epidemiology Treatment and Outcome of Childhood Arthritis; ESR = erythrocyte sedimentation rate; NAJ = number of joints with active arthritis; NLJ = number of joint with limitation on motion; PhGA = physician global assessment of overall disease activity; SJC = swollen joint count; TJC = tender joint count; VAS = visual analog scale.

The level of correlation of the tested parent measures with the cJADAS10 remained stable after grouping patients by ILAR category (Figure 1A). Similar results were obtained for patient measures (see Supplementary Figure 1, available on the Arthritis Care & Research website at http://onlinelibrary.wiley.com/doi/10.1002/acr.24855). In the same analysis with patients grouped in 8 geographic areas, correlation levels were similar, although on average, they were higher in Latin America and slightly lower in North America (Figure 1B for parents’ measures, and for patients see Supplementary Figure 2, available on the Arthritis Care & Research website at http://onlinelibrary.wiley.com/doi/10.1002/acr.24855).

Figure 1.

Comparison of Spearman's correlations of morning stiffness duration, active joint count, level of disease activity, and level of pain assessed by parents with the clinical Juvenile Arthritis Disease Activity Score in 10 joints among the International League of Associations for Rheumatology categories of juvenile idiopathic arthritis (A) and the different geographic areas (B). RF = rheumatoid factor; VAS = visual analog scale.

In 6,287 patients in the EPOCA data set for whom these data were available, the level of correlation of the 4 measures with the cJADAS10 did not change according to the level of education of the parent completing the questionnaire (data not shown). Finally, in 7,336 subjects, correlations remained in the same category across 3 different categories of socioeconomic status (low, moderate, or high) of the patient's family (Table 3). The correlations with cJADAS10 of the 4 measures obtained from patients progressively increased from the lower age group to the higher age group (Table 3).

Table 3.

Spearman's correlations of the parent‐reported outcomes with cJADAS10 by socioeconomic status and education level and correlations of the patient‐reported outcomes with cJADAS10 by age group^*

	No.	Parent joint count	Morning stiffness	Pain VAS	Disease activity VAS
Socioeconomic status
Low	1,401	0.75 (0.72, 0.78)	0.55 (0.51, 0.59)	0.78 (0.75, 0.8)	0.74 (0.71, 0.76)
Average	4,954	0.68 (0.67, 0.7)	0.52 (0.5, 0.54)	0.72 (0.7, 0.73)	0.7 (0.69, 0.72)
High	786	0.72 (0.68, 0.75)	0.53 (0.47, 0.58)	0.75 (0.71, 0.78)	0.72 (0.68, 0.76)
Education
Elementary or lower	1,492	0.71 (0.68, 0.74)	0.52 (0.48, 0.56)	0.74 (0.72, 0.77)	0.69 (0.66, 0.72)
High school	2,823	0.71 (0.69, 0.73)	0.51 (0.48, 0.54)	0.74 (0.72, 0.76)	0.71 (0.69, 0.73)
Degree	1,790	0.71 (0.68, 0.73)	0.56 (0.52, 0.59)	0.75 (0.72, 0.77)	0.73 (0.7, 0.75)
Age group, years
6–10	1,768	0.65 (0.62, 0.68)	0.51 (0.47, 0.55)	0.66 (0.64, 0.69)	0.69 (0.66, 0.71)
11–13	1,801	0.67 (0.64, 0.69)	0.49 (0.45, 0.53)	0.69 (0.66, 0.71)	0.7 (0.67, 0.72)
14–18	2,305	0.7 (0.67, 0.72)	0.53 (0.5, 0.56)	0.72 (0.69, 0.74)	0.73 (0.71, 0.75)
>18	114	0.73 (0.62, 0.82)	0.52 (0.37, 0.65)	0.73 (0.62, 0.81)	0.77 (0.66, 0.84)

^*Values are the correlation (95% confidence interval) unless indicated otherwise. cJADAS10 = clinical Juvenile Arthritis Disease Activity Score in 10 joints; VAS = visual analog scale.

Reliability measurement

Interrater reliability

Paired data for parents and patients were available in 5,947 visits. The Spearman's correlations between the parent's and the patient's rating were 0.83 for the disease activity scale, 0.84 for the morning stiffness scale, and 0.88 for both the pain scale and the joint count. As a reference, the correlation of the well‐being scale between parent's and patient's rating was 0.80.

Test–retest reliability

After a median of 7 (interquartile range 6–7) and 7 (6; 7) days from first completion, the questionnaire was filled a second time by 442 parents and 344 patients, respectively. ICCs showed almost perfect reproducibility (ICC >0.80) for all measures, with the exception of the disease activity VAS for parents’ assessment (ICC = 0.78) and the well‐being VAS for parents’ assessment (ICC = 0.73) (Table 4).

Table 4.

Test–retest reliability: intraclass correlation coefficients (ICCs) of parent‐ and patient‐reported outcomes between first and second assessment^*

	No.	ICC
Parent‐reported outcomes
Parent joint count	442	0.83
Morning stiffness	442	0.86
Pain VAS	442	0.87
Disease activity VAS	442	0.78
Well‐being VAS	441	0.73
Patient‐reported outcomes
Patient joint count	344	0.84
Morning stiffness	344	0.88
Pain VAS	344	0.81
Disease activity VAS	344	0.83
Well‐being VAS	344	0.86

^*Second assessment was performed no more than 2 weeks after first assessment. VAS = visual analog scale.

Figure 2 presents Bland‐Altman plots for each of the 4 disease activity indices, demonstrating the mean difference between measurements with 95% limits of agreement (morning stiffness 0.05 [–1.3, 1.4], joint count 0.03 [–2.9, 3.0], VAS disease activity 0.3 [–3.1, 3.7], and VAS pain 0.3 [–2.6, 3.3]) according to the baseline value. Bland‐Altman plots for patients’ measures are shown in Supplementary Figure 3, available on the Arthritis Care & Research website at http://onlinelibrary.wiley.com/doi/10.1002/acr.24855.

Figure 2.

Agreement between scores obtained by the morning stiffness duration (A), parent assessment of joint count (B), level of disease activity by visual analog scale (VAS) (C), and level of pain by VAS (D) measures at first and second assessment illustrated by Bland‐Altman plots. Interval between first and second assessment was 7 (interquartile range 6–7) days. Broken lines indicate the mean and 95% limit of agreement. Each dot represents an individual patient.

DISCUSSION

Patient self‐assessment or parent proxy‐assessment are nowadays considered of foremost importance in the care of chronic conditions, and in particular, of JIA, with a disease course that is mostly unpredictable. Remote patient self‐assessment could foster the early recognition of disease flares, leading to timely and effective medical treatment.

This study describes the assessment of validity and reliability of 4 parent/child–reported outcomes for JIA. The choice of the 4 measures to be tested was based on the updated OMERACT core domain set for studies in JIA. In fact, 3 of these measures (pain, disease activity, and joint count) refer to domains indicated as mandatory by the OMERACT workshop, whereas stiffness is considered an important, even though optional, domain. To provide adequate strength to the validation process, the criterion validity and reliability were assessed in a large sample, including >6,000 patients from several different countries. These patients are likely to be representative of the whole spectrum of JIA phenotypes, as well as cultural background, education, and socioeconomic status. Although the patient sample was skewed toward a low level of disease activity, the EPOCA study data set was large enough to include a representative number of subjects for each disease state based on recent JADAS10 thresholds (31).

All tested measures demonstrated good criterion validity, by yielding moderate correlations with the physician‐reported measures, such as PhGA and the number of joints with active arthritis, and strong correlations with the JADAS10 and cJADAS10, with the exception of morning stiffness, which remained moderately correlated with the composite disease activity scores. Correlations with cJADAS10 were similar after grouping patients by ILAR category and geographic area, suggesting that our results could be representative of different clinical settings. Furthermore, the level of correlation remained stable irrespective of the socioeconomic status of the family and the parent education level, indicating that the criterion validity of the 4 measures is not significantly affected by the social context of the family. On the other hand, the correlations with cJADAS10 of the 4 measures obtained by the patients increased in the older age group, suggesting that the higher the patient age the more reliable the parent/child–reported outcome. This finding is in line with previously reported results on the general pediatric population (32).

The 4 parent/child–reported outcomes were also found to be very reliable tools, by obtaining correlations in a strong range both in interrater and in test–retest reliability analysis. Bland‐Altman plots showed 95% limits of agreement, with approximately ±3 for VAS pain, disease activity, and joint count, meaning that a difference of >3 could be interpreted as a real change, with a 5% risk of being wrong. Furthermore, the plots showed that differences between test–retest evaluations were more pronounced in the middle of the scales (almost all test–retest combinations outside the limits of agreement occur between 2.5 and 7.5 points), whereas scores toward the lower end of the scales tended to be reproduced more accurately. Thus, parents and children deeming themselves in remission or low disease activity could report this fact trustworthily. Also, children with at least some disease activity would probably report that fact again, if asked to re‐evaluate their disease activity, even though the exact score attributed to their disease activity might vary by ±3 points.

Pain perception in children with JIA is multifactorial and results from the combination of biologic, psychological, and environmental factors (11). Despite being the most common and distressful symptom of JIA, pain has been widely neglected in the development of outcome measures for JIA (33). Indeed, pain assessment is not included in the Wallace criteria for clinically inactive disease (34) or in the American College of Rheumatology Pediatric response criteria (23), which have been used as outcome measure in all the recent trials on biotechnologic drugs in JIA. Yet pain evaluation has been included in the updated core domain set for studies in JIA by OMERACT as a mandatory domain (9). The use of age‐appropriate, reliable, and valid tools is recommended to assess pain in children with chronic arthritis (35). In fact, a reliable appraisal of pain in patients with JIA requires the use of well‐validated pain assessment tools that could capture the multifaceted aspects of the pain experience (32). The 21‐numbered circular VAS has been found to be a simpler and more feasible measure for pain self‐report compared to the 100‐mm VAS (28). Our study confirmed the good criterion validity of the pain 21‐numbered circular VAS, which yielded strong correlations with the composite scores for disease activity JADAS10 and cJADAS10 and moderate correlations with physician‐reported measures, such as the PhGA and the active joint counts. In the reliability analysis, the pain scale performed better among the 4 measures tested. Altogether, these results confirm that the 21‐numbered circle is a feasible tool for pain self‐ or proxy‐report in JIA, and its use should be encouraged both in standard clinical practice and in research settings to allow clinicians and researchers to track child pain over time.

To our knowledge, only 2 studies have investigated the role of self‐ or proxy‐reported joint count in JIA (15, 36). Even though both showed that patients and/or parents tended to overestimate the presence of arthritis when marking active joints on a manikin‐format joint, Dijkstra et al found a moderate agreement between the physician and the patient total joint count. In line with that, in our analysis, both parent and patient joint count yielded moderate correlation with the number of active, swollen, and tender joint counts provided by the physician, demonstrating good criterion validity. Furthermore, parents’ joint counts correlated strongly with the patient's count, and both demonstrated a very high interrater and retest reliability. In many instances, such as when evaluating whether treatment needs to be escalated, the exact number and location of active joints is of less importance, as long as the overall evaluation of joint activity is in agreement between parents, patients, and physicians. This result suggests that, even though parent/patient–reported joint count cannot replace the physician's joint assessment in clinical practice, it could be helpful in JIA disease activity remote monitoring. Admittedly, the tested joint count is based on a reduced and selected list of joints as it is included in the JAMAR (20).

So far, the patient's perception of the level of disease activity in JIA has been measured through the parent/child overall well‐being VAS, both in disease activity scores and in a core set of multiple criteria for the definition of different disease activity states (9, 13). However, the well‐being VAS measures a broader construct than the level of disease activity, including all the aspects of the disease burden affecting the patient's health‐related quality of life. In this study, we provided evidence supporting the efficacy of a VAS specifically designed to assess the level of disease activity, as disease level is perceived by the patient or by caregivers. Notably, of the 2 most widely adopted disease activity scores for adults with RA, the DAS incorporates a patient global health tool (19), whereas the Simplified Disease Activity Index incorporates a patient global disease activity tool (17). Further discussion is urgently needed to identify the measure that better serves the purpose of describing the parents’ or patients’ perspective of the disease course. In the present study, the correlation of the disease activity scale with physician‐reported measures reached greater levels compared to the overall well‐being VAS. On this basis, parent and child disease activity VAS may be a suitable indicator of disease status in children with JIA, and its incorporation in the composite disease activity scores should be further investigated.

Among the 4 parent/child–reported outcomes tested, morning stiffness was the one with the lower performance in the correlation analysis, although still moderately correlated with the PhGA and the JADAS10 and highly reliable. This finding may be at least in part due to the use of a 5‐point Likert scale, transformed to a 0–10 scale. Although not included in the OMERACT core‐set list of mandatory variables (9), the duration of morning stiffness is included in the ACR provisional definition of inactive disease (23). Recently, some discussion has been raised on the possibility of allowing a morning stiffness duration of 15 minutes in the definition of remission, as most parents do not consider their child to be in remission in the presence of morning stiffness, even of a short duration (24).

Our results should be interpreted in the light of some potential limitations. First, multiple tools are available to measure the selected domains. Our analysis was limited to the instruments included in the JAMAR. Second, test–retest reliability was assessed with a time interval of 7–14 days between the first and second assessment. We believe this time span is appropriate to assess test–retest reliability in a chronic disease like JIA on a large scale, but we did not formally assess whether the level of disease activity was the same at the 2 time points. Another key aspect of the evaluation of outcome measures is responsiveness to change and determining minimal clinically important differences, which requires longitudinal data analysis.

In conclusion, we have provided further evidence of validity and reliability of 4 parent/child–reported outcome measures, whose referring domains are included in the OMERACT JIA core domain set. By documenting these key measurement properties, we have shown that these measures are valid instruments for patient/parents’ evaluation of disease activity in JIA and are, therefore, potentially applicable not only in a research setting but also in the standard clinical care. In particular, these parent/child–reported outcomes are ideally suited to be included in a parent/patient–reported disease activity score for remote monitoring of patients.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be submitted for publication. Dr. Consolaro had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

van Dijkhuizen, Ridella, Naddei, Trincianti, Ruperto, Ravelli, Consolaro.

Acquisition of data

Avrusin, Mazzoni, Sutera, Ayaz, Penades, Constantin, Herlin, Oliveira, Rygg, Sanner, Susic, Sztajnbok, Varbanova.

Analysis and interpretation of data

van Dijkhuizen, Ridella, Consolaro.

Supporting information

Disclosure Form

Supplementary figure I Comparison of Spearman correlations of morning stiffness duration, active joint count, level of disease activity and level of pain assessed by patients with the clinical Juvenile Arthritis Disease Activity Score 10 among the ILAR categories of JIA

Supplementary figure II Comparison of Spearman correlations of morning stiffness duration, active joint count, level of disease activity and level of pain assessed by patients with the clinical Juvenile Arthritis Disease Activity Score 10 grouped by geographic areas

Supplementary figure III Agreement between scores obtained by the morning stiffness duration, patient assessment of joint count, level of disease activity and level of pain measures at first and second assessment illustrated by Bland– Altman plots. Interval between first and second assessment was 7 (6;7) days