Validation of two simple patient-centered outcome measures for virtual monitoring of patients with idiopathic inflammatory myositis

Abstract

Background

Telemedicine has changed the landscape of patient care with wider use of patient-centered outcome measures (PCOMs). We evaluated two novel task–based PCOMs namely ten times arm lift (AL) test and two-minute walk distance (2MWD) in idiopathic myositis (IIM).

Methods

This was a cross-sectional observational study with the enrolment of adult IIM (ACR/EULAR criteria) patients with active/inactive disease. Active disease was defined as any two of increase in immunosuppression within 3 months, elevated muscle enzymes, physician VAS ≥ 2, worsened cutaneous disease, or fall in MMT8 < 76. Standard myositis core set measures (CSMs) were evaluated and test–retest validity [Cronbach’s alfa (CA)], construct validity (Pearson’s correlation), and discriminant validity (between active/inactive IIM) were assessed. The results were further validated in a separate tele-rheumatology cohort.

Results

Among 22 IIM patients (68%-female) of age 30.5(19–62) years, AL and 2MWD showed excellent test–retest reliability (CA-0.987, 0.99). AL exhibited moderate-strong correlation with all CSMs except CK levels and MDI. In contrast, 2MWD values were highly variable without CSM correlation. A higher AL time discriminated active and inactive myositis (16.6 vs 11 s, p = 0.006) with an AUC of 0.882 (p = 0.006). AL > 12.8 s had 94% negative predictive value (NPV) for active muscle disease. In the validation cohort (47 patient visits among 26 patients), AL significantly differentiated between active vs. inactive disease with an NPV of 95%.

Conclusions

AL test exhibits pilot evidence of construct and discriminant validity in patients with IIM requiring further evaluation. 2MWD was not a good test for outcome evaluation of IIM patients.

Key Points • Novel task–based patient-centered outcome measures were evaluated for remote monitoring of muscle strength in IIM. • Ten times arm lift (AL) test showed strong test–retest reliability as well as provide pilot evidence of construct and discriminant validity in patients with IIM unlike 2-min walk distance. • This provides preliminary evidence to further evaluate the role of AL as patient-centered outcome measure in patients with IIM for virtual clinical trials.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10067-021-05990-3.

Introduction

The idiopathic inflammatory myopathies (IIM), i.e., myositis are chronic and disabling autoimmune disorders with a primary target of muscle and/or skin. Myositis leads to significant morbidity and lower quality of life partly due to limitation in mobility and activities of daily living as well as endurance and fatigue challenges [1, 2]. Furthermore, limited motility often translates to limited access to healthcare and potential delays in treatment. The onset of the coronavirus disease 2 (COVID-19) pandemic precipitated a global health emergency, with a detrimental impact on the delivery of healthcare to patients with IIM as well as other chronic rheumatic diseases [3, 4]. However, the widespread use of tele-rheumatology and rapid digitalization of healthcare helped maintain significant continuity of chronic care in the pandemic period [5–8].

A visible shift of in-person clinical practice to virtual or remote consulting in times of a global pandemic has necessitated a wider use of patient-centered outcomes (PCOMs) and device-based objective measures of disease assessment for remote monitoring of the patients [9–11]. A few studies have explored the potential tool for monitoring disease remotely, a realm that may assume larger importance for future optimal disease management [12–14].

If valid for longitudinal assessment of disease activity, these PCOMs could be a valuable addition to existing clinical measures used in IIM, because they would provide an objective assessment of muscle function remotely. Moreover, these would be a feasible and convenient means of tracking patient’s longitudinal changes in disease activity and severity remotely, while reducing travel and healthcare costs to the patient and the administrative system. The convenience and rapidity of assessment also hand the baton of self-care to patients themselves, allowing physicians to manage disease flares in a timely fashion [6].

Therefore, we aimed to prospectively validate 2 novel task–based PCOMs for evaluation of upper and lower extremity physical function, namely arm lift (AL) test and two-minute walk distance (2MWD), respectively, in adults with idiopathic myositis (IIM).

Methods

Patient population

Patients were identified from 280 IIM patients enrolled into a retro-prospective MyoCite registry cohort (December 2017 to March 2021) at a rheumatology tertiary-care center in India. Inclusion criteria were all adults with the diagnosis of PM or DM as per 2017 ACR/EULAR myositis classification criteria who were evaluated at rheumatology clinic between January and March of 2021. Children and those with known symptomatic ILD (dyspnoea VAS ≥ 2), PAH, or significant lower limb arthritis (pain VAS ≥ 2) were excluded due to the nature of the tests being evaluated and the pilot nature of the study [13–15]. Patients with other conditions interfering with their physical function, such as severe fibromyalgia, pregnancy, or fractures, were also considered ineligible for enrolment. We used the STROBE checklist for reporting methods and results.

Study design and measures

This was a prospective, cross-sectional observational study in which physician assessments, PCOMs, functional assessments, and laboratory studies were carried out for patients enrolled from the center during their clinical visit (in-clinic observation cohort). A separate tele-rheumatology cohort was established for patients seen and enrolled during the audio-visual teleconsultation visit.

Data collection

In-clinic observation cohort

Patients with IIM and satisfying inclusion criteria were asked to perform two novel PCOMs, including time to lift arms up ten times in seconds, and distance walked in 2 min under physician guidance in the outpatient clinic. Both active and inactive disease patients were enrolled. Active disease was defined as any of the two features including escalation of immunosuppression within 3 months, elevated muscle enzymes, physician global disease activity (10 cm visual analogue scale VAS) of > 2, clinical worsening in dermatomyositis skin rash, or manual muscle testing (MMT8) < 76 (our of 80 total). Muscle disease remission was defined as a composite assessment of muscle disease activity by the physician, encompassing PhGA, muscle enzymes, and MMT8, all of which were normal or no disease activity. We also evaluated all six myositis core set measures (CSMs) [MMT8, patient and physician global disease activity (PtGA and PhGA), muscle enzymes, health assessment questionnaire disability index (HAQ-DI), extra-muscular disease activity using the myositis disease activity assessment test (MDAAT), and myositis damage index (MDI)]. Demographic and medication history was also recorded.

Arm lift test

Patients were requested to lift their arms (abduction) ten times as fast as feasible. Arm lift (AL) was by abduction, with the elbows straight, above the head, until both arms were parallel, followed by bringing arms to the side. The manoeuver was to be repeated as fast as possible ten times, with a timer set to record the total number of seconds needed to complete the task (Fig. 1). The task was repeated thrice and values were recorded for each time. Mean scores were calculated for analysis.

Fig. 1.

Arm lift test and 2-min walk distance (infographic)

Two-minute walk distance

For the 2MWD, the patients were asked to walk in a designated corridor with a length of 35 m back and forth on a prescribed line. The patients were motivated to walk as fast as possible, spending the least amount of time turning comfortably. The total number of rounds was recorded to obtain the total distance walked in 2 min until the timer went out.

Tele-rheumatology cohort

A cohort of patients who were evaluated over tele-rheumatology was also enrolled into the study during the period when COVID pandemic restricted in-person clinic visits. Patients were explained about the two PCOMs procedures over teleconsultation with verbal and/or written instructions. Patients had the opportunity to ask questions and call back if further explanations were required. Patients were also assessed for disease activity remotely by the physician, based on all the information available and limited examination during teleconsultation. Routine blood investigations (including muscle enzymes, hemogram, liver and kidney function tests) were also recorded. Active disease was defined as any two of the following: escalation of immunosuppression within 3 months, elevated muscle enzymes, physician disease activity of 10 cm VAS ≥ 2, or worsening of dermatomyositis skin rash.

Data retrieval

The MyoCite cohort has detailed clinical and biorepository archives as previously detailed [6, 15–18]. Patient demographics (age, gender), baseline characteristics, such as myositis subtype, and myositis-specific and -associated antibodies (line immunoassay, Lubeck, Germany) were obtained from the archives. Definitions for organ involvement and type of IIM are as previously described [6, 15–17, 19]. Data were collected from 1st Jan 2021 till 31st March 2021.

Statistical analysis

Disease-related variables including the clinico-serologic subtypes were compared between those with active and inactive disease. Physician VAS, patient VAS, MMT8, HAQ-DI, MDAAT (MYOACT and MITAX subcomponents), muscle enzymes, and MDI at the time of assessment were correlated with the novel outcome measures (construct validity). All values are in the median and inter-quartile ranges. Non-parametric tests were used. SPSS version 26 was used for analysis.

Discriminant validity was assessed for AL and 2MWD between active and inactive disease and its variation with MMT8 and HAQ-DI. Test–retest ability was checked between various repeated recordings of the patients within 1 month period using Cronbach’s alpha. Sensitivity to change was assessed for followed up patients with paired t-test.

ROC curve was used to calculate AUC, cut-offs of the outcome measures. Performance of the PCOMs based on the cut-off was assessed in the validation cohort. Further differences in the PCOMs between active and inactive diseases were tested in the validation cohort.

Results

In-clinic observation cohort

Twenty-two adult IIM patients (68% female) with a median (IQR) age of 30.5 (19–62) years were enrolled in the study. Patients with varying severity of disease activity and muscle weakness were enrolled with a median (IQR) MMT-8 of 79 (74–80) and physician disease activity of 0 (0–2). Median AL time and 2MWD were 11.9 (10.5–14) seconds, 198 (167–225) meters, respectively. Other baseline characteristics are detailed in Table 1.

Table 1.

Patient characteristics

	Observational cohort		Tele-rheumatology cohort (n = 47 consults of 26 patients)
	Clinic visit (n = 22)	Tele-follow-up (n = 30 consults of 12 patients)
Age (years)	30.5 (19–62)	25 (18–50)	33 (22–45)
M:F	7:15	4:8	5:21
PtGA	0 (0–2)	-	-
PhGA	0 (0–2)	-	-
MMT8	79 (74–80)	-	-
HAQ-DI	0.25 (0–0.56)	-	-
AST (IU/L)	36 (26–51)	38 (37–47)	37 (28–43)
ALT (IU/L)	36 (25–50)	40 (32–50)	35 (24–51)
CK (IU/L)	164 (109–613)	418 (220–792)	209 (65–443)
LDH (IU/L)	371 (262–566)	271 (247–314)	293 (221–392)
Ex-Musc GA	0 (0–2.25)	-	-
Muscle disease VAS	0 (0–4)	-	-
MYOACT (0–60)	0.49(0–3)	-	-
MITAX (0–63)	1.9(0–12)	-	-
AL test (seconds)	11.9 (10.5–14)	11 (9–16)	13 (10–18)
2MWD (meters)	196 (167–225)	177 (135–209)	148 (107–185)
Number of patients with active muscle disease	5	6	8

AST aspartate transaminase, ALT alanine transaminase, CK creatine kinase, HAQ-DI health assessment questionnaires disability index, LDH lactic acid dehydrogenase, MDI myositis damage index, MITAX myositis intention to treat activity index, MITAX em myositis intention to treat activity index extra-muscular, MYOACT myositis disease activity assessment visual analogue scales, MYOACT em myositis disease activity assessment visual analogue scales extra-muscular, PtGA patient global assessment VAS, PhGA physician global assessment VAS, PhEMS physician extra-muscular global assessment VAS, VAS visual analogue scale

Test–retest reliability

AL and 2MWD showed excellent test–retest reliability (Cronbach’s alpha 0.987 and 0.99, respectively, n = 12). AL test had moderate to strong correlation with all myositis CSMs including PhGA (r = 0.725), PtGA (r = 0.710), MMT8 (r =  − 0.784), HAQ-DI (r = 0.483), AST (r = 0.576), ALT (r = 0.586), LDH (r = 0.753), muscle disease VAS (r = 0.414), and extra-muscular GA (r = 0.479) (Table 2, Fig. 2, Supplementary Fig. 1). 2MWD correlated moderately with muscle disease VAS (r = 0.426) and poorly with all other core set measures (Table 2).

Table 2.

Correlation of arm lift test and 2MWD with disease parameters (construct validity) in observation cohort (n = 22)

	Arm lift test		2MWD
	r	p	r	p
Core set measures
PtGA	0.725	0.000	0.106	0.632
PhGA	0.710	0.000	0.100	0.651
MMT-8	− 0.784	0.000	− 0.074	0.736
HAQ-DI	0.483	0.023	0.204	0.363
Muscle enzymes
AST (U/L)	0.576	0.004	0.103	0.639
ALT (U/L)	0.586	0.003	0.260	0.231
CK (U/L)	0.286	0.221	− 0.048	0.841
LDH (U/L)	0.753	0.000	0.112	0.640
Ex-Musc GA	0.479	0.024	− 0.413	0.056
Other measures
Muscle disease VAS	0.414	0.050	0.426	0.043
MDI severity of damage score	− 0.012	0.743	0.051	0.858
MDI extent of damage score	− 0.059	0.789	− 0.013	0.955
MDI extended damage score	− 0.182	0.406	− 0.133	0.545

AST aspartate transaminase, ALT alanine transaminase, CK creatine kinase, ESR erythrocyte sedimentation rate, HAQ-DI health assessment questionnaires disability index, LDH lactic acid dehydrogenase, MDI myositis damage index, MITAX myositis intention to treat activity index, MITAX em myositis intention to treat activity index extra-muscular, MYOACT myositis disease activity assessment visual analogue scales, MYOACT em myositis disease activity assessment visual analogue scales extra-muscular, PtGA patient global assessment VAS, PhGA physician global assessment VAS, PhEMS physician extra-muscular global assessment VAS, VAS visual analogue scale

Those bolded are significant (p < 0.05)

Fig. 2.

Scatter plot of correlation between Arm lift test with A patient global assessment VAS, B physician global assessment VAS, C MMT8, D aspartate transaminase (AST), E lactate dehydrogenase (LDH), F health assessment questionnaire disability index (HAQ-DI)

Construct validity

AL test had moderate to strong correlation all myositis CSMs including MMT8, PtGA, PhGA, HAQ-DI, Extra-muscular global (Ex-MuscGA), and muscle enzymes (LDH, AST, ALT) but not with CK levels (Table 2, Fig. 2, Supplementary Fig. 1). In addition, it showed a moderately strong association with muscle disease activity VAS and muscle disease remission, whereas no association with disease damage as measured by MDI. In contrast, 2MWD had huge variability and no significant association with any of the core set or other measures except a moderate correlation with muscle disease activity 10 cm VAS (Table 2, Supplementary Fig. 2).

Discriminant validity

A higher AL time discriminated active and inactive myositis at baseline (16.6 vs 11 s, p = 0.006) and discriminated between patients with and without moderate to severe muscle weakness (significant) and high vs. low HAQ score (non-significant trend) (Fig. 3A–C). However, 2MWD did not discriminate based on muscle disease activity or other measures.

Fig. 3.

Arm lift test variation based on muscle disease remission (A), MMT8 (B), and HAQ-DI (C)

AUC and cut-off values

AUC for active versus inactive disease was 0.882 (p 0.006) with the AL test (Fig. 4A). A cut-off of 12.8 s had 83% sensitivity, 83% specificity, 57% positive predictive value, and 94% negative predictive values for active muscle disease.

Fig. 4.

A ROC between arm lift time and active disease. B Arm lift test in active versus inactive disease in validation cohort (47 patient visits)

Tele-rheumatology cohort

Twenty-six patients contacted the tele-rheumatology service during the lockdown period amounting to 47 remote patient visits. The median age of the virtual cohort was 33 years and 81% were women (Table 1). AL significantly differentiated between active vs. inactive patients (Fig. 4B). A cut-off of 12.8 s for the AL test discriminated patients with active and inactive disease with a sensitivity of 97%, specificity of 51%, a positive predictive value of 27%, and a negative predictive value of 95%.

Discussion

We evaluated the utility of AL and 2MWD as two novel and easy, patient-centered outcome measures for monitoring in IIM especially with the possibility of remote assessment of patient and self-administration of the test by the patient at home. AL test correlated well with standard myositis disease activity CSMs including muscle weakness, providing preliminary evidence for its role as PCOM in IIM especially in clinical trials and clinical practice, especially for remote clinical trials and telemedicine consultation. We validated our observations in a teleconsultation-based cohort to understand utility in day-to-day practice. 2MWD failed to show good psychometric properties for an outcome measure in IIM. Larger longitudinal studies are needed to further explore the effects of age, muscle damage, and variations within subgroups of IIM on these two tests. To our knowledge, this is the first study looking et al. as an outcome measure for IIM.

Recent times of a global pandemic have brought remote care to the forefront of healthcare delivery [5, 6]. In times as this, patients with disabling diseases like IIM found it particularly challenging to travel to the clinics [3]. IIM contributes to significant debility, often hampering mobility, and limiting travel for patients without ample social support. Therefore, diseases like IIM may particularly benefit from the development of patient-reported outcomes for remote monitoring, even after the global pandemic [20].

Muscle examination is rather complex, with a plethora of clinical tests used to ascertain muscle strength and endurance. Although the MMT-8 is used as the gold standard for objective assessment of IIM, it may be impacted by subjectivity and interobserver variability, necessitating the need for training physicians on this technique. The prime symptom of myositis is muscle weakness, manifesting as difficulty in performing tasks involving proximal muscle. This may be difficult to diagnose and/or monitor remotely, as similar symptoms may result out of fatigue, arthritis, general deconditioning, ILD, PAH, and disuse atrophy, all of which may occur in patients with IIM. Therefore, objective measures of remote self-assessment may empower patients to keep a diary of their performance, while enabling them to identify early changes of relapse or worsened muscle strength. Functional index 2, a validated tool for testing muscle endurance, requires weights and a metronome device to be administered. The metronome device may not be accessible for remote home-based assessment. Thus, there is a need for simple device-independent tests to access muscle strength and endurance that can be assessed remotely.

Early disease recognition and assessment of severity may go a long way in improving outcomes for these patients. Timely treatment is necessary to reduce damage and disability in IIM patients. Although the established IMACS CSMs are used in clinical trials and clinical practice, they are influenced by inter-rater differences, the requirement of training as well as lack of self-administration by patients remotely [21]. Patient-centered assessments like AL allow more regular and frequent measurements by patients at home or simpler more objective measurements during clinic follow-up may mitigate some of these limitations. Additionally, these may serve as important secondary outcomes to supplement the collection of validated CSMs in clinical trials [14].

Previously, simple home-based assessment tests have been found useful to document muscle strength in IIM [22, 23]. These include TUG (timed up and go test), 30-s arm lift, and sit-to-stand. IIM often affects the elderly and those with arthritis, wherein knee osteoarthritis and pain/stiffness are often reported. These changes may potentially confound home-based assessment in this group of the lower extremity, which includes a large proportion of those with IIM. 2MWD probably had the same confounders. AL test obviated these challenges, providing a reasonable PCOM of proximal upper extremity weakness which is simple to understand and use, reliable, and convenient.

Telemedicine care is equally effective in the follow-up of other chronic rheumatic diseases [24–27]. Furthermore, PCOMs can be easily self-administered at home and conveyed via teleconsultation. This paves way for novel simple patient-centered outcome measures to assess various domains of a particular disease [24, 25].

We fully acknowledge the limitations arising from limited sample size and cohort heterogeneity (for disease subsets and disease activity) as well as a milder phenotype in our in-clinic cohort, which call for larger external validation studies with more diverse cohorts. Finally, 2MWD assessments may be marred by turning time, which depends on hall length, an aspect to be explored in future studies. We also attempted to minimize the effects of inter-rater reliability on CSM in this study by having the same experienced physician performed all assessments for the patient. Participants were also familiarized with function tests before their first performance to reduce any effects of familiarity/training. Furthermore, small sample size deterred us from studying the effect of age and muscle damage (by myositis damage index) in the performance of these two tests.

In conclusion, the AL test correlated well with myositis disease activity measures including muscle weakness providing preliminary evidence for its role as PCOM in IIM especially in clinical trials and telemedicine consultation. 2MWD was not a good test for outcome evaluation of IIM patients. Larger longitudinal studies are needed to further validate these findings. Especially, 2MWD needs to be performed supervised in-clinic in a larger cohort before ruling out its usefulness. Novel PCOMs, particularly arm-based assessments, may be used for remote assessment of muscle strength, as a reflection of disease activity, pending validation in larger cohorts of IIM.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contribution

LG, RA, and VA conceptualized the study. LG, DT, and NR collected and analyzed the clinical data. LG and NR prepared the first draft. All authors were involved in writing and reviewing the manuscript for critical intellectual inputs.

Data availability

The data underlying this article will be shared on reasonable request to the corresponding author.

Declarations

Disclosures

None relevant to this work.

Disclaimer

The abstract is accepted at ACR 2021 conference as a poster presentation.