Abstract
Objectives
The aim was to evaluate patient self-assessment of RA disease activity in terms of Routine Assessment of Patient Index Data (RAPID) scores via a Web-based smartphone application (WebApp).
Methods
In this prospective, multicentre study, adult RA patients were examined by a rheumatologist at baseline and after 3 months. Patients were asked to complete WebApp questionnaires weekly. The time course of patient-assessed RAPID3/4 scores and their correlations with rheumatologist-assessed DAS28, as well as Clinical and Simplified Disease Activity Indices (CDAI/SDAI), were evaluated.
Results
Eighty patients were included in the analysis (median RA duration, 4.5 years; age, 57 years; 59% female). At baseline, there was a moderate to strong correlation between RAPID3 and DAS28 (r = 0.63), CDAI (r = 0.65) and SDAI (r = 0.61) scores. Similar or stronger correlations were seen at the 3-month follow-up visit (DAS28 r = 0.66, CDAI r = 0.71 and SDAI r = 0.61). Similar correlations were seen between RAPID4 and rheumatologist assessments. Correlations were not influenced by demographics or RA treatment. In the 3-month period, the RAPID3 score changed into a higher severity category than the category at baseline at least once in 47% of patients. When DAS28 scores were predicted from the RAPID3, 11% of patients had an increase of > 1 DAS28 unit during the 3-month observation period.
Conclusion
Web-based patient assessments were strongly correlated with rheumatologist assessments of RA activity and showed considerable variation during follow-up. This provides a rationale for further exploration of their use as cost-effective tools to monitor RA activity between outpatient visits and to optimize tight control strategies.
Keywords: rheumatoid arthritis, patient’s self-assessment, patient-reported outcome, tight control, RAPID score
Rheumatology key messages
Patients can reliably self-assess their RA activity by means of a WebApp.
Patients’ assessed RA activity fluctuates considerably in between rheumatologist visits.
Introduction
Current RA treatment guidelines recommend the use of clinical measures, for instance, the DAS based on 28 joints (DAS28), in the assessment of disease activity and treatment success to achieve tight disease control [1]. Tight disease control implies that physicians must enable timely adaptation of therapy, which requires intensive disease activity monitoring [2]. In clinical practice, however, patients with RA are usually seen by rheumatologists only every 3–6 months as, although desirable, more frequent visits are often not possible because of limited resources.
Patient involvement in disease management and a shared decision-making approach is not only recommended by the EULAR, but also preferred by most RA patients [3–5]. Innovative solutions are thus needed from the physicians’ and from the patients’ perspective to monitor disease activity in between clinic visits to improve clinical management of patients with RA.
Patient-reported outcomes (PROs) are increasingly recognized for their value in providing the patient’s perspective on their health and quality-of-life status and are hence essential in the judicious care of RA patients [6, 7]; the use of PROs in clinical practice is also recommended by EULAR to complement disease monitoring [4, 5]. With limited resources of health-care professionals and an increasing focus on patient-centred care, continuous monitoring of RA disease activity might be achievable because of regular patients’ self-assessment of disease activity using valid and reliable PROs. With the development of new technology, PRO questionnaires are becoming available as Web-based applications (WebApps) for computers and smartphones [8].
The Routine Assessment of Patient Index Data 3 and 4 (RAPID3/RAPID4) are PRO tools for RA self-assessment by the patient [9]. The RAPID3 is one of the most extensively validated PRO tools, with strong internal consistency and structural validity, and is recommended by the ACR as a disease activity measure for use in clinical practice because of its sensitivity to change and ability to discriminate well between disease activity states [10, 11]. Additionally, the RAPID3 takes only ∼1.5 min to complete [12]. Several studies have demonstrated good correlations between paper-based RAPID-based assessments and clinical measures of RA activity, such as the DAS28 and the Clinical Disease Activity Index (CDAI) [9, 13]. Additionally, RAPID-based assessments have been shown to be cost effective and time efficient in a home-based monitoring or busy clinical setting [9, 13].
The COmPASS study aimed to evaluate the COrrelation between PAtient self-ASSessment of RA activity using RAPID scores via WebApp and physician RA activity assessments using traditional scores. Additionally, this study aimed to evaluate RA activity dynamics between routine clinical visits using RAPID scores.
Methods
Study population and design
This prospective, multicentre study was conducted between November 2012 and March 2014 across five sites in Switzerland after approval from the centres’ independent ethics committees. Adult RA patients were eligible if they received or began treatment with conventional or s.c. DMARDs or both, according to the Swiss label (see Inclusion and exclusion criteria section of supplementary data, available at Rheumatology Online). This study was approved by the following ethical committees: Ethikkommission Nordwest und Zentralschweiz, Kantonale Ethikkommission Bern, Ethikkommission des Kantons St Gallen, Ethikkommission Thurgau and Kantonale Ethikkommission Zürich. Each patient provided written informed consent.
Patients were equipped with smartphones (iPhone 3), were educated how to use the WebApp, and had the chance to familiarize themselves with the application for 2 weeks before the baseline visit. At baseline and the 3-month follow-up visit, the RA activity of patients was assessed by a rheumatologist. At baseline and during the subsequent 3 months, patients were asked to provide weekly self-assessments on the WebApp.
Study measures and outcomes
The following measures were collected by the rheumatologists: DAS28 (based on CRP [14]), the CDAI and the Simplified Disease Activity Index [15]. For more information, see the Collected outcome measures section of supplementary data, available at Rheumatology Online.
Information captured with the WebApp included the RAPID3 and RAPID4 (range 0–30 and 0–40, respectively, with higher scores indicating more active disease), the latter of which also includes the patient’s self-reported joint count [9].
After 3 months, the user friendliness and usability of the WebApp was evaluated by patients using the System Usability Scale (range 0–100; see System usability scale section of supplementary data, available at Rheumatology Online) [16]. The value of the WebApp for patient–physician communication was assessed by patients and physicians using a 100-mm visual analog scale (see Patient–physician communication section of supplementary data, available at Rheumatology Online).
Statistical analysis
Baseline and follow-up scores were compared using Wilcoxon–Mann–Whitney tests. Correlations between the outcome measures were analysed using the Spearman’s correlation coefficient. Multiple linear regression analyses were carried out to assess the potentially confounding effect of variables defined a priori.
The absolute dynamics of the RAPID3 score was calculated as the sum of RAPID3 fluctuations (differences from the baseline RAPID3 score) divided by the number of RAPID3 entries. Likewise, the percentage of the average fluctuation in RAPID3 score was calculated. The DAS28 and CDAI scores between clinical visits were predicted from the RAPID3 scores based on a linear regression model. The number of patients who had an increase in the predicted DAS28 and CDAI of more than the minimally clinically important difference (MCID) at any time was assessed [17]. All data were analysed using Stata 13.1 (StataCorp, College Station, TX, USA).
Results
Patient and disease characteristics
Five Swiss centres screened 92 patients, of whom 80 patients were included in this analysis (supplementary Fig. S1, available at Rheumatology Online). Demographic, socio-economic and baseline disease activity characteristics are provided in Table 1. There were no significant differences between baseline and follow-up scores with regard to all clinical or patient self-assessed disease activity measures (all P > 0.15).
Table 1.
Demographic, socio-economic and clinical characteristics of the study population at baseline and the follow-up visit
| Characteristic (n = 80) | Baseline | 3-month follow-up |
|---|---|---|
| Female, % | 58.8 | |
| Age, median (IQR), years | 57.3 (44.9–65.9) | |
| Education | ||
| Compulsory schooling not completed, % | 5.0 | |
| Compulsory schooling completed, % | 48.8 | |
| Secondary school level, % | 20.0 | |
| University or similar, % | 26.3 | |
| Prior smartphone use, % | 50.0 | |
| Native language | ||
| German, % | 85.0 | |
| Other, % | 15.0 | |
| Currently employed, % | 43.6 | |
| RA duration since diagnosis, median (IQR), years | 4.5 (1.1–9.5) | |
| RA symptom duration, median (IQR), years | 5.1 (1.6–12.2) | |
| DMARD use, % | 82.5 | 92.5 |
| Biologic DMARD use, % | 31.3 | 39.2 |
| DAS28 | ||
| Median (IQR) | 2.4 (1.9–3.5) | 2.4 (1.7–3.7) |
| ≤2.6 (remission), % | 55.8 | 54.8 |
| >2.6 to ≤ 3.2 (low disease activity), % | 15.6 | 12.3 |
| >3.2 to ≤ 5.1 (moderate disease activity), % | 26.0 | 26.0 |
| >5.1 (high disease activity), % | 2.6 | 6.9 |
| CDAI | ||
| Median (IQR) | 7.6 (4.3–16.5) | 5.7 (2.0–14.2) |
| ≤2.8 (remission), % | 16.9 | 33.3 |
| >2.8 to ≤ 10 (low disease activity), % | 44.1 | 35.9 |
| >10 to ≤ 22 (moderate disease activity), % | 27.3 | 15.4 |
| >22 (high disease activity), % | 11.7 | 15.4 |
| SDAI | ||
| Median (IQR) | 10.5 (6.6–20.4) | 8.6 (4.9–22.1) |
| ≤3.3 (remission), % | 10.7 | 11.0 |
| >3.3 to ≤ 11 (low disease activity), % | 44.0 | 47.9 |
| >11 to ≤ 26 (moderate disease activity), % | 28.0 | 23.3 |
| >26 (high disease activity), % | 17.3 | 17.8 |
| RAPID3 | ||
| Median (IQR) | 6.6 (3.1–11.9) | 6.4 (2.6–12.3) |
| ≤3 (near remission), % | 25.0 | 28.7 |
| 3.1 to ≤ 6 (low severity), % | 21.2 | 18.8 |
| 6.1 to ≤ 12 (moderate severity), % | 28.8 | 27.5 |
| >12.1 (high severity), % | 25.0 | 25.0 |
| RAPID4 | ||
| Median (IQR) | 7.8 (3.5–15.1) | 7.6 (3.0–15.8) |
| ≤4 (near remission), % | 30.0 | 32.5 |
| 4 to ≤ 8 (low severity), % | 20.0 | 18.8 |
| 8 to ≤ 16 (moderate severity), % | 30.0 | 23.8 |
| >16 (high severity), % | 20.0 | 25.0 |
| SF-36 PCS, median (IQR) | 43.8 (36.5–52.6) | 46.1 (38.6–53.2) |
| SF-36 MCS, median (IQR) | 50.7 (37.4–58.0) | 50.1 (35.6–58.9) |
| HAQ-DI, median (IQR) | 0.38 (0.13–1.13) | 0.38 (0.13–1.25) |
| FACIT-F, median (IQR) | 38 (15–47) | 38 (26–42) |
CDAI: Clinical Disease Activity Index; Education: compulsory schooling is defined as 9 years of schooling, and secondary school level includes vocational training; FACIT-F: Functional Assessment of Chronic Illness Therapy—Fatigue; HAQ-DI: HAQ Disability Index; IQR: interquartile range; RAPID: Routine Assessment of Patient Index Data; SDAI: Simplified Disease Activity Index; SF-36 MCS: Short Form-36 Mental Component Summary; SF-36 PCS: Short Form-36 Physical Component Summary.
Correlations between RAPID3/4 and clinical scores
The correlation between the RAPID3 and DAS28 was strong at baseline and at the 3-month follow-up (r = 0.63 and r = 0.66, respectively; Table 2). The same was true for the correlations between RAPID3 and the other physician assessments of RA activity (Table 2).
Table 2.
Correlation coefficients of Routine Assessment of Patient Index Data 3/4 vs clinical scores and patient-reported outcome measures
| Scores | RAPID3 (Spearman’s coefficient, 95% CI) | RAPID4 (Spearman’s coefficient, 95% CI) | ||
|---|---|---|---|---|
| Baseline | 3-month follow-up | Baseline | 3-month follow-up | |
| DAS28 | 0.63 (0.47, 0.75) | 0.66 (0.50, 0.77) | 0.64 (0.48, 0.75) | 0.67 (0.52, 0.78) |
| CDAI | 0.65 (0.50, 0.76) | 0.71 (0.57, 0.81) | 0.66 (0.52, 0.77) | 0.72 (0.59, 0.81) |
| SDAI | 0.61 (0.45, 0.74) | 0.61 (0.43, 0.74) | 0.61 (0.45, 0.74) | 0.61 (0.44, 0.74) |
| SF-36 PCS | −0.75 (−0.84, 0.64) | −0.78 (−0.85, −0.67) | −0.76 (−0.84, −0.65) | −0.80 (−0.87, −0.70) |
| SF-36 MCS | −0.51 (−0.66, −0.32) | −0.70 (−0.80, −0.56) | −0.52 (−0.66, −0.33) | −0.70 (−0.80, −0.56) |
| HAQ-DI | 0.74 (0.62, 0.83) | 0.82 (0.72, 0.88) | 0.77 (0.66, 0.85) | 0.83 (0.74, 0.89) |
| FACIT-F | −0.70 (−0.80, −0.57) | −0.72 (−0.81, −0.59) | −0.71 (−0.80, −0.57) | −0.73 (−0.82, −0.60) |
CDAI: Clinical Disease Activity Index; FACIT-F: Functional Assessment of Chronic Illness Therapy—Fatigue; HAQ-DI; HAQ Disability Index; RAPID: Routine Assessment of Patient Index Data; SDAI: Simplified Disease Activity Index; SF-36 MCS: Short Form-36 Mental Component Summary; SF-36 PCS: Short Form-36 Physical Component Summary.
Notably, the correlation between the RAPID4 and DAS28 was similar to that of RAPID3 and DAS28, both at baseline (r = 0.63 vs 0.64) and at the 3-month follow-up visit (r = 0.66 vs 0.67; Table 2). The same was true for the correlations between the RAPID4 and the other physician-reported assessments (Table 2).
At baseline, the relationship between the clinical scores and the RAPID3/4 scores was not confounded by sex, age, native language, current DMARD use or prior smartphone experience (supplementary Table S1, available at Rheumatology Online).
RA activity at baseline and follow-up
On average, there were 90 days (s.d. 11) between the baseline and the follow-up visits; the average interval between WebApp entries was 10 days (s.d. 5.3).
When comparing DAS28 activity strata at baseline and after 3 months (Table 1), the RA activity did not change as measured by DAS28 activity strata in 59% of individuals. At the follow-up visit, 16% of patients were in a higher DAS28 activity category than at baseline, whereas 25% of patients were categorized into a lower DAS28 activity category.
RA activity dynamics: RAPID3
During the 3-month observation period, the RAPID3 severity category was higher than the baseline RAPID3 category at least once in 47% of the patients. Of these patients, 61% changed into a higher category once, 25% twice, 11% three times and 3% four times (supplementary Table S2, available at Rheumatology Online). In between these changes, the patients either remained in the same RAPID3 category or fell into a lower severity category. Among the patients who were in DAS28 remission at baseline, 53% changed into a higher RAPID3 category at least once during the follow-up compared with 33% of patients with low RA disease activity and 35% of patients with moderate RA activity.
Details on the variation of the RAPID3 scores over the study period are provided in supplementary Fig. S2, available at Rheumatology Online. The greatest absolute fluctuations of the RAPID3 score were observed in those patients who were already in the high RAPID3 category at baseline (supplementary Fig. S2D, available at Rheumatology Online). In these patients, the median RAPID3 fluctuation from the baseline RAPID3 score was 3.7 units [interquartile range (IQR): 1.9–5.5], compared with 2.3 units (IQR: 1.7–4.1) in patients with moderate RAPID3 disease severity, 2.0 units (IQR: 1.4–3.4) in patients with low RAPID3 disease severity and 0.9 units (IQR: 0.4–1.4) in patients in RAPID3 remission.
RA activity dynamics: predicted DAS28
For further analysis of the relevance of observed changes in RAPID3 scores during the 3-month observation period, we predicted the DAS28 scores in between the baseline and follow-up visits based on the RAPID3 during the observation period. Using this prediction, 11.3% of patients would have increased by >1 DAS28 unit during the observation period, which is regarded the DAS28 MCID [17]. This increase was first observed at a median of 47 days (IQR: 21–81) after baseline. The highest percentage of patients with an increase in the predicted DAS28 higher than the MCID was observed in patients with a low DAS28 disease activity at baseline (16.7%) compared with 11.6% in patients in remission, 10.0% in patients with moderate disease activity and 0% in patients with high disease activity at baseline. For results on the predicted CDAI, see RA activity dynamics over time—predicted CDAI section of supplementary data, available at Rheumatology Online.
WebApp evaluation
The System Usability Scale indicated a median score of 85 (IQR: 73–93) at the follow-up visit, suggesting that patients rated the WebApp as very positive in terms of ease of use; a score of 68 is considered average [16]. Patients and physicians rated the benefit of the WebApp on the patient–physician communication with a median score of 74 (IQR: 43–93) and 50 (IQR 18–60), respectively, on the 100-mm visual analog scale.
Discussion
In this study, patient self-assessment of RA disease activity via smartphone technology was demonstrated to be correlated strongly with clinical outcome parameters recorded by physicians. Although RAPID scores have already been shown to be correlated reproducibly with clinical outcomes in clinical practice and research [9, 13], our study now extends the data derived from paper-based data recordings to the use of WebApps. This offers benefits in terms of automated scoring, continuous data capture outside the physician’s office and partial outsourcing of RA monitoring to the patient, saving time for the healthcare professional. This WebApp-based approach has demonstrated feasibility, as there was a high degree of engagement with the technology in terms of regular data entry by patients. Furthermore, patients assessed the tool as easy to use and saw value in the WebApp as a positive support tool to physician–patient communication. Owing to the ease of use and intuitive nature of the WebApp, elderly patients and patients with no prior smartphone experience were also able to use the WebApp; hence, this WebApp-based approach seems applicable to a wide range of patients.
One of the biggest challenges in the management of RA is monitoring disease effectively, in order to avoid missing flare-ups. EULAR guidelines suggest that disease activity should be monitored every 1–3 months in active RA, because tight control is more effective in controlling disease and preventing radiographic progression [4]. During the 3-month period of this study, ∼1 in 10 patients reported an increase in RAPID3 scores that would translate into >1 DAS28 unit, which is regarded as the MCID. These data suggest that clinically relevant flares may be missed when physicians are seeing patients at a 3-month interval, even in supposedly well-controlled patients with low disease activity or in remission. The population of patients deemed to have quiescent RA could potentially benefit from the WebApp by enabling treatment adjustment. The median time to flare was as short as 47 days, suggesting that WebApp-based disease monitoring could be used to enable earlier interventions if RAPID flares triggered a timely rheumatologist visit. It is, however, unclear at present whether an earlier response to patient-reported flares would indeed offer better control of RA activity; further research is needed to quantify this.
Although our exploratory study has limited statistical power, implying that negative results with respect to potentially confounding factors must be interpreted cautiously, the results of this study support the feasibility of the continuous patient self-monitoring of RA activity using a WebApp-based approach in order to capture disease flares, even in patients deemed quiescent. WebApp-based RAPID3 data capture may evolve into a means to collect detailed information about the fluctuations of RA activity in between outpatient visits and may in the future optimize RA management.
To further research about continuous self-monitoring, the randomized COmPASS-II study was initiated in order to evaluate the effect of patients’ continuous disease activity self-assessment by WebApp on RA outcome in a routine clinical setting [18].
Supplementary Material
Acknowledgements
We would like to acknowledge Dr D. Aeberli from the University Hospital of Bern and B. Flückiger, MD Consultant Rheumatologist in Zurich for participating in this study. Contributorship: Study conception and design: P.D., D.K. Acquisition of data: U.A.W., R.B.M., R.T., A.F., D.K. Analysis and interpretation of data: U.A.W., R.B.M., V.K.J., P.D., F.G., D.K. Manuscript drafting: U.A.W., V.K.J., F.G. All authors were involved in revising the article critically for important intellectual content, and all authors approved the final version to be published.
Funding: This work was supported by AbbVie AG, Switzerland. The design and study conduct were provided by AbbVie. AbbVie participated in the interpretation of data, review and approval of the manuscript.
Disclosure statement: U.A.W. has served as a consultant and speaker for AbbVie and other companies that produce RA medication and has received an unrestricted grant support from Abbvie for a follow-up study of the submitted COmPASS study. F.G. is an employee and shareholder of Abbvie. D.K. has received speaker and consultancy fees and unrestricted research grants by AbbVie, Pfizer, Roche, Bristol-Myers Squibb (BMS), Union Chimique Belge (UCB), Merck Sharp & Dohme (MSD). P.D. is an employee of Abbvie. R.B.M. has received consultancy fees from AbbVie, UCB, Roche, BMS, Antares and Pfizer and research funding from Roche, BMS and UCB. A.F. has served as a consultant and speaker for AbbVie, UCB, Pfizer, BMS, Roche and Merck. All other authors have declared no conflicts of interest.
Supplementary data
Supplementary data are available at Rheumatology Online.
References
- 1. Smolen JS, Aletaha D, Bijlsma JWJ. et al. Treating rheumatoid arthritis to target: recommendations of an international task force. Ann Rheum Dis 2010;69:631–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Grigor C, Capell H, Stirling A. et al. Effect of a treatment strategy of tight control for rheumatoid arthritis (the TICORA study): a single-blind randomised controlled trial. Lancet 2004;364:263–9. [DOI] [PubMed] [Google Scholar]
- 3. Nota I, Drossaert CH, Taal E, Vonkeman HE, van de Laar MA.. Patient participation in decisions about disease modifying anti-rheumatic drugs: a cross-sectional survey. BMC Musculoskelet Disord 2014;15:333. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Combe B, Landewe R, Daien CI. et al. 2016 update of the EULAR recommendations for the management of early arthritis. Ann Rheum Dis 2017;76:948–59. [DOI] [PubMed] [Google Scholar]
- 5. Smolen JS, Landewé R, Bijlsma J. et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2016 update. Ann Rheum Dis 2017;76:960–77. [DOI] [PubMed] [Google Scholar]
- 6. Pincus T, Wolfe F.. Patient questionnaires for clinical research and improved standard patient care: is it better to have 80% of the information in 100% of patients or 100% of the information in 5% of patients? J Rheumatol 2005;32:575–7. [PubMed] [Google Scholar]
- 7. Kojima M, Kojima T, Suzuki S. et al. Patient-reported outcomes as assessment tools and predictors of long-term prognosis: a 7-year follow-up study of patients with rheumatoid arthritis. Int J Rheum Dis 2015; doi:10.1111/1756-185X.12789. [DOI] [PubMed] [Google Scholar]
- 8. Grainger R, Townsley H, White B, Langlotz T, Taylor WJ.. Apps for people with rheumatoid arthritis to monitor their disease activity: a review of apps for best practice and quality. JMIR mHealth uHealth 2017;5:e7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Pincus T, Swearingen CJ, Bergman M, Yazici Y.. RAPID3 (Routine Assessment of Patient Index Data 3), a rheumatoid arthritis index without formal joint counts for routine care: proposed severity categories compared to disease activity score and clinical disease activity index categories. J Rheumatol 2008;35:2136. [DOI] [PubMed] [Google Scholar]
- 10. Hendrikx J, de Jonge MJ, Fransen J, Kievit W, van Riel PL.. Systematic review of patient-reported outcome measures (PROMs) for assessing disease activity in rheumatoid arthritis. RMD Open 2016;2:e000202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Anderson J, Caplan L, Yazdany J. et al. Rheumatoid arthritis disease activity measures: American College of Rheumatology recommendations for use in clinical practice. Arthritis Care Res 2012;64:640–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Anderson JK, Zimmerman L, Caplan L, Michaud K.. Measures of rheumatoid arthritis disease activity. Arthritis Care Res 2011;63:S14–36. [DOI] [PubMed] [Google Scholar]
- 13. Sullivan MB, Iannaccone C, Cui J. et al. Evaluation of selected rheumatoid arthritis activity scores for office-based assessment. J Rheumatol 2010;37:2466–8. [DOI] [PubMed] [Google Scholar]
- 14. van Gestel AM, Haagsma CJ, van Riel PL.. Validation of rheumatoid arthritis improvement criteria that include simplified joint counts. Arthritis Rheum 1998;41:1845–50. [DOI] [PubMed] [Google Scholar]
- 15. Aletaha D, Smolen JS.. The Simplified Disease Activity Index (SDAI) and Clinical Disease Activity Index (CDAI) to monitor patients in standard clinical care. Best Pract Res Clin Rheumatol 2007;21:663–75. [DOI] [PubMed] [Google Scholar]
- 16. Sauro J. Measuring Usability with the System Usability Scale (SUS). 2011. http://www.measuringu.com/sus.php. (2 October 2015, date last accessed).
- 17. Ward MM, Guthrie LC, Alba MI.. Clinically important changes in individual and composite measures of rheumatoid arthritis activity: thresholds applicable in clinical trials. Ann Rheum Dis 2015;74:1691–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Walker UA. COmPASS II - Selbstbeurteilung der Krankheitsaktivität von Rheuma per WebApp. 2016. www.compass2.ch (16 June 2016, date last accessed).
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.