Characterizing Adherence to a Mobile Health App for Patient-Reported Outcomes in Rheumatoid Arthritis

Abstract

Introduction

Adoption of mobile health applications (“apps”) that collect patient-reported outcomes (PROs) into clinical practice remains limited, and studies about patient usage preferences are lacking. Our aim was to characterize adherence to a PRO app designed for patients with rheumatoid arthritis (RA).

Methods

We performed a post hoc analysis of a non-randomized trial of a PRO app for patients with RA seen at an academic medical center. The 12-month trial enrolled 149 participants with RA. The app collected one of four PROs every other day. We calculated adherence using two methods: (1) percent of PROs completed, and (2) time until app non-use. These calculations used different intervals for PRO completion and different periods of non-use to define adherence. We also investigated the association of potential predictors on the outcome using multivariable regression models.

Results

The percentage of completed PROs was 43.5% (standard deviation, SD = 27.8) using a 2-day interval between PROs; this included all possible PRO questionnaires. Adherence was 68.0% (SD = 34.0) for any completion in an 8-day interval, and 77.3% (SD = 30.9) for a 32-day interval. In survival analyses, describing time until app non-use, the mean number of days until cessation was 153.8 days (SD = 94.1), defining non-use as ≥ 32 days without PRO completion and 152.3 days (SD = 95.8) for ≥ 64 days. We observed that older age and more frequent physician engagement with the PROs data were associated with greater adherence and increased time until cessation.

Conclusions

Adherence to longitudinal PRO symptom reporting through an app was moderate and varied depending on how it was measured; the 8-day interval (approximately 1 week) is likely the most realistic expectation for patient completion of PRO symptoms. Adherence correlates with patient and physician factors.

Key Summary Points

Why carry out this research?

This paper describes various methods for the assessment of adherence with longitudinal symptom reporting using a mobile health application (“app”), a digital health technology becoming more popular among patients.

What was learned from the study?

We found that different methods—interval-based completion rates and survival analysis for time-to-cessation—provided different depictions of adherence.

Regression analyses highlighted provider engagement with app data and advanced patient age as significant predictors of patient adherence, highlighting the bilateral value of digital technology between patient and provider.

Longitudinal symptom reporting through a PRO app likely should occur every 1–4 weeks to achieve optimal adherence.

Introduction

Digital health technologies, such as smartphone applications (“apps”) are a rapidly growing health care technology [1]. Their integration into clinical practice offers significant potential to improve healthcare by engaging patients more actively in the management of chronic diseases [2–4]. One of the most commonly adopted technologies is the smartphone app [5]. Approximately 91% of the US population owns a smartphone [6], and widespread use of a smartphone facilitates accessibility to mobile health apps [1]. Patient-reported outcomes (PROs) collected via smartphones or tablets can be used for symptom tracking, helping patients to recognize disease patterns and potential disease triggers. Routine longitudinal collection of PROs may reduce clinical visit frequency when the symptom burden is stable and low [7, 8]. They may also guide decision-making during medical appointments [9]. Certain apps provide a bidirectional communication channel between patients and healthcare providers, giving patients a method to send symptom information to their providers. Despite these advantages, their integration into routine care remains limited [3], and their long-term use has not been widely studied [3, 10].

Studies of apps across various chronic diseases have been pursued, especially for patients with rheumatoid arthritis (RA) [2, 11–19]. RA is an example of a chronic condition marked by fluctuating symptoms [16] and periodic flares. Patients typically visit their rheumatologist 3–5 times a year [9], and patients often manage their own symptoms between visits. Apps can bridge this gap by supporting remote symptom tracking and facilitating communication with healthcare providers [20]. Therefore, PROs have been widely advocated as tools to navigate the disease course in RA. Recommendations have been proposed to generalize the adoption of digital health technologies into rheumatology [21, 22]. Studies have outlined critical implementation challenges [3, 16, 23–26], with patient adherence a primary concern [5, 27–31]. Prior studies have reported a gradual decline over months in long-term adherence to reporting PROs via mobile health apps [20, 21]. App adherence in prior studies has typically been described by usage frequency and/or duration until discontinuation [18]. Adherence with technologies, such as an app that collects longitudinal PROs, is more nuanced. For example, how often should the technology be used? Is once a week or once a month enough? How should brief interruptions be considered when estimating adherence? Is app adherence related to RA disease activity or patient outcomes?

Prior studies have explored factors influencing app adherence [32] and have demonstrated conflicting results [33]. For example, older age and lower baseline RA disease activity (CDAI < 10) were predictors of higher adherence in several studies [5, 16], and other studies concluded that women had a significantly higher likelihood of dropping out earlier from use of apps that monitor PROs [16, 20]. Female sex appeared to facilitate engagement in another study. Adherence appears to decrease over time, with one study observing decreased adherence from 88 to 62% over 6 months [23]. Dutch investigators reported declining adherence to PRO completion using an app from rates > 90% in the first week to less than 50% in week four [24]. Another factor that can be relevant to adherence is the clinician’s interest in the data generated by the app. One study showed that patients expected their doctors to assess the data provided [34].

In a recent trial, we developed and implemented a PRO app platform [35]. Here, we examine adherence to the PRO app, explore various adherence measures, and assess patient and provider variables that may be associated with these measures.

Methods

Study Design and Population

This study is a post hoc analysis of a non-randomized trial of a mobile health app for patients with RA [35]. The original trial of the app has been described [12, 13]. The trial enrolled 149 patients with RA who were seen by rheumatologists at one academic medical center. All patients had a clinical diagnosis of RA and were cared for by any of 11 rheumatologists whose primary role was clinical rheumatology (versus administration or research). Patients gave written informed consent to use the app developed by the senior investigator (DHS). Some patients consented but never downloaded the app and were not included in these analyses. Those who downloaded the app and answered the first PRO questionnaire received a $25 gift card. The development of the app has been described previously [18, 35] and collects one of four PROs from patients every other day: PROMIS fatigue short form, PROMIS function short form, PROMIS pain interference short form, and RA Disease Activity Index (RADAI)‐5 [36, 37]. The app is integrated within the electronic health record, allowing clinicians to view PRO data between visits or at the time of a clinical visit. In this set of analyses, we examined patient adherence with the app over the 12-month study period.

Human Ethics

The study was reviewed and approved by the Partners Healthcare Human Subjects Committee (#2021-P000790). All patients provided written informed consent to participate in research with the express understanding that the results would be published. The study was conducted in accordance with the Helsinki Declaration of 1964 and its later amendments.

Smartphone Application

The app included four different PRO measures; each one available for 48 h during an 8-day cycle. Thus, during each 8-day cycle, patients were given the opportunity to complete all four PROs. After consenting to be in the trial, patients were sent a unique code that allowed them to download the app. At 8:00 am and 8:00 pm, the app would send a push notification to remind patients of available PROs to complete. Within the app, participants could view a line graph summarizing their responses to each survey over time. Participants could also look up answers to frequently asked questions about RA: general information, diet and lifestyle, medications, etc. The time to complete each PRO was brief, approximately 1 min or less.

Data from the app were viewable by clinicians in the electronic health record (EHR). The data were summarized in separate graphical displays for each PRO. In addition, clinicians received three different types of EHR messages regarding the PRO data. First, 48 h prior to the next rheumatology visits, if a patient had viewable PRO data, the clinician received a message reminding them to review the data. Second, if patients had PROs that were substantially worsening over consecutive weeks, the clinician received a message that suggested considering moving up the visit date. Finally, 2 weeks prior to the next scheduled visit, if all PRO data were stable, the clinician received a message suggesting considering postponing the visit. Trial results describing visit frequency and medication use were previously reported [12]. The design of the app and its implementation were based on focus groups with relevant patients and providers [12].

Outcome: App Adherence

We investigated adherence using different definitions; these were somewhat arbitrary and, to some extent, based on the frequency of PRO questionnaires. However, they were based on approximately weekly and monthly reporting, two intervals commonly used in prior literature.

First, we examined the percent PRO completion over different intervals. The original interval was the total percent of completed PRO’s out of all available PRO’s asked every 2 days. Since the four PROs were available every other day, we considered 8 days to be one cycle of PROs. Thus, we defined whether any PRO was completed in each 8-day period (one PRO cycle) for the study year, in each 16-day period (two PRO cycles), and in each 32-day period (four PRO cycles).

Second, we estimated the time until a patient stopped completing the PROs (“cessation”); this measure approximates a survival analysis. We defined three different time periods without a PRO completed: at least 32 days (four cycles), at least 64 days (eight cycles), and at least 96 days (12 cycles). The day of cessation was defined as the first day of the 32-day (or 64-day or 96-day) period without completing a PRO. We decided to use the term “cessation” over “dropout”, as our calculation was not based on participants permanently discontinuing app use, but rather on prolonged periods of non-use.

Covariates

We identified several baseline variables that might be associated with adherence, consisting of patient-related and clinical factors. Patient-related factors included age, gender, and work status (employed, unemployed, disabled, retired). Clinical factors were baseline disease activity, measured as the average of the first two RADAI-5 scores [19, 31, 36]. In analyses that considered disease activity, the first two RADAI-5 scores were removed from the adherence outcome. Additionally, disease duration was categorized (early ≤ 2 years, established > 2 years) and RA treatments were characterized as baseline RA treatment (0: none, 1: conventional disease-modifying antirheumatic drugs (DMARD) only, 2: biologic or targeted treatment only, 3: both), pain medication (0: nothing, 1: nonsteroidal anti-inflammatory drugs (NSAID) only, 2: opioids only, 3: both) and glucocorticoids (yes/no). We also considered the rheumatologist’s level of engagement with the app data as a potential predictor. This was characterized by assessing the total number of times each rheumatologist accessed, or viewed, the PRO data for a given participant [33]. There were no missing data requiring imputation.

Statistical Analyses

We first characterized the study population at the start of app use. Means and standard deviations (SDs) were calculated for continuous variables and percentages for categorical variables. Then, the different measures of adherence—completion and time until cessation—were estimated. For the completion measure, we estimated the mean completion across all time points using each definition (every 2 days, every 8 days, every 16 days, and every 32 days). We plotted the completion rates over time using each definition and estimated the total completion for each of the four PROs separately. We used the 8-day definition of completion as out outcome in multivariable linear regression models with all baseline covariates in one model.

For time until cessation analysis, we calculated the mean time to cessation using the 32-, 64-, and 96-day definitions and plotted Kaplan–Meier curves. For modeling, we used the 32-day definition as our outcome in multivariable Cox proportional hazards regression with all baseline covariates in one model. No variable selection was attempted, as all variables were hypothesized to have a relationship with the outcome. Model fit was assessed with the c-statistic. The R statistical program was used for analyses.

Results

We recruited a total of 149 patients with RA into the study, who received a notification to download the app, and downloaded the app. The mean age of participants was 59.5 years (SD = 13.8), 124 (83%) were female, and 135 (91%) self-identified as White (Table 1). Regarding medication use, 62% were taking conventional synthetic (cs) DMARDs, 60% were on biologic/targeted synthetic (b/ts) DMARDs, 51 (34%) were using both csDMARD and b/tsDMARDs, 36% were using glucocorticoids, 17% opioids, and 20% NSAIDs. In terms of employment status, 53% were employed, 27% were retired, and 20% were either unemployed or had unknown employment status.

Table 1.

Baseline characteristics of the 149 patients with rheumatoid arthritis receiving the mobile health application

Characteristic
Age, mean (SD), years	59.5 (13.8)
Female sex, n (%)	124 (83%)
Race, n (%)
White	135 (91%)
Black	5 (3.4%)
Other	9 (6.0%)
Seropositive, n (%)	74 (50%)
High-sensitivity C-reactive protein, mean (SD), mg/l	4.2 (8.9)
Disease duration, n (%)
Established	138 (93%)
Early (< 2 years)	11 (7.4%)
Conventional synthetic (cs) DMARD prescription, n (%)	92 (62%)
Biologic or targeted (bt) DMARD prescription, n (%)	89 (60%)
Oral corticosteroid prescription, n (%)	54 (36%)
NSAID prescription, n (%)	30 (20%)
Opioid analgesic prescription, n (%)	25 (17%)
Employment status, n (%)
Employed	79 (53%)
Not employed/unknown	30 (20%)
Retired	40 (27%)

DMARD disease-modifying antirheumatic drug, NSAID non-steroidal anti-inflammatory drug

Over the course of the 12-month study, participants’ mean completion rate of all PROs was 43.5% (SD = 27.8). Completion rates did not differ among the four different PRO measures. The mean PRO completion rates for different cycles were as follows: 67.9% (SD = 34.0) for 8-day cycles (any PRO answered in each 8-day period over the 12-month study); 72.4% (SD = 32.5) for 16-day cycles; and 77.3% (SD = 30.9) for 32-day cycles (Fig. 1).

Fig. 1.

Percent adherence based on different definitions of the cycle length for the patient-reported outcome questionnaires

We calculated the percentage of study participants who stopped completing PROs (“cessation”) using different definitions. Using the ≥ 32-day non-use criteria, 81 participants (54.4%) were classified as having ceased PRO completion at some point, with a mean number of days to cessation of 153.8 (SD = 94.1) (Fig. 2). Using the ≥ 64-day non-use criteria, 66 participants (44.3%) met the cessation criteria, with a mean time to cessation of 152.3 days (SD 95.8). Using the ≥ 96-days non-use criteria, 53 participants (35.6%) were classified as having ceased PRO completion, with a mean number of days to cessation of 134.1 (SD 84.3).

Fig. 2.

The probability of remaining adherent based on the definition of non-response to the patient-reported outcome questionnaires

Adjusted models revealed a few significant factors related to greater PRO adherence. For the percent completion analysis (Table 2, Model 1), for each time the physician accessed the patient data, the completion rate increased by 6.2% (95% CI 1.1–11). Other variables were not significant. In the time to cessation analysis, each 1-year increase in the patient age was associated with a decreased hazard of app cessation (HR 0.97, 95% CI 0.95–0.99) (Table 2). Additionally, a greater number of views of the PRO data by rheumatologists (used as a representation of physician interaction with app-generated data) was associated with a decreased hazard of app cessation (HR 0.66, 95% CI 0.49–0.88). Other variables were not significant. The model fit was relatively weak with a c-statistic of 0.62.

Table 2.

Multivariable regression model for patients’ 8-day cycle adherence and time until 32 days of non-use

Covariates	Multivariable model for 8-day cycle adherence			Multivariable model for time until 32 days of non-use
	β	[95% CI]	p value	β	[95% CI]	p value
Age, per year	0.43	– 0.06, 0.92	0.083	0.97	0.95, 0.99	0.009
Sex
Female	1.0			1.0
Male	– 1.7	– 17, 13	0.8	1.04	0.53, 2.02	> 0.9
Race
White	1.0			1.0
Black	– 16	– 45, 14	0.3	1.04	0.28, 3.90	> 0.9
Other	– 8.9	– 32, 14	0.4	0.94	0.35, 2.55	> 0.9
Disease duration
Established	1.0			1.0
Early	11	– 9.9, 32	0.3	0.73	0.29, 1.83	0.5
csDMARD prescription	5.6	– 5.6, 17	0.3	0.84	0.53, 1.33	0.5
btDMARD prescription	– 6.3	– 18, 5.6	0.3	1.19	0.68, 2.06	0.5
Oral corticosteroid prescription	– 2.4	– 15, 9.7	0.7	0.93	0.55, 1.56	0.8
NSAID prescription	– 13	– 27, 1.1	0.071	1.52	0.81, 2.84	0.2
Opioid analgesic prescription	13	– 2.8, 28	0.11	0.52	0.24, 1.10	0.087
Employment status
Employed	Ref			Ref
Not employed/unknown	6.1	– 7.7, 20	0.4	0.72	0.40, 1.32	0.3
Retired	7.5	– 8.0, 23	0.3	0.63	0.30, 1.31	0.2
Number of times physician accessed the data	6.2	1.1, 11	0.019	0.66	0.49, 0.88	0.005
Baseline disease activitya	– 1.7	– 5.6, 2.3	0.4	1.18	0.99,1.40	0.064

csDMARD conventional synthetic DMARD, btDMARD biologic or targeted DMARD, DMARD disease-modifying antirheumatic drug, NSAID non-steroidal anti-inflammatory drug

β unstandardized regression coefficient, denoting the change in adherence for a given value of the covariate

Discussion

The aim of this study was to assess adherence patterns across multiple definitions, including interval-based completion rates and time until cessation analyses, and analyze demographic and clinical factors associated with higher adherence to a mobile health app designed to collect PROs in patients with RA. Although adherence has usually been cited as a barrier in the rapidly emerging field of mobile health technologies, we realized that it remains unstandardized in its definition [30]. To address this gap, we incorporated adherence definitions previously described in the literature and introduced alternative measures of app adherence. This study revealed a range of adherence values based on various definitions. Additionally, we identified significant correlations between greater app adherence and longer time until cessation of the app with increasing patient age as well as clinician engagement with the app data.

Our 12-month study is one of the more intensive digital monitoring efforts in rheumatology. A similar study also extended up to 12 months [22], but most have been shorter in duration: a few lasted 6 months or less [13, 18, 38] and others were limited to short-term interventions of 3 months or less [17, 39–42]. These studies also varied in how often PROs were collected, ranging from daily [17, 18, 43] to weekly [17, 38, 42, 44]. Among the studies that addressed adherence, most reported completion rates, while others calculated dropouts. Some studies chose to focus on the concept of feasibility, which led to inconsistencies in definitions and created challenges in making direct comparisons across studies [42, 44].

One prior study defined weekly adherence as completing weekly PRO’s for at least 75% of weeks and observed an adherence rate of 87% [17]. We looked at adherence for 100% of weeks and our 8-day cycle adherence rate was 68%. The subtle differences in definitions (75% vs. 100% of weeks) might explain the discrepancies observed in the findings. In another study that lasted 6 months and collected daily PROs, the adherence rate was reported as 79.3% [18]. In contrast, in our study, the adherence rate over 12 months for completing PROs every 2 days was only 43.5%. Since adherence rates decline with time [18, 40, 44], the longer study period may have correlated with the reduced aggregate completion rate. One study defined dropout as seven days of non-use and reported a cumulative dropout rate of 47.5% for daily and 24.6% for weekly PROs across 3 months [17]. In the current study, the shortest definition of cessation was 32 days of non-use and the rate was 45.6%.

One study reported that being over the age of 65 and having lower baseline clinical disease activity (CDAI) were correlated with higher app adherence [18]. Another study demonstrated that being between the ages of 55 and 73, longer duration of app usage, and higher socioeconomic status were associated with greater adherence [19]. In the current study, older age and higher physician interaction with app-generated data were significantly associated with higher adherence. We found no association with other potential predictors. In a previous study, patients surveyed about their experience with a PRO app indicated that greater physician engagement with app-generated data would improve their adherence [45]. Our study results are in line with this prior patient survey.

The current study offers new insights into how mobile health apps function in real-world, longitudinal settings for patients with RA. The current results demonstrate that frequent PRO questionnaires correlated with survey fatigue and reduced long-term adherence. Another factor that should be taken into consideration is the nature of the disease. Given that RA is a chronic inflammatory disease with episodic flares, relatively frequent PRO input (e.g., weekly) might be favorable. On the other hand, less frequent PRO input may be suitable for a slowly progressive disease.

The association between older age and higher adherence and time using an app may provide insight into which patient populations are best suited for mobile health apps. It is possible that older patients derive greater benefits from such tools. Additionally, our finding that higher clinician interaction with app data is associated with better adherence and time using the app emphasizes the dual role of both patient and provider engagement in sustaining long-term use.

The strength of this study is the long follow-up period and frequent PRO collection, which provided detailed adherence dynamics over time. Inclusion of patient and provider-related variables enabled us to explore a range of potential predictors of adherence. However, this study has several limitations. As a post hoc analysis, it was not designed to test adherence as the outcome. Employment status was self-reported, baseline disease activity was calculated as the mean of the first two RADAI surveys from the app, and the study was single-center. We also had no information on digital literacy. Finally, the lack of a standardized definition of adherence in the field limited comparability with other studies.

Conclusions

In conclusion, this paper summarizes how patients with RA engage with a mobile health app designed to collect PROs. We found that reducing the frequency of app use would have likely improved app adherence. For example, assessing the use of the app every 8 days instead of every 2 days was associated with increased overall adherence. We also described the time course of longitudinal app use. These results demonstrate that digital symptom monitoring is practical for chronic diseases, and that monitoring every 1–2 weeks may be enough. More attention needs to be paid to standardizing definitions of adherence; this will facilitate smarter app design.

Author Contribution

Zeynep S. Tuzun: study design, data collection, manuscript drafting, interpretation of results. Leah Santacroce: data analysis, interpretation of results. Hilde S. Ørbo, Sara K. Tedeschi, Sho Fukui: interpretation of results, critical review. Daniel H. Solomon: study design, data collection, supervision, manuscript revision.

Funding

This work was supported by NIH-P30-AR072577 (National Institute of Arthritis, Musculoskeletal and Skin Research; Bethesda MD, USA). The journal’s Rapid Service Fee was funded by the authors.

Data Availability

The datasets generated and/or analyzed during the current study are not publicly available due to institutional restrictions but are available from the corresponding author on reasonable request.

Declarations

Conflict of Interest

Sara K. Tedeschi reports consulting fees from Amgen, Avalo Therapeutics, Fresenius Kabi, Merck, and Novartis, as well as research grant support from Alexion. Daniel H Solomon receives salary support from research contracts to Brigham and Women’s Hospital from Amgen, CorEvitas, and Janssen. He receives royalties from UpToDate on chapters unrelated to this paper. He also holds stock options in GreenCape Health, a digital health company. Zeynep S Tuzun, Leah Santacroce, Hilde S Ørbo, and Sho Fukui declare that they have no conflicts of interest.

Ethical Approval

The study was reviewed and approved by the Partners Healthcare Human Subjects Committee (#2021-P000790). All patients gave written informed consent to participate in research with the express understanding that results would be published. The study was conducted in accordance with the Helsinki Declaration of 1964 and its later amendments.