Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 May 1.
Published in final edited form as: Circ Cardiovasc Qual Outcomes. 2019 May;12(5):e005509. doi: 10.1161/CIRCOUTCOMES.119.005509

Corrie Health Digital Platform for Self-Management in Secondary Prevention After Acute Myocardial Infarction: Myocardial infarction, COmbined-device Recovery Enhancement (MiCORE) Rationale and Design

Erin M Spaulding 1,*, Francoise A Marvel 2, Matthias A Lee 3, William E Yang 4, Ryan Demo 3, Jane Wang 4, Helen Xun 4, Lochan Shah 4, Daniel Weng 4, Oluwaseun E Fashanu 2, Jocelyn Carter 5, Julie Sheidy 6, Renee McLin 6, Jennifer Flowers 6, Maulik Majmudar 7, Eric Elgin 6, Valerie Vilarino 8, David Lumelsky 8, Vinayak Bhardwaj 9, William Padula 9, Jerilyn K Allen 1,4,9, Seth S Martin 2,3,4
PMCID: PMC6697167  NIHMSID: NIHMS1526201  PMID: 31043065

Abstract

Background:

Unplanned readmissions after hospitalization for acute myocardial infarction (AMI) are among the leading causes of preventable morbidity, mortality, and healthcare costs. Digital health interventions (DHI) could be an effective tool in promoting self-management, adherence to guideline-directed therapy, and cardiovascular risk reduction. A DHI developed at Johns Hopkins—the Corrie Health Digital Platform (Corrie)—includes the first cardiology Apple CareKit smartphone application, which is paired with an Apple Watch and iHealth Bluetooth-enabled blood pressure cuff. Corrie targets: (1) self-management of cardiac medications, (2) self-tracking of vital signs, (3) education about cardiovascular disease through articles and animated videos, and (4) care coordination that includes outpatient follow-up appointments.

Methods and Results:

MiCORE’s three phases include: (1) the development of Corrie, (2) a pilot study to assess the usability and feasibility of Corrie, and (3) a prospective research study to primarily compare time to first readmission within 30 days post-discharge among patients with Corrie to patients in the historical standard of care comparison group. In Phase 2, feasibility of deploying Corrie in an acute care setting was established among a sample of 60 AMI patients. Phase 3 is ongoing and patients from four hospitals are being enrolled as early as possible during their hospital stay if they are 18 years or older, admitted with AMI (STEMI or type I NSTEMI), and own a smartphone. Patients are either being enrolled with their own personal devices or they are provided an iPhone and/or Apple Watch for the duration of the study. Phase 3 started in October 2017 and we aim to recruit 140 participants.

Conclusions:

This manuscript will provide an in-depth understanding of the feasibility associated with implementing a DHI in an acute care setting, and the potential of Corrie as a self-management tool for AMI recovery.

Introduction

Every year in the United States (US), approximately 720,000 people experience their first acute myocardial infarction (AMI), and about 335,000 people suffer from recurrent AMIs.1 The Agency for Healthcare Research and Quality (AHRQ) released national data from 2011 showing that nearly 20% of Medicare patients hospitalized for an AMI are subsequently readmitted within 30 days of leaving the hospital, resulting in a total cost of $693 million from actual expenses incurred in the delivery of hospital services.2 According to the Medicare Payment Advisory Commission, 76% of 30-day readmissions are potentially preventable using proven standards of care such as adherence to medications, attending follow-up appointments, eating a Mediterranean diet, engaging in moderate-to-vigorous physical activity, and ceasing smoking.3

The American College of Cardiology (ACC) and the Institute for Healthcare Improvement pioneered the Hospital to Home (H2H) initiative, a nationwide quality improvement approach to increase patient self-management and to reduce 30-day readmissions. H2H recommendations to reduce AMI readmissions include the following: (a) developing programs that deliver general information about diagnosis and related comorbidities, (b) individualized education on medications, lifestyle modifications and cardiac rehabilitation, (c) information regarding post-discharge follow-up, and (d) the importance of adhering to one’s care plan.46 Despite the well-established factors that contribute to AMI readmissions, a gap exists between the ACC’s recommendations for post-AMI recovery and the current care delivery model. The standard of care is a paper-based discharge process that provides all of the medication, follow-up, and lifestyle modification directions in the last few moments of hospitalization. The failures of this system are evident because: (1) the majority of patients do not understand medication changes,7 (2) 40% of patients cannot accurately describe why they were hospitalized,8 and (3) 54% of patients cannot recall instructions about their follow-up appointments and treatment plan.8

The Food and Drug Administration defines DHIs as the software or hardware used to improve the quality, access, efficacy, or efficiency of healthcare delivery.9 With the ubiquitous growth in smartphone ownership among adults in the US (from 35% in 2011 to 77% in 2018),10 DHIs are currently an underutilized patient-centered resource that may be integrated into patients’ daily routines and improve self-management. This is especially relevant for adults ages 65 and older who are more likely to suffer an AMI and for whom smartphone ownership has increased 22%, from 24% in 2013 to 46% in 2018.1014 Smartphone ownership also remains ubiquitous with minimal variability across sex, race, and socioeconomic status.1114 In a recently conducted systematic review and meta-analysis on the effect of DHIs on the secondary prevention of cardiovascular disease (CVD), Gandhi and colleagues found that DHIs, including text messaging and smartphone applications (apps), increased medication adherence, attainment of blood pressure targets and exercise goals, increased awareness of diet and exercise, and reduced anxiety.15 This analysis showed no impact on hospital readmissions; however, only five studies assessed this outcome and only one included a smartphone app intervention.15 Thus, further research is needed to determine whether smartphone apps can help reduce 30-day hospital readmissions post-AMI.

Methods

The ongoing Johns Hopkins Myocardial infarction, COmbined-device, Recovery Enhancement (MiCORE) study—approved by the Johns Hopkins University School of Medicine Institutional Review Board (IRB00099938)—has three phases: (1) development of the DHI, (2) a pilot study to assess the usability and feasibility of the DHI, and (3) a prospective study to primarily compare time to first readmission within 30 days post-discharge among patients with Corrie to patients in the historical standard of care comparison group.

Phase 1: Development of the DHI

An interdisciplinary team comprising patients, nurses, physicians, design experts from Apple (Cupertino, California), and engineers partnered to develop a DHI that empowered patients to learn about their diagnosis and risk factors as early as possible during their hospital stay and to become actively involved in self-management throughout their care. The development of the DHI, Corrie (‘Cor’ is Latin for heart), started with a beta version of the smartphone app developed at Johns Hopkins University to address a clinical need. The technology progressed from paper renderings to wireframes, to an early-stage web-based app and finally, in collaboration with Apple, to an Apple CareKit App with a collaborative Apple Watch app, Bluetooth blood pressure cuff, and backend data monitoring.

The app was developed based on clinical expertise and constructs identified in widely-accepted theories of health behavior change, including the Health Belief Model (HBM) and Social Cognitive Theory (SCT).16 The HBM theorizes that people’s beliefs about whether they are at risk for a health problem, and their perceived benefits of taking action, influence readiness to change.17 The HBM has most frequently been applied for prevention-related, asymptomatic conditions such as CVD.18,19 The SCT synthesizes concepts from cognitive, behavioristic, and emotional models of behavior change and can be applied to interventions for disease prevention and management.20 Both theoretical approaches have overlapping constructs that, when included in behavior change interventions, have been associated with better outcomes.18, 2125 Behavior change strategies based in these theories such as education, self-monitoring, goal-setting, feedback, and prompts are particularly useful components of effective interventions that are used in Corrie. In Supplemental Table 1, we include theoretical components of the HBM and SCT, an explanation of each, and how they were integrated into Corrie to promote behavior change.

Corrie gained traction through the Johns Hopkins research team’s collaboration with Apple in designing an intuitive user interface. Apple’s open source software development framework, CareKit, provided out-of-the-box encryption and integration with Apple tools. An overview of Corrie is provided in Figure 1, which includes the smartphone app, smartwatch app on Apple Watch, and iHealth Ease Bluetooth-enabled blood pressure cuff (Model BPL3). The Corrie Data Platform, or backend, was engineered at the Johns Hopkins Whiting School of Engineering on Amazon Web Services (AWS) and is fully HIPAA compliant, enabling the team direct control over app-generated data and app content. The Corrie app, as seen in Figure 2, allows patients to (a) manage their medications (track daily adherence, indication, and side effects), (b) monitor their vital signs (heart rate, blood pressure, weight, mood, and steps), (c) learn about the risk factors for CVD and lifestyle modification through educational articles (all at a sixth or seventh grade reading level as determined by the Flesch-Kincaid Readability Test Tool) and animated videos (Nucleus Media) as seen in Table 1, (d) schedule and track follow-up appointments, (e) connect with their providers, and (f) upload stent and insurance cards for readily available health information. Follow-up appointments are entered manually into the app by the participant by choosing a clinician from a pre-populated directory or adding one manually. A reminder is sent automatically via Corrie one day before the appointment. Apple Watch integration allows participants to monitor their heart rate and physical activity, receive reminders on medications and appointments (also delivered on the iPhone home screen), and track medications directly on the watch. The iHealth blood pressure cuff integration allows participants to monitor, save, and review previous blood pressure recordings within Corrie. There is no monitoring of real time data by the study or patient’s clinical team. At this stage, Corrie is solely a self-management tool for patients.

Figure 1.

Figure 1.

MiCORE DHI: Corrie Digital Health Platform (Corrie) and Backend

Figure 2.

Figure 2.

Corrie CareKit App for AMI recovery

Table 1.

Educational article and animation topics

ABC’s of Heart Health Educational articles Educational animations
A’s 1) Age 1) About your heart attack and recovery
2) Antiplatelet medications 2) Angiography
3) Atherosclerosis
B’s 1) Blood pressure 1) Treating high blood pressure
C’s 1) Cholesterol 1) Managing high cholesterol
2) Cigarettes: cessation 2) Health effects of smoking
D’s 1) Diabetes 1) Treating high blood sugar
2) Diet 2) Treatment for obesity: lifestyle changes
E’s 1) Exercise 1) Treatment for obesity: lifestyle changes
2) Echocardiogram

While there are many behavior change apps available, a recent systematic review on the effect of mobile health on the secondary prevention of CVD found only three apps available for this purpose.15 One focused solely on just two lifestyle factors and two medications for risk reduction.26 For the secondary prevention of stroke, Shen delivered an app to hospitalized patients with educational videos about stroke prevention and management.27 Lastly, Widmer and colleagues provided an app to patients undergoing cardiac rehab so they could perform daily tasks and track physiologic data.28 There are apps that offer one or two features of medication tracking or education; however, Corrie offers patients an all-in-one approach for managing post-AMI recovery, including, the ability to: track any medication, weight, heart rate, blood pressure, physical activity, mood, and follow-up appointment; educate oneself about medications, hospital procedures they may undergo, CVD in general, and the various risk factors for CVD; and receive reminders to manage their care plan. Corrie also distinguishes itself from other apps by providing integration with the latest digital health sensors to provide real time data to patients.

Phase 2: Pilot Study

Design.

In the pilot study, participants were enrolled with Corrie as early as possible during their hospital stay and, for a small subsample, follow-up was conducted at 3 and 30 days post-discharge from the hospital.

Objectives.

The objective of the pilot study was to assess the feasibility of deploying Corrie to patients in the hospital and the usability of the intervention.

Sample, setting, and eligibility criteria.

From October 1, 2016 to September 30, 2017 convenience sampling was used to recruit 60 adults admitted to Johns Hopkins Hospital (JHH) and Johns Hopkins Bayview Medical Center (JHBMC) for ST elevation (STEMI) or non-ST elevation (NSTEMI) myocardial infarction, type 1. Patients were screened to determine whether they met the following additional inclusion criteria: (1) 18 years and older, (2) owned a smartphone, and (3) were approved to participate by their inpatient care team. Patients were excluded from the study if they (1) were non-English speaking, (2) had a visual, auditory, cognitive, or motor impairment that precluded use of Corrie, and (3) were unable to participate due to severity of illness. Smartphone ownership was required because the learning curve associated with smartphone use may have constituted a significant amount of time within the 30-day window and adversely affected perceived app usability. Corrie was downloaded directly onto the personal device of participants who owned an iPhone 5 or newer model. For patients who owned an earlier iPhone model or a different type of smartphone, a loaner iPhone (iShare) was provided for the study. Patients with a history of alcohol or substance abuse were considered for inclusion on a case-by-case basis considering the impact of the use disorder on the ability to meaningfully engage with the DHI.

Variables and measurement.

The following variables were collected for Phase 2:

Demographics.

Demographic and clinical data for those enrolled in Corrie.

Feasibility.

Feasibility was defined by successful download of the Corrie app onto a patient’s iPhone or deployment of the iShare, including deployment to older adults and low socioeconomic populations in acute care settings.

Usability.

Among a small subsample, the 10-item Systems Usability Scale (SUS), comprising five-point Likert scales, was used to assess perceived systems usability of Corrie both 3 and 30 days post-discharge.29 In a study of 2,324 cases the internal reliability of the SUS was found to be 0.91 and the concurrent validity to be 0.806.30

Procedures.

The pilot study procedures were similar to those conducted in Phase 3 and are described in detail below in accordance with the larger study with a few exceptions: participants in the pilot study did not receive an iHealth blood pressure cuff and were not given $10 Amazon gift cards for survey completion.

Data collection and management.

The SUS was sent out via email to a subsample of participants via Qualtrics.

Results.

Table 2 compares demographic data between those enrolled with Corrie to those in a National TRIUMPH AMI cohort.31 The two samples are largely comparable except the TRIUMPH cohort has a larger percentage of uninsured participants, smokers, and STEMIs.

Table 2.

Comparison of Corrie and TRIUMPH sample characteristics

Corrie (N=60)
n (%)
TRIUMPH31 (N=3,536)
n (%)
Age(yrs) mean±SD 58±11 59±12
<65 37 (62%) Unknown
≥65 23 (38%) Unknown
Women 18 (30%) 1,170 (33%)
White 41 (68%) 2,475 (70%)
No Health insurance 2 (3%) 738 (21%)
Obesity (BMI ≥30kg/m2) 30 (50%) 1,411 (40%)
Diabetes 18 (30%) 1,040 (29%)
Hypertension 32 (53%) 2,315 (66%)
Current Smoking 16 (27%) 2,095 (59%)
Dyslipidemia 29 (48%) 1,737 (49%)
Heart failure 4 (7%) 258 (7%)
STEMI/NSTEMI 21/39 (35%/65%) 1,583/1,953 (45%/55%)
Device enrolled w/
iShare 30 (50%) NA
Personal iPhone 30 (50%) NA

Deploying Corrie was deemed feasible in an acute care setting. Barriers to participation included: (1) distrust, lack of interest, or feeling too overwhelmed to engage in research, (2) perceiving they already had a system in place for managing CVD, and (3) inability to recall Apple ID and/or password on a personal iPhone to download Corrie. Initially, using a beta version on TestFlight, efficiency of the onboarding process was suboptimal. We addressed this by launching Corrie in the app store with access restricted to study participants. The iShare program was initiated in December 2016 to reduce selection bias based on smartphone ownership. Usability of Corrie is still being assessed in Phase 3 but based on preliminary feedback we: (1) enhanced the watch app so that participants could track their medications through the watch, (2) enlarged the medication buttons to make marking medications as taken easier, (3) added a warning if the patient attempted to take medications off schedule, (4) grouped the medication reminders together for each timepoint, and (5) developed a page for participants to view overall adherence and completion of activities.

Phase 3: Prospective Study

Design.

In this prospective study, participants are being enrolled with Corrie as early as possible during their hospital stay; follow-up is being conducted at 3 and 30 days post-discharge from the hospital.

Objectives.

The primary objective is to compare time to first readmission within 30 days post-discharge among patients with Corrie to patients in the historical standard of care comparison group. The secondary objectives are to evaluate: (1) the cost-effectiveness of Corrie as an intervention to reduce 30-day readmission rates in post-AMI patients; (2) in-hospital satisfaction, perceived app usability, app satisfaction, and patient activation among post-AMI patients with Corrie 3 days post-discharge; (3) perceived app usability, app satisfaction, patient activation, user engagement, medication adherence, emergency department (ED) visits, hospital observations, and attendance of follow-up appointments and cardiac rehab among post-AMI patients with Corrie 30 days post-discharge; and (4) change in perceived app usability, app satisfaction, and patient activation over time from 3 to 30 days post-discharge among post-AMI patients with Corrie. Exploratory analyses will be conducted to assess: (1) trends in the physiologic data (weight, mood, heart rate, blood pressure, physical activity) and (2) whether abnormal physiologic data precede a hospital readmission, observation, or ED visit.

Sample, setting, and eligibility criteria.

Consecutive sampling is being used to recruit adults admitted to JHH, JHBMC, Massachusetts General Hospital (MGH), and Reading Hospital for a STEMI or type 1 NSTEMI. We aim to recruit 140 Corrie users in Phase 3 meeting the eligibility criteria as described in Phase 2. Recruitment for the Corrie group at JHH and JHBMC started on October 1, 2017. Recruitment at MGH and Reading Hospital started later on June 8 and June 29, 2018, respectively. The historical comparison group consists of English-speaking AMI patients admitted from October 1, 2015 to October 1, 2016, prior to the availability of Corrie. Both groups receive(d) the standard of care.

Variables and measurement.

The following variables are being collected for Phase 3:

Baseline variables.

Baseline variables are being collected at the time of admission from the hospitals’ administrative databases. Demographic information includes age, sex, race, smoking, and insurance status. Hospitalization characteristics include the following: (a) hospital admitted to, (b) diagnosis of STEMI or NSTEMI, (c) length of stay, (d) type of intervention received (percutaneous coronary intervention or coronary artery bypass surgery), (e) a count of AHRQ identified Elixhauser comorbidities based on ICD-9-CM and ICD-10 diagnosis codes, (f) discharge disposition, and (g) for patients admitted to JHH and JHBMC, the Health Services Cost Review Commission expected readmission rate, defined by All-Patient–Refined Diagnosis-Related Group (APR-DRG) and severity-of-illness (SOI) combinations.3235

Among participants in the Corrie group, the following potential predictors of engagement with Corrie are being collected at baseline: whether a participant is enrolled with a personal device or iShare, mHealth literacy, and the perceived value of Corrie to help participants achieve their heart attack recovery goals. mHealth literacy is being measured using the mHealth Apps’ Trialability and mHealth Literacy five-point Likert scale to assess participants’ ability to use mobile devices to seek, find, understand, appraise, and apply health information to address or solve a health problem.36 The mHealth literacy subscale items were shown to have Cronbach’s alphas ranging from 0.75 to 0.85.36 Medications prescribed at discharge will also be collected among Corrie participants to assess the accuracy of medication tracking. Percent of hospitalization stay with access to Corrie will also be assessed.

Primary outcome variable.

Unplanned all-cause 30-day readmissions will be obtained from claims data/hospital administrative databases. JHH and JHBMC collect claims data including readmissions to any hospital in Maryland. MGH is part of a centralized clinical data registry; therefore, we will know if an MGH patient is readmitted to any of the six hospitals participating in this registry. The Reading Hospital readmissions data are limited to that particular site. For Corrie group participants we are also querying them on the 30-day surveys as to whether they were readmitted to any other hospitals to reduce missing data. The number of cardiac and AMI-related readmissions will be evaluated for each group.

Secondary outcome variables.

The following variables, except for cost-effectiveness, ED visits, and hospital observations, are solely being collected among Corrie participants:

In-hospital care satisfaction.

A 10-item subset of the Hospital Consumer Assessment of Healthcare Providers and Systems survey, comprising five-point Likert scales plus one dichotomous question, is used 3 days post-discharge to assess participants’ satisfaction with the care they received during their hospitalization.37 Internal consistency reliabilities for the survey are greater than 0.77.37

Usability.

The SUS described in Phase 2 is used to assess perceived usability of Corrie 3 and 30 days post-discharge.29,30

App satisfaction.

A five-item scale developed by the study team, comprising five-point Likert scales, is used to assess participant satisfaction with Corrie as a tool to improve AMI recovery 3 and 30 days post-discharge.

User engagement.

This variable is assessed as (1) a subjective experience and (2) through app usage to capture the behavioral manifestations of user engagement:

  1. Subjective experience: The 31-item User Engagement Scale, comprising five-point Likert scales, is used to assess the subjective experience of user engagement with Corrie 30 days post-discharge. This scale includes items to capture the perceived aesthetics, novelty, involvement, usability of the app, focused attention (perception of time passing while using the app), and endurability (willingness to return to the app or recommend it to others).38,39 Internal consistencies for the subscales range from 0.74 to 0.92.40

  2. App usage: We monitor the number of user sessions as defined by the number of times the app transitions to the foreground of the phone; time spent on the app overall and the individual pages; time elapsed between sessions; and the number of session interactions, including: medications tracked, vitals tracked, educational articles read, videos watched, and follow-up appointments added.

Patient activation.

The 10-item Patient Activation Measure, comprising four-point Likert scales plus a “not applicable” option, is used to assess patient activation 3 and 30 days post-discharge. Concepts assessed include: believing the patient role is important, having the confidence and knowledge necessary to act, acting to maintain and improve one’s health, and staying the course even under stress.41 Among a sample of hospitalized multimorbid patients, the estimated reliability of the scale was 0.88 and the content validity index was 0.91.42

Medication adherence.

The eight-item Adherence to Refills and Medications Scale subscale, comprising four-point Likert scales, is used to assess cardiac medication adherence 30 days post-discharge.43 The scale has demonstrated an internal consistency of 0.79 and stability across literacy levels.43 Medication adherence is also measured from the smartphone and smartwatch app usage data.

Emergency department visits.

This variable is assessed in both groups 30 days post-discharge via claims data/administrative databases and by querying patients on the 30-day surveys for the Corrie group.

Hospital observations.

This variable is assessed in both groups 30 days post-discharge via claims data/administrative databases and by querying patients on the 30-day surveys for the Corrie group.

Attendance of follow-up appointments and cardiac rehab.

We are querying participants in the surveys sent out 30 days post-discharge as to whether they attended an appointment with a primary care provider, cardiologist, or cardiac rehab. Follow-up appointment scheduling is also measured via app usage data.

Procedures.

Every 24 hours, a study team member reviews a list of patients admitted with either a STEMI or type 1 NSTEMI. At JHH/JHBMC, we built an automatic trigger within the Epic electronic medical record to alert the study team to patients admitted with an AMI diagnosis or an elevated troponin to reduce the likelihood of missing eligible patients. At Reading Hospital, the research team receives a census every 24 hours of patients admitted with an AMI and at MGH the team reviews the patient census on the study units. Potentially eligible participants are entered into REDCap (Research Electronic Data Capture), a secure web-based data application designed to support data capture for research studies,44 and screened by the team member to determine whether they meet the eligibility criteria. The team member then approaches the patient to determine whether he/she owns a smartphone and if so provides more information to determine whether the patient is interested in participating. If the patient is eligible and elects to participate, the team member consents the patient via an electronic informed consent app built on the Apple ResearchKit platform (Figure 3). The informed consent app, downloaded onto study team iPads or iPhones, enables a team member to guide participants through the informed consent process and to provide them with the necessary information regarding the risks and benefits to participating, all in an easy-to-understand format. A copy of the electronic informed consent is automatically emailed to both the participant and the team. Once the consent is complete, the participant’s username and password to access Corrie is set up. An email is required to create a username for Corrie. In the rare instance a patient does not have an email address we create a free Gmail account for them.

Figure 3.

Figure 3.

Electronic Informed Consent ResearchKit App

The Corrie app is then downloaded onto the participant’s personal device or the participant is given an iShare, preloaded with the Corrie app. All participants receive the Corrie app, Apple Watch, and iHealth blood pressure cuff, with the exception of those who own an iPhone 5 which is not compatible with WatchOS 4. Touch ID or Facial Recognition, when available, is activated to streamline the login process. Corrie requires WiFi or a data plan for logging-on and syncing with the backend; however, the majority of the features are stored locally and do not require the user to have internet connection. SIM cards are also included in the iShares. A team member provides a Corrie orientation which includes reviewing main features of the app, showing the participant how to add, edit, track, and review indications and common side effects of his/her medications, monitor his/her heart rate and step count using the Watch, monitor his/her blood pressure using the iHealth blood pressure cuff, access the educational articles and videos, schedule follow-up appointments, add providers’ or health advocates’ contact information, and upload stent or insurance cards to the app.

When instructing the participant on how to add his/her medications, the team member prints out the participant’s medication list at the time of enrollment and shows him/her how to add one medication. The team member then has the participant enter the other medications to ensure comprehension and reviews them to make sure the medications were entered correctly. The team member notifies the patient that his/her medications are likely to change at discharge and potentially after seeing his/her primary care provider or cardiologist, thus it is his/her responsibility to update them as prescribed. Each participant is informed that he/she is expected to use the app with at least the same frequency as his/her medications are due, typically once or twice daily. The app allows patients to enter their vitals at any point throughout the day. As the optimal frequency of vitals monitoring following AMI is unknown, the app does not dictate a rigid schedule. Rather, participants enter vitals per their preferences and are not reminded to enter them at scheduled times. Patients can review vitals tracked on previous days. The goal of tracking physiologic parameters within Corrie is to increase awareness of individual susceptibility for CVD risk factors. Goal completion graphics for vitals also serve as a call to action. Paired with the other app components, including education on these topics and medications to manage many of these parameters we provide patients with a way to comprehensively self-manage their care. The participant is encouraged to review the educational content while hospitalized or within the first week post-discharge, and prior to discharge is encouraged to schedule and enter at least two follow-up appointments.

For participating in the study, participants are given a Corrie tote bag containing the following: a visual step-by-step quick-start guide on the key features of Corrie and study team contact information, standard Corrie pill box (does not track medication adherence), iHealth blood pressure cuff, and prepaid return mailer to return loaned devices following study completion. They are instructed to return equipment following 30 days post-discharge. After hospital discharge, participants have no scheduled follow-up appointments with the study team. Follow-up data is collected from participants via emailed REDCap surveys.

Data collection and management.

Data are collected at various timepoints throughout the study either through extraction from the administrative databases, participant self-report, or Apple ResearchKit, a software framework for iPhone apps that allows researchers to access user data analytics.

REDCap automatically emails surveys to patients 3 and 30 days post-discharge. Survey emails contain one link that connects participants to the first survey and upon completion seamlessly transitions them to the following survey. If a participant does not respond to his/her surveys within 3 days, he/she receives an automated reminder email. If the participant still has not completed the surveys within 7 days, a team member attempts to contact the participant by phone to encourage survey completion.

The user data analytics collected via the app are received continuously while the app is open and active. The app is designed to be offline-first, meaning data is first written locally on the phone and then synchronized with the Corrie Data Platform when WiFi or Cellular connections are available. Our platform is fully HIPAA compliant following industry-standard best practices such as end-to-end encryption (both in-flight and at-rest). We use Apple’s encryption on the device which encrypts all user and app data.

Statistical plan.

Participants from Phases 2 and 3 (n=200) will be combined to assess the primary outcome and secondary outcomes of cost-effectiveness, ED visits, and hospital observations. The other Phase 3 secondary outcomes will be assessed among the sample of 140. Descriptive statistics will be provided for the baseline variables. We will use frequencies (percentages) to describe categorical variables and mean (standard deviation) or median (interquartile range) to describe continuous variables. Inferential statistics will be used to compare demographic and clinical characteristics between Corrie and non-Corrie (including those in the historical comparison group and those excluded from prospective enrollment) patients using chi-square for categorical variables and t-tests for continuous variables. Inferential statistics will also be used to compare demographic and clinical characteristics between Corrie users enrolled in Phase 2 versus Phase 3 using chi-square for categorical variables and t-tests for continuous variables. For inferential statistics, alpha will be set at 0.05 to determine statistical significance. Stata will be used to conduct analyses. Microsoft Excel will be used for the calculations and modeling related to the cost-effectiveness analyses. Non-conflicted study team members that do not hold equity in Corrie Health perform patient recruitment and will analyze the results independently.

Primary objective.

Claims data/hospital databases will be used to conduct Cox proportional hazard models to test differences in time to all-cause first readmission within 30 days between groups, controlling for covariates. Covariates, selected based on the literature, include the following: age, sex, race, smoking status, insurance status, diagnosis of STEMI or NSTEMI, count of comorbidities, and any additional baseline variables assessed that differ significantly between the two groups. Site will be included as a fixed effect in the models. Among participants in the Corrie group, exploratory Cox proportional hazard models will also be used to test differences in time to all-cause first readmission within 30 days between those who use Corrie on a personal versus loaner iPhone, between those enrolled in Phase 2 versus 3, and by expected readmission rate and user engagement, controlling for covariates. In the Cox regression models, we will include and control for an indicator variable for study phase, followed by adding cross-product terms of this variable with each exposure of interest to evaluate the interaction effects. Finally, in a sensitivity analysis, we will restrict our analysis to participants enrolled in Phase 3 to test the robustness of the results obtained from the whole sample.

Secondary objectives.

(1) To assess cost-effectiveness of Corrie, a Markov Model of cost-effectiveness will be built, based on 30-day readmission rates, to assess the Incremental Cost-Effectiveness Ratio of adopting Corrie. A Markov Model will be used to model probabilities of events over time. The perspective considered is the healthcare system (provider/payer). By attaching estimates of resource use and health outcome consequences to the states and transitions (no complications, readmissions, or deaths) in the model and then running the model over a large number of cycles, it is possible to estimate the long-term costs and outcomes associated with a disease (AMI) and a particular healthcare intervention (Corrie).45 A Markov Model is particularly useful in the early stages of cost-effectiveness assessment when there is uncertainty regarding the cost and effectiveness of the intervention.45 This model provides a way of systematically managing uncertainty using sensitivity and threshold analysis.45 A 10,000-cycle multivariate probabilistic sensitivity analysis approach will be employed. All currencies will be provided in US dollars and utility provided in Quality-Adjusted Life Years.

(2) We will use descriptive statistics to assess in-hospital care satisfaction, perceived app usability, app satisfaction, and patient activation among post-AMI patients with Corrie 3 days post-discharge. (3) We will use descriptive statistics to assess perceived app usability, app satisfaction, patient activation, user engagement, medication adherence, ED visits, and attendance of follow-up appointments and cardiac rehab among post-AMI patients with Corrie 30 days post-discharge. Multiple linear regressions will be conducted to assess the associations between the proposed baseline predictors and user engagement with Corrie, controlling for covariates. The various medication adherence measures (ARMS and Corrie self-monitoring data) will be compared to see whether they are positively correlated with each other. (4) T-tests will be used to assess change in mean scores from 3 to 30 days post-discharge, for perceived app usability, app satisfaction, and patient activation.

Missing data plan.

Readmissions to hospitals outside the data trusts may be missed; however, this impacts the intervention and comparison group similarly. We expect some missing data resulting from noncompletion for the secondary objective outcomes collected via REDCap surveys. We will examine differences between those with and without missing data (chi-square and t-tests on baseline variables) to determine if there are factors related to noncompletion. As we only anticipate missing data for the secondary objectives we will conduct complete case analyses.

Sample size and power analysis.

Allowing for 10% of drop-outs during 30-day follow-up, the aim was to recruit N=200 in the intervention group and N=1000 patients in the comparison group in order to provide around 90% power at the two-sided 5% level of significance to detect a hazard ratio of 0.5 (i.e. 50% reduction in a hazard of the intervention group), assuming 85% of patients in the comparison group survive or are not readmitted by the end of study (Stata stpower).46 The incident readmission rate of patients in our historical comparison group admitted to JHH and JHBMC was 16 cases per 100 person-months. We aim to detect a 50% risk reduction in readmission rates because our solution is comprehensive in nature and it is estimated that 76% of 30-day readmissions are potentially preventable using proven standards of care.3

Results

Phase 2 was initiated October 1, 2016 and ended September 30, 2017 after recruiting 60 participants from JHH and JHBMC. After determining that deploying Corrie was feasible and obtaining initial data on usability, the research team transitioned to Phase 3. Recruitment for Phase 3 was initiated on October 1, 2017 and is expected to be complete by March 2019.

Discussion

The main goal of this prospective study is to compare time to first readmission within 30 days post-discharge among patients with Corrie to patients in the historical standard of care comparison group. Placing intuitive technology in the hands of patients may empower them to manage their CVD during their hospitalization as they transition home and back to their community. This novel approach to prevention after AMI and our multi-phase study design, incorporating changes in the DHI over time, may be of interest to other researchers.

We recognize risks of bias in our research design and have taken steps to address these. First, the generalizability of our results may be limited as we are using nonprobability sampling and our sample is limited to English-speaking adults. Corrie is currently only available in English, and therefore people from ethnic minorities, who are at increased risk for CVD, may be excluded. It also raises the possibility of selecting a more motivated sample, limiting generalizability to patients who are less motivated and possibly to those with a higher risk of readmission. We will assess whether Corrie group participants are significantly different from those in the comparison group in terms of number of Elixhauser comorbidities, a proxy for readmission risk, and control for this variable when conducting our primary analysis. It is possible that those who agree to participate may be more motivated towards self-care which is the group who would ultimately utilize the intervention in the future. The Corrie sample in Phase 2 was similar in age to the national TRIUMPH AMI cohort. Ultimately, age distribution in the total MiCORE sample after completion of Phase 3 will determine generalizability to older adults. In patients who were otherwise eligible but excluded for not owning a smartphone, we will assess the relationship between age and smartphone ownership. We will examine the relationship between age and user engagement. However, we have attempted to increase the generalizability of our results through the creation of the iShare program and enrolling from multiple sites. The iShare program has helped us to expand our sample, potentially reduce selection bias stemming from different levels of motivation toward health behavior change between iOS and Android users, and reach people of lower socioeconomic status, as iOS devices are typically more expensive than other brands. At this stage, however, it is unclear how user engagement with Corrie will vary between native iPhone owners and participants in the iShare program.

Second, Corrie is being developed iteratively across the course of the study based on participant feedback. As a result, participants may have slightly different experiences with Corrie depending on when they are enrolled in the study. We track when changes are made and will examine if any substantial differences in user engagement are seen over time.

Third, it will be difficult to differentiate what improvement in health outcomes may be due to the smartphone application versus the Apple Watch or blood pressure cuff. Future studies will be necessary to differentiate between these components and the impact on engagement and health outcomes. Fourth, the following are threats to internal validity that arise from using a historical comparison group: (1) variables for the primary analysis being limited to certain areas/hospitals, (2) differences between groups, and (3) the potential for maturation bias. Only being able to assess 30-day readmission rates to certain areas/hospitals may lead to an underestimation of readmissions in both groups. For the comparison group, we are unable to exclude those with an impairment that would preclude the use of Corrie and those without a smartphone as we do not have data available on these variables. We are also unable to compare other app use between the two groups which may result in residual confounding. Over the course of the study period there have been significant changes in DHIs; therefore, DHIs independent of Corrie may influence outcomes. However, by not limiting Corrie participants access to other apps, the groups are more likely to be similar in this regard. When interpreting the results, we will consider any changes that occurred in policy and care at the hospital sites that may affect 30-day readmissions and the potential for maturation bias.

Conclusions

The MiCORE study will provide a wealth of information about clinical integration of Corrie, an innovative, patient-centered, action-oriented DHI aimed at self-management, adherence to guideline-directed therapy, and cardiovascular risk reduction. In addition, the theoretical framework for developing Corrie based on the HBM and SCT concepts of self-management will offer valuable guidance to other researchers when developing future apps.

Supplementary Material

Supplemental Material

Acknowledgments

Thank you, Jie Ding, PhD, statistician at the Johns Hopkins Ciccarone Center for the Prevention of Cardiovascular Disease for your contributions to the statistical plan of this study.

Sources of Funding

This study has received material support from Apple and iHealth and funding from the Maryland Innovation Initiative, Wallace H. Coulter Translational Research Partnership, Louis B. Thalheimer Fund, and the Johns Hopkins Individualized Health Initiative. Erin Spaulding, RN, PhDc, has received the following financial support for the research, authorship, and publication of this article: NIH/NINR F31 NR017328, Ruth L. Kirschstein National Research Service Award and NIH/NINR T32 NR012704, Pre-Doctoral Fellowship in Interdisciplinary Cardiovascular Health Research.

References

  • 1.Benjamin EJ, Virani SS, Callaway CW, Chang AR, Cheng S, Chiuve SE, Cushman M, Delling FN, Deo R, de Ferranti SD, Ferguson JF, Fornage M, Gillespie C, Isasi CR, Jimenez MC, Jordan LC, Judd SE, Lackland D, Lichtman JH, Lisabeth L, Liu S, Longenecker CT, Lutsey PL, Matchar DB, Matsushita K, Mussolino ME, Nasir K, O’Flaherty M, Palaniappan LP, Pandey DK, Reeves MJ, Ritchey MD, Rodriguez CJ, Roth GA, Rosamond WD, Sampson UKA, Satou GM, Shah SH, Spartano NL, Tirschwell DL, Tsao CW, Voeks JH, Willey JZ, Wilkins JT, Wu JHY, Alger HM, Wong SS, Muntner P. Heart disease and stroke statistics–2018 update: a report from the American Heart Association. Circulation. 2018;137:e67–e492. doi: 10.1161/CIR.0000000000000558. [DOI] [PubMed] [Google Scholar]
  • 2.Hines AL, Barrett ML, Jiang HJ, Steiner CA. Conditions with the Largest Number of Adult Hospital Readmissions by Payer, 2011 HCUP Statistical Brief #172. April 2014; Agency for Healthcare Research and Quality, Rockville, MD: Available: http://www.hcup-us.ahrq.gov/reports/statbriefs/sb172-Conditions-Readmissions-Payer.pdf (Accessed October 10, 2018) [PubMed] [Google Scholar]
  • 3.Medicare Payment Advisory Commission Report to the Congress: promoting greater efficiency in Medicare. Washington (DC): The Commission; 2007. p.107–108. Available: http://www.medpac.gov/docs/default-source/reports/Jun07_EntireReport.pdf?sfvrsn=0 (Accessed October 10, 2018) [Google Scholar]
  • 4.Wiggins BS, Rodgers JE, DiDomenico RJ, Cook AM, Page RL. Discharge counseling for patients with heart failure or myocardial infarction: a best practices model developed by members of the American College of Clinical Pharmacy’s Cardiology Practice and Research Network based on the Hospital to Home (H2H) Initiative. Pharmacotherapy. 2013;33:558–580. doi: 10.1002/phar.1231 [DOI] [PubMed] [Google Scholar]
  • 5.Jneid H, Anderson JL, Wright RS, Adams CD, Bridges CR, Casey DE, Ettinger SM, Fesmire FM, Ganiats TG, Lincoff AM, Peterson ED. 2012 ACCF/AHA focused update of the Guideline for the Management of Patients with Unstable Angina/Non -ST-Elevation Myocardial Infarction. Circulation. 2012;126:875–910. doi: 10.1161/CIR.0b013e318256f1e0. [DOI] [PubMed] [Google Scholar]
  • 6.O’Gara PT, Kushner FG, Ascheim DD, Casey DE, Chung MK, De Lemos JA, Ettinger SM, Fang JC, Fesmire FM, Franklin BA, Granger CB, Krumholz HM, Linderbaum JA, Morrow DA, Newby LK, Ornato JP, Ou N, Radford MJ, Tamis-Holland JE, Tommaso CL, Tracy CM, Woo YJ, Zhao DX. 2013 ACCF/AHA guidelines for the management of ST-elevation myocardial infarction: executive summary. A report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. Circulation. 2012;127:529–55. doi: 10.1161/CIR.0b013e3182742cf6 [DOI] [PubMed] [Google Scholar]
  • 7.Horwitz LI, Moriarty JP, Chen C, Fogerty RL, Brewster UC, Kanade S, Ziaeian B, Jenq GY, Krumholz HM. Quality of discharge practices and patient understanding at an academic medical center. JAMA Intern Med. 2013;173:1715–1722. doi: 10.1001/jamainternmed.2013.9318 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Ziaeian B, Araujo KL, Van Ness PH, Horwitz LI. Medication reconciliation accuracy and patient understanding of intended medication changes on hospital discharge. J Gen Intern Med. 2012; 27:1513–1520. doi: 10.1007/s11606-012-2168-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Food and Drug Administration. U.S. Department of Health and Human Services. 2017; Available: https://www.fda.gov/MedicalDevices/DigitalHealth/default.htm (Accessed October 10, 2018)
  • 10.Pew Research Center. Device Ownership Over Time. Pew Res Center’s Internet Am Life Proj. 2018; Available: http://www.pewinternet.org/data-trend/mobile/device-ownership/ (Accessed October 10, 2018)
  • 11.Pew Research Center. Mobile Technology Fact Sheet. Pew Research Center’s Internet & American Life Project. 2014; Available: http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/ (Accessed October 10, 2018)
  • 12.Anderson M Technology Device Ownership: 2015. Pew Research Center. 2015; Available: http://www.pewinternet.org/2015/10/29/technology-device-ownership-2015/ (Accessed October 10, 2018)
  • 13.Anderson M & Perrin A. Tech Adoption Climbs Among Older Adults. Pew Research Center. 2017; Available: http://www.pewinternet.org/2017/05/17/tech-adoption-climbs-among-older-adults/ (Accessed January 7, 2019)
  • 14.Ericsson AB. Ericsson Mobility Report: On the Pulse of the Networked Society Ericsson; Sweden, Tech. Rep. EAB-14, 61078. 2015; Available: https://www.ericsson.com/assets/local/news/2015/6/ericsson-mobility-report-june-2015.pdf (Accessed October 10, 2018) [Google Scholar]
  • 15.Gandhi S, Chen S, Hong L, Sun K, Gong E, Li C, Yan LL, Schwalm JD. Effect of mobile health interventions on the secondary prevention of cardiovascular disease: systematic review and meta-analysis. Can J Cardiol. 2017;33:219–231. doi: 10.1016/j.cjca.2016.08.017 [DOI] [PubMed] [Google Scholar]
  • 16.Painter JE, Borba CP, Hynes M, Mays D, Glanz K. The use of theory in health behavior research from 2000 to 2005: A systematic review. Ann Behav Med. 2008;35:358–362. doi: 10.1007/s12160-008-9042-y [DOI] [PubMed] [Google Scholar]
  • 17.Rosenstock IM. Historical origins of the health belief model. Health Educ Monogr. 1974;2:328–325. [Google Scholar]
  • 18.Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Anna Rev Public Health. 2010;31:399–418. doi: 10.1146/annurev.publhealth.012809.103604 [DOI] [PubMed] [Google Scholar]
  • 19.Will JC, Farris RP, Sanders CG, Stockmyer CK, Finkelstein EA. Health promotion interventions for disadvantaged women: overview of the WISEWOMAN projects. J Womens Health. 2004;13:484–502. doi: 10.1089/1540999041281025. [DOI] [PubMed] [Google Scholar]
  • 20.Bandura A Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ, US: Prentice-Hall, Inc; 1986. [Google Scholar]
  • 21.Ammerman AS, Lindquist CH, Lohr KN, Hersey J. The efficacy of behavioral interventions to modify dietary fat and fruit and vegetable intake: A review of the evidence. Prev Med. 2002;35:25–41. doi: 10.1006/pmed.2002.1028. [DOI] [PubMed] [Google Scholar]
  • 22.Legler J, Meissner HI, Coyne C, Breen N, Chollette V, Rimer BK. The effectiveness of interventions to promote mammography among women with historically lower rates of screening. Cancer Epidemiol Biomarkers Prev. 2002;11:59–71. [PubMed] [Google Scholar]
  • 23.Noar SM, Benac CN, Harris MS. Does tailoring matter? Meta-analytic review of tailored print health behavior change interventions. Psychol Bull. 2007;133:673–693. doi: 10.1037/0033-2909.133.4.673. [DOI] [PubMed] [Google Scholar]
  • 24.Taylor N, Conner M, Lawton R. The impact of theory on the effectiveness of worksite physical activity interventions: A meta-analysis and meta-regression. Health Psychol Rev. 2012;6:33–73. doi: 10.1080/17437199.2010.533441. [DOI] [Google Scholar]
  • 25.Webb TL, Joseph J, Yardley L, Michie S. Using the Internet to promote health behavior change: A systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010;12:e4. doi: 10.2196/jmir.1376. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Tian M, Ajay VS, Dunzhu D, Hameed SS, Li X, Liu Z, Li C, Chen H, Cho K, Li R, Zhao X, Jindal D, Rawal I, Ali MK, Peterson ED, Ji J, Amarchand R, Krishnan A, Tandon N, Xu LQ, Wu Y, Prabhakaran D, Yan LL. A cluster-randomized, controlled trial of a simplified multifaceted management program for individuals at high cardiovascular risk (SimCard Trial) in rural Tibet, China, and Haryana, India. Circulation. 2015;132:815–824. doi: 10.1161/CIRCULATIONAHA.115.015373 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Widmer RJ, Allison T, Lerman L, Lerman A. The augmentation of usual cardiac rehabilitation with an online and smartphone-based program improves cardiovascular risk factors and reduces rehospitalizations. Journal of the American College of Cardiology 2014;63:A1296. [Google Scholar]
  • 28.Shen L Application of video information technology on health education for stroke patients. Nurs Recov. 2015;14:10–2. [Google Scholar]
  • 29.Brooke J SUS-A quick and dirty usability scale. Usability evaluation in industry. 1996;189:4–7. [Google Scholar]
  • 30.Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24:574–594. doi: 10.1080/10447310802205776. [DOI] [Google Scholar]
  • 31.Dreyer RP, Dharmarajan K, Kennedy KF, Jones PG, Vaccarino V, Murugiah K, Nuti SV, Smolderen KG, Buchanan DM, Spertus JA, Krumholz HM. Sex Differences in 1-Year All-Cause Rehospitalization in Patients After Acute Myocardial Infarction: A Prospective Observational Study. Circulation. 2017;135:521–531. doi: 10.1161/CIRCULATIONAHA.116.024993. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Healthcare Cost and Utilization Project. NIS Description of Data Elements. Healthcare Cost and Utilization Project (HCUP). 2008; Agency for Healthcare Research and Quality, Rockville, MD: Available: https://www.hcup-us.ahrq.gov/db/vars/aprdrg/nisnote.jsp (Accessed October 10, 2018) [PubMed] [Google Scholar]
  • 33.Hoyer EH, Brotman DJ, Apfel A, Leung C, Boonyasai RT, Richardson M, Lepley D, Deutschendorf A. Improving outcomes after hospitalization: a prospective observational multicenter evaluation of care coordination strategies for reducing 30-day readmissions to Maryland hospitals. J Gen Intern Med. 2018;33:621–627. doi: 10.1007/s11606-017-4218-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Hoyer EH, Needham DM, Miller J, Deutschendorf A, Friedman M, Brotman DJ. Functional status impairment is associated with unplanned readmissions. Arch Phys Med Rehabil. 2013;94:1951–8. doi: 10.1016/j.apmr.2013.05.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Oduyebo I, Lehmann CU, Pollack CE, Durkin N, Miller JD, Mandell S, Ardolino M, Deutschendorf A, Brotman D. Association of self-reported hospital discharge handoffs with 30-day readmissions. JAMA Intern Med. 2013;173:624–9. doi: 10.1001/jamainternmed.2013.3746 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Lin TT, Bautista JR. Understanding the Relationships between mHealth Apps’ Characteristics, Trialability, and mHealth Literacy. J Health Comm. 2017;22:346–354. doi: 10.1080/10810730.2017.1296508. [DOI] [PubMed] [Google Scholar]
  • 37.Dyer N, Sorra JS, Smith SA, Cleary P, Hays R. Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) clinician and group adult visit survey. Med Care. 2012;50:S28–34. doi: 10.1097/MLR.0b013e31826cbc0d. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.O’Brien HL, Toms EG. What is user engagement? A Conceptual Framework for defining user engagement with technology. J Am Soc Inf Sci Technol. 2008;59:938–955. doi: 10.1002/asi.20801.1. [DOI] [Google Scholar]
  • 39.O’Brien HL, Toms EG. The development and evaluation of a survey to measure user engagement. J Am Soc Inf Sci Technol. 2010;61:50–69. doi: 10.1002/asi.21229. [DOI] [Google Scholar]
  • 40.O’Brien HL, Toms EG. Is there a Universal Instrument for Measuring Interactive Information Retrieval? The Case of the User Engagement Scale. In Proceedings of the third symposium on Information interaction in context. 2010;335–340. doi: 10.1145/1840784.1840835. [DOI] [Google Scholar]
  • 41.Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Serv Res. 2004;39:1005–26. doi: 10.1111/j.1475-6773.2004.00269.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Schmaderer M, Pozehl B, Hertzog M, Zimmerman L. Psychometric properties of the patient activation measure in multimorbid hospitalized patients. J Nurs Meas. 2015;23:128–41. doi: 10.1891/1061-3749.23.3.E128. [DOI] [PubMed] [Google Scholar]
  • 43.Kripalani S, Risser J, Gatti ME, Jacobson TA. Development and Evaluation of the Adherence to Refills and Medications Scale (ARMS) among Low-Literacy Patients with Chronic Disease. Value Health. 2009;12:118–123. doi: 10.1111/j.1524-4733.2008.00400.x [DOI] [PubMed] [Google Scholar]
  • 44.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–81. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Briggs A, Sculpher M. An Introduction to Markov Modelling for Economic Evaluation. Pharmacoeconomics. 1998;13:397–409. [DOI] [PubMed] [Google Scholar]
  • 46.Freedman LS. Tables of the number of patients required in clinical trials using the logrank test. Statistics in medicine. 1982;1:121–129. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplemental Material

RESOURCES