Remote digital smart device follow-up in prospective clinical trials: early insights from ORBITA-2, ORBITA-COSMIC, and ORBITA-STAR

Abstract

Smart devices are a fundamental media for acquisition, processing, storage, and transfer of digital health data. The global penetration and high frequency usage of smart devices such as smartphones and fitness monitors provide us an opportunity for incorporation into clinical trials to generate more clinically meaningful data. Reporting of angina can significantly vary between patients and also within patients at different timepoints. Furthermore, the nature of angina can lead to variation in ways patients adapt their activities of daily living and hence reporting of symptoms and quality of life. Current clinical trials investigating the effects of intervention on angina do not accurately incorporate these patient centred outcomes and considerations. Hence, methods to contemporaneously assess daily angina burden in a convenient, patient focused, and cost-effective manner are priorities for contemporary clinical trials to address. In this article, we provide our insights into the use of remote digital smart devices in clinical trials of stable coronary artery disease conducted by our research group. We discuss how our experiences from previous trials necessitated its incorporation and will provide us with important data that will inform clinical practice. We discuss the benefits and current challenges and limitations of smart device incorporation while providing our procedural workflow for how we incorporated smart devices into our clinical trials for others to consider. We hope that this approach will allow us to understand the perceptions and implications of angina on patient lives with greater granularity than previously explored.

Introduction

Digital health is a general term, encompassing eHealth (i.e. the utility of information and communication technology in health-related fields), mHealth (mobile wireless technologies including wearables), and advancing areas such as artificial intelligence and omics.¹ These technologies have the potential to establish a more personalized model of care, while allowing enhanced communication between healthcare providers and patients. They may also increase patient safety if the large amounts of health data generated through these processes can be managed, filtered, and interpreted correctly.

Leveraging advancements in digital technology, smart devices have become the cornerstone of digital health data acquisition, processing, storage, and transfer. By definition, a smart device is one that is able to connect to a network (e.g. bluetooth or Wi-Fi), transmit data via a communication channel, has a decision trigger (in built algorithms where certain input conditions execute certain actions), or has a sensor with aggregator function (able to measure a physical property and transform/aggregate this sensor data into information).² Modern-day smartphones and wearables such as smartwatches and fitness trackers allow us to measure various physical properties such as step count, step intensity, quality of sleep, heart rate, heart rhythm, and anthropometric composition.³ They can also be used to promote positive behavioural change such as to increase physical activity via motivational text messaging, gamification, and even social and financial incentives.³

The COVID-19 pandemic has further widened the remit of digital health and smart device usage whether through the lens of teleconsultation and telemonitoring of patients or even digital health education.^4,5 Beyond the pandemic, the increased chronic disease burden, shortage of healthcare professionals (relative to an aging population), and inevitable rollout of wireless network communication over larger territories at lower costs for access requires enhanced deployment and utilization of digital technology.⁶ This is especially true for the future of cardiovascular care. Recent position statements by the European Society of Cardiology and American College of Cardiology have further emphasized this need.^6,7

Digitalizing clinical trials

Clinical trials are the bedrock upon which guideline recommendations, funding decisions, and contemporary clinical practice are based. However, the exceptional knowledge that these evidence generating experiments can offer sometimes comes at exceptional cost. The conduct of a clinical trial can be remarkably challenging, with difficulties often encountered in patient recruitment and retention and time-consuming and burdensome data collection.^8,9 For these reasons, many clinical trials fail to deliver the answer to the question they set out.¹⁰ This is wasteful for both patients, researchers, and funders.

In recognition of these contemporary challenges and advancements in digital health, efforts to incorporate digital technology into clinical trials have reached governmental priority. The potential benefits are obvious: streamlined data collection, reduced patient burden, and the possibility for big data acquisition are just some advantages. In April 2019, the United States National Institute of Health and National Science Foundation held a workshop in Maryland, bringing together eminent clinical trialists and experts in digital technology and analytics to discuss strategies in digitalizing clinical trials and urging trialists to adopt these technologies into their protocols where possible.¹¹ Moreover, global penetration of smartphones and high frequency of fitness monitor ownership make the adoption of these technologies attractive to clinical trialists.¹²

The application of smart devices in clinical trials is wide. In trials of Parkinson’s disease, progression of disease and response to therapy is monitored using subjective scales; however, smart device incorporation allows these measurements to be quantified objectively using sensors.¹³ Studies identifying triggers in asthma utilize a global positioning system to determine weather and air pollution information, to correlate asthma systems with environmental factors.¹⁴ In cardiology, step counting and position trackers allow for calculations of a six-minute walk test, an important predictor of mortality.¹⁵

The recent industry-sponsored Apple Heart Study, however, highlighted both the power and limitations of digital innovation in clinical trials.¹⁶ While this pragmatic, site-less study provides a foundation for large-scale recruitment (approximately 400 000 participants in over 8 months), it was possibly the very nature of this study design and ease of app-based enrolment that may have led to high dropout rate and skewed patient demographic (average age 41 years) given the study was trying to assess the ability of a smartwatch application to identify atrial fibrillation during real-world use. Another trial of smart device-based intervention using a smartphone and social media based cardiac rehabilitation programme (via WeChat) for secondary prevention in China was able to increase participant engagement, knowledge about coronary heart disease and rated as a very acceptable tool, led to an improvement in 6-minute walk tests at 6 months, low density lipoprotein cholesterol and adherence to cardioprotective medications at 12 months.¹⁷ Whether this leads to an improvement in more long-term mortality and morbidity data while remaining cost-effective and acceptable to patients remains unclear.

Digital follow-up for revascularization trials in stable coronary artery disease

Data from the International Study of Comparative Health Effectiveness with Medical and Invasive Approaches (ISCHEMIA) and Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) trials show no prognostic benefit of an initial invasive strategy compared to an initial conservative strategy, in patients with moderate-to-severe burden of ischaemia.^18,19 This suggests that the primary remit for revascularization in patients with stable CAD (with preserved ejection fraction and no significant left main stem disease) is to improve symptoms.

Angina symptoms can be reported by the patient [e.g. with the Seattle Angina Questionnaire (SAQ)], assessed by the physician (e.g. with the Canadian Cardiovascular Society Class angina grading) or quantified through other modalities (e.g. exercise testing). Patient-reported outcomes are key endpoints of studies²⁰ and in stable CAD, outcome measures include the SAQ, Rose Angina Questionnaire and EuroQoL-5 (EQ-5D-5L). With symptomatic endpoints however, there is significant between-patient (inter-variability) and within-patient (intra-variability) variability at different timepoints, due to the multifactorial nature of symptom perception. Daily fluctuations in angina can be affected by physical, psychological, social and economic circumstances and can often go undetected in trial-related intermittent protocolized in-clinic evaluations.²¹

Hence, it is vital for standardized and reproducible qualitative and quantitative tools to contemporaneously record angina symptoms and their effect on patient quality of life. Digital smart device follow-up allows us to record this in ad libitum frequency and environment, providing us with better understanding of the nature, impact, and change in symptoms experienced by patients. This has the added advantage of reducing recall bias (e.g. the SAQ requires accurate independent recall of symptom occurrence and inducibility over a 4-week period).²² Increasing the frequency of data acquisition also improves the precision of the estimate and allows us to track the relationship of these symptoms with personalized daily functioning. This gold-standard method of daily documentation of angina has been used by previous trials²³ and used to validate current questionnaires, although through paper-based angina diaries. A further opportunity/benefit for collecting data via smartphone application will provide an opportunity to further validate angina questionnaires which have already shown to be discordant with physician assessments of angina.²⁴

Here, we describe our early insights into the use of remote digital smart devices in the clinical trials conducted by our research group at our institution, particularly focusing on its use for follow-up. We present the benefits and challenges faced in its generation and implementation along with our procedural workflow for others to consider implementing smart device follow-up into their trials.

Lessons learnt from ORBITA

The Objective Randomized Blinded Investigation with optimal medical Therapy of Angioplasty in stable angina (ORBITA) was the first placebo-controlled trial of percutaneous coronary intervention of stable coronary artery disease.²⁵ The primary endpoint was the difference in change in treadmill exercise time between the PCI and placebo groups. In this group of medically treated patients with severe single vessel coronary stenosis, PCI did not increase exercise time by significantly more than a placebo procedure.

ORBITA revealed no significant improvement with PCI beyond placebo, in symptoms as assessed by CCS class, or SAQ or quality of life metrics (EQ-5D-5L).²⁶ The only symptom endpoint where placebo-controlled efficacy of PCI was seen was in the non-prespecified analysis of SAQ freedom from angina in which 20% more patients in the PCI arm were free from angina at follow-up than in the placebo arm.²⁷

ORBITA-2 study design

To address whether PCI truly provides symptomatic relief in patients with stable CAD, ORBITA-2 (NCT03742050) will recruit 400 patients (double the size of ORBITA), including patients with multivessel disease, without intensive introduction and uptitration of antianginal medication,²⁸ and with a longer blinded follow-up period of 12 weeks.²⁹ At enrolment, regular antianginal medications are stopped to assess the sole effect of PCI versus placebo. If symptoms become intolerable, antianginal medications are re-introduced according to a pre-specified protocol. At randomization, any regular antianginals are stopped again and are restarted during the blinded follow-up phase using the same protocolized approach if symptoms are intolerable.

The full study design of ORBITA-2 is seen in Figure 1.

Figure 1.

Shows the study design of ORBITA-2. Following enrolment, daily symptom assessment using mobile application is required daily until exit from the trial.

A key element of ORBITA-2 is the incorporation of a patient-centred primary endpoint, a 79-level clinical outcome angina scale. This incorporates the daily anginal frequency recorded on the smartphone application, the presence or absence of angina each week during pre-specified activities, ‘units’ of antianginal medications, unblinding due to intolerable angina, acute coronary syndrome, and death. The incorporation of the smartphone application will allow this primary endpoint to be tracked over 12 weeks rather than at a single time-point.

Design and implementation of smartphone application

Here, we report our experience of incorporating smartphone/digital health-based technology into clinical trials.

STEP 1: patient involvement

Patient participation was central to the design of ORBITA-2. This was achieved by working with the previous participants and trialists in ORBITA, who formed a focus group. In fact, it was through these deliberations that the primary endpoint was adapted to reflect what was important to patients. Exercise time was considered relevant but did not capture the complete picture of their anginal health status.³⁰ Furthermore, on analysis of the secondary outcomes of ORBITA, it was symptoms rather than exercise time that displayed greater benefits. Hence ORBITA-2 is designed to provide PCI the best opportunity to demonstrate greater benefits over the placebo procedure.

Participants echoed the importance of accurately recalling their episodes of angina and previous questionnaires placed a large burden on them to do this (SAQ needed a 28-day recall period) and daily symptom reporting was proposed by the ORBITA focus group. Having also understood from the participants that they were limited by their angina in different ways and were having to make modifications to their daily activities and routines to cope with this, we employed personalized symptom reporting.³¹ As well as defining their subjective experience of angina themselves, participants will also pre-specify two activities that currently induce angina. Each week, participants perform these two activities and record if they have angina. This will allow us to understand if participants are angina free due to being truly asymptomatic or due to behavioural modification to avoid angina.³²

STEP 2: survey of clinicians

ORBITA benefited from the participation of cardiologists, both from large tertiary teaching hospitals and smaller district general hospitals, with extensive experience managing angina. It was important for us to incorporate their views on the best way of capturing patient angina symptoms. As science progresses in this era of digital technology, the views of clinicians on the best way of measuring angina will likely change.

We surveyed cardiologists for their opinion on the smartphone-based primary endpoint. There was a good consensus for the use of the ordinal scale, angina-based primary endpoint over the conventional angina questionnaires and exercise tests, which have formed the basis of previous unblinded trials advocating for PCI in medically treated patients with coronary artery disease.^33–35

STEP 3: creation of smartphone application

Following the deliberation and consensus from clinicians and patients, the smartphone application was designed and built by the research team. It is a web app, which allows it to work identically on iOS, Android, or other smartphone operating systems or on tablets or computers, as long as they have an internet connection.

The study team monitors developments in smartphone operating systems to ensure that the app continues to work correctly on current versions of all systems used by participants.

For each patient, clinical trial data is entered onto an electronic case report form on OpenClinica V4.0, an open-source clinical trial data management system. Smartphone data are, however, stored on a central server.

STEP 4: delivery and training of the smartphone application

Following an eligibility check and written informed consent at enrolment, patients are trained how to use the symptom application on their smartphone. Importantly, not having a smartphone is not an exclusion criterion as participants who do not possess a smartphone are provided one and taught how to use it.

Initially, patients are required to be set-up on the application with their unique identification number provided at enrolment. Next, they set a unique password only known to them as the second layer of identity confirmation (the first being their password to unlock their smartphone).

Following a successful one-time set-up, participants can login to the application. They are then required to answer some practice questions—for our application, these consist of three parts (Figure 2). These questions were designed to provide the participants familiarity with the user interface and be able to accurately describe their symptoms.

Figure 2.

Shows the three part practice question for participants during the set-up of the smartphone application.

STEP 5: daily input and maintenance of application

Participants are asked to input their symptoms into the application daily. These questions pertain to the previous day and include if they had angina (yes/no), frequency of these episodes, and severity of the worst episode (mild, moderate and severe, given a numerical grading from 0 to 600, shown in Figure 3).

Figure 3.

Shows the daily angina symptom questionnaire employed for the smartphone application. This will form part of the primary endpoint of the ORBITA-2 trial.

Along with the responses from participants, the application will notify the research team when the participants have failed to report their symptoms. We employed the traffic light system as a visual aid for research staff who prompt participants on this daily symptom reporting list (Figure 4). If three or more days are missed, participants will be prompted by the research staff to enter their symptoms (seen in red) via a standardized text message.

Figure 4.

Shows our traffic light prompting system for the participant symptom reporting app on the 30th of September 2021. The participant in red (highest on list) has not filled in their symptom application for 4 days (last data entry was 25th of September 2021 as shown in the left sided column) and has been reminded via text by a research staff. Participants in yellow (second and third on list) have not filled in their symptoms for two days. If these two participants do not fill in their questionnaire tomorrow, they will become red and a text message will be sent to them that morning.

If participants are struggling to fill in the questionnaire (be it with issues with their smartphone, application or internet connectivity), we have employed a live-assist function for researchers to manually input participant data over a phone call or email if needed. Researchers are able to troubleshoot the participant with their smartphone application issues during this time as well. The helpline number is provided to the participants during enrolment and is also clearly visible in the smartphone application under the Help section (Figure 5). This helpline number is constantly monitored by a clinical fellow who is blinded to treatment allocation.

Figure 5.

Shows the help section for the ORBITA-2 symptom application available to all participants.

Step-by-step flowchart for the design and implementation of our symptom application is shown in Figure 6.

Figure 6.

Shows our procedural workflow for incorporation of remote digital smart device follow-up in clinical trials conducted by our research group.

ORBITA-COSMIC and ORBITA-STAR

ORBITA-COSMIC (Coronary Sinus Reducer Objective Impact on Symptoms, MRI Ischaemia and Microvascular Resistance, NCT04892537)³⁶ and ORBITA-STAR (Symptomatic Trial of Angina Assessment Prior to Revascularization, NCT04280575)³⁷ are two independent clinical trials conducted by our research group to further our understanding of the link between coronary stenosis, ischaemia and angina and the impact of blinding on these endpoints.

ORBITA-COSMIC aims to investigate the mechanism of action of the coronary sinus reducer and its placebo-controlled impact on myocardial ischaemia, coronary flow, microvascular resistance, and symptoms in patients with refractory angina who have no further options for revascularization. The primary endpoint is a change in myocardial perfusion reserve on MRI between the groups at 6 months, and we aim to recruit 40 patients to this trial.

ORBITA-STAR aims to determine whether symptoms, induced by confirmed experimental ischaemia, can help us predict which patients will respond to PCI. The primary endpoint is a placebo-controlled similarity and intensity scores correlated to change in angina symptom score at 8 weeks following PCI.

The study design of ORBITA-COSMIC and ORBITA-STAR are seen in Figures 7 and 8, respectively.

Figure 7.

Shows the study design of ORBITA-COSMIC. Smartphone and Apple Watch^™ are provided to patients at enrolment and used to monitor patents until study completion.

Figure 8.

Shows the study design of ORBITA-STAR. Smartphone is provided to patients at enrolment and also used to monitor patents until study completion.

Similar to ORBITA-2, ORBITA-COSMIC and ORBITA-STAR include a symptoms assessment phase prior to the randomization invasive procedure in which patients will record their daily symptoms on the smart-phone application for a 2-week and 4-week period respectively. Again, patients who are asymptomatic during this phase are excluded from the trials. These trials also follow up patients with daily symptom reporting.

Both ORBITA-COSMIC and ORBITA-STAR incorporate the Apple Watch^™ to understand the daily activity level of participants. For example, participants in ORBITA-COSMIC are given an Apple Watch^™ to be worn daily for the whole duration of the study. Apple Watch^™ activity measures (steps per day, calories burned per day, flights climbed per day, and daily heart rate variability) will be collected as secondary endpoints.

With this, we can capture more personal, individual level data on the daily function, duration and intensity of activity levels and angina symptoms. These data allow us to further interrogate if patients are exerting themselves to reproduce these anginal symptoms and this forms an important confounding factor to consider in outcome trials for intervention in stable CAD and contributes to Hawthorne bias. Next, we will be able to further our understanding of the effects of intervention. Heart rate, haemodynamic responses and inducible symptoms have been shown to improve following PCI in unblinded trials. However, at single time points in any trial, these conclusions can equally be problematic. With more data points from wearables, we can more precisely determine this. Furthermore, we hope to determine from these parameters which patients are more likely to benefit from PCI. In unblinded trials, symptomatic benefits for use of PCI exist for those with frequent angina.³⁵ The use of step count has also been suggested as a marker of improvement in treating angina.³⁸ However, would it be participants who are highly symptomatic from a high activity level correlating with their angina, or those that describe significant angina on minimal activity duration and intensity, that will benefit most from this invasive procedure?

Limitations and challenges of smartphone/wearable use for patient follow-up

We have found that remote monitoring of certain endpoints (e.g. daily monitoring of symptoms and activity levels), to allow us to achieve our trial objectives, is both cost-effective and convenient to us as the research team and to patients. This is particularly important for our central trial group to obtain comprehensive data collection on large sample sizes, who are enrolled from multiple centres. During the COVID-19 pandemic, when patient facing elements of the trial protocol (such as exercise testing and stress echocardiography) were minimized, we were able to continue smartphone monitoring of anginal symptoms and anti-anginal use which is the primary endpoint of ORBITA-2.

However, key limitations remain for remote smart device use in participant follow-up. Firstly, it is not suitable for all endpoints. As described, exercise testing, echocardiography and other invasive procedures require in-person assessment. However, digital technology could assist in the data collection of these endpoints. For example, many devices such as blood pressure monitors, fitness trackers and weighing scales can connect directly to smartphones and store this data.

Incorporation of smartphone or digital technology follow-up potentially introduces a further selection bias in already selective clinical trial populations, towards those with technological literacy (typically a younger population). Ease of smart device usage was an important consideration in our trials and app utility was discussed with the focus group. Participants were specifically taught how to use the device during enrolment and were followed-up more closely during this initial period through the daily symptom reporting list to determine if any participants are struggling with their smart devices. Those who remain unable to operate their device are asked to record their symptoms manually (on paper or diary) and research fellows periodically contact participants to transcribe their results onto the application. We have found that so far, almost all participants have been successfully able to operate their device and provide meaningful data. Furthermore, with the inevitable progressive increase in uptake of and familiarity with smartphone usage, this issue will hopefully dissipate overtime.

Design of a smartphone application can be time-consuming and requires constant monitoring from the app development perspective. There are multiple operating systems which evolve over time. At any one time there will be many versions of each operating system in use. Within each operating system there are many devices which have different capabilities, screen resolutions and shapes, each of which needs to display the app appropriately. For monitoring purposes, this also requires surveillance and prompting beyond standard working hours. In our trials, the helpline number is available all hours of the day and there is daily monitoring of the smartphone application to prompt participants if there is delay in symptom reporting.

It can also be difficult to determine if responses recorded on these remote devices are independently from the patient. Anyone with the password or access to the smart device could in theory respond. Where it may be more pertinent is when carers or family members partake in the reporting on this application with the participants. It is then difficult to distinguish what the patient truly feels and what other individuals believe the patient is feeling and this is a key limitation of any form of remote reporting. Participants and their carers can be briefed at enrolment on the importance of this.

Finally, digital health follow-up creates challenges in the efforts to ensure privacy and safety of the data generated. In ORBITA, data from the smartphone are stored on a central server. Breach of confidential databases remains a risk even with methods of de-identification as meta-data associated with the user can theoretically be used to re-identify them.³⁹ Methods to mitigate this risk include the use of distributed ledgers, such as blockchain, or decentralized databases.⁴⁰ Patient views on their personal data and what the default privacy and security standards for sharing this data should be, will depend on societal principles. An opt-in system for stringent standards of data security and privacy although might cause some not to protect their privacy as much as they would like, it could however meaningfully improve patient engagement with the health care systems and therefore improve health outcomes whilst respecting patient autonomy.⁴¹

Conclusion

Digital health and smart devices utilization have allowed us to develop personalized, effective, and safe healthcare that is more accessible to the global population. Its incorporation in clinical trials has also furthered the remit of our best evidence generating methods, randomized double-blinded controlled trials. In ORBITA-2, ORBITA-STAR, and ORBITA-COSMIC, we have employed daily digital smart device follow-up on smartphones and smartwatches. This should provide a clinically meaningful and patient-centered approach to determine whether PCI or other coronary interventions truly provide symptomatic relief in patients with stable CAD, and most importantly will allow us to accurately understand the perception and implications of angina on patients’ lives.

Funding

This paper was published as part of a supplement sponsored by Abbott.

Data availability

No new data were generated or analysed in support of the present work.