Development and usability testing of HEARTPA♀N: protocol for a mixed methods strategy to develop an integrated smartphone and web-based intervention for women with cardiac pain

Abstract

Introduction

More women experience cardiac pain related to coronary artery disease and cardiac procedures compared with men. The overall goal of this programme of research is to develop an integrated smartphone and web-based intervention (HEARTPA♀N) to help women recognise and self-manage cardiac pain.

Methods and analysis

This protocol outlines the mixed methods strategy used for the development of the HEARTPA♀N content/core feature set (phase 2A), usability testing (phase 2B) and evaluation with a pilot randomised controlled trial (RCT) (phase 3). We are using the individual and family self-management theory, mobile device functionality and pervasive information architecture of mHealth interventions, and following a sequential phased approach recommended by the Medical Research Council to develop HEARTPA♀N. The phase 3 pilot RCT will enable us to refine the prototype, inform the methodology and calculate the sample size for a larger multisite RCT (phase 4, future work). Patient partners have been actively involved in setting the HEARTPA♀N research agenda, including defining patient-reported outcome measures for the pilot RCT: pain and health-related quality of life (HRQoL). As such, the guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols (SPIRIT-PRO) are used to report the protocol for the pilot RCT (phase 3). Quantitative data (eg, demographic and clinical information) will be summarised using descriptive statistics (phases 2AB and 3) and a content analysis will be used to identify themes (phase 2AB). A process evaluation will be used to assess the feasibility of the implementation of the intervention and a preliminary efficacy evaluation will be undertaken focusing on the outcomes of pain and HRQoL (phase 3).

Ethics and dissemination

Ethics approval was obtained from the University of Toronto (36415; 26 November 2018). We will disseminate knowledge of HEARTPA♀N through publication, conference presentation and national public forums (Café Scientifique), and through fact sheets, tweets and webinars.

Trial registration number

NCT03800082.

Strengths and limitations of this study.

• Robust methods guided by the individual and family self-management theory, mobile device functionality and the sequential phased approach recommended by the Medical Research Council.

• Sustainable HEARTPA♀N design and development based on the real needs of women with oversight by a Patient Advisory Committee.

• Extensive recruitment and solid retention strategies using gender and culturally sensitive research methods.

• Larger pilot randomised controlled trial (RCT) focused on feasibility and primary evaluation of efficacy will increase precision of estimates and provide robust data to inform the design of a future full-scale RCT.

Introduction

Cardiac pain is a key symptom of coronary artery disease (CAD) and acute coronary syndrome. Women have a varied pattern and distribution of cardiac pain and/or cardiac pain symptoms associated with both obstructive (macrovascular) and non-obstructive (microvascular) CAD. Women with obstructive CAD are usually 7–10 years older than men¹ and present with coronary atherosclerosis and risk of atherosclerotic plaque rupture and/or erosion.² Compared with men, women with obstructive CAD who undergo a percutaneous coronary intervention (PCI)³ and/or cardiac surgery^4–6 have more persistent pain of moderate to severe intensity.⁷ The origin of this pain is complex, and thought to be pathophysiologic (eg, scar tissue, damage to intercostal nerves) and/or psychological (eg, anxiety) in origin.⁸ Non-obstructive CAD is cardiac pain without evidence of coronary artery obstruction,⁹ defined as less than a 50% epicardial coronary lesion on angiography.¹⁰ Coronary microvascular dysfunction/coronary spasm and coronary microembolism also contribute to ischaemia in non-obstructive CAD.¹¹ Recent evidence suggests that up to 67% of women who present with cardiac pain and/or cardiac pain symptoms have ischaemia related to non-obstructive CAD.¹² Non-obstructive CAD is more prevalent in younger, middle-aged women and evidence suggests that more extensive, non-obstructive CAD is associated with major adverse cardiac events similar to those with obstructive CAD.¹³ Obstructive/non-obstructive CAD is the leading cause of death of women across all ages, and recent data show an increase in CAD incidence and deaths among women 45–54 years of age.¹⁴

Many women describe typical obstructive and non-obstructive cardiac pain as tight, heavy and dull with additional symptoms that include nausea and palpitations,¹⁵ and/or dyspnoea, weakness and unusual fatigue.¹⁶ Women also report that their cardiac pain is more likely to radiate to their left arm, back and/or jaw and neck.¹⁵ Women describe persistent poststernotomy pain as aching, tender and exhausting.⁴ This varied pattern and distribution of symptoms make it difficult for women to interpret as cardiac related (ie, obstructive/non-obstructive or post-PCI/cardiac surgery).^{13 17 18} Women also minimise symptoms, prefer to consult with family and friends and have caring responsibilities and concerns for their family.¹⁹ As a result, women delay seeking appropriate care for their cardiac pain.²⁰ The time from symptom onset to emergency department (ED) arrival for women is 85–320 min, this has not changed in the last decade.²¹

Women with cardiac pain due to obstructive/non-obstructive CAD and/or post-PCI/cardiac surgery pain are frequent users of healthcare services²² and at risk for impaired function, depression, poor health-related quality of life (HRQoL) and death.²³ Women have been historically under-represented in cardiovascular clinical trials,^{24 25} with much of the current evidence comparing cardiac pain and/or cardiac pain symptoms in women to men. There is little evidence focused on interventions to assist women to recognise and manage cardiac pain and/or cardiac pain symptoms.²⁶ Self-management interventions allow people to take an active part in the management of their own conditions²⁷ and are important predictors of successful behaviour change.²⁸ In addition to reducing pain, self-management interventions improve HRQoL.^29–34 A current mixed methods systematic review of self-management programmes (HEARTPA♀N, phase 1), which included women greater than 18 years of age with cardiac pain, found self-management interventions for cardiac pain were more effective if they included a greater proportion of women (p=0.02), goal setting (p=0.03) and collaboration/support from healthcare providers (HCP) (p=0.01).³⁵ Mobile health (mHealth) technologies have been developed to help women self-manage weight,^36–40 increase physical activity,⁴¹ monitor for perinatal depression and assist with postpartum smoking cessation.⁴² Many women view mobile health technologies as novel and supportive,³⁶ and indicate these technologies motivate healthy behaviours, reduce symptoms⁴³ and improve HRQoL.⁴³ Health app usage across all ages is on the rise,^{44 45} yet there is little objective rigorous research evaluating outcomes of smartphone-based interventions.⁴⁶ The benefits of mHealth interventions in healthcare are compelling; smartphones are portable, they offer connectivity and they provide access to women who are difficult to reach, yet no smartphone or web-based self-management programme has been developed and tested with women who have cardiac pain and/or cardiac pain symptoms.

Objectives

The overall goal of this programme of research is to develop and systematically evaluate an integrated smartphone and web-based intervention (HEARTPA♀N) to provide evidence-informed symptom triage and self-management support to reduce pain and increase HRQoL in women with cardiac pain and/or cardiac pain symptoms. Specific objectives for each phase of development/evaluation include: (1) develop the HEARTPA♀N content and core feature set (phase 2A), (2) conduct usability testing (phase 2B), and (3) assess feasibility in terms of implementation (accrual rates, acceptability and level of engagement) and determine an initial estimation of effectiveness outcomes (estimates of magnitude of effect) in a pilot randomised controlled trial (RCT) (phase 3). The phase 3 pilot study will enable us to refine the prototype, inform the methodology and calculate the sample size for a larger multisite RCT (phase 4, future work).

Methods and analysis

Phases 2A and 2B

We are using the individual and family self-management theory,^{47 48} mobile device functionality and the pervasive information architecture of mHealth interventions,⁴⁹ and following the sequential phased approach recommended by the Medical Research Council (MRC)^50–52 and used by Stinson and others^{52 53} to develop HEARTPA♀N. We will develop the HEARTPA♀N content/core feature set and conduct usability testing (phases 2A and 2B) to ensure it is easy to use, efficient and satisfying to operate.

Study design

Focus group interviews in phase 2A will assist to: (1) learn about the experiences and healthcare needs of women with cardiac pain/cardiac pain symptoms from the perspectives of women, (2) design content and the core feature set of HEARTPA♀N, and (3) validate the HEARTPA♀N triage algorithms with HCPs. This feature set will include evidence-informed symptom triage algorithms to help women recognise their cardiac pain/cardiac pain symptoms and seek appropriate care. The usability testing in phase 2B will focus on user performance (ease of use, efficiency, ease of learning and errors) and satisfaction with programme content and functionality (reports, goal setting).⁵⁴

Eligibility criteria

Women living in Canada greater than 18 years of age with obstructive/non-obstructive CAD pain and/or pain post-PCI/cardiac surgery lasting greater than 3 months. All women will be required to speak and read English and will be excluded if they have severe cognitive impairment assessed using the Six-Item Screener administered by telephone or in face-to-face interview,^{55 56} or major comorbid medical or psychiatric illness that could preclude their ability to participate in an interview. HCPs will include physicians and nurses/nurse practitioners who have worked in cardiology, family medicine or in an ED for at least 1 year; trainees, whose presence in the clinical setting is often transient, will be excluded.

Study setting

Phase 2A 1-hour focus group interviews will be scheduled at a mutually convenient time for participants, and conducted by telephone, using ZOOM online video conferencing technology, or face to face in a location suitable to participants and free from distractions.⁵⁷Phase 2B participants will complete a one-on-one observation for 60–90 min in a quiet room within the labs at Healthcare Human Factors in Toronto, Ontario, Canada.

Procedures

Phase 2A. Following ethics approval, a purposive sample of women with obstructive (n=10)/non-obstructive (n=10) CAD pain and post-PCI/cardiac surgery pain (n=10) will be recruited for focus group interviews through cardiology, cardiac surgery and pain clinics, and using social media platforms (eg, Twitter, Facebook). HCPs (n=10) will also be recruited for a separate focus group interview via letters and emails. We will use a semistructured interview guide to explore the views, experiences and beliefs/motivations⁵⁷ of women with cardiac pain. We will also use a semistructured interview guide to validate the triage algorithms with HCPs. Interviews will be conducted by two team members experienced in conducting interviews and techniques will be used to minimise power differentials, such as establishing rapport, active listening and relaxed body language.⁵⁸

Phase 2B. Based on previous experience^59–61 and recommendations that usability testing by three to five users finds approximately 85% of interface usability problems,^{62 63} each usability cycle will include five end users (per pain type—obstructive (n=5)/non-obstructive (n=5) CAD, and pain post-PCI/cardiac surgery (n=5)). Women will be provided with a brief explanation of the HEARTPA♀N intervention and then asked to move through standardised scenarios and list of features including the about you, event profile, goal-setting features, graphics, audio and video clips and interactive components (reporting, symptom triage algorithms, self-management skills). We will employ a ‘think aloud’ approach⁶⁴ to gather insight into the way users solve problems as they move through the application in a systematic way. Comments will be recorded, and the project coordinator will make field notes about any problems encountered on the Usability Testing Error and Efficiency Documentation Form. At the end of the session, participants will be asked to complete the System Usability Scale (SUS).⁶⁵ The SUS has been used across a wide range of user interfaces, including web pages and web applications.⁶⁶ The ten 5-point Likert questions can be scored to provide a point estimate of usability with a reported reliability of 0.85.⁶⁷ In addition, four semistructured questions will be asked to determine users’ overall impression of HEARTPA♀N, what they liked and why, what could be improved and if anything was missing.⁶¹ Observations will be conducted in iterative cycles. After the first cycle, changes will be made to the interface based on comments from the content analysis of the audiotapes and field notes. The revised user interface will then be evaluated in a subsequent cycle. These iterations usually require two to three testing cycles with each end user group until no further comments are identified.^{61 62 67}

Outcomes

We will use the summary matrix from our integrated mixed methods systematic review (phase 1)³⁵ and the results from focus group interviews (phase 2A) to discuss HEARTPA♀N designs with women who have cardiac pain in a consensus workshop with our Human Factors Designers. HEARTPA♀N will be designed for a consistent experience for women, and developed on a web-based platform, with easy access on any device with a web browser, including smartphones and tablets.^{68 69} HEARTPA♀N’s web-based approach will allow for faster maintenance, easier updates to content, as well as improved accessibility for users.⁷⁰ All HEARTPA♀N contents will be written at a grade 5–6 reading level,⁷¹ and communication with a central database server will occur through secure internet connections. Women with cardiac pain will be able to participate in realistic scenarios in a simulated environment (phase 2B) in order to assess the appropriateness and ease of use of HEARTPA♀N prior to the phase 3 pilot RCT.

Phase 3

The HEARTPA♀N intervention is the first of its kind; there are no previous trials of the efficacy of such an intervention to decrease pain and improve HRQoL in women with cardiac pain/cardiac pain symptoms. We will undertake a process and preliminary effect evaluation of the HEARTPA♀N intervention for women with cardiac pain, as guided by the MRC framework.^50–52 The guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols (SPIRIT-PRO) are used to report the protocol for this pilot RCT.

Study design

A two-group parallel single-blind pilot RCT.

Eligibility criteria

Inclusion/exclusion criteria have been previously described (phases 2A and 2B). Additional exclusion criteria will include women who participated in phase 2A or phase2B studies.

Study setting

Participants will attend one in-person session to learn about the trial, obtain informed written consent and complete demographic, clinical and baseline measures (T1). Participants allocated to the intervention group will also learn how to use the HEARTPA♀N intervention. The intervention will be delivered on restricted password-protected applications.

Procedures

Following ethics approval, a single coordinating centre (University of Toronto) will recruit women using the methods described previously (phases 2A and 2B). Interested participants will contact the project coordinator by telephone or express their interest using the HEARTPA♀N website. Eligibility criteria will be confirmed, verbal consent obtained and an appointment for an initial study visit will be made. The project coordinator will track the number of eligible participants approached and reasons for refusal using a study log. We will use multiple methods to promote recruitment and retention, such as reimbursing participants for travel costs related to the initial study visit and reimbursement for use of their smartphone and data plan ($85) for the duration of the study. The project coordinator will send email and postcard reminders and at 3 months, participants will be telephoned (standardised script) to complete post-test measures online at home. Gift cards will be provided at study completion ($25). We anticipate minimal loss to follow-up as reported in previous pilot studies.⁷² However, logins every 1–2 days for 3 months may be burdensome for women, which we will assess in our process evaluation. The project coordinator will also be available to address questions, issues and concerns without delay and all T2 assessments will be completed online, eliminating the need for participants to return to the study centre.

Randomisation. Following completion of baseline measures, participants will be randomised to the control or intervention group at a 1:1 ratio in blocks of 4 stratified by type of cardiac pain^{73 74} (obstructive CAD, non-obstructive CAD and post-PCI/cardiac surgery). Randomisation will be managed centrally using a web-based randomisation service (www.randomize.net/).

Allocation. Participants allocated to the control group will receive the usual care and supports provided to women with cardiac pain/cardiac pain symptoms, including usual clinic appointments and follow-up. With detailed informed consent procedures, it is expected that women will accept their group allocation following randomisation. Participants randomised to the intervention group will consist of use of the HEARTPA♀N intervention every 1–2 days, in addition to usual care, for a period of 3 months. The HEARTPA♀N intervention will be delivered on restricted password-protected applications that will permit tracking of adherence (number of logins to app and website using Google Analytics). Participants will be encouraged to log in to HEARTPA♀N every 1–2 days (via automated alerts) over the 3-month period to develop and track goals related to pain, activities, sleep and emotions. Participants will be directed to the project coordinator for technical problems.

Blinding. It is not possible to blind the participants to group allocation due to the specific nature of the HEARTPA♀N intervention; however, a data analyst at the University of Toronto’s Faculty of Nursing who is blinded to treatment allocation will conduct the analysis ensuring neutrality of the outcome assessment.

Outcomes

A process evaluation will be used to assess the feasibility of the implementation of the intervention. Recruitment and retention will be determined through the use of the study log, which will document each potential participant contacted, whether or not they chose to participate in the trial, reasons for non-participation, whether or not they completed follow-up assessments and reasons for dropout. Issues and/or difficulties encountered during trial implementation will be tracked. Adverse events will be recorded on an Adverse Event Form and engagement will be assessed using Google Analytics. We will assess acceptability and satisfaction at the end of the 3-month period in all participants in the intervention group using a modified Acceptability e-Scale (AES).⁷⁵ A preliminary efficacy evaluation will also be undertaken focusing on the outcomes of pain and HRQoL. Pain will be measured using the Brief Pain Inventory-Short Form (BPI-SF), which rates pain severity and the degree to which pain interferes with mood, sleep and other physical activities such as work, social activity and relations with others. It has good construct validity,^{76 77} reliability is reported at 0.86–0.91⁷⁶ and it has detected clinically important differences.^{4 72 78} HRQoL will be measured using the SF-36v2, which contains 36 items and yields a score for each of the eight domains of health: physical functioning, role limitations due to physical health (role—physical), bodily pain, general health perceptions, vitality, social functioning, role limitations due to emotional problems (role—emotional) and mental health.⁷⁹ It has an internal consistency of 0.76–0.94^{80 81} with construct, criterion and predictive validity.⁸¹ A participant flow diagram is included in figure 1.

Figure 1.

Anticipated participant flow-through in pilot randomised controlled trial (RCT). CABG, coronary artery bypass graft; PCI, percutaneous coronary intervention.

Sample size

As this is a pilot trial focused on feasibility and primary evaluation of efficacy, we are not testing for statistical significance.⁸² To decide on a sample size, we used the CI approach based on the feasibility outcomes of recruitment and retention. For a one-sided 95% CI for the proportion of women recruited and a margin of error of 0.05 (the lower bound) we would need at least 81 participants to estimate an overall recruitment rate of 0.70. A sample of n=49 would be needed to estimate an overall retention rate of 85%; however, to estimate retention separately in the intervention and control groups, we will need a total sample size of 98 (49*2). As attrition is one of our measured feasibility outcomes, we have not accounted for it in the sample size calculations.

Data management

Data will be collected using the HEARTPA♀N application, as well as surveys and stored on a password-protected server. The trial steering committee includes all research and Patient Advisory Committee (PAC) team members. As this is a pilot trial, there is not a separate data monitoring and safety committee.

Statistical methods

Process evaluation. Prevalence of refusal, retention, engagement with the intervention and technical difficulties reported will be calculated, along with their 95% CIs. Mean acceptability and satisfaction will be calculated from the total score of the AES, along with its SD. We will record symptom descriptions and use of the symptom triage algorithms, what women did as a result of this recommendation (eg, self-management, contact with primary HCP, ED visit). Qualitative process data collected will be analysed using methods appropriate to the data obtained.

Primary effect evaluation. We will investigate the variability and sensitivity to change for outcomes of pain and HRQoL (T2-T1). We will calculate the number of participants who report clinically meaningful decreases in pain, which has been defined for the BPI-SF as a 2-point difference in worst pain.⁸³ Variability will be estimated using the mean/median scores and SD, in each group separately, at pretest and post-test. Similarly, sensitivity to change will be assessed by determining the number of participants who had a clinically meaningful increase in HRQoL scores over time. Although the study will not be powered to detect significant differences, we will use multiple regression to estimate the effect of group allocation on each outcome (separately) at post-test, adjusting for baseline scores. This will help determine the magnitude and direction of effect and provide a signal of the intervention’s effectiveness. The analysis will be conducted using an intent-to-treat approach. As this is a pilot trial, no interim analyses are planned.

Patient and public involvement

Seven women (LC, CA, CF, DP, MP, BR, VSD) with cardiac pain formed the HEARTPA♀N PAC. They were actively involved in phase 1 of this research programme (eg, defining search terms for our systematic review) and continue to be actively involved in setting the HEARTPA♀N research agenda for phases 2A, 2B and 3. This includes assisting to define the scope of the project (eg, defining patient-reported outcome measures for the pilot RCT), active involvement in recruitment activities, assisting to write project quarterly newsletters and participation in all team meetings. They will be invited to be copresenters at scientific conference meetings and public forums (Café Scientifique) and will assist to write lay summaries and fact sheets for each phase of our project.

Ethics and dissemination

Ethics approval was obtained from the University of Toronto (36415; 26 November 2018). This is a 3-year study, phase 2A recruitment began in March 2019. Informed consent will be obtained from participants (online supplementary material 1). To ensure privacy during the pilot RCT, all personally identifying information will be stored on a separate database from health data on the app. Information that is sent to the smartphone or used by the reporting system will be independent of their personal information. No personal information will be transmitted after the initial set-up. For security issues, information that is transmitted will be sent securely via encrypted Hypertext Transfer Protocol Secure connection, preventing interception by a third party. All electronic entries will be backed up on a central server and communication with the central database server will occur through secure internet connections. Only the principal investigator and project coordinator will have access to the data. We will disseminate knowledge of HEARTPA♀N through publication, conference presentation and educational national public forums (Café Scientifiques), and through fact sheets, tweets and webinars posted in the Women’s Xchange Knowledge Translation and Exchange Centre as well as to key stakeholders and programmes.