GOALS AND VISION OF THE PROGRAM
Angina predicts outcomes from CAD, including mortality, morbidity, reduced quality of life, and increased health care spending (1). However, patient symptoms are underreported and incompletely documented, limiting understanding of the efficacy of therapies (2). Angina can be reliably assessed using patient reported outcomes measures (PROMs): short, well-validated questionnaires assessing symptoms and quality of life (3–4). However, implementation barriers continue to hinder routine use of PROMs. We describe two strategies designed to evaluate the feasibility of PROMs assessment. In Strategy #1, we implemented a system for collecting the Seattle Angina Questionnaire-7 (SAQ-7), 7 questions evaluating angina; the Rose Dyspnea Scale (RDS), 4 questions evaluating dyspnea; and the Patient Health Questionnaire-2 (PHQ-2), two questions evaluating depression, via electronic self-administered surveys prior to elective coronary angiography at Beth Israel Deaconess Medical Center (BIDMC), and at three follow-up times. In Strategy #2, we created a system for phone-based administration of the SAQ-7 in CAD patients receiving care at most of Massachusetts General Hospital’s (MGH) primary care clinics and affiliated health centers.
DESIGN AND IMPLEMENTATION OF THE INITIATIVE
Strategy #1
We conducted a pilot study, 12/18/16-1/19/18, administering web-based versions of the SAQ-7, RDS, and PHQ-2 via tablets to patients presenting for non-emergent coronary angiography (5). Catheterization lab nurses and front-desk staff provided tablets to patients in the waiting room or pre-procedure holding area. Tablets were configured to display the survey and a brief study explanation with minimal assistance. Responses were transmitted via WiFi to an encrypted REDCap database, and uploaded by a research coordinator (L.R.V.) as portable document format (PDF) files to the electronic health record (EHR). A project request, currently pending, was submitted to BIDMC to create a workflow to automatically upload surveys. Patients desiring to use their own mobile device were given a survey link. Participants were automatically sent emails at 30-days, 6-months, and 12-months following the procedure with requests to complete the questionnaire again. Non-responders were contacted by phone by a research coordinator (L.R.V.). Catheterization lab, nursing, and front desk staff and clinicians (N = 40) attended a 10-minute training by study staff prior to study initiation to learn about the goals of the initiative, their role in its conduct, and advice for troubleshooting (eFigure 1–2). Attendance was documented and efforts were made to contact those unable to attend. Nursing and administrative leadership were included in all planning meetings.
Strategy #2
We conducted a pilot study, 2/1/17-7/31/17, to assess the feasibility of administering the SAQ-7 to MGH primary care patients ≥ age 30 with CAD receiving care through one of 15 primary care clinics and community health centers. Patients were excluded if dementia was documented in the EHR, or they were deemed unable to complete a telephone survey due to dementia (ascertained based on caregiver or family report), hearing impairment, or language barrier.
We obtained a list of patients with CAD from the MGH Primary Care Practice Based Research Network (PBRN), a database containing EHR data for 161,000 patients receiving primary care through the MGH network. SAQ-7 surveys were administered by trained population health coordinators (PHCs) as an extension of their current work. Seven full-time PHCs are employed by MGH, each working with 1–3 MGH primary care clinics, ensuring patients receive appropriate disease screening and reach disease targets. Our strategy was to expand the pool of PHCs so each would be able to commit time to survey administration weekly. We hired an eighth PHC who assumed responsibility for part of the PHCs’ workload, enabling each PHC to devote six hours weekly to survey administration.
Seven PHCs administered surveys in English, one in Spanish. One study investigator (D.M.B.) administered up to ten surveys weekly on evenings or weekends per patient request. PHCs were given the option to administer the SAQ-7 via secure email (“Patient Gateway”) per preference. PHCs were instructed to perform ≥ 1 follow-up call for patients not answering or returning a voice message within 24 hours. All responses were recorded in the EHR. We aimed to survey 50 patients from each of the 15 primary care clinics. We drafted scripts and protocols for contacting patients, administering surveys, and recording survey responses in the EHR, and trained PHCs in their use (eFigures 4–10).
For patients reporting concerning symptoms, we developed a protocol to promptly identify them and contact a cardiologist to evaluate need for urgent medical evaluation. These included survey responses or informal patient reports of daily, severe, and/or progressive angina, dyspnea, increasingly frequent episodes of angina, or rest angina.
SUCCESS OF THE INITIATIVE
Strategy #1
As of January 19, 2018, a total of 613 baseline surveys were collected on 1,448 elective coronary angiography procedures (average response rate 42.3%). Of the 613 patients completing a baseline survey, 156 (25.4%) underwent PCI with the remainder undergoing coronary angiography alone (eFigure 3). Of those eligible for follow-up at the time of this report, 56.1% (331/590) completed the 30-day survey, 50.3% (186/370) completed the 6-month survey, and 60.6% (40/66) completed the 1-year survey. A total of 479 respondents (77.0%) provided an e-mail address on the baseline survey, but only 27.3% (161/590) of participants completed a follow-up survey by e-mail at 30 days, 23.5% (87/370) at 6 months, and 22.7% (15/66) at one year. The remaining surveys were conducted by phone, representing slightly more than half of completed follow-up surveys.
The main reason for non-completion of follow-up surveys was lack of response to e-mail and phone contact, accounting for 34.4% (203/590) of participants at 30 days, 36.5% (135/370) at 6 months, and 21.2% (14/66) at 1 year. The majority of other non-responders either declined the survey (30 days: 12/590; 6 months: 16/370; 1 year: 4/66), did not speak English (30 days: 10/590; 6 months: 7/370; 1 year: 4/66), or provided an incorrect e-mail address or phone number (30 days: 15/590; 6 months: 23/370; 1 year: 2/66). A minority were hard of hearing or cognitively impaired (30 days: 3/590; 6 months: 2/370; 1 year: 1/66), or had died (30 days: 2/590; 6 months: 1/370; 1 year: 1/66). Respondents were healthier than non-respondents, and those with a positive stress test were more likely to respond to the survey (eTable 1). Thus, it is possible that expectation of need for PCI could have influenced participation.
Strategy #2
PHCs intended to survey 4,789 CAD patients, but excluded 687 patients (14.3%) at the time of survey administration (553 [11.5%] due to no longer seeing an MGH-affiliated PCP, being in hospice, a nursing home, or hospitalized; 77 [1.6%] for hearing impairment; 53 [1.1%] for dementia; and 4 [0.1%] for difficulty with speech). Of the 4,102 remaining individuals, 1,612 (34%) completed the SAQ-7 (eTable 2). Of these, 1,598 (99.1%) completed the SAQ-7 by phone and 14 (0.9%) used the Patient Gateway (eFigure 4). Among the 2,490 non-responders, reasons for non-response included: 900 (28.3%) did not answer the phone; 888 (27.9%) did not return a voice message from the PHC; 408 (12.9%) refused or did not complete the survey; 213 (6.7%) requested follow-up at a later date but never completed the survey; and 37 (1.2%) did not respond to the SAQ-7 when administered via Patient Gateway. In 44 cases (1.4%), the EHR contained inaccurate contact information. We surveyed at least 50 patients for 14 of 15 participating primary care clinics. Median clinic-specific and PHC-specific survey completion rates were 34.9% (range: 16.7%─51.1%) and 35.2% (range: 9.3%─46.2%), respectively. Eighteen surveys (1.0%) met criteria for a “concerning survey response.” Follow-up within one week was arranged for two patients; none went to the emergency room.
LOCAL CHALLENGES IN IMPLEMENTATION
Strategy #1
The most significant barriers were technological. Tablets required reconfiguration daily to ensure data privacy, making integration into workflow difficult. Additionally, patient disability or poor dexterity limited electronic PROMs completion. Patients were allowed to use their own device to complete surveys, and family members were encouraged to help, but many declined or elected to complete a paper version. Additionally, PROMs were only available in English, a barrier to non-English speakers.
Strategy #2
Contacting patients proved challenging, as many patients were busy during daytime hours. Language barriers also limited administration to non-English, non-Spanish speaking patients. Additionally, PHCs were not initially prepared to explain to asymptomatic patients reasons for completion, so some patients refused to complete the survey. PHCs were subsequently trained to provide an explanation to these patients, mitigating this barrier. The authors are considering further research to understand reasons for the differential response rates among PHCs.
TRANSLATION TO OTHER SETTINGS
Phone-based administration of PROMs may be particularly useful for evaluating symptoms longitudinally among several patient populations. Health systems participating in risk-based contracts including a mandate to assess patient-reported health status may also find it useful to develop web- or phone-based PROMs programs. Further comparison of web vs. phone-based strategies is warranted to elicit which strategies work best locally.
SUMMARY OF THE EXPERIENCE, FUTURE DIRECTIONS, AND CHALLENGES
We describe the motivation, design, implementation, and early outcomes of two complementary programs to evaluate the feasibility of collecting PROMs in the inpatient and outpatient setting among patients with CAD (Figure).
Figure: Challenges of PROMs Implementation in Inpatient and Outpatient Settings.
Venn-Diagram illustrating the overlap between challenges experienced in both inpatient and outpatient settings with provision of PROMs surveys to cardiovascular patients.
In Strategy #1, PROMs were administered by tablet to patients undergoing coronary angiography. Response rates were reasonable (42.3%), and the innovation demonstrated that PROMs can be administered in a semi-automated fashion in a busy catheterization laboratory. Challenges included competing responsibilities among staff, hospital technological requirements that didn’t integrate well into workflow, and lack of incentives for participation. While feedback was given on low response rates to staff, the perception of increased workload was a sufficient barrier to implementation. Whether staff incentives for participation could improve response rates is unknown.
In Strategy #2, the SAQ-7 was administered by phone to a population of primary care patients with stable CAD, with two findings: 1) the program’s 34% response rate compares favorably with the 15–20% response rates achieved when the SAQ-7 is administered via tablet device to patients during outpatient clinic appointments with MGH providers (unpublished data), and 2) the infrastructure for PROMs administration can be embedded within existing population health management programs. Challenges included contacting patients, responding promptly to concerning surveys, and off-hours administration. The human capital needed to administer PROMs by phone could substantially limit efforts to scale this approach.
Challenges common to both programs include integration into busy clinical workflows and engaging patients in follow-up. Both innovations confirmed higher success rates with phone contact compared to e-mail, including in elderly, comorbid populations. These two innovations demonstrate the need to standardize workflow for PROMs administration, and underscore the importance of providing training and support to administering personnel. Integrating PROMs into daily workflow may not be possible without increasing staff capacity to offset additional workload. Both innovations achieved low absolute response rates suggesting that full participation may not be realistic. Strategies for improving response rates include tailoring surveys to participants’ language and literacy levels, conducting interviews face-to-face, setting deadlines for participation, and using non-monetary incentives. Additionally, it is possible that iterative improvement through Plan-Do-Study-Act (PDSA) cycles may improve response rates in the future, similar to observational registries. Using a structured approach, such as the Reach-Effectiveness-Adoption-Implementation-Maintenance schematic (www.RE-AIM.org) could result in improved responses. While clinicians were encouraged to share PROMs with patients and use them for counseling, it is unclear how many clinicians chose to do so, and whether this was associated with increased follow-up response rates. Moreover, while the current projects were designed to evaluate the feasibility of PROMs collection, with appropriate training, routine integration of PROMs into clinical workflow as a “vital sign” may help to improve response rates and clinical utilization. Both MGH and BIDMC have plans to continue and modify existing programs through adding staffing and electronic work-arounds, but these interventions may not apply to all institutions. Until evidence-based strategies exist to improve response rates and overcome common barriers to PROMs administration, institutions are encouraged to develop pilot programs to start collecting PROMs to identify the most effective local strategies for administration. Through pooled knowledge and expertise, we can start to build the evidence base for extracting meaningful and reproducible information from our most important and underutilized resource, the patient.
Supplementary Material
Acknowledgements:
The authors would like to acknowledge Kim Fong for his contribution to this work and the Centers for Healthcare Delivery Science at Beth Israel Deaconess Medical Center.
Funding: Dr. Blumenthal reports receiving funding for this work through a Population Health Management Innovation Pilot Award from Partners Healthcare and the John S. LaDue Memorial Fellowship at Harvard Medical School. Dr. Yeh is funded by a grant from the National Heart, Lung, and Blood Institute (1R01HL136708-01). Dr. Strom is funded by a grant from the American Heart Association (18CDA34110267). Dr. Wasfy reports grant support from the National Institutes of Health and Harvard Catalyst (KL2 TR001100). Dr. Yeh reports additional support through an Innovation Grant from the Center for Healthcare Delivery Sciences at Beth Israel Deaconess Medical Center for the current project, as well as grant support from Boston Scientific and Abiomed, and consulting fees from Abbott, Medtronic, and Teleflex, outside the submitted work. Dr. Wasfy is medical director for population health which includes work implementing PROMs at the Massachusetts General Physicians Organization.
Footnotes
Disclosures: The authors report nothing to disclose, relevant to the current work.
LITERATURE CITED
- 1.Arnold SV, Morrow DA, Lei Y, Cohen DJ, Mahoney EM, Braunwald E, Chan PS. Economic impact of angina after an acute coronary syndrome: insights from the MERLIN-TIMI 36 trial. Circulation: Cardiovascular Quality and Outcomes. 2009;2:344–53. [DOI] [PubMed] [Google Scholar]
- 2.Arnold SV, Grodzinsky A, Gosch KL, Kosiborod M, Jones PG, Breeding T, Towheed A, Beltrame J, Alexander KP, Spertus JA. Predictors of Physician Under-Recognition of Angina in Outpatients With Stable Coronary Artery Disease. Circulation: Cardiovascular Quality and Outcomes. 2016;9:554–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Spertus JA, Winder JA, Dewhurst TA, Deyo RA, Fihn SD. Monitoring the quality of life in patients with coronary artery disease. AJC. 1994;74:1240–4. [DOI] [PubMed] [Google Scholar]
- 4.Beltrame JF, Weekes AJ, Morgan C, Tavella R, Spertus JA. The prevalence of weekly angina among patients with chronic stable angina in primary care practices: The Coronary Artery Disease in General Practice (CADENCE) Study. Arch Intern Med. 2009;169:1491–9. [DOI] [PubMed] [Google Scholar]
- 5.McNamara RL, Spatz ES, Kelley TA, Stowell CJ, Beltrame J, Heidenreich P, Tresserras R, Jernberg T, Chua T, Morgan L, Panigrahi B, Rosas Ruiz A, Rumsfeld JS, Sadwin L, Schoeberl M, Shahian D, Weston C, Yeh R, Lewin J. J Am Heart Assoc. 2015;4:e001767. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.