Abstract
Background
Patient reminders for influenza vaccination, delivered via an electronic health record patient portal and interactive voice response calls, offer an innovative approach to engaging patients and improving patient care.
Objective
The goal of this study was to test the effectiveness of portal and interactive voice response outreach in improving rates of influenza vaccination by targeting patients in early September, shortly after vaccinations became available.
Methods
Using electronic health record portal messages and interactive voice response calls promoting influenza vaccination, outreach was conducted in September 2015. Participants included adult patients within a large multispecialty group practice in central Massachusetts. Our main outcome was electronic health record–documented early influenza vaccination during the 2015-2016 influenza season, measured in November 2015. We randomly assigned all active portal users to 1 of 2 groups: (1) receiving a portal message promoting influenza vaccinations, listing upcoming clinics, and offering online scheduling of vaccination appointments (n=19,506) or (2) receiving usual care (n=19,505). We randomly assigned all portal nonusers to 1 of 2 groups: (1) receiving interactive voice response call (n=15,000) or (2) receiving usual care (n=43,596). The intervention also solicited patient self-reports on influenza vaccinations completed outside the clinic. Self-reported influenza vaccination data were uploaded into the electronic health records to increase the accuracy of existing provider-directed electronic health record clinical decision support (vaccination alerts) but were excluded from main analyses.
Results
Among portal users, 28.4% (5549/19,506) of those randomized to receive messages and 27.1% (5294/19,505) of the usual care group had influenza vaccinations documented by November 2015 (P=.004). In multivariate analysis of portal users, message recipients were slightly more likely to have documented vaccinations when compared to the usual care group (OR 1.07, 95% CI 1.02-1.12). Among portal nonusers, 8.4% (1262/15,000) of those randomized to receive calls and 8.2% (3586/43,596) of usual care had documented vaccinations (P=.47), and multivariate analysis showed nonsignificant differences. Over half of portal messages sent were opened (10,112/19,479; 51.9%), and over half of interactive voice response calls placed (7599/14,984; 50.7%) reached their intended target, thus we attained similar levels of exposure to the messaging for both interventions. Among portal message recipients, 25.4% of message openers (2570/10,112) responded to a subsequent question on receipt of influenza vaccination; among interactive voice response recipients, 72.5% of those reached (5513/7599) responded to a similar question.
Conclusions
Portal message outreach to a general primary care population achieved a small but statistically significant improvement in rates of influenza vaccination (OR 1.07, 95% CI 1.02-1.12). Interactive voice response calls did not significantly improve vaccination rates among portal nonusers (OR 1.03, 95% CI 0.96-1.10). Rates of patient engagement with both modalities were favorable.
Trial Registration
ClinicalTrials.gov NCT02266277; https://clinicaltrials.gov/ct2/show/NCT02266277
Keywords: electronic health records, influenza vaccination, patient care, patient engagement
Introduction
Influenza infections contribute to increased health care costs and loss of productivity and can lead to serious complications and even death [1]. Effective strategies for the prevention of influenza are of critical importance as we enter the 2020-2021 influenza season. The confluence of the upcoming influenza season and the ongoing coronavirus disease 2019 (COVID-19) pandemic is expected to place additional stress on our health care system, fueled in part by similarities in presenting complaints for these two illnesses, as well as by the potential for increased risk of poor outcomes in patients co-infected with COVID-19 and influenza [2-4].
An estimated 5%-20% of the US population contracts influenza every year, with several hundred thousand people hospitalized annually due to influenza-related complications [5]. Estimates of annual influenza and pneumonia-associated deaths over the past decade reached as high as 61,000 in the 2017-2018 season [6-12]. According to Centers for Disease Control and Prevention (CDC) estimates, during the 2018-2019 influenza season, vaccinations prevented approximately 4.4 million flu illnesses, 58,000 hospitalizations, and 3500 deaths [13,14]. Since it takes approximately 2 weeks for antibodies to develop in response to influenza vaccination, the CDC recommends getting vaccinated before flu begins to spread within a community, by the end of October [15].
Despite widespread publicity promoting influenza vaccination, vaccines are underutilized [16-19]. In 2017, national vaccination coverage among adults for influenza was 37.1% [20], while the Healthy People 2020 target was 70% [21]. The CDC estimated flu vaccination coverage among adults aged ≥18 years as of mid-November 2018 was 44.9% [22]; at the end of the 2018-2019 influenza season, national influenza vaccination coverage was 45.3%[23].
Clinical decision support has been shown to improve health outcomes by supporting the delivery of timely, evidence-based and guideline-concordant medical care, including annual vaccinations [24-26]. Clinical decision support includes computerized reminders both to providers and patients. Many health systems effectively use provider-directed clinical decision support, including noninterruptive or interruptive (pop-up) alerts, reminding providers of recommended prevention or screening measures [27,28]. While frequently effective, provider-directed clinical decision support is subject to important limitations. If alerts are triggered by erroneous or incomplete electronic health record data, or if providers experience alert fatigue from an overwhelming number of notifications, alerts may be ignored or overridden [29-32]. Considering these challenges, and in the setting of nationwide adoption of electronic patient portals, patient-directed clinical decision support delivered via a patient portal offers an innovative approach to the promotion of timely influenza vaccination.
Electronic patient portals are secure websites that provide patients with 24-hour online access to limited electronic health record information. A portal provides patients with a personal health record that is tethered to their electronic health record. Accessible information within a tethered portal varies by health system but may include vaccinations, laboratory results, problem lists, allergies, and information from recent doctor visits or hospitalizations [33,34]. A core function of portals is secure messaging—electronic communication with the physician or health care team [35]. Patient portals have the potential to improve patient-provider communication, improve medication adherence, decrease office visits, increase self-management of disease and disease awareness, increase use of preventative medicine, and increase inclusion of patients in medical decision making [36-38]. Previous patient outreach interventions have been shown to improve rates of vaccination completion and have been tested using multiple options including mailed letters, postcards, live phone calls, automated phone messages, and combination postcard/phone-based [39,40]. Few studies have tested the use of patient-directed vaccination reminders sent via patient portals [25,26,41].
We conducted a randomized controlled trial aimed at improving rates of influenza vaccination among eligible adult patients in a large multispecialty group practice in central Massachusetts. We used electronic health record patient portal messages as well as interactive voice recognition calls to (1) promote early season influenza vaccination completion and (2) solicit patient self-report on vaccinations completed outside the clinic.
Methods
Study Objectives
The overarching goal of this study was to improve rates of early influenza vaccination (by the end of October) among eligible adults in an outpatient population. Our primary objective was to determine whether our outreach increased completion of influenza vaccinations and, if so, whether one mode of outreach was most effective. Additional objectives were to improve documentation of influenza vaccinations administered outside the practice by inviting patient self-report (improving the accuracy of existing decision support tools) and to track process measures (eg, rates of portal message opening and interactive voice response call answering).
Study Design
We conducted a nonblinded randomized controlled intervention (NCT02266277) at a large multispecialty medical group in central Massachusetts. Previously, our team developed and tested interactive voice response and portal outreach which we targeted to patients who had no documented vaccination 2 months after the start of the season [25,26]; this study adapted our previous approach, targeting a broader population in early September in order to promote early vaccination and provide information on September and October flu clinic dates.
Using a computer-generated randomization table, we assigned all active portal users to receive either a portal message promoting influenza vaccination, listing upcoming clinics, and offering online scheduling of vaccination appointments (n=19,506) or usual care with no portal message (n=19,505). Separately, we randomized all portal nonusers to receive either an interactive voice response call (n=15,000) or usual care with no interactive voice response call (n=43,596) (Figure 1). For portal users only, after the conclusion of the study and assessment of outcomes on November 3, messages were sent to the usual care group if they still did not have an electronic health record–documented influenza vaccination for the 2015-2016 season (n=14,118). The cost of calls prevented us from being able to send interactive voice response messages to the usual care group in the portal nonusers.
Figure 1.
CONSORT (Consolidated Standards of Reporting Trials) Diagram describing randomized controlled trial to improve rates of early season influenza vaccinations using EHR portal messages and Interactive Voice Response calls.
The study was reviewed and approved in 2014 by the Reliant Medical Group institutional review board; due to administrative changes, oversight was transferred to the University of Massachusetts institutional review board in 2015. A waiver for informed consent for patient outreach was approved by these institutional review boards. Patients were not compensated for their participation. The principal investigator (SC) oversaw the trial and data analysis.
Study Population
Eligibility Criteria
Patients were eligible for the study if they (1) had a primary care provider at the medical group during the 12 months prior to randomization; (2) were aged ≥18 on the date of randomization; (3) had a recent office visit or telephone encounter with an internal medicine practitioner or family practitioner (defined as having had an office visit, phone encounter, consult, or complete physical exam within the 12 months prior to randomization). This requirement was intended to minimize inclusion of patients who had moved to another practice but whose names were retained in the medical group records. To ensure capture of patients transitioning from pediatric to adult care, the recent office visit could also be with a pediatrician.
A patient was eligible for inclusion in the electronic patient portal portion of the trial if the patient was an active user, which was defined as having an activated portal with a log-in at least once in the year preceding randomization.
Exclusion Criteria
For both interactive voice response and portal outreach, patients were excluded if there was electronic health record documentation of an allergy to influenza vaccines. For interactive voice response calls only, exclusion criteria also included the presence of any of the following on the date of randomization: (1) electronic health record documentation of influenza vaccination completion in the 2014-2015 influenza season (or documented influenza vaccination after the end of the 2014-2015 influenza season but before the start of the 2015-2016 season); (2) no listed phone number.
Study Procedures
This study consisted of interactive voice response or portal-based outreach. Qualitative interviews conducted with patients, physicians, nurses, and staff informed the development of our outreach material [25,26].
Electronic Patient Portal Intervention
We designed an outgoing secure portal message to be sent via patient portal to patients randomized to the portal message arm (Multimedia Appendix 1). Portal message content appeared in letter format with the signature line reflecting the name of the patient’s primary care provider. Portal messages were delivered through standard channels used for portal-based correspondence between the medical group’s health care providers and patients (ie, a generic message that contained neither personal health information nor any reference to vaccinations was delivered to patient’s email account; the message prompted patients to log in to secure portal account via hyperlink). Once logged in to the portal accounts, patients clicked on a message labeled “A Message from Your Primary Care Provider” to view the outreach message. The outreach message included information about upcoming flu clinics for September and October 2015.
Unique to the portal message (compared to interactive voice response phone messages) was the option of direct online scheduling of appointments for influenza vaccination. Information about accessing the CDC vaccination website appeared within the body of the portal message as a hyperlink (and was conveyed verbally in the interactive voice response script). Opportunities to report external influenza vaccinations and to report intent to get vaccinated matched the interactive voice response call content.
Portal Message Delivery
Messages were sent out to 500 to 1500 patients daily over a period of 15 days in September 2015 in order to reduce the risk of being blocked by network bulk-spam filters.
Interactive Voice Response Call Intervention
Interactive voice response calls appeared on caller ID as originating from the medical group. This is consistent with identification of interactive voice response calls used for appointment reminders at the time of the study. Combining voice response with branching logic, calls elicited patient self-reports of influenza vaccinations completed outside the medical group (Multimedia Appendix 2). For patients reporting no influenza vaccination completed, calls included information about upcoming flu clinics for September and October 2015. Patients reporting no influenza vaccination completed were also asked whether they intended to get vaccinated. Patients reporting that they were unsure or did not intend to get vaccinated were asked further questions on specific reasons why they did not plan to get a flu vaccination.
Interactive Voice Response Call Delivery
Intervention interactive voice response calls, initiated on September 11, 2015, began by confirming that the person answering the phone was the intended patient recipient of the call. If voicemail was encountered or if the person reached was someone other than the patient, the interactive voice response system left a message asking patients to call back and provided an inbound call line number. The last interactive voice response outbound calls were placed on September 25, 2015. The inbound call line was maintained throughout the duration of outgoing calls and for 3 weeks after the final outgoing call was placed; patients who called this number from the phone number of record heard the interactive voice response call script in its entirety, beginning with questions confirming the identity of the caller.
Study Outcomes
Primary Outcome
Our primary outcome was percentage of eligible patients with influenza vaccinations documented in the electronic health record as of November 3, 2015. We pulled data on vaccination rates as of this date, chosen in order to assess the impact of early outreach on completion of early immunization. Immunizations captured solely through the patient portal questionnaire or through the interactive voice response were excluded from the primary analysis in order to enable comparison with the control groups. The origins of all entered influenza vaccinations were tracked to allow our team to distinguish between sources of information on completed vaccinations.
Process Measures and Additional Outcomes of Interest
For portal messages, we calculated (1) percentage of recipients who logged in to the patient portal during early flu season (through November 3, 2015), (2) percentage of recipients who opened messages during early flu season, and (3) percentage of recipients who completed questionnaires. We tracked self-reports (via the portal) of influenza vaccinations completed. We also tracked patient-reported intent to get an influenza vaccination during the 2015-2016 flu season.
For interactive voice response calls, we calculated (1) percentage of recipients reached and (2) percentage of recipients who completed the calls by responding to questions. We tracked self-report (via interactive voice response) of influenza vaccinations completed. We also tracked patient-reported intent to get an influenza vaccination during the 2015-2016 flu season.
Sample Size
With our proposed sample size, power calculations based on estimates of baseline vaccination rates indicated that 4286 participants per arm would give 80% power to detect a 3% improvement in influenza vaccination rates between groups (α=.05; 2-sided).
Statistical Methods
Primary Outcome
To determine the impact of our interventions on early vaccination rates for the 2015-2016 influenza season, we calculated frequencies and performed intention-to-treat bivariate analyses of randomized patients, assessing whether vaccination completion was associated with group assignment. Due to different rates of electronic health record–recorded vaccination measured at baseline (in 2014) between portal users (35.9%) and nonusers (25%), and due to the differences in intervention (portal versus interactive voice response call), analyses for these groups were conducted separately.
We then performed multivariate logistic regression analyses, adjusting for demographic and practice-level covariates, we modeled the odds of receiving an influenza vaccination in the 2015-2016 influenza season.
Results
Baseline Characteristics
Baseline characteristics of both patient portal users and portal nonusers are reported in Table 1. Baseline characteristics were similar among portal users and portal nonusers. However, compared to the portal nonusers, portal users were more likely to be women, older, and have a higher level of health care utilization.
Table 1.
Baseline characteristics of participants.
| Characteristics | All (N=97,607), n (%) | Portal users | Portal nonusers | |||
|
|
|
|
Portal message (n=19,506), n (%) | Usual care (n=19,505), n (%) | Call (n=15,000), n (%) | Usual care (n=43,596), n (%) |
| Sex |
|
|
|
|
|
|
|
|
Female | 53,250 (54.5) | 12,230 (62.7) | 12,249 (62.8) | 7325 (48.8) | 21,446 (49.2) |
|
|
Male | 44,349 (45.4) | 7275 (37.3) | 7256 (37.2) | 7672 (51.1) | 22,146 (50.8) |
|
|
Missing | 8 (0.01) | 1 (0.0) | 0 (0.0) | 3 (0.0) | 4 (0.0) |
| Age |
|
|
|
|
|
|
|
|
18-34 | 30,556 (31.3) | 4508 (23.1) | 4597 (23.6) | 5615 (37.4) | 15,836 (36.3) |
|
|
35 - 49 | 25,579 (26.2) | 5096 (26.1) | 5062 (26.0) | 3896 (26.0) | 11,525 (26.4) |
|
|
50 - 64 | 26,495 (27.1) | 6275 (32.2) | 6224 (31.9) | 3547 (23.6) | 10,449 (24.0) |
|
|
65-74 | 8593 (8.8) | 2419 (12.4) | 2388 (12.2) | 965 (6.4) | 2821 (6.5) |
|
|
75+ | 6384 (6.5) | 1208 (6.2) | 1234 (6.3) | 977 (6.5) | 2965 (6.8) |
| Race |
|
|
|
|
|
|
|
|
White | 66,020 (67.6) | 14,180 (72.7) | 14,223 (72.9) | 9608 (64.1) | 28,009 (64.2) |
|
|
Black | 3417 (3.5) | 456 (2.3) | 440 (2.3) | 630 (4.2) | 1891 (4.3) |
|
|
Asian | 3502 (3.6) | 737 (3.8) | 715 (3.7) | 528 (3.5) | 1522 (3.5) |
|
|
American Indian or Alaska Native | 1021 (1.1) | 140 (0.7) | 144 (0.7) | 194 (1.3) | 543 (1.2) |
|
|
Other | 10,366 (10.6) | 1672 (8.6) | 1745 (8.9) | 1769 (11.8) | 5180 (11.9) |
|
|
Missing | 13,281 (13.6) | 2321 (11.9) | 2238 (11.5) | 2271 (15.1) | 6451 (14.8) |
| Health care utilization level |
|
|
|
|
|
|
|
|
Had office visita | 80,748 (82.7) | 17,896 (91.7) | 17,908 (91.8) | 11,609 (77.4) | 33,335 (76.5) |
|
|
Did not have office visit | 16,859 (17.3) | 1610 (8.3) | 1597 (8.2) | 3391 (22.6) | 10,261 (23.5) |
a12 months prior to randomization.
Portal Users
Among portal users, 28.4% (5549/19,506) of message recipients and 27.1% (5294/19,505) of the usual care group had documentation in their electronic health records that they received that influenza vaccinations on or before November 3, 2015 (P=.004). Portal users who received the messages were significantly more likely to have received the influenza vaccination compared to the usual care group (odds ratio [OR] 1.07, 95% CI 1.02-1.12). This finding was consistent even after adjusting for age, race, sex, and health care utilization (OR 1.07, 95% CI 1.02-1.12) (Table 2).
Table 2.
Likelihood of receiving an early-season influenza vaccination.
| Recipients | n | Unadjusted | Adjusteda | ||
|
|
|
ORb | 95% CI | OR | 95% CI |
| Portal messagec | 39,011 | 1.07 | (1.02, 1.12) | 1.07 | (1.02, 1.12) |
| Interactive voice response callc | 58,596 | 1.03 | (0.96, 1.10) |
1.03 | (0.96, 1.11) |
aAdjusted for age, sex, race, and health care utilization level (where utilization was defined as office visit, phone encounter, consult, or complete physical exam within the 12 months prior to randomization).
bOR: odds ratio.
cThe reference is usual care.
Portal Nonusers
Among portal nonusers, 8.4% (1262/15,000) of call recipients and 8.2% (3586/43,596) of usual care recipients received vaccinations (P=.47). Bivariate and multivariate analysis showed nonsignificant differences in influenza vaccination rates between intervention and usual care groups (Table 2).
Process Measures
Portal Message
Among patient portal message recipients, 71.2% (13,862 recipients out of 19,479 to whom the message was sent) logged in to the patient portal on or after the date of message delivery through the end of early flu season (September 9, 2015 to November 3, 2015). Messages were opened by 51.9% (10,112/19,479) recipients; 13.2% (2570/19,479) responded to the first question asking if they received a flu vaccination on or after August 1, 2015, 2.0% (386/19,479) reported already receiving a vaccination and 11.2% (2176/19,479) responded to a second question assessing their intention to receive a flu vaccination during the flu season (asked only of those who were not already vaccinated) (Table 3).
Table 3.
Process measures and self-reported influenza vaccinations for portal message recipients.
| Action | Portal users, n (%) |
| Randomized | 19,506 |
| Message senta | 19,479 (100) |
| Logged in to patient portal | 13,862 (71.2) |
| Opened message | 10,112 (51.9) |
| Responded to “Have you received a flu vaccination on or after August 1, 2015?” | 2570 (13.2) |
| Reported receiving a flu vaccination | 386 (2.0) |
| Responded to “Do you plan to get a flu vaccination this flu season?” | 2176 (11.2) |
| Reported that they planned to get a flu vaccination | 1814 (9.3) |
a27 patients were no longer portal users by the time we sent the messages (due to change in medical record numbers, invalid patient portal IDs, etc).
Of those opening messages, 25.4% (2570/10,112) responded to our question on receipt of influenza vaccination, 3.8% (386/10,112) reported already receiving vaccinations and 21.5% (2176/10,112) responded to our question on whether they planned to get vaccinated.
Interactive Voice Response Call
Among interactive voice response call recipients, 50.7% of patients were reached (7599/14,984); 36.8% (5513/14,984) responded to the first question asking if they received a flu vaccination on or after August 1, 2015, 3.8% (575/14,984) reported receiving their vaccination and 24.1% (3613/14,984) responded to a second question assessing their intention to receive a flu vaccination during the flu season (asked only of those who were not already vaccinated) (Table 4).
Table 4.
Process measures and self-reported influenza vaccinations for interactive voice response call recipients.
| Action | Call recipients, n (%) |
| Randomized | 14,999 |
| Call attempteda | 14,984 (100) |
| Target reached (inbound and outbound calls) | 7599 (50.7) |
| Responded to “Have you received a flu vaccination on or after August 1, 2015?” | 5513 (36.8) |
| Reported receiving a flu vaccination | 575 (3.8) |
| Responded to “Do you plan to get a flu vaccination this flu season?” | 3613 (24.1) |
| Reported that they planned to get a flu vaccination | 1415 (9.4) |
a15 patients had invalid/blank phone numbers, or the patient was on a “Do Not Call” list, by the time we placed the calls.
Of those reached, 72.5% (5513/7599) responded to our question on receipt of influenza vaccination, 7.6% (575/7599) reported already receiving a vaccination, and 47.5% (3613/7599) responded to our question on whether they planned to get vaccinated.
Discussion
Principal Results
Our study showed a clinically small but statistically significant improvement in completion of early season influenza vaccinations among those randomized to receive outreach via patient portal, compared to a usual care control group (OR 1.07, 95% CI 1.02-1.12). There was no significant increase of early season influenza vaccinations among those randomized to receive an interactive voice response call (OR 1.03, 95% CI 0.96-1.10).
This outreach was designed to deliver a relatively time-sensitive message (eg, reminding patients of the importance of influenza vaccination and alerting them of upcoming vaccination clinics in September and October), and as such, represents a successful, brief patient engagement effort. We attained greater than 50% opening rates for portal messages and reached the targeted patient on over 50% of interactive voice response calls, achieving similar levels of exposure to the messaging for the two interventions. Among portal message recipients, more than one quarter of those who opened the message responded to our subsequent question on whether they had received or intended to receive their influenza vaccination. Among interactive voice response call recipients, close to three-quarters of those reached responded to the question.
A small number of patients reported influenza vaccinations that had been completed in the community; data on patient-reported vaccinations performed in the community were then uploaded into the electronic health record in order to improve the accuracy of existing influenza vaccination alerts directed at primary care providers. The relatively small number of self-reported flu vaccinations (961/20,978 participants across the portal message intervention and the interactive voice response intervention arms combined) has several possible explanations. It is possible that participants who had already been vaccinated found the outreach less relevant; and therefore, chose not to engage (eg, chose not to open the message or chose to hang up the phone). Relatively low rates of self-reported vaccination completion may also be attributable to the timing of our outreach—intentionally positioned at the start of the flu season. In contrast, our past work [26] captured much higher proportions of self-reported vaccinations (2591/20,000 intervention patients) when participants were approached several months past the start of flu season, which highlights the importance of asking patients to self-report vaccinations administered outside the clinic.
Although these “discovered” immunizations were incorporated into the electronic health record, they were not counted in the primary analysis; this allowed comparison with the usual care groups who were not queried for outside immunizations. While recognizing that the actual immunization rate was higher than the documented rate, any differences in the documented rate between intervention and control groups can reasonably be attributed to increased vaccination rates. Data on intention to be vaccinated were also elicited. Studies have shown that stating intent to complete a behavior can enhance likelihood of follow-through [42,43], thus it is possible that engaging patients in this manner may have contributed to the modest success of our intervention.
Once designed, portal messages (sent out in several batches) required minimal staff time to deliver. Our messages were designed to be easily adapted for use in future years; our messages have already been adapted and implemented by our medical center to target high-risk populations (eg, children with asthma).
Comparison With Prior Work
There have been multiple studies [44-47] documenting success in use of patient-directed reminders for influenza vaccination. Effective reminders described include letters, telephone invitations, phone calls from a peer, tailored communications, customized letters/phone calls, and client-based appraisals [44]. Text message reminders have been shown to be successful for influenza vaccination reminders in pediatric and adolescent populations [45], in patients with rare diseases [46], and among high-risk patients [47].
While portal-based outreach directed at patients has been described for a variety of preventive measures [48], few studies have tested use of patient-directed influenza vaccination reminders sent via patient portals; one study [49] that has done so focused exclusively on untethered (ie, personally controlled) patient health records among a university student and staff population in Australia, yielding a 6.7% greater likelihood of influenza vaccination among users of the personal health records compared to those randomized to a 6-month waitlist for portal activation [49]. In our past work [25,26], we tested similar portal and interactive voice response outreach messages using a factorial design and targeting a population more likely to be nonadherent to vaccination guidelines (patients who had no documented vaccination 2 months after the start of the flu season). In that study [25,26], we found small but statistically significant improvements in influenza vaccination rates among recipients of either outreach method compared to usual care (portal message alone (OR 1.20, 95% CI 1.06-1.35); interactive voice response call alone (OR 1.15, 95 % CI 1.02-1.30); both messages (OR 1.29, 97.5% CI 1.13-1.48). In their recent randomized trial, Szilagyi et al [41] studied the effect of 1, 2, or 3 patient portal reminders on influenza vaccination rates among 164,205 patients in 52 primary care practices. They noted small but statistically significant improvements in vaccination rates with an attenuating effect of repeated messaging (37.5% for those receiving no reminders; 38.0% for 1 reminder; 38.2% for 2 reminders, and 38.2% for 3 reminders). Higher overall rates of vaccination were documented in their trial in comparison to ours; this is likely due in part to the study’s inclusion of children (who have a higher rate of influenza vaccination than that of the overall adult population) as well as their measurement of vaccination rates at the end of the influenza season (March 31).
The rates of patient engagement (measured by message opening and subsequent action) demonstrated by our patient portal recipients were comparable to those from previously published patient portal outreach studies. In our previous influenza outreach intervention [25,26] (in a population unvaccinated 2 months into the flu season), messages were opened by slightly more than half of message recipients and interactive voice response targets were reached in just over 60%. In that prior study, among those who opened portal messages, 28.6% responded to subsequent questions. Among those reached via interactive voice response call, 78.3% responded to questions. In their patient portal intervention aimed at improving influenza vaccination rates, Szilagyi and colleagues [41] found opening rates of 52.9% among patients receiving a single portal message (increasing to 55.9% in a 2-reminder group and 58.8% in a 3-reminder group).
Fischer and colleagues [50] found that among patients randomized to active reminders about multiple health maintenance services (eg, hemoglobin A1c testing, lipids, etc), nearly 65% of patients logged in to the portal after receiving the first of several messages. In a study [51] of colorectal cancer screening reminders delivered via patient portal, among the 552 patients randomized to receive messages, 54% viewed the message and 9% performed a suggested web-based risk assessment tool. In a study describing the reach and feasibility of an interactive lung cancer screening decision aid delivered by patient portal, Dharod and colleagues [52] found that 86% of lung cancer screening eligible patients identified by an electronic health record algorithm to receive a patient portal message read the message, 40% then visited a web-based decision aid for lung cancer screening, and 35% completed questionnaires to determine their eligibility for lung cancer screening.
Limitations
Our study had several limitations. Despite responding to inquiries on flu vaccinations receipt at higher rates, the interactive voice response group did not show a significant impact on vaccination rates (while the portal message group showed a small statistically significant improvement in vaccination rates). This discrepancy could be due in part to differences in the baseline characteristics of portal nonusers compared to portal users. For several reasons, interactive voice response call recipients may have been more resistant to completion of influenza vaccination than portal message recipients. Portal nonusers were less likely to have had an office visit in the previous 12 months and might have been less actively engaged in their health care overall. In addition, eligibility for the interactive voice response study was intentionally more stringent, with patients included only if they lacked documentation of an influenza vaccination in the previous flu season. This study design choice was made with guidance from the medical center where our study was implemented and was intended to make available the limited resource of interactive voice response calls to the broadest possible number of nonadherent patients (our study covered the cost for only 15,000 calls but we had no limit on the number of portal messages that we could send). It is possible that the interactive voice response calls might have yielded a greater impact if we had not opted to employ this more stringent eligibility criterion.
Conclusions
There are compelling reasons to use existing functionality within electronic health record–tethered portals to promote influenza vaccination. For vaccination outreach, data recorded in the electronic health record through routine care delivery can inform real-time identification of unvaccinated populations. Portal-based outreach can be more cost-effective than phone calls or mailings and easier to implement than a new app; simple messages can be sent out by office staff without additional informatics expertise. Studies [37,38,53,54] show that patient portals can enhance patient empowerment and sense of autonomy, enhance patient engagement, improve medication adherence, decrease office visits, increase self-management of disease and disease awareness, increase use of preventive medicine, and increase inclusion of patients in medical decision making.
Our parallel interventions to patient portal users and portal nonusers allowed us to assess the impact of outreach supporting influenza vaccination in both groups. Our study demonstrated a small improvement in influenza vaccination rates among portal message recipients, and successful patient engagement in both portal message recipients and interactive voice response call recipients. This method is readily applicable to current practice.
As we simultaneously face the upcoming influenza season and the ongoing COVID-19 pandemic, expanding influenza vaccination coverage in ambulatory populations can decrease the strain on our overtaxed health system and may help avert poor outcomes in patients at risk for co-infection with influenza and COVID-19.
Our findings of a small but significant improvement in influenza vaccination rates resulting from portal-based outreach represent an important contribution to the national conversation on caring for and protecting our patients in the upcoming months and for years to come.
Acknowledgments
This study was funded by Pfizer Independent Grants for Learning and Change. The authors designed and implemented the study with full oversight and responsibility for data collection, analysis, and manuscript preparation. Our study team acknowledges the contributions made by Eliza, an HMS Company, toward the design and implementation of the interactive voice response calls. A portion of MIF’s work was supported by the National Heart, Lung, and Blood Institute through Grant 1K12HK138049-01. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Abbreviations
- CDC
Centers for Disease Control and Prevention
- OR
odds ratio
Appendix
Example of portal outreach message.
Example of interactive voice response call script.
Footnotes
Conflicts of Interest: None declared.
References
- 1.Molinari NM, Ortega-Sanchez IR, Messonnier ML, Thompson WW, Wortley PM, Weintraub E, Bridges CB. The annual impact of seasonal influenza in the US: measuring disease burden and costs. Vaccine. 2007 Jun 28;25(27):5086–96. doi: 10.1016/j.vaccine.2007.03.046. [DOI] [PubMed] [Google Scholar]
- 2.Gostin LO, Salmon DA. The Dual Epidemics of COVID-19 and Influenza: Vaccine Acceptance, Coverage, and Mandates. JAMA. 2020 Jul 28;324(4):335–336. doi: 10.1001/jama.2020.10802. [DOI] [PubMed] [Google Scholar]
- 3.Antony SJ, Almaghlouth NK, Heydemann EL. Are coinfections with COVID-19 and influenza low or underreported? An observational study examining current published literature including three new unpublished cases. J Med Virol. 2020 Jun 12;:26167. doi: 10.1002/jmv.26167. [DOI] [PubMed] [Google Scholar]
- 4.Yue H, Zhang M, Xing L, Wang K, Rao X, Liu H, Tian J, Zhou P, Deng Y, Shang J. The epidemiology and clinical characteristics of co-infection of SARS-CoV-2 and influenza viruses in patients during COVID-19 outbreak. J Med Virol. 2020 Jun 12;:26163. doi: 10.1002/jmv.26163. http://europepmc.org/abstract/MED/32530499. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.NIH fact sheets - influenza. NIH. 2011. [2019-09-27]. https://archives.nih.gov/asites/report/09-09-2019/report.nih.gov/nihfactsheets/ViewFactSheet44bd.html?csid=133&key=I#I.
- 6.National CFHS. Health, United States. Health, United States. 2015;With special feature on racial and ethnic health disparities. Hyattsville (MD):104–112. doi: 10.15620/cdc:82212. [DOI] [PubMed] [Google Scholar]
- 7.Kochanek KD, Murphy SL, Xu J. Deaths: Final Data for 2011. Natl Vital Stat Rep. 2015 Jul 27;63(3):1–120. https://www.cdc.gov/nchs/data/nvsr/nvsr63/nvsr63_03.pdf. [PubMed] [Google Scholar]
- 8.Kochanek KD, Xu J, Murphy SL, Miniño AM, Kung H. Deaths: final data for 2009. Natl Vital Stat Rep. 2011 Dec 29;60(3):1–116. https://www.cdc.gov/nchs/https://www.cdc.gov/nchs/data/nvsr/nvsr60/nvsr60_03.pdf. [PubMed] [Google Scholar]
- 9.Murphy SL, Kochanek KD, Xu J, Heron M. Deaths: Final Data for 2012. Natl Vital Stat Rep. 2015 Aug 31;63(9):1–117. https://www.cdc.gov/nchs//data/nvsr/nvsr63/nvsr63_9.pdf. [PubMed] [Google Scholar]
- 10.Murphy SL, Xu J, Kochanek KD. Deaths: final data for 2010. Natl Vital Stat Rep. 2013 May 8;61(4):1–117. https://www.cdc.gov/nchs/https://www.cdc.gov/nchs/data/nvsr/nvsr61/nvsr61_04.pdf. [PubMed] [Google Scholar]
- 11.Xu J, Murphy SL, Kochanek KD, Bastian BA. Deaths: Final Data for 2013. Natl Vital Stat Rep. 2016 Feb 16;64(2):1–119. https://www.cdc.gov/nchs/data/nvsr/nvsr64/nvsr64_2.pdf. [PubMed] [Google Scholar]
- 12.Prevention CDC. Estimated Influenza Illnesses, Medical visits, Hospitalizations, and Deaths in the United States - 2017-2018 influenza season. 2019. [2019-09-27]. https://www.cdc.gov/flu/about/burden/2017-2018.htm.
- 13.Rolfes MA, Flannery B, Chung J, O'Halloran A, Garg S, Belongia EA, Gaglani M, Zimmerman R, Jackson ML, Monto AS, Alden NB, Anderson E, Bennett NM, Billing L, Eckel S, Kirley PD, Lynfield R, Monroe ML, Spencer M, Spina N, Talbot HK, Thomas A, Torres S, Yousey-Hindes K, Singleton J, Patel M, Reed C, Fry AM, U.S. Flu VE Network‚ the Influenza Hospitalization Surveillance Network (FluSurv-NET)‚the Assessment Branch‚ Immunization Services Division‚ Centers for Disease ControlPrevention Effects of Influenza Vaccination in the United States during the 2017-2018 Influenza Season. Clin Infect Dis. 2019 Feb 02;:1845–1853. doi: 10.1093/cid/ciz075. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Prevention CDC. Estimated Influenza Illnesses, Medical visits, Hospitalizations, and Deaths in the United States - 2018-2019 influenza season. 2020. [2019-09-27]. https://www.cdc.gov/flu/about/burden/2018-2019.html.
- 15.Frequently asked flu questions 2018-2019 influenza season. Centers for Disease Control and Prevention. 2019. https://www.cdc.gov/flu/about/season/flu-season-2018-2019.htm.
- 16.National Vaccine Advisory Committee A pathway to leadership for adult immunization: recommendations of the National Vaccine Advisory Committee: approved by the National Vaccine Advisory Committee on June 14, 2011. Public Health Rep. 2012;127 Suppl 1:1–42. doi: 10.1177/00333549121270s101. http://europepmc.org/abstract/MED/22210957. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Appel A. Improving adult immunization rates: overcoming barriers. Am Fam Physician. 2011 Nov 01;84(9):977–8. http://www.aafp.org/link_out?pmid=22046935. [PubMed] [Google Scholar]
- 18.Bean-Mayberry B, Yano EM, Mor MK, Bayliss NK, Xu X, Fine MJ. Does sex influence immunization status for influenza and pneumonia in older veterans? J Am Geriatr Soc. 2009 Aug;57(8):1427–32. doi: 10.1111/j.1532-5415.2009.02316.x. http://europepmc.org/abstract/MED/19515114. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Vaughn JA, Miller RA. Update on immunizations in adults. Am Fam Physician. 2011 Nov 01;84(9):1015–20. http://www.aafp.org/link_out?pmid=22046942. [PubMed] [Google Scholar]
- 20.Estimates of influenza vaccination coverage among adults - United States, 2017-18 flu season. Centers for Disease Control and Prevention. 2018. [2019-06-14]. https://www.cdc.gov/flu/fluvaxview/coverage-1718estimates.htm.
- 21.HealthyPeople.gov Immunization and infectious diseases data details. [2019-10-17]. https://www.healthypeople.gov/node/3527/data-details.
- 22.Early-season flu vaccination coverage - United States, November 2018. Centers for Disease Control and Prevention. 2018. [2019-06-14]. https://www.cdc.gov/flu/fluvaxview/nifs-estimates-nov2018.htm.
- 23.Prevention CDC. Flu Vaccination Coverage, United States, 2018-19 Influenza Season. 2019. [2019-09-27]. https://www.cdc.gov/flu/fluvaxview/coverage-1819estimates.htm.
- 24.Clinical decision support: more than just 'alerts' tipsheet Clinical decision support: more than just 'alerts' tipsheet. US Centers for Medicare & Medicaid Service. https://www.cms.gov/regulations-and-guidance/legislation/EHRincentiveprograms/downloads/clinicaldecisionsupport_tipsheet-.pdf.
- 25.Cutrona SL, Sreedhara M, Goff SL, Fisher LD, Preusse P, Jackson M, Sundaresan D, Garber LD, Mazor KM. Improving Rates of Influenza Vaccination Through Electronic Health Record Portal Messages, Interactive Voice Recognition Calls and Patient-Enabled Electronic Health Record Updates: Protocol for a Randomized Controlled Trial. JMIR Res Protoc. 2016 May 06;5(2):e56. doi: 10.2196/resprot.5478. http://www.researchprotocols.org/2016/2/e56/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Cutrona SL, Golden JG, Goff SL, Ogarek J, Barton B, Fisher L, Preusse P, Sundaresan D, Garber L, Mazor KM. Improving Rates of Outpatient Influenza Vaccination Through EHR Portal Messages and Interactive Automated Calls: A Randomized Controlled Trial. J Gen Intern Med. 2018 May;33(5):659–667. doi: 10.1007/s11606-017-4266-9. http://europepmc.org/abstract/MED/29383550. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Cutrona SL, Fouayzi H, Burns L, Sadasivam RS, Mazor KM, Gurwitz JH, Garber L, Sundaresan D, Houston TK, Field TS. Primary Care Providers' Opening of Time-Sensitive Alerts Sent to Commercial Electronic Health Record InBaskets. J Gen Intern Med. 2017 Nov;32(11):1210–1219. doi: 10.1007/s11606-017-4146-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Amroze A, Field TS, Fouayzi H, Sundaresan D, Burns L, Garber L, Sadasivam RS, Mazor KM, Gurwitz JH, Cutrona SL. Use of Electronic Health Record Access and Audit Logs to Identify Physician Actions Following Noninterruptive Alert Opening: Descriptive Study. JMIR Med Inform. 2019 Feb 07;7(1):e12650. doi: 10.2196/12650. https://medinform.jmir.org/2019/1/e12650/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Cash JJ. Alert fatigue. Am J Health Syst Pharm. 2009 Dec 01;66(23):2098–2101. doi: 10.2146/ajhp090181. [DOI] [PubMed] [Google Scholar]
- 30.van der Sijs H, Aarts J, Vulto A, Berg M. Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc. 2006;13(2):138–47. doi: 10.1197/jamia.M1809. http://jamia.oxfordjournals.org/lookup/pmidlookup?view=long&pmid=16357358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Ridgely M, Greenberg M. Too many alerts, too much liability: sorting through the malpractice implications of drug-drug interaction clinical decision support. J Health Law Policy. 2012;5:257–96. https://www.slu.edu/law/academics/journals/health-law-policy/pdfs/issues/v5-i2/ridgely_greenberg_article.pdf. [Google Scholar]
- 32.Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation. Health Aff (Millwood) 2011 Dec;30(12):2310–7. doi: 10.1377/hlthaff.2010.1111. http://content.healthaffairs.org/cgi/pmidlookup?view=long&pmid=22147858. [DOI] [PubMed] [Google Scholar]
- 33.Amante DJ, Hogan TP, Pagoto SL, English TM. A systematic review of electronic portal usage among patients with diabetes. Diabetes Technol Ther. 2014 Nov;16(11):784–93. doi: 10.1089/dia.2014.0078. [DOI] [PubMed] [Google Scholar]
- 34.Irizarry T, DeVito DA, Curran CR. Patient Portals and Patient Engagement: A State of the Science Review. J Med Internet Res. 2015;17(6):e148. doi: 10.2196/jmir.4255. http://www.jmir.org/2015/6/e148/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.HealthIT.gov What is a patient portal? 2014. [2019-09-27]. https://www.healthit.gov/faq/what-patient-portal.
- 36.Britto MT, Hesse EA, Kamdar OJ, Munafo JK. Parents' perceptions of a patient portal for managing their child's chronic illness. J Pediatr. 2013 Jul;163(1):280–1.e1. doi: 10.1016/j.jpeds.2013.02.041. [DOI] [PubMed] [Google Scholar]
- 37.Kruse CS, Bolton K, Freriks G. The effect of patient portals on quality outcomes and its implications to meaningful use: a systematic review. J Med Internet Res. 2015;17(2):e44. doi: 10.2196/jmir.3171. http://www.jmir.org/2015/2/e44/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res. 2015;17(2):e40. doi: 10.2196/jmir.3703. http://www.jmir.org/2015/2/e40/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Odone A, Ferrari A, Spagnoli F, Visciarelli S, Shefer A, Pasquarella C, Signorelli C. Effectiveness of interventions that apply new media to improve vaccine uptake and vaccine coverage. Hum Vaccin Immunother. 2015;11(1):72–82. doi: 10.4161/hv.34313. http://europepmc.org/abstract/MED/25483518. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Jacobson VJC, Szilagyi P. Patient reminder and patient recall systems to improve immunization rates. Cochrane Database Syst Rev. 2005;(3):CD003941. doi: 10.1002/14651858.CD003941.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Szilagyi PG, Albertin C, Casillas A, Valderrama R, Duru OK, Ong MK, Vangala S, Tseng C, Rand CM, Humiston SG, Evans S, Sloyan M, Lerner C. Effect of Patient Portal Reminders Sent by a Health Care System on Influenza Vaccination Rates: A Randomized Clinical Trial. JAMA Intern Med. 2020 Jul 01;180(7):962–970. doi: 10.1001/jamainternmed.2020.1602. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Orbell S, Hodgkins S, Sheeran P. Implementation Intentions and the Theory of Planned Behavior. Pers Soc Psychol Bull. 1997 Sep;23(9):945–954. doi: 10.1177/0146167297239004. [DOI] [PubMed] [Google Scholar]
- 43.Pirolli P, Mohan S, Venkatakrishnan A, Nelson L, Silva M, Springer A. Implementation Intention and Reminder Effects on Behavior Change in a Mobile Health System: A Predictive Cognitive Model. J Med Internet Res. 2017 Nov 30;19(11):e397. doi: 10.2196/jmir.8217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Thomas RE, Lorenzetti DL. Interventions to increase influenza vaccination rates of those 60 years and older in the community. Cochrane Database Syst Rev. 2018 May 30;5:CD005188. doi: 10.1002/14651858.CD005188.pub4. http://europepmc.org/abstract/MED/29845606. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Stockwell MS, Kharbanda EO, Martinez RA, Vargas CY, Vawdrey DK, Camargo S. Effect of a text messaging intervention on influenza vaccination in an urban, low-income pediatric and adolescent population: a randomized controlled trial. JAMA. 2012 Apr 25;307(16):1702–8. doi: 10.1001/jama.2012.502. [DOI] [PubMed] [Google Scholar]
- 46.Esteban-Vasallo MD, Domínguez-Berjón MF, García-Riolobos C, Zoni AC, Aréjula Torres JL, Sánchez-Perruca L, Astray-Mochales J. Effect of mobile phone text messaging for improving the uptake of influenza vaccination in patients with rare diseases. Vaccine. 2019 Aug 23;37(36):5257–5264. doi: 10.1016/j.vaccine.2019.07.062. [DOI] [PubMed] [Google Scholar]
- 47.Regan AK, Bloomfield L, Peters I, Effler PV. Randomized Controlled Trial of Text Message Reminders for Increasing Influenza Vaccination. Ann Fam Med. 2017 Nov;15(6):507–514. doi: 10.1370/afm.2120. http://www.annfammed.org/cgi/pmidlookup?view=long&pmid=29133488. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Krist AH, Woolf SH, Bello GA, Sabo RT, Longo DR, Kashiri P, Etz RS, Loomis J, Rothemich SF, Peele JE, Cohn J. Engaging primary care patients to use a patient-centered personal health record. Ann Fam Med. 2014;12(5):418–26. doi: 10.1370/afm.1691. http://www.annfammed.org/cgi/pmidlookup?view=long&pmid=25354405. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Lau AYS, Sintchenko V, Crimmins J, Magrabi F, Gallego B, Coiera E. Impact of a web-based personally controlled health management system on influenza vaccination and health services utilization rates: a randomized controlled trial. J Am Med Inform Assoc. 2012;19(5):719–27. doi: 10.1136/amiajnl-2011-000433. http://jamia.oxfordjournals.org/cgi/pmidlookup?view=long&pmid=22582203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Fischer GS, Hess R, Landeen BM, Weimer M, Zieth CR, Dong X, Clark S, Roberts MS. Electronic reminders to patients within an interactive patient health record. Telemed J E Health. 2013 Jun;19(6):497–500. doi: 10.1089/tmj.2012.0116. [DOI] [PubMed] [Google Scholar]
- 51.Sequist TD, Zaslavsky AM, Colditz GA, Ayanian JZ. Electronic patient messages to promote colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2011 Apr 11;171(7):636–41. doi: 10.1001/archinternmed.2010.467. http://europepmc.org/abstract/MED/21149743. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Dharod A, Bellinger C, Foley K, Case LD, Miller D. The Reach and Feasibility of an Interactive Lung Cancer Screening Decision Aid Delivered by Patient Portal. Appl Clin Inform. 2019 Jan;10(1):19–27. doi: 10.1055/s-0038-1676807. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Kipping S, Stuckey MI, Hernandez A, Nguyen T, Riahi S. A Web-Based Patient Portal for Mental Health Care: Benefits Evaluation. J Med Internet Res. 2016 Nov 16;18(11):e294. doi: 10.2196/jmir.6483. http://www.jmir.org/2016/11/e294/ [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Sieck CJ, Hefner JL, McAlearney AS. Improving the patient experience through patient portals: Insights from experienced portal users. Patient Experience Journal. 2018 Nov 06;5(3):47–54. doi: 10.35680/2372-0247.1269. [DOI] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Example of portal outreach message.
Example of interactive voice response call script.