This cluster randomized clinical trial aims to determine whether provision of COVID-19 vaccine messaging platforms in emergency departments increases COVID-19 vaccine acceptance and uptake in unvaccinated patients.
Key Points
Question
Does provision of COVID-19 vaccine educational messaging increase vaccine acceptance and uptake in unvaccinated emergency department (ED) patients?
Findings
In this cluster randomized clinical trial of 496 participants conducted at 7 EDs in the US, delivery of COVID-19 vaccine messaging platforms resulted in statistically significant higher vaccine acceptance among intervention group participants compared with the control group (25.8% vs 12.0%; adjusted difference, 11.9 percentage points) as well as uptake within 30 days (20.0% vs 8.7%; adjusted difference, 7.9 percentage points). Outcome effect sizes of the intervention were greater in Latinx persons and participants without primary care physicians.
Meaning
These findings support the delivery of COVID-19 vaccine messaging platforms in EDs nationally to improve vaccine acceptance and uptake in underserved populations whose primary health care access occurs in EDs.
Abstract
Importance
Large segments of the US population’s primary health care access occurs in emergency departments (EDs). These groups have disproportionately high COVID-19 vaccine hesitancy and lower vaccine uptake.
Objective
To determine whether provision of COVID-19 vaccine messaging platforms in EDs increases COVID-19 vaccine acceptance and uptake in unvaccinated patients.
Design, Setting, and Participants
This prospective cluster randomized clinical trial was conducted at 7 hospital EDs in 4 US cities from December 6, 2021, to July 28, 2022. Noncritically ill adult patients who had not previously received COVID-19 vaccines were enrolled.
Interventions
A 3-pronged COVID-19 vaccine messaging platform (an English- or Spanish-language 4-minute video; a 1-page informational flyer; and a brief, scripted message from an ED physician or nurse) was delivered during patient waiting times.
Main Outcomes and Measures
The 2 primary outcomes were (1) COVID-19 vaccine acceptance, assessed by survey responses in the ED, and (2) receipt of a COVID-19 vaccine within 30 days, ascertained by ED confirmation of vaccination, electronic health record review, and telephone follow-up.
Results
Of the 496 participants enrolled (221 during intervention weeks and 275 during control weeks), the median (IQR) age was 39 (30-54) years, 205 (41.3%) were female, 193 (38.9%) were African American, 97 (19.6%) were Latinx, and 218 (44.0%) lacked primary care physicians. More intervention group participants, compared with control participants, stated that they would accept the vaccine in the ED (57 [25.8%] vs 33 [12.0%]; adjusted difference, 11.9 [95% CI, 4.5-19.3] percentage points; number needed to treat [NNT], 8 [95% CI, 5-22]). More intervention group participants than control participants received a COVID-19 vaccine within 30 days of their ED visit (44 [20.0%] vs 24 [8.7%]; adjusted difference, 7.9 [95% CI, 1.7-14.1] percentage points; NNT, 13 [95% CI, 7-60]). The intervention group had greater outcome effect sizes than the control group in participants who lacked a primary care physician (acceptance, 38 of 101 [37.6%] vs 16 of 117 [13.7%] [P for interaction = .004]; uptake, 31 of 101 [30.7%] vs 11 of 117 [9.4%] [P for interaction = .006]), as well as in Latinx persons (acceptance, 23 of 52 [44.2%] vs 5 of 48 [10.4%] [P for interaction = .004]; uptake, 22 of 52 [42.3%] vs 4 of 48 [8.3%] [P for interaction < .001]).
Conclusions and Relevance
Results of this cluster randomized clinical trial showed that with low NNT, implementation of COVID-19 vaccine messaging platforms in EDs leads to greater vaccine acceptance and uptake in unvaccinated ED patients. Broad implementation in EDs could lead to greater COVID-19 vaccine delivery to underserved populations whose primary health care access occurs in EDs.
Trial Registration
ClinicalTrials.gov Identifier: NCT05142332
Introduction
COVID-19, the illness caused by SARS-CoV-2, has resulted in more than 1 million deaths in the US as of May 20, 2022.1 A quarter to as many as 58% of these deaths could have been prevented by timely COVID-19 vaccination, which has been widely available in the US since the early months of 2021.2,3 With approximately 23% of the US population still reluctant to be vaccinated, vaccine hesitancy persists as the principal barrier to universal COVID-19 vaccination in the US.4,5,6
As a primary health care access point for many underserved groups, especially African American and Latinx persons, and “the safety net of the safety net” for other populations at risk for poor outcomes from COVID-19 infection (eg, immigrants, persons experiencing homelessness, people who are medically uninsured), emergency departments (EDs) are positioned to serve as key public health intervention sites for a large, medically underserved segment of the US population.7,8,9,10,11,12,13 In a study of this population during visits at 15 geographically representative EDs across the US, we previously demonstrated that patients whose primary health care access occurs in EDs had greater COVID-19 vaccine hesitancy and other health care access barriers to COVID-19 vaccination than those with primary care.14 In response to this health care disparity and with a specific goal of decreasing COVID-19 vaccine hesitancy and increasing vaccine uptake in underserved populations, we conducted in-depth qualitative interviews of 65 ED patients who were vaccine hesitant and whose primary health care access occurred in EDs in the summer and fall of 2021. We analyzed interview transcripts with a content-analysis approach for reasons for COVID-19 vaccine hesitancy, other vaccination barriers, trusted messengers, and what specific messages might address their hesitancy.15 We used these findings to develop the ED population-specific Promotion of COVID-19 Va(X)ccination in the Emergency Department (PROCOVAXED) messaging platforms (videos, informational flyers, and scripts for face-to-face ED clinician messaging) to address identified COVID-19 vaccine concerns and promote vaccine uptake.
The objective of this cluster randomized clinical trial was to determine whether implementation of these 3-pronged PROCOVAXED messaging platforms in EDs would be associated with increased COVID-19 vaccine acceptance and uptake in unvaccinated ED patients. We hypothesized that delivery of these messaging platforms during patient waiting times in EDs would be associated with greater COVID-19 vaccine acceptance at the time of ED visit and uptake 30 days after ED visits.
Methods
Study Design
We conducted a cluster randomized clinical trial of implementation of the PROCOVAXED COVID-19 messaging platforms in 7 EDs at a mix of academic, community, and safety net hospitals in 4 US cities: San Francisco, California; Philadelphia, Pennsylvania; Seattle, Washington; and Durham, North Carolina. All study procedures were approved by the University of California, San Francisco Committee on Human Research as a central institutional review board with multisite reliance mechanisms in place at the other sites. We followed the Consolidated Standards of Reporting Trials (CONSORT) reporting guidelines herein and in our previously published full study protocol.16 Procedures are also listed in Supplement 1.
Study Population
Given practical considerations and limits on research personnel in patient care areas during the COVID-19 pandemic that precluded 24/7 study enrollment, we conducted this study during five 6- to 10-hour daytime blocks per week at each site (typically 9 AM to 5 PM Monday through Friday). Research staff reviewed patient age, chief complaint, and COVID-19 vaccination status on ED dashboards to identify potentially eligible patients and queried ED clinicians and patients regarding inclusion and exclusion criteria. We included adults (>17 years old) who were: (1) not already vaccinated for COVID-19; (2) able to provide informed consent; (3) fluent in English or Spanish (Spanish speakers at 2 sites with Spanish-speaking research staff); and (4) anticipated ability to complete the study intervention in the ED (watch the short video). We excluded patients who: (1) were unable to complete a survey because of major trauma, intoxication, altered mental status, or critical illness; (2) were in police custody or incarceration; (3) had a psychiatric chief complaint or hold; (4) had a medical reason (as per the ED clinician or patient) that they could not receive a COVID-19 vaccine; and (5) were suspected of acute COVID-19 illness. Because many patients were receiving COVID-19 testing for surveillance reasons (eg, routine admission testing), performance of a COVID-19 test was not an exclusion; however, if a test resulted positive, the patient was excluded.
Randomization
Using pseudorandom numbers, the study statistician randomized thirty 1-week periods to the intervention group and thirty 1-week periods to the control group. Three centers were in the control condition and 4 centers in the experimental condition at any time (or vice versa) to minimize the effect of vaccine hesitancy secular trends.
Study Procedures and Intervention
All study procedures were conducted during patient waiting times in ED rooms. We obtained scripted verbal consent for enrollment, then administered the intake survey, which asked about participant characteristics and was identical in both study arms. Participants were informed that they would not be compensated for participation (eMethods 1 in Supplement 2).
After the intake survey during intervention weeks, research staff attempted to deliver the intervention, consisting of 3 COVID-19 messaging platforms (Box). Staff asked participants if they were willing to watch a short (4 minute) audiovisual recording about COVID-19 vaccines. If they agreed, research staff showed participants the video on an electronic tablet. After watching the video (or refusal to watch), research staff asked participants if they were willing to read an informational flyer about COVID-19 vaccines. If they agreed, staff handed the participant the flyer. As per qualitative interview summary recommendations, research staff matched videos and informational flyers with participants’ ethnic and racial characteristics, as self-reported during their intake surveys (eg, Latinx messengers on video for Latinx participants). After leaving the room, research staff gave the 6-sentence scripted message to 1 of the participant’s ED clinicians (physician, nurse, or advanced care practitioner) and asked them to deliver the message orally to the patient by reading the message directly or giving it in their own words.
Box. COVID-19 Messaging Platforms Comprising the Intervention.
-
A 4-minute public service announcement–type video (5 versions, all with the same wording in the message but each with a different pair of physician messengers):
1. African American physicians
2. Latinx physicians (English version)
3. Latinx physicians (Spanish version)
4. Physicians of more than 1 race
5. White physicians
-
A single-page informational flyer (5 versions, all with the same format and wording/captions but each with different pictures of patients receiving the vaccine and clinicians administering the vaccines):
1. Predominantly African American patients and clinicians
2. Predominantly Latinx patients and clinicians (English version)
3. Predominantly Latinx patients and clinicians (Spanish version)
4. Patients and clinicians of more than 1 race
5. Predominantly White patients and clinicians
A 6-sentence script for a COVID-19 vaccine message (from an emergency department physician or nurse) on a sheet of paper
Study procedures during nonintervention (control) blocks were identical to procedures in intervention blocks with the exception that participants were not given the intervention; after the intake survey, research staff left the room. Randomization to the control group did not preclude delivery of vaccine messaging by ED clinicians (ie, research staff did not tell clinicians to avoid vaccine messaging).
Approximately 1 hour after the intake survey, research staff administered the vaccine acceptance survey (eMethods 2 and 3 in Supplement 2). Considering variability in ED length of stay and care plans (eg, patients away from their rooms for radiography), research staff were allowed to conduct this survey anytime between 30 minutes and 6 hours after the intake survey. The last vaccine acceptance survey question in both study arms was, “Would you accept the COVID vaccine in the emergency department today if your doctor asked you to get it?” Given that this was an acute care ED setting, we withdrew patients from the study who became too ill to continue participation, had a change in mental status, left suddenly without notification of clinicians, or withdrew consent before the vaccine acceptance survey.
During the study period, the study EDs usually had the capability of administering COVID-19 vaccines; however, sometimes they ran out of vaccines or could not administer them prior to participant discharge. When participants said that they would accept the vaccine in the ED, research staff asked participants if they could notify their clinician and confirm whether they had received it. They did not provide counseling or tell participants that they qualified for the COVID-19 vaccine. When participants agreed to ED clinician notification of vaccine acceptance, research staff notified clinicians and later confirmed receipt or nonreceipt in the ED.
When participants said that they would not accept the vaccine in the ED, research staff asked (through written informed consent and HIPAA [Health Insurance Portability and Accountability Act] document agreements) if they could contact the participants by telephone and review their electronic health records (EHRs) in a month. They also asked for 1-month follow-up in participants who agreed to the vaccine in the ED but did not get it there. Research staff were blinded to study arm and conducted EHR review and telephone follow-up 28 to 32 days later (maximum of 3 telephone call attempts).
Study Outcomes
We designated 2 primary outcomes a priori because we believed that they were equally important: (1) COVID-19 vaccine acceptance in the ED, assessed by participant response to the vaccine acceptance survey question, “Would you accept the COVID vaccine in the emergency department today if your doctor asked you to get it?” (yes indicated acceptance and no or unsure indicated nonacceptance); and (2) COVID-19 vaccine uptake within 30 days of the ED visit, ascertained via confirmation of receipt of a COVID-19 vaccine during participant index ED visit, review of EHR at 30 days, and telephone follow-up at 30 days with the question, “Have you received a COVID-19 vaccine since you were in the emergency department?” Participants who did not have confirmed receipt of a COVID-19 vaccine by 1 of these 3 mechanisms or did not have follow-up after declining the vaccine in the ED were deemed not to have received the vaccine (eFigure in Supplement 2).
Sample Size Considerations
The investigator team reviewed relevant literature and considered the commonality of COVID-19 vaccine hesitancy, high benefit of increasing vaccine acceptance, and negligible risk of the intervention (a trusted messaging program).17,18,19,20,21 In consultation with health policy experts, we determined that even a small increase in vaccine acceptance and uptake would be clinically important. The study sample size of 1290 (645 participants per arm) was therefore selected to detect an absolute increase in vaccine acceptance and uptake of 7% (from a control rate of 15%) with 90% power on a 2-sided 0.05 level test.
Using the Lan-DeMets flexible spending approach with an O’Brien-Fleming boundary, an independent data monitoring committee monitored the trial for early superior efficacy termination criteria.22 Analyses for early termination were planned for the one-third and two-thirds points of study enrollment (430 and 860 participants, respectively). Thirty days after reaching the enrollment target for the first interim analysis on June 8, 2022, we presented data on 431 participants to the interim analysis committee and continued enrollment until we were informed that the study had met early termination criteria on July 28, 2022.
Statistical Analysis
We planned the primary analysis on the intent-to-treat population of participants who consented to initial surveys and were not withdrawn from the study. Participants were analyzed by the study arm (ED visit during intervention period vs visit during control period) to which they were allocated, regardless of whether they received vaccine PROCOVAXED messaging platforms. The 2 primary outcomes were modeled using mixed effects logistic regression with a random center effect to accommodate potential within-center characteristics, as well as terms for time and randomized intervention.23 Results of this analysis were averaged over the distribution of the enrollment times and the random effects for site to obtain the adjusted difference in outcome proportions by arm. A pooled test of the linear coefficients for the 2 primary outcomes was calculated to provide a single test that corrected for multiple comparisons.24,25
We conducted preplanned analyses of patient subgroups, stratifying according to the following characteristics: tertiles of age, gender, race (self-selected as African American, American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, White, or Other), Latinx ethnicity, having primary care, primary language, and previously having had COVID-19. Subgroup differences were considered statistically significant if the omnibus test of treatment by subgroup interaction was significant on a 2-sided 0.05 level test. Using multiple imputation and inverse weighting, we conducted sensitivity analyses for the 30-day vaccine uptake outcome for those who were not vaccinated in the ED and did not have 30-day follow-up. We managed data using REDCap (Vanderbilt University)26 and analyzed data using Stata statistical software, version 17 (StataCorp).
Results
Study Participants
We screened 2327 patients who had unknown COVID-19 vaccination status on ED dashboards; 1022 patients stated that they were already vaccinated, and 201 had other exclusions. Of the remaining 1104 eligible patients, 541 (49.0%) agreed to participate; 22 of these patients left the ED before study procedures could be enacted, 12 became too ill, and 11 withdrew consent. Of the 496 participants, 221 were enrolled during intervention weeks and 275 were enrolled during control weeks (Figure). The median (IQR) age of patients was 39 (30-54) years, 205 (41.3%) were female, 193 (38.9%) were African American, and 97 (19.6%) were Latinx. Additionally, 218 patients (44.0%) lacked primary care physicians, 95 (19.2%) had no health insurance, 47 (9.5%) spoke Spanish primarily, and 87 (17.5%) were unhoused or marginally housed (Table 1).
Figure. CONSORT Flow Diagram.
aParticipants were withdrawn if they became too ill, left the emergency department (ED) prior to completion of study procedures, or asked to be withdrawn.
Table 1. Participant Characteristics.
| Characteristic | No. (%) | ||
|---|---|---|---|
| All (N = 496) | Intervention (n = 221) | Control (n = 275) | |
| Age, median (IQR), y | 39 (30-54) | 39 (31-53) | 39 (29-54) |
| Gender | |||
| Female | 205 (41.3) | 86 (38.9) | 119 (43.2) |
| Male | 289 (58.3) | 134 (60.6) | 155 (56.4) |
| Othera | 2 (0.4) | 1 (0.5) | 1 (0.4) |
| Race and ethnicity | |||
| African American | 193 (38.9) | 71 (32.1) | 122 (44.3) |
| American Indian or Alaska Native | 18 (3.6) | 14 (6.3) | 4 (1.5) |
| Asian | 12 (2.4) | 5 (2.3) | 7 (2.5) |
| Latinx | 97 (19.6) | 50 (23.5) | 47 (17.1) |
| Native Hawaiian or Other Pacific Islander | 10 (2.0) | 4 (1.8) | 6 (2.2) |
| White, non-Latinx | 183 (36.9) | 85 (38.5) | 98 (35.6) |
| Otherb | 48 (9.7) | 23 (10.4) | 25 (9.1) |
| Health insurance provider | |||
| City-based health care service | 28 (5.6) | 12 (5.4) | 16 (5.8) |
| Health maintenance organization/Kaiser | 7 (1.4) | 4 (1.8) | 3 (1.1) |
| Medicaid | 173 (34.8) | 79 (35.7) | 94 (34.2) |
| Medicare | 97 (19.6) | 38 (17.2) | 59 (21.5) |
| None | 95 (19.2) | 47 (21.3) | 48 (17.5) |
| Affordable Care Act | 21 (4.2) | 10 (4.5) | 11 (4.0) |
| Private | 103 (20.8) | 49 (22.2) | 54 (19.6) |
| Other | 23 (4.6) | 11 (5.0) | 12 (4.4) |
| Veterans Health Administration | 5 (1.0) | 2 (0.9) | 3 (1.1) |
| Has a primary care clinic or physician | |||
| Yes | 277 (55.8) | 119 (53.8) | 158 (57.5) |
| No | 218 (44.0) | 101 (45.7) | 117 (42.5) |
| Unsure | 1 (0.2) | 1 (0.5) | 0 |
| Primary language | |||
| English | 435 (87.7) | 196 (88.7) | 239 (86.9) |
| Spanish | 47 (9.5) | 22 (10.0) | 25 (9.1) |
| Other | 14 (2.8) | 3 (1.4) | 11 (4.0) |
| Housing status | |||
| Housed | 408 (82.3) | 172 (77.8) | 236 (85.8) |
| Marginal | 19 (3.8) | 10 (4.5) | 9 (3.3) |
| Unhoused | 68 (13.7) | 38 (17.2) | 30 (10.9) |
Other was self-selected by participants and includes transgender and nonbinary/gender nonconforming individuals.
Other was self-selected by participants if they identified as a race or ethnicity other than one of the listed categories.
Of 221 intervention group participants, 210 (95.0%) received at least 1 of the 3 messaging platforms: 143 participants (64.7%) received the video, 171 (77.4%) received the flyer, and 131 (59.3%) received vaccine messaging from an ED clinician. Of 275 control group participants, 49 (17.8%) stated that they received vaccine messaging from an ED clinician.
Primary Outcomes
More intervention group participants stated that they would accept the vaccine in the ED than control participants (57 [25.8%] vs 33 [12.0%]; absolute difference, 13.8 percentage points; adjusted absolute difference, 11.9 [95% CI, 4.5-19.3] percentage points; number needed to treat [NNT], 8 [95% CI, 5-22]). More intervention group participants received a COVID-19 vaccine within 30 days of their ED visit than control participants (44 [20.0%] vs 24 [8.7%]; absolute difference, 11.3 percentage points; adjusted absolute difference, 7.9 [95% CI, 1.7-14.1] percentage points; NNT, 13 [95% CI, 7-60]) (Table 2). Sensitivity analyses for 30-day uptake yielded similar adjusted differences (multiple imputation, 8.8 [95% CI, 0.6-17.1] percentage points; inverse probability weighting, 7.8 [95% CI, 0-15.6] percentage points; eTable 1 in Supplement 2). Vaccinations at 30 days outside of the ED were rarely reported, with 4 of 108 participants (3.7%) in the intervention group and 9 of 141 participants (6.4%) in the control group receiving vaccines.
Table 2. Primary Outcomes, Unadjusted Analysis and Adjusted for Site and Calendar Time.
| Primary outcome | Unadjusted analysis, No. (%) | Adjusted for site and calendar time, % | |||||
|---|---|---|---|---|---|---|---|
| Intervention (n = 221) | Control (n = 275) | Percentage point change (95% CI) | Intervention | Control | Percentage point change (95% CI) | P value | |
| Would accept COVID-19 vaccine in the emergency department, if offered | 57 (25.8) | 33 (12.0) | 13.8 (6.9-20.7) | 22.2 | 10.3 | 11.9 (4.5-19.3) | <.001 |
| Received COVID-19 vaccine within 30 d | 44 (20.0) | 24 (8.7) | 11.2 (4.9-17.4) | 14.6 | 6.7 | 7.9 (1.7-14.1) | <.001 |
Subgroup Analyses
The intervention group, when compared with the other subgroups analyzed, had greater outcome effect sizes in participants who lacked a primary care physician (acceptance, 38 of 101 [37.6%] vs 16 of 117 [13.7%] [P for interaction = .004]; uptake, 31 of 101 [30.7%] vs 11 of 117 [9.4%] [P = .006]), and in Latinx persons (acceptance, 23 of 52 [44.2%] vs 5 of 48 [10.4%] [P for interaction = .004]; uptake, 22 of 52 [42.3%] vs 4 of 48 [8.3%] [P for interaction < .001]) (Table 3 and eTables 2 and 3 in Supplement 2). Effect sizes of the intervention group increased with increasing numbers of platforms received (eTable 4 in Supplement 2).
Table 3. Subgroup Analyses, Adjusted for Site and Calendar Time.
| Characteristic | Total, No. (%) (N = 496) | Accepted COVID-19 vaccine in emergency department, % | Vaccinated in 30 d, % | ||||
|---|---|---|---|---|---|---|---|
| Intervention | Control | Percentage point change (95% CI) | Intervention | Control | Percentage point change (95% CI) | ||
| Age tertiles, y | |||||||
| 18-32 | 172 (34.5) | 21.0 | 11.5 | 9.5 (−1.4 to 20.4) | 12.4 | 4.1 | 8.2 (−0.3 to 16.8) |
| 33-49 | 160 (32.3) | 21.8 | 10.0 | 11.8 (0.8 to 22.8) | 17.5 | 7.9 | 9.7 (−0.1 to 19.5) |
| 50-85 | 164 (33.1) | 24.0 | 9.4 | 14.7 (2.5 to 26.8) | 13.4 | 8.1 | 5.3 (−3.6 to 14.3) |
| Gender | |||||||
| Female | 205 (41.3) | 16.9 | 10.4 | 6.5 (2.7 to 15.8) | 11.0 | 7.1 | 3.9 (−3.7 to 11.4) |
| Male | 289 (58.3) | 26.0 | 9.8 | 16.1 (6.4 to 25.9) | 17.1 | 6.5 | 10.6 (2.4 to 18.8) |
| Race | |||||||
| African American | 192 (38.7) | 22.6 | 12.6 | 10.0 (1.0 to 21.0) | 10.2 | 7.3 | 2.9 (−4.7 to 10.5) |
| Asian/Other Pacific Islandera | 20 (4.0) | 0 | 6.3 | −6.3 (−18.7 to 6.1) | 5.5 | 4.4 | 11.0 (−12.5 to 14.8) |
| White | 197 (39.7) | 21.6 | 8.5 | 13.1 (2.5 to 23.7) | 16.2 | 6.1 | 10.1 (0.7 to 19.5) |
| Otherb | 87 (17.5) | 29.3 | 9.8 | 19.4 (3.5 to 35.4) | 22.0 | 7.8 | 14.2 (0.3 to 28.0) |
| Ethnicity | |||||||
| Latinx | 100 (20.2) | 37.8 | 8.8 | 29.0 (11.8 to 46.2) | 29.2 | 5.4 | 23.8 (7.7 to 39.8) |
| Not Latinx | 396 (79.8) | 18.5 | 11.1 | 7.4 (0.2 to 14.6) | 11.0 | 7.5 | 3.4 (−2.1 to 9.0) |
| Have primary care | |||||||
| No | 218 (44.0) | 33.6 | 12.1 | 21.5 (9.4 to 33.6) | 21.8 | 6.8 | 15.0 (4.5 to 25.5) |
| Yes | 277 (55.8) | 14.0 | 9.6 | 4.4 (−3.1 to 11.9) | 9.6 | 7.3 | 2.3 (−4.0 to 8.6) |
| Primary language | |||||||
| English | 435 (87.7) | 20.4 | 10.7 | 9.7 (2.4 to 17.0) | 13.3 | 7.0 | 6.3 (0.3 to 12.3) |
| Spanish | 47 (9.5) | 37.8 | 9.7 | 28.2 (4.7 to 51.6) | 28.0 | 6.8 | 21.2 (0.9 to 41.5) |
| Other | 14 (2.8) | 54.7 | 7.8 | 46.9 (−13.1 to 106.9) | 19.7 | 5.4 | 14.4 (−24.0 to 52.8) |
| Previously had COVID-19 | |||||||
| No | 324 (65.3) | 23.5 | 11.4 | 12.1 (3.2 to 21.0) | 15.6 | 7.2 | 8.4 (1.0 to 15.9) |
| Unsure | 8 (1.6) | 63.2 | 0 | 63.2 (7.9 to 118.5) | 27.0 | 0 | 27.0 (20.4 to 74.3) |
| Yes | 164 (33.1) | 18.6 | 8.6 | 9.9 (−0.4 to 20.3) | 12.4 | 6.1 | 6.3 (−2.2 to 14.8) |
These races were grouped together owing to small sample sizes.
Other was self-selected by participants if they identified as a race or ethnicity other than one of the listed categories.
Discussion
We have demonstrated that the delivery of PROCOVAXED COVID-19 vaccine messaging platforms leads to greater COVID-19 vaccine acceptance and uptake in unvaccinated ED patients. These effects on acceptance and uptake are particularly pronounced in Latinx persons and patients who lack primary care—2 groups who have experienced disproportionate morbidity and mortality during the COVID-19 pandemic.27,28,29,30,31
The COVID-19 pandemic has highlighted multiple health care disparities, including the problem that traditional primary care clinic-based public health messaging and interventions often cannot reach the populations who need them most.32 In our previous national study, more than a fifth of people who would accept a COVID-19 vaccine reported that they had no primary care or other place to get it.14 The overarching premise of our research is that to address this public health delivery gap and reach a number of medically underserved populations in the US (and elsewhere), you must go where they go for care: the ED. In this regard, the present study population was diverse (62% were African American, Latinx, or Native American) and medically underserved (nearly half lacked primary care and a fifth lacked health insurance), and had disproportionately high percentages of other social determinants of health susceptibilities (housing insecurity and a primary language that was not English). Notably, the finding of high intervention effect size in Latinx participants supports another underlying principle of our work—that diverse populations may be more receptive to messages when they are delivered by clinicians who look and speak like them.
The high number of ED visits across the US (approximately 160 million in 2019)33 translates to potential for substantial effect of our work. If implemented nationally in EDs, an 8% to 12% increase in vaccine uptake and acceptance could lead to the delivery of tens of thousands of COVID-19 vaccines to people who would not otherwise get vaccinated. In terms of scalability, the 3 messaging platforms (videos, flyers, and scripts) are freely available on the National Institutes of Health Disaster Research Response Resources open access portal.34 Although we presented videos on electronic tablets, they can instead be shown on televisions and kiosks or accessed by patients on personal smartphones using quick response codes during waiting times in EDs.
These findings support the notion that interventions that are developed systematically and specifically for these underserved populations can lead to improvements in public health outcomes. A logical extension is the ED development of other vaccine messaging, especially for influenza and pneumococcal vaccines, which have been offered in some EDs since the early 1990s.35,36,37,38 Beyond vaccination campaigns, our research lays the groundwork for the real-time ED delivery of other public health measure messaging, such as for hypertension and diabetes, to vulnerable populations who lack access to traditional clinic-based primary care.
Given that most participants who received the COVID-19 vaccine received it during their ED visit and not at other health care sites, future efforts should be focused on getting shots in arms in EDs after messaging, rather than waiting or directing them elsewhere for vaccination. Many participants who said that they would accept the vaccine in the ED did not get it there, indicating a need for optimization of ED vaccine supply and delivery workflow protocols.
Limitations
When we conducted the preliminary study underlying this research in December 2020, more than 40% of the patients in the EDs expressed vaccine hesitancy.14 However, by the initiation of this trial a year later, 76% to 80% of the 4 states’ populations had been vaccinated.39 The remaining limited pool of patients who had not received vaccines may have been particularly resistant to COVID-19 vaccine messaging, possibly limiting the absolute effect size of the intervention.
Although we conducted the messaging platform content development qualitative interviews and cluster randomized clinical trial in a mix of public (county), tertiary care, and community hospital EDs, all were located in urban, coastal cities; the messaging platform interventions may be less effective in dissimilar ED populations. Nevertheless, we believe that the methods and messaging platform production can be easily adapted to other ED (and urgent care) settings. We have deposited the interview guides into the National Institutes of Health Disaster Research Response Resources open access portal to allow for refinement and adaptation to other populations.34
Approximately 30% of participants who did not get vaccinated in the ED declined follow-up, limiting 30-day follow-up ascertainment in these participants to their ED vaccination status. Nevertheless, the proportions who refused follow-up were nearly identical between study arms, their characteristics were similar, and the sensitivity analyses showed no meaningful difference in 30-day vaccine uptake effect size as compared with the primary analysis.
Conclusions
Results of this cluster randomized clinical trial showed that with a low NNT and higher effect sizes in Latinx persons and participants who lacked primary care, delivery of PROCOVAXED vaccine messaging platforms in EDs resulted in greater COVID-19 vaccine acceptance and uptake in unvaccinated ED patients. If implemented nationally in EDs, these messaging platforms could lead to greater COVID-19 vaccine delivery to underserved populations whose primary health care access occurs in EDs.
Trial Protocol
eMethods 1. PROCOVAXED First Survey
eMethods 2. Vaccine Acceptance Survey: Intervention Group
eMethods 3. Vaccine Acceptance Survey: Control Group
eFigure. Study Flow and Procedures
eTable 1. Proportion Receiving COVID-19 Vaccine at 30 Days: Sensitivity Analyses for Those With No Follow-Up Outside the ED
eTable 2. Subgroup Analyses: Unadjusted Analysis
eTable 3. Interaction P Values for Treatment Effect Across Characteristics
eTable 4. Outcomes by Number of Messaging Platforms Received by Participants
Nonauthor Collaborators. PROCOVAXED Study Network
Data Sharing Statement
References
- 1.COVID data tracker. Centers for Disease Control and Prevention . Accessed November 18, 2022. https://covid.cdc.gov/covid-data-tracker/#trends_totaldeaths_select_00
- 2.Amin K, Ortaliza J, Cox C, Michaud J, Kates J. COVID-19 mortality preventable by vaccines. Petersen-KFF Health Systems Tracker . April 21, 2022. Accessed November 18, 2022. https://www.healthsystemtracker.org/brief/covid19-and-other-leading-causes-of-death-in-the-us/
- 3.Steele MK, Couture A, Reed C, et al. Estimated number of COVID-19 infections, hospitalizations, and deaths prevented among vaccinated persons in the US, December 2020 to September 2021. JAMA Netw Open. 2022;5(7):e2220385. doi: 10.1001/jamanetworkopen.2022.20385 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.KFF COVID-19 vaccine monitor dashboard. Kaiser Family Foundation . Accessed November 18, 2022. https://www.kff.org/coronavirus-covid-19/dashboard/kff-covid-19-vaccine-monitor-dashboard/
- 5.Szilagyi PG, Thomas K, Shah MD, et al. National trends in the US public’s likelihood of getting a COVID-19 vaccine—April 1 to December 8, 2020. JAMA. 2020;325(4):396-398. doi: 10.1001/jama.2020.26419 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Siegler AJ, Luisi N, Hall EW, et al. Trajectory of COVID-19 vaccine hesitancy over time and association of initial vaccine hesitancy with subsequent vaccination. JAMA Netw Open. 2021;4(9):e2126882. doi: 10.1001/jamanetworkopen.2021.26882 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kellermann AL, Hsia RY, Yeh C, Morganti KG. Emergency care: then, now, and next. Health Aff (Millwood). 2013;32(12):2069-2074. doi: 10.1377/hlthaff.2013.0683 [DOI] [PubMed] [Google Scholar]
- 8.Lane BH, Mallow PJ, Hooker MB, Hooker E. Trends in United States emergency department visits and associated charges from 2010 to 2016. Am J Emerg Med. 2020;38(8):1576-1581. doi: 10.1016/j.ajem.2019.158423 [DOI] [PubMed] [Google Scholar]
- 9.HCUP fast stats: trends in emergency department visits. Agency for Healthcare Research and Quality . Accessed November 18, 2022. https://www.hcup-us.ahrq.gov/faststats/NationalTrendsEDServlet
- 10.Weber EJ, Showstack JA, Hunt KA, Colby DC, Callaham ML. Does lack of a usual source of care or health insurance increase the likelihood of an emergency department visit? results of a national population-based study. Ann Emerg Med. 2005;45(1):4-12. doi: 10.1016/j.annemergmed.2004.06.023 [DOI] [PubMed] [Google Scholar]
- 11.Health, United States. Centers for Disease Control and Prevention . Accessed November 18, 2022. https://www.cdc.gov/nchs/hus/contents2017.htm
- 12.Walls CA, Rhodes KV, Kennedy JJ. The emergency department as usual source of medical care: estimates from the 1998 National Health Interview Survey. Acad Emerg Med. 2002;9(11):1140-1145. doi: 10.1197/aemj.9.11.1140 [DOI] [PubMed] [Google Scholar]
- 13.Tang N, Stein J, Hsia RY, Maselli JH, Gonzales R. Trends and characteristics of US emergency department visits, 1997-2007. JAMA. 2010;304(6):664-670. doi: 10.1001/jama.2010.1112 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rodriguez RM, Torres JR, Chang AM, et al. ; REVVED UP Investigators . The rapid evaluation of COVID-19 vaccination in emergency departments for underserved patients study. Ann Emerg Med. 2021;78(4):502-510. doi: 10.1016/j.annemergmed.2021.05.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Gentsch AT, Butler J, O’Laughlin K, et al. Perspectives of COVID-19 vaccine hesitant emergency department patients to inform messaging platforms to promote vaccine uptake. Acad Emerg Med. Published online October 30, 2022. doi: 10.1111/acem.14620 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rodriguez RM, O’Laughlin K, Eucker SA, et al. PROmotion of COvid-19 VA(X)ccination in the Emergency Department-PROCOVAXED: study protocol for a cluster randomized controlled trial. Trials. 2022;23(1):332. doi: 10.1186/s13063-022-06285-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Larson HJ, Jarrett C, Eckersberger E, Smith DMD, Paterson P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: a systematic review of published literature, 2007-2012. Vaccine. 2014;32(19):2150-2159. doi: 10.1016/j.vaccine.2014.01.081 [DOI] [PubMed] [Google Scholar]
- 18.Jarrett C, Wilson R, O’Leary M, Eckersberger E, Larson HJ; SAGE Working Group on Vaccine Hesitancy . Strategies for addressing vaccine hesitancy—a systematic review. Vaccine. 2015;33(34):4180-4190. doi: 10.1016/j.vaccine.2015.04.040 [DOI] [PubMed] [Google Scholar]
- 19.Schmid P, Rauber D, Betsch C, Lidolt G, Denker M-L. Barriers of influenza vaccination intention and behavior—a systematic review of influenza vaccine hesitancy, 2005-2016. PLoS One. 2017;12(1):e0170550. doi: 10.1371/journal.pone.0170550 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Reiter PL, Pennell ML, Katz ML. Acceptability of a COVID-19 vaccine among adults in the United States: how many people would get vaccinated? Vaccine. 2020;38(42):6500-6507. doi: 10.1016/j.vaccine.2020.08.043 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Paltiel AD, Schwartz JL, Zheng A, Walensky RP. Clinical outcomes of a COVID-19 vaccine: implementation over efficacy. HealthAffairs . January 2021. Accessed November 18, 2022. https://www.healthaffairs.org/doi/pdf/10.1377/hlthaff.2020.02054 [DOI] [PMC free article] [PubMed]
- 22.Proschan MA, Lan KKG, Wittes JT. Spending functions. Statistical Monitoring of Clinical Trials: A Unified Approach. Springer; 2008: 81-97. [Google Scholar]
- 23.Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182-191. doi: 10.1016/j.cct.2006.05.007 [DOI] [PubMed] [Google Scholar]
- 24.Pan Q. Multiple hypotheses testing procedures in clinical trials and genomic studies. Front Public Health. 2013;1:63. doi: 10.3389/fpubh.2013.00063 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Wei LJ, Lachin JM. Two-sample asymptotically distribution-free tests for incomplete multivariate observations. J Am Stat Assoc. 1984;9:653-661. doi: 10.1080/01621459.1984.10478093 [DOI] [Google Scholar]
- 26.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. doi: 10.1016/j.jbi.2008.08.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Karmakar M, Lantz PM, Tipirneni R. Association of social and demographic factors with COVID-19 incidence and death rates in the US. JAMA Netw Open. 2021;4(1):e2036462. doi: 10.1001/jamanetworkopen.2020.36462 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Mackey K, Ayers CK, Kondo KK, et al. Racial and ethnic disparities in COVID-19-related infections, hospitalizations, and deaths: a systematic review. Ann Intern Med. 2021;174(3):362-373. doi: 10.7326/M20-6306 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Magesh S, John D, Li WT, et al. Disparities in COVID-19 outcomes by race, ethnicity, and socioeconomic status: a systematic review and meta-analysis. JAMA Netw Open. 2021;4(11):e2134147. doi: 10.1001/jamanetworkopen.2021.34147 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Khanijahani A, Iezadi S, Gholipour K, Azami-Aghdash S, Naghibi D. A systematic review of racial/ethnic and socioeconomic disparities in COVID-19. Int J Equity Health. 2021;20(1):248. doi: 10.1186/s12939-021-01582-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Tai DBG, Shah A, Doubeni CA, Sia IG, Wieland ML. The disproportionate impact of COVID-19 on racial and ethnic minorities in the United States. Clin Infect Dis. 2021;72(4):703-706. doi: 10.1093/cid/ciaa815 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Healthy people 2030: building a healthy future for all. US Department of Health and Human Services . Accessed November 18, 2022. https://health.gov/healthypeople
- 33.National emergency department inventory: USA. Emergency Medicine Network . Accessed November 18, 2022. https://www.emnet-usa.org/research/studies/nedi/nedi2019/usa/
- 34.Disaster Research Response (DR2) Resources Portal. National Institute of Environmental Health Sciences . Accessed November 22, 2022. https://tools.niehs.nih.gov/dr2/index.cfm/resource/24262
- 35.Rodriguez RM, Baraff LJ. Emergency department immunization of the elderly with pneumococcal and influenza vaccines. Ann Emerg Med. 1993;22(11):1729-1732. doi: 10.1016/S0196-0644(05)81313-7 [DOI] [PubMed] [Google Scholar]
- 36.Wrenn K, Zeldin M, Miller O. Influenza and pneumococcal vaccination in the ED: is it feasible? J Gen Intern Med. 1994;9:425-429. doi: 10.1007/BF02599056 [DOI] [PubMed] [Google Scholar]
- 37.Slobodkin D, Zielske PG, Kitlas JL, et al. Demonstration of the feasibility of ED immunization against influenza and pneumococcus. Ann Emerg Med. 1998;32:537-543. [PubMed] [Google Scholar]
- 38.Stack SJ, Martin DR, Plouffe JF. An emergency department-based pneumococcal vaccination program could save money and lives. Ann Emerg Med. 1999;33(3):299-303. doi: 10.1016/S0196-0644(99)70366-5 [DOI] [PubMed] [Google Scholar]
- 39.Mathieu E, Ritchie H, Rodés-Guirao L, et al. Coronavirus pandemic (COVID-19). Our World in Data . Accessed November 18, 2022. https://ourworldindata.org/coronavirus
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Trial Protocol
eMethods 1. PROCOVAXED First Survey
eMethods 2. Vaccine Acceptance Survey: Intervention Group
eMethods 3. Vaccine Acceptance Survey: Control Group
eFigure. Study Flow and Procedures
eTable 1. Proportion Receiving COVID-19 Vaccine at 30 Days: Sensitivity Analyses for Those With No Follow-Up Outside the ED
eTable 2. Subgroup Analyses: Unadjusted Analysis
eTable 3. Interaction P Values for Treatment Effect Across Characteristics
eTable 4. Outcomes by Number of Messaging Platforms Received by Participants
Nonauthor Collaborators. PROCOVAXED Study Network
Data Sharing Statement