System Interventions to Reduce Disparities in Covid-19 Vaccine Offer Rates

A simple strategy to reduce disparities in Covid-19 vaccination is to consistently offer the vaccine during primary care visits, a process that can be measured using standardized documentation practices.

Summary

The Covid-19 pandemic has exposed stark disparities in health outcomes among communities of color. Black patients are at higher risk of being infected with and dying from Covid-19 but are less likely to be vaccinated relative to the overall population. At the San Francisco Health Network in March 2021, the vaccination rate for primary care patients aged 65 and older was 69.8% overall versus just 61.5% for Black patients. In this article, the authors share data that Black patients had lower documented offer rates for the Covid-19 vaccine in their primary care network. The network iterated a series of interventions to address the disparity in vaccine offer rate documentation and used process measure data and stakeholder engagement to understand this disparity. They observed a 7–percentage-point gap in March 2021, which briefly narrowed to a 4-point difference but increased to 5 to 6 points for most of 2021. By March 2022, the disparity had decreased to 2 points, and the offer rates for all patients reached 95% and for Black patients, 93%. This improvement journey generated conversations about implicit bias and burnout and ultimately led to reduced disparities in both the documented offer rate and the vaccination rate for Black patients in the network. In navigating conversations about implicit bias, the network found it most effective to focus on actionable system-level changes rather than on blaming individual behaviors.

Key Takeaways

• Leverage data to identify disparities in quality metrics. We learned that we cannot successfully intervene on disparities if we do not measure them.

• Share data widely and transparently. While a first-pass analysis may focus on disparities in outcome measures, we must also look closely at process measures to uncover root causes and opportunities for improvement.

• Engage diverse voices, including those of patients and frontline staff, to understand the nuances of a complex problem.

• Recognize the limits of quantitative data. Stubborn problems often require on-site observation and discussion with patients and those who do the work each day. Qualitative data pave the way for important insights and innovative solutions.

• Talk openly with your teams about implicit bias. Acknowledge that bias is universal and something that must be addressed through system fixes rather than blaming individuals. An effective way to overcome bias is through the transparent use of data and adherence to standard work.

• Counter the burnout associated with vaccine counseling by showing that repeatedly offering patients the vaccine makes a difference. Our data suggest that consistently offering the vaccine results in higher vaccination rates and lower disparities for Black patients.

The Challenge

Covid-19 has disproportionately impacted communities of color. Black, Latinx, and American Indian/Alaskan Native individuals are at greater risk of contracting Covid-19 and have higher rates of hospitalization and death.^1,2 Despite wide availability of vaccines, vaccination rates in these communities lag behind the general population.^3,4 The root causes for these disparities are complex.^5-7 High-effort countermeasures, including vaccine pop-up sites in low-income neighborhoods,⁸ door-to-door campaigns, recruitment of community navigators,⁹ and provider training, have been implemented to reduce vaccine disparities.¹⁰ One simple strategy to improve vaccine uptake, however, is to consistently offer patients the vaccine during primary care visits. To our knowledge, no data have been published on how often the vaccine is offered to patients in this setting.

Implicit bias refers to unconscious assumptions we make about others.¹¹ Most health care providers show evidence of implicit bias that negatively impacts Black, Latinx, and dark-skinned people.^12-14 People of color, for example, are less likely to be offered preventive health services such as smoking cessation counseling or the influenza vaccine,^15,16 despite the well-known benefits of these interventions.¹⁷

As leaders in primary care quality, equity, and operations, we sought to understand whether disparities existed in offering the Covid-19 vaccine to Black patients in the San Francisco Health Network (SFHN). SFHN is a safety-net health care system in San Francisco serving approximately 100,000 low-income Medicaid, undocumented, and uninsured patients. Our network consists of 14 primary care clinics, dozens of specialty clinics, and a 397-staffed-bed urban county hospital.¹⁸

The Goal

Address disparities in Covid-19 vaccination outcomes among Black patients by:

1. Quantifying disparities in vaccine offer rate between the overall active primary care population and Black patients in primary care

2. Assessing whether the documented vaccine offer rate correlates with higher rates of vaccine administration

3. Using data and iterative cycles of improvement to reduce disparities in offer rate

4. Decreasing disparity in vaccination rate

We set a department-wide goal of documenting a vaccine offer to at least 95% of patients with a primary care office or telehealth encounter, with a 0% disparity for Black patients.

The Execution

Quantifying Disparities

Shortly after Covid-19 vaccines became widely available early in 2021, we observed lower vaccination rates among our Black patients, consistent with nationwide trends.¹⁹ In March 2021, the overall vaccination rate for our patients aged 65 and older was 69.8% versus 61.5% for our Black patients aged 65 and older. To better understand this disparity, we began collecting data in March 2021 on vaccine offer rates during primary care visits in our network. To collect offer rate data, we leveraged a care gap alert in our electronic health record (Figure 1).

Figure 1.

Covid-19 Vaccine Care Gap Alert

Auth/Cert = authorization/precertification, DPH = Department of Public Health. Source: San Francisco Health Network Electronic Health Record

The care gap alert is visible to providers and staff accessing the patient’s chart. We defined the denominator for vaccine offer rate as any patient who was eligible for the first dose of the Covid-19 vaccine and had engaged in either an office or telehealth visit in primary care. The numerator included all patients whose care gap was resolved by: (1) administering a vaccine, (2) scheduling a future vaccine appointment, (3) entering a historical immunization, or (4) documenting a decline or postponement (Figure 2).

Figure 2.

Documentation of Covid-19 Vaccine Decline/Postponement

DPH = Department of Public Health. Source: San Francisco Health Network Electronic Health Record

We set a department-wide goal of documenting a vaccine offer to at least 95% of patients with a primary care office or telehealth encounter, with a 0% disparity for Black patients.

Iterative Cycles of Improvement

Phase 1: Disseminating Disparities Data

Our first report of vaccine offer rate in March 2021 suggested that Black patients were being offered the Covid-19 vaccine less often (74%) than the overall active primary care population (81%) (Figure 3).

Figure 3.

Covid-19 Vaccine Offer Rate, March 2021–March 2022

Source: The authors and San Francisco Health Network Electronic Health Record

To communicate this disparity with clinic teams, we disseminated the data in multiple ways. Reports were emailed out to primary care clinic management teams weekly and were periodically reviewed at twice-monthly management meetings. These meetings served as a venue to solicit feedback on the process and to reinforce documentation workflows. Management teams were encouraged to review the reports with staff at huddles and to post the graphs on clinic data walls. In addition, a Missed Opportunities report (Figure 4) was developed to help managers identify clinic-specific disparities by race/ethnicity and coach staff and providers who needed additional training in the documentation workflow.

Figure 4.

Covid-19 Vaccine Missed Opportunities Report

DPH = Department of Public Health, HC = health center, HM = health management, MRN = Medical Record Number, PC = primary care. Source: San Francisco Health Network Electronic Health Record

Phase 2: Input from Vaccine Equity Committee

While we initially saw a reduction in offer rate disparity from 7.0 percentage points to 4.0 percentage points after 1 month simply by sharing and talking about the data, disparities in documented offer rates persisted. To brainstorm solutions, we consulted with our SFHN Vaccine Equity Committee, a 15-member cross-departmental collaboration that brings together patient advisors²⁰ and system leaders. This committee was created at the behest of SFHN executive leadership in April 2021 to address disparities in vaccination rates and vaccine offer rates. The SFHN Vaccine Equity Lead chaired the committee and provided periodic status updates to SFHN leadership. Feedback from this committee included the suggestions in Table 1.

Table 1. Input from SFHN Vaccine Equity Committee: Strategies for Reducing Disparities in Documented Vaccine Offer Rate^*.

Recommendation 1	Conduct provider training on how to offer vaccine and have difficult conversations.
Recommendation 2	Retrain clinics on documenting vaccine offer.
Recommendation 3	Perform site visit to clinics with greater disparities in offer rate to explore root causes and solutions.

^*SFHN = San Francisco Health Network. Source: The authors

Patient advisors, who make up three of the 15 members of the Vaccine Equity Committee, contributed content for a primary care provider training. The group suggested incorporating several key elements: recognize the connection between the history of racism perpetrated by the medical system and vaccine hesitancy among Black patients, review antivaccine propaganda to provide insight into these narratives, ask patients for permission to talk about the vaccine, and open the conversation with a statement of caring for patients’ well-being.

As we navigated conversations with clinic managers and providers about implicit bias, we found it most effective to focus on actionable system-level interventions such as trainings and transparent data sharing rather than on blaming individual behaviors.

The training occurred in October 2021 at a dedicated meeting with approximately 50 primary care providers across our network. This 20-minute virtual training was developed and presented by the SFHN Vaccine Equity Lead. The intention of the training was to: (1) call attention to disparities in documented vaccine offer rates, (2) place disparities in the larger context of the history of medical racism, (3) highlight the importance of consistently offering the vaccine and documenting correctly, and (4) equip providers with vaccine talking points (Table 2).

Table 2. Talking Points from Vaccine Equity Training for Providers.

Vaccine Talking Points	Suggested Script for Providers
Validate concerns and address trauma	“I hear you. It is kind of scary how rapidly the vaccine was created and how quickly the recommendations are changing every day.”
Find mutual truths and common ground	“It is frustrating that people are still getting Covid-19 even though they got vaccinated. I wish there was a perfect prevention strategy.”
Redirect misinformation with facts	“Even if people who are vaccinated get Covid-19, they tend to be much less sick and don’t need to be hospitalized, compared to people who get Covid-19 who weren’t vaccinated.”
Show respect and understanding	“It is a difficult decision, and we respect your decision! It is a lot to think about.”
Keep door open for continued medical services and getting the vaccine later	“We are still your providers no matter what. Please feel free to come back if you want to talk about this in more detail. Here is information on how to get vaccinated if you change your mind.”

Source: The authors

Phase 3: Site Visit Observations and Interviews

While the provider-based training modestly reduced gaps in vaccine offer rates, we continued to observe small, persistent disparities in our data. At the suggestion of the Vaccine Equity Committee, the Equity Lead and Director of Population Health conducted site visits at several primary care clinics to carry out structured observations and stakeholder interviews. We selected four clinics with documented offer rate disparities exceeding 5 percentage points. At each site, we interviewed at least two medical assistants or nurses and three providers (physicians, resident physicians, and advanced practice providers). Sites had clinical staff full-time equivalent ranging from 12 to 32. We also observed patient–provider interactions when possible. We queried the providers and staff about: (1) their comfort with and approach to offering the vaccine, (2) their process for documenting vaccine offer, and (3) challenges with talking about the vaccine.

We analyzed the qualitative feedback by using affinity clustering²¹ to categorize our observations. To translate observations into countermeasures, we used multivoting, a Lean Six Sigma technique, and identified four actionable interventions (Table 3).

Table 3. Clinic Site Visits: Observations and Countermeasures to Reduce Disparity in Vaccine Offer.

Observation	Countermeasure
Vaccine not consistently available in clinics at all times	Ensure clinics have Covid-19 vaccine on site during all hours of operation
Ambiguous ownership of vaccine counseling and documentation between staff and providers	Standardize workflow for vaccine counseling and documentation
Provider perception that vaccine counseling does not change patients’ minds	Disseminate data linking higher documented offer rate with: (1) greater vaccine uptake, and (2) smaller disparities in vaccine administration
No educational material about the vaccine	Distribute vaccine handouts for clinic waiting areas and examination rooms

Source: The authors

Hurdles

Questioning the Ask

Our data on vaccine offer rate are a process measure²² documenting whether vaccine was offered to patients. As such, the data are only accurate to the extent that the documentation process is followed. Despite data showing that accurate documentation improves quality metrics,²³ we observed resistance among providers to the requirement that they document in a standard way. Some voiced a belief that documentation does not correlate to patient outcomes. One provider commented during a structured interview: “I don’t get why it matters how we document.” In response to this feedback, we continuously reinforced the rationale for documenting vaccine offer in a capturable way; namely, that we cannot successfully intervene on disparities if we do not measure them.

Using Data to Justify the Ask

Our data raised some uncomfortable questions about implicit bias, an unconscious behavior that leads to differential treatment. Some managers, for example, expressed discomfort in sharing disparity data with their teams, citing concerns that the data would be received as an accusation of racism. Acknowledging one’s own implicit bias can be difficult, but it is possible to mitigate bias by consistently following standard processes and using data to identify disparities. As we navigated conversations with clinic managers and providers about implicit bias, we found it most effective to focus on actionable system-level interventions such as trainings and transparent data sharing rather than on blaming individual behaviors.

In our messaging to providers and staff, being able to demonstrate a clear relationship between process measure and outcome measure helped reinforce the importance of following the standard documentation workflow.

To bolster the argument that documenting accurately mattered, we examined the correlation between vaccine offer for Black patients and vaccination rate among Black patients (Figure 5).

Figure 5.

Vaccine Offer Rate Compared with Vaccination Rate for Black Patients, April 2021–March 2022

Source: The authors

Lastly, we assessed the number of times the vaccine was offered and declined before the patient agreed to be vaccinated (Figure 6).

Figure 6.

Total Number of Documented “Postponed” Before Completing Covid-19 Vaccination for Patients Ages 5 and Up, March 2021–April 2022

BAA = Black/African American. Source: San Francisco Health Network Electronic Health Record

In our messaging to providers and staff, being able to demonstrate a clear relationship between process measure and outcome measure helped reinforce the importance of following the standard documentation workflow. We shared these data through various channels, including newsletters, trainings, webinars, and management meetings.

Anecdotally, we found it unusual for a patient to say, “Don’t ask me about the vaccine again.” As long as the vaccine was offered in a respectful way that honored the patient’s autonomy, patients tended to understand it is the provider’s responsibility to offer the vaccine, even after repeated declinations. Further, we witnessed patient beliefs surrounding the vaccine evolve over time; patients who were initially resistant to vaccination ultimately changed their minds for a variety of reasons. Table 4 includes representative patient quotes from a survey conducted at an SFHN vaccine site in June 2021, after the vaccine had been widely available for almost 2 months. Patients were queried about their reasons for getting vaccinated in June rather than earlier.

Table 4. Survey of Patients at SFHN Vaccine Site Regarding Attitudes Toward the Covid-19 Vaccine, June 2021^*.

Survey Question	Representative Quote
Why did you decide to get vaccinated now?	My son got the second dose shot a week ago and kept asking me to get it. The father of my son also got the vaccine a while ago. I thought to myself, “Ok, I’m the last one.”
When did you start considering whether to get the vaccine?	I probably wouldn’t get it if I didn’t need to for my job. Only when other countries started to roll out vaccines and increase supply did I really consider it.
Was there anything stopping you from getting the vaccine?	I was against it at first because my body should be able to fight viruses on its own. I am still kind of against it. Not that I think you’re lying, but you could be injecting something else that’s not the vaccine in me. When my dad got it and my older sister got it, that convinced me to get it.

^*SFHN = San Francisco Health Network. Source: The authors

Variable Comfort with Vaccine Counseling

During sites visits, we observed significant variability in vaccine counseling practices among medical assistants. At some sites, medical assistants proactively initiated vaccine conversations with patients. At other sites, medical assistants largely deferred the discussion to providers.

When asked about their comfort in talking about the vaccine, two different medical assistants cited assessing patient body language²⁴ to gauge if the patient wanted to talk about the vaccine. Relying on subjective assessments rather than standard protocols could allow implicit bias to influence which patients receive vaccine counseling.

In interviews with providers, one doctor expressed hesitancy toward having conversations about the vaccine with patients who had previously declined it: “I just feel like my patients are sick of being bothered about this, so I’ve decided to lay off the vaccine talk for now.” Other providers articulated a belief that patients — particularly patients of color — no longer wanted to hear from their provider about the vaccine. A theme of vaccine counseling burnout among providers emerged. One physician expressed a sense of futility with vaccine counseling: “People have already made up their minds.”

Data linking vaccine offer with increased likelihood of vaccine administration could be especially important in countering burnout. Our data suggest that hesitancy is a fluid emotion that can change with repeated vaccine offers to patients. Sharing these data with providers and staff could allay avoidance of and fatigue with this topic.

Our data suggest that hesitancy is a fluid emotion that can change with repeated vaccine offers to patients. Sharing these data with providers and staff could allay avoidance of and fatigue with this topic.

The Team

• Primary Care Population Health Team: oversees clinical metrics and quality improvement for the entire population of primary care patients in SFHN

  • Primary care physician (1)

  • Director of population health (1)

  • Population health manager (1)

  • Health program coordinator (1)

  • Care experience manager (1)

  • Primary care director and deputy director (2)

• Vaccine Equity Committee: monitors disparities in Covid-19 vaccination rates among patients at SFHN and develops interventions to reduce inequities

  • Vaccine equity lead/primary care physician (1)

  • Patient advisors (3)

  • Vaccine equity committee members at large (11)

• Managers, providers, staff at 14 primary care clinics in SFHN: provide full-scope primary care services to the entire population of primary care patients in SFHN

Metrics

We began collecting data on the vaccine offer rate in March 2021. Initially, we saw a 7–percentage-point disparity (Figure 3) between Black patients and the overall primary care population (74% vs. 81%, respectively). As we disseminated these data and implemented changes to reduce the disparity, we saw the gap in offer rate progressively narrow. By March 2022, the disparity had shrunk to 2 percentage points (93% offer rate for Black patients vs. 95% offer rate for primary care patients overall).

Further analysis of our data demonstrated a positive correlation between vaccine offer to Black patients and vaccination rate for Black patients. In other words, Black patients at clinics with high offer rates to Black patients were more likely to be vaccinated than at clinics with lower offer rates (Figure 5, correlation coefficient 0.626).

We also demonstrated that some patients were offered the vaccine multiple times before eventually consenting to vaccination (Figure 6). While most patients accepted the vaccine after declining it once (70.5% of patients overall and 66.7% of Black patients), a smaller proportion were offered the vaccine between two and 11 times before agreeing to it.

Population-level vaccination rates are represented in Figure 7. While disparities in vaccination rates for Black patients have lagged behind those of the overall population from the beginning, the disparity has shrunk from a peak of 18.3 percentage points in May 2021 to 12.6 percentage points in March 2022.

Figure 7.

Covid-19 Fully Vaccinated Rate for Patients Ages 5 and Up, January 2021–January 2022

BAA = Black/African American, SFHN = San Francisco Health Network. Source: San Francisco Health Network Electronic Health Record

Where to Start

The path to reducing disparities starts with data. Proving that disparities exist is a first step, but further analysis is needed to uncover root causes and understand how the disparity impacts patients. Engaging the diverse voices of frontline staff, clinic leadership, and patient advisors sparks creative solutions and drives important conversations among health care teams.

While it can be difficult to get buy-in for strict adherence to a process measure, linking process measures to outcome measures helps make the case for why the process matters. Conversely, outcome measures alone are often not sufficient to understand a problem. Analyzing process measures can provide critical information as to root causes and what is happening on the ground.

Talking about implicit bias is hard. It is also incredibly important. Discussing bias nonjudgmentally and making data visible to all team members can help raise awareness of unconscious behaviors that impact patients. To start addressing implicit bias, approach the topic with transparency, respect, and an unwavering commitment to eliminating disparities.