Knowledge and perception regarding effectiveness in influenza vaccines among General Practitioners in Germany: A national survey

Highlights

• •The terms vaccine efficacy and vaccine effectiveness are not interchangeable.
• •Vaccine effectiveness happens in the real world, and efficacy in clinal trials.
• •However, 41% of physicians said that the terms can be used interchangeably.
• •Knowledge of the difference in terms can help support well-informed decisions.
• •Getting the annual flu shot is more important than the type of flu vaccine.

Abstract

Objectives

To understand physicians’ knowledge and perception regarding the effectiveness of influenza vaccines and to communicate the importance of understanding the differences in terms of vaccine efficacy and vaccine effectiveness.

Methodology

This cross-sectional quantitative online survey was conducted using a questionnaire comprising 20 questions, between September 11 and 19, 2021. The survey was conducted across 14 cities in Germany, including physicians actively involved in influenza vaccine purchasing decisions. Descriptive statistics were used to summarize the data and paired t-test was performed to compare the physicians’ understanding of efficacy and effectiveness.

Results

Eighty physicians (21%) completed the survey. Physicians defined the terms vaccine efficacy and effectiveness similarly, with only minimal distinctions. Forty-one percent agreed that both terms can be used interchangeably in clinical practice. A higher proportion used the phrase “observational study” for vaccine efficacy and 21% associated “controlled environment” with effectiveness. The majority of physicians indicated that antigen match to circulating strain plays a large role in overall effectiveness and vaccine coverage strongly influences overall influenza case prevention. Vaccine performance in observational studies under so-called “real-world conditions” and (vaccine independent) strain match were the most important factors to assess vaccine performance and vaccine choice.

Conclusion

These findings show that physicians in Germany use the terms vaccine efficacy and vaccine effectiveness interchangeably. A better knowledge of the differences between these terms will help to make informed decisions on the choice of influenza vaccine for its population. Finally, and most important, increasing the annual flu vaccine uptake rates will have more and the greatest beneficial impact on reducing flu-related disease and public health, regardless of the expression of the benefits for different vaccine types.

Introduction

Seasonal influenza is a public health problem with 5 to 20 % of the European population contracting this infection every year. The infection has high incidence and mortality rates of 5.9 infections per 100,000 inhabitants and 5.89 deaths per 100,000 inhabitants every year [1], [2]. In Germany, as per 2018–2019 epidemiology reports, there were 3.8 million influenza-attributable medical conditions, 18,000 hospitalization cases, and 25,000 influenza-related deaths [3].

Vaccination remains the cornerstone for the prevention of seasonal influenza. Broadly, the influenza vaccine is available in trivalent and quadrivalent forms [4], and has efficacy rates of 70 to 90 % in controlled trials [5], and effectiveness rates of 30 to 60 % [6] in real-world data from immunization programs. Though efficacy and effectiveness represent different expressions of vaccine performance, they are often used interchangeably.

In the literature, and therefore also in the context of this survey, the commonly used operational definitions of vaccine efficacy (controlled RCTs) and vaccine effectiveness (observational non-controlled studies) were used. However, definitions of these terms based on the biological process of viral exposure, infection and disease seem more appropriate (see discussion).

The objective of the current survey was to understand the knowledge and perceptions of physicians in Germany, regarding the difference between vaccine efficacy and vaccine effectiveness to assess and compare influenza vaccines’ performance. This paper will focus on the physicians’ perceptions of effectiveness across different seasons, factors influencing effectiveness, the influence of vaccine coverage and effectiveness rates on case prevention, knowledge gaps regarding effectiveness, decision criteria used when choosing influenza vaccines, and influenza vaccine usage across different age groups.

Methods

Survey design

A descriptive cross-sectional quantitative survey was conducted on 80 physicians working across Germany. The survey was conducted between September 11 and 19, 2021 and was designed to target a demographic composition of German physicians based on geographic location, surveying 14 of the 16 federal German states. This was a face-to-face online interview and the survey was conducted in German.

The research done is consistent with European Union (EU) General Data Protection Regulation law and European Pharmaceutical Market Research Association (EphMRA) Legal and Ethical Guidelines.

Eligibility criteria

Included in the survey were board-certified physicians who primarily specialized in General Practice or General Medicine with 2–35 years of experience and were actively involved in influenza vaccine purchasing decisions. Their involvement was either as the sole or joint decision-maker with at least equal influence on decisions as other members. Physicians were excluded if they had any pharmaceutical company or market research firm affiliations. Participants were also given notice about the adverse event (AE) or product complaint reporting requirements, should any be mentioned during the survey; however no AEs were mentioned.

Survey questionnaire

The questionnaire comprised of 20 questions in 4 sections: general awareness, vaccine effectiveness, choice of influenza vaccines, and physician demographics. Some questions were open-ended and some questions were aided with a list of options.

In section one, the physicians were asked about their perspectives on vaccine efficacy and effectiveness. The physician had to select the words and phrases that they associate with efficacy and effectiveness. Later, the physicians were given the opportunity to associate the same terms with either efficacy or effectiveness while presented with a table with columns for both.

In section two, the physicians were asked about their knowledge on vaccine effectiveness, including questions on the reasons for differences in published effectiveness rates, factors influencing effectiveness, and influences of effectiveness and vaccine coverage on influenza case prevention.

In section three, physicians’ knowledge and perception regarding the choice of an influenza vaccine were evaluated. The physicians were asked about the attributes they find most important and depend on when selecting an influenza vaccine, their awareness of the currently available influenza vaccine products, and key sources of evidence they depend on when selecting the influenza vaccine.

In the last section, physicians were asked about their primary area of practice, the patient volume they managed in the last 12 months (by age category), and the percentage of patients who received an influenza vaccine during the 2020–2021 season (by age category) were collected.

Statistical analysis

The statistical analysis was conducted using Q Research Software. Descriptive data were presented as frequencies and percentages. The proportion of physicians selecting the efficacy and effectiveness for each attribute was compared using paired t-tests. To examine if the distributions were the same between efficacy and effectiveness, Kolmogorov-Smirnov Test was conducted. Significant differences were denoted by a letter B on every summary statistic. The absence of such lettering indicates that figure is not statistically significantly different to its comparator.

Results

A total of 384 unique invitations that would achieve the desired sample size were randomly sent to invite the physicians within each region from our sample pool. The sample pool consists of physicians from the national database who have agreed to participate in market research. Of the invitations sent, 105 (27 %) have accessed the survey, and 80 (21 %) completed the survey. Thirteen physicians did not meet the inclusion criteria, 5 did not complete the survey and 7 were excluded due to data quality purposes.

To ensure representativeness across Germany, sampling was done proportionally (approximately) within the 16 federal states. However, there were no participants from two of the smaller provinces. The majority of the physicians were from Nordrhein Westfalen (n = 16, 20 % of final sample), Bayern (n = 16, 20 % of final sample), and Baden-Württemberg (n = 11, 14 % of final sample) provinces. Full list of the physician distribution is shown in Fig. 1. Nearly all the participating physicians (99 %) practice in a private, community setting, and the majority (71 %) were sole decision-makers and/or have veto authority over influenza vaccine purchase choices. The remaining 29 % make decisions jointly with other staff. The average years of clinical practice experience of the participating physicians were 20 years.

Fig. 1.

Sample Distribution by Federal State (Germany). Figure legends: Final sample = 80; Please note, while recruitment efforts were nationwide, there were no respondents successfully recruited from Bremen or Thüringen.

The majority of the patients they managed in the past 12 months were between 18 and 65 and greater than 65 years old. According to the physicians, only 56 % of the elderly patients received a vaccination during the 2020–21 influenza season. The rate of vaccination was lower in the younger age groups (11 % for < 18 years old; 27 % for 18–49 years old).

Knowledge and perception on vaccine efficacy and effectiveness

Physicians used similar phrases to define vaccine efficacy and effectiveness, and the phrases used for each were mentioned with similar frequency. Sixty percent of them associated the term “protection” with effectiveness and 49 % with efficacy. The phrase “how well it works”, “risk reduction” and “desired effect/target response achieved” were more often used in defining efficacy than effectiveness. The phrases “observational study” and “real-world conditions/practice” were less often used for both efficacy (6 % and 5 %, respectively) and effectiveness (0 % and 1 %, respectively) (Table 1). Overall, a statically significantly greater number of physicians associated the terms “observational study” and “desired effect/ target response achieved” with efficacy. Nineteen percent of physicians provided responses that were exactly the same as, or synonymous with, both the terms.

Table 1.

Unaided definitions of vaccine efficacy and effectiveness.

(Unaided) Definitions of Vaccine Efficacy & Vaccine Effectiveness	TOTAL Physicians (n = 80) (% of General Practitioners)
	Vaccine Efficacy	Vaccine Effectiveness
(RATE OF) PROTECTION (NET)	49 %	60 %
Against infection / disease	25 %	25 %
Against getting (seriously) sick from infection / disease	13 %	21 %
Prevention (general)	5 %	9 %
DATA / STUDY CHARACTERISTICS (NET)	30 %B	16 %
Comparison of vaccinated & unvaccinated groups	18 %	14 %
Observational study	6 %B	–
Real-world conditions / practice	5 %	1 %
RESPONSE (NET)	30 %B	11 %
Desired effect / target response achieved	11 %B	1 %
Developed immune response / immunity / antibodies	6 %	9 %
EFFECTIVENESS / HOW WELL IT WORKS (NET)	25 %	18 %
(RELATIVE) RISK REDUCTION (NET)	18 %	11 %
Reduced likelihood / probability of contracting disease	10 %	6 %

[B] denote statistically significant differences at 90 % CI.

After the open-ended questions, physicians were aided by being shown a set of relevant words to associate with vaccine efficacy or effectiveness. Most physicians agreed that both efficacy and effectiveness were measures that demonstrate “vaccine performance”. Thirty-five percent of the respondents associated “clinical practice” with effectiveness (vs 25 % with efficacy), and 55 % associated “virus prevention” with effectiveness (vs 46 % for efficacy). Fifty-five percent of respondents associated “clinical study” with efficacy, and 43 % with effectiveness. Twenty-eight percent of respondents associated “real-world evidence” (RWE) with effectiveness and 21 % associated it with efficacy. However, there is a surprising misalignment when it comes to the term “controlled environment”. Twenty percent of the physicians associated “controlled environment” with effectiveness, which is surprisingly similar to the 18 % associating it with efficacy. However, effectiveness relates to performance outside of a controlled environment, as compared with the efficacy measures from the controlled environment of a clinical trial. Overall, only one attribute “vaccine performance” was statistically significantly associated with efficacy. The two terms, vaccine efficacy and effectiveness were shown side by side, and respondents were asked again to associate the phrases with effectiveness or efficacy. Twenty-nine percent of physicians made adjustments to their initial associations. A large number of answers were modified; however, the changes did not lead to more statistically significant differences between efficacy and effectiveness. “Risk Reduction” is the only phrase that flipped its association, that is, more associated with effectiveness initially, but then more associated with efficacy after viewing the two phrases side-by-side (Fig. 2).

Fig. 2.

Vaccine Efficacy vs Effectiveness Term Association (% of General Practitioners). Figure legends: GP = General Practitioners; [A/B] denote statistically significant differences at 90 % CI.

Knowledge and perception on factors influencing vaccine effectiveness

Unaided, the responding physicians most commonly cited differences in the population (38 %) and/or other study characteristics (34 %), such as differences in study designs and/or time periods of evaluation, as the possible reasons for differences in published effectiveness rates. Nineteen percent of the respondents cited differences in data collection or choice of assessment (statistical) tools, and 18 % cited vaccine-specific reasons such as differences in vaccines, ingredients, or manufacturers as the possible reasons. Only 4 % of the responding physicians specifically mentioned “different time periods of evaluation” as a potential reason for differences in published effectiveness rates.

The majority of respondents (89 %) felt that an antigen match to a circulating strain strongly influences effectiveness rates. While pre-vaccination status (74 %), virus circulation (65 %), and vaccine coverage (64 %) were also reported to have an impact to some degree, 24 % felt that vaccine coverage does not play an important role in vaccine effectiveness rates (Fig. 3). Twenty-one percent of the respondents indicated that effectiveness has more influence on overall influenza case prevention and 19 % placed a higher emphasis on the coverage. Respondents were also presented with two scenarios and were asked to select one scenario that would result in the prevention of a greater number of clinical influenza cases. Scenario One had a vaccine with 20 % effectiveness and 70 % coverage and Scenario Two had a vaccine with 50 % effectiveness and 20 % coverage. A greater number of physicians (68 %) expressed the opinion that Scenario One, with higher vaccine coverage of 70 %, had more influence on influenza case prevention than Scenario Two, which had the higher effectiveness rates.

Fig. 3.

Perceptions about Factors Influencing Vaccine Effectiveness (% of General Practitioners). Figure legends: GP = General Practitioners; 1 = No influence at all; 7 = Strong influence.

When physicians were asked about their level of agreement on a series of statements regarding effectiveness, there was a split as to whether or not the terms vaccine efficacy and effectiveness are interchangeable in clinical practice. Opinion was fairly evenly split, with 41 % agreeing that the terms can be used interchangeably and 40 % disagreeing. Sixty-one percent said effectiveness rates can be compared across different influenza seasons, and the majority mentioned that there was no reliable effectiveness data in the country. About three out of five did not feel that booster doses provided any additional value over single-dose forms in terms of effectiveness (Fig. 4).

Fig. 4.

Level of Agreement with Statements About Vaccine Effectiveness (% of General Practitioners). Figure legends: GP = General Practitioners; 1 = Strongly disagree; 4 = Neither agree nor disagree; 7 = Strong agree.

Knowledge and perception regarding the choice of influenza vaccine

Clinical performance measures emerged as the most influential attributes in influenza vaccine choice. They include real world data and evidence demonstrating effectiveness, antigen match to circulating strains, RCT data demonstrating efficacy and adverse reactions to vaccines. Specific features of the influenza vaccine, such as vaccine technology and adjuvant ingredients, and patient characteristics, such as age, were presented as having a lesser role to play in the decision (Fig. 5).

Fig. 5.

Attribute Relative Importance When Selecting an Influenza Vaccine.

Perceptions varied among physicians in terms of defining a meaningful difference between overall effectiveness rates for two influenza vaccines. Twenty-eight percent said that a difference of more than 60 %, while 26 % said that a difference of only 16–20 %, in effectiveness rates was needed for a meaningful difference when choosing an influenza vaccine. The mean and median meaningful difference was 30 % and 42 %, respectively.

Physicians most commonly rely on local and/or regional guidelines when choosing influenza vaccines to stock and administer to their patients. Information sourced from journals, sales representatives, professional associations, & health care professional (HCP) social groups also aid in their decision-making (Fig. 6).

Fig. 6.

Key Sources of Evidence Relied Upon When Selecting Influenza Vaccines (% of General Practitioners). Figure legends: GP = General Practitioners; CSL = Clinical Science Liaison; MSL = Medical Science Liaison. Other key sources of evidence include recommendations from Health insurance, Pharmacist, and Personal experience.

Discussion

To the best of our knowledge, this is the first time a survey on the views of physicians on influenza vaccine efficacy and effectiveness was conducted in Germany.

Though vaccine efficacy and effectiveness measure the proportion of reduction in infection/disease among vaccinated persons [7], there are important differences between these terms. As mentioned in the introduction, this survey, following practices from the literature, used the trial design (RCTs vs observational studies) as the basis to discriminate between the two terms.

However, based on the biological infection cycle (viral circulation/exposure, infection and disease), vaccine efficacy should be more properly defined as the proportion of reduced infection in a vaccinated compared to an unvaccinated group. This can be measured by post-immunization antibody titers, corrected for pre-vaccination titers [8]. Such studies are usually done in controlled RCTs. The advantage of RCTs compared to non-controlled observational studies is that they ascertain equal viral exposure and infection rates between vaccinated and non-vaccinated study groups [9]. Thus RCTs minimize confounding factors between vaccinated and non-vaccinated study groups related to environmental factors in the two study arms.

Vaccine effectiveness is defined as the proportion of reduced disease in a vaccinated compared to a non-vaccinated group. Often (and for practical reasons) vaccine effectiveness is assessed in non-controlled, observational studies (often referred to as “real-world data” [9]. Outcomes from non-controlled observational studies, however, do not depend on vaccine performance alone, but also on the (different) epidemiological circumstances between the study groups, such as exposure and infection rates. Study outcomes are therefore (much) more sensitive to biases compared to RCTs.

So, understanding the distinction between vaccine efficacy and vaccine effectiveness is important for the interpretation of clinical outcomes and their potential implications [8]. Nevertheless, while the two terms are not interchangeable by their definitions, 41 % of physicians said in the survey that the terms can be used interchangeably in clinical practice.

As mentioned, the definitions based on trial design as used in this survey, the phrase “real-world conditions/practice” (5 %) was used 5 % of the times. However, in the aided scenario (when provided with a list of options), 33 % of physicians associated it with effectiveness. Also, the phrase “controlled environment” was more associated with effectiveness, which is not consistent with the definition. For many of these phrases, we would expect clear alignment with either efficacy or effectiveness. While many trend in the right direction, the fact that there were no statistically significant differences between these associations supports the idea that many physicians viewed the terms synonymously. In particular, we expect these to be more distinguished with “real-world evidence” being more strongly associated with effectiveness and “controlled environment” with efficacy.

In the survey, the reasons given for differences in published effectiveness rates for a given vaccine were mainly attributed to differences in study populations and study design. The time period of evaluation was not considered by many physicians. However, the time period for evaluation is important and can cause variation in vaccine effectiveness due to differences in attack rates and circulating strains. Further, the vaccine effectiveness is calculated from the infection or disease risk rates in vaccinated and unvaccinated persons. For example, if a vaccine shows 100 % efficacy in a RCT, but there is no flu in the following season, its effectiveness is 0 %. Hence, it is important not to compare results of effectiveness studies done in one year vs another, without taking these points into consideration [10], [11].

In the survey, many physicians rightly mentioned that the antigen match to circulating strains is an important factor influencing effectiveness. When the circulating strain and the vaccine strain do not match with each other, vaccination does not provide or offer very little protection [12], [13]. Other factors such as prior infection or vaccination immune status and virus circulation also influence effectiveness. Hence, a thorough understanding of these epidemiological factors, which vary by time and place, helps in the proper interpretation of rates of vaccine effectiveness [10].

Vaccine coverage is another important parameter that determines the level of reduction in disease burden. As the vaccine coverage increases, there is a reduction in prevalence or incidence rates of the disease and an increase in the development of herd immunity. This improves vaccine effectiveness rates [14]. When initially asked, 21 % of physicians felt that vaccine coverage did not influence vaccine effectiveness rates; however, when provided with a scenario, 68 % of them agreed that vaccine coverage has more influence in case prevention. Vaccine coverage of greater than 50 % with effectiveness rates as low as 10 to 20 %, can substantially reduce the risk of influenza infection rates [10]. Notwithstanding this view by the majority, coverage of the influenza vaccine remains low in Europe [15]. Furthermore, the Standing Committee on Vaccination (STIKO) Germany [16], has prioritized the use of high-dose quadrivalent influenza vaccine (compared with the standard trivalent and quadrivalent vaccines) in the elderly population aged 65 years and older. The Joint Committee on Vaccination and Immunization (JCVI), United Kingdom [17] also supports the use of high-dose adjuvanted quadrivalent influenza vaccine in the elderly population along with the standard trivalent and quadrivalent vaccines. The goal of universal vaccination should put greater weight on increasing patient access and promoting vaccine coverage, rather than prioritization of one vaccine over the other.

In the survey, the majority of physicians (61 %) mentioned that effectiveness rates can be compared across different influenza seasons. However, this is not true and vaccine effectiveness varies from season to season. The rates seen in one season occur under specific epidemiological conditions and cannot be over-generalized or extrapolated to different epidemiological settings. Ignoring this can lead to serious bias on the perception of influenza vaccination in the public domain [10]. Generalized conclusions can be obtained only by meta-analysis studies because only meta-analysis resolves the influence of different epidemiological factors that exist in various influenza seasons and provide a general conclusion on vaccine effectiveness rates [10], [18].

In our survey results physician reported that just half of their elderly population (56 %) got their flu vaccine, and the vaccine coverage is less in younger age groups (11 % in < 18 years old; 27 % in 18–49 years, and 43 % in 50–64 years old). This is despite the WHO and European council recommendations of increasing the vaccination coverage to high-risk groups and achieving a 75 % target for older patients [15], [19]. National guidelines must communicate the importance of vaccine coverage. Increasing the influenza vaccine coverage will result in a reduction of the disease burden and therefore improve the health outcomes and quality of life of people.

The study was limited to general practitioners, involved in decision-making, and not affiliated with a pharmaceutical company or market research firm. These were selected, as general practitioners would be more likely to see flu patients and because the interest was in understanding those with the greatest influence on vaccine choice. This can be viewed as a limitation of the study, if one were more interested in the level of understanding in the German physician community more broadly or among specialists. While about 73 percent did not accept the invitation, but we have no reason to expect that has anything systematic to do with their understanding of vaccine efficacy and effectiveness.

Conclusions

Our survey showed that physicians in Germany are not clear on the difference between vaccine efficacy and vaccine effectiveness and often use these terms interchangeably. This finding suggests that there is no clarity about the difference between vaccine efficacy and vaccine effectiveness. There are different definitions used in the literature, which may contribute to this confusion. Because the scientific robustness of study outcomes are influenced by study design, definitions, group sizes and environmental circumstances, a careful interpretation of vaccine performance from individual studies is warranted. Discrimination between the terms vaccine efficacy and vaccine effectiveness is relevant to judging the performance of the vaccine. We have presented the definitions as they are proposed and used in the literature, where vaccine efficacy expresses the reduction in infections between vaccinated and non-vaccinated groups (usually assessed in RCTs), whereas vaccine effectiveness expresses the reduction in disease (mostly assessed in observational studies). Since vaccine effectiveness is usually assessed in observational studies, the study outcomes can be influenced by other factors related to environmental/epidemiolocal differences between the study groups. Such considerations are of public health relevance when assessing the performance of influenza vaccines for national immunization programs.

Funding support

The study was funded by Viatris.

Author contributions

Both the authors meet the ICMJE criteria for authorship. Conceptualization, SH; JC; methodology and data collection, JC; SH; data analysis, SH, JC; writing - original draft, SH; JC; writing - review and editing, SH, JC. Both the authors have read and agreed to the published version of the manuscript.

Declaration of Competing Interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: [SH is an employee of Viatris, Bangalore, India; JC is an employee of Viatris, Canonsburg, USA, and holds stocks in Pfizer and Viatris].

Data availability

Data will be made available on request.