Abstract
The Human Papillomavirus (HPV) Prevention and Control Board convened a meeting in Bucharest, Romania (May 2018), to discuss the role of healthcare providers (HCPs) in prevention programs, with a focus on HPV vaccination and cervical cancer screening. International and local experts discussed the role that HCPs can play to increase the uptake of HPV vaccine and screening. Experts recommended: 1) increasing HCP norms of getting vaccinated; 2) training providers to make effective recommendations; 3) making culturally appropriate materials available, in local languages; and 4) centralizing and coordinating education and information material, to direct both HCPs and the general public to the best material available.
1. Introduction
The HPV Prevention and Control Board (www.hpvboard.org) organizes technical and country meetings, where international and local experts exchange experiences and insights to strengthen countries’ efforts to secure HPV prevention and control and discuss technical items related to HPV prevention programs [[1], [2], [3], [4]]. On May 15–16, 2018, the fifth meeting in the series was held in Bucharest, Romania. This technical meeting targeted the role of healthcare providers in prevention programs, with a focus on HPV vaccination and cervical cancer screening. For the sake of clarity, we use two terms in this report: healthcare providers (HCPs) and general practitioners (GPs). HCPs includes all people working in healthcare (e.g. physicians, nurses, physician assistants, but also technicians). GPs (sometimes called family doctors) are physicians who focus on primary care at the outpatient clinic or community level. Similarly, the terms confidence and hesitancy are used in this report. Note that these are not synonyms: confidence is an attitude that vaccines are good or bad (people can believe both), whereas hesitancy is a motivation (or lack of it) related to getting vaccinated or giving the vaccine.
2. The role of HCPs in the vaccination program
2.1. Lessons learned from influenza vaccination acceptance among HCPs
The rationale for vaccination of HCPs against seasonal influenza is clear: 1) HCPs are at increased risk for occupational exposure to influenza [5]; 2) HCPs with influenza-like illness may continue to work, placing patients (and colleagues) at risk [6,7]; 3) HCPs care for patients at high-risk for serious morbidity, complications and death due to intrinsic factors such as age (neonates, elderly) or underlying health conditions (e.g. immunosuppression or other chronic disease); 4) unvaccinated HCPs may be the source of infection in outbreaks, especially in neonatal intensive care units [[8], [9], [10]]; 5) influenza vaccine is effective in preventing influenza and absenteeism in HCPs, with absenteeism leading to an increased workload for the remaining (vaccinated) HCPs [[11], [12], [13], [14], [15]]; and 6) HCPs are role models for their patients [16]. Hence, vaccination of HCPs is justified to directly protect HCPs, indirectly protect their patients, colleagues and families, and to maintain optimal healthcare services during the flu season. Nevertheless, influenza vaccination rates among HCPs vary widely, ranging from <5% to >90% worldwide. In Europe, vaccination coverage among HCPs is generally low (often less than 30%) despite recommendations. Influenza vaccination coverage among HCPs in the United States (US) is higher than in Europe and other countries [17].
Requiring influenza vaccination of hospital personnel in US hospitals has led to a vaccination coverage of over 98% in this population [18]. In Europe, acceptance of policies requiring vaccination for HCPs ranges between 50% and 70%, with higher acceptance for HCP who care for high-risk patients. Barriers to increasing coverage of influenza vaccination among HCPs are similar to those that exist for the general public, namely insufficient knowledge of, and misconceptions about, vaccine safety and effectiveness, lack of awareness of vaccine recommendations, and mistrust of health authorities [19].
Given that HCPs often are trusted role models for their patients, refusal of vaccination by HCPs may undermine vaccine uptake in the general population. To increase vaccination among HCPs, several general principles seem to apply: 1) Increase education of HCPs related to vaccination and vaccines; 2) Emphasize the norms of vaccination (e.g., to protect patients, colleagues and family) among HCPs; and 3) Make it easier for HCPs to get vaccinated (e.g. by making vaccines free of charge, improving access to vaccination by supplying them at the work space). When HCPs truly are hesitant, education will not always solve these issues, in which case engagement is needed, as well as providing a space to listen to and address their concerns.
2.2. Is there vaccine hesitancy among european HCPs?
HCPs are among the most trusted source of immunization information [20], and therefore are a core tool to address low or declining public confidence in vaccination. However, reliance on HCPs to provide optimal guidance to patients (the word patient reflecting all healthcare users, including healthy (adolescent) individuals [21]) may be jeopardized if HCPs, themselves, have doubts about vaccines or they do not communicate effectively with their patients. HCPs, regardless of their attitudes towards vaccination, may also delay or inappropriately alter the vaccination schedule. A qualitative study of HCPs from four different European countries (Croatia, France, Greece and Romania [22]) showed that, while in general the benefits of vaccination were appreciated, perceived risks played an important role in their attitudes towards vaccination. Concerns about possible side-effects, especially for vaccines proposed for addition to the national immunization program (such as HPV vaccine in Romania) were also raised, together with questions about HCP's responsibility for these side-effects if they should occur in their patients. Furthermore, the study found that HCPs sometimes mistrust pharmaceutical companies, and, albeit less frequently, health authorities, especially in France and Greece. While some HCPs see it as their role to respond to hesitancy and influence the patients' decision, others feel they should remain neutral, and leave it to the patients to decide. In general, HCPs felt that vaccine confidence could be improved by providing more information to HCPs and their patients (e.g., on side-effects), provider training (e.g., communication skills), as well as stricter legislation (vaccination requirements, action against vaccine-hesitant HCPs). Finally, it is clear from the study of these four different countries [22], that while some elements appear across all countries, most issues are country- and context-specific.
2.3. The role of HCPs in measles control and prevention in Bulgaria
During the 2009–2011 measles outbreak in Bulgaria, 90% of cases occurred in the Roma population as a result of the accumulation of a large susceptible population due to low measles-mumps-rubella (MMR) vaccine coverage [23]. Between April 2009 and December 2010, 188,700 MMR vaccine doses were administered free of charge through the routine immunization system and special outreach teams in collaboration with Roma health mediators (HMs), to specifically target Roma communities. The HMs played a vital role in the success of this campaign, which started in 2009 in the affected regions for persons of Roma ethnicity and was extended to a national level, targeting persons aged 13 months to 20 years who had not received two MMR vaccine doses. From March 2010, persons aged 30 years and older who had not received two MMR doses were targeted. HCPs were offered a dose of MMR, regardless of their immunization status or age.
This outbreak helped develop and strengthen national and local vaccination activities through enhanced collaboration between HCPs and Roma organizations with the aim of improving the Roma community's integration into the healthcare system [24]. Bulgaria was one of the first countries in the WHO EURO region to test the Guide to Tailoring Immunization Programs (TIP) [25]. Bulgaria participated in the European Centre for Disease Prevention and Control (ECDC) and World Health Communication Associates (WHCA) project “Let's talk about protection: an ECDC Action Guide to enhance childhood vaccination uptake”, aiming to coordinate a group of experts to develop content for an Action Guide on childhood immunization and different vaccines, with a special focus on MMR vaccine, targeting health professionals, parents/grandparents, underserved/poorly served population and media. WHCA helped ECDC to facilitate the dissemination of the Bulgarian version of the childhood vaccine uptake communication action guide, by health care providers in selected communities in Bulgaria as a central part of a pilot project initiative to increase MMR vaccination uptake by underserved population groups.
A practical vaccination guide, adapted to the context in Bulgaria, was published and distributed for use by HCPs and HMs [26].
2.4. Should we monitor HCPs’ attitudes towards vaccination?
Currently, limited activities are undertaken to try to understand and measure HCPs' concerns about vaccines and immunization across countries. The Vaccine Confidence Project's “2018 State of Vaccine Confidence in the EU” report provides a clear correlation between HCPs' attitudes regarding vaccination and those of their patients [27]. Patients trust HCPs, so if HCPs do not appear knowledgeable about the pros and cons of the vaccine and confident in their recommendation, the patients may opt to look for other, potentially less accurate, sources.
Optimally, measurement of vaccine confidence among HCPs should be introduced into vaccination surveillance systems, together with the collection of vaccine safety and effectiveness data. Ideally, assessment of HCPs vaccine confidence could be achieved via routine periodic surveys, respecting the country-specific context. To maximize response rate, surveys would need to balance open-ended (non-leading) data collection with data collection tools that take minimal time and are easy to complete. An important objective would be to identify the existence of, and reasons for, vaccine confidence in HCPs, as well as in the general public.
While vaccine confidence among HCPs may be an important indicator of vaccine confidence in the general population, other important measurements include provider behavior (recommendation) and patient behavior (how often patients who want to delay or refuse vaccination are encountered). Surveillance should also include HCPs' reactions to parent hesitancy, e.g., how do HCPs follow-up with parents who refuse a vaccine for their child? Further, it is important to identify the questions HCPs may have about vaccination or about parent's and patient's concerns, to be able to address them appropriately. Finally, HCPs are no longer the only information source related to vaccination and, as such, may be challenged by patients with information from other, perhaps less reliable, sources [28,29].
Education of HCPs related to vaccination and communication is essential to permit them to provide appropriate, accurate information to parents.
3. The role of HCPs in implementing and expanding coverage of HPV vaccination: country examples
3.1. Bulgaria: the role of health mediators in reaching the roma population
The Bulgarian National Network of Health Mediators was established in 2007 to represent more than 200 HMs working in the field, among the most marginalized people. Continuing education on immunizations is provided to members via regional and national meetings. The HMs connect the GPs and the Regional Health Inspectorate to the Roma community, actively searching for under-vaccinated children. For example, during the measles epidemic in 2010, HMs, teaming up with the Regional Health Inspections, managed to vaccinate more than 188,000 children. Similarly, the HPV prevention program is only successful in regions where Regional Health Inspectorate, GPs and HMs work closely together. In the Roma communities, HMs generally do not encounter anti-vaccine sentiments, but challenges occur due to a lack of information, poor communication, and access to GPs.
3.2. Scotland: school immunization teams and the HPV program
The Scottish HPV immunization program started in September 2008, as a school-based program targeting 12- and 13-year-old girls, although a catch-up program for girls under 18 years old ran for three years. For the first four years the bivalent vaccine was used and from 2012, the programme changed to the quadrivalent vaccine. Vaccine uptake in the routine population was high, with around 90% of eligible girls receiving all doses (Fig. 1), whereas uptake in the catch-up population was much lower, around 65% receiving all doses [30].
These high routine vaccination rates with the bivalent HPV vaccine led to a large reduction in prevalence of HPV vaccine types 16/18 (from 30% to 4.5%), and in HPV types 31/33/45 (from 14.2% to 2.6%), whereas no impact was seen on other HPV types [30]. In recent years, the role of school nurses in immunization has been taken over by dedicated school immunization teams. These teams send out letters to the schools in May with suggested vaccination dates, distribute consent forms and information leaflets to schools in mid-November, and hold HPV vaccination sessions between January and March. They have time to establish and maintain a relationship with schools and parents, and to increase awareness and understanding amongst girls. Vaccine uptake rates have remained high, despite some growth in anti-vaccine sentiment, and attempts at coordinated anti-HPV campaigns [31].
3.3. France: the role of HCPs in vaccination
Recent work has shown that France had the lowest vaccine confidence in Europe [32,33]. However, French health barometer data, which is based on telephone surveys, show that this low confidence bottomed out in 2010, and has gone up again, although it has not reached the pre-2010 level. A study has shown that people who trust the health care system and health professionals were more likely to have been vaccinated [34]. Furthermore, those who were advised by HCPs to be vaccinated showed the intention to do so [34]. An HCP recommendation, together with subjective norms, had the strongest association with self-reported vaccination.
Vaccine confidence in France may be influenced by several trends: 1) a general, increasing feeling of distrust toward governmental institutions and, in particular, distrust of public health authorities; 2) the rise of complementary and alternative medicines that promote the utilization of “natural” means to tackle diseases, to the exclusions of vaccines; 3) the impact of (non-evidence-based, anti-vaccine) information amplified through social media, and 4) the indifference of global leaders to act upon these trends. The combination of these phenomena may have facilitated the rapid propagation of incorrect or suspicious information, which may have affected acceptance of some vaccines.
3.4. Flanders (Belgium): training of medical students and vaccinators
At the University of Antwerp, medical students come across vaccine-related topics throughout their curriculum, with applied training on syringe and needle use in Bachelor year 1; an introduction to vaccinology, including skills training in Bachelor year 2; and a one-week module on vaccinology in Bachelor year 3. In year 7, the first year of GP training, a case-based update on vaccinology is provided. Training is not restricted to medical students; a 12-h module on vaccinology is also provided to pharmacy students. At the European level, together with other universities, a two-year educational Master program has been developed (EU-LIVE master course1). As a less intensive alternative, a teaching week on infectious diseases and vaccines is organized at four European universities, combined with a summer school program in Paris (EU-IDEAL-course2).
The University of Antwerp also organizes a Summer School on Vaccinology course3 for students, including topics on immunology, public health, infectious diseases, vaccines, immunization programs, vaccine safety communication with practical training on how to vaccinate. Finally, the University of Antwerp – with the support of the Flemish Health Agency - organizes a yearly Valentine Vaccination Symposium4 for vaccinators (doctors and nurses) with scientific presentations covering current vaccination topics relevant to the healthcare sector. In 15 years, the symposium has grown from less than 100 to over 500 participants. To keep the topics as up-to-date as possible, questions are requested from participants beforehand, and answered during the symposium by an expert panel.
3.5. Flanders (Belgium): the flemish vaccination board
In Belgium, the inclusion of vaccines into the vaccination program is based upon recommendations of the (federal) national immunization technical advisory group. However, public health prevention efforts are a regional responsibility and duty. In Flanders, vaccination advice is provided by the Flemish Vaccination Board, which consists of members representing all professionals involved in vaccination and academics. The Vaccination Board advises about the implementation of the vaccination program, about communication and vaccination campaigns and prepares common communications about vaccinations. In case of a potential crisis, the Vaccination Board provides pro-active communication (in one voice) to vaccinators and the Minister of Health.
3.6. Colombia: the role of the HCP/specialist after a mass psychogenic event
Although several key epidemiological studies on the relationship between HPV and cervical cancer and phase 3 efficacy trials of the HPV vaccines were conducted in Colombia [[35], [36], [37], [38], [39], [40]], at the time of HPV vaccine introduction, confusion about the vaccine was high among both GPs and parents. The intense communication campaign for the introduction was not supported by providing information and education to those responsible for administration of the vaccine, leaving them unprepared for questions from the public. Hence, following the launch of the HPV vaccination program in 2012, Colombia was not prepared for the crisis that happened in Carmen de Bolivar in 2014, when 15 girls from one school presented with tachycardia, shortness of breath, chest pain, paresthesia and fainting. An outbreak study involving 517 girls ruled out that the events were vaccine-related, but the response to the crisis was too slow and technical and lacked empathy for the affected girls and parents. In contrast, the anti-vaccine movement was well-organized and quick to respond. Vaccine confidence was further eroded by the request from the National Academy of Medicine to the Minister of Health to screen for, and exclude girls with autoimmune conditions, or a family history of such conditions from vaccination. This crisis, and the follow-up actions, led to a sharp decline in HPV vaccine coverage from 88% in 2012 to 14% in 2016. (Fig. 2)
To increase HPV vaccine advocacy, a roundtable has been established (including the Liga Colombiana contra el Cancer, the Ministry of Health, and Clinical Societies) to improve cervical cancer control through HPV vaccination and introduction of HPV testing as a primary screening test. Furthermore, an e-course on HPV vaccine efficacy and safety has been developed by the Catalan Institute of Oncology and the National Cancer Institute of Colombia targeting Colombian physicians and HCPs. Together with the HPV Prevention and Control Board, a statement was issued to refute the proposal to screen all vaccine recipients before immunization (December 2017). Finally, in some cities (Cali and Pasto), school-based HPV vaccination will be reactivated.
In May 2018, Colombian media paid attention to the Cochrane report on the safety of HPV vaccines [41], which was the first positive media coverage on HPV since the crisis and may prove to be a turning point for HPV vaccination in Colombia. To boost vaccine coverage, the next country meeting of the HPV Prevention and Control Board was scheduled in Colombia, in November 2018.
4. Strategies to enhance the impact of HCPs in improving HPV vaccination coverage
4.1. Making effective HPV vaccine recommendations
Patient-provider communication is key to ensuring a successful HPV vaccination program and optimal coverage rates. There is a clear need for improved strategies to communicate valid information concerning HPV vaccines, especially in countries that provide the vaccine through clinics or GPs. In the US, challenges to routine recommendation include that many HCPs expect conversations about HPV vaccine for adolescents will be uncomfortable, believe that parents do not want the vaccine for their children, and think that the discussion will take a long time. Hence, HCPs often discuss HPV vaccination at the end of a consultation, or not at all [42].
In addition, a key factor in HPV vaccine uptake is the strength of the HCPs' recommendation: in a nationally representative study, only 35% of children without a HCP recommendation were vaccinated, while 65% with a HCP recommendation received the vaccine [43]. Presumptive announcements are effective in increasing uptake, but providers rarely make announcements (1.5% of visits) [44]. This finding led to the development of ‘the Announcement Approach.’ For all families, the provider states that three vaccines will be provided at the end of the consultation, mentioning HPV vaccine in the middle. If the parent is hesitant, the provider asks for the main concern, addresses it and ends with a strong recommendation. In a randomized controlled trial, the announcement training improved HPV vaccine uptake, both in girls and boys [45]. Furthermore, the HCPs were satisfied with the approach, as they felt that it was both easier for parents and for HCPs. A different training, based on open-ended patient-directed conversations, led to no improvement in HPV vaccine coverage [45].
Based on these findings, the announcement approach was further refined and is now: Announce + Connect, Clarify, Counsel. Messages for the Clarify step were obtained from a review of online materials and interviews with providers and then tested with US parents [46].
4.2. Working with media: helping HCPs to address HPV queries
The first step towards working with media is the realization that media work differently from healthcare or science: they have their own deadlines, headlines, sources and influencers. When interacting with the media, HCPs should be prepared: know how HPV is being covered in the news and social media; understand the main themes and arguments presented on various sides of the issue; and be aware of the gaps in stories, facts, or perspectives that could help improve the support for HPV vaccination. Furthermore, HCPs are advised to develop effective communication skills to avoid common pitfalls. It is important to build relationships with the media (if they know you in times of peace, they may turn to you first in times of crisis). One must strive for evidence-based communication, get your facts right, while keeping in mind that cultural adaptation of materials may be necessary. It is important to help HPV vaccine advocates tell their stories and then amplify messages through social media. Another important strategy is to be opportunistic by linking in HPV vaccination information to important events, taking advantage of breaking news, and providing key information when the target population may be receptive. Communication with the media should always be based on mutual respect and ethical communication standards.
4.3. The value of training for HCPs
As stated previously, HCPs are an important source of immunization information. National vaccination programmes in England are supported by a series of cross-sectional surveys exploring attitudes to the infant and teenager immunization programmes [47,48]. The 2017 surveys showed that parental confidence in their decision to immunize their child was increased following discussion with HCPs. Additionally, 16% of parents who planned not to immunize their child and 27% of parents who were previously undecided, decided to immunize after talking to an HCP. The information provided by HCPs is well-trusted and, when immunization programs are supported by HCPs, confidence is heightened. Fewer than 10% parents and teenagers reported having come across information that would make them concerned about any teenage vaccination. The small number of those with any concerns were more likely to have seen this online. Only 10% of parents and 7% of teens reported getting any immunization information online. Ninety three percent of teenagers trusted immunization advice given by health professionals [48]. This compares to only 17% who reported that they would trust immunisation information on social media. [Joanne Yarwood, personal communication, data submitted for publication, reference will be updated once accepted].
At the same time, HCPs feel more confident in giving advice when they receive appropriate training, indicating the importance of appropriate training of HCPs [49]. Therefore, it is essential to ensure that all those delivering immunizations are well trained and confident when providing immunization information. Recently, in the UK, the national minimum standards and core curriculum for immunization training, which describes the training for all registered HCPs involved in immunization, has been revised through a consultative process [50]. Furthermore, a new e-learning program has been developed covering topics such as: the national immunization policy; immunology; vaccine preventable diseases; communication with patients and parents; legal aspects; vaccine storage and administration [Joanne Yarwood, personal communication].
5. Strengths and weaknesses of different school and GP-based HPV vaccination programs
School-based and GP-based HPV vaccination programs are used extensively world-wide.
Several strengths of school-based vaccination programs have been defined [[51], [52], [53]]: the ease to distribute and collate information within the school environment; the cost-effectiveness of having many vaccinees in the same place; and given the well-defined number of vaccinees in each setting, vaccine uptake is easy to calculate. Furthermore, children, including pre-adolescents in secondary school, are a captive audience in their classrooms, potentially leading to higher uptake. Similarly, because students are among friends, vaccination may turn into a shared experience, again potentially leading to higher uptake. As a weakness, this group setting may turn against the program, because vaccination of groups may be more prone to mass psychogenic illness, e.g. fainting, especially if the actual vaccination process is witnessed by the peers. This is particularly the case in adolescent age groups, and less in pre-teen age groups: a recent systematic review described eight clusters of anxiety-related adverse events following immunization (AEFI) occurring in both rural and urban settings, as well as in high-, middle- and low-income countries. Among the patients, males and females were affected equally, and seven out of eight clusters occurred in school-aged population, with 6 out of 7 occurring in 12–17-year-olds [54]. Another possible weakness is parents’ concerns about giving up their control [Emilie Karafillakis, personal communication]. Other weaknesses of school-based vaccination programs include the need for a centralized database for tracking vaccinations, and possible shortage of appropriately trained personnel, especially in settings requiring immunizations to be given by HCPs. Finally, school-based vaccination programs may not reach all children, particularly in countries where school is not compulsory, but also for children not attending compulsory school.
The major strength of the GP-based program is the personal relationship between the GP and the vaccinees and their parents; the GP knows the child and the family and has insight into their medical history. Weaknesses of a GP-based program include the need for a centralized, national database for tracking vaccinations, similar to the school-based program; the fact that the patient must visit the GP to get vaccinated – healthy people may not visit their GP on a regular basis; vaccine uptake is more difficult to calculate; and the GP-based system is sub-optimal for reaching underserved groups (although this can be remedied by health mediators as shown in Bulgaria). Finally, a major weakness is vaccine-hesitant GPs, which may affect vaccination of all patients in the practice.
Other vaccine delivery programs do exist, such as clinic-based programs. The main weakness of clinic-based programs is that the HPV vaccination rates are lower in countries with clinic-based provision. In countries using both systems, such as Canada [55], or the UK, early in the program [56], rates are lower in clinics than in school programs. Other vaccine delivery programs were not specifically discussed.
6. Information and training materials
6.1. US centers for disease control and prevention (CDC)
The US CDC has a web portal on HPV,5 directed towards parents, HCPs and partner organizations. To support clinicians, factsheets have been developed: addressing the 2-dose topic; providing specific vaccine information; and “tips and timesavers,” helping clinicians to provide parents with the clearest answers to their HPV questions.
To allow clinicians to see and hear fellow clinicians talk about HPV and HPV vaccination, an online series of videos called #HowIRecommend,6 is available and others are in development.
Slide decks with information on the burden of disease, HPV vaccine recommendations and effective communication with parents are available to anyone who would like to use it, as well as web-on-demand continuing education courses for immunization providers. Finally, a #preteenvaxscene webinar series7 has been produced on a variety of topics related to increasing the use of recommended vaccines in preteens and teens.
6.2. European Centre for Disease Prevention and Control (ECDC)
The ECDC8 provides a wealth of material, including communication guides and publications on general immunization, vaccine hesitancy, cultural adaptation of communication materials and vaccine-specific material. A communication toolkit,9 aiming to increase immunization uptake, provides template materials. Finally, infographics and brochure on a range of vaccination-related topics are also available.
6.3. World health organization, regional office for europe (WHO/Europe)
Within the vaccine-preventable diseases and immunization program (VPI), WHO-Europe established an informal HPV peer group of immunization managers from Austria, Denmark, France, Ireland, Netherlands, Sweden and the United Kingdom, who participate in bimonthly teleconferences and semi-annual face-to-face meetings to share lessons learned, advise each other and discuss the latest evidence on HPV. VPI also developed supporting materials, including a series of HPV videos10 and a comprehensive HPV ‘questions and answers’ package11 segmented by key target audiences.
Furthermore, in 2017, VPI provided comprehensive support and worked closely with three Member States in the region which were introducing the HPV vaccine: Armenia, Georgia and the Republic of Moldova. This support included formative research, in-depth interviews and focus groups to identify barriers and enablers relevant to the introduction of the HPV vaccine. This support also identified possible communication channels, trusted sources and development of key messages related to the vaccination program which resulted in the development of a field guide that was piloted by the three member states upon introduction of the HPV vaccine. The use and usefulness of the guide was evaluated, and based on encouraging results, the field guide was consequently revised into a tool for new vaccine introduction in general.
An online library with more than 20 supporting documents for building vaccination confidence and responding to crisis is available from WHO-Europe. The package consists of a “vaccination and trust” background document; support documents; and a training program12.
Finally, training modules are available for HCPs on vaccine safety and contraindications13 to prepare confident key trainers and provide training materials to educate front line HCPs. Topics included in the modules are: monitoring of vaccines' safety during pre-and post-licensure periods; WHO recommendations on contraindications and supporting evidence; the basics of AEFI classification and causality assessment; and routine and new vaccines’ impact assessment.
6.4. The Catalan Institute of Oncology online oncology community
E-oncologia14 is the virtual oncology training program of the Catalan Institute of Oncology, with 71 courses and 2300 hours of training materials, currently in nine languages. One part of the program is devoted to cervical cancer prevention. This module is endorsed by the International Federation of Gynecology and Obstetrics, the International Atomic Energy Agency, the Unions for International Cancer Control, the International Agency for the Research on Cancer and the WHO. The module is fully online, with three moderated forum discussions on epidemiology and natural history, vaccination and screening. As trainees can become tutors, a snowball effect in training is envisioned. The module can be calibrated to local needs, such as an adapted course for Colombia after the vaccine crisis. As described earlier, the HCP educational needs during the Colombian crisis were addressed by the e learning program at ICO/NCI in Bogota. Within the first three months of offering the course, over 4000 HCP (GPs, nurses, midwifes, etc.) in Colombia participated. The course has national tutors, who are ready to address any questions.
Furthermore, translation of the course into new languages is possible, as is currently ongoing for a Greek translation. In order to contribute to the education of HCPs in Greece and subsequently to the promotion of vaccination and screening among Greek women, the Hellenic HPV Society decided to collaborate with ICO to translate the online cervical cancer prevention course.
6.5. Other resources
-
-
The US National HPV Vaccination Roundtable provides a short video, explaining the association between HPV and head and neck cancers. This video can be used at staff meetings or conferences. Furthermore, clinician and systems action guides, as well as, a school nurse toolkit have been developed.15
-
-
The American Academy of Pediatrics has developed a toolkit16 to help clinicians educate other HCPs; discuss HPV vaccination with parents; and make necessary changes to improve HPV vaccination rates. They also provide a Nurse Tip Sheet, a 2-page vaccination resource for primary care nurses.
-
-
The American Cancer Society has developed a “Steps” guide17 for increasing HPV vaccination, including a toolkit, a road map and a portal with resources.
-
-
Vaccine makers also often have educational material available on their websites.
6.6. Discussion
While many resources are available, further needs were raised during the meeting, especially related to the need for answers to specific, frequently asked questions in a Q&A format targeting vaccines, parents and HCPs. This material should be in plain language and easy to comprehend, but with high-quality and evidence-based information, to circumvent concerns from escalating and ultimately to increase vaccine uptake. The information should be country-specific, which goes beyond simple translation, as the message should ideally be locally and culturally adapted [57].
Because of the important role of HCPs, they should be given time to study through continuing (medical) education, with support from their professional societies. Many forms of training should be offered, including face-to-face settings, webinars, e-learning courses, conferences and peer-to-peer learning, so HCPs can choose the most appropriate form for themselves, and feel more comfortable in addressing the subject of HPV vaccination. In addition, efforts should be made to train current medical and paramedical students on vaccinology, as they are the health care providers of the future.
As many resources are already available, it is important not to re-invent the wheel, and to learn from what has already been created. But it might be useful to review and rank the available resources, and make sure that the best resources are available in several languages, to reach a broad public, in a one-stop shop.
7. Lessons learned
The burden of addressing public vaccine hesitancy is increasingly being placed on HCPs. However, it is easy to forget that HCPs are also members of the public: they can have the same questions, the same doubts, and the same fears about vaccines as their patients have. This is of greater importance for HCP with specializations not traditionally involved in vaccinations (e.g. obstetricians and gynecologists). Insufficient communication, engagement and education of HCPs related to vaccination can jeopardize vaccine uptake and attempts at improving public confidence. To address this, it is essential to take the time to actively listen to HCPs’ concerns about vaccination. HCPs need more support to manage the changes in society (more assertive patients, actively seek information, which is not always scientifically correct) as well as the quickly evolving vaccine environment, through training, incorporation of vaccinology into the medical curriculum, providing access to tools and resources. Those involved in vaccine program design and implementation need to rebuild trust among HCPs by including them at the earliest possible stage: in decision making for vaccine recommendations and policies, as well as the design of communication materials. Finally, as vaccinated HCPs are more likely to recommend vaccination to their patients, there is a need to restore and maintain vaccination as a norm among the health community: the HCP as role model.
HCPs can be trained to use the announcement approach to increase HPV vaccination uptake. The approach should be: Announce, (and only if parents hesitate) Connect, Clarify, Counsel, while keeping in mind that parents who say no, may say yes later. Therefore, vaccination should be brought up again at each successive visit.
Education and information material is available from many sources, but centralization and coordination of these materials might be useful, to direct both HCPs and the general public to the best material available.
Consistent introduction of vaccinology into the medical and paramedical curriculum should be pursued. In general, the focus in training is still too much on cure, and not enough on prevention. The apparent lack of training on vaccinology during the curriculum can be remedied through summer courses, and later with inclusion of modules on vaccinology. Post-academic training can be provided through symposia for vaccinators (e.g., Flemish Valentine Vaccination symposia). This provides an opportunity to discuss questions and possible concerns, while a survey of these questions may provide insight into vaccine confidence locally.
Disclaimer
The HPV Prevention and Control board is supported by in kind contributions and support from the international experts involved and their institutions. To set up the activities, the secretariat obtains unrestricted grants from industry (GlaxoSmithKline Biologicals, Merck, Abbott, Sanofi Pasteur and MSD). All funds are handled according to the rules of the University of Antwerp. No remuneration for experts or speakers is provided.
Conflicts of interest
AV University of Antwerp obtained unrestricted educational grants from GSK, Merck and SPMSD; speakers fees from Merck were paid directly to an educational fund held by the University of Antwerp.
PB received research funding for epidemiological research and HTA evaluations from MSD, Pfizer, GSK, Sanofi Pasteur and Seqirus, and participated to advisory boards and symposia sponsored by the same companies.
HCM declares no conflict of interest but has received honoraria for lectures in symposia and meetings by SF and MSD.
JY declares no conflict of interest.
NTB has received research grants from and served as a paid consultant for Merck and FDA.
FXB received research funding via his institution from GSK, Merck, Qiagen, Roche, and SPMSD, and reimbursement of travel expenses for attending symposia, meetings and/or speaking at conferences from GSK, Merck, Qiagen, Roche, and SPMSD.
SH declares no conflict of interest.
RC declares no conflict of interest.
ELF declares no conflict of interest but discloses that he occasionally serves as advisor for companies involved with HPV diagnostics (Roche) and HPV vaccination (Merck, GSK). His university has received unconditional grants from Roche and Merck for studies that he initiated.
MA declares no conflict of interest.
NM declares no conflict of interest.
MK declares no conflict of interest.
JP declares no conflict of interest.
MB received medical writing fees from vaccine-producing pharmaceutical companies, including Merck, SPMSD and GSK.
EK has received funding through her institution from Merck to convene a research symposium and from Merck and SPMSD for speaking at conferences, symposiums, and meetings.
PVD acts as principal investigator for HPV vaccine trials conducted on behalf of the University of Antwerp, for which the University obtained research grants from vaccine manufacturers; speakers fees for presentations were paid directly to an educational fund held by the University of Antwerp.
Acknowledgements
We thank the session chairs and speakers for their valuable slides, presentations, and comments; and the meeting participants for their thorough and insightful discussions. The authors are grateful to Lisa Lindsay (P95) for critical review of the manuscript.
References
- 1.Vorsters A., Arbyn M., Baay M., Bosch X., de Sanjose S., Hanley S. Overcoming barriers in HPV vaccination and screening programs. Papillomavirus Res. 2017;4:45–53. doi: 10.1016/j.pvr.2017.07.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.HPV Prevention and Control Board . 2017. Prevention and Control of HPV and HPV Related Cancers in Denmark: Lessons Learned and the Way Forward. Meeting Report Copenhagen. Denmark, November 17-18, 2016. [Google Scholar]
- 3.HPV Prevention and Control Board . 2017. Building Trust, Managing Risks: Vaccine Confidence and Human Papillomavirus Vaccination. Meeting Report London. England, June 7-8, 2017. [Google Scholar]
- 4.HPV Prevention and Control Board . 2018. Prevention and Control of HPV and HPV Related Cancers in Ireland and the UK: Lessons Learned and the Way Forward. Meeting Report Dublin, Ireland. November 30 - December 1, 2017. [Google Scholar]
- 5.Kuster S.P., Shah P.S., Coleman B.L., Lam P.P., Tong A., Wormsbecker A. Incidence of influenza in healthy adults and healthcare workers: a systematic review and meta-analysis. PLoS One. 2011;6(10) doi: 10.1371/journal.pone.0026239. e26239. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Weingarten S., Riedinger M., Bolton L.B., Miles P., Ault M. Barriers to influenza vaccine acceptance. A survey of physicians and nurses. Am. J. Infect. Contr. 1989;17(4):202–207. doi: 10.1016/0196-6553(89)90129-6. [DOI] [PubMed] [Google Scholar]
- 7.Ridgway J.P., Bartlett A.H., Garcia-Houchins S., Carino S., Enriquez A., Marrs R. Influenza among afebrile and vaccinated healthcare workers. Clin. Infect. Dis. 2015;60(11):1591–1595. doi: 10.1093/cid/civ163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Maltezou H.C., Drancourt M. Nosocomial influenza in children. J. Hosp. Infect. 2003;55(2):83–91. doi: 10.1016/s0195-6701(03)00262-7. [DOI] [PubMed] [Google Scholar]
- 9.Eibach D., Casalegno J.S., Bouscambert M., Benet T., Regis C., Comte B. Routes of transmission during a nosocomial influenza A(H3N2) outbreak among geriatric patients and healthcare workers. J. Hosp. Infect. 2014;86(3):188–193. doi: 10.1016/j.jhin.2013.11.009. [DOI] [PubMed] [Google Scholar]
- 10.Payet C., Voirin N., Ecochard R., Vanhems P. Influence of observable and unobservable exposure on the patient's risk of acquiring influenza-like illness at hospital. Epidemiol. Infect. 2016;144(10):2025–2030. doi: 10.1017/S0950268816000145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wilde J.A., McMillan J.A., Serwint J., Butta J., O'Riordan M.A., Steinhoff M.C. Effectiveness of influenza vaccine in health care professionals: a randomized trial. JAMA. 1999;281(10):908–913. doi: 10.1001/jama.281.10.908. [DOI] [PubMed] [Google Scholar]
- 12.Vanhems P., Baghdadi Y., Roche S., Benet T., Regis C., Lina B. Influenza vaccine effectiveness among healthcare workers in comparison to hospitalized patients: a 2004-2009 case-test, negative-control, prospective study. Hum. Vaccines Immunother. 2016;12(2):485–490. doi: 10.1080/21645515.2015.1079677. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Pearson M.L., Bridges C.B., Harper S.A. Influenza vaccination of health-care personnel: recommendations of the healthcare infection control practices advisory committee (HICPAC) and the advisory committee on immunization practices (ACIP) MMWR Recomm. Rep. (Morb. Mortal. Wkly. Rep.) 2006;55(Rr-2):1–16. [PubMed] [Google Scholar]
- 14.Saxen H., Virtanen M. Randomized, placebo-controlled double blind study on the efficacy of influenza immunization on absenteeism of health care workers. Pediatr. Infect. Dis. J. 1999;18(9):779–783. doi: 10.1097/00006454-199909000-00007. [DOI] [PubMed] [Google Scholar]
- 15.Van Buynder P.G., Konrad S., Kersteins F., Preston E., Brown P.D., Keen D. Healthcare worker influenza immunization vaccinate or mask policy: strategies for cost effective implementation and subsequent reductions in staff absenteeism due to illness. Vaccine. 2015;33(13):1625–1628. doi: 10.1016/j.vaccine.2015.01.048. [DOI] [PubMed] [Google Scholar]
- 16.Maltezou H.C., Poland G.A. Vaccination policies for healthcare workers in Europe. Vaccine. 2014;32(38):4876–4880. doi: 10.1016/j.vaccine.2013.10.046. [DOI] [PubMed] [Google Scholar]
- 17.To K.W., Lai A., Lee K.C., Koh D., Lee S.S. Increasing the coverage of influenza vaccination in healthcare workers: review of challenges and solutions. J. Hosp. Infect. 2016;94(2):133–142. doi: 10.1016/j.jhin.2016.07.003. [DOI] [PubMed] [Google Scholar]
- 18.Babcock H.M., Gemeinhart N., Jones M., Dunagan W.C., Woeltje K.F. Mandatory influenza vaccination of health care workers: translating policy to practice. Clin. Infect. Dis. 2010;50(4):459–464. doi: 10.1086/650752. [DOI] [PubMed] [Google Scholar]
- 19.Maltezou H.C., Tsakris A. Vaccination of health-care workers against influenza: our obligation to protect patients. Influenza Other Respir. Viruses. 2011;5(6):382–388. doi: 10.1111/j.1750-2659.2011.00240.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Paterson P., Meurice F., Stanberry L.R., Glismann S., Rosenthal S.L., Larson H.J. Vaccine hesitancy and healthcare providers. Vaccine. 2016;34(52):6700–6706. doi: 10.1016/j.vaccine.2016.10.042. [DOI] [PubMed] [Google Scholar]
- 21.Neuberger J. Do we need a new word for patients? Lets do away with “patients”. BMJ. 1999;318(7200):1756–1757. doi: 10.1136/bmj.318.7200.1756. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Karafillakis E., Dinca I., Apfel F., Cecconi S., Wurz A., Takacs J. Vaccine hesitancy among healthcare workers in Europe: a qualitative study. Vaccine. 2016;34(41):5013–5020. doi: 10.1016/j.vaccine.2016.08.029. [DOI] [PubMed] [Google Scholar]
- 23.Muscat M., Marinova L., Mankertz A., Gatcheva N., Mihneva Z., Santibanez S. The measles outbreak in Bulgaria, 2009-2011: an epidemiological assessment and lessons learnt. Euro Surveill. 2016;21(9):30152. doi: 10.2807/1560-7917.ES.2016.21.9.30152. [DOI] [PubMed] [Google Scholar]
- 24.Marinova L., Parmakova K., Kojouharova M. Striving for better communication with underserved communities in Bulgaria - a step towards improving immunisation coverage. Probl. Infect. Parasit. Dis. 2014;42(2):35. [Google Scholar]
- 25.World Health Organization Regional Office for Europe . 2014. Tailoring Immunization Programmes (TIP): Outputs of Pilot Implementation in Bulgaria. Copenhagen, Denmark. [Google Scholar]
- 26.National Centre of Infectious and Parasitic Diseases and National Network of Health Mediators Association . 2013. Let's Talk about Disease Protection: How to Increase the Number of Children's Immunizations. Practical Guide for Health Care Providers and Health Mediators. Sofia, Bulgaria. [Google Scholar]
- 27.Larson H.J., de Figueiredo A., Karafillakis E., Rawal M. 2018. State of Vaccine Confidence in the EU 2018. Luxembourg. Report No.: EW-06-18-233-EN-N Contract No. [Google Scholar]
- 28.Kennedy J. Populist politics and vaccine hesitancy in Western Europe: an analysis of national-level data. Eur. J. Public Health. 2019;29(3):512–516. doi: 10.1093/eurpub/ckz004. [DOI] [PubMed] [Google Scholar]
- 29.Dimirtova V., Kurchatova A., Minkova A., Georgieva T., Naseva E., Kojouharova M. Is social media a realistic information channel during epidemics and pandemics? Results from the citizen consultation conducted in Bulgaria. Epidemics Pandemics: Response Soc. 2017;(4):6–10. [Google Scholar]
- 30.Palmer T., Wallace L., Pollock K.G., Cuschieri K., Robertson C., Kavanagh K. Prevalence of cervical disease at age 20 after immunisation with bivalent HPV vaccine at age 12-13 in Scotland: retrospective population study. BMJ. 2019;365:l1161. doi: 10.1136/bmj.l1161. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.National Health Services Scotland . 2017. HPV Immunisation Statistics Scotland. [Google Scholar]
- 32.Larson H.J., Cooper L.Z., Eskola J., Katz S.L., Ratzan S. Addressing the vaccine confidence gap. Lancet. 2011;378(9790):526–535. doi: 10.1016/S0140-6736(11)60678-8. [DOI] [PubMed] [Google Scholar]
- 33.Larson H.J., de Figueiredo A., Xiahong Z., Schulz W.S., Verger P., Johnston I.G. The state of vaccine confidence 2016: global insights through a 67-country survey. EBioMedicine. 2016;12:295–301. doi: 10.1016/j.ebiom.2016.08.042. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Bish A., Yardley L., Nicoll A., Michie S. Factors associated with uptake of vaccination against pandemic influenza: a systematic review. Vaccine. 2011;29(38):6472–6484. doi: 10.1016/j.vaccine.2011.06.107. [DOI] [PubMed] [Google Scholar]
- 35.Munoz N., Bosch F.X., de Sanjose S., Tafur L., Izarzugaza I., Gili M. The causal link between human papillomavirus and invasive cervical cancer: a population-based case-control study in Colombia and Spain. Int. J. Cancer. 1992;52(5):743–749. doi: 10.1002/ijc.2910520513. [DOI] [PubMed] [Google Scholar]
- 36.Bosch F.X., Munoz N., de Sanjose S., Izarzugaza I., Gili M., Viladiu P. Risk factors for cervical cancer in Colombia and Spain. Int. J. Cancer. 1992;52(5):750–758. doi: 10.1002/ijc.2910520514. [DOI] [PubMed] [Google Scholar]
- 37.Bosch F.X., Munoz N., de Sanjose S., Navarro C., Moreo P., Ascunce N. Human papillomavirus and cervical intraepithelial neoplasia grade III/carcinoma in situ: a case-control study in Spain and Colombia. Cancer Epidemiol. Biomark. Prev. 1993;2(5):415–422. [PubMed] [Google Scholar]
- 38.Munoz N., Bosch F.X., de Sanjose S., Vergara A., del Moral A., Munoz M.T. Risk factors for cervical intraepithelial neoplasia grade III/carcinoma in situ in Spain and Colombia. Cancer Epidemiol. Biomark. Prev. 1993;2(5):423–431. [PubMed] [Google Scholar]
- 39.Munoz N., Manalastas R., Jr., Pitisuttithum P., Tresukosol D., Monsonego J., Ault K. Safety, immunogenicity, and efficacy of quadrivalent human papillomavirus (types 6, 11, 16, 18) recombinant vaccine in women aged 24-45 years: a randomised, double-blind trial. Lancet. 2009;373(9679):1949–1957. doi: 10.1016/S0140-6736(09)60691-7. [DOI] [PubMed] [Google Scholar]
- 40.Munoz N., Kjaer S.K., Sigurdsson K., Iversen O.E., Hernandez-Avila M., Wheeler C.M. Impact of human papillomavirus (HPV)-6/11/16/18 vaccine on all HPV-associated genital diseases in young women. J. Natl. Cancer Inst. 2010;102(5):325–339. doi: 10.1093/jnci/djp534. [DOI] [PubMed] [Google Scholar]
- 41.Arbyn M., Xu L., Simoens C., Martin-Hirsch P.P. Prophylactic vaccination against human papillomaviruses to prevent cervical cancer and its precursors. Cochrane Database Syst. Rev. 2018;5:Cd009069. doi: 10.1002/14651858.CD009069.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Gilkey M.B., Malo T.L., Shah P.D., Hall M.E., Brewer N.T. Quality of physician communication about human papillomavirus vaccine: findings from a national survey. Cancer Epidemiol. Biomark. Prev. 2015;24(11):1673–1679. doi: 10.1158/1055-9965.EPI-15-0326. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Lindley M.C., Jeyarajah J., Yankey D., Curtis C.R., Markowitz L.E., Stokley S. Comparing human papillomavirus vaccine knowledge and intentions among parents of boys and girls. Hum. Vaccines Immunother. 2016;12(6):1519–1527. doi: 10.1080/21645515.2016.1157673. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Sturm L., Donahue K., Kasting M., Kulkarni A., Brewer N.T., Zimet G.D. Pediatrician-parent conversations about human papillomavirus vaccination: an analysis of audio recordings. J. Adolesc. Health. 2017;61(2):246–251. doi: 10.1016/j.jadohealth.2017.02.006. [DOI] [PubMed] [Google Scholar]
- 45.Brewer N.T., Hall M.E., Malo T.L., Gilkey M.B., Quinn B., Lathren C. Announcements versus conversations to improve HPV vaccination coverage: a randomized trial. Pediatrics. 2017;139(1) doi: 10.1542/peds.2016-1764. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Shah P.D., Calo W.A., Gilkey M.B., Boynton M.H., Alton Dailey S., Todd K.G. Questions and concerns about HPV vaccine: a communication experiment. Pediatrics. 2019;143(2) doi: 10.1542/peds.2018-1872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Campbell H., Edwards A., Letley L., Bedford H., Ramsay M., Yarwood J. Changing attitudes to childhood immunisation in English parents. Vaccine. 2017;35(22):2979–2985. doi: 10.1016/j.vaccine.2017.03.089. [DOI] [PubMed] [Google Scholar]
- 48.Letley L., Campbell H., Edwards A., Saliba V., J Y. European Public Health Conference Winds of Change: towards New Ways of Improving Public Health in Europe; 28 November - 1 December 2018. vol. 26. Eur J Public Health; Ljubljana, Slovenia: 2018. What do parents and young people in England think about immunisation? A national interview survey. S4. [Google Scholar]
- 49.Vishram B., Letley L., Jan Van Hoek A., Silverton L., Donovan H., Adams C. Vaccination in pregnancy: attitudes of nurses, midwives and health visitors in England. Hum. Vaccines Immunother. 2018;14(1):179–188. doi: 10.1080/21645515.2017.1382789. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Public Health England . 2018. National Minimum Standards and Core Curriculum for Immunisation Training for Registered Healthcare Practitioners. [Google Scholar]
- 51.Paul P., Fabio A. Literature review of HPV vaccine delivery strategies: considerations for school- and non-school based immunization program. Vaccine. 2014;32(3):320–326. doi: 10.1016/j.vaccine.2013.11.070. [DOI] [PubMed] [Google Scholar]
- 52.Shah P.D., Gilkey M.B., Pepper J.K., Gottlieb S.L., Brewer N.T. Promising alternative settings for HPV vaccination of US adolescents. Expert Rev. Vaccines. 2014;13(2):235–246. doi: 10.1586/14760584.2013.871204. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Jacobson R.M., Agunwamba A.A., St Sauver J.L., Finney Rutten L.J. The most effective and promising population health strategies to advance human papillomavirus vaccination. Expert Rev. Vaccines. 2016;15(2):257–269. doi: 10.1586/14760584.2016.1116947. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Loharikar A., Suragh T.A., MacDonald N.E., Balakrishnan M.R., Benes O., Lamprianou S. Anxiety-related adverse events following immunization (AEFI): a systematic review of published clusters of illness. Vaccine. 2018;36(2):299–305. doi: 10.1016/j.vaccine.2017.11.017. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Musto R., Siever J.E., Johnston J.C., Seidel J., Rose M.S., McNeil D.A. Social equity in Human Papillomavirus vaccination: a natural experiment in Calgary Canada. BMC Public Health. 2013;13:640. doi: 10.1186/1471-2458-13-640. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Department of Health England . 2012. Annual HPV Vaccine Coverage in England in 2010/2011 - Routine Programme for School Year 8 Females (12 to 13 Years Old) [Google Scholar]
- 57.European Centre for Disease Prevention and Control . A five-step guide; Stockholm: 2016. Translation Is Not Enough – Cultural Adaptation of Health Communication Materials. [Google Scholar]