Abstract
In 2012, Colombia implemented a school-based HPV vaccination program of a 3-dose series for nine year old girls. Following a mass psychogenic response after vaccination in a Colombian town, vaccination rates dropped from 80% in 2012–2013 to 5% in 2016.
The study aimed to identify barriers and facilitators of HPV vaccine uptake among girls eligible for vaccination in the initial years of vaccine implementation from 2012 to 2014, and their parents. We conducted 19 individual qualitative interviews and 18 focus groups with an average of 5 girls, in Manizales, Colombia between 2016 and 2017. In total, 49 girls from six schools and 58 of their parents participated in the study.
Participants had some degree of awareness about cervical cancer, especially among those of middle and upper socioeconomic level. However, the vaccine was known as a prevention measure only after pap-smears and condoms. The main facilitator for vaccine uptake for parents was the desire to prevent diseases in general and for girls, it was facilitated by receiving positive information about the vaccine. The main barriers for vaccine uptake or for three doses completion were the event in Carmen de Bolivar, fear of adverse effects and fear of needles. Girls and parents stated that they received little or no information from schools or health care services about the HPV vaccine prior to vaccination. Our results suggest that improving HPV vaccination rates in Colombia will require a comprehensive education program including mass media information about HPV vaccine.
Keywords: Papillomavirus vaccines, Qualitative research, Patient acceptance of health care, School age population, Immunization programs, Vaccination refusal
Highlights
-
•
In general, girls were unaware of the causes of invasive cervical cancer.
-
•
Adults and girls from the middle and upper socioeconomic have more awareness of HPV.
-
•
Carmen de Bolivar event was an important barrier for HPV vaccination uptake.
-
•
Information provided to girls and parents prior vaccination seemed to be insufficient.
1. Introduction
The recognition of Human Papilloma Virus high-risk types (HR-HPV) as the etiologic agent of cervical and other ano-genital and upper respiratory tack malignancies has led to the development of prophylactic vaccines for the prevention of these cancers. Currently, there are three prophylactic vaccines against HPV infection: the Bivalent (Cervarix®) vaccine contains Viral Like Particles (VLPs) targeting HPV16 and 18, the quadrivalent (Gardasil ®) vaccine targets an additional two low-risk types, HPV 6 and 11, that causes 90% of genital warts, and the Nonavalent (Gardasil-9 ®) vaccine contains the same VLPs of the quadrivalent vaccine but also high-risk types 31, 33, 45, 52, and 58 (Gallagher et al., 2017).
Following the approval of HPV vaccines by regulatory agencies in North America and Europe and prequalification from the World Health Organization, many countries have adopted HPV vaccination as a key strategy for reducing cervical cancer burden. Colombia included the quadrivalent HPV vaccine in the Expanded Program on Immunization (PAI, for the initials of Programa Ampliado de Inmunizacion) in August 2012 (Reina and Muñoz, 2014). The routine program was to offer the quadrivalent HPV vaccine 3-dose regimen to the target population of girls aged 9 years in fourth grade elementary education, attending both public and private schools. The second and third doses were scheduled at two and six months after the first dose. In 2013, the program was expanded to reach non-school registered girls aged 9 years old, and in 2014, the program was further expanded to include girls attending up to the last year of high-school. Although there were differences between states, vaccination coverage in 2012 was 80% or higher for all three doses of the HPV vaccine (Simas et al., 2018). Despite the successful uptake of the vaccine, the program was adversely affected by the controversy and misinformation regarding its safety, specifically after the event of “El Carmen de Bolivar”, which occurred from May to October 2014. During this period, about 600 girls reported a variety of symptoms that required medical attention (Simas et al., 2018). As a result, news and social media platforms were dominated by hundreds of videos and images of girls fainting and being carried into hospitals unconscious (Simas et al., 2018). Following this event, the Colombian National Institute of Health conducted an epidemiological investigation and concluded that the event was due to a mass psychogenic reaction and not to biological factors associated with the vaccine (Martinez et al., 2015). Despite this finding, the event caused a dramatic reduction in uptake of the HPV vaccine with completion rates of all three doses dropping from 88% in 2012 to 5% by 2016 (Simas et al., 2018).
In Colombia, where the prevalence of cervical cancer is high, the Carmen de Bolivar event has decimated the effective and successful implementation of the HPV vaccine. Low levels of knowledge about the disease and the effectiveness of the HPV vaccine for preventing cervical cancer in Colombia were identified in studies conducted before (Wiesner et al., 2010) and after (Torrado-Arenas et al., 2017) the introduction of the vaccine. There was also evidence that parents, especially those of high socioeconomic level, believed that the vaccine could promote promiscuity (Wiesner et al., 2010). Despite the findings from the study conducted prior to the implementation of the vaccination program, information about what the vaccine is and what it could prevent may have been limited within the implementation of the program. The lack of information may have contributed to the general mistrust in vaccines and the attribution of the El Carmen de Bolivar event to the HPV vaccine (Grillo-Ardila, 2014).
Given the lack of public confidence of the HPV vaccine, uptake remains low. Therefore, it is imperative to understand barriers and facilitators of vaccination in Colombia in order to improve girls and parents' knowledge and their willingness to be vaccinated in the future. The degree of knowledge and cultural beliefs about the vaccine as well as religious, socio-economic factors and barriers of health systems along with the acceptance of the vaccine by government agencies, parents, educators and adolescents have an important role in the success or failure of vaccination programs worldwide (Wigle et al., 2013; Winkler et al., 2008). Studies in different countries have found that the lack of knowledge about cervical cancer, HPV and the vaccine (Camano-Puig and Sanchis, 2015; Cordeiro et al., 2014; Firenze et al., 2015; Godoy Verdugo et al., 2013), fear of adverse effects (Cordeiro et al., 2014; Firenze et al., 2015; Godoy Verdugo et al., 2013; Rambout et al., 2014) and perception of low susceptibility to the disease (Godoy Verdugo et al., 2013; Hendry et al., 2013) constitute the main barriers for the acceptance and intake of the HPV vaccine. In contrast, factors that promote the acceptability of the vaccine in the parents of adolescents were the prevention of the disease, the desire to protect their daughters and the perception of the illness as something serious (Alder et al., 2015; Bair et al., 2008; Godoy Verdugo et al., 2013). To date barriers and facilitators to HPV vaccine uptake remain unknown following the Carmen de Bolivar event.
This qualitative study was conducted among girls eligible for vaccination in 2012 to 2014 and their parents to identify important themes about barriers and facilitators towards acceptability and uptake of the HPV vaccine prior to the Carmen de Bolivar event. Results are important to inform a larger population-based survey that will be conducted to identify factors that may be important for the girl's and/or parents vaccination decision.
2. Materials and methods
2.1. Study design and participants
This qualitative study was conducted between September 2016 and February 2017. Participants were recruited from ten schools, representative of the 207 schools in Manizales, a town located in the central region of Colombia. Schools were selected from an official list provided by the Secretary of Education, based on their socioeconomic area, location (rural and urban), school funding type (public or private), gender enrollment (female only or mixed gender), and religious practices (religious or secular). Among the six schools that accepted to participate, two schools each were of low, middle and of high socioeconomic levels, only one school was located in a rural location, an equal number of schools were public and private, two schools were religious and the rest were secular, and two schools enrolled girls only.
Eligible girls were those aged at least 9 years old who resided in Manizales from 2012 to 2014 and were registered in sixth, seventh or eighth grade of high school at the time of the study. Four schools provided official school registration lists of vaccinated or unvaccinated girls which were used to randomly select 10–12 girls from each classroom. After selecting the girls, they received a letter inviting them and their parents to participate in the study. In the other two schools, the selection was based on convenience sampling. School officers presented the project and asked for voluntary participation during regular parents' meetings. In all instances, the acceptance of the parent was a requirement for the girls' participation. Given the low number of eligible unvaccinated girls, all unvaccinated girls and their parents were invited to participate.
The study was approved by the Bioethics Committee of University of Antioquia. All parents signed an informed consent for themselves and for their daughter's participation. In addition, all girls signed an informed assent. To protect the anonymity of all participants, a unique study number was assigned. The study complied with Colombian regulation for studies conducted among adolescents. Recruitment and interviews were supervised by a professional psychologist with experience in adolescent psychological issues. The interviews and focus groups took place at schools in empty classrooms. At the end of the study participation, every participant received information and a brochure about cervical cancer, HPV and HPV vaccine.
2.2. Data collection
2.2.1. Interview tools
Individual interviews and focus groups were conducted separately by two trained female researchers, a psychologist for girls and a nurse for parents. A standardized questionnaire was administered by the trained researchers privately to obtain sociodemographic information. Information about the knowledge of cervical cancer, HPV infection, and barriers and facilitators for vaccine uptake was obtained through individual interviews and focus groups. The interviews took an average of 20 min and the focus groups took an average of 40 min.
For the interviews and focus groups, a guide with open questions was designed (Supplementary Table 1) based on the Health Belief Model (HBM) (Glanz et al., 2008). In the context of HPV vaccination, the HBM suggests that the intention to uptake the vaccine can be predicted by the individual's perceived severity of and susceptibility to HPV infection or cervical cancer, their beliefs about the benefits and barriers to being vaccinated, or signals for action or facilitators (e.g. a doctor's recommendation) (Bowyer et al., 2014). The perceived susceptibility was defined as the perception of the possibility of developing cervical cancer or being HPV infected; the perceived benefit, as the perception of the effectiveness of the vaccine to reduce the incidence of cervical cancer and the perceived barriers or facilitators, as the situations that prevent or promote access for the uptake of the vaccine. Specific questions were adapted specifically for girls and parents.
The interview guide was validated by experts in the field of cervical cancer and adolescents. After conducting a literature review, a preliminary guide was developed. This guide was assessed by a nurse, a physician and four microbiologists with experience in public health and infectious diseases and a psychologist with a PhD in Health and Clinical Psychology. Each expert provided recommendations, which were reviewed at two meetings and revised accordingly. Then the guide underwent pre-cognitive testing in an interview with two girls and their parents to identify any questions that required further clarification or revision based on areas of confusion revealed in the interviews. The interviews were evaluated and only minor changes were made in terms of the language used and the order of the questions.
2.2.2. Interview and focus groups
Interviews were conducted individually in a private room. Focus groups were conducted separately for girls and for their parents. Focus groups were convened according to the participant's vaccination status (vaccinated or unvaccinated), socioeconomic level (low, middle or high), location (rural or urban), funding (public or private), religious practices (religious or secular), gender enrollment (only girls or both genders). Participants were provided with a transportation incentive and a snack.
Nineteen individual interviews (four vaccinated girls, six unvaccinated girls, four parents of vaccinated girls, and five parents of vaccinated girls) and 18 focus groups with an average of 5 participants per group (six groups of vaccinated girls, two groups of unvaccinated girls, seven groups of parents of vaccinated girls, and three groups of parents of unvaccinated girls) were conducted. The number of interviews and focus groups were determined by saturation of the information. Both the interviews and the focus groups were audio-taped and verbatim transcribed.
Qualitative rigor was maintained in data collection by reproducing the conditions of application in every interview and focus group sessions by using the interview guide and recording the audio of every interview and focus groups. Additionally an independent researcher that did not conducted the focus group reviewed the information. For the data analysis, rigor was maintained by codifying the transcripts independently using Atlas T1 software and reaching consensus in a posterior meeting.
2.3. Data analysis
Descriptive statistics of sociodemographic characteristics of girls and parents was carried out using the Statistical Package for Social Science, version 24.0 (SPSS v 24.0). A content analysis of the information of verbatim transcripts was conducted. Two researchers independently read and coded all the verbatim transcripts identifying emerging issues, concepts, and themes according to research questions. Example codes included awareness of cervical cancer and HPV, perceived susceptibility, sources of information and fear of needles. These codes were entered to Atlas TI version 7.54 to create a model that was adjusted throughout the process of coding and recoding. Disagreement between researchers was resolved through subsequent discussion.
3. Results
The sample consisted of 49 girls and their 58 parents or guardians (referred to as parents). The parents of eight girls but not their girls and both parents of one girl participated in the study. No girl refused participation. Table 1 presents the sociodemographic information of the 49 girls (38 vaccinated and 11 unvaccinated). The majority was 13–15 years old (81.6%), in sixth or seven grade (70%), of low and middle socioeconomic level schools (75%), and from secular urban schools that enrolled both genders (81%).
Table 1.
Socio-demographic characteristics of girls. Manizales, Colombia, September 2016 to February 2017.
| Girls | Unvaccinated |
Vaccinated |
Total |
|||
|---|---|---|---|---|---|---|
| N = 11 | % | N = 38 | % | N = 49 | % | |
| Age | ||||||
| 9–12 | 0 | 0,0 | 4 | 100,0 | 4 | 8,2 |
| 13–15 | 9 | 22,5 | 31 | 77,5 | 40 | 81,6 |
| ≥16 | 2 | 40,0 | 3 | 60,0 | 5 | 10,2 |
| School grade | ||||||
| 6–7 | 9 | 26,5 | 25 | 73,5 | 34 | 69,4 |
| 8–9 | 2 | 15,4 | 11 | 84,6 | 13 | 26,5 |
| ≥10 | 0 | 0,0 | 2 | 100,0 | 2 | 4,1 |
| School socio-economic levela | ||||||
| Low | 1 | 5,0 | 19 | 95,0 | 20 | 40,8 |
| Middle | 7 | 41,2 | 10 | 58,8 | 17 | 34,7 |
| High | 3 | 25,0 | 9 | 75,0 | 12 | 24,5 |
| School location | ||||||
| Rural | 0 | 0,0 | 8 | 100,0 | 9 | 18,4 |
| Urban | 11 | 26,8 | 30 | 73,2 | 40 | 81,6 |
| School funding | ||||||
| Private | 4 | 25,0 | 12 | 75,0 | 16 | 32,7 |
| Public | 7 | 21,2 | 26 | 78,8 | 33 | 67,3 |
| School character | ||||||
| Secular | 8 | 21,1 | 30 | 78,9 | 38 | 77,6 |
| Religious | 3 | 27,3 | 8 | 72,7 | 11 | 22,4 |
| School gender | ||||||
| Female only | 3 | 27,3 | 8 | 72,7 | 11 | 22,4 |
| Mixed | 8 | 21,1 | 30 | 78,9 | 38 | 77,6 |
This level corresponds to the socioeconomic level of the neighborhood in which the school is located. In Colombia there are six stratums to classify households according to their physical characteristics and its surroundings. Stratum 1 and 2 correspond to low socioeconomic levels, 3 and 4 to middle, 5 and 6 to high. National Administrative Department of Statistics (DANE Spanish for Departamento Administrativo Nacional de Estadistica).
Table 2 shows the sociodemographic characteristics of the 58 corresponding parents, of whom 45 (77%) had vaccinated daughters. The majority of parents were female (89.7%) with an average age of 44.1 years (range 29–70), of low or middle socioeconomic level (59.2%), and approximately 20% had a university degree. The majority was married or was in a marital union (67%) and more than half were employed or self-employed (53%).
Table 2.
Socio-demographic characteristics of parents. Manizales, Colombia, September 2016 to February 2017.
| Parents | Unvaccinated |
Vaccinated |
Total |
|||
|---|---|---|---|---|---|---|
| N = 13 | % | N = 45 | % | N = 58 | % | |
| Gender | ||||||
| Female | 11 | 21,2% | 41 | 78,8% | 52 | 89,7 |
| Male | 2 | 33,3% | 4 | 66,7% | 6 | 10,3 |
| Age | ||||||
| 29–50 | 10 | 20,8% | 38 | 79,2% | 48 | 82,8 |
| 51+ | 3 | 33,3% | 6 | 66,7% | 9 | 15,5 |
| School socio-economic levela | ||||||
| Low | 2 | 9,1% | 20 | 90,9% | 20 | 40,8 |
| Middle | 7 | 35,0% | 13 | 65,0% | 17 | 34,7 |
| High | 4 | 25,0% | 12 | 75,0% | 12 | 24,5 |
| Education level | ||||||
| No education/complete/some elementary school | 2 | 14,3% | 12 | 85,7% | 6 | 10,3 |
| Complete/some secondary school | 4 | 23,5% | 13 | 76,5% | 25 | 43,1 |
| Technician | 5 | 31,3% | 11 | 68,8% | 16 | 27,6 |
| Professional or higher | 2 | 18,2% | 9 | 81,8% | 11 | 19 |
| Marital status | ||||||
| Married/cohabiting | 11 | 28,2% | 28 | 71,8% | 24 | 41,4 |
| Single | 1 | 10,0% | 9 | 90,0% | 10 | 17,2 |
| Divorced/widowed | 1 | 14,3% | 6 | 85,7% | 22 | 37,9 |
| Missing data | 0 | 0,0% | 2 | 100,0% | 2 | 3,4 |
| Occupation | ||||||
| Employed | 6 | 27,3% | 16 | 72,7% | 25 | 43,1 |
| Self-employed | 3 | 33,3% | 6 | 66,7% | 6 | 10,3 |
| Housewife | 4 | 19,0% | 17 | 81,0% | 21 | 36,2 |
| Missing data | 0 | 0,0% | 6 | 100,0% | 6 | 10,3 |
This level corresponds to the socioeconomic level of the neighborhood in which the school is located. In Colombia there are six stratums to classify households according to their physical characteristics and its surroundings. Stratum 1 and 2 correspond to low socioeconomic levels, 3 and 4 to middle, 5 and 6 to high. National Administrative Department of Statistics (DANE Spanish for Departamento Administrativo Nacional de Estadistica).
3.1. Cervical cancer awareness
Table 3 shows the textual quotations from participants about their perceptions of cervical cancer. Most parents, especially those of middle and upper socioeconomic level had some degree of awareness about the disease (30 of 58 parents). Most parents believed cervical cancer is caused by sexual intercourse, the man (a transmitter) and “asleep condition” that becomes “awakened” in some people, others referred it to “fate” and very few mentioned HPV (11 of 58 parents). In general, parents considered the Pap smear and use of condoms as important prevention measures for cervical cancer while the vaccine was important only for very few parents of vaccinated girls of high socioeconomic status. Female parents also acknowledged that they were susceptible to developing cervical cancer as well as their daughters. In regards to the consequences of having this cancer, most parents mentioned death and hysterectomy.
Table 3.
Participant's perceptions of cervical cancer.
| Theme | Topic | Illustrative quotations |
|---|---|---|
| Awareness | Aware nP = 30/58 nG = 16/49 |
A: I have heard that it is very dangerous, that sometimes it can lead to death (Vaccinated girl, low socioeconomic level) |
| Unaware nP = 2/58 nG = 12/49 |
J: it may be something that appears here, like balls [she touches her neck] (Unvaccinated girl, middle socioeconomic level) | |
| Causes | Sex nP = 23/58 nG = 6/49 |
N: because sexual intercourse, when you don't use protection (Mother of unvaccinated girl, low socioeconomic level) |
| It awakes nP = 14/58 nG = 0 |
AM: We all have cancer but it has not awaken yet. (Mother of vaccinated girl, low socioeconomic level) | |
| Men nP = 11/58 nG = 1/49 |
A: It is a virus that is transmitted by men, during sexual intercourse (Mother of vaccinated girl, high socioeconomic level). | |
| HPV nP = 11/58 nG = 0/49 |
T: One of the causes of this cancer is the human papilloma virus (Mother of vaccinated girl, high socioeconomic level) | |
| Fate nP = 6/58 nG = 1/49 |
M: Well, I imagine that if it get me it would be because it is my fate, because it had to be that way (Mother of unvaccinated girl, middle socioeconomic level) | |
| Doesn't know nP = 1/58 nG = 23/49 |
M: I do not really know what is the cause of that (Mother of unvaccinated girl, middle socioeconomic level) | |
| Consequences | Hysterectomy nP = 7/58 nG = 1/49 |
L: Well, from the people I have met, some have had a total hysterectomy and others have died. (Mother of vaccinated girl, middle socioeconomic level) |
| Death nP = 4/58 nG = 1/49 |
A: Sometimes it can cause death. (Vaccinated girl, low socioeconomic level) | |
| Prevention | Pap smear nP = 12/58 nG = 0/49 |
M: Well, I have heard that abnormalities can start to be seen during the pap smear and the colposcopy (Mother of vaccinated girl, middle socioeconomic level) |
| Condoms nP = 4/58 nG = 1/49 |
C: Protecting yourself, because if it is due to sexual intercourse, protecting yourself, with condoms (Mother of vaccinated girl, high socioeconomic level) | |
| Vaccine nP = 4/58 nG = 5/49 |
G: That's why you get the human papilloma vaccine, so you do not get that type of cancer (Vaccinated girl, high socioeconomic level) | |
| Education nP = 3/58 nG = 0/49 |
S: I think it is impossible to prevent cancer. I mean, we are all at risk of getting it. The vaccine I do not know, to be honest that's why I had my eldest daughter vaccinated but not my youngest, because today I don't agree with the vaccine, I agree more with teaching my daughters the care and respect for the body […] I see prevention more on the side of teaching them to love and respect their body than to have the vaccine (Mother of vaccinated girl, high socioeconomic level) | |
| Does not know nP = 1/58 nG = 10/49 |
I do not know that. Cancer is a disease that according to what I have heard, seen and read does not have a cure (Mother of vaccinated girl, middle socioeconomic level) | |
| Self-susceptibility | Present nP = 41/58 nG = 10/49 |
L: Well I have heard that it develops in women only. (Mother of vaccinated girl, middle socioeconomic level) |
| Absent nP = 3/58 nG = 0/49 |
P: Well, I do not think so, because… my sexual partner is only my husband. I have never had sexual intercourse with any other person except with my husband. (Mother of unvaccinated girl, high socioeconomic level) | |
| Does not know nP = 0/58 nG = 6/49 |
X: I do not know because I really do not know why the infection occurs. (Vaccinated girl, middle socioeconomic level) | |
| Daughter's susceptibility | Present nP = 43/58 nG = N/A |
L: I think so because we as women, we all have that risk anyway. (Mother of unvaccinated girl, middle socioeconomic level) |
| Absent nP = 2/58 nG = N/A |
L:N to my daughter anymore because I had her vaccinated (Mother of vaccinated girl, low socioeconomic level) |
nP: number of parents.
nG: number of girls.
Vaccinated girls showed more awareness of cervical cancer than those who were unvaccinated. In general girls were unaware of the causes of disease. They also mentioned death, hysterectomy and sterility as consequences of having this cancer. Regardless of vaccination status, vaccination was noted as the main form of cervical cancer protection among girls of high socioeconomic status, followed by Pap smear, condoms and healthy eating. In contrast, girls of lower socioeconomic status mentioned hygiene. The girls had very low awareness about their susceptibility to cervical cancer and vaccine as a prevention strategy. Although girls of middle and upper socioeconomic level had a greater perception of their susceptibility, this perception was generalized to all types of cancers, not only to cervical cancer.
3.2. Awareness of human papillomavirus
Table 4 presents the perspectives of HPV infection among participants. Parents and girls from middle and upper socioeconomic level showed some awareness about HPV, while those from lower socioeconomic level and even among those who received the vaccination were unaware. Parents attributed the infection to sexual intercourse and men as transmitters. The girls only attributed HPV infection to sexual intercourse. Parents named warts and cervical cancer as consequences of infection; whereas girls, generally did not know the consequence of infection, although some mentioned warts, serious infections and death. In relation to the prevention of HPV infection, parents reported condom use more frequently, while Pap smear was reported more frequently only in parents of unvaccinated girls. Among parents of middle and upper socioeconomic level, they mentioned education, vaccination, monogamy, choice of sexual partner and sexual abstinence as means of prevention. In contrast, girls mentioned condom and vaccines as means of prevention, particularly among those who have been vaccinated. Susceptibility to HPV infection was more frequent in parents of vaccinated girls of high socioeconomic level. For girls, the perception of susceptibility to infection was less frequent than their susceptibility to cervical cancer.
Table 4.
Participant's perceptions of HPV.
| Category | Topic | Illustrative quotations |
|---|---|---|
| Awareness | Aware nP = 33/58 nG = 41/49 |
C: It is a virus, something that can be transmitted, but it is also something that women have inside of them and it is something you can get by sexual transmission (Vaccinated girl, high socioeconomic level) |
| Unaware nP = 1/58 nG = 3/49 |
A: The truth is that we do not know, because that came out recently, it came out a short time ago. (Mother of vaccinated girl, low socioeconomic level) | |
| Causes | Sex nP = 21/58 nG = 21/49 |
C: Well, everything happens through sexual intercourse and that's all I know (Vaccinated girl, middle socioeconomic level) |
| Men nP = 14/58 nG = 3/49 |
N: it is a disease that is transmitted by men (Mother of vaccinated girl, low socioeconomic level) | |
| Does not know nP = 2/58 nG = 3/49 |
A: I do not what to say about it (Mother of unvaccinated girl, middle socioeconomic level) | |
| Consequences | Warts nP = 5/58 nG = 2/49 |
L: You can get ugly warts in the whole body and in order to avoid them you need to get an injection (vaccinated girl middle socioeconomic level) |
| CCU nP = 5/58 nG = 0/49 |
N: The disease that produces the human papillomavirus is cervical cancer. (Mother of vaccinated girl, low socioeconomic level) | |
| Does not know nP = 9/58 nG = 11/49 |
A: One is incline to listen, yes, to listen to what it is said, but one does not pay much attention to that (Mother of vaccinated girl, low socioeconomic level) | |
| Prevention | Condoms nP = 11/58 nG = 7/49 |
Y: Well, to use protection, the condom and all that (Vaccinated girl, middle socioeconomic level) |
| Pap smear nP = 5/58 nG = 0/49 |
S: With often Pap smears and colposcopy, like every 6 months or so, to get studied that the virus is not going to progress (Mother of unvaccinated girl, high socioeconomic level) | |
| Vaccine nP = 5/58 nG = 8/49 |
Y: It is assumed that the vaccine was for that, to avoid the human papillomavirus (Vaccinated girl, middle socioeconomic level) | |
| Education nP = 4/58 nG = 0/49 |
L: There is one thing that is fundamental and that is that I can prevent at educational level many situations that could put my daughter at risk (Father of unvaccinated girl, high socioeconomic level) | |
| Does not know nP = 1/58 nG = 8/49 |
M: I do not know, I have never investigated about that (Unvaccinated girl, high socioeconomic level) | |
| Self-susceptibility | Present nP = 14/58 nG = 8/49 |
C: Let's say that if one has intercourse and does not use protection and besides does not have the vaccine, one can also get infected (Unvaccinated girl, high socioeconomic level) |
| Does not know nP = 1/58 nG = 2/49 |
Y: Maybe yes, maybe not (Vaccinated girl, high socioeconomic level) | |
| Daughter's susceptibility | Present nP = 8/58 nG = N/A |
L: I think they can get the disease after having sex (Mother of unvaccinated girl, middle socioeconomic level) |
| Absent nP = 1/58 nG = N/A |
L: We have taught her that she must wait for her husband, she should keep herself for her husband and I hope she does not get that [HPV] (Mother of unvaccinated girl, middle socioeconomic level) |
nP: number of parents.
nG: number of girls.
3.3. Perspectives, facilitators and barriers to HPV vaccination
Table 5 presents the participant's perspectives, facilitators and barriers of HPV vaccine uptake. The majority of parents reported having some degree of awareness about the HPV vaccine. When asked about the positive and negative consequence of vaccination, parents reported that the positive consequences were prevention of cervical cancer and the perception that it would reduce the effects of the infection. Whereas, for negative consequences of the vaccine, they referred to the event that occurred in Carmen de Bolívar.
Table 5.
Participant's perceptions, barriers and facilitators of HPV vaccines.
| Category | Topic | Illustrative quotations |
|---|---|---|
| Awareness | Aware nP = 34/58 nG = 26 |
M: girls are being vaccinated because women are getting a lot of cancer. So the girls are being prevented from now on. (Mother of vaccinated girl, low socioeconomic level) |
| Unnaware nP = 9/58 nG = 5/49 |
L: I think I have listened about it but I do not have it very present. I do not know what it is (Mother of vaccinated girl, middle socioeconomic level) | |
| Positive Information nP = 21/58 nG = 26/49 |
L: It is said that the vaccine helps to prevent cancer. (Mother of vaccinated girl, low socioeconomic level) | |
| Negative information nP = 10/58 nG = 9/49 |
C: It has been heard that there are many studies, not here but in Europe and abroad, about the effects that the vaccine has caused at central nervous system level. I do not know. I have read studies but I do not know if they are true. If it is actually because of the virus or it is a coincidence that these people have generated this type of diseases or affectations in their central nervous system. I do not know if it is true but it raises doubt in me. (Mother of vaccinated girl, high socioeconomic level) | |
| Facilitators | General prevention nP = 22/58 nG = 16/49 |
F: To be able to avoid diseases when I get older (Vaccinated girl, low socioeconomic level) |
| HPV prevention nP = 5/58 nG = 1/49 |
M: Let's hope it's like this, that they will not get the human papillomavirus (Mother of vaccinated girl, middle socioeconomic level) | |
| Cervical cancer prevention nP = 1/58 nG = 4/49 |
M: Well, I had them [my daughters] vaccinated to prevent that they will get that papillomavirus and thus prevent cancer in a certain way. (Mother of vaccinated girl, middle socioeconomic level) | |
| Barriers | Carmen de Bolivar nP = 20/58 nG = 18/49 |
L: The news said that the girls had been vaccinated against the human papilloma virus and that from there they showed symptoms of fainting in several schools, they fainted very often and presented pain… Their joints hurt, some of them couldn't almost move, then I was frightened and I preferred not to vaccinate my daughter (Mother of unvaccinated girl, middle socioeconomic level) |
| Fear of needles nP = 3/58 nG = 12/49 |
L: I did not want to, not because everything that had happened in the news, but because I was afraid of the needle. (Unvaccinated girl, middle socioeconomic level) | |
| Fear of adverse events nP = 3/58 nG = 6/49 |
C: Well, my mom said that it could be dangerous, because there had been many cases of girls who suffered very serious side effects or that the vaccine caused them harm. That is why they did not put it on me. (Unvaccinated girl, high socioeconomic level) | |
| Decision of vaccinate | Both parents nP = 12/58 nG = 10/49 |
And: the truth is we both decide it but we … I mean after hearing so much in the news, at least I was aware that it was something very good for her (Mother of vaccinated girl, middle socioeconomic level) |
| Only mother nP = 9/58 nG = 18/49 |
L: Well, my husband told me that if it was a vaccine then it was important, but then, with all that came out, then he said I should see what to do. Then I got scared. (Mother of unvaccinated girl, middle socioeconomic level) | |
| Girl nP = 7/58 nG = 14/49 |
F: my mom asked me if I wanted to, if I said yes then yes or if I did not, then no. Then I said yes of course. (Vaccinated girl, low socioeconomic level) | |
| Explanation about the vaccine | Present explanation nP = 9/58 nG = 15/49 |
Here at the school we were also informed, they gave us little information about what the vaccine was about (Mother of vaccinated girl, middle socioeconomic level) |
| Absent or insufficient nP = 27/58 nG = 35/49 |
J: they did not give us any explanation, they just gave us the authorization form, so the parents could authorize the vaccination and that was it (Unvaccinated girl, middle socioeconomic level) | |
| Vaccine authorization | Present nP = 28/58 nG = 23/49 |
M: We were sent some forms to our home to authorize the vaccination at school (Mother of vaccinated girl, middle socioeconomic level) |
| Absent nP = 9/58 nG = 7/49 |
V: they [the vaccinators] arrived from nowhere and called those in the classroom to put on the vaccine and they did not send any form home (Vaccinated girl, low socioeconomic level) | |
| Sources of information | School nP = 2/58 nG = 12/49 |
A: the school called us, they gave us a talk, the doctor from the school talked to us. (Mother of unvaccinated girl, high socioeconomic level) |
| Health provider nP = 11/58 nG = 3/49 |
S: When I took her for the health checkup [programa de crecimeinto y desarrollo] they started to talk us about the vaccine, then I took her to the Health Provider and I had her vaccinated with the first dose (Mother of vaccinated girl, high socioeconomic level) | |
| Vaccination campaign nP = 8/58 nG = 8/49 |
S: Just at the time that the campaign was already launched we were informed. (Mother of vaccinated girl, middle socioeconomic level) | |
| TV nP = 9/58 nG = 19/49 |
D: there was a time when it was even on TV commercials. I think there was a commercial on television that said you had to vaccinate the girls (Mother of vaccinated girl, high socioeconomic level) | |
| Internet nP = 2/58 nG = 2/49 |
Q: the truth is that it was on Facebook where I read about it. (Mother of unvaccinated girl, middle socioeconomic level) | |
| Did not received information from school nP = 3/58 nG = 10/49 |
A: Well, my daughter at school was not told about the vaccine. What she barely saw, and even scared her, were the news, because she said: “Oh mom, you had me vaccinated and look what is happening to those girls in Cartagena or in Bolívar”. But in the school my girl was never told about the papillomavirus vaccine. (Mother of vaccinated girl, middle socioeconomic level) | |
| Did not received from health provider nP = 6/58 nG = 12/49 |
L: No, the doctors where I have attended have not spoken to me about that. (Mother of unvaccinated girl, middle socioeconomic level) |
nP: number of parents.
nG: number of girls.
For facilitators of HPV vaccination parents generally had some degree of awareness about it as a general prevention measure and mentioned most frequently that it was for the prevention of HPV and cervical cancer. Among those of middle and upper socioeconomic level, it was observed that there was a greater number of facilitators than those of lower socioeconomic level. For girls, general prevention and cervical cancer prevention were the most frequent facilitators.
The main barriers for parents not to vaccinate their daughters or complete the 3-dose course of the vaccine were the fear of adverse effects and the event of Carmen de Bolívar. For the girls, the fear of needles and the Carmen de Bolívar event were the main barriers to start or complete vaccination. Other barriers to HPV vaccination included the belief that the prevention of STIs should be done through education and not with vaccines among parents of unvaccinated girls of middle and high socioeconomic level.
Parents and girls had similar answers about the decision to vaccinate. The decision was generally taken by both parents. In some cases, the decision was mainly made by the mother and in a few instances, the decision was taken by the father or the girl herself.
Most parents of vaccinated girls affirmed that they were provided with some type of information prior to their daughter's vaccination but believed the information was inadequate. This affirmation was also made by parents of unvaccinated girls of middle socioeconomic level. Girls reported similar responses about the lack of information provided from the school or their healthcare provider. A significant number of parents reported that the information provided by the schools and the healthcare provider was limited. They mentioned that the main sources of information were from the vaccination campaign, some doctors, the media and from the girl herself. Only parents of unvaccinated girls from middle and high socioeconomic level mentioned having obtained information from the internet. For girls, the main sources of information were from the vaccination campaign, television and parents.
4. Discussion
Coverage of HPV vaccine has been in decline since its implementation in Colombia in 2012. This qualitative study was able to identify perceptions of parents and girls about cervical cancer, HPV and vaccination from a Colombian municipality. The results were largely consistent with those reported in the literature (Camano-Puig and Sanchis, 2015; Das et al., 2010; Rambout et al., 2014; Torrado-Arenas et al., 2017; Wiesner et al., 2010; Wigle et al., 2013; Winkler et al., 2008).
In general, awareness about cervical cancer in this population was limited. It was not surprising that vaccinated girls had more information about HPV e, whereas very few parents mentioned HPV as the cause of cervical cancer. In general, girls were unclear about the relationship between HPV and cervical cancer. These observations were similar to those reported previously from another Colombian municipality, Rivera, where there were also low levels of knowledge about cervical cancer and HPV transmission and prevention (Torrado-Arenas et al., 2017). Similar to other studies, there is a lack of information provided on HPV as a cervical cancer precursor within the Latino community of low- and middle–income, which can in effect be a barrier to vaccination uptake (Arrossi et al., 2012; Bair et al., 2008; Osis et al., 2014; Wigle et al., 2013; Winkler et al., 2008).
In the study conducted prior to the introduction of the HPV vaccine in Colombia (Wiesner et al., 2010), it was recommended “to promote the vaccine to prevent cervical cancer rather than to prevent a sexually transmitted infection”. As a result, the campaign carried out in the first year of vaccine implementation in August 2012 had the following message of “Que vivan las mujeres, que viva la vida” (Long live women, long live life). This message emphasized that the vaccine prevents against HPV and consequentially cervical cancer. However, unexpectedly, both parents and girls did not mention that the vaccine against HPV infections is also the main form of prevention of cervical cancer in our study. This result indicates that the campaign message did not reach the population before vaccination.
Both female parents and girls perceived themselves to be susceptible to cervical cancer but less susceptible to HPV, which suggests that the link between HPV as an etiological factor of cervical cancer is unclear. According to the HBM (Glanz et al., 2008) perceived susceptibility and perceived severity are critical to increase the likelihood of undertaking behavior to prevent disease which have been associated with vaccine acceptability (Reina and Muñoz, 2014; Winkler et al., 2008).
Our study suggests that participants eligible for vaccination after the Carmen de Bolivar event generated a negative impact on the acceptability of the vaccine, especially among parents, who are ultimately the decision-maker for their daughters. Our findings are consistent with several studies (Camano-Puig and Sanchis, 2015; Torrado-Arenas et al., 2017), which support the fact that the population still retained the belief of the supposed adverse effects of vaccination even several years after the occurrence of the Carmen de Bolivar event,
Our study did not identify religious beliefs, promoting early initiation of sex or general opposition to vaccines as barriers to vaccination. However, there was a clear fear of possible side effects. These fears were manifested particularly among parents who did not want to harm their daughters for the decision they made to vaccinate them.
Despite the challenges of running a routine school-based vaccination program (Cheruvu et al., 2017; Clements, 2007; Gallagher et al., 2017), participants consider schools to be ideal places to achieve high uptake of the HPV vaccine given that adolescents are generally healthy and do not need to see their doctor regularly as compared to younger age groups (e.g. under 5 years). In fact, there have been difficulties in achieving high vaccine coverage in countries where the vaccine is offered opportunistically (Choi et al., 2013; Skinner and Cooper Robbins, 2010; Turiho et al., 2014). They would only be offered the vaccine if they went to see their doctor. This could explain why medical advice was not mentioned as a frequent facilitator of vaccination for participants in our study.
Following the mass psychogenic illness event in El Carmen de Bolivar, it is imperative that governmental and local authorities act promptly and appropriately to contain widespread misinformation and to minimize any long-term impact such an event can have on public health prevention of cervical cancer. It is also recommended not to suspend active recommendation of the HPV vaccine because this would increase uncertainty and loss of confidence in the vaccine (Simas et al., 2018). To encourage informed decision making about HPV vaccination, it is important that doctors become educators for girls eligible for vaccination and their parents. This would support the parents and girls in weighing the benefits and risks to vaccination and would be an opportunity for “deciding with the patients and not for the patients” (Grillo-Ardila, 2014).
Similar declines of coverage in HPV vaccination have occurred following similar events around the world, which have been resolved successfully. For example, in Ireland, the government created an initiative called the HPV Vaccination Alliance, which united 35 organizations across the country to encourage all key stakeholders to actively promote the vaccine through a wide range of cross-sectoral strategies, such as mass media advertisement and differential training programs (Corcoran et al., 2018).
The study has several limitations. Our study included a limited number of girls who were unvaccinated and it was not possible to interview girls not attending school. Therefore, section bias is possible, and results may not be generalizable to non-school attending girls. For some selected schools, a convenience sample was used which could also potentially limit generalizability. Also, social desirability might have influence the participant responses. Although, interviewer bias was minimized by the use of professionals with experience in qualitative data collection, the possibility cannot be ignored.
Our findings suggest the need for a comprehensive education program including mass media information to provide adequate information on how the HPV vaccine can prevent cervical cancer to help girls and parents make an informed decision about vaccination.
The following is the supplementary data related to this article.
Focus group and interview question guide. Manizales, Colombia, September 2016 to February 2017.
Declaration of competing interest
The authors declare that there are no conflicts of interests.
Acknowledgments
This study was financed by COLCIENCIAS project 886-2015 and Universidad de Antioquia.
The authors wish to thank all the schools and parents who participated on the study.
Contributor Information
Verónica Cordoba-Sanchez, Email: veronica.cordoba@udea.edu.co.
Olga Lucía Tovar-Aguirre, Email: otovar@ucm.edu.co.
Sandra Franco, Email: sandra.franco_i@ucaldas.edu.co.
Nelson Enrique Arias Ortiz, Email: nelson.arias@ucaldas.edu.co.
Karly Louie, Email: karly.louie@gmail.com.
Gloria Ines Sanchez, Email: gloria.sanchez@udea.edu.co.
Isabel C. Garces-Palacio, Email: icristina.garces@udea.edu.co.
References
- Alder S., Gustafsson S., Perinetti C., Mints M., Sundström K., Andersson S. Mothers' acceptance of human papillomavirus (HPV) vaccination for daughters in a country with a high prevalence of HPV. Oncol. Rep. 2015;33:2521–2528. doi: 10.3892/or.2015.3817. [DOI] [PubMed] [Google Scholar]
- Arrossi S., Maceira V., Paolino M., Sankaranarayanan R. Acceptability and uptake of HPV vaccine in Argentina before its inclusion in the immunization program: a population-based survey. Vaccine. 2012;30:2467–2474. doi: 10.1016/j.vaccine.2012.01.032. [DOI] [PubMed] [Google Scholar]
- Bair R.M., Mays R.M., Sturm L.A., Zimet G.D. Acceptability of the human papillomavirus vaccine among Latina mothers. J. Pediatr. Adolesc. Gynecol. 2008;21:329–334. doi: 10.1016/j.jpag.2008.02.007. [DOI] [PubMed] [Google Scholar]
- Bowyer H.L., Forster A.S., Marlow L.A.V., Waller J. Predicting human papillomavirus vaccination behaviour among adolescent girls in England: results from a prospective survey. J. Fam. Plann. Reprod. Health Care. 2014;40:14–22. doi: 10.1136/jfprhc-2013-100583. [DOI] [PubMed] [Google Scholar]
- Camano-Puig R., Sanchis M.M. Vacuna contra el virus del papiloma humano en adolescentes: Análisis mediante grupos focales. Rev. Salud Pública. 2015;16:647–659. doi: 10.15446/rsap.v16n5.41021. [DOI] [PubMed] [Google Scholar]
- Cheruvu V.K., Bhatta M.P., Drinkard L.N. Factors associated with parental reasons for “no-intent” to vaccinate female adolescents with human papillomavirus vaccine: National Immunization Survey - Teen 2008–2012. BMC Pediatr. 2017;17 doi: 10.1186/s12887-017-0804-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Choi H.C., Leung G.M., Woo P.P., Jit M., Wu J.T. Acceptability and uptake of female adolescent HPV vaccination in Hong Kong: a survey of mothers and adolescents. Vaccine. 2013;32:78–84. doi: 10.1016/j.vaccine.2013.10.068. [DOI] [PubMed] [Google Scholar]
- Clements C.J. Gardasil and mass psychogenic illness. Aust. N. Z. J. Public Health. 2007;31:387. doi: 10.1111/j.1753-6405.2007.00101.x. [DOI] [PubMed] [Google Scholar]
- Corcoran B., Clarke A., Barrett T. Rapid response to HPV vaccination crisis in Ireland. Lancet. 2018;391(10135):2103. doi: 10.1016/S0140-6736(18)30854-7. [DOI] [PubMed] [Google Scholar]
- Cordeiro V.G., Pérez C.S., Iñarrea F.A., Simón V.D., Reboredo R.C., Couceiro N.E., Ramón y Cajal C. ¿Por qué no se vacunan nuestras pacientes?: Motivos por los que las pacientes fuera de los programas de vacunación sistemática, con infección por el virus del Papiloma Humano, deciden decir no a la vacuna. Rev. Chil. Obstet. Ginecol. 2014;79:390–395. [Google Scholar]
- Das A., Madhwapathi V., Davies P., Brown G., Dearnley E., Spencer A., Williams H. Knowledge and acceptability of the HPV vaccine by school children and their parents in Birmingham. Vaccine. 2010;28:1440–1446. doi: 10.1016/j.vaccine.2009.11.041. [DOI] [PubMed] [Google Scholar]
- Firenze A., Marsala M.G.L., Bonanno V., Maranto M., Ferrara C., Giovannelli L., Restivo V. Facilitators and barriers HPV unvaccinated girls after 5 years of program implementation. Hum. Vaccin. Immunother. 2015;11:240. doi: 10.4161/hv.36158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Gallagher K.E., Howard N., Kabakama S., Mounier-Jack S., Griffiths U.K., Feletto M., Burchett H.E.D., LaMontagne D.S., Watson-Jones D. Lessons learnt from human papillomavirus (HPV) vaccination in 45 low- and middle-income countries. PLoS One. 2017;12 doi: 10.1371/journal.pone.0177773. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glanz K., Rimer B.K., Viswanath K. Wiley; 2008. Health Behavior and Health Education: Theory, Research, and Practice. [Google Scholar]
- Godoy Verdugo M.K., Zonana Nacach A., Anzaldo Campos M.C. Acceptance of the vaccine against human papilloma virus from mothers to daughters aged 9 to 13 years old. Ginecol. Obs. Mex. 2013;81:645–651. [PubMed] [Google Scholar]
- Grillo-Ardila C.F. El Carmen de Bolívar: una lección que no debemos olvidar. Rev. Colomb. Obstet. Ginecol. 2014;65:198. doi: 10.18597/rcog.46. [DOI] [Google Scholar]
- Hendry M., Lewis R., Clements A., Damery S., Wilkinson C. “HPV? Never heard of it!”: a systematic review of girls' and parents' information needs, views and preferences about human papillomavirus vaccination. Vaccine. 2013;31:5152–5167. doi: 10.1016/j.vaccine.2013.08.091. [DOI] [PubMed] [Google Scholar]
- Martinez M., Estevez A., Quijada H., Walteros D., Tolosa N., Paredes A., Alvarez C., Armenta A., Osorio L., Castillo O., De la Hoz F. Brote de evento de etiología desconocida en el municipio de El Carmen de Bolívar, Bolívar, 2014. Inf. Quinc. Epidemiol. Nac. 2015:41–77. [Google Scholar]
- Osis M.J., Duarte G.A., Sousa M.H. SUS users' knowledge of and attitude to HPV virus and vaccines available in Brazil. Rev. Saude Publica. 2014;48:123–133. doi: 10.1590/S0034-8910.2014048005026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rambout L., Tashkandi M., Hopkins L., Tricco A.C. Self-reported barriers and facilitators to preventive human papillomavirus vaccination among adolescent girls and young women: a systematic review. Prev. Med. 2014;58:22–32. doi: 10.1016/j.ypmed.2013.10.009. [DOI] [PubMed] [Google Scholar]
- Reina J.C., Muñoz N. Vaccine against human Papilloma Virus. Colomb. Med. 2014;45 [PMC free article] [PubMed] [Google Scholar]
- Simas C., Munoz N., Arregoces L., Larson H.J. HPV vaccine confidence and cases of mass psychogenic illness following immunization in Carmen de Bolivar, Colombia. Hum. Vaccin. Immunother. 2018:1–4. doi: 10.1080/21645515.2018.1511667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Skinner S.R., Cooper Robbins S.C. Voluntary school-based human papillomavirus vaccination: an efficient and acceptable model for achieving high vaccine coverage in adolescents. J. Adolesc. Health. 2010;47:215–218. doi: 10.1016/j.jadohealth.2010.07.002. [DOI] [PubMed] [Google Scholar]
- Torrado-Arenas D.M., Álvarez-Pabón Y., González-Castañeda J.L., Rivera-Contreras O.E., Sosa-Vesga C.D., González B.X., Sepúlveda-Agudelo J. Conocimientos sobre el virus del papiloma humano y su vacuna en padres de familia de Rivera, Huila en el 2015. Rev. MÉDICAS UIS. 2017;30:13–19. [Google Scholar]
- Turiho A.K., Okello E.S., Muhwezi W.W., Harvey S., Byakika-Kibwika P., Meya D., Katahoire A.R. Effect of school-based human papillomavirus (hpv) vaccination on adolescent girls' knowledge and acceptability of the HPV vaccine in Ibanda District in Uganda. Afr. J. Reprod. Health. 2014;18:45–53. [PMC free article] [PubMed] [Google Scholar]
- Wiesner C., Piñeros M., Trujillo L.M., Cortés C., Ardila J. Aceptabilidad de la vacuna contra el Virus Papiloma Humano en padres de adolescentes, en Colombia. Rev. Salud Pública. 2010;12:961–973. [PubMed] [Google Scholar]
- Wigle J., Coast E., Watson-Jones D. Human papillomavirus (HPV) vaccine implementation in low and middle-income countries (LMICs): health system experiences and prospects. Vaccine. 2013;31:3811–3817. doi: 10.1016/j.vaccine.2013.06.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Winkler J.L., Wittet S., Bartolini R.M., Creed-Kanashiro H.M., Lazcano-Ponce E., Lewis-Bell K., Lewis M.J., Penny M.E. Determinants of human papillomavirus vaccine acceptability in Latin America and the Caribbean. Vaccine. 2008;26:73–79. doi: 10.1016/j.vaccine.2008.05.027. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Focus group and interview question guide. Manizales, Colombia, September 2016 to February 2017.