Abstract
Improving human papillomavirus (HPV) vaccination rates is a public health priority and a crucial cancer prevention goal. We designed a survey to estimate HPV vaccination coverage and understand factors associated with HPV vaccination among American Indian adolescents aged 9 to 17 years in Cherokee Nation, United States. The final survey contains 37 questions across 10 content areas, including HPV vaccination awareness, initiation, reasons, recommendations, and beliefs. This process paper provides an overview of the survey development. We focus on the collaborative process of a tribal–academic partnership and discuss methodological decisions regarding survey sampling, measures, testing, and administration.
Keywords: American Indian, HPV vaccination, indigenous data sovereignty, survey design, questionnaire
1. Introduction
American Indian and Alaska Native women bear a disproportionate burden of cervical and other human papillomavirus (HPV)-associated cancers nationally and in Oklahoma [1,2]. From 2013 to 2017, the incidence of HPV-associated cancers nationally was 1.2 times higher in American Indian and Alaska Native (15.9 per 100,000) than non-Hispanic White (13.7 per 100,000) women [1]. In addition, during the same period, regional variations in HPV-associated cancers among American Indian and Alaska Native populations have been observed, with the highest incidence of all HPV-associated cancers among males (14.9 per 100,000) and cervical cancer among females (13.8 per 100,000) reported in the Southern Plains (Kansas, Oklahoma, and Texas) [1]. In Oklahoma, American Indian and Alaska Native women had the highest age-adjusted incidence rate for cervical cancer (14.8 per 100,000 women) among all racial and ethnic groups. This incidence rate was significantly higher than the rate in White (rate ratio [RR]: 1.6; 95% confidence interval [CI]: 1.5, 1.8) and African American (RR: 1.5; 95% CI: 1.3, 1.8) women [2].
Vaccines to prevent HPV-associated cancers have been available and recommended for use in the United States (US) since their introduction in 2006 [3]. However, data on HPV vaccination coverage for individual American Indian and Alaska Native tribes are not readily available. In addition, patterns and policies for vaccination among American Indian and Alaska Native populations differ. For example, in a survey of HPV vaccination policies among American Indian tribes in Washington State, 25% of the facilities serving tribal members had a clinic-only policy, 4% had a tribal-only policy, 18% had both clinic and tribal policies, and 46% had no policy [4]. While tribal policies regarding HPV vaccine administration followed guidelines from the Advisory Committee on Immunization Practices (ACIP), some clinics followed guidelines from the ACIP, Vaccines for Children (VFC) program, or Washington State [4]. Examples of differing clinic policies included providing the HPV vaccine unless the family opted out, developing a standing order protocol to enable nurses to administer the vaccine without a physician’s order, and offering vaccines to 19-to-26-year-olds who may not meet the VFC eligibility criteria [4]. Similar to differences in HPV-associated cancer burden, these variations in policies likely result in regional- and tribal-dependent differences in HPV vaccination coverage. Furthermore, the lack of tribal-specific or missing estimates for HPV vaccination likely hinders the opportunity to monitor trends and improve coverage in their tribal communities.
Compared with other racial and ethnic groups in the US, surveys assessing HPV vaccination coverage and factors among American Indian and Alaska Native people are limited. Among the surveys focused on American Indian and Alaska Native populations, most examined HPV vaccination policies and practices among health care professionals [4,5,6,7]; some examined HPV vaccination decision-making in American Indian college students [8,9,10]; and few examined attitudes on HPV vaccination among American Indian women [11] and Alaska Native parents [12]. Other survey studies including American Indian and Alaska Native communities have investigated HPV vaccine-related knowledge [13] and HPV vaccine information sources [14]. As HPV vaccination coverage and factors differ by tribe, and in the absence of any study assessing these factors in Cherokee Nation, we developed a survey to address this research gap.
This article is part of a larger ongoing project on HPV vaccination in Cherokee Nation, which also includes assessing perceptions and attitudes towards HPV vaccination among parents through focus groups. Cherokee Nation Public Health initiated these HPV vaccination quantitative and qualitative analyses projects to inform their public health programs better. The focus of this article is to provide an overview of the design and implementation of an American Indian HPV vaccination survey currently underway in Cherokee Nation. In this process paper, we describe the community partnership, instrument development, testing, sampling, and survey dissemination. We also present, wherever applicable, checklist items from the reporting guidelines of the CONSIDER statement (Consolidated criteria for strengthening reporting of health research involving indigenous peoples) [15].
2. Methods
2.1. Research Team
The research team for this survey project consisted of the Cherokee Nation Comprehensive Cancer (CNCC) program director (MB), Cherokee Nation epidemiologist (AC), Cherokee Nation Public Health employees (SP; MF and KNP), and researchers from the University of Oklahoma Health Sciences Center (OUHSC) who are members of a federally recognized tribe (AJ, SAM, ASA) or who each have experience working with American Indian tribes in Oklahoma (SVG, JEC, AHW, SC, and SFP) (CONSIDER statement #6 and #7). The research team formed a working group in May 2019 and has had bi-weekly project meetings since June 2019. The process, as described below, was iterative with the ultimate goal of improving the health of Cherokee Nation citizens while respecting tribal sovereignty. Further, the epidemiologist (AC) is an early stage investigator at Cherokee Nation interested in reducing health disparities among American Indians, especially among Cherokee citizens. This project aims to facilitate her goal of research independence and increase the evaluation and research capacity at Cherokee Nation that will lead to tribally driven and community-engaged evaluation and research (CONSIDER statement #13). In addition, the process is part of the practice of data decolonization of evaluation and research in indigenous communities [16].
2.2. Survey Aims
Before the survey was developed, the aims of the HPV vaccination in Cherokee Nation study were discussed and refined. The overall study aims to: (1) estimate HPV vaccine initiation, defined as the receipt of at least one dose of the HPV vaccine, among American Indian adolescents aged 9 to 17 years accessing Cherokee Nation Health Services in Oklahoma, and (2) identify factors associated with HPV vaccine acceptance or refusal by parents or guardians of American Indian adolescents aged 9 to 17 years accessing Cherokee Nation Health Services. This study also aims to collect data on HPV vaccine awareness, completion, intentions, and beliefs.
After finalizing the aims, the best approach to achieve the study aims was carefully considered, and following discussions, the research team concluded that a survey would be the most appropriate tool.
2.3. Population
Located in northeastern Oklahoma, Cherokee Nation is the largest federally recognized tribe in the US, with more than 389,000 citizens. Approximately 37% of the tribal population lives on the reservation, spanning 14 counties and more than 7000 square miles in Oklahoma [17]. The Cherokee Nation Health Services serve over 100,000 patients through two inpatient and nine outpatient health facilities throughout the reservation [18].
As parents and guardians play a prominent role in the vaccine uptake and vaccination behaviors of their children [19], the target population for this survey was parents or guardians of American Indian adolescents aged 9 to 17 years who used the Cherokee Nation Health Services from 1 January 2018 to 31 August 2020.
2.4. Sampling and Sample Size
We used electronic health records (EHR) from Cherokee Nation Health Services as the sampling frame. Using EHR as the frame reduced the potential for coverage error—the bias that occurs when the sampling frame does not cover the target population [20]. From 1 January 2018 to 31 August 2020, 15,803 adolescents aged 9 to 17 years were seen at the Cherokee Nation Health Services. Using this frame, a sample size of 2000 was drawn through a simple random sampling without replacement design in SAS (version 9.4; SAS Institute, Cary, NC, USA) [21].
For this survey, the sampling and observational units for this survey were individual parents or guardians of American Indian adolescents aged 9 to 17 years. Based on the HPV vaccination coverage of 70.1% for at least one dose among American Indian and Alaska Native teens nationally [22], the sample size needed for the survey was estimated to be 257, with a ±4% margin of sampling error.
2.5. Survey Measures
We selected survey measures based on how well the measures addressed the research aims and aligned with the priorities of Cherokee Nation Public Health and Cherokee Nation policymakers. To inform the selection and development of survey measures, the research team reviewed the Cherokee Nation Comprehensive Cancer Prevention and Control Plan 2018–2022 [17] and undertook a literature review of existing instruments on HPV vaccination.
Based on the literature review, we identified and examined adult- and adolescent-directed surveys with questions on HPV vaccination, including the National Immunization Survey (NIS)–Teen (2019) [23], National Health Interview Survey (NHIS) (2019) [24], Health Information National Trends Survey (HINTS) (2018) [25], National Survey of Family Growth (NSFG) (2015–2017) [26], National Health and Nutrition Examination Survey (NHANES) (2017–2018) [27], Behavioral Risk Factor Surveillance System (BRFSS) (2019) [28], UNC National Parents Survey (2017) [29], and Carolina HPV Immunization Attitudes and Beliefs Scale (CHIAS) (2007) [30]. In addition, we searched Q-Bank, which provides access to tested questions from over 50 national surveys [31]. These standardized questionnaires have been extensively tested and shown to have valid and reliable items, thereby reducing the potential of measurement error stemming from the instrument. However, before inclusion, we reviewed the question text, response options, reference period, respondents, data collection method, survey coverage, sample description, and any additional information about the question or scale. As a result, we adapted some of these questions for our survey.
2.5.1. HPV Vaccine Awareness
Awareness of the HPV vaccine is a strong predictor of vaccine initiation [32,33]. In this survey, we assessed HPV vaccine awareness by asking, “A vaccine to prevent HPV infection is available. The vaccine is also known as or sometimes called the HPV vaccine, cervical cancer vaccine, Gardasil, or Cervarix. Before this survey, had you ever heard of the HPV vaccine?” Response options for the question included “yes” and “no”.
2.5.2. HPV Vaccine Initiation
One of the main study outcomes, HPV vaccine initiation, was defined as the reported receipt of at least one dose of the vaccine. We estimated initiation through the question: “Has your child ever received an HPV shot or vaccine?” Response options were “yes”, “no”, and “I don’t know”. Estimating HPV vaccine initiation among American Indian adolescents accessing Cherokee Nation Health Services aligns with the strategy outlined in objective 1.3 of the Cherokee Nation Comprehensive Cancer Prevention and Control Plan 2018–2022 to increase HPV vaccination rates [17] (CONSIDER statement #4).
2.5.3. HPV Vaccination Factors
To understand HPV vaccination factors, another primary outcome of interest, we ascertained the main reason for HPV vaccination refusal and acceptance. We assessed HPV vaccination refusal through the following question: “If your child has NOT received any shots of the HPV vaccine, what is the MAIN reason that your child has NOT received the HPV shot or vaccine? (select ONE response only).” Similarly, we assessed HPV vaccination receipt by asking: “If your child has received any shots of the HPV vaccine, what is the MAIN reason that your child DID receive the HPV shot or vaccine? (select ONE response only)”.
2.5.4. HPV Vaccine Recommendation
As provider recommendation is a strong predictor of HPV vaccine initiation [34,35,36], we assessed this measure in our survey by asking the following question: “Has a doctor, nurse, or other health care provider ever recommended that your child receive an HPV shot or vaccine?” If the HPV vaccine was recommended by a provider, we also asked, “At what age did the doctor, nurse, or other health care provider recommend that your child should start receiving the HPV shot or vaccine?”
2.5.5. Adolescent Vaccinations
To compare HPV vaccination coverage with other recommended vaccines for adolescents, we included questions about meningitis and tetanus booster shots in the survey. The question on meningitis vaccination read as: “Has your child ever received a meningitis shot or vaccine, sometimes called Menactra, Menveo, or Menomune?” The question on tetanus booster vaccination read as: “Has your child ever received a tetanus booster shot? This is usually given at 11–12 years of age. The tetanus booster shot is also known as or sometimes called the Td or Tdap shot or vaccine.” Response options for both questions were “yes”, “no”, and “I don’t know”.
2.5.6. HPV Vaccine Beliefs
Participants were presented with statements on HPV vaccine importance, intent, and safety and were asked to rate agreement or disagreement using a five-point Likert scale of “strongly disagree”, “somewhat disagree”, “neither disagree or agree”, “somewhat agree”, and “strongly agree”. For instance, we assessed beliefs on the vaccine’s safety by these two statements: “the HPV vaccine can cause side effects, such as fever and discomfort” and “the HPV vaccine can cause lasting health problems”.
2.5.7. Sociodemographic Variables
We ascertained sociodemographic information about adolescents and their parents or guardians. For American Indian adolescents, we asked questions about their gender, race, ethnicity, and age. For parents or guardians of American Indian adolescents, we asked questions about their gender, race, ethnicity, age, relationship to the adolescent, marital status, and highest level of education.
2.5.8. COVID-19 and Vaccination
We also assessed COVID-19 vaccination intent among parents and adolescents by asking the following two questions: “How likely are you to get an approved COVID-19 vaccine when it becomes available?” and “How likely are you to get an approved COVID-19 vaccine for your child when it becomes available?” To ascertain whether the pandemic has caused any difficulty in getting the HPV vaccine, we also asked, “Has the COVID-19 pandemic made it difficult to get the HPV vaccine for your child?” These questions were selected and included in our survey as part of the Rapid Acceleration of Diagnostics-Underserved Populations (RADxSM-UP) Native American Research Centers for Health (NARCH) program for Cherokee Nation (Grant Number: 3S06GM127983-03S1).
2.6. Questionnaire Check
To ensure clarity and reduce the potential for measurement error, we avoided questions that were leading, loaded, double-barreled, negatively worded, and vague. We also avoided any acronyms or abbreviations. The question on HPV vaccination intent was asked twice in different sections of the survey to check for consistency of participant responses. We also assessed the readability of the survey using the Flesch Reading Ease score. We obtained a Flesch Reading Ease score of 64.6 out of 100 using Microsoft Word (version 2016; Microsoft Corporation, Redmond, WA, USA) and a score of 79.5 using an online Readability Test Tool (WebFX, Harrisburg, PA, USA) [37], indicating that the reading difficulty ranged from standard to fairly easy [38].
2.7. Survey Testing
Expert- and respondent-driven testing of the survey questions was undertaken in multiple ways, including stakeholder meetings, testing, and cognitive interviews.
2.7.1. AIDCoP Meeting
At the American Indian Data Community of Practice (AIDCoP) meeting on 16 October 2019, in Oklahoma City, we tested questions on the primary outcomes and HPV vaccine beliefs. The organization’s membership includes over 80 members from 12 tribal nations and 10 sectors of community, state, tribal, and federal entities. Before the meeting, we shared the background and objectives of the survey with meeting participants.
For each question on the primary outcomes and HPV vaccine beliefs, AIDCoP members were asked to provide feedback by discussing and responding to: “What is the best way to ask this question? Are the response options appropriate? Do any response options need to be modified? Are there any response options that are missing (that should have been included)?” Discussion points and suggestions to improve the question and response options were captured and incorporated to update the survey.
2.7.2. Cognitive Interviews
As part of the testing process, we conducted cognitive interviews to ensure that the respondents understood the survey questions. The interview was designed to elucidate comprehension of the survey question and understanding of the response process.
Seven participants were enrolled for cognitive interviews, as this sample size was considered adequate for identifying the measurement errors that the interviews were designed to detect [39]. We recruited participants through an e-mail message sent out to all Cherokee Nation employees.
We employed the think-aloud method (e.g., “tell me what you are thinking”) and asked the participants to actively verbalize their thoughts as they answered survey items [20,40]. We used the think-aloud method to identify any problems in answering the survey questions [20]. We also used the verbal probing method (e.g., “what does the term vaccination intent mean to you?”) to elicit detailed information beyond that usually provided by respondents [20,40]. The verbal probing method allowed us to assess participants’ understanding of the question and its key terms [20]. The interviews took place in September 2020 over Zoom (Zoom Video Communications, Inc., San Jose, CA, USA).
2.8. Survey Mode and Administration
The research team was unable to identify the parents or guardians by name specifically. Therefore, the team mailed the surveys to the listed address of the randomly selected adolescents and addressed the survey cover letter to “the parents/guardians of <INSERT ADOLESCENT’S NAME>”. Parents (biological, step, foster, adoptive) or guardians (grandparent, elder sibling, aunt or uncle, or other) of American Indian adolescents received paper surveys through the mail in April 2021. Either parent could complete the survey, or both parents could complete the survey together. The paper survey contained a weblink to an online version of the survey, offering participants the opportunity to take the survey either on paper or online through REDCap (Research Electronic Data Capture) (version 10.0.1; Vanderbilt University, Nashville, TN, USA). REDCap was selected as it is a secure, HIPAA-compliant web-based application designed for data collection for surveys. Before moving the REDCap survey to production, we thoroughly tested the survey for navigational and compatibility issues using various internet browsers on different platforms and operating systems. The survey will be self-administered. REDCap access to confidential information was limited to Cherokee Nation researchers and staff, and paper surveys were returned directly to Cherokee Nation. Only de-identified data were shared with researchers from the University of Oklahoma Health Sciences Center for analysis (CONSIDER statement #8).
In mixed-mode surveys, inconsistencies in presentation may lead to inconsistencies in measurement [41]. To reduce the potential for this measurement error, we adhered to the guidelines with the underlying principle of universal presentation developed by the U.S. Census Bureau [42]. The principle of universal presentation states that all respondents should be presented with the same question and response categories, regardless of mode, i.e., the meaning and intent of the question and response options must be consistent [42]. One of the trade-offs of the universal presentation was to eliminate skip logic from the online version of the survey as it could have created confusion in the paper survey, as seen in other studies [43].
To inform the prospective participants about the survey, an advance letter from Cherokee Nation Public Health was sent before the survey. As the letter came from a recognized organization in the community, participants may be more likely to respond to the survey, potentially reducing nonresponse error [44]. Furthermore, a round of reminder letters with the survey was sent to a subset of non-respondents in June 2021 to improve response and reduce nonresponse error. The survey was initially disseminated in April 2021, and we collected participant responses until July 2021. Data analysis will commence in August 2021.
2.9. Ethics
The study was approved by the Institutional Review Boards (IRB) of Cherokee Nation (approved on 27 July 2020) and the University of Oklahoma Health Sciences Center (IRB Number 12246; approved on 30 July 2020) and was conducted in compliance with its requirements (CONSIDER statement #5). To compensate for participation and reduce nonresponse error [45,46], we provided participants with a USD 30 gift card for the cognitive interview and a USD 10 gift card for the survey. The amount was agreed upon to be enough for appropriately compensating participants for their time and valuable contribution.
For cognitive interviews, we obtained verbal consent from participants using an information sheet. For the survey, we obtained a waiver of signed informed consent and added the following statement at the end of the consent form: “by taking the survey, you are agreeing to participate in this study” (CONSIDER statement #10).
The Cherokee Nation IRB reviewed and approved this paper before submission to the journal (CONSIDER statement #16).
3. Results
Based on the feedback from the AIDCoP meeting and cognitive interviews, the survey underwent several revisions. Participants provided input on survey measures, content, structure, terminology, and formatting (Table 1). For instance, we added a new question on assessing the awareness of HPV vaccine recommendation based on the cognitive interviews: “Before this survey, did you know that an HPV shot or vaccine was recommended for children aged 9–17 years?”
Table 1.
Examples of select revisions to survey questions based on testing.
| Content Area | Feedback | Rationale | Source | Final Revised Question |
|---|---|---|---|---|
| HPV awareness | Remove the following sentence from the question: HPV is different from HIV. | Participants felt that the sentence was unnecessary. | Cognitive interview | Human papillomavirus, also known as HPV, is a common sexually transmitted virus that can cause genital warts, cervical and other types of cancer in men and women. Before this survey, had you ever heard of human papillomavirus or HPV? |
| HPV and other vaccinations status | Remove the following sentence from the question: the Tdap booster shot also protects against pertussis or whooping cough. | Participants shared that the additional information on protection would be unnecessary. | Cognitive interview | Has your child ever received a tetanus booster shot? This is usually given at 11–12 years of age. The tetanus booster shot is also known as or sometimes called the Td or Tdap shot or vaccine. |
| HPV vaccine intent | Revise the question on intent (original question: how likely is it that your child will receive HPV shots in the next 12 months? Would you say) | Participants stated that the revised question would be much easier for parents or guardians to understand. | Cognitive interview | Will your child receive an HPV shot or vaccine in the next 12 months? |
| HPV vaccine recommendation | Include the following new question: Before this survey, did you know that an HPV shot or vaccine was recommended for children aged 9–17 years? | Participants stated that a question was needed to assess the awareness of the HPV vaccine recommendation. | Cognitive interview | Before this survey, did you know that an HPV shot or vaccine was recommended for children aged 9–17 years? |
| Change “health care professional” to “health care provider” | Participants stated that the term “health care provider” would be more accurate. | Cognitive interview | Has a doctor, nurse, or other health care provider ever recommended that your child receive an HPV shot or vaccine? | |
| HPV vaccine refusal | Remove the word “refuse” from the question. | Participants shared that the word “refuse” might be judgmental and alienate some participants. | AIDCoP meeting | If your child has NOT received any shots of the HPV vaccine, what is the MAIN reason that your child has NOT received the HPV shot or vaccine? (select ONE response only). |
| Bold and capitalize the following words in the question: not and main | Participants stated that bolding and capitalizing the words would help with clarity and emphasis. | Cognitive interview | ||
| HPV vaccine beliefs | Revise the following statement: The HPV vaccine is effective in preventing several cancers. | Participants asked to include and specify cervical cancer in the statement. | AIDCoP meeting | The HPV vaccine prevents cervical and other cancers. |
Abbreviations: AIDCoP, American Indian Data Community of Practice; HPV, human papillomavirus.
Participants also provided feedback on terminology, such as using “health care provider” instead of “healthcare professional” or “health care worker” in the survey questions; consistency, such as using HPV vaccine or shot throughout the survey instead of just HPV vaccine; formatting, such as bolding and capitalizing words in the question for the main measure, “If your child has NOT received any shots of the HPV vaccine, what is the MAIN reason that your child has NOT received the HPV shot or vaccine? (select ONE response only)”; and simplifying words, such as “risk” instead of “probability” and “getting” instead of “acquiring” in the response option “my child has a high risk of getting an HPV infection”.
The final survey contained 37 questions across ten content areas. The content areas and their purpose are presented in Table 2. All questions were closed-ended questions with mutually exclusive and exhaustive response options.
Table 2.
Information about the American Indian HPV Vaccination Survey in Cherokee Nation.
| # | Content Area | # of Question(s) | Purpose |
|---|---|---|---|
| 1 | Eligibility | 2 | To assess participant eligibility for the survey. |
| 2 | Demographic information | 11 | To obtain demographic data from the survey participants. |
| 3 | HPV awareness | 1 | To assess awareness of the virus. |
| 4 | HPV vaccine awareness | 1 | To assess awareness of the HPV vaccine. |
| 5 | HPV and other vaccinations status | 5 | To assess uptake of the HPV, meningitis, and tetanus booster vaccines. |
| 6 | HPV vaccination intent | 1 | To assess intent to receive the HPV vaccine (if not already received). |
| 7 | HPV vaccine recommendation | 4 | To assess whether the HPV vaccine was discussed or recommended by a health care provider. |
| 8 | HPV vaccine refusal | 1 | To assess the main reason why the adolescent has not received the HPV vaccine. |
| 9 | HPV vaccine acceptance | 1 | To assess the main reason why the adolescent received the HPV vaccine. |
| 10 | HPV vaccine beliefs | 7 | To assess beliefs about the importance, safety, and side effects of the HPV vaccine. |
| 11 | COVID-19 and vaccination | 3 | To assess intent to receive the COVID-19 vaccine and whether the pandemic has caused difficulty in getting the HPV vaccine for the adolescent. |
Abbreviations: HPV, human papillomavirus. # Total number of questions in each content area.
The final survey, informed consent form, advance letter, and cognitive interview guide are provided as Supplementary Materials.
4. Discussion
In this paper, we describe the development and dissemination of an HPV vaccination survey for American Indians in Cherokee Nation. As this survey is a new tool to advance our understanding of HPV vaccination factors in American Indian populations, we highlight our methodological choices and explain our decisions regarding the target population, sampling strategy, survey measures, and testing.
Increasing HPV vaccination rates in adolescents is an important step for cancer prevention efforts. It is a tribal priority affirmed in the Cherokee Nation Comprehensive Cancer Prevention and Control Plan 2018–2022 [17] and a national public health priority noted in the Healthy People 2030 initiative [47]. To advance and measure progress towards these priorities, there is a need for data that are specific to the tribe. Our survey was designed to estimate HPV vaccination coverage and assess factors associated with vaccination for American Indian adolescents aged 9 to 17 years in Cherokee Nation. By understanding vaccination factors, this survey will provide information to guide interventions and policies aimed towards improving HPV vaccine initiation and completion among Cherokee Nation adolescents (CONSIDER statement #17).
American Indian populations face data inequities. Existing data for their communities are often unavailable, inadequate, or incomplete [48,49]. In response to data inequities, Native nations are working towards advancing Indigenous data sovereignty and governance. Indigenous data sovereignty is “the right of a nation to govern the collection, ownership, and application of its own data” [50]. In a Tribal Data Practices Survey conducted by the National Congress of American Indians in 2017, responses representing 122 tribes revealed that 83% indicated that it was extremely important for tribes to collect or access data on their tribal citizens for governance purposes [49]. In the same survey, health (57%) was one of the most frequently identified need areas for more or better data on their tribal members [49]. Our survey represents a small effort among the numerous initiatives of data sovereignty and governance at Cherokee Nation. This project could provide guidance for other indigenous populations on how to conduct their research and evaluation projects to understand the health of their communities. Further, this project may guide academic and other research-based institutions on working with tribal entities and facilitate research and evaluation projects that are tribally directed and engaged. This may lead to better implementation of interventions aimed at improving the health and well-being of indigenous populations.
We anticipate several limitations with our survey. First, we plan to estimate HPV vaccination coverage for American Indian adolescents based on self-reported data provided by the parents. As a result, parental reports may under- or over-estimate the number of doses or shots received. However, previous studies have shown high concordance [51,52], indicating that parental reports of HPV vaccination status may be reasonably accurate. Second, and closely related, we rely on parental recall for HPV vaccination factors, which may vary depending on the time elapsed and information regarding the adolescents’ immunization, among other factors. Third, self-reported responses are subject to social desirability bias and may not correlate with actual or future behaviors, such as the intention to receive the HPV vaccine. However, we tried to minimize any resulting bias by assuring the participants in the consent letter that their responses were confidential. Fourth, we anticipate low response rates based on prior surveys of HPV vaccination [53,54], but have provided an incentive for participation, sent an advance letter, and mailed a reminder letter to mitigate nonresponse. In addition, we will adjust for nonresponse in our analysis. Fifth, due to the dynamic nature of the COVID-19 pandemic and response, participants’ attitudes and perceptions regarding the COVID-19 vaccine may evolve and not reflect the current reasons for vaccination. Lastly, we asked parents to select only the main reason for vaccinating or not vaccinating their child. Some parents may have had multiple reasons; however, capturing the main reasons allowed us to identify the most important reasons.
5. Conclusions
In summary, we developed a survey to estimate HPV vaccination coverage and factors. Survey design is a multi-step, collaborative, and iterative process. Through this paper, we have explained some of the steps with a view to make the process transparent and replicable. Researchers and communities interested in developing surveys for similar American Indian populations may benefit from the information presented in this paper.
Acknowledgments
The authors would like to thank the Cherokee Nation staff members who contributed to the implementation of this survey, including Melissa Foreman and Kylie N. Parnell. The authors appreciate the assistance of Kathy Kyler (Staff Editor, Office of the Vice President for Research, University of Oklahoma Health Sciences Center) in preparing this manuscript for publication.
Supplementary Materials
The following are available online at https://www.mdpi.com/article/10.3390/ijerph18179239/s1, Supplementary File 1: Final Survey; Supplementary File 2: Informed Consent Form; Supplementary File 3: Advance Letter; Supplementary File 4: Cognitive Interview Guide.
Author Contributions
All authors: S.V.G., A.E.J., M.B., J.E.C., S.A.M., A.H.W., S.C., A.S.A., S.F.P., S.P., and A.C. contributed to the design, development, and implementation of the survey. S.V.G. wrote the first draft of the manuscript. All authors provided feedback on the manuscript. All authors have read and agreed to the published version of the manuscript.
Funding
This project received funding from the Centers for Disease Control and Prevention (Grant NU58DP006345). SVG was supported by the Hudson Fellowship in Public Health sponsored by the Hudson College of Public Health at the University of Oklahoma Health Sciences Center. SC was supported by the National Institute on Minority Health and Health Disparities (NIMHD) at the National Institutes of Health (NIH) (1R21MD014658-01A1). SC and JEC were partly supported by the Oklahoma Shared Clinical and Translational Resources (U54GM104938) with an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences. AC, AEJ, ASA, and JEC were supported by the RADxSM-UP program from the National Institute of General Medical Sciences (3S06GM127983-03S1). JC was partly funded by the National Cancer Institute Cancer Center Support (Grant P30CA225520) awarded to the University of Oklahoma Stephenson Cancer Center for the use of the Biostatistics and Research Design Shared Resources. SAM was supported by the National Institute on Minority Health and Health Disparities of the National Institutes of Health (Grant R25MD011564). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or Cherokee Nation.
Institutional Review Board Statement
The study was approved by the Institutional Review Boards (IRB) of Cherokee Nation (approved on 27 July 2020) and the University of Oklahoma Health Sciences Center (IRB Number 12246; approved on 30 July 2020) and was conducted in compliance with its requirements.
Informed Consent Statement
For cognitive interviews, we obtained verbal consent from participants using an information sheet. For the survey, we obtained a waiver of signed informed consent and added the following statement at the end of the consent form: “by taking the survey, you are agreeing to participate in this study”.
Data Availability Statement
The survey, informed consent form, advance letter, and cognitive interview guide are provided as Supplementary Materials.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Melkonian S.C., Henley S.J., Senkomago V., Thomas C.C., Jim M.A., Apostolou A., Saraiya M. Cancers Associated with Human Papillomavirus in American Indian and Alaska Native Populations—United States, 2013–2017. MMWR Morb. Mortal Wkly. Rep. 2020;69:1283–1287. doi: 10.15585/mmwr.mm6937a2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Gopalani S.V., Janitz A.E., Campbell J.E. Trends in cervical cancer incidence and mortality in Oklahoma and the United States, 1999–2013. Cancer Epidemiol. 2018;56:140–145. doi: 10.1016/j.canep.2018.08.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Markowitz L.E., Gee J., Chesson H., Stokley S. Ten Years of Human Papillomavirus Vaccination in the United States. Acad. Pediatr. 2018;18:S3–S10. doi: 10.1016/j.acap.2017.09.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Duvall J., Buchwald D. Human papillomavirus vaccine policies among american Indian tribes in Washington State. J. Pediatr. Adolesc. Gynecol. 2012;25:131–135. doi: 10.1016/j.jpag.2011.11.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Jim C.C., Lee J.W., Groom A.V., Espey D.K., Saraiya M., Holve S., Bullock A., Howe J., Thierry J. Human papillomavirus vaccination practices among providers in Indian health service, tribal and urban Indian healthcare facilities. J. Womens Health. 2012;21:372–378. doi: 10.1089/jwh.2011.3417. [DOI] [PubMed] [Google Scholar]
- 6.Bruegl A.S., Bottsford-Miller J.N., Bodurka D.C. HPV vaccination practices among American Indian/Alaska Native providers. Gynecol. Oncol. 2016;141:132. doi: 10.1016/j.ygyno.2016.04.349. [DOI] [Google Scholar]
- 7.Schmidt-Grimminger D., Frerichs L., Black Bird A.E., Workman K., Dobberpuhl M., Watanabe-Galloway S. HPV knowledge, attitudes, and beliefs among Northern Plains American Indian adolescents, parents, young adults, and health professionals. J. Cancer Educ. 2013;28:357–366. doi: 10.1007/s13187-013-0468-y. [DOI] [PubMed] [Google Scholar]
- 8.Hodge F.S., Itty T., Cardoza B., Samuel-Nakamura C. HPV vaccine readiness among American Indian college students. Ethn. Dis. 2011;21:415–420. [PubMed] [Google Scholar]
- 9.Hodge F.S. To get the shot or not: HPV vaccines and American Indian female student decision-making. Commun. Nurs. Res. 2011;44:131. [Google Scholar]
- 10.Samuel-Nakamura C., Hodge F.S. Modifiable and Non-Modifiable Factors Associated with HPV Vaccine Decision-Making among American Indian Women College Students. Am. Indian Cult. Res. J. 2016;40:71–81. doi: 10.17953/aicrj.40.4.samuel-nakamura. [DOI] [Google Scholar]
- 11.Buchwald D., Muller C., Bell M., Schmidt-Grimminger D. Attitudes toward HPV vaccination among rural American Indian women and urban White women in the northern plains. Health Educ. Behav. 2013;40:704–711. doi: 10.1177/1090198113477111. [DOI] [PubMed] [Google Scholar]
- 12.Toffolon-Weiss M., Hagan K., Leston J., Peterson L., Provost E., Hennessy T. Alaska Native parental attitudes on cervical cancer, HPV and the HPV vaccine. Int. J. Circumpolar Health. 2008;67:363–373. doi: 10.3402/ijch.v67i4.18347. [DOI] [PubMed] [Google Scholar]
- 13.Kepka D., Bodson J., Lai D., Sanchez-Birkhead A.C., Davis F.A., Lee D., Tavake-Pasi F., Napia E., Villalta J., Mukundente V., et al. Diverse caregivers’ HPV vaccine-related awareness and knowledge. Ethn. Health. 2018;26:811–826. doi: 10.1080/13557858.2018.1562052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Lai D., Bodson J., Davis F.A., Lee D., Tavake-Pasi F., Napia E., Villalta J., Mukundente V., Mooney R., Coulter H., et al. Diverse Families’ Experiences with HPV Vaccine Information Sources: A Community-Based Participatory Approach. J. Community Health. 2017;42:400–412. doi: 10.1007/s10900-016-0269-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Huria T., Palmer S.C., Pitama S., Beckert L., Lacey C., Ewen S., Smith L.T. Consolidated criteria for strengthening reporting of health research involving indigenous peoples: The CONSIDER statement. BMC Med. Res. Methodol. 2019;19:173. doi: 10.1186/s12874-019-0815-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Lucero E. From tradition to evidence: Decolonization of the evidence-based practice system. J. Psychoact. Drugs. 2011;43:319–324. doi: 10.1080/02791072.2011.628925. [DOI] [PubMed] [Google Scholar]
- 17.Cherokee Nation . Cherokee Nation Comprehensive Cancer Prevention and Control Plan 2018–2022. Cherokee Nation Public Health; Tahlequah, OK, USA: 2018. [Google Scholar]
- 18.Health Center and Hospital Locations. [(accessed on 15 May 2021)]; Available online: https://health.cherokee.org/health-center-and-hospital-locations/
- 19.Reiter P.L., Brewer N.T., Gottlieb S.L., McRee A.L., Smith J.S. Parents’ health beliefs and HPV vaccination of their adolescent daughters. Soc. Sci. Med. 2009;69:475–480. doi: 10.1016/j.socscimed.2009.05.024. [DOI] [PubMed] [Google Scholar]
- 20.Lavrakas P.J. Encyclopedia of Survey Research Methods. SAGE Publications, Inc.; Thousand Oaks, CA, USA: 2008. [Google Scholar]
- 21.Cochran W.G. Sampling Techniques. 3rd ed. John Wiley & Sons; New York, NY, USA: 1977. [Google Scholar]
- 22.Walker T.Y., Elam-Evans L.D., Yankey D., Markowitz L.E., Williams C.L., Fredua B., Singleton J.A., Stokley S. National, Regional, State, and Selected Local Area Vaccination Coverage Among Adolescents Aged 13-17 Years—United States, 2018. MMWR Morb. Mortal Wkly. Rep. 2019;68:718–723. doi: 10.15585/mmwr.mm6833a2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.NIS-Child Data and Documentation for 2015 to Present. [(accessed on 10 December 2019)]; Available online: https://www.cdc.gov/vaccines/imz-managers/nis/datasets.html.
- 24.NHIS Data, Questionnaires and Related Documentation. [(accessed on 10 December 2019)]; Available online: https://www.cdc.gov/nchs/nhis/data-questionnaires-documentation.htm.
- 25.Health Information National Trends Survey Survey Instruments. [(accessed on 10 December 2019)]; Available online: https://hints.cancer.gov/data/survey-instruments.aspx.
- 26.National Survey of Family Growth Questionnaires, Datasets, and Related Documentation. [(accessed on 10 December 2019)]; Available online: https://www.cdc.gov/nchs/nsfg/nsfg_questionnaires.htm.
- 27.NHANES Questionnaires, Datasets, and Related Documentation. [(accessed on 10 December 2019)]; Available online: https://wwwn.cdc.gov/nchs/nhanes/Default.aspx.
- 28.BRFSS Questionnaires. [(accessed on 10 December 2019)]; Available online: https://www.cdc.gov/brfss/questionnaires/index.htm.
- 29.Shah P.D., Calo W.A., Gilkey M.B., Boynton M.H., Alton Dailey S., Todd K.G., Robichaud M.O., Margolis M.A., Brewer N.T. Questions and Concerns About HPV Vaccine: A Communication Experiment. Pediatrics. 2019;143:e20181872. doi: 10.1542/peds.2018-1872. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.McRee A.L., Brewer N.T., Reiter P.L., Gottlieb S.L., Smith J.S. The Carolina HPV immunization attitudes and beliefs scale (CHIAS): Scale development and associations with intentions to vaccinate. Sex. Transm. Dis. 2010;37:234–239. doi: 10.1097/OLQ.0b013e3181c37e15. [DOI] [PubMed] [Google Scholar]
- 31.Q-Bank. [(accessed on 20 December 2019)]; Available online: https://wwwn.cdc.gov/qbank/home.aspx.
- 32.Bastani R., Glenn B.A., Tsui J., Chang L.C., Marchand E.J., Taylor V.M., Singhal R. Understanding suboptimal human papillomavirus vaccine uptake among ethnic minority girls. Cancer Epidemiol. Biomark. Prev. 2011;20:1463–1472. doi: 10.1158/1055-9965.EPI-11-0267. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Laz T.H., Rahman M., Berenson A.B. An update on human papillomavirus vaccine uptake among 11-17 year old girls in the United States: National Health Interview Survey, 2010. Vaccine. 2012;30:3534–3540. doi: 10.1016/j.vaccine.2012.03.067. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Newcomer S.R., Caringi J., Jones B., Coyle E., Schehl T., Daley M.F. A Mixed-Methods Analysis of Barriers to and Facilitators of Human Papillomavirus Vaccination Among Adolescents in Montana. Public Health Rep. 2020;135:842–850. doi: 10.1177/0033354920954512. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Berenson A.B., Brown V.G., Fuchs E.L., Hirth J.M., Chang M. Relationship between maternal experiences and adolescent HPV vaccination. Hum. Vaccine Immunother. 2017;13:2150–2154. doi: 10.1080/21645515.2017.1332551. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Donahue K.L., Hendrix K.S., Sturm L.A., Zimet G.D. Human Papillomavirus Vaccine Initiation among 9-13-Year-Olds in the United States. Prev. Med. Rep. 2015;2:892–898. doi: 10.1016/j.pmedr.2015.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Readability Test Tool. [(accessed on 15 March 2020)]; Available online: https://www.webfx.com/tools/read-able/
- 38.D’Alessandro D.M., Kingsley P., Johnson-West J. The readability of pediatric patient education materials on the World Wide Web. Arch. Pediatr. Adolesc. Med. 2001;155:807–812. doi: 10.1001/archpedi.155.7.807. [DOI] [PubMed] [Google Scholar]
- 39.Willis G.B. Cognitive Interviewing: A Tool for Improving Questionnaire Design. Sage Publications; Thousand Oaks, CA, USA: 2004. [Google Scholar]
- 40.Willis G.B., Artino A.R., Jr. What Do Our Respondents Think We’re Asking? Using Cognitive Interviewing to Improve Medical Education Surveys. J. Grad. Med. Educ. 2013;5:353–356. doi: 10.4300/JGME-D-13-00154.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Blair J., Czaja R.F., Blair E.A. Designing Surveys: A Guide to Decisions and Procedures. Sage Publications; Thousand Oaks, CA, USA: 2013. [Google Scholar]
- 42.Martin E., Childs J.H., DeMaio T., Hill J., Reiser C., Gerber E., Styles K., Dillman D. Guidelines for Designing Questionnaires for Administration in Different Modes. US Census Bureau; Washington, DC, USA: 2007. p. 20233. [Google Scholar]
- 43.Halista C.E., Kline R.J., Bepko J. Understanding Barriers to HPV Vaccination: Perspectives from Air Force Family Medicine Physicians and Active Duty Air Force Males. Mil. Med. 2019;185:e878–e886. doi: 10.1093/milmed/usz403. [DOI] [PubMed] [Google Scholar]
- 44.Fox R.J., Crask M.R., Kim J. Mail survey response rate: A meta-analysis of selected techniques for inducing response. Public Opin. Q. 1988;52:467–491. doi: 10.1086/269125. [DOI] [Google Scholar]
- 45.Millar M.M., Dillman D.A. Improving response to web and mixed-mode surveys. Public Opin. Q. 2011;75:249–269. doi: 10.1093/poq/nfr003. [DOI] [Google Scholar]
- 46.VanGeest J.B., Johnson T.P., Welch V.L. Methodologies for improving response rates in surveys of physicians: A systematic review. Eval. Health Prof. 2007;30:303–321. doi: 10.1177/0163278707307899. [DOI] [PubMed] [Google Scholar]
- 47.Vaccination Objectives. [(accessed on 1 April 2021)]; Available online: https://health.gov/healthypeople/objectives-and-data/browse-objectives/vaccination.
- 48.Rodriguez-Lonebear D. Building a data revolution in Indian country. In: Kukutai T., Taylor J., editors. Indigenous Data Sovereignty: Toward an Agenda. ANU Press; Acton, Australia: 2016. pp. 253–272. [Google Scholar]
- 49.NCAI Policy Research Center . The State of Tribal Data Capacity in Indian Country: Key Findings from the Survey of Tribal Data Practices. National Congress of American Indians; Washington, DC, USA: 2018. [Google Scholar]
- 50.Carroll S.R., Rodriguez-Lonebear D., Martinez A. Indigenous Data Governance: Strategies from United States Native Nations. Data Sci. J. 2019;18:31. doi: 10.5334/dsj-2019-031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Ojha R.P., Tota J.E., Offutt-Powell T.N., Klosky J.L., Ashokkumar R., Gurney J.G. The accuracy of human papillomavirus vaccination status based on adult proxy recall or household immunization records for adolescent females in the United States: Results from the National Immunization Survey-Teen. Ann. Epidemiol. 2013;23:281–285. doi: 10.1016/j.annepidem.2013.02.002. [DOI] [PubMed] [Google Scholar]
- 52.Rolnick S.J., Parker E.D., Nordin J.D., Hedblom B.D., Wei F., Kerby T., Jackson J.M., Crain A.L., Euler G. Self-report compared to electronic medical record across eight adult vaccines: Do results vary by demographic factors? Vaccine. 2013;31:3928–3935. doi: 10.1016/j.vaccine.2013.06.041. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Fontenot H.B., Fantasia H.C., Charyk A., Sutherland M.A. Human papillomavirus (HPV) risk factors, vaccination patterns, and vaccine perceptions among a sample of male college students. J. Am. Coll Health. 2014;62:186–192. doi: 10.1080/07448481.2013.872649. [DOI] [PubMed] [Google Scholar]
- 54.McCave E.L. Influential factors in HPV vaccination uptake among providers in four states. J. Community Health. 2010;35:645–652. doi: 10.1007/s10900-010-9255-4. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The survey, informed consent form, advance letter, and cognitive interview guide are provided as Supplementary Materials.