Abstract
Background
Health literacy is a key determinant of health, shaping individuals ability to understand and act on health information. Low cancer literacy is associated with fatalistic beliefs and reduced participation in prevention and screening programmes. Although around 40% of all cancers are preventable, public awareness of modifiable risk factors remains low. The European Code Against Cancer (ECAC), developed under the World Code Against Cancer Framework, aims to strengthen prevention knowledge; however, tools to measure cancer prevention literacy, and therefore evaluate its impact, are lacking. This study aimed to develop and validate a questionnaire to assess ‘cancer prevention literacy’ in adults from the general population.
Methods
We conducted a methodological study to develop a questionnaire measuring ‘cancer prevention literacy’ in adults from the general population using the Integrative Model of Behavioural Prediction as theoretical framework. Questionnaire development followed a three-stage process: (1) a literature review and two-round Delphi process to select and refine items; (2) a third Delphi round, complemented by readability checks, face-validity assessment, and cognitive testing to evaluate item relevance, clarity, and representativeness; and (3) a pre-test and psychometric analysis to assess construct validity.
Results
The final Cancer Prevention Literacy Questionnaire (CPL-Q) consists of a 10-question validated instrument comprising 58 items, with selected items incorporating the verbatim wording of the recommendations from the ECAC, 4th edition. Strong expert agreement was achieved through an iterative Delphi process, and content and face validity indicated that the items were comprehensible, relevant, and appropriate. Psychometric analyses, including exploratory and confirmatory factor analysis, provide preliminary support for the instrument’s construct validity, with initial evidence suggesting acceptable reliability.
Conclusion
The CPL-Q provides a standardised instrument to measure ‘cancer prevention literacy’ and to assess the impact of Regional Codes Against Cancer through monitoring and benchmarking changes over time. It also offers a practical tool to identify gaps in ‘cancer prevention literacy’ within specific populations and to support international comparisons and longitudinal evaluations, thereby contributing to the evidence base for cancer prevention policies.
Supplementary Information
The online version contains supplementary material available at 10.1186/s12889-026-26641-9.
Keywords: Cancer prevention, Literacy, Questionnaire, Regional Codes Against Cancer
Background
Health literacy is a major determinant of health. The World Health Organization (WHO) defines health literacy as the cognitive and social skills that determine an individual’s motivation and ability to access, understand, and use information to promote and maintain good health [1]. Literacy affects people’s capacity to interact effectively with providers, make informed decisions, adopt healthy behaviours, manage their own health, and navigate health systems [2]. Low health literacy is often linked to reduced use of preventive services, delays in diagnosis, poor adherence to medical recommendations, higher hospitalization and mortality rates, and substantially greater healthcare costs [3], as understanding how to prevent disease and navigate health-care systems may contribute to better health outcomes. In this context, cancer literacy refers to all the knowledge a layperson needs to possess to understand the information and advice the health system provides concerning cancer prevention, diagnosis, and treatment [4].
Cancer remains a major cause of illness and death in the European Union (EU), with approximately 2.9 million new cases and 1.3 million deaths in 2022 [5]. These numbers are projected to increase by 25.3%, reaching an estimated 3.8 million new cases annually by 2050 [6]. However, a significant portion of the burden, around 40% of cases, could be prevented by reducing exposure to modifiable risk factors of cancer and promoting behaviours that enhance protection against cancer [7]. This underscores the critical role of prevention as the most cost-effective long-term strategy for cancer control [8].
Adults with low cancer literacy are less informed about cancer and more likely to hold fatalistic views regarding cancer and its prevention [9]. Poor literacy is also linked to avoidance of medical visits and confusion or apprehension about screenings. Factors such as health beliefs, attitudes, cancer fatalism, low self-efficacy, and mistrust of healthcare providers may contribute to worse health outcomes among adults with low health literacy [10]. Disadvantaged social and socioeconomic conditions, particularly low socioeconomic status and educational attainment, are key determinants of low health literacy levels [11]. These social determinants contribute to inequalities in cancer prevention and diagnosis. Improving access to information about cancer risk factors and its determinants, and enhancing understanding, is essential to reducing inequalities in health outcomes.
Although improving health literacy requires more than simply transmitting health information, health education remains a fundamental tool for promoting health and preventing disease [12]. Accordingly, the Europe’s Beating Cancer Plan [8] seeks to raise awareness of, and address, key cancer risk factors to improve health literacy. The European Code Against Cancer (ECAC) is an educational tool designed to improve individuals’ cancer prevention literacy through evidence-based recommendations on preventive actions and interventions, thereby helping reduce their cancer risk [13].
Taking the ECAC, 4th edition (ECAC4) as a model, the International Agency for Research on Cancer (IARC/WHO) launched the World Code Against Cancer Framework in 2022 [14], an initiative aimed at promoting cancer prevention globally through the development of Regional Codes Against Cancer [15]. A gap identified during projects on the development and dissemination of the Regional Codes Against Cancer in the EU [16, 17] and in Latin America and the Caribbean [18] is the generally low level of public awareness of cancer risk factors and preventive interventions [19–21], along with the need to periodically monitor and evaluate changes in cancer prevention literacy to measure and compare the impact of the Regional Codes [22].
Building on existing cancer prevention awareness questionnaires, this study was framed by the need to define the concept of ‘cancer prevention literacy’ within the World Code Against Cancer Framework. The main aim was to develop a comprehensive questionnaire for adults in the general population with evidence of validity suitable for evaluating the impact of Regional Codes Against Cancer and related initiatives.
Methodology
Study design and framework
This is a methodological study aimed to develop a questionnaire to measure ‘cancer prevention literacy’ in adults of the general population with evidence of validity, guided by the Integrative Model of Behavioural Prediction [23]. To achieve this objective, we used a three-stage approach (Fig. 1): (1) a literature review and two-round Delphi process to select and refine questionnaire items; (2) a third Delphi round, together with readability checks, face validity assessment and cognitive testing to assess item relevance, clarity and representativeness; and (3) psychometric analysis to evaluate construct validity [24]. All these methodological components are complementary steps designed to support the main objective of the study.
Fig. 1.
Stepwise process of questionnaire development and subsequent validation
The Integrative Model of Behavioural Prediction [23] posits that behavioural intention which leads to behaviour implementation, arises from specific beliefs, shaped by three determinants: attitude, perceived norm, and self-efficacy. In addition, several theoretical frameworks of prevention behaviour, including the Health Behaviour Model [25] and the Preventive Health Care Behaviour Model [26], have proposed knowledge as an antecedent of preventive behaviour; the latter model also considers self-efficacy, health motivation and health consciousness as influences on preventive health care behaviours. The association of health literacy with cancer-related attitudes, knowledge, and behaviours have also been studied as a way to educate and increase patient trust, self-efficacy, and engagement in decision-making [27]. Given the interrelated nature of these determinants, the term ‘cancer prevention literacy’ is used as a unifying framework for the questionnaire, encompassing all these constructs, as well as others selected from the behavioural change models mentioned above, and from other models such as the Tripartite Model of Risk Perception (TRIRISK) [28]. The constructs included at this stage were beliefs about cancer, awareness about cancer prevention, knowledge of cancer prevention and risk factors, attitudes towards cancer prevention, intention to change behaviours, self-efficacy to engage in cancer prevention, adherence to behaviours, informational needs on cancer prevention, risk perception, and health literacy [see Supplementary Table 1 (Additional file 1) for the full list of selected constructs and definitions [1, 23, 25, 28–35]].
Questionnaire development
A literature review was undertaken to identify questionnaires administrated to the public in any part of the world and available in English, that included the selected constructs related to cancer prevention [Supplementary Table 1 (Additional file 1)], in order to extract relevant questions. Validated and recurring (periodic) questionnaires were prioritised.
Seventeen existing questionnaires identified through a literature search were reviewed. Items were sourced from nine of these instruments, including national health and public opinion surveys, and peer-reviewed publications. Sixteen questions related to eight of the selected constructs, either taken verbatim or adapted from existing questionnaires, were included in the first draft of the questionnaire [Supplementary Table 2 (Additional file 1)]. A specific construct on awareness of the ECAC was added to allow monitoring and evaluating the impact of the implementation of the tool in the EU. Where applicable, the verbatim wording of the risk factors, behaviours or interventions mentioned in ECAC4 served as the basis for adapting the questions.
Initial feedback was obtained from close collaborators who are experts in cancer prevention and health literacy. They were asked to review the first draft of the questionnaire and suggest the inclusion, exclusion or modification of items; classify questions under the most appropriate constructs; and identify any additional relevant constructs for ‘cancer prevention literacy’. Along with the draft questionnaire, experts received a preliminary database of alternative questions identified from our literature search, allowing them to select alternatives if they considered any original question inappropriate. They were also asked to recommend other useful questionnaires or sources. Feedback was then assessed and discussed internally among the research team (AF, PRC, CE), and changes were made where deemed appropriate based on relevance, clarity, and alignment with the objectives of the questionnaire.
Delphi method
The Delphi technique is a well-established method for generating content for questionnaires [36]. It involves structured procedure to reach consensus on complex issues where knowledge is uncertain or incomplete, which are assessed by a panel of experts through an iterative and structured process. The Delphi process typically includes at least two rounds of anonymous surveys. After each round, panellists’ individual responses are fed back in an anonymised manner so that they can be reconsidered before the next round, with the aim to achieve consensus [36, 37]. Consensus was defined as a minimum of 70% agreement among panellists, in accordance with standard practice in Delphi studies [38].
Panellists
Delphi panels typically include between 10 and 100 experts, with no agreed-upon standard [39]. Fifteen experts in cancer prevention and health literacy were invited and agreed to participate. They represented research centres, universities, and national health and cancer institutes in the EU (France, Portugal, Romania, and Spain) and Latin America (Argentina, Brazil, Chile, Colombia, Costa Rica and Perú), corresponding to the regions where a Regional Code Against Cancer existed at the time of this study.
Panellists were informed that there would be two to three rounds of deliberation, involving de-identified surveys, and that the time required to respond might vary across rounds. They were also informed that the rounds would take place over a maximum of five months, with 10 days to two weeks allocated for responses, followed by an additional one to two weeks for reviewing comments, providing feedback, and preparing the questionnaire for the next round. After each round, panellists received a summary report outlining the main decisions taken.
All communication with panellists were conducted via email. For each round, messages were sent to the panel distribution list created and managed by the research team, using ‘blind carbon copy’ (bcc) to maintain confidentiality. The names of the panellists were known only to the research team (AF, RI, PRC, and CE).
Data collection and analysis
Round 1
Panellist were informed that with Round 1 the Delphi process had officially begun, some had already been consulted during preliminary phase. The purpose of Round 1 was to agree on the questionnaire items, their order, and the suitability of the proposed response format (e.g., Likert scale, dichotomous, multiple choice). Panellists could suggest alternative response formats and provide feedback on the wording of any questions and items. Items not reaching consensus were either removed or substantially revised for consideration in the following round.
Round 2
The purpose of this intermediate round was to reach consensus on the proposed response format and item order for the questions that had not reach consensus in Round 1. Panellists were asked to assess whether the revised options, based on Round 1 feedback, were appropriate. In this round, panellists were not allowed to propose alternatives beyond those provided. Content was retained only if consensus was reached.
Final round (Round 3)
The purpose of the final round was to assess the content validity (relevance and representativeness) of the questions, prioritise them, and reduce the questionnaire’s length by eliminating items deemed less important. They were asked to rate each question on a 7-point Likert scale, with 1 representing ‘not very relevant’ and 7 representing ‘highly relevant’. Panellists were also asked to select the five most-relevant questions and the three least relevant questions for prioritisation [40]. At this stage, panellists could not suggest changes to item wording or propose new questions.
In this phase, data analysis focused on the median and interquartile range (IQR) of panellists’ responses, the preferred approach for Delphi studies [41]. Median scores, IQRs, and prioritisation rankings were used to determine the degree of agreement among panel members, with the selection criteria for consensus detailed in Table 1.
Table 1.
Criteria for selecting (prioritising) items included in the questionnaire
| Baseline | The median Likert scale scores of each question had to be 5 or higher. |
|---|---|
| #1 | Primary prioritisation criterion: Questions with higher median Likert scale scores were prioritised. |
| #2 | Secondary prioritisation criterion: Among questions with the same median Likert scale scores, those with higher mean score were prioritised. |
| #3 | Tertiary prioritisation criterion: Among questions with the same mean score, those with a smaller IQR were prioritised, as a smaller IQR indicated higher the consensus. |
| #4 | Quaternary prioritisation criterion: Among questions with the same IQR, those with a higher third quartile (P75) value, closer to 7 (high priority), were prioritised. |
| #5 | Final prioritisation criterion: For questions identical across all previous criteria, the number of times they were ranked among the top 5 most relevant or bottom 3 least relevant was used as the decisive criteria. |
Once consensus was reached on the final set of questions and items, the questionnaire underwent additional procedures to explore aspects of validity, including content, face, and construct validity. These involved assessing whether the instrument’s content adequately reflected the construct being measured (content validity), whether it appeared to measure what it was intended to (face validity), and whether the items collectively measure each of the theoretical construct they are intended to assess (construct validity) [42, 43].
Content and face validity
Readability assessment
Readability assessment is a critical aspect of content validity, ensuring that the questionnaire is comprehensible for the target audience. A readability analysis was conducted using the Hemingway Editor (https://hemingwayapp.com/), a digital tool designed to assess the reading age of documents. An a priori reading age threshold of 10 years was set, as it is recommended that health questionnaires for adults are written at a fifth to sixth grade reading level, corresponding to a reading age of 10–11 years of age [44].
External expert review of clarity and appropriateness
A separate sample of cancer prevention experts who were not part of the Delphi panel was asked to evaluate the clarity, comprehensibility, and appropriateness of the questionnaire for the target audience, and to identify potential ambiguities, misinterpretations, or omissions, as a measure of face validity.
The questionnaire, along with a de-identified online form, was distributed to a convenience sample of 10 experts, selected within IARC and other institutions in Europe and Latin America working in cancer prevention. Participants rated each item’s clarity, comprehensibility, and appropriateness for the general population using a 4-point Likert scale: 1 = Not clear and understandable; 2 = Somewhat clear and understandable; 3 = Clear and understandable, and 4 = Very clear and understandable [45]. The questionnaire also included general items on overall appropriateness, potential gaps in constructs relevant to ‘cancer prevention literacy’, and the adequacy of structure and format. Raters received instructions to guide the validation process. The only exclusion criteria included non-response and incomplete questionnaires.
Ratings of 1 and 2 indicated insufficient clarity, while ratings of 3 or 4 indicated acceptable clarity and comprehensibility. The Item-Level Face Validity Index (I-FVI) was calculated by dividing the number of ratings scored 3 or 4 by the total number of raters. The Scale-Level Face Validity Index (S-FVI/Ave) was calculated as the average of all I-FVI scores. Values ranged from 0 to 1, with thresholds set at ≥ 0.75 for I-FVI and ≥ 0.80 for S-FVI. Items with I-FVI scores below 0.75 were not accepted and flagged for revision through cognitive testing interviews [45].
Cognitive testing
Cognitive testing, another key technique to ensure face and content validity, was conducted with a convenient sample of native English speakers with the aim to (1) evaluate whether the wording contained ambiguous or unfamiliar terms, (2) identify any structural issues that could confuse respondents, (3) check whether response categories included or excluded what respondents expected, (4) assess whether recalling dates or frequencies posed any challenges; and (5) determine whether the question wording was perceived as upsetting or intrusive.
Participants were recruited through the researchers’ informal networks. The only inclusion criteria were being at least 18 years old and a native English speaker. We endeavoured to select between 6 and 10 participants to ensure enough diversity in terms of gender, age, and educational level. A total of 16 potential participants who expressed initial interest in the study were finally invited to participate.
We prioritised testing the questions that did not achieve consensus during the face validity assessment and those with grade levels above the threshold. Interview probes were developed to elicit in-depth responses from participants. Additionally, a data collection grid (Additional file 2) was created to standardise the assessment of responses and classify them into one of the following categories: (1) ‘understood with ease’ (quick to answer, showed understanding and/or probe questions aligned with question meaning); (2) ‘understood with some difficulty’ (verbal or non-verbal communication as pauses and hesitation, asked for repetition, expressed difficulty answering or asked for clarification) and (3) ‘unable to understand effectively’ (couldn’t provide/qualify response, responded outside the scope of the question). Scoring the interviews provided a quantitative approach to identify recurring points of confusion and frustration. In parallel, qualitative data –captured through recordings or handwritten notes –offered detailed insights into respondents’ answers, helping to determine whether questions posed issues related to comprehension, recall, judgment, or response [46].
The interviews were conducted online via videoconferencing by two researchers (RI and AS) and were expected to last approximately one hour. They were conducted in two rounds. After the first five interviews, identified issues were discussed by the research team, and appropriate revisions were made to the questions for testing with the remaining participants. All interviews were recorded, and written informed consent was obtained from all participants.
Written notes from the interviews were carefully reviewed, and recordings were consulted as needed to fill in any gaps. Notes and scores for each question were compiled and analysed across all interviews to identify recurring issues or deviations from the intended meaning. When a clear pattern of misunderstanding emerged, the questions were reassessed in conjunction with the face validity results. Based on these findings and the researchers’ collective judgment, appropriate revisions were made to improve clarity.
Construct validity
Pilot testing and preliminary psychometric analysis
A factor analysis (FA) was conducted to assess construct validity and to explore the underlying dimensions explaining relationships among multiple questionnaire items [47], using data from a pilot test conducted among adults (≥ 18 years) with no prior diagnosis of cancer. Sample size was determined based on the questionnaire scale with the largest number of items, using the conventional rule of thumb of 10–15 participants per item [48, 49]. The primary scales designed for FA contained 4–12 items per scale, as the 25-item awareness about cancer prevention question (Q4, Table 2) was initially conceptualised as a comprehensive checklist rather than a multi-item scale. English-speaking participants were recruited through market research panels using both probability- and non-probability-based sampling methods. All data were collected online.
Table 2.
Overview of questionnaire’s constructs, questions, items, and response formats following the final round of the Delphi study, as used in subsequent validation steps
| Construct | Question | Response formats |
|---|---|---|
| Beliefs about cancer |
Q1. Please, indicate your level of agreement or disagreement with the following statements: a) It seems like everything causes cancer b) When I think about cancer, I automatically think about death c) There is not much that you can do to lower your chances of getting cancer d) There is so much information out there about cancer that it is hard to know what to do |
5-item Likert scale |
| Risk perception | Q2. How often do you worry about getting cancer? | 5-item Likert scale |
| Q3. How do you think your chance of developing cancer in the future compares to the average person of your gender and age? | 5-item Likert scale | |
| Awareness about cancer prevention (general) | Q4. Which of the following, if any, increases a person’s chance of developing cancer? (list of risk factors, e.g., smoking, exposure to another person’s smoking, being overweight, not being physically active, not eating enough whole grains, eating processed meat, etc.) | Dichotomous (Yes/No) |
| Knowledge about cancer prevention | Q5. What percentage of cancer cases do you think could be prevented with cancer prevention? | Multiple choice |
| Intention to change behaviours: unhealthy behaviours, and uptake cancer prevention interventions | Q6. I intend to make changes to: (list of behaviours, e.g., have a healthier diet, limit my alcohol intake, avoid too much sun, etc.) | Dichotomous (Yes/No) |
|
Q7. If your children were invited to have the following vaccination, would you agree to him/her having it sometime soon? a) Hepatitis B b) Human Papillomavirus |
5-item Likert scale | |
| Adherence to behaviours | Q8. Is there anything that you do regularly to consciously prevent cancer? (list of behaviours, e.g., do not smoke, take action to be a healthy body weight, be physically active in everyday life, avoid too much sun, etc. ) |
Multiple choice (select all that apply) |
| Q9. Have you taken any steps in the last 12 months to prevent cancer? If yes, what steps have you taken with the specific aim of preventing cancer? Please select all that apply. (list of behaviours, e.g., refraining from smoking, having a healthy diet, limiting my alcohol intake, etc.) |
Multiple choice (select all that apply) |
|
| Awareness about the European Code Against Cancer | Q10. Are you familiar with the European Code Against Cancer? |
Dichotomous (Yes/No) |
All questionnaire items were considered in the analysis, except for the single question on knowledge about cancer prevention (Q5, Table 2). This question was excluded due to its objective nature and response format, which involved multiple discrete options (percentage estimates), making it unsuitable for standard FA.
A preliminary psychometric analysis was conducted in several steps. First, descriptive analyses were performed to summarise and explore the key characteristics of the multivariate data, providing an overview of item distribution, variability, and potential inter-item relationships. Associations among items within each question were examined using Spearman’s rank-order correlation coefficients (ρ). The suitability of the data for FA was then assessed using the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity. A KMO value ≥ 0.7 and a significant Bartlett’s test were considered to indicate that the data were suitable for FA, reflecting sufficient inter-item correlations. Following this, FA was conducted only for constructs that met these criteria and for those with a sufficient number of items to investigate the underlying structure of the questionnaire. The questions under the intention to change behaviours construct were excluded: the one related to unhealthy behaviours (Q6, Table 2), due to some items not being applicable to all participants or being relevant only to specific subgroups, limiting their suitability for inclusion in a consistent factor structure; and the one related to the uptake of cancer prevention interventions (Q7, Table 2), as it contained only one question with two items, which was insufficient for meaningful FA. While no missing values were observed in the dataset, for Q6 (Table 2), the responses ‘don’t know’ and ‘not applicable’ were excluded from the corresponding analyses. To avoid potential biases related to this exclusion (e.g., exclusion of non-smokers), only descriptive analyses were performed for these items.
Depending on the construct, the specific aims, and whether a theory-driven model or framework was available, either EFA, Confirmatory Factor Analysis (CFA), or both were applied. EFA was appropriate and was used when the factor structure was unknown, to identify latent factors underlying item variation and correlations. Ordinal and binary items were analysed using their observed response categories, without transformation. Pearson correlation matrices were used to examine inter-item relationships. The number of extracted factors (or factor retention) was guided by theoretical considerations, the Kaiser criterion (eigenvalues > 1), and visual inspection of scree plots. Factors were extracted using principal axis factoring and rotated using an oblique rotation, allowing factors to correlate, which might be consistent with theoretical expectations for cancer prevention literacy constructs. Factor loadings ≥ 0.40 were retained [48, 50].
CFA was used when a theory-based or previously validated structure existed to test how well the observed data from the pilot study fit the hypothesised model, thereby confirming or rejecting the proposed structure. When CFA was applied without a preceding EFA, a one-factor model was specified, assuming a single latent construct. The CFA estimation method used was the weighted least squares mean and variance adjusted (WLSMV) estimator for ordinal/categorical data. CFA model fit was evaluated using standard global fit indices, including the Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR). Acceptable fit was determined according to conventional thresholds (e.g., CFI/TLI ≥ 0.90, RMSEA ≤ 0.08, SRMR ≤ 0.08)] [48, 50, 51].
Cronbach’s alpha (α) coefficients were calculated to assess internal consistency (i) within each multi-item construct, and (ii) across all individual items within each question. Values of α ≥ 0.70 were considered acceptable, indicating satisfactory internal consistency among items measuring the same underlying construct or among the overall item set, respectively. Corrected item-total correlations were also examined, with ≥ 0.30 indicating adequate item contribution to the overall item set [52–54].
All factor analyses were conducted in R statistical software (version 4.0.3) using the lavaan package, which offers comprehensive functions for both EFA and CFA analyses.
Ethical approval
This study was conducted in accordance with the principles outlined in the Declaration of Helsinki and adhered to all relevant national and institutional guidelines for research involving human participants. Ethical approval was obtained from IARC’s Ethics Committee (IEC 24–39 and IEC 25–25) and from Instituto Nacional de Saúde Doctor Ricardo Jorge (INSA 163/2024). Participation was voluntary, and informed consent was obtained from all participants prior to data collection. Data were collected anonymously and stored securely to ensure confidentiality. No identifying information was linked to the responses, and only aggregated results are reported.
Results
The preparation for the Delphi methods took place between late March and early April 2024. During this phase, experts provided feedback through open-ended questions. They offered general comments regarding the questionnaire structure, suggesting that it should start by assessing current beliefs, risk perceptions, and general awareness about cancer prevention; and end with questions directed toward the future: intention to change behaviours, self-efficacy to engage in cancer prevention behaviours, adherence to behaviours and awareness about the European Code Against Cancer. In addition, they recommended dropping the question on the construct information needs [Q15, Supplementary Table 2 (Additional file 1)], as it was considered outside the scope of the questionnaire. No further constructs were added.
Based on experts’ comments, two questions were deleted. One, under the construct knowledge about cancer prevention (Q8), was deemed too technical for the public, ambiguous, and potentially misleading. The other, under the construct of intention to change behaviours (Q11), was considered inadequate to capture respondents’ stage of change and was replaced with a new question assessing the same construct, and addressing specifically primary prevention behaviours. For Q6, items 3 and 4 were removed because experts considered them too technical for the public. Finally, one additional question was added to the construct adherence to behaviours as a potential alternative to the questions initially selected by the research team [Supplementary Table 2 (Additional file 1)].
Regarding the specific questions, the main comments and suggestions concerned the formulation of the questions and items, the response format, their order, and the response options.
Delphi method
Round 1
The Round 1 took place in early July 2024. Thirteen out of the 15 panellists (86.7% response rate) completed the de-identified online form. Most questions reached consensus (11/15 questions, 70%), with some response items slightly modified, mostly word changes, based on panellists’ suggestions. These questions were carried forward to the Final Round of the Delphi process.
Some questions, however, required further modifications as they did not reach consensus. Discrepancies arose mainly regarding the type of response (3/15 questions), response items (1/15), and the order of response items (1/15). Changes were made to incorporate panellists’ feedback. The revised version of these questions was included in Round 2 for reassessment.
Round 2
The Round 2 took place in late July 2024. The response rate of this round was lower than the previous with only eight panellists filling in the form (53.4%). Consensus was achieved for all the questions included (4/4 questions).
Round 3
The Final Round took place in August and early September 2024. Eleven out of the 15 panellists (73.3% response rate) completed the form. As shown in Supplementary Table 3 (Additional file 1), only one question (Q1) was directly excluded for not reaching at least a median score of 5. Three questions had a median Likert score of 7 (highest relevance) and, therefore, were prioritised among the rest (Q2, Q5, and Q15). All three questions were included in the top 5 most relevant questions ranking.
Eleven out of the 15 questions had a median score of 6. Among these questions, the four with the highest mean value (mean (x̄): 5.8) were prioritised (Q3, Q7, Q9, and Q12). Two of these were also included in the top 5 most relevant questions ranking. Following, we included the questions with the subsequent highest mean values (x̄= 5.7 for Q13 and x̄= 5.5 for Q8, respectively). Among questions with the same mean value (x̄= 5.4), Q6 was excluded because it had the highest IQR, indicating lower consensus among panellists. Finally, among questions with the same IQR, the one included in the top 3 least relevant ranking (Q10) was excluded. Only the 10 questions deemed most relevant were retained to limit the questionnaire length to 10 question stems or fewer. Among the excluded questions (Q1, Q6, Q10, Q11, and Q14) were those related to cancer screening (Q6 and Q10) [Supplementary Table 3 (Additional file 1)].
The resulting questionnaire of this Final Round had 10 questions (Table 2), including at least one question from each of the selected constructs that conforms our definition of ‘cancer prevention literacy’ (Fig. 2), except for self-efficacy to engage in cancer prevention actions, which was not included in the final questionnaire.
Fig. 2.
Selected constructs influencing behaviours that frame our definition of ‘cancer prevention literacy’
Readability test
All selected questions in the Final Round of the Delphi method showed good readability (Grade 10 or lower), with six out of the 10 questions considered appropriate for a reading level of a ten-year-old or younger. However, some questions contained hard-to-read sentences (e.g., passive voice) and difficult words and were therefore slightly above our initial threshold. These questions were pre-selected to be included in the cognitive interviews to ask respondents of the general public specific questions about the item to ensure that words are chosen adequately and are understood in the same way by both the research team and the respondent.
External expert review of clarity and appropriateness
The review took place from mid to late September 2024. From the 10 experts invited to participate in this phase, nine accepted with a response rate of 100%.
The I-FVI was calculated for clarity, comprehension and appropriateness for each of the questions. Out of the ten questions only three showed I-FVI scores over 0.75 for the three components, seven for clarity (70%), and three for comprehension (30%). Conversely, all questions showed high scores for I-FVI (> 0.75) for appropriateness, seven of them between 0.89 and 1.00, confirming their suitability for the target audience: adults from the general population. Those questions that did not meet the threshold for clarity and comprehension (7/10) were included in the cognitive interviews to confirm experts’ suspicions and ask the target audience how to improve these questions.
In line with the I-FVI results, we found that S-FVI/Ave of the overall questionnaire did not meet the threshold of 0.80 for clarity nor for comprehension (0.69 and 0.63, respectively), but showed a high S-FVI/Ave for appropriateness (0.89).
Cognitive interviews
Of the 16 participants invited, six agreed to take part in the interviews, with a mean age of 56.7 years (range: 26–78 years). The sample included two women and four men, all native English speakers, residing in France, Ireland, Portugal, Sweden, and the United Kingdom. Most participants (5/6) had completed higher education, while one reported secondary education. Their professional backgrounds were varied, including a housing officer, nurse, IT professional, medical statistician, osteopath, and homemaker.
The interviews lasted between 50 and 86 min. Among the eight questions asked at this stage, three were found to be hard or moderately difficult to understand by at least one participant, and two were somewhat difficult. In the question listing cancer risk factors (Q4, Table 2), some participants hesitated when responding to the negative statements (e.g. ‘Not eating enough whole grains’). However, no changes were made to these items, as the research team had previously agreed all items should convey the evidence for increased cancer risk, such as ‘Not eating enough whole grains’, rather than a positive statement. Participants did suggest adding examples to clarify certain food groups (e.g., legumes, processed meat) and harmful substances, and these suggestions were incorporated.
The question measuring intention to change behaviours (Q6, Table 2) caused confusion, as it was not clearly framed in the context of cancer prevention. Some participants reported engaging in these behaviours for other health reasons, making the intention to change ‘not applicable’. As a result, the question was revised to ‘To prevent cancer, I intend to make changes by:’ to better clarify its purpose.
Two questions on current adherence to healthy behaviours raised a similar issue (Q8 and Q9, Table 2). Many respondents were already engaging in healthy behaviours for reasons other than cancer prevention. To avoid losing this information, two additional items were added: ‘I do regularly at least one of the actions above but for reasons other than to prevent cancer (for example, general health, feeling good)’ and ‘There is nothing in particular I regularly do to prevent cancer’. These options were slightly adapted for the second question to fit its formulation.
Following a suggestion from the Delphi panel, participants were also asked whether they preferred the term ‘chance’ or ‘risk’ of getting cancer throughout the survey. There was a strong preference for ‘risk’, as ‘chance’ was associated with a positive connotation (e.g., winning the lottery), whereas ‘risk’ was seen as negative but potentially controllable. However, we decided not to replace ‘chance’ with ‘risk’, since the original questions including this term came from a validated questionnaire widely used in cancer prevention awareness [55].
Preliminary psychometric analysis
A total of 240 adults without a cancer diagnosis were included in the pre-test. The sociodemographic characteristics of the sample are summarised in Table 3.
Table 3.
Sociodemographic characteristics of the pre-test sample
| N (%) | ||
|---|---|---|
| Total | 240 | |
| Sex | Male | 116 (48.3) |
| Female | 124 (51.7) | |
| Age | 18–24 | 19 (7.9) |
| 25–34 | 31 (12.9) | |
| 35–44 | 39 (16.3) | |
| 45–54 | 48 (20.0) | |
| 55–64 | 42 (17.5) | |
| 65+ | 61 (25.4) | |
| Education attainment | No formal education | 4 (1.7) |
| Basic education | 23 (9.6) | |
| Lower secondary education | 74 (30.8) | |
| Upper secondary education | 87 (36.2) | |
| Higher education | 52 (21.7) | |
| Employment status | Employed | 123 (51.3) |
| Self-employed | 19 (7.9) | |
| Full-time homemaker | 21 (8.8) | |
| Student/unpaid work | 10 (4.2) | |
| Retired/Disabled | 44 (18.3) | |
| Disabled or too ill to work | 3 (1.2) | |
| Unemployed | 19 (7.9) | |
| Other | 4 (1.7) | |
| Community type | City | 76 (31.7) |
| Small or mid-sized town | 120 (50.0) | |
| Rural area | 44 (18.3) |
Beliefs about cancer (Q1, Table 2)
For most items, the average score was approximately 3, indicating that participants were generally uncertain about their beliefs about cancer, which may reflect indecision in their responses or a lack of understanding of the question. The internal consistency of the scale (Table 4), assessed using Cronbach’s alpha (α), was 0.62, indicating questionable reliability. Item-level α values, assessed after removing each item individually, were all slightly lower than the overall α, suggesting that excluding any individual item would not significantly improve the scale’s reliability.
Table 4.
Internal consistency
| Questions from Table 2 | N | Cronbach’s alpha (α) |
|---|---|---|
| Beliefs about cancer (Q1) | 240 | 0.62 |
| Risk perception (Q2 & Q3) | 240 | 0.66 |
| Awareness about cancer prevention (Q4) | 240 | 0.91 |
| Intention to change behaviour: uptake of cancer prevention interventions (Q7) | 240 | 0.85 |
| Adherence to cancer prevention behaviour (Q8 & Q9) | 240 | 0.82 |
Overall, the one-factor CFA model demonstrated poor fit, indicating that the model required refinement [Supplementary Table 4 (Additional file 1)]. Standardized factor loadings ranged from 0.55 to 0.63, falling within the recommended range of 0.50 to 0.95, and all were statistically significant (p < 0.001). However, the single latent construct hypothesised accounted for only 34% of the total variance across all items, and the composite reliability was 0.65, slightly below the conventional threshold of 0.70. Despite these modest psychometric indicators, we retained the question unchanged, as it is theoretically central to the instrument, consistent with the Integrative Model of Behavioural Prediction [23] and the Health Beliefs Model [25].
To enhance construct validity, the Likert-scale response option ‘Not sure’ (numeric value = 3) was revised to ‘Neither agree nor disagree’ to reduce ambiguity and minimise the likelihood that responses reflected misunderstanding of the question. This modification was applied to all questions assessing agreement using 5-point Likert-scale in the questionnaire.
Risk perception (Q2 & Q3, Table 2)
For both items considered, the average scores were approximately 3. For the first item, this indicates that, on average, respondents held a neutral stance, perceiving worry about developing cancer as occurring occasionally or with moderate frequency. For the second item, respondents, on average, perceived no substantial difference in their likelihood of developing cancer in the future compared to others. To improve clarity, the Likert-scale response option ‘Not sure’ was revised.
The Spearman’s (ρ) correlation between the two items was ρ = 0.45, indicating a moderate positive association, suggesting that the items are somewhat related but not strongly correlated. The internal consistency (α) of the two-item scale was estimated at 0.66, indicating borderline reliability (Table 4). Despite this, items were retained as they are key determinants of behavioural intentions within the Integrative Model of Behavioural Prediction [23] and therefore central to our definition of ‘cancer prevention literacy’.
Awareness about cancer prevention (Q4, Table 2)
Descriptive results highlighted lifestyle-related factors as the most recognised contributors to cancer risk. Among the 25 items, smoking was the most widely acknowledged, with approximately 75% of participants recognising its impact. In contrast, factors such as not breastfeeding, hormonal replacement therapy, and high salt intake were minimally recognised, with fewer than 17% of participants identifying them as risk factors. The scale demonstrated excellent internal consistency (α = 0.91, Table 4), which remained stable regardless of item exclusion, indicating that excluding any item would not improve overall reliability.
Although this question was not initially conceptualised as a multi-item scale, a supplementary EFA was conducted to explore potential underlying dimensions, identifying two latent constructs underlying the variation among the 25 items: the first represented unhealthy lifestyle behaviours, while the second reflected environmental and external factors affecting health (e.g., work-related exposures, Human Papillomavirus (HPV) infection, and second-hand smoke) [Supplementary Table 5 (Additional file 1)]. CFA subsequently validated the EFA-derived factor structure, with a good fit to the data, as all model fit indices met the recommended thresholds [Supplementary Table 6 (Additional file 1)].
Although five items were found to contribute minimally to the underlying factor structure, all items were retained to preserve the completeness of the questionnaire, which aims to evaluate awareness (recognition) of the risk factors included in the ECAC4 [16].
Intention to change behaviours: unhealthy behaviours (Q6, Table 2)
After excluding missing values (defined as ‘don’t know’ and ‘not applicable’ responses, which ranged from 16.7% to 29.6% across all items), correlations between participants’ intentions to change various unhealthy behaviours ranged from moderate to strong (ρ = 0.31–0.60). The strongest correlation was observed between quitting smoking and limiting alcohol consumption (ρ = 0.60), suggesting that participants who intended to quit smoking were also likely to limit alcohol consumption. In contrast, avoiding excessive sun exposure showed the lowest correlations with other items (ρ = 0.31–0.43), indicating that this behaviour was less strongly associated with other health-related intentions.
Overall, correlations were moderate to strong; however, the high proportion of ‘don’t know’ and ‘not applicable’ responses warrants caution, as these findings may not fully reflect participants’ true intentions. To improve reliability and reduce missing data, ‘Don’t know’ was replaced by ‘Not sure’ and an explanatory introductory statement was added: “Select ‘Not applicable’ only if you already practice one of the healthy behaviours listed below and do not need to change it (e.g., do not smoke or drink).”
Intention to change behaviours: uptake of cancer prevention interventions (Q7, Table 2)
The average scores for the two items were 3.6 and 3.7, respectively, indicating a generally positive attitude toward vaccinating children to prevent cancer, with responses leaning toward agreement rather than disagreement. This tendency was particularly pronounced for the item referring to the HPV vaccine. The correlation between the two items was ρ = 0.77, indicating a strong positive association. The internal consistency (α) of the two-item scale was estimated at 0.85, demonstrating good internal consistency (Table 4).
Adherence to cancer prevention behaviours (Q8 & Q9, Table 2)
Adherence to dietary recommendations was the most common and consistently practiced cancer prevention behaviour (41%), followed by smoking cessation (35%), reduced alcohol intake (34%), and sun avoidance (32%). Similarly, over the past year, adopting a healthy diet remained the most frequent behaviour, while fewer participants reported newly initiating other behaviours, likely because many had already adopted them.
Correlations between regular (Q8) and recent (past 12 months, Q9) behaviours were low to moderate (ρ = 0.37–0.50), with the strongest relationship for smoking cessation/reduction. Within both time frames, correlations among behaviours were also low to moderate, indicating that changes in one behaviour were often accompanied by changes in others, though associations were weak.
The scale showed good internal consistency (α = 0.82, Table 4), stable regardless of item exclusion. While EFA identified a single underlying latent factor where all 12 items showed moderate to strong association with the single factor (factor loadings > 0.4) [Supplementary Table 7 (Additional file 1)]. CFA indicated only moderate fit to the data, with a relatively low proportion of the total variance across all 12 items explained by the latent factor, suggesting that further refinement might be needed [Supplementary Table 8 (Additional file 1)]. However, we decided to retain these two questions in their original form, given that they were derived from a previously validated longitudinal questionnaire.
The final Cancer Prevention Literacy Questionnaire (CPL-Q) is a 10-question validated instrument comprising 58 items (Additional file 3), using the verbatim wording of the recommendations of ECAC4 where applicable, and forms the basis of the ‘cancer prevention literacy’ instrument for the World Code Against Cancer Framework.
Discussion
The CPL-Q as a first-generation ‘cancer prevention literacy’ measurement tool
The CPL-Q is a 10-question instrument developed through an iterative Delphi process with strong expert agreement, supported by evidence of content and face validity indicating that it is comprehensible, relevant, and appropriate for its intended audience. Psychometric testing using EFA and CFA offers preliminary support for construct validity, with initial evidence suggesting acceptable reliability.
While the CPL-Q was developed using established methodological approaches and has demonstrated overall good validity, some constructs, particularly beliefs about cancer and risk perception, showed explained variance below recommended thresholds and only modest reliability. These constructs may, therefore, be best interpreted as brief indices rather than fully developed latent scales at this stage. This highlights the complexity to operationalise certain ‘cancer prevention literacy’ constructs and suggests they may benefit from further refinement, expansion and evaluation in larger and more diverse samples.
Particular attention was given to the interpretation of ‘don’t know’ and ‘not applicable’ responses. Rather than treating these responses as random missing data, they are conceptualised as informative indicators of uncertainty, lack of knowledge, or irrelevance due to current behaviour; dimensions that are meaningful for assessing ‘cancer prevention literacy’. This approach supports more consistent scoring, enhances comparability over time, and is expected to improve the reliability and validity of the instrument in future applications. Moreover, systematic differences in ‘don’t know’ and ‘not applicable’ responses by education level or socioeconomic status may themselves reflect disparities in 'cancer prevention literacy' and should therefore be considered in future analysis and equity-focused research.
Future research should therefore prioritise further validation in larger and more heterogeneous populations, including assessments of measurement invariance across social groups and countries. Additional work is needed to explore the CPL-Q’s relationship with health inequalities and other components of health literacy, as well as its predictive validity in relation to behavioural change and cancer-related health outcomes. Such efforts will be essential to strengthen the utility of the CPL-Q for population monitoring and policy-relevant research.
Taken together, these considerations support viewing the CPL-Q as a well-founded first-generation instrument designed to capture a broad range of ‘cancer prevention literacy’ constructs, rather than a fully psychometrically finalised questionnaire.
Implications for cancer prevention research and policy in the EU
The CPL-Q addresses an important gap in cancer prevention and health literacy research as, to our knowledge, it is the first questionnaire specifically designed to comprehensibly assess ‘cancer prevention literacy’, understood as an overarching term encompassing several interrelated constructs. While health literacy instruments exist [56–59], few are tailored specifically to cancer prevention, and even fewer have been validated. Most focus on beliefs [60], recognition of risk factors, signs and symptoms, and help-seeking [55, 61, 62], or on specific prevention interventions, such as vaccination or screening, usually addressed independently [63, 64] or by cancer type [64–66].
Compared with existing tools, the CPL-Q is distinct in offering a multidimensional approach that integrates constructs from behavioural change models. It provides a comprehensive measure of beliefs, risk perception, awareness, knowledge, intentions to change, adherence, and other behaviours relevant to cancer prevention, while using the verbatim wording of evidence-based risk factors, behaviours and interventions from ECAC4. This broader scope aligns with recent calls for health literacy measures that capture the complexity of decision-making and behaviour change in cancer prevention.
This first-generation validated questionnaire is expected to serve several functions. First, it offers an easy-to-adapt instrument to measure the impact of different Regional Codes Against Cancer in adults from the general population, under the World Code Against Cancer Framework, thereby contributing to the monitoring and benchmarking of changes in ‘cancer prevention literacy’. Since its inception in the 1980s, few studies have evaluated the impact of the ECAC at the EU level [19, 67, 68], and this questionnaire may facilitate such monitoring in future editions, subject to minor adaptations for new versions. Second, it may serve as a practical tool for policymakers and practitioners seeking to identify gaps in ‘cancer prevention literacy’ within specific populations, thus guiding the design of targeted education and awareness campaigns. Third, it offers a standardised measure that can facilitate international comparisons and longitudinal studies, contributing to the evidence base for cancer prevention policies.
Strengths and limitations
This study has several strengths. First, it addresses a recognised gap by providing a first-generation theory-informed validated questionnaire specifically designed to assess ‘cancer prevention literacy’ in the general population without a prior cancer diagnosis. Second, the rigorous scientific approach underpinning its development enhances its credibility and applicability, including a systematic review of existing instruments, structured expert consensus through the Delphi method, and multiple layers of validation. The participation of a heterogeneous panel of international experts from both Europe and Latin America ensured broad representation and contextual relevance, strengthening the generalisability of the findings. Another strength is the use of established theoretical models to guide the selection of constructs, ensuring that the questionnaire captured the multidimensional nature of ‘cancer prevention literacy’ in a systematic and evidence-based manner. In addition, cognitive testing and readability assessments helped ensure that the questionnaire is understandable and appropriate for the target audience, while psychometric analyses provided initial evidence of construct validity. Although originally developed taking ECAC4 as a basis and currently being applied in several EU Member States, it has been designed for easy adaptation to the next editions of the ECAC [17] and to other geographic and cultural contexts under the World Code Against Cancer Framework. At the time of publication, the CPL-Q for the Latin America and the Caribbean Code Against Cancer [18] will be available at https://cancer-code-world.iarc.who.int/literacy and soon for the forthcoming edition of the ECAC.
Some limitations should also be acknowledged. Although the Delphi panel included experts from multiple countries and disciplines, participation rates varied across rounds, which may have influenced the range of perspectives captured. Cognitive testing was conducted in a small sample of highly educated individuals and was limited to eight of the 10 questions, prioritising those with unresolved face validity issues and higher-grade levels due to resource constraints. This may limit the questionnaire’s suitability for populations with lower literacy. Further testing is recommended when using the CPL-Q in such groups. In addition, some items, including previously validated ones, may benefit from further wording refinement over time to better adapt to population needs.
The pilot testing involved 240 adults recruited from online panels with relatively high educational attainment, which may limit the generalisability of the findings, as the sample may not fully represent the socioeconomic, cultural, and linguistic diversity of the EU population. Although the sample size was below the traditional 10:1 participant-to-item ratio for the awareness about cancer prevention question (Q4, Table 2), the EFA for this question was conducted as a preliminary analysis and would benefit from replication in larger samples to ensure fully stable solutions based on polychoric correlations. Nevertheless, the analysis is justified according to contemporary psychometric guidance, which recognises that sample size adequacy depends on multiple factors beyond participant-to-item ratios, including communalities, factor loadings, and model complexity, and given the moderate to strong factor loadings observed in this study [69].
Despite explained variance below recommended threshold and only modest reliability, item retention for the beliefs about cancer and risk perception constructs was methodologically justified, as all factor loadings (0.55–0.63) exceeded established thresholds [48, 70] and each item in these brief scales represents a theoretically essential component of the construct. Another limitation was that FA was not performed for Q6 (Table 2), given the relatively high number of ‘don’t know’ and ‘not applicable’ responses. Yet, while descriptive analyses still provided relevant information regarding intention to change behaviours, future studies with larger samples may allow further validation of this question and a more robust assessment of how these responses can be handled (e.g., stratified analyses, modelling ‘don’t know’ and ‘not applicable’ responses as separate response categories).
Beyond item-specific consideration, other potential sources of bias should also be considered when interpreting pilot testing results and in future applications of the questionnaire, including mode effects, social desirability and recall bias in self-reported behaviours, and cross-cultural or translation equivalence. Finally, based on Delphi panellist prioritisation, the CPL-Q questions focus on general cancer prevention, risk and protective factors, unhealthy behaviours, and vaccination; therefore, it does not capture literacy related to cancer screening, which may be addressed in future adaptations or expansions of the questionnaire.
Conclusion
This study developed and validated a first-generation, theory-informed questionnaire to assess ‘cancer prevention literacy’ among adults with no prior cancer diagnosis, within the World Code Against Cancer Framework. Unlike existing tools, it captures multiple dimensions –not only including awareness of risk factors but also beliefs about cancer, risk perception, knowledge about cancer prevention, intentions to change behaviours, and adherence to behaviours– providing a comprehensive, evidence-based instrument. By addressing a critical gap in existing instruments and providing evidence of content and face validity, as well as promising construct validity, this questionnaire offers a valuable resource to support research, guide policymakers in identifying literacy gaps, and enable international comparisons and longitudinal studies. Ultimately, it has the potential to contribute to more effective and equitable cancer prevention strategies worldwide.
Supplementary Information
Acknowledgements
We would like to thank Silvina Arrossi, Centro de Estudios de Estado y Sociedad, Argentina; Fernanda de Albuquerque Melo Nogueira, Instituto Nacional de Câncer (INCA), Brazil; Catterina Fereccio, Pontificia Universidad Católica de Chile, Chile; Carolyn Finck, Universidad Los Andes, Colombia; Patricia J Garcia, Universidad Cayetano Heredia, Perú; Marius Geantă, INOMED, Romania; Rolando Herrero, Agencia Costarricense de Investigaciones Biomédicas, Costa Rica; Mariantonia Lemos, Universidad EAFIT, Colombia; Ana Molina, FISABIO, Spain; Raul Murillo, Pontificia Universidad Javeriana, Colombia; Dafina Petrova, Escuela Andaluza de Salud Pública, Spain; Marina Pinto, FISABIO, Spain; Ana Santos, Universidade de Aveiro, Portugal; Mafalda Uva, Instituto Nacional de Saúde Doutor Ricardo Jorge, Portugal; Loredana Vlado, Eurocomunicare, Romania for their participation in the Delphi study as panellists; and Tilman Brand, Leibniz Institute for Prevention Research and Epidemiology - BIPS, Germany; Erica D’Souza, IARC, France; Isabel Garcés, Universidad de Antioquia, Colombia; Kate Hamilton-West, Cancer Research UK, the United Kingdom; Isabel Mosquera, IARC, France; Kimberley Neve, Cancer Research UK, the United Kingdom; Sathishrajaa Palaniraja, IARC, France; David Ritchie, IARC, France; Stephan Van den Broucke, Université catholique de Louvain, Belgium; Hajo Zeeb, Leibniz Institute for Prevention Research and Epidemiology - BIPS, Germany for their input in the face-validity assessment. Where authors are identified as personnel of the International Agency for Research on Cancer/World Health Organization, the authors alone are responsible for the views expressed in this article, and they do not necessarily represent the decisions, policies, or views of the International Agency for Research on Cancer/World Health Organization.
Abbreviations
- CFA
Confirmatory Factor Analysis
- CPL-Q
Cancer Prevention Literacy Questionnaire
- ECAC
European Code Against Cancer
- ECAC4
European Code Against Cancer, 4th edition
- EFA
Exploratory Factor Analysis
- EU
European Union
- FA
Factor Analysis
- HPV
Human Papillomavirus
- IARC
International Agency for Research on Cancer
- I-FVI
Item-level Face Validity Index
- IQR
Interquartile Range
- S-FVI
Scale-level Face Validity Index
- WHO
World Health Organization
Authors’ contributions
Ariadna Feliu: Conceptualisation, Methodology, Formal analysis, Investigation, Resources, Data curation, Visualisation and Writing-Original Draft, Project administration. Rubana Islam: Methodology, Formal analysis, Investigation, Resources, Data curation, Writing-Review & Editing; Paula Romeo-Cervera: Methodology, Formal analysis, Investigation, Resources, Data curation, Writing-Review & Editing; Liacine Bouaoun: Formal analysis, Writing-Review & Editing; Victoria Whitelock: Validation, Writing-Review & Editing; Mirela Pirvan: Investigation; Luis Roxo: Formal analysis, Writing-Review & Editing; Ana Santos: Methodology, Investigation, Validation, Writing-Review & Editing; and Carolina Espina: Conceptualisation, Methodology, Formal analysis, Investigation, Resources, Data curation, Writing-Review & Editing, Project administration, Funding acquisition. Ariadna Feliu and Carolina Espina are the guarantors of the study and accept full responsibility for the integrity of the data and the accuracy of the data analysis and interpretation.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:
Funding from the HORIZON-MISS-2022-CANCER-01-01 (Project ID 101104432) as part of the 4P-CAN project has been used to conduct the cognitive interviews, the pre-test and the psychometric analysis; funding from EU4Health programme under Grant Agreement No. 101075240 as part of the ECAC project has supported part of the research team. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them.
Data availability
The datasets generated and analysed during the current study available from the corresponding author on reasonable request.
Declarations
Ethical approval and consent to participate
The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the International Agency for Research on Cancer (IARC/WHO) Ethics Committee (IEC 24–39 and IEC 25–25) and by the Instituto Nacional de Saúde Doctor Ricardo Jorge (INSA) (163/2024).
All subjects gave their written informed consent prior to enrolment in the study.
Consent for publication
Not applicable, as all data were de-identified prior to analysis and no individual details are presented.
Competing interests
The authors declare no competing interests.
Footnotes
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Change history
5/19/2026
Following the article's publication, an error was found in Supplementary Material 3. This has been corrected.
References
- 1.World Health Organization. Health literacy. Geneva: WHO; 2024. Available from: https://www.who.int/news-room/fact-sheets/detail/health-literacy. Cited 3 Sept 2025.
- 2.Diviani N, Schulz PJ. What should laypersons know about cancer? Towards an operational definition of cancer literacy. Patient Educ Couns. 2011;85(3):487–92. [DOI] [PubMed] [Google Scholar]
- 3.EUR/RC69/14 Rev. 1 Copenhagen, 16–19 September 2019: draft WHO European roadmap for implementation of health literacy initiatives through the life course. Available from: https://www.who.int/europe/publications/i/item/EUR-RC69-14Rev.1. Cited 15 Aug 2025.
- 4.Powers BJ, Trinh JV, Bosworth HB. Can this patient read and understand written health information? JAMA. 2010;304(1):76–84. [DOI] [PubMed] [Google Scholar]
- 5.Ferlay J, Ervik M, Lam F, Laversanne M, Colombet M, Mery L, et al. Global Cancer Observatory: Cancer Today. Lyon, France. 2024. Available from: https://gco.iarc.fr/today/en. Cited 1 Oct 2024.
- 6.Ferlay J, Laversanne M, Ervik M, Lam F, Colombet M, Mery L, et al. Cancer Tomorrow. Global Cancer Observatory. Lyon, France. 2024. Available from: https://gco.iarc.who.int/tomorrow. Cited 8 Jan 2025.
- 7.Cabasag CJ, Vignat J, Ferlay J, Arndt V, Lemmens V, Praagman J, et al. The preventability of cancer in europe: A quantitative assessment of avoidable cancer cases across 17 cancer sites and 38 countries in 2020. Eur J Cancer. 2022;177:15–24. [DOI] [PubMed] [Google Scholar]
- 8.EuropeanCommission. Europe’s Beating Cancer Plan [Internet]. Luxembourg: Publications Office of the European Union; 2021. Available from: https://health.ec.europa.eu/system/files/2022-02/eu_cancer-plan_en_0.pdf. Cited 13 Feb 2026.
- 9.Fleary SA, Paasche-Orlow MK, Joseph P, Freund KM. The relationship between health Literacy, cancer prevention Beliefs, and cancer prevention behaviors. J Cancer Educ. 2019;34(5):958. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Morris NS, Field TS, Wagner JL, Cutrona SL, Roblin DW, Gaglio B, et al. The association between health literacy and cancer-related attitudes, behaviors, and knowledge. J Health Commun. 2013;18(Suppl 1):223–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Stormacq C, Van Den Broucke S, Wosinski J. Does health literacy mediate the relationship between socioeconomic status and health disparities? Integrative review. Health Promot Int. 2019;34(5):E1–17. [DOI] [PubMed] [Google Scholar]
- 12.Hollands GJ, French DP, Griffin SJ, Prevost AT, Sutton S, King S, et al. The impact of communicating genetic risks of disease on risk reducing health behaviour: systematic review with meta-analysis. BMJ (Online). 2016;352:i1102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.International Agency for Research on Cancer. European Code Against Cancer: 12 ways to reduce your cancer risk. Lyon: IARC; 2014. Available from: https://cancer-code-europe.iarc.fr/index.php/en/. Cited 20 May 2022.
- 14.International Agency for Research on Cancer. World Code Against Cancer Framework. Lyon: IARC; 2022 Available from: https://cancer-code-world.iarc.who.int/. Cited 5 Mar 2025.
- 15.Espina C, Ritchie D, Feliu A, Canelo-Aybar C, D’Souza E, Mitrou PN, et al. Developing evidence-based cancer prevention recommendations: methodology of the world code against cancer framework to create region-specific codes. Int J Cancer. 2026;158(1):9–18. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Schüz J, Espina C, Villain P, Herrero R, Leon ME, Minozzi S, et al. European code against cancer 4th edition: 12 ways to reduce your cancer risk. Cancer Epidemiol. 2015;39:S1–10. [DOI] [PubMed] [Google Scholar]
- 17.Espina C, Ritchie D, Riboli E, Kromhout H, Franceschi S, Lansdorp-Vogelaar I, et al. European code against cancer 5th edition: 14 ways you can help prevent cancer. Lancet Reg Health – Europe. 2026;63:101592. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Espina C, Feliu A, Maza M, Almonte M, Ferreccio C, Finck C, et al. Latin America and the Caribbean code against cancer 1st edition: 17 cancer prevention recommendations to the public and to policy-makers (World code against cancer Framework). Cancer Epidemiol. 2023;86(Suppl 1):102402. [DOI] [PubMed] [Google Scholar]
- 19.Ritchie D, Mallafré-Larrosa M, Ferro G, Schüz J, Espina C. Evaluation of the impact of the European code against cancer on awareness and attitudes towards cancer prevention at the population and health promoters’ levels. Cancer Epidemiol. 2021;71:101898. [DOI] [PubMed] [Google Scholar]
- 20.Lemos M, Restrepo J, Espina C, Feliu A, Ferreccio C, Garcés-Palacio IC, et al. Latin America and the Caribbean code against cancer 1st edition: formative research on the comprehension and persuasiveness of the recommendations by the general population. Cancer Epidemiol. 2023;86(Suppl 1):102456. [DOI] [PubMed] [Google Scholar]
- 21.Mantzari E, Brain K, Bessems K, Bigaard J, Bouaoun L, D’Souza E, et al. Optimising the European code against Cancer, 5th edition to increase awareness of avoidable cancer risks in all socioeconomic groups: impact of message content, length and format. Mol Oncol. 2026;20(1):154–69. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Espina C, Yared W, Ritchie D, Lipponen S, Anttila A, Tamminiemi K, et al. Sustainability and monitoring of the European code against cancer: recommendations. Cancer Epidemiol. 2021;72:101933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Yzer M. The Integrative Model of Behavioral Prediction as a Tool for Designing Health Messages. In: Cho H, editor. Health communication message design: Theory and practice. Thousand Oaks (CA): Sage Publications; 2012. p. 21–40.
- 24.Aithal A, Aithal S. Development and validation of survey questionnaire and experimental Data–A systematical Review-based statistical approach. IJMTS. 2020;5(2):233–51. [Google Scholar]
- 25.Becker MH. The health belief model and sick role behavior. Health Educ Monogr. 1974;2(4):409–19. [Google Scholar]
- 26.Jayanti RK, Burns AC. The antecedents of preventive health care behavior: an empirical study. J Acad Mark Sci. 1998;26(1):6–15. [Google Scholar]
- 27.Simmons RA, Cosgrove SC, Romney MC, Plumb JD, Brawer RO, Gonzalez ET, et al. Health literacy: cancer prevention strategies for early adults. Am J Prev Med. 2017;53(3):S73–7. [DOI] [PubMed] [Google Scholar]
- 28.Ferrer RA, Klein WMP, Persoskie A, Avishai-Yitshak A, Sheeran P. The tripartite model of risk perception (TRIRISK): distinguishing Deliberative, Affective, and experiential components of perceived risk. Annals of behavioral medicine. Ann Behav Med. 2016;50(5):653–63. [DOI] [PubMed] [Google Scholar]
- 29.Lykins ELB, Graue LO, Brechting EH, Roach AR, Gochett CG, Andrykowski MA. Beliefs about cancer causation and prevention as a function of personal and family history of cancer: a national, population-based study. Psychooncology. 2008;17(10):967–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Trevethan R. Deconstructing and assessing knowledge and awareness in public health research. Front Public Health. 2017;5:194. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Uribe FAR, De Souza Godinho RC, Machado MAS, Da Silva Gonçalves Oliveira KR, Espejo CAN, De Sousa NCV, et al. Health knowledge, health behaviors and attitudes during pandemic emergencies: A systematic review. PLoS ONE. 2021;16(9):e0256731. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Fishman J, Yang C, Mandell D. Attitude theory and measurement in implementation science: a secondary review of empirical studies and opportunities for advancement. Implement Sci. 2021;16(1):1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Witte K. Putting the fear back into fear appeals: the extended parallel process model. Commun Monogr. 1992;59(4):329–49. [Google Scholar]
- 34.Howren MB. Adherence. In: Gellman MD, Turner R, editors. Encyclopedia of behavioral medicine. New York: Springer; 2013. pp. 33–9. [Google Scholar]
- 35.Afzal W. The specification of the conceptual domain of information need. J Librariansh Inf Sci. 2025;0(0).
- 36.Avella JR. Delphi panels: research design, procedures, advantages, and challenges. Int J Doctoral Stud. 2016;11:305–21. [Google Scholar]
- 37.Spranger J, Homberg A, Sonnberger M, Niederberger M. Reporting guidelines for Delphi techniques in health sciences: A methodological review. Z Evid Fortbild Qual Gesundhwes. 2022;172:1–11. [DOI] [PubMed] [Google Scholar]
- 38.Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9. [DOI] [PubMed] [Google Scholar]
- 39.Shang Z. Use of Delphi in health sciences research: A narrative review. Medicine. 2023;102(7):e32829. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Van den Wyngaert I, Van Pottelbergh G, Coteur K, Vaes B, Van den Bulck S. Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method. BMC Health Serv Res. 2024;24(1):433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Heuzenroeder L, Ibrahim F, Khadka J, Woodman R, Kitson A. A Delphi study to identify content for a new questionnaire based on the 10 principles of dignity in care. J Clin Nurs. 2022;31(13–14):1960–71. [DOI] [PubMed] [Google Scholar]
- 42.de Souza AC, Alexandre NMC, de Guirardello E. Psychometric properties in instruments evaluation of reliability and validity. Epidemiologia E Serviços De Saúde. 2017;26(3):649–59. [DOI] [PubMed] [Google Scholar]
- 43.Allen MS, Robson DA, Iliescu D. Face validity: A critical but ignored component of scale construction in psychological assessment. Eur J Psych Assess. 2023;39(3):153–6. [Google Scholar]
- 44.McHugh RK, Behar E. Readability of Self-Report measures of depression and anxiety. J Consult Clin Psychol. 2009;77(6):1100–12. [DOI] [PubMed] [Google Scholar]
- 45.Yusoff MSB. ABC of content validation and content validity index calculation. Educ Med J. 2019;11(2):49–54. [Google Scholar]
- 46.Tourangeau R. Cognitive Science and Survey Methods: A Cognitive Perspective. In: Jabine T, Tanur J, Tourangeau R, editors. Cognitive Aspects of Survey Methodology Building a Bridge between Disciplines. Washington DC: National Academy Press; 1984. p. 73–100. Available from: https://www.scirp.org/reference/referencespapers?referenceid=2896468. Cited 22 Sept 2025.
- 47.Tavakol M, Wetzel A. Factor analysis: a means for theory and instrument development in support of construct validity. Int J Med Educ. 2020;11:245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best practices for developing and validating scales for Health, Social, and behavioral research: A primer. Front Public Health. 2018;6:149. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Lorenzo-Seva U, Ferrando PJ. Determining sample size requirements in EFA solutions: A simple empirical proposal. Multivar Behav Res. 2024;59(5):899–912. [DOI] [PubMed] [Google Scholar]
- 50.DeVellis RF, Thorpe CT. Scale development: theory and applications. 5th ed. Washington DC: SAGE Publications Ltd; 2021. [Google Scholar]
- 51.Field A. Discovering statistics using IBM SPSS statistics. 6th ed. Washington DC: SAGE Publications Ltd; 2024. [Google Scholar]
- 52.Streiner DL, Norman GR, Cairney J. Health measurement scales: A practical guide to their development and use. 6th ed. Oxford: Oxford University Press; 2024. [Google Scholar]
- 53.Kline P. A handbook of test construction. A handbook of test construction (Psychology Revivals). London, UK: Methuen; 1986. [Google Scholar]
- 54.Ghiselli EE, Campbell JP, Zedeck S. Measurement theory for the behavioral sciences. San Francisco, CA: W.H. Freeman; 1981. [Google Scholar]
- 55.Stubbings S, Robb K, Waller J, Ramirez A, Austoker J, Macleod U, et al. Development of a measurement tool to assess public awareness of cancer. Br J Cancer. 2009;101(Suppl 2):S13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.M-POHL-WHO Action Network on Measuring Population and Organizational Health Literacy. Health Literacy Survey. 2019. Available from: https://m-pohl.net/HLS19. Cited 18 Aug 2025.
- 57.Osborne RH, Batterham RW, Elsworth GR, Hawkins M, Buchbinder R. The grounded psychometric development and initial validation of the health literacy questionnaire (HLQ). BMC Public Health. 2013;13:658. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Liu H, Zeng H, Shen Y, Zhang F, Sharma M, Lai W, et al. Assessment tools for health literacy among the general population: A systematic review. Int J Environ Res Public Health. 2018;15(8):1711. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Tavousi M, Mohammadi S, Sadighi J, Zarei F, Kermani RM, Rostami R, et al. Measuring health literacy: A systematic review and bibliometric analysis of instruments from 1993 to 2021. PLoS ONE. 2022;17(7):e0271524. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Shahab L, McGowan JA, Waller J, Smith SG. Prevalence of beliefs about actual and mythical causes of cancer and their association with socio-demographic and health-related characteristics: findings from a cross-sectional survey in England. Eur J Cancer. 2018;103:308–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Simon AE, Forbes LJL, Boniface D, Warburton F, Brain KE, Dessaix A, et al. An international measure of awareness and beliefs about cancer: development and testing of the ABC. BMJ Open. 2012;2(6):e001758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Cancer Research UK. Cancer Awareness Measure ‘Plus’ (CAM+). London: CRUK; 2024. Available from: https://www.cancerresearchuk.org/sites/default/files/cam_2024_-_final_for_web_page.pdf. Cited 18 Aug 2025.
- 63.Waller J, Ostini R, Marlow LAV, McCaffery K, Zimet G. Validation of a measure of knowledge about human papillomavirus (HPV) using item response theory and classical test theory. Prev Med (Baltim). 2013;56(1):35–40. [DOI] [PubMed] [Google Scholar]
- 64.Marlow LAV, Waller J, Wardle J. Parental attitudes to pre-pubertal HPV vaccination. Vaccine. 2007;25(11):1945–52. [DOI] [PubMed] [Google Scholar]
- 65.Simon AE, Wardle J, Grimmett C, Power E, Corker E, Menon U, et al. Ovarian and cervical cancer awareness: development of two validated measurement tools. J Family Plann Reproductive Health Care. 2012;38(3):167–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Power E, Simon A, Juszczyk D, Hiom S, Wardle J. Assessing awareness of colorectal cancer symptoms: measure development and results from a population survey in the UK. BMC Cancer. 2011;11:366. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Commission of the European Communities. Europeans and the Prevention of Cancer. Degree of awareness of the programme and the European Code against Cancer: Attitudes and behaviour with regard to the rules in the Code. Brussels: Commission of the European Communities; 1988.
- 68.Commission of the European Communities. Eurobarometer 49: Europeans and their Views on Cancer. Brussels: Commission of the European Communities; 1998.
- 69.Mundfrom DJ, Shaw DG, Ke TL. Minimum sample size recommendations for conducting factor analyses. Int J Test. 2005;5(2):159–68. [Google Scholar]
- 70.Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res. 2018;27(5):1159–70. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The datasets generated and analysed during the current study available from the corresponding author on reasonable request.


