Development and validity testing of the Adolescent Health Literacy Questionnaire (AHLQ): Protocol for a mixed methods study within the Irish school setting

Abstract

Introduction

Health literacy research has focused predominantly on the adult population, and much less is understood about this concept from an adolescent perspective. The tools currently available to measure adolescent health literacy have been adapted from adult versions. This limits their applicability to young people because of the developmental characteristics that impact on adolescents’ behaviour, including impulse control and judgement skills. This protocol describes the intended development and validity testing of a questionnaire to measure health literacy in adolescents.

Methods and analysis

This protocol describes this mixed methods study that has three phases: the first phase will involve grounded research with adolescents using qualitative group interviews, co-design and concept mapping workshops to understand what health and healthy behaviours mean to adolescents and to explore their health literacy needs and the potential domains for the questionnaire. The draft health literacy domains identified will be presented to the youth advisory panel, and the questionnaire will be altered based on their feedback. Cognitive pretesting of the questionnaire items will also be conducted. Phase 2 will involve piloting the questionnaire to a two-stage random sample of young people in five urban and rural schools in Ireland. Test–retest reliability will be conducted using Pearson correlation coefficient. Confirmatory factor analysis will also be conducted to analyse the psychometric properties of the questionnaire. Phase 3 will involve the questionnaire being rolled out to a nationally representative sample of adolescents (n=6052) in Ireland to assess their levels of health literacy.

Ethics and dissemination

Ethical approval to conduct this study has been granted from the University College Dublin Human Research Ethics Committee – Sciences (LS-20–08). Informed assent from adolescents and informed consent from parents/guardians will be sought. The findings of this research will be disseminated at national and international conferences, as well as through publication in peer-reviewed journals.

Strengths and limitations of this study.

• A strength of this study is that it takes a grounded approach to understanding health literacy needs and health behaviours of young people using co-design workshops and group interviews and shifts beyond viewing health literacy from a functional perspective to one that incorporates interactive and critical health literacy.

• The questionnaire developed will be specifically targeted for use with young people and will be context and content specific.

• A potential limitation of this study is that the validity of the questionnaire will be assessed using a cross-sectional study, thereby not allowing us to track differences in the health literacy of participants over time. Interventions developed using the OPtimising HEalth LIteracy and Access approach would seek to use the Adolescent Health Literacy Questionnaire as a pre–post assessment tool to see if health literacy improvements occur among adolescents.

Introduction

Health literacy has been defined as being ‘linked to literacy and entails people’s knowledge, motivation and competences to access, understand, appraise, and apply health information in order to make judgments and take decisions in everyday life concerning healthcare, disease prevention and health promotion to maintain or improve quality of life during the life course’.¹ Understandings of health literacy have evolved from focusing on a person’s ability to read and comprehend health information, to understandings that encompass knowledge, attitudes, personal skills, self-efficacy and behavioural intentions relating to health.^2–4 It is a personal and empowering resource for individuals that allows them to make informed health decisions in everyday life.^{1 5–7} Individual characteristics, including age and gender, as well as contextual factors, such as the sociopolitical environment, can impact on health literacy. Limited health knowledge, poorer health outcomes and inappropriate use of healthcare services are associated with low health literacy.^{6 8} The WHO has been engaged in a range of actions to promote health literacy as an enabling factor in promoting health in a whole-of-society approach.^9–11 Specifically, the WHO has emphasised the important role improved health literacy can make in the prevention and reduction of non-communicable diseases, which have been exacerbated by factors including globalisation, urbanisation and growing health inequities.^{11 12}

Despite the establishment of health behaviours in adolescence, there is a paucity of health literacy research specifically focusing on child and adolescent populations. Until relatively recently, health literacy research has predominantly been conducted with adult populations, while adolescent groups have been largely neglected, with some notable exceptions.1 3 5 6 13–19 Adolescence is a time where life-long health behaviours are being developed, therefore potentially providing a key juncture for health literacy interventions to improve health behaviours and outcomes into adulthood.^{16 20} The few studies that have investigated adolescent health literacy have shown that low health literacy is associated with risky behaviours,^{21 22} obesity^{23 24} and poorer health behaviours.²⁵ There is a lack of an appropriate tool to measure health literacy in adolescents that uses grounded qualitative approaches with young people to determine key attributes of and acknowledge health literacy as a latent construct.²⁶ One tool that was specifically developed for use in adolescents and treats health literacy as a multidimensional and latent concept is the Health Literacy for School-aged Children (HLSAC).²⁷ The HLSAC was developed for use in Finnish, Polish, Slovakian and Belgian adolescents and represents a promising tool for comparing health literacy across international contexts,²⁷ but the applicability of the tool has yet to be tested. Additionally, this tool was not developed following a grounded co-design process with young people and does not generate a set of scale scores that map the potential mechanisms related to health literacy strengths and weaknesses that are required for the development of interventions within the OPtimising HEalth LIteracy and Access (Ophelia) process. To date, four systematic reviews have considered the measurement of child and adolescent health literacy.5 6 14 16 However, many of the current adolescent health literacy measures have been adapted from adult versions, which many have argued is inappropriate given that health literacy in adolescents is likely to have very different elements and changes as the individual matures.^{15 16 28} Consequently, several authors have stated that the measurement of health literacy is lacking in quality.^{16 29–31} This then represents a severe weakness in a measure termed construct under-representation.^{32 33}

Adolescent health literacy is distinct from adult health literacy5 6 14 16 and needs to be adapted to the needs of a specific population in various contexts. To ensure relevance for specific populations or contexts, research needs to be conducted with core health promotion principles in mind, including research as empowering, participative, holistic and equitable. There has been little empirical work on how adolescent health literacy may be defined, or frameworks that may be used to guide the development of an adolescent health literacy measure. A recent systematic review by Bröder et al¹⁴ noted that there are 12 variations in how health literacy has been defined in children and adolescents, varying from those tailored towards a life course perspective or a settings approach. Just two of the studies included in this review by Bröder et al^{28 34} involved children and adolescents in the development and conceptualisation of health literacy through focus group discussions. Three core dimensions of health literacy for children and adolescents were identified in the systematic review by Bröder et al¹⁴: cognitive, behavioural/operational and affective attributes. This review also identified affective attributes including self-awareness, self-regulation, self-efficacy and motivation as core dimensions of health literacy.¹⁴ The review concluded that there is a wide variety of definitions of health literacy that provides a starting point to consider and describe this multidimensional construct.¹⁴ It is important to consider however that these constructs have been developed based on the extant literature from a limited range of cultural contexts and settings. Therefore, additional domains may exist that have not yet been articulated.

The challenge is to develop a questionnaire for adolescents that will serve the context and needs of adolescents in Ireland, given that health literacy is context and content specific.^{4 7 31} Specifically, in order to develop a tool that is sensitive to the wide range of potential health literacy needs of adolescents, including potential key mechanisms to support intervention development, as well as measuring potential intervention effects, a well-targeted measure of health literacy is vital.⁶ Research has underscored the value of actively involving participants, including improving the quality and appropriateness of research.³⁵ From a health promotion perspective, research should be ‘carried out “with” or “by” members of the public rather than “to”, “about” or “for” them’.³⁶ Therefore, a co-design approach, where young people are actively involved in the research process, is crucial to develop a questionnaire to measure adolescent health literacy using a validity-driven approach.³⁷

In response to the findings of previous research, this project aims to define adolescent health literacy and develop an adolescent health literacy questionnaire. This will be achieved in three phases. The first phase encompasses qualitative research using co-design workshops with young people to determine how the concept of health literacy would be best operationalised to suit the purposes of the intended measure. Further mixed methods research will elicit the measurement domains of the questionnaire. Phase 2 involves validity testing of the data generated from the draft questionnaire. Phase 3 involves rolling out the questionnaire to a nationally representative sample of young people in Ireland to understand adolescents’ health literacy needs and levels.

Methods and analysis

Study aim, design and setting

The primary aim of this study is to conceptualise adolescent health literacy to inform the development and testing of the constructs for the Adolescent Health Literacy Questionnaire (AHLQ). Secondary objectives are described in table 1. This mixed methods study began in July 2019 and is envisaged to be completed by February 2022. This research will involve postprimary school sites across Ireland with adolescents aged 12–18 years.

Table 1.

Specific objectives and methodology for the adolescent health literacy project

Objectives	Method	Sample	Analysis
1. Identify preliminary understanding of health concepts with adolescents	Group interviews.	(Composition): n=31 (16 girls, 15 boys).	(Proposed): thematic analysis.
2. Explore domains of health literacy from an adolescent perspective	Co-design workshops, vignettes and concept mapping.	(Anticipated composition): n=30 approx.	(Proposed): thematic analysis.
3. Exploring if the items are understood as intended by the adolescents	Group interviews.	(Anticipated composition): n=15 approx.	(Proposed): refining item wording and composition from adolescents’ feedback.
4. Further refining of the items within the questionnaire	Cognitive interviews.	(Anticipated composition): n=4 adolescents and n=4 education workers approx.	(Proposed): Further refining of items within questionnaire via observation and interview questioning.
5. Pilot testing of the questionnaire	Questionnaire completed by representative sample of Irish adolescents within school settings.	(Anticipated composition): n=200 approx.	Test–retest reliability and confirmatory factor analysis.
6. Validity testing of questionnaire with nationally representative sample of Irish adolescents	Questionnaire completed by nationally representative sample of Irish adolescents.	(Anticipated composition based on sample size calculation): n=6052 approx.	Test–retest reliability and confirmatory factor analysis.

Patient and public involvement

The study will be supported by a youth advisory panel that will provide input for the conceptualisation and development of the AHLQ. This advisory group will meet on a regular basis and are partnered with us for the design of the questionnaire and the informational material for the national rollout study.

Theoretical approach

The specific approach that we will take to guide the qualitative and psychometric construction of the AHLQ will be through the application of a validity-driven approach, as outlined by Buchbinder et al³⁷:

Grounded approaches to a concept definition that includes consultation with a broad range of stakeholders and deliberately eschews prevailing theories until later in the development process; stakeholder participation in the organization of ideas into groups that form the basis for hypothesizing scales to be included in the measurement tool; the development of a priori hypotheses about the way in which items co-vary and can be used to form measurement scales; recognition that construct validation is an ongoing process, and that an instrument is never validated but that each interpretation of the scores needs to be validated; and the specification of a program of research to support the valid application of the tool in relation to an increasing range of interpretations (uses).

Consistent with recommendations by Hawkins et al, we will explicate the intended use and purpose of the AHLQ to ensure the development and validity testing processes are consistent and will support the development of an argument-based approach to validation.^{38 39}

In undertaking the development of the AHLQ, we acknowledge several explicit assumptions regarding adolescent health literacy:

• It is a multidimensional concept that can be represented by several independent constructs.

• It is influenced by the contexts in which the adolescent lives, learns, works and plays.

• It changes over time.

• It can be represented by a series of questions that an adolescent can understand and attend to.

This research is being guided by the Ophelia principles (table 2). The Ophelia principles are a structured framework that ensures the research is conducted in a participatory, sustainable, equity-driven and community-based fashion.¹⁸ Following the Ophelia process will allow us to gain an in-depth understanding of the health literacy mechanisms that we need to influence to inform subsequent intervention development.

Table 2.

The Ophelia principles guiding this research^{18 47}

Principles	Description
1. Outcomes focused	Improved health and well-being outcomes.
2. Equity driven	Emphasis placed on the individual to optimise equity.
3. Co-design/co-production approach	Relevant stakeholders actively collaborating to identify solutions.
4. Needs-diagnostic approach	Responding to locally identified needs.
5. Driven by local wisdom	Intervention solutions driven by local wisdom and lived experience.
6. Sustainable	Optimal practice identified becomes normal policy and practice.
7. Responsiveness	Recognition that needs/responses will vary across contexts.
8. Systematically applied	Focused on sustained improvements.

Ophelia, OPtimising HEalth LIteracy and Access.

Sampling and data collection

Phase 1: conceptualising adolescent health literacy

Five post-primary school sites in Dublin, Ireland, were approached about being involved in this research as part of the first phase of this project (figure 1). These schools were selected as the study investigators have already established research links with the principals in each of the schools. Initial research providing a foundation for this study has involved five group interviews with adolescent groups (31 adolescents; 16 girls and 15 boys) across five sites in disadvantaged urban (n=4) and rural (n=1) areas. The schools included a community school, a comprehensive school, a vocational school and two secondary schools. Three schools were mixed-sex education, while the latter two were single-sex education schools. These group interviews were conducted in May 2019. The aim of these group interviews was to scope what health and healthy behaviours meant to adolescents, while simultaneously aiming to identify preliminary health literacy domains. The group interviews were semi-structured in nature, where open-ended questions were used to elicit adolescents’ experiences and perspectives on health literacy.

Figure 1.

Steps undertaken in the development of the Adolescent Health Literacy Questionnaire.

Several co-design and concept mapping workshops will be conducted with a youth advisory group (YAP), which was established by the Irish Heart Foundation (study funders) to ensure adolescent representation from as early as possible in the research process. The YAP comprises 32 adolescents (16 males and 16 females) aged between 12 and 18 years who are currently in post-primary school. Parents have given their written informed consent for their child to be in the YAP and to take part in group discussions related to work conducted in the Irish Heart Foundation. The adolescents have also given their written informed assent to be part of the research. Informed assent is sought verbally before the start of each YAP discussion, and the members are reminded that they can withdraw their consent at any time, without negative consequence. The purpose of these additional co-design workshops is to allow for feedback of the information collected through the initial group interviews to identify the domains of health literacy from an adolescent perspective. Researchers will meet with the YAP up to four times throughout the study to: (1) explore health behaviours and health literacy needs of adolescents; (2) explore potential health literacy domains for the instrument; (3) present the draft health literacy domains; and (4) conduct cognitive pretesting on the questionnaire items.

Concept mapping, domain specification and item generation

A concept mapping process will be undertaken.⁴⁰ The steps include the development of the focus of the conceptualisation, brainstorming to generate a range of statements, sorting the statements and generating and revising the concept map as appropriate.^{37 40} An initial sensitising session will be conducted with the YAP where we will discuss 'what health is' so there is a common understanding among the group. Two seeding statements will be generated and presented to the adolescents in the YAP. They will be asked to consider these statements individually for 5 min and generate their ideas. This allows for a highly structured process to elicit a breadth of ideas and equal input from all participants. The following are the seeding statements for the young people: ‘What do adolescents need to live a healthy life?’ and ‘What is important to you in your life now’. The young people will be asked to place each idea per seeding statement onto a piece of card and to continue until they have included all their own ideas. They will be asked to rank the ideas they have generated by importance. The group facilitator will ask each young person to present their ideas to the group. Once this has been completed, each participant will sort through each of the ideas they generated and sort the statements into conceptually similar groups in any way which makes sense to them. Again, each participant will complete this exercise on their own to ensure equal participation of all within the group. Following this, group interviews will also be conducted to explore young peoples’ views of how the school environment, their physical environment and the commercial determinants of health impact their health and well-being. The facilitator will create a series of vignettes for each topic and will display them in a storyboard format. A series of seeding statements will be used to elicit participants’ views:

• ‘What does he/she need to have a healthy life?’

• ‘For him/her to make a change, what does he/she need to do?’

• ‘Is there anything stopping him/her from making a change? If so, what?’

• ‘How can he/she overcome the things that are stopping him/her from making a change?’

Multidimensional scaling will then be used to create a concept map, based on participants’ inputs. In this way, statements that are sorted into similar piles appear closer together on the map. Once the concept map has been created, participants will be asked to label each of the clusters and to ensure that each of the statements within the cluster are appropriate. Participants will be asked to nominate a different cluster for any statement they feel does not fit within a cluster. This will be done individually by each participant. The final grouping of the clusters will be done as a group, where we will also verify if there are any missing constructs.

Item refinement

Questionnaire items will be derived from concept mapping statements (organised into the theoretical constructs) and supplemented by text from group interviews and the YAP. Items will be reviewed by the authors to ensure they specifically fit their intended construct and are distinct from other constructs. To guide item writing within each construct, two vignettes will be created based on the emergent construct definition and potential pool of items from the concept mapping, specifying a young person with very high levels of the domain’s attributes and one with low levels of the attribute in question. Consistent with Bloom’s taxonomy,^41–43 items will be written that cover a range of difficulty (ie, items that are easy to endorse with a low level of the trait and hard to endorse with a high level of the trait). The use of the vignette and attention to item difficulty will ensure that the content of the items for each construct cover the full range of adolescents’ health literacy attributes. An initial set of 6–10 items for each construct will be generated. For each draft item an item intent narrative will be drafted – this is text that fully describes what the item is intended to convey, and not convey.

Cognitive pretesting

The draft items will then be initially pre-tested with the YAP using a group interview technique to determine if the items are understood as intended (ie, according to the item intent). Word choice and sentence structure and complexity will also be reviewed and revised where needed. Members will read each item themselves, one at a time, and then discuss what they think the item means. If there is ambiguity, suggestions for improvement will be gathered and the item(s) will be changed accordingly. Once a refined and prioritised set of six to eight items per construct has been generated, cognitive interviews will take place, as outlined below.

Cognitive interviews

Consistent with the methodology defined in Osborne et al,⁴⁴ the refined questionnaire will be tested using a cognitive testing process where each item will be tested with at least four children and four education workers. The questionnaire will be administered using pen and paper format while a member of the research team will carefully observe each participant. Once the respondent has completed the questionnaire, the researcher will specifically probe the participant on items they hesitated or appeared to find difficult to answer. Participants will be asked ‘What were you thinking about when you were answering that question?’. This will elicit the cognitive process behind participants’ answers. If necessary, a further prompt question will be used: ‘Why did you select that answer?’.

Phase 2: quantitative pilot of the AHLQ

The questionnaire developed to measure adolescent health literacy will be pilot tested in a two-stage random sample of the five schools that were approached for phase 1: the qualitative component of the study. The school principals will be contacted by post or telephone and briefed about the pilot testing of the questionnaire. They will be given a research pack that will contain letters and information sheets describing the study for adolescents and parents/guardians, a consent form and a paper copy of the questionnaire. In terms of inclusion criteria, participants will be aged between 12 and 18 years and give written informed assent. Recruitment with respect to age, geographic location and the socioeconomic profile of the school and surrounding area will be targeted. Parents/guardians will also provide written informed consent.

This phase of the study will involve field-testing the questionnaire by administering the final self-report questionnaire to a study population of approximately 200 adolescents aged between 12 and 18 years from the five schools across Dublin who took part in the initial group interview discussions for this research. This is to ensure the questionnaire’s content and appropriateness for the target population. This is envisaged to take place in September 2021. Demographic data will also be collected as part of the questionnaire. Questionnaires will be web based and completed on tablet devices and will target a completion duration of no more than 30 min. Confirmatory factor analysis will be used to test for construct validity. Test–retest reliability (2 weeks apart) will also be conducted as part of the pilot study with a smaller subsample of adolescents.

Phase 3: national rollout of the AHLQ

Following the successful completion of the pilot study, a cross-sectional survey will be conducted. This is envisaged to take place in late 2021. A random sample of postprimary schools in Ireland will be approached to take part in the national rollout of the AHLQ. For this large-scale national rollout, a sample of 6052 is required for adequate power to ensure representativeness of the sample. The sampling strategy will be similar to an Irish national study of school children’s health outcomes and physical activity⁴⁵ and will include all postprimary schools in Ireland. Schools will be stratified by gender (male, female and mixed), socioeconomic status (designated disadvantage vs non-designated), area of residence (urban vs rural), type of school (secondary, community, comprehensive or vocational) and school classification (free education or fee paying).⁴⁵ It is expected the questionnaire will take no longer than 30 min for adolescents to complete. This is to ensure the questionnaire can be completed in one sitting during class. Validity testing of this large sample will take place and the psychometric testing of this sample will also be repeated.

Data analysis

Phase 1: conceptualising adolescent health literacy

Five group interviews will be conducted and will be transcribed verbatim and analysed by CMD and AS. The group interviews and the concept mapping workshops with the YAP will be transcribed verbatim and analysed by AS. Pseudonyms will be used throughout the transcription process to protect anonymity. NVIVO V.12 software will be used to organise the qualitative data collected and facilitate the analysis process.

Phase 2 and phase 3

In terms of the expected scoring procedures, the response options for the items will be determined by the nature of the content of the emerging items and may be agree/disagree scale or a difficulty scale or other scale. We expect the validity-driven approach with concept mapping will generate several constructs that form multiple item scales. These scales will have items that are equally weighted, summed and divided by the number of items in each scale. Scale scores will be presented as means and SD, and 95% CIs. Descriptive statistics will be generated for each item to determine any missing values, floor and ceiling effects, scale reliability (Cronbach’s alpha) and p value for item fit tests. Confirmatory factor analysis will be used to test construct validity, given that the constructs were specified a priori within the AHLQ. The reliability of the scale will be tested using Cronbach’s alpha. Acceptable levels for internal consistency using Cronbach’s alpha are values between ≥0.80 and ≤0.90,⁴⁶ as levels greater than 0.90 may point to redundancy among the scale items.⁴⁶ Acceptable levels for composite reliability in this study will be set at >0.7. A correlation matrix will also be run to examine the item–total correlations. A mean interitem correlation value between 0.15 and 0.50 is acceptable.⁴⁶ Test–retest reliability will be run using Pearson correlation coefficient. The questionnaire will be revised as appropriate from the information gathered by the factor analysis and the Cronbach’s alpha coefficient.

Ethics and dissemination

Ethical approval for this study has been approved by the University College Dublin Human Research Ethics Committee – Sciences (LS-20–08). This research is currently in phase 1, where qualitative research is being conducted to conceptualise adolescent health literacy. The aim of this study is to conceptualise an understanding of adolescent health literacy and generate an AHLQ using a validity-drive approach³⁷ to measure health literacy in adolescents. The AHLQ will be rolled out to a nationally representative sample of young people in postprimary schools across Ireland.

The study output will allow for the accurate measurement of health literacy in adolescents. This research is part of a larger study involving the development of a health literacy intervention for postprimary schools in Ireland. It is intended that the AHLQ will be used for pre–post evaluation of health literacy interventions. The authors will produce one paper outlining the development and validation of the measure, as well as one main outcomes paper. The findings of this research will be disseminated at regional, national and international scientific conferences, as well as through peer-reviewed publications in international journals. This research will also be disseminated via the WHO National Health Literacy Demonstration Projects programme channels.