Development and initial validation of the health-related quality of life pictorial inventory for children (HEALTH-PIC)

Abstract

Purpose

This 3-study paper aimed to develop and validate a self-reported Health-Related Quality of Life Pictorial Inventory for Early Childhood Children (HEALTH-PIC). The scale was designed to overcome existing barriers of parent-proxy response styles such as observation bias and offer an alternative to age-suited literary questionnaires to assess self-reported health-related quality of life, including physical health, emotional health, social health and school health in early childhood.

Methods

Each study targeted a specific aspect of scale development, employing distinct samples to refine and validate the inventory. Study 1 involved item development/revision, which utilized a panel of 10 experts (Mean_age = 34.8, SD = 4.9) and 25 parents (Mean_age = 38.9; SD = 4.1) via the Delphi method to revise the scale and establish agreement. Study 2 is a cross-sectional study that invited a sample of 22 primary school students (Mean_age = 6.18, SD = 0.39) and 20 kindergarten students (Mean_age = 4.55, SD = 0.50) to establish face validity amongst primary respondents. Finally, Study 3 is a cross-sectional study that invited 342 parent and child (Mean_age = 6.30, SD = 1.31) dyads to complete the HEALTH-PIC and reference health-related quality of life (HRQoL) questionnaires to establish the questionnaire’s factorial, concurrent, discriminant and criterion validity in addition to internal consistency.

Results

Scale items in Study 1 were developed alongside experts and parents with a strong theoretical and statistical support calculated using Aiken’s agreement (Aiken’s V p < 0.05), ensuring that the items were clear, accurate and applicable for children. In Study 2, we ensured that primary respondents of different ages were able to accurately identify the pictorial images (Aiken’s V p < 0.05) and complete the questionnaire when the scripted instructions were read aloud. In Study 3, structural equation modelling of the HEALTH-PIC demonstrated a robust factor structure (CFI and TLI > 0.99; RMSEA and SRMR ≤ 0.08), concurrent validity, discriminant validity, criterion validity and an acceptable level of internal consistency (i.e., Cronbach’s α = 0.60–0.66).

Conclusion

The findings of the three studies provide preliminary evidence to support the content validity and construct validity of the HEALTH-PIC. This novel pictorial scale not only provides a reliable and valid assessment of the multidimensional aspects of health-related quality of life among children but can also overcome existing barriers of parent-proxy or age-suited questionnaires.

Supplementary Information

The online version contains supplementary material available at 10.1007/s11136-025-03988-x.

Introduction

There are many existing definitions of health-related quality of life (HRQoL), all of which have a consensus that it encompasses multidimensional constructs. According to the popular PedsQL (Pediatric Quality of Life Inventory) measurement model, HRQoL for children encompasses physical health, emotional health, social health and school/daily functioning [1–4]. HRQoL is important in the scientific literature, as it is an outcome variable used to monitor health, track treatment effectiveness, and differentiate health status between sick and healthy individuals [1, 5–9]. Increased HRQoL in early childhood can even extend to long-term outcomes such as lower health care costs and family burdens [10, 11]. The present study aimed to develop and preliminarily test a pictorial measure of HRQoL that overcomes the previous barriers of traditional questionnaire measures.

Measures of HRQoL for children: parent proxy and child self-report

Assessing HRQoL in early childhood is especially important as children are experiencing a crucial period of growth and development, the outcomes of which may even have lasting effects in later life [7, 12–15]. Despite its importance and usage in research, clinical and educational fields, current measures of HRQoL for young children (age < 8 years old) have relied heavily on parent proxy questionnaires where parents respond on the child’s behalf due to language barriers and cognitive limitations.

Research investigating HRQoL scores by parent proxy and by child self-reports has revealed a significant disparity when comparing both reporting methods [14, 16]. Qualitative investigations have considered the possibility of recall and observational bias, as parents tended to recall more instances of physical externalizations (rather than emotional internalizations) of child behaviours when they evaluate their child’s HRQoL [5, 16, 17].

Alternative methods of quantifying young child HRQoL include the use of age-suited questionnaires, where both appropriate language and vocabulary have been adapted, allowing young children (even those aged 5+) to self-report [18, 19]. Although this advancement addresses previous barriers, it remains questionable whether traditional questionnaires that involve literary items are suitable for children to complete. This is especially the case when the items are intended to measure complex concepts that require attentional focus to read and understand the questions and cognitive recollection of a specific situation or scenario [20, 21]. In turn, the quality of the data may present as a lower standard than that of 8+years old [22].

Other questionnaires, such as the Big 5 personality questionnaire, which encounter similar methodological constraints in obtaining first-person responses from children younger than 8 years old, have hence moved towards the use of pictorial questionnaires [e.g., 20, 23, 24].

Pictorial questionnaires

Pictorial questionnaires make use of image-based elements to replace text-based literary items and/or the rating scale of traditional questionnaires. In particular, items in the pictorial questionnaires are presented as static pictures instead of words. The respondents then rate the extent to which the concepts are relatable to them or how often the scenarios have recently occurred to them. Children aged 8 years or younger may then make direct interpretations of the items by looking at the pictures that incorporate daily life examples, ultimately lowering the necessary cognitive load on reading and recalling specific scenarios that match the description [20]. It has also been suggested that children have more pleasure in completing pictorial questionnaires because the questions are more visually appealing. With the formatting like a children’s book, it may also allow children to engage more and hold their attention for longer, which can prevent young children from defaulting to mindless agreement, ultimately making them more susceptible to acquiescence bias [24–27].

Pictorial questionnaires have also been statistically shown to be as accurate and reliable as traditional surveys [21, 28]. For example, a pictorial scale of children’s swimming competence has captured reliable responses that are significantly associated with an objective measure of swimming competence [28]. The supportive evidence of pictorial scales has thus paved the way for extending their application in measuring other psychological or behavioural patterns of children.

Sauer and colleagues [21] reviewed existing pictorial scales and identified 57 existing studies that developed and provided examples of pictorial scales in research. In their findings, a wide variety of existing pictorial scales covering complex concepts such as emotions, personality and family aggression were found [20, 29, 30]. However, as far as we know, there has not been a pictorial scale developed for measuring young children’s general HRQoL even when presented with the theoretical and practical value of measuring this construct in early childhood. In addition, Sauer and colleagues [21] noted that there have been some inconsistencies in the way the pictorial scales were developed, and these inconsistencies might lead to measurement bias and generalizability problems. In combating this, the authors subsequently recommended a 3-phase structured protocol, directing future pictorial scales to be developed with a strong foundation. Therefore, the development and subsequent validation of a pictorial HRQoL scale that follows the 3-phase protocol recommended by Sauer and colleagues [21] is highly warranted.

The present studies

The aim of the present study was to develop and establish a Health-related Quality of Life Pictorial Inventory for Children (HEALTH-PIC) and to conduct a preliminary test of the validity of the scale by referring to Sauer and colleagues’ [21] 3-phase protocol. In this paper, we present three complementary studies that contribute to the ultimate development and validation of the HEALTH-PIC. Study 1 followed Phase 1 of Sauer and colleagues [21] guidelines, which involved the item development and revision of HEALTH-PIC items on the basis of feedback from experts and parents. Study 2 followed Phase 2, which aimed to establish face validity and preliminarily test HEALTH-PIC in primary respondents aged 4–8. Study 3 followed Phase 3 to preliminarily test the HEALTH-PIC’s psychometric properties, namely, the factorial, concurrent, discriminant, criterion validity and internal consistency of the scale in parent–child dyads.

Study 1

The ultimate purpose of Study 1 was to follow the Phase 1 guidelines of Sauer and colleagues [21] and to establish items of a pictorial HRQoL scale suitable for children aged 4–8.

Literature research and item generation

Although pictorial scales exist that measure components of the concept of HRQoL (i.e., emotion), their developmental framework often diverges from the specific focus of HRQoL and may not have been designed for children aged 4–8. Therefore, the literature regarding gross motor skills [i.e., Walking, 31], daily skills [i.e., Chores, putting toys away; 32], primary emotions [i.e., happiness, sadness, 33, 34], peer social interactions [i.e., bullying, teasing, 33, 34] and cognitive functioning [i.e., participation in class, 35] was examined during initial item generation, as they corresponded to the subscales of physical health, emotional health, and social health and school/daily functioning in children [4, 18].

During the initial development of the pictorial scale, we brainstormed key items or components within the literature that could reflect common situations that children encountered on a daily basis. Ultimately, a total of 23 items reflecting 4 factors (7 physical health, 6 primary emotions, 5 social health and 5 school functioning) were initially developed.

A dichotomous Yes/No Smiley Face Likert Scale [36] in addition to an “I don’t know” option [20] was also adopted as a response scale. Dichotomous scales can be preferred in younger populations, as children may be prone to extreme positive/negative answers when presented with Likert scales [17, 23, 37, 38]. Additionally, to avoid forcing children to conform to one answer, an “I don’t know” option was also included.

The recall period of one month was chosen for the HRQoL scale, as current research on HRQoL topics utilizes both weekly and monthly recall periods [39]. Weekly, shorter recall periods can capture immediate experiences, whereas monthly recall periods can capture broader patterns or gradual developments [particularly in social health where changes take time to manifest; 39]. Therefore, a one-monthly recall period was chosen for the current scale to better capture comprehensive experiences.

To account for gender differences in pictorial items and to make the scale more relatable, boy and girl versions were developed where the pictures differed only in the hair length of the main character [23, 40–42].

Expert consultation

In addition to the literature, both Sauer and colleagues [21] and Tang and colleagues [43] suggested to utilize a group of experts in the field to gather ideas for items while being mindful of the target population (e.g., children or those who are sick). These expert consultations are imperative, as they are familiar with the framework of child HRQoL and are able to provide valuable feedback for scale development. The experts in the current investigation were recruited by following the recommendations of Dunn, Bouffard, and Rogers [44] and were defined as those who had a publication record in research areas of children and HRQoL. The research protocol for the expert and parent consultation was approved by the Human Research Ethics Committee of the first author’s institution (Ref no. 2021-2022-0424).

A total of 10 experts were invited to provide valuable feedback on the initial items, leading to the refinement and revision of the initial pool across each subscale. The items were reviewed independently by the expert panel, and they were re-drawn via their comments until the expert panel felt that the items were the clearest and encompassed the closest representation for children’s HRQoL in each of the subscales.

After initial item confirmation and following Aiken [45]content validity guidelines, each expert was then asked to rate the pictorial item in terms of its clarity (i.e., Do you think this picture clearly conveys “walking” to children), accuracy (i.e., Do you think this picture is an accurate representation of “walking” for children) and applicability (i.e., Do you think the pictured scenario is realistic and applicable for children) for children (defined as 4–8 years old) on a scale of 1 (strongly disagree) to 5 (strongly agree) using an online platform. Additionally, to assess content relevance, they were also asked to indicate which subscale the item most likely represented via multiple choices of physical health, emotional health, social health or school functioning.

Parent consultation

In addition to an expert panel, we also formed a parent panel group. Specifically, the parent panel consisted of 25 eligible parents who were current fathers/mothers of at least one child who was between 4 and 8 years old. Each parent participant was asked to independently review the HEALTH-PIC initial items via an online platform. The parents were not involved in initial item generation, as their feedback may contradict the expert’s previous input. However, for each item, they were able to suggest feedback to improve the pictorial representation of the target item and rated each item in terms of its clarity, accuracy and applicability for children (defined as younger than 8 years old) on a scale of 1 (strongly disagree) to 5 (strongly agree). Similarly, they were also asked to rate which subscale the item most likely represented: physical health, emotional health, social health or school functioning.

Item revision and analysis

Data from the expert and parent groups were analysed together as we wanted to reach a consensus across groups. The content relevance of the scale was coded as incorrect (1)/correct (2) to the preconceived factor prior to calculating Aiken’s significance. All of the agreement results were calculated via Aiken’s formula (V = S/[n(c−1)]), where S is the sum of the ratings on the same item minus the number of raters, and n is the number of raters, and c is the highest score in the rating scale. The resulting number is then determined via Aiken’s significance (p < 0.05) table [45]. Items that did not reach significance were subsequently revised according to parents’ and experts’ comments. These revised versions were then presented to the same panel again for another round of ratings/comments until satisfactory (indicated by Aiken’s significance of p < 0.05). To ensure an eventual consensus, the Delphi method was employed, where the panel was given the previous pictorial item, average ratings and sample comments of the previous round as a reference, allowing them to reflect on others’ opinions before their second round of ratings/comments on the revised pictorial item [46, 47]. The current sample of 25 parents/experts is considered acceptable for the Delphi Method [48, 49].

Results

Results of study 1

There was a total of 2 rounds of expert (M_age = 34.8; SD = 4.9; range = 28–46) and parent (M_age = 38.9; SD = 4.1; range = 31–45) revisions before all the items reached significance. In the first round, a total of 7 items satisfied all the criteria of significant agreement of clarity (V = 0.80 to 0.94; p < 0.05), accuracy (V = 0.76 to 0.94; p < 0.05), applicability (V = 0.73 to 0.91; p < 0.05) and content relevance (V = 0.77 to 0.97; p < 0.05). As 16 items did not reach significance (p > 0.05) on all the requirements, we reconstructed these items according to the experts’ and parents’ comments and asked the parents/experts to re-evaluate these items.

Examples of these initial comments include “maybe sweeping the floor is better than mopping the floor, both are chores” (item 6); “the box can be empty, you can draw more toys in the box to make it look heavier” (item 4); and “the other classmates can seem more relaxed when doing the test” (item 19).

In the second round of the panel review, 10 experts and 15 parents agreed to rate the revised version, and all 16 items met the required criteria for content relevance, clarity (V = 0.64 to 0.89; p < 0.05), accuracy (V = 0.64 to 0.86; p < 0.05), applicability (V = 0.63 to 0.91; p < 0.05) and content relevance (V = 0.69 to 1.0; p < 0.05). The specific V values for each item are displayed in Table 1, and the final pictorial items can be found in online Appendix A.

Table 1.

Aiken’s V coefficient for Study 1 expert’s and parent’s ratings of HEALTH-PIC items’ clarity, accuracy, applicability and content relevance

Item no	Clarity	Accuracy	Applicability	Content relevance
Item 1	.79**	.75**	.88**	.84**
Item 2	.80**	.76**	.91**	.97**
Item 3	.89*	.86**	.91**	.96**
Item 4	.78*	.75**	.63*	.75**
Item 5	.81**	.76*	.86**	.91**
Item 6	.85**	.84**	.82**	.76**
Item 7	.81*	.79*	.82*	.77**
Item 8	.76**	.70*	.81**	1.0**
Item 9	.89**	.82**	.73*	.97**
Item 10	.94*	.94*	.91**	1.0**
Item 11	.91**	.91**	.90**	1.0**
Item 12	.77*	.78**	.75**	.84**
Item 13	.82**	.80**	.85*	.76**
Item 14	.82**	.77**	.88**	1.0
Item 15	.79**	.80**	.81**	.92**
Item 16	.86**	.84**	.77**	.89**
Item 17	.74**	.74**	.77**	.69**
Item 18	.66**	.65**	.75**	.72**
Item 19	.66**	.64**	.69**	.96**
Item 20	.72**	.71**	.74**	.74**
Item 21	.68**	.67**	.71**	.69**
Item 22	.71**	.71**	.79**	.88**
Item 23	.64*	.64*	.73*	.8 0**

^*p < 0.05

^**p < 0.001

Conclusions of study 1

Following protocols for scale development, literature research and expert/parent consultation, Study 1 developed 23 items of the HEALTH-PIC. Evidence from expert and parent panel reviews provided initial support for face validity in terms of clarity, accuracy, applicability and content relevance on the HEALTH-PIC.

Study 2

The purpose of Study 2 is to establish face validity of the HEALTH-PIC in primary respondents of early childhood children. Specifically, we tested whether children could accurately identify what the pictorial items portrayed in addition to their ability to complete the questionnaire. To account for differing levels of cognitive development based on age and school year in early childhood, we invited both primary school and kindergarten students.

Method

Participants

We recruited 22 primary school students and 20 kindergarten students for Study 2 by sending school invitations to participate in a cross-sectional study. The participants were all between 4 and 8 years old (n_Total = 42, 50% female) and resided in Hong Kong.

Procedure

The research protocol for this study was approved by the Human Research Ethics Committee of the first author’s institution (Ref no. 2021–2022-0424).

Item identification

After consent, the children were interviewed in person by a trained researcher. In the beginning, a researcher first read a script outlining the instructions for questionnaire completion. Once they understood the instructions, each child was presented with a single item from the HEALTH-PIC, followed by the question “Can you tell me what is happening in the picture?”. The child was encouraged that there was no right or wrong answer and that they could verbalize their thought process via the think-aloud technique. After the child responded, the researcher marked the answer in verbatim.

Questionnaire completion

After identifying the item, the researcher revealed the second part of the question, where the child participants were asked whether they had experienced similar situation(s) within the past month or last 4 weeks and were presented with a smiley face Likert scale alongside the labels of “yes”, “no”, and “I don’t know”. After the child responded, the researcher encouraged them to circle their answers by themselves.

This process was repeated with all 23 items of the HEALTH-PIC, and the whole process was audio recorded. At the very end, the child was also asked by the researcher whether they found the questionnaire too long or too difficult (yes/no).

Analysis

Data from 22 primary school students (M_age = 6.18; SD = 0.39; range = 6–8) and 20 kindergarten students (M_age = 4.55; SD = 0.50; range = 4–6) were analysed separately to account for differing levels of cognitive development across different age ranges and school grades in early childhood.

Item identification

Participants’ verbal responses were coded into numerical values that indicated correct/incorrect responses that corresponded to the questionnaire item/intended pictorial depiction. Responses were classified as correct if they were consistent with the preconceived item description established in Study 1. For example, if the item represented “running” (item 2), the responses of “walking”, “moving”, and “playing” were not accepted. These responses had to be equivalent to the preconceived item description in Study 1 (i.e., running); any other description was otherwise marked as wrong. In cases where the correctness of the responses was unclear, a second researcher was consulted to ensure objective classification. Aiken’s validity coefficient and significance [45, p < 0.05] were used to reveal whether both kindergarten children and primary school children were able to correctly identify the picture to the corresponding item question (for evidence of face validity).

Questionnaire completion

To observe whether children could complete the questionnaire by following the instructions of the scripted directions, we investigated whether there were missing or “I don’t know” values for at least 11 items (approx. 50% of the questionnaire). These “I don’t know” values were counted as missing data as they do not directly conform to a yes/no answer [50]. Additionally, to determine the statistical significance and agreement of Aiken’s validity coefficient [45, p < 0.05], the subsequent yes(1)/no(2) responses on whether the questionnaire was too long or too difficult were numerically coded.

Results of study 2

Item identification

In terms of item identification, Aiken’s validity was computed similarly to that of Study 1. All 23 items satisfied the criteria of Aiken’s significance (kindergarten children V = 0.75–1.0; primary school children V = 0.73–1.0; p < 0.05; see Table 2), indicating that both kindergarten and primary school children were able to accurately identify all the pictorial items.

Table 2.

Study 2 Aiken’s V coefficient on coded children’s response on item identification of the HEALTH-PIC

Item no	Kindergarten students (aged 4–6)	Primary school students (aged 6–8)
Item 1	.85**	1.0**
Item 2	.95**	.95**
Item 3	.95**	1.0**
Item 4	1.0**	.82**
Item 5	1.0**	1.0**
Item 6	1.0**	1.0**
Item 7	.80**	.91**
Item8	.90**	.91**
Item 9	.95**	.81**
Item 10	1.0**	1.0**
Item 11	1.0**	1.0**
Item 12	.75*	.81**
Item 13	.80**	.73*
Item 14	.90**	1.0**
Item 15	.85**	.95**
Item 16	.85**	.95**
Item 17	.80**	.73*
Item 18	.80**	.81**
Item 19	.90**	1.0**
Item 20	.80**	.77**
Item 21	.85**	.81**
Item 22	.80**	.73*
Item 23	.75*	.91**
Not too long	.80**	.86**
Not too difficult	.85**	.91**

^*p < 0.05

^**p < 0.001

Questionnaire completion

All child participants (n = 42) were able to follow the scripted directions to complete the questionnaire. In terms of length and difficulty, Aiken’s validity also revealed that the current questionnaire was considered suitable in terms of length (showing agreement that kindergarten and primary school children did not find it too long; kindergarten children V = 0.80; p < 0.05; primary school children V = 0.86; p < 0.05) and difficulty (showing agreement that kindergarten and primary school children did not find it too difficult; kindergarten children V = 0.85; p < 0.05; primary school children V = 0.90; p < 0.05) for the current sample of children.

Conclusions of study 2

Through Study 2, we investigated children’s perspectives of the HEALTH-PIC items and determined that children in kindergarten/primary school in our current sample were able to accurately identify the pictorial depictions. Our investigation also revealed that children within our sample were able to follow the scripted directions and complete the questionnaire without finding the questionnaire too long or difficult.

Study 3

The purpose and aim of Study 3 is to examine the HEALTH-PIC in terms of its factorial, concurrent, discriminant, and criterion validity in addition to its internal consistency in a cross-sectional study of children aged 4–8. Additionally, we aimed to recruit participants from the United Kingdom to provide initial support for the generalizability of the scale across countries. To demonstrate scale validity, our specific hypothesis is as follows in relation to the current study measures:

(H1) Factorial validity

Consistent with development, items in the factors of physical health, emotional health, social health and school functioning, would load on their respective factors, and the four-factor model would yield an acceptable goodness-of-fit indices.

(H2) Concurrent validity

The overall score of the HEALTH-PIC would be positively associated with parent proxy HRQoL and Child HRQoL overall score. Moreover, the subscales of the HEALTH-PIC would also be positively associated with the subscales of parent proxy HRQoL and Child HRQoL in terms of physical health, emotional health, social health and school functioning, respectively. These associations are expected to show moderate correlations (r = 0.3) as defined by Cohen [51, 52] as previous HRQoL demonstrated similar levels of association [53].

(H3) Discriminant validity

The average variance extracted from the HEALTH-PIC would be greater than its shared variance with a mental health questionnaire.

(H4) Criterion validity

The overall score of the HEALTH-PIC would significantly and negatively correlate with child illness. Moreover, the physical subscale would be positively associated with daily functioning, the emotional health subscale will be negatively associated with mood disturbances, the social subscale would be positively associated with social life and school functioning would be positively associated with academic status. The associations are expected to show moderate correlations (r = 0.3) as defined by Cohen [52, 54] as previous HRQoL subscale correlations demonstrated similar levels of associations [55].

(H5) Internal consistency

Cronbach’s alpha of the overall scale and subscales of the HEALTH-PIC would be acceptable.

Method

Participants

Parent participants were recruited via an online questionnaire platform, Prolific (https://www.prolific.com/), and were eligible if they had a healthy child between the ages of 4—8 years old, were currently residing in the United Kingdom, had a normal vision and understood English. Both parents and their children were required to complete a cross-sectional questionnaire and were compensated for their time. We recruited 342 healthy parent‒child dyads.

Materials

The online questionnaire was split into two parts: a parent questionnaire and a child questionnaire.

Parents were invited to complete a series of self-reported measures of demographic information, child illness, parent proxy HRQoL, mental health and daily functioning with respect to their children.

Children were invited to complete a series of self-reported measures measuring HRQoL (pictorial/literary), mood disturbances, academic status and social life with the assistance of their parents in reading the directions and items.

Demographic information

Data on the characteristics of the participating parents and children were collected. Examples include parent/child age, parent/child gender, and the current child’s grade level.

Child illness

A total of 8 questions were designed to capture observational aspects of particular situations indicative of a child’s health, specifically whether they were ill within the past month. Parents were presented with statements (e.g., days absent from class, days visited the hospital), where they indicated if the situation had happened to their children in the past month (yes/no). If the parent participant indicated “yes”, they were asked to indicate how many times/days it had happened. Example questions include “Gone to the hospital to receive treatment” and “Felt sick/unwell in school and had to go home early”. The number of time/days was then added together to indicate a child’s recent overall health status, where a higher number indicated poorer health.

Parent proxy HRQoL

The parent proxy PedsQL assessed the health-related quality of life of their child from the parents’ perspective [3, 18α = 0.86]. There were a total of 23 items where parents reported how much of a problem the child had in the past month with physical functioning, emotional functioning, social functioning and school functioning. Response styles were based on a 5-point Likert scale ranging from 0 (“Never A Problem”) to 4 (“Almost Always A Problem”). Ultimately, after reverse scoring was applied, a higher average score indicated better HRQoL.

Mental health

The Patient Health Questionnaire (PHQ-4) is a brief 4-item questionnaire designed to detect symptoms of anxiety and depression [56, 57, α = 0.81]. Parents were asked to answer based on their child’s emotions in the past month on a 4-point Likert scale ranging from 0 (Not At All) to 3 (Nearly Everyday). Higher summed scores indicated greater symptom severity.

Daily functioning

Only the 3 items that refer to daily movements from the EQ-5D-3L scale were measured [58, 59, α = 0.72]. Parents were asked to rate whether their child had problems with mobility, self-care and conducting usual activities on a three-point Likert scale of “No Problems”, “Some problems” and “Unable to Perform”. After reverse scoring, the scores were summed so that a higher score indicated better daily functioning health.

Pictorial HRQoL/HEALTH-PIC

The Health-PIC is a novel pictorial questionnaire developed (in Study 1 and Study 2) to assess self-reported HRQoL in children in the past month. There were a total of 23 items across multiple dimensions of HRQoL (i.e., physical health, emotional health, social health and school functioning), and a dichotomous smiley face response scale was used with an “I don’t know” neutral face. I don’t know scores were counted as missing data [50], and a higher average score indicated better HRQoL. Parent participants assisted child participants in completing the items by reading aloud the scripted directions and items while allowing their child to select their answers.

Child HRQoL

The 23-item Child PedsQL acute version measures health-related quality of life [18, α = 0.88], where children are presented with common problems that they may face. Parents assisted their child by reading aloud the scripted directions and items while allowing their child to select their answers. Example items are “Is it hard for you to play sports or exercise; do you have hurts or aches?”. The respondents were then asked to point on a 3-point Smiley Face Likert scale to indicate whether they had experienced such problems in the past month. To ensure fidelity and understanding, each child was asked in the beginning “can you snap your fingers”, children were subsequently required to respond and perform the action (of which had to correspond to their answers). After reverse scoring was applied of the questionnaire, a higher average score indicated better HRQoL.

Mood disturbances

The 13-item Mood & Feelings Questionnaire [60, α = 0.88; 61, 62] used a three-point Likert scale with a response of “Not True”, “Sometimes” and “True” to assess children’s feelings. Parents assisted their child by reading aloud the scripted directions and items while allowing their child to select their answers. Scores were calculated via summation, and higher scores indicated greater emotional functioning problems. Example items include “I felt miserable or unhappy; I hated myself”.

Academic status

Only the subscale of intellectual and school status of the 2nd Edition Piers-Harris Children Self-Concept Scale was used in the current study [63, 64; α = 0.77]. This subscale has a total of 11 true/false statements where child participants respond based on their past month academic status [64]. Parents first read aloud the scripted directions and items while allowing their child to select their true/false answers. Example items include “I am smart; I am good in my schoolwork”. After reverse scoring for some items was applied, a higher summed score indicated greater self-concept of academic status/intellect.

Social life

Parents were told to ask their children “Do you have any friends? Can you tell me their names” as an indicator of their social relationships and immediate social circles [65, 66]. Parents were also strictly told not to remind their child of any friends and subsequently counted the number of friends they named.

Procedure

The research protocol for this study was approved by the Human Research Ethics Committee of the first author’s institution (Ref no. 2021-2022-0424).

Eligible parents first completed the first section of the questionnaire on their own (parent section) and assisted their child by strictly reading the scripted directions and items of the latter child questionnaire.

Analysis

(H1) Factorial validity

Exploratory structural equation modelling (ESEM) was employed via Mplus 8 [67] to assess the factor structure of the HEALTH-PIC. Considering that the HEALTH-PIC has dichotomous categorical responses, we used the weighted least square estimator (WLSMV) because it does not assume normality and provides modelling for categorical data [68, 69]. Multiple indicators were used to assess the goodness-of-fit of the current proposed model via traditional fit indices of CFI and TLI values exceeding 0.90 and RMSEA and SRMR values below 0.08 [70]. The COSMIN (Consensus-based Standards for the selection of health Measurement Instruments) good measurement guidelines of CFA/TLI > 0.95 OR RMSEA < 0.06 were also considered [71].

(H2) Concurrent validity

Concurrent validity is established when there is a significant relationship between the overall and subscale scores of the HEALTH-PIC and the PedsQL. A significant correlation coefficient of 0.30 or higher indicates an acceptable level of concurrent validity [72].

(H3) Discriminant validity

To determine discriminant validity, we estimated the average shared variance (AVE) and the shared variance between the factors in the HEALTH-PIC and a measure of mental health. Specifically, the AVE is calculated by summing the squared factor loadings and dividing it by the number of items. With this, discriminant validity can be established when the AVE is greater than the shared variance between the constructs [HEALTH-PIC and PHQ-4; 70, 72].

(H4) Criterion validity

To establish criterion validity, the calculated overall score of the HEALTH-PIC should be significantly and negatively associated with child illness. Moreover, the subscales of physical health, emotional health, social health and school functioning of the HEALTH-PIC should be significantly correlated with daily functioning, mood disturbances, social life, and academic status, as we hypothesized.

(H5) Internal consistency

To establish criterion validity, the calculated overall score of the HEALTH-PIC should be significantly and negatively associated with child illness. Moreover, the subscales of physical health, emotional health, social health and school functioning of the HEALTH-PIC should be significantly correlated with daily functioning, mood disturbances, social life, and academic status, as we hypothesized.

Cronbach’s α coefficient of the HEALTH-PIC was calculated to determine the internal consistency of the scale. A value greater than 0.60 was used as a criterion for acceptable internal consistency [73, 74].

Results

Two participants were excluded from the analysis for an apparent pattern of missing data (i.e., > 50% missing data/ “I don’t know” responses in the HEALTH-PIC). Therefore, our final sample included 340 pairs of parent‒child dyads. The parent participants were aged between 35 and 44 years (63.2%), and most were mothers (as opposed to fathers) who completed the questionnaire (65.3%). The child participants were all aged between 4 and 8 years (M_age = 6.30, SD = 1.31), with a balanced female/male child percentage (52.1% female).

(H1) Factorial validity

ESEM analysis revealed that one item from the physical subscale and one item from the emotional subscale (items 7 and 8) had cross-loading indifferences where they loaded highly on two different factors. They were hence eliminated because of the inability to disentangle specific factors [67]. Moreover, we investigated the inter-item correlations and items within the same subscale, and one item (item 3) from the physical subscale was eliminated because it formed negative correlations with the other items of the same subscale [item 3, 67].

After these three items were eliminated, the 4-factor model showed an acceptable fit of (x² = 121.55 (df = 116), CFI = 0.99, TLI = 0.99, RMSEA = 0.01, 90%CI [0.00,0.03], SRMR = 0.08, establishing factorial validity. The factor loading range of each subscale can be found in Table 3.

Table 3.

Study 3 Correlation matrix, factor loadings, validity indices of the HEALTH-PIC

	HEALTH-PIC-physical health	HEALTH-PIC-emotional health	HEALTH-PIC-social health	HEALTH-PIC-school functioning	HEALTH-PIC-total	PedsQL –child	PedsQL –parent proxy	Child illness	EQ-5D-3L	PHQ-4	MFQ	Social life	PHCSCS
HEALTH-PIC—Physical Health	–
HEALTH-PIC-Emotional Health	.15**	–
HEALTH-PIC- Social Health	.10	.36**	–
HEALTH-PIC—School Functioning	.20**	.34**	.21**	–
Health-PIC Total	.41**	.79**	.65**	.67**	–
PedsQL-Child	.37**	.61**	.37**	.53**	.74**	–
PedsQL-Parent Proxy	.23**	.44**	.27**	.34**	.50**	.48**	–
Child Illness	− .15**	− .33**	− .01	− .35**	− .34**	− .27**	− .36**	–
EQ-5D-3L	.18**	.09	.14**	.12*	.19**	.28**	.25**	− .10	–
PHQ-4	− .07	− .33**	− .12*	− .22**	− .30**	− .37**	− .52**	.33**	− .26**	–
MFQ	− .27**	− .51**	− .21**	− .34**	− .54**	− .65**	− .40**	.22**	− .17**	.34**	–
Social Life	.12*	− .01	.07	.05	.07	.08	.04	.05	− .02	− .02	.01	–
PHCSCS	.18**	.23**	.22**	.38**	.39**	.46**	.28**	− .07	.14*	− .19**	− .35**	.13*	–
Range of Item Loadings to Target Factors	.61–1.13	.36–1.21	.52–.96	.44–.66	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Range of Mean (SD)	94.41 (13.69)	57.34 (32.51)	83.94 (22.82)	77.55 (25.44)	78.52 (15.92)	84.82 (11.80)	90.85 (8.61)	1.55 (2.63)	8.90 (.400)	4.62 (1.09)	1.89 (2.67)	8.14 (5.00)	7.75 (1.38)
Cronbach’s α Coefficient	.60	.66	.63	.60	.76	.85	.87	N/A	.61	.69	.81	N/A	.60
AVE	.60	.46	.47	.34	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Shared Variance (R)2	.04	.11	.01	.05	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A

HEALTH-PIC health-related quality of life pictorial inventory for children, PedsQL pediatric quality of life inventory, EQ-5D-3L subscales of mobility, mobility, self-care and usual activities only, PHQ-4 patient health questionnaire, MFQ mood and feelings questionnaire, PHCSCS piers-harris children self-concept scale 2nd edition, AVE average variance extracted

^* p < 0.05

^**p < 0.001

The final model had 5 items for each dimension of physical health, emotional health, social health and school functioning (see online Appendix B).

(H2) Concurrent validity

The current HEALTH-PIC overall score demonstrated significant and acceptable correlations with the overall scores of the child PedsQL (r = 0.74, p < 0.001) and the parent PedsQL (r = 0.50, p < 0.001).

The subscales of the HEALTH-PIC also correlated with the subscales of parent proxy HRQoL, namely, physical (r = 0.18, p < 0.001), emotional (r = 0.57, p < 0.001), social (r = 0.33, p < 0.001) and school (r = 0.49, p < 0.001) health, in addition to the subscales of child HRQoL; physical (r = 0.43, p < 0.001), emotional (r = 0.66, p < 0.001), social (r = 0.52, p < 0.001) and school (r = 0.69, p < 0.001) health.

(H3) Discriminant validity

Discriminant validity can be established when the HEALTH-PIC is not significantly correlated with other theoretically different measures. In the case of HEALTH-PIC, the average variance extracted (AVE) was calculated by summing the squared factor loadings and dividing it by the number of items (AVE range = 0.34–0.60). Additionally, the shared variance between the factors of the HEALTH-PIC and mental health (measured via the PHQ-4) was also calculated (shared variance range = 0.01–0.11) and compared. Ultimately, discriminant validity is established as the AVE is greater than the shared variance between the constructs [70]. Specific subscale’s AVE and shard variance can be found in Table 3.

(H4) Criterion validity

In terms of criterion validity, the current HEALTH-PIC overall score demonstrated a significant negative correlation with parent-observed child illness (r = – 0.34, p < 0.001). Looking at the subscales, the physical health subscale of the HEALTH-PIC correlated significantly with daily functioning (r = 0.18, p = 0.003); and the emotional health subscale of the HEALTH-PIC correlated significantly with mood disturbances (r = − 0.51, p < 0.001). The social health subscale of the HEALTH-PIC, however, did not correlate with children’s social life (r = 0.07, p = 0.181), but the school functioning subscale of the HEALTH-PIC also significantly correlated with academic status (r = 0.38, p < 0.001). All correlations for criterion validity can be found in Table 3.

(H5) Internal consistency

The Cronbach’s α coefficient of the HEALTH-PIC was also calculated to determine the internal consistency of the scale. The current scale shows an acceptable reliability of α = 0.76 across all 20 items and α = 0.60–0.66 for each factor [73, 74; see Table 3].

Conclusions of study 3

The findings of Study 3 support HEALTH-PIC's factorial, concurrent, discriminant, criterion validity and internal consistency.

General discussion

The current three-study paper aimed to develop and preliminarily test the HEALTH-PIC, a pictorial measure of early childhood self-reported health-related quality of life. In the first study, we were able to follow the recommendations of Boateng and colleagues [75] and Sauer and colleagues [21] by utilizing the literature and experts’ opinions to develop and revise the items. Subsequently, experts and parents rated the scale’s properties, ensuring that it was clear, relatable and accurate for children. Our actual child population in Study 2 was able to show that both kindergarten and primary school children were able to correctly interpret the items in the picture and complete the questionnaire via scripted directions and that the overall questionnaire was not too long or difficult.

In our last study, we were able to successfully determine the scale’s statistical properties, namely, factorial, concurrent, discriminant, criterion validity and internal consistency.

Despite its general statistical agreement, there was no significant association between the social subscale of the HEALTH-PIC and children’s social life. However, because there were significant correlations between the HEALTH-PIC and parent-proxy HRQoL and child HRQoL, the lack of the aforementioned association might not necessarily indicate the inability of the HEALTH-PIC to assess social health. Instead, this could be because our measure of social life did not account for the quality of children’s friendships and did not quantify the volume of playtime they had with their friends [76, 77]. Future investigations could therefore aim to incorporate detailed assessments that consider friendship quality or closeness to evaluate a child’s social life.

Overall, the HEALTH-PIC serves as a great addition to existing literature, as most measures of child HRQoL are completed via parent proxy. As these parent proxy questionnaires are subject to bias, and may not accurately reflect the underlying child HRQoL [14, 16]. This pattern can also be observed in Study 3, where the correlations between the parent proxy-reported overall HRQoL and its subscales showed lower strength in correlation than when comparing the child self-reported form. Specifically, when looking at the physical subscale and the emotional subscale of the parent-proxy PedsQL, although significant correlations were observed, the magnitude of that compared to child self-reported PedsQL along the HEALTH-PIC demonstrated weaker correlations. This disparity could be explained such that parents may not see their children in school and hence may only base their judgments on home interactions [78].

In addition to parent proxy reports, most HRQoL self-report questionnaires for children in the literature often involve literary items [i.e., KIDSCREEN, 79]; however, the newly developed HEALTH-PIC has demonstrated that even children as young as 4 years of age can understand and complete a self-report questionnaire with assistance from others in reading the instructions. More importantly, we were able to observe a stronger association of the HEALTH-PIC with child illness than the child PedsQL scale (age-suited). Although, this was not the case for the PedsQL parent proxy, which demonstrated a comparable correlation. This may suggest that the HEALTH-PIC is better suited for children aged 4–8 in capturing HRQoL aspects that are directly linked to our measure of child illness.

The nature of pictorial questionnaires can serve as a tool that allows child respondents to better understand and represent the items in the questionnaire when it is supported by relatable pictures [20, 21, 23, 41]. These findings therefore suggest that the HEALTH-PIC can be a more suitable measurement method for our current population of early childhood participants. Unlike some existing scales, our pictorial questionnaire underwent scale validation as suggested by Sauer and colleagues and Tang and colleagues [21, 43], addressing potential gaps in the literature of pictorial scales [21]. Researchers interested in using the HEALTH-PIC can obtain a copy by contacting the corresponding author.

Limitations & further direction

With the strengths and novelty of the HEALTH-PIC, we acknowledge several study limitations. First, in the test of the psychometric properties of the HEALTH-PIC, the adoption of a cross-sectional design in Study 3 limited the level of evidence in our validation study. In particular, the criterion validity of the HEALTH-PIC that we demonstrated in Study 3 was only at a cross-sectional level, which meant that we could not determine whether the HEALTH-PIC was predictive of future health outcomes [80]. Similarly, given its reliability, we were unable to examine the temporal stability of HEALTH-PIC. Future studies could investigate predictive power and test–retest reliability by adopting longitudinal designs [81].

Second, Studies 2 and 3 involved only healthy participants, as we aimed to investigate whether children could successfully complete the questionnaire and validate it without the need to account for confounding variables such as treatment effects or symptom severity. As such, we could not investigate the scale’s sensitivity to changes across diverse health statuses. Future studies could investigate its applicability and sensitivity across different illnesses and symptoms [82, 83].

Third, while developing and testing the HEALTH-PIC, we did not examine whether the reliability and validity of the scale were generalizable across countries or cultures. Although the samples of our studies involved parent‒child dyads from Hong Kong (Study 2) and the United Kingdom (Study 3), we did not collect sufficient cross-cultural data for comparison, which could allow us to examine any measurement invariances between cultural contexts [84]. Future studies could compare the properties of HEALTH-PIC by conducting cross-country/cross-cultural studies [85].

Fourth, while the scale’s development was guided by established frameworks and developmental guidelines, Study 3’s reference questionnaires aimed to target slightly older children (i.e., 5 years old for the PedsQL and 6 years old for the Mood and Feelings Questionnaire/Piers-Harris Children Self-Concept Scale). However, owing to the limited number of existing scales available in the literature that effectively measure children aged 4, we adapted these scales for younger children by involving parents to read aloud the instructions and items. Previous research has also supported the notion of parent-assisted self-reported measures of health and mood in a child population [86]. On a similar note, it is also believed that the properties of the smiley face Likert scale may bias children’s responses in terms of social desirability towards happy faces. Future research could therefore incorporate alternative visual response styles and investigate whether or not child responses would be affected by the response types [38, 87].

Finally, since the current Delphi Method study is heavily based on the existing framework of Aiken[45], Sauer and colleagues [21], Boateng and colleagues [75] and Tang and colleagues [43], future research could incorporate questions that align with the COSMIN recommendations, as this can investigate the relevance of the items within each subscale [88].

Conclusion

The present study aimed to develop the HEALTH-PIC, a pictorial questionnaire for measuring HRQoL in early childhood children aged 4–8. We investigated the face validity and examined the psychometric properties (factorial validity, concurrent validity, discriminant validity, criterion validity, and internal consistency) of the HELATH-PIC through 3 studies. Our findings revealed that the children in our studies were able to complete the developed HEALTH-PIC with the assistance of their parents in reading the instructions. More importantly, the initial evidence supports HEALTH-PIC as a promising assessment tool that may supplement existing measures of HRQoL in early childhood research.

Supplementary Information

Below is the link to the electronic supplementary material.

Author contributions

All authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by TCWT. The first draft of the manuscript was written by TCWT and DKC, all other authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Funding

This work was supported by the Early Childhood Education Departmental Research Grant (DRG) of the corresponding author’s institution.

Data availability

Data can be made available upon request of the corresponding author.

Declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Ethical approval

This research was approved by the Human Research Ethics Committee of the first author’s institution (Ref no. 2021–2022-0424).

Consent to participate

Informed consent was obtained from all individual participants included in the study. For child participants, written informed consent was obtained from their parents.