Measuring health-related quality of life measures in children: lessons from a pilot study

Abstract

There is a debate in the health outcomes literature regarding who the most appropriate respondent is when assessing children’s health-related quality of life (HRQoL). In some cases, parent-proxy may be the only practical option where children are unable to self-complete an HRQoL questionnaire. However, children’s self-reported values may be preferable because HRQoL is subjective and represents the respondent own perception of health. We collected the youth version of the EQ-5D-3L as part of a feasibility study comparing psychoanalytic child psychotherapy with usual care for children aged 5-11 years with treatment resistant conduct disorders. The questionnaires were completed at baseline and 4-month follow-up by the child via face-to-face researcher administration, and by one parent as a proxy respondent. We present percentages of completion at each time-point and investigate the level of agreement between child and proxy-respondent on the child’s health. About two thirds of children (65.5%) were able to complete the EQ-5D-Y at baseline and 34.4% at follow-up. Children and primary carers were mostly concordant regarding overall child’s health. Parents reported more problems in ‘doing usual activities’ and ‘feeling worried, sad or unhappy’ and fewer problems with ‘pain’ and ‘looking after oneself’ than children did. The reports regarding ‘mobility’ were very similar between children and proxy-respondents. The assessment of quality of life by children using selfreport questionnaires is possible with the help of a face-to-face researcher, providing evidence that children should be asked to self-complete HRQoL questionnaires in trial studies.

Introduction

Measuring health-related quality of life (HRQoL) is an important aspect of economic evaluation within health economics. HRQoL questionnaires usually interrogate individuals on a series of dimensions of health that impacts upon their quality of life; these dimensions include physical, mental, emotional and social domains of health (Wolstenholme 2018).

It is preferred that individuals self-complete these questionnaires as individuals are regarded as the best judges of their own health and as such, they will provide an appropriate account of their perception of their own health (Khadka et al., 2019). The importance of reporting an accurate approximation of HRQoL relates to the fact that responses to HRQoL questionnaires will be used when evaluating the effectiveness and the costeffectiveness of competing treatments for various health conditions (Drummond et al., 2015). A very common HRQoL in health research is the EuroQoL 5-Dimension (EQ-5D), which is a generic HRQoL instrument composed of a two-part questionnaire (Wang et al., 2021). The first part consists of a visual analogue scale (EQVAS) which records self-rated health on a vertical scale (similar to a thermometer) where the endpoints are labelled ‘best imaginable - 100’ and ‘worst imaginable health state - 0’ (Rabin & Charro, 2001). The second part is a descriptive system where individuals can rate the state of their health using a descriptive system of the following five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression using three possible levels of severity: i) no problem; ii) some problems; iii) extreme problems¹.

Although self-completion of HRQoL questionnaires is ideal, it is not always feasible for some individuals. For example, individuals who have learning disabilities or child populations may experience challenges in understanding the language within questionnaires and might not be able to complete them. Inaccurate or incomplete completion is likely to produce biased measurement of HRQoL and jeopardise their use in related studies (Jiang et al., 2021). In the context of children and adolescents, there are several commonly used generic instruments that have either been adapted from adult questionnaires or developed especially for their application in child populations (Jiang et al., 2021; Di Biase et al., 2021). The EQ-5D questionnaire is considered suitable for children aged 12 years, however for children younger than this age, the child-friendly version of EQ-5D-Youth version (EQ- 5D-Y) is often used instead. The EQ-5D-Y is a modified version of the standard form of the EQ-5D that has been tested in a survey of 3000 children and young people aged 7 to 17 years (Willie et al., 2010). The modified version comprises the same five dimensions but uses a wording that is comprehensible to younger children (Willie et al., 2010); for example, the ‘self-care’ item was changed to ‘looking after myself’, ‘anxiety and depression’ to ‘feeling worried, sad or unhappy’². Several studies have investigated the validity of EQ-5D-Y as a child HRQoL measure and have shown that it is highly feasible for children (Wolstenholme et al., 2018).

Still, despite research advances in developing child-specific questionnaires, children, especially those of a young age, might lack the cognitive and/or linguistic capacity to self-complete existing child-friendly versions of HRQoL questionnaires (Kwon et al., 2019; Khadka et al., 2019). In these circumstances, proxy-reporting is considered the practical option. This is when parents or caregivers rate the child’s health state based on their behaviour, usually in the presence of an interview guide and as a child-primary carer dyad (Jiang et al., 2021). Proxy-reporting is commonly used in generating HRQoL for cost-effectiveness studies (Otero et al., 2013) and is not limited to quality of life outcomes (Mack et al., 2020). Condition-specific questionnaires have also compared the quality of questionnaire completions with child-primary carer dyads (see for example, DISABKIDS a disease specific instrument for children with chronic conditions (Baars et al., 2005) or Mack et al. (2020) for cancer patients.

Evidence shows discrepancies between HRQoL when reported by parents or reported by children (Jiang et al., 2021; Khadka et al., 2019). A systematic review and meta-analysis comparing self-reported and proxy-reported health utilities found significant differences between proxy respondents and children aged between 3 and 18 years (Jiang et al., 2021). The directions and magnitudes of these differences were inconsistent across different health conditions and valuation methods. For example, proxy-reported HRQoL of children who were obese or overweight were better than when children assessed their own health, whereas in conditions such as infections, mobility impairments and chronic illnesses, proxy-reported HRQoL was worse than that reported by children (Jelsma & Ramma., 2010; Belfort et al., 2011; Robertson et al., 2016; Bray et al., 2017; Kulpeng et al., 2017;Perez-Souza et al., 2018). Additionally, in their systematic review comparing child and proxy-derived child HRQoL, Khadka et al., (2019) found poor agreement between the parents and children for subjective attributes such as cognition, emotion and pain, relative to physical attributes of HRQoL, such as mobility, self-care and speech. Khadka et al. (2019) found that parent-proxies were more likely to underestimate than overestimate HRQoL compared with children, but the magnitude of difference between child and proxy responses varied between the type and severity of health condition being assessed, as well as the age of the child, with adolescents often rating their quality of life slightly lower than their parents.

In this article, we use secondary data from a randomised controlled feasibility trial comparing manualised psychoanalytic child psychotherapy with heterogeneous treatment-as-usual for children aged 5-11 with treatment resistant conduct disorders and their primary carers. HRQoL for the child was reported by the child and by a primary carer (mostly one parent) as a proxy-respondent as part of the trial at baseline and at 4-month follow-up. This paper aims to investigate whether it is a challenge for children aged 5-11 years to answer questions about their own health and whether a primary carer is a suitable proxy respondent when the child cannot self-complete. It must be noted that the study is not about identifying who is the correct respondent when measuring HRQoL of children. As a matter of fact, discrepancies within reports of health between parents and children are not expected to be a measurement problem but may reflect the fact that different people genuinely have different perspectives, and researchers need to work out how they can deal with the differences (Reyes et al., 2015).

Our paper’s addition to the literature is two-fold. First, we consider children with mental health problems, who are seldom considered in this literature. In a very recent systematic review, Jiang et al. (2021) identified only 2 out of 30 studies where comparisons of reporting between children and proxies concerned mental health conditions. Second, the age range of children in the current study is young and is younger than in most studies comparing reports by children and proxies; most studies included in Jiang et al. (2021) concerned children with a median age of 12 and the two studies about mental health conditions included children with median ages of 9 and 15 years old.

Materials and methods

Trial design and participants

The trial on improving intergenerational attachment for children undergoing behavioural problems (TIGA-CUB) was a multi-centre, two-arm, pragmatic, parallel-group, individually- randomised (1:1) controlled feasibility trial. The objective of TIGA-CUB was to determine the practicality and design of a confirmatory trial of the clinical and costeffectiveness of manualised psychoanalytic child psychotherapy (mPCP) compared with heterogeneous treatment as usual (TaU) for children aged 5-11 with treatment resistant conduct disorders and their primary carers. The study included primary carers who had previously been offered a first-line group or individual parenting programme or structured parenting intervention which had not been successful. Primary carers with severe mental health difficulties or severe adverse parental functioning were not recruited in the study (exclusion criteria). The study was conducted in four National Health Service (NHS) Trusts and a total of 7 Child and Adolescent Mental Health Services (CAMHS) in Yorkshire and the West Midlands in the UK. The complete study protocol including ethical aspects is reported elsewhere (Edginton et al., 2017). Eligible dyads comprising a child and a primary carer were randomised on a 1:1 basis to receive either mPCP or TaU. Participants and clinicians were aware of treatment allocation. Participant assessments were undertaken at baseline and at 4 months post-randomisation. This trial feasibility study was carried out to establish robust recruitment and retention strategies and not to measure significant differences between treatment arms.

Health-related quality of life questionnaires

While the health economics component alongside a trial study usually evaluates the cost-effectiveness of a new treatment comparative to treatment as usual (Drummond et al., 2015), in a feasibility trial, this component is designed to check the completeness and the ability to obtain HRQoL data that could then be used to measure health gains in any subsequent cost-effectiveness analysis. Both the child and their parent were invited to complete the two components of the EQ-5D-Y questionnaire (the 5 dimensions descriptive system along with the visual analogue scale) to describe the child’s health at baseline and 4-month follow-up. For children’s responses, we enlisted the help of a face-to-face researcher. The two researchers were recruited because they had prior experience in conducting in-depth, exploratory interviews and received specific training so that they interacted minimally during data collection. To avoid biasing children’s responses, they remained blind to the treatment.

Data analysis

Questionnaires were analysed using descriptive statistics and considered the overall sample independently of the feasibility study arms. Firstly, we calculated the completion of HRQoL questionnaire by both the child and the primary carer at each time-point. We then studied the level of agreement between the child and the proxy-respondent in completing the descriptive system EQ-5D-3L and the EQ-VAS of the EQ-5D-Y, comparing means and standard deviations as well as distributions across the two groups, and we calculated Pearson correlation coefficients. A coefficient value between 0.50 and 1 exhibit strong correlation, between 0.30 and 0.49 moderate correlation, and below 0.29 a low correlation. Analyses were carried out using STATA V15.0³.

Results

Descriptive analysis

Thirty-two dyads were recruited; child mean age at baseline was 7.7 (SD 1.8) years old, 69% were male, 19% were on psychotropic medication (ADHD stimulant medication and/or hypnotics). Amongst the sample, 25% of the children were currently being or had historically been treated for physical health conditions (asthma, eczema), 6.3% were treated for a neurological condition (epilepsy), 15.6% were treated for a mental health condition (anxiety, ADHD, mutism, attachment disorder) and 25% were currently being or had historically been treated for sleep disturbance. For all but two participants, the birth mother was the consenting primary carer, in the other two cases it was the birth father. Out of the parent respondents, 50% had a historic or current mental health condition, however these were not considered severe conditions (Edginton et al., 2018).

Questionnaire completion

Among the original sample of 32 dyads, 4 dyads did not attend the treatment, 26 received partial treatment, and only 2 dyads completed all sessions. Overall, 21 dyads participated in the postal and researcher follow-up. For the purpose of the current paper, we focus on completion of questionnaires related to HRQoL that were part of the follow-up. The percentages of completion for these questionnaires overall are provided in Table 1. For questionnaires with a proxy respondent, there was no missing data for baseline but 8 questionnaires were missing at follow-up (25%); this is due to partial completion of the treatment. About two thirds of the children self-completed the EQ-5D-Y at baseline but only half of them completed it at follow-up. Reasons for the child not to self-complete included: the child was at school when the researcher visited (n=2), the child was unable to concentrate (n=1), the child was selectively mute (n=2), the child had moved to live with relatives (n=1).

A simple descriptive analysis was also performed to see if the missing data for EQ-5D-Y reported by the child were related to child age. Where the child did not selfcomplete their questionnaire at baseline, the average age was 7.8 (range: 5-11), whereas where both child and parent responded, the mean age was 7.6 (range: 5-10).

Level of parent and child agreement

The aim of the current study was to assess the level of agreement between the child self-completed EQ-5D-Y components (visual analogue scale - VAS and descriptive system) and the parent-completed version as a proxy-respondent, so we only focussed on dyads where both questionnaires are available. While at baseline a sample of N=21 dyads were available (65.6% of the baseline study sample), a sample of 11 dyads were available at 4-month follow-up (47.8% of the follow-up study sample).

Table 2 presents the average scores in the EQ-VAS in the full sample. At baseline, child-reported scores for EQVAS were higher overall than proxy-reported (self- 78.0, proxy- 74.1). The reported VAS scores increased between baseline and 4-month follow-up in both child and proxy respondent; however, the increase was higher for proxyrespondents and we observed an opposite pattern at follow- up, with parents self-reporting better overall health for the child than the child did for themselves (self- 80.1, proxy- 85.8). Overall, there is moderate to strong correlation between children’s and parents’ ratings, with higher correlation at follow-up.

Table 3 presents the detailed reports at baseline and follow-up for each of the five dimensions of the EQ-5DY descriptive system. Moderate to strong correlation is observed between ratings of the following three dimensions: ‘looking after oneself’, ‘pain and discomfort’, and ‘feeling sad or unhappy’.

In the mobility level, 100% of the proxy-respondents reported that the child had no problem, whereas 81% of children reported no problem in mobility, 3 (14.3%) children reported they had some problems with mobility and 1 (4.8%) reported a lot of problems. At 4 months, only 2 (18.2%) children reported having some problems while 100% of their proxy respondents reported they had no problems. Thus, with respect to mobility, overall, children reported having more problems than their proxy respondents did.

For the dimension ‘looking after oneself’, the distribution of reports of child and proxy-respondents was very similar at baseline and identical at follow-up. The observed difference at baseline was one proxy-respondent reporting some problems while the child reported no problem.

There were more important differences in the reports concerning ‘doing usual activities’, with parents rating children at worse levels than they used for themselves. While 86% of children reported having no problems with doing some usual activities, only 57% of parents reported no problems. Similarly, 24% of parents reported their children had some problems (versus 9.5% in children) and 19% reported lots of problems (versus 5% in children). The differences between children and proxy-respondents were less marked at the 4-month follow-up.

Table 1.

Completion of health-related quality of life questionnaires by the child and by proxy-respondent at baseline and 4 months follow-up.

Questionnaires	Baseline	4 month follow-up
Overall sample	N (%)	N (%)
EQ-5D-Y (self-completed)	21 (65.6%)	11 (34.4%)
EQ-5D-Y (proxy-respondent)	24 (100%)	24 (75.0%)

Table 2.

Average scores on EQ-VAS self-reported by the child and by proxy-respondent at baseline and 4 months follow-up.

Visual analogue scale (VAS)	Baseline mean (SD)	4 month follow-up mean (SD)
Overall sample	N=21	N=11
EQ-5D-Y (self-completed)	78.0 (25.33)	80.1 (23.45)
EQ-5D-Y (proxy respondent)	74.1 (24.75)	85.8 (20.01)
Pearson’s correlation coefficient	0.43	0.94

In terms of experiencing ‘pain and discomfort’, at baseline, 67% of children reported ‘no problems’, 24% reported some problems and 10% reported a lot of problems. The reports of proxy respondents regarding the child’s pain levels were higher with 81% reporting no problem and 19% reporting some problems. At followup, self-reports and proxy-reports were closer.

In terms of ‘feeling sad or unhappy’ at baseline and follow- up, reports were very different between children and proxy-respondents. While 43% of children reported no problem, only 24% proxy-respondents reported no problem. Inversely, 38% proxy-respondents reported severe problems with sadness and happiness, while 19% of children did. The differences were less marked at follow-up but still present. Proxy-respondents reported higher levels of sadness and unhappiness in children than they did themselves.

Discussion

Two thirds of children completed the EQ-5D-Y at baseline. Age appeared unrelated to child completion. Children and primary carers were concordant in their report of overall health when using the EQ-VAS at baseline. While quality of life appeared to increase over the 4- month follow-up for both the child and their proxy-respondent, their reports differed slightly at follow-up and parents rated children’s overall health with the EQ-VAS higher than the children did themselves. These results are similar to a study exploring interventions in children with obesity, where child self-reports were lower than those of proxies at post-treatment follow-up time points (Robertson et al., 2016). One reason for this might be that children with conduct disorders tend to have quite low self-esteem, which is possibly reflected in their overall HRQoL scores at the end of treatment compared with the parent-reported scores. While children can have complex feelings about the end of their therapy (Wittenberg, 1999) their parents may already perceive the benefits of the treatment on their child’s health status. Moreover, previous studies have shown that parents’ view of their children’s health could also stem from their own health profiles and beliefs (Ungar, 2011).

Table 3.

Responses by dimension and level for self-reported EQ-5D-Y by the child and proxy-reported by the primary carer at baseline and 4 months follow-up.

		Baseline		Overall sample		4 month
	Self-completed		Proxy-completed		Self-completed		Proxy-completed
			Mobility N (%)
Level 1	17 (80.95)		11(100)	Level 1	9 (81.81)		11(100)
Level 2	3 (14.29)		0	Level 2	2 (18.18)		0
Level 3	1 (4.76)		0	Level 3	0		0
Pearson’s coefficient		n/a*		Pearson’s coefficient commercial		n/a*
			Looking after oneself
Level 1	15 (71.43)		14 (66.67)	Level 1	7 (33.33)		7 (33.33)
Level 2	5 (23.81)		6 (28.57)	Level 2	2 (18.18)		2 (18.18)
Level 3	1 (4.76)		1 (4.76)	Level 3	2 (18.18)		2 (18.18)
Pearson’s coefficient		0.43		Pearson’s coefficient		0.54
			Doing usual activities N (%)
Level 1	18 (85.71)		12 (57.14)	Level 1	6 (63.63)		5 (45.45)
Level 2	2 (9.52)		5 (23.81)	Level 2	4 (36.36)		4 (36.36)
Level 3	1 (4.76)		4 (19.04)	Level 3	1 (9.09)		2 (18.18)
Pearson’s coefficient		0.09		Pearson’s coefficient		0.54
			Having pain or discomfort N (%)
Level 1	14 (66.67)		17 (80.96)	Level 1	8 (72.73)		9 (81.82)
Level 2	5 (23.81)		4 (19.04)	Level 2	3 (27.27)		2 (18.18)
Level 3	2 (9.52)		0	Level 3	0		0
Pearson’s coefficient		0.63		Pearson’s coefficient		0.76
			Feeling worried, sad or unhappy N (%)
Level 1	9 (42.86)		5 (23.82)	Level 1	6 (54.55)		3 (27.27)
Level 2	8 (38.09)		8 (38.09)	Level 2	3 (27.27)		6 (54.55)
Level 3	4 (19.05)		8 (38.09)	Level 3	2 (18.18)		2 (18.18)
Pearson’s coefficient		0.31		Pearson’s coefficient		0.76

*Non applicable due to low variability.

Regarding the five specific dimensions of EQ-5D-Y, we found that parents reported the children had more problems in ‘doing usual activities’ and ‘feeling worried, sad or unhappy’ than children did about themselves. This is in contrast to what has previously been reported in the literature, where children tend to self-report more emotional problems than their parents (Reyes et al., 2015). Conversely, parents reported fewer problems with ‘pain’ and ‘looking after oneself’ than children did. The reports regarding ‘mobility’ were, however, very similar between children and proxy-respondents. Coherence in reported EQ-5D dimensions between self and proxy respondents has been investigated in previous studies; they show that parents tend to underestimate domains associated with pain, emotion and cognition and that there is consistently poor agreement between children and their parents within the psychosocial domains in general (Khadka et al., 2019; Wolstenholme et al., 2018).

Furthermore, consistent with our results, studies have shown correlations between child and parent responses for the physical attributes of quality of life such as ‘mobility’ (Khadka et al., 2019; Wolstenholme et al., 2018). This is thought to be due to the fact that it is easier for parents to make judgments and assumptions about what they can observe physically rather than the more subjective aspects of health such as emotion and pain, which are more difficult to ascertain if they are not clearly communicated between parent and child. As with children who have experienced trauma, it might be that children with conduct disorders experience increased difficulty in verbalising their feelings, or feel if they do, their parents will not respond appropriately (Jackson, 2004; Sharp, 2014). Furthermore, a study investigating the appropriateness of using children aged 3-5 years to self-complete the EQ- 5D-Y found poor retest reliability across this age group for answering questions on the domain ‘looking after oneself’ (Verstraete et al., 2020). Most parents of children across these age groups did not find it appropriate for young children to self-report on this domain, as they do not have the same autonomy and agency over this as adults do, which may in part explain the discrepancy between children and proxy response with this domain (Verstraete et al., 2020). Children with conduct disorders may have felt more omnipotent at the beginning of treatment but may have developed a more realistic perception of their place within the world subsequently (Magagna & Piercey, 2020).

Interestingly, we observed more coherence between self- and parent-reported HRQoL at 4-month follow-up than baseline. In particular, differences between children and proxy-respondents were less marked at 4-month follow- up for ‘mobility’ and ‘doing usual activities’, however this was not the case for ‘feeling sad or unhappy’ and it might be that this domain was more a reflection of the parent’s state of mind rather than an account of the child’s feelings (Manfredi et al., 2016; Misciosa et al., 2018). This higher agreement at follow-up between parents and child could be related to the child receiving treatment and both the parent and the child being more aware of the health status of the child. There is potential for this to be explored in a large-scale confirmatory trial.

Our results converge with studies finding discrepancies between child and parent’s report of the child’s health when using psychiatric specific instruments such as the Strengths and Difficulties Questionnaire (SDQ) or the Child Behaviour Checklist (CBCL) (Achenbach & Rescorla 2000; 2001; Goodman, 1997). Studies on child populations with psychiatric diagnoses show moderate to low agreement between parents and children when reporting on internalising (less observable) disorders such as anxiety and depression using both CBCL and SDQ, whereas there are higher levels of agreement between children and adults when reporting on externalising disorders (more observable) such as ADHD and conduct disorders. Children tend to report more internalising problems than their parents as these may be concealed from the parents’ view, emphasising the importance of child self-reporting. On the other hand, children tend to under report externalising problems while parents over report, which may be a result of the externalising conditions being directed at the parent or suggest that children may not be accurate reporters of these symptoms (van der Meer et al., 2008; Salbach- Andrae et al., 2009). Using the CBCL, there was higher parent-child disagreement where the child suffered from comorbid psychiatric disorders (Salbach-Andrae et al., 2009). This is important to note as psychiatric comorbidities can suggest greater impairment in the child response to therapy. This is relevant to our study, as children recruited to the TIGA-CUB trial also presented with psychiatric and/or physical comorbidities which may have impacted on how they responded to treatment.

This study presents some limitations. First, this is a pilot study and we were provided with limited data that did not allow analysis beyond descriptive statistics; it would be valuable to investigate how these findings sustain with a larger scale sample. Second, missing data due to loss of follow- up or withdrawal from treatment is common in clinical studies, especially in mental health settings, and resulted in a data analysis of only 24 dyads from the initial 32 dyads recruited. Third, despite face-to-face researchers being trained to collect data, we cannot fully confirm that they did not interfere with the children’s responses. Finally, the study only collected the quantitative completion of the HRQoL instrument and did not consider using qualitative methods to assess the discrepancies between parents and children when measuring the health of children.

Conclusions

This study adds new results to the limited literature looking into valuing HRQoL in children and adolescents with mental health disorders. We found that the assessment of HRQoL by children using self-report questionnaires is possible with the help of a face-to-face researcher even in children as young as 5 years old. Children should be asked to self-complete HRQoL questionnaires in trial studies ab initio. The coherence of self- and proxy-health reports for mental health conditions in children has not been studied much before. Our results showed that parents can respond as proxy for children; however, they are only second best and their reports might differ from children’s self-reports. This is not to say that parents’ perception of their children’s health is not a valuable sense of information, as this information can be used in adjunct with the child self-report to gain a further understanding of treatment impact and how children internalise or externalise certain symptoms, reflected in HRQoL domains. Furthermore, where adults have mental health difficulties of their own, this could be an impacting factor in the way they perceive their children’s health (Misciosa et al., 2018), but this question is beyond the scope of this paper and should be explored as a future research avenue.

This study contributes to the growing literature in assessing child’s quality of life within mental health and may be drawn upon to help evaluate therapeutic directions for this population. We show that a multi-informant approach is valuable as parent and child’s agreement on the problem impacts how well they might co-operate emotionally in terms of achieving their therapy goals. These original, but preliminary, results relying on a feasibility trial should be confirmed in a large-scale trial and this is the next step on our research agenda.