Measuring quality of life in pediatric palliative care: challenges and potential solutions

Abstract

Annually, about 500,000 children are coping with life-limiting illnesses in the USA. Integrated pediatric palliative care program could benefit some of these children by improving their health-related quality of life (HRQOL). To measure the effect of pediatric palliative care programs on HRQOL, a valid and reliable tool must be identified. This study aimed to validate the psychometric properties of a generic HRQOL instrument, the Pediatric Quality of Life 4.0, for children with life-limiting illnesses. Analyses were conducted using telephone survey data collected from 266 parents whose Medicaid-enrolled children had life-limiting illnesses. Results of the analyses suggest the Pediatric Quality of Life 4.0 does not have valid psychometric properties for measuring HRQOL within this population. Our study documents several challenges in using the generic instrument to measure HRQOL in pediatric palliative care setting. We point out future directions to refine or develop HRQOL instruments for this population of vulnerable children.

Introduction

Each year in the USA about 500,000 children are coping with life-limiting illnesses,^1–4 and there is growing interest in addressing their physical and psychosocial needs through comprehensive programs.^5–9 Experts from the American Academy of Pediatrics and the Institute of Medicine recently called for the integration of pediatric palliative care into ongoing medical management from the point of diagnosis to the end of life.^10,11 To answer that call, a more appropriate model of integrated pediatric palliative care was developed. Since 2005, two states (Florida and Colorado) have developed and implemented programs for publicly insured children.¹²

The purpose of the integrated pediatric palliative care program is to help children with life-limiting illnesses and their families receive both curative and palliative care services simultaneously. For the purposes of program eligibility, the State of Florida defined life-limiting illnesses as those conditions where death might occur by the time the child reaches his or her 21st birthday. Palliative care services include play therapy for the child and his/her siblings, sibling counseling, parent counseling, music therapy, and other supportive services. The healthcare team includes physicians, nurses, psychologists, child life specialists, and music and art therapists.³ Prior to the implementation of this program, parents had to choose between curative and life-prolonging therapies and palliative care because Medicaid would not reimburse for both types of services simultaneously. Now these children can receive both types of care, which is believed to enhance their health and quality of life.³

As a matter of fact, a specific aim of the integrated palliative care program for children is to improve their health-related quality of life (HRQOL). This is done through expert management of pain and other physical symptoms such as shortness of breath, nausea, vomiting, and anxiety. It is also done through emotional and spiritual support services, offering the patient and family specialized counseling to help them cope with emotional distress that results from dealing with a serious illness or condition.¹³ Given that a key goal of these programs is to improve HRQOL, health plan administrators, providers, researchers, advocates, and policy-makers want to measure and determine the effectiveness of pediatric palliative care programs on HRQOL.

HRQOL instruments are either disease-specific or generic. As pediatric palliative care programs usually provide care to children with diverse diseases, generic instruments are preferred. Generic HRQOL instruments assess basic functioning for physical, emotional, and social health. Some studies also include additional items measuring pain, fatigue, and depression. Our comprehensive literature review using PubMed yielded over 50 studies of HRQOL with children who have chronic conditions, including those that are life-limiting. Among these studies, the most commonly used instrument is the Pediatric Quality of Life 4.0 Generic Core Scales (PedsQL).¹⁴ Not only is the PedsQL widely used, it has been validated in a variety of conditions such as asthma, cancer, heart disease, rheumatology, and diabetes.¹⁵

Although the PedsQL has been validated in children who have specific chronic illnesses, including some that are potentially life-limiting such as cancer, its usefulness for children enrolled in integrated pediatric palliative programs has not been determined. To our knowledge, only one pediatric palliative care study used the PedsQL to measure HRQOL for 21 children enrolled in the Seattle Pediatric Palliative Care Project.¹⁶ Findings from this study suggest that between baseline and follow-up, HRQOL was improved in three out of the four PedsQL domains. Yet, the characteristics of children who have chronic illnesses that are not life-limiting and those who are eligible for integrated pediatric palliative care programs may be very different. It is critical to validate psychometric properties of a HRQOL instrument for use in integrated pediatric palliative care programs.

As the PedsQL has been validated for use in children with a range of conditions, including some that are life-limiting, it may be a useful alternative. The aim of this study was to evaluate the psychometric properties of the PedsQL using a sample of parents whose children have life-limiting illnesses. We used the following psychometric methods to validate the PedsQL: construct validity, scale reliability, item-domain convergent and discriminant validity, and known-groups validity. Information derived from this study is crucial for exploring the challenges in measuring HRQOL for this population and suggesting potential solutions that would be useful for those interested in advancing integrated pediatric palliative care.

Methods

Sample

We identified children enrolled in Florida’s Medicaid program between April 2006 and March 2007 who had life-limiting illnesses and met Florida’s Medicaid guidelines for integrated pediatric palliative care admission (N = 1251).¹⁷ All children in the study were aged 2 to 18 years. We sent an introductory letter to a random sample of 936 parents whose children meet our criteria for selection. Telephone surveys were conducted between November 2007 and April 2008. Four hundred and eighty-nine of the potential subjects had invalid contact information. Overall, 266 surveys were completed (response rate 59.5%). The final sample size provided sufficient power for performing psychometric analyses, which requires at least 10 completed surveys per item or a minimum of 230 subjects. The University of Florida’s Institutional Review Board approved this study.

Measures

The PedsQL is a generic pediatric HRQOL instrument, which has 23 items measuring four relevant domains: physical functioning (eight items), emotional functioning (five items), social functioning (five items), and school functioning (five items).¹⁴ The PedsQL can be administered to parents of children aged 2 to 18 years or to children themselves. Four age-specific versions (2–4, 5–7, 8–12, and 13–18 years) were developed with minor modifications in the wording of items based on the children’s ages. In this study, we focus on parents’ reports of their children’s HRQOL rather than children’s self-reports because one of our previous studies suggests that many children with life-limiting illnesses are too sick to answer the survey.¹⁷ In addition, shared decision-making with physicians about the child’s treatment relies on the parent’s observation of their child’s illness progress, including HRQOL.

The parent was asked how much of a problem a specific function has been for their child in the past month. Response categories for each item are ‘never’, ‘almost never’, ‘sometimes’, ‘often’, and ‘almost always’. Domain scores are calculated by summing item scores in a specific domain and then linearly transforming to a 0–100 scale, where 0 indicates the lowest HRQOL and 100 the highest.

Information about the children’s health status was collected to validate the PedsQL. Specifically, we used the clinical risk groups (CRGs) to validate the PedsQL.¹⁸ The CRGs query over 2000 diagnoses and procedures from the healthcare claims and encounters and assign children to one of five categories. The five categories are: (i) non-significant, non-acute including children whose underlying chronic condition was not recorded in the claims data but were seen for routine care or whose primary expenditures were pharmacy services; (ii) significant acute conditions including children with acute illnesses that could be precursors to, or place the child at risk for developing a chronic disease; (iii) minor chronic conditions including children with illnesses that can usually be managed effectively with few complications; (iv) moderate chronic conditions including children with illnesses that are variable in their severity and progression, can be complicated, and require extensive care; and (v) major chronic conditions including children with illnesses that are serious, and often result in progressive deterioration, debility, or death. Demographic data including the children’s age, gender, race/ethnicity, and parents’ educational attainment were also collected.

Quantitative analyses

Descriptive analyses were performed to estimate the mean, median, and standard deviation of the item and domain scores. To understand the magnitude of the impairment of HRQOL among children with life-limiting illnesses, we compared HRQOL scores of the present population to a ‘reference group’, which was based on a previous study that we conducted.¹⁹ The reference group comprised of 1745 representative children in Florida who were between 2 and 18 years old and enrolled in the Florida KidCare Program, which consists of Medicaid and the Title XXI State Children’s Health Insurance Program. Health status of children in the reference group varied, from healthy children to children with special healthcare needs, including acute and complex chronic conditions, and life-limiting illnesses.

In addition to the descriptive analyses, the following psychometric analyses were conducted: scale reliability, construct validity, item-domain convergent/discriminant validity, and known-groups validity.²⁰

Scale reliability

To measure scale reliability, known as internal consistency, Cronbach’s alpha coefficients were estimated to indicate the degree to which items of the same domain yield consistent results. Alpha coefficients above 0.7 were deemed acceptable for the purpose of group comparisons.²⁰

Construct validity

To measure construct validity, a confirmatory factor analysis was performed to examine how the constructs of the PedsQL correspond to the scale structure as designed. The degree to which factorial structures of the PedsQL can be well replicated in the population of children with life-limiting illnesses was tested. Two indicators were used to determine the goodness of model fit to the data: the comparative fit index (CFI) and root mean square error of approximation (RMSEA). Values of CFI >0.9 and RMSEA <0.06 were used to indicate a satisfactory goodness of fit.²⁰

Item-domain convergent/discriminant validity

Item-convergent and discriminant validity measures whether the designed items capture the concept the corresponding domains intend to measure. Pearson’s correlation coefficient was used to examine item-domain convergent/discriminant validity. Specifically, convergent validity estimates the correlation between the score of a specific item with the score of the corresponding domain (with overlap adjustment). Discriminant validity examines the correlation of the score of a specific item with the score of the other domains. To be satisfied with the convergent and discriminant validity, correlation coefficients larger for convergent validity as compared with discriminant validity should be observed.

Known-groups validity

Known-groups validity was used to examine the extent to which the instrument discriminates between health status groups which are significantly associated with HRQOL. The instrument should be sensitive enough to detect underlying differences between children with higher and lower HRQOL. Mplus version 5.1 was used to perform construct validity and STATA version 10.0 was used to perform the remaining analyses.

Results

Sample characteristics

Table 1 shows the characteristics of the sample. Fifty-five percent of the respondent’s children were male and the majority of respondents had graduated from high school (35%). Race/ethnicity of the respondents was 41% White, non-Hispanic, 24% Black, non-Hispanic, and 30% Hispanic. Most respondents were married (49%). Although not shown in Table 1, mean age of respondents was 43 years old (SD = 11.5) and mean age of the respondent’s children was 12 years old (SD = 5.5).

Table 1.

Sample characteristics

Characteristics	Percent, number of observations
Child’s gender
Male	55.3%, 147
Female	44.7%, 119
Parent’s education background
Below high school	22.2%, 59
High school or general educational development	35.0%, 93
Vocational, technical or business certificate or diploma	22.2%, 59
Associate degree or above	20.7%, 55
Parent’s race/ethnicity
Hispanic	29.7%, 79
White/non-Hispanic	41.4%, 110
Black/non-Hispanic	24.4%, 65
Other	4.5%, 12
Parent’s marital status
Married/common law	48.5%, 129
Divorced/separated	24.1%, 64
Single	25.2%, 67
Widowed	2.3%, 6

Domain scores

Table 2 shows the distribution of PedsQL domain scores. The ‘reference group’ column in Table 2 shows scores from a prior study where we assessed HRQOL for 1745 children enrolled in Florida’s Medicaid program.¹⁹ Across all domains, children with life-limiting illnesses scored significantly lower than the reference group (p < 0.001). The mean scores of all domains were slightly skewed to the right. Floor effects were not significant in any of the domains, while ceiling effects were slightly larger in the domains of emotional (10.2%) and social functioning (9.7%) compared with other domains.

Table 2.

Distribution of domain scores^a

Domain	Mean	Median	SD	Floor effect (%)	Ceiling effect (%)	Reference group
Physical	52.57	26.52	28.30	3.52	2.64	81.08
Emotional	65.56	65.00	22.31	1.33	10.18	75.80
Social	58.38	60.00	24.46	0.88	9.65	76.95
School	52.20	50.00	24.85	5.26	4.39	71.57

^aThe range of score is 1–100; higher score indicates better HRQOL.

At the item level, the distribution of item scores was not normal. A greater portion of parents reported that their children ‘never’ had a problem in the items of emotional (mean: 36%; range: 24–42%) and social functioning (mean: 35%; range: 21–48%), followed by physical (mean: 32%; range: 19–48%) and school functioning (mean: 23%; range: 14–32%). By contrast, a moderate portion of parents reported that their children ‘almost always’ had a problem in the items of physical functioning (mean: 21%; range: 13–25%) and social functioning (mean: 19%; range: 6–37%), followed by school functioning (mean: 14%; range: 8–20%) and emotional functioning (mean: 7%; range: 4–10%). Importantly, parents reported that the content of several items was not applicable to their children, particularly related to physical functioning (mean: 9%; range: 1–15%) and school functioning (mean: 5%; range: 3–8%), as opposed to social functioning (mean: 3%; range: 2–3%) and emotional functioning (mean: 2%; range: 0–6%). For example, related to physical functioning, a follow-up question was asked to determine why the item was not applicable and parents reported that their children used a wheelchair, remained in bed, or were otherwise unable to walk.

Scale reliability

As shown in Table 3, Cronbach’s alpha coefficients suggest that the PedsQL demonstrates acceptable internal consistency reliability for all domains (alpha greater than 0.7), except the domain of social functioning.

Table 3.

Reliability and item-domain convergent/discriminant validity

Domain	Reliability (Cronbach’s alphaa)	Convergent validity	Discriminant validity
Physical	0.85	0.37–0.74	0.16–0.45
Emotional	0.74	0.45–0.60	0.19–0.43
Social	0.60	0.34–0.54	0.05–0.57
School (2–4 years)	0.85	0.63–0.80	0.25–0.63
School (5–18 years)	0.70	0.26–0.57	0.05–0.44

^aAcceptable Cronbach’s alpha: >0.70.

Item-domain convergent/discriminant validity

Table 3 also shows results for the item-domain convergent/discriminant validity. Pearson’s correlation coefficients suggest that convergent/discriminant validity was not satisfied for the PedsQL. Correlations between the scores for a specific item with its own domain (with overlap adjustment) were not significantly larger than the correlations between the scores for a specific item with other domains. For example, correlations between item scores of social functioning and the domain score of social functioning (0.34–0.54) were not discernable when compared with the correlations between items of social functioning with the other domains (0.05–0.57).

Construct validity

Although not shown in the tables, findings from confirmatory factor analysis suggest that the sample data do not replicate the hypothesized four-factor structure of the PedsQL. The CFIs were 0.90 and 0.69 for age groups 2–4 and 5–18 years, respectively. However, the RMSEAs were 0.20 and 0.25 for the same age groupings, implying an unacceptable model fit.

Known-groups validity

Findings from Table 4 suggest that the PedsQL does not satisfy known-groups validity. Using the CRGs as the known groups, parents of children with more severe conditions were no more likely to report lower HRQOL than parents of children with less severe conditions across all domains of the PedsQL.

Table 4.

Known-groups validity

Clinical risk group	Physical	Emotional	Social	School
Minor	52.08	65.42	57.22	56.11
Moderate	50.74	67.78	51.70	52.58
Major	52.86	64.70	60.18	51.79
F-ratio*	0.11	0.37	2.37	0.14

All comparisons with p > 0.05.

Discussion

This study investigated the psychometric properties of the PedsQL within a population of children who were eligible for an integrated pediatric palliative care program. We used standard psychometric tests, including Cronbach’s alpha coefficients, confirmatory factor analysis, and clinically relevant known-groups. Comparing the results from this study with a previous study where we assessed the HRQOL for 1745 children enrolled in Medicaid, the results show that HRQOL for children with life-limiting illnesses was significantly impaired across all domains of the PedsQL compared with the reference group. Moreover, variations of the domain scores were larger for children with life-limiting illnesses compared with the reference group. However, this comparison may not be valid because we were unable to replicate the psychometric properties of the PedsQL in this study.

Our confirmatory factor analysis did not support construct validity of the PedsQL, implying that the hypothesized HRQOL structures between children with life-limiting illnesses and other populations may be different. Furthermore, item-domain convergent and discriminant validity of the PedsQL was not satisfied, suggesting that items meant to be clustered on the hypothesized domains did not really measure that concept. These findings may be partially due to the fact that the parents of children with life-limiting illnesses interpreted the meaning of a specific item in a way that was not originally intended for that item.²¹ For example, parents who have children with limited mobility may incorporate information about their children’s social and school functioning when responding to items about physical functioning. It is beyond the scope of the current study to determine how parents were truly interpreting the items. Qualitative interviews with parents whose children have life-limiting illnesses about their perceptions of the items would be important for future research.

Evidence of known-groups validity did not support the hypothesized relationships of HRQOL with the CRGs. Our study found that the most severely ill group, as measured by the CRGs, did not demonstrate impaired HRQOL on the PedsQL as compared with their less severely ill counterparts, which contradicts the extant pediatric HRQOL literature. In our prior study that assessed HRQOL among children enrolled in Florida’s Medicaid program, we found physical functioning scores of 70.3, 65.4, and 56.4 for those with minor, moderate, and major chronic conditions, respectively.¹⁹ The association between high illness severity and impaired HRQOL was also observed in the emotional, social, and school functioning.¹⁹ Perhaps using the CRGs to validate the PedsQL was less appropriate for this population in that data gathered for generating the CRGs was based on a 1-year window prior to HRQOL assessment. For children with life-limiting illnesses whose diseases progress rapidly, there may be a time lag between assessed HRQOL, as determined by the CRGs, and genuine health status. It is also possible that the unexpected findings are related to parental adjustment and adaptation about their children’s functioning in the face of a life-limiting illness. The parents may view their children’s functioning in a more positive way than the children’s clinical condition warrants. The finding may be related to confounding between different illness severity groups on unmeasured variables such as parents’ psychosocial functioning and social support.

Measuring HRQOL is a challenging endeavor for children with life-limiting illnesses because some items in the extant HRQOL instruments, especially generic instruments, may not be appropriate for many of them. For example, in this study we found that the items in physical functioning which measured ‘problem walking more than one block’ and ‘problem with running’ were not applicable for 15% of children whose parent reported that they were in bed or used a wheel-chair. This finding implies that the use of generic instruments with a static module, whereby subjects are required to answer all items regardless of their applicability to their children, is imprecise. Similarly parents of children who are non-verbal, non-communicative, and/or have cognitive impairment struggled to answer questions such as ‘does your child worry about what will happen to him or her?’. The third challenge is that the use of generic instruments may not fully capture the impact of life-limiting illnesses on daily functioning and well-being. The domains which are important to this group of children, but are not covered in generic instruments, may include fatigue, pain and symptoms, ancillary health states (e.g. vision, hearing, speech, etc.), financial impact, and family functioning and family cohesion.^22,23

Although the findings may be discouraging to programs that use, or have considered using the PedsQL, this study does build a foundation for improving HRQOL instruments to allow the integrated pediatric palliative care community to move toward the ultimate goal of measuring HRQOL. There are several potential strategies that can be pursued to improve HRQOL measurement, including revising the extant instruments or developing a new measurement module. As previously described, many items in the PedsQL are inappropriate because the tasks/situations (e.g. walking more than one block) that the items measure are not possible for some children with life-limiting illnesses to accomplish. We believe that adding some condition-specific items, such as ‘moving more than one block using equipment such as a wheelchair’ will be beneficial to our population. In addition, all items may not be equally useful for our study population, as evidenced by the wide variation of score distribution in our study population relative to the reference population. This finding suggests that the use of static models, meaning administering all items to all subjects, will increase measurement error and decrease precision. Therefore, the use of item response theory, along with dynamic methods, such as computerized adaptive tests, might better assess the HRQOL for this population.²⁴ Finally, given the poor construct validity of the instrument for use on our heterogeneous population, it may be more appropriate to consider the use of individualized measurement tools rather than standard health profiles.²⁵ This individualized approach allows children and/or parents to identify the relative importance of domains and items in the domain, leading to customization of HRQOL measures. This approach also addresses the challenge that some domains of HRQOL are not measurable, such as mental functioning or pain for children who are non-verbal, non-communicative, and/or have cognitive impairment, and these aspects of HRQOL may be regarded as less important or ignored by parents.

Several study limitations merit attention. First, the response rate for the survey was 59.5%. While this response rate is consistent with other surveys conducted with Medicaid eligible pediatric populations,^19,26 there may be inherent differences between responders and non-responders. Second, the generalizability of our findings to the overall population of children with life-limiting illnesses is limited because the underlying characteristics of parents of Medicaid children in this study may be different from parents of children from other socioeconomic backgrounds. These underlying factors may lead to different response patterns to the quality-of-life surveys. Third, we chose the CRGs as our known-groups, which queries diagnostic and procedure codes to classify children into health status categories. Inherent biases may exist in the CRGs in that they do not account for gaps in coverage and provider miscoding that may have occurred. Finally, this study collected and validated cross-sectional data, which does not take into account changes in the children’s HRQOL associated with their disease progression. It is important to validate longitudinally HRQOL instruments within this population and over disease trajectories.

Despite these limitations, to our knowledge, this is the first study to provide evidence about the psychometric properties of any HRQOL instrument within a population of children who are eligible for pediatric palliative care. In the USA accurate measurement of HRQOL is increasingly important as more states receive approval to provide publicly funded, integrated pediatric palliative care programs. Recent proposed legislation (US House Resolution 6931) would require all children eligible for Medicaid to receive hospice services earlier than the last 6 months of life.²⁷ Consequently, our study results could have immediate impacts on states as they seek guidance on how to measure pediatric HRQOL for their participants. Finally, our study emphasizes the need for further testing of existing HRQOL instruments or even the development of a new HRQOL instrument.