Comparison of four pediatric health-related quality of life Instruments: a study on a Medicaid population

Abstract

Background

Few studies have compared multiple health-related quality of life (HRQOL) instruments simultaneously for pediatric populations. This study aimed to test psychometric properties of four legacy pediatric HRQOL instruments: the Child Health and Illness Profile (CHIP), the KIDSCREEN-52, the KINDL, and the Pediatric Quality of Life Inventory (PedsQL).

Methods

This study used data of 908 parents whose children (ages 2–19) were enrolled in Florida Medicaid. Parents were asked via telephone interview to complete each instrument appropriate to the age of their children. Structural, convergent/discriminant, and known-group validities were investigated. We examined structural validity using confirmatory factor analyses. We examined convergent/discriminant validity by comparing Spearman rank correlation coefficients of homogeneous (physical functioning and physical well-being) vs. heterogeneous (physical and psychological functioning) domains of the instruments. We assessed known-groups validity by examining the extent to which HRQOL differed by the status of children with special health needs (CSHCN).

Results

Domain scores of the four instruments were not normally distributed and ceiling effects were significant in most domains. The KIDSCREEN-52 demonstrates the best structural validity, followed by the CHIP and the KINDL, and the PedsQL. The PedsQL and the KIDSCREEN-52 show better convergent/discriminant validity than the other instruments. Known-groups validity in discriminating CSHCN versus no needs was the best for the PedsQL, followed by the KIDSCREEN-52, the CHIP, and the KINDL.

Conclusion

No one instrument was fully satisfactory in all psychometric properties. Strategies are recommended for future comparison of item content and measurement properties across different HRQOL instruments for research and clinical use.

BACKGROUND

There is a growing interest in using pediatric health-related quality of life (HRQOL) measures to evaluate effectiveness of clinical interventions and/or health care programs (1). HRQOL measures aim to assess various aspects of a patient’s functional status and well-being, including physical, psychological, and social domains (1, 2). Evidence suggests exploring and discussing HRQOL issues can improve communication and promote shared decision-making between physicians and patients (3–5). HRQOL reported by patients is particularly important in pediatric clinical settings because it helps physicians detect children’s psychosocial issues in routine practice (6, 7).

In the last two decades, more than 30 generic and 60 disease-specific HRQOL instruments have been developed for pediatric populations (8). Several of the commonly used instruments include the Child Health and Illness Profile (CHIP) (9), the KIDSCREEN-52 (10), the KINDL (11), and the Pediatric Quality of Life Inventory (PedsQL) (12). Ideally, the development of good pediatric HRQOL instruments should be based on a conceptual framework that accommodates multiple aspects of the child’s health (e.g., physical, emotional/psychological, and social) and developmental issues (1). In addition, they should be as brief as possible to reduce administrative burden and maintain good psychometric properties including reliability, validity, and responsiveness (1). Although the aforementioned four instruments were developed based on the concept of health (13), each instrument did not measure exactly the same aspects of children’s health and functional status. These include, but are not limited to, physical health, psychological/emotional health, social interaction, and school activity. The PedsQL focuses on physical, emotional, social and school functioning, and has the shortest length compared to the KIDSCREEN-52, the KINDL, and the CHIP. The KIDSCREEN-52 and the CHIP are the lengthiest (52 and 45 items, respectively) among the four instruments, but they include unique domains that are not present in the PedsQL and the KINDL. Specifically, the KIDSCREEN-52 includes financial resources and autonomy domains, and the CHIP includes the domains that related to the child’s future health and development such as risk avoidance and resiliency. Each of these instruments has demonstrated adequate psychometric properties in their original evaluations, yet these instruments have not been fairly compared to each other based on the same study sample.

The design and administration of HRQOL measures is a challenging endeavor in pediatric research. Although the FDA (14) and previous research (15) suggest collecting HRQOL data directly from children to capture their own perception of health and functional status and to avoid the potential bias for data derived from parents, parent-proxy reports still provide unique information and are demanded in clinical settings (16, 17). If a child is too young to comprehend and report HRQOL or cannot respond due to physical, psychological, or cognitive problems, the use of parent-proxy report is important. The parents’ perception of the child’s HRQOL influences decision-making for health care utilization on behalf of their children (18, 19), reflects the quality of health care services the child receives (20), and links to risk factors of poor health outcomes associated with the socioeconomic position (1, 21, 22). Importantly, few studies focus on psychometric properties of HRQOL measures for children enrolled in Medicaid, who are an under-studied population with a greater risk for chronic conditions and worse health outcomes than high-income, privately insured children (21, 23).

Given the practical needs of parent-proxy reports of HRQOL instruments for a publicly insured pediatric population (21), it is important to compare the measurement properties of frequently used instruments to identify if an instrument has superior quality in measuring pediatric HRQOL than other instruments (8, 24). Using an HRQOL instrument with inferior measurement properties may bias the comparisons of different treatment regimen outcomes and mislead decisions made by clinical and policy stakeholders. Previous research comparing pediatric instruments is limited to review studies which evaluate HRQOL instruments across different populations and different study designs (13, 24–27) and empirical studies which do not deliver each instrument to the same population (28). Important design factors for fair comparisons among HRQOL instruments include a large sample size, the use of the same sample to evaluate multiple HRQOL instruments, and application of comprehensive and sophisticated psychometric methods for data analysis (29). A good pediatric HRQOL instrument must demonstrate acceptable measurement properties such as reliability and construct validity (2, 30). The comparison of four commonly used pediatric HRQOL instruments may guide the appropriate selection of an instrument for use in future studies or healthcare settings involving a Medicaid population.

To our knowledge, no large-scale studies have compared measurement properties of multiple pediatric HRQOL instruments using the same sample simultaneously. In light of limited empirical studies, the present study aimed to compare four renowned generic pediatric HRQOL instruments (the CHIP, KIDSCREEN-52, KINDL, and PedsQL) based on children enrolled in the Florida Medicaid program. The primary research question asks what are the measurement properties of parent-proxy versions of the aforementioned four instruments? We evaluated the measurement properties of reliability and construct validity (structural validity, convergent/discriminant validity, and known-groups validity). The second research question asks do these instruments measure the same concept of HRQOL, and does one instrument have superior validity and reliability compared to the others?

METHODS

Data collection

The study sample was children enrolled in the Florida Medicaid program. Throughout the manuscript, we use the terms “children” and “child” to refer to the individuals from the time of birth to their 18^th birthday (31). In this Medicaid sample, the majority of families are within 100% of the federal poverty level (FPL), with some participants within 155% of the FPL if the child is between the ages of 2 and 5. To be eligible for study participation, families were required to have 12 months of continuous Medicaid enrollment. For the eligible families, 5,879 phone numbers were available for contact for phone interview. However, 2,873 phone numbers were disconnected, non-working, or fax lines. Of the remaining 3,006 eligible numbers, 908 parents agreed to participate and completed the interview. The overall response rate was 30.2% (i.e., 908/3006). Ages of children were stratified by three strata (2–7, 8–12, and 13–17 years). The survey was conducted through the telephone interview and participants spent between 40 to 50 minutes to complete the survey. This study was approved by the University of Florida Institutional Review Board.

Survey instruments

Four pediatric HRQOL instruments (the CHIP, the KIDSCREEN-52, the KINDL and the PedsQL) and the Children with Special Health Care Needs (CSHCN) Screener were administered using parent-proxy reports. The instruments were delivered in the following sequence: the CHIP, the CSHCN, the KIDSCREEN-52, the PedsQL, and the KINDL (Kiddy or Kid/Kiddo). Age-appropriate versions of the instruments were administered for each child. Each instrument was scored according to the developers’ guidelines and all items were scored so that higher scores indicate better HRQOL. The domain scores and the total score were transformed to a 0–100 point scale, with 100 representing the best HRQOL. Missing item information was imputed based on each instrument’s respective guidelines. Because missing on all items was less than 5% for each participant, we retained the entire sample in the analyses (n=908).

Table 1 compares the characteristics of the pediatric HRQOL instruments. Briefly, the CHIP was designed to identify risk populations of poor functional status and HRQOL in epidemiology studies, and to assess the effects of health services and public policy on children’s health outcomes (9, 32). The CHIP contains 5 domains (45 items) including satisfaction (5 items), comfort (12 items), resilience (8 items), risk avoidance (8 items), and achievement (8 items).

Table 1.

Description of instruments used in this study

Instrument	Age ranges designed in the instruments	Age ranges used in this study	Domains (number of items)	Response category*
Child Health and Illness Profile (CHIP) (9)	Child: 6–11 Adolescent : 12–17	Child : 6–11	5 domains (45) Satisfaction (9) Comfort (12) Resilience (8) Risk avoidance (8) Achievement (8)	1=“never”, 2=“almost never”, 3=“sometimes”, 4=“almost always”, 5=“always”
KIDSCREEN-52 (10)	8–18	8–18	10 domains (52) Physical well-being (5) Psychological well-being (6) Moods & emotions (7) Self-perception (5) Autonomy (5) Parent relationship & home life (6) Social support & peers (6) Social acceptance & bullying (3) School environment (6) Financial resources (3)	1=”never”, 2=“seldom”, 3=”sometimes”, 4=”often”, 5=”always”
KINDL (11)	Kiddy KINDL: 4–7 Kid/Kiddo KINDL: 6–16‡	Kiddy KINDL: 4–7 Kid/Kiddo KINDL: 8–16‡	6 domains (24) Physical well-being (4) Psychological well-being (4) Self-esteem (4) Family functioning (4) Friends (or social functioning†)(4) School functioning (or everyday functioning relevant to school or nursery school/kindergarten†) (4)	1=”never”, 2=”seldom”, 3=”sometimes”, 4=“often”, 5=”all of the time”
Pediatric Quality of Life Inventory 4.0 (PedsQL) (12)	Toddler: 2–4 Young child: 5–7 Child: 8–12 Teen: 13–18	Toddler: 2–4 Young child: 5–7 Child: 8–12 Teen: 13–18	4 domains (21 for 2–4 years old; 23 for 5–18 years old) Physical functioning (8) Emotional functioning (5) Social functioning (5) School functioning (3 for 2–4 years old) School functioning (5 for 5–18 years old)	1=”never”, 2=”almost never”, 3=”sometimes”, 4=”often”, 5=”almost always”

^*Higher scores indicate better HRQOL for all items

^†Domain names for the Kiddy KINDL (4–7 years)

^‡The Kid/Kiddo KINDL (8–16 years old) is a combined version of the Kid (8–12 years old) and the Kiddo (13–16 years old) KINDL

The KIDSCREEN-52 is the most commonly administered pediatric HRQOL instrument in Europe (33). The instrument contains 10 domains (52 items): physical well-being (5 items), psychological well-being (6 items), moods and emotions (7 items), self-perception (5 items), autonomy (5 items), parent relationship and home life (6 items), social support and peers (6 items), social acceptance and bullying (3 items), school environment (6 items), and financial resources (3 items).

The KINDL was developed to assess HRQOL among healthy and chronically or acutely ill children (11). The instrument has two versions (i.e., Kiddy KINDL (4–7 years of age) and the Kid/Kiddo KINDL (8–16 years of age)) with different wordings for the friends and school functioning domains. Each version has six domains, each with four items: physical well-being, psychological well-being, self-esteem, family functioning, friends (or named social functioning in Kiddy KINDL), and school functioning (or named everyday functioning relevant to nursery school/kindergarten in Kiddy KINDL).

The PedsQL 4.0 was developed to assess the World Health Organization’s core concept of health (physical, emotional, and social functioning) plus school functioning for children (12). This instrument contains 23 items measuring problems of performing daily functioning over the past month. The four domains include physical functioning (8 items), emotional functioning (5 items), social functioning (5 items), and school functioning (3 items for 2–4 years of age, and 5 items for 5–18 years of age).

The CSHCN screener (34) was administered to assess known-groups validity for the HRQOL instruments. Known-groups validity is described in the Statistical Analysis section. A child with special health care needs is defined as having a chronic condition (i.e. allergies, asthma, and/or attention deficit disorder) and requiring health-related services beyond a child’s normal requirements. This screener is comprised of 5 question sequences evaluating the presence and duration of health conditions. It uses a primary health consequence item to determine whether the “sub-items” need to be answered. The screener is comprised of 3 domains: dependency on prescription medications, service use above routine levels, and functional limitations. If the parent responds “yes” to a primary health consequence item, then 2 follow-up items are asked to determine if the consequence is due to a medical or health condition and whether the duration or expected duration is 12 months or longer. Both follow-up items must be answered “yes” to qualify the child as a CSHCN.

Statistical analyses

Comparisons across the four instruments were performed based on standard psychometric methods, including distribution of domain scores, reliability, structural validity, convergent/discriminant validity, and known-groups validity.

To examine the distribution of domain scores, summary measures (mean, standard deviation, range, 25 percentile, 50 percentile, and 75 percentile) of each domain in the four instruments were calculated. In addition, floor and ceiling effects were examined to determine whether the domain scores are collapsed at the extreme ends of the domain. Ceiling effect refers to when scores are at the maximum possible value for a domain, and floor effect refers to when scores are at the minimum possible value for a domain. The Shapiro-Wilk method was used to test the normality of domain score distribution. Cronbach’s alpha coefficients were calculated to represent the internal consistency of the domains. If the alpha coefficient of a domain is above 0.7, it is deemed acceptable for the purpose of group comparisons (35).

Structural validity refers to how well operationalized items measure the theoretical domains (i.e., factorial structure) of HRQOL within each instrument. We used confirmatory factor analysis (CFA) to confirm the constructs of individual HRQOL domains in each instrument. We used two fit indices to determine an acceptable goodness-of-fit, including the Comparative Fit Index (CFI >0.95) and Root Mean Square Error of Approximation (RMSEA <0.06) (36).

Convergent/discriminant validity refers to how well the domains of the target instruments are associated with domains of well-established instruments. We evaluated the convergent and discriminant validity based on the multitrait-multimethod (MTMM) that demonstrates the correlation between homogenous and heterogeneous domains of each instrument (37). For example, homogenous domains between instruments would be physical functioning of the PedsQL and physical well-being of the KIDSCREEN-52. Heterogeneous domains would be the psychological domain of the KIDSCREEN-52 and the achievement of the CHIP. Specifically, we calculated Spearman’s rank coefficients to account for non-normality of the data. In the present study, the PedsQL was chosen as the anchor because this instrument measures the concept of health suggested by the World Health Organization (38). The domains of the PedsQL (physical, emotional, social and school functioning) capture generic and basic functional status for general children population and are almost all included in other three HRQOL instruments. The PedsQL can be used by the broadest age range of children (2 to 18 years old) compared to other three instruments. Correlations among the domains of the PedsQL (anchor instrument) and the CHIP, the KIDSCREEN-52, and the KINDL (as target instruments) were compared. Moderate (r = 0.50–0.69) to strong (r ≥0.7) correlations among homogenous domains of the target and the anchor instrument indicate good convergent validity. In contrast, small (r = 0.30–0.49) or negligible (r <0.30) correlations among heterogeneous domains of the target and the anchor instrument indicate good discriminant validity (39).

Known-groups validity refers to the extent to which the mean domain scores of each instrument can discriminate between clinically meaningful groups (i.e., CSHCN status) which are known to differ in the underlying HRQOL construct being investigated (40). Bivariate and multivariate linear regression were used to examine the mean difference in HRQOL scores between groups with and without adjusting for covariates (i.e., parent age, child age, sex, income and education). Huber-White robust standard errors were used to account for non-normality in the distribution of HRQOL scores. Effect sizes (ES) were calculated by using the difference in domain scores between CSHCN and those without needs and dividing by pooled standard deviation. Cohen’s d criteria were used (<0.2 as negligible, 0.2–0.49 as small, 0.5–0.79 as moderate, and >0.8 as large) to determine the magnitude of ES (41).

As part of known-groups validity, we calculated relative validity (RV) to examine the extent to which an instrument is more efficient (i.e., more systematic variation is explained by the items relative to variation due to error) versus the other instruments (42, 43). Essentially, RV compares two or more instruments’ ability to discriminate between participants’ varying levels of the underlying HRQOL. To determine the RV for a domain of an instrument, the F-statistics (squared t-statistics) of individual domains were contrasted against the domain with the lowest F-statistic (42, 44). Linear regression with robust standard errors, using domain score as the dependent variable and CSHCN status as the independent variable, was applied to obtain F-statistics (45). A domain of one instrument demonstrating a higher ratio of F-statistics represents superior RV to the other domains. STATA version 9 (46) was used for all analyses.

RESULTS

Participant characteristics

Table 2 shows the characteristics of study participants (N=908). The average age of parents completing the survey was 39.9 years old (SD=11.9). The majority of parents was White, non-Hispanic (48%), and received high school or equivalent degree (39%). One third (33%) of the families had a family income of $20,000 and above. The average age of children was 9.9 years old (SD=5.1), and approximately half of the children were girls (52%). Of the children, 37% were classified with special health care needs.

Table 2.

Subject characteristics (N=908)

Characteristics	Category	Mean (SD) or N (%)
Child age	-	9.9 (5.1/2–19)
Parent/guardian age	-	39.9 (11.9/20–83)
Child gender	Male	47.6
	Female	52.4
Child race/ethnicity	White	42.3
	Black	28.4
	Hispanic	14.4
	Other	14.9
	Don’t know/refused	0.4
Parent race/ethnicity	White	47.7
	Black	27.4
	Hispanic	13.6
	Other	11.3
	Don’t know/refused	0.3
Parent education	Less than HS	23.8
	GED/HS* degree	39.0
	Vocational/some college/AA† degree	28.5
	College graduate	6.2
	Graduate degree	2.4
Marital status	Married	40.2
	Single	32.5
	Other	27.4
Family income	<$9,999	28.2
	$10,000 – $19,999	32.6
	$20,000+	33.3
	Don’t know/refused	6.0
CSHCN	Yes	36.7
	No	63.3

^*General Educational Development test/High school

^†Associate of Arts degree

Distribution of domain scores

The distributions of domain scores in the four instruments were skewed left (Table 3). The Shapiro-Wilk tests suggest that the scores of all domains from the four instruments were not normally distributed. Floor effects were not significant across the four instruments. Ceiling effects were more significant in the domains of the KINDL and less significant in the CHIP. For example, 60% and 40% of participants reported the maximum or highest scores in psychological well-being and friends domains of the Kiddy KINDL, respectively. For ceiling effects in the total scores, the percentage is 0% for the CHIP and KIDSCREEN-52, 1.62% for the Kid/Kiddo KINDL, 5.56% for the Kiddy KINDL, and 6.99% for the PedsQL.

Table 3.

Distribution of the domain scores^† and reliability

	Mean (SD)	Range	25p	50p	75p	Ceiling effects	Floor effects	Cronbach’s alpha
CHIP
Satisfaction	85.75 (12.65)	36.11–100	77.77	88.89	94.44	10.72%	0	0.82
Comfort	83.17 (13.59)	25–100	75	87.50	93.75	8.35%	0	0.83
Resilience	82.60 (12.02)	31.25–100	75	84.38	90.63	6.63%	0	0.70
Risk avoidance	80.27 (15.10)	28.13–100	68.75	81.25	93.75	11.08%	0	0.75
Achievement	74.14 (19.27)	7.14–100	62.50	78.13	90.63	4.97%	0	0.85
Total score	81.22 (10.96)	40.35–99.44	74.73	83.40	89.79	0	0	-
KIDSCREEN-52
Physical well-being	73.14 (19.67)	9.99–100	60	75.00	90	11.13%	0	0.77
Psychological well-being	80.05 (16.82)	8.33–100	70.83	83.33	91.70	16.93%	0	0.86
Moods/emotions	75.65 (18.82)	3.57–100	64.29	78.57	92.86	11.85%	0	0.86
Self-perception	79.13 (18.74)	20–100	65	79.99	95	22.15%	0	0.74
Autonomy	83.45 (14.41)	20–100	75	85.00	95	22.24%	0	0.73
Parent relationship/home life	82.03 (16.35)	20.83–100	75	85.33	95.83	19.72%	0	0.79
Social support/peers	73.64 (18.54)	0–100	62.50	75.00	90	9.62%	0.17%	0.85
School acceptance/bully	77.87 (22.14)	0–100	66.67	83.33	100	34.33%	0.88%	0.78
School environment	73.31 (20.96)	0–100	58.33	75.00	87.50	13.29%	0.17%	0.88
Financial resources	62.56 (25.02)	0–100	50	66.67	83.33	12.22%	1.92%	0.85
Total score	76.05 (12.77)	34.50–99.23	66.90	77.56	86.40	0	0	-
Kiddy KINDL
Physical well-being	85.12 (17.66)	0–100	75	87.50	100	39.69%	0.52%	0.67
Psychological well-being	92.18 (13.01)	37.5–100	87.50	100	100	59.28%	0	0.60
Self-esteem	77.45 (24.28)	0–100	62.50	81.25	100	34.72%	2.59%	0.86
Family functioning	78.78 (17.2)	25–100	66.67	81.25	93.75	20.54%	0	0.45
Friends	84.72 (17.42)	25–100	75	87.50	100	39.89%	0	0.66
School functioning	80.28 (21.50)	12.5–100	68.75	87.50	100	31.55%	0	0.70
Total score	82.79 (14.45)	34.72–100	75	86.46	93.75	5.56%	0	-
Kid/Kiddo KINDL
Physical well-being	81.15 (19.08)	6.25–100	68.75	87.50	100	28.27%	0	0.73
Psychological well-being	84.79 (16.63)	25–100	75	87.50	100	34.69%	0	0.69
Self-esteem	75.99 (21.28)	0–100	62.50	75	93.75	24.51%	0.22%	0.83
Family functioning	75.33 (17.46)	12.5–100	62.50	75	87.50	16.70%	0	0.53
Friends	76.6 (20.06)	0–100	62.50	81.25	93.75	17.77%	0.43%	0.75
School functioning	70.73 (20.31)	0–100	50	75	87.50	13.07%	0.23%	0.57
Total score	77.62 (13.95)	19.79–100	68.58	79.17	88.54	1.62%	0	-
PedsQL
Physical functioning	84.57 (18.36)	0–100	75	90.62	100	29.01%	0.11%	0.83
Emotional functioning	79.86(18.33)	0–100	70	80	95	22.59%	0.11%	0.78
Social functioning	80.43 (20.96)	0–100	70	87.50	100	33.41%	0.22%	0.76
School functioning	73.18 (21.59)	10–100	60	75	91.67	20.29%	0	0.73
Total score	79.42 (16.49)	13.28–100	69.69	82.26	92.97	6.99%	0	-

^†The Shapiro-Wilk test for normality for all domain scores were statistically significant at p< 0.001 except for the KIDSCREEN-52 financial resources domain (p=0.003).

Reliability

The Cronbach’s alpha in all domains of the CHIP, the KIDSCREEN-52, and the PedsQL was acceptable with coefficients above 0.7 (Table 3, last column). However, reliability of some domains in the KINDL was not satisfied; these domains included physical well-being (0.67), psychological well-being (0.60), family functioning (0.45) and friends domain (0.66) of the Kiddy KINDL, as well as psychological well-being (0.69), family functioning (0.53) and school functioning (0.57) of the Kid/Kiddo KINDL.

Structural validity

The KIDSCREEN-52 is the only instrument that met both criteria of acceptable structural validity (CFI>0.95 and RMSEA <0.06). The CHIP (CFI=1.00, RMSEA=0.19), Kiddy KINDL (CFI=1.00, RMSEA=0.43), and Kid/Kiddo KINDL (CFI=1.00, RMSEA=0.08) satisfied the criteria of CFI, but not the RMSEA. However, the PedsQL did not satisfy the CFI and RMSEA criteria, with 0.84 and 0.23, respectively

Convergent/discriminant validity

Table 4 shows the convergent/discriminant validity of the CHIP, the KIDSCREEN-52, and the KINDL against the anchor instrument, the PedsQL. The KIDSCREEN-52 demonstrates superior convergent/discriminant validity when compared to the KINDL. The homogenous domains between the KIDSCREEN-52 and the PedsQL were moderately correlated, whereas the heterogeneous domains were weakly correlated. For example, correlation coefficient of physical well-being of the KIDSCREEN-52 with physical functioning of the PedsQL was 0.55, and psychological well-being of the KIDSCREEN-52 with emotional functioning of the PedsQL was 0.58. In contrast, correlation coefficients of physical well-being of the KIDSCREEN-52 with the domains other than physical functioning of the PedsQL were between 0.26 and 0.41.

Table 4.

Convergent/discriminant validity^† of the CHIP, the KIDSCREEN-52, the Kiddy KINDL, and the Kid/Kiddo KINDL versus the anchor instrument (the PedsQL)

	PedsQL‡
	Physical	Emotional	Social	School
CHIP
Satisfaction	0.50	0.50	0.45	0.37
Comfort	0.57	0.60	0.47	0.41
Resilience	0.34	0.24	0.32	0.29
Risk avoidance	0.48	0.61	0.34	0.54
Achievement	0.47	0.54	0.59§	0.59
KIDSCREEN-52
Physical	0.55	0.34	0.40	0.35
Psychological	0.45	0.58	0.41	0.37
Mood/emotions	0.50	0.69	0.46	0.43
Self-perception	0.40	0.54	0.40	0.30
Autonomy	0.37	0.38	0.35	0.26
Parent/home	0.33	0.40	0.34	0.27
Social support	0.37	0.39	0.45	0.37
Social acceptance	0.33	0.59	0.64	0.53
School environment	0.41	0.49	0.40	0.49
Financial	0.29	0.28	0.26	0.37
Kiddy KINDL
Physical	0.32	0.48	0.33	0.27
Psychological	0.37	0.41	0.34	0.41
Self-esteem	0.40	0.34	0.32	0.39
Family	0.37	0.35	0.29	0.31
Friends	0.47	0.33	0.38	0.29
School	0.43	0.37	0.46	0.39
Kid/Kiddo KINDL
Physical	0.48	0.48	0.40	0.39
Psychological	0.47	0.58	0.51	0.46
Self-esteem	0.38	0.44	0.40	0.35
Family	0.31	0.44	0.32	0.31
Friends	0.37	0.47	0.49	0.35
School	0.37	0.45	0.38	0.45

^†Spearman’s Rank coefficients; all values were significant at p< 0.05

^‡Values in bold indicate the highest correlation coefficients between a specific domain of the PedsQl and individual domains of other three instruments; values in italic indicate hypothesized as a priori convergent validity: the moderate to high correlation coefficients between a specific domain of the PedsQL and homogeneous domains of other three instruments (e.g., physical domain of the PedsQL and physical domains all other instruments); values in regular font indicate hypothesized as a priori discriminant validity: the small correlations coefficients between a specific domain of the PedsQL and heterogeneous domains of other three instruments (e.g., physical domain of the PedsQL and financial domain of the KIDSCREEN-52).

^§No domain was hypothesized a priori to be highly correlated between the CHIP and the PedsQL.

In comparing the Kiddy KINDL to the PedsQL, correlation coefficients of homogenous domains in two instruments were not larger than the coefficients of heterogeneous domains in the two instruments. This provides evidence of poor convergent/discriminant validity. For example, the correlation coefficient between physical well-being of the Kiddy KINDL and physical functioning of the PedsQL was 0.32; coefficient between the psychological well-being of the Kiddy KINDL and emotional functioning of the PedsQL was 0.41. However, the correlation coefficient between physical well-being of Kiddy KINDL and emotional functioning of the PedsQL was 0.48; coefficient for psychological well-being of the Kiddy KINDL and physical functioning of the PedsQL was 0.37. Similarly, poor convergent/discriminant validity was evident for the Kid/Kiddo KINDL to the PedsQL because correlation coefficients of homogenous domains in the two instruments were not larger than the coefficients of heterogeneous domains in the two instruments.

Comparing the CHIP to the PedsQL, the achievement domain of the CHIP was conceptually comparable with school functioning of the PedsQL. Achievement domain of the CHIP was moderately correlated with school functioning (0.59) and weakly correlated with physical functioning (0.47) of the PedsQL. In addition, the comfort domain of the CHIP was moderately correlated with the physical and emotional functioning domains of the PedsQL (0.57 and 0.60, respectively).

Known-groups validity

Table 5 shows known-groups validity of the four instruments using the CSHCN status as the known-groups. Overall, the instruments demonstrated acceptable ability to distinguish the HRQOL of children with and without special health care needs. The domain scores of HRQOL among CSHCN were significantly more impaired than children without needs in the four instruments (p<0.05), except for resilience domain of the CHIP, financial resources of the KIDSCREEN-52, and physical and psychological well-being of the Kiddy KINDL. The magnitudes of the PedsQL ES values were larger than the other three instruments.

Table 5.

Known-groups validity for mean domain and total scores

Individual domain scores	CSHCN (Median)	No special health care needs (Median)	Difference 1 (effect size)†	Relative Validity	Difference 2 (effect size)‡	Relative Validity
CHIP
Satisfaction	81.06 (83.33)	88.95 (91.67)	7.89 (0.62)**	1.65	7.59 (0.61)**	1.75
Comfort	78.44 (79.17)	86.37 (89.58)	7.92 (0.58)**	1.53	9.62 (0.70)**	2.20
Resilience	82.61 (84.38)	82.59 (84.38)	−0.01 (−.001)	N/A^	1.44 (0.12)	1.00#
Risk avoidance	73.68 (75.00)	84.76 (87.50)	11.08 (0.73)**	2.41	12.16 (0.79)**	2.49
Achievement	73.68 (75.00)	77.99 (81.25)	9.57 (0.50)**	1.00#	11.72 (0.61)**	1.72
KIDSCREEN-52
Physical	64.65 (65.00)	79.23 (80.00)	14.58 (0.74)**	5.83	12.66 (0.69)**	63.58
Psychological	73.00 (75.00)	85.13 (87.50)	12.13 (0.72)**	3.90	10.53 (0.70)**	59.67
Mood/emotions	69.30 (67.86)	80.14 (84.52)	10.84 (0.58)**	3.23	7.42 (0.44)*	38.46
Self-perception	74.29 (75.00)	82.56 (90.00)	8.27 (0.44)*	3.57	8.80 (0.50)**	21.70
Autonomy	81.02 (80.00)	85.18 (90.00)	4.16 (0.29)*	1.00#	1.93 (0.14)	9.39
Parent/home	78.59 (79.58)	84.49 (87.50)	5.91 (0.36)*	2.33	3.61 (0.23)	14.06
Social support	69.49 (75.00)	76.61 (79.17)	7.12 (0.38)*	3.31	6.19 (0.33)*	15.67
Social acceptance	70.02 (66.67)	83.41 (91.67)	13.35 (0.60)**	4.64	12.96 (0.60)***	40.42
School environment	66.00 (66.67)	78.56 (79.17)	12.55 (0.60)**	3.84	12.62 (0.67)**	39.84
Financial	61.17 (58.33)	63.54 (66.67)	2.38 (0.09)	1.00#	0.722 (0.03)	1.00#
Kiddy KINDL
Physical	82.67 (87.50)	86.21 (93.75)	3.54 (0.20)	6.50	0.707 (0.04)	1.00#
Psychological	91.46 (100.0)	92.50 (100.0)	1.05 (0.08)	1.00#	−0.36 (0.03)	N/A^
Self-esteem	72.01 (75.00)	79.90 (83.33)	7.89 (0.20)	15.00	5.15 (0.25)*	27.41
Family	75.00 (71.88)	80.51 (83.33)	5.51 (0.32)*	16.92	5.56 (0.35)*	49.00
Friends	81.21 (87.50)	86.27 (87.50)	5.06 (0.29)*	12.33	4.27 (0.25)*	29.29
School	75.61 (81.25)	82.55 (87.50)	6.95 (0.32)*	12.79	5.93 (0.30)*	44.97
Kid/Kiddo KINDL
Physical	75.63 (78.13)	84.80 (87.50)	9.18 (0.48)*	2.76	10.78 (0.57)**	2.71
Psychological	81.01 (87.50)	87.29 (93.75)	6.28 (0.38)*	1.69	6.66 (0.41)*	2.00
Self-esteem	70.88 (75.00)	79.37 (81.25)	8.50 (0.39)*	1.95	7.04 (0.33)*	1.02
Family	72.30 (75.00)	77.34 (75.00)	5.04 (0.29)*	1.00#	5.43 (0.32)*	2.00
Friends	70.15 (75.00)	80.83 (87.50)	10.68 (0.53)**	2.02	9.43 (0.46)*	1.00#
School	65.82 (68.75)	73.89 (75.00)	8.07 (0.40)*	1.85	10.03 (0.52)**	1.73
PedsQL
Physical	77.17 (81.25)	88.86 (93.75)	11.68 (0.64)**	1.00#	11.43 (0.65)**	1.00#
Emotional	71.54 (75.00)	84.67 (90.00)	13.12 (0.72)**	1.34	13.38 (0.74)**	1.38
Social	71.75 (75.00)	85.47 (90.00)	13.72 (0.65)**	1.10	13.20 (0.62)**	1.52
School	64.48 (60.00)	78.26 (80.00)	13.78 (0.64)**	1.14	14.69 (0.71)**	1.77
Total scores
CHIP	76.87 (79.15)	84.21 (86.32)	7.34 (0.67)**	11.82	8.47 (0.77)**	6.05
KIDSCREEN-52	70.98 (69.10)	79.67 (80.83)	8.69 (0.60)**	20.42	7.40 (0.68)**	3.65
Kiddy KINDL	79.82 (82.99)	84.24 (87.50)	4.42 (0.31)*	1.00#	2.77 (0.22)*	1.00#
Kid/Kiddo KINDL	73.04 (73.96)	80.59 (81.94)	7.54 (0.54)**	9.59	8.33 (0.58)**	3.00
PedsQL	71.14 (73.13)	84.24 (97.92)	13.10 (0.79)**	42.64	13.07 (0.80)***	8.38

^{^}N/A: Not applicable because the t-statistic was negative

^†Without covariate adjustment

^‡With covariate adjustment (child’s age, child’s sex, child’s race/ethnicity, parent’s age, parent’s race/ethnicity, parent’s educational background, and family income)

^#A reference domain (with the lowest t-statistic) used for calculating relative validity

^*p<0.05;

^**p<0.01;

^***p<0.001

The total HRQOL scores of CSHCN were significantly more impaired than the total HRQOL scores of children without needs across four instruments (p<0.05). The ES in the PedsQL was larger than the other three instruments: the magnitudes were 0.79 for the PedsQL, 0.67 for the CHIP, 0.60 for the KIDSCREEN-52, 0.54 for the Kidd/Kiddo KINDL, and 0.31 for the Kiddy KINDL. ES were similar or increased after adjusting for covariates. In addition, the PedsQL demonstrates a superior relative validity (42.64) in the total scores, followed by the KIDSCREEN-52 (20.42), the CHIP (11.82), the Kid/Kiddo KINDL (9.59), and the Kiddy KINDL (1.00). These findings were replicated after adjusting for covariates, where the relative validity was superior for the PedsQL (8.38), followed by the CHIP (6.05), the KIDSCREEN-52 (3.65), the Kid/Kiddo KINDL (3.00), and the Kiddy KINDL (1.00).

DISCUSSION

Several studies have made recommendations and suggested criteria for evaluating pediatric HRQOL instruments (1, 26). These criteria include an operationalized definition of HRQOL, inclusion of domains relevant to the population of interest, versions for child and parent, satisfactory psychometric properties, and recognition of developmental process by providing different age-related forms. The present study focuses on testing and comparing psychometric properties of four parent-proxy versions of pediatric HRQOL instruments, the CHIP, the KIDSCREEN-52, the KINDL, and the PedsQL, based on children who were enrolled in Florida Medicaid. The findings suggest that none of the instruments are superior to one another with respect to the psychometric properties assessed in this study. However, the PedsQL and the KIDSCREEN-52 performed slightly better than the others in the psychometric evaluation and contain the recommended domains for measuring the essential construct of HRQOL, including physical, psychological, and social functioning. The suboptimal psychometric performance of the CHIP might reflect the inclusion of unique constructs related to future health and development issues such as resilience and risk avoidance. These unique domains were designed to describe the child’s functional and behavioral outcomes, and evaluate how complex areas of child’s health may be impacted by health policy or services interventions (9, 47).

Domain scores of the four instruments are not normally distributed with significant ceiling effects, especially in the KINDL. Ceiling effects indicate that the sample assessed had very good HRQOL, which will limit the instruments’ capability to differentiate those whose HRQOL scores were in the upper range or to detect the change of HRQOL scores over time. The reliability of the Kiddy and Kid/Kiddo KINDL was lower in comparison to the other three instruments. Low Cronbach’s alpha indicates that some items within the domain of the KINDL may measure different concepts of HRQOL, and are not highly correlated with other items. These findings are consistent with a previous study that reported Cronbach’s alpha for school and friends domains of the KINDL was low with the values of 0.62 and 0.64, respectively (48).

Several findings of the construct validity analyses warrant further discussion. First, using the PedsQL as an anchor for evaluating convergent/discriminant validity, correlation coefficients were moderate overall, and the KIDSCREEN-52 demonstrated slightly greater convergent/discriminant validity than the KINDL and the CHIP. It is not surprising that the CHIP demonstrated suboptimal convergent/discriminant validity because the domains capture unique content (i.e. resilience, risk avoidance) compared to the other instruments. The CHIP does not include the same standard domains of HRQOL as the PedsQL does; instead, the CHIP measures different aspects of a child’s health related to developmental issues such as engagement in risk behaviors that may threaten future health (i.e. risk avoidance) and factors that enhance future health, such as family involvement or social problem-solving skills (i.e., resilience) (9, 32) Second, the psychological and social domains are higher in known-groups validity than physical domains within most instruments, with the exception of the Kid/Kiddo KINDL. This finding indicates that the psychological and social domains were the domains best able to distinguish the difference between CSHCN and those without needs. This may suggest that CSHCN status is associated with greater psychosocial demands than physical demands in this population. Review studies suggest that children with chronic health problems may have more difficulty adapting to psychosocial demands than healthy peers (49, 50). Third, in the known-groups validity analysis, the relative validity of the total scores was greater in the PedsQL, followed by KIDSCREEN-52, the CHIP, and the KINDL, suggesting that the PedsQL was the most sensitive in detecting differences between CSHCN and those without special health care needs. The evidence of fair construct validity implies that items from the same domain of an instrument may capture multi-dimensional concepts of HRQOL. A more comprehensive CFA approach (e.g., bi-factor models) and item response theory (IRT; see below) may be implemented to better understand the specific measurement properties of items from different instruments in future studies.

The present study compared different instruments at the domain level and assumed that the items in similar domains of different instruments (e.g., psychological well-being in the KIDSCREEN-52 and emotional functioning in the PedsQL) capture the same underlying concept the domains intend to measure. However, the comparisons derived from domain level information rather than item level can be biased because items from different instruments were not created on the same foundation (i.e., same underlying construct of HRQOL). The rationale for classifying items into different domains might be different from one instrument to another instrument. Future studies may use the International Classification of Functioning, Disability and Health (ICF) to provide supplementary evidence for comparisons (51, 52). The ICF is a framework of health and functional status proposed by the World Health Organization that can be used to compare measurement content of different HRQOL tools (53). The ICF is comprised of two main sections: functioning and disability and contextual factors. Functioning and disability is divided into body functions, body structures, and activities and participation. The contextual factor section separates into environmental and personal factors. ICF may inform on whether the four pediatric instruments cover the comprehensive domains and appropriate items of HRQOL for pediatric population. Comparing the instruments to domains defined by the ICF framework demonstrates that some instruments might provide more comprehensive coverage than others (52). For example, Shiariti and colleagues have compared the contents of items from four condition-specific and three generic pediatric HRQOL instruments on the basis of the ICF-CY framework; they found that the Child Health Questionnaire captured a broader range of content in the ICF-CY domains of body functioning, activities and participation, and environmental factors compared to other pediatric HRQOL instruments (54).

Given the evidence that no one specific instrument was superior to the other instruments, we suggest two approaches for future pediatric HRQOL research. First, IRT may be used to build new item banks by selecting appropriate items from each of instruments that are capable of measuring the same concept across different instruments and calibrating these selected items to the same metric or scale. Many pediatric instruments, including the ones utilized in this analysis, were developed using classical test theory (CTT) which has several limitations for instrument development. CTT is test/scale-driven rather than item-driven, meaning that the entire set of items must be administered to ensure the scale’s reliability even though some items may not fit a child’s underlying HRQOL (55, 56). For example, for a child with severe health conditions, items measuring capability to walk one block may be infeasible for a specific child, but must be asked because of the instrument’s design. In addition, CTT cannot differentiate between a child’s level of an underlying HRQOL and the scale’s measurement properties (55, 56). A second approach for future research is to equate all of the original items of different instruments. Equating is a statistical process that adjusts item scoring on instruments so that the item scores and total scores can be used interchangeably and compared between studies if different instruments are delivered (57, 58).

Several limitations are identified in this study. First, the surveys are based on parent-proxy rather than child self-report. Although the FDA guidelines recommend obtaining reports from patients directly, the parent-proxy retains important value in pediatric HRQOL research as described in the Introduction. Second, the instruments were delivered in the same order for all participants. Given the lengthy content, this may have introduced response fatigue for the final instrument that was administered. Third, we were not able to obtain test-retest reliability due to budget and time constraints. Finally, CSHCN status was determined through the non-categorical approach, which might provide different information compared to the use of the categorical approach or clinical diagnosis (e.g., asthma, cancer, cystic fibrosis, diabetes, etc.). Despite the validity of the CSHCN screener, using clinical diagnostic information for the HRQOL validation might help interpret the specific impact on HRQOL resulting from different health conditions.

CONCLUSION

Although we found that no one instrument was superior to the other instruments in different psychometric properties, other criteria such as the instrument lengths and unique contents of HRQOL should be taken into account in the selection of instrument. In particular, the use of the CHIP might be able to capture unique development issues beside physical, emotional and social functioning status for children. Nevertheless, we suggest that future studies should utilize well-established content structure, such as the ICF framework, to guide the comparisons for the content in different pediatric HRQOL instruments, followed by applying IRT to test item-level measurement properties across different instrument. Before these sophisticated methods are implemented, we remind researchers to carefully select pediatric HRQOL instruments for their population of interest in research and clinical use.