The IWQOL-Kids^©: Establishing minimal clinically important difference scores and test-retest reliability

Abstract

This study presents additional psychometric testing of the Impact of Weight on Quality of Life-Kids (IWQOL-Kids) with aims to establish distribution-based minimal clinically important difference scores (MCIDs) and evaluate test-retest reliability. Participants (N=263) represent a pooled sample of treatment-seeking obese adolescents (11–19 years) from four large studies examining HRQOL and psychosocial outcomes (Mz_BMI=2.6± 0.4; M_age=15.1±1.9; 64% female; 51% Black, 46% White). Adolescents completed the IWQOL-Kids©. Standard errors of measurement, which represent the MCID for each scale, were: Physical Comfort=8.8; Body Esteem=7.7; Social Life=8.1; Family Relations=6.2; Total QOL=4.8. Test-retest reliabilities ranged from 0.75–0.88. These data provide further support for the excellent psychometric properties of the IWQOL-Kids. In addition, preliminary MCIDs for IWQOL-Kids scales have now been established, which can be used in clinical trials.

Health-related quality of life (HRQOL), a multidimensional construct that assesses several domains (e.g., physical, emotional, social), is an important patient-reported outcome (PRO) in obesity research. PROs are used as primary and secondary endpoints for clinical trials if basic psychometric standards (i.e., good reliability and validity) are met and information is provided to evaluate the smallest clinically relevant change a patient perceives, known as the minimal clinically important difference (MCID)(1).

The sensitivity and specificity of weight/obesity-specific measures make them well-suited as PROs for clinical trials. Several such measures exist(2,3), including a self-report for adolescents, the Impact of Weight on Quality of Life-Kids(4). Initial psychometric evaluation of the IWQOL-Kids, has demonstrated excellent scale reliabilities, convergent and discriminant validity, and responsiveness(4). However, test-retest reliability and the establishment of MCIDs have not been conducted.

MCIDs can be established using anchor-based or distribution-based methodologies. Anchor-based methods utilize changes on clinically-relevant rating scales (e.g., within-patient global ratings of change) to serve as anchors when establishing the MCID. Although easy to obtain and based on the patient’s perspective, these methods are limited by measurement imprecision, unknown reliability, and a lack of reliability of the specific global rating of change scales(5). Distribution-based methods rely on statistical procedures to determine the MCID, including the one standard error of measurement (SEM) method, (5,6) which has been used to establish MCIDs for a popular pediatric generic HRQOL measure, the PedsQL(7). This approach is sample-independent, accounts for measurement precision, and is expressed in the units of the measure. Furthermore, studies have demonstrated that one SEM consistently mapped on to MCIDs established with anchor-based methods(8). The aims of the current study were to evaluate test-retest reliability and establish distribution-based MCIDs for the IWQOL-Kids scales.

Methods and Results

Participants and Procedures

Participants (N=263, ages 11–19 years) represent a pooled sample from four separate studies examining HRQOL of obese youth at Cincinnati Children’s Hospital Medical Center (CCHMC) between 2004–2007 (2,3,9,10). Participants sought treatment in either the CCHMC behavioral weight management program (HealthWorks!; n=232; Study 1: n=146; Study 2: n=59; Study 3: n=27) or adolescent bariatric surgery program (Study 4: n=31). Both programs required a physician referral although differed based on entry body mass index (BMI: kg/m²) (behavioral: BMI ≥ 95^th percentile; bariatric: BMI ≥ 40 kg/m²). Across studies, eligibility criteria included written informed consent/assent and exclusion of youth with developmental disabilities or significant reading difficulties. Age eligibility varied by study. Questionnaires were completed prior to intervention (i.e., behavioral, surgery), with the exception of one protocol (10) which was a follow-up study of youth who had previously sought behavioral weight management (n = 59; Mean time since treatment = 4.2±0.8 years). All personnel were trained in study procedures. Participants were compensated for their time. Study protocols were approved by the local Institutional Review Board.

Test-retest reliability of the IWQOL-Kids was assessed with a sub-sample of participants (n=21) from one behavioral weight management study protocol (2,3) who were approached for follow-up approximately 2–4 weeks later and prior to weight management intervention. Height and weight measurements were taken again to ensure no significant BMI changes, which would compromise stability.

Measures

IWQOL-Kids©

The IWQOL-Kids is a weight-related HRQOL measure validated on youth ages 11 years and older(4). Four subscales and a Total Score have been identified. These scales assess the impact of weight on an individual’s physical mobility and comfort (Physical Comfort), how an individual feels about themselves and their body (Body Esteem), how an individual is treated in their social environment (Social Life), and the individual’s perception of what family members may think and feel about them (Family Life). Scaled scores range from 0–100, with higher scores representing better HRQOL. This measure has demonstrated excellent reliability (Cronbach’s alphas=0.88 to 0.95) and validity, discriminated among weight status groups, and was responsive to weight change(4).

Weight and height

Height and weight were measured by trained staff using calibrated equipment in the clinic (Height: Holtain stadiometer, Crymych, UK; weight: Scaletronic, Wheaton, IL) or at home (Height: custom portable stadiometer, Creative Health Products, Plymouth, MI; weight: SECA digital scale, Hamburg, Germany). Anthropometric data were used to calculate BMI and the standardized zBMI using the LMS method(11) based on the CDC 2000 growth curves(12).

Demographic Questionnaire

Caregivers provided adolescent race/ethnicity, as well as information to determine family socioeconomic status (SES) using the Revised Duncan (TSE12; 13,14).

Statistical and Data Analyses

Descriptive data (e.g., means and standard deviations) were calculated. MCID scores were calculated for each scale using the SEM with the following equation: SEM=SD√[1-α], SD=standard deviation of mean IWQOL-Kids score; α=scale reliability(8). Test-retest reliability was determined using intraclass correlation coefficients (ICC). An ICC of ≥ 0.80 suggests excellent agreement and between 0.61 and 0.80 moderate agreement (15).

Participants

Participants included 263 adolescents (M_age=15.1±1.9; 64% female; 50.6% Black; 46.0% White, non-Hispanic; 3.5% Other). M_BMI was 43.1±11.2 (zBMI=2.6± 0.4). Mean family SES was 35.6 ± 20.7, representing occupations such as bank tellers, teacher’s aides, and cleaning staff. The test-retest reliability subsample (n=21) had similar demographics.

SEMs, which represent the MCID for each scale, ranged from 4.8–8.8 units (Table 1). Regarding test-retest reliability, the average time between visits was 17.9 days (SD=7.1) with no significant BMI changes observed (t (20)=−1.1; p=0.27). Test-retest reliability was strong for all scales (Table 1).

Table 1.

IWQOL-Kids: Means (SD), Reliability Coefficients and MCID

IWQOL-Kids Scales	Mean(SD)	Cronbach’s Alpha	Test-Retest Reliability (n=21)	MCID
Physical Comfort	72.6 (25.1)	0.88	0.75	8.8
Body Esteem	62.2 (29.1)	0.93	0.87	7.7
Social Life	76.8 (23.7)	0.88	0.77	8.1
Family Relations	90.7 (16.9)	0.87	0.88	6.2
Total QOL	74.1 (19.5)	0.94	0.88	4.8

Discussion

While medical indices (e.g., BMI) provide information regarding health status, PROs are increasingly utilized to ascertain how behavioral, pharmacological, and surgical treatments impact HRQOL. The current study provides further evidence of the excellent psychometric properties of the IWQOL-Kids and extends findings by demonstrating excellent test-retest reliability and preliminary documentation of MCIDs using an SEM distribution-based method.

MCIDs enable researchers and clinicians to identify the minimal amount of patient-perceived change in a construct such as HRQOL. This information can be used to evaluate whether treatments are beneficial, or should be changed or discontinued due to their impact on HRQOL. MCIDs can also be used to identify changes in daily functioning that warrant attention. For example, a 10-point negative change on the Social Life scale may lead a clinician to initiate a discussion about the adolescent’s obesity and need for intervention.

Although consensus regarding the best approach to determine MCIDs is lacking(5), anchor-based methods or other distribution-based methods may have yielded different results and should be considered in future research. In addition, although ICCs were strong, the current test-retest sample was small, necessitating validation with a larger sample.