Modification and Evaluation of a Velopharyngeal Insufficiency Quality of Life Instrument

Abstract

Objective

Modify the existing 45-item velopharyngeal insufficiency (VPI) quality of life (QOL) instrument (VPIQL), assess the modified instrument for reliability and provide further validation. There are patient and parent versions of the instrument.

Design

Validation convenience sample from a previously conducted pilot study.

Setting

Two academic tertiary referral medical centers.

Participants

De-identified data were used from 29 subjects with VPI and 29 control subjects age 5–17 years, and parents.

Outcome measures

Subjects and parents completed VPIQL and a generic pediatric QOL instrument (PedsQL^4-0).

Data Analysis

Twenty-two items were removed from the VPIQL for ceiling effects, floor effects, and redundancy, to produce the modified instrument, VPI Effects on Life Outcomes (VELO) instrument. VELO was tested for internal consistency (Chronbach’s alpha), discriminant validity (paired t-test with control subjects), and concurrent validity (Pearson correlation with the PedsQL^4-0). These analyses were also completed for parents.

Results

The 45-item VPIQL instrument was reduced to the 23-item VELO instrument. The VELO had excellent internal consistency (Chronbach’s alpha 0.96 for parents and 0.95 for VPI subjects). The VELO discriminated well between VPI and control subjects, with mean score (SD) was significantly lower (worse) for VPI subjects (67.6 [23.9]) than for control subjects (97.0 [5.2]) (p<0.0001). The VELO total score was significantly correlated with the PedsQL^4.0 (r=0.73) among subjects with VPI. Similar results were seen in parent responses.

Conclusions

The VELO is a 23-item QOL instrument that was designed to measure and follow QOL in subjects with VPI, with less burden than the original VPIQL. VELO demonstrates internal consistency, disciminant validty, and concurrent validity with the PedsQL^4-0.

Introduction

Health-related quality of life (QOL) refers to judgment of value placed on patient’s health related experiences. QOL instruments can be categorized as generic or condition specific. Generic QOL instruments are able to capture QOL differences in children with a wide variety of difficulties. Condition-specific QOL instruments are tailored to measure how the condition affects children’s QOL and are better able to detect changes in QOL that are important to patients.¹

Velopharyngeal insufficiency (VPI) is a condition that affects speech, swallowing, and many psychosocial aspects of a child’s life in a way that is different from other conditions. Children with VPI report a lower (or worse) QOL than peers without VPI.² Generic QOL instruments may not be sensitive to these differences. Accurately measuring QOL in children with VPI is an area in need of further research.

One condition-specific measure that has been developed for children with VPI is the VPIQL instrument. This VPI specific QOL instrument was developed to capture the many ways VPI affects children’s lives. It was developed from focus groups comprised of VPI subjects and their parents, with input from otolaryngologists and speech and language pathologists with extensive experience caring for these children.³ This is a crucial step in developing a QOL instrument and gives the VPIQL content validity.⁴

The development and initial limited validation produced an instrument with 48 items (or questions) organized in seven domains. These domains include speech limitation, swallowing problems, situational difficulty, emotional impact, perception by others, activity limitation and caregiver impact. While the VPIQL was developed for this population with tailored content, its length with 48 items may render the instrument too burdensome for routine use. An ideal instrument would balance two competing interests; being short enough to minimize patient and family burden while being long enough to fully capture all of the items relevant to VPI specific QOL.

The goal of this study was to condense the VPIQL and evaluate the resulting shortened instrument in terms of reliability, discriminant validity, concurrent validity and construct validity.

Methods

Initial Development of VPIQL

To develop a list of items (questions) to measure the way VPI affects these children’s lives, focus groups were conducted. Focus group participants included subjects with VPI, their parents and a moderator (pediatric otolaryngologist, pediatric otolaryngology fellow, or speech and language pathologist). The content was recorded during the focus groups and the moderator ensured all people’s thoughts could be expressed. Individual focus groups were conducted until the group was not adding new items and new focus groups were repeated until most items discussed were repeated twice (thematic saturation). This approach resulted in three focus groups. Following these focus groups, a national panel of clinicians who manage VPI reviewed the content in 2003 and additional items were added.² This process produced a list of 48 items organized into domains including speech limitation, swallowing problems, situational difficulty, emotional impact, perception by others, activity limitation and caregiver impact.

Study Subjects

This study used data from the pilot study of the VPIQL instrument previously described.² Briefly, subjects age 5 to 17 years with VPI diagnosed by an otolaryngologist or speech pathologist were recruited at one of two centers. Additional study participants were recruited from retrospective review of administrative data sets. Potential study subjects were identified by International Classification of Disease, Ninth revision code for VPI (750.29). Medical records were reviewed to ensure inclusion criteria were met. A total of 29 subjects with VPI were enrolled after obtaining informed consent. The VPI group had a mean age of 8.7 years (range 5–15) and included 15 males and 14 females (Table 1). In order to test discriminant validity, 29 control subjects without VPI and their parents were also enrolled randomly from the clinical practices. Additional inclusion criteria (for VPI subjects and control subjects) included being a native speaker of English. Exclusion criteria for control subjects included a previously diagnosed speech or language disorder or prior pharyngeal or laryngeal surgery.

Table 1.

Description of Sample - VPI subjects and controls

	VPI Subjects n=29	Control Subjects n=30
Age - yrs	8.7 (3.0)	8.1 (3.0)
Male - n (%)	15 (51.7%)	17 (56.7%)
VPIQL score	72.0 (18.5)	86.3 (10.6)
PedsQL score	67.6 (23.9)	97.0 (5.2)

All data reported as mean (SD) unless otherwise noted

De-identified data from this pilot study was obtained from Dr. Thibeault after IRB approval was obtained from the University of Washington, University of Utah and University of Wisconsin.

Patient reported outcomes

VPIQL and Pediatric Quality of Life Inventory - version 4.0 (PedsQL^4.0) questionnaires were completed by both subjects and parents with parents assisting their children as necessary. The questionnaires were completed at one time point.

VPIQL

The VPIQL is a 48-item VPI-specific QOL instrument with seven domains, each domain included 3–10 items. The domains include speech limitation (9 items), swallowing problems (3 items), situational difficulty (10 items), emotional impact (9 items), perception by others (7 items), activity limitation (5 items) and caregiver impact (5 items). Domains can be thought of as different dimensions or elements of health-related QOL.⁵ The caregiver impact domain is only included on the parent version. Respondents are prompted to “in the past four weeks, how much of a problem has your child had with [].” Items are presented with a response format of a 5-point Likert-type scale ranging from Never to Almost Always. The instrument score is the average of all items and is converted to a zero to 100-point scale with zero representing worse QOL. Domain scores are the average of all items in the domain, similarly converted to a zero to 100-point scale. The VPIQL was previously shown to have discriminant validity, with lower QOL among VPI subjects than controls, and parents were shown to be adequate proxy’s for children responses, utilizing the data presented here.²

PedsQL-4

The PedsQL^4.0 is 23-item validated generic pediatric QOL instrument^6–7 after which the VPIQL was modeled. The items are organized into four domains (physical functioning, emotional functioning, social functioning and school functioning). Because the VPIQL was modeled after the PedsQL^4.0, they have similar prompts and Likert scale response formats. The PedsQL^4.0 is also scored on a 0 to 100 point scale with zero representing worse QOL.

Statistical Analysis

Instrument Modification - Item Reduction

The item reduction analyses were conducted utilizing responses from VPI subjects and their parents with a number of analyses to identify redundant and poorly functioning questions. The statistical attributes of the VPIQL instrument items were analyzed to identify large floor or ceiling effects. Items were marked for potential elimination if the endorsement frequency (proportion answering “Never”) was greater than 50% or if the item-total correlation was less than 0.70. To identify potentially redundant items, the remaining items were tested for item-item correlation greater than 0.80. Internal consistency with Cronbach’s alpha was also calculated with the removal of each of the remaining items. There was no significant increase in alpha (no significant improvement in reliability without a given item), so no additional items were marked for elimination.

Each of the items marked for potential elimination were reviewed by a panel of clinicians managing VPI (two pediatric otolaryngologists and one speech and language pathologist) and items were removed only if consensus was obtained. The panel reviewed the item to ensure the content of the item being removed was still captured in the remaining items. For items marked for potential elimination because of item-item correlation, the items were reviewed by the panel to ensure they contained related content.

Instrument Modification - Readability

The VPIQL instrument was reviewed for readability to identify problematic items and wording. Readability was assessed by determining the Flesch-Kincaid Grade Level for the instrument, domains and individual items. The Flesch-Kincaide grade level is a formula used to provide an estimate of the average number of years in school required to understand a piece of written material.⁸ Items above the 3rd grade level in the youth version of the instrument and 6th grade in the parent version were reviewed for potential rewording. Each item was reviewed to determine if it contained individual words above the 3rd and 6th grade level for the youth and parent versions, respectively, utilizing a standardized vocabulary list.⁹ Potential changes to the instrument were reviewed by the panel to obtain consensus.

Reliability and Validation

Reliability and validation testing was conducted on the modified/reduced instrument. Reliability of an instrument is the degree to which repeated iterations of the instrument yield the same result.¹ Reliability in this study was assessed by internal consistency testing utilizing Cronbach’s alpha.¹⁰ Cronbach’s alpha was calculated for the reduced instrument and domains for all VPI subjects and then for subgroups of VPI subjects ages 5– 9 years and 10–15 years. A Cronbach’s alpha of greater than 0.70 was considered acceptable.⁴

Validity testing, in general, assesses the extent to which the instrument is measuring what it purports to measure.¹¹ There are a number of specific methods of validation and utilizing these methods can be thought of as accumulating evidence to support an instrument's validity. This study uses a variety of analyses for validation.

Discriminant validity tests an instrument’s ability to detect a difference in QOL among VPI subjects and control subjects. The primary analysis tested for a difference between mean total scores with t-test and secondarily analysis tested for a difference between mean domain scores. A p-value of less than 0.05 was considered statistically significant. Only discriminant validity utilized data from control subjects and parents.

Concurrent validity seeks to correlate the QOL instrument to another QOL instrument that has already undergone rigorous validation. This was assessed by calculating the Pearson correlation between the modified instrument total score and the PedsQL^4.0 with a correlation of greater than 0.50 considered sufficient because this accounts for 25% of the variance in the modified instrument score. A correlation too high, say above 0.9, would suggest that the new instrument adds little information over the existing instrument. To further validate domain scores, secondary analyses included correlation between domains including VPIQL-emotional impact with PedsQL^4.0 emotional functioning; VPIQL-perception by others with PedsQL^4.0 school functioning; as well as both VPIQL-situational difficulty and perception by others with PedsQL^4.0 social functioning.

Establishing construct validity involves a process of hypothesis testing of theorized associations.¹² Principal factor analysis was conducted for construct validation on both VPI subject responses and parent responses. Principal factor analysis clusters items that are statistically related. More specifically, it is a method of identifying the underlying structure of the variance in item responses. The underlying statistical structure often suggests content domains of related items. The analysis produces factors (or latent variables) around which the item responses vary. Factors are sequentially analyzed and retained in the model (explaining less variance with each additional factor) until the latent variables do not significantly add to the model. The resultant factor loadings can be interpreted as the correlation of the QOL item to the underlying factor. Orthogonal varimax rotation was conducted keeping factors with loadings of greater than 1.0. A scree plot of eigen values was reviewed to ensure the appropriate number of factors were retained in the final model. The factor loadings of items after orthogonal varimax rotation were compared to the proposed (hypothesized) content domains and factor loadings of greater than 0.5 were considered relevant.¹³

Parent Proxy Assessment

Assessment of parental response as a proxy for VPI patient response was assessed by testing the difference between parents reported total score and VPI subject total score using the paired t-test. This analysis was repeated for each domain score. To test the inter-rater reliability (comparing parents and subjects), we calculated the intra-class correlation coefficient (ICC) for the total score as well as each domain. An ICC > 0.5 indicates at least moderate agreement. Since the parent-subject inter-rater reliability might be different for younger versus older subjects, we divided the VPI subjects into those 9 years old or less and those 10 year and older.

Results

Instrument Modification - Item Reduction

The item reduction process identified 23 items for potential elimination. After review by the panel, 22 items were eliminated which resulted in a 23-item instrument for VPI patients and a 26-item instrument for parents. One item was retained to allow three items in the swallowing domain despite low item-total correlation. Many of the items were marked for potential elimination by multiple techniques. The overall composition of the modified VPIQL instrument, the VPI Effects on Life Outcomes (VELO) is the same as the original version with questionnaires being administered to both children and their parents. The VELO instrument has six domains including speech limitation (7 items), swallowing problems (3 items), situational difficulty (5 items), emotional impact (4 items), perception by others (4 items) and caregiver impact (3 items, answered only by parents). The domain of “Activity Limitation” was eliminated with the five items each being eliminated from the instrument. In addition to the poor performance of the individual items, the initial sub-scale had Cronbach’s alpha of 0.48 for parents and 0.69 for children’s responses. The content of the individual items in this domain were retained on the remaining items. The instrument’s initial prompt, response format and scoring were not changed.

Instrument Modification - Readability

In the youth version administered to children, several words above the 3^rd grade reading level, including nasal, depressed, abnormal and perception, were identified. Items were edited to avoid these and other problematic words. The words “difficult” and “difficulty” appeared in several items and significantly increased the reading level for these items. The word “difficult” is rated as a 3^rd grade reading level⁹, but difficult and difficulty were changed to hard and trouble in the youth version to improve readability. One item was also identified that did not clearly match the response format. This item also had marginal performance on internal consistency testing and additionally was correlated with several other items at the 0.75 level. This item was removed after panel consensus. After modifications, the youth version had a Flesch-Kincaid Grade Level of 2.5 with four items having grade 4 or higher and the parent version had a Flesch-Kincaid Grade Level of 3.7 with five items grade 7 or higher. All parent items with grade 7 or higher had the word difficult or difficulty.

Reliability – Internal Consistency

Cronbach’s alpha for the modified instrument total score was 0.96 for parents and 0.95 for VPI subjects, and each domain had a Cronbach’s alpha greater than 0.70 (Table 2). The VELO instrument also had adequate internal consistency for each age group with Cronbach’s alpha 0.96 for subjects age 5–9 years and 0.95 for subjects 10–15 years.

Table 2.

Internal Consistency by Cronbach's Alpha of VELO for total and domains

	Cronbach’s Alpha
Domain Description	Parent	VPI subjects
Total Score	0.96	0.95
Speech Limitation	0.87	0.88
Swallowing	0.85	0.80
Situational Difficulty	0.93	0.92
Emotional Impact	0.89	0.78
Perception by Others	0.84	0.86
Caregiver Impact	0.77	-

Discriminant Validity

The parent-reported mean (SD) VELO score was significantly lower for VPI patients than for control patients (61.4 (21.4), 98.1 (4.0), p<0.0001). Similarly, VPI subject reported mean (SD) VELO score significantly lower than control subjects (67.6 (23.9), 97.0 (5.2), p<0.0001). Lower scores denote worse QOL. Each of the VELO domains also had discriminant validity (p<0.01, Table 3).

Table 3.

Discriminant validity of VELO: Difference in mean VELO total and domain scores among VPI subject and controls

Domain Description	VPI Parents	Control Parents	p-value*	VPI Subjects	Control Subjects	p-value*
Total Score	61 (21)	98 (4)	<0.0001	68 (24)	98 (4)	<0.001
Speech Limitation	51 (24)	96 (8)	<0.0001	59 (27)	95 (9)	<0.0001
Swallowing	90 (18)	100 (0)	0.006	86 (22)	99 (4)	0.004
Situational Difficulty	46 (27)	97 (7)	<0.0001	57 (32)	96 (9)	<0.0001
Emotional Impact	67 (28)	99 (3)	<0.0001	73 (26)	99 (4)	<0.0001
Perception by Others	75 (24)	100 (1)	<0.0001	79 (25)	99 (2)	<0.001
Caregiver Impact	61 (24)	99 (3)	<0.0001	-	-	-

All data reported as mean (SD)

^*p-value for t-test

Concurrent Validity

Both parent-reported and subject-reported VELO total scores were significantly correlated with the PedsQL^4.0 (Pearson correlation coefficient r=0.78, p<0.0001 and r=0.73, p<0.0001, respectively). Secondary analysis of hypothesized domain correlations showed VELO-emotional impact and PedsQL^4.0 emotional functioning were sufficiently correlated for parent reports (r=0.59) but not for subject reports (r=0.42). Similarly, VELO-perception by others and PedsQL^4.0 school functioning were sufficiently correlated for parent reports (r=0.52) but not subject reports (r=0.45). VELO-situational difficulty was sufficiently correlated with PedsQL^4.0 social functioning (r=0.56 and 0.55 for parent reports and subject reports, respectively) as was VELO-perception by others (r=0.63 and 0.68).

Construct Validity

Factor analysis of VPI subject’s responses resulted in a four-factor solution that explained 77.5% of the variance in VELO responses. The parent’s responses initially resulted in a five-factor solution with the fifth factor having an eigen value of 1.07 and with one item loading on this factor. A four-factor solution was chosen as the fifth factor was associated with only one item. The four-factor solution of parent’s responses explained 75.1% of the variance. The factor loading after varimax rotation largely followed hypothesized domains (Table 4) though items from several of the items from domains loaded on the same factor. Among VPI subjects and parents, speech limitation items loaded on several factors, though Factor #2 in the VPI subject responses had fairly high loading for all except speech question 7. Swallowing difficulty items loaded on the same factor and was associated with several of the speech limitation items for both groups. Among VPI subjects, the situational difficulty items and the emotional impact items loaded highly on the same factor (Factor #1) while among parents emotional impact items and perception by others items loaded on the same factor (Factor #1). Caregiver impact items loaded highly on the same factor as situational difficulty. Overall, the items largely loaded on the hypothesized domains with the speech limitation items loading on several different factors. Among parent responses, Factor #1 represents emotional impact and perception by others, Factor #2 represents situational difficulty and caregiver impact, Factor #3 represents swallowing problems and Factor #4 speech limitations. Among subject responses, Factor #1 represents situational difficulty and emotional impact, Factor #2 represents speech limitation and swallowing problems, Factor #3 and 4 represents perception by others and situational difficulty. Oblique rotations were also attempted, in case the underlying factors were correlated, and this did not significantly change the interpretation of the factor loadings.

Table 4.

Factor Analysis with four retained factors among parents and children. Factor loadings reported with all factor loadings above 0.5 shown in bold

	Parent Factors				VPI Subject's Factors
Speech Limitations	#1	#2	#3	#4	#1	#2	#3	#4
Air comes out my nose when I talk	0.12	0.49	0.3	0.55	0.16	0.69	0.42	−0.1
I run out of breath when I talk	0.18	0.24	0.06	0.84	0.27	0.7	0.10	0.34
It is hard talking in long sentences	0.52	0.03	0.47	0.53	0.44	0.52	0.30	0.38
My speech is too weak	0.02	0.18	0.72	0.18	0.48	0.74	0.00	0.06
I have trouble being understood when I’m in a hurry	0.45	0.57	0.2	0.28	0.57	0.47	0.40	0.08
My speech gets worse toward the end of the day	0.2	0.28	0.62	0.5	0.28	0.46	0.12	0.51
My speech sounds different than other kids	0.36	0.61	0.33	0.11	0.68	0.27	−0.2	0.31
Swallowing Problems
Liquids come out my nose while drinking	0.04	0.47	0.71	0.26	0.12	0.71	0.45	0.2
Food comes out my nose while eating	0.06	0.11	0.91	0.01	0.17	0.72	−0.2	0.33
Others make fun of me when food or liquids come out my nose	0.21	0.2	0.79	0.02	0.02	0.76	0.28	0.41
Situational Difficulty
My speech is hard for strangers to understand	0.50	0.64	0.21	0.26	0.81	0.29	0.04	0.25
My speech is hard for friends to understand	0.65	0.56	0.32	0.16	0.64	0.08	0.13	0.61
My speech is hard for family to understand	0.35	0.73	0.19	0.24	0.45	0.26	0.27	0.69
I have trouble being understood when others can't see my face, for example, in a car	0.24	0.64	0.31	0.35	0.59	0.32	0.18	0.5
I have trouble being understood on the phone	0.49	0.56	0.03	0.35	0.75	0.05	0.37	0.41
Emotional Impact
I am teased because of how I talk	0.71	−0	0.14	0.43	0.06	0.18	0.83	0.15
I get sad because of how I talk	0.82	0.22	−0	0.21	0.74	0.19	0.32	0.08
I get frustrated or give up when I am not understood	0.78	0.42	0.08	0.16	0.57	0.3	0.49	0.22
I am shy because of how I talk	0.79	0.08	0.32	−0	0.58	0.02	0.42	−0
Perception by Others
I am treated like I am not smart because of how I talk	0.58	0.53	0.06	−0.1	0.03	0.14	0.84	0.38
Others ignore me because of how I talk	0.85	0.21	−0.1	0.08	0.36	0.06	0.88	0.09
Others do not like to talk on the phone with me because of how I talk	0.66	0.12	0.24	0.32	0.2	0.3	0.43	0.7
My family or friends tend to talk for me	0.76	0.46	0.15	0.08	0.4	0.32	0.28	0.59
Caregiver Impact
I am worried or concerned about my child’s speech	0.48	0.47	0.12	0.34
I find it difficult to understand my child	0.35	0.61	0.21	0.39
My child’s speech problem slows me down or inconveniences me	0.07	0.72	0.52	−0.1

Parent Proxy Assessment

Parent ratings of their VPI subject’s QOL is analogous to a second rater for the subject’s VPI, and we compared parent ratings to subject ratings with a test of inter-rater reliability using the ICC. Parents reported lower or worse mean (SD) VELO total score (61(21)) than VPI subjects (68 (24), p=0.05) which was driven largely by the two domains, speech limitation and situational difficulty. Despite this difference in mean scores, the parent proxy report is reasonable with an ICC >0.6 for the total score as well as the domains. To ensure there wasn’t a difference between the proxy reliability of younger and old VPI subjects, the ICC was calculated for those up to age 9 years old and those 10 years and older. The ICC may be smaller for the older group but was only less than 0.6 for the domains of emotional impact and perception by others. The sample size in these subgroups may limit the interpretation of the age specific ICC.

Discussion

This study provided an important step in the refinement of a QOL instrument for evaluating children with VPI. Most previous research related to VPI has utilized postoperative perceptual speech analysis (by speech and language pathologists) or closure of the velopharyngeal orifice by endoscopic exam as their primary surgical outcomes. There is a paucity of patient reported outcomes of validated condition-specific functional status or QOL. Aside from the VPIQL, the Pediatric Voice Outcome Survey (PVOS) has been used in a small (n=12) study of subjects with VPI and was found to be responsive to changes in QOL after surgical correction.¹⁴ The Pediatric Voice Outcome Survey is a four-item instrument that was modified from the adult version and validated in a general pediatric otolaryngology patient population.^15–16

While the PVOS has the advantage of low patient time burden with just four items, it likely does not measure many of the issues important to children with VPI. Conversely, the 48-item VPIQL is too long for routine use. The 48-item VPIQL was modified to preserve content validity. The item reduction analysis was conducted to reduce the patient burden (from 45 items to 23 items for VPI subjects and 48 items to 26 items for parents) while maintaining important concepts and content. With the elimination of poorly performing items, the domains initially established were largely retained.

Ensuring readability of an instrument is an important and recommended step⁴ that is sometimes overlooked when developing a new instrument. In addition to improving readability of the instrument, the process of review and panel discussion helps to ensure thorough and thoughtful review of each and every item for content and wording. The modifications to the VPIQL (48-item instrument) to produce the new VELO instrument will hopefully improve the functioning of the instrument in future studies.

The internal consistency testing, with Cronbach’s alpha, shows that the instrument as well as all of the domains appear internally reliable. The original 48-item instrument had an overall alpha of 0.97 for both VPI subjects and parents, indicating redundancy. Chronbach’s alpha for the total instrument may still indicate redundancy, but the current length is necessary to achieve adequate content. Because repeated measures will be necessary for future longitudinal studies, test-retest reliability should be conducted in future studies. The initial study of the 48-item VPIQL utilized one time point, so test-retest reliability could not be conducted. Future test-retest reliability will ensure that item scores (and domain scores) are stable enough to analyze changes in QOL. The internal consistency testing done here is an important first step in reliability testing.

The modified instrument (VELO) retained its ability to detect differences in QOL among subjects with and without VPI (discriminant validity). The instrument total score retained discriminant validity as did all of the domains. Additionally, the VELO was shown to have concurrent validity (correlation with a previously validated instrument) with the generic pediatric QOL instrument. This helps to show that the VELO is measuring QOL though in a way more specific to VPI. Condition-specific QOL measures have been shown to be better able to detect change (responsiveness) in QOL than generic instruments, which is an important goal for this instrument. Pre- and post-treatment longitudinal measurements were not collected in this sample, so responsiveness testing was not possible with these data. Future responsiveness testing will be important to determine whether the VELO will be useful for outcomes studies

The factor analysis conducted here provides some first steps towards construct validation. Construct validity seeks to confirm hypothesized correlations related to the responses. Factor analysis is a statistical tool that analyzes the underlying association among a group of variables.¹⁷ When used with a priori hypotheses, this allows for content validation of an instrument’s domains showing item responses are correlated along the hypothesized domains. If an instrument measured only one domain, the hypothesis would be that all items would load on one factor. In our analysis, the factor loadings largely followed the hypothesized content domains, though some of the domains showed overlap in the underlying factor. The domains of situational difficulty, emotional impact and perception by others may all draw from an underlying domain of psychosocial difficulty. Adequate sample size for factor analysis is typically described as 10 times the number of items⁴, so these results should be interpreted with caution and need to be repeated in future studies. When factor analysis is conducted in future studies, a larger sample size will be essential to further understanding of the underlying associations.

Criterion-related validation (validation against a “gold standard” measure) is also necessary with this instrument. While no true “gold-standard” exists for VPI, perceptual speech analysis is the most widely accepted and utilized measure in diagnosis of VPI^18–19 and validation against this measure should be conducted. We did not have access to the perceptual speech analysis results for this cohort.

This analysis supports parent proxy assessment of VPI specific QOL. Parents report worse QOL related to speech limitation and situational difficulty (Table 5 and 6), which might reflect different emotional reactions by subjects and parents when the subject is facted with these difficulties. Despite the lower reported VELO among parents, the inter-rater reliability for parents and VPI subjects is adequate (Table 6). This data supports the initial research and discussion of parent proxy for VPI subjects in the initial 48-item VPIQL study.²

Table 5.

Parent and VPI Subject reported VELO scores by paired t-test

Domain Description	Parents mean (SD) VELO score	VPI Subjects mean (SD) VELO score	p-value*
Total Score	61 (21)	68 (24)	0.05
Speech Limitation	51 (24)	59 (27)	0.05
Swallowing	90 (18)	86 (22)	0.15
Situational Difficulty	46 (27)	57 (32)	0.01
Emotional Impact	67 (28)	73 (26)	0.16
Perception by Others	75 (24)	79 (25)	0.16
Caregiver Impact	61 (24)	-	-

^*paired t-test

Table 6.

Parent as proxy for VPI subjects response: Intra-Class Correlation Coefficient for VPI subjects and parents

Domain Description	All Ages n=29*	<10 years n=16*	10–15years n=13*
	ICC+ (95% CI)	ICC+ (95% CI)	ICC+ (95% CI)
Total Score	0.71 (0.51–0.89)	0.80 (0.63–0.98)	0.62 (0.28–0.96)
Speech Limitation	0.68 (0.48–0.88)	0.60 (0.30–0.92)	0.71 (0.44–0.99)
Swallowing	0.70 (0.51–0.89)	0.91 (0.82–0.99)	0.63 (0.30–0.96)
Situational Difficulty	0.65 (0.45–0.87)	0.69 (0.43–0.95)	0.65 (0.33–0.97)
Emotional Impact	0.71 (0.53–0.89)	0.89 (0.78–0.99)	0.44 (0.00–0.89)
Perception by Others	0.60 (0.37–0.83)	0.79 (0.61–0.98)	0.33 (0.00–0.82)

^*number of VPI subjects

⁺Intra-class correlation coefficient

Understanding and measuring QOL is important for understanding and advancing the treatment of children with VPI. Having a rigorously tested and refined instrument is necessary to measure patient-centered outcomes. The VELO has been refined to reduce the time burden on participants and improve readability while maintaining its content validity. Future studies should be conducted, and are currently underway by our group, to test this instrument further. This work will provide a foundation for future investigations of the impact of VPI and treatment outcomes with a focus on a patient-centered measure.