The Lymphatic Malformation Function (LMF) Instrument

Abstract

Objective

The Lymphatic Malformation Function (LMF) instrument is a preliminary parent-report assessment designed to measure outcomes in children with cervicofacial lymphatic malformation (LM). This study aimed to assess the measurement properties of the LMF, refine it, test criterion validity, and evaluate the test-retest reliability.

Study Design

Cross sectional

Setting

Two pediatric tertiary referral centers

Subjects

Parents of 60 children from 6 months – 15 years old with cervicofacial LM.

Methods

Parents were recruited via mail and online. The LMF was administered on paper or online initially and again within 21 days. Response distributions and inter-item correlations were examined for item reduction. Exploratory factor analysis was conducted on retained items. Cronbach’s α, Spearman correlation, and intraclass correlation (ICC) coefficients were calculated to test internal consistency, criterion validity (compared to stage), and test-retest reliability, respectively.

Results

One item was removed due to a floor effect. The response scale was collapsed from a 5-point scale to a 3-point scale due to skewness. Six items were discarded due to redundancy (inter-item correlations >0.7); two items were discarded due to factor loadings <0.4. Exploratory factor analysis revealed a 2-factor structure explaining 84% of variance, and the domains, “Signs” and “Impacts”, had good internal consistency (all Cronbach’s α >0.80 and <0.90), significant association with stage (p<0.05), and good overall test-retest reliability (ICC 0.82).

Conclusion

The LMF has been refined into a 12-item, 2-domain instrument measuring LM-specific signs and impacts with internal consistency, criterion validity, and test-retest reliability.

Introduction

Head and neck lymphatic malformations (LM) are anomalous tangles of lymphatic vessels that impact basic daily functions such as breathing, eating, and speaking ^1–4. There are currently no established, validated measures of head and neck LM disease burden incorporating patient- or family-centered functional criteria. Current classification schema of LM focus on anatomic and histopathology descriptions ^5–7, and most outcome measures have focused on clinician assessment of lesion resolution or on treatment complications ^6,7. The Cologne Disease Score was the first staging system to incorporate non-anatomic measures of disease burden, and has demonstrated correlation with outcomes in at least one study ⁷. This instrument includes functional variables designed to be assessed by clinical providers, but does not include patient- or family-centered impact of LM on function and daily activities.

In order to address this need, we have developed a disease-specific parent-reported function assessment for children with head and neck LM. The content of the instrument came from interviews with parents of children with LM who were recruited through the Seattle Children’s Hospital Vascular Anomalies Clinic. Pediatric otolaryngologists reviewed interview responses to define clinically relevant items within functional and clinical sign categories. Parents then rated the impacts on their child’s daily life for each category. This resulted in a 21-item parent-report instrument called the Lymphatic Malformation Function (LMF) instrument ⁴.

This instrument has undergone the first phase of development, which established content validity, but its other measurement properties remain to be tested. For this study, we aimed to evaluate measurement properties of the LMF, reduce and refine it, measure its internal consistency, test criterion validity, and evaluate test-retest reliability.

Methods

Patients and recruitment

Packets were mailed to 148 families of children age birth to 11 years with cervicofacial LM selected from the vascular anomalies clinic databases of two pediatric tertiary care centers. The population was comprised of children from a full range of the clinical staging system for head and neck LM ^8–11 who had not previously undergone complete surgical resection. Parents of 39 children responded (Table 1). Parents of an additional 21 children responded to advertising on the websites of two LM foundations. For test-retest reliability, all parents who were recruited through the hospitals and 10 parents recruited online who agreed to re-contact were invited to complete a second questionnaire within 21 days post-baseline. This study was approved by the Institutional Review Boards of Seattle Children’s Hospital and Rady Children’s Hospital in San Diego.

Table 1.

Demographic Characteristics of the Sample (N=60)

Variable	Mean (SD)	Range
Age (years)	7 (4)	0.5 – 15
Variable	N	%
Gender
Male	30	50
Female	30	50
Lymphatic Malformation Stage*
I – Unilateral infrahyoid	16	26
II – Unilateral suprahyoid	13	22
III – Unilateral infra- & suprahyoid	6	10
IV – Bilateral suprahyoid	13	22
V – Bilateral infra- & suprahyoid	6	10
Could not determine	7	10

SD – Standard deviation

^*Chi² goodness of fit p>0.05, suggesting an even distribution of stage

Data Collection

The LMF asks parents to indicate on a 5-point Likert scale the frequency of disease-related signs and functional impacts over the previous 30 days. All signs were designed to be directly observable by parents, with the possible exceptions of pain and sadness/anger. Parents completed the LMF on paper or online via REDCap, a secure, web-based application designed to support data capture for research studies¹². Paper questionnaires were entered into the REDCap database manually and checked for accuracy by a separate member of the research team. Chart review was conducted to collect information on child age, sex and LM stage. Because chart review was not possible for those recruited via website, these parents were asked to report child age, sex and stage based on diagrams embedded within the online questionnaire.

Analysis

Statistical analysis was conducted with Stata/SE 12 software (StataCorp LP, College Station, Texas). Descriptive summaries for continuous variables are reported as ranges and means ± standard deviations, whereas frequencies are reported for categorical variables.

The distribution of responses for each item were examined for floor and ceiling effects, defined as 80% or more responses in the lowest or highest value of the response scale, respectively. Items with either ceiling or floor effects that were deemed clinically non-essential were eliminated. The distributions of responses were also examined for skewness to evaluate for collapsing the Likert scale for simplicity. Item-to-item Pearson correlations were examined and those >0.70 were considered for removal due to redundancy.

The Kaiser-Meyer-Olkin Measure of Sampling Adequacy was calculated to determine the degree of common variance, an assessment of whether the sample is adequate for factor analysis. The interpretation of the measure is as follows: 0.90–0.99 is considered “marvelous”, 0.80–0.89 “meritorious”, 0.70–0.79 “middling”, 0.60–0.69 “mediocre”, 0.50–0.59 “miserable”, and below 0.50 “unacceptable” for factor analysis ¹³. The Bartlett Test of Sphericity was conducted to determine whether the items were sufficiently inter-correlated for factor analysis; a significant p-value indicates that the inter-correlation is sufficient ¹³. After the data were deemed adequate, exploratory factor analysis using the principal factors method due to non-normality of data was conducted, and the scree plot and eigenvalues were examined to determine the number of factors to retain ^14–17. Retained factors were rotated using both promax (oblique) and varimax (orthogonal) rotation to determine factor structure (domains). Items that did not load onto at least one factor with a loading >0.4 were removed.

Internal consistency was tested with Cronbach’s α by domain. Spearman correlation was used to test the hypothesized association between the total and domain scores with stage (criterion validity). We tested for association with age or sex to determine if the instrument functioned differently in these subpopulations. For parents who completed the instrument a second time within 21 days of the baseline, intraclass correlation (ICC) for agreement was calculated using the summed score of the total instrument and the summed scores of each of the domains with ≥ 0.70 typically considered to be the standard for reliability of health status measures ¹⁸. Twenty-one days was chosen as a period long enough to avoid exact recall of previous answers while short enough that clinical change would be unlikely to have occurred ¹⁸.

Results

Description of sample

The children with LM were a mean age 7 +/− 4 years, evenly distributed by sex, and represented the full range of LM stages (Table 1). Thirty-six respondents were from Seattle Children’s Hospital, three from Rady Children’s Hospital, and 21 were recruited online from foundation websites. Respondents to the survey did not differ significantly in age or sex from non-respondents for whom we had this demographic information (both p > 0.05).

Reduction of items and refinement of response scale

Two items, “Difficulty producing vocal sounds, and “Difficulty speaking” and were included in the initial instrument to cover the full range of language developmental stages. However, they had a high inter-item correlation (r=0.68) and were deemed redundant based on content. To refine the instrument, these two items were combined into a single item, and re-worded as “Difficulty vocalizing or speaking clearly.” For the purposes of factor analysis, the response of parents to “Difficulty producing vocal sounds” was retained for this item children ≤2 years of age and that for “Difficulty speaking” was retained for children >2 years.

One item “Difficulty moving his/her head or neck” was removed due to a floor effect (85% responding “None of the time,” Table 2). One additional item, “Difficulty breathing” had a floor effect (83% responding “None of the time,” Table 2); however, this item was considered clinically essential and was not captured elsewhere in the instrument; thus it was retained. The remaining items also had a distribution skewed toward the floor of the 5-point scale. For this reason, the decision was made to reduce the 5-point response categories into 3-point scales by collapsing the sparsely populated top three response categories into a single top response category (Figure). This change was made in order to even the distribution of responses while maintaining a useful level of clinical information collected on each item.

Table 2.

Descriptive Statistics for 20 Items (N=60)

Item	Mean (SD)	% Floor	% Ceiling
1 Mouth bleeding	0.5 (0.8)	71	19
2 Tongue swelling	0.7 (0.9)	55	28
3 Difficulty chewing	0.8 (0.9)	56	31
4 Difficulty swallowing	0.5 (0.8)	66	19
5 Change in usual diet	0.6 (0.8)	64	19
6 Drooling	0.6 (0.8)	55	18
7 Difficulty vocalizing	0.7 (0.9)	55	28
8 Difficulty brushing teeth	0.5 (0.8)	68	22
9 Facial swelling	1.0 (0.9)	40	42
10 Head or neck swelling	0.6 (0.8)	62	23
11 Difficulty moving head/neck	0.2 (0.6)	85*	8
12 Difficulty breathing	0.2 (0.6)	83	7
13 Symptoms preventing usual activities	0.4 (0.7)	68	10
14 Miss out on things he/she wanted to do	0.5 (0.7)	67	12
15 Have difficulty doing thing he/she enjoys	0.5 (0.8)	70	15
16 Stay home from school	0.5 (0.8)	67	17
17 Avoid going out in public	0.4 (0.7)	75	15
18 Appear sad or angry	0.5 (0.7)	58	12
19 Appear to be in pain	0.7 (0.8)	51	17
20 Have difficulty sleeping	0.6 (0.8)	62	20

Mean = mean score on 5-point likert scale. SD = standard deviation. % Floor = percent of subjects who chose the lowest value on the 5-point scale. % Ceiling = percent of subjects who chose the lowest value on the 5-point scale.

^*Item removed due to floor effect

Figure.

Refined Instrument Response Scale

The item-to-item correlations revealed two clusters of highly inter-correlated items resulting in removal of five items (Table 3). “Difficulty chewing” was highly correlated with “Symptoms causing a change in usual diet” (r=0.80) and “Difficulty brushing his/her teeth” (r=0.73). To optimize item reduction, the latter two were removed and the former retained. “Miss out on things he/she wanted to do” was highly correlated with “Have difficulty doing things he/she enjoys” (ρ=0.74), “Have symptoms preventing usual activities” (ρ=0.79), and “Miss school” (ρ=0.71). “Have difficulty doing things he/she enjoys” and “Have symptoms preventing usual activities” were also highly correlated (r=0.77). “Miss out on things he/she wanted to do” was retained and the other three items were removed.

Table 3.

Inter-Item Correlations for 19 Retained Items (N=60)

Item	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19
1 Mouth bleeding	1.00	0.62	0.51	0.41	0.47	0.34	0.37	0.46	0.41	0.02	0.40	0.28	0.34	0.35	0.30	0.25	0.39	0.38	0.12
2 Tongue swelling	0.62	1.00	0.56	0.49	0.42	0.64	0.50	0.34	0.30	0.04	0.26	0.25	0.23	0.16	0.32	0.11	0.17	0.28	0.06
3 Difficulty chewing	0.51	0.56	1.00	0.62	0.80	0.45	0.52	0.73	0.48	0.21	0.49	0.46	0.49	0.35	0.53	0.34	0.30	0.47	0.42
4 Difficulty swallowing	0.41	0.49	0.62	1.00	0.59	0.36	0.44	0.32	0.20	0.05	0.44	0.25	0.33	0.23	0.45	0.36	0.28	0.41	0.42
5 Change in usual diet	0.47	0.42	0.80	0.59	1.00	0.26	0.32	0.57	0.42	0.24	0.35	0.47	0.55	0.37	0.54	0.33	0.41	0.62	0.52
6 Drooling	0.34	0.64	0.45	0.36	0.26	1.00	0.52	0.23	0.22	0.04	0.35	0.30	0.25	0.17	0.28	0.14	0.26	0.26	0.16
7 Difficulty vocalizing	0.37	0.50	0.52	0.44	0.32	0.52	1.00	0.47	0.41	0.29	0.50	0.25	0.23	0.37	0.25	0.09	0.25	0.13	0.03
8 Difficulty brushing teeth	0.46	0.34	0.73	0.32	0.57	0.23	0.47	1.00	0.39	0.15	0.31	0.35	0.45	0.37	0.29	0.28	0.34	0.35	0.21
9 Facial swelling	0.41	0.30	0.48	0.20	0.42	0.22	0.41	0.39	1.00	0.36	0.32	0.49	0.49	0.61	0.52	0.44	0.38	0.34	0.22
10 Head or neck swelling	0.02	0.04	0.21	0.05	0.24	0.04	0.29	0.15	0.36	1.00	0.26	0.41	0.33	0.41	0.35	0.25	0.13	0.15	0.01
11 Difficulty breathing	0.40	0.26	0.49	0.44	0.35	0.35	0.50	0.31	0.32	0.26	1.00	0.59	0.46	0.51	0.47	0.34	0.23	0.19	0.18
12 Symptoms preventing usual activities	0.28	0.25	0.46	0.25	0.47	0.30	0.25	0.35	0.49	0.41	0.59	1.00	0.79	0.77	0.67	0.41	0.45	0.35	0.36
13 Miss out on things he/she wanted to do	0.34	0.23	0.49	0.33	0.55	0.25	0.23	0.45	0.49	0.33	0.46	0.79	1.00	0.74	0.71	0.66	0.43	0.52	0.31
14 Have difficulty doing thing he/she enjoys	0.35	0.16	0.35	0.23	0.37	0.17	0.37	0.37	0.61	0.41	0.51	0.77	0.74	1.00	0.49	0.50	0.50	0.34	0.26
15 Stay home from school	0.30	0.32	0.53	0.45	0.54	0.28	0.25	0.29	0.52	0.35	0.47	0.67	0.71	0.49	1.00	0.65	0.44	0.59	0.45
16 Avoid going out in public	0.25	0.11	0.34	0.36	0.33	0.14	0.09	0.28	0.44	0.25	0.34	0.41	0.66	0.50	0.65	1.00	0.46	0.55	0.42
17 Appear sad or angry	0.39	0.17	0.30	0.28	0.41	0.26	0.25	0.34	0.38	0.13	0.23	0.45	0.43	0.50	0.44	0.46	1.00	0.64	0.51
18 Appear to be in pain	0.38	0.28	0.47	0.41	0.62	0.26	0.13	0.35	0.34	0.15	0.19	0.35	0.52	0.34	0.59	0.55	0.64	1.00	0.64
19 Have difficulty sleeping	0.12	0.06	0.42	0.42	0.52	0.16	0.03	0.21	0.22	0.01	0.18	0.36	0.31	0.26	0.45	0.42	0.51	0.64	1.00

Bolded items indicate inter-item correlation >0.7 (redundancy)

Exploratory Factor Analysis

The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.79, and the Bartlett Test of Sphericity was significant (p<0.001), indicating the data were sufficient for factor analysis^15,19. The first three eigenvalues obtained from the principal components analysis were 4.96, 1.63, and 0.91, explaining 63%, 84%, and 95% of the cumulative variance, respectively. These results suggested that two to three factors should be extracted based on the Kaiser-Guttman rule¹⁴ and examination of the screeplot, which demonstrated an inflection at the third factor. Principal axis factoring was conducted extracting two and three factors, and the solutions were rotated using orthogonal (Varimax) and oblique (Promax) rotation methods. Overall, the two-factor solution with oblique promax rotation provided the optimal grouping of items. All but two of the items loaded onto one of the two factors with loadings >0.4. These items (“Facial swelling” and “Head or neck swelling other than face”) did not load substantially onto either factor and were excluded. After excluding these items, the principal axis factoring and rotation were repeated on the remaining items. No further items were found to have weak factor loadings, and there was no cross-loading of factors.

The final factor loadings for the two-factor model are reported in Table 4. The two-factor (or, two-domain) structure that emerged divided the items into those addressing LM-related Signs and those addressing LM-related Impacts (Table 4). The Signs domain had seven items with loadings ranging from 0.51 to 0.86. The Impacts domain had five items with loadings ranging from 0.61 to 0.82. Correlation analysis and Cronbach’s α statistics for the 12 retained items are shown in Table 5. Each of the items showed good correlation with the overall instrument. Removal of any one of the items did not improve the overall Cronbach’s α statistic for the instrument, suggesting all items should be retained. Cronbach’s α was 0.86 for Signs and 0.84 for Impacts (Table 6). The two domains were correlated at 0.50 (p<0.001), suggesting they are related but not measuring identical concepts.

Table 4.

Items Included in the Lymphatic Malformation Function Instrument and Factor Loadings

Content Summary	Factor 1	Factor 2
Mouth bleeding	0.59	--
Tongue swelling	0.86	--
Difficulty chewing	0.61	--
Difficulty swallowing	0.52	--
Drooling	0.69	--
Difficulty speaking or vocalizing	0.79	--
Difficulty breathing	0.51	--
Miss out on things that he or she wanted to do	--	0.61
Avoid going out in public	--	0.74
Appear sad or angry	--	0.67
Appear to be in pain	--	0.82
Difficulty sleeping	--	0.77

Table 5.

Correlation Analysis of 12 Retained Items (N=60)

Item	Item-Total Correlation	Cronbach’s Alpha if Item Deleted*
1 Mouth bleeding	0.66	0.86
2 Tongue swelling	0.65	0.86
3 Difficulty chewing	0.80	0.85
4 Difficulty swallowing	0.73	0.86
5 Drooling	0.62	0.87
6 Difficulty vocalizing	0.61	0.87
7 Difficulty breathing	0.60	0.87
8 Miss out on things he/she wanted to do	0.66	0.86
9 Avoid going out in public	0.59	0.87
10 Appear sad or angry	0.62	0.87
11 Appear to be in pain	0.70	0.86
12 Have difficulty sleeping	0.54	0.87
Test scale		0.87

^*Cronbach’s Alpha did not improve from overall test scale with removal of any item

Table 6.

Description of the Lymphatic Malformation Function Instrument and Domains

Scale	Number of items	Mean Total Score	Range	Standard Deviation	Cronbach’s α
Signs	7	0.58	0–2	0.59	0.86
Impacts	5	0.56	0–2	0.51	0.84

Criterion validity and test-retest reliability

The correlations between the refined instrument and stage were measured. As hypothesized, there was a significant positive correlation between instrument scores (total, Signs domain, and Impact domain) and stage (ρ=0.61, p<0.01 ρ=0.73, p<0.001; and ρ=0.30, p=0.04, respectively). There was no significant correlation between total and domain scores and age (all p>0.30), and the mean total and domain scores did not differ significantly by sex (all p>0.10) or by source of recruitment (clinical versus online; all p>0.05).

Twenty of 60 respondents completed the instrument a second time post-baseline. Seventeen of those did so within 21 days and were included in test-retest reliability analysis. ICC for test–retest reliability were as follows: total instrument 0.82 (95% CI 0.58–0.93; p<0.001), Signs domain 0.88 (95 % CI 0.70–0.95, p<0.001), and Impacts domain 0.58 (95 % CI 0.16–0.82, p=0.006), indicating good reliability for the total instrument and Signs domain, but marginal reliability for the Impacts domain.

Discussion

These incremental data and analyses of the LMF have reduced the instrument to 12 items and two domains (Signs and Impacts). The refined instrument demonstrates internal consistency, criterion validity, and good overall test-retest reliability, which complement the face validity from originally being developed from parent interviews ⁴.

The two domains of the refined instrument had a medium strength of correlation with one another, suggesting they are related but not overlapping, and are thus measuring distinct constructs. The refined instrument had a significant positive correlation with disease stage; as stage worsened, so did the function score, which demonstrates the instrument’s criterion validity. The correlation for the Signs domain was much stronger than that for the Impacts domain. This difference suggests that the signs measured within the Signs domain are related closely to lesion severity, but that the variation in impacts on daily life may not be reflected as fully by lesion stage alone. As expected, they did not have significant correlations with age or differ significantly by sex, which suggest the instrument behaves consistently across these populations. The difference in reliability between domains could be due to the relatively small sample size, or more likely that the Impacts domain assesses a more subjective construct than Signs, with Impacts more prone to variable parental report.

This analysis resulted in substantial item reduction and scale refinement. Due to an overall skew toward the response “None of the time” on all of the items, responses on the five-point scale were collapsed into a three-point response scale, which normalized the response distribution. This three-point scale simplifies respondent burden while still allowing clinicians to determine whether signs, symptoms or impacts represents no problem, a moderate problem, or a large problem for a patient. In addition, nine of the initial 21 items were removed due to a combination of factors, which improved the quality of the instrument and decreased respondent burden.

This analysis has several limitations. Though there are no strict rules regarding sample size for exploratory factor analysis¹⁵, the sample size of 60 subjects is smaller than is recommended by some¹⁸. Further, a non-representative sample with regard to age, sex or disease severity could potentially bias the analysis. For instance, if sampling was done from a segment of the population with less severe disease, this may result in elimination of items that may be important to those with more severe forms of the disease. Finally, even with a well-distributed sample, the respondents might have had greater or lesser functional impacts of LM than non-respondents. However, the Kaiser-Meyer-Olkin test of Sampling Adequacy and Bartlett Test of Sphericity both indicated that the sample was adequate for exploratory factor analysis. In addition, the sixty respondents represented an even sex distribution, a wide range of ages (from infant to adolescent) and were well distributed among the five LM stages, suggesting we obtained responses from a representative sample. In addition, the factor structure that emerged had high clinical face validity, lending credence to the results. Baseline demographic characteristics (age and sex) of the respondents were not significantly different from the non-respondents for whom we had baseline data, which reduces the concern for fundamental differences of the respondents. Nonetheless, the generalizability of the instrument is not fully established, and its characteristics are described as relevant to LM patients with parental response to the survey questionnaire.

Patient-reported outcomes measurement aims to strike a balance between comprehensive data collection from patients and reasonable respondent burden. This analysis allowed us to refine the LMF instrument, simplify the response scale, reduce items, and propose a scoring algorithm based on two domains. We believe that the current parent-report instrument may be useful in the clinical setting for collecting proxy-reported information on head and neck LM-related function in children and adolescents. However, we would like to emphasize that this version of the instrument does not represent a final, fully validated instrument, and several steps still remain. Future work will include collecting responses with this revised instrument in order to conduct confirmatory factor analysis, construct validation, and to measure responsiveness to treatment-related changes. In addition, the minimal clinically importance difference will need to be determined. Finally, proxy-report instruments are important to pediatric practice, but do not always mirror self-report. Thus a self-report version of the instrument for adolescents is also in development.