Item Reduction of the Wisconsin Upper Respiratory Symptom Survey (WURSS-21) Leads to the WURSS-11

Abstract

Purpose

To develop a shorter version of the Wisconsin Upper Respiratory Symptom Survey (WURSS-21), -a self-report questionnaire for evaluating daily symptoms and functional impairments during acute respiratory illness (ARI).

Method

WURSS-21 data were retrieved from 4 studies (n=1167) spanning the years 2002–2010. Similar methodologies were employed among these studies. Degree of missingness, ceiling/floor effects, and exploratory (EFA) and confirmatory (CFA) factor analyses were investigated and used to guide item retention. Stability of the reduced WURSS was evaluated across the 1^st 3 days of ARI-illness.

Results

Degree of missingness was < 1% and appeared to be completely at random. Seven WURSS items with >30% of ratings of zero (floor effects) were eliminated. Cross-loading items (Head-congestion, Sleep-well and Breathe-easily) were excluded following EFA on subset-1. Subsequent CFA using subset-2 showed satisfactory indices of fit. The reduced WURSS-11 instrument demonstrated 3 dimensions of 3 items each, and was stable across 3 days of illness. The indicated dimensions (items) include; Nasal (Runny-nose, Plugged-nose, Sneezing), Throat (Cough, Sore-throat, Scratchy-throat) and Quality of life (Feeling-tired, Think-clearly, Accomplish-daily-activities).

Conclusion

The WURSS-11 has similar dimensional structure as the WURSS-21. This shorter version may reduce the time and burden required for completing the survey.

Introduction

A shorter yet valid version of the Wisconsin Upper Respiratory Symptom Survey (WURSS) is desirable for reducing the burden involved in filling out the questionnaire frequently during a time of illness. The current WURSS-21 has been shown to be a reliable and responsive research tool useful in evaluating the severity of acute respiratory illness (ARI-illness).[1]

It was developed from the WURSS-44 using item-domain reliability, responsiveness and importance-to-participants as main criteria.[2,3] This work suggested that general symptoms and quality of life functions were valued more highly than specific symptoms.[1,4] These original WURSS versions showed 3 factors for WURSS-21 and 8 factors for WURSS-44. Both versions have been validated[4] against surveys and laboratory-assessed biomarkers.[5,6]

At least one version of the WURSS instrument has been utilized by investigators in 200 institutions in 40 countries and translated into 7 languages (http://www.fammed.wisc.edu/research/external-funded/wurss). With the increasing interest and use of the WURSS, the aim of this study was to derive a shorter version which could reduce survey completion time and perhaps increase response rates.

Method

Data sources

Data for this work came from 4 previous studies generating WURSS-21 information including: #1) WURSS-21 development[4] and #2) WURSS-21 validation[1] which spanned the years 2002–2007. Additional data came from 2 clinical trials titled #3) “Placebo: Physician or Pill? RCT in a Common Cold Model” (2004–2008)[7,8] and #4) “Meditation or Exercise for Preventing Acute Respiratory Infections” (2009–2010).[9]

Briefly, n=1167 people with ARI-illness self-reported daily information on the WURSS-21, if they had symptoms of <48hours, a score of >2 on the Jackson scale, and felt they were having a cold. Two consecutive daily responses with “Yes” to the question “Do you think you have a cold?” and “No” to “Do you think you are still sick?” respectively confirmed the start and end of the illness episode.

Statistical analysis

The WURSS introductory (“How sick do you feel today?”) and concluding items (“Compared to yesterday…”) were excluded from this analysis because they are designed to measure different time frames and are usually analyzed separately. Analysis was restricted to the 1^st three days of illness to maximize sample size and because ARI-illness severity tends to peak within this time.[10]

Following the Kroonenberg and Lewis approach[11], we randomly divided the combined dataset into 2 equal sized subsets. Approach-1 then employed exploratory factor analysis (EFA) on subset-1 while approach-2 tested these results using confirmatory factor analysis (CFA) on subset-2. Subsequently, the combined subsets were used to obtain final parameter estimates of the reduced WURSS model. Model invariance was assessed across the 1^st three days of illness.[12]

Approach-1 used EFA to identify the underlying dimensional/factor structure towards a reduced number of WURSS-21 items with significant loadings. An oblique solution accounted for correlations between the factors while weighted least squares with mean and variance adjustment (WLSMV) was used to model the ordered categorical nature of the WURSS items. Several models involving multiple dimensions were evaluated and the best dimensional model was selected. Model choice was based on dimensional cohesion and satisfactory applications of factor analysis.[13]

Approach-2 employed CFA to examine the model fit and dimensional structure of the selected items. Assessing the measurement model validity occurs when the theoretical measurement model is compared with the reality model to see how well the data fits. Chi-square testing of goodness of fit statistics including comparative fit index (CFI), Tucker-Lewis index (TLI) and root mean square error of approximation (RMSEA) were used to check measurement model validity.[14–16] [17]

Longitudinal measurement invariance was conducted using multiple CFA models with WLSMV. Following procedures suggested by Vandenberg and Lance, progressively more stringent nested models were constructed and evaluated.[18] All models were constructed in Mplus version 7.11.[19] Model 1 consisted of two correlated common factors corresponding to two time periods (day1 and day 3) with correlated residual variances for the same items over the time periods. Model 2 was the same as model 1 but constrained the loadings to be the same or invariant over time. Model 3 extended model 2 by constraining the factor variances to be invariant over time. The DIFFTEST option in Mplus was used to obtain a correct chi-square difference test when the WLSMV estimators are used because the difference in chi-square values for two nested models using the WLSMV chi-square values is not distributed as chi-square.[20]

Results

Of the n=1167 participants included in this analysis, most were female (66%), non-smokers (65%), and college graduates (49%), with a mean age of 35 years (standard deviation=15). Randomization (without replacement) yielded n=584 participants in subset-1 and n=583 in subset-2. Results showed a decrease in number of observations with each consecutive increase in days of illness, due primarily to early recovery. Data from illness day-1 (n=1167) was used for most analyses, as contained more observations than day-2 (n=1155) or day-3 (n=1152).

The proportion of missing data was less than 1%, with most from Head-congestion 0.9% (n=11) and least from Cough 0.2% (n=2). Little’s missing completely at random test failed to reject the null hypothesis that no identifiable pattern exists to the missing data.[21] The combined random pattern of missing values and estimated low degrees of missingness may reflect ease of WURSS use among participants.

Frequency of rating items as “0” (no symptom or no interference with daily function) varied among the WURSS items, with least for Feeling-tired (6%) and most for Chest-congestion (55%). Six items with >30% of their responses having “0” rating were excluded from further investigation because they demonstrated substantive floor effects and therefore may be less able to assess severity of ARI-illness over time (Figure-1).

Figure 1.

Frequency Ratings of Zero on WURSS-21 during Acute Respiratory Illness

*Any WURSS item with >30% its total count (n=1167) indicating ratings of zero were not retained for further investigation. These 6 items excluded from further investigation because they demonstrated substantive floor effects were; Live your personal life (n=376), Hoarseness (n=377), Interact with others (n=420), Work inside the home (n=467), Work outside the home (n=505) and Chest congestion (n=640).

To minimize redundancy, Live-your-personal-life, Interact-with-others, Work-outside, or-inside-the-home, and Walk-climb-stairs-exercise were excluded, while Accomplish-daily-activity was retained because it showed the highest correlation with most of these items and may reasonably encompass their individual meanings.

Approach-1

Of the 19 WURSS items examined, EFA was performed on 12 while 7 were eliminated based on >30% floor effects and redundancy. The EFA indicated 3 clinically meaningful dimensions/factors: 1-Head/Nasal; 2-Chest/Throat; and 3-Quality of life (QOL)/Functional impairment (3-factor model) with reasonably strong loadings of the WURSS items. The item assessing Cough showed a somewhat weaker loading estimate but was retained because it is known to occur frequently during ARI-illness, with substantive patient-oriented impact. Three significant cross-loading items were excluded while the remaining 9 items were retained for confirmatory analysis (Table-1).

Table 1.

Approach -1(Exploratory Factor Analysis) ^*

	Dimension/Factor (F)
WURSS items	F1-loading (Nasal) (Standard error)	F2-loading (Throat) (Standard error)	F3-loading (Quality of Life) (Standard error)
Runny nose	0.76 (0.025)	0.030 (0.044)	−0.0090 (0.020)
Plugged nose	0.71 (0.036)	−0.040 (0.030)	0.16 (0.054)
Sneezing	0.70 (0.035)	0.043 (0.044)	−0.021 (0.038)
Sore throat	−0.030 (0.017)	0.64 (0.063)	0.15 (0.055)
Scratchy throat	0.057 (0.072)	1.035 (0.071)	−0.003 (0.0040)
Cough	0.18 (0.054)	0.25 (0.046)	0.28 (0.055)
Head congestion	0.46 (0.040)	0.040 (0.030)	0.40 (0.046)
Feeling tired	−0.030 (0.041)	0.064 (0.043)	0.74 (0.037)
Think clearly	−0.0040 (0.010)	−0.025 (0.027)	0.83 (0.022)
Sleep well	0.33 (0.045)	0.048 (0.033)	0.52 (0.043)
Breathe easily	0.52 (0.039)	−0.043 (0.029)	0.45 (0.047)
Accomplish daily activities	0.040 (0.045)	−0.018 (0.024)	0.85 (0.035)

WURSS-21 excluding 2 items “How sick do you feel today?” and “Compared to yesterday, I feel…?”

Utilized applications include observed break point on scree plot, number of eigenvalues > 1, cumulative percent of variance explained by model, factor loadings ≥ 0.30, minimal number of cross loading items, and each factor having ≥ 3 WURSS items[13].

^*ARI day-1 data were explored using subset-1.These are unstandardized loadings.

Loading estimates in bold indicate domains where WURSS items belong.

Bold borders indicate cross-loaders with loadings >0.3, to be excluded. Three significant cross-loading items (Head congestion, Breathe easily and Sleep well) were excluded.

Nine items were retained for confirmatory analysis: Nasal (Runny nose, Plugged nose and Sneezing), Throat (Sore throat, Scratchy throat and Cough) or QOL dimension (Feeling tired, Think clearly and Accomplish daily activities).

Approach-2

CFA using the retained 9 items showed significant reliability and dimensional cohesion. Exclusion or inclusion of Cough resulted in satisfactory indices of model fit with good coefficient estimates. The model excluding the Cough item showed slightly better fit (CFI=0.97, TLI=0.96, RMSEA=0.09) compared to the model including cough (CFI=0.93, TLI=0.89, RMSEA=0.13). However, Cough was retained because of the widespread recognition that cough is an important symptomatic and quality-of-life component of ARI-illness.

Following satisfactory EFA-CFA, both subsets were combined and the item-dimension structure showed high internal consistency (Cronbach’s α) and adequate convergent validity (composite reliability and average variance extracted-AVE,Table-2). Finally, the 3-factor model with retained 9 items explained a significant proportion of variance in the original WURSS-21 (R-squared=0.92). It also showed significant factor correlations (Figure-2). Discriminant validity was satisfactory because construct’s AVE was greater than the shared variance.[22]

Table 2.

Estimates of Reliability and Validity of the Wisconsin Upper Respiratory Symptom Survey (WURSS-11).

Dimensions	Nasal	Throat	Quality of life
Cronbach’s α	0.74	0.71	0.81
Composite reliability	0.77	0.78	0.84
Average variance extracted	0.53	0.55	0.64

Adequate convergent validity = Composite reliability> Average variance extracted or Average variance extracted >0.5

Discriminant validity was satisfied because the average variances extracted for Nasal (0.53), Throat (0.55) and QOL (0.64) were larger than their shared variances (square of the factor correlation)[22].

Figure 2.

3-dimensional factor model of Wisconsin Upper Respiratory Symptom Survey (WURSS-11)*

*Standardized values

The 3 dimensions are Nasal (Runny nose, Plugged nose, Sneezing), Throat (Cough, Sore and Scratchy throat) and QOL (Feeling tired, Think clearly, Accomplish daily activities). Introductory “How sick do you feel today?” and concluding item “Compared to yesterday, I feel…? which measure different time frames were not included during the analysis but will be added to the retained 9 items to form the WURSS-11.

Invariance was based on non-significant chi-square difference tests (p> 0.05), and minimal change in the goodness of fit indices of CFI and TLI (<=0.01) between models.[23] Results from Table 3 indicate that the WURSS demonstrated evidence of longitudinal scalar invariance for factors-1 and-3, but not for factor variances. The reason factor-2 failed the invariance assessment was due to the incorporation of a fairly weak item (i.e., cough based on clinical rather than statistical justification).

Table 3.

Longitudinal invariance of the WURSS over 3 day period.

Factors	Models		df	p-value	CFI	TLI	RMSEA	WRMR
Factor 1
	Model 1	2.533	5	0.772	1.00	1.00	0.000	0.143
	Model 2	5.020	7	0.658	1.00	1.00	0.000	0.210
	Model 3	23.108	8	0.003	0.997	0.994	0.040	0.534
Factor 2
	Model 1	21.779	5	<0.001	0.998	0.994	0.054	0.439
	Model 2	51.097	7	<0.001	0.995	0.988	0.073	0.722
	Model 3	137.23	8	<0.001	0.984	0.970	0.118	1.247
Factor 3
	Model 1	5.527	5	0.354	1.00	1.00	0.010	0.190
	Model 2	8.587	7	0.283	1.00	1.00	0.014	0.261
	Model 3	73.833	8	<0.001	0.993	0.987	0.084	0.949
Comparisons of the WURSS over 3 day period.
Factors		Model comparisona	diff	dfdiff	p-value	CFI	TLI
Factor 1
		Model 1 vs 2	3.017	2	0.22	0.000	0.000
		Model 2 vs 3	9.730	1	0.01	0.003	0.006
Factor 2
		Model 1 vs 2	26.512	2	<0.001	0.003	0.006
		Model 2 vs 3	65.976	1	<0.001	0.011	0.018
Factor 3
		Model 1 vs 2	3.490	2	0.17	0.000	0.000
		Model 2 vs 3	33.013	1	<0.001	0.007	0.013

Note. -

^aChi-square difference test based on Satorra and Bentler correction[17,20]. χ²=Chi-square; DF=Degree of Freedom; root mean square error of approximation (RMSEA <0.06 acceptable fit); comparative fit index (CFI > 0.95 acceptable fit) or Tucker-Lewis index (TLI > 0.95 acceptable fit); weighted root mean square residual (WRMR).

Discussion

This study demonstrates that the WURSS-21 can be reduced to WURSS-11, preserving reliability and domain structure. The introductory and concluding items will be added to the retained 9 items to form the WURSS-11.

Among the retained items, QOL measures appeared to perform somewhat better, with higher parameter estimates, compared to symptom severity. This supports previous studies which showed that QOL items perform slightly better and are more important to patients than specific symptoms [1,4,24].

The reduced WURSS-11 demonstrates satisfactory indices of fit with good reliability estimates. Most importantly, it retains and reflects similar dimensional construct as the parent WURSS-21. While not yet assessed, WURSS-11’s retention of roughly half of the items from the parent survey may suggest a reduction of WURSS completion time by as much as 50%.

These results show stability of WURSS-11 across the 1^st three days of ARI-illness. It also suggests WURSS-11 retains content validity while construct validity is likely to be satisfactory, based on previous findings of significant convergent and discriminant validities.[1,4,6] However, prospective studies are needed to compare the performance of the WURSS-11 to its parent WURSS-21 and external comparators such as general and illness-specific self-report health measures, and viral nucleic acids.

Study limitations include limited diversity in the study population, and lack of external replicated validation. However, generalizability is strengthened by the large sample size (n=1167), assorted age groups, large time span (2002–2010), different seasons of ARI-illness with incidental viral microbes, and most importantly the similarities between the inclusion and exclusion criteria among all 4 WURSS studies.

In conclusion, this study has successfully created the WURSS-11, which is a shorter version of the current WURSS-21. It suggests that the WURSS-11 has similar dimensional structure as the parent WURSS-21, and may be as reliable and responsive as the longer versions of WURSS. This shortened self-reporting survey may reduce WURSS completion time and increase response rate, but will need additional validation in future studies.