Validity and Reliability of the Fatigue Severity Scale in a Real-World Interstitial Lung Disease Cohort

Abstract

Rationale

Fatigue is a common and debilitating symptom for people living with interstitial lung disease (ILD). Studies on fatigue in ILD are limited, and little headway has been made toward developing interventions targeting the alleviation of fatigue. A barrier to progress is a lack of knowledge around the performance characteristics of a patient-reported outcome measure to assess fatigue in patients with ILD.

Objectives

To assess the validity and reliability of the Fatigue Severity Scale (FSS) for measuring fatigue in a national cohort of patients with ILD.

Methods

FSS scores and several anchors were measured in 1,881 patients from the Pulmonary Fibrosis Foundation Patient Registry. Anchors included the Short Form 6D Health Utility Index (SF-6D) score and a single vitality question from the SF-6D; the University of California, San Diego, Shortness of Breath Questionnaire; FVC; Dl_CO; and 6-minute-walk distance. Internal consistency reliability, concurrent validity, and known-groups validity were assessed. Structural validity was assessed using confirmatory factor analysis.

Measurements and Main Results

The FSS demonstrated high internal consistency (Cronbach’s α = 0.96). There were moderate to strong correlations between FSS score and patient-reported anchors (vitality question from the SF-6D [r = 0.55] and University of California, San Diego, Shortness of Breath Questionnaire total score [r = 0.70]) and weak correlations between FSS score and physiological measures (FVC [r = −0.24], percentage predicted Dl_CO [r = −0.23], and 6-minute-walk distance [r = −0.29]). Higher mean FSS scores, indicating greater fatigue, were observed among patients using supplemental oxygen, those prescribed steroids, and those with lower percentage predicted FVC and percentage predicted Dl_CO. The confirmatory factor analysis results suggest that the nine questions of the FSS reflect one dimension of fatigue.

Conclusions

Fatigue is an important patient-centered outcome in ILD that is poorly correlated with physiological measures of disease severity, including lung function and walk distance. These findings further support the need for a reliable and valid measure of patient-reported fatigue in ILD. The FSS possesses acceptable performance characteristics for assessing fatigue and distinguishing different degrees of fatigue among patients with ILD.

At a Glance Commentary

Scientific Knowledge on the Subject

Studies on fatigue in patients with interstitial lung disease (ILD) are limited, and little headway has been made toward developing interventions targeting alleviation of fatigue. A barrier to progress is a lack of knowledge around the performance characteristics of a patient-reported outcome measure to assess fatigue in patients with ILD.

What This Study Adds to the Field

Fatigue is poorly correlated with commonly collected physiological measures of disease severity, including lung function and walking distance in ILD. The Fatigue Severity Scale is a short, valid, and reliable patient-reported outcome measure that can capture fatigue severity in patients with a variety of ILD subtypes and treatment groups.

Fatigue, a multifaceted and profoundly debilitating symptom, is highly prevalent among patients with interstitial lung disease (ILD) (1, 2). People who live with ILD describe fatigue as distinguishable from and, for some, more bothersome than breathlessness (1, 3–6). This phenomenon has been described as a total lack of energy or a feeling of complete exhaustion that negatively affects one’s ability to function both physically and mentally and impairs health-related quality of life (HRQOL) (7). There are several potential etiologies of fatigue in ILD, including those that stem from the pathophysiology of the disease, its treatments (8), and comorbidities (e.g., obstructive sleep apnea, heart disease, connective tissue disease, mood disturbance) (9–11). Despite how intrusive and profoundly taxing fatigue may be, there remains a poor understanding of the optimal way to evaluate or treat it in patients with ILD. Thus, fatigue remains underdiagnosed and undertreated (12). Aside from research conducted in sarcoidosis (13–16), few studies have targeted fatigue as a high-tier outcome in ILD (2). Fatigue is experienced differently among patients with sarcoidosis compared with those with idiopathic pulmonary fibrosis (IPF) (17). Given the potential for systemic involvement in sarcoidosis (and lack of lung involvement in a percentage of patients) and its slower rate of progression, data from sarcoidosis do not generalize to other forms of ILD (17, 18). The lack of robust data and the absence of attention to fatigue have created a conspicuous gap in ILD patient-centered care.

Experts have designated fatigue as a high-priority endpoint in ILD patient-centered research (19). However, fatigue is challenging to study because it is a complex symptom with multiple, potentially overlapping causes. A more glaring obstacle is that we have yet to identify a valid patient-reported outcome measure (PROM) of fatigue in patients with ILD (other than sarcoidosis, a multisystem disease) (13, 20–22). Although several fatigue scales have been developed and have undergone validity studies in other patient populations (23), these data are lacking in the broader ILD population. Guidelines from experts in psychometrics, clinical trial design, and regulatory agencies require robust psychometric testing of instruments that measure patient-reported outcomes in the population of interest before their use in therapeutic trials (24–27). Before we can evaluate, select, and implement interventions that reduce patients’ fatigue in ILD, we must first identify the appropriate PROMs to use. Determining which PROMs have acceptable performance characteristics in ILD also provides a way to systematically assess fatigue in clinical practice. The objective of the present study was to address gaps in our understanding of the clinical relevance of fatigue in patients with ILD and determine the reliability and validity of the Fatigue Severity Scale (FSS) in a national geographically diverse cohort of patients with ILD.

Methods

Study Population

The Pulmonary Fibrosis Foundation (PFF) Patient Registry is a longitudinal observational study of ∼2,000 adults with ILD across 42 PFF Care Centers in the United States (28). Details on the registry are provided in the online supplement. We included registry participants who completed all FSS items at enrollment.

FSS

The FSS is a nine-item, self-reported questionnaire assessing perceived fatigue severity in various functional and behavioral aspects of life, with a subjective measurement of daytime fatigue that is independent of sleepiness and depression (29). The survey was designed as a measure of fatigue in patients with chronic neurologic disease, and it is now administered to patients with several other chronic illnesses (30–33). Each item includes a seven-point, Likert-type response where 1 = “strongly disagree” and 7 = “strongly agree.” Higher scores indicate more severe fatigue. Mean scores are 2.3 (SD, 0.7) among healthy adults (32), 4.6 (SD, 1.5) (34) among subjects with systemic lupus erythematosus, and 5.3 (SD, 0.9) among subjects with multiple sclerosis (35).

Statistical Analysis

Descriptive statistics were calculated for baseline variables. Frequency tables were generated for each item; floor and ceiling effect thresholds (e.g., when a considerable number of participants chose either the highest or lowest scoring response option) were prespecified at 15% (36).

We assessed FSS reliability using Cronbach’s α. Interitem and item-total correlations were calculated using polychoric correlations and the intraclass correlation coefficient.

Concurrent validity (e.g., a measure of how well the FSS compares with a prior validated test in ILD) was assessed using Spearman correlation coefficients between the mean FSS score and several anchors collected at baseline. We chose anchors (questionnaires or metrics of disease severity collected per registry protocol) that we hypothesized would be associated with fatigue. These included 1) a single question on vitality from the Short Form 6D Health Utility Index (SF-6D) (37–39); 2) the total score on the SF-6D, representing HRQOL; 3) the University of California, San Diego, Shortness of Breath Questionnaire (UCSD-SOBQ) total score (40, 41); 4) percentage predicted FVC; and 5) percentage predicted Dl_CO, and 6) 6-minute-walk distance. FVC and Dl_CO are universally used to describe the severity of ILD (42, 43). For the FVC and Dl_CO analyses, we included subjects with data available within 30 days of completion of the FSS. We considered |r| < 0.4 to indicate a weak correlation, 0.4 < |r| < 0.7 to indicate a moderate correlation, and |r| > 0.7 to indicate a strong correlation (44, 45).

Given the large sample size, we assessed known-groups validity (e.g., the ability of the FSS to distinguish among distinct groups of participants) using four one-way ANOVAs in which we compared mean FSS score (the dependent variable) across categories of the following independent variables: 1) ILD disease type (collagen vascular disease, hypersensitivity pneumonitis, IPF, and the heterogeneous grouping “other”); 2) use of supplemental oxygen; 3) percentage predicted FVC (⩽50%, 50–80%, or >80%); 4) percentage predicted Dl_CO (⩽40%, 40–60%, 60–75%, or >75%); and 5) use of steroids or immunosuppressants (steroids only, immunosuppressants only, both, or neither). We hypothesized that fatigue would be higher among patients with collagen vascular disease (46), those reliant on supplemental oxygen, those with worse FVC or Dl_CO, and those prescribed steroids.

To test the hypothesis of an underlying unidimensional fatigue construct for the FSS (e.g., that the nine FSS questions measure only one primary concept of fatigue), we conducted a confirmatory factor analysis (CFA) with robust SEs. Model fit was assessed using the Satorra-Bentler scaled chi-square statistic, comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean square error of approximation (RMSEA), and standardized root mean squared residual (SRMR). Thresholds for fit were set at ⩾0.95 for the CFI and TFI, <0.06 for the RMSEA, and <0.08 for the SRMR (47).

Sensitivity analyses were conducted to assess the ordinal structure and ceiling effects present in the FSS responses. These are included in the online supplement. Analyses were conducted using SAS version 9.4 (SAS Institute, Inc.). R version 4.1 (R foundation for Statistical Computing) was used to generate CFA and boxplots.

Results

The analytic sample included 1,881 PFF Patient Registry participants with a mean age of 68.1 (SD, 10.1) years. Sixty-two percent of the cohort had IPF. The mean FSS score was 4.25 (SD, 1.78). All additional relevant baseline characteristics are shown in Table 1. Floor and ceiling effects are displayed in Table 2 (accompanying histograms for each individual item are available in the online supplement). Eight of the nine FSS items demonstrated ceiling effects, and five demonstrated floor effects.

Table 1.

Patient Demographics and Baseline Characteristics

Characteristic	Distribution (n = 1,881)
Women, n (%)	686 (36)
Age, yr	68.1 (10.1)
Duration of ILD, yr, median (interquartile range)	2.74 (0.56–3.85)
Race, n (%)
Asian	43 (2)
Black	93 (5)
White	1,694 (90)
Other or unknown	51 (3)
Ethnicity, n (%)
Hispanic	110 (6)
Non-Hispanic	1,708 (91)
Unknown	63 (3)
ILD subtype at enrollment, n (%)
CVD/autoimmune disease	308 (16)
Hypersensitivity pneumonitis	144 (8)
IPF	1,172 (62)
Other*	257 (14)
FSS mean score	4.25 (1.78)
UCSD-SOBQ total score	42.16 (26.5)
SF-6D total score	0.68 (0.11)
SF-6D vitality	3.29 (1.04)
FVC, L†	2.62 (0.86)
ppFVC, %†	68.54 (18.2)
DlCO, ml/min/mm Hg†	12.30 (5.4)
ppDlCO†	42.83 (16.5)
6 min-walk distance, m	361 (128)
Oxygen use, n (%)	843 (45)
Antifibrotic use, n (%)	771 (41)
Steroid use, n (%)	642 (34)
Steroid-sparing immunosuppressant, use n (%)	395 (21)
Biologic use, n (%)	24 (1.3)
Cough suppressants (narcotic based), n (%)	122 (6.5)
Comorbidities, n (%)
OSA	469 (25)
OSA treated with NIPPV	394 (21)
Anemia	139 (7.4)
Cancer‡	212 (11)
CHF	79 (4.2)
Depression	325 (17)
PAH	141 (7.5)
Thyroid disease	296 (16)

Definition of abbreviations: CHF = congestive heart failure; CVD = collagen vascular disease; FSS = Fatigue Severity Scale; ILD = interstitial lung disease; IPF = idiopathic pulmonary fibrosis; NIPPV = noninvasive positive pressure ventilation; OSA = obstructive sleep apnea; PAH = pulmonary arterial hypertension; ppDl_CO = percentage predicted Dl_CO; ppFVC = percentage predicted FVC; SF-6D = Short Form 6D Health Utility Index; UCSD-SOBQ = University of California, San Diego, Shortness of Breath Questionnaire.

Data are mean (SD) on the basis of the total number of patients in each group, unless stated otherwise.

^*“Other” includes drug-induced, non-IPF idiopathic interstitial pneumonia, occupational exposure, and other specific ILD.

^† For FVC, n = 1,307; for ppFVC, n = 1,302; for Dl_CO, n = 1,154; and for ppDl_CO, n = 1,150.

^‡ History of cancer at any point in time before or at enrollment (excluding skin cancer).

Table 2.

Floor and Ceiling Effects

Item	Response = 1	%	Response = 7	%
1. My motivation is lower when I am fatigued	160	8.51	559	29.72
2. Exercise brings on my fatigue	144	7.66	414	22.0
3. I am easily fatigued	217	11.54	329	17.49
4. Fatigue interferes with my physical functioning	220	11.70	331	17.60
5. Fatigue causes frequent problems for me	355	18.87	243	12.92
6. Fatigue prevents sustained physical functioning	285	15.15	354	18.82
7. Fatigue interferes with carrying out certain duties and responsibilities	314	16.69	332	17.65
8. Fatigue is among my three most disabling symptoms	353	18.77	395	21.00
9. Fatigue interferes with my work, family, or social life	428	22.75	306	16.27

Reliability

Cronbach’s α was 0.96. There was a strong correlation (intraclass correlation coefficient = 0.71) among the nine FSS items.

Concurrent Validity

The strengths of the correlations were moderate and in the hypothesized directions between FSS score and the vitality question from the SF-6D (r = 0.55) and the total SF-6D score (r = 0.61). There was a strong correlation in the hypothesized direction between FSS score and UCSD-SOBQ total score (r = 0.70). There were weak correlations in the hypothesized directions between FSS score and percentage predicted FVC (r = −0.24), percentage predicted Dl_CO (r = −0.23), and 6-minute walk distance (r = −0.29) (Figure 1).

Figure 1.

Correlations with baseline Fatigue Severity Scale mean score and anchors. 6MWD = 6-minute-walk distance; SF-6D = Short Form 6D Health Utility Index; UCSD-SOBQ = University of California, San Diego, Shortness of Breath Questionnaire.

Known-Groups Validity

Mean FSS scores did not differ significantly across ILD subgroups (P = 0.12) (Figure 2A). Patients on supplemental oxygen had a significantly higher mean FSS score (4.80) than those not on supplemental oxygen (3.80) (P < 0.0001) (Figure 2B). Individuals with FVC ⩽ 50% predicted had a significantly higher mean FSS score (4.83) than those in higher FVC subgroups (FVC 50–80%, mean = 4.28; FVC > 80%, mean = 3.67) (P < 0.0001) (Figure 2C). Individuals with Dl_CO ⩽ 40% predicted had a significantly higher mean FSS score (4.54) than those in higher Dl_CO subgroups (Dl_CO 40–60%, mean = 4.03; Dl_CO 60–75%, mean = 3.44; Dl_CO > 75%, mean = 3.82) (P < 0.0001) (Figure 2D). Individuals prescribed either steroids or immunosuppressants plus steroids had significantly higher mean FSS scores (4.50 and 4.69, respectively) than those prescribed immunosuppressants alone or neither immunosuppressants nor steroids (4.16 and 4.07, respectively) (P < 0.0001) (Figure 2E).

Figure 2.

Known-groups validity of the FSS with anchors. *P < 0.001. CVD = collagen vascular disease; FSS = Fatigue Severity Scale; HP = hypersensitivity pneumonitis; ILD = interstitial lung disease; ImmunoSup = immunosuppressants; IPF = idiopathic pulmonary fibrosis.

Structural Validity

The nine FSS questions loaded strongly on a single dimension (concept) of fatigue, with standardized loadings (e.g., correlation coefficients between the item and fatigue) >0.86, except for question 1 (“My motivation is lower when I am fatigued”) and question 2 (“Exercise brings on my fatigue”), with loadings of 0.61 and 0.71, respectively. The results of the CFA suggest that there is an acceptable fit and support the FSS as a unidimensional scale (an important characteristic of a high-quality PROM). The CFA model fit was satisfactory on the basis of the CFI (0.949), TLI (0.932), and SRMR (0.031). Although the statistically significant Satorra-Bentler chi-square test and the higher than acceptable RMSEA indicate that the hypothesized model is not a perfect fit (χ²[df = 27] = 960; P < 0.001; RMSEA = 0.134), the CFI (0.949), TLI (0.932), and SRMR (0.031) provide evidence that the FSS adequately reflects the dimensionality of the construct of fatigue in patients with ILD. Our unidimensional factor model for FSS is presented in the online supplement.

Discussion

We report findings from the first validation study of a PROM that assesses fatigue in a large cohort of patients with ILD. We found that high degrees of fatigue are prevalent among patients with ILD and that this is poorly correlated with pulmonary function testing, further strengthening the argument that a separate valid and reliable measure of fatigue is needed. FSS scores possess requisite reliability and validity for assessing fatigue in patients with ILD. FSS scores were correlated with anchors that either directly measured fatigue (e.g., amount of energy according to the single SF-6D item on vitality) or were hypothesized to be related to fatigue. FSS scores satisfactorily discriminated between patients hypothesized to have differing degrees of fatigue severity on the basis of physiological measures of ILD severity (FVC and Dl_CO), the need for supplemental oxygen, and the use of medications that affect fatigue.

As expected, baseline FSS scores were higher than in the general population, suggesting a substantial degree of fatigue in the cohort. However, in contrast to our hypothesis, we did not observe significant differences in fatigue between patients with different subtypes of ILD. These results remind us that in general, all patients with ILD, even those without underlying systemic conditions, have significant fatigue, a result that confirms and extends the findings of studies with much smaller samples in which the FSS was administered (10, 48).

Our analyses support the unidimensionality of the FSS in ILD (49, 50). The strong correlations between the nine FSS items and a single dimension of fatigue support our a priori hypothesis that all items in the instrument are measuring the same construct of fatigue and confirm its structure as a valid scale for assessing fatigue in this population.

The receipt of a prescription of supplemental oxygen has been previously identified as a significant milestone in the disease journey of patients with ILD. To many of our patients, this milestone signifies that there has been substantial progression of their disease, often bringing to light changes in quality of life and in perspective on the future (51, 52). The FSS measured higher fatigue among patients who were using supplemental oxygen than those who were not. Although we did not incorporate detailed information on flow rate and the frequency/duration of use, this analysis supports the discriminant validity of the FSS: it can differentiate patients with differing degrees of disease severity (as defined by receipt of a prescription of supplemental oxygen or not) and, by extension, differing degrees of fatigue (53). FSS scores were higher in patients taking corticosteroids (either with or without immunosuppressants) compared with those who were not. This is an important finding suggesting that the FSS score could be useful in the clinic or as an endpoint in therapeutic trials for agents that may improve or worsen fatigue. Although studies have shown that steroids may alleviate fatigue due to chronic fatigue syndrome or rheumatoid arthritis (54, 55), steroids suppress the adrenal axis and may unmask fatigue when withdrawn (56). We believe our results warrant further investigation.

Although we found that the FSS correlated moderately well with a generic measure of fatigue and strongly with a measure of breathlessness, there were weak but statistically significant correlations between the FSS score and the percentage predicted FVC and Dl_CO. This reminds us that patients with more severe ILD are likely to have more severe fatigue. However, the weak correlation supports the argument that the FSS captures unique information apart from what these objective measures capture. The finding that fatigue is prevalent among patients with ILD, but is only weakly associated with lung function, has striking clinical relevance. If we continue to evaluate the severity of disease solely by lung function, we risk missing vital, clinically relevant information about patients’ well-being: how they feel and function in their daily lives. Weak correlations between physiological measures and scores from PROMs are not uncommon in patients with ILD (41, 57). These findings remind us of the importance of using metrics other than pulmonary function testing data alone if we want a comprehensive assessment of disease severity (19, 22, 58).

We used a prespecified cutoff of 15% or higher for floor and ceiling effects, which is a commonly chosen cutoff in instrument validation studies (36, 59). When using this conservative cutoff, we found that eight of the nine items demonstrated ceiling effects. This is not uncommon when an existing instrument developed in a certain target population is administered in a new target population. On a scale such as the FSS, with seven potential response options, if individuals were evenly distributed across response options, we would expect to see 14.3% of responses for each option. As we observed, it is not unusual for an option to be selected by 15–20% of individuals. The ceiling effect would potentially come into play only if an intervention makes fatigue worse; the instrument would not capture worsening in respondents with the worst possible scores at baseline. For interventions aimed at alleviating fatigue, in which the expected change is toward the lower end of the scale (e.g., away from the ceiling), there is no concern (60). In future investigations, the FSS could be subjected to item response theory analysis in ILD to further investigate the floor and ceiling effects found in this analysis (58).

A challenge in measuring fatigue (as with nearly every PROM) is the absence of a gold standard. Although some may consider fatigue to be a central lack of energy or exhaustion, others, especially those with chronic respiratory disease, may interpret the word to mean breathlessness (6, 61), whereas others with chronic neuromuscular disease may consider this to be muscular in nature (62). A challenge with the FSS is that it does not provide an explicit definition of fatigue in the instructions, which may influence how the questions are interpreted by patients with ILD. To address this concern, we included the vitality question from the SF-6D (a question specifically asking about fatigue by using the word “energy” in the item) as a separate anchor, which was found to be moderately correlated with the FSS score (r = 0.55), increasing our confidence that the FSS measures a construct of fatigue separate from breathlessness.

The present analysis provides evidence that FSS scores possess excellent validity and reliability at a single point in time in patients with ILD; however, more research is needed to better understand the instrument’s responsiveness to change and to assess minimal important change thresholds in this population.

This is the first study to highlight fatigue as an important outcome using real-world data from a large, geographically diverse registry of patients with ILD other than sarcoidosis. As we describe in the introduction, sarcoidosis is a multisystem granulomatous disease, which contrasts with several forms of ILD and pulmonary fibrosis, in which the pathology is localized to the lungs. The PFF Patient Registry specifically excludes patients with sarcoidosis (28). Furthermore, although other fatigue scales have been used to measure fatigue in different patient populations (cancer, chronic fatigue syndrome, after critical illness) (23), these data cannot be extrapolated to patients with ILD. As emphasized in established guidelines from measurement experts (including from the International Society for Quality of Life Research, Consensus-Based Standards for the Selection of Health Measurement Instruments, and the U.S. Food and Drug Administration) (25, 26, 63, 64), like any other clinical outcome assessment, fatigue scales must undergo rigorous analyses to ensure that they possess the requisite measurement properties to assess fatigue in the population of interest (i.e., to establish that they are “fit for purpose” in the target population). This is the first study to assess the validity and reliability of a fatigue scale in ILDs other than sarcoidosis using established guidelines.

The strengths of our study include the large, national cohort of patients who were diagnosed and/or had diagnosis confirmation at an accredited PFF center of excellence and the diverse group of ILD subtypes. The use of the registry provides us with real-world evidence, which is not subject to the constraints and often limited generalizability of clinical trial data. The combination of rich clinical data with PROMs is also an important strength to note.

There are several limitations to our study. There is no gold-standard measure of fatigue to apply as an anchor in this patient population. In this analysis, we used the closest valid scales available to us in the registry (e.g., the vitality question on the SF-6D, the total SF-6D score measuring HRQOL, and the USCD-SOBQ, which measures breathlessness) as anchors and were able to provide some information regarding correlation with those instruments that measure similar or interrelated domains that patients experience with ILD. Given that this is a registry using real-world data, the pulmonary function testing data were not collected on the same day as the surveys were completed. We chose the 30-day interval for FVC and Dl_CO to be consistent with what has been done in other rigorous patient-reported outcome validation studies in ILD (65) and allow some proximity to the test within the confines of the limitations of data collection for the registry. A majority of the cohort is composed of patients with IPF, so we interpret the lack of significant differences in fatigue across the different types of ILD with some caution. In addition, patients were enrolled from a single country and administered the instrument in a single language. As such, we cannot generalize to patients with ILD in other countries. Confirmation of these findings in other populations across the world would be valuable. This may require additional work to perform both linguistic and cross-cultural validity testing.

Conclusions

Fatigue is a frequently encountered symptom and thus an important patient-centered outcome in ILD that is often overlooked and has been understudied. We found that fatigue is weakly correlated with commonly collected physiological measures of disease severity, including lung function and 6-minute-walk distance. This finding is striking because it underscores how little such “objective” metrics reflect the perceptions patients have of their own amounts of energy. Furthermore, the FSS is a valid and reliable instrument to capture fatigue severity in patients with a variety of ILD subtypes and treatment groups. As a brief, easily and quickly administered patient-reported outcome scale, the FSS can be useful to assess fatigue from the patient perspective in both clinical practice and therapeutic trials.