Evaluation of the Psychometric Properties of the PROMIS Cancer Fatigue Short Form with Cancer Patients

Abstract

Objective

Fatigue is common among cancer patients and adversely impacts quality of life. As such, it is important to measure fatigue accurately in a way that is not burdensome to patients. The 7-item Patient Reported Outcome Measurement Information System (PROMIS) Cancer Fatigue Short Form scale was recently developed using item response theory (IRT). The current study evaluated the psychometric properties of this scale in two samples of cancer patients using classical test theory (CTT).

Methods

Two samples were used: 121 men with prostate cancer and 136 patients scheduled to undergo hematopoietic cell transplantation (HCT) for hematologic cancer. All participants completed the PROMIS Cancer Fatigue Short Form as well as validated measures of fatigue, vitality, and depression. HCT patients also completed measures of anxiety, perceived stress, and a clinical interview designed to identify cases of cancer -related fatigue.

Results

PROMIS Cancer Fatigue Short Form items loaded on a single factor (CFI = 0.948) and the scale demonstrated good internal consistency reliability in both samples (Cronbach’s alphas > 0.86). Correlations with psychosocial measures were significant (p-values < .0001) and in the expected direction, offering evidence for convergent and concurrent validity. PROMIS Fatigue scores were significantly higher in patients who met case definition criteria for cancer-related fatigue (p < .0001), demonstrating criterion validity.

Conclusion

The current study provides evidence that the PROMIS Cancer Fatigue Short Form is a reliable and valid measure of fatigue in cancer patients.

Introduction

Research suggests that fatigue is one of the most common and distressing symptoms for cancer patients (1, 2). As such, measurement of fatigue in cancer patients has been the focus of significant research interest. While there are several existing measures of fatigue for cancer patients, few have been developed using item response theory (IRT) (3).

The Patient Reported Outcome Measurement Information System (PROMIS) Fatigue Scale was developed as part of a National Institutes of Health (NIH) funded effort to build and validate item banks using item response theory to measure important health outcomes across clinical and non-clinical populations (4, 5). As part of this effort, item pools were developed from identification of existing items, focus group input, expert item review and revision, and cognitive interviewing (4). The resulting item banks were further refined and calibrated using IRT, and can be used to create short form measures (6, 7).

Assumptions of IRT dictate that the instrument be unidimensional and demonstrate local independence, meaning that the items should load on one factor and not be highly related to each other. Moreover, one of the strengths of instruments developed using IRT is that they are based upon ability scores which are test independent, meaning that the test can be developed to be sensitive across a range of impairment and regardless of the particular choice of test items. This differs from measurement development based upon Classical Test Theory (CTT), which is based upon observed scores and true scores that are test and sample dependent. PROMIS item banks can be administered either uniformly using a defined set of items or interactively using computerized adaptive testing (CAT) (4). Thus, the PROMIS initiative has the potential to advance measurement of patient-reported outcomes using standardized measures that are easy to administer, adaptable, and allow for comparison across clinical and non-clinical samples.

The measure that is the focus of the current study, the 7-item PROMIS Cancer Fatigue Short Form, was developed from a bank of 95 items (7). The final set of items was selected so that there was consistency in the response scale options, broad coverage across the fatigue continuum, and good precision of measurement (7). To the best of our knowledge, only one published study has examined the psychometric properties of the PROMIS Cancer Fatigue Short Form (4). The study used data collected from a sample that combined a representative U.S. general population with multiple disease populations including people with cancer (4). The study found that the scale demonstrated good concurrent validity and was highly correlated in the expected direction with the SF-36 Vitality Scale, and the Functional Assessment of Chronic Illness Therapy-Fatigue Scale.

The goal of the current study was to build on this work by examining the reliability and validity of the PROMIS Cancer Fatigue Short Form in two samples of cancer patients, men diagnosed with prostate cancer (Sample 1) and men and women diagnosed with hematologic cancer (Sample 2). It was hypothesized that the measure would be unidimensional as demonstrated by factor analysis, free of locally dependent items as demonstrated by residual correlations, and have strong internal consistency. It was also hypothesized that the measure would demonstrate concurrent, convergent and criterion validity. It was predicted that PROMIS Cancer Fatigue Short Form scores would be significantly correlated with other measures of fatigue as well as measures of vitality, depression, anxiety, and perceived stress, and that PROMIS Cancer Fatigue Short Form scores would be significantly higher in patients meeting the criteria for a case definition of cancer-related fatigue.

Method

Participants

Sample 1

Participants were recruited to a larger study examining cognitive functioning and quality of life among men with prostate cancer and included two subsamples: men starting androgen deprivation therapy (ADT) for prostate cancer (ADT+) and men previously treated with surgery for prostate cancer who had not received ADT (ADT−). Participants were eligible if they were greater than 18 years of age, were able to speak and read English, had at least a sixth grade education, had no history of cerebrovascular accident, scored in the normal range (i.e., less than three errors) of mental functioning on the Short Portable Mental Status Questionnaire (8), and were able to provide informed consent. The ADT+ participants were also required to meet the following criteria: they were diagnosed with non-metastatic or asymptomatic metastatic prostate cancer, had not been receiving treatment within the past 12 months for another cancer diagnosis, had no clinical evidence of another diagnosed cancer at the last follow-up visit, had never been diagnosed with primary brain cancer and/or received cranial radiation, and were to be treated with ADT continuously for at least 6 months. The ADT− participants were also required to meet the following criteria: they were diagnosed with non-metastatic prostate cancer, had not been diagnosed with any other form of cancer (except non-melanoma skin cancer), had undergone prostatectomy, had no history of recurrent disease since undergoing prostatectomy, had no history of other forms of prostate cancer treatment, were not scheduled for additional prostate cancer treatment, and were not receiving testosterone supplementation.

Sample 2

Participants were recruited to a larger study examining quality of life and cognitive functioning among patients undergoing hematopoietic cell transplantation (HCT) for hematologic malignancies. Participants were eligible if they were 18 years of age or older, were diagnosed with hematologic cancer; were scheduled to receive an allogeneic HCT with peripheral blood stem cells, had no history of cerebrovascular accident or head trauma with loss of consciousness, had completed six or more years of formal education, were capable of speaking and reading standard English, and were able to provide written informed consent.

Procedure

Sample 1

Following Institutional Review Board (IRB) approval, participants were recruited between September 2008 and July 2012. Participants were compensated $80. Participants were screened for eligibility using medical record review. ADT+ participants were recruited during outpatient appointments at Moffitt Cancer Center or James A. Haley Veterans’ Hospital. Potential participants were approached in clinic to verify eligibility and inform them about the study. Those who agreed to participate provided signed informed consent, and were escorted to a private room in the clinic. Patients then had the option of completing the self-report measures that day, or within one month of the start of ADT. Potential ADT− participants were recruited by mail and telephone. They were initially mailed a letter and directions for opting out of the study. Those who did not opt out were contacted by phone to have the study explained. Those who were eligible and interested scheduled an appointment to obtain written informed consent and complete the self-report measures. For the larger study, 484 patients were approached for participation; of these, 189 refused, 4 withdrew, 6 were ineligible after consent, 4 failed screening, 63 were unable to be matched, and 218 signed consent and completed the baseline assessment (45% of those contacted). Of those who completed the baseline assessment, PROMIS Cancer Fatigue Short Form was administered to 121 participants. Thus, analyses were conducted on 121 participants with evaluable data.

Sample 2

Following IRB approval, participants were recruited between September 2010 and July 2012. Participants were compensated $20. Potential participants were identified by their transplant physicians and approached during regularly scheduled outpatient visits at Moffitt Cancer Center. All potential participants were informed about the study. Those who wished to participate provided signed informed consent, and were escorted to a private room in the clinic. Participants had the option of completed the self-report measures that day or on another day prior to starting pre-transplant conditioning. For the larger study, 273 patients were approached for participation; of these, 48 refused, and 225 signed consent and completed the baseline assessment (82% of those contacted). Of those who completed the baseline assessment, PROMIS Cancer Fatigue Short Form was administered to 136 participants. Thus, analyses were conducted on 136 participants with evaluable data.

Measures Administered to Both Samples

Demographic and clinical characteristics

Demographic variables (i.e., age, race/ethnicity, marital status and education level) were collected via self-report. Clinical characteristics (i.e., disease type, time since diagnosis) were collected by medical record review at study entry.

Fatigue

The 7-item PROMIS Cancer Fatigue Short Form assesses the frequency of fatigue in the past seven days (7). Table 2 presents a list of the items. Items are measured on a five point scale (1 = never; 5 = always) and summed, after reverse scoring item 7, with higher scores indicating greater fatigue. Raw total scores served as the primary outcome for the purposes of this study. Normalized T-scores were also computed based upon a sample representative of the general U.S. population. The Fatigue Symptom Inventory (FSI) is a 14-item scale that assesses the frequency, severity, and disruptiveness of fatigue (9). Analyses in the current study focused on average fatigue severity and disruptiveness. Average fatigue severity was measured using an item that asks participants to rate their average fatigue in the past week on an 11-point scale, with higher scores indicating greater fatigue (0 = not at all fatigued; 10 = as fatigued as I could be). Disruptiveness of fatigue was assessed using seven items rated on an 11-point scale (0 = no interference; 10 = extreme interference). Responses to these seven items are summed yielding a total interference score. Previous research has demonstrated validity of the FSI with individuals diagnosed with cancer (1). The fatigue disruptiveness index demonstrated adequate internal consistency reliability (Cronbach’s alphas > 0.92).

Table 2.

PROMIS Cancer Fatigue Short Form item factor loadings

Item number	Item	Loading Sample 1 (PC)	Loading Sample 2 (HCT)
1	How often did you feel tired?	.782	.833
2	How often did you experience extreme exhaustion?	.781	.824
3	How often did you run out of energy?	.850	.863
4	How often did your fatigue limit you at work (include work at home)?	.748	.873
5	How often were you too tired to think clearly?	.748	.679
6	How often were you too tired to take a bath or shower?	.687	.434
7	How often did you have enough energy to exercise strenuously?	.354	.391

PC = prostate cancer (N=121); HCT = hematopoietic cell transplantation (N=136)

Vitality

The vitality subscale of the Medical Outcomes Study Short Form (SF-36) is composed of 4 items that assess energy/fatigue (10). Respondents use a six-point scale (1 = all of the time; 6 = none of the time) to rate how frequently they have experienced each feeling in the past week. Items are scored and averaged, with higher scores indicating greater vitality (i.e., less fatigue). This measure has demonstrated validity among cancer patients (11). The vitality subscale demonstrated adequate internal consistency reliability (Cronbach’s alphas > 0.93).

Depressive Symptomatology

The Center for Epidemiological Studies – Depression Scale (CES-D) is a 20-item measure of depressive symptomatology (12). Respondents use a four-point scale (0 = rarely or none of the time; 3 = most or all of the time) to rate how frequently they have experienced each depressive symptom in the past week. Items are summed to produce a total score ranging from 0 to 60, with higher scores indicating more depressive symptoms. The validity of the CES-D has been demonstrated with a wide range of populations, including cancer patients (13, 14). The CES-D demonstrated adequate internal consistency reliability (Cronbach’s alphas > 0.90).

Additional Measures Administered to Sample 1

Insomnia

The Insomnia Severity Index (ISI) is a 7-item measure of insomnia severity keyed to the past week (15). A five-point Likert scale is used to rate each item (0 = no problem; 4 = very severe problem). Items are summed to produce a total score, with higher scores indicating worse insomnia. The validity of the ISI has been demonstrated among individuals diagnosed with cancer (16). The ISI demonstrated adequate internal consistency reliability (Cronbach’s alpha = 0.91).

Additional Measures Administered to Sample 2

Case Definition of Cancer-Related Fatigue

The Diagnostic Interview Guide for Cancer-Related Fatigue (DIG-CRF) is a semi-structured interview used to determine if patients meet criteria for a case definition of cancer - related fatigue (17, 18). To meet the case definition, individuals must: report a 2-week period of significant fatigue and/or lack of energy in the preceding month; report at least five of ten symptoms consistent with cancer -related fatigue (e.g., feel weak or heavy all over); report significant distress or functional impairment from these symptoms; be considered to have fatigue symptoms as a consequence of cancer or cancer treatment; and not be considered to have fatigue symptoms due primarily to a co-morbid psychiatric disorder. The validity and utility of this case- definition approach has been demonstrated with cancer patients (19).

Anxiety

The state version of the State-Trait Anxiety Inventory (STAI) consists of a 20-item scale that assesses current anxiety (20). Participants rate symptoms of anxiety on a four-point Likert scale (1=not at all, 4=very much so). Item scores are summed to produce a total score ranging from 20 to 80, with higher scores indicating more anxiety. This measure has demonstrated excellent psychometric properties (20). The STAI demonstrated adequate internal consistency reliability (Cronbach’s alpha = 0.94).

Perceived Stress

The Perceived Stress Scale (PSS) is a measure that assesses participants’ perceptions of life as stressful over the past month (21). The PSS consists of ten items rated on a five-point Likert scale (0=never, 4=very often) that asks participants to think about the past month. Total scores range from 0 to 40 with higher scores indicating greater perceived stress. The PSS has demonstrated high reliability in cancer patients (22). The PSS demonstrated adequate internal consistency reliability (Cronbach’s alpha = 0.88).

Statistical Analyses

To determine if the PROMIS Cancer Fatigue Short Form was unidimensional and to evaluate the generalizability of the factor structure across groups, confirmatory factor analysis (CFA) was performed simultaneously in both samples using Mplus 6 (23). A single factor was specified. The measurement model produced from Sample 1 was used as the basic model without any constraints on parameters across the two samples. Metric equivalence was examined by constraining the estimated factor loadings to be equal across the two samples and model fit compared to the unconstrained model. Model fit was evaluated using a chi-square test, the root mean square error of approximation (RMSEA), and the comparative fit index (CFI). In order to assess the internal consistency of the PROMIS Cancer Fatigue Short Form for the samples combined and separately Cronbach’s alpha was computed.

Concurrent validity was examined via correlations between the PROMIS Cancer Fatigue Short Form summed total and FSI average fatigue, FSI fatigue disruptiveness, and SF-12 Vitality scores. Convergent validity was examined by computing Pearson’s correlations between the PROMIS Fatigue total score and CES-D, and ISI scores (Sample 1), and between the PROMIS Cancer Fatigue Short Form total score and CES-D, STAI and PSS scores (Sample 2). Additionally, confidence intervals were computed for correlations using Fisher’s r to z transformation to compare the magnitude of correlations between the PROMIS Cancer Fatigue Short Form and other measures of fatigue, and correlations between the PROMIS Cancer Fatigue Short Form and other psychosocial measures. Criterion validity was examined using an independent t- test to compare PROMIS Cancer Fatigue Short Form scores between patients did versus did not meet the criteria for fatigue caseness.

Results

Participant characteristics

Sample 1 was comprised of 48 men with prostate cancer beginning ADT for prostate cancer and 73 men previously treated with surgery for prostate cancer (mean age = 67 years, range 49–92). Sample 2 was comprised of 136 patients scheduled to undergo HCT (mean age = 51 years, range 20–75). As shown in Table 1, both samples were predominantly Caucasian, non-Hispanic, married, and college-educated. For Sample 1 and Sample 2, mean T-scores for the PROMIS Cancer Fatigue Short Form were 48 and 51, respectively. For Sample 1 and Sample 2, mean raw total scores for the PROMIS Cancer Fatigue Short Form were 14.59 (SD=5.05) and 16.55 (SD=4.65), respectively.

Table 1.

Participant characteristics

	Sample 1 (PC) (N = 121)	Sample 2 (HCT) (N = 136)
Characteristic
Age, years
Mean	66.64	51.37
SD	7.95	13.14
Gender (n, %)
Male	121 (100)	88 (61)
Female	0 (0)	56 (39)
Race (n, %)
White	108 (90)	129 (96)
Non-White	12 (10)	6 (4)
Ethnicity (n, %)
Hispanic	5 (4)	19 (14)
Non-Hispanic	114 (96)	116 (86)
Marital status (n, %)
Married	88 (73)	90 (66)
Not married	33 (27)	46 (34)
Years of education (n, %)
12 or less	27 (22)	32 (24)
13 to 16	74 (61)	86 (63)
17 or more	20 (17)	18 (13)

PC = prostate cancer; HCT = hematopoietic cell transplantation

Scale composition and factor structure

Table 2 provides the standardized factor loadings of the PROMIS Cancer Fatigue Short Form calculated using the unconstrained model. In Sample 1, factor loadings ranged from .354 to .893. In Sample 2, the item loadings ranged from .391 to .873. Fit indices of the unconstrained model indicated acceptable fit [χ²(34)=85.04, p<.001; CFI=.948; RMSEA=.104]. Fit indices of the constrained model also indicated acceptable fit [χ²(40)=95.08, p<.001; CFI=.944; RMSEA=.101] and these additional constraints did not statistically degrade model fit [Δχ²(8)=10.04, p=.12], suggesting model equivalence. Modification indices indicated that residual variances of items 5 and 6 were related but only in Sample 2; thus, no modifications to the CFA model were implemented. The internal consistency assessed yielded Cronbach’s alphas of 0.866 in Sample 1, 0.864 in Sample 2, and 0.876 for the two samples combined.

Validity

Sample 1

Table 3 shows correlations by group between PROMIS Cancer Fatigue Short Form scores and scores on the FSI, SF-36 Vitality Scale, CES-D, and ISI. Scores on the PROMIS Cancer Fatigue Short Form were significantly positively associated with both measures of fatigue (p values <.0001). Scores on the PROMIS Cancer Fatigue Short Form were also significantly correlated with measures of vitality, depression, and insomnia in the expected directions (p values <.001). Specifically, greater fatigue as measured by the PROMIS Cancer Fatigue Short Form was related to lower vitality and greater depression and insomnia. Confidence intervals created using Fisher’s r to z transformation demonstrated that correlations between the PROMIS Cancer Fatigue Short Form and other measures of fatigue were significantly greater in magnitude than correlations between the PROMIS Cancer Fatigue Short Form and measures of depression and insomnia (p < .05).

Table 3.

Correlations of the PROMIS Cancer Fatigue Short Form with measures of fatigue, depression, insomnia, anxiety, and perceived stress.

Variable	Sample 1 (PC) Correlation [95% CI]	Sample 2 (HCT) Correlation [95% CI]
Fatigue severity (FSI)	.72 [.62, .79]	.78 [.71, .84]
Fatigue disruptiveness (FSI)	.78 [.69, .84]	.79 [.72, .85]
Vitality (SF-36)	−.66 [−.75, −.54]	−.77 [−.83, −.68]
Depression (CES-D)	.51 [.36, .63]	.56 [.43, .66]
Insomnia (ISI)	.42 [.26, .56]	--
Anxiety (STAI)	--	.45 [.29, .57]
Perceived stress (PSS)	--	.44 [.30, .57]

All p’s < .0001

Note: PC = prostate cancer; HCT = hematopoietic cell transplantation; FSI = Fatigue Symptom Inventory; SF-36 = Medical Outcomes Study Short Form; CES-D = Center for Epidemiological Studies Depression Scale; ISI = Insomnia Severity Index; STAI = State Trait Anxiety Inventory; PSS = Perceived Stress Scale

Sample 2

Table 3 displays correlations between the PROMIS Cancer Fatigue Short Form scores and average fatigue, fatigue disruptiveness, vitality, depression, anxiety, and perceived stress. Scores on the PROMIS Cancer Fatigue Short Form were significantly positively associated with fatigue severity, and fatigue disruptiveness (p values <.0001). In addition, PROMIS Cancer Fatigue Short Form scores were significantly higher in patients meeting criteria for fatigue caseness t (147) = −5.55 (p < .0001). Consistent with predictions, lower scores on the PROMIS Cancer Fatigue Short Form were significantly associated with higher vitality scores (p <.0001). PROMIS Cancer Fatigue Short Form scores were also significantly positively related to depression, anxiety, and perceived stress (p values <.0001). Confidence intervals demonstrated that correlations between the PROMIS Cancer Fatigue Short Form and other measures of fatigue were significantly greater in magnitude than correlations between the PROMIS Cancer Fatigue Short Form and measures of depression, anxiety and perceived stress (p < .05).

Discussion

The goal of this study was to examine the psychometric properties of the 7-item PROMIS Cancer Fatigue Short Form among cancer patients. Results provide additional evidence for the reliability and validity of the PROMIS Cancer Fatigue Short Form. The internal consistency of the PROMIS Cancer Fatigue Short Form total score was adequate in both samples. As expected, CFA demonstrated that in both samples items the PROMIS Cancer Fatigue Short Form loaded on a single factor. Mean PROMIS Cancer Fatigue Short Form T-scores for both samples clustered around the average for the general population. However, these scores should be interpreted with caution given the differences in demographic characteristics (e.g., age and gender) between the current samples and the general population.

As hypothesized, scores on the PROMIS Cancer Fatigue Short Form were correlated with other well-validated measures of fatigue, demonstrating concurrent validity. Furthermore, criterion validity was demonstrated by showing that PROMIS Cancer Fatigue Short Form scores were significantly higher in patients meeting a case definition of fatigue than those who did not in Sample 2. Convergent validity was assessed by examining correlations between the PROMIS Cancer Fatigue Short Form and other measures expected to be associated with fatigue, including depression, anxiety, perceived stress, and insomnia. Results show that while these measures were significantly associated with PROMIS Cancer Fatigue Short Form scores, the strength of the association was significantly less than that of the PROMIS Cancer Fatigue Short Form with other measures of fatigue (i.e., FSI, SF-36 Vitality).

Study limitations should be noted. First, the study employed a sample of mostly non-Hispanic, white, well- educated patients with cancer. Due to the homogeneity of the sample, these results may not generalize to other more diverse populations of cancer patients. Furthermore, it is unknown whether the results generalize to cancer patients who do not have prostate cancer or hematologic malignancies. Future research should examine the psychometric properties of this instrument among more diverse samples of cancer patients and among patients with cancers (such as lung, colorectal and breast cancer) where fatigue is commonly experienced. Second, this study was cross- sectional. Longitudinal studies are needed to determine test-retest reliability and sensitivity to change of the PROMIS Cancer Fatigue Short Form. Lastly, the CFA performed in this study only examined a unidimensional factor structure, because there no a priori hypotheses regarding which items would load on additional factors. Consequently, it was not possible to compare fit across different factor structures.

In conclusion, the 7-item PROMIS Cancer Fatigue Short Form offers a number of advantages over existing measures of fatigue. Instead of being based on classical test theory, it was developed using IRT, a more modern approach to test development that allows for assessment across a full spectrum of fatigue using a minimal number of items to reduce participant burden. Furthermore, the PROMIS Cancer Fatigue Short Form is part of a larger measurement system that assesses not only fatigue, but a number of other important patient-reported outcomes across clinical and non-clinical samples. These measures are standardized, easy to administer, and adaptable (7). Our study demonstrates the reliability and validity of the PROMIS Cancer Fatigue Short Form for use with cancer patients. These results are similar to the results of studies evaluating the measure among other populations (24, 25). Thus, it is expected that the PROMIS Cancer Fatigue Short Form will be commonly used in future research.

Highlights.

• The PROMIS Cancer Fatigue Short Form items loaded on a single factor.

• The scale demonstrated good internal consistency reliability in two samples of patients with cancer.

• Significant relationships with psychosocial measures offer evidence for convergent and concurrent validity.

• PROMIS Cancer Fatigue Short Form scores were higher in patients who met case definition criteria for cancer-related fatigue.

• The PROMIS Cancer Fatigue Short Form is a reliable and valid measure of fatigue in cancer patients.