Cancer and Treatment Distress (CTXD) Psychometric Evaluation over Time: a BMT CTN 0902 Secondary Analysis

Abstract

BACKGROUND

Routine monitoring of cancer-related distress is recognized as essential to quality care and mandated by a major accrediting organization. However, few cancer-specific measures have been developed to assess the multiple cancer-related factors contributing to this distress. We examined the psychometric properties of the Cancer and Treatment Distress (CTXD) measure over time in hematopoietic cell transplantation (HCT) recipients.

METHODS

As a secondary analysis of a multicenter randomized controlled clinical trial, adult patients undergoing autologous or allogeneic HCT completed patient-reported outcomes including the CTXD and the Short-Form 36 (SF-36) at pretransplant and 100 and 180 days after HCT.

RESULTS

Across 21 transplant centers, 701 patients consented, received transplants and were included in analyses, 645 were alive at 100 days, and 618 were alive at 180 days. Internal consistency reliability was strong for the overall CTXD at the three time points: Cronbach's alphas were 0.94, 0.95, and 0.95, respectively. Subscale reliability met hypothesized levels of alpha >0.70 across time, with lowest reliability for the Identity subscale at 180 days, alpha =0.77. Correlations with the SF-36 Mental Health were higher than with Physical Functioning at each time point, supporting convergent and discriminant validity. Strong correlations of the pretransplant CTXD with post-transplant CTXD and SF-36 Mental Health supported predictive validity.

CONCLUSIONS

The CTXD is reliable and valid as a measure of cancer distress both before and after HCT. It may be a useful tool for measuring dimensions of distress and for defining patients needing distress treatment during and after transplant.

Graphical Abstract

Precis: Although emotional distress is widely recognized as contributing to the suffering of cancer patients, few measures of the sources and intensity of emotional distress specific to cancer and its treatment have been evaluated in patients over time. In this psychometric testing with hematopoietic cell transplantation patients at pretransplant, 100 days and 180 days, the Cancer and Treatment Distress measure was found to be reliable and valid, and thus could be useful for tracking multiple dimensions of distress, determining need for psychosocial treatment, and evaluating response to treatment.

Background

Emotional distress related to the diagnosis and treatment of cancer impacts quality of life for many patients with cancer and is now widely accepted as a symptom that requires routine assessment.¹ The National Comprehensive Cancer Network has provided distress screening guidelines to assist clinicians since 1999.^2,3 More recently, the American College of Surgeons Commission on Cancer has mandated psychosocial distress screening for accredited cancer programs.⁴ Uncertainty and fear of recurrence are widely recognized as major sources of stress during cancer treatment, along with other demands such as physical changes, financial strain, and managing medical needs. However, these domains are not included in most measures of distress or in standardized health-related quality of life measures. While overall distress level is important to evaluate, it is also important to identify factors contributing to distress in order to target appropriate interventions. For example, a patient with distress due to family or financial concerns may benefit from couples-based or financial counseling, whereas a patient struggling mainly with uncertainty, fear of recurrence, or depression may benefit from individual therapy including medication evaluation.

Despite recognition that distress is important to measure, few tools have been developed to assess the multiple sources of cancer-related distress and even fewer have had their psychometric properties over time evaluated. Measures such as the single-item Distress Thermometer have been used over the course of treatment with distress decreasing from pretreatment to the end of chemotherapy.^5,6 While the Distress Thermometer is an attractive measure because of its brevity, it may not be ideal when evaluating response to treatment or when determining the sources of distress in regard to needs for clinical care.

We developed the Cancer and Treatment Related Distress (CTXD) measure to assess not only an overall level of distress but also to evaluate cancer-related factors contributing to distress.⁷ These factors, which are subscales of the CTXD, include uncertainty, family strain, health burden, finances, loss of Identity, and medical demands. Previous work has confirmed the reliability and validity of the CTXD in a multi-site cohort of hematopoietic cell transplantation (HCT) recipients at pretransplant.⁷ However, the psychometric properties of the CTXD post-transplant have not been examined.

HCT patients undergo demanding treatment with higher risk for acute toxicities and late effects than cancer patients receiving many other treatments.^8-11 Their related distress has not been well characterized but may differ from other cancers given the intensity of treatment and uncertainty of long-term complications. In research comparing HCT survivors with survivors in the Childhood Cancer Survivor Study, the HCT survivors demonstrate elevated risks of severe/life-threatening conditions and functional impairments.⁸ They also have a 9-fold increased risk of late mortality,^12-14 and are at risk for poorer psychosocial health, with more depression and post-traumatic stress symptoms than age-matched norms.^15-17 Distress in HCT survivors continues over a lengthy period extending well beyond their physical recovery.¹⁵ To facilitate addressing the psychological needs of HCT survivors, there is value in establishing the psychometric reliability, validity, and stability of the CTXD in post-treatment survivors of HCT.

This secondary analysis had three aims: (1) to replicate previous findings of acceptable internal consistency of the CTXD pretransplant, (2) to determine whether the CTXD exhibits acceptable internal consistency post-transplant at 100 and 180 days, and (3) to assess the convergent, discriminant, and predictive validity of the CTXD using a standardized measure of physical and mental health. For acceptable reliability, we required the internal consistency of the CTXD as measured by Cronbach's alpha be greater than 0.80 for the total mean score and greater than 0.70 for the subscales at each time point. To establish acceptable construct validity, we required that at each time point the CTXD total score correlate greater than 0.50 with the Short Form (SF)-36 Mental Health subscale for convergent validity and correlate less than 0.50 with the SF-36 Physical Functioning subscale for discriminant validity. For predictive validity, we required pretransplant CTXD total score to have large magnitude correlations (r >0.50) with post-transplant CTXD scores and at least moderate negative correlation (r <−0.30) with post-transplant Mental Health scores.

Materials and Methods

Participants

Patients were recruited for a multicenter randomized controlled trial (RCT) examining the effects of a stress management and exercise program on quality of life in patients undergoing autologous or allogeneic HCT (the Blood and Marrow Clinical Trials Network 0902 study; NCT01278927). The RCT did not demonstrate differences between the intervention and control groups.¹⁸ Participants were recruited from 21 centers in the United States. Eligible patients were: 1) 18 years or older, 2) able to speak and read English, 3) able to exercise at low to moderate intensity as determined by physician and self-reported judgment, and 4) preparing to undergo autologous or allogeneic transplant within six weeks. Patients were excluded if they: 1) had problems which prevented safe ambulation, 2) were participating in another clinical trial with quality of life or functional status as a primary endpoint, 3) were planning to receive anti-cytotoxic therapies other than tyrosine kinase inhibitors, 4) were planning to receive donor lymphocyte infusion within 100 days of transplant, 5) were planning to receive tandem transplant (i.e., a planned autologous/autologous or autologous/allogeneic HCT).

Procedure

All study procedures were approved by the protocol review committee set by the National Heart, Lung and Blood Institute in addition to each institution's Institutional Review Board. Eligibility was determined via chart review and in consultation with the attending physician. Eligible patients were recruited and informed consent was obtained from each participant prior to the day of graft infusion (i.e., day 0). Following informed consent, participants completed the baseline assessment prior to randomization.¹⁸ Participants completed follow-up assessments at 100 days and 180 days post-HCT.

Measures

Demographic and clinical data

Demographic data obtained prior to HCT included age, gender, race, ethnicity, marital status, education, and income. Clinical data were collected via BMT CTN reporting and included disease type, transplant type, pre-HCT Karnofsky score, and survival at 100 days and 180 days post-HCT.

Cancer-related distress

The Cancer and Treatment Distress (CTXD) measures cancer-related distress over the past week.⁷ Items were generated initially based on qualitative interviews with patients asking what aspects of their diagnosis and treatment were most distressing to them. Physicians, nurses, and other healthcare providers then reviewed the items and generated additional content. Additional interviews with patients and factor analyses were used to refine the item pool, resulting in a 22-item measure comprised of a total mean score and six subscales: Uncertainty (“not knowing what the future will bring”), Health Burden (“not being able to do what I used to do”), Family Strain (“wondering about the emotional toll on my family or other caregivers”), Identity (“changes in my appearance”), Finances (“cost of treatment”), and Medical Demands (“dealing with the medical system”). Response options are on a four-point scale from 0 = “none” to 3 = “severe”. Scores are the mean response across the scale/subscale items. Higher scores indicate greater distress.

Quality of life

The acute (one week) version of Medical Outcomes Study Short Form – 36 (SF-36) was used to assess health-related quality of life.¹⁹ The Physical Functioning (PF) and Mental Health (MH) subscales were used to assess convergent and divergent validity, respectively. Published scoring procedures were followed producing scores on these subscales range from 0 to 100 with higher scores reflecting better quality of life. The other six SF-36 subscales were deemed less relevant for validity testing. The often-used physical component and mental component summaries were not used because the summaries include a weighted contribution from each of the SF-36 subscales potentially confounding the two scales when used for validity testing.²⁰

Statistical Analyses

To examine change in internal consistency across time, Cronbach's alpha was calculated for the total mean CTXD score and the 6 subscales at each time point. We used Pearson correlations to examine convergent and divergent validity of the CTXD with the SF-36 MH and PF subscales, respectively, at each time point. Cohen's criterion was used to interpret the magnitude of correlation coefficients (r <0.3 = small; r >0.3 and <0.5 = moderate; and r ≥0.5 = large/strong).²¹ Analyses were completed with SAS v. 9.3 (SAS Institute, Cary, NC).

Results

Participants

In sociodemographic characteristics (Table 1), participants were majority male (57%), with an average age of 54.6 years (SD 12.8) but with a broad range from 18-75, and largely white (88%) and non-Hispanic/Latino (93%) consistent with the HCT population. The cohort was well-educated with 80% having more than a high school education, and mostly married or living with a partner (74%).

Table 1.

Demographic characteristics (N=701)

	N (%) or M (SD)
Women	302 (43.1)
Age (range of 18 to 75)	54.6 (12.8)
Race:
American Indian/Alaskan Native	1 (0.1)
Asian	11 (1.6)
Black/African-American	58 (8.3)
Hawaiian/Pacific Islander	3 (0.4)
White	614 (87.6)
More than one race	6 (0.9)
Unknown	8 (1.1)
Ethnicity:
Hispanic/Latino	33 (4.7)
Not Hispanic/Latino	650 (92.7)
Unknown	18 (2.6)
Education:
Grade school	2 (0.3)
Some high school or high school graduate	138 (19.7)
Some college or college graduate	419 (59.8)
Postgraduate	137 (19.5)
Unknown	5 (0.7)
Marital Status:
Married or living with partner	519 (74.0)
Single, Never married	83 (11.8)
Divorced or separated	72 (10.3)
Widowed	18 (2.6)
Unknown	9 (1.3)
Annual Family Income:
Under $-$24,999	90 (12.9)
$25,000-$49,999	131 (18.7)
$50,000-$74,999	148 (21.0)
$75,000-$99,999	99 (14.1)
$100,000 or above	191 (27.3)
Unknown	42 (6.0)

In clinical characteristics (Table 2), the cohort was about half autologous HCT recipients (50%) with the other half evenly split between myeloablative (25%) and non-myeloablative (25%) allogeneic transplants. Diagnoses were diverse, with multiple myeloma the most common (27%). Of the 701 participants, 645 were alive to complete the 100-day assessment, and 618 were alive to complete the 180-day assessment. All completed the CTXD at least once and are included in analyses, 2 did not complete the baseline CTXD (0.3%), and of the survivors 79 (12%) and 125 (20%) did not complete the CTXD at 100 days and 180 days respectively. Among those who were alive, baseline CTXD scores did not differ between respondents versus non-respondents at 100 days and 180 days or when tested for interactions of responder status with the sociodemographic and clinical variables listed in Tables 1 and 2.

Table 2.

Clinical Characteristics (N=701)

	N (%)
Type of Transplant Procedure:
Autologous/Syngeneic	354 (50.5)
Myeloablative Allogeneic	175 (25.0)
Reduced Intensity/Non-myeloablative Allogeneic	172 (24.5)
Primary Diagnosis:
Multiple Myeloma/ Plasma Cell Disorder	191 (27.3)
Non-Hodgkin Lymphoma	177 (25.3)
Acute Myelogenous Leukemia	124 (17.7)
Myelodysplastic/ Myeloproliferative Disorders	59 (8.4)
Hodgkin Lymphoma	48 (6.9)
Acute Lymphoblastic Leukemia	37 (5.3)
Other Leukemia	30 (4.2)
Chronic Myelogenous Leukemia	14 (2.0)
Other Disease	21 (2.9)
Survival:
100 days	645 (92.0)
180 days	618 (88.2)

Internal consistency

Table 3 presents means, standard deviations, and Cronbach's alphas for the entire scale and the 6 subscales at each time point. The full CTXD met requirements for internal consistency over time with alpha ≥0.94. Each of the 6 subscales exceeded the criterion level of >0.70 at each time point, considered a minimum acceptable level, and these internal consistency levels were also stable over time for each subscale with the largest range of 0.06 for the Family Strain subscale. Consistent with the high alpha for the total mean scale, subscale correlations ranged from 0.44 to 0.65 at baseline, 0.44 to 0.65 at 100 days, and 0.39 to 0.72 at 180 days (P's <.001).

Table 3.

Means, standard deviations, and Cronbach's alpha for the Cancer and Treatment Distress (CTXD) total mean score and subscales at each assessment time point

Scale/ Subscale	N items	Baseline		100 days		180 days
		M (SD)a	alpha	M (SD)	alpha	M (SD)	alpha
CTXD total	22	1.12 (0.60)	0.940	0.93 (0.60)	0.945	0.85 (0.59)	0.945
Health Burden	4	1.43 (0.73)	0.832	1.32 (0.74)	0.862	1.24 (0.76)	0.874
Family Strain	3	1.47 (0.77)	0.800	1.06 (0.80)	0.839	0.94 (0.79)	0.863
Uncertainty	4	1.24 (0.78)	0.867	1.02 (0.75)	0.878	0.95 (0.80)	0.904
Finances	3	1.00 (0.85)	0.789	0.84 (0.81)	0.782	0.75 (0.79)	0.811
Identity	4	0.80 (0.77)	0.816	0.69 (0.72)	0.813	0.60 (0.62)	0.772
Medical Demands	3	0.71 (0.66)	0.780	0.56 (0.66)	0.793	0.51 (0.63)	0.811

^aItems are measured on a 4-point scale with 0=none, 1=mild, 2=moderate, and 3=severe. Scores are the mean item response value across the items for each scale/subscale.

Construct validity

Table 4 presents descriptive statistics at each time point for mean the MH and PF subscales along with correlations assessing convergent, discriminant, and predictive validity. Convergent validity was demonstrated by strong concurrent correlations of the CTXD with the MH subscale at each assessment (r's >−0.66). Discriminant validity was demonstrated by lower concurrent correlations with the PF subscale (r's <−0.48) at each assessment. Predictive validity was demonstrated by the moderate negative correlations of the baseline CTXD measure with MH at 100 and 180 days (r's >−0.44). Patients with higher levels of distress pretransplant exhibited lower mental health post-transplant. As with concurrent correlations supporting convergent and divergent validity, the correlations between the pretransplant CTXD and post-transplant PF (r's =−0.22 and −0.26) were lower than for MH. It is noteworthy that these correlations with MH and PF were different even though MH and PF were significantly, positively correlated with each other at each assessment. Finally, predictive validity was also supported by strong correlations of the baseline CTXD with CTXD at 100 and 180 days.

Table 4.

Validity testing for the Cancer and Treatment Distress (CTXD) total mean score versus the SF-36 Physical Functioning (PF) and Mental Health (MH) scales at each time point ^a

Time point / Measure	N	M (SD)	Correlation with CTXD total b	Correlation with SF-36 PF b	Correlation with CTXD total at baseline c
Baseline
SF-36 PF	688	68.3 (23.3)	−.337***
SF-36 MH	688	74.6 (17.4)	−.660***	+.297***
100 days
CTXD	565	0.93 (0.60)			+.628***
SF-36 PF	555	66.6 (23.9)	−.423***		−.224***
SF-36 MH	552	78.0 (16.9)	−.653***	+.281***	−.463***
180 days
CTXD	491	0.85 (0.59)			+.617***
SF-36 PF	485	70.7 (23.8)	−.481***		−.260***
SF-36 MH	484	78.8 (16.7)	−.709***	+.372***	−.437***

* P <.05

** P <.01

^***P <.001

^aPF and MH higher score = better function, CTXD higher score = more distress

^bCorrelation with concurrent measures, addressing convergent (SF-36 MH) and divergent (SF-36 PF) validity

^cCorrelation of baseline CTXD with CTXD at 100 and 180 days (partly addressing test-retest consistency) and with PF and MH at 100 and 180 days (addressing predictive validity)

Finally, we evaluated construct validity of the 6 subscales. The pattern of correlations for each subscale with MH and PF paralleled those of the CTXD total mean score. Within time points all CTXD subscales correlate with MH between −0.43 and −0.63. The highest correlations with MH consistently are with Health Burden and Uncertainty, the lowest are with Finances. Within time points, the PF correlations with CTXD subscales were generally lower, ranging from −0.20 to −0.35. The exceptions were for Health Burden ranging from −0.44 to −0.59 and Family Strain with r=−0.43 at 100 days and r=−0.48 at 180 days. Predictive validity of the subscales was generally supported with r's > .50 between baseline and follow-ups at 100 and 180 days. The two minor exceptions were for Medical Demands (r >0.45) and Health Burden from baseline to 100 days (r =0.48)].

Discussion

In this large cohort of HCT patients followed from before to six months after their transplants, the CTXD demonstrated strong internal consistency reliability, as well as strong convergent, divergent and predictive validity. Psychometric properties were stable over time not only for the total mean score but also within the subscales. Distress on the CTXD total mean score declined on average nearly a half a standard deviation by 180 days. Based on established definitions of clinically meaningful change of a half standard deviation,^22,23 this suggests that CTXD distress declines slowly but meaningfully over the half year after HCT. Among the subscales, patterns of change from pretransplant to 180 days were descriptively more variable, with the least change seen in Health Burden over time and with the sharpest drop in Family Strain, declining more than a half standard deviation from pretransplant to 180 days.

Results support hypothesized levels of reliability and indicate that the CTXD has a consistent structure over time. The six subscales remained reliable, ranging from Cronbach's alphas of 0.90 to 0.77, with the reliabilities and standard deviations for each subscale remaining within a small range from pretransplant to 180 days even as the means declined. These reliabilities are in the range from acceptable to excellent without adding unnecessary redundancy to the measure.²⁴ This is noteworthy given that each subscale has only 3 or 4 items, and the most frequent response to items was ‘not at all’. Thus we believe the CTXD reduces patient response burden to as low a level as possible while preserving the ability to reliably assess multiple dimensions of the distress experience in HCT patients over time.

The reported correlations support the construct validity and to some extent the test-retest reliability of the CTXD. Ideally, test-retest reliability would be confirmed with assessments closer together in time and without the major events that occur during the six months of the HCT process. We tested predictive validity with the measures available from the parent study, so were restricted to the SF-36. Of note, the MH and PF results were consistent with the physical component and mental component summaries of the SF-36 (data not shown), even though for validity testing we considered the cleaner separation of the subscales using the MH and PF to be advantageous.

Numerous aspects of the CTXD remain to be examined to determine the value of measuring cancer-related distress as distinct from standardized global measures of mental health or the widely used single-item Distress Thermometer (DT). The DT has been examined in HCT populations in a few longitudinal studies.^25-27 Two studies that have considered factors contributing to HCT-related distress by examining the problem checklist categories that accompany the distress thermometer found results similar to ours except for family stress in that physical difficulties were most frequently checked as contributing to distress, followed by emotional concerns, practical concerns such as finances and work, then family issues, with spiritual concerns receiving the lowest endorsement rates.^26,27 We found high means for family strain, but otherwise similar ordering of the means for distress within our factors. In development of the CTXD we did not retain items measuring spiritual distress because we found low endorsement of spiritual distress in HCT recipients and those items did not contribute unique variance or reliability to the measure.⁷ Additionally, research is needed to compare the sensitivity and specificity of the CTXD with standardized general population measures to assure added value of the distress measure.

While it is important that the CTXD predicts later values on similar measures and the CTXD itself, predictive value of the CTXD will be enhanced if it also predicts later health outcomes of HCT such as psychosocial needs during and after HCT, health care needs such as hospitalization, disability, or costs of care. Now that the reliability and validity of the CTXD, including subscales, is established, there is opportunity to address these questions with prospective longitudinal studies.

In previous research we established a cut point for clinically meaningfully elevated distress on the CTXD total mean score of >1.1.⁷ Using that criterion with these data, 47% of the pretransplant participants had elevated distress, 35% at 100 days and 32% at 180 days (data not shown). The value of the CTXD as a clinical tool remains to be demonstrated. Results do align with a meta-analysis of interview-based studies identifying the prevalence of depression, anxiety and adjustment disorder, which found that overall 30-40% of cancer patients had one or more psychiatric mood disorders.²⁸ Given the intensity of HCT, we would expect rates to be at least that high. Future research will examine whether the CTXD can be useful for identifying patients needing intervention for their distress.

Other topics remain to be investigated with the CTXD including whether it has value for assessing distress in patients with other cancer diagnoses or treatments. Although numerous items were tested that could be specific to HCT, such as distress related to chronic graft versus host disease, none of those items were retained in the final CTXD factor analyses during development.⁷ Thus the items in the CTXD are not specific to HCT, and the measure could be relevant to other cancer diagnostic groups or treatments. Psychometric testing is needed to establish its value with these other populations. Furthermore, whether the measure provides useful identification of those at higher risk of poor outcomes during or after HCT remains to be tested, as does its sensitivity to detecting treatment differences. The CTXD has been translated into German with similar results,²⁹ but studies in other languages and with greater cultural diversity are needed.

Several strengths and limitations need to be noted. This is the first published longitudinal analysis of the psychometric properties of the CTXD to our knowledge. The sample is a large cohort from 21 transplant centers, suggesting that it may be representative of the broader HCT population. Limitations include the possibility of biases in the enrollment because the parent protocol was an RCT that may have been of more interest to some HCT recipients than others. The participants are largely well educated, and mostly middle to higher income. Normative data on the CTXD over time in a cohort of HCT patients not enrolled on a clinical trial would be valuable. The measures collected during this clinical trial do not provide a ‘gold standard’ measure of cancer-related distress from which to evaluate sensitivity, specificity and clinically meaningful cut points. In addition, since the study only followed patients for six months, the psychometric stability of the CTXD as a multidimensional measure of distress for long-term survivors is unknown.

In conclusion, the CTXD as tested in this secondary analysis from an RCT dataset demonstrates strong reliability and validity. With further testing relative to established measures such as clinical diagnostic interviews, the CTXD with its subscales could be useful for screening distress and identifying the types of intervention patients need during and after HCT. This evidence supports the value of the CTXD as a research or clinical tool for assessing not only overall distress but also the domains contributing to distress in HCT patients over the first six months after treatment.