Abstract
Objective
To examine the validation and reliability of the distress thermometer (DT) recommended by National Comprehensive Cancer Network (NCCN) in Chinese cancer patients.
Methods
A total of 574 Chinese cancer patients from Beijing Cancer Hospital completed the detection of DT, the Hospital Anxiety and Depression Scale (HADS) and Symptom Checklist 90 (SCL-90), Receiver Operating Characteristic (ROC) curve and Area Under the Curve (AUC) were used to analyze the validation relative to HADS and SCL-90. The patients with DT≥4 and whose distress caused by emotional problems were interviewed with the MiNi International Neuro-psychiatric Interview (MINI) (Chinese Version 5.0). This version was used to analyze cancer patients’ psychological and Psychiatric symptoms during the cancer process; 3. Another 106 cancer patients in rehabilitation stage and stable condition were asked to fill in DT two times, at the base time and after 7-10 days.
Results
Data of ROC indicates that a DT cutoff score of 4 yielded AUC of 0.80 with a optimal sensitivity (0.80) and specificity (0.70) relative to HADS, and AUC of 0.83 with the greatest sensitivity (0.87) and specificity (0.72) against SCL-90. The DT also has acceptable test-retest reliability (r=0.800, P=0.000); According to the interview results, the most common psychiatric problems cancer patients have adjustment disorder, depression, and anxiety.
Conclusion
The data suggest that DT has acceptable overall accuracy and reliability as a screening tool for testing distress severity and specific problems causing distress in Chinese cancer patients. It is worth being used in oncology clinic, the rapid screening and interview could help caregivers to identify psychological and psychiatric problems of cancer patients and provide useful information for further treatment.
Key words: Validation, Reliability, Cancer patients, Distress thermometer
INTRODUCTION
Psychosocial problems have been suggested by a voluminous literature currently, medical care has been improved gradually and the physical symptoms of patients are well controlled, but the psychological problems and symptoms are often overlooked, even though psychological and social factors affect the incidence, development and outcome of cancer.
Psychosocial distress is a broad concept defined by National Comprehensive Cancer Network (NCCN). It is an unpleasant emotional experience of a psychological, social and/or spiritual nature which extends on a continuum from normal feelings of vulnerability, sadness and fears to disabling problems such as depression, anxiety, panic, social isolation and spiritual crisis[1]. The term “distress” is chosen because it is more acceptable and less stigmatizing than “psychiatric”, “psychological” “emotional”. “Distress” caused by life-threatening diseases like cancer. It could be measured by self report.
Despite many reports in medical literatures about psychological distress[2-4], it is often unrecognized and untreated by health care professionals[5, 6]. Clinic oncology could benefit from identifying cancer patients’ distress as it could relive patients’ negative emotion, improve their quality of life and adherence to treatment recommendations, and reduce treatment cost[7]. Because of this, Distress Management Guideline by panel of the NCCN recommends screening all cancer patients regularly for psychological distress as a part of routine care.
A number of well-validated measures exist that can be used as distress screeners, including the Hospital Anxiety and Depression Scale (HADS), Symptom Checklist 90 (SCL-90), and some psychiatric interview tools, but these measures require time commitments and explanation by medical staff that can limit their use in busy oncology clinics. Some research suggest Distress Thermometer (DT) recommended by NCCN is a single-item scale accompanied by a problem list, which can be used as a routine screening tool because of its simplicity and ability to test the distress level and identify causes concurrently. It has been translated into Chinese and the preliminary clinical application in cancer patients has showed it is a rapid and effective screening measure for distress[8]. In this study, we examined the reliability and validity of DT and provides theoretical support for its extensive application of DT in cancer patients in China.
MATERIALS AND METHODS
Participants for Validation and Psychiatric Interview
Participants were cancer patients recruited from Beijing Cancer Hospital in Beijing, China between March 2009 and January of 2010. Who: (1) aged 18 years old or above; (2) could read and understand the questionnaires; (3) were prior informed consent; (4) were diverse in terms of cancer sites and stages.
Participants for Reliability
Study participants were also recruited from Beijing Cancer Hospital in Beijing, China. Who: (1) aged 18 years old or above with KPS ≥60; (2) could read and understand the questionnaires; recovery or improvement by assessment of clinical efficacy after clinical treatment; (3) were prior informed consent; (4) were diverse in terms of cancer sites and stages.
The study was approved by the Ethical Committee for Clinical Research of Beijing Cancer Hospital.
Research Tools
Distress Thermometer (DT)
DT consists a single-item self report measure of psychological distress, which consists of an 11-point scale with the endpoints labeled “No distress”(0) and “Extreme distress” (10), participants were instructed to circle the number that describes their level of distress in the past seven days. It developed by Roth at the first time and used to screen the distress of prostate cancer patients’ distress[9]. NCCN added a Problem List which contains 34 items divided into 5 groups (practical problems, family problems, emotional problems, physical problems, spiritual/religious). Problem List in the Chinese version is adjusted to 40 items (5 groups). The validation of NCCN Problem List has been proved in many countries and a cutoff score 4 is the most sensitive and specific[10-13], which is also recommended by NCCN distress management guidelines. That means cancer patients whose score is equal or surpass 4 should better receive evaluation and treatment by professional psychologists and psychiatrists[14].
Hospital Anxiety Depression Scale (HADS)
HADS developed by Zigmond AS and Snaith RP in 1983[15] consists of two parts, 14 items, including 7 items for anxiety sub-scale, 7 items for depression sub-scale. This scale is widely used in screening for anxiety and depression in general hospital patients, and screening study in psychosomatic disease[16, 17]. The reliability and validity of Chinese version has been proved[18]. In this study, cutoff score 9 is used to evaluate cancer patients’ anxiety and depression.
Symptom Checklist 90 (SCL-90)
SCL-90 developed by Derogaitis LR in 1975[19], contains 90 items and is widely used in China. The scale covers the feelings, emotions, thinking, consciousness, behavior, habits, relationships, diet and sleep. Screening criteria are as follows: (1) Total score ≥160; or (2) The number of positive items (item scores ≥2) ≥43; or (3) Any one factor score ≥2[20].
Karnofsky Performance Status (KPS)
KPS developed by Karnofsky in 1948[21], reports patents’ physical functions with acceptable reliability and validation in China[22]. Scores range from 0-100.
MiNi-International Neuropsychiatric Interview (MINI)
MINI developed by David Sheehan and Yves Lecrubier in 1997[23] at the base of the 4th version of Diagnostic and Statistical Manual of Mental Disorders Fourth Edition (DSM-IV) and The International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10). Many studies have showed its reliability and validity[19-26].
Statistical Analysis
Patients are asked to fill in DT at the beginning and 7-10 days later. SPSS16.0 is used to analyze the correlation between the twice scores.
Patients of validation test were asked to fill in the questionnaires, including DT, HADS, SCL-90, KPS; Patients whose score ≥4 received psychiatry interview. Receiver operating characteristics (ROC) curves is calculated in SPSS16.0 to determine which cutoff score on the DT was most effective in distinguishing distressed patients from those who were not distressed. The DT’s area under the curve (AUC) represents the measure’s accuracy in screening distress. AUC value 0.5-0.7 means low accuracy while 0.7-0.9 moderate accuracy, 0.9-1.0 high accuracy.
RESULTS
Reliability of DT
A total of 110 patients met the criterion involved in the study and completed the initial test questionnaires, of which, 106 completed the re-test questionnaires and all were valid. The mean age was 57 years (SD=10); 57.5% of the participants were male; retest correlation coefficient was 0.80, (P<0.01).
Validation of DT
A total of 600 eligible patients participated in and completed questionnaire package for validation study (DT, HADS, SCL-90), of which, 574 were valid, questionnaires response rate was 95.7%. The sample averaged approximately 55 years of age (SD=14) and had slightly more female (53.8%) than male (46.2%) and the majority was married (89.5%). Other socio-demographic and clinical characteristics are shown in Table 1.
Table 1. Socio-demographic and clinical characteristics of the study participants.
| Item | No. of cases | Percentage (%) |
|---|---|---|
| Age (y) | ||
| 18-88 (Mean age 55±14) | ||
| Gender | ||
| Male | 265 | 46.2 |
| Female | 309 | 53.8 |
| Race | ||
| Han | 547 | 95.3 |
| Minorities | 27 | 4.7 |
| Religious belief | ||
| Yes | 52 | 9.1 |
| No | 522 | 90.9 |
| Marital Status | ||
| Unmarried(Single, Separated, Divorced, Widowed) |
60 | 4.9 |
| Married | 514 | 89.5 |
| Education | ||
| Middle School | 168 | 29.3 |
| High School | 272 | 47.4 |
| College | 134 | 23.4 |
| Hospital Charges Paid by | ||
| Public Health Service | 82 | 14.3 |
| Self-Paid | 122 | 21.3 |
| Medical Insurance | 370 | 64.5 |
| Family Economic Status | ||
| High | 8 | 1.4 |
| Ordinary | 446 | 77.7 |
| Low | 120 | 20.9 |
| Cancer Site | ||
| Lung Cancer | 99 | 17.2 |
| Breast Cancer | 75 | 13.1 |
| Colorectal Cancer | 88 | 15.3 |
| Hepatobiliary Cancer | 25 | 4.4 |
| Gynecological malignancies | 63 | 11.0 |
| Lymphoma | 88 | 15.3 |
| Gastric cancer | 65 | 11.3 |
| Esophagus cancer | 18 | 3.1 |
| Pancreas Cancer | 9 | 1.6 |
| Head and neck carcinoma | 13 | 2.3 |
| Malignant tumor of urinary system | 8 | 1.4 |
| Duodenum neoplasm | 5 | 0.9 |
| Others | 18 | 3.1 |
| Recurrence or metastasis | ||
| Yes | 179 | 31.2 |
| No | 395 | 68.8 |
ROC Analysis
ROC analysis were performed to confirm the efficiency of DT in cancer patients and identify the valid DT cutoff score. The greater the height of curve above the chance line (AUC=0.5), the more accurate the measure is. Area under the Curve (AUC) was 0.803 (against HADS), P<0.01; AUC 0.834 (against SCL-90), P<0.01 (Figure 1).
Figure 1.
ROC curves against HADS and SCL-90. A: DT against HADS; B: DT against SCL-90.
A cutoff of 4 maximized sensitivity and specificity, which were 0.803 and 0.699. A cutoff of 4 maximized sensitivity and specificity, which were 0.872 and 0.718 as shown in Table 2 and Table 3.
Table 2. Sensitivity and Specificity against HADS.
| Positive if greater than or equal to | Sensitivity | 1- Specificity |
|---|---|---|
| -1.00 | 1.000 | 1.000 |
| 0.50 | 0.985 | 0.814 |
| 1.50 | 0.944 | 0.654 |
| 2.50 | 0.859 | 0.457 |
| 3.50 | 0.803 | 0.301 |
| 4.50 | 0.712 | 0.213 |
| 5.50 | 0.497 | 0.122 |
| 6.50 | 0.333 | 0.066 |
| 7.50 | 0.247 | 0.037 |
| 8.50 | 0.136 | 0.027 |
| 9.50 | 0.091 | 0.019 |
| 11.00 | 0.000 | 0.000 |
Table 3. Sensitivity and Specificity against SCL-90.
| Positive if greater than or equal to | Sensitivity | 1- Specificity |
|---|---|---|
| -1.00 | 1.000 | 1.000 |
| 0.50 | 0.984 | 0.819 |
| 1.50 | 0.968 | 0.651 |
| 2.50 | 0.914 | 0.442 |
| 3.50 | 0.872 | 0.282 |
| 4.50 | 0.749 | 0.209 |
| 5.50 | 0.529 | 0.116 |
| 6.50 | 0.342 | 0.070 |
| 7.50 | 0.246 | 0.044 |
| 8.50 | 0.139 | 0.028 |
| 9.50 | 0.091 | 0.021 |
| 11.00 | 0.000 | 0.000 |
Psychiatric Interview Result
Totally 50 patients with DT ≥4 and whose distress was caused by ‘emotional problems’ was interviewed in this study. The common mental disorders in cancer patients are shown in Table 4.
Table 4. The common mental disorders in cancer patients.
| Assessment results | No. of cases | Percentage (%) |
|---|---|---|
| Adjustment disorder | 47 | 94 |
| Adjustment disorder with anxiety | 10 | 20 |
| Adjustment disorder with depression | 8 | 16 |
| Adjustment disorder with depression and anxiety | 29 | 58 |
| Anxiety disorder | 2 | 4 |
| Depression disorder | 1 | 2 |
DISCUSSION
Test-retest reliability, which also means coefficient of stability, reflects the cross-time stability of a scale. Generally, an acceptable scale requires test-retest correlation coefficient of 0.7 or above. In this study, the test-retest correlation coefficient was 0.80, which suggested that DT had acceptable reliability.
Criterion validity was used to test the effectiveness of DT. Many literatures had proven that DT could screen for the psychological distress in cancer patients and their family members. The effectiveness and the best cutoff point of DT in Chinese cancer patients was verified in this study through comparing DT with other standard scales. ROC curve analysis showed that area under the curve was 0.803 (HADS as a reference) and 0.834 (SCL-90 as a reference), which indicates that DT has a high degree diagnostic accuracy, can differentiates distressed patients from those who were not distressed, or distinguishes suspected patients from those under cutoff score of SCL-90. We got the same result when determine DT’s cutoff point with the reference of HADS and SCL-90. A cutoff point of 4 maximized sensitivity at 0.803 (against HADS) and 0.872 (against SCL-90) and specificity at 0.699 (against HADS) and 0.718 (against SCL-90), respectively.
DT can be only used as a screening tool, not a diagnostic tool in the clinical application. Accurate diagnosis for mental disorder in cancer patients needs further detailed assessment. Compared with other screening measures, DT has unique advantages in clinical distress screening as follows: (1) It is simple and easily understood as a single-item scale. Medical staff can guide patients to complete DT through a short explanation,which only takes 1 or 2 minutes. Therefore, distress screening will not increase the workload of medical staff and cancer patients in the busy oncology clinics; (2) Problem List covers most of the factors resulting in distress of cancer patients and provides clues for further treatment for psychiatrists and psychologists.
Adjustment disorder, anxiety and depression were the most common mental disorders according to the psychiatric interview in this study. Derogatis et al. found that nearly 50% of cancer patients meet the standard of DSM-III, most of which were adjustment disorder[27]. Mental problems assessment are very important for cancer patients, because anxiety and depression have significant negative impacts: patients have lower quality of life, seek more medical services, have more difficulty in making decisions, are less adherent to their treatment regimens, and are less satisfied with their medical care and have reduced effectiveness of chemotherapy, increased risk of suicide and stay in the hospital longer[28-30]. However, the psychological distress of cancer patients are often ignored or underestimated. Passik et al. has shown that only 13% of severe depressed patients were identified in time; other patients did not receive proper treatments[31]. DT could detect psychological distress of cancer patients in clinical applications; psychiatric interview could provide further assessment and diagnosis for their mental problems and disorders.
However, the limitations in this study should be noted. Due to time and resource constraints, some types of cancer patients are not involved, such as brain tumors and leukemia. So the sample collection for DT’s reliability and validity test should be improved and the efficiency of DT in those patients uncovered in this study needed further confirmation. Psychiatric interview process were interfered by many factors, such as that some participants who were of low performance status didn’t want to accept the interviews, so the results of interview are not enough to explain mental disorder prevalence in cancer patients because of the biased collection of cases; the result could only provide a reference for clinical treatment in psychiatry and psychology department.
REFERENCES
- 1.American Cancer Society & National Comprehensive Cancer Network. (2005). Distress Treatment guidelines for patients-Version II/July 2005. Retrieved October 3, 2005, Available from http://www.nccn.org/ patients_ gls/_ english/_distress/contents.aps
- 2.Maguire P.Improving the detection of psychiatric problems in cancer patients. Soc Sci Med 1985; 20:819-23 [DOI] [PubMed] [Google Scholar]
- 3.Zabora J, Brintzenhofeszoc K, Curbow B, et al. The prevalence of psychological distress by cancer site. Psychooncology 2001; 10:19-28 [DOI] [PubMed] [Google Scholar]
- 4.Stark D, Kiely M, Smith A, et al. Anxiety disorders in cancer patients: their nature, associations, and relation to quality of life. J Clin Oncol 2002; 20:3137-48 [DOI] [PubMed] [Google Scholar]
- 5.Söllner W, DeVries A, Steixner E, et al. How successful are oncologists in identifying patient distress, perceived social support, and need for psychosocial counselling? Br J Cancer 2001; 84:179-85 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.McDonald MV, Passik SD, Dugan W, et al. Nurses’ recognition of depression in their patients with cance. Oncol Nurs Forum 1999; 26:593-9 [PubMed] [Google Scholar]
- 7.Bultz BD, Carlson LE. Emotional distress: the sixth vital sign in cancer care. J Clin Oncol 2005; 23:6440-1 [DOI] [PubMed] [Google Scholar]
- 8.Zhang YN, Zhang HW, Tang LL. Application of the NCCN Distress Thermometer in Chinese cancer Patients. Chin Ment Health J (in Chinese)2010; 12:897-902 [Google Scholar]
- 9.Roth AJ, Kornblith AB, Batel-Copel L, et al. Rapid screening for psychological distress in men with prostate carcinoma: a pilot study. Cancer 1998; 82:1904-8 [DOI] [PubMed] [Google Scholar]
- 10.Jacobsen PB, Donovan KA, Trask PC, et al. Screening for psychological distress in ambulatory cancer patients. Cancer 2005; 103:1494-502 [DOI] [PubMed] [Google Scholar]
- 11.Shim EJ, Shi YW, Jeon HJ, et al. Distress and its correlates in Korean cancer patients: pilot use of the distress thermometer and the problem list. Psychooncology 2008. 17:548-55 [DOI] [PubMed] [Google Scholar]
- 12.Gessler S, Low J, Daniells E, et al. Screening for distress in cancer patients: is the distress thermometer a valid measure in the UK and does it measure change over time? A prospective validation study. Psychooncology 2008; 17:538-47 [DOI] [PubMed] [Google Scholar]
- 13.Recklitis CJ, Licht I, Ford J, et al. Screenning adult survivors of childhood cancer with the distress thermometer: a comparison with the SCL-90-R. Psychooncology 2007; 16:1046-9 [DOI] [PubMed] [Google Scholar]
- 14.National Comprehensive Cancer Network. NCCN Clinical Practice Guidelines in Oncology: Distress management, version 1. 2008. Available from: http://www.nccn.org/professionals/physician_gls/ PDF/distress.pdf [DOI] [PMC free article] [PubMed]
- 15.Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 198367:361-70 [DOI] [PubMed] [Google Scholar]
- 16.Barczak P, Kane N, Andrews S, et al. Patterns of psychiatric morbidity in a genitor-urinary clinic. A validation of the Hospital Anxiety Depression scale (HAD). Br J Psychiatry 1988; 152:698-700 [DOI] [PubMed] [Google Scholar]
- 17.Bjelland I, Dahl AA, Haug TT, et al. The validity of the Hospital Anxiety and Depression Scale. An update literature review. J Psychosom Res 2002; 52:69-77 [DOI] [PubMed] [Google Scholar]
- 18.Ye WF, Xu JM. Application and evaluation of hospital anxiety and depression scale in general hospital patients. Chin Behav Med J (in Chinese)1993; 2:17-9 [Google Scholar]
- 19.Derogatis LR. How to use the Symptom Distress Checklist (SCL-90) in clinical evaluations, Psychiatric Rating Scale, Vol II. Self-Report Rating Scale, Hoffmann-La Roche Inc.
- 20.Jin H, Wu WY, Zhang MY. Preliminary analysis of SCL-90 results in normal Chinese. Chin J Nerv Ment Dis (in Chinese)1986; 12:260-3 [Google Scholar]
- 21.Karnofsky DA. The use of nitrogen mustards in the palliative treatment of carcinoma. Cancer 1948; 1:634-56 [Google Scholar]
- 22.Yates JW, Chalmer B, McKegney FP. Evaluation of patients with advanced cancer using karnofsky performance status. Cancer 1980; 45:2220-4 [DOI] [PubMed] [Google Scholar]
- 23.Lecrubier Y, Sheehan DV, Weiller E, et al. The mini international neuropsychiatric interview (MINI). A short diagnostic structured interview: reliability and validity according to CIDI [J] Eur Psychiatry 1997; 12:224-31 [Google Scholar]
- 24.Spitzer RL, Williams JB, Gibbon M, et al. The structured clinical interview for DSM-III-R (SCID). I. History, rationale, and description. Arch Gen Psychiatry 1992; 49:624-9 [DOI] [PubMed] [Google Scholar]
- 25.World Health Organization. The Composite International Diagnostic Interview (CIDI), Authorized Core, Version 1.0. Geneva: WHO Press; 1990. [Google Scholar]
- 26.Si TM, Shu L, Dang WM, et al. Evaluation of the reliability and validity of Chinese version of the mini international neuropsychiatric interview in patients with mental disorders. Chin Mental Health J (in Chinese)2009; 23:439-503 [Google Scholar]
- 27.Derogatis LR, Morrow GR, Fetting J, et al. The prevalence of psychiatric disorders among cancer patients. JAMA 1983; 249:751-7 [DOI] [PubMed] [Google Scholar]
- 28.Glover J, Dibble SL, Dodd MJ, et al. Mood states of oncology outpatients: does pain make a difference? J Pain Symptom Manage 1995; 10:120-8 [DOI] [PubMed] [Google Scholar]
- 29.Ayres A, Hoon PW, Franzoni JB, et al. Influence of mood and adjustment to cancer on compliance with chemotherapy among breast cancer patients. J Psychosom Res 1994; 38:393-402 [DOI] [PubMed] [Google Scholar]
- 30.Lee MS, Love SB, Mitchell JB, et al. Mastectomy or conservation for early breast cancer: psychological morbidity. Eur J Cancer 1992; 28A:1340-4 [DOI] [PubMed] [Google Scholar]
- 31.Passik SD, Dugan W, McDonald MV, et al. Oncologists’ recognition of depression in their patients with cancer. J Clin Oncol 1998; 16:1594-600 [DOI] [PubMed] [Google Scholar]