Examining variation across treatment clinics in cancer patients’ psychological outcomes: results of a cross sectional survey

Abstract

Purpose

The majority of research on psychological outcomes for cancer patients has focussed on the role of individual characteristics, and disease and treatment factors. There has been very little exploration of the potential contribution of the treatment clinic to these outcomes. This study explored whether there is variation among clinics in cancer patients’ psychological outcomes.

Methods

Cancer outpatients were recruited from 22 medical oncology and haematology clinics in Australia. Participants completed a pen and paper survey including the Hospital Anxiety and Depression Scale (HADS), as well as sociodemographic, disease and treatment characteristics.

Results

Of those eligible to participate, 4233 (82%) consented and 2811 (81% of consenters) returned the completed survey. There was no statistically significant variation in HADS depression scores across clinics. Some difference in anxiety scores derived from the HADS questionnaire between clinics (p = 0.03) was found with the percentage of between-clinic variation estimated to be 1.11%. However, once all demographic, disease and treatment predictors were adjusted for there was no statistical differences between clinics (percent of between-clinic variation = 0.53%; p = 0.1415).

Conclusions

Psychological outcomes were not found to vary between clinics. Other sources of variation including patient characteristics may over-ride between-clinic variability, if it exists.

Background

There is an abundance of research indicating that people with cancer experience high rates of depression and anxiety [1–3]. Despite recognition of the importance of addressing adverse psychological outcomes, these conditions remain under-detected and under-treated in many oncology and haematology settings [4, 5]. Consequently, there is a need to delineate the contributing factors that may have an effect on patient psychological outcomes.

What factors may influence psychological outcomes for people with cancer?

Non-modifiable individual characteristics such as age and gender [2] have been associated with psychological outcomes. Potentially modifiable individual factors such as coping strategies [6] and social support [7] have also been linked to poorer psychosocial outcomes. The association between disease and treatment characteristics has also been examined. Compared to other people with cancer, those undergoing current treatment [8], and people diagnosed with poor prognosis cancers such as lung, or cancers with complex treatment regimens such as haematological cancers tend to have poorer psychological outcomes [2, 9, 10].

It is also possible that there are differences between clinics in patient psychosocial outcomes. Variation in patient outcomes has been observed across a number of other areas of health care. For example, the Dartmouth Atlas of Health Care has documented a fivefold regional variation in rate of leg amputations for patients with diabetes [11]. This may be due to regional variation in the management of diabetes across indicators such as blood glucose testing and control, blood cholesterol testing and control, smoking cessation and foot care to identify and treat ulcers [11]. Similarly, regional variation in unplanned readmissions following treatment for peripheral artery disease ranged from 1 in 3 to 1 in 10 [11]. After adjustment for casemix, analyses of national claims data in the US showed that variability in mortality following cancer resection was influenced by both hospital volume and surgeon volume [4].

Several other studies have provided evidence of variation in factors that potentially relate to psychological outcomes. Jacobsen and colleagues reported variation in indicators of psychological care across 11 medical outpatient clinics. The study relied on medical records data, so it was unclear to what extent the variation observed related to actual differences in care patterns or to variation in the quality of documentation. Variation in patients’ psychological outcomes was not examined. There is some evidence of variation in related outcomes such as patient experience and pain across cancer clinics. A Korean study involving 34 palliative care centres revealed significant variation in pain management outcomes between centres. Higher scores for human resource adequacy were associated with better pain outcomes suggesting that organisational factors played a role in the observed variation in pain outcomes across centres [12].

Despite evidence for between-clinic variations in care practices which may be expected to influence psychosocial outcomes, there has been limited exploration of whether psychosocial outcomes vary across clinics. Our previous study involving radiation oncology clinics indicated that rates of anxiety varied from 24 to 34%; while depression varied from 13 to 20% [13]. These differences were not statistically significant, and conclusions of the study were limited by the small sample of clinics. To our knowledge, no other studies have explored the question of whether psychological outcomes vary across cancer clinics.

Therefore, the aim of the present study was to examine in a large sample of medical and haematology clinics: (a) whether there is significant variation in patient anxiety and depression across clinics independent of sociodemographic, disease and treatment factors.

Methods

Setting and design

A cross sectional survey was undertaken involving patients recruited from medical oncology and haematology clinics in Australia. Ethics approval, in accordance with the Helskini Declaration of the World Medical Association, was obtained from the Cancer Institute of New South Wales (2011/10/351) and the University of Newcastle Human Research Ethics (H-2010-1324) Committees, as well from ethics committees associated with each participating site.

Recruitment of clinics

Medical oncology clinics were eligible if they treated at least 400 new patients each year. A leading medical oncologist from each state acted as a ‘clinical champion’ to help with recruitment for the study. These individuals identified clinics which met the inclusion criteria, and sent a written invitation. Additional information was provided by phone, email and/or in-person by the research team.

Patient sample

Patients with a confirmed diagnosis of cancer, aged 18 years or older, English speaking and presenting for an outpatient haematology or medical oncology consultation were eligible to participate. Those unable to complete the survey independently were excluded. As it would be unreasonable to attribute psychological outcomes to clinic characteristics for patients attending the clinic for the first time, these patients were also excluded.

Recruitment of patients

Consecutive patients were recruited in each of the participating clinics. Eligible patients were invited to participate and written informed consent was obtained. The age and sex of non-consenters were collected to assess potential consent bias; no other information was collected due to privacy restrictions. Participants were asked to complete a pen and paper survey either in clinic or at home. Those who took the survey home were asked to return it within 1 week. Non-responders were mailed a second copy of the survey and a reminder letter at 2–3 weeks. A second reminder letter was sent 2–3 weeks after the first reminder.

Measures

Explanatory variables

Demographic variables: Age, sex, highest level of education, Aboriginal and/ or Torres Strait Islander status, marital status, country of birth, home postal code, living situation, employment, private health insurance, concession card status, and smoking status were obtained by patient self-report. Concession cards are government issued cards that enable the bearer to obtain discounts on health services and medicines (e.g. for people on low incomes).

Disease and treatment variables included self-reported cancer type, perceived stage of disease at diagnosis (early or advanced), remission status, time since diagnosis, cancer treatments, relocation for treatment and main reason for hospital visit on the day of recruitment.

Outcome variable

The Hospital Anxiety and Depression Scale (HADS) was used to assess psychological morbidity. The scale includes an anxiety subscale (7 items), and a depression subscale (7 items) [14]. Each item is scored from 0 to 3, giving a maximum score of 21 for each subscale. The HADs has been widely used with cancer patients [15], and meets psychometric criteria for internal consistency, construct validity, and discriminant validity [15]. A score of 8 or above on the depression or anxiety subscales was used to indicate possible caseness for depression or anxiety, respectively [15].

Statistical analysis

Between-clinic variation in anxiety or depression scores (as determined by the HADS) was initially examined through boxplots, and then formally assessed through random effect linear regression models with anxiety or depression score as the outcome and a random intercept for clinic effect. Model-based estimates of the site level means (i.e. partially pooled estimates) with 95% confidence intervals were displayed as caterpillar plots for comparison. The statistical significance of the clinic random effect variance was assessed by comparing the model with a random effect to the equivalent linear model without the random effect, the corresponding likelihood ratio test (LRT) is assumed to follow a 50:50 mixture of chi-square distributions. The intra-class correlation (ICC) was reported, and where clinic variation in anxiety or depression was found to be statistically significant (p < 0.05), we reported the corresponding 95% confidence interval for the ICC obtained from bootstrapping. We also determined if this variation could be explained through differing patient level characteristics by including the following patient demographic (age, sex, marital status, highest level of education, smoking status, concession card status, private health insurance status, Aboriginal and /or Torres Strait Islander status and country of birth) and disease and treatment characteristics (cancer type, time since diagnosis, stage at diagnosis and treatments received) as fixed effects in the mixed-effects regression model and assessed the significance of the between-clinic variation using the method described above. Classification of categorical variables was as shown in Table 1. We also present results for estimates of the ICC on the probability scale for depression and anxiety dichotomised at a cut point of 8 or more. Logistic regression within a generalised estimating equation framework assuming a compound symmetric residual correlation matrix was used for these analyses.

Table 1.

Demographic, disease and treatment characteristics of sample

Variable	Category	Oncology (n = 2096) n (%)	Haematology (n = 715) n (%)
Gender	Male	808 (39%)	408 (57%)
Age	Less than 65	1187 (58%)	416 (59%)
Aboriginal and Torres Strait Islander origin	Yes	29 (1.4%)	7(1.0%)
Marital status	Married or partnered	1355 (66%)	469 (66%)
	Single, divorced, separated or widowed	711 (34%)	239 (34%)
Highest level of education	High school or below	1030 (50%)	321 (46%)
	Vocational training, university or other	1026 (50%)	384 (54%)
Birth place	Australia	1441 (70%)	495 (70%)
	Other	627 (30%)	214 (30%)
Health insured	Yes	861 (42%)	374 (52%)
Concession card	Yes	1190 (58%)	399 (56%)
Smoking status	Current smoker	206 (10%)	59 (8.3%)
	Former smoker	882 (43%)	265 (37%)
	Never smoked	975 (47%)	389 (55%)
Location of residence	City	1229 (59%)	665 (94%)
	Regional or remote	844 (41%)	39 (5.5%)
Living arrangements	With others	1647 (80%)	564 (80%)
Employment	Paid employment	641 (31%)	244 (34%)
	Retired	847 (41%)	266 (37%)
	No paid employment (home duties, unemployed or disability pension)	570 (28%)	200 (28%)
Cancer type	Breast	727 (36%)
	Colorectal	332 (16%)
	Lung	171 (8.4%)
	Melanoma	59 (2.9%)
	Other cancer or more than 1	564 (28%)	357 (50%)
	Myeloma	26 (1.3%)	148 (21%)
	Non-Hodgkin lymphoma	46 (2.3%)	203 (29%)
	Prostate	117 (5.7%)
Time since diagnosis	12 months or less	992 (48%)	203 (29%)
	13 to 24 months	354 (17%)	121 (17%)
	Over 24 months	723 (35%)	383 (54%)
Treatments ever received	Surgery	1475 (71%)	146 (21%)
	Chemotherapy	1684 (82%)	575 (82%)
	Radiotherapy	1003 (51%)	163 (23%)
	Hormone therapy	489 (25%)	16 (2.2%)
	Biological therapy	250 (13%)	49 (6.9%)
	Bone marrow transplant	6 (0.3%)	100 (14%)
	Stem cell	17 (0.9%)	153 (21%)
	Other*	30 (1.5%)	55 (7.7%)
Cancer stage	Early	1113 (62%)	208 (30%)
	Advanced	507 (28%)	166 (24%)
	Do not know or in remission, or not applicable	163 (9.1%)	322 (46%)

*Numbers reported for each variable may not add to total sample size due to missing data

Sample size

A sample of approximately 2800 patients from 22 clinics (an average of 127 participants per clinic) would enable a detection of differences in depression or anxiety scores between any two sites of at least 0.35 standard deviation with 80% power and a 5% significance level.

Results

Twenty-two clinics from 19 hospitals participated in the study. Three hospitals had both a medical oncology and a haematology clinic participate. One clinic was located in a private hospital in a regional area, two clinics were located in regional public hospitals, and the remaining 20 clinics were located in public metropolitan hospitals. Most clinics (n = 21) were located in teaching hospitals. Two clinics were located in hospitals with less than 200 beds, 8 in hospitals with 200–500 beds and 12 in hospitals with more than 500 beds.

Of the 6036 patients screened, 4233 (70%) were considered eligible. Of these, 3472 agreed to participate (82%); 2811 returned a completed survey (81% of consenters) and were included in the analysis. There were no statistically significant differences in terms of the gender of consenters and non-consenters (consenters 57% female, non-consenters 52% female, p = 0.056). The median number of participants per clinic was 150 (minimum 17, maximum 195). Sociodemographic characteristics of participants are presented in Table 1. Most participants (n = 2561, 94%) reported that they had received most of their cancer care from the hospital from which they were recruited.

Depression scores across treatment clinics

Mean depression scores ranged from 3.7 to 5.4, with an overall mean of 4.57 (SD = 3.78). No evidence was found for variation in HADS depression scores among clinics (random effect variance LRT p value = 0.1232), (see Fig. 1), therefore no further analysis for the depression outcome was undertaken. The overall proportion of those reporting elevated depression scores (≥ 8) was 21%, (n = 588) with rates varying from 12 to 32% across clinics and an intra-class correlation (ICC) of 0.0001.

Fig. 1.

Caterpillar plot of the mean depression scores across haematology and medical oncology clinics (estimated from a random effects model). Model-based means are a weighted average of site-specific sample mean and the grand mean. Estimates from sites with large variability and/or small sample size are pooled toward to the grand mean

Anxiety scores across treatment clinics

The overall mean anxiety score was 5.24 (SD = 4.07) with mean scores ranging from 3.9 to 6.4. There was evidence for variation in HADS anxiety scores between clinics (random effect variance LRT p value = 0.0002), with the ICC estimated to be 0.011 (95% CI 0.005, 0.024) (see Fig. 2). The overall proportion of those reporting elevated anxiety scores (≥ 8) was 27%, with rates varying from 12 to 38% across clinics and an ICC of 0.0067.

Fig. 2.

Caterpillar plot of the mean anxiety scores across haematology and medical oncology clinics (estimated from a random effects model). Model-based means are a weighted average of site-specific sample mean and the grand mean. Estimates from sites with large variability and/or small sample size are pooled toward the grand mean

The variation between treatment clinics in HADS anxiety scores was not statistically significantly different than zero once all demographic, disease and treatment predictors were adjusted for (icc < 1%, between-site variation LRT p = 0.1351).

Discussion

The results from this study indicate that approximately one fifth of medical oncology patients reported elevated levels of depression and a quarter indicated clinical levels of anxiety using standardised instruments. These rates are consistent with other studies in the field [1, 2] suggesting the external validity of the findings.

No variation in levels of clinical anxiety or depression across cancer treatment clinics

To our knowledge, this is the first large scale study to examine whether treatment clinics potentially contribute to variation in psychological outcomes for cancer patients. Previous studies suggest that clinic characteristics are associated with unwarranted variation in important outcomes such as pain [12] and mortality rates following surgery [4]. Jacobsen’s study [16] found that indicators of psychosocial care varied across 11 medical oncology clinics; therefore, it is reasonable to expect variation in psychological outcomes may also occur. Therefore, our finding that, once patient factors were controlled for, there was no significant variation across clinics for depression or anxiety is surprising.

A number of hypotheses may explain why between-clinic variation in psychological outcomes was not identified in the present study.

There is a high degree of within-clinic variability in psychosocial care

In Australia, while clinical practice guidelines were developed for psychosocial care of adults with cancer in 2003 [17], these have subsequently been rescinded. There is no requirement for specific distress screening processes, or provision of specific interventions, in order to gain accreditation. In the absence of policy levers to ensure implementation of specific care processes, individual clinicians have a high level of discretion regarding the processes (or lack thereof) which they put in place to detect and treat anxiety, depression and related outcomes. Therefore, our results may reflect that within clinic variability in the process of care used by clinicians within a clinic was greater than between clinic variation in processes.

There is a high degree of similarity in the psychosocial care provided across clinics

It is also possible that there is little within-clinic variation, and that clinics implement very similar care processes. This would also account for our finding of no variation in outcomes. While private hospitals represent approximately 47% of all hospitals in Australia [18], only one private hospital was included in our sample due to prohibitive costs associated with obtaining research ethics and governance approvals. A greater representation of private clinics may have introduced greater variability as it is the nature of private enterprise to strive to differentiate oneself from the crowd by applying innovations in products and processes.

Efficacious psychosocial care strategies are not effective in real world settings, hence variability (or lack of) in delivery across clinics does not impact on outcomes

While numerous studies have demonstrated the efficacy of specific treatments to improve the psychological well-being of cancer patients [19–22], effectiveness studies are more limited. Efficacy studies are undertaken in optimal circumstances with a focus on internal validity [23]. Often, this is achieved by controlling for context in a way that does not reflect the real world. Patient eligibility criteria, for example, are carefully delineated (often excluding individuals with commonly occurring co-morbidities, culturally diverse backgrounds and limiting age range) [23]. Furthermore, clinicians involved in intervention trials may be biased toward those who are already committed to improving psychological care. Additionally, in well-controlled research, the fidelity of adherence to the treatment protocol is usually carefully monitored and maintained [23]. These experimental conditions may not be achieved in everyday clinical practice, diluting the potential effectiveness of an intervention and limiting the generalisability of results. Therefore, if efficacious interventions are implemented poorly, or are ineffective in real-world settings, then variability in care processes may not translate into variability in psychological outcomes of patients. There is a need for the field to undertake effectiveness studies (sometimes referred to as pragmatic trials) where the power of the intervention is examined in real-world circumstances. In such studies, the evaluation design is able to take proper account of context by allowing a more realistic eligibility criteria, ensuring a range of clinicians are involved, and allowing for the expected variation in treatment fidelity.

Measurement error may have hindered the ability to identify variability across clinics

The Hospital Anxiety and Depression Scale is a commonly used measure of psychosocial well-being. It has acceptable psychometric properties, and scores are not confounded by the physical symptoms of cancer or its treatment [15]. However, it remains a screening tool, and unlike a structured clinical interview, is not diagnostic. Therefore, it is possible that measurement error may have impacted on the accuracy of the anxiety and depression scores. In an effort to overcome uncertainty about the optimal clinical cut-point for cancer populations [13], analysis for this paper was undertaken using the HADS score as a continuous variable. Examining the HADS score in this manner increases the statistical power of the analysis.

Limitations

As discussed above, our results must be interpreted in light of several limitations which may have affected the results. These include potential measurement error, and a hospital sample which consisted almost entirely of large metropolitan public hospitals. Further, our study did not allow us to rule out possible explanations for the lack of variability observed such as there being a high degree of within clinic variability in the delivery of care.

Conclusions

This study provides the first large scale examination of whether variability in psychological outcomes occurs across oncology clinics. Our results indicate that there is no variation in anxiety and depression outcomes across medical oncology and haematology clinics.

Funding

This research was funded by a National Health and Medical Research Council Project Grant (ID 1010536), a Strategic Research Partnership Grant (CSR11-02) from Cancer Council NSW to the Newcastle Cancer Control Collaborative (New-3C), infrastructure funding from the Hunter Medical Research Institute (HMRI) [no grant number applicable], National Health and Medical Research Council (NHMRC) Translating Research into Practice Fellowship (ID 1073031) to MC, NHMRC Career Development Fellowship (ID 1061335) to CP, NHMRC (ID 1073317) and Cancer Institute New South Wales (13/ECF/1-37) Early Career Fellowships to AB, and Leukaemia Foundation of Queensland Post-doctoral Fellowship to TCM.

Conflicts of interest

All authors declare that they have no conflicts of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.