Evaluation of collaborative models of care in the management of patients with depression: protocol and progress

Abstract

Introduction Depression is highly prevalent and has a considerable impact on the quality of life of affected individuals, and on healthcare resources. Evidence indicates that collaborative care models can improve patient outcomes within a primary care setting. The Primary Care Services Improvement Project (PCSIP) aims to investigate the costs and outcomes of different models of care for the management of patients with depression. These models have been defined based on the level of involvement of practice nurses in management processes within the primary care setting in Australia. This paper describes our study protocol and its progress.

Methods PCSIP is an observational study that will link retrospective data from a range of sources to estimate costs and intermediate outcomes (such as relapse rate) over a 3-year time horizon. The main sources of primary data include the medical records of patients held at participating practices and Medicare Australia.

Initial report We recruited 15 practices from a metropolitan area and allocated them to three models of care. Two hundred and sixty-one patients agreed to participate. Appropriate regression-based analyses will be used to evaluate the association between different models of care and patient-level outcomes while controlling for several covariates such as age and gender.

Discussion/conclusions This project will generate the knowledge required to promote investment in the most cost-effective initiatives, and to ensure that waste of resources due to the implementation of comparatively inefficient interventions is minimised. Given the scarcity of resources, the increasing costs of providing healthcare and the increasing prevalence of chronic diseases, such research is essential.

Introduction

The rapidly increasing burden of chronic diseases is a major public health challenge around the world. Because of their high prevalence and significant contribution to social and economic costs, some chronic conditions such as depression have been identified as national health priority areas.¹ Depression is the most common mental health disorder and is associated with an increased risk of suicide at rates 10–20 times above those in the general population.^2–4 Approximately 15% of the Australian population will experience depression at least once in their lifetime.⁵ In 2008–09, depression accounted for 34.3% of all mental-health-related problems managed in Australia, making it the most frequently managed mental-health-related problem.⁶

In Australia, it has been estimated that 78% of people with affective disorders, of which depression is the most significant, seek help in the primary care setting.⁷

Therefore, by providing support for assessment, education and proactive follow-up of patients, there is a great potential to improve disease control and health outcomes within primary care. Several randomised clinical trials (RCTs), including a meta-analysis of 35 RCTs, concluded that collaborative care models that incorporate ongoing assessments, patient education and regular follow-up of patients with depressive episodes to monitor adherence to the management plan, can improve patient outcomes.^8–11 Economic evaluations of collaborative models of care have also demonstrated that such models can be cost-effective.^12,13 The key aspect of these models regularly includes practice nurses (PNs). There is increasing evidence that the active involvement of PNs in management processes can improve quality of care in patients with depression.^14,15

In recent years, a number of primary-care-based funding initiatives have been implemented in Australia to provide incentives for general practices to take a more multidisciplinary approach and to improve the recognition and management of a number of national health priority areas, including depression.¹⁶ Examples include financial incentives for general practices through the Practice Incentive Programme to encourage the employment of PNs.

Undertaking an RCT, Eley et al¹⁷ are currently investigating the role of PNs in collaborative care models within the Australian primary care setting by examining changes in clinical outcomes for patients with chronic conditions, such as diabetes, heart failure and cardiovascular disease. With a focus on patients with depression, the TrueBlue study is a cluster-randomised trial that aims to investigate the impact of a PN-led collaborative care model (compared with usual care) on clinical outcomes (e.g. a reduction in the depression score).¹⁸

The available evidence on the effect of PNs^17,18 on patient outcomes has been derived from controlled experiments in which an intervention is introduced as part of the evaluation process. Such studies are always subject to uncertainty around the transferability of effects observed under controlled conditions to routine clinical practice within primary healthcare. Such uncertainties are likely to be magnified in the context of behavioural interventions, where there is more scope for variation in the application of such interventions. In addition to the assessment of effectiveness, any decision around the funding of a healthcare intervention will depend upon its expected cost-effectiveness.

The Primary Care Services Improvement Project (PCSIP) was set up to provide a framework for evaluating the costs and consequences of alternative models of care, as applied in routine clinical practice. The novel aspect of this project is that we do not implement an intervention, rather we recruited general practices and assigned them to one of alternative models of primary care, based on their current activities.

The PCSIP is an observational study and uses linked, routinely collected data to evaluate the costs and benefits of alternative models of care in terms of reducing depression relapse in patients with depression over a 3-year time horizon. However, it is generally acknowledged that the cost and effects associated with interventions in the management of chronic conditions such as depression span a longer time frame. To estimate the long-term costs and outcomes, a modelling approach is required. The PCSIP estimates lifetime costs and benefits of different models of care in terms of the incremental cost per quality-adjusted life year (QALY), using a decision analytic model.

This paper details the design of the PCSIP and describes its progress in terms of practice and patient recruitment, and the data collection process.

Methods/design

The following sections describe alternative models of care, recruitment and classification of practices and patients, the data collected and the data analysis.

Definition of alternative models of care

In recognition of the key role of PNs in management processes in patients with depression, we defined three models of care which show a transition from a medical care model (in which the general practitioner [GP] is the custodian of all care within the general practice) to a team-based approach:

• Model I: no PN;

• Model II: low level of PN involvement in depression management;

• Model III: high level of PN involvement in depression management.

PNs have diverse roles in general practices, including clinical-based activities (e.g. patient education), integration (networking with other healthcare providers), clinical organisation (e.g. clinical data entry) and procedural-based activities (e.g. taking blood).¹⁹

In this study, the level of PN involvement is measured by: (a) the proportion of patients of a general practice that are seen by PNs for the management of depression, and (b) the percentage of the PN time spent on clinical-based activities. These activities include patient education and self-management advice, monitoring clinical progress, and assessing and enhancing treatment adherence. A threshold of 50% or more on both measures has been set to identify a high level of PN involvement, hence allocating the general practice to Model III. Model III is a team-based approach with a high level of PN involvement in depression management, and represents the complementary role of PNs in patient care. All other general practices with PNs are allocated to Model II. Model II includes a low level of involvement of PNs in the management of patients with depression. This model fails to reflect the complementary role of PNs in patient care, where PNs may be seen as replacements for GPs in some clinical situations rather than as professionals that can add value to patient care in their own right.²⁰

A key strength of this project is our consideration of the quality of care provided by PNs, as few studies have attempted to measure this aspect. The 50% threshold has been carefully considered by the PCSIP steering committee comprising experienced primary-care-based researchers, GPs and PNs, and informed by the findings of a scoping survey which was piloted with GPs and PNs working in three practices.

Practice recruitment process and classification

General practices within the Adelaide Northern Division of General Practice (ANDGP) were recruited to the study. Divisions are regional networks of general practices whose main role, among others, is to provide support for GPs to establish an infrastructure for chronic diseases management. ANDGP is located within the northern suburbs of metropolitan Adelaide, South Australia. In 2007, the ANDGP catchment population was approximately 205,000, with 11% aged over 65 years.²¹ The population demonstrates a relatively low socio-economic status profile, with scores well below the average for both Australia and Adelaide on the Socio-Economic Index for Australia (SEIFA).²² There are 66 general practices within the division’s boundaries, and a relatively higher number of patients with mental health disorders compared with the national figure (116 per 1,000 population compared with 98 per 1,000 for Australia as a whole).²²

All general practices within the ANDGP catchment area were approached by an invitation letter sent to the practice manager or lead GP giving information about the project. Fifteen of the 66 general practices within the division’s boundaries agreed to participate in this study, with five practices with no PN, hence classified as Model I. To inform the classification of general practices into Models II and III, PNs employed at participating practices (i.e. 10 practices) were surveyed. On the basis of these surveys, seven practices were classified as Model II (i.e. low level of PN involvement) and three as Model III (i.e. high level of PN involvement). In order to validate our two models of PN involvement, a GP survey was conducted about their agreement on the models. A general consensus was found regarding the validity of the models of care.

Patient recruitment process

In each participating general practice, the Pen Computer Systems Clinical Audit Tool (CAT) was used to identify eligible patients, defined as patients with at least three visits within the last 2 years, aged 18–75 years with major depression (based on clinical diagnosis). Exclusion criteria included pregnancy, a severe mental disorder (e.g. psychosis) or dementia, or living in residential care facilities.

Patients received a letter from their GP informing them of the study, and relevant consent forms, which they were asked to return to the research team. A reminder postcard was sent 2 weeks after the initial letter. After a two-step process (recruitment letters mailed to eligible patients followed by a postcard reminder), 261 of the 1,168 eligible patients approached agreed to participate. The target of 100 patients per model of care was achieved for Model II, but not for Models I and III. The process is described in Figure 1.

Figure 1.

Flowchart of the recruitment process. ANDGP, Adelaide Northern Division of General Practice; PCSIP, Primary Care Services Improvement Project; CAT, Clinical Audit Tool.

Figure 1 shows the PCSIP flowchart to recruit practices and patients.

Data collection and sources

Data are drawn from a range of sources. The following sections describe the data sources and the variables extracted from each source.

The primary outcome measure is a reduction in the relapse rate. The perspective for the cost-effectiveness analysis is that of the healthcare system, thus only direct healthcare costs are considered.

Medical records held at participating practices

An encrypted patient identification code was created for each participating patient. Data from patient records are extracted directly onto a database that contains only patients’ encrypted identification codes, and no patient identifiers (i.e. name, address, Medicare card number). The data collected from medical records provide information on a range of patient-level variables including:

• sociodemographic characteristics (e.g. age, gender, employment status);

• clinical characteristics (e.g. years since depression diagnosed, management history including pharmacological and non-pharmacological interventions, referrals, history of other medical conditions); and

• intermediate outcome measures. The project’s primary outcome measure is reduction in relapse rate. Secondary outcomes are the duration and number of previous episodes, and alcohol abuse and suicide attempt.

Clinical outcome measures, such as relapse, using standard assessment tools are not typically recorded in medical records. Based on a literature review^20,23 and clinical knowledge, we use the following proxy measures to identify relapses:

• increased dose/frequency of treatment;

• switching between treatment options (e.g. switching from one antidepressant to another);

• adding another antidepressant (i.e. combination therapy); and

• re-starting treatment after a time without treatment.

We also consider physician notes to identify relapses, increased symptoms, new symptom development, hospital admission and suicide attempts.

Medicare Australia

Data requested from Medicare provided the following healthcare utilisation data (i.e. number of health service encounters and cost estimates):

• out-of-hospital services, e.g. GP, specialist, pathology and imaging services;

• pharmaceuticals, data on all pharmaceuticals that attract a government contribution (e.g. anti-diabetic agents).

The Integrated South Australian Activity Collection (ISAAC)

The ISAAC provided diagnostic data, procedural data and length of stay for all hospital separations (public and private) in South Australia. Patient-level costs were available for separations at the four main public hospitals. For remaining hospitals, AR-DRG cost weights were used to estimate inpatient costs.

Surveys

The cost of the financial incentives to the government, including payment to general practices to support practices to employ PNs, are being obtained from participating practices.

Other services are provided by the Division of General Practice, including:

• services provided to participating patients if they are referred to the ANDGP by their GPs, e.g. psychological assessment; and

• PN education for participating practices.

Data describing the frequency of these services, and the associated costs are being obtained from the ANDGP.

Sample size calculation

Sample size was estimated on the basis of identifying a statistically significant effect. We estimated that 50 patients per model are required to detect an effect size of 0.4 between models of care at the 5% level of significance with the 80% power under an analysis of variance test. The choice of the effect size was based on previous studies of collaborative care models within primary care settings, and is consistent with the findings of a systematic review.^24–27 Assuming an intraclass correlation coefficient (ICC) of 0.04, the design effect was calculated to be 1.5 (based on previous works in primary care).^16,20 This inflation factor gives an adjusted number of patients per model of care of 75. Finally, to account for regression-based adjustment for potential confounding variables, we increased the estimated sample size by 5% per control variable, 28 and hence a sample size of approximately 100 patients per model of care was estimated.

Data analysis

The short-term (i.e. within-trial analysis) and long-tem (i.e. model-based analysis) costs and benefits of different models of care are currently being evaluated. A key strength of the data analysis is around risk adjustment (i.e. controlling for covariates).

Within-trial analysis

An appropriate regression-based analysis will be used to evaluate the association between different models of care and patient-level outcomes while controlling for both general practice and patient-level covariates. The analysis will estimate adjusted mean estimates of costs and outcomes for each of the three models of care, which will be compared with estimated incremental costs and effects between the different models of care. Bootstrapping methods will be used to represent the uncertainty around the adjusted mean parameter estimates, and the incremental results.

Model-based analysis

The short-term data (3 years) derived from the PCSIP will be used as a basis to estimate lifetime costs and benefits of alternative models of care. We will develop a decision analytic model to extrapolate short-term findings. Decision analytic models include a set of health states (e.g. response, remission, chronic depression) and transitions between them. These models are used to synthesise data from a variety of sources, to link intermediate outcomes to final outcomes (e.g. QALYs) and to extrapolate beyond the data observed in clinical trials.¹⁸ The aim is to estimate differences in the time spent in each state by patients receiving alternative interventions, over a defined time horizon (e.g. patients’ lifetimes). Costs and utility weights (representing quality of life on a scale of 0 to 1) are then attached to the time spent in each state to estimate the costs and QALYs associated with alternative management strategies (e.g. pharmaceuticals, models of care, etc.).

The PCSIP depression model will be populated by using data from a variety of sources, and will include primary data and evidence from the literature. The details of the PCSIP model structure have been described elsewhere.²⁹ Briefly, the model was designed on the basis of the natural history of depression and includes all relevant clinical events and patient attributes.

The PCSIP depression model will estimate lifetime costs and QALYs for each model of care, which will inform incremental cost-effectiveness ratios between the three models of care in terms of the expected lifetime costs and QALYs. Because of high levels of uncertainty in the parameter estimates, appropriate methods will be used to test the robustness of model outputs (i.e. sensitivity analysis).

Discussion

This paper reports the protocol of an observational study that uses routinely collected data to compare the cost-effectiveness of alternative models of primary care with respect to the treatment of depression. The management of chronic diseases is now one of the major challenges facing healthcare systems. It has been noted that providing collaborative models of care can improve the management and control of chronic diseases such as depression. Given the large amount of funding currently used to maintain primary-care-based initiatives in general practices in Australia, the results of studies such as the PCSIP will provide important information to optimise allocation of inevitably scarce healthcare resources.

The PCSIP facilitates assessment of the impact of initiatives in a ‘real-world’ population to which they have been applied, rather than in an ideal population, as traditionally seen with RCTs. Real-world practice is represented, which improves its generalisability. The study also explores opportunities in using linked, routinely collected data to evaluate primary-care-based interventions. Furthermore, it eliminates reporting, observer and assessment bias, and is able to track patient history effectively. The value of such analyses is derived from the feedback they provide to policymakers, which may inform an iterative process of policy development, implementation and evaluation. From the findings of this study, policymakers may learn that there is variation in the roles and responsibilities assigned to practice nurses, and that this variation has potential implications for the effectiveness, and cost-effectiveness of primary care PNs with respect to the management of depression.

The limitations of the project are inherent in practice-based research and observational studies, which require use of appropriate regression-based analyses to control for differences in patient, healthcare professional, and practice characteristics. The non-random selection of practices and recruitment of those GPs and patients who agree to participate may lead to self-selection bias, which may affect the external validity of the results.