Including Economic Evaluations in Implementation Science

The percent of US adults reporting poor mental health continues to rise,¹ and 62% of those who need treatment for a mental health problem do not receive it.² The integration of mental health care in primary care settings can be instrumental in addressing these unmet needs. Substantial evidence of integrated models points to better mental health access and outcomes³; however, many health care organizations do not implement integrated care due to organizational challenges in implementing these models as well as the resources needed for implementation.⁴ The field of implementation science is focused on improving the uptake of evidence-based practices such as integrated care, and various implementation strategies are often tested. Implementation strategies can vary in their intensity and resource use as well as their effectiveness in aiding implementation, so weighing the relative cost and benefits of implementation strategies is critical. Unfortunately, implementation studies often lack information on the costs and value of implementation despite the importance of this information to decision makers.

Ritchie et al. conducted a study to estimate the costs of facilitation as a strategy to implement integrated primary care and mental health services (PCMHI) in two Veterans Affairs (VA) health care networks comprised of several primary care clinics.⁵ Facilitation is a process by which designated individuals engage in problem solving while planning and monitoring the implementation process with stakeholders, such as providers and clinic administrators, to address challenges in implementation. Facilitation, using an external expert and internal staff employed by the health care networks, was previously found to increase the effectiveness of implementing PCMHI compared with standard implementation; for example, it substantially increased the likelihood that patients were treated in PCMHI in practices that used the strategy.⁶ The current study used detailed records of facilitators’ activities and time spent on each activity to estimate the total costs of internal and external facilitator and stakeholder time, ranging from $208,314 to $236,263 in the two networks, over 28 months. Stakeholder participation and internal facilitator costs varied by network and individual clinic, as did the types of facilitation activities. While facilitation has been used to implement various evidenced-based practices in primary care,⁷ few studies have attempted to estimate facilitation costs, and none has estimated facilitation costs for implementing PCMHI.

Dedicating more than $200,000 worth of personnel time across several practices to aid implementation of PCMHI is no trivial matter for a health care organization. Implementation costs alone of PCMHI have been previously estimated at roughly $44,000 for a single practice.⁸ Therefore, the costs of facilitation can add substantially more to implementation costs. However, these one-time costs occurring during the implementation process to adopt a more effective PCMHI model may be relatively low compared with years of potential savings from patients receiving PCMHI services. For example, collaborative care programs have been associated with costs savings of $1300 per patient due to lower downstream health care utilization.⁹ Even without cost savings, effective PCMHI models can achieve more efficient use of health care dollars. A review of 79 randomized controlled trials found improvements in mental health symptoms and quality of life associated with collaborative care models,¹⁰ so higher costs to implement PCMHI can still provide higher value when we compare the cost of PCMHI per unit of health outcome gained relative with usual care.

To inform decision making around implementation of PCMHI and other innovations, economic evaluation of implementation strategies such as this one can be valuable for health care organizations. Knowing the costs of activities and personnel associated with facilitation allows for the planning of required resources and short-term trade-offs for primary care practices and health care systems considering similar strategies. There are limitations when these implementation strategy costs are reported though, since they may provide an accurate accounting of resources for a particular setting but may not generalize to other settings. For example, the VA health care system maintains a national electronic medical records system, so organizations that do not have medical records easily shared between providers may face additional hurdles to implementing PCMHI, and facilitation costs could be higher.

Ritchie et al.⁵ reports on outcomes that are rarely studied in implementation science. While implementation science is widely recognized as central to enhancing dissemination of effective practices to improve health care and health outcomes, implementation costs are not commonly included in evaluations¹¹ despite its potential as a barrier to implementation. One possible reason for this is that guidance on measuring implementation costs has lagged behind guidelines for more traditional economic evaluations such as cost-effectiveness analysis. Recent work now provides a primer on the different types and methods of economic evaluations used in implementation science.^{12, 13} The study by Ritchie et al.⁵ used methods to document the various activities performed by facilitators and their time spent on these activities on a regular basis. The study’s approach can provide a template to guide similar efforts to measure implementation costs in other practices or systems. These methods are consistent with the Cost of Implementing New Strategies (COINS) framework, which is an implementation framework suggested for measuring implementation costs of direct and indirect expenses.¹⁴ Under COINS, costs are assessed for implementation activities as they occur. Another framework, called the Stages of Implementation Completion (SIC), can be used in conjunction with COINS. SIC proposes 8 stages of the implementation process where activities and their associated costs occur in each stage of implementation.¹⁵

The results in this study also demonstrate that collecting resource use of implementation strategies is both feasible and does not require extensive expertise in economic methods. Microcosting is the process of enumerating and costing of inputs of health care services. Microcosting of implementation strategies can be done through activity-tracking forms or time-and-motion studies to obtain estimates of provider and staff time dedicated to implementation activities. Using self-reported tracking forms may impose burdens for participants as more frequent and consistent reporting of participant time and activities improves accuracy but requires more time spent on documentation. On the other hand, less frequent reporting can lead to difficulties with recall and potentially inaccurate information. Streamlining the process to collect relevant information remains a challenge. Translating participant time into costs, however, is relatively straightforward. Information on provider and staff salaries is usually accessible within an organization or can be estimated through public sources of information. Therefore, few hurdles exist to adding analysis of the costs of implementation strategies to existing evaluations.

The field of economic evaluation in implementation science could benefit from greater consensus on methods and tools used in evaluations. For example, in order to assess costs relative to outcomes from an implementation perspective, standardized implementation outcomes could be used to evaluate implementation strategies; fidelity to the intervention, patients’ health outcomes, and clinician preference for implementation strategy have been suggested as such outcomes.¹³ Some implementation activities may lend themselves to measurement through a standard set of required resources akin to relative value units assigned to clinical services, and this area remains unexplored. Additionally, clear methods to capture indirect costs (e.g., clinic space, clinic overhead) of implementation strategies are often lacking.

Given the need by decision makers to understand the resources involved in implementation, evaluators should incorporate economic considerations in implementation studies more routinely. VA’s Quality Enhancement Research Initiative now requires implementation evaluations to include analysis of costs that are appropriate for each program’s economic implications.¹⁶ The more work that is conducted to measure economic outcomes of implementation and implementation strategies similar to the study by Ritchie et al.,⁵ the more the knowledge base on the economics of implementation science can grow. Uncertainty of the resource use needed for implementation strategies may lead to poor adoption of these strategies and consequently slow dissemination of evidence-based practices overall. Widespread conduct of studies that include economics of implementation can aid broader dissemination of health care programs to benefit the health care system.