Remember the denominator: improving population impact of translational behavioral research

Implications.

Research: The papers in this special issue advance the science in four important ways consistent with the National Institute of Mental Health priorities in mental health services research by (a) better operationalizing and validating core components of integrated care models, (b) leveraging technology to improve population health, (c) extending the reach of interventions into novel care settings, and (d) studying mechanism of action associated with interventions.

Practice: Findings of papers submitted to this special issue inform practice by encouraging implementation of effective and sustainable integrated care models to improve population health, by effectively identifying and treating patients in novel settings who would otherwise not be reached by existing approaches, by leveraging technology in innovative ways, and by objectively examining the value of social support as a component of treatment.

Policy: To realize the National Institute of Mental Health vision for a world “in which mental illnesses are prevented and cured,” interventions need to be clinically effective and also able to be reached, accessed, and fully engaged by the people who need them.

Introduction

To realize the National Institute of Mental Health (NIMH) vision for a world “in which mental illnesses are prevented and cured,” [1] interventions need to be clinically effective and also able to be reached, accessed, and fully engaged by the people who need them. Strategic Objective 4 of the NIMH Strategic Plan calls for research to improve the reach and effectiveness of mental health services to strengthen the public health impact of all NIMH-supported research [2]. This message is echoed in NIMH’s active funding announcements for mental health services research (e.g., [3, 4]), and most recently, a $50 million investment in research to optimize the collaborative care model for people with opioid use disorder and mental health conditions, as part of the National Institutes of Health’s (NIH’s) efforts to stem the national opioid crisis and issues related to chronic pain [5].

Collaborative care as a standard for improving population impact

Findings from NIMH-supported epidemiological studies demonstrate that most people with mental health needs are not accessing adequate care. And, of the minority of patients with mental health needs who find their way into care, most are not seen by a specialist. Over 40 years ago, Regier and colleagues [6] aptly identified that primary care is the de facto mental health service system in the United States, after noting that only approximately 20% of people with mental health problems were treated in specialty mental health settings and approximately 60% were identified (but not necessarily treated) in primary care/outpatient medical settings. The Epidemiologic Catchment Area Study showed that less than 1 in 5 respondents with a mental health disorder received treatment in the previous 12 months [7]. And from the National Co-Morbidity Survey conducted a decade later, only about 1 in 4 respondents sought care in the previous 12 months [8]. Findings from the National Co-Morbidity Survey-Revised [9] showed that 41% of respondents with a mental health problem received treatment in the previous year. And of those who sought care, only 22% of respondents received care from a psychiatrist and/or a nonpsychiatrist mental health specialist.

The collaborative care model [10, 11], a specific model of service delivery, extends the expertise of a mental health specialist into primary care settings and improves the quality and outcomes of care in panels of patients with mental health needs. The model puts into practice several key components of effective care integration: (a) population-based care through real-time use of a disease registry; (b) implementation of measurement-based care and treatment to target principles; (c) care management to support the primary care provider with recommended practices (e.g., assist with screening and follow-up and troubleshooting issues of treatment engagement), (d) dedicated psychiatric consultation, which provides case consultation on panels of patients and advises on psychotropic medications, psychosocial interventions (which may be delivered by other members of the care team), referrals to more intensive care, and treatment planning. In this way, collaborative care enhances mental health service delivery in a system where people with mental health problems already go—primary care.

Born out of the RE-AIM analytic framework (Reach, Effectiveness, Adoption, Implementation, and Maintenance; www.RE-AIM.org; [12]), which introduces the idea that an intervention’s overall impact could be estimated as the product of its effectiveness and reach, Koepsell and colleagues [13] developed a practical methodology to estimate population impact using clinical trial data. Their method has since been applied yielding important information. RE-AIM calls attention to essential program elements that can improve the sustainable adoption and implementation of effective, generalizable, evidence-based interventions. For example, in a simulation comparing collaborative care versus behavioral activation for PTSD, Zatzick and colleagues [14] noted that even though behavioral activation outperformed collaborative care (50% vs. 7% PTSD prevention) the reach of collaborative care into the simulated target population far exceeded behavioral activation (0.18% vs. 0.0027%), resulting in a 9.5 greater population impact over behavioral activation. Belsher and colleagues [15] applied this framework to assess the population impact of enhanced collaborative care over usual collaborative care for PTSD and depression, using data from a multi-site clinical trial conducted within the military health system. Findings also demonstrated improved population impact though the use of the enhanced collaborative care model.

NIMH first invested in trials to test the effectiveness of collaborative care models in the early 1990s [16] and continues to fund collaborative care trials where evidence is limited and where the need exists. Since the early 1990s, numerous public and private funders have supported scores of clinical trials [17] and numerous others since that review. These clinical trials generated evidence of effectiveness that contributed to a “strong for” and “weak for” recommendation in the VA/DoD clinical practice guidelines for depression [18] and PTSD [19], along with other complementary data that led to the Centers for Medicare and Medicaid Services and a growing number of pubic and commercial payors ([20, 21]) now reimbursing for services furnished via the collaborative care model.

Subsequent comments attend to not only intervention effectiveness but also the ability of effective interventions to reach the people who need them, that is, the so-called denominator.

PERSPECTIVES

The authors who submitted to this special issue [22–31] report or comment on translational research whose findings have potential to improve practice. The treatment outcome papers cover a variety of settings (e.g., primary care, community, hospital, tertiary care, and Veterans Affairs medical centers), populations (e.g., providers, and people with depression, substance use disorders, sickle cell disease), and interventions, to include interventions using technology. Other papers review novel technologies for behavioral interventions, comment on social support (as a primary target for intervention), operationally define common services (e.g., care management, case management, and care coordination), and develop frameworks to functionally identify core elements of behavioral health integration with the express purpose of helping practice move toward integration and improving care. For this commentary, they are sorted into four categories: rubrics for care integration, integration of technology into integrated care systems, case finding in novel settings, understanding the value of a personal relationship.

Rubrics for care integration

Despite a robust evidence base of clinical and health services outcomes supporting the implementation of collaborative care, implementing effective collaborative care with fidelity is difficult and not always successful [32]. As a systems-level multi-component intervention, busy practices with limited resources may need a guiding framework toward better integration of mental health services into primary care services to realize the positive outcomes seen in the clinical trial literature. Three of the submitted papers provide this guidance.

Williams [31] clarifies and operationalizes the roles and responsibilities of care managers, case managers, and care coordinators, positions which have historically been somewhat confusing to distinguish from one another. Care managers who are furnishing collaborative care services may be asked to perform a variety of tasks in service to their patients. It is crucially important to know what tasks are both valuable and feasible to accomplish under resource-constrained environments, especially if collaborative care is to be a sustainable business practice. For example, for collaborative care to be cost neutral or modestly profitable for practices treating people with opioid use disorder, Lee and colleagues [33] estimated that patient panel sizes should be 85 to 120. When job descriptions are clear and job definitions are practical, patients can be more appropriately matched to needed services. System leaders can ensure they offer the right services to the patient population. And patients and care team members can better share a mutual understanding of the expectations of services provided.

Goldman and colleagues [23], field-tested a framework continuum composed of core components of behavioral health integration. Despite being the gold standard of care integration, they argue that implementing the core elements of collaborative care may be untenable to implement simultaneously (vs. sequentially over time) or worse, untenable altogether. However, Goldman and colleagues suggest that there would be value if clinics took an incremental and intentional approach and opted to integrate one or more of the eight domains (e.g., domains of ongoing care management; decision support for measurement-based stepped care; and/or case finding, screening, and referral to care) over time. In this way, clinics could move along a continuum toward integration.

In a mixed-methods evaluation, Stephens and colleagues [30] identified five core integrated behavioral health principles and mapped them to 25 processes (e.g., principle of providing population-based care mapped to a process of using appropriate assessments of key indicators to triage patients to behavioral health services). They also defined nine clinic structures (e.g., a structure that includes financial billing strategies to ensure sustainability of integrated behavioral health services). The authors obtained stakeholder ratings about importance of these constructs, feasibility of measuring them, and an assessment of whether and how strong these constructs mapped to other measure sets like the National Committee for Quality Assurance (NCQA) criteria for Distinction in Behavioral Health. Generally, these constructs were viewed as important, measurable and associated with key measure sets.

These three papers [23, 30, 31] offer face valid and practical rubrics for primary care practices to improve mental health integration. However, evidence supporting behavioral health integration models is not equal. The evidence supporting the implementation of the collaborative care model is superior to other models and includes a robust body of high-quality clinical trial evidence [34]. In contrast, the evidence supporting other behavioral health integration models is of low quality [35], limiting generalizability and clinical usefulness of the data. For example, although many outcomes were positive in single cohort designs, the review by Possemato and colleagues noted that no differences in patient health status were found when a comparison group was included in the design. Despite positive but low-quality evidence, “[t]he implementation of [non collaborative care primary care behavioral health] PCBH services is ahead of the science supporting the usefulness of these services” [35].

The popularity of other (noncollaborative care) care integration models, in contrast to collaborative care, suggests a need to improve the quality and outcomes associated with these other care integration models. Here, research could inform how best to implement those components of collaborative care that are not already in place, as part of usual care. Research could inform the de-implementation of ineffective care integration approaches and simultaneously, inform the implementation of collaborative care. Through factorial designs or optimization approaches [36, 37], research could also identify the most effective components (i.e., key ingredients or modules) or combinations of those components associated with care integration as defined by Williams, Goldman and colleagues, and/or Stephens and colleagues. Clinical trial testing of noncollaborative care integrated care models would need strong scientific and economic justifications, given the general clinical superiority of collaborative care to other models of care. Optimizing the core components by both clinical outcome and intervention reach associated with each component or combination of components would offer important information to primary care practices about how to best impact their patient populations.

Plugging technological innovations into integrated care

Carleton and colleagues [22] evaluate a multipurpose app that automates several care management functions (e.g., appointment and goal reminders, symptom monitoring, and health education materials) and offers an additional means of communication with the care team through secure messaging. The authors suggest that app-based services can improve productivity, but the authors did not directly measure this important construct in their study. Though the use of the app was not associated with improved clinical outcomes, it was associated with improved quality of care outcomes (e.g., more contacts, fewer missed visits, shorter time to follow-up) and more optimal delivery of mental health services than standard care. By making care more efficient, clinics are better able to address the needs of more patients, thereby improving reach. The authors find weakness with their nonrandomized design. However, it is not clear that a randomized design would accurately reflect the needs of a practice setting, in part because patients could choose or not chose to use the app. The authors’ relatively large convenience sample of willing volunteers helps demonstrate appetite for and ability to engage with the app.

Jonassaint and colleagues [25] and Leung and colleagues [26] capitalize on previously made investments in technology development. Jonassaint and colleagues test an off-the-shelf computerized cognitive behavioral therapy (cCBT) program in patients being treated for sickle cell disease who also have depression. Prior studies suggest that the cCBT program is generally efficacious for those who use it. They reasoned that patients with sickle cell disease could benefit from cCBT, especially because they might not get adequate depression treatment elsewhere, and cCBT is easily scalable. Leung and colleagues assessed provider perceptions of an underutilized cCBT developed by the Department of Veterans Affairs. Providers were found to be accepting of cCBT, and the authors reason that if available to patients enrolled in collaborative care programs, where access to live psychotherapy could be limited, patients would have ready access to an evidence-based treatment alternative.

In both these studies, there is a face valid rationale for use of cCBT—it holds promise to increase access to care because it is so scalable and, therefore, can reach deeply into the target population. “Digital Health technology, particularly online interventions, mobile health applications (apps), and electronic patient portals have tremendous potential to increase the reach of existing evidence-based treatments, to deliver interventions targeted to the unique symptom profile of the individual, and to do so in a manner that allows for as-needed and timely delivery. Technology can extend care to those who cannot access it by overcoming access barriers related to time constraints, transportation problems, and cost.” [38]

To date, this tremendous potential has been a mirage. “In spite of the overwhelming evidence that technology-based treatments can be effective, particularly when coupled with some human support, there are almost no instances of successful and sustainable implementation in real-world clinical settings.” [38] Mounting evidence from multiple trials suggests that despite need, uptake remains poor. While Jonassaint et al. achieved a 90% follow-up rate with the research assessments, participants received marginal benefit from cCBT, arguably due to only completing four of eight sessions. The authors noted that a minor and readily fixed programming error was associated with patient discontinuation, suggesting that even modest barriers can impede patient engagement. Researchers learned this lesson using the same cCBT as Jonassaint and colleagues [39–42].

Leung and colleagues [26] rely on provider attitudes and beliefs that cCBT to realize the reach and effectiveness of cCBT in their patient populations. Extant literature suggests that attitudes and beliefs are poor predictors of cCBT uptake at the scale Lueng and colleagues envision. However, if investments in cCBT have already been made, and minimal resources are required to maintain availability of cCBT to patients, then there is a reasonable opportunity to make good use of cCBT. This situation might be akin to that presented by Belsher and colleagues [43], where patients engaged with and benefited from a web-based intervention for PTSD who declined specialty care services and who would likely otherwise receive no care.

Though more challenging than once thought, the promise of using technological innovations to improve the reach of mental health services is still alluring. This allure is especially true when those technologies begin to emulate human interactions, but use of those technologies is not bounded by pedestrian barriers to services like the business hours of a mental health professional. Sezgin and colleagues [27] conducted a scoping review of the use of voice assistant technologies to improve self-management and healthy lifestyle behaviors. The authors define this technology as an inanimate object (e.g., smartphone) programmed with some type of artificial intelligence capable of two-way dialog (e.g., Apple Siri, Amazon Alexa, or Google Assistant), differentiating it from one-way technology such as voice commands. Unsurprisingly, the clinical evidence of voice assistant efficacy or effectiveness is limited, arguable due to the newness of these technologies and the fact (as identified in this scoping review) that they are not being used or tested as stand-alone interventions.

Reaching the target population in expected and unexpected settings

Another way to increase intervention reach is to find new cases. This is the approach of Sirey and colleagues [28] took with their intervention, SMART-MH. While screening for depression in nontreatment seeking older adults affected by Hurricane Sandy in a community (nonclinical) setting, the authors identified 333 respondents (12% of screened sample) with depression. Of those, 201 (60%) were not in treatment and of those, 143 agreed to receive their “Engage” intervention. One hundred nine participants completed all six sessions, and two out of three patients reported clinically significant improvement in symptoms.

At the other end of the care continuum, Smith and colleagues [29] adapt and port a brief evidence-based intervention for smoking cessation designed for hospital inpatients and outpatients to a substance use disorder treatment program, where the rate of tobacco use was twice that of the general population. Two out of five smokers enrolled in the intervention. Although the authors anticipate that approximately 1/3 to 1/2 of patients will likely drop out of care, the authors noted that rates of enrollment were three times higher than published enrollment rates for people who are in general inpatient settings. The authors posit that the added staff time could be feasibly incorporated into existing workloads. And, by accounting for patient dropouts, the staff time required to deliver the intervention is further reduced.

While reduced staff time may make the intervention seem more implementable to substance use disorder treatment programs, there is a downside to this marketing approach. The authors make a compelling case that the intervention is highly effective, and the benefits of smoking cessation in this population could be life-saving. It seems like an opportunity to more optimally use staff time to promote enrollment and minimize dropout for those patients who may benefit.

The value of a relationship

Graham and colleagues [24] remind us that intervention popularity without effectiveness requires a fundamental shift in thinking about the value of that intervention. The authors comment on the state of science of social support interventions for smoking cessation. Here, meaningful social relationships, which could include relationships with fellow support group members, family members or friends, a support “buddy,” or even relationships from an online community (in lieu of an in-person group), would foster accountability and exchange of information and would drive behavior change. The authors contend that 30 years of research has yielded minimal evidence that social support its effective for smoking cessation, in part due to low engagement with the social support interventions.

However, Graham and colleagues [24] aptly point out that research designs often presume a one size fits all perspective, which may bias results toward the null, because participants who would otherwise not engage with an intervention outside of research context are randomized into a study. Rather, they argue that rigorous observational approaches, combined with more sophisticated data capture techniques available through technology, can inform how social interactions can change behavior. Their position on alternative ways to study social support intervention is supported by research from Carleton and colleagues [22], where a notable number of volunteers (in a nonrandomized study) engaged with a care management app associated and experienced a number of quality of care improvements though automated services provided by the app.

Graham and colleagues [24] suggest that research can and should focus on the mechanism through which social relationships (to include social networks) can impact behavior change, rather than examine the impact of engagement itself. It will be interesting to see how the artificial intelligence/voice assistant technologies reviewed in Sezgin and colleagues [27] evolve and better emulate the mechanics of human interaction sans actual human interaction. It remains an open question as to how the mechanisms of social relationships can positively impact behavior change, when patients engage with these lifelike and highly scalable technologies.

A CALL TO ACTION

The papers in this special issue advance the science in four important ways consistent with NIMH priorities in mental health services research. First, Williams [31], Goldman and colleagues [23], and Stephens and colleagues [30] identify the need to better operationalize and validate core components of integrated care models. Doing so would improve the population impact of clinically effective integrated care models through better reach, broader scale, and sustainable financing models. NIMH continues to call for research in these areas under strategic research priorities 4.1.C (to optimize financing models to provide efficient and effective care) and 4.3.B (to develop and validate service delivery models, to include models of care coordination, to dramatically include outcomes from sustainable service delivery approaches) [44]. While (noncollaborative care) integrated care approaches are more widely adopted than the collaborative care model, they are not well operationalized and supported by a weak evidence base. Innovative approaches (e.g., [36, 37]) to optimizing key ingredients of integrated care models will prove useful.

Second, Carleton and colleagues [22], Jonassaint and colleagues [25], Leung and colleagues [26], and Sezgin and colleagues [27] demonstrate both the need for and challenge with leveraging technological innovations to improve the population health. NIMH recognizes that digital health technology offers unprecedented opportunities to help consumers, clinicians, and researchers measure, manage, and improve mental health services [45] and that service delivery systems using social networking and other new and unconventional technological platforms can exponentially expand the reach of services to the general population and populations who would otherwise not be identified [44]. Third, Sirey and colleagues [28] and Smith and colleagues [29] extend the reach of their interventions by delivering and testing them in novel settings. Their research aligns closely with NIMH strategic research priority 4.3.A.2, which calls for research to make evidence-based practices scalable, deliverable, and effective in nontraditional settings and 4.4 where the reach and effect of innovations is necessary to impact target populations [44].

And fourth, Graham and colleagues [24] suggest that research is needed to better understand mechanism of action associated with the engagement with and delivery of social support interventions. The research directions described by Graham align closely to NIMH’s priorities outlined in active funding announcements (e.g., [46]) to not just understand whether (or not) interventions are effective, but also to understand how and why. In this way, intervention research, to include research on mental health services interventions, can explicitly inform whether the intervention engages the associated change mechanisms and will have utility regardless of trial outcomes (e.g., in the event of negative results, information about whether the intervention was successful at engaging its mechanisms can facilitate interpretation). Applications of this experimental therapeutics approach to mental health services interventions can enhance the rigor of studies for people suffering from mental illness [47] and can potentially inform ways to improve patient access and engagement so that effective interventions can better reach their intended target populations.

Compliance with Ethical Standards

Conflict of Interest: Michael C. Freed has no conflict of interest to report.

Copyright: The author is a federal employee at the National Institutes of Health. No content in this commentary may be copyrighted.

Human Rights/Informed Consent/Animal Welfare: N/A. This is a commentary on papers submitted to this special issue.