Evaluating the Implementation of the Re-Engineering Systems of Primary Care Treatment in the Military (RESPECT-Mil)

Abstract

A RAND team conducted an independent implementation evaluation of the Re-Engineering Systems of Primary Care Treatment in the Military (RESPECT-Mil) Program, a system of care designed to screen, assess, and treat posttraumatic stress disorder and depression among active duty service members in the Army's primary care settings. Evaluating the Implementation of the Re-Engineering Systems of Primary Care Treatment in the Military (RESPECT-Mil) presents the results from RAND's assessment of the implementation of RESPECT-Mil in military treatment facilities and makes recommendations to improve the delivery of mental health care in these settings. Analyses were based on existing program data used to monitor fidelity to RESPECT-Mil across the Army's primary care clinics, as well as discussions with key stakeholders. During the time of the evaluation, efforts were under way to implement the Patient Centered Medical Home, and uncertainties remained about the implications for the RESPECT-Mil program. Consideration of this transition was made in designing the evaluation and applying its findings more broadly to the implementation of collaborative care within military primary care settings.

The Re-Engineering Systems of Primary Care Treatment in the Military (RESPECT-Mil) Program is a system of care designed to screen, assess, and treat posttraumatic stress disorder (PTSD) and depression among active duty service members in the Army's primary care settings. A team of researchers from RAND evaluated the implementation of the program in military treatment facilities based on existing program data and discussions with key stakeholders. This report presents results from the evaluation and makes recommendations intended to improve the implementation of collaborative care models such as RESPECT-Mil in military treatment settings. Lessons learned may apply to other primary care–based models that facilitate and coordinate care for behavioral health needs (e.g., Patient Centered Medical Home, or PCMH).

Background

Improving Access to Mental Health Services for Active Duty Service Members

Despite high rates of need, many service members do not seek mental health care. Among active duty service members with probable PTSD or depression, nearly half have not sought any mental health care in the prior year (Schell and Marshall, 2008). The Department of Defense (DoD) Task Force on Mental Health put forth a recommendation to embed mental health professionals in primary care settings as one way to increase the accessibility of mental health services among service members (Department of Defense Task Force on Mental Health, 2007). This recommendation is based on the view that primary care settings may be less stigmatizing than specialty mental health clinics. Identifying service members with unmet mental health needs in primary care and connecting them to services in the same setting may be an effective means for increasing access to treatment. Service members make an average of three primary care visits per year; women and those with PTSD access primary care at even higher rates (Frayne et al., 2011).

Effective approaches to integrating mental health treatment within primary care have been established for depression and anxiety in civilian settings (Archer et al., 2012; Thota et al., 2012). Similar efforts have been under way in the Department of Veterans Affairs (VA) for several years now but have mostly targeted depression (Felker et al., 2006; Fortney, Enderle, et al., 2012). DoD has launched programs to integrate mental health care into military primary care settings with active duty service members, but these programs have not been evaluated extensively (Weinick et al., 2011).

In 2009, the Defense Centers of Excellence for Psychological Health and Traumatic Brain Injury (DCoE) asked RAND to evaluate DoD-sponsored programs aimed at promoting the psychological health of service members and their families. One of the programs selected for evaluation was RESPECT-Mil.

Origins and Components of the RESPECT-Mil Program

RESPECT-Mil is based on a collaborative care model known as Re-Engineering Primary Care Treatment of Depression (RESPECT-D) (Dietrich et al., 2004). At the center of RESPECT-D is its Three Component Model (3CM), a systematic integrated team approach to depression care involving three core components: (1) the primary care clinician and practice, (2) care management, and (3) a collaborating mental health specialist. RESPECT-D was designed to account for challenges in implementing and sustaining collaborative care programs in real-world settings that lack intensive, externally funded research support. RESPECT-D is composed of practical, evidence-based clinical routines for depression management intended to facilitate broad dissemination using available resources within health care organizations. RESPECT-D has been found to substantially improve depression-related outcomes and treatment satisfaction compared with treatment as usual in primary care (Dietrich et al., 2004). RESPECT-Mil builds on RESPECT-D by using the 3CM to improve the management of PTSD and depression in the Army's primary care clinics.

Analogous to RESPECT-D, RESPECT-Mil has three components: (1) the primary care provider and prepared practice, (2) the RESPECT-Mil care facilitator, and (3) the behavioral health specialist, more commonly referred to as the behavioral health champion (Engel et al., 2008). The primary care provider and prepared practice provide routine screening, assessment, and management of PTSD and depression. The primary care provider works with service members to develop a treatment plan, which may include psychotropic medication, counseling, and self-management strategies. The care facilitator plays a supportive role by promoting service members' adherence to treatment plans. Care facilitators achieve this via monthly follow-up contacts during which they attend to service member needs, monitor treatment adherence and response, and encourage self-management strategies. Care facilitators keep the primary care provider and the behavioral health champion abreast of the service members' treatment experiences. The behavioral health champion, typically a psychiatrist, provides clinical and pharmacotherapy advice to the primary care provider, monitors service members' treatment progress with the care facilitator, and facilitates referrals to specialty care when indicated.

In addition to the 3CM, the U.S. Army Medical Command ordered the formation of the RESPECT-Mil Implementation Team to provide program management, training, oversight, and assistance to Army installations assigned to implement RESPECT-Mil (U.S. Army Medical Command Operation Order 07–34; see U.S. Army Medical Command, 2007).* Beginning in 2007, the program was implemented at 15 Army military treatment facilities, involving 43 primary care clinics. By the summer of 2012, RESPECT-Mil had expanded to 37 Army installations and more than 90 clinics (Deployment Health Clinical Center, 2012). RESPECT-Mil represents one of the largest undertakings to implement collaborative care for PTSD and depression in real-world settings.

RAND's Evaluation Goals

RAND's evaluation focused on the implementation of RESPECT-Mil. The evaluation had three specific aims: (1) assess the degree to which RESPECT-Mil is being implemented in the Army's primary care settings; (2) identify facilitators and barriers to implementation; and (3) examine the sustainability of RESPECT-Mil according to the perspectives of key stakeholders in the military health system. We map these three aims onto the Reach, Efficacy, Adoption, Implementation, Maintenance (RE-AIM) framework (Glasgow, Vogt, and Boles, 1999). The RE-AIM framework has been used as a model for evaluating the implementation or translation of evidence-based practices into real-world settings (Meyer et al., 2012; Rogers et al., 2013).

According to the RE-AIM framework, the implementation and public health impact of an evidence-based intervention depends on the following factors: the scope and extent to which the intervention reaches the targeted population; the efficacy of the intervention in yielding positive outcomes; the degree of adoption of the intervention across a substantial proportion of settings; the level of implementation fidelity with respect to whether the intervention is being delivered as designed; and the viability of the long-term maintenance of the intervention.

For aim 1, we examined the implementation of RESPECT-Mil in relation to the reach, adoption, implementation fidelity, and efficacy of the program.** Aims 2 and 3 explore factors associated with the maintenance of collaborative care programs like RESPECT-Mil. Aim 2 identifies facilitators and barriers to implementing RESPECT-Mil from the perspective of providers and the RESPECT-Mil Implementation Team. Aim 3 assesses the sustainability of the program from the vantage point of stakeholders in the military health system.

Methodological Approach

For aim 1, we relied on two data sources that the RESPECT-Mil Implementation Team maintains for the purposes of program oversight—Monthly Screening and Referral Clinic Reports and the Fast Informative Risk and Safety Tracker and Stepped Treatment Entry and Planning System (FIRST-STEPS). Data were extracted from these two sources for the period of August 2011 to March 2012 for 37 U.S. Army installations with 84 primary care clinics. The Monthly Screening and Referral Clinic Reports track screening and referral practices for PTSD and depression. FIRST-STEPS is an electronic case-management tracking tool designed for use by care facilitators and behavioral health champions. FIRST-STEPS contains records on care facilitator contacts, clinical assessments, medication and counseling adherence, engagement in psychoeducation and self-management goals, and final dispositions of case closures.***

For aim 2, we spoke with RESPECT-Mil providers and the RESPECT-Mil Implementation Team to identify facilitators and barriers to implementation. A total of 35 RESPECT-Mil providers participated in the study (i.e., 11 care facilitators, seven behavioral health champions, eight primary care champions, and nine primary care providers), while all 11 of the RESPECT-Mil Implementation Team members participated (e.g., program director, deputy director, behavioral health and care facilitator proponent, program evaluators, database managers and analysts, and administrative assistants). For aim 3, we relied on discussions with key stakeholders within the military health system and with the RESPECT-Mil Implementation Team to gain further insight to factors that may influence the sustainability of the program. Personnel from the Regional Medical Commands, U.S. Army Medical Department, U.S. Army Medical Commands, the Office of the Assistant Secretary of Defense for Health Affairs, and Tricare Management Activity were invited to take part in the study. A total of 24 of 43 key stakeholders who were contacted agreed to participate.

Findings

Aim 1: Extent of Implementation of RESPECT-Mil in Army Primary Care Settings

Reach

To determine the reach of RESPECT-Mil, we asked the following question: To what extent does RESPECT-Mil facilitate the identification of service members with mental health needs related to depression and/or PTSD?

• Of the primary care visits made from August 2011 to March 2012, 93 percent (599,760) included screens for PTSD and depression.

• Of the screened visits, 13 percent (77,998) resulted in a positive screen.

• Of the positive screens, 61 percent (47,797) resulted in a probable diagnosis of a mental health disorder:

  • ▪Forty-six percent (36,231) of positive screens had a diagnosis of PTSD, depression, or both.
  • ▪Fifteen percent (11,566) of positive screens had a diagnosis other than PTSD or depression.

Of the approximately 80,000 positive screens, a breakdown of their final disposition is as follows (final dispositions of the remaining 2 percent [1,617] were unknown due to missing data):****

• Sixteen percent (12,835) resulted in a referral being accepted to one or more of the following sources of care: enhanced primary care treatment (i.e., RESPECT-Mil), behavioral health, or another psychosocial resource.

• Eight percent (6,353) resulted in a referral being declined to RESPECT-Mil, behavioral health, or both.

• Five percent (4,033) resulted in the need being addressed in primary care “as usual.”

• Thirteen percent (10,172) resulted in no behavioral health need being identified.

• Fifty-five percent (42,988) were recorded as already being in one or more sources of mental health treatment.

Findings indicate that at an absolute level, RESPECT-Mil is identifying a considerable number of service members who are reporting depression and PTSD symptoms, as evidenced by the number of positive screens resulting from routine screening. Moreover, a substantial proportion of positive screens are resulting in the detection of probable diagnoses of depression and PTSD and other mental diagnoses. Of the total positive screens, only a smaller proportion (13 percent) had no behavioral health need identified. Of the 19,188 referrals provided (referrals accepted plus referrals declined), approximately two-thirds were accepted, resulting in a sizeable number of service members being connected to needed mental health care. More than half of the positive screens were composed of service members who were already in treatment but were still symptomatic. To the extent that routine screenings can facilitate additional support for service members who are engaged in treatment but are not progressing sufficiently, this may be another venue in which RESPECT-Mil can address unmet mental health needs.

Adoption

To examine the level of adoption of RESPECT-Mil,***** we analyzed the screening and referral rates for each of the 37 Army installations that had implemented RESPECT-Mil. This helped answer the following question: How do Army installations vary with respect to the identification and referral of service members with mental health needs?

• A majority of installations (25 out of 37) were screening a high proportion of visits (ranging from 91 to 99 percent); 31 of 37 installations were screening at least 80 percent of their visits.

• In many cases, service members who screened positive were already in treatment. At 20 of the 37 installations, 50 percent or more of those screening positive were recorded as already receiving care.

• For ten of the 37 installations, at least 20 percent of positive screens resulted in an accepted referral to RESPECT-Mil; 24 of the installations had rates between 10 percent and 19 percent.

• Of positive screens, five of the 37 installations had rates of declined referrals to RESPECT-Mil that were 10 percent or higher.

Most installations were conducting depression and PTSD screening for a large proportion of primary care visits. Rates of positive screens were also fairly uniform across sites. In contrast, rates of probable diagnosis resulting from positive screens were more variable, with no evident or consistent relationship to length of implementation time. With respect to rates of referrals of positive screens, variation also occurred across sites but with no clear relationship to length of implementation time. Thus, implementing RESPECT-Mil over a longer duration does not guarantee greater rates of identification and referral of service members with mental health needs. Further, the extent to which variations in probable diagnoses and referral rates are due to service member factors (e.g., differences in clinical symptoms, willingness to disclose, or preferences for certain types of services is unknown) versus provider factors (e.g., administration of clinical assessments, willingness to address mental health needs) is unknown.

Implementation Fidelity

The RAND team analyzed data from FIRST-STEPS, an electronic case-management tracking tool, to examine the implementation of key components of RESPECT-Mil. We extracted clinical assessment and treatment monitoring information for 3,043 service members who were enrolled in RESPECT-Mil during the period between August 2011 and March 2012. Below are the main findings to the questions we posed regarding the implementation fidelity of RESPECT-Mil. We asked: To what degree is RESPECT-Mil enrolling service members with depression and/or PTSD?

Of the 3,403 service members enrolled in RESPECT-Mil:

• Sixteen percent (549) had no symptoms or minimal symptoms.

• Another 14 percent (473) were missing clinical assessments at baseline.

• The remaining 70 percent (2,381) were classified according to one of the baseline clinical status categories created for this study (i.e., depression prominent, PTSD prominent, depression plus PTSD prominent). Of the 3,403 service members enrolled in RESPECT-Mil:

  • ▪Twenty-one percent (716) were classified as “depression prominent” (mild to severe depression).
  • ▪Twenty-six percent (875) were classified as “PTSD prominent” (mild to severe PTSD).
  • ▪Twenty-three percent (790) were classified as “depression + PTSD prominent” (mild to severe depression and PTSD).

RESPECT-Mil enrolled, at somewhat comparable rates, service members with depression prominent, PTSD prominent, and depression plus PTSD prominent symptoms. Enrolled service members also exhibited a wide range of clinical symptom severity levels. A proportion of service members are also presenting with no or minimal depression and/or PTSD symptoms.

We also asked: To what extent are care facilitators able to implement their RESPECT-Mil responsibilities? To facilitate comparisons between our findings and other collaborative care studies, we focused on the 2,381 service members who had mild to severe depression and/or PTSD symptoms (i.e., service members classified with one of the baseline clinical status categories). These are the main findings regarding care facilitator contacts:

• Of the 2,381 service members who were classified with one of the baseline clinical status categories:

  • ▪Thirty-eight percent (897) had their cases closed after the initial primary care referral to RESPECT-Mil and had no contacts with the care facilitator. This proportion is in the middle range compared with similar studies of collaborative care programs (e.g., Wells, Sherbourne, Schoenbaum, et al., 2000; Chaney et al., 2011).
  • ▪Twenty-three percent (541) only had an initial care facilitator contact.
  • ▪Thirty-nine percent (943) had an initial care facilitator contact and at least one monthly follow-up contact. This rate of follow-up contacts is lower than in the original RESPECT-D study (Dietrich et al., 2004), in which 64 percent of depressed patients had follow-up contacts.

• After the initial care facilitator contact, service members had an average of 2.6 follow-up contacts, which falls slightly below RESPECT-Mil recommended guidelines.

• Nearly 80 percent of all monthly follow-up contacts were conducted early or on time.

• Ninety percent of monthly follow-up contacts had a clinical assessment recorded.

Similar to other collaborative care studies, care facilitators experienced challenges with establishing contact with a substantial proportion of service members after their initial primary care referral to RESPECT-Mil. Furthermore, care facilitators were unable to engage service members in the recommended number of follow-up contacts. Of the follow-up contacts that were made, a high proportion were conducted on time and included clinical assessments to track responses to treatment. Service members who had established contact with the care facilitator remained enrolled in RESPECT-Mil for approximately two months, on average, which is a shorter time frame than outlined by the program.

We then asked: Are service members participating in the full course of recommended treatment? What is the degree of engagement in psychotropic medication, counseling, self-management goals, and psychoeducation?

Regarding psychotropic medication:

• Approximately 39 percent of service members who established contact with the care facilitator reported starting psychotropic medication. Psychotropic medication use was relatively lower than rates found in comparable studies with ranges between 73 and 83 percent (Fortney, Pyne, Edlund, et al., 2007; Hedrick et al., 2003; Schnurr, Friedman, Oxman, et al., 2013).

• At baseline, only 9 percent of service members reported that they were taking or that their primary care provider had suggested that they take medication.****** This increased to 53 percent by last follow-up contact.

• Among service members who had been prescribed medication, adherence rates stayed about the same from baseline (60 percent) to last follow-up contact (61 percent).

Regarding counseling:

• Twenty-three percent of service members who had contact with a care facilitator had started counseling while enrolled in RESPECT-Mil.

• At baseline, only 14 percent of service members were recorded as either attending counseling or having been referred to counseling by a primary care provider.******* This increased to 67 percent by last follow-up contact.

• Among service members attending counseling appointments, approximately 30 percent reported “attending all,” “almost always,” or “often.” This increased to 49 percent by the last follow-up contact. Comparable rates of engagement in counseling were found in the RESPECT-D study (Dietrich et al., 2004).

Regarding self-management goals and psychoeducation:

• Sixty-seven percent of service members who made contact with a care facilitator reported working on self-management goals. This rate of engagement is substantially higher than found in a VA collaborative care study (Fortney, Enderle, et al., 2012).

• Sixty percent were recorded as having read psychoeducational materials. This figure is comparable to the 71 percent of individuals who reported being offered psychoeducational materials (a less stringent criteria) in RESPECT-D.

Overall, 46 percent of service members had started either psychotropic medication or counseling while enrolled in RESPECT-Mil. Although primary care provider recommendations for psychotropic medication and service member initiation of psychotropic medications appeared to increase over the course of RESPECT-Mil, only 39 percent of service members had started psychotropic medications, which is relatively lower than rates found in other collaborative care studies. Similarly, rates of attending counseling or having a primary care provider recommend counseling increased during the course of being enrolled in RESPECT-Mil. In total, 23 percent of service members were recorded as having started counseling while enrolled in RESPECT-Mil, which is comparable to some other collaborative care studies (Dietrich et al., 2004; Hedrick et al., 2003). More than 60 percent of service members were recorded as having engaged in self-management goals and in psychoeducation, rates that are comparable to or higher than those found in other collaborative care studies.

Efficacy

The RAND team used the same FIRST-STEPS data to examine outcomes associated with RESPECT-Mil. We summarize our findings in response to the following questions: What proportion of service members experience improvement in depression and/or PTSD symptoms? What proportion shows improvement in functioning?

• Forty-two percent of service members in the “depression prominent” category experienced a 50 percent reduction in depression symptoms from baseline to the last follow-up assessment. Other studies have reported a range of 19 percent to 53 percent of patients showing similar improvement.

• Thirty-three percent of service members in the “PTSD prominent” category experienced similar decreases in symptoms. In a civilian collaborative care study with PTSD patients, 50 percent experienced decreases in symptoms with a less stringent criterion (i.e., 40 percent reduction in symptoms) and with a longer follow-up period (six months) (Craske et al., 2011).

• Twenty-nine percent of service members in the “depression prominent” category experienced remission, which is within the range of other depression collaborative care studies, with rates of 26 percent to 30 percent (Dietrich et al., 2004; Fortney, Enderle, et al., 2012).

• Twenty-six percent of service members in the “PTSD prominent” category experienced remission. This rate is substantially higher than that found in the Re-Engineering Systems for the Primary Care Treatment of PTSD (RESPECT-PTSD) study (Schnurr, Friedman, Oxman, et al., 2013), which may have enrolled patients with greater levels of clinical symptom severity.

• Across the clinical status categories, the proportion of service members who reported no longer experiencing depression and/or PTSD-related impairment in functioning at their last follow-up visit ranged from 17 to 28 percent.

Overall, rates of treatment response and remission seen in RESPECT-Mil were within the range of other collaborative care studies. A slightly larger proportion of service members in the depression prominent category reported no longer experiencing functional impairment at the last follow-up assessment compared with service members in the PTSD prominent or depression plus PTSD prominent categories.

Aim 2: Facilitators and Barriers to Implementation

Based on interviews with providers (care facilitators, behavioral health champions, primary care champions, and primary care providers), we identified factors that facilitate or hinder the various provider roles.

Facilitators

• Screening for mental health needs is valued as a means of reaching a broader population that otherwise might not receive mental health care.

• Regular communication and consultation with the behavioral health champion eases primary care providers' comfort with prescribing and managing medication for depression and PTSD.

• Solid linkages between primary care providers and care facilitators facilitate successful care coordination.

Barriers

• Stigma (that is, fear of negative repercussions by one's unit and for career advancement) impedes engagement in mental health treatment.

• Some primary care providers do not buy into the program and refuse to refer service members to RESPECT-Mil.

• Short appointment times and heavy caseloads create challenging time constraints, especially for primary care providers and behavioral health champions.

• Primary care providers can experience discomfort with handling behavioral health issues. Many viewed certain behavioral health problems as requiring treatment outside of primary care.

• Insufficient coordination and communication between providers can pose problems. In particular, the handoff between the primary care provider and the care facilitator presented difficulties.

• Care facilitators are perpetually challenged in their ability to maintain contact with service members due to deployment, permanent change of station, block leave, and so on.

• Lack of engagement by top command is seen as a lost opportunity to strengthen the program, if not an outright barrier.

Interviews with the RESPECT-Mil Implementation Team also provided insight into factors that facilitate and impede their role in providing training, monitoring, and oversight of program implementation.

Facilitators

• Command support (e.g., Medical Command, Installation Command, chief of primary care, chief of family medicine) assists in the development of an implementation plan that is good for the site.

• Site performance reports are a valuable tool for monitoring and enforcing program fidelity.

• Monthly phone “coaching” calls provide sites with support around hiring, staff turnover, and training, as well as with problem solving around any implementation issues.

Barriers

• RESPECT-Mil Implementation Team staff are “too small” and “stretched too thin” to perform effectively.

• Unfilled provider positions due to long hiring processes and staff turnover can hinder program implementation.

• There are challenges in providing real-time feedback to sites due to delays in obtaining data from clinics as well as in creating and disseminating performance reports.

Aim 3: Sustainability of RESPECT-Mil

We also asked key Army and DoD stakeholders, many of whom are responsible for implementing health care initiatives in the Army, about sustainability issues regarding RESPECT-Mil, particularly in the context of the transition to the PCMH model of care that is now under way. The discussions focused on the following issues:

Addressing stigma in the Army. RESPECT-Mil was cited as a promising way to destigmatize mental health care, suggesting that the program meets an important need. Other comments noted the possibility of adapting RESPECT-Mil for other deployment environments and for explicitly targeting suicide prevention.

Meeting the mental health care needs of service members. There was general support among respondents, who saw value in the RESPECT-Mil program. At the same time, there was consensus that the time was ripe for a reevaluation of mental health initiatives. Circumstances have changed since RESPECT-Mil was first launched. In addition, future funding for mental health initiatives may not be as robust as once envisioned, suggesting a possible need to consolidate mental health programs and potentially integrate RESPECT-Mil with other initiatives. Further, though some considered RESPECT-Mil's performance monitoring system as advanced for Army medicine programs, others were more critical and stressed the need to examine the evidence base for the program and service member outcomes.

Transitioning to the PCMH model. Most respondents agreed that the RESPECT-Mil program would likely be integrated into the PCMH model over time, meaning that the program would lose its status as a separate program with fenced funding. However, some aspects of the program were perceived as valuable and worthy to be continued. In particular, respondents pointed to the screening aspect of RESPECT-Mil, as well as the continuing integration of primary care providers into behavioral health care, especially for medication management for depression and PTSD. In addition, the program's activities for monitoring and tracking patient progress were noted as strengths.

Interviews with the RESPECT-Mil Implementation Team identified other factors that may affect the sustainability of RESPECT-Mil. Factors include the amount and source of funding (e.g., U.S. Army Medical Command versus Defense Centers of Excellence), which can affect critical implementation processes like hiring, training, and the use of the FIRST-STEPS system; leadership support and buy-in; and the increased workload resulting from the expansion of services to dependents of service members, and to the Navy, Marines, and Air Force, which is occurring under the rollout of the PCMH.

Conclusions and Recommendations

Overall, our results indicate that RESPECT-Mil is performing in ways that are comparable to other primary care collaborative care efforts that have been analyzed in the published scientific literature. Of course, RESPECT-Mil is difficult to compare with other collaborative care studies because of differences in selection criteria for enrollment, settings, and types of interventions. Given this caveat, however, both processes of care and outcomes for RESPECT-Mil are generally comparable with those of other programs. Nonetheless, some aspects of program implementation lagged behind expectations delineated in the program design and manuals, indicating opportunities for improvement in the future.

Based on these results, we developed several recommendations for refining the RESPECT-Mil program and for improving the access and quality of behavioral health services for military service members. The highlights of these recommendations appear below.

Improving the Recognition and Assessment of Depression and PTSD

Consider ways to streamline screening and assessment. Routine screening is seen as a major strength of RESPECT-Mil. Potential areas where streamlining might be explored include bypassing clinical assessments among service members who have been recently screened and diagnosed by the program and finding ways to ease the administrative burden involved in conducting and recording assessments.

Determine the value of screening service members already enrolled in behavioral health care. Half of the positive screens were already in behavioral health care. Flagging service members who are experiencing clinically significant depression and/or PTSD symptoms despite receiving care in behavioral health settings may provide an opportunity to intervene to ensure that adequate levels of treatment are being obtained.

Enhance command support. Findings from stakeholder and RESPECT-Mil provider discussions indicated that service members may not report PTSD and depression symptoms during routine screening because of anticipated negative repercussions from their fellow service members and commanders. Due to the use of different screening and diagnostic instruments as well as scoring algorithms in other research studies, there are no benchmarks to accurately gauge whether service members are underreporting on RESPECT-Mil screening and diagnostic assessments. However, concerns regarding underreporting of PTSD and depression symptoms due to stigma have been well documented (Institute of Medicine, 2012). Continuing and enhancing command support for the RESPECT-Mil program, as well as other evidence-based programs for psychological health, may foster greater openness and disclosure of PTSD and depression among service members.

Explore expanding routine screening and evidence-based primary care management practices for depression and PTSD. Findings suggest that RESPECT-Mil is catching people who may have previously fallen through the cracks. Based on this observation, stakeholders and RESPECT-Mil providers recommended that routine screening for PTSD and depression should be expanded to all primary care settings.

Improving Referrals and the Management of Depression and PTSD in Primary Care

Increase primary care provider engagement and comfort. RESPECT-Mil provider discussions revealed that some primary care providers do not feel comfortable managing the mental health needs of service members. Identified concerns include fears of being held liable for adverse behavioral health outcomes and beliefs that PTSD should be handled in behavioral health. Ways to increase primary care provider engagement in the program and comfort with addressing behavioral health needs could include monitoring individual primary care provider performance, providing additional training with primary care champions, and strengthening the consultative relationship with behavioral health champions, as well as structural or cultural changes to the primary care environment that better allow for the time and effort needed to address mental health issues.

Incentivize and support primary care champions. Primary care champions face severe constraints and need to demonstrate productivity outside the RESPECT-Mil program. Opportunities should be expanded for incentivizing and supporting those in the champion positions so that they can continue to train, monitor, and assist primary care providers in maintaining fidelity to the program.

Consider whether modifications are needed given the range of symptom severity among service members referred to the program. Of the service members referred to and enrolled in RESPECT-Mil, fewer than half met criteria for a probable depression or PTSD diagnosis. Further investigation is needed to understand the reasons for referral and enrollment of service members with no or minimal depression or PTSD symptoms.

Strengthen the handoff between the primary care provider and the care facilitator. A significant proportion of service members with mild to severe depression or PTSD symptoms (38 percent) who were referred to the program never established contact with the nurse serving as the care facilitator. More than half of these service members either withdrew from the program or could not be engaged or contacted. It would be helpful to explore strategies to prevent service member dropout after the initial primary care referral, including training primary care providers to better orient and introduce service members to the program as well as providing warm handoffs within clinics.

Facilitate engagement and communication with service members. Service members with mild to severe depression and/or PTSD who successfully established their initial contact with nurse care facilitators have an average of 2.6 subsequent follow-ups with the facilitators. Moreover, during the period of enrollment in the program, only 46 percent of service members report starting any medication or counseling. Given that the level of treatment engagement is below optimal for a substantial proportion of service members, strategies for facilitating engagement and communication should be explored. This may include the use of newer technologies for communication (e.g., texting, social media) as well training nurse care facilitators in motivational interviewing strategies.

Enlist command in support of service members' treatment engagement and adherence while recognizing that some service members may want to keep their treatment confidential. According to provider and stakeholder discussions, barriers to treatment engagement include service member concerns about the potential negative repercussions on job performance and career advancement, as well as the lack of flexibility and support on the part of commanders to accommodate treatment requirements (e.g., modifying schedules to attend treatment appointments). Enlisting the support of commanders could play an integral role in creating incentives for service members to engage in and adhere to treatment. Avenues that can be explored include educating commanders on the effect of policies that discourage treatment seeking among service members, building collaborative relationships between commanders and primary care providers, and promoting commander awareness of the program via trainings delivered by behavioral health champions or primary care champions.

Fortify communication between providers. Based on discussions with providers, several areas of communication between them were identified as possible targets for improvement. To strengthen the coordination of care among providers, the following could be considered: Explore ways to integrate and streamline record management systems (e.g., Armed Forces Health Longitudinal Technology Application, FIRST-STEPS), expand venues for communication outside the medical record systems (e.g., colocation, cross-unit meetings focused on service member care), and identify strategies to ensure that primary care providers are obtaining feedback (positive and negative) about the service members they have referred to RESPECT-Mil.

Ensure that the behavioral health champion role is adequately supported. Barriers to carrying out the responsibilities of behavioral health champions include severe constraints on staff time, competing priorities related to their primary occupational responsibilities within behavioral health, and few incentives to participate in RESPECT-Mil. In order to enable behavioral health champions to perform optimally, consider ways to provide adequate and protected time for RESPECT-Mil duties, incentivize participation in the program, and ensure efficient staffing of cases.

Consider enhancing the behavioral champion role. Behavioral health champions were depicted as functioning positively in their role as informal consultants to primary care providers regarding the diagnosis and management of depression and PTSD. Consideration may be given to enhancing the behavioral health champion role in providing more-intensive support to primary care providers. Expansion of the behavioral health champion role may occur through changes in location, availability, and incentives. Behavioral health champion engagement may be especially important during the initial phases of implementation, when primary care providers are being trained in the program and their comfort and skill level in managing behavioral health issues are developing.

Improving Quality Assurance Monitoring

Augment individualized and real-time performance feedback. Currently, no apparent, routinized protocol is in place to provide primary care providers or behavioral champions with performance feedback on fidelity to the program. Care facilitator performance can be monitored via FIRST-STEPS, but the type and frequency of feedback provided are unclear. For example, performance feedback for care facilitators could include the rate at which service members are being successfully connected to medication and counseling, engaging in the full course of recommended treatment, and appropriately referred to other behavioral resources. The development of targets for optimal performance for each provider role will be important so that sites can gauge their own performance against target metrics.

Create incentives for sites and providers to buy into quality improvement processes. More routine, localized, on-site monitoring may increase ownership and investment in quality improvement processes. In addition, localized monitoring may facilitate more real-time and personalized feedback, which is more challenging to conduct when the monitoring of all Army sites is centralized.

Continue support for the RESPECT-Mil Implementation Team or similar centralized quality improvement programs. The RESPECT-Mil Implementation Team data collection efforts on clinic screening and referral practices and care management activities via FIRST-STEPS allow for valuable tracking on implementation fidelity as well as program effectiveness. This is in line with one of the major recommendations issued in an Institute of Medicine report, Treatment for Posttraumatic Stress Disorder in Military and Veteran Populations: Initial Assessment, which called for the DoD and the VA to “institute programs of research to evaluate the efficacy, effectiveness, and implementation of all their PTSD screening, treatment, and rehabilitation services” (Institute of Medicine, 2012, p. 13).

Establish a self-monitoring process for the RESPECT-Mil Implementation Team oversight efforts. Ongoing evaluations of the relative merit and impact of different monitoring strategies (e.g., site visits, site calls) may be beneficial for targeting which activities should be continued and supported, particularly in light of limited resources.

Implementation of RESPECT-Mil Within the Military Health System

Given that RESPECT-Mil is slated to be the model that is used to implement behavioral health treatment within the PCMH, careful consideration is needed to determine the aspects of RESPECT-Mil that add value and can be preserved within the PCMH. Continued monitoring and oversight of the RESPECT-Mil program and the PCMH will be necessary as these programs change and adapt over time, since they have similar goals but different structural elements.

Conclusions

The real-world implementation of RESPECT-Mil in Army primary care settings is comparable to other collaborative care efforts that are often conducted under more tightly controlled research conditions. As with other collaborative care efforts, RESPECT-Mil encountered significant implementation barriers. Challenges included establishing initial contact with service members on referral to the program, procuring service member engagement in the full course of recommended treatment, obtaining provider buy-in, provider time constraints and competing demands, and the provision of oversight and accountability to program fidelity. Factors that facilitated the implementation of RESPECT-Mil included valuing routine screening for depression and PTSD as an effective way to reach service members who may otherwise fall through the cracks; behavioral health champions' support and consultations with primary care providers; and solid linkages between primary care providers and care facilitators. Our findings highlight key junctures where opportunities for engaging service members in needed treatment for depression or PTSD may be improved. Potential avenues for improving program fidelity include increasing the comfort of primary care providers and incentives to address depression and PTSD within primary care settings, ensuring warm handoffs between the initial primary care referral and the care facilitator in order to protect against dropouts, equipping providers with additional skills and strategies to improve treatment engagement, and providing individualized provider performance feedback. Even if perfect program fidelity were achieved, barriers such as stigma and lack of leadership support for recommended treatment plans are unlikely to be completely overcome without corresponding increases in organizational and policy support. Recommendations issued in this report are targeted at the provider, clinic administration, and military organizational levels on how to improve the implementation of primary care collaborative care programs aimed at enhancing mental health care. Recommendations are relevant to efforts currently under way to usher in the PCMH by building on the foundations and infrastructure developed by RESPECT-Mil.