Using the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ) to identify practice patterns

Abstract

Primary care-mental health integration (PC-MHI) is growing in popularity. To determine program success, it is essential to know if PC-MHI services are being delivered as intended. The investigation examines responses to the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ) to explore PC-MHI provider practice patterns. Latent class analysis was used to identify clusters of PC-MHI providers based on their self-report of adherence on the PPAQ. Analysis revealed five provider clusters with varying levels of adherence to PC-MHI model components. Across clusters, adherence was typically lowest in relation to collaboration with other primary care staff. Clusters also differed significantly in regard to provider educational background and psychotherapy approach, level of clinic integration, and previous PC-MHI training. The PPAQ can be used to identify PC-MHI provider practice patterns that have relevance for future clinical effectiveness studies, development of provider training, and quality improvement initiatives.

Electronic supplementary material

The online version of this article (doi:10.1007/s13142-015-0325-0) contains supplementary material, which is available to authorized users.

INTRODUCTION

Integration of mental health services into primary care settings is on the rise worldwide [1]. Overarching goals of integrated care models include delivery of effective, efficient, accessible, and equitable mental health services [2]. Driving factors for their implementation have included the need to address notable gaps in service delivery and to reduce costs associated with mental health specialty services. Empirical evidence for the effectiveness of integrated care on patient outcomes has since been demonstrated in terms of improved patient functioning, symptomatic reduction, and increased access to mental health care [3–5].

Various approaches to integrated care exist, such as care management, co-located, collaborative, consultative, and staff advisory models, among others [6–8]. Two core models of primary care-mental health integration (PC-MHI) have existed within the Veterans Health Administration (VHA) since national implementation in 2007–2008 [9]: care management and co-located, collaborative care (CCC). Care management is designed as an algorithm-based platform of care that revolves around ongoing assessment and monitoring of patient needs, as well as promoting treatment adherence, patient education, and referral management. Care management approaches supported in VHA are the Behavioral Health Laboratory [10], and Translating Initiatives in Depression into Effective Solutions [11].

Whereas care management services are often telephone-based and delivered by nursing staff, CCC providers are embedded in multidisciplinary primary care clinics. CCC providers support patients by providing brief assessment and intervention, including facilitating referral to specialty mental health care for those patients with more severe or complex conditions. Services are often delivered same day, coordinated by an in-person transition of care (i.e., a warm handoff) from primary care providers or patient self-referral. CCC providers also directly support primary care providers by following up on positive mental health screens, holding conjoint appointments, educating providers on assessment and treatment of mental health concerns, and collaborating in comprehensive treatment planning, which often includes brief behavioral interventions appropriate for the primary care setting. Ideally, PC-MHI providers are equipped to perform both care management and CCC functions. However, significant local variation in PC-MHI implementation exists across VHA sites nationally, with providers engaging in CCC-only, care management-only, or combined CCC and care management functions.

PC-MHI program efforts have typically emphasized program-level implementation at either the national or local level [12, 13]. Similarly, there have been multiple detailed descriptions of patients served within various PC-MHI models [9, 14, 15]. Yet, relatively little attention has been paid to the assessment of PC-MHI provider behaviors, such as their level of adherence to either care management or CCC. Assessment of adherence, or the degree to which providers engage in model-specific tasks or procedures, is integral to successful program implementation and comprehensive evaluation [16]. Indeed, such assessment is necessary to interpret a variety of patient, provider, and clinic-level outcomes, as well as for quality improvement initiatives. Simply put, it is difficult to gauge the success of a health service delivery program without knowing whether it was delivered as intended.

The lack of literature on adherence among PC-MHI providers is in part attributable to historical challenges in measurement. Recently, we developed and validated the Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ), a provider-focused self-report measure designed to evaluate adherence to CCC practice parameters [17, 18]. Despite some overlapping aspects between CCC and other integrated care models (e.g., care management), the PPAQ was specifically designed to focus on measurement of CCC because it is the most widely adopted approach within VHA nationally. The PPAQ was based on a conceptual framework for implementation fidelity developed by Carroll and colleagues [16] which uses the terms “fidelity” and “adherence” interchangeably. This framework suggests that provider adherence is the best indicator of the dose of an intervention’s active ingredients that is delivered to patients. This framework also identifies a multitude of factors that impact provider adherence: the organizational and political context of the clinic setting, the level of complexity of the intervention, provider acceptability of the intervention, availability of strategies (e.g., manuals or protocols, training, audit-and-feedback) to support high-quality delivery of services, and the nature of the strategies used to enroll patients in care [16, 19].

Although previous investigations have ensured the PPAQ’s validity and reliability, detail on its utility in quantitatively discerning practice variation among providers has yet to be documented. Therefore, this study conducted a latent class analysis of data from our prior validation study [18] in order to more closely examine naturally occurring variability in provider adherence. Specifically, the primary aim of this study was to compare clusters of PC-MHI providers based on self-reported levels of adherence measured by the PPAQ. To better understand the nature of these clusters, secondary aims of this study included the following: identification of PPAQ items with uniformly high and low adherence across clusters (aim 2), comparison of clusters by PPAQ practice domains representing broader components of CCC practice (aim 3), and comparison of clusters by provider background and clinic setting (aim 4).

METHODS

PPAQ validation study

The methods of our validation study were described in detail previously [18]. Briefly, our aim was to conduct a psychometric evaluation of the PPAQ with a sample of PC-MHI providers in either CCC-only roles or hybrid CCC and care management roles. Five-hundred and eighty participants were contacted by e-mail and invited to complete an anonymous Web-based survey. Of those contacted, 173 VHA mental health providers (30 % response rate) who provided clinical services in primary care for at least 25 % of their duties in 2012 participated in the study. Analysis revealed that the internal consistency reliability of PPAQ subscales (alpha = 0.70 to 0.92) was sufficient to warrant validity assessment. In line with our a priori hypotheses regarding differences in adherence scores based on provider and setting characteristics, higher adherence scores were noted among providers who (1) identified cognitive behavioral therapy as a primary psychotherapy approach, (2) worked in clinics at VHA medical centers versus VHA community-based outpatient clinics, and (3) reported having adequate local resources to engage in CCC. Convergent and divergent validity was demonstrated via correlations with a clinic-level measure of the degree of mental and behavioral health integration of their primary care setting (r = 0.35–0.55). This study was approved by the VA Western New York Healthcare System Institutional Review Board.

Participant characteristics

Providers were from a range of educational backgrounds, including doctoral level psychologists (46 %), master’s level social workers (24 %), nurses (19 %), or psychiatrists (11 %). They endorsed a range of psychotherapy approaches, including cognitive behavioral therapy (60 %), humanistic or client-centered (15 %), interpersonal (8 %), and other (18 %). On average, participants reported 2.7 (SD = 1.8) prior training experiences related to PC-MHI, such as graduate coursework or supervised clinical experience. Participants were primarily working in medical centers (68 %) versus outpatient clinics (32 %). All participants provided mental health services in primary care, with the majority noting their roles as exclusively CCC (48 %) or both CCC and care management (43 %). Participants reported that they had worked in their current clinical role for 2 or fewer years (29 %), 3 to 4 years (36 %), or 5 or more years (35 %).

Measures

Provider background survey

This brief survey requested information from participants on their professional background, selected characteristics of their VHA clinic setting, and training experiences related to integrated care.

Primary Care Behavioral Health Provider Adherence Questionnaire (PPAQ)

The PPAQ is a 48-item scale designed to assess mental health provider adherence to the CCC model [17]. The details of the initial development of PPAQ have been described previously [17]. The PPAQ prompts participants to self-rate how frequently they engaged in 48 CCC behaviors across domains of clinical practice, from assessment and intervention to collaboration with primary care providers. The PPAQ uses a 5-point, Likert-type response scale ranging from “never” to “always.” The PPAQ includes essential items, which measure behaviors that are highly consistent with the CCC model, and prohibited items, which measure behaviors that are inconsistent with the model. As noted above, the psychometric properties of the PPAQ have been reported previously [18]. For this secondary analysis, emphasis was primarily on interpretation of the 48 individual prohibited and essential items to assess specific adherence behaviors. To aid in interpreting individual items, we first applied a cutoff of four to identify scores at the optimal level of adherence (i.e., engaging in essential behaviors “often” or “always” or engaging in prohibited behaviors “rarely” or “never”). Additionally, because of the large number of individual items, we organized similar PPAQ items by the larger content domains of CCC practice they represented: Clinical Scope and Interventions (e.g., “During clinical encounters with a patient, I implement behavioral and/or cognitive interventions”); Practice and Session Management (e.g., “During clinical encounters with patients, I address the primary care provider’s reason for referral”); Referral Management and Care Continuity (e.g., “I provide advice to the primary care team about appropriate referrals to specialty behavioral health services”); and Consultation, Collaboration, and Interprofessional Communication (e.g., “I meet briefly with primary care staff as a team to provide both a behavioral health perspective and behavioral data”) [17]. The PPAQ items are listed in Table 1 of the electronic supplemental material.

Level of integration measure

The Level of Integration Measure provides a reliable assessment of the level of behavioral and physical health integration in primary care [20]. The Level of Integration Measure consists of 35 items rated on a 5-point, Likert type scale ranging from “strongly disagree” to “strongly agree,” with higher scores suggesting higher levels of integration. It includes items that address clinic system integration, provider beliefs, commitment to integrated care, clinical practices, interdisciplinary alliances, training and consultation, and leadership. This measure was included in the current study in order to determine if cluster membership was correlated with the degree of clinic integration.

Statistical analysis

Latent class analysis was utilized to identify clusters of adherence behaviors based on observed responses from 48 individual PPAQ prohibited and essential items. The number of clusters of adherence was selected by running sequential models (1, 2, 3 clusters) and comparing the Bayesian Information Criterion statistic. The model with the lowest Bayesian Information Criterion statistic implies the best fit to the observed data. Other criteria such as classification error, cluster size, and graphs were additionally used to determine the most informative model [21].

Each participant was assigned to the cluster with the highest postposterior probability. Cluster membership was then compared with background characteristics using Pearson’s chi-square test. Fisher’s exact test was used for comparisons with cell counts of less than five. Cluster membership was compared with Level of Integration Measure scores using analysis of variance. Low precision due to small cell sizes prohibited analyses beyond bivariate associations (e.g., multiple variable regression model).

RESULTS

Aim 1: cluster identification and PPAQ item-level comparisons

Five clusters, ranging in size from 21 to 49 providers each, resulted in the best fitting model. Figure 1 displays clusters by individual PPAQ item means. Item means are organized by domain to improve interpretability. Overall, this display illustrates that scores follow generally similar patterns within domains, yet differ across clusters by magnitude of adherence. Means in Clinical Scope and Interventions show the least separation by cluster (i.e., cluster lines are closest to each other) and the highest degree of adherence in relation to the optimal cutoff score (i.e., the most items fall at or above the optimal four point cutoff). In contrast, means in Consultation, Collaboration, and Interprofessional Communication show the most notable separation across clusters while also indicating the lowest degree of overall adherence. Clusters 4 and 5 appear to be the highest and lowest adherence groups, respectively, regardless of domain.

Fig 1.

PPAQ item means by cluster. Notes: Bold horizontal line indicates the four-point cutoff for optimal level of fidelity. CSI Clinical Scope and Interventions; PSM Practice and Session Management; RMCC Referral Management and Care Continuity; CCIC Consultation, Collaboration, and Interprofessional Communication

Aim 2: identification of PPAQ items with uniform adherence

Examination of PPAQ items also revealed which CCC behaviors were more consistently adhered to than others, regardless of cluster assignment. As shown in Table 1, uniformly high adherence, or mean scores at or above the four point cutoff in all clusters, was endorsed by the sample for five (10 %) of PPAQ items (i.e., 4, 18, 20, 42, and 45). Scores on these five items reflect optimal adherence to a variety of essential and prohibited behaviors, such as being certain to address the primary care provider’s reason for referring the patient for mental health services (i.e., item 45). Uniformly low adherence, or average scores below the four point cutoff in all clusters, was indicated for 14 (30 %) of PPAQ items (i.e., 1, 10, 11, 16, 22, 24, 25, 26, 29, 31, 37, 38, 43, and 48). These items appear to be the key behaviors to which all providers find it challenging to adhere. Six of the uniformly low adherence items (i.e., 1, 16, 22, 24, 37, and 43) focus on delivering time-limited, brief mental health services. Low adherence on these items suggests that participants in our sample (1) struggled to provide encounters of 30 min or less, (2) typically spend more than 30 min documenting encounters, and (3) see patients for 10 or more encounters per episode of care. The remaining eight low adherence items are diverse in content, but primarily include behaviors related to interprofessional communication and integration with the primary care team.

Table 1.

PPAQ items with uniformly high or low adherence

High adherence		Low adherence
Item summary	Mean	Item summary	Mean
18. Clarifies, confirms, and discusses patient concerns	4.72	37. Sees patients for ≥ 10 appointments per episode of care (reverse scored)	3.56
45. Addresses Primary Care Providers’ reason for referral	4.70	29. Includes findings from assessments and screening in progress notes	3.52
4. Accepts referrals for patients with common mental health problems	4.56	10. Includes focused recommendations for the Primary Care Provider/team	3.45
42. Provides long-term (i.e., greater than 8 sessions) group therapy (reverse scored)	4.39	38. Accepts referrals for issues specific to older patients.	3.45
20. Provides full neuropsychological, cognitive, or personality assessments (reverse scored)	4.36	1. Sees patients for ≤ 30 min	3.44
		43. Meets with patients for ≥50 min to gather history (reverse scored)	3.44
		25. Uses local community resources	3.13
		22. Explains that appointments will be ≤30 min	3.12
		31. Consults with Primary Care Providers	3.10
		24. See patients for 50-min appointments (reverse scored)	3.04
		26. Provides education to the primary care team on behavioral health issues	2.89
		11. Meets with primary care team to provide a behavioral health perspective/data	2.79
		16. Takes ≥30 min to complete documentation (reverse scored)	2.72
		48. Participates in clinical pathways for common health conditions	2.50

Aim 3: differences in PPAQ domains across clusters

Having examined the differences in clusters at the item level, we then examined cluster differences in relation to level of adherence by PPAQ content domains (Fig. 2). Content domain scores for the full sample suggested moderately high adherence, with scores for domains Clinical Scope and Interventions, Practice and Session Management, and Referral Management and Care Continuity approaching the four-point cutoff (denoted with a bold horizontal line) indicating the optimal level of adherence. Adherence was poorest in the Consultation, Collaboration, and Interprofessional Communication domain. Similar to the full sample, content domain scores for clusters 1 and 3 scored close to the four-point cutoff. Cluster 5 typically showed the lowest levels of adherence for all domains, but particularly Consultation, Collaboration, and Interprofessional Communication. In contrast, clusters 2 and 4 showed generally higher levels of adherence, with cluster 4 as the only group of participants scoring in the optimal range of adherence across domains.

Fig 2.

PPAQ domain means, full sample, and by clusters. Notes: Bold horizontal line indicates the four-point cutoff for optimal level of fidelity. CSI Clinical Scope and Interventions; PSM Practice and Session Management; RMCC Referral Management and Care Continuity; CCIC Consultation, Collaboration, and Interprofessional Communication

Aim 4: variations in adherence by provider and setting characteristics

Table 2 provides an overall depiction of how clusters varied by provider background and clinic setting characteristics. There were significant associations between cluster assignment and psychotherapy approach (p = 0.002), with fewer providers endorsing a cognitive behavioral therapy approach in cluster 5 (19 %) than in other clusters (61.2 to 67.7 %). Provider educational background was also associated with cluster assignment (p = 0.004), with the highest percentage of psychologists in cluster 2 (62 %), the highest percentage of social workers in cluster 4 (42 %), and the highest percentage of nurses and physicians in cluster 1 (47 %). However, there was no significant association between cluster and practice setting, provider role, or length of time in current position. The Level of Integration Measure was strongly associated with cluster assignment, with total scores significantly higher in cluster 4, indicating a higher level of integration, compared to clusters 2 or 5 (p = 0.001). In terms of PC-MHI-related training experiences, total number of experiences was not associated with cluster assignment, with each cluster averaging about 2.3 to 2.9 prior trainings. Attendance at a VA Center for Integrated Healthcare workshop training was associated with cluster assignment (p = 0.005) as was completion of an integrated care certification course (p = 0.012), although only about 5 % (n = 8) of the sample reported this certification. There was no association between clusters and the remaining sources of integrated care training, including completion of graduate level coursework, clinical rotations, postdoctoral training, mentorship or clinical supervision, independent readings, or attendance at other various integrated care workshops.

Table 2.

Comparisons of cluster means by provider background characteristics

Characteristics	Cluster					p
	One n = 49	Two n = 39	Three n = 33	Four n = 31	Five n = 21
CBT orientation	30 (61)	26 (67)	22 (67)	21 (68)	4 (19)	0.002
Provider education						0.004
PhD	21 (43)	24 (62)	18 (55)	10 (32)	7 (33)
SW	5 (10)	10 (26)	6 (18)	13 (42)	7 (33)
Nurse/physician	23 (47)	5 (13)	9 (27)	8 (26)	7 (33)
Practice setting—VAMC	35 (71)	24 (62)	16 ( 48)	23 (74)	14 (67)	0.185
Provider role—CCC only	23 (47)	23 (59)	19 (58)	11 (35)	7 (33)	0.143
Years in role
≤2	15 (31)	10 (26)	9 (27)	8 (26)	8 (38)	0.803
2–4	19 (39)	17 (44)	10 (30)	9 (29)	7 (33)
>4	15 (31)	12 (31)	14 (42)	14 (45)	6 (29)
LIMa	127 (17)	112 (21)	126 (17)	139 (16)	112 (20)	<0.001
Total traininga	2.8 (1.6)	2.3 (1.9)	2.9 (2)	2.8 (1.5)	2.6 (1.9)	0.626
CIH workshop	17 (35)	7 (18)	17 (52)	18 (58)	7 (33)	0.005
Integrated care certification	0 (0)	0 (0)	4 (12)	2 (6)	2 (10)	0.012

^aValues presented are n (%) except for LIM and total training presented as mean (standard deviation)

CBT cognitive behavioral therapy, VAMC Veterans Affairs Medical Center, CCC colocated, collaborative care, LIM Level of Integration Measure, CIH Center for Integrated Healthcare

DISCUSSION

Variation in provider adherence to the CCC model within a real-world context is to be expected given PC-MHI providers’ broad scope of duties. However, it has been difficult to both quantify these differences and describe the precise nature of these differences without appropriate measurement. Our results demonstrate that latent class analysis can provide a detailed description of provider practice patterns using self-report data captured with a psychometrically valid questionnaire, the PPAQ. The novel application of latent class analysis allowed us to avoid traditional approaches often based on averaging scores across items and responses to identify high and low adherence groups, which can mask more nuanced patterns within each set of providers’ responses. This method allowed us to maintain each individual’s set of responses to the PPAQ items and then determine where clusters exist through the use of probabilities estimated directly from the model [22].

The majority of providers within our sample demonstrated moderate—if suboptimal—levels of adherence with the highest levels of adherence among a single cluster representing only 18 % of our sample. These findings are not surprising as there have been calls for more integrated care training programs to help prepare early career professionals for these types of positions [23, 24]. In addition, the national implementation of PC-MHI occurred quickly across the last few years within the VHA [25], which led to many providers transitioning from positions in traditional specialty mental health care to PC-MHI. In our prior qualitative work, we found that PC-MHI providers tended to develop local models of CCC that were dependent upon both their level of knowledge regarding population-based models of mental health care and local resource limitations, such as provider staffing levels and physical layout of the clinic [26]. Thus, both provider and contextual factors came together to determine the degree of provider fidelity.

The five clusters of practice patterns we observed among PC-MHI providers generally followed similar patterns within the four content domains of the PPAQ. The 11 items from the PPAQ that were identified within the Clinical Scope and Interventions domain demonstrated the highest degree of reported adherence and the least separation by cluster. This finding suggests that although this sample of providers had different professional backgrounds (i.e., training and prior experiences) and work in different primary care settings, they were generally able to adhere to and perform regularly the expected CCC elements focused on routine clinical interventions, such as conducting suicide risk assessments. In contrast to the Clinical Scope and Interventions domain, providers in this sample appeared to have the most separation across clusters and the lowest degree of adherence overall for the Consultation, Collaboration, and Interprofessional Communication domain, which includes elements such as providing routine feedback to primary care providers or consulting with them regarding medical aspects of mental health conditions. Whereas the Clinical Scope and Interventions domain is entirely dependent on providers’ clinical skills, behaviors in the Consultation, Collaboration, and Interprofessional Communication domain reflect providers’ interpersonal skills and ability to engage with health care providers of other disciplines. Achieving high levels of interprofessional collaboration remains a challenge for primary care teams, and there are only scant resources to assist PC-MHI providers with team integration that so often depends on learning the culture and language of primary care [27–29]. The warm handoff, for example, is often seen as a critical aspect of high-quality integrated care that depends on effective communication and trust across providers. Yet, handoff-type transitions have only recently been examined closely to understand the nature of these transitions and to develop suggestions for how to maximize their utility [30].

When assessing the associations among the level of clinic integration, provider background characteristics, and provider clusters, our data confirmed our prior hypothesis that lower adherence (e.g., cluster five) would be associated with a lower percentage of providers with cognitive behavioral therapy orientation. This finding suggests that additional training or support in conducting cognitive and behavioral interventions may assist PC-MHI providers in becoming more adherent. Additionally, our finding is consistent with other efforts in VHA to improve the implementation of cognitive behavioral therapy in primary care thereby increasing the availability of evidence-based mental health treatments [31–33]. Although our validation study found that, on average, psychologists evidenced higher levels of adherence [18], the most adherent cluster (i.e., cluster 4) had the largest number of social workers compared to the other clusters. The professional training obtained by social workers may serve as a good foundation to becoming highly adherent PC-MHI providers. Social workers may also have an advantage over other providers because of their emphasis on providing holistic care to patients and knowledge of how to navigate large health care systems to connect patients to a wide range of services [26]. The most adherent cluster was also associated with a higher level of clinic integration as measured by the Level of Integration Measure. It is possible that clinics which are highly integrated offer a number of contextual features that promote high levels of adherence among PC-MHI providers, but this association should be examined in future work.

LIMITATIONS

Limitations to this study include a potential for PC-MHI providers to over-report their behaviors on the PPAQ in order to appear highly adherent due to social desirability bias [34], despite the anonymity of our survey approach. However, we feel that this is a minor concern as we still found a large number of PC-MHI providers reporting behaviors that were only on the cusp of preferred performance. Further validation of the PPAQ using objective measures of fidelity would strengthen assertions regarding the validity of the PPAQ. Direct observation of the full scope of CCC as described in the PPAQ remains challenging. CCC practice includes a wide range of behaviors directed at not only the patient but also the primary care team. Thus, traditional methods of observing clinical interventions in clinical trials, such as audio recording patient-provider interactions, are likely inadequate for assessing the full CCC platform of care. Because this study was a secondary data analysis, we could not assess a wider range of specific contextual factors (e.g., adequate CCC provider staffing levels, mental health and primary care leadership support, provider and patient acceptability) that nonetheless could contribute to provider self-assessments of adherence. Thus, this study focused primarily on provider characteristics documented previously to be related to adherence [18]. The PPAQ was developed with input from integrated care experts from VHA, Department of Defense, and academic health centers to ensure its potential relevance outside of VHA. However, the use of only VHA providers in our sample does restrict the generalizability of our current study findings to non-VHA settings. Future research would benefit from including a larger number of providers across federal and non-federal health care settings who have had a wider range of training experiences through integrated care certification courses or doctoral training programs.

STUDY IMPLICATIONS

Overall, this study’s results have several broad implications. The most immediate relevance of our findings is for directing future clinical effectiveness research that needs to be conducted to examine the impact of PC-MHI on clinical outcomes, such as patients’ distress, functional impairment, and quality of life. Because provider adherence often moderates the impact of an intervention on outcomes, it is essential to quantify the magnitude of variation in provider practices. The PPAQ was originally designed to provide an estimate of adherence based on assessment of a provider’s engagement in both essential and prohibited practices. The current study adds to previous work by demonstrating that the PPAQ can identify discrete clusters of providers, suggesting an alternative method by which to assess the relation between adherence and outcome. Furthermore, examination of consistencies across clusters also revealed that 40 % of the PPAQ items representing specific CCC functions did not appear to be helpful in differentiating across providers, even though they were deemed to be content valid in previous work. Therefore, future research will need to determine if the use of the full PPAQ subscales scores, a modified score derived from excluding those items with very high or low uniformity, or use of latent class analysis-derived clusters may be the ideal approach when examining the moderating effect of provider’s adherence on clinical outcomes.

Study findings also have practice implications regarding refining system-wide efforts to address gaps in practice across a broad array of PC-MHI providers. Although it will be important to continue to monitor adherence to the uniformly high items on the PPAQ, future provider training may potentially be able to de-emphasize support in areas where providers are readily able to achieve high adherence. Targeting the areas of lowest adherence may assist in prioritizing how PC-MHI implementation strategies are delivered. Based on the theoretical framework that guided this study [16], provider fidelity is unlikely to be optimized by addressing provider education alone. For example, PC-MHI providers could clearly benefit from multiple support strategies regarding overcoming barriers to integration with primary care teams. The use of provider tools and additional training in interprofessional communication may be important, but achieving high levels of clinic integration likely requires that the larger organizational context be addressed through improved leadership support, articulation of shared goals across teams, and systematic quality improvement efforts [35].

As PC-MHI evolved in response to a national mandate within the VHA [25], our study results provide some insight into PC-MHI implementation status nationally. Overall, PC-MHI providers appear to be adhering to a number of key elements of integrated care, particularly in the domain with the most immediate impact on patients (i.e., Clinical Scope and Interventions). These findings provide empirical evidence to support the continued implementation of policy that aims to improve the availability and uniformity of mental health services in VHA facilities. Similarly, these findings and those from continued research into the effectiveness of integrated care programs hold potential to begin to inform the development of PC-MHI practice guidelines. PC-MHI practice guidelines may be especially beneficial in addressing undesirable variations in care stemming from idiosyncratic provider preferences, traditions in clinical practice, or lack of knowledge regarding evidence-based standards [36].

In summary, by applying latent class analysis to data captured by the PPAQ, we were able to identify subgroups of providers based on their levels of adherence to specific practice domains using a fully data-driven approach. These subgroups are a first step in characterizing the significant variation in practice patterns that exist nationally in the VHA’s PC-MHI efforts and have potential to direct future activities to support frontline providers. The ultimate utility of applying these practice patterns to the study of clinical outcomes will need to be tested in future research.