Measuring the integration of primary care and behavioral health services

Abstract

Objective

To perform a factor analysis of the Practice Integration Profile (PIP), a 30‐item practice‐level measure of primary care and behavioral health integration derived from the Agency for Healthcare Research and Quality's Lexicon for Behavioral Health and Primary Care Integration.

Data Sources

The PIP was completed by 735 individuals, representing 357 practices across the United States.

Study Design

The study design was a cross‐sectional survey. An exploratory factor analysis and assessment of internal consistency reliability via Cronbach's alpha were performed.

Data Collection Methods

Participant responses were collected using REDCap, a secure, web‐based data capture tool.

Principal Findings

Five of the PIP's six domains had factor loadings for most items related to each factor representing the PIP of 0.50 or greater. However, one factor had items from two PIP domains that had loadings >0.50. A five‐factor model with redistributed items resulted in improved factor loadings for all domains along with greater internal consistency reliability (>0.80).

Conclusions

Five of the PIP's six domains demonstrated excellent internal consistency for measures of health care resources. Although minor improvements to strengthen the PIP are possible, it is a valid and reliable measure of the integration of primary care and behavioral health.

1. INTRODUCTION

Primary care settings are integrating behavioral health providers in greater numbers using a variety of clinical models.1 One model, Primary Care Behavioral Health (PCBH), calls for a Behavioral Health Consultant (BHC) to be included as a full member of the care team and for the BHC to provide treatment according to a primary care model of brief, focused, high‐volume interventions instead of conducting traditional independent psychotherapy sessions.2, 3, 4 Behavioral Health Consultants in this model are available for same‐day interventions and educate the rest of the team to improve behavioral health knowledge and skills. Another model, the Collaborative Care Model (CoCM) of integration focuses on strengthening the connections between primary care and specialty mental health providers through the employment of care managers who work with patients and providers to foster shared focus on problems and treatments. The care managers provide ongoing follow‐up, monitoring, and liaison functions to connect primary care providers with consulting psychiatrists. The Collaborative Care Model often focuses narrowly on patients with conditions such as depression or anxiety.5, 6 These models of integrating behavioral health services into primary care practices are distinct from approaches that add primary care services to a community mental health center, sometimes referred to as “reverse integration” or “bidirectional health homes.”7

Quality and performance metrics have been used to guide primary care and behavioral health integration, but provide inadequate guidance for integrating care.8 For example, the Center for Medicare and Medicaid Services has only one minimum quality standard for Accountable Care Organizations related to behavioral health: screening for depression.9 Patient Centered Medical Home (PCMH) recognition by the National Committee for Quality Assurance (NCQA) goes a bit further, requiring comprehensive behavioral health assessments and monitoring of clinical quality measures for behavioral health.10 National Committee for Quality Assurance recognition as a PCMH with Distinction in Behavioral Health Integration requires practices have the capacity to address behavioral health needs, including screening and brief intervention. To date, five out of 14,500 practices have achieved this distinction;11 however, the range of allowable evidence supporting this distinction allows for substantial variability. The PCBH and CoCM models of integrated behavioral health reflect some overlap with each other and NCQA's PCMH criteria. They also have important differences. Nevertheless, all three are often referred to as “integrated care.”12, 13

In 2008, the Agency for Healthcare Research and Quality concluded “the field had not identified the core elements of successful integration.”14 Little has changed since that report. There are many checklists assessing integrated care including The Integrated Practice Assessment Tool15 which is used by the Substance Abuse and Mental Health Services Administration. The AHRQ's Integration Academy has developed an Atlas of Integrated Behavioral Health Care Quality Measures.16 Unfortunately, none of these resources have been psychometrically tested, leaving the field without a validated, reliable, measure of primary care and behavioral health integration.17

1.1. Lexicon for behavioral health and primary care integration

In 2013, AHRQ published the Lexicon for Behavioral Health and Primary Care Integration: Concepts and Definitions Developed by Expert Consensus. The Lexicon is a set of concepts and definitions developed by expert consensus aimed to provide “a common definitional framework for building behavioral health integration as one important way to improve health care quality.”18 The Lexicon's approach to describing integrated behavioral health is distinct from other approaches such as the SAMHSA‐HRSA (2013) Standard Framework for Integrated Care.19 The Lexicon includes a set of defining core concepts to define what is meant by integrated behavioral health. Six paradigm, case‐defining clauses map similarities and differences in integrated behavioral health to twelve corresponding parameters that highlight how one instance of integrated behavioral health might differ from another on structural and procedural dimensions such as type of work space arrangement, method for identifying patients in need of services, and degree to which protocols are followed.

Although the Lexicon was a major advance, there is still little consensus about the ideal financial, operational, or clinical characteristics of ideal primary care and behavioral health integration. The AHRQ Lexicon of Collaborative Care provides a broadly accepted theoretical construct of integration; however, it does not provide a method to reliably measure integration. Without a measurement tool, the field struggles to compare results of integration efforts across practices or provide a standard that specifies necessary integration components. Policy makers, administrators, and researchers could benefit from using standard measures, or descriptors of integration.

1.2. The Practice Integration Profile

Beginning in 2013, the authors assembled a group of national integration clinicians and researchers to develop an instrument for measuring the integration of behavioral health services into primary care practices, the Practice Integration Profile (PIP).20 The PIP was tested for construct validity in a convenience sample of 152 clinics in 35 states and found to accurately distinguish among clinics with varying degrees of integration assessed by other standards.21 Rater agreement was consistently high, with total integration scores differing by an average of 7.1 points out of 100. A subset of raters repeated tests to confirm inter‐rater reliability over time. A separate set of providers completed an evaluation of four hypothetical clinic settings based on degree of integration, resulting in a high level of correlation with degree of integration (P = 0.0005). Overall, the PIP was found to be reliable and feasible, with good face validity, low response burden, and good discrimination.21, 22, 23 The PIP is designed to allow for meaningful comparisons between practices and to aide in the identification of integration activities that positively impact health and other meaningful outcomes. Although the PIP's association with clinical outcomes is not known, the measure provides an efficient, repeatable, and actionable evaluation of clinical structure and process. At this time, more than 1,300 respondents from more than 600 practices of varying size, ownership, and geographic location have completed the PIP.

1.3. Objectives

We conducted a factor analysis to accomplish two goals. First, we sought to confirm the extent to which the PIP's six domains are discrete, observable phenomenon in a large sample of primary care practices. Second, we sought additional data for a potential revision to improve the PIP's ability to measure differences between primary care practices and changes within a practice over time.

2. METHODS

2.1. Data source

A convenience sample from primary care practices across the United States completed the PIP between September 2014 and January 2017. Members of these practices were invited to complete the PIP by direct email invitation, email listserv announcements, and snowball sampling. The instrument was also made available on a public website where respondents could find and complete the survey (www.practiceintegrationprofile.com). The raters were primary care providers, behavioral health providers, nurses, and practice administrators. Study data were collected and managed using REDCap, a secure, web‐based electronic data capture tool hosted at the University of Vermont.24

2.2. Measures

The PIP has 30 items, each comprised of three parts (Table 1). The first part is the “stem.” All stems lead off with, “In our practice…” followed by a statement (e.g, “…we use registry tracking for patients with identified BH issues”). The second part provides an example of the stem (e.g, “Insomnia registry”). The third part is the scoring criteria (e.g, “Numerator = # of patients in BH registries” and “Denominator = # of patients with BH needs”). Most stems are followed with the response option of “None: 0 percent,” “Never: 0 percent,” “Some: 1‐33 percent,” “About half: 34‐66 percent,” “Most: 67‐99 percent,” and “All: 100 percent.” Further description of stems, examples, and scoring have been published.21

Table 1.

Sample practice integration profile item

In our practice, …	Example	Scoring criteria	Score
We use registry tracking for patients with identified BH issues.	Insomnia registry	Numerator = # of patients in BH registries Denominator = # of patients with BH needs	Never 0%	Sometimes 1%‐33%	Often 34%‐66%	Frequently 67%99%	Always 100%

Each of the 30 items belongs to one of six domains: Practice Workflow, Clinical Services, Workspace Arrangement and Infrastructure, Integration and Sharing Methods, Case Identification, and Patient Engagement. Domain sizes range from two items (i.e, Work Arrangement and Infrastructure) to nine items (i.e, Clinical Services). For each domain, participants are given a score from 0 (no integration) to 100 (full integration). The Total Integration Score is the unweighted numeric average of the six‐domain scores.

Respondents completing the PIP also provided demographic information including practice name, location, practice type (Community Health Center, Pediatrics, Family Medicine, Internal Medicine, Other), practice size, respondent's position in the practice, and length of time the practice has been integrating primary care and behavioral health services. Participants were not compensated for their efforts. The University of Vermont IRB deemed this study nonhuman subjects research and exempt from review.

2.3. Data analysis

2.3.1. Descriptive statistics for practices in sample

Frequency distributions were created for key practice characteristics, such as practice type, practice size, position in practice, and integration efforts. Respondent data are missing for three questions, since some descriptive questions were added after the survey was activated online. Hence, data measuring stage of the integration effort were not ascertained for 77 respondents (10.5 percent).

2.3.2. Factor analyses and internal consistency reliability

An initial exploratory factor analysis using SPSS version 24 assessed the fit of PIP survey items into key domains (IBM Inc., Armonk, NY, USA). The factor analysis used default settings to determine the number of factors (i.e, retaining factors with eigenvalues >1). Additional factor analyses were conducted to match the six conceptual domains of the PIP and explore the extent to which alternate domains may more accurately describe practice integration. Cronbach's alpha coefficients were calculated for all composite measures in factor analyses to assess internal consistency reliability.25

3. RESULTS

3.1. Description of sample of practices included in analysis

A total of 735 respondents from 357 unique practices completed the PIP (Table 2). The majority of respondents represented community health centers (27 percent) or family medicine practices (41 percent). Most of the practices had more than 10 employees (85 percent). Two out of three respondents did not report the NCQA level of their practices. Practices were located in diverse settings, including almost one‐third from urban locations, about one‐quarter, respectively, from suburban and rural locations, and 14 percent from inner cities. Integration has been active in nearly half (46.9 percent) of the practices for more than 1 year. Over one‐half of respondents (54.0 percent) reported that a behavioral health provider was employed by the practice or practice organization (as opposed to functioning as an outside contractor).

Table 2.

Descriptive statistics for respondents included in analyses

Characteristic	N	Percent
Practice type
Community Health Center	201	27.3
Pediatrics	33	4.5
Family Medicine	304	41.4
Internal Medicine	75	10.2
Other Specialty Medical Practice, OB/Gyn	117	15.9
NCQA level
Do not Know	483	65.7
Level 1	9	1.2
Level 2	57	7.8
Level 3	133	18.1
No NCQA Level	53	7.2
Size
Less than 5 employees	48	6.5
6 to 10 employees	61	8.3
10+ employees	626	85.2
Position in the practice
Managing Director	93	12.7
Senior Behavioral Health Clinician	102	13.9
Managing Physician	110	15.0
Behavioral Health Clinician	74	10.1
Physician	65	8.8
Nurse	30	4.1
Student Intern	69	9.4
Practice Manager	38	5.2
Administration	78	10.6
Other Clinical Staff (Nurse Practitioner, Physician Assistant, or Medical Assistant)	26	3.5
Did not respond	50	6.8
Practice location
Inner City	104	14.1
Urban	237	32.2
Suburban	189	25.7
Rural	189	25.7
Frontier	16	2.2
Length of time integration effort has been active
Not ascertained	77	10.5
Do not have a behavioral health clinician in our practice	71	9.7
Less than 6 mo	51	6.9
More than 6 mo to 1 y	72	9.8
More than 1 y	345	46.9
Status of behavioral health clinician
Not ascertained	82	11.2
Employed by the practice or practice organization	397	54.0
Contracted with the clinician	27	3.7
Contracted for services with a different organization	90	12.2
Do not have a behavioral health clinician in our practice	139	18.9
How long has BH clinician been part of the practice?
Not ascertained	82	11.2
Do not have a behavioral health clinician in our practice	186	25.3
Less than 6 mo	39	5.3
6 mo to 1 y	59	8.0
1‐2 y	55	7.5
More than 2 y	298	40.5
Do not know	16	2.2

Notes: Some respondents did not provide responses to some items reported above. For this reason, the totals for each category in Table 2 vary.

NCQA, National Committee for Quality Assurance.

3.2. Factor analyses

Analyses were conducted to produce both six‐ and five‐factor models (Tables 3 and 4). Exploratory factor analyses using principal components extraction produced five factors that had eigenvalues >0.99 (Table 4), accounting for 65 percent of explained variance. To reproduce the original PIP's six domains, an alternate model was constructed in which the sixth factor had an eigenvalue of 0.95, accounting for slightly over 68 percent of explained variance (Table 3).

Table 3.

Six‐factor model for practice integration profile

	Component
	1	2	3	4	5	6
Clinical services
Clinicians available on site who provide noncrisis focused BH services	0.78	0.16	0.15	0.29	0.22	0.15
Clinicians available on site to respond to patients in behavioral crisis	0.77	0.13	0.17	0.25	0.24	0.16
BH clinicians can respond to seriously mentally ill and substance‐dependent patients	0.78	0.19	0.10	0.19	0.17	0.29
Behavioral interventions for patients with chronic/complex medical illnesses	0.53	0.29	0.24	0.21	0.26	0.38
Clinicians with a background and training in complex or specialized BH therapies	0.67	0.30	0.07	0.21	0.19	0.26
Evidence‐based substance abuse interventions	0.53	0.27	0.19	0.11	0.27	0.38
Prescription meds for routine mental health and substance abuse diagnoses	0.25	0.01	0.09	0.13	0.21	0.84
Prescription meds for complex co‐occurring mental health/substance abuse diagnoses	0.33	0.15	0.06	0.12	0.10	0.81
Referral to nonclinical services outside of our practice	0.08	0.20	0.15	0.15	0.61	0.26
Practice workflow
Use standard protocol for patients who need/can benefit from integrated BH	0.34	0.51	0.31	0.14	0.30	0.03
Use registry tracking for patients with identified BH issues	0.24	0.69	0.19	0.01	0.17	0.03
We use a standard approach for documenting patients’ self‐management goals	0.33	0.40	0.21	0.18	0.60	0.06
Provide coordination of care for patients with identified BH issues	0.27	0.17	0.15	0.13	0.80	0.08
Provide referral assistance to connect patients to community resources	0.22	0.15	0.17	0.13	0.77	0.15
Provide referral assistance to connect patients to specialty mental health resources	0.18	0.44	0.28	0.22	0.46	0.00
Patient engagement
Successfully engage identified patients in Behavioral Care	0.32	0.50	0.22	0.37	0.23	0.26
Successfully retain patients in Behavioral Care	0.33	0.52	0.16	0.38	0.13	0.30
Specific systems to identify/intervene on patients who did not initiate or complete care	0.11	0.72	0.23	0.20	0.19	0.10
Has follow‐up plans for all patients who complete BH interventions	0.14	0.55	0.20	0.42	0.26	0.17
Workspace arrangement and infrastructure
BH and Medical Clinician's physical work space	0.49	0.09	0.12	0.58	0.03	−0.06
Patient treatment plans documented in a medical record for BH and medical clinicians	0.14	0.02	0.14	0.75	0.04	0.14
Integration methods
BH and Medical Clinicians regularly, actively exchange information about patient care	0.25	0.26	0.19	0.71	0.30	0.08
Regular educational activities include BH and Medical Clinicians	0.31	0.33	0.10	0.53	0.21	0.19
BH and Medical Clinicians regularly spend time together collaborating on patient care	0.44	0.31	0.21	0.59	0.28	0.10
Patients have care plans developed jointly by the patient & BH/Medical clinicians	0.13	0.51	0.18	0.55	0.23	0.07
Case identification
Screen eligible adults for BH conditions using a standardized procedure	0.17	0.04	0.73	0.25	0.14	0.15
Use practice‐level data to screen for patients at risk for complex or special needs	0.18	0.48	0.65	0.07	0.05	−0.02
Patients are screened annually for behavioral conditions related to a medical problem	0.18	0.18	0.81	0.11	0.12	0.07
Patients are screened annually for lifestyle or behavioral risk factors	0.04	0.15	0.79	0.08	0.21	0.09
Screening data are presented to clinicians with recommendations for patient care	0.06	0.39	0.63	0.20	0.18	−0.01
Percent of variance for each factor	14.7	12.8	11.5	11.4	10.7	7.6

Notes: Extraction method: principal component analysis.

Rotation method: Varimax with Kaiser normalization.

Six factors account for 68.6% of total variance.

Factor loadings ≥0.45 are presented in bold.

Rotation converged in seven iterations.

Table 4.

Five‐factor model for practice integration profile (Revised)

	Component
	1	2	3	4	5
Clinical services
Clinicians available on site who provide noncrisis focused BH services	0.63	0.46	0.23	0.11	0.19
Clinicians available on site to respond to patients in behavioral crisis	0.64	0.41	0.20	0.13	0.21
BH clinicians can respond to seriously mentally ill and substance‐dependent patients	0.74	0.34	0.24	0.07	0.15
Behavioral interventions for patients with chronic/complex medical illnesses	0.63	0.28	0.30	0.23	0.25
Clinicians with a background and training in complex or specialized BH therapies	0.64	0.33	0.34	0.04	0.17
Evidence‐based substance abuse interventions	0.63	0.18	0.29	0.18	0.26
Prescription meds for routine mental health and substance abuse diagnoses	0.77	0.05	−0.07	0.13	0.23
Prescription meds for complex co‐occurring mental health/substance abuse diagnoses	0.81	0.07	0.08	0.09	0.11
Practice workflow
We use a standard approach for documenting patients’ self‐management goals	0.27	0.23	0.43	0.19	0.59
Provide coordination of care for patients with identified BH issues	0.25	0.17	0.20	0.14	0.79
Provide referral assistance to connect patients to community resources	0.26	0.15	0.16	0.16	0.77
Provide referral assistance to connect patients to specialty mental health resources	0.12	0.25	0.45	0.27	0.46
Referral to nonclinical services outside of our practicea	0.24	0.11	0.17	0.16	0.62
Patient engagement
Successfully engage identified patients in Behavioral Care	0.39	0.38	0.49	0.22	0.23
Successfully retain patients in Behavioral Care	0.42	0.39	0.50	0.16	0.14
Specific systems to identify/intervene on patients who did not initiate or complete care	0.14	0.19	0.71	0.23	0.20
Has follow‐up plans for all patients who complete BH interventions	0.20	0.39	0.53	0.21	0.27
Use standard protocol for patients who need/can benefit from integrated BHb	0.26	0.21	0.55	0.29	0.29
Use registry tracking for patients with identified BH issuesb	0.19	0.05	0.71	0.18	0.16
Workspace and integration methodsb
BH and Medical Clinician's physical work space	0.26	0.69	0.14	0.10	0.02
Patient treatment plans documented in a medical record for BH and medical clinicians	0.14	0.72	0.00	0.16	0.06
BH and Medical Clinicians regularly, actively exchange information about patient care	0.18	0.72	0.26	0.19	0.31
Regular educational activities include BH and Medical Clinicians	0.31	0.54	0.33	0.11	0.22
BH and Medical Clinicians regularly spend time together collaborating on patient care	0.34	0.66	0.33	0.20	0.28
Patients have care plans developed jointly by the patient & BH/Medical clinicians	0.11	0.53	0.50	0.19	0.24
Case identification
Screen eligible adults for BH conditions using a standardized procedure	0.21	0.26	0.05	0.73	0.14
Use practice‐level data to screen for patients at risk for complex or special needs	0.11	0.10	0.50	0.63	0.04
Patients are screened annually for behavioral conditions related to a medical problem	0.17	0.14	0.20	0.80	0.12
Patients are screened annually for lifestyle or behavioral risk factors	0.09	0.07	0.15	0.79	0.21
Screening data are presented to clinicians with recommendations for patient care	0.03	0.20	0.39	0.63	0.19
Percent of variance for each factor	16.6	13.6	13.4	11.2	10.6

Notes: Extraction method: principal component analysis.

Rotation method: Varimax with Kaiser normalization.

Five factors account for 65.4% of total variance.

Factor loadings ≥0.45 are presented in bold.

Rotation converged in seven iterations.

^aMoved from Clinical Services domain to Practice Workflow domain.

Moved from Practice Workflow domain to Patient Engagement domain.

^bNew domain combines items previously in Workspace Arrangements and Infrastructure domain and Integration Methods domain.

3.2.1. Six‐factor model (original PIP)

Table 3 shows factor loadings for the items in which the analyses were set to produce six factors. The six‐factor model reproduces the domains of the PIP very well with a few exceptions. For example, Factor 1 has loadings >0.50 for six of the nine items assigned to this PIP domain. Two items related to prescription drugs have high factor loadings on factor 6 and one item related to referrals loads highly on factor 5, which has other questions related to referral services. Five of the items related to Case Identification have factor loadings that align with factor 3 (i.e., loadings >0.60). Factor 2 has items that fit with Workflow and Patient Engagement Domain (i.e, two Workflow items and all four Patient Engagement items have factor loadings ≥0.50).

3.2.2. Five‐factor model (proposed PIP)

The five‐factor model shown in Table 4 suggests that the two items representing Workspace Arrangements and Infrastructure (location of behavioral health providers in the practice, and accessibility of medical records by both medical and behavioral health clinicians) fit well with the Integration and Sharing Methods domain with factor loadings >0.50 for factor 2. Hence, this domain combines six items about the physical arrangements, shared medical records, and the joint activities of medical and behavioral health providers.

One of the nine items from the Clinical Services domain, “referral to non‐clinical services outside of our practice” is moved to the Practice Workflow domain in the five‐factor model. The revised Clinical Services domain has eight items, including the two prescription drug items (prescription drugs for routine conditions and prescription drugs for complex conditions) that have high factor loadings on factor 1 in the five‐factor model (i.e, loadings >0.75).

In the five‐factor model, two items are moved from the Practice Workflow domain to the Patient Engagement domain. These items, “use standard protocol for patients who need/can benefit from integrated BH” and “use registry tracking for patients with identified BH issues,” load on factor two which includes all of the items in the Patient Engagement domain.

3.3. Reliability

Table 5 shows the item‐to‐scale correlations and internal consistency reliability estimates (Cronbach's alpha) for the six‐domain model and a revised five‐domain model in which Workplace Arrangements and Infrastructure are combined with Integration Efforts and Sharing Methods. The item “referral to non‐clinical services outside of our practice” is included in the Practice Workflow domain, rather than Clinical Services. The original six‐domain model has alpha coefficients >0.80 for five of the domains. The domain with the lowest reliability was Workplace Arrangements and Infrastructure, a two‐item domain with an alpha coefficient of 0.59. The Clinical Services domain has one item‐to‐scale correlation (r = 0.46) that is much lower than the other items in this domain (all r > 0.60). All internal consistency coefficients in the five‐domain model were >0.80. The new six‐item Integration and Sharing Methods domain had an alpha of 0.87 and all item‐to‐scale correlations are at least 0.56.

Table 5.

Item‐to‐scale correlations (corrected for overlap) and reliability estimates for PIP domains

PIP domain and survey questions	PIP v1.0						PIP Rev.
	1	2	3	4	5	6	4&5
Internal consistency reliability (Cronbach's alpha)	0.92	0.87	0.85	0.59	0.87	0.86	0.87
1. Clinical services
Clinicians available on site who provide noncrisis focused BH services	0.78
Clinicians available on site to respond to patients in behavioral crisis	0.77
BH clinicians respond to seriously mentally ill & substance‐dependent pts	0.83
Behavioral interventions for patients with chronic/complex medical illnesses	0.76
Clinicians with a background & training in complex, specialized BH therapies	0.74
Evidence‐based substance abuse interventions	0.71
Prescription meds for routine mental health and substance abuse diagnoses	0.61
Prescription meds for complex health/substance abuse diagnoses	0.66
2. Workflow
Use standard protocol for patients who need/can benefit from integrated BH		0.65
Use registry tracking for patients with identified BH issues		0.54
Use a standard approach for documenting patients’ self‐management goals		0.76
Provide coordination of care for patients with identified BH issues		0.68
Provide referral assistance to connect patients to community resources		0.71
Provide referral assistance to connect patients to specialty MH resources		0.67
Referral to nonclinical services outside of our practice		0.52
3. Patient engagement
Successfully engage identified patients in Behavioral Care			0.76
Successfully retain patients in Behavioral Care			0.73
Systems to identify/intervene on patients who didn't initiate or complete care			0.64
Has follow‐up plans for all patients who complete BH interventions			0.68
4. Workspace arrangements
BH and Medical Clinician's physical work space				0.42			0.58
Treatment plans documented in medical record for BH & medical clinicians				0.42			0.54
5. Integration efforts
BH & medical Clinicians regularly, actively exchange info about pt care					0.77		0.79
Regular educational activities include BH and Medical Clinicians					0.67		0.66
BH & medical clinicians regularly collaborate on patient care					0.81		0.80
Patients care plans developed jointly by patient & BH/Medical clinicians					0.69		0.66
6. Case identification
Screen eligible adults for BH conditions using a standardized procedure						0.64
Use practice data to screen for patients at risk for complex/special needs						0.67
Patients screened annually for BH conditions related to a medical problem						0.74
Patients are screened annually for lifestyle or behavioral risk factors						0.68
Screening data presented to clinicians with recommendations for patient care						0.66

PIP, practice integration profile.

4. DISCUSSION

4.1. Performance of the PIP's six domains

The PIP's original six domains were derived from the AHRQ's Lexicon for Behavioral Health, which itself was a summary of expert opinion by thought leaders in the field of behavioral health integration.18 Expert opinion alone informed the development of the PIP, as no previously existing, reliable, validated measure of behavioral health integration was available. The current study describes the real‐world performance of the PIP's six domains in a sample of 357 unique primary care practices. The results of this analysis demonstrated five of the six domains perform surprisingly well, with measures of internal consistency that are more than adequate for measures of primary care practice such as the PIP. In a proposed five‐domain PIP, all Cronbach's alpha coefficients were higher than 0.80, which is more than adequate for making group comparisons.26 In this proposed model, most of the alpha coefficients approached or exceeded 0.90, which has been proposed as a threshold suggesting redundancy.27, 28 Minor modifications to the PIP are recommended to further improve its performance.

4.2. Recommended improvements to the PIP

As initially constructed, the Workspace Arrangements and Infrastructure domain included only two items. These items focused on the physical location of behavioral health providers in the primary care practice and the sharing of documentation in patients’ health records. Upon reflection, it is easy to imagine how these two items could vary independently. Providers could be physically adjacent but use separate health records. Or, providers could be physically distant while sharing a tightly integrated health record. It is not surprising, then, that the internal consistency of the Workspace Arrangements and Infrastructure domain was relatively low in this analysis. Furthermore, this poor performance was partly attributable to the general difficulty of establishing internal consistency in a domain with only two items.

We recommend that the two‐item Workspace Arrangements and Infrastructure domain be combined with the four items in the Integration and Sharing Methods domain to create a new, merged, six‐item domain. The new six‐item Integration and Sharing Methods domain retains a high degree of internal consistency, even after including the two items formally in the Workspace Arrangements and Infrastructure domain. This change also makes intuitive sense as the Integration and Sharing Methods domain, “covers the type and degree of interactions among medical and behavioral providers.”21 The inclusion of questions assessing physical proximity and shared documentation in this domain is consistent with the domain's focus.

Three changes to the distribution of items in the PIP are recommended by this analysis. The item from the Clinical Services domain, “we offer referral to non‐clinical services outside of our practice” is more closely aligned with the Practice Workflow domain. Moving this item from the Clinical Services domain to the Practice Workflow domain would reduce the Clinical Services domain from nine items to eight, increase the Practice Workflow domain from six items to seven, and improve the internal consistency of both domains (Table 5). Furthermore, this item does not assess a clinical service, but rather a practice's ability to establish reliable workflows for connecting patients to nonclinical services such as “spiritual advisors, schools, criminal justice (probation and parole, drug courts), or vocational rehabilitation.” Given this item's focus on “referral to non‐clinical services,” it makes intuitive sense to attribute it to Practice Workflow rather than Clinical Services.

Two additional changes are advised. The items “use standard protocol for patients who need/can benefit from integrated BH” and “use registry tracking for patients with identified BH issues” should be moved from the Practice Workflow domain to the Patient Engagement domain. These items assess patient identification and tracking which are closely related to patient engagement.

The changes described above also have the benefit of reducing the influence of domains with fewer items on the PIP total score. In its initial form, the PIP had six domains ranging in number of items from two to nine. After adopting the changes described above, the range in the revised five‐domain version is five to eight. Given that the PIP total score is calculated by using the unweighted mean of each domain score, this narrowing in the number of items in each domain also narrows the range of the weight of each item in determining the PIPs total score.

4.3. Strengths and limitations of this study

The current study has a number of strengths. The sample size allows for a statistically sound factor analysis of the PIP's original six domains. In addition, the analysis provides empirical support for the PIP's domain structure. Finally, the results provide a clear set of recommendations for the improvement of the PIP.

A limitation of this study is the unknown representativeness of the sample of practices. The PIP is available at no cost on a publicly available website. However, practices that are enthusiastically advancing behavioral health integration may be more likely to use the PIP. This convenience sample may have impacted our analysis in ways that cannot be fully understood. In addition, this analysis was focused on the optimal organization of the PIP domain scores and did not consider whether the addition, elimination, or changes in individual items would improve the PIP.

4.4. Future of the PIP

A validated measure of behavioral health integration will help identify integration activities that are associated with improved patient outcomes. Previous studies have demonstrated the reliability and validity of the PIP.21 Since the initial validation studies were conducted, 1000 additional PIPs have been completed, resulting in a more robust database. The initial validation of the PIP's inter‐rater reliability, test‐retest reliability, construct validity, and discriminant validity should be repeated with the more robust dataset. Second, anecdotal responses from users have indicated that some PIP items are not always interpreted as intended. Structured cognitive interviews should be conducted to explore respondents’ interpretations of each item (i.e, meaning, scoring methodologies, and available responses). Finally, a short‐form version of the PIP could have utility in some settings and data analyses should be conducted to determine whether a subset of PIP items can be identified which have suitable reliability, validity, and correlation with the full 30‐item PIP.

This PIP is a valid and reliable measure that is suitable for making comparisons between practices and describing changes over time. The present analysis demonstrates that a revised, five‐domain PIP is superior to the existing six‐domain PIP. This improvement should be incorporated with other PIP improvements to further strengthen the measure.

CONFLICT OF INTEREST

Lee Hargreaves, PhD, was paid at a consultant rate by University of Massachusetts Medical School, not salaried. Other authors have no conflict of interest.

Supporting information

 

Mullin DJ, Hargreaves L, Auxier A, et al. Measuring the integration of primary care and behavioral health services. Health Serv Res. 2019;54:379–389. 10.1111/1475-6773.13117