Advancing Integrated Care through Psychiatric Workforce Development: A Systematic Review of Educational Interventions to Train Psychiatrists in Integrated Care

Abstract

Objective:

Integrated or collaborative care is a well-evidenced and widely practiced approach to improve access to high-quality mental health care in primary care and other settings. Psychiatrists require preparation for this emerging type of practice, and such training is now mandatory for Canadian psychiatry residents. However, it is not known how best to mount such training, and in the absence of such knowledge, the quality of training across Canada has suffered. To guide integrated care education nationally, we conducted a systematic review of published and unpublished training programs.

Method:

We searched journal databases and web-based ‘grey’ literature and contacted all North American psychiatry residency programs known to provide integrated care training. We included educational interventions targeting practicing psychiatrists or psychiatry residents as learners. We critically appraised literature using the Medical Education Research Study Quality Instrument (MERSQI). We described the goals, content, and format of training, as well as outcomes categorized according to Kirkpatrick level of impact.

Results:

We included 9 published and 5 unpublished educational interventions. Studies were of low to moderate quality and reflected possible publication bias toward favourable outcomes. Programs commonly involved longitudinal clinical experiences for residents, mentoring networks for practicing physicians, or brief didactic experiences and were rarely oriented toward the most empirically supported models of integrated care. Implementation challenges were widespread.

Conclusions:

Similar to integrated care clinical interventions, integrated care training is important yet difficult to achieve. Educational initiatives could benefit from faculty development, quality improvement to synergistically improve care and training, and stronger evaluation.

Systematic review registration number:

PROSPERO 2014:CRD42014010295.

Integrated mental health care models improve access to high-quality mental health care outside of specialized settings and address the high burden of mental illness.^1,2 Known as collaborative, shared, or integrated mental health care (henceforward ‘integrated care’), these approaches involve psychiatrists, primary care providers (PCPs), and other health care professionals working together to improve outcomes for a defined population.^2,3 There is a spectrum of integration from colocation or consultation-liaison of mental health specialists in a primary care setting (physically or virtually via telepsychiatry) to chronic care models that include support for patient self-management, measurement-based care, clinical registries enabling proactive outreach, and decision support to guide steps in care targeting remission.⁴ These innovations in care delivery have well-demonstrated efficacy in improving timeliness of care, clinical outcomes, and cost effectiveness of care, with the most empirical support for the chronic care model.^2,5–8

However, transferring and scaling up integrated care from clinical trials to ‘real-world’ settings has proven challenging.^9,10 One significant barrier is the need for health care providers to be trained in new ways of working with patients and clinical teams and using individual and population data to inform care.^2,11–16 Although large-scale efforts to train psychiatrists are now under way across North America, including mandatory training for all Canadian psychiatry residents, it is not known how best to prepare psychiatrists to work in these evolving models of mental health service delivery.^17,18 This knowledge gap has had significant consequences for the variability and overall quality of existing training across the country.¹⁹ Kates’s chapter in Approaches to Postgraduate Education in Psychiatry in Canada has provided a useful starting point during the first few years of mandatory training across Canada by suggesting learning objectives and educational strategies²⁰; however, to advance the field, an empirical approach is also needed.

We conducted a systematic review of the literature in which we aimed to identify all integrated care training experiences that have been evaluated in published and unpublished sources; examine their goals, contents, methods and outcomes; and distill, where possible, evidence-informed recommendations for psychiatric training.

Methods

The primary objective of this systematic review was to synthesize and critically appraise training curricula and methods that have been used to develop psychiatrists’ competence for the practice of integrated care. The study objectives were outlined a priori in a published protocol.²¹ We developed the following eligibility criteria based on our knowledge of the field as integrated care providers, educators, and researchers.

Population

We selected interventions that targeted junior and senior psychiatric residents, fellows, faculty members, and practicing psychiatrists. We excluded undergraduate medical trainees who are undifferentiated as learners working across many disciplines. We excluded training initiatives targeted exclusively toward other health care providers.

Intervention

We included any educational intervention that trained the study population in integrated care and included an evaluative component. To be considered integrated care, the focus of training had to meet 3 parameters of the Agency for Healthcare Research and Quality typology, including a) an interprofessional team, b) working with some degree of collaboration (e.g., co-located, coordinated, consultation-liaison, or chronic care models) in person or at a distance (e.g., telepsychiatry), and c) with a shared population and mission.²²

Comparator

We included studies regardless of the presence or absence of a comparison group in the evaluation.

Outcomes

We examined all psychiatric learner, co-learner, and patient outcomes at all Kirkpatrick levels.²³ The Kirkpatrick model outlines 4 levels of educational impact, including 1) reaction/satisfaction: learner views on the learning experience, its organization, content, teaching methods, materials, and quality; 2) learning: changes in perceptions and attitudes (including plans for future practice) and acquisition of knowledge and skills, including problem solving and social skills; 3) behaviour: changes in workplace performance; and 4) results: organizational and clinical outcomes.

Literature Search

We conducted an extensive literature search of English-language published and unpublished literature to August 2015 that included all quantitative and qualitative study designs and time periods (see supplemental file for details). We also reached out to directors of psychiatry residency programs in North America who were known to include integrated care in the curriculum, inquiring about both residency training and faculty development. In Canada, integrated care training is mandatory for psychiatry residents, so we contacted all programs via the Council of Psychiatric Educators. For the United States, we relied upon the American Association of Directors of Psychiatry Residency Training (AADPRT) database.

Abstract and Full-Text Screening

Two research team members independently reviewed all titles and abstracts using DistillerSR software to determine inclusion, resolving any differences of opinion by consensus discussion between team members and the study lead. Then, at least 2 researchers conducted a full-text review in a similar fashion. We assessed interobserver agreement for inclusion of studies using a chance-corrected kappa statistic.

Data Collection and Critical Appraisal

Two research team members abstracted data for each included citation. We collected quantitative and qualitative information related to the learners, educational intervention, and methods and results of evaluation. For unpublished programs, we collected the same data by e-mail and/or telephone. We assessed study quality using the Medical Education Research Study Quality Instrument (MERSQI), which is a 10-item tool that appraises study design, sampling, type of data, validity, data analysis, and outcomes.²⁴ Each of the 6 domains can produce a maximum score of 3, resulting in a maximum possible MERSQI score of 18. We used the Critical Appraisal Skills Programme (CASP) or the Newcastle-Ottawa Scale-Education (NOS-E) as supplemental tools to assess the risk of bias of qualitative and observational studies, respectively, if applicable.^24,25

Data Analysis

We prepared descriptive statistics of the learners, evaluation design, and outcomes measured. We extracted descriptions of each educational intervention: program goals, curriculum content and/or clinical workplace activities, supervision provided, and program duration. For each outcome measured, we prepared a brief description and assessed the Kirkpatrick level and strength to draw conclusions about the intervention.^23,26,27 We summarized a list of unique outcomes evaluated in this body of literature and also described broad themes. The data were predominantly textual; therefore, we employed qualitative content analysis techniques.²⁸ We inductively developed categories or ‘codes’ describing outcomes of training as they emerged from our close reading and immersion in the data; then we refined, sorted, and grouped these codes based on their relationships to each other; and finally, we interpreted the data using written summaries and tables.^29,30

Results

We screened 3871 educational interventions, including 3813 published literature citations, 44 grey literature documents, and 14 residency programs (see Figure 1 for PRISMA diagram). Interrater reliability for the abstract screening and full-text screening questions ranged from 0.73 to 0.98 and 0.69 to 0.98, respectively. We located 10 published and 4 unpublished educational interventions that met our criteria. Most programs were in the United States (n = 8) or Canada (n = 5), and almost all targeted residents (n = 12) (see Tables 1 and 2 for descriptions of included programs). Clinical training experiences were often based in primary care or, alternatively, targeted specific populations such as women, people living with addictions or human immunodeficiency virus, or those in rural settings.

Figure 1.

PRISMA diagram.

Table 1.

Descriptive Statistics of Included Sources.

	No.
Publication status	Peer reviewed published = 10
	Residency program grey literature = 4
Country where study conducted	United States = 8
	Canada = 5
	Australia = 1
Psychiatric learner breakdown	Junior residents only = 2
	Senior residents only = 5
	Junior and senior residents = 4
	Residents (not specified) = 1
	Staff = 2
	Fellows = 0
	Faculty = 0
Core or elective	Elective = 6
	Core curriculum = 5
	Mix of core + elective = 1
	Not reported = 2
Type of educational intervention	Didactic only = 3
	Clinical placement only = 2
	Didactic + clinical placement = 8
	Didactic + clinical + other = 1
	Other = Research project, joint quality
	improvement, practice groups (case-based learning)
Research design (no control groups included any studies)	Noncomparative = 14 (100%)
Data collection methods/sources	Self-report quantitative = 13
	Self-report qualitative = 2
	Other (patient self-report, partner provider self-report) = 5
	(5 studies had more than one method)
Highest level outcome (Kirkpatrick level) for each study	Level 4b: Patient outcomes = 2
	Level 4b: Patient satisfaction = 1
	Level 4a: Organizational outcomes = 1
	Level 2a: Learner perceptions/attitudes = 7
	Level 1: Learner satisfaction = 4

Table 2.

Characteristics of Included Integrated Care Training Programs.

First Author, Year Published	Training Setting(s)	Target Population	Core or Elective	Type of Education	Training ‘Dose’, Duration	Concurrent Trainees	No. of Trainees (Start, End)	Highest Level of Evaluation	MERSQI Scorea
Cowley,35 2000	Family medicine, general internal medicine, global health and HIV, women’s health clinics	Senior residents	Elective	Clinical, didactic	Half-day per week over 12 months or full-day per week over 6 months	None	14, 14	Learner perceptions/ attitudes	9
DeGaetano,36 2015	Family medicine, addictions clinics (Veterans Affairs)	Junior residents	Core	Clinical	NR, 6 months	None	15, 15	Learner perceptions/ attitudes	6
Engel,37 n.d.	Patient-aligned care team (Veterans Affairs)	Junior, senior residents	Mixed core, elective	Clinical, didactic	Full-time over 4 weeks	Internal medicine residents	18, 18	Learner satisfaction	6.5
Henrich,40 2003	Inner-city women’s health clinic	Senior residents	Elective	Clinical, didactic	Half-day per week, NR	Ob/Gyn, internal medicine residents	NR	Learner satisfaction	8
Huang,31 2015	Nonclinical	Junior, senior residents	NR	Didactic	Two 2-hour modules, plus readings and videos	None	46, 46	Learner perceptions/ attitudes	7.5
Kates,41 2011	Urban family medicine clinic	Junior, senior residents	Core	Clinical, didactic	Half-day per week over 6 months for junior residents, NR for senior residents	Family medicine residents and staff physicians	15, 15	Learner perceptions/ attitudes	5
McMahon,42 1983	Psychiatry and family medicine clinic at academic hospital	Junior residents	NR	Other (case-based learning, Balint group, QI, research project)	Monthly didactic, 2-3 times per week case-based learning, NR	Family medicine residents	NR	Learner satisfaction	6
Merkel,32 n.d.	Rural family medicine clinics via telepsychiatry	Senior residents	Core	Clinical, didactic	Half-day every other week over 1 year	None	NR	Patient satisfaction	9
Naimer,43 2012	Psychiatry and family medicine departments at academic hospital	Junior, senior residents	Elective	Clinical, didactic	NR, 1 year	Family medicine residents	4, 4	Learner perceptions/ attitudes	7.5
Ratzliff,38 n.d.	Family medicine clinic	Senior residents	Elective	Clinical, didactic	Half-day per week over 6 to 12 months	None	19, 19	Learner perceptions/ attitudes	7
Rockman,33 2004	NR	Staff psychiatrists	Elective	Didactic	2 CME events, NR	General practitioners (including psychotherapists)	10, NR	Patient outcomes	5
Sunderji,39 n.d.	Family medicine clinics	Senior residents	Core	Clinical, didactic	2 days per week over 3 months	Variable; family medicine residents and interprofessional trainees	NR	Learner satisfaction	6
Sved Williams,34 2006	NR	Staff psychiatrists	Elective	Didactic	2 hours	None	31, 27	Organizational outcomes	8
Teshima,44 2015	Rural family medicine clinics via telepsychiatry from academic children’s hospital	Residents (year NR)	Core	Clinical	Two or more 1- to 2-hour consultations	None	NR, 381	Learner perceptions/ attitudes	7.5

HIV, human immunodeficiency virus; MERSQI, Medical Education Research Study Quality Instrument; n.d., no date; NR, not reported; QI, quality improvement.

Critical Appraisal of Included Literature

The median MERSQI score for included studies was 7.25 (range, 5-9; see supplemental file). All programs except one³¹ were limited to a single site, and many had response rates that were low or not reported. Few programs reported on higher order organizational or patient outcomes.^32–34 The overall quality of included studies is low to moderate. Despite the prevalent weak study designs and the predominant focus on lower level outcomes (e.g., learner satisfaction and attitudes), it is likely that this body of literature reflects some publication bias toward favourable outcomes, considering that few programs reported any negative outcomes.^35–37

Comparison of Included Programs with the Larger Body of Known Canadian and US Programs

Of the 3 educational interventions included in our review and based in Canadian postgraduate training, 2 originated from the University of Toronto residency program, which is the largest residency program by far, and 1 from McMaster University, another moderately sized program also in Ontario. A previous survey of Canadian psychiatry residency programs that we conducted (unpublished observation, n = 13 respondents out of 16 residency programs in 2012) suggests that these experiences are representative of the range of training experiences currently offered across Canada, although the implementation climate would likely differ in Toronto’s large metropolitan academic environment.

With respect to the US residency programs that are known to provide training in integrated care, of the 34 distinct training experiences offered at 27 residency programs, only 3 programs responded to our email invitation, met all inclusion criteria, and are reported here (further information available from the authors upon request). They do not appear to capture the breadth of clinical settings, outcomes, or implementation challenges described in the AADPRT database.

Goals of Training

Programs emphasized learners’ skill development to collaborate with other health care providers outside of specialist psychiatric settings, such as initiating and managing collaborative working relationships, providing direct patient consultation and indirect provider-to-provider consultations, and communicating effectively. Few programs taught knowledge and skills specific to the more empirically supported chronic care models, such as the evidence, supervision of care managers, delivery of measurement-based care, and/or population-based care.^31,38

Co-learners

Seven of the 14 programs reported having some other ‘co-learners’ involved besides psychiatrists, such as family medicine or other residents, or practicing primary care physicians; only 1 program included interprofessional health care provider trainees (e.g., social work or psychology), and it did so variably.³⁹ None reported learning objectives or outcomes for these co-learners, nor were they published separately according to a targeted search.

Outcomes Evaluated and Strength of Findings

We identified 39 unique outcomes that represent the ways in which integrated care educational interventions have been evaluated. These included commitment or buy-in to integrated care among all stakeholders (21% of the outcomes measured); learner experience and satisfaction (15%); learner awareness and knowledge (15%); learner skill development (13%); effects on educational resource requirements, such as teachers’ time (5%); structures and processes of care, including collegial/professional networks, communication, and coordination of care (10%); and patient-oriented outcomes such as experience, wait time, or clinical outcomes (21%) (see Table 3 for outcomes by type and strength of finding). No studies used validated rating scales.

Table 3.

Outcomes of Integrated Care Training by Type and Strength of Finding.

Type of Outcome, % (n) of All Outcomes	Conclusions Can Probably Be Based on the Results	Ambiguous Results; There Appears to Be a Trend	Not Significant or No Conclusions Could Be Drawn
Buy-in to integrated care among diverse stakeholders, 23% (9)	Recruitment and retention of psychiatrists into the IC program	Use of the service (e.g., by PCPs) (±) Referring provider (e.g., PCP) experience, perceptions of advice provided Learner intent to incorporate skills into practice Learner integration into primary care setting (±) Learner intent to practice telepsychiatry in their career	Learner intention to practice IC in their career PCP perceived need to refer to IC
Learner experience, 15% (6)	Experience of providing IC Satisfaction with training in IC	Recommend the training experience to continue	Perceived supervision quality Perceived relevance to current and future practice Satisfaction with use of telepsychiatry technology Experience of collaboration via telepsychiatry (±)
Learner awareness, knowledge, 15% (6)		Knowledge of relationship between physical and mental health Clinical knowledge (of all providers) Knowledge of evidence for IC models Knowledge of telepsychiatry as a useful mode of service delivery to underserved areas	Exposure to culturally diverse population Learner clarity re: their role (–)
Learner skill, 13% (5) development		Preparation to function as a psychiatric consultant at the interface with primary care Assessment and management of mental illnesses Quality of learner documentation	Assessment and management of mental illness via telepsychiatry Assessment and management of risk and emergencies via telepsychiatry (±)
Educational resource requirements, 5% (2)		Minimized duplication of teach efforts across departments Workload of mentoring trainees (–)
Communication and coordination of care, 10% (4)		More collegial relationships Improved communication and continuity of care	Enhanced professional practice network PCP access to specialist opinion
Patient-oriented outcomes, 20% (8)		Decrease in patient symptoms Patient experience of care providers’ helpfulness Decrease in wait times Patient engagement in care planning Patient experience of clinic/logistics	PCP perception of overall improved patient care Patient experience of telepsychiatry

All outcomes positive unless noted (–) for negative outcomes or (±) for conflicting studies. IC, integrated care; PCP, primary care provider.

The overall low to moderate quality of studies included in this review was reflected in our assignment of the strength of the findings. Of the 39 unique outcomes extracted from the studies, 36% (14) were identified as of the lowest strength (not significant or no conclusions could be drawn), 56% (22) were slightly better (ambiguous results, appears to be a trend), 8% (3) were stronger still (conclusions can probably be based on the results), and none were of the highest strength (results are clear and very likely).

Descriptions of and Trends in Effective Integrated Care Training

While the educational interventions included in this review are markedly heterogeneous, there are several major variants in the types of interventions: a) mentoring networks whereby psychiatrists provide case-based input (e.g., telephone, fax, or e-consultation) upon PCPs’ request, b) brief didactic events, and c) longitudinal supervised clinical experiences for residents.

Mentoring networks

Three studies provided brief training to practicing psychiatrists or residents to provide just-in-time case-based consultation and ongoing mentorship to PCPs.^33,34,43 Two of these studies also trained the PCPs together with the psychiatrists, thus jointly preparing mentors and mentees to make use of the consultative relationship through fostering rapport, raising awareness of the service and how to use it appropriately, and addressing medicolegal issues, including confidentiality and documentation.^33,43 In all 3 studies, there were trends toward use of the service and PCPs’ perceptions of the utility of the advice (conclusions could reasonably be drawn in Naimer et al.⁴³). In 2 studies, there were trends toward positive impacts on PCPs’ self-reported knowledge^33,34 and in 1 case on PCPs’ self-reported time to implement optimal treatment.³³ Conclusions could reasonably be drawn that one of these programs was successful in recruiting and retaining psychiatrists to provide case-based provider-to-provider consultation upon request by PCPs.³⁴

Brief didactic teaching

Two programs consisted solely of didactic teaching to residents, including interactive workshops with readings, videos, case-based discussions, and/or hospital-based rounds.^31,42 These were provided for as little as two 2-hour modules to as much as twice per week for 1 academic year. Of note, 1 of these programs focused on the collaborative/chronic care model.³¹ In the results of our analysis, few conclusions could be drawn about impact; learners may be willing to incorporate new techniques but may prefer greater realism (e.g., case-based learning and role-plays).

Longitudinal supervised clinical experiences

Six studies outlined longitudinal clinical workplace activities in primary care settings, on average 6 months in duration (ranging from a half to a full day per week for between 3 and 12 months).^{32,35,36,38,39,41} Residents provided consultation to PCPs and their patients in person or via telemedicine. Most programs supplemented this with didactic content about integrated care models and effective approaches for collaboration. Some programs showed trends toward satisfaction with the training experience,³⁹ improved resident knowledge and skills (e.g., empirically supported models of integrated care, using screening and monitoring tools, supporting care managers),³⁸ impact on future career plans,³⁸ and patient experience.³² However, in other cases, no conclusions could be drawn.^36,41,45

Implementation Challenges

Many authors described challenges in training residents within models of care that are complex, evolving, and difficult to implement and sustain in varied ‘real-world’ clinical settings.^{19,32,35–37,39,40,41} Infrastructural barriers were prevalent, such as a) sustainability of funding,^32,37,40 b) physical space that was inadequate or not conducive to collaborative interactions,^32,35,37 c) insufficient staffing to maintain the service, and d) staff lacking interest and ability to role-model good consultation-liaison attitudes and skills.^32,37,41 Engagement and support of collaborators (e.g., PCPs) were repeatedly flagged as crucial to ensure stability of the service and to develop attitudes conducive toward collaboration with psychiatry trainees.^32,35,41 In the absence of this type of buy-in, it can be difficult for trainees to have clarity about their roles.³⁵

Discussion

In this study, we rigorously review published and unpublished literature describing training programs that prepare current or future psychiatrists to provide integrated care. We have distilled training programs’ goals, pedagogical methods, outcomes, and implementation challenges hampering sustainability and spread. We balanced the desire to illuminate key issues in integrated care training with judicious interpretation of included studies, given the overall heterogeneity and low to moderate quality of the studies. Thus, we provide a comprehensive and well-considered overview of the current state of the field. To our knowledge, this is the first study of its kind and can provide a useful guide to academic and clinical leaders in this area.

Several limitations common to systematic reviews merit consideration. The topic under study is at the intersection of several complex concepts, and the search terms and parameters we used would have influenced the integrated care educational interventions that we located, with some risk of unintentional omission of relevant literature. We suspect some degree of publication bias has also affected the availability and quality of information pertaining to the defined research question. We attempted to mitigate this by seeking out unpublished literature, but we were limited by the low response rate among residency programs. Another limitation is the overall low to moderate methodological rigor of available studies. We critically appraised each study design using the MERSQI and carefully considered the strength of each outcome.

Based on the existing literature, it is difficult to determine which, if any, approaches to training improve educational, patient, or health system outcomes. This is concerning considering that training is already widespread and has been resource intensive to implement in already overburdened curricula. There is a strong appetite across Canada for practical recommendations to facilitate implementation and improvement of training across Canada. Although we are hard-pressed to make such recommendations based on empirical knowledge from this systematic review, our results do provide some direction for this important but immature field. Drawing upon a combination of this scientific knowledge, our pragmatic knowledge as providers of integrated care (N.S., P.B., A.G.R., A.C.) and our experiential knowledge as integrated care educators and leaders (N.S., A.G.R.), clinical supervisors (N.S., A.C.), and learners (P.B., D.H.), we suggest several recommendations to advance psychiatric training in integrated care, with an emphasis on postgraduate training and with the caution that further evaluation is needed (see Table 4).

Table 4.

Practical Recommendations to Improve Postgraduate Psychiatric Training.

Aspect of Training	Specific Recommendations
Clinical exposure	We recommend a longitudinal clinical experience that will afford residents ample time to experience and reflect on the delivery of integrated care, as well as opportunities for its improvement (e.g., 1 day per week for 6 or ideally 12 months); this approach may also contribute to developing and sustaining emerging integrated care clinical programs whereby organizational and interprofessional relationships are formed and the setting and its staff become an ongoing resource for learning. By working from a defined set of integrated care competencies that can be attained in a variety of settings,14 it is possible to create ‘minimum specifications’ for training settings (ours are available upon request). This can ensure a robust and sufficiently standardized experience while diversifying and expanding the range of different training contexts, including primary care and community agency settings, telemental health clinics, geriatric long-term care facilities, schools, and child protection agencies.19,47 Enabling senior residents to demonstrate competency in multiple domains simultaneously (e.g., geriatric or child psychiatry and integrated care) can enhance feasibility of implementation and encourage residents to consider working in integrated care regardless of their intended area of future practice.19 While mentorship networks appear promising in the reviewed literature,33,34,43 only 1 study has applied this in postgraduate training.43 We suggest that they should not be used as an exclusive strategy for training residents, as it may be difficult to provide an adequate ‘dose’ of training in this model or to develop the range of practical skills needed beyond mentoring.
Core curriculum	A strong nonclinical curriculum can mitigate gaps in the clinical training experience (e.g., related to the evidence-to-practice gap in the models of integrated care),31,48 emphasize topics relevant to transitioning to practice (e.g., leadership, career planning, managing patient safety, and medicolegal liability in collaborative practice), and ideally introduce feedback loops that engage faculty in helping residents apply the curriculum learning to the clinical environment. A ‘flipped classroom’ model49,50 can engage senior residents as adult learners and minimize the faculty teaching burden, which can be particularly high if there are few faculty equipped to teach this content.32,35,41 However, ensuring adequate time for the independent study component is key to resident acceptance, and there is a need for further evaluation of this approach.51
Assessment	Assessment should be congruent with the move to competency-based education, focused, and relevant to integrated care (e.g., multisource feedback, case-based application of population-based care concepts, written or oral application of learning about evidence-based models of integrated care).52
Faculty development	The importance of faculty development cannot be overstated. Given the newness of integrated care training in Canada, faculty may not have been exposed to this mode of practice during their own training and/or they may not be familiar with teaching and assessing it. An initial ‘bolus’ of preparation may be needed, ideally accompanied by longitudinal peer group learning and support.
Implementation issues and change management	It is crucial that any efforts to expand or improve training anticipate and proactively address issues with funding models, physical space, and readiness of the clinical training environment (e.g., culture, stability).32,35,37,40,41 This requires at least 1 champion with dedicated time as it involves extensive engagement work with diverse stakeholders, including faculty, collaborating primary care and community sites, academic leaders, and learners.32,35,41 Ideally, collaboration with primary care and/or community partners should be built into all aspects of curriculum, assessment, and faculty development, if there is willingness and capacity (or if this can be cultivated over time).

Beyond these practical recommendations, we wish to offer a few general observations and broader recommendations. Training should better align with empirically supported models of integrated care, and reciprocally, integrated care training experiences require better evaluation. This issue runs parallel to the overall advancement of integrated care, where further work is needed to a) disseminate and implement evidence-informed integrated care models (‘evidence-based practice’) and b) engage practitioners in participatory evaluation and improvement of what has been implemented and the creation of generalizable knowledge (‘practice-based evidence’).⁴⁶

This necessary progression in the field can be achieved by explicitly orienting integrated care training toward how to simultaneously work in and improve integrated care models.¹⁴ Training will need to include knowledge of the evidence in integrated care, population health principles, chronic disease management, stepped care models, health policy, and skills in collaborative leadership, program consultation, and using patient- and clinic-level data to guide improvements in practice (i.e., quality measurement and improvement).^13,14 These competencies align with a modern conceptualisation of basic science (i.e., incorporating improvement science) and clinical science (i.e., leadership skills for system-based practice), which learners can attain while ‘actively participat[ing] in high-quality institutional initiatives to improve patient outcomes’.⁵³ Among the studies included in our review, only 2 were specifically oriented toward teaching the collaborative/chronic care model, including population-based care and case finding (both programs)^31,38 and quality improvement and leadership (1 program).³⁸

Furthermore, improvements to integrated care training will necessitate much greater attention to faculty development. Despite an exhaustive search, we did not identify any initiatives aimed at faculty development in integrated care. This is particularly concerning considering a lack of sufficient and skilled supervisors is a widely identified barrier to implementing integrated care training.^19,32,37,41 Faculty need all the same competencies as residents plus the ability to teach and assess in this area.

Co-learning of residents and faculty together may be a worthwhile approach to efficiently build capacity for integrated care training across Canada. For example, an innovative and well-received experiential co-learning curriculum in quality improvement (QI) for internists at the University of Toronto could serve as a model. It enabled faculty to learn QI and/or learn to teach and mentor residents in QI flexibly according to their readiness and interest, thus training up faculty at the same time as teaching current residents.⁵⁴ Following this model, psychiatry faculty and residents could jointly engage in a mentored effort to improve integrated care implementation in their clinical setting, thus harnessing the benefits of experiential learning, workplace-based learning, communities of practice, and mentorship.^55,56 Findings from the co-learning QI curriculum also suggest potential benefits for collegial networking and professional identity formation.⁵⁴ Having faculty and residents co-produce learning could also help reduce dissonance between the curriculum as planned by clinical and education leaders, as taught by supervisors and collaborators and as experienced by learners.^57–59

Concurrently, training in integrated care should better prepare all members of the integrated care team and do so jointly. Many of the studies included in our review involved co-learners of other disciplines in the training setting alongside psychiatric learners, yet their roles and learning objectives and outcomes were rarely described. Individual competencies for care managers and other team members have been suggested and developed into curricula that go beyond the scope of this review, particularly in the United States, where the collaborative/chronic care model is seeing more rapid implementation.^13,15,16 Equally important is the concept of team competence. Successful integrated care is the result of effective interactions between diverse professionals,^60–62 and the literature on health care teams increasingly recognizes that ‘competent individual professionals can—and do, with some regularity— combine to create an incompetent team’.⁶³ Pedagogy in health care professionals’ education has yet to reflect the need to train and evaluate training outcomes together.

Finally, clinical training experiences in integrated care require better evaluation and reporting. Clinician innovators and educators should consider a) using a framework for integrated care evaluation that can help identify improvement opportunities in the clinical training environment,^9,64 b) focusing initially on implementation evaluation to assess what is actually occurring in the training setting and why,⁶⁵and c) using a logic model to guide outcome evaluation of practice and organizational changes that are linked to the educational and clinical care processes that have been implemented.^23,66 With these elements in place, quasi-experimental study designs for ‘real-world’ clinical practice settings, such as time trends analysis and qualitative studies, could yield compelling new insights.

Conclusion

This systematic review provides an overview of current educational initiatives in integrated care that can be used to improve the quality of psychiatric training. We highlight the common approaches to training and their challenges, as well as the overall limitations of the current literature. We propose that the following are needed: a competency-based approach to clinical training, curricula and assessment, faculty development, team training, and the application of QI methods. The advancement of training and the implementation of evidence-informed integrated mental health care are reciprocally linked and will be aided by data-informed reflection on practice, harnessing improvement and implementation sciences.