Table 3.

Summary of evaluation design and measures from studies identified in the systematic literature review

Authors, year of publication	Evaluation Design	Evaluation Measures	Summary of results
Fisman et al., 1996	Post-training questionnaire. No control group	Likert scale items to indicate self-report confidence in ability (4-point scale) and self-report helpfulness of program (5-point scale) in regards to 17 identified knowledge, attitudes, and skills learning objectives.	76% response rate (n=25). Majority reported confidence in knowledge, attitudes, and skills. Majority of participants across the 5 groups reported sessions as helpful (69%–91%) with didactic sessions reported as least helpful (38%).
Garcia-Ortega et al., 2013	Baseline and post-training surveys. No control group.	Likert scale items to assess self-report confidence in using practice support tools provided in program and self-report confidence in mental health care delivery. 20-item true/false knowledge measure on child/youth mental health.	100% response rate at baseline, 47.4% response rate at post training for group 1, 64.3% response rate for post-training survey for group 2. Comparison from baseline to post-training shows an improvement in confidence using tools, confidence in providing mental health care, and increased knowledge of child/youth mental health.
Gaylord et al., 2015	Retrospective comparison of referrals from clinics with shared mental health care program to a (control) clinic without a shared care program.	Chart reviews to assess frequency of referrals to mental health services for depression, anxiety, psychosocial problems and psychotic symptoms.	Assessed 4600 referrals. On average the clinics with a shared care program made more referrals to mental health services compared to the clinic without a shared care program.
Hunter et al., 2008	Descriptive/qualitative study of mentoring network	Quality assurance survey conducted in 2004 with and qualitative analysis of email	From survey data: Satisfaction with program was high (88%). Most common requests for help are for nonemergency clinical issues, pharmacotherapy, psychotherapy, treatment review, and diagnosis. From email correspondence: Mentors convey information directly and indirectly. Trusted relationship between the mentor and mentee was important for effective communication.
Kates et al., 1998	Qualitative description	Participants’ self-report of benefits of the program for providers and patients after program implementation.	Benefits reported include support of family physician to provide timely mental health care, enhanced continuity of care, patients are able to receive more accessible mental health care at their family doctor’s office.
Kirmayer et al., 2003	Post-program surveys. Post-program chart reviews. No control group.	Provider self-report satisfaction with program. Chart reviews for first 100 of those referred to service consultations.	86% of providers reported satisfaction with the program. Useful aspects of program identified as increased cultural understanding, improved communication, and improved treatment. Chart reviews show that most common reasons for consultation was for clarification of diagnosis or symptoms (58%), request for help with treatment planning (45%), and requests for information related to a specific ethnocultural groups (25%).
Kisley & Chisholm, 2009	Pre- and post-program surveys.	Wait times for consultation, mental health functioning (based on DUKE scale), patient self-report satisfaction, self-report provider satisfaction, self-report provider knowledge, self-report provider confidence.	Median wait time for referral was 6 days for sites with program, 39.5 days for control site. 71–78% patients were satisfied with service. Improvement in general and mental health from pre to post in sites with program, but not at control site. Providers reported satisfaction with service and increased sense of knowledge and confidence from pre to post at program sites.
Kutcher et al., 2002	Pre- and post-program surveys. No control group.	Depression knowledge measure created for purposes of this study, chart reviews to assess diagnosis and pharmacotherapy. Self-report patient satisfaction.	Increase in knowledge from pre to post. Enhanced group more likely to use diagnostic tool, more likely to record depression diagnosis, and more likely to initially prescribe citalopram than general group. No difference between general and enhanced group in patient satisfaction, compliance, or treatment outcome.
Kutcher et al., 2003	Pre- and post-program surveys. No control group.	25 item yes/no depression knowledge measure created for the purposes of this study. Had two parts to assess general depression knowledge and depression treatment.	Participants had high baseline knowledge of depression. Significant increases in general depression knowledge and depression treatment knowledge among participants from pre to post.
Naimer et al., 2012	Post-program surveys only. No control group.	Self-report provider utilization of program, self-report provider perceptions of program usefulness, self-report provider satisfaction with program.	80% of family medicine residents used their psychiatry resident buddy during that academic year. In year 1, 93% of family medicine residents reported being satisfied, in year 2, 50% of family medicine residents reported being satisfied. All residents reported the program was useful.
Pignatiello et al., 2008	Qualitative evaluation methodology using surveys, focus groups, and interviews after program implementation. Case reviews of 100 consultations.	Self-report provider perspectives of service providers, family members, and consulting psychiatrists. Case reviews to assess outcomes of consultations.	Providers reported enhanced capacity to support clients, and increased confidence in diagnosis. Families reported reduced burden to travel for speciality care. Consulting psychiatrists reported valuing their role as consultants and were able to provide services without increasing ongoing caseload. Case reviews showed that multiple recommendations were made for youth with extremely complex needs. Successful implementation of recommendations was facilitated by specific treatment directives, availability of follow up sessions, and meaningful engagement of the young person and their family. Barriers were brief nature of consultation, and scarcity of community resources.
Rockman et al., 2004	Pre- and post-program surveys. Post-program chart reviews. No control group.	Baseline survey of providers assesses self-report access to specialized mental health care and self-report mental health care knowledge. Assessed self-report use of program and satisfaction every 3 months. Chart reviews to assess consultation patterns.	Survey results show that providers reported less difficulty accessing mental health care help, increased knowledge and confidence in providing mental health care, and the majority of providers were very satisfied with the program. Chart reviews indicated that most family physician visits were for depression and family dysfunction. Consultations were most frequently for bipolar disorder, schizophrenia, and addictions.
Spenser et al., 2009	Post-program surveys. No control group.	Questionnaires assessed provider self-report confidence and perceived effectiveness of program. Surveys to assess educational needs of paediatricians and mental health clinicians.	100% of providers reported satisfaction with having paediatrician on the mental health team. The top three medical problems posing challenges for mental health clinicians: sleep problems, seizures, and obesity. 83% of mental health clinicians reported satisfaction with CME sessions, 78% reported feeling more confident and knowledgeable. All paediatricians reported satisfaction with CME sessions.
Spenser et al., 2009	Post-program surveys. No control group.	Questionnaires assessed provider self-report confidence and perceived effectiveness of program. Surveys to assess educational needs of paediatricians and mental health clinicians.	100% of providers reported satisfaction with having paediatrician on the mental health team. The top three medical problems posing challenges for mental health clinicians: sleep problems, seizures, and obesity. 83% of mental health clinicians reported satisfaction with CME sessions, 78% reported feeling more confident and knowledgeable. All paediatricians reported satisfaction with CME sessions.
Stretch et al., 2009	No control group. Pre-, post- and 3 month follow-up questionnaires. One month follow up interviews.	Questionnaires assessed self-report provider confidence in managing ADHD and behaviour disorders. Self-report provider satisfaction and self-report provider perceptions of usefulness of program/impact on practice.	Pre-post training questionnaires showed an increase in self-report confidence in providing care (for ADHD more so than behaviour disorders). All participants reported being satisfied with the workshop. The discussions and didactic information were identified as key features of the workshop. Participants suggested a need for up to date written material, as well as local resources and collaborating with mental health network.
Swenson et al., 2008	Qualitative methodology using focus groups and interviews.	Focus groups and individual interviews after program implementation to assess provider and patient perceptions of the program.	Physicians reported the SHARE program useful. Physicians used program to help with medication, diagnosis, and development of treatment plans. Physicians reported consultations had an educational benefit, and that program facilitated a wide range of mental health services. 89% of patient reported the care they received as excellent or very good. 77% of patients were satisfied with the service. Patients liked the speed and flexibility of referrals, the communication between the family doctor and mental health team, and reported relief for finally getting treatment.
Vingilis et al., 2007	Mixed-method (quantitative and qualitative) design. Post-program implementation evaluation. No control group.	Quantitative data from patient database. Qualitative data from focus groups with practitioners to assess perceptions of practitioners.	Quantitative data: Over 12 month period (2002–2003): 3223 patients referred to mental health staff, 3431 patients referred to dieticians. Continuing care was provided by specialists and family physician. Most common problems were depression, marital problems, and anxiety. Most common treatments were individual counselling, Cognitive Behavioral Therapy (CBT), assessment/recommendations, and supportive therapy. Qualitative data: Collaboration and education opportunities were highlighted by providers, with time constraints being a barrier to care. Providers reported less burden on external services and better access to care for patients, and shorter wait times for referral.