Evaluation of a mentorship program to support chronic kidney disease care

Abstract

Problem addressed

Primary care providers (PCPs) are ideally situated to detect and manage patients with chronic kidney disease (CKD), but they could use more support from nephrologists to accomplish this.

Objective of program

To improve early detection and management of CKD in primary care, and improve referrals to nephrologists through education and greater partnership between nephrologists and PCPs.

Program description

Nephrologists provided mentorship to PCPs in Ontario through a collaborative relationship. Nephrologists provided PCPs with educational orientation sessions and need-based advice on patient cases.

Conclusion

Primary care providers with more than 5 years of experience were more likely to use the program. Primary care providers expressed high satisfaction with the program and reported that it was effective in supporting routine CKD screening efforts, management of early CKD, appropriate referrals, and building a collaborative relationship with nephrologists.

A 2013 study estimated that 12.5% of adults (3 million adults) in Canada were living with chronic kidney disease (CKD) between 2007 and 2009, including 0.73 million adults with stage 3 to 5 CKD.¹ Worsening of kidney function might lead to end-stage renal disease (ESRD), which is associated with high morbidity and health care costs. In Canada, the incidence of ESRD almost doubled between 1993 and 2012, with 5431 new cases of ESRD in 2012.² By the end of 2012, more than 40 000 Canadians were being treated for ESRD. The estimated annual cost of care for a patient with ESRD in Canada is $55 466.³

Given the rising prevalence of ESRD and the high cost of care for these patients, it has become a priority within Ontario to identify and improve management of CKD at an early stage to prevent or slow the progression of CKD to ESRD.⁴ Primary care in Canada is generally regarded as the entry point and system coordinator of health care. As such, primary care providers (PCPs) are ideally situated to detect and manage CKD at an early stage, as well as to refer to nephrologists when appropriate. However, PCPs might not be leveraging opportunities to discuss CKD management with patients owing to a variety of patient-, provider-, and system-level barriers, including PCPs’ self-perceived lack of adequate knowledge or skills to educate patients.^5,6 Additionally, it might be challenging for PCPs to adopt CKD-specific clinical practice guidelines into their range of care.⁷

Program objective

An online needs assessment of PCPs in Ontario recognized a need for stronger nephrologist support and further CKD education. Based on successes seen with other mentoring programs,^8,9 the Ontario Renal Network (ORN) implemented the ORN Mentorship Pilot Program (MPP) to link PCPs in Ontario to nephrologists in a collaborative relationship. The ORN MPP, modeled after the Ontario College of Family Physicians’ Collaborative Mental Health Care Network, was implemented in October 2012 as a 1-year program. The objectives of the ORN MPP were as follows:

• to promote early detection through improved knowledge and comfort testing for CKD and identifying patients with CKD;

• to prevent progression of disease and delay the need for dialysis;

• to increase appropriate and timely referrals to nephrologists from PCPs;

• to foster partnerships between nephrologists and PCPs; and

• to enrich CKD education available to PCPs.

This program was pilot-tested to provide the ORN with a better understanding of the feasibility of using a mentorship model to improve the management of CKD in primary care by improved communication and coordination with nephrologists. This report provides results from the pilot program’s evaluation, including overall PCP use of the ORN MPP, predictors of PCPs’ use of the program, changes in PCPs’ knowledge of and comfort levels with CKD care, changes in ease of access to and satisfaction with nephrologist support, and PCPs’ overall experience in the program.

Program description

In the ORN MPP, PCPs were designated as mentees and nephrologists were designated as mentors.

At the onset of the ORN MPP, all of the mentors attended an ORN-delivered training session to receive instructions and to highlight their roles and what was expected of them as mentors. In turn, each mentor arranged an orientation workshop for their respective mentees. These standardized orientation sessions familiarized mentees with the program, including the support that their mentors could offer them, as well as provided an overview of CKD care, including a primer on CKD, screening and diagnosis, medication, and lifestyle management.

After the initial sessions, the ORN MPP involved 2 components: nephrologist-led regional group learning sessions and needs-based nephrologist–primary care case consultation. The group learning sessions were generally delivered through telephone conferences and were focused on nephrologist interactions with PCPs. Mentors were required to deliver a minimum of 2 group learning sessions. Mentees were encouraged to initiate one-on-one case consultations with their mentors whenever required, and mentors were expected to respond within 48 hours. Mode of communication was based on the preferences of the mentees and mentors, and included e-mail, fax, telephone, video conference, or in person.

Participant recruitment

Nephrologists were recruited as mentors through ORN opportunity postings, and successful candidates were reimbursed by the ORN for their participation in the program. Primary care providers were recruited through brochures and with assistance from ORN-affiliated nephrologists. Participants enrolled online and provided information regarding their profession, geographic location, and contact information. Mentees were awarded up to 8 Mainpro-C credits for full participation in the program.

Data collection

Mentors were required to log all interactions with their assigned mentees. The mentor logs provided information on the frequency and mode of interactions and were used to evaluate the demand for mentorship and predictors of its use.

Mentees completed an online survey at the start and close of the program. The pre-program and post-program surveys were available online for completion in October 2012 and October 2013, respectively. Both surveys consisted primarily of close-ended questions. The surveys provided the ORN with information regarding the mentees’ characteristics, knowledge of and comfort levels with CKD care, ease of access to nephrologist support, and satisfaction with nephrologist consultations. The 2 surveys used identical questions, when applicable, so that changes in perceptions could be assessed. The post-program survey also asked mentees about their overall experience with the program, directly and through proxy questions.

Statistical analysis

Mentee baseline characteristics were described using descriptive statistics.

Univariate and multivariate linear regression models were constructed to assess the independent association of mentees’ baseline characteristics (profession, sex, years of practice, type of practice, and mean number of CKD patients seen per year) with the number of mentee-mentor interactions. The models included all of the mentee characteristics collected with the pre- program survey and were believed to be potential indicators of ORN MPP use. All characteristics were assessed as dichotomous variables (the category with the highest responses vs all other categories). Type of practice compared solo with group practices (small and large) because there was no clear indication to differentiate a small and a large group. Results were reported as odds ratios (ORs) and 95% CIs.

Changes in knowledge of and comfort levels with CKD topics, ease of access to nephrologists, and satisfaction with nephrologist support were all determined with 5-point ordinal scales and were statistically compared using paired-sample t tests. Categorical variables were compared using ² tests. Mentees who responded to both the pre-program and the post-program surveys were included in the analysis.

All statistical analyses were performed using IBM SPSS, version 22. For 2-sided tests, a P value less than .05 was considered statistically significant.

Results

Seven nephrologists (including 1 nephrologist who volunteered to participate) and 154 PCPs were recruited. Assignment of mentees to mentors was based primarily on participants’ geographic locations. Participants included PCPs and nephrologists from regions across Ontario, including Central East, Erie St Clair, Hamilton Niagara, North East, North West, Champlain, South East, and Waterloo Wellington. A total of 89 mentees completed both the pre-program and the post-program surveys and were included in our analysis. Table 1 shows the mentees’ baseline characteristics. Most mentees cared for CKD patients daily or weekly (76%), and a little more than half (51%) cared for more than 20 patients with CKD each year.

Table 1.

Baseline characteristics of primary care provider mentees: N=89.

CHARACTERISTIC	SURVEY RESPONDENTS,* N (%)
Sex
• Male	30 (33.7)
• Female	59 (66.3)
Profession
• MD (ie, primary care physician)	68 (76.4)
• Nurse practitioner	20 (22.5)
• Registered nurse	1 (1.1)
No. of years in practice
• 0–5	23 (25.8)
• 6–10	11 (12.4)
• 11–15	13 (14.6)
• 16–20	7 (7.9)
• 21–25	20 (22.5)
• > 25	15 (16.9)
Type of practice
• Solo	16 (18.0)
• Small group	56 (62.9)
• Large group	17 (19.1)
Frequency of CKD patient visits
• Daily	24 (27.0)
• Weekly	44 (49.4)
• Monthly	14 (15.7)
• Rarely	7 (7.9)
• Never	0 (0.0)
Mean no. of CKD patients seen per year
• 0	0 (0.0)
• 1–10	11 (12.4)
• 11–20	33 (37.1)
• > 20	45 (50.6)

CKD—chronic kidney disease, MD—medical doctor.

^*Percentages might not add to 100% owing to rounding.

Predictors of mentees’ use of the ORN MPP.

The mentor logs revealed that 24% of mentees did not engage their mentor after the initial orientation workshop. On average, active mentees participated in 2 one-on-one consultations; however, the median number of consultations varied across regions (range of 1 to 6 consultations). Consultations most commonly occurred via e-mail (88%) and were often resolved without in-person follow-up.

No univariate predictors of increased mentee-mentor interactions were identified (Table 2), but multivariate logistic regression indicated that increased years of practice (> 5 years vs ≤ 5 years) was associated with increased frequency of interactions (OR = 2.34, 95% CI 1.03 to 5.33; P = .043). Mentees who cared for fewer CKD patients showed a trend toward using the program more (OR = 0.49, 95% CI 0.23 to 1.03; P = .058).

Table 2.

Univariate and multivariate predictors of increased mentee-mentor interactions

VARIABLE	UNIVARIATE		MULTIVARIATE
	ODDS RATIO (95% CI)	P VALUE	ODDS RATIO (95% CI)	P VALUE
MD vs non-MD	0.59 (0.29–1.22)	.16	0.54 (0.24–1.22)	.140
Female vs male	1.11 (0.53–2.32)	.78	NA	NA
> 5 y in practice vs ≤ 5 y in practice	1.98 (0.90–4.34)	.09	2.34 (1.03–5.33)	.043
Group vs solo practice	1.20 (0.46–3.14)	.71	NA	NA
>20 CKD patients per y vs ≤ 20 CKD patients per y	0.55 (0.27–1.13)	.10	0.49 (0.23–1.03)	.058

CKD—chronic kidney disease, MD—medical doctor, NA—not applicable.

Knowledge and comfort levels.

Self-rated knowledge of and comfort levels with all CKD topics increased, ranging from a 0.51- to 1.19-point increase, representing statistically significant change (P < .001) (Table 3).

Table 3.

Changes in mentees’ knowledge of and comfort levels with various CKD topics: Items ranked on a 5-point scale (1 = not at all knowledgeable or comfortable; 2 = not very knowledgeable or comfortable; 3 = somewhat knowledgeable or comfortable; 4 = very knowledgeable or comfortable; 5 = extremely knowledgeable or comfortable).

CKD TOPIC	KNOWLEDGE MEAN		P VALUE	COMFORT LEVEL MEAN		P VALUE
	PRE PROGRAM	POST PROGRAM		PRE PROGRAM	POST PROGRAM
Counseling about smoking cessation (N = 87)	3.71	4.22	< .001	3.77	4.30	< .001
Counseling about diet modification (N = 86)	3.16	4.21	< .001	3.13	4.30	< .001
Diagnosing and treating comorbid conditions (N = 89)	3.21	4.11	< .001	3.20	4.27	< .001
Medication management (N = 89)	3.48	4.02	< .001	3.40	4.15	< .001
Screening appropriate patients for CKD (N = 89)	3.07	3.91	< .001	3.07	4.13	< .001
Appropriate referrals (N = 88)	2.83	3.74	< .001	2.75	3.94	< .001

CKD—chronic kidney disease.

Mentees reported that participation in the program improved their knowledge of and comfort levels with testing for CKD and identifying patients with CKD (79% and 83% responded “very much” or “extremely,” respectively). Similarly, 79% of mentees reported that participation “very much” or “extremely” improved their knowledge of and comfort levels with managing patients with CKD.

Ease of access to and satisfaction with nephrologist support.

Self-rated scoring for ease of access to a nephrologist increased 1.22 points (P < .001), and the post-program survey mean of 4.21 suggests that mentees were able to easily access their mentor (Table 4).

Table 4.

Changes in ease of access to and satisfaction with nephrologist support

ITEM	PRE-PROGRAM MEAN	POST-PROGRAM MEAN	P VALUE
Ease of access (N = 87)*	2.99	4.21	< .001
Satisfaction with nephrologist support
• Overall (N = 86)†	3.45	4.33	< .001
• In person (N = 37)‡	3.57	4.65	< .001
• E-mail (N = 64)‡	3.42	4.48	< .001
• Small group (N = 53)‡	3.43	4.51	< .001
• Telephone or teleconference (N = 70)‡	3.47	4.06	< .001
• Webinar or videoconference (N = 19)‡	3.47	3.63	.667

^*Item ranked on the following 5-point scale: 1 = always difficult; 2 = often difficult; 3 = sometimes easy or difficult; 4 = very easy; 5 = extremely easy.

^†Overall satisfaction represents the mean across all modes of communication for each respondent.

^‡Item ranked on the following 5-point scale: 1 = not at all satisfied; 2 = slightly satisfied; 3 = moderately satisfied; 4 = very satisfied; 5 = extremely satisfied.

Similarly, mean satisfaction scores in the post-program surveys were higher for all measured modes of communication when compared with the mean satisfaction scores in the pre-program surveys (P < .001), except for the webinar or videoconferencing communication mode (P = .667). The mentor logs showed that one-on-one case consultations were most commonly conducted through e-mail. Thus, satisfaction levels seen for e-mail interactions might be most representative of the ORN MPP. A comparison of pre-program and post-program survey results for e-mail interactions revealed that extreme satisfaction increased from 10% to 59%.

Overall experience.

About two-thirds (68%) of mentees reported that the program was “very helpful” or “extremely helpful.” Almost half (44%) of mentees reported that the program education was “always applicable” to their practice, and more than half (53%) reported that they were “extremely likely” to recommend the program to a colleague or friend. Most of the mentees (87%) reported that they would use the program again if it were offered in the future (n = 87).

Discussion and limitations

The ORN MPP was developed to support improved CKD care by Ontario PCPs in a number of ways: targeted detection, management, referral, education, and collaborative relationships with nephrologists. The program was found to be effective in meeting these objectives. Participating PCPs reported that the experience was highly helpful and applicable to their practices, and that they were extremely likely to recommend the program and would participate again if it were re-offered.

Surprisingly, mentees with more experience (> 5 years in practice) were 2.34 times more likely to use the ORN MPP. This result might be biased by self-selection into the ORN MPP. For example, PCPs with more experience who enrolled in the program might have a specific interest in improving CKD care, compared with PCPs with relatively fewer years of practice experience who might have enrolled owing to an interest in overall learning. This study also found that mentees who saw more patients with CKD (> 20 patients per year) were 0.49 times as likely to use the ORN MPP. This might be because PCPs who frequently care for patients with CKD become more comfortable and seek less support. In contrast, PCPs who are not often exposed to patients with CKD might use the ORN MPP more in order to build their expertise. Note that all PCP characteristics were captured through self-report, and frequency of visits with patients with CKD and the number of patients with CKD were not verified through medical records.

The ORN MPP provided PCP education through the initial mentee workshops, as well as through ongoing support from mentors in the form of one-on-one case consultations and group learning sessions. Based on self-report, this evaluation showed that mentees found the program to be effective in increasing knowledge of and comfort levels with all of the measured CKD topics, which included CKD detection, management approaches (medication, lifestyle modification, treatment of comorbid conditions), and appropriate referrals. Mentees’ baseline knowledge was not assessed through other means. The surveys were unable to capture which component of the ORN MPP most influenced this change. Perceived helpfulness of the ORN MPP might also vary owing to differences in mentors’ presentation style, level of enthusiasm and commitment, and timeliness and quality of responses to ad hoc consultations. Owing to the small number of PCP participants, the study did not stratify results by geographic location or mentors. Future evaluations might consider assessing how these factors might affect the use of the ORN MPP. It might be interesting to better delineate this in subsequent programs to focus resources.

Mentees also indicated that the program provided improved access to nephrologist support and satisfaction with nephrologist consultations when compared with access and satisfaction outside of the program before implementation. Ease of access to nephrologists was expected to improve, as mentors were required to respond to questions within 48 hours, as per the responsibilities outlined in their roles. Satisfaction with nephrologist support might have increased for a number of reasons. First, the mentors were accountable under their contracts with the ORN and had been informed by the ORN that they were required to provide high-quality support to the mentees. Primary care providers who enrolled in the ORN MPP might represent a population of PCPs who had more need for nephrologist support than other PCPs do. For example, this population might have lacked access to nephrologist support or were inexperienced in caring for patients with CKD. Use of more in-depth data collection, such as interviews, might help us better understand how to best target mentees who would benefit from the program and how to increase participation in the program.

Both the mentees and mentors might represent a group of providers who are more interested in quality improvement and are more engaged in CKD care than their peers are. Additionally, the pre-program and post-program surveys were completed by only 89 of 154 ORN MPP participants, presenting the possibility of nonresponse bias; those who did not respond might have had a different experience than those who did. This might limit the generalizability of the evaluation findings to other PCPs in Ontario. We were also unable to evaluate whether this program actually led to an improvement in CKD care or in patient health outcomes. Future evaluations of any subsequent mentorship model might consider including patients to assess if they believed their primary care management of CKD improved or to assess health outcomes.

Conclusion

Based on the mentee perspective, this study supports a mentorship model that fosters partnership between PCPs and nephrologists and supports PCPs in the care of patients with CKD. The results of this pilot study were similar to those of the Ontario College of Family Physicians’ Collaborative Mental Health Care Network, which linked family physicians with mental health specialists, and was found to satisfy family physicians’ need for support from other specialists.⁸ Other Canadian mentorship models to support PCPs have reported positive results, such as the Champlain BASE (Building Access to Specialist Care through eConsultation) project, which provides PCPs with an opportunity for electronic consultations via an online interface, and the RACE (Rapid Access to Consultative Expertise) program in Vancouver, BC, which uses a hot-line for PCPs to contact other specialists via telephone for advice. These programs increased PCP access to support from other specialists and bidirectional communication despite not having face-to-face interactions.^10,11 The ORN will use the findings from the ORN MPP and similar models seen in the literature to develop a sustainable provincial model for supporting PCPs in the management of CKD. Future renal mentorship models should consider the adoption of the ORN KidneyWise Clinical Toolkit.¹² This tool kit was developed in parallel with the ORN MPP and provides PCPs with guidance on which patients are at high risk of developing CKD and recommendations on how to properly diagnose and best manage patients to reduce the risk of further disease progression. It includes a clinical algorithm, an evidence summary, and a standardized outpatient nephrology referral form, and has been vetted by nephrology and primary care leaders within Ontario and across Canada.

EDITOR’S KEY POINTS

• Identification of chronic kidney disease (CKD) at an early stage is critical to improving patient outcomes. Primary care providers (PCPs) are ideally situated to detect, manage, and appropriately refer patients with CKD.

• The Ontario Renal Network Mentorship Pilot Program was developed to support improved CKD care by PCPs. The program provided PCPs with educational workshops, as well as with ongoing support from nephrologists in the form of one-on-one case consultations and group learning sessions.

• This program was effective in increasing PCPs’ knowledge of and comfort levels with many CKD topics (eg, CKD detection). It also improved PCPs’ access to nephrologist support and satisfaction with nephrologist consultations when compared with access and satisfaction before program implementation.

POINTS DE REPÈRE DU RÉDACTEUR

• Diagnostiquer une maladie rénale chronique (MRC) à un stade précoce est une étape critique pour améliorer les issues d’un patient. Les soignants de première ligne (SPL) sont en situation idéale pour identifier et traiter les patients souffrant de MRC et les diriger en spécialité.

• Le programme pilote de mentorat du réseau rénal de l’Ontario a été créé pour favoriser un meilleur traitement de la MRC par les SPL. Ce programme offre aux SPL des ateliers de formation de même qu’une assistance continue de la part de néphrologues sous la forme de consultations cas par cas et de sessions d’apprentissage en groupe.

• Grâce à ce programme, les SPL ont amélioré leurs connaissances sur plusieurs sujets liés à la MRC (p. ex. sur son identification) et se sont sentis plus à l’aise pour traiter cette maladie. Grâce au programme, les SPL ont eu un meilleur accès aux néphrologues; ils se sont dits très satisfaits des consultations par rapport à la situation qui existait avant la mise en place du programme.