Milestones on the social accountability journey

Abstract

Objective

To assess the effect of different levels of exposure to the Northern Ontario School of Medicine’s (NOSM’s) distributed medical education programs in northern Ontario on FPs’ practice locations.

Design

Cross-sectional design using longitudinal survey and administrative data.

Setting

Canada.

Participants

All 131 Canadian medical graduates who completed FP training in 2011 to 2013 and who completed their undergraduate (UG) medical degree or postgraduate (PG) residency training or both at NOSM.

Intervention

Exposure to NOSM’s medical education program at the UG (n = 49) or PG (n = 31) level or both (n = 51).

Main outcome measures

Primary practice location in September of 2014.

Results

Approximately 16% (21 of 129) of FPs were practising in rural northern Ontario, 45% (58 of 129) in urban northern Ontario, and 5% (7 of 129) in rural southern Ontario. Logistic regression found that more rural Canadian background years predicted rural practice in northern Ontario or Ontario, with odds ratios of 1.16 and 1.12, respectively. Northern Canadian background, sex, marital status, and having children did not predict practice location. Completing both UG and PG training at NOSM predicted practising in rural and northern Ontario locations with odds ratios of 4.06 to 48.62.

Conclusion

Approximately 61% (79 of 129) of Canadian medical graduate FPs who complete at least some of their training at NOSM practise in northern Ontario. Slightly more than a quarter (21 of 79) of these FPs practise in rural northern Ontario. The FPs with more years of rural background or those with greater exposure to NOSM’s medical education programs had higher odds of practising in rural northern Ontario. This study shows that NOSM is on the road to reaching one of its social accountability milestones.

The shortage of rural physicians is particularly acute in northern Ontario, where 29% of physicians’ practices are in rural areas compared with 39% of the population.^1–4 The Northern Ontario School of Medicine (NOSM) has a social accountability goal to educate physicians to practise in northern Ontario.^5,6 We examined practice location of FPs who received their undergraduate (UG) or postgraduate (PG) medical education or both at NOSM to assess NOSM’s success in reaching one of the first milestones on its social accountability journey.

There is emerging global interest in measuring NOSM’s success, as NOSM is one of only a few medical schools in the world with an explicit social accountability mandate.^7–11 Evidence from northern Ontario,^12,13 Ontario,¹⁴ and Canada,¹⁵ as well as international reviews,^16–19 shows that if medical schools accept learners who have lived in rural areas and educate them in rural areas in a positive manner, then these learners are more likely to practise in rural locations.

Fulfilling a social accountability mandate is particularly challenging in northern Ontario, where 833000 people are scattered across 807000 km².⁴ Approximately 56% of the population is clustered in 5 large urban areas (Timmins, North Bay, Sault Ste Marie, Thunder Bay, and Sudbury), each with 43 000 to 161 000 people. Another 5% reside in 3 smaller urban centres of 11 000 to 15 000 people. Northern Ontario also has a higher proportion of 2 cultural-linguistic minority groups relative to the whole province. Indigenous people constitute 14% of the northern Ontario population versus 2% provincially, and Francophones constitute 18% versus 5% provincially.^20,21 Northern Ontarians, particularly indigenous peoples and Francophones, have poorer health status than the average Ontarian.^22–25

At NOSM, all of northern Ontario is viewed as the campus, and NOSM serves as the Faculty of Medicine for Laurentian University in Sudbury (2011 census metropolitan area population of 161 000) and for Lakehead University in Thunder Bay (122 000)—cities that are located 1000 km apart by road. The NOSM selects academically qualified medical school applicants from northern, rural, indigenous, or Francophone backgrounds to reflect northern Ontario’s demographic characteristics.²⁶ The distributed, community-engaged learning model at NOSM enables skilled clinicians to provide medical learners with education and clinical training in the smaller cities, towns, and communities of northern Ontario, with less dependence on traditional acute care hospitals in urban areas or large regional hospitals.^27,28 Learners provide care to patients with different socioeconomic backgrounds, cultures, and care needs while training in interprofessional settings.

We assessed the overall influence of NOSM’s UG admission criteria and UG and PG educational programs on FPs’ practice locations. While the international literature has identified rural background as a predictor of rural practice location,^16–19 it is not known if NOSM’s selection process has achieved the desired effect or if other factors (eg, northern background) would be important predictors.

METHODS

This cross-sectional study of practice location compares UG and PG school groups (described below) as part of a prospective, longitudinal research program.²⁹ We used data from surveys, NOSM, and medical licensing agencies to predict practice location for NOSM’s learners who completed their FP training in 2011 to 2013. Research ethics boards at Laurentian University and Lakehead University provided ethical approval.

Data collection and participants

We started surveying learners and extracting administrative data in 2005 when NOSM accepted its first cohort of medical students. We distribute voluntary questionnaires to medical learners during their NOSM UG education or NOSM PG residency training.

We categorized learners who completed family medicine programs during 2011 to 2013 (n = 131)—both NOSM UG and NOSM PG (n = 51); NOSM UG only (n = 49); and NOSM PG only (n = 31)—to reflect different exposure to NOSM’s admission criteria and medical education programs. We excluded FPs who completed training in 2014 (n = 57) because they would only be 2 to 3 months into full licensed practice or might be pursuing additional training at the time of analysis. We excluded international medical graduates (IMGs) (n = 29) because their UG training differed from that of Canadian medical graduates (CMGs), and because IMGs were significantly older by 5.6 years (t = 4.77, df = 158, P < .001) and had a significantly lower proportion of women (24% vs 66%; χ² = 17.51, df = 1, P < .001). In addition, a gap in survey coverage caused by delayed funding meant that key information on rural or northern background, practice intentions, and other variables was not available for IMGs. As well, all IMGs belonged to the NOSM PG–only group, and their data cannot be used to assess different levels of exposure to NOSM programs.

We used NOSM’s social accountability mandate to identify 3 overlapping outcomes for primary practice location: rural northern Ontario, northern Ontario, and rural Ontario. We extracted primary practice location in September 2014 from publicly available data on provincial or territorial medical regulatory agencies’ websites (eg, College of Physicians and Surgeons of Ontario). We matched practice location (outcome) and locations where the doctor had lived up to age 18 (independent variable) to census subdivisions categorized by geographic region (eg, northern Ontario, rest of Ontario, other Canadian province or territory, other country). Northern Ontario was defined as the area of 3 (former) northern Ontario district health councils³⁰ that constitute NOSM’s service region. Northern Canada was defined by the respective province or territory’s Ministry of Health. We used Statistics Canada’s Statistical Area Classification that categorizes census metropolitan areas or census agglomerations as urban and all other census subdivisions as rural.³¹

Analyses

We defined 3 predictors a priori for the logistic regression models: years lived in rural Canada, years lived in northern Canada, and UG-PG medical school combination. We also analyzed intent to practise rurally, which can be predictive of rural practice,^32,33 and whether the physicians had contracts that bound them to practise initially in a specific area. Other potential predictors included age, marital or partnership status, and presence of children (all at PG entry) and sex, French-language ability, and cultural or linguistic background. These alternative predictors were analyzed with hierarchical log-linear models to identify a subset of predictors with the highest number of significant interactions with practice location. Selected alternative predictors replaced predictors that were non-significant (P > .05) in previous logistic regression models. For the population of 131 learners, we excluded cases with missing data, restricted models to 3 predictors and 1 outcome, and used SPSS, version 20, for all analyses.

RESULTS

We obtained demographic data, practice intentions, and service obligations for 60% (79 of 131) to 100% of practising FPs and primary practice location for 98% (129 of 131) (Tables 1 and 2).^30,31 A total of 16% (21 of 129) of FPs located their primary practices in rural northern Ontario, 45% (58 of 129) practised in urban northern Ontario, and 5% (7 of 129) practised in rural southern Ontario. Of the 29 IMG FPs, 4 (14%) had located in rural Ontario (2 each in the north and south), with most practising in urban areas in northern (n = 15) and southern (n = 10) Ontario (Table 2).^30,31

Table 1.

Characteristics of practising FPs who finished their PG training in 2011 to 2013 and completed their UG or PG medical education or both at NOSM

CHARACTERISTIC*	VALUE	N	DATA COVERAGE, %
Age at entry into PG program, y	Mean (SD) = 31 (5.51) Median = 29 Range = 24–50	131	100
Female, n (%)	87 (66.4)	131	100
Married or partnered at entry into PG program, n (%)	59 (56.2)	105	80
Had children at entry into PG program, n (%)	33 (34.4)	96	73
Rural Canadian background up to age 18 y	25th percentile = 0 Median = 5 75th percentile = 18 Bimodal distribution, n (%): 42 (42.9) had 0 y; 25 (25.5) had 18 y	98	75
Northern Canada background up to age 18 y	25th percentile = 11 Median = 18 75th percentile = 18 Bimodal distribution, n (%): 18 (17.3) had 0 y; 61 (58.7) had 18 y	104	79
Evidence of ability to communicate in French, n (%)	49 (37.4)	131	100
Indigenous background, n (%)	10 (9.2)	109	83
Intention to practise in a rural community as reported at end of UG training, n (%)†	51 (64.6)	79	60
Intention to practise in a northern Ontario community as reported at end of UG training, n (%)‡	73 (92.4)	79	60
UG-PG medical school, n (%)	NOSM UG and PG: 51 (38.9) NOSM UG only: 49 (37.4) NOSM PG only: 31 (23.7)	131	100
Service contract, n (%)§	Yes: 34 (35.8)	95	73

NOSM—Northern Ontario School of Medicine, PG—postgraduate, UG—undergraduate.

^*Characteristics with less than 98% data coverage were obtained from surveys of NOSM UG or PG trainees.

^†Strongly or moderately inclined to practise in at least 1 community of population size < 1000, 1000–5000, or 5000–10 000.

^‡Strongly or moderately inclined to practise in northeast or northwest Ontario.

^§The NOSM PG–only group has 22 of 31 missing values.

Table 2.

Primary practice location of practising FPs (N = 129 CMGs, N = 29 IMGs) who finished their PG training in 2011 to 2013 and completed their UG or PG medical education or both at NOSM

PRACTICE LOCATION	NORTHERN ONTARIO*		SOUTHERN ONTARIO		OTHER NORTHERN CANADA		OTHER SOUTHERN CANADA		TOTAL
	CMG, N (%)	IMG, N	CMG, N (%)	IMG, N	CMG, N (%)	IMG, N	CMG, N (%)	IMG, N	CMG, N (%)	IMG, N
Rural†	21 (16.3)	2	7 (5.4)	2	2 (1.6)	0	4 (3.1)	0	34 (26.4)	4
Urban	58 (45.0)	15	23 (17.8)	10	0 (0.0)	0	14 (10.9)	0	95 (73.6)	25
Total	79 (61.2)	17	30 (23.3)	12	2 (1.6)	0	18 (14.0)	0	129 (100.0)	29

CMG—Canadian medical graduate, IMG—international medical graduate, NOSM—Northern Ontario School of Medicine, PG—postgraduate, UG—undergraduate.

^*Northern Ontario was defined by 3 district health council boundaries in 2003³⁰ and northern Canada was defined in 2014 by the applicable provincial or territorial Ministry of Health.

^†Rural location was defined in 2011 by Statistics Canada as any census subdivision that was not a census metropolitan area or census agglomeration.³¹

Increasing number of rural background years was a statistically significant predictor of rural northern Ontario and rural Ontario practice location for CMG FPs, with odds ratios of 1.16 and 1.12, respectively (Table 3). The number of northern Canadian background years did not predict any location. The UG-PG school combination predicted practice location, with the NOSM UG and NOSM PG combination displaying the highest odds ratios: 8.62 for practice in rural northern Ontario, 48.62 for northern Ontario, and 4.06 for rural Ontario.

Table 3.

Predicting rural or northern Ontario practice location of practising FPs (97 of 131 CMGs) who finished their PG training in 2011 to 2013 and completed their UG or PG medical education or both at NOSM

PREDICTOR	β	STANDARD ERROR (β)	WALD χ2	P VALUE	OR	95% CI FOR OR
Rural northern Ontario vs other location*
• Rural background, y	0.15	0.04	11.31	< .01	1.16	1.06–1.27
• Northern Canada background, y	−0.03	0.05	0.30	.58	0.97	0.88–1.08
• NOSM UG only	NA	NA	7.83	.02	1.00	NA
• NOSM PG only	1.84	1.57	1.36	.24	6.27	0.29–136.74
• NOSM UG and PG	2.15	0.77	7.72	< .01	8.62	1.89–39.36
• Constant	−4.17	1.12	13.90	< .001	0.02	NA
Northern Ontario vs other location†
• Rural background, y	−0.005	0.04	0.02	.89	1.00	0.92–1.07
• Northern Canada background, y	0.04	0.05	.064	.43	1.04	0.94–1.16
• NOSM UG only	NA	NA	30.80	< .001	1.00	NA
• NOSM PG only	2.41	1.21	3.98	.05	11.13	1.04–118.84
• NOSM UG and PG	3.88	0.71	30.15	< .001	48.62	12.15–194.48
• Constant	−1.81	0.99	3.38	.07	0.16	NA
Rural Ontario vs other location§
• Rural background, y	0.12	0.04	10.06	< .01	1.12	1.05–1.21
• Northern Canada background, y	−0.01	0.05	0.04	.85	0.99	0.91–1.08
• NOSM UG only	NA	NA	5.20	.07	1.00	NA
• NOSM PG only	0.98	1.41	0.48	.49	2.67	0.17–42.35
• NOSM UG and PG	1.40	0.62	5.19	.02	4.06	1.22–13.59
• Constant	−3.13	0.94	11.15	< .01	0.04	NA

CMG—Canadian medical graduate, NA—not applicable, NOSM—Northern Ontario School of Medicine, OR—odds ratio, PG—postgraduate, UG—undergraduate.

^*Model ${\chi }_4^2=19.4$, P < .01; goodness-of-fit test ${\chi }_6^2=1.3$, P = .97, Nagelkerke R² = 0.31. Correctly classified cases: 84.5% for constant only vs 85.6% for full model. Hosmer-Lemeshow goodness-of-fit test: P > .05 indicated that the model was a good fit with Nagelkerke R² interpreted as a pseudo R². The assumption of linearity in the logit for rural or northern background years was not rejected for any model (P > .11).

^†Model ${\chi }_4^2=54.3$, P < .01; goodness-of-fit test ${\chi }_7^2=6.4$, P = .50, Nagelkerke R² = 0.58. Correctly classified cases: 59.8% for constant only vs 84.5% for full model.

^§Model ${\chi }_4^2=14.4$, P < .01; goodness-of-fit test ${\chi }_7^2=6.6$, P = .47, Nagelkerke R² = 0.22. Correctly classified cases: 79.4% for constant only vs 81.4% for full model.

After controlling for rural background years and UG-PG school, practice contract status (ie, yes, no, unknown) was not a significant predictor of any outcome (P > .58) nor was intention to practise in a rural community of 10 000 people or less as reported at the end of NOSM UG education (P > .17). Alternative predictors (ie, sex, married or partnered or not, presence of children, French-language ability) were not significant (P > .05) based on forward selection or backward elimination logistic regression models.

A significantly higher percentage of FPs who went to NOSM for UG plus PG training practised in rural northern Ontario (26%, P = .03) or northern Ontario (94%, P < .001) compared with those with NOSM UG degrees only (6% and 20%, respectively). Those with NOSM PG training only exhibited an intermediate percentage (Table 4).

Table 4.

Primary practice location by UG-PG training location for practising FPs (N = 129 CMGs) who finished PG training in 2011 to 2013 and completed UG or PG medical education or both at NOSM: A) Rural northern practice location (Pearson ${\chi }_{\mathit{2}}^{\mathit{2}}=\mathit{6.90}$, P = .03); B) Northern practice location (Pearson ${\chi }_{\mathit{2}}^{\mathit{2}}=\mathit{59.17}$, P < .001); C) Rural practice location (Pearson ${\chi }_{\mathit{2}}^{\mathit{2}}=\mathit{3.39}$, P = .19).

A)
UG-PG SCHOOL	PRIMARY PRACTICE LOCATION			TOTAL, N (%)
	RURAL NORTHERN ONTARIO, N (%)*	ELSEWHERE, N (%)*	OTHER RURAL NORTHERN CANADA, N (%)†
NOSM UG and PG	13 (25.5) ↑	38 (74.5)↓	0 (0.0)	51 (100.0)
NOSM UG only	3 (6.1) ↓	46 (93.9) ↑	2 (4.1)	49 (100.0)
NOSM PG only	5 (17.2)	24 (82.8)	0 (0.0)	29 (100.0)
Total	21 (16.3)	108 (83.7)	2 (1.6)	129 (100.0)

B)
UG-PG SCHOOL	PRIMARY PRACTICE LOCATION			TOTAL
	NORTHERN ONTARIO, N (%)*	ELSEWHERE, N (%)*	OTHER NORTHERN CANADA, N (%)†
NOSM UG and PG	48 (94.1) ↑	3 (5.9) ↓	0 (0.0)	51 (100.0)
NOSM UG only	10 (20.4) ↓	39 (79.6) ↑	2 (4.1)	49 (100.0)
NOSM PG only	21 (72.4)	8 (27.6)	0 (0.0)	29 (100.0)
Total	79 (61.2)	50 (38.8)	2 (1.6)	129 (100.0)

C)
UG-PG SCHOOL	PRIMARY PRACTICE LOCATION			TOTAL
	RURAL ONTARIO, N (%)	ELSEWHERE, N (%)	OTHER RURAL CANADA, N (%)†
NOSM UG and PG	15 (29.4)	36 (70.6)	0 (0.0)	51 (100.0)
NOSM UG only	7 (14.3)	42 (85.7)	6 (12.2)	49 (100.0)
NOSM PG only	6 (20.7)	23 (79.3)	0 (0.0)	29 (100.0)
Total	28 (21.7)	101 (78.3)	6 (4.7)	129 (100.0)

CMG—Canadian medical graduate, NOSM—Northern Ontario School of Medicine, PG—postgraduate, UG—undergraduate.

^*Arrows indicate that the observed count was significantly higher (↑) or lower (↓) than the expected count based on adjusted standardized Pearson residuals.

^†Included in the “elsewhere” category for the χ² tests.

Of the 100 FPs who completed at least their UG training at NOSM and went to NOSM or went elsewhere for their PG training, 58% were in northern Ontario, 22% were in rural Ontario, and another 6% were in other rural Canadian communities (Table 4). Approximately 86% of the 80 FPs who went to NOSM for at least their PG training and went to NOSM or elsewhere for their UG medical education were practising in northern Ontario and 26% were in rural Ontario.

DISCUSSION

Practice location predictors

More rural Canadian background years predicted rural practice location in northern Ontario or Ontario, which agrees with the literature^16–19 and supports NOSM’s UG admission policies. Years of northern background did not predict any practice location after controlling for rural background years and UG-PG school. This was unexpected as the literature suggests, albeit weakly, that growing up in underserved areas might predispose graduates to practise in underserved areas.¹⁸ However, not all areas in northern Canada are underserved and this might dilute any effect. In addition, 59% of FPs had the maximum 18 years of northern background and there might be insufficient variability to differentiate among practice locations.

That NOSM PG education was a strong predictor of a northern Ontario practice location was consistent with a study of earlier family medicine programs in northern Ontario.¹³ Further, NOSM UG plus PG medical education also predicted rural practice in northern Ontario or Ontario. Studies from the University of Manitoba³⁴ and Memorial University of Newfoundland³⁵ have shown that exposure to both UG and PG medical education at the same school was associated with a higher likelihood of practising in the province or in rural areas. For example, 94% of the 51 CMG FPs who completed both their UG and PG training at NOSM were practising in northern Ontario. By comparison, 76% of FPs and other specialists enrolled as medical students during 1998 to 2009 had located their first practice in Manitoba after UG and PG training at the University of Manitoba.³⁴ In 2014, 49% of Memorial University of Newfoundland’s UG- and PG-trained FPs and other specialists who graduated as medical students during 1973 to 2008 were practising in Newfoundland, with 16% in rural Canada.³⁵ Approximately 29% of the FPs who completed both UG and PG training at NOSM were practising in rural Ontario.

Our study did not find any significant association between practice location and learners’ sex, marital or partnered status, presence or absence of children, French-language ability, intention to practise rurally, or practice contract status. The literature provides varying evidence: most Canadian studies found no evidence of an association with sex,^13,34,35 with one exception³⁶; married or partnered FPs and other specialists were more likely to be practising in rural areas of Manitoba³⁴; and systematic literature reviews suggest a positive association between intent to practise rurally and rural practice.^16–18

Limitations and strengths

There were systematic differences among UG-PG groups because NOSM UGs, but not necessarily NOSM PGs, were selected to have strong rural or northern backgrounds. Thus, there might be some redundancy when school and background were analyzed together. All groups experienced NOSM’s educational programs, and NOSM’s training effect might be underestimated. There likely is social desirability bias in self-reported practice intention, which might peak early in the UG program, and so we used intention just before UG completion. We excluded 29 IMG residents because we did not have key survey data and because their age, sex proportions, and medical education background differed significantly from CMGs. Subsequent studies might investigate why 86% of these IMGs practise in urban areas.

A strength of this study is that basic demographic data and practice location were available for 73% to 100% and 98% of the 131 CMG FPs, respectively. Future studies will assess if practice profile (eg, scope of practice, patient populations) and other practice characteristics (eg, interprofessional care teams) of NOSM-trained physicians will meet northern Ontarians’ needs.

Conclusion

More than 60% of CMG FPs who completed UG or PG training or both at NOSM had located their medical practices in northern Ontario in 2014. Slightly more than a quarter of these FPs practising in northern Ontario are in rural communities. The distribution in northern Ontario of FPs who trained at NOSM for UG or PG degrees or both does not yet match the 39% of the population who live in rural areas and so further research is needed into the roles of nature (eg, rural background) and nurture (eg, medical education, incentives) in influencing practice location. However, it is promising that FPs were more likely to practise in rural northern Ontario if they had greater exposure to NOSM’s educational programs. Our study shows that NOSM has increased the number of CMG FPs who practise in rural and northern Ontario and is on the road to reaching one of its main social accountability milestones.

EDITOR’S KEY POINTS

• The Northern Ontario School of Medicine (NOSM) has a social accountability mandate to educate physicians to practise in northern Ontario. This longitudinal tracking study of NOSM medical learners reports on an analysis of primary practice locations in 2014 for Canadian medical graduates (CMGs) trained as FPs.

• Overall, 61% of CMG FPs who completed undergraduate or postgraduate training or both at NOSM were practising in northern Ontario; 22% of CMG FPs were practising in rural Ontario; and 5% were practising in other rural Canadian communities.

• A total of 94% of CMG FPs who completed both their undergraduate and postgraduate training at NOSM were practising in northern Ontario.

POINTS DE REPÈRE DU RÉDACTEUR

• Pour répondre à un objectif d’imputabilité sociale, l’École de médecine du Nord de l’Ontario (ÉMNO) s’est donné le mandat de former des médecins capables de pratiquer dans le Nord de l’Ontario. Cette étude longitudinale pour retracer les étudiants en médecine formés à l’ÉMNO présente une analyse des endroits où, en 2014, les diplômés canadiens en médecine familiale (DCMF) ont commencé leur pratique.

• Dans l’ensemble, 61 % des DCMF qui ont complété leur formation des premier et deuxième cycles à l’ÉMNO pratiquaient dans le Nord de l’Ontario; 22 % travaillaient en Ontario rural; et 5 % dans d’autres communautés rurales au Canada.

• Au total, 94 % des DCMF qui ont effectué leurs études de premier et de deuxième cycles à l’ÉMNO pratiquaient dans le Nord de l’Ontario.