Does clinical exposure matter?

Abstract

Objective

To determine the number of patient visits, patient demographic information, and diagnoses in an urban ambulatory care setting in a family medicine residency program, and assess the correlation between the number of patient visits and residents’ in-training examination (ITE) scores.

Design

Retrospective analysis of data from resident practice profiles, electronic medical records, and residents’ final ITE scores.

Setting

Family medicine teaching unit in a community hospital in Barrie, Ont.

Participants

Practice profile data were from family medicine residents enrolled in the program from July 1, 2013, to June 30, 2014, and electronic medical record and ITE data were from those enrolled in the program from July 1, 2010, to June 30, 2015.

Main outcome measures

Number of patient visits, patient characteristics (eg, sex, age), priority topics addressed in clinic, resident characteristics (eg, age, sex, level of residency), and residents’ final ITE scores.

Results

Between July 1, 2013, and June 30, 2014, there were 11 115 patient visits. First-year residents had a mean of 5.48 patient visits per clinic, and second-year residents had a mean of 5.98 patient visits per clinic. A Pearson correlation coefficient of 0.68 was found to exist between the number of patients seen and the final ITE scores, with a 10.5% difference in mean score between residents who had 1251 or more visits and those who had 1150 or fewer visits. Three diagnoses (ie, epistaxis, meningitis, and neck pain) deemed important for Certification by the College of Family Physicians of Canada were not seen by any of the residents in clinic.

Conclusion

There is a moderate correlation between the number of patients seen by residents in ambulatory care and ITE scores in family medicine. It is important to assess patients’ demographic information and diagnoses made in resident practices to ensure an adequate clinical experience.

Ambulatory care experiences serve as the backbone of postgraduate training in family medicine.¹ In the United States, a few studies have reported on the number of visits to residents in family medicine clinics,^2,3 and the Accreditation Council for Graduate Medical Education requires that a resident have at least 1650 patient visits in the ambulatory care setting over the course of the 3-year residency.⁴ There exists a paucity of literature on how many patients are seen during family medicine training in Canada. There is also no accreditation standard that relates to a minimum number of patient visits. Moreover, it is important to track the clinical exposures that family medicine residents (FMRs) receive to ensure they graduate with the necessary knowledge and skills. Tracking of clinical experiences also allows for curricular adjustments at a program level.⁵

The Department of Family and Community Medicine (DFCM) at the University of Toronto in Ontario has developed a competency-based curriculum that defines areas of clinical exposure required by all trainees in order to become practice-ready family physicians.⁶ To assess the breadth of the clinical experience, the DFCM has introduced a resident practice profile (RPP) tool. In addition, as part of the assessment of the family medicine expert role, FMRs write 4 in-training examinations (ITEs) during residency, with the last examination written in the 21st month of a 24-month residency. In order to obtain an independent licence to practise family medicine in Canada, FMRs must pass the College of Family Physicians of Canada (CFPC) Certification examination. This examination is based on the CFPC’s priority topics and key features for assessment in family medicine document^7,8 and is usually written in the 22nd month of residency.

The objectives of this study were to determine the overall number of patient visits to residents in a teaching unit in an ambulatory care setting, the training levels of the FMRs who encountered these patients, and how the overall number of patient visits correlated with ITE scores. In addition, this study sought to describe the types of patients seen by FMRs and to determine which of the CFPC’s priority topics were encountered during training.

METHODS

Setting

Royal Victoria Regional Health Centre (RVH) in Barrie, Ont, is a 319-bed acute care community hospital. Barrie is a suburban city located north of Toronto with a population of approximately 145 000.

There are 18 FMRs trained at RVH per year—9 first-year residents and 9 second-year residents. The program operates in a horizontal fashion in which residents spend 3 clinical half-days per week in the teaching unit caring (under supervision) for their own roster of up to 200 patients. Each half-day is 3 hours in length. There is also a weekly mandatory academic half-day, which addresses core topics in family medicine delivered via lectures, seminars, and workshops. Family medicine residents also participate in in-patient and ambulatory care experiences to address the learning outcomes defined in the curriculum.

Data collection

The 3 sources of data used to assess the ambulatory experiences of FMRs were the RPP, electronic medical records (EMRs), and final ITE results.

The DFCM RPP is a Web-based system used to track patient profiles. Family medicine residents are expected to enter all the information about their patient visits (ie, patient age, sex, procedures performed, and diagnosis or ICD-9 [International Classification of Diseases] diagnostic code) for all of their ambulatory care clinics in family medicine. They are able to enter up to 3 diagnoses or codes per visit. Data from the RPP for this study were generated from July 1, 2013, to June 30, 2014, by 18 FMRs spanning 2 cohorts. The first cohort (9 residents) began training in July 2012, and the second (9 residents) began in July 2013. We also mapped our tracking codes in the RPP to 71 of the 101 CFPC priority topics.⁷ Some of the topics were too general to be mapped to an ICD-9 code and some did not exist on lists of common codes billed by family physicians (Box 1).⁷

Box 1. Priority topics not included in the RPP data.

Excluded from analysis because topic was too general

• Chronic disease, counseling, difficult patient, disability, elderly, family issues, immigrants, in children, infections, learning, lifestyle, multiple medical problems, newborn patients, patients, palliative care, self-learning, skin disorder, travel medicine

Excluded from analysis because topic did not have a common billing code for ambulatory care

• Advanced cardiac life support, antibiotics, bad news, crisis, dehydration, domestic violence, gender-specific issues, poisoning, rape or sexual assault, somatization, trauma, violent or aggressive patient

Priority topics from the College of Family Physicians of Canada.⁷

The EMR generated the number of patient visits in ambulatory care clinics for all FMRs who were enrolled at RVH from July 1, 2010, to June 30, 2015. The EMR data from July 1, 2013, to June 30, 2014, were used to determine the mean number of visits FMRs had per clinic. The FMR ITE results were used for second-year residents who wrote the test in the 21st month of training.

The RVH Research Ethics Board approved the study.

Data analysis

Total numbers and percentages were used for univariate (ie, total number of patient visits, total number of visits per clinic, total number of priority topics seen, and total number of residents in the age, sex, and source of medical degree categories) and bivariate (ie, total number of visits per clinic by residency level; and total number of visits, patient demographic information of those visits, and the residents’ characteristics of those patient visits [ie, sex, source of medical degree, and residency level]) relationships. The relationship between the number of patient encounters and ITE scores was determined using the Pearson correlation coefficient.

RESULTS

Eighteen FMRs were included in the sample assessing the RPP. There were 8 (44.4%) men and 10 (55.6%) women. The mean (SD) age of the FMRs entering the family medicine residency program was 28.6 (4.5) years. We dichotomized the resident age variable: 8 (44.4%) residents were younger than age 27 and 10 (55.6%) residents were aged 27 or older. There were 9 first-year residents and 9 second-year residents; 14 (77.8%) residents were Canadian medical graduates and 4 (22.2%) were international medical graduates.

The EMR data from July 1, 2013, to June 30, 2014, showed FMRs (N = 18) had 11 115 patient visits; first-year residents had 5903 patient visits over 1078 half-day clinics (5.48 visits per clinic) and second-year residents had 5212 visits over 871 half-day clinics (5.98 visits per clinic). Figure 1 reports the average number of visits per half-day by residency level and training period.

Figure 1.

Average number of visits per half-day, by resident level and academic quarter

There were 29 second-year residents who wrote their final ITE in the 21st month of training, including 16 FMRs whose RPP data were analyzed as they began residency in July 2012 and July 2013. Two of the 18 residents whose RPP data were analyzed had not written the ITE in their 21st month of residency.

A Pearson correlation coefficient of 0.68 (95% CI 0.42–0.84; P < .001) was found between the number of patient visits and final ITE scores. Table 1 shows FMRs’ mean ITE scores by number of patient visits. Mean ITE scores increased when the number of patient visits was higher: compared with the ITE mean score when the number of patient visits was between 1051 and 1150, the mean score increased by 10.5% when there were more than 1251 patient visits and by 3.3% when there were between 1151 and 1250 patient visits.

Table 1.

Relationship between residents’ mean ITE score and the number of patient visits

NO. OF PATIENT VISITS	NO. OF RESIDENTS	MEAN (SD) ITE SCORE
1051–1150	11	66.5 (4.2)
1151–1250	10	69.8 (5.6)
1251–1350	8	77.0 (3.8)*

ITE—in-training examination.

^*Pair-wise t test suggests residents with 1251 to 1350 patient visits have significantly higher ITE scores than residents with fewer visits do (P = .0003).

Using the RPP data, FMRs recorded 9641 patient visits in the RPP (an 86.7% data capture rate compared with EMR data). Table 2 summarizes patient characteristics stratified by FMR sex, residency level, and source of medical degree. There was a statistically significant association between the sex of the patient and the sex of the resident (P < .001), with male residents seeing more male patients and female residents seeing more female patients.

Table 2.

Number of patient visits to, as well as patient characteristics of those who visited, residents as captured in the RPP

RESIDENTS AND RESIDENT CHARACTERISTICS	NO. OF PATIENT VISITS	PATIENT SEX, N (%)		PATIENT AGE, Y, N (%)
		MALE	FEMALE	0–2	3–10	11–20	21–40	41–64	65–74	≥ 75
All residents (N = 18)	9641	3789 (39.3)	5852 (60.7)	1082 (11.2)	571 (5.9)	651 (6.8)	2400 (24.9)	3337 (34.6)	971 (10.1)	629 (6.5)
Male residents (N = 8)	4538	2038 (44.9)	2500 (55.1)	494 (10.9)	269 (5.9)	224 (4.9)	1100 (24.2)	1664 (36.7)	465 (10.3)	322 (7.1)
Female residents (N = 10)	5103	1751 (34.3)	3352 (65.7)	588 (11.5)	302 (5.9)	427 (8.4)	1300 (25.5)	1673 (32.8)	506 (9.9)	307 (6.0)
First-year residents (N = 9)	5056	2009 (39.7)	3047 (60.3)	609 (12.0)	298 (5.9)	321 (6.3)	1312 (26.0)	1743 (34.5)	450 (8.9)	323 (6.4)
Second-year residents (N = 9)	4585	1780 (38.8)	2805 (61.2)	473 (10.3)	273 (6.0)	330 (7.2)	1088 (23.7)	1594 (34.8)	521 (11.4)	306 (6.7)
CMGs (N = 14)	7848	3124 (39.8)	4724 (60.2)	909 (11.6)	451 (5.7)	479 (6.1)	1954 (24.9)	2755 (35.1)	778 (9.9)	522 (6.7)
IMGs (N = 4)	1793	665 (37.1)	1128 (62.9)	173 (9.6)	120 (6.7)	172 (9.6)	446 (24.9)	582 (32.5)	193 (10.8)	107 (6.0)

CMG—Canadian medical graduate, IMG—international medical graduate, RPP—resident practice profile.

^*All percentages might not add to 100 owing to rounding.

Table 3 shows the percentage of FMRs who had seen specific priority topics at least once. Three priority topics (ie, epistaxis, meningitis, and neck pain) were not documented as being seen by any of the FMRs. An additional 8 topics were seen at least once by fewer than 9 FMRs. Forty-nine (73.1%) of the topics were seen at least once by at least 14 of the 18 FMRs.

Table 3.

Priority topics seen at least once by residents

RESIDENTS WHO ADDRESSED PRIORITY TOPIC IN A CLINICAL ENCOUNTER AT LEAST ONCE, %	NO. OF TOPICS (N = 67)	LIST OF TOPICS
100	25	Abdominal pain, anxiety,* cancer, chest pain, chronic obstructive pulmonary disease, contraception, cough, depression, diabetes, diarrhea, earache, headache, hypertension, immunizations, joint disorder, low back pain, periodic health assessment or screening, pregnancy, sexually transmitted infections, smoking cessation, substance abuse, upper respiratory tract infection, urinary tract infection,† vaginal bleeding, well-baby care
90–99	11	Anemia, asthma, atrial fibrillation, behavioural problems, dizziness, dyspepsia, hyperlipidemia, ischemic heart disease, neurodegenerative diseases,‡ thyroid, vaginitis
75–89	13	Allergy, fatigue, fever, gastrointestinal bleed, hepatitis, insomnia, loss of weight, obesity, osteoporosis, pneumonia, prostate, red eye, schizophrenia
50–74	7	Fracture, loss of consciousness, menopause, personality disorder, seizure, sex, stroke
1–49	8	Breast lump, croup, deep venous thrombosis, eating disorder, infertility, laceration, mental competence, suicide
0	3	Epistaxis, meningitis, neck pain

RPP—resident practice profile.

^*Anxiety in the RPP included the stress and grief priority topics.

^†Urinary tract infection in the RPP included both urinary tract infection and dysuria priority topics.

^‡Neurodegenerative diseases in the RPP included parkinsonism and dementia priority topics.

DISCUSSION

Our study is the first in family medicine to assess the number of residents’ patient visits in an ambulatory setting and to demonstrate the relationship between the number of patient visits and residents’ ITE scores.

Using EMR data, we identified 11 115 patient visits to FMRs from July 1, 2013, to June 30, 2014. First-year residents saw on average 5.48 patients per clinic and second-year residents saw 5.98 patients per clinic. This translates to second-year residents seeing 50 more patients over 100 clinics and is consistent with a study of internal medicine residents in which first-year trainees had an average of 4.8 patient visits per half-day clinic and second- and third-year residents had 5.6 patient visits per half-day clinic.⁹ When the patient visit data were analyzed by 3-month periods in training, there was a trend toward increased number of visits in the first year of residency, which stabilized over the second year of residency. This supports the need to monitor patient volumes in residency and helps program directors to modify visit benchmarks to provide greater clinical exposure.

The number of patients seen during training moderately correlates with performance on the final ITE, suggesting that clinical exposure correlates with performance on more objective measures of competence. An association between the number of patient admissions and ITE scores has also been reported in the literature in both internal medicine¹⁰ and pediatric¹¹ settings. When the mean ITE scores for FMRs with the highest number of patient visits were compared with those with the lowest number of patient visits, the difference was 10.5%. This is a larger difference than demonstrated previously when the performance of Canadian medical graduates and international medical graduates (5.0%) and first- and second-year trainees (1.6%) were compared.¹² However, it is possible that more patient visits might not result in higher ITE scores but rather that both might be markers of highly functioning FMRs.

We also described the types of patients seen with respect to sex, age, and diagnoses. There were 9641 patient visits recorded in the RPP from July 1, 2013, to June 30, 2014. These data were derived from resident logging of patient visits. When compared with EMR data, residents logged 86.7% of all patients they actually saw; 60.7% of these logged visits were made by female patients and 39.3% were made by male patients. This is consistent with previously reported data on proportions of female-to-male ambulatory care visits.^13,14 Female residents tended to have a higher percentage of visits made by women and male residents tended to have a higher percentage of visits made by men, which is consistent with previously published data.¹³ Given this finding, it is important to pay close attention to the sex distribution for both male and female residents to ensure that their practices are not skewed toward their sex. The age distribution of the patient visits followed a bell-shaped pattern with almost 60% of visits made by individuals aged 21 to 64. Based on these data, it is important to be vigilant with respect to visits made by pediatric and geriatric age groups in order to provide a balanced experience for residents. This also supports the need for experiences with these age groups outside of the ambulatory care setting.

Previous literature has reported on the most common diagnostic and procedural experiences in family medicine residency.^13,15,16 No studies have reported on gaps in the clinical ambulatory experience. We found that 16.4% (11 of 67) of conditions essential for Certification were seen by less than 50% of FMRs in an academic year. Programs in emergency medicine have come to similar conclusions.^17,18 Although conditions like neck pain might have been seen but not documented, the absent documentation for meningitis, croup, and deep vein thrombosis likely indicates lack of exposure. By identifying diagnoses that are seen less than 50% of the time by an entire cohort of FMRs, correction strategies can be developed. This might include adjustment of the clinical cases seen or supplementation through academic sessions or reading programs. Our program provides 1-page summaries of clinical topics through the RPP tool for many of the diagnoses seen by FMRs.

Limitations

This is an exploratory study describing the patient demographic information and diagnoses seen by FMRs in an urban ambulatory care setting that demonstrates a correlation between the number of patient visits and performance on ITEs. As such, there are a number of limitations. First, the study describes the practice profile for an urban centre that recruited patients for resident practices. This might have resulted in more balanced practices than might be seen in other centres.

Second, the RPP data represented 86.7% of patient visits during the academic year, which might have underreported the types of patients and conditions seen.

Third, there might be differences in the absolute number of patient visits required to obtain competence based on the characteristics of the training site and the population served, including complexity. Future studies using larger resident cohorts are required to confirm and build on our findings.

Conclusion

Successful family medicine residency training requires broad exposure to problems encountered in practice. Monitoring the types and number of patient encounters is crucial to establishing whether a program is achieving training goals. This pilot study found that FMRs received good exposure to a variety of ambulatory clinical problems; however, exposure to the entire spectrum of CFPC’s priority topics is lacking. A correlation between the number of patients seen and residents’ ITE scores exists. These findings need to be confirmed using larger cohorts of residents.

Editor’s key points

• ▸ This study assessed the number of patient visits among residents in an ambulatory care setting. First-and second-year residents saw an average of 5.48 and 5.98 patients per half-day clinic, respectively. Analysis of 3-month training periods revealed a trend toward an increased number of visits in the first year of residency, which stabilized over the second year. To provide greater clinical exposure in residency, patient volumes must be monitored and visit benchmarks should be modified.

• ▸ The number of patients seen during residency moderately correlated with final in-training examination scores, suggesting that clinical exposure correlates with performance on more objective measures of competence.

• ▸ Female residents had more visits from female patients and male residents had more visits from male patients. Also, almost 60% of visits were by patients aged 21 to 64. In order to provide a balanced experience for residents, it is important to consider sex distribution for residents so that their practices are not skewed toward their sex and to be vigilant with respect to visits made by pediatric and geriatric age groups.

Points de repère du rédacteur

• ▸ Dans cette étude, on a évalué le nombre de patients vus par les résidents dans une clinique de soins ambulatoires. En moyenne, les résidents 1 et 2 ont vu 5,48 et 5,98 patients par demi-journée de clinique respectivement. Une analyse portant sur des périodes de 3 mois a montré que le nombre de visites avait tendance à augmenter durant la première année de résidence, pour ensuite se stabiliser durant la deuxième. Pour s’assurer que les résidents ont une meilleure exposition à la clinique, on devrait mesurer le nombre de patients et modifier leur répartition.

• ▸ On a observé une corrélation modérée entre le nombre de patients vus durant la résidence et les résultats à l’examen de fin du stage, ce qui laisse entendre que le degré d’exposition à la clinique est lié au rendement d’après des mesures plus objectives de la compétence.

• ▸ Parmi les résidents, les femmes avaient vu plus de femmes et les hommes, plus d’hommes. De plus, près de 60 % des patients avaient entre 21 et 64 ans. Pour que les résidents aient une expérience bien équilibrée, on devra tenir compte du sexe des patients lors de leur attribution aux résidents pour éviter qu’ils ne développent un biais envers leur propre sexe dans leur pratique; il faut aussi être vigilant pour ce qui est des groupes d’âge, comme pour les enfants et les personnes âgées.