Incorporating CanMEDS and subspecialty training into paediatric residency programs: Why are we still deficient?

Abstract

BACKGROUND:

The Royal College of Physician and Surgeons of Canada mandates that paediatric training programs in Canada incorporate subspecialty training and the teaching and evaluation of the seven CanMEDS roles into their curriculum. The literature suggests that newly practicing paediatricians feel inadequately prepared in many subspecialties and CanMEDS roles.

HYPOTHESIS:

That either current training programs underestimate the importance of these areas for future practice, or that residents themselves feel that these areas are less important.

METHOD:

An online survey of Canadian paediatric residents and paediatric residency program directors was conducted to determine their views on various subspecialty areas and CanMEDS roles.

RESULTS:

Fourteen of 16 Canadian paediatric programs participated, and 127 of 486 (26%) paediatric residents completed the survey. Overall, trainees were satisfied with their current training (86%), and 90% believed they would be adequately prepared for independent practice. Forty-six residents (40%) believed training programs place less importance on 10 of the subspecialties that newly practicing paediatricians felt less comfortable with (from a previous study conducted in 2006). However, at least 25% of residents themselves placed less importance on nine of these 10 areas. Residents also place less importance on two CanMEDS competencies which practicing paediatricians felt less comfortable with, including the medical aspects of palliative care (medical expert) and managing an efficient office practice (manager).

CONCLUSIONS:

Residents and programs place less importance on specific areas of paediatric training, thus creating potential deficiencies in graduating paediatricians. Promotion of these topics during training may better prepare residents for future practice.

Recently, the core components that comprise the field of paediatrics have continued to expand, forcing training programs to readjust their curricula to offer the most up to date education. Each paediatric residency program in Canada has developed their own curriculum, individualized to the strengths and limitations of each institution, while simultaneously needing to meet the fundamental objectives set out by the Royal College of Physicians and Surgeons of Canada (RCPSC) (1). Paediatric residency programs also need to expose trainees to the various subspecialities in paediatric medicine; the exact frequency and duration of rotations that one should complete in these subspecialties is unclear.

Given this, it is unclear whether residents graduating from paediatric training programs in Canada have obtained the core skills to begin independent practice. A recent study by Lieberman and Hilliard (2) surveyed newly graduated paediatricians to determine how prepared they felt in various subspecialties and in the seven CanMEDS competencies. These paediatricians identified themselves as feeling less than adequately trained in 10 subspecialty areas, including gynecology (73% felt training was inadequate), child psychiatry (64%), behavioural paediatrics (58%), surgical subspecialties (54%), adolescent medicine (46%), dermatology (44%), nutrition (43%), allergy and immunology (39%), metabolics (36%) and genetics (31%). There were no differences in overall perceptions of preparedness between community-based and university-based paediatricians. However, compared with paediatric subspecialists, general paediatricians were less satisfied with respect to their training in child development behavioural paediatrics and child psychiatry. More university-based paediatricians than community-based paediatricians felt they had too much training in neonatology.

The RCPSC has adopted the CanMEDS framework to ensure that essential physician competencies (medical expert, communicator, collaborator, scholar, health advocate, manager and professional) (3) are addressed in residency training. This framework has been integrated into the RCPSC accreditation standards, objectives of training, and final in-training evaluations. Lieberman and Hilliard’s study (2) examined the adequacy of training in various CanMEDS roles, selected in consultation with the Canadian Paediatric Society and the RCPSC Paediatric Specialty Committee. The roles examined included: medical expert (anticipatory guidance/well-child care, dealing with maltreatment/abuse, chronic care, palliative care, dealing with death and bereaved parents, and procedural skills); communicator (working with patients/ families, working with cultural/socio-economic differences); collaborator (working as a team); manager (learning principles of quality management, managing an efficient office practice); health advocate (for individual patients, for disadvantaged patients); scholar (ability to conduct a research project, ability to critically appraise literature); and professional (including ethical issues). Paediatricians in Lieberman and Hilliard’s study expressed concerns of inadequate training in three of the CanMEDS competencies, including the medical aspects of palliative care (medical expert), dealing with death and the bereaved family (medical expert), and managing an efficient office practice (manager).

These findings are corroborated by other studies from the United States, Australia and Turkey (4–11). Preliminary data from the Residency Review and Redesign (R3P) Project in the United States has also outlined many of these topics as needing greater focus according to both recently graduated paediatricians and current fellows (12).

Given that specific areas of deficiency have been identified, reasons behind these deficiencies on graduation need to be explored. We hypothesized that either current training programs underestimate the importance of, or the amount of emphasis placed on, these areas for future practicing paediatricians, or that residents themselves feel that these areas are less important to them during their training. To investigate this, we conducted a Canada-wide survey of paediatrics residents and paediatric residency program directors (PDs) to determine their views on these subspecialty areas as well as the various CanMEDS roles.

METHODS

A nine-page online questionnaire was designed using an online survey tool (www.surveymonkey.com) and was sent to all current Canadian paediatric residents across 16 institutions, through their respective PDs. The survey was a modified version of the previously validated questionnaire designed by Lieberman and Hilliard. Modifications included three additional pages that focused on personal demographics, and resident opinions on their current paediatric training program. Additionally, Lieberman and Hilliard’s survey grouped all surgical subspecialities together, whereas the present survey specifically asked questions about ophthalmology, otolaryngology and orthopedics to obtain viewpoints on these specific areas.

The present survey consisted of questions regarding the adequacy of current paediatric residency programs. Residents’ impressions on the importance of various subspecialties were contrasted with their program’s perceived emphasis on these same topics during their training. Similar comparisons were made for the seven CanMEDS roles and the importance they play in resident education. The present survey consisted primarily of Likert scale questions rated from 1 to 4 as follows: 1 (not important); 2 (somewhat important); 3 (important); 4 (very important). A translated francophone version of the survey was available to accommodate French-speaking residents. Identical questionnaires were also sent to all current Canadian PDs. Initial distribution was in September of 2008 and data were collected for a total of 10 weeks, and a reminder e-mail was sent to all eligible participants at the six-week mark. Ethics approval was obtained from the Children’s Hospital of Eastern Ontario Research Ethics Board.

The survey data were analyzed usign SPSS 16.0 (IBM Corporation, USA) for Windows (Microsoft Corporation, USA). Summary statistics were generated for the responses. For analysis, responses were dichotomized, in which responses 1 and 2 were classified in the overall category ‘Less Important’ and responses 3 and 4 were classified in the overall category ‘Important’. χ² tests were used to compare opinions of residents and PDs. P <0.05 was considered to be statistically significant. Cohen’s Kappa statistics were calculated with 95% CIs to assess the agreement between the residents’ ratings of the importance of the subspecialties and the residents’ perception of the importance that their programs place on these subspecialties beyond that expected by chance. A Kappa of 0.81 to 0.99 was interpreted as indicating almost perfect agreement, 0.61 to 0.80 was considered to be substantial agreement, 0.41 to 0.60 was considered moderate agreement, 0.21 to 0.40 was considered fair agreement, 0.01 to 0.20 was considered slight agreement and below 0 indicated poor or less than chance agreement (13).

RESULTS

General demographics

Fourteen of 16 accredited Canadian paediatric programs chose to participate. One-hundred twenty-seven of 486 active paediatric residents completed the survey (response rate 26%). Men accounted for 23% of respondents, while 77% were women, which at least in part, reflects the relative sex bias in paediatrics programs. The majority of respondents (70%) were between 20 and 30 years of age. There was an even distribution of post-graduate years (PGY) represented in the survey, with 31% of respondents in PGY-1, 31% in PGY-2, 22% in PGY-3 and the remainder in PGY4/5. Of the institutions participating in the present survey, 10 of 14 PDs completed the survey.

Of the residents surveyed, 54% planned to pursue a career in subspecialty paediatrics, and 46% planned to practice in general paediatrics. Only 10% of respondents felt they would not be adequately prepared for independent practice on graduation, and 86% either agreed or strongly agreed that they were satisfied with their current training.

Importance of subspecialty training

Residents were asked to rate how important they felt various subspecialty rotations were to their learning. The subspecialties rated as most important are shown in Table 1 (columns 1 and 2). Similarly, the subspecialties that residents rated as least important to their learning are shown in Table 2 (columns 1 and 2). At least 25% of residents rated nine of the 10 deficient areas as less important to their training (exact percentages shown in table). The majority of subspecialties identified as deficient in graduated paediatricians, as per Lieberman and Hilliard’s study, were considered to be less important by current residents.

TABLE 1.

Subspecialty topics rated as more important to resident education

Subspecialty topic	Residents rating topic as more important (n=27)	Programs viewed as placing more importance on topic (n=127)	PD rating topic as more important (n=10)
Areas in which recent graduates felt deficient
Cardiology	99	94	100
Endocrinology	98	80	100
Infectious diseases	97	83	100
Respirology	97	81	100
Neurology	96	85	100
Emergency	95	92	100
Neonatology	95	99	100
Gastroenterology	94	86	100
Nephrology	92	76	90
Development	92	85	100
Hematology & oncology	91	88	100
Areas in which recent graduates felt competent
Nutrition	78	21	70
Dermatology	75	18	50

Data presented as %. Column 1 lists the subspecialty topics felt to be most important by residents themselves; column 2 indicates the actual percentage of residents rating these topics as more important (only those topics rated as more important by at least 75% of residents have been displayed). Column 3 shows the percentage of programs perceived by residents to place more importance in these subspecialty areas. Column 4 lists the percentage of program directors that rated these topics as more important to resident learning. The table is subdivided into two sections: the areas where recently graduated Canadian paediatricians felt deficient, and the areas in which they felt competent (as shown by Leiberman and Hilliard)(3). PD Paediatric residency program directors

TABLE 2.

Subspecialty topics rated as less important to resident education

Subspecialty topic	Residents rating topic as less important (n = 127)	Programs viewed as placing less importance on topic (n = 127)	PD rating topic as less important (n=10)
Areas in which recent graduates felt deficient
Ophthalmology	74	93	60
Gynecology	69	92	60
ENT	59	89	40
Genetics	55	63	20
Orthopedics	49	83	60
Metabolic disease	48	64	50
Allergy & immunology	44	75	40
Child psychiatry	38	60	20
Adolescent medicine	33	42	0
Behavioral paediatrics	28	64	10
Dermatology	25	82	50
Areas in which recent graduates felt competent
General surgery	51	26	60
Rheumatology	38	67	30
Radiology	38	77	40

Data presented as %. Column 1 lists the subspecialty topics believed to be least important by residents themselves; column 2 indicates the actual percentage of residents rating these topics as less important (only those topics rated as more important by at least 75% of residents have been displayed). Column 3 shows the percentage of programs perceived by residents to place less importance in these subspecialty areas. Column 4 lists the percentage of program directors that rated these topics as less important to resident learning. The table is subdivided into two sections: the areas where recently graduated Canadian paediatricians felt deficient, and the areas in which they felt competent (as shown by Leiberman and Hilliard [3]).ENT Ear, nose, and throat; PD Paediatric residency program directors

Column 3 of Table 2 shows the percentage of residents that perceived their program to place less emphasis on each of the listed topics. At least 40% of programs were viewed as placing less importance on all 10 of Lieberman and Hilliard’s deficient areas (exact percentages shown in Table 2), indicating that residents also believe their programs place less emphasis in the exact same areas that are deficient in practicing physicians.

Column 4 (Table 2), which lists the percentage of PDs who felt that each of the corresponding subspecialty areas was less important to resident training, indicates that more than 50% of PDs felt that less importance should be placed in the areas of dermatology, metabolic disease, gynecology, orthopedics, general surgery and ophthalmology. When compared with PD’s, resident opinions differed significantly in two categories: a higher proportion of residents than PDs rated genetics and adolescent medicine as not important (Fisher’s exact test P<0.05).

The degree of emphasis residents placed on these topics versus the perceived importance their programs placed on these areas, was also compared. Kappa values were calculated to determine which topics showed statistically significant discrepancies between the two groups. Table 3 lists all of the subspecialties in which there were significant discrepancies (kappa ≤0.2) between how important residents felt a subject was and their perception of how much emphasis they felt their program was placing on that subject area. For example, only 25% of residents felt that dermatology was less important to their learning. However, 82% of residents perceived their programs as placing less emphasis on this topic. Residents therefore felt that their program should place more importance in this area, and this is reflected in the very low kappa (0.02). There were significant discrepancies between how much emphasis residents felt should be placed on a topic and how much emphasis they felt their programs placed on eight of the 10 areas identified as deficient in practicing paediatricians. Thus, residents and programs poorly agree on how much importance should be placed on these future areas of deficiency. Residents placed a higher percentage of importance on every topic compared with their programs except for general surgery and neonatology. These were the only two topics in which a higher number of programs were seen as placing more importance compared with residents themselves.

TABLE 3.

Comparison of specialty importance between residents and training programs. Topics showing poor agreement in perceived level of importance between residents and programs are shown (defined by a kappa (ĸ) value of less than 0.2)

Subspecialty Topic	Residents rating topic as less important (n =127)	Programs viewed as placing less importance on topic (n =127)	KAPPA (ĸ)
Areas in which recent graduates felt deficient
Dermatology	25	82	0.02
Nutrition	22	79	0.10
Allergy and immunology	44	75	0.11
Adolescent medicine	33	42	0.12
Orthopedics	49	83	0.13
Child psychiatry	38	60	0.17
Behavioural paediatrics	28	64	0.17
Development	8	15	0.17
Areas where recent graduates felt competent
Radiology	38	77	0.16
General surgery	51	26	0.20

Data presented as %, unless otherwise indicated. The table is subdivided into two sections: the areas where recently graduated Canadian paediatricians felt deficient, and the areas in which they felt competent (as shown by Leiberman and Hilliard [3]).

Importance of CanMEDS roles

The CanMeds roles were felt to play an important part in paediatric training for 78% of trainees surveyed. The roles rated as less important by residents are shown in column 2 of Table 4. Thirty-eight per cent of respondents felt that the ability to conduct a research project (scholar role) was less important to their education. The medical aspects of palliative care (medical expert role) and managing an efficient office practice (manager role) were two of the three roles identified as deficient by Lieberman and Hilliard, and were felt to be less important by 30% and 25% of all residents, respectively.

TABLE 4.

CanMEDS roles rated as less important in resident education

Subspecialty Topic	Residents rating topic as less important n =127	Programs viewed as placing less importance on topic n =127	PD rating topic as less important n=10
Areas in which recent graduates felt deficient
The medical aspects of palliative care (medical expert)	30	60	20
Managing an efficient office practice (manager)	25	74	20
Dealing with death and bereaved parents (medical expert)	14	46	0
Areas in which recent graduates felt competent
Ability to conduct a research project (scholar)	38	11	20
Learning principles of quality management (manager)	28	54	20
Conflict resolution (manager)	24	52	20
Effective use of resources (manager)	21	53	10
Working with socioeconomic differences (communicator)	19	42	10
Working with cultural differences (communicator)	17	46	10
Anticipatory well child care (medical expert)	15	43	0
Complex chronic care (medical expert)	13	18	0
Advocate for disadvantaged infants/children (health advocate)	13	40	10
Teaching skills (medical expert)	11	26	0

Data presented as %. Column 1 lists the CanMEDS roles believed to be least important by residents themselves; column 2 indicates the actual percentage of residents rating these topics as less important. Column 3 shows the percentage of programs perceived by residents to place less importance in these roles. Column 4 lists the percentage of program directors that rated these topics as less important to resident learning. The table is subdivided into two sections: the areas where recently graduated Canadian paediatricians felt deficient

Column 3 of Table 4 displays the CanMEDS roles that were perceived by trainees as having less emphasis by training programs. Seventy four per cent of programs were viewed as placing less emphasis on managing an efficient office practice (manager role, and 60% of programs were viewed as placing less emphasis on the medical aspects of palliative care (medical expert); both are roles that are deficient in newly practicing paediatricians. In total, more than 46% of all residents believe that training programs place less emphasis on all three of CanMEDS deficiencies identified by Lieberman and Hilliard.

PDs views on the importance of the various CanMEDS roles are also shown in Table 4. Column 4 lists the percentage of PDs who rated the corresponding role as less important to resident training. More than 80% of PDs felt that all of the CanMEDS roles were either important or very important to resident learning. When asked to evaluate training programs however, more than 50% of PDs felt that less emphasis was placed on the medical aspects of palliative care (medical expert role), learning principles of quality management (manager), managing an efficient office practice (manager) and effective use of resources (manager).

DISCUSSION

Lieberman and Hilliard showed that Canadian paediatricians felt inadequately trained in several areas, including gynecology, child psychiatry, behavioural paediatrics, surgical subspecialties, adolescent medicine, dermatology, nutrition, allergy and immunology, metabolics and genetics. Determining the reasons why these areas of deficiency arise was the primary goal of our study. Our findings point to both resident perceptions, as well as insufficient emphasis by training programs as potential explanations for these deficiencies.

Reassuringly, trainees were satisfied with their current training and felt they would be adequately prepared for independent practice. This is similar to the findings of Lieberman and Hilliard (2), who found that 96% of respondents felt they were adequately or well trained. However, we demonstrated that during training, residents place lesser importance on nine of the 10 areas identified in Lieberman and Hilliard’s study as deficient in future practicing paediatricians. It is possible that during training, residents may not appreciate the practical significance of these topics until after graduation.

It is also possible that training programs themselves place less emphasis on these topics. More than 40% of residents viewed their programs as placing less importance on all 10 areas deemed deficient by practicing paediatricians. There are several reasons why training programs may place less emphasis on these topics. There may be a lack of awareness that these are topics are important for future clinical practice. It may also be due to the absence of established subspecialty clinical programs at some institutions or a lack of available teachers. Programs may be able to improve knowledge in these areas by encouraging electives at other institutions, encouraging independent reading on these topics or presenting these topics as academic half days.

We found significant discrepancies in certain areas that were deemed important to residents, but were perceived as being less important by their training programs. These discrepancies may be the initial seed that has skewed resident interest and motivation in these topics in the first place. Programs may be giving the perception that these areas are not worth focusing on, and perhaps if training programs were to actively promote these topics, residents may be encouraged to place more emphasis on them during their training.

How then, should programs incorporate all of these areas into already full curricula? One option may be to extend the length of training; however, only 20% of the respondent’s in Lieberman and Hilliard’s study (3) felt that paediatrics training should be extended to five years. Programs were seen as placing more importance on general surgery and neonatology compared with residents themselves. This may mean that residents feel programs place more emphasis than is needed on these topics and may indicate areas that could be adjusted to accommodate other ‘deficient’ areas, although qualitative focus group research would help to better clarify this.

Although most residents feel the CanMEDS roles are important, one-quarter of residents did not place a great deal of importance in them. Specifically, residents placed less importance in the medical aspects of palliative care (medical expert role) and managing an efficient office practice (manager role), which were two of the three roles that were deficient in practicing physicians. Again, a lack of importance felt by residents in these areas during training is likely contributing to deficiencies on graduation. Similarly, residents felt their programs placed less emphasis on three of the CanMEDS roles that were identified as deficient in practicing paediatricians. Incorporating these roles more effectively into the paediatric curriculum may help to promote their importance, making residents more inclined to realize their impact on future practice.

Overall, PDs views on the importance of different areas of subspecialty training mirrored residents’ views At least 50% of PDs rated seven topics identified as being deficient in newly practicing paediatricians by Lieberman and Hillard as less important to resident learning. It would be interesting to know the representation of these topics in paediatric residency curriculums in Canada, and whether this is representative of PDs views on their importance. As PDs have a direct role-model influence on residents, the effects that their opinions have on resident education is substantial.

The main limitation of our study was the low response rate, and this needs to be taken into account when interpreting the data. Internet surveys are relatively new and have many potential advantages such as easier data analysis, faster response time and less paper waste. Unfortunately, they tend to produce significantly lower response rates than traditional paper versions. Several recent studies comparing internet and paper surveys found internet response rates vary anywhere between 11% and 58% (14–19). Thus, our response rates still fall well within the expected range for this type of survey. Our study could also have been strengthened by subdividing responses accordign to PGY, thereby allowing us to determine whether more senior residents had a different viewpoint on what should be emphasized in their training; this was not possible due to the low response rate. Additionally, it would have been ideal to compare resident perceptions of importance with external measures of resident knowledge, such as performance on the American Board of Pediatrics in-training examination or the RCPSC examination in pediatrics, to determine whether in fact, knowledge deficiencies were found in certain areas. However, this was beyond the scope of our study.

Thus, this perceived lack of importance felt by both residents and programs in various subspecialites and CanMEDS roles may contribute to deficiencies in newly graduated paediatricians. Qualitative research may be needed to understand the reasons why; knowing these reasons would better assist us in determining how we may alter the attitudes and training toward these future areas of deficiency. Promotion of these topics by training programs may also help to emphasize these areas, and better prepare residents for future practice.