Procedural skills in paediatric residency: Re-evaluating the competencies

Abstract

BACKGROUND:

The Royal College of Physicians and Surgeons of Canada (RCPSC) sets objectives for residency training, including many skills that may not be commonly performed in practice.

OBJECTIVE:

To describe attitudes of residents enrolled in Canadian paediatric residency programs toward procedural skills training, including perception of importance and perceived proficiency of the RCPSC-required procedures.

METHODS:

An anonymous electronic survey was distributed to all senior residents enrolled in Canadian paediatric residencies, using a five-point Likert scale to address procedural importance and corresponding proficiency. Data were analyzed using descriptive statistics, with Pearson correlation coefficients used to describe relationships among variables.

RESULTS:

A total of 68 residents responded. Fifteen skills had a mean importance ≥4 (very or extremely important) and five skills had a mean rating <3 (somewhat or not important). Residents believed they were extremely or very proficient (mean rating ≥4) for three skills (bag-mask ventilation, lumbar puncture and chest x-ray interpretation). They reported 23 procedures for which they felt somewhat to not proficient (mean <3). The correlation between importance and proficiency was high (Pearson’s correlation coefficient = 0.87). However, proficiency was significantly lower than importance (P<0.05) for the majority of procedures (88%). The largest gaps between importance and proficiency were observed for chest tube insertion, gathering evidence of child maltreatment, defibrillation and intraosseous insertion.

CONCLUSION:

Many, but not all, RCPSC-required procedures are believed to be important. Residents do not believe that they are adequately proficient in many of these procedures. Skills with the greatest gap between importance and proficiency may be targets for curricular interventions.

Procedural skills have been identified across countries and medical specialties as an area of weakness in training programs (1,2). Paediatric residents have specifically been shown to have difficulty with acute-care procedural skills, and often perform these tasks incorrectly (3–7). Several of these skills, however, have been identified as essential for practice in a general paediatric setting (8). The Royal College of Physicians and Surgeons of Canada (RCPSC), specifically the RCPSC Pediatrics Specialty Committee, is the standard-setting body for accreditation of Canadian paediatric residency training programs, and determines the Objectives of Training (OTR) for paediatrics, which lists the procedural skills in which residents are required to gain competence by training completion (9). The list of skills outlined in the paediatric OTR, therefore, includes several acute-care procedural skills, as well as several other procedures that may be performed in other clinical settings. Resident competence in skills outside of an acute-care setting is not well characterized in the literature.

Studies investigating procedural-skills training in American residency programs have demonstrated that residents did not develop competence in several important procedural skills. Furthermore, the skills required by the American Residency Review Committee (the analogous standard setting body in the United States) were not necessarily congruent with the skills considered to be most important by program directors (1). Similar concerns have been raised within the Canadian system (10–13).

It is unclear how Canadian paediatric residency programs train their residents in the acquisition of procedural skills, and resident proficiency in these skills at graduation remains uncertain. Furthermore, the importance of skills that are required by the RCPSC has not been examined recently, particularly as perceived by trainees. Therefore, we sought to describe resident attitudes toward the importance of each of the procedural skills required by the RCPSC, as well as to quantify their perceived proficiency at performing these procedures.

METHODS

Study subjects

The present study was approved by the Research Ethics Board of The Hospital for Sick Children (Toronto, Ontario). An anonymous electronic survey was distributed to all third- and fourth-year residents (as well as fifth-year residents in Quebec) enrolled in all established Canadian paediatric residency training programs via an online survey distribution tool. Consent for participation was acquired; respondents were advised that consent was implied by completion of the survey. Surveys were distributed through administrators of each program to maintain anonymity of subjects. A total of 231 residents received surveys. Three reminder e-mails were sent over a three-month collection period.

Survey tool

Demographic, curriculum and program-related data were collected. For each skill required by the RCPSC Pediatric OTR, residents were asked to rate both importance and perceived proficiency on a five-point Likert scale. Proficiency was defined as the ability to perform a skill independently, without supervision, with a high likelihood of successful completion and with understanding of risks, indications and contraindications. Presence of a specific curriculum and the requirement of documentation of completion for each skill were also recorded.

All invasive, noninvasive and test-interpretation skills were extracted from the OTR, resulting in a total of 35 skills. For the purpose of clarity, certain skills were subdivided into components: eg, “cardiopulmonary resuscitation (CPR) (neonatal and paediatric)” was divided into three discrete skills (defibrillation, neonatal CPR and paediatric CPR). “Bag-mask ventilation (BMV) and tracheal intubation (neonatal and paediatric)” was divided into BMV, neonatal tracheal intubation and paediatric tracheal intubation. Finally, “bladder catheterization and/or suprapubic aspiration” was reclassified as two separate skills. Therefore, a total of 40 skills were assessed.

Data analysis

Data are presented as frequencies, medians with minimums and maximums, or means and SDs, as appropriate. Means of importance and perceived proficiency scores were calculated, and classified as ≥4 (rated very or extremely important; residents felt very or extremely well-prepared to perform the skill) or <3 (rated somewhat to not important; residents felt somewhat or not at all prepared to perform). Gap scores were calculated for each skill by subtracting the mean proficiency from the mean importance, as described by Amin et al (10). Pearson correlation coefficients were used to describe relationships among variables. Skills were categorized as vascular, acute, resuscitation and ambulatory for further analysis. All analyses were performed using SPSS version 17.0 (IBM Corporation, USA).

RESULTS

Demographic and program characteristics

Residents from 17 Canadian paediatric programs were surveyed. A total of 68 surveys (29%) were collected from residents in a variety of program sizes, ranging from <10 to >80 residents total. Residents reported a mean (± SD) of 86±10% of their time being devoted to a tertiary care or university hospital. Eighty-eight percent of residents reported the presence of a phlebotomy team at their primary institution. Forty-seven percent of residents reported that they ‘never’ draw blood samples, versus 5% who reported ‘frequently’ or ‘very frequently’ drawing blood samples. Similarly, 38% reported ‘never’ starting intravenous lines, versus 7% who ‘frequently’ or ‘very frequently’ start intravenous lines.

Skill importance and perceived proficiency

Figure 1 presents the mean importance and perceived proficiency scores for each of the evaluated skills, listed in order of descending importance score. Table 1 lists the percentage of residents reporting a formal curriculum, documentation of completion of a skill, as well as the calculated gap score (the difference between mean importance and proficiency) for each required skill. A total of 15 skills were given a mean importance score of ≥4, corresponding to a rating of ‘very important’ or ‘extremely important’, most of which pertained to resuscitation. Skills with the highest mean importance scores included BMV (mean 5.0), neonatal CPR (mean 4.9) and paediatric CPR (mean 4.9). Five skills were rated ‘somewhat important’ to ‘not important’ (mean <3); these included indwelling catheter management (mean 2.8), breast examination (mean 2.7), tympanogram interpretation (mean 2.6), suprapubic aspiration (mean 2.5) and gastric lavage (mean 2.1).

Figure 1).

Mean importance and perceived proficiency scores for each of the Royal College of Physicians and Surgeons-required procedural skills. BMV Bag-mask ventilation; CPR Cardiopulmonary resuscitation; CVL Central venous line; IO Intraosseous; ECG Electrocardiogram; FB Foreign body; GU Genitourinary; ID Infectious disease; IV Intravenous; NG Nasogastric; OG Orogastric; PFT Pulmonary function test; TB Tuberculosis; UA Umbilical artery; UV Umbilical vein

Table 1.

Presence of formal curriculum, resident documentation and calculated gap scores for each procedural skill required by the Royal College of Physicians and Surgeons of Canada, subdivided according to category

Skill	Presence of curriculum, %	Presence of documentation, %	Gap score*
Resuscitation skills
Cardiopulmonary resuscitation (neonatal)	100.0	86.7	1.16
Cardiopulmonary resuscitation (paediatric)	100.0	86.7	1.27
Bag-mask ventilation	95.7	81.8	0.77
Tracheal intubation (neonatal)	93.5	90.9	1.14
Tracheal intubation (paediatric)	93.5	86.4	1.29
Intraosseous insertion	91.3	70.5	1.56
Defibrillation	89.1	68.2	1.47
Vascular skills
Umbilical venous and umbilical arterial catheterization	69.6	75.6	0.96
Intravenous access and blood-drawing	47.8	59.1	1.03
Arterial line/central venous catheter placement	28.3	59.1	1.34
Arterial puncture	26.1	59.1	1.11
Acute care skills
Lumbar puncture	60.9	68.2	0.23
Chest tube placement and thoracentesis	50.0	55.8	1.51
Cervical spine immobilization	43.5	34.1	1.29
Suture of a one layer laceration, simple wound closure	43.5	66.7	0.66
Immobilization of acute limb injury including fractures	39.1	23.3	1.15
Gastric tube placement (oro- or nasogastric)	19.6	51.2	0.64
Foreign body removal from eye/nose/upper airway	15.2	18.2	1.24
Tracheostomy tube care, including replacement	15.2	16.3	1.09
Access and care for indwelling catheters	10.9	20.9	1.05
Suprapubic aspiration	8.7	46.5	0.90
Gastric lavage	2.2	20.9	0.62
Ambulatory skills
Reliably interpret an electrocardiogram in all age groups	82.6	40.9	0.73
Interpretation of chest x-rays	71.7	29.5	0.71
Interpretation of pulmonary function tests	69.6	31.8	0.59
Interpretation of abdominal x-rays	67.4	34.1	0.63
Interpretation of soft tissue x-rays in acute upper airway obstruction	60.9	20.5	1.26
Interpret common abnormalities seen on urine microscopy	58.7	29.5	0.71
Interpret bone x-rays for fractures	58.7	27.3	0.91
Perform and interpret oximetry	54.3	31.8	0.64
Gather child maltreatment evidence appropriately, including documentation and specimen collection	41.3	20.9	1.67
Bedside measurement of glucose	35.6	30.2	0.29
Immunizations (storage, administration and documentation)	30.4	23.3	0.64
Perform and interpret tuberculin skin testing	28.3	14.0	0.01
Procurement of appropriate specimens for diagnosis of infections	23.9	20.5	0.45
Gynecological, genitourinary and pelvic examination and specimen procurement	21.7	27.3	0.72
Interpretation of the tympanogram	19.6	16.3	0.78
Bladder catheterization	15.2	58.1	0.93
Perform curettage under direct visualization of the ear	13.0	16.3	0.58
Breast examination	8.7	14.0	0.47

^*Mean importance score minus mean proficiency score

Residents reported that they felt ‘somewhat prepared’ or ‘not at all prepared’ (mean <3) to perform 23 of the required skills. They felt ‘very well-prepared’ or ‘extremely well-prepared’ (mean ≥4) to complete only three skills; these included BMV (mean 4.2), lumbar puncture (mean 4.5) and chest x-ray interpretation (mean 4.0).

Perceived importance and perceived proficiency were strongly correlated (r=0.87, P<0.0001), with all skills receiving higher mean scores for importance than proficiency. Skills with the largest gap between mean importance and proficiency included gathering evidence for child maltreatment (gap score 1.67), intraosseous line insertion (gap score 1.56), chest-tube insertion (gap score 1.51) and defibrillation (gap score 1.47). Both importance and perceived proficiency were strongly correlated with the presence of a formal curriculum to teach the skill in question (r=0.92; P<0.0001, and r=0.76; P<0.0001, respectively). The requirement to document completion of a skill also correlated with both importance and proficiency, albeit less strongly (r=0.58; P<0.0001, and r=0.40; P=0.01, respectively).

Subcategory analysis

Seven skills were categorized as resuscitation, four as vascular, 11 as acute and 18 as ambulatory (Table 1). Figure 2 presents the mean importance and perceived proficiency scores for skills according to category. Residents’ perceived proficiency was rated significantly lower than importance for each of the skill categories (P<0.0001). Resuscitation skills, according to category, received the highest mean scores for both importance and perceived proficiency (mean 4.67 and 3.43, respectively), while vascular skills received the lowest mean proficiency scores (mean 2.31).

Figure 2).

Mean importance and perceived proficiency of skills categorized according to subgroup

DISCUSSION

We characterized the attitudes of third- and fourth-year paediatric residents enrolled in Canadian training programs toward procedural skill requirements and performance. Several RCPSC-required procedures were perceived to be less important. While the utility of resident perceptions in determining program requirements has obvious limitations, the exposure and clinical value of those skills rated the lowest in terms of importance may be considered. This provides a unique opportunity to further examine these skills in terms of actual practical use.

With regard to perceived proficiency, residents did not feel adequately prepared in all of the required procedural skills. While limited conclusions can be drawn from resident attitudes alone, our group previously surveyed Canadian paediatric program directors to explore their attitudes toward procedural skills training and proficiency, and similar results were obtained. Specifically, program directors rated a total of 15 required skills with mean proficiency scores <3 (‘somewhat prepared’ to ‘not at all prepared’) (14). Both residents and program directors believed that trainees were least prepared to perform gastric lavage. Suprapubic aspiration was rated second lowest by residents and third lowest by program directors (breast examination was given slightly lower proficiency scores by program directors). Notably, these skills were also given the lowest importance scores in our study. Skills pertaining to resuscitation, on the other hand, were identified as most important, and residents rated these skills relatively high in terms of perceived proficiency. Resuscitation skills had the highest rates of curricula, possibly accounting for the high rates of proficiency. Conversely, gastric lavage and suprapubic aspiration had extremely low rates of curricular presence, perhaps contributing to their poor proficiency scores.

We did, however, identify several important discrepancies between importance and proficiency. In general, the gap between these scores was large; 43% of skills had differences between importance and proficiency >1 point on the Likert scale. Furthermore, intraosseous line insertion, defibrillation and cervical spine immobilization were rated as very or extremely important, but residents felt somewhat or not at all prepared to complete these tasks. While the vast majority of residents reported a formal curriculum for both intraosseous insertion and defibrillation (91.3% and 89.1%, respectively), only 43.5% of residents reported a formal curriculum for cervical spine immobilization, indicating that residents’ views on importance and proficiency are not merely a reflection of how much they are taught a specific skill. This discrepancy between importance and perceived proficiency is consistent not only with the study of paediatric procedural skills training in the United States (1), but also with comparable studies conducted in other medical specialty training programs including family and internal medicine (8,15–18).

Relying on clinical exposure to develop expertise in procedural skills may be inadequate. Polled residents in our study reported a majority of time (mean 86%) spent in tertiary care hospitals, with 88% reporting the presence of a phlebotomy team and 47% reporting that they never perform blood draws. Tertiary-care institutions may create a training environment in which opportunities to develop procedural skills are unevenly distributed (11). Factors diluting procedural experience may include competition among trainees, both within medical training programs and other fields of study such as respiratory therapy, as well as the presence of experienced ancillary services such as phlebotomy teams, large resuscitation teams or easy access to multiple medical subspecialties. A recent study demonstrated very little procedural skill performance on paediatric wards, with more time devoted to indirect patient care and paperwork (19).

Development of a specific training curriculum is one strategy used to overcome these limitations and ensure equitable access to procedural skills. Gaies et al (1) demonstrated that the presence of any formal teaching method in procedural skills training was associated with improved perceived competence among residents, which is consistent with our findings. Procedural training curricula have been well described in other medical specialties (17,20) and more recently in paediatric training (21).

We did, however, identify skills that received poor proficiency scores despite the presence of formal curricula, including both intraosseous line insertion and defibrillation. Many studies have demonstrated that deliberate and repeated practice is necessary to develop expertise in psychomotor tasks (22–25) and that a greater volume of experience is correlated with greater proficiency in procedural skills (25–27). Limited clinical exposure and time constraints on formal training time collectively make skill acquisition more difficult. The lengthy list of procedural skills in which residents are required by the RCPSC to achieve competence compounds this difficulty. The main goal in the creation of the RCPSC OTR was to aid individual residency training programs in forming their curricula to optimally meet the needs of residents and paediatricians, as well as the health care needs of children and youth (12). While 40 skills are deemed necessary by the RCPSC, in comparison, the US Pediatric Residency Review Committee requires training in only 16 procedures; furthermore, residents in training programs in the United States still struggle to achieve proficiency in these skills (1). Refining the list of required procedural skills would allow for increased focus on the skills considered to be most important, and increase available time and resources to allow for the volume of repeated, deliberate practice necessary to achieve proficiency, particularly in the skills not encountered regularly in clinical practice. In addition, skills with large gaps between importance and proficiency could be targeted for curricular reform to reduce these discrepancies.

Our study was limited by the relatively low response rate. After three e-mail reminders, our survey response rate was 29.4%, with the potential for response-related bias. Surveys were not translated for French-speaking residents, which could have contributed to additional bias. Limited demographic data were collected to prioritize the anonymity of the participants. Study-derived limitations could be overcome in the future with more robust qualitative methods such as structured surveys. Little can be definitively concluded strictly from resident attitudes, particularly when there may be variability between perceived and actual proficiency. However, similar deficiencies obtained from a previous survey of program directors indicates that further, more detailed study of procedural skills acquisition in Canada is required (14). Future areas of study may include exploration of the procedures used most widely in community paediatric practice, as well as identification of the skills with strong evidence to support their utility, to help further explore refinement of the RCPSC-generated list of required skills.

CONCLUSION

Our study demonstrates that residents may not feel adequately prepared in all RCPSC-required procedures at or near the time of residency completion. While further study is required to more accurately determine resident performance of required procedural skills, skills with the largest gap scores could be targets for curricular reform. Furthermore, the list of required skills is lengthy, and many skills were considered to be less important from the viewpoint of residents. A thorough examination of the skill set required by the RCPSC is required so that the list can be refined to better reflect the skills that are most important for independent practice as a general paediatrician.