Exploring paediatric residents’ perceptions of competency in neonatal intensive care

Abstract

Assessment and stabilization of the newborn are expected competencies of trainees graduating from Canadian paediatrics residency training programs. There is limited evidence regarding the optimal approach to training, and whether such competencies are actually achieved by graduates. A national, cross-sectional survey was developed to explore paediatrics residents’ self-reported experiences in performing neonatal procedures and resuscitation skills. Survey questions were constructed based on the review of the Royal College of Physicians and Surgeons of Canada objectives of training in paediatrics to include activities necessary in the assessment and stabilization of the newborn. The survey was distributed to residents across Canada. A total of 138 residents from 15 Canadian paediatrics residency training programs completed the survey. A minority of residents (17%) reported independently performing resuscitative skills (positive pressure ventilation, intubation and umbilical line insertion). Of all the different neonatal procedural skills, only lumbar puncture was reported as an activity that residents on average performed independently by senior years of training. Our study showed a direct relationship between the number of completed blocks of Neonatal Intensive Care Unit (NICU) and self-reported experiences in providing NICU resuscitation skills and procedures. We found an inverse relationship between the exposure to cross-cover calls and such experiences. Our study showed that a minority of paediatrics residents self-report evidence of competency in performing neonatal procedures and resuscitation skills. As residency programs are transitioning toward competence-based education, it is important to gain more insights with respect to strengths, deficiencies and opportunities for paediatrics residency training in terms of NICU experiences.

BACKGROUND

Newborn care is a core competency expected of graduates from Canadian paediatrics residency training programs. The Royal College of Physicians and Surgeons of Canada (RCPSC) mandates that all paediatrics residents complete ‘three to five blocks, or longitudinal equivalent, of newborn care, which must include neonatal resuscitation in the case room, care of the newborn and rotations in level II and level III neonatal units’ (1). From this exposure, it is expected that trainees will gain sufficient experience, becoming competent in recognizing, resuscitating and stabilizing critically ill newborns.

Resuscitation training benefits from simulation-exposures during such teaching as the Neonatal Resuscitation Program, as well as experiential learning with actual patients during rotations in Neonatal Intensive Care Units (NICUs). The use of deliberate practice and related activities in simulation can improve resuscitation performance (2–4). Yet, it is unclear whether these targeted educational interventions actually improve long-term retention of resuscitation skills or knowledge (5–8). Anecdotally, some graduates have expressed that despite completion of paediatrics residency training, they do not feel comfortable with neonatal procedures and resuscitation, and therefore seek out additional supervised time in NICUs before independent practice. A 2010 study revealed that Canadian paediatrics trainees were not meeting the intubation standards set by the American Heart Association Neonatal Resuscitation Program (9).

There is limited evidence regarding the optimal, holistic approach to training residents in newborn care. In keeping with the RCPSC standards for training, Canadian residency programs either employ a block-based approach, a longitudinal approach or a combination of both. A block-based approach (such as rotation-based) is one in which paediatrics residents have discrete periods of time spent in the NICU. A longitudinal approach (such as cross-coverage) is one in which paediatrics residents have ongoing yet discontinuous exposures to neonatal care by doing shifts in the NICU outside of dedicated NICU rotations. This latter approach foreseeably offers exposure to a greater variety of patients of varying levels of acuity, promoting ongoing skill development and competency retention.

At this time, several changes are occurring within medical training in Canada: changes to resident duty hours, a shift to competency-based education and a possible shift to more in-house attending coverage within intensive care units. All of these factors are anticipated to influence how residents learn and achieve their training goals. As the training landscape changes, it is important to describe how our current model is training residents—thus allowing us to make more informed decisions about how to improve training curricula. This is particularly important as training programs are shifting towards a competence-by-design framework. The objective of this study is to explore Canadian paediatrics residents’ self-reported experiences in providing NICU resuscitation skills and procedures.

METHODS

A cross-sectional survey design was employed to gather paediatrics residents’ reported experiences of neonatal procedures and resuscitation skills. All trainees enrolled in Canadian paediatrics residency programs during the 2016 to 2017 academic year were eligible to participate.

Study data were collected and managed using Research Electronic Data Capture (REDCap), an electronic data capture tools hosted at University of Alberta (10). REDCap is a secure, web-based application designed to support data capture for research studies, providing 1) an intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages and 4) procedures for importing data from external sources. Survey questions were constructed based on review of the RCPSC objectives of training in paediatrics. A draft version of the survey was reviewed for face validity and approval by the paediatrics program directors of the Canadian Paediatric Program Directors network. Domains of the survey included: 1) Institution of study; 2) Year of residency; 3) NICU experience up to the time of the questionnaire (number of months of NICU training completed, level of acuity of NICUs worked in, and whether residents were buddied during their calls); 4) Whether trainees cross-cover NICU calls while on other rotations and 5) Self-reported experiences in performing specific neonatal procedures and resuscitation skills. Likert scales were used to quantify these experiences: never performed procedure (score 0), only performed procedure in simulation setting (score 1), performed procedure only under direct supervision with real patient (score 2), performed procedure independently with real patient (score 3) and taught other learners the procedure with real patient (score 4).

An electronic survey link with information about the survey was e-mailed to trainees via their program directors. All program directors in Canada were contacted to invite their trainees to participate. Participation was voluntary and anonymous. No efforts were made to identify respondents.

Self-reported evidence of competency was considered as scoring 3 or 4 on NICU procedures and resuscitative skills. A resuscitation skill index was calculated based on scores of 3 or 4 on all of the basic activities and procedures utilized in neonatal resuscitation: bag and mask or T-piece positive pressure ventilation; intubation of infant more than 28 weeks gestation in either case room or NICU and umbilical venous catheterization. Since the provision of chest compressions is an infrequent event, it was not included in the resuscitation skill index score.

Following verification that continuous variables were distributed normatively, variables were described by means and standard deviations for continuous variables. Frequencies and percentages were used to describe categorical variables. T–tests and chi-square was used to explore relationships between variables. A P-value less than 0.05 was considered statistically significant. A binomial logistic regression was performed to ascertain the effects of total number of blocks completed, participation in cross-cover calls, and whether a resident was considering specialized training in NICU on the resuscitation skill index score as an outcome variable. Statistical analysis was performed using SPSS, version 24 (IBM Corp, Armonk, NY, USA).

RESULTS

A total of 138 paediatrics residents completed the survey from an estimated eligible 625 trainees giving a response rate of 22%. Demographics of respondents are presented in Table 1. The participating residents were distributed fairly evenly according to their year of training: year 1 (37/138; 27%), year 2 (39/138; 28%), year 3 (46/138; 33%) and year 4 (16/138; 12%). There were relatively fewer participating year 4’s, which was expected given some paediatrics residents enter subspecialty training after year 3. There was participation from 15 out of 16 paediatrics residency programs across Canada.

Table 1.

Demographics of participants

Demographics
	Year 1	Year 2	Year 3	Year 4	Total
Participants from eastern universities	10	10	11	3	34
Participants from universities in Ontario	15	14	21	1	51
Participants from central and western universities	12	14	14	12	52
Total	37	38	46	16	137*

Eastern universities: Memorial University, Dalhousie University, Université Laval, Université de Sherbrooke, McGill University

Universities in Ontario: University of Ottawa, Northern Ontario School of Medicine, University of Toronto, McMaster University, Western University

Central and western universities: University of Manitoba, University of Saskatchewan, University of Alberta, University of Calgary, University of British Columbia

*Please note, one participating resident abstained from specifying their residency program

Procedures and resuscitation skills

Table 2 reports means of scores for individual procedures and resuscitation skills. Lumbar puncture was the only procedure to achieve a mean score of 3 or greater by year 3 and 4 of training.

Table 2.

Participants scores in neonatal procedures and skills

Average score in individual procedures and skills
	Year 1	Year 2	Year 3	Year 4	All Years
Procedures forming competence index
Positive pressure ventilation	1.6	2.3	2.9	2.5	2.3
Intubation (more than 28 weeks)	1	1.3	2	1.8	1.5
Umbilical venous catheterization	2	2.4	2.9	2.8	2.5
Other procedures and skills
Blood draw	1.1	0.9	1.4	1.6	1.2
Peripheral intravenous line	1.2	1	1.7	2.3	1.4
Peripherally inserted central catheter	0.1	0.4	0.2	0.7	0.3
Arterial puncture	0.2	0.5	1	2	0.7
Umbilical arterial catheterization	1.4	2	2.5	2.4	2
Peripheral arterial line	0.2	0.3	0.4	1	0.4
Bladder catheterization	0.6	0.5	1.3	0.6	0.8
Suprapubic catheterization	0.2	0.2	0.2	0.3	0.2
Nasogastric tube	0.8	1	1.6	1.3	1.2
Naso-jejunal tube	0.2	0.2	0.4	0.6	0.3
Lumbar puncture	1.8	2.6	3.4	3.6	2.8
Thoracocenthesis	0.3	0.5	1.2	1.3	0.8
Chest tube	0.4	0.3	1	1	0.7
Intubation for meconium	1	0.7	2.2	2.6	1.7
Intubation infant less than 28 weeks in case room	0.5	0.5	1	1.1	0.8
Intubation infant less than 28 weeks in Neonatal Intensive Care Unit	0.8	0.8	1.4	1.8	1.1

Resuscitation skill index

Twenty-three residents (17%) of the respondents satisfied the resuscitation skill criterion (score of 3 or greater in bag and mask or T-piece positive pressure ventilation; intubation of infant more than 28 weeks gestation in either case room or NICU and umbilical venous catheterization). Scores varied according to the current year of training at the time of the survey.

Further specialization in NICU

The majority of residents in the study reported not considering further specialized training in NICU (91%). Of these, the majority did not meet the resuscitation skill criterion (85%). Of those considering further specialized training in NICU, 33% met the resuscitation skill criterion. The association of consideration of specialized training in NICU and achieving the criterion was not statistically significant (P=0.12).

Autonomy, cross-cover calls and number of NICU blocks

The majority of residents reported being given increasing autonomy during their paediatrics residency with respect to NICU experiences (72%). Of these, 77% did not meet the resuscitation skill criterion. All of the residents who did not report increasing autonomy did not meet the criterion. The difference in proportions was statistically significant (P=0.0002). Fifty-seven per cent of the residents had exposure to cross-cover calls; of these, 8% met the resuscitation skill criterion. There was an inverse relationship between the exposure to cross-cover calls and criterion (P=0.001). Subanalysis of variables such as location of paediatrics training program and the year in which the residents participated in cross-cover calls showed no particular pattern with respect to the resuscitation skill criterion.

The number of blocks of NICU completed at the time of survey varied from 0 and 6. The logistic regression exploring the relationship between total number of NICU blocks completed, cross-cover NICU calls, and consideration for specialized training in neonatology to criterion score as an outcome is shown in Table 3. In this model, number of blocks of NICU completed was positively associated with the criterion score (P=0.003); while, cross-cover calls was negatively associated with the criterion score (P=0.027).

Table 3.

Binominal regression with resuscitation skill index as outcome

Variable	P-value	Odds ratio	95% CI for odds ratio
			Lower limit	Upper limit
Total number of blocks completed*	0.030	1.792	1.058	3.037
Cross-cover calls made (Yes vs. No)*	0.027	0.166	0.034	0.811
Considering specialized training in Neonatal Intensive Care Unit (Yes vs. No)	0.207	2.950	0.551	15.806

CI Confidence interval.

DISCUSSION

Newborn care is an expected competency of graduates of Canadian paediatrics residency training programs yet we report that a minority of trainees actually self-report evidence of competency in procedural skills and resuscitation activities in training (performing an activity independently with real patient or teaching other learners an activity with a real patient). To become proficient in neonatal resuscitation, a resident needs a comprehensive understanding of physiology, anatomy, equipment, medications, appropriate interventions and teamwork dynamics (3). Furthermore, these competencies require participation in resuscitation events (11). It would seem that the current minimum requirements of three blocks of newborn care, as established by the RCPSC, is potentially inadequate for trainees to meet the training objective to recognize, resuscitate and stabilize a critically ill newborn. Possibly residents are not attending enough resuscitations (12). Alternatively, exposure is sufficient yet residents are not being meaningfully engaged in activities such as practice, reflection, and feedback. Learning models have been articulated, such as deliberate practice and mastery learning, with such features cited as crucial for effective learning (13,14). Learning, however, may be complicated by many factors including the multidisciplinary nature of the resuscitation team, the presence of trainees at different levels of training needing to learn, practice and master certain skills, as well as nontrainee team members needing to maintain their expertise in neonatal resuscitation.

Educating trainees in neonatal resuscitation presents many challenges (15). With advances in perinatal care and changes in resuscitation algorithms, fewer babies are requiring breathing support, let alone intubation, chest compressions and/or medication administration after birth (16,17). Even the care of premature infants in NICU has become less invasive (15,18,19). Yet, with regionalization of care in Canada, paediatricians working in community and rural settings often do have to attend and lead neonatal resuscitations. The question that needs to be asked is whether paediatrics residency training needs to ensure each trainee achieves these competencies or only those who will be working in community and rural settings with responsibilities to potentially critically ill newborns. As paediatricians provide coverage for the delivery rooms and nurseries in many centres in Canada, this potentially has direct consequences on the provision of healthcare to newborns. Our study expectantly demonstrated a relationship between the number of completed blocks of NICU and self-reported experiences in providing NICU resuscitation skills and procedures. Interestingly, and contrary to what we had thought, there was an inverse relationship between cross-coverage calls in the NICU and self-reported resuscitation and procedural experiences. Exposure to cross-cover calls needs to be explored in more detail in future research to understand the reason for this inverse relationship. It is possible that sporadic and/or discontinuous exposures in the NICU ultimately are not very meaningful for trainees if the multidisciplinary team does not afford them autonomy and support to develop in their NICU competencies. As paediatrics programs transition to competency-by-design, program directors may need to anticipate that trainees will need to spend more time in the NICU unless additional educational strategies are adopted to maximizing learning from limited exposures.

We recognize multiple limitations of our study. First, the use of survey has intrinsic limitations, including the difficulty of providing strong evidence of cause and effect. Our study was also based upon self-reported experiences in providing NICU resuscitation skills and procedures. Actual competence does not necessarily always correlate with self-reported experiences of performance. Contrasts between self-confidence and actual performance have been observed in real-life interactions and simulated resuscitations (20–22). Finally, our limited sample of the total population of Canadian paediatrics residents limits generalizing findings to the broader population of residents who may have had different experiences.

CONCLUSION

Our study showed that a minority of paediatrics residents self-report evidence of competency in performing neonatal procedures and resuscitation skills. Future research needs to systematically address whether we are graduating paediatrics trainees with necessary neonatal competencies. Such research foreseeably may need to occur at the level of the RCPSC incorporating results from in-training and certifying exams to appreciate which educational strategies are ultimately benefiting trainees in achieving and maintaining competencies.

Funding Source: The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: Women & Children’s Health Research Institute.

Financial Disclosure: The authors declared no financial relationships that could be broadly relevant to the research, authorship, and/or publication of this article.

Conflict of Interest: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Clinical Trial Registration: Not applicable

Ethics Board: Approved by the University of Alberta Research Ethics Board.