Addressing barriers to evidence-based medicine in pediatric surgery: an introduction to the Canadian Association of Paediatric Surgeons Evidence-Based Resource

Abstract

Background

Pediatric surgical practice lags behind medicine in presence and use of evidence, primarily due to time constraints of using existing tools that are not specific to pediatric surgery, lack of sufficient patient data and unstructured pediatric surgery training methods.

Method

We developed, disseminated and tested the effectiveness of an evidence-based resource for pediatric surgeons and researchers that provides brief, informative summaries of quality-assessed systematic reviews and meta-analyses on conflicting pediatric surgery topics.

Results

Responses of 91 actively practicing surgeons who used the resource were analysed. The majority of participants found the resource useful (75%), improved their patient care (66.6%), and more than half (54.2%) found it useful in identifying research gaps. Almost all participants reported that the resource could be used as a teaching tool (93%).

Conclusion

Lack of awareness of the resource is the primary barrier to its routine use, leading to potential calls for more active dissemination worldwide. Users of the Canadian Association of Paediatric Surgeons Evidence-Based Resource find that the summaries are useful, identify research gaps, help mitigate multiple barriers to evidence-based medicine, and may improve patient care.

Key messages.

What is already known about this subject?

• Reasons for low uptake of evidence-based resources by pediatric surgeons include difficulties in decision-making and lack of time for performing surgery.

• Although evidence-based medicine (EBM) is considered the gold standard for medical and surgical interventions, its development and application in pediatric surgery is lacking.

• Barriers to EBM implementation include time and resource limitations, overall lack of high-quality evidence, and the continuing culture of apprentice-style teaching.

What are the new findings?

• The majority of respondents (75%) found the resource summaries from the Canadian Association of Paediatric Surgeons Evidence-Based Resource (CAPS EBR) to be useful.

• Numerous respondents (66.6%) stated that the resource improved patient care, and more than half mentioned that the resource could identify research gaps.

• Nearly all respondents (93%) reported that the resource could be used as a teaching tool.

How might it impact on clinical practice in the foreseeable future?

• Potential for future studies to integrate the CAPS EBR into their practices.

• Studies can evaluate the effectiveness of the tool in transferring knowledge in real time.

• To increase awareness of the CAPS EBR from a patient perspective, we aim to promote the resource to families in hospital to facilitate shared decision-making in pediatric surgery.

Introduction

Reasons for low uptake of evidence-based resources (EBR) by pediatric surgeons include difficulties in decision-making and lack of time for performing surgery.^{1 2} Sackett et al describe evidence-based medicine (EBM) as ‘the integration of best clinical expertise and patient values’.³ Although EBM is considered the gold standard for medical and surgical interventions, its development and application in pediatric surgery is lacking.^4–6 In addition, analyses of the quality of EBM studies in pediatric surgery have reported methodological flaws (eg, lack of adherence to guidelines).^{5 7 8} Although there has been a response to the need for higher quality research in the field,^{6 9} generation and uptake of EBM in pediatric surgical practice is lacking.

We previously investigated barriers to the implementation of EBM by pediatric surgeons through semistructured interviews.¹⁰ Results stemming from this study indicated barriers that included time and resource limitations, overall lack of high-quality evidence, lack of critical appraisal skills, and the continuing culture of apprentice-style teaching. Similar results were found in a previous systematic review.¹¹

We have developed an online EBR for pediatric surgery residents, fellows, surgeons, and researchers to maximize accessibility to current and reliable content. This resource was established in 2012 and is accessible through the Canadian Association of Paediatric Surgeons (CAPS) website.¹² The goal of the resource is to improve patient care by providing easily accessible, up-to-date, evidence-based educational information. Essentially, CAPS Evidence-Based Resource (EBR) provides readers with information on conflicting topics by briefly summarizing the results of published systematic reviews and meta-analyses, most of which have been conducted by the CAPS EBR team. These summaries also contain information on the quality of evidence, as well as recommendations for further research within areas lacking high-quality evidence to avoid duplication of effort. The summaries help to direct research resources toward topics where evidence is insufficient. Details regarding the process of identifying conflicting topics are published on the CAPS EBR website.¹²

To identify key issues of particular interest to the CAPS members, a survey is developed and distributed. Once the clinical problem receiving the most votes is identified, a systematic review is subsequently undertaken, guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.¹³ After completion of the systematic review, evidence is disseminated on the CAPS EBR website, and summaries are written containing components of the methods involved, the main results, and clinical implications. A discussion regarding the possibility of future research is explored if there is a paucity of high-quality evidence or if it is difficult to draw conclusions based on the available evidence (eg, if the evidence is contradictory).

The objectives of the current study were to evaluate whether the CAPS members are aware of our resource and if they are using it or not. Specifically, we wanted to measure the members’ perception of the CAPS EBR in relation to its application during daily practice.

Methods

Participants

This questionnaire was designed for pediatric surgeons who are members of CAPS.

Study design

This was a survey for CAPS members. The survey development was informed by our previous research on barriers and facilitators to the implementation of EBM.¹⁰

An electronic survey was created via the Research Electronic Data Capture (REDCap) database, was sent to CAPS members electronically during the period, March 2019 to September 2020, and was distributed in person as a paper version during the CAPS 2019 and 2020 annual meetings in Toronto and Quebec City. REDCap is a widely recognized, secure, web-based application for building and managing surveys.¹⁴

The questionnaire (figure 1) included a combination of closed and open-ended questions. Questions included demographic information (eg, gender, qualifications, years of experience) and experiences with CAPS EBR (eg, Does it mitigate challenges to implementing evidence-based practice? Does it facilitate evidence-based practice?).

Figure 1.

Survey: evaluating the impact of evidence-based resource on pediatric surgery residents, fellows, and surgeons.

Pretesting the questionnaires

A preliminary review of the questionnaires was conducted with a team of content experts (AN and VG) whereby items were evaluated for redundancy, reduction, clarity, relevancy, and considerations for respondent fatigue. The survey was also reviewed and approved for distribution by the CAPS research committee.

Analysis

A descriptive analysis was conducted to investigate the impact of the CAPS EBR.

Results

A total of 115 participants completed the surveys, of whom 91 (79%) reported having used the CAPS EBR. Of those who reported sex (n=52), 50% were male. Most were practicing in Canada (49%), and a few had retired (5%). Participant characteristics can be found in table 1.

Table 1.

Characteristics of survey participants (N=115)

	n (%)
Sex (n=52)
Male	26 (50)
Female	21 (40.4)
Did not disclose	5 (9.6)
Country of practice (n=115)
Canada	57 (49)
USA	18 (15)
Asia	15 (13)
Europe	12 (10)
Middle East	7 (6)
Retired	6 (5)
Years of experience, mean (range)	15.6 (2–40)

Experience with using CAPS EBR

The majority of participants found the resource to have improved their patient care (66.6%), and more than half (54.2%) found the resource to be useful in identifying research gaps. 89% of respondents found the resource summaries useful, while 84% found the quality assessments (evidence ratings) to be useful. Almost all participants reported that the resource could be used as a teaching tool (93%).

Participants reported having used the CAPS EBR one to three times overall (52%), with few using it more than 10 times. The most common reasons for using the CAPS EBR included teaching, general learning and treatment decision-making. The vast majority of participants found the summaries and research quality appraisals provided by CAPS EBR to be useful (table 2).

Table 2.

Experience with CAPS EBR (n=91)

CAPS EBR users
Overall frequency of use
1–3 times	48 (52%)
4–6 times	26 (29%)
7–10 times	12 (13%)
10 times	5 (6%)
Reason for use
General learning	67 (74%)
Treatment decision-making	61 (67%)
Teaching	85 (93%)
Usefulness of summaries
Useful	81 (89%)
Somewhat useful	10 (11%)
Not useful	0
Usefulness of evidence ratings
Useful	76 (84%)
Somewhat useful	14 (15%)
Not useful	1 (1%)

CAPS EBR, Canadian Association of Paediatric Surgeons Evidence-Based Resource.

Reasons for not using CAPS EBR

A total of 28 (28/52, 54%) participants reported not using the CAPS EBR. Reasons for non-use included a lack of awareness of the resource (15/28, 53.6%), time constraints (3/28, 10.7%), not finding the resource to be useful (2/28, 7.1%), and other reasons, such as forgetting about the resource (5/28, 17.9%) and retirement (4/28, 14.3%).

EBR’s mitigation of barriers to implementation of evidence-based practices

CAPS EBR was found to mitigate common challenges for incorporating EBM in practice (figure 2). Most participants agreed the CAPS EBR addressed implementation challenges, such as a lack of resources (70.8%), time constraints (79.1%), and staying up to date with literature (75%). Half agreed the resource mitigated challenges of not knowing how to critically appraise available evidence and poor knowledge mobilization. Only some (25%) felt the resource mitigated preferences for apprentice-style teaching.

Figure 2.

Canadian Association of Paediatric Surgeons Evidence-Based Resource (CAPS EBR) mitigation of barriers to implementation of Evidence-based practice (EBP).

Discussion

We have developed an EBR on inconclusive surgical management topics to address barriers to evidence-based practice in pediatric surgery. The EBR is publicly available via the CAPS website.¹² The EBR currently covers around 14 conflicting topics with another 10 topics being peer reviewed before posting. The goal of the current study was to measure the impact of the EBR.

Prior literature has demonstrated that simple passive diffusion of new research and guidelines is insufficient to achieve change in physician practice, citing the inability to translate results of EBM research into everyday practice as a major challenge.¹⁵ Surgeons, and physicians in general, have limited time available to identify high-quality research informing their delivery of care. Therefore, finding information is important. Disseminating research in a timely fashion for immediate application and development of valid clinical guidelines are both necessary for the creation of evidence-based decision-making in healthcare.^16–19 Moreover, current EBRs need to be increasingly accessible, user-friendly, relevant, widely disseminated, and updated regularly to preserve their clinical relevance.¹⁵ Despite the utility of systematic reviews in condensing large bodies of evidence for clinical application, they are only as reliable as the primary studies they synthesize.^{20 21} This is particularly true in the setting of rare diseases with limited and low-quality evidence. As such, recommendations can be made to increase quality within the pediatric surgery literature. First, high-quality guidelines for observational studies and consequently systematic reviews must be produced and consistently followed, according to previously established international methodologies.²² Moreover, elements of a study in pediatric surgery must be reported clearly for readers to appraise the study and to apply it to clinical practice.²³ In fact, Rangel et al developed one such checklist for reporting observational studies in pediatric surgery, with improved reporting after implementation.^{24 25} Such checklists can aid during manuscript writing and protocol development for various study designs and thus can improve reporting quality.^{13 26 27} Another suggestion is to mandate the use of checklists by all pediatric surgery journals. Following these suggestions may be a first step to ensure that all literature is evidence based and is appropriate for clinical decision-making, especially in the rare disease setting.

Overall, our study demonstrates that the CAPS EBR is an effective teaching tool, and the majority of surgeons felt that it informed patient care. Additionally, surgeons found the summaries and quality appraisals to be useful. Importantly, the CAPS EBR filled necessary gaps in medical practice, such as a lack of resources, time constraints, and accessing current evidence-based literature. However, challenges to its implementation continue to be lack of awareness of the resource and limited time. Although our study sample size is relatively small, the findings yield promising results regarding the benefits of accessing and learning from the CAPS EBR.

To address the challenge of lack of awareness, the CAPS EBR team has partnered with Cochrane Canada and the Canadian Agency for Drugs and Technologies in Health (CADTH). Both partners actively disseminate new research resulting from the CAPS EBR through social media, with CADTH additionally agreeing to disseminate a one-page summary of the resource to their stakeholders, as appropriate. The CAPS EBR team also promotes the resource via Twitter (@CAPS_EBR) and at national and international scientific meetings. Moreover, for CAPS members, we will send automatic email notifications when evidence-based summaries are disseminated on the website, and we develop a dashboard with completion rates,^{28 29} indicating the number of systematic review summaries consulted on the website. We can also conduct performance reviews regarding accessibility of our website by obtaining data on the number of times the CAPS EBR resource was consulted. We can create more open-ended surveys embedded in the website to ask about utility of the resource. To ensure the reviews stay up to date, we will perform updates of existing systematic reviews every 3 years. With respect to time constraints, we will continue to create brief, one-page summaries and will consider delivering these summaries on convenient, oral platforms (eg, podcasts) too.

To increase awareness of the CAPS EBR from a patient perspective, our team is examining alternative routes for active dissemination. These include providing patient and family-friendly summaries and promoting the resource to families in hospital and remotely using different internet platforms during families’ initial hospital or subsequent clinic visits for purposes of facilitating shared decision-making in pediatric surgery.

In conclusion, our EBR conducts systematic reviews and meta-analyses and provides summaries, evidence ratings, and parent-friendly summaries (in the near future). Users of the CAPS EBR find that the resource helps to mitigate multiple barriers to EBM in pediatric surgery and helps to improve patient care. There is a need for more active dissemination to increase awareness of the existence of CAPS EBR for surgeons worldwide.

Data availability statement

Data are available upon reasonable request.

Ethics statements

Patient consent for publication

Not required.

Ethics approval

This study has been approved by the Research Ethics Board at the Children’s Hospital of Eastern Ontario (18/126X).