Global Survey of Perceptions of the Surgical Safety Checklist Among Medical Students, Trainees, and Early Career Providers

Abstract

Background

The Surgical Safety Checklist (SSC) has been shown to reduce perioperative complications across global health systems. We sought to assess perceptions of the SSC and suggestions for its improvement among medical students, trainees, and early career providers.

Methods

From July-September 2019, a survey assessing perceptions of the SSC was disseminated through InciSioN, the International Student Surgical Network comprising medical students, trainees, and early career providers pursuing surgery. Individuals with ≥2 years of independent practice after training were excluded. Respondents were categorized according to any clinical versus solely non-clinical SSC exposure. Logistic regression was used to evaluate associations between clinical/non-clinical exposure and promoting future use of the SSC, adjusting for potential confounders/mediators: training level, human development index, and first perceptions of the SSC. Thematic analysis was conducted on suggestions for SSC improvement.

Results

Respondent participation rate was 24%. Three-hundred and eighteen respondents were included in final analyses; 215 (67%) reported clinical exposure and 190 (60%) were promoters of future SSC use. Clinical exposure was associated with greater odds of promoting future SSC use (aOR 1.81 95%CI[1.03–3.19],p=0.039). A greater proportion of promoters reported “Improved Operating Room Communication” as a goal of the SSC (0.21 95%CI[0.15–0.27]-vs.−0.12 [0.06–0.17],p=0.031), while non-promoters reported the SSC goals were “Not Well Understood” (0.08 95%CI[0.03–0.12]-vs.−0.03 [0.01–0.05],p=0.032). Suggestions for SSC improvement emphasized context-specific adaptability and earlier formal training.

Conclusions

Clinical exposure to the SSC was associated with promoting its future use. Earlier formal clinical training may improve perceptions and future use among medical students, trainees, and early career providers.

Introduction

The World Health Organization Surgical Safety Checklist (SSC) was introduced in 2009 as a tool for surgical teams in global health systems to improve the quality of surgical care delivery[1]. In the ten years since its introduction, the SSC continues to demonstrate associations with improved long-term patient outcomes through enhanced teamwork and communication[2], [3]. Although the SSC has potential to continue to improve quality and safety, there remain barriers to sustaining its future implementation[3].

A missing component in the existing literature on SSC implementation is the role of the “Checklist Generation” – the medical students, trainees, and early career providers in independent practice after training whose education occurred after the introduction of the SSC, and therefore may have been exposed to the SSC. These individuals remain well-positioned to identify patient safety issues and quality improvement opportunities in surgery from their boots-on-the-ground, patient-facing experiences[4]. Furthermore, there is a growing interest to engage this next generation of clinicians in patient safety and quality improvement initiatives, as these individuals represent the stewards of similar current and future efforts[5]–[7]. Measuring and incorporating the voice of the Checklist Generation is therefore critical for further evolution and implementation of the SSC, and maintaining its sustainability.

The purpose of this study was to determine perceptions of the SSC and suggestions for its improvements among medical students, trainees, and early career providers through a global survey. We aimed to determine how these individuals were exposed to the SSC and if they would promote its future use. We hypothesized that certain types of exposures to the SSC would be associated with differential odds of promoting its future use. These findings have the potential to provide valuable information for evolving the SSC and engaging those providers responsible for ensuring its future implementation.

Materials and Methods

Survey Design

This study was approved by the Committee on the Use of Human Subjects at Harvard University. An anonymous, English language, multiple-choice survey was designed in accordance with published guidelines to collect information on perceptions of the SSC among medical students, trainees, or early career providers[8], [9]. Early career providers were physicians in independent practice within 2 years of completion of medical and/or surgical training. Questions were developed based on literature review and discussions with content experts. The questions were reviewed by survey methodologists for clarity.

The survey was pre-tested and delivered using the web-based platform Qualtrics (Qualtrics, Provo, Utah, USA). Pre-testing included iterative cognitive interviewing and pilot testing until saturation in feedback was achieved; this included a total of three cycles with 5–10 people in each cycle[8], [9]. These participants, including medical students, trainees, and early career providers, represented 9 countries and 12 different spoken languages. As the survey was written in English, this pre-testing was important to ensure the phrasing of the survey was interpretable and acceptable to the intended study population. At the end of this iterative process, the final survey included 36 items (Appendix 1). The survey incorporated skip logic navigating respondents to items based on their training level and SSC exposure status. Only survey items completed by medical students, trainees, and early career providers were incorporated in final analyses.

Survey Dissemination and Study Population

To evaluate perceptions of the SSC among medical students, trainees, and early career providers, the survey was distributed through InciSioN, the International Student Surgical Network, a not-for-profit international organization of medical students, trainees, and early career providers interested in surgery[10]. At the time of this study, InciSioN was comprised of 42 country-level National Working Groups (NWG) representing approximately 5,000 individuals. NWG leaders were contacted with a request to opt-in to participate in survey distribution. Participating leaders received the study protocol with a survey link and were responsible for e-mailing their NWG listserv. The leaders did not participate in the survey. After one month of listserv-based distribution, the survey was disseminated through NWG and InciSioN social media channels (Facebook, Instagram, and Twitter) for an additional month.

Partial survey responses were recorded, and no measures were taken to limit multiple submissions from the same respondent or Internet Protocol address. This allowed respondents located in areas with intermittent internet access to participate. Respondents were excluded from the final analyses if they did not report exposure to the SSC, were not a medical student, trainee, or did not consent. Respondents were also excluded if they self-identified as mid- or late-career providers (i.e. ≥2 years of independent practice after training), as the survey was designed specifically for medical students, trainees, and earlier-career providers. Additionally, respondents were excluded if they did not report a type of SSC exposure, perceptions of the SSC, or had ≥15% missing responses.

Exposure and Outcome Measures

Respondents were categorized according to clinical versus non-clinical exposure to the SSC (Question 9, Appendix 1). Clinical exposure included any exposure during medical school clinical rotations, as well as any SSC experience while a trainee or early career provider. Non-clinical exposure included lectures or other didactic sessions in medical school; through global organizations or at conferences; through reading journal articles about the SSC; or by viewing The Checklist Effect documentary or reading The Checklist Manifesto: How to Get Things Right book. Respondents who selected both clinical and non-clinical exposures were considered to have been exposed clinically.

The outcome, “promoting future SSC use,” was assessed through a single item asking respondents how likely they were to use of the SSC in the future (Question 28, Appendix 1). Options for responses included integer values on a 1–10 scale, where 1 represented the lowest likelihood and 10 the highest likelihood of future SSC use. Individuals were classified as promoters of future SSC use if they selected 9 or 10; any score ≤8 was considered a non-promoter. These definitions have been previously used in medical research to summarize individuals’ satisfaction with a certain initiative, treatment, or intervention[11].

The primary endpoint was the association between type of exposure to the SSC (clinical versus non-clinical) and promoting future SSC use. Secondary endpoints were to determine perceived goals of the SSC (Question 26, Appendix 1) and suggestions for its improvement. Suggestions were provided in predetermined categories (e.g., SSC content, structure, and implementation) and through additional free responses (Questions 29, Appendix 1).

Statistical Analysis

Survey participation rates were determined using the American Association for Public Opinion Research Standard Definitions Report [12]. Participation rates were determined by counting the number of usable partial/complete responses divided by the number of potential respondents. Data were summarized using frequency with percentages for categorical variables. Bivariate analysis was performed using Fisher’s exact test to determine significant covariates among respondents reporting clinical versus non-clinical SSC exposure. Tests of equality of proportions were used to determine differences in SSC perceptions among promoters versus non-promoters.

Logistic regression analysis was used to determine the association between type of exposure to the SSC and promoting its future use, adjusting for potential confounders, effect modifiers, or mediators selected a priori. Training level (Question 4, Appendix 1) was included as a potential confounder given more senior respondents (e.g., trainees versus medical students) may have greater exposure the SSC through experiential learning in clinical settings, and therefore may have greater buy-in for patient safety tools in general[13]. HDI classification of country where medical education occurred (Question 5, Appendix 1) was also selected as a potential confounder given known variation in uptake and use of the SSC in different resource settings[2]. Level of motivation to try the SSC at first exposure was scored on a 1–10 scale (1 represented lowest and 10 the highest motivation); individuals were classified as motivated if they scored ≥9 (Question 12, Appendix 1). This was included as a potential mediator because initial impressions (e.g., perceived value, ease of use, and appropriateness for local health systems) may influence promoting the future use of the SSC.

For the free response items on suggestions for SSC improvement, qualitative analysis was performed to categorize respondents’ feedback into themes. This occurred independently by two members of the research team by systematically and iteratively reviewing line-by-line responses and coding each using inductive reasoning. The final representative themes were reviewed and agreed to by consensus. All statistical analyses were performed using STATA software version 15.1 (StataCorp LLC, College Station, Texas). Statistical significance was defined as a two-sided p-value<0.05.

Results

Survey Response and Baseline Characteristics of Respondents

Following a request for participation sent to the 42 NWGs, 19 (45%) opted in and agreed to disseminate the survey to a theoretical study base of 1,985 members. Six-hundred and ten individuals from 21 countries initiated the survey and 478 provided usable complete or partial responses, corresponding to a 24% participation rate (478/1985, Appendix 2). Among all respondents, 120 were ineligible for inclusion because they had never been exposed to or were not familiar with the SSC (n=96); they self-identified as an experienced provider (≥2 years as an independent provider, n=19); or they did not provide consent (n=5). Of those eligible for inclusion, 172 were excluded because their survey was ≤85% complete (n=142); they did not report how they were exposed to the SSC (n=9); or they did not score their likelihood of promoting the SSC (n=31). A total of 318 respondents were included in final analyses; 215 (67%) reported any clinical exposure to the SSC and 103 (33%) reported non-clinical exposure only (Figure 1). Two-hundred and thirty-three (73%) reported that English was one of the languages commonly spoken during their medical education.

Fig 1. Survey dissemination and selection of participants.

Survey was distributed through InciSioN NWG (n=1,985) and social media efforts (e.g., Facebook, Instagram, Twitter NWG and InciSioN accounts). Abbreviations: surgical safety checklist (SSC)

Compared with individuals clinically exposed to the SSC, those reporting non-clinical exposure were younger (68.0% versus 37.1% reporting age 18–24 years, p<0.001) and in medical school (90.3% versus 63.7%, p<0.001). A greater number of respondents who were clinically exposed to the SSC were from “Very High HDI” countries (27.4% versus 16.5%, p=0.032). There were no other differences based on type of SSC exposure (Table 1).

Table 1: Baseline characteristics of survey responders included in final analysis.

Distribution of demographics among those exposed to surgical safety checklists in non-clinical and clinical settings. Abbreviations: World Health Organization (WHO); Human Development Index (HDI). Bivariate analysis performed using Fisher’s exact test. Statistical significance defined as a two-sided p-value<0.05.

		Non-Clinical Exposure	Clinical Exposure	p-value
		n=103 (33%)	n=215 (67%)
Age	18–24y/o	70 (68.0%)	79 (37.1%)	<0.001
	25–30 y/o	28 (27.2%)	106 (49.8%)
	31–34 y/o	4 (3.9%)	22 (10.3%)
	35–40 y/o	1 (1.0%)	4 (1.9%)
	Prefer not to disclose	0 (0.0%)	2 (0.9%)
Self-Identified Gender Pronoun	He/him	43 (41.7%)	118 (54.9%)	0.15
	She/her	55 (53.4%)	86 (40.0%)
	They/their	3 (2.9%)	4 (1.9%)
	Other	1 (1.0%)	1 (0.5%)
	Prefer not to report	1 (1.0%)	5 (2.3%)
	Did not report	0 (0.0%)	1 (0.5%)
Level of Training	Early Career Provider (of <2 years)	1 (1.0%)	7 (3.3%)	<0.001
	Medical School	93 (90.3%)	137 (63.7%)
	Trainee (Resident, Registrar, Fellow)	9 (8.7%)	71 (33.0%)
WHO Region	Europe	20 (19.4%)	46 (21.4%)	0.11
	Western Pacific	3 (2.9%)	5 (2.3%)
	Americas	22 (21.4%)	37 (17.2%)
	Africa	37 (35.9%)	91 (42.3%)
	South-East Asia	0 (0.0%)	2 (0.9%)
	Eastern Mediterranean	6 (5.8%)	22 (10.2%)
	Other/Did not report	15 (14.6%)	12 (5.6%)
HDI Classification	Very High HDI	17 (16.5%)	59 (27.4%)	0.03
	High HDI	20 (19.4%)	19 (8.8%)
	Medium HDI	5 (4.9%)	13 (6.0%)
	Low HDI	43 (41.7%)	97 (45.1%)
	Other	17 (16.5%)	26 (12.1%)
	Not Reported	1 (1.0%)	1 (0.5%)

Perceptions of the SSC

Table 2 shows the baseline perceived goals of the SSC and suggestions for areas of improvement. There were no differences in level of motivation to use the SSC at first exposure or perceived goals among those reporting non-clinical versus clinical exposure. Respondents with clinical exposure to the SSC more frequently identified SSC implementation as an area for improvement compared with respondents reporting non-clinical exposure (29.3% versus 14.4%, p=0.008). A greater proportion of respondents with non-clinical exposure did not identify suggestions for improvement compared with those reporting clinical exposure (74.4% versus 54.8%, p=0.008).

Table 2: Surgical safety checklist perceptions.

Distribution of perceptions among those exposed to surgical safety checklists in non-clinical and clinical settings. Bivariate analysis performed using Fisher’s exact test. Statistical significance defined as a two-sided p-value<0.05.

		Non-Clinical Exposure	Clinical Exposure	p-value
		n=103 (33%)	n=215 (67%)
Motivated to Use SSC at Initial Exposure		43 (41.7%)	68 (31.6%)	0.180
Perceived Goals	Improves OR Team Communication	15 (14.6%)	40 (18.6%)	0.110
	Improved Patient Safety	56 (54.4%)	130 (60.5%)
	Malpractice Protection	23 (22.3%)	37 (17.2%)
	Not Well Understood	7 (6.8%)	8 (3.7%)
	Did not report	2 (1.9%)	0 (0.0%)
Suggestions for Improvement	Content	2 (2.2%)	15 (8.0%)	0.008
	Structure	5 (5.6%)	9 (4.8%)
	Implementation	13 (14.4%)	55 (29.3%)
	No Changes	67 (74.4%)	103 (54.8%)
	Other	3 (3.3%)	6 (3.2%)

Respondents clinically exposed to the SSC had greater odds of promoting its future use, compared with respondents who reported non-clinical exposure (OR 1.66 95%CI [1.02–2.67], p=0.038). This association remained significant after adjusting for level of training, HDI classification, and level of motivation to try using the SSC at first exposure (aOR 1.81 95%CI [1.03–3.19], p=0.039). A greater proportion of promoters perceived “Improved Operating Room Communication” as a goal of the SSC (0.21 95%CI [0.15–0.27]) compared with non-promoters (0.12 [0.06–0.17], p=0.031). A greater proportion of non-promoters reported the SSC goals were “Not Well Understood” (0.08 95%CI [0.03–0.12]) compared with promoters (0.03 [0.01–0.05], p=0.032, Figure 2).

Fig 2. Perceived goals of SSC.

Proportion of 318 responses among promoters and non-promoters of checklist goals are shown. Count of responses shown within each bar. Tests of equality of proportions were used to determine differences in perceived goals of the SSC among promoters versus non-promoters.*Denotes statistical significance at two-sided alpha <0.05

Suggestions for SSC Improvement

The results of qualitative analysis of 72 free response suggestions are shown in Table 3. Themes for content change included modifiable checklist items for context-specific use, such as resource-constrained environments and conflict zones. Themes for structural changes to the SSC included that it should be designed such that it could be executed by any member of the operating theater, including medical trainees. Themes for implementation changes identified formal curricula in using the SSC earlier in medical training. Among 278 suggestions for improvements to the SSC, there were similar proportions of promoters and non-promoters who identified improvements in the domains of SSC content, structure, and implementation (Figure 3).

Table 3: Summary of major themes on SSC improvement.

Seventy-two suggestions provided for changes to SSC content, structure, and implementation. Abbreviations: point-of-care (POC); venous thromboembolism (VTE); surgical site infection (SSI); operating room (OR); surgical safety checklist (SSC).

Content	Structure	Implementation
Modifiable checklist for POC use (e.g., consider resources, acuity of surgery, type of procedure, patient characteristics)	Structure for execution by any member of the OR team (e.g., trainees and anesthesia)	Formal training curricula and/or introduction on SSC
Incorporate perioperative quality measures (e.g., VTE, antibiotics, SSI prevention)	Provide structured themes rather than individual questions which may be redundant/unnecessary	Quality-control on proper implementation and use of SSC
Review intraoperative resource availability (e.g., equipment, instruments)		Earlier exposure to SSC (medical school, training)
		Mandatory use

Fig 3. Suggestions for improvement of SSC.

Proportion of 278/318 responses among promoters and non-promoters of suggestions for checklist improvement are shown. Count of responses shown within each bar

Discussion

The results of this study suggest that among medical students, trainees, and early career providers, clinical exposure to the SSC was associated with greater odds of promoting its future SSC use compared with non-clinical exposure. Respondents further in their education, training, or independent practice more frequently reported clinical exposure to the SSC. In addition, the perceived goals of the SSC were different among promoters and non-promoters of future use. There were similar proportions of promoters and non-promoters who identified areas for improvement in SSC content, structure, and implementation.

There is a growing body of literature describing the effectiveness of different approaches to implementing patient safety and quality improvement initiatives among medical students, trainees, and early career providers[5]. The increased attention to these strategies underscores the important role of the next generation of clinicians in adopting, sustaining, and improving current patient safety and quality improvement programs[6], [7]. While both observational experiences, such as didactic or similar traditionally pre-clinical education experiences, and experiential (e.g., real-life or simulation-based clinical exposures) approaches have value, the latter is associated with a greater willingness to change preferences, enhance knowledge-acquisition, and translate into future clinical practice[13], [14]. Our findings reflect this by demonstrating that clinical exposure was associated with greater odds of promoting future use of the SSC. In addition, a greater number of respondents reporting clinical exposure to the SSC identified areas for its future improvement, which may similarly reflect the importance and influence of different types of exposure.

Prior global studies, which focused primarily on measuring uptake of the SSC, have also demonstrated differential odds of SSC use based on level of experience. Vohra et al. conducted an international survey in 2013 and found that older age and professional seniority were independent predictors of SSC use, compared with being a trainee or medical student[15]. A similar cross-sectional study by Kilduff et al. in 2018 found that among 935 senior-level medical and nursing students in the United Kingdom, fewer than one-third understood the purpose of the SSC and over one-half reported no teaching in SSC use in clinical or non-clinical settings[16]. The results of this study add to these findings by demonstrating the need to improve SSC penetration among medical students, trainees, and early career providers (96/610 reported no exposure to the SSC) and variation in the perceived goals of the SSC. In addition, these findings extend from those of previous research by reporting that the type of exposure to the SSC may be important in regard to promoting its future use.

Over the last decade, the SSC has been adapted and modified in a variety of operating room settings based on local needs and preferences. In a 2018 study by Solsky et al.,155 publicly available checklists were reviewed. All were modified, most frequently with additional items addressing perioperative quality measures (e.g., anticoagulation for venous thromboembolism prophylaxis) or equipment availability[17]. Few versions of the SSC included modifications to improve communication and teamwork among the operating room staff. In this study, similar themes emerged in the suggestions for SSC content changes. However, it is important to also recognize the additional structural and implementation changes, such as student-participation in the SSC execution and earlier, formal training in SSC use, that were identified by the respondents. These suggestions for SSC modifications and adaptations should be considered not only as the SSC evolves, but also when attempting to identify potential barriers to its implementation.

Strengths and Limitations

These findings should be interpreted in the context of the study design. A strength of our approach was collaborating with an international research, education, and advocacy organization during the design and dissemination of a survey with quantitative and qualitative components[18]. In doing so, the survey incorporated perspectives of different training and practice environments, such as low- and middle-income settings, and resulted in participation from a diverse study base.

There were also several limitations. First, in regard to generalizability, the study sample was a relatively small proportion of the targeted population. Several logistical issues reported by participating NWG may be responsible for this lower rate. These include unreliable internet connectivity in certain regions and within-country conflict (e.g., Democratic Republic of Congo’s Ebola crisis). This may have also contributed to why certain NWG did not participate and partially completed individual surveys. Despite these unforeseen issues, we were able to reach participants from 21 countries, the majority of which identify as low- and middle-income countries. Second, the reach of the survey through social media efforts was difficult to quantify. Third, the survey was offered only in English. Translating the survey into different languages based on the InciSioN NWG may have resulted in a greater number and completeness of surveys. However, there were no differences in the proportion of English-speaking countries among those NWG that participated versus opted-out (58% versus 54%) [19]. Fourth, the majority of respondents were medical students, and we were unable to contextualize their clinical versus non-clinical exposure with their institutions’ training or use related to the SSC. Similarly, facility-level data outside of the survey on each respondent were not collected; therefore, we were unable to validate or contextualize self-reported exposure to the SSC. Lastly, the study base was comprised of individuals who were interested in global surgery, and therefore were likely to be aware of patient safety and quality improvement programs like the SSC. This convenience sampling may have produced unmeasurable selection bias. However, it is important to report the perspectives from these medical students, trainees, and early career providers, who may be invested in improving interventions like the SSC at baseline, as they are likely to remain stakeholders in similar and new patient safety and quality improvement programs in the future[20].

Conclusion

We demonstrate that exposure to the SSC in the clinical setting is associated with promoting its future use among medical students, trainees, and early independent providers. Future work includes collecting facility-level data from respondent’s hospitals to contextualize quality of SSC use and validate exposure to the SSC. Mixed-methods approaches to measuring the influence and relative contribution of specific clinical exposures on promoting long-term SSC use are also needed. Ultimately, we must identify innovative methods to expose medical students, trainees, and early career providers to the SSC in ways that are structured, intentional and lead to promoting its future use. Continuing to measure and incorporate the perspectives of the Checklist Generation will allow for its ongoing improvement and sustainability.

Appendix 1: Final survey disseminated for data collection.

The version disseminated through social media platforms included an additional question asking respondents if they belonged to an InciSioN-affiliated NWG. The skip logic mapping of survey branching is censored for simplicity; only responses to items completed by medical students, trainees, and early career providers were incorporated into final analyses.

1. Which InciSioN National Working Group are you currently a member of?

2. What is your age?

   • 18–24 years old
   • 25–30 years old
   • 31–34 years old
   • 35–40 years old
   • Greater than or equal to 41 years old
   • Prefer not to disclose
3. What is your preferred gender pronoun?

   • She/her
   • Him/he
   • They/their
   • Other (please specify):
   • Prefer not to disclose
4. What is your highest level of clinical experience?

   1. Medical school (current student or recent graduate)
   2. Trainee (i.e. intern, resident, registrar, fellow, etc)
   3. Independent provider (i.e. completed clinical training)
5. In which country are you obtaining, or did you obtain most of your medical school education?

   • 5a.
     What language(s) were/are commonly spoken in the environment where you obtained/are obtaining most of your medical school education? (select all that apply):

     1. English
     2. Russian
     3. Arabic
     4. Chinese
     5. French
     6. Spanish
     7. Other (please specify):
   • 5b.
     (If you are a student) What is the current stage of your education as a medical student?

     1. Pre-clinical training (i.e. classroom)
     2. Clinical training
     3. My education combines clinical and classroom learning from the beginning
     4. Medical School graduate
     5. Other (please specify):
   • In which country are you obtaining or did you obtain most of your post-medical school training?
   • 6a.
     What language(s) were/are commonly spoken in the environment where you obtained/are obtaining most of your post-medical school training? (Select all that apply)

     1. English
     2. Russian
     3. Arabic
     4. Chinese
     5. French
     6. Spanish
     7. Other (please specify):
   • 6c.
     (If you are a trainee) How many years of post-medical school training have you completed?

     1. < 2 years of training post medical school graduation
     2. ≥ 2 years of training post medical school graduation
   • In which country are you currently practicing independently?
   • 7a.
     What language(s) are commonly spoken in your current clinical practice environment? (select all that apply)

     1. English
     2. Russian
     3. Arabic
     4. Chinese
     5. French
     6. Spanish
     7. Other (please specify):
   • 7b.
     (If you are an early-career provider) how long have you practiced independently?

   1. less than 6 months
   2. More than or equal to 6 months and less than 1 year
   3. More than or equal to 1 year and less than 2 years
   4. More than or equal to 2 years
6. (If you are a trainee or provider) what is your clinical focus (i.e. specialty)?

   1. Anesthesia
   2. Surgery (please list sub-specialty)
   3. Obstetrics/gynecology
   4. General practice (i.e. family practice)
   5. General/Internal Medicine
   6. Other (please describe):

Definition: By “exposure,” we mean having seen, heard of, used, or discussed the Surgical Safety Checklist

1. Where have you been exposed to the Surgical Safety Checklist (select all that apply)?

   1. During Medical school - in classroom setting
   2. During Medical school - in clinical setting
   3. As a trainee (i.e. intern, resident, registrar, fellow, etc)
   4. As an independent provider (i.e. Completed clinical training)
   5. Through InciSioN or other global organization
   6. At Conference(s)
   7. In the Academic literature or journal articles
   8. The Checklist Manifesto by Atul Gawande
   9. The Checklist Effect documentary through InciSioN showing
   10. The Checklist Effect documentary independent of InciSioN showing
   11. Other: {please describe}
   12. I am not familiar with the Surgical Safety Checklist
2. (Among the selected exposure), where were you first exposed to the Surgical Safety Checklist (one option only)

   1. During Medical school - in classroom setting
   2. During Medical school - in clinical setting
   3. As a trainee (i.e. intern, resident, registrar, fellow, etc)
   4. As an independent provider (i.e. completed clinical training)
   5. Through InciSioN or other global organization
   6. At Conference(s)
   7. In the Academic literature or journal articles
   8. The Checklist Manifesto by Atul Gawande
   9. The Checklist Effect documentary through InciSioN showing
   10. The Checklist Effect documentary independent of InciSioN showing
   11. Other: {copy what was described in Q9 k}
   12. I do not recall
3. How was it incorporated into the classroom setting in your medical school?

   1. Part of the formal curricula
   2. Other: please specify
   3. I don’t know or I don’t recall

4. When you were first exposed to the Surgical Safety Checklist, how motivated were you to try it? Likert scale 1–10 with legend (1 - not motivated; 5 - somewhat motivated; 10 - very motivated)

Definition: By “training” we mean instruction on using the Surgical Safety Checklist in a patient care setting.

1. Have you been formally or informally trained on how to use the Surgical Safety Checklist?

   1. Yes
   2. No

      • 13a.
        When were you trained on how to use the Surgical Safety Checklist? (select all that apply) (only display if trainee or provider)
   3. Medical school
   4. Trainee (i.e. intern, resident, registrar, fellow, etc)
   5. Independent Practicing provider (i.e. completed clinical training)
2. When you were trained to use the Surgical Safety Checklist, what method(s) were utilized during the training (Select all that apply)?

   1. Didactic/lecture
   2. Role play/simulation
   3. Seeing proper Surgical Safety Checklist use modeled for me
   4. Other (please describe)
3. What training method for the use the Surgical Safety Checklist do you think was the most successful?

   1. Didactic/lecture
   2. Role play/simulation
   3. Seeing proper Surgical Safety Checklist use modeled for me
   4. Other (please describe)
   5. I don’t know

Definition: We define using the Surgical Safety Checklist as being involved in a patient care episode in which the Surgical Safety Checklist was used

1. Have you ever personally used the Surgical Safety Checklist (or a version of it) in an operating theatre during a clinical case?

   1. Yes
   2. No
2. Do you currently care for surgical patients (as a provider or trainee)?’

   1. Yes
   2. No
3. Select all the characteristics that describe the facility where you spend most of your time caring for surgical patients? (select all that apply)

   1. Rural
   2. Urban
   3. Private
   4. Public
   5. Community/district hospital
   6. Academic
   7. Other: (please describe)
4. To the best of your knowledge, how was the Surgical Safety Checklist implemented (i.e. introduced into practice) in the facility where you currently practice or are being trained? (Select all that apply)

   1. Mandated
   2. Quality improvement initiative
   3. Other: (please describe)
   4. I don’t know
   5. The Surgical Safety Checklist is not currently used at my facility
5. What professions are using the Surgical Safety Checklist in your facility (select all that apply)?

   1. Perioperative staff (i.e. nurse, scrub tech, etc)
   2. Surgeon (specify specialty:)
   3. Anesthesiologist
   4. Resident/trainee (specify specialty:)
   5. Other: (please describe)
   6. I don’t know
6. In your facility, in what proportion of patients undergoing surgery is the Surgical Safety Checklist used?

   1. Rarely (0–25% of patients)
   2. Occasionally (25–50% of patients)
   3. Most of the time (51–75% of patients)
   4. Majority of the time (76%−100% of patients)
   5. I don’t know
7. Why is the Surgical Safety Checklist used with this frequency in your facility? (Select all that apply)

   1. Not mandated
   2. Only some specialties use it
   3. Not always available
   4. Time consuming
   5. Low or little/no perceived value
   6. Patients don’t like it
   7. Implementation challenges
   8. Other: (please describe)
   9. The Surgical Safety Checklist is used for 100% of cases, every day
8. How do you think the Surgical Safety Checklist positively impacts patient care? (select all that apply)

   1. I think it reduces errors in the operating room
   2. It makes me feel more confident to speak up when I see something going wrong
   3. I feel the team works better together
   4. Other: (please describe)
   5. I don’t think it positively impacts patient care
9. In contrast, why do you think the Surgical Safety Checklist does not improve patient care? (select all that apply)

   1. We already do everything on it
   2. We don’t have the resources to do everything on it
   3. It was never properly introduced into my facility
   4. There is limited buy-in
   5. Other: (please describe)
10. Overall, how do you think the Surgical Safety Checklist affects patient care?

    1. Positively
    2. No effect
    3. Negatively
    4. I don’t know
11. Which of the following best describes the goal of the Surgical Safety Checklist?

    1. To protect healthcare professionals from malpractice
    2. A tool to remind you to complete key perioperative patient safety processes
    3. A tool to improve communication among the operating room team
    4. I don’t really understand the goal of the Surgical Safety Checklist
12. Why would you use the Surgical Safety Checklist in the future? (select all that apply)

    1. It is mandated
    2. Improves patient safety
    3. Improves team communication
    4. Standard of care
    5. Easy to use
    6. Quick to use
    7. Patients like it
    8. Other: (please describe)
    9. I don’t know
13. Overall, how likely are you to use the Surgical Safety Checklist in your future practice? Likert scale 1–10; 1 being “not likely at all,” and 10 being “very likely.”

    1. Please describe why:
14. How could the Surgical Safety Checklist be improved (select all that apply)?

    1. Changes to what is included on the Checklist (content) (please describe):
    2. Changes to how the Checklist is structured (design and organization) (please describe);
    3. Changes related to how it is implemented/introduced (please describe);
    4. Other (please describe):
    5. I wouldn’t change anything

Appendix 2: Report of Survey Respondents.

Among 1985 InciSioN members eligible to receive the survey, the following responses were determined using the American Association for Public Opinion Research Standard Definitions.

Complete Response = 451

Partial Response = 27

Refusal and Break Off = 5 (Refusal) + 132 (Break Off)

Non-Contact = 1375

Other = 0

Unknown Household = not applicable

Unknown Other = not applicable

Please note above, break-off was considered to be <85%, partial response 85–99%, and complete response 100%. These thresholds were used given what was felt to be proportional of applicable/crucial questions. In addition, similar thresholds have been offered in the Standard Definitions report of the AAPOR[12].

The survey developed for this study was most consistent with the AAPOR definition of a non-probability web-based survey because that the exact number of people who were exposed to the survey were not known (e.g., estimates of InciSioN’s national working group (NWG) size was determined based on size of e-mail listserv, not through direct contact). In such studies, the AAPOR recommends reporting participation and/or completion rates, rather than response rates. The definition used in the Standard Definition report for these rates is the “number of respondents who have provided a usable response divided by the number of personal invitations requesting participation.” Overall NWG participation rate was 45% (19/42). Respondent participation rate was determined by considering usable responses (451 complete and 27 partial responses) divided by a total of 1985 potential respondents. This respondent participation rate is 24%. Because we do not know the receipt of the survey when disseminated through social media platforms, we do not estimate a response or participation rate, consistent with AAPOR recommendations.