Development and validation of a Self‐assessment Instrument for Perioperative Patient Safety (SIPPS)

Abstract

Background

Patient safety is a fundamental value of healthcare to avoid patient harm. Non‐compliance with patient safety standards may result in patient harm and is therefore a global concern. A Self‐assessment Instrument for Perioperative Patient Safety (SIPPS) monitoring and benchmarking compliance to safety standards was validated in a multicentre pilot study.

Methods

A preliminary questionnaire, based on the Dutch perioperative patient safety guidelines and covering international patient safety goals, was evaluated in a first digital RAND Delphi round. The results were used to optimize the questionnaire and design the SIPPS. For measurement and benchmarking purposes, SIPPS was categorized into seven main patient safety domains concerning all care episode phases of the perioperative trajectory. After consensus was reached in a face‐to‐face Delphi round, SIPPS was pilot‐tested in five hospitals for five characteristics: measurability, applicability, improvement potential, discriminatory capacity and feasibility.

Results

The results of the first Delphi round showed moderate feasibility for the preliminary questionnaire (81·6 per cent). The pilot test showed good measurability for SIPPS: 99·8 per cent of requested information was assessable. Some 99·9 per cent of SIPPS questions were applicable to the selected respondents. With SIPPS, room for improvement in perioperative patient safety compliance was demonstrated for all hospitals, concerning all safety domains and all care episode phases of the perioperative trajectory (compliance 76·1 per cent). SIPPS showed mixed results for discriminatory capacity. SIPPS showed good feasibility for all items (range 91·9–95·7 per cent).

Conclusion

A self‐assessment instrument for measuring perioperative patient safety (SIPPS) compliance meeting international standards was validated. With SIPPS, improvement areas for perioperative patient safety and best practices across hospitals could be identified.

Introduction

Patient safety is a fundamental value of healthcare to avoid patient harm. Non‐compliance with patient safety standards may result in permanent injury, increased length of stay in healthcare facilities or even death, and is a major global concern. Therefore, ensuring the safety of patient care is to be given the highest priority. Since the launch of the WHO patient safety programme in 2004, more than 140 countries have taken the challenge to identify risks in patient safety and improve safety performance, to prevent avoidable harm1. It is commonly reported that around one in ten hospitalized patients are harmed as a result of adverse events; at least 50 per cent of these events are considered preventable2. Most adverse events are related to surgical procedures (40 per cent) and medication errors (15 per cent)2.

National audits3, 4, 5 by the Dutch Health Care Inspectorate (IGZ) in the period 2007–2009 showed that perioperative care in the Netherlands could be improved with regard to information transfer, clinical documentation, teamwork and coordination. In response to the IGZ recommendations, national perioperative safety guidelines6, 7, 8 were developed in 2010–2012. These guidelines are in line with international patient safety goals (IPSGs)9 (Table  1), and are applicable to both surgical and non‐surgical interventions performed in hospitals. Adherence to evidence‐based guidelines is associated with safer perioperative care and improved outcome10, 11, 12, 13.

Table 1.

Definitions of international patient safety goals9

IPSG no.	Goal	Standard
1	To identify patients correctly	The hospital develops and implements a process to improve the accuracy of patient identification
2	To improve effective communication	The hospital develops and implements a process to improve the effectiveness of verbal and/or telephone communication among caregivers
3	To improve the safety of high‐alert medications	The hospital develops and implements a process to improve the safety of high‐alert medications
4	To ensure correct‐site, correct‐procedure, correct‐patient surgery	The hospital develops and implements a process for ensuring correct‐site, correct‐procedure, correct‐patient surgery
5	To reduce the risk of healthcare‐associated infection	The hospital adopts and implements evidence‐based hand hygiene guidelines to reduce the risk of healthcare‐associated infections
6	To reduce the risk of patient harm resulting from falls	The hospital develops and implements a process to reduce the risk of patient harm resulting from falls

ISPG, international patient safety goal.

To support hospitals in their efforts to improve patient safety, the non‐profit Dutch Safe Curative Care Association (VVCZ; http://www.vvcz.nl) was established in 2011. Within the VVCZ, 15 Dutch hospitals cooperate to improve patient safety by exchanging knowledge, best practices and well designed tools. To facilitate monitoring and benchmarking, the VVCZ developed and introduced an integrated PeriOperative Patient Safety audit (iPOPS). iPOPS examines the extent to which current perioperative patient safety performances meet (inter)national standards of care, and facilitates hospitals and departments to improve areas that are currently below standards. In this way, iPOPS helps staff to improve continuously the organization's perioperative patient safety performance. The iPOPS audit consists of four elements that are complementary to one another, each focusing on a different aspect of perioperative care; used together, they provide a complete assessment of the perioperative practice performance in daily healthcare (Fig. 1). This assessment includes the following four parts: a prospective Self‐assessment Instrument for Perioperative Patient Safety (SIPPS) by means of a questionnaire, conducted by professionals working in the perioperative trajectory; observations of perioperative practice performance in the workplace by peers, using the Surgical Patient safety Observation Tool (SPOT)14; assessment of locally used protocols based on (inter)national guidelines; and interviews with professionals directly involved in perioperative care.

Figure 1.

Components of the Integrated PeriOperative Patient Safety audit (iPOPS). SIPPS, Self‐assessment Instrument for Perioperative Patient Safety; SPOT, Surgical Patient safety Observation Tool

The aim of the present multicentre pilot study was to evaluate and optimize SIPPS as one of the four iPOPS core elements for further improvement and scientific validation of the instrument. The pilot study included testing of SIPPS in five hospitals to assess the clinimetric characteristics in line with previous studies14, 15, 16, 17, 18, 19.

Methods

The study consisted of two parts (Fig. 2): evaluating and improving the preliminary questionnaire by means of a RAND‐modified Delphi consensus procedure20 21; and validating SIPPS by pilot‐testing the instrument on its clinimetric characteristics in five hospitals14, 15, 16, 17, 18, 19 (Table  2). The self‐assessment questionnaire is a translation of the content of the perioperative guidelines into questions answered by a preselected group of healthcare providers working in the perioperative trajectory. This questionnaire was originally developed in Dutch and translated into English by an official translating agency. For measuring and benchmarking purposes, SIPPS was categorized into both seven patient safety domains (Table  3) and four perioperative care episode phases. Questions concerning safety culture were added to ascertain whether the organizational culture encouraged individual staff members to report concerns about safety or quality of care without retaliatory action from the hospital. These additional questions were excluded from the present study, because they were not based on the Dutch perioperative guidelines and international patient safety goals. Ethical approval was not required as no patients were involved.

Figure 2.

Evaluation and validation steps for a validated Self‐assessment Instrument for Perioperative Patient Safety (SIPPS)

Table 2.

Definitions of clinimetric characteristics14, 15, 16, 17, 18, 19

Criterion	Definition	Score
Measurability	Questions are measurable	Good: at least 80 per cent of questions are answered
		Moderate: more than 20 to less than 80 per cent of questions are answered
		Poor: 20 per cent or less of questions are answered
Applicability	Questions are applicable to the selected respondents	Good: at least 80 per cent of questions are applicable
		Poor: less than 80 per cent of questions are applicable
Improvement potential	Room for improvement of current practice (topic level)	Good: compliance with the standard is less than 90 per cent
		Poor: compliance with the standard is at least 90 per cent
Discriminatory capacity for comparison	Discrimination of practice performance (compliance with the standards) between different topics and between departments or hospitals	Good: more than 20 per cent variation between lowest and highest scores
		Poor: 20 per cent or less variation between lowest and highest scores
Feasibility	The questionnaire is clear, applicable and easy to use	Good: at least 90 per cent of respondents agree
		Moderate: 50–89 per cent of respondents agree
		Poor: less than 50 per cent of respondents agree

Table 3.

Definitions of patient safety domains covered by the Self‐assessment Instrument for Perioperative Patient Safety (SIPPS)

Domain	Definition
Verification	The process of checking the validity and completeness of a clinical or other requirement from the source that issued the requirement
Medical record	A written account by healthcare professionals of a variety of patient health information, such as assessment findings, treatment details, progress notes and discharge summary
Behaviour	Demonstrated (non)compliance with standards by healthcare professionals
Organization	Demonstration of the standards that are organized according to what is done directly and indirectly to provide for a safe, effective and well managed organization
Transfer	The formal shifting of responsibility for the care of a patient from: one care unit to another; one clinical service to another; one healthcare provider to another (also known as handover); or one organization to another
Patient communication	Standards that are organized according to what is done directly or indirectly to inform the patient
Standard operating procedure	A (combination of) protocol, procedure or process documentation
	Protocol: a scientific medical treatment plan or study outline for a procedure or treatment
	Procedure: a written document describing how a task is performed, usually including step‐by‐step instructions
	Process: a definition of a task that needs to be done and by whom

Evaluating and improving the preliminary questionnaire

Part 1 of the study (Fig. 2) was performed between January 2015 and July 2017 during iPOPS audits conducted in VVCZ member hospitals, in two Delphi rounds.

Based on the existing perioperative self‐assessment questionnaire used during iPOPS audits, the respondents of VVCZ‐audited hospitals (professionals working in the perioperative trajectory) were invited to comment on the content of this questionnaire. For this reason, in addition to each question, respondents could choose between four simple evaluation statements: ‘question is good’, ‘question is unclear’, ‘question is not relevant’ and ‘question is not applicable to my work’. Supplementary comments could be added to each question.

Results from the first Delphi round were analysed and prepared to present in a face‐to‐face VVCZ members' meeting, to reach consensus on the content of an optimized self‐assessment instrument, called SIPPS.

Validating SIPPS

In part 2 of the study (Fig. 2), SIPPS was pilot‐tested on five clinimetric characteristics: measurability, applicability, improvement potential, discriminating capacity and feasibility (Table  2). To evaluate SIPPS on the first four characteristics, the respondents could choose between five answers for each question: ‘yes’, meaning ‘we are 90–100 per cent compliant with this standard’; ‘partly’, meaning ‘we are 50–89 per cent compliant with this standard’; ‘no’, meaning ‘we are less than 50 per cent compliant with this standard’; ‘unknown’, meaning ‘I do not know the answer to this question’; or ‘not applicable’, meaning ‘this question is not applicable to my job’. Supplementary comments could be added to each question.

To evaluate the fifth characteristic (the feasibility of SIPPS), respondents were invited to comment according to a six‐point scale, varying from ‘totally agree’ to ‘totally disagree’ on four statements after completing SIPPS: ‘the questions were clearly formulated’, ‘the questions were relevant to my work’, ‘the answering categories provided enough possibilities to give the answer(s) I had in mind’ and ‘the time to complete the self‐evaluation SIPPS was acceptable’.

Results

Evaluating and improving the preliminary questionnaire

In the period from April to December 2016, respondents of the preliminary questionnaire commented on the instrument. Thirteen VVCZ member hospitals participated in this first Delphi round, and digitally and anonymously filled out 345 self‐assessment questionnaires and additional evaluation questions. All respondents were professionals working in the perioperative trajectory: anaesthesia nurses (31), anaesthetists (21), pharmacists (13), nurses working in the preanaesthesia care unit (17), nurses working in the admissions department (18), operating room (OR) managers (25), quality and safety managers (14), medical technologists (19), anaesthesiology employees working in the outpatient clinic (25), OR planners (27), surgical physicians (16), OR nurses (28), employees working in the polyclinic department with the exception of anaesthesia (15), postanaesthesia care nurses (25), surgical ward nurses (19), medical technology employees (14) and logistics employees (18). In total, 12 139 evaluating answers and 495 comments were received. The results of the first Delphi round showed moderate feasibility for the preliminary questionnaire (81·6 per cent) (Table  4).

Table 4.

Results of the first Delphi round for evaluation of the preliminary questionnaire

Evaluation statement	Score
Question is not applicable to my work	1243 (10·2)
Question is not relevant	129 (1·1)
Question is unclear	861 (7·1)
Question is good	9906 (81·6)
Total score	12 139 (100)

Values in parentheses are percentages.

A single researcher processed and analysed the results and suggestions from the first Delphi round, and then compared the questions with the perioperative guidelines in detail, translating the items resulting from the first Delphi round into clear questions for specific groups of professional caregivers. Existing categories of the preliminary questionnaire were considered and proposals made for adaptations. This researcher also studied the existing categories of the preliminary questionnaire and made proposals for adaptations. Subsequently, consensus was reached in a face‐to‐face VVCZ members' meeting on three remaining issues: combining or renaming certain employee groups and adding ICU employees (both nurses and physicians) to the list of professionals; adding a perioperative phase to the pre‐existing preoperative, intraoperative and postoperative phases (for questions concerning the total perioperative trajectory); and adding culture‐related questions to SIPPS.

SIPPS was then developed in a web‐based application. SIPPS contains 118 questions with 0–10 subquestions. Table  5 gives examples of SIPPS questions, categorization and professionals addressed. The complete SIPPS questionnaire is provided in Table  S1 (supporting information).

Table 5.

Examples of SIPPS questions and categorization

Question*	Care episode phase	Safety domain	Professionals
Do you obtain the following information from the referring specialist: reason for admission, relevant medical history, medication, specific details?	Preoperative	Transfer	Surgeon
Is anaesthesia induced with no background noise in the OR?	Intraoperative	Behaviour	Anaesthetist, anaesthesia nurse, OR nurse
Do at least two people transport the patient after leaving the OR?	Postoperative	Organization	Anaesthetist, anaesthesia nurse, OR nurse, postanaesthesia care nurse
Do you have access at all times to the patient data you require to work in a patient‐safe manner?	Perioperative	Medical record	Anaesthetist, anaesthesia nurse, pharmacist, ICU nurse, ICU physician, OR anaesthesia care manager, OR surgical care manager, OR nurse, planning employee, postanaesthesia care nurse, preanaesthesia care nurse, surgeon, surgical ward nurse

^*Answering categories: yes, partly, no, not known, not applicable, not answered. OR, operating room.

Validating SIPPS

Between March and June 2017, iPOPS audits were performed in five Dutch VVCZ hospitals (2 tertiary care and 3 regional care hospitals). During these audits, a total of 140 SIPPS questionnaires were digitally and anonymously filled out by professionals working in the perioperative trajectory: anaesthesia nurses (11), anaesthetists (10), pharmacists (5), medical technicians (6), preanaesthesia care nurses (9), ICU ward nurses (6), ICU physicians (5), OR anaesthesia care managers (8), OR surgical care managers (9), OR planners (10), quality and safety staff members (11), physicians (16), OR nurses (11), postanaesthesia care nurses (9) and surgical ward nurses (14).

Results with respect to measurability, applicability, improvement potential, discriminating capacity and feasibility of SIPPS are shown in Tables  6, 7 and 8. Feasibility results of SIPPS are shown in Table  9. Based on these results, five measurable characteristic results of SIPPS were defined.

Table 6.

SIPPS results per hospital per safety domain

Safety domain	Response	Yes	Partly	No	Not known	Not applicable	No response
Hospital 1
Verification	205	181 (88·2)	14 (6·9)	4 (1·8)	6 (3·0)	0 (0)	0 (0)
Medical record	272	223 (82·0)	26 (9·7)	5 (1·7)	17 (6·2)	0 (0)	1 (0·4)
Behaviour	63	53 (84)	5 (8)	3 (5)	2 (3)	0 (0)	0 (0)
Organization	369	288 (78·0)	46 (12·4)	14 (3·9)	21 (5·7)	0 (0)	0 (0)
Transfer	94	71 (75)	11 (12)	9 (9)	3 (3)	0 (0)	0 (0)
Patient communication	70	49 (70)	10 (15)	9 (13)	0 (0)	0 (0)	1 (1)
SOP	139	103 (74·3)	12 (8·9)	10 (7·2)	12 (8·9)	0 (0)	1 (0·7)
Total	1212	968 (79·9)	125 (10·3)	53 (4·4)	62 (5·1)	0 (0)	3 (0·3)
Hospital 2
Verification	112	109 (97·0)	2 (1·5)	0 (0)	1 (1·1)	0 (0)	0 (0)
Medical record	179	144 (80·2)	20 (11·0)	5 (2·6)	10 (5·6)	0 (0)	1 (0·6)
Behaviour	51	44 (85)	7 (13)	0 (0)	1 (2)	0 (0)	0 (0)
Organization	271	212 (78·1)	34 (12·6)	9 (3·1)	15 (5·6)	0 (0)	2 (0·6)
Transfer	95	73 (77)	8 (9)	7 (8)	6 (6)	0 (0)	0 (0)
Patient communication	26	19 (72)	3 (13)	3 (12)	1 (4)	0 (0)	0 (0)
SOP	68	57 (83)	3 (4)	2 (3)	5 (8)	0 (0)	1 (1·5)
Total	802	656 (81·8)	77 (9·6)	26 (3·2)	40 (5·0)	0 (0)	4 (0·4)
Hospital 3
Verification	141	121 (86·1)	11 (7·7)	3 (1·9)	6 (4·3)	0 (0)	0 (0)
Medical record	218	169 (77·4)	19 (8·8)	7 (3·0)	22 (10·3)	0 (0)	1 (0·5)
Behaviour	55	42 (75)	12 (22)	1 (2)	1 (1)	0 (0)	0 (0)
Organization	406	305 (75·0)	59 (14·6)	15 (3·8)	27 (6·7)	0 (0)	0 (0)
Transfer	117	89 (76·4)	9 (7·6)	12 (10·5)	2 (2·1)	4 (3·4)	0 (0)
Patient communication	38	23 (60)	9 (23)	0 (0)	6 (16)	0 (0)	0 (0)
SOP	147	101 (68·4)	13 (8·6)	5 (3·1)	29 (19·6)	0 (0)	1 (0·3)
Total	1122	849 (75·7)	132 (11·7)	42 (3·8)	93 (8·3)	4 (0·4)	2 (0·1)
Hospital 4
Verification	78	69 (88)	4 (6)	5 (6)	0 (0)	0 (0)	0 (0)
Medical record	109	81 (74·6)	18 (16·6)	5 (4·6)	5 (4·2)	0 (0)	0 (0)
Behaviour	32	22 (69)	8 (25)	2 (6)	0 (0)	0 (0)	0 (0)
Organization	192	113 (59·0)	40 (20·8)	15 (7·9)	24 (12·5)	0 (0)	0 (0)
Transfer	54	37 (69)	5 (9)	12 (22)	0 (0)	0 (0)	0 (0)
Patient communication	15	11 (71)	2 (15)	2 (13)	0 (0)	0 (0)	0 (0)
SOP	66	37 (56)	3 (4)	9 (13)	17 (26)	0 (0)	0 (0)
Total	546	370 (68·0)	81 (14·8)	49 (9·0)	46 (8·4)	0 (0)	0 (0)
Hospital 5
Verification	101	76 (75·0)	14 (14·1)	8 (7·5)	3 (2·6)	1 (0·9)	0 (0)
Medical record	164	121 (74·0)	28 (17·1)	4 (2·4)	11 (6·6)	0 (0)	0 (0)
Behaviour	45	31 (69)	11 (24)	2 (4)	1 (2)	0 (0)	0 (0)
Organization	330	233 (71·0)	60 (18·0)	11 (3·3)	26 (7·8)	0 (0)	1 (0·3)
Transfer	90	54 (60)	20 (22)	8 (9)	8 (9)	0 (0)	0 (0)
Patient communication	17	14 (82)	2 (13)	0 (0)	1 (6)	0 (0)	0 (0)
SOP	108	81 (75·2)	5 (4·6)	2 (2·2)	19 (18·0)	0 (0)	0 (0)
Total	855	610 (71·0)	140 (16·3)	35 (4·1)	69 (8·0)	1 (0·1)	1 (0·1)

Values in parentheses are percentages; due to rounding, values and percentages may not sum or calculate correctly. ‘Yes’ indicates 90–100 per cent compliance with this standard; ‘partly’ indicates 50–89 per cent compliance; ‘no’ indicates less than 50 per cent compliance; ‘not known’ indicates the answer to the question was unknown; ‘not applicable’ means the question was not applicable to the person's job. SIPPS, Self‐assessment Instrument for Perioperative Patient Safety; SOP, standard operating procedure.

Table 7.

SIPPS results per hospital per care episode phase in the perioperative trajectory

Care phase	Response	Yes	Partly	No	Not known	Not applicable	No response
Hospital 1
Perioperative	528	408 (77·3)	65 (12·2)	18 (3·4)	36 (6·9)	0 (0)	1 (0·2)
Preoperative	137	125 (91·0)	12 (8·7)	0 (0)	0 (0)	0 (0)	0 (0)
Intraoperative	312	258 (82·6)	26 (8·3)	14 (4·5)	13 (4·2)	0 (0)	1 (0·3)
Postoperative	235	178 (75·6)	23 (9·8)	21 (9·0)	12 (5·1)	0 (0)	1 (0·5)
Total	1212	968 (79·9)	125 (10·3)	53 (4·4)	62 (5·1)	0 (0)	3 (0·3)
Hospital 2
Perioperative	362	267 (73·7)	52 (14·3)	12 (3·3)	29 (7·9)	0 (0)	3 (0·8)
Preoperative	42	37 (88)	3 (7)	1 (2)	1 (2)	0 (0)	0 (0)
Intraoperative	223	201 (90·3)	10 (4·5)	7 (3·3)	4 (1·8)	0 (0)	0 (0)
Postoperative	175	150 (86·0)	12 (7·0)	5 (3·1)	6 (3·6)	0 (0)	1 (0·3)
Total	802	656 (81·8)	77 (9·6)	26 (3·2)	40 (5·0)	0 (0)	4 (0·4)
Hospital 3
Perioperative	593	439 (74·0)	81 (13·6)	18 (3·1)	54 (9·1)	0 (0)	2 (0·3)
Preoperative	55	44 (81)	6 (11)	2 (4)	2 (4)	0 (0)	0 (0)
Intraoperative	271	210 (77·5)	31 (11·6)	19 (6·9)	7 (2·6)	4 (1·5)	0 (0)
Postoperative	203	156 (76·8)	13 (6·5)	3 (1·7)	30 (14·9)	0 (0)	0 (0)
Total	1122	849 (75·7)	132 (11·7)	42 (3·8)	93 (8·3)	4 (0·4)	2 (0·1)
Hospital 4
Perioperative	269	174 (65·0)	44 (16·4)	12 (4·6)	38 (14·2)	0 (0)	0 (0)
Preoperative	34	23 (68)	4 (11)	7 (20)	0 (0)	0 (0)	0 (0)
Intraoperative	144	111 (77·0)	13 (8·8)	16 (11·0)	4 (2·9)	0 (0)	0 (0)
Postoperative	99	61 (62)	20 (20)	14 (14)	4 (4)	0 (0)	0 (0)
Total	546	370 (68·0)	81 (14·8)	49 (9·0)	46 (8·4)	0 (0)	0 (0)
Hospital 5
Perioperative	469	318 (68·0)	88 (18·8)	14 (3·1)	48 (10·2)	0 (0)	1 (0·2)
Preoperative	39	28 (71)	6 (15)	3 (7)	2 (5)	1 (2)	0 (0)
Intraoperative	215	157 (73·0)	35 (16·3)	17 (7·8)	7 (3·1)	0 (0)	0 (0)
Postoperative	132	108 (82·0)	11 (8·1)	1 (1·1)	12 (9·0)	0 (0)	0 (0)
Total	855	610 (71·0)	140 (16·3)	35 (4·1)	69 (8·0)	1 (0·1)	1 (0·1)

Values in parentheses are percentages; due to rounding, values and percentages may not sum or calculate correctly. ‘Yes’ indicates 90–100 per cent compliance with this standard; ‘partly’ indicates 50–89 per cent compliance; ‘no’ indicates less than 50 per cent compliance; ‘not known’ indicates the answer to the question was unknown; ‘not applicable’ means the question was not applicable to the person's job. SIPPS, Self‐assessment Instrument for Perioperative Patient Safety.

Table 8.

Yes (compliant) scores per safety domain and perioperative care phase for hospitals 1–5

	Total response	Hospital yes score					Total yes	Range (%)	Difference (%)*
		1	2	3	4	5
Safety domain
Verification	637	181 (88·2)	109 (97·0)	121 (86·1)	69 (88)	76 (75·0)	555 (87·2)	75·0–97·0	22
Medical record	942	223 (82·0)	144 (80·2)	169 (77·4)	81 (74·6)	121 (74·0)	738 (78·3)	74·0–82·0	8
Behaviour	246	53 (84)	44 (85)	42 (75)	22 (69)	31 (69)	191 (77·6)	69–85	16
Organization	1568	288 (78·0)	212 (78·1)	305 (75·0)	113 (59·0)	233 (71·0)	1150 (73·3)	59·0–78·1	19
Transfer	450	71 (75)	73 (77)	89 (76·4)	37 (69)	54 (60)	325 (72·1)	60–77	17
Patient communication	166	49 (70)	19 (72)	23 (60)	11 (71)	14 (82)	115 (69·5)	60–82	22
SOP	528	103 (74·3)	57 (83)	101 (68·4)	37 (56)	81 (75·2)	379 (71·8)	56–83	27
Total	4537	968 (79·9)	656 (81·8)	849 (75·7)	370 (68·0)	610 (71·0)	3453 (76·1)	68·0–81·8	14
Perioperative care phase
Perioperative	2221	408 (77·3)	267 (73·7)	439 (74·0)	174 (65·0)	318 (68·0)	1606 (72·3)	65·0–77·3	12
Preoperative	307	125 (91·0)	37 (88)	44 (81)	23 (68)	28 (71)	257 (83·7)	68–91·0	23
Peroperative	1165	258 (82·6)	201 (90·3)	210 (77·5)	111 (77·0)	157 (73·0)	937 (80·4)	73·0–90·3	17
Postoperative	844	178 (75·6)	150 (86·0)	156 (76·8)	61 (62)	108 (82·0)	653 (77·4)	62–86·0	24
Total	4537	968 (79·9)	656 (81·8)	849 (75·7)	370 (68·0)	610 (71·0)	3453 (76·1)	68·0–81·8	14

Values in parentheses are percentages; due to rounding, values and percentages may not sum or calculate correctly.

^*Maximum minus minimum range value. SOP, standard operating procedure.

Table 9.

Evaluation of feasibility of the SIPPS

Evaluation statement	Responded	Did not respond	Agreed totally	Agreed	Agreed slightly	Disagreed slightly	Did not agree	Disagreed totally	Total agreed	Total disagreed
The questions were clearly formulated	137	3	6	104	18	8	0	1	128 (93·4)	9 (6·6)
The questions were relevant to my work	136	4	12	89	24	9	2	0	125 (91·9)	11 (8·1)
The answering categories provided me enough possibilities to give the answers I had in mind	138	2	14	94	24	5	1	0	132 (95·7)	6 (4·3)
The time to complete the self‐evaluation SIPPS was acceptable	136	4	16	106	8	4	2	0	130 (95·6)	6 (4·4)
Total	547	13	48	393	74	26	5	1	515 (94·1)	32 (5·9)

Values in parentheses are percentages. SIPPS, Self‐assessment Instrument for Perioperative Patient Safety.

Measurability

SIPPS showed good measurability: 99·8 per cent (4527 of 4537) of the requested information was provided by the respondents (Tables  6 and 7).

Applicability

SIPPS showed good applicability: 99·9 per cent (4532 of 4537) of the questions were answered by the preselected respondents (Tables  6 and 7).

Improvement potential

Table  8 shows the patient safety compliance results (the total ‘yes’ scoring answers of the questionnaire of the five participating hospitals). Both in total and at an individual hospital level, the 90 per cent score was not reached: 76·1 (range 67·8–81·8) per cent. Room for improvement in perioperative practice performance is indicated by the SIPPS results, in all five pilot hospitals.

Discriminating capacity

Table  8 shows the discriminating capacity of SIPPS – the compared differences between the total ‘yes’ (compliant) results of the participating hospitals. For patient safety, good variation (more than 20 per cent) in compliance between hospitals was shown for three domains: 75– 97 per cent for the domain verification; 60–82 per cent for patient communication; and 56–83 per cent for standard operating procedure. The domains organization, transfer, behaviour and medical record showed poor variation (20 per cent or less): 19, 17, 16 and 8 per cent respectively. For the care episode phase, variation between hospitals was good with respect to comparison for the preoperative and postoperative phases (23 and 24 per cent respectively). Variation within intraoperative and total perioperative phases was 17 and 12 per cent respectively.

Feasibility

SIPPS showed good feasibility for all four quality criteria (94·1 per cent): clear formulation (93·4 per cent), relevance (91·9 per cent), good answering possibility (95·7 per cent) and acceptable time effort (95·6 per cent) (Table  9).

Discussion

SIPPS, a comprehensive self‐assessment instrument to measure perioperative patient safety, was pilot‐tested and validated for prospective monitoring, benchmarking and improving perioperative safety performance. SIPPS was evaluated by means of a RAND‐modified Delphi consensus procedure and pilot‐tested on its clinimetric characteristics.

The present study resulted in a validated and feasible measurement instrument for perioperative patient safety performance in daily clinical practice. All perioperative patient safety questions appeared to be easily measurable with SIPPS and were generally applicable to surgical procedures. The overall improvement potential appeared to be good. SIPPS also helped to identify patient safety domains with room for improvement. A good discriminatory capacity of the tool was shown for several topics, whereas the total hospital scores showed lower variation in the pilot period. These results indicate there is no need to adapt the content of SIPPS.

Based on the pilot SIPPS results, benchmarking of best practices between hospitals and different departments or phases in the perioperative trajectory is facilitated. For instance, hospital 4 could learn from the best practice performance of hospital 2, especially for the domains ‘organization’ and ‘standard operating procedure’ (Fig. 3), and in the preoperative and postoperative care episode phase (Fig. 4). SIPPS is currently used in all VVCZ member hospitals, as part of iPOPS. The iPOPS audit team consists of an audit leader together with two expert professionals in perioperative care (1 physician and 1 non‐physician). The hospitals use iPOPS results to support internal perioperative patient safety improvement initiatives. Results are anonymized and then used for benchmarking between VVCZ member hospitals. SIPPS is complementary to the previously evaluated SPOT14, which shows the results of observable patient safety topics. Together with protocol insight (judging clarity, currency and conformity with perioperative guidelines) and interviews with caregivers, a comprehensive perioperative audit is provided for Dutch hospitals, in a safe setting by and for professionals themselves. To the authors' knowledge, no similar self‐assessment tool is currently available to characterize perioperative patient safety. As well as surgical disciplines, SIPPS is also suitable for other medical specialties and activities with an interventional character, such as radiology, cardiology, bronchoscopy and endoscopy.

Figure 3.

Self‐assessment Instrument for Perioperative Patient Safety (SIPPS) results for compliance at the safety domain level in a hospital 2 and b hospital 4. ‘Yes’ indicates 90–100 per cent compliance with the standard; ‘partly’ indicates 50–89 per cent compliance; ‘no’ indicates less than 50 per cent compliance; ‘not known’ indicates the answer to the question was unknown. SOP, standard operating procedure

Figure 4.

Self‐assessment Instrument for Perioperative Patient Safety (SIPPS) results for compliance at care episode level in a hospital 2 and b hospital 4. ‘Yes’ indicates 90–100 per cent compliance with the standard; ‘partly’ indicates 50–89 per cent compliance; ‘no’ indicates less than 50 per cent compliance; ‘not known’ indicates the answer to the question was unknown.

Worldwide, patient safety programmes are designed to measure and improve safety in order to control risks and minimize potentially avoidable patient harm, by systems and teamwork approaches22. These programmes are intended not only to prevent adverse events and complications, but also to develop a ‘culture of safety’ within hospitals, setting up an infrastructure for surveillance and management of preventable patient harm13. New insights into the mechanisms by which compliance with safety and quality measures lead to improvements in patient outcomes suggest that hospitals taking steps to implement comprehensive sets of safe practices may benefit from improvements in the management of complications among higher‐risk surgical patients13. These findings highlight the importance of hospitals having systems to identify and treat surgical complications. Nonetheless, it remains difficult to evaluate the impact of hospital safety initiatives on patient outcomes, as the effectiveness may depend on the cumulative effect of many different coordinated care systems and safety interventions.

Extensive guidelines are transformed into a comprehensive and feasible self‐assessment instrument for professional caregivers, to measure perioperative patient safety performance. Perioperative experts from various hospitals participated in the present study, which resulted in broad support for the instrument. SIPPS offers a structured method for measuring and monitoring perioperative compliance and risks in order to improve patient safety. The pilot test showed that SIPPS is a suitable and easy‐to‐use tool for identifying patient safety risk areas throughout the perioperative process. Spreading this standardized validated assessment tool across institutions can be used to provide ongoing, comparable data to maintain and improve safety standards.

This SIPPS study has limitations. The evaluation characteristics used to validate the instrument were based on measurable clinical characteristics. Although the results showed good ability to identify opportunities for improvement of patient safety performance, results in outcome and/or quality of care cannot be measured by SIPPS. Because only a small number of representatives participated, the results do not provide a full representation of all those working in the perioperative trajectory. It is therefore suggested that hospitals use the tool regularly to stimulate a broad response. Greater attention needs to be focused on understanding the hospital's ability to improve patient outcomes through safety programmes and practices. Finally, the current English version of the questionnaire has some limitations because it has not been translated back into the original Dutch language by an independent person, to ensure that no meanings have been altered.

Supporting information

Table S1. SIPPS questions and categories