Understanding Patient and Clinician Reported Non-Routine Events in Ambulatory Surgery

Abstract

Objectives:

Non-routine events (NREs, i.e., deviations from optimal care) can identify care process deficiencies and safety risks. NREs reported by clinicians have been shown to identify systems failures, but this methodology fails to capture the patient perspective. The objective of this prospective observational study is to understand the incidence and nature of patient and clinician-reported NREs in ambulatory surgery.

Methods:

We interviewed patients about NREs that occurred during their perioperative care using a structured interview tool prior to discharge and in a 30-day follow-up call. Concurrently, we interviewed the clinicians caring for these patients immediately post-operatively to collect NREs. We trained two experienced clinicians and two patients to assess and code each reported NRE for type, theme, severity and likelihood of reoccurrence (i.e., likelihood that the same event would occur for another patient).

Results:

101 out of 145 ambulatory surgery cases (70%) contained at least one NRE. Overall, 214 NREs were reported – 88 by patients and 126 by clinicians. Cases containing clinician-reported NREs were associated with increased patient BMI (p=0.023) and lower post-case patient ratings of being treated with respect (p=0.032). Cases containing patient-reported NREs were associated with longer case duration (p=0.040), higher post-case clinician frustration ratings (p<0.001), higher ratings of patient stress (p=0.019) and lower patient ratings of their quality of life (p=0.010), of the quality of clinician teamwork (p=0.010), being treated with respect (p=0.003), and being listened to carefully (p=0.012). Trained patient raters evaluated NRE severity significantly higher than did clinician raters (p<0.001), while clinicians rated recurrence likelihood significantly higher than patients for both clinician (p=0.032) and patient-reported NREs (p=0.001).

Conclusions:

Both patients and clinicians readily report events during clinical care that they believe deviate from optimal care expectations. These two primary stakeholders in safe, high quality surgical care have different experiences and perspectives regarding NREs. The combination of patient- and clinician-reported NREs appears to be a promising patient-centered method of identifying healthcare system deficiencies and opportunities for improvement.

INTRODUCTION

There is substantial opportunity to improve healthcare quality and safety. Many believe that this could be achieved in part by making care more ‘patient-centered.’ However, care processes are currently designed to meet quality and safety targets set by clinicians and institutions, not by patients or their families. There are limited data on differing views of patients and families versus clinicians about care experiences [1-3], and the evidence is sparse about what constitutes suboptimal (low quality or unsafe) care processes from the patient’s perspective [4-6].

Prior studies showed that patients can discriminate clinical quality when reporting satisfaction with health care services [7, 8], can accurately identify adverse events in their care [9], and are interested in participating in quality improvement efforts [10]. Further, Weissman et al., [11] found that patients identified important adverse events not captured in the medical record. However, some previous studies found that clinician experts considered the majority of patient-reported events to be “misunderstandings” of appropriate care [12] or “service quality issues” [6]. A patient-centered care process must address such “misunderstandings” and embrace lay perspectives in operational definitions of safe practice.

We previously developed a novel method to assess system safety using the “non-routine event” (NRE) framework. A NRE is defined as any event that deviates from optimal or expected care for a specific patient in a specific clinical situation [7, 13-15]. Studies have found that clinician-reported NREs are: 1) frequent (≥1 NRE in 20-40% of all care periods studied); 2) reliably reported (based on observer validation and video recordings of care episodes); 3) associated with other patient outcome measures; 4) associated with important clinician performance-shaping factors like workload, distractions, and poor usability; and 5) captured a wide cross-section of system failures [16-22].

This is the first study to describe the incidence and nature of patient-reported NREs during ambulatory surgery concurrently with NREs reported by the clinicians caring for these patients. To further delineate differences between patients’ and clinicians’ perspectives of the care delivered, we trained both clinicians and patients to assess and code the reported NREs. While several studies have previously investigated patient reporting of adverse events [6, 9, 11] and safety concerns [23], this study is novel as we focus on non-routine events, which offer a broader perspective on potential system failures and opportunities for safety improvements.

We hypothesized that patients would report ample NREs, some of which would represent serious safety or quality issues within the surgical care process. Further, we hypothesized that there would be appreciable differences in the nature of NREs reported by patients as compared to their clinicians. Finally, we anticipated that trained patient raters would have a different perspectives concerning the severity of and likelihood of recurrence of the reported NREs than clinician raters.

METHODS

This study was approved by the Vanderbilt University Institutional Review Board prior to data collection.

Participants

We enrolled 145 adult patients who were scheduled to undergo elective surgery and be discharged from the Vanderbilt University Hospital (Nashville, TN) within 23 hours of completion of their surgical procedure. Patients who were unexpectedly admitted after ambulatory or ‘short stay’ surgery were retained in the study. We concurrently enrolled the clinicians participating in each study patient’s surgical care – the surgeons, anesthesia providers, OR nurses and technicians, and post-anesthesia nurses who took care of the study patient.

Materials

We collected data from clinicians using the Comprehensive Open-ended Non-routine Event Survey (CONES) [15], a standardized interview instrument used to identify and elaborate the nature of NREs. The patient and family version, PCONES was iteratively developed with patients using both patient partners (members of the research team) as well as based on focus groups with a diverse set of patients [24].

Procedure

We identified patients undergoing ambulatory/short stay surgery using the daily operating theatre schedule. A trained research assistant approached eligible patients for possible study inclusion in the pre-operative holding area. After obtaining written informed consent from each patient, their family members were also informed of the study and encouraged to participate. We interviewed patients (with family caregivers, if present) before they left the post-operative recovery area using the PCONES to elicit events which deviated from their perspective of an ideal or expected care experience. When possible, short-stay patients (i.e., 23 hour stay) were also seen near the time of discharge the following morning. We then conducted a follow-up PCONES interview at the post-operative clinic visit or by phone a week after discharge. Patient-reported NRE’s were reported by both patients, and their family members, with the patient’s permission. In parallel, we interviewed the study patient’s intra- and post-operative clinicians using the CONES, either in the operating theatre after skin closure or in the immediate post-operative period. All study data were deidentified and kept secure.

We obtained additional information from the respondents pre- and post-operatively. From hospital records, we obtained patient age, gender, weight and height (and thus BMI), and American Society of Anesthesiologists Physical Status Score [25, 26]. Participating patients provided information about their race, level of employment, level of education, and their health literacy using a single item question [27] (see Table 1). Participating clinicians provided their age, gender, length of clinical experience, estimated amount of sleep obtained the previous night, how sleepy or fatigued they felt in the moment (0-9 scale from ‘drowsy’ to ‘awake’) and self-rating of their mood on a 0-9 scale (bad to good).

Table 1.

Study patient characteristics

		Clinician-reported NREs			Patient-reported NREs
Patient Attribute	N*	No NRE (N=64)	≥ 1 NRE (N=81)	p-value	No NRE (N=88)	≥ 1 NRE (N=57)	p- value
Age (yrs)	137	58.8±13.6	57.6±14.0	0.624	59.7±13.3	55.6±14.3	0.107
Percent female	137	39% (24)	49% (37)	0.213	43% (36)	46% (25)	0.737
Weight (kg)	141	86.3±22.3	89.6±24.8	0.716	88.0±23.7	88.4±24.0	0.851
Height (cm)	120	174.7±10.1	170.8±9.8	0.037	173.3±9.9	171.4±10.4	0.287
BMI	120	28.5±6.6	31.5±8.2	0.023	29.7±6.6	30.8±9.0	0.482
Health Literacya	135			0.376			0.147
1: Not at all		2% (1)	8% (6)		6% (5)	4% (2)
2: A little bit		3% (2)	3% (2)		4% (3)	2% (1)
3: Somewhat		0	0		0	0
4: Quite a bit		18% (11)	20% (15)		25% (20)	11% (6)
5: Extremely		77% (47)	69% (51)		65% (53)	83% (45)
Education:	134			0.852			0.192
No diploma		5% (3)	4% (3)		7% (6)	0
High school or GED		27% (16)	30% (22)		25% (20)	33% (18)
Some college		20% (12)	24% (18)		23% (18)	22% (12)
College degree		48% (29)	42% (31)		45% (36)	45% (24)
Race	136			0.119			0.980
White, non-Hispanic		94% (58)	85% (63)		89% (73)	89% (48)
Other		6% (4)	15% (11)		11% (9)	11% (6)
Employment	137			0.581			>0.999
Employed		73% (45)	68% (51)		70% (58)	70% (38)
Unemployed		3% (2)	1% (1)		2% (2)	2% (1)
Retired/disabled		24% (15)	31% (23)		28% (23)	28% (15)
ASA Physical Status	136			0.429			0.116
1: A normal healthy patient		2% (1)	1% (1)		2% (2)	0
2: A patient with mild systemic disease		36% (21)	47% (36)		37% (30)	50% (27)
3: A patient with severe systemic disease		63% (37)	51% (39)		61% (50)	48% (26)
4: A patient with severe systemic disease that is a constant threat to life		0	1% (1)		0	2% (1)

^aHealth literacy question: “How confident are you filling out medical forms by yourself?”

^*N is the number of non-missing values

Data are presented as mean ± standard deviation or as a percent of total (number).

Comparisons that are significant at the P<0.05 level are in bold text.

Abbreviations: NRE=Non-Routine Event, BMI=Body Mass Index, ASA=American Society of Anesthesiologists.

Patient and clinician raters

We recruited two patients from the Vanderbilt University Medical Center Family Advisory Council, both with substantial prior experiences as patients and who were also active in promoting patient- and family-centered care across the Medical Center. These two patients and two anesthesiologists, who were not involved in any of the included surgical cases, were trained as NRE raters. These clinician and patient raters underwent rigorous training on the definition and verification of non-routine events as well as the types of NREs, the severity of patient injury, and the likelihood that the event would recur analogous to methods described in detail in prior work [19, 20]. Training used a random subset of the NRE data. All raters reviewed every event and followed a structured coding process. The two anesthesiologist raters coded each NRE independently and then met to discuss and resolve any disagreements [28]. The two patient raters followed the same process. If and when there was a disagreement between the pairs of raters (clinician or patient rater pairs), three project leaders (MBW, JS, SA – human factors researchers) served as arbiters to resolve any disagreements between them.

Statistical analysis

Descriptive statistics of select participant characteristics including NRE occurrence, type, severity and likelihood of reoccurrence were used to summarize the study sample. We calculated mean and standard deviation (SD) for continuous variables; median and interquartile range (IQR) were used in cases of large divergences from normally distributed data. Frequencies and proportions were used for categorical variables. Unadjusted comparisons across groups were performed using the Kruskal-Wallis test and Pearson chi-squared test. The Wilcoxon signed rank test was used to compare clinician and patient ratings of NRE severity. We used complete-case analyses for variables with missing observations. All analyses were conducted using R statistical software [29]. All statistical hypothesis tests were implemented at the conventional 5% type-I error rate, which corresponds to a p-value threshold of 0.05. No familywise hypotheses were specified or considered [30, 31]. Because of the descriptive exploratory nature of this study, we did not feel that multi-variate statistical analyses were appropriate.

RESULTS

We studied 145 ambulatory surgery patients. At least one NRE was reported in 101 of the study cases (70%) (Table 2). We captured a total of 126 clinician- and 88 patient-reported NREs. As more than one NRE could be reported per case, we found a total of 81 cases with clinician-reported NREs and 57 cases with patient-reported NREs. More than half (55.9%) of the cases contained at least one clinician-reported and 39.3% of the cases contained at least one patient-reported NRE. For the most part, patients and clinicians reported fundamentally different NREs – this dichotomy can best be seen in cases containing both patient and clinician NREs (see Table 6).

Table 2.

Study case characteristics

Case Attribute	N*	Clinician-reported NREs			Patient-reported NREs
		No NRE (N=64)	≥ 1 NRE (N=81)	p-value	No NRE (N=88)	≥ 1 NRE (N=57)	p-value
Case duration (min)	144	92.6±52.0	101.2±48.5	0.165	91.3±49.9	106.6±49.4	0.040
Type of Anesthesia	139			>0.999			0.276
General anesthesia		92% (58)	90% (68)		89% (76)	93% (50)
Monitored anesthesia care		8% (5)	9% (7)		11% (9)	5% (3)
Epidural		0	1% (1)		0	2% (1)
Primary Surgical Service	113			0.321			0.178
General		38% (18)	27% (18)		30% (21)	34% (15)
Orthopedic		2% (1)	2% (1)		1% (1)	2% (1)
Urologic		47% (22)	39% (26)		49% (34)	32% (14)
Ear, nose or throat		2% (1)	3% (2)		3% (2)	2% (1)
Colorectal		0	3% (2)		3% (2)	0
Endocrine		6% (3)	15% (10)		9% (6)	16% (7)
Eye		2% (1)	2% (1)		0	5% (2)
Gynecological		2% (1)	2% (1)		0	5% (2)
Plastics		0	8% (5)		4% (3)	5% (2)

^*N is the number of non-missing observations

Data are presented as mean ± standard deviation or percent of total (number).

Comparisons that are significant at the P<0.05 level are in bold text.

Abbreviations: NRE=Non-Routine Event

Table 6.

Representative sample of NREs during cases containing both a patient- and clinician-reported

Case Number	Clinician-reported NRE(s) [role of reporter]	Patient-reported NRE(s) [role of reporter]
53398	[Surgeon] Laparoscopic stapler misfired and cut patient. Replaced equipment and VERITAS report done.	[Patient] Post-operative nausea and vomiting; PACU nurse unprofessional behavior, including loud socializing and their use of social network sites. Nurses not responding to alarms. [Spouse] Updates were inconsistent from OR and then from PACU – Only 3 updates during 12 hour process; Delay in getting transferred to a room - other patients were seemingly "skipping in front of" her husband.
53445	[Anesthesiologist] Delay getting to OR because prior case ran over; [OR-RN] Delay getting to OR because hose was missing; [OR-RN] Intraoperative wait due to equipment malfunction, new scope components had to be autoclaved.	[Patient] Was not taught how to use the catheter and the various bags. He had to Google it. Bleeding following catheter removal, resulting in an ED visit
53479	[Surgeon] Difficult dissection resulting in temporary misidentification of the cystic duct; Too much chatter primarily by the scrub tech during stressful components of the procedure.	[Patient] Pt woke up with a [surgical drainage] tube which was unexpected; Patient had to share restroom with another patient during their overnight stay.
53572	[Anesthesiologist] Infusion pump malfunction at the beginning of the case.	[Patient] Poor directions and signage from parking lot caused them to wander around for more than 20 minutes; Anesthesiologist was felt to be very condescending (e.g., correcting the patient’s English); Anesthesiologist had hoarse voice and was clearly sick; Patient had a list of meds that he couldn't take because of intolerance, which was ignored by providers.
53615	[Anesthesiologist] Case delay due to previous case running over; [OR-RN] Surgeon planned to do open procedure but team had prepared for minimally invasive. Surgeon did not communicate plan to staff. [OR-RN] Came to work ill and has gotten worse throughout the day (“Once you come in, they won't let you go”).	[Patient] MRI contrast caused pain and tech was seemingly dismissive and did nothing to manage it. Patient described as psychological trauma and would not recommend this institution based on this event.
53625	[Surgeon] Has requested patient paralysis, but anesthesiologist put in LMA – Unparalyzed patient resulted in suboptimal surgical conditions.	[Patient] Pain postop much more than prior surgeries. Resolved with laxatives; Patient had driven himself to the hospital and was not able to drive himself home – Surgeon had to make up a reason to admit.
53675	[OR-RN] Machine failed during procedure resulting in 10 min case delay; [Anesthesiologist] Difficulty fitting BP cuff because of pt body habitus resulting in difficulty monitoring – Treated "low" BPs which may or may not have been real; Frustrated because a different attending anesthesiologist took over responsibility mid-case resulting in a change in the care plan.	[Patient] Rushed out of recovery; Vomited when sat up but handed clothing and left to dress herself while still groggy; Two care partners argued about who had to wheel patient out to the car; Near fall while getting into car due to excessive sedation.
53479	[OR-RN] Delay in case start due to: Holding room RN did not execute anesthesia orders, wrong bed in OR, appropriate supplies not available; OR music was loud and distracting; and Patient vomited on extubation.	[Patient] Rushed out of recovery room despite nausea; Felt he still had kidney stones though doctor claimed he removed them. Ended up passing some stones at home.

Tables 1-4 detail the characteristics for two categories of cases – those with and without clinician-reported NREs and those with and without patient-reported NREs. As can be seen in Table 1, the study patient population was characterized by being middle age, a majority male, well-educated, having high reported health literacy and a reasonable number of prior encounters with the healthcare system. Patients had significantly higher BMI and shorter height in cases with clinician NREs compared to no clinician NREs. All but six of the patients were classified as either ASA Physical Status 2 or 3. The preponderance of patients received general anesthesia for surgical procedures lasting about an hour and a half. The majority of cases were either general surgery (e.g., inguinal hernia repair) or involving the ear, nose or throat (ENT). Case duration was significantly longer in cases with patient-reported NREs compared to cases with no NREs reported by patients.

Table 4.

Post-case clinician and patient variables

		Clinician-reported NREs			Patient-reported NREs
	N*	No NRE (N=64)	≥ 1 NRE (N=81)	p-value	No NRE (N=88)	≥ 1 NRE (N=57)	p-value
Clinician
Mood in the moment (0-9)	37	2.1±0.3	2.2±0.5	0.389	2.2±0.5	2.1±0.3	0.749
Quality of teamwork	95	8.4±1.1	8.5±1.0	0.686	8.5±1.1	8.4±0.9	0.239
Clinician rating of patient QOL	94	7.4±1.6	7.6±1.6	0.477	7.4±1.8	7.6±1.4	0.880
NASA-TLX subscales
Performance	124	6.7±3.7 9 (6.5 – 9)	6.0±4.1 9 (0 – 9)	0.588	6.7±3.8 9 (6.25 – 9)	5.8±4.0 8 (0 – 9)	0.065
Frustration	95	1.0±2.4 0 (0 – 0)	1.3±2.1 0 (0 – 1.5)	0.102	0.6±1.8 0 (0 – 0)	1.9±2.5 1 (0 – 3.5)	<0.001
Patient
HCAHPS Question #1 – “How often were you treated with courtesy and respect” (n=131)				0.032			0.003
Always		100% (59)	90% (65)		99% (78)	89% (46)
Usually		0	8% (6)		0	11% (6)
Sometimes		0	1% (1)		1% (1)	0
HCAHPS Question #2 – “How often were you listed to carefully” (n=135)				0.457			0.012
Always		94% (58)	88% (64)		95% (78)	83% (44)
Usually		6% (4)	11% (8)		4% (3)	17% (9)
Sometimes		0	1% (1)		1% (1)	0
HCAHPS Question #3 – “Would you recommend this hospital to your friends and family” (n=134)				>0.999			0.252
Definitely yes		98% (61)	96% (69)		99% (81)	94% (49)
Probably yes		2% (1)	3% (2)		1% (1)	4% (2)
Probably no		0	1% (1)		0	2% (1)
Patient quality of life (0-9)	91	7.2±1.9	7.7±1.5	0.449	7.8±1.6	7.0±1.7	0.010
Quality of Clinician Teamwork (0-9)	96	8.8±0.7	8.5±1.4	0.094	8.8±0.6	8.3±1.6	0.010
Patient Stress (0-9)	96	1.3±2.7 0 (0 – 0)	1.0±2.1 0 (0 – 1)	0.446	0.7±1.9 0 (0 – 0)	1.8±2.9 0 (0 – 3)	0.019

^*N is the number of non-missing observations

Data presented as mean ± standard deviation. Categorical scores show median and (range).

Comparisons that are significant at the P<0.05 level are in bold text.

Abbreviations: NRE=Non-Routine Event, QOL=Quality of life, NASA-TLX=National Aeronautics and Space Administration Task Load Index, HCAHPS=Hospital Consumer Assessment of Healthcare Providers and Systems.

A majority of the clinicians who filled out CONES were nurses (circulating, scrub, or post-operative) with the next most common being anesthesia providers (Table 3). We did not identify any significant differences in clinical experience, clinicians’ prior sleep, or clinician post-mood score between cases with and without NREs reported.

Table 3.

Study clinician characteristics

Clinician attributes	N*	Clinician-reported NREs			Patient-reported NREs
		No NRE (N=64)	≥ 1 NRE (N=81)	p-value	No NRE (N=88)	≥ 1 NRE (N=57)	p-value
Age	129			0.515			0.205
20-29		79% (44)	69% (50)		75% (58)	69% (36)
30-39		12% (7)	15% (11)		14% (11)	13% (7)
40-49		7% (4)	10% (7)		9% (7)	8% (4)
50 and above		2% (1)	7% (5)		1% (1)	10% (5)
Gender (% female)	134	79% (48)	70% (51)	0.247	72% (58)	77% (41)	0.458
Clinical experience (months)	102	80.5±87.0 49.5 (25 – 81)	111.6±143.7 41.0 (17–150)	0.752	86.1±116.0 32.0 (15.3 – 108.0)	118.5±134.5 50.5 (27.0 – 174.5)	0.067
Number of consecutive days worked	134	2.3±1.7	2.5±1.6	0.337	2.3±1.3	2.7±2.1	0.725
Amount of sleep the prior night’s (hours)	134	6.4±1.4	6.6±1.5	0.275	6.6±1.4	6.4±1.5	0.753
Level of post-case drowsiness (0-9)	130	4.1±2.8	4.3±3.1	0.934	4.3±3.1	4.0±2.8	0.597
Type of clinician	134			0.390			0.435
Nurses		67% (41)	53% (39)		64% (52)	53% (28)
Anesthesia providers		25% (15)	29% (21)		22% (18)	34% (18)
Surgeons		8% (5)	18% (13)		14% (11)	13% (7)

^*N is the number of non-missing observations

Data are presented as mean ± standard deviation or a percent of total (number). Categorical scores show median and (range).

Comparisons that are significant at the P<0.05 level are in bold text.

Abbreviations: NRE=Non-Routine Event

Table 4 shows post-case clinician and patient variables. We found significantly higher (p<0.001) post-case clinician frustration in cases with patient-reported NREs than in cases with no patient-reported NREs. Both patient (p=0.003) and clinician (p=0.032) reported NREs were associated with a lower score on the question, “How often were you treated with courtesy and respect?”. Further, cases with patient-reported NREs were associated with higher ratings of patient stress and lower patient ratings of the quality of the clinicians’ teamwork, of the patients’ quality of life, and of patients being listened to carefully when compared to cases with no patient NREs.

Clinician and Patient Ratings of NREs

Ratings of 126 of the 132 clinician-reported NREs were completed by both clinician and patient raters. For patient-reported NREs, matched rating data were available for 88 of the 92 NREs. We identified significant differences in the classification and severity between trained clinician and patient raters of the NREs (Table 5). Clinician raters scored clinician and patient NREs as significantly less severe compared to the patient raters. While the patient raters deemed both patient- and clinician-reported NREs as significantly more severe, the patient investigators rated NREs as less likely to recur compared to the clinician raters. Clinicians and patients also categorized the NREs differently. Clinicians rated approximately one quarter of clinician- (29%) and patient- (25%) reported NREs as relating to health care process deficiencies. In contrast, patient raters deemed 58% of clinical-reported and only 16% of patient-reported NREs as falling in this focus group theme. Patient raters were able to assign all of the NREs to a theme group whereas clinician raters were unable to assign 38% of the clinician-reported NREs into any theme group. Thirty-eight percent of patient-reported NREs were classified in the communication of health information theme by patient-raters, whereas, clinician raters only classified 19% of the patient NREs as related to this theme. The most consistently rated (at least between raters) theme was ‘diagnostic and therapeutic issues’ with 14–17% of clinician-reported and 20–25% of patient-reported NREs categorized as such.

Table 5.

NRE severity and classification according to patient and clinician raters

NRE severity and classification	Clinician-reported NREs (N=126)			Patient-reported NREs (N=88)
	Clinician rated	Patient rated	p-valuea	Clinician rated	Patient rated	p-valuea
Severity of injury (9-point scale – see Appendix 1)	1.38±1.2	2.45±0.8	< 0.001	1.64±0.9	2.06±0.9	< 0.001
Likelihood of recurrence (1-5 scale – see Appendix 2)	3.02±0.5	2.87±0.8	0.032	3.05±0.6	2.77±0.6	0.001
Likely severity if recurred (1-5 scale – see Appendix 3)	1.68±0.7	2.05±0.7	< 0.001	1.84±0.7	1.97±0.6	0.166
Non-Routine Event Theme
	Clinician selected	Patient selected	P valueb	Clinician selected	Patient selected	P valueb
Diagnostic and therapeutic issues (e.g., unfamiliarity with the patient’s condition, mistakes and errors, diagnostic delays or misdiagnoses)	14% (17)	25% (32)	< 0.001	17% (15)	20% (18)	< 0.001
Health care process deficiencies (e.g., unexpected care, failure to get access, delays in treatment, care disruptions or variability)	29% (36)	58% (73)		25% (22)	16% (14)
Communication of health information (e.g., getting the wrong amount of information, or wrong content, or mistimed delivery)	6% (8)	4% (5)		19% (17)	38% (33)
Environment of care (e.g., available food choices, incorrect diet, cleanliness)	5% (6)	1% (1)		12% (11)	9% (8)
Staffing issues (e.g., too few nurses, adequately trained clinicians unavailable)	8% (10)	12% (15)		5% (4)	14% (12)
Patient-clinician relationship (e.g., dismissal of patient concerns, not talking with or listening to patients, being rude or inflexible)	1% (1)	0		9% (8)	2% (2)
No relevant focus group theme	38% (47)	0		12% (11)	1% (1)

^aWilcoxon signed rank test

^bPearson Chi-squared test

Data presented as mean ± standard deviation. Categorical scores show median and (range).

Comparisons that are significant at the P<0.05 level are in bold text.

Abbreviations: NRE=Non-Routine Event

Reported NREs were diverse and differed appreciably between those reported by clinicians and those reported by patients. Despite these apparent differences, for both groups, the NREs most commonly describe failures of process or of communication either between individuals or between an individual and technology or systems. A representative sample of cases, in which at least one patient- and one clinician-reported NRE were reported can be found in Table 6. As but a few illustrative examples of patient-reported NREs: Post-operative nausea and vomiting; delay in getting transferred to a room; nurses not responding to alarms; patient was not taught how to use the catheter - bleeding following catheter removal, resulting in an ED visit; pain postop much more than prior surgeries; and rushed out of recovery room despite nausea. Illustrative examples of clinician-reported NREs include: Laparoscopic stapler misfired and cut patient [surgeon reported]; intraoperative wait due to equipment malfunction [OR nurse]; infusion pump malfunction [anesthesiologist]; and wrong bed in OR and appropriate supplies unavailable [OR nurse]. The dichotomy between patient and clinician perspectives can be illustrated by case in which a patient had an intraoperative cardiac arrest during routine surgery. The problem was quickly resolved but the surgery was cancelled and the patient was monitored in the ICU over-night. An involved anesthesiologist, nurse, and surgeon all reported an NRE that was about the intraoperative cardiac event. When interviewed in the ICU the following morning prior to discharge home, the patient reported two NREs. The first NRE was that he had to stay overnight and he didn’t get his planned surgical procedure because “something happened to my heart”. The second NRE was that his ICU room’s door was squeaky and this precluded him from sleeping very much.

DISCUSSION

In this study, we describe the incidence and nature of patient- and clinician-reported NREs during adult ambulatory surgery. Among 145 ambulatory surgery cases, 101 (70%) contained at least one NRE; In total, there were 126 clinician-reported and 88 patient-reported NREs. The most common type of clinician-reported NRE was a process deficiency (often associated with a delay of care). The most common types of patient-reported NREs were related to failures of communication between the patient or their family and members of the clinical team. Despite similar patient, case, and clinician attributes, we found significantly higher post-case clinician frustration in cases with patient-reported NREs than in cases without NREs. Cases with patient-reported NREs had significantly higher ratings of patient stress and lower patient ratings of the quality of clinician teamwork, the patients’ quality of life, and patient ratings of being treated with respect and listened to carefully. Finally, trained independent patient and clinician raters had different perspectives on the nature and severity of the events. This research expands on previous NRE research by highlighting the ability of surgical patients to report NREs post-operatively, the differences between patient and clinician NREs, and the potential value of patient-reported NREs for identifying suboptimal care processes that may not be discerned using clinician-centered methods.

Patient NREs are frequent, meaningful, and distinct from clinician NREs

Numerous patient-reported NREs impacted the patient experience and care delivery. Some were clear safety issues while others might be considered by clinicians to be “service” issues. Regardless, events that patients deem as non-routine can have unanticipated effects on traditional care quality domains beyond just not being patient centered. Overall, patient-reported NREs were most frequently related to the themes: diagnostic and therapeutic issues, health care process deficiencies, and communication of health information. Despite extensive literature on how clinician-reported NREs can reflect low quality or unsafe care processes [7, 13, 14, 17, 19, 20], there is limited evidence on what patients and laypersons consider suboptimal care processes [4, 32]. Patients undergoing ambulatory surgery and the clinicians who care for them appear to inhabit two concurrent but largely distinct experiences (or perspectives). This is perhaps not surprising since the patient is generally under anesthesia during much of the time of the intraoperative clinicians’ experience of the encounter. In contrast, most patient experiences after discharge are largely unknown to their clinicians. The NREs reported by clinicians were most commonly related to process failures, whereas NREs reported by patients were most commonly related to communication failures. Patients may have more knowledge and a broader perspective on their entire care experience since the surgery itself represents only a part of longer patient health journey [33]. Therefore, patients bring a unique point of view of care processes and what constitutes ‘quality care’. As healthcare continues to move toward a more patient-centered approach, it is important to incorporate and act on all sources of data on the patient experience. This study extends previous research by capturing patient care experiences and perspectives that appear to delineate healthcare system weaknesses. Future research should further investigate NREs reported by patients and their lay caregivers in other care setting to identify care deviations and potential areas to improve patient care that will improve system safety and quality.

Patients have different perspectives than clinicians about NREs

We found that trained patients rated NRE severity significantly higher than clinicians for both patient and clinician-reported NREs. In contrast, trained clinicians rated the likelihood of NRE recurrence significantly higher than patients. This likely reflects differences in knowledge of context and of perspective. For instance, a wrist sprain from surgical positioning may not seem very severe to a clinician, however, this may be perceived as more severe to a patient who is a typist. Further, patient and clinician reviewers categorized the same NREs into different themes. For instance, patient raters more frequently attributed NREs to “communication of health information” and “staffing issues” compared to clinician raters. Our results suggest that patients and families have unique perspectives and information critical to safe, high-quality, patient-centered care that is not currently utilized or effectively shared with their clinical team. Thus, patient-reported NREs are a promising patient-centered method of identifying healthcare system deficiencies and opportunities for improvement. Further, eliciting patient feedback on clinician NREs could provide a unique perspective on the impact of suboptimal care processes on patients.

Limitations and future research

This study took place in a single ambulatory surgery facility in one academic medical center. Our patient population was primarily white and educated. While our study showed no difference between white and other races in patient-reporting of events, non-white clinicians reported relatively more events than whites (not statistically significant). Future research should investigate NREs experienced by a more diverse patient population across multiple institutions and domains. Despite robust rater training, the ratings could have been influenced by outcome and/or hindsight bias. The two independent trained rater pairs did not agree on their NRE ratings in some cases thus necessitating a consensus approach. The two rater pairs were able to reach consensus agreement in the vast majority cases; an investigator served as the arbiter in the others. It is likely that both patient- and clinician-reported NREs were under-reported – some study participants, particularly patients, may have been reluctant to share suboptimal aspects of the care delivered. Nevertheless, we still captured a significant number and wide diversity of NREs from both populations. Finally, we had some missing data due to participants not always completing all questions in the CONES and PCONES. However, we used complete-case analyses to account for this.

This work extends previous NRE research methods by including patients and families as sources of NRE reports. The results of this study can lead to several important research directions, including: (1) exploring NREs reported by patients and their family caregivers in other care settings, (2) examining clinicians’ acceptance of patient and caregiver observations and assessments related to care quality and safety; and (3) investigating the effects of individual- and group-level factors that facilitate, propagate and mitigate the effects of NREs. Elucidating relationships between NREs and patient outcomes will expand research opportunities to define, develop and test interventions to improve care. With NREs as an effective measure of care delivery system vulnerability, researchers, managers and policy makers will be better able to develop targeted interventions whose effectiveness can be more readily measured and addressed.

CONCLUSION

Patient- and clinician-reported NREs provide important and differing perspectives on suboptimal care deviations that occur during ambulatory surgery. Understanding NREs from the patients’ perspective can help identify unrecognized suboptimal processes and can inform how healthcare systems could become more patient-centered. The inclusion of patient NREs provides additional breadth and depth about non-routine events that can be a useful source of information for care quality and safety efforts. Future research should leverage these methods to identify valuable information to help systems develop improved patient-centered care.

Appendix 1: Severity scale

Code #	Category	Examples
0	No injury
1	Temporary emotional injury	Transient stress/frustration; transient grieving
2	Temporary insignificant injury	Loss of toenail; facial abrasion; discharge delay after elective hospitalization;
3	Temporary minor injury	Persistent post-operative nausea/vomiting requiring hospital admission; bruise from femoral bleed; statin-induced muscle cramps;
4	Temporary major injury	Sepsis/ARDS without any permanent injuries; Ruptured appendicitis with successful treatment; Myocardial infarction with timely re-vascularization (and no permanent myocardial muscle damage);
5	Permanent minor injury	Loss of range of motion of finger joint; peripheral neuropathy; chronic chest pain (angina) with significant exertion;
6	Permanent significant injury	Neurogenic bladder; chronic angina with minimal exertion;
7	Permanent major injury	End-stage renal disease; chronic respiratory compromise that severely limits quality of life (e.g., can’t perform ADLs without dyspnea)
8	Permanent grave injury	Life debilitating injuries including significant brain damage; quadriplegia;
9	Death

Appendix 2: Likelihood of recurrence scale

Code #	Severity	Description
1	Negligible	The severity of the event results in extra steps or slight delays that will not affect proper usage.
2	Mild	The severity of the event results in minor system performance changes. This might cause nuisance alarms, require additional process steps, or increased delay. The results could affect proper usage.
3	Moderate	The severity of the event could result in non-serious injury that does not require intervention or major system performance changes.
4	Serious	The severity of the event could result in a non-serious injury that requires intervention.
5	Catastrophic	The severity of the event could result in death or serious injury.

Appendix 3: Likely severity if recurrence scale

Code #	Qualitative Likelihood	Description	Prob.
1	Improbable	So unlikely, it can be assumed that occurrence will not be experienced.	<0.01%
2	Remote	Unlikely, but may occur during the life of the product	0.1-5%
3	Occasional	Likely to occur in the life of the product	5-10%
4	Probable	Will occur a number of times in the life of the product	10-30%
5	Frequent	Likely to occur routinely during regular use	>30%