The Accuracy of Surgeons' Provided Estimates for the Duration of Hysterectomies: A Pilot Study

Abstract

Study Objective

To determine the accuracy of gynecologic surgeons' estimate of operative times for hysterectomies and to compare these with the existing computer-generated estimate at our institution.

Design

Pilot prospective cohort study (Canadian Task Force classification II-2).

Setting

Academic tertiary women's hospital in the Northeast United States.

Participants

Thirty gynecologic surgeons including 23 general gynecologists, 4 gynecologic oncologists, and 3 urogynecologists.

Intervention

Via a 6-question survey, surgeons were asked to predict the operative time for a hysterectomy they were about to perform. The surgeons' predictions were then compared with the time predicted by the scheduling system at our institution and with the actual operative time, to determine accuracy and differences between actual and predicted times. Patient and surgery data were collected to perform a secondary analysis to determine factors that may have significantly affected the prediction.

Measurements and Main Results

Of 75 hysterectomies analyzed, 36 were performed abdominally, 18 vaginally, and 21 laparoscopically. Accuracy was established if the actual procedure time was within the 15-minute increment predicted by either the surgeons or the scheduling system. The surgeons accurately predicted the duration of 20 hysterectomies (26.7%), whereas the accuracy of the scheduling system was only 9.3%. The scheduling system accuracy was significantly less precise than the surgeons, primarily due to overestimation (p = .01); operative time was overestimated on average 34 minutes. The scheduling system overestimated the time required to a greater extent than the surgeons for nearly all data examined, including patient body mass index, surgical approach, indication for surgery, surgeon experience, uterine size, and previous abdominal surgery.

Conclusion

Although surgeons' accuracy in predicting operative time was poor, it was significantly better than that of the computerized scheduling system, which was more likely to overestimate operative time. Journal of Minimally Invasive Gynecology (2015) 22, 57-65 © 2015 AAGL. All rights reserved.

Accurate estimation of operative time is integral to efficient scheduling of the operating room (OR). Overestimation of surgical time leads to underutilization of operating rooms, resulting in fewer scheduled procedures on any given day. Alternatively, underestimation of surgical time may lead to unplanned overtime and potential cancellation or delay of procedures. Therefore, inaccurate scheduling of elective operations can lead to both increased costs and suboptimal use of the operating room and to patient and surgeon/staff dissatisfaction.

Hysterectomy is the most frequently performed major gynecologic procedure in the United States [1]. The surgical approach (laparotomy, vaginal, or laparoscopic) is influenced according to patient characteristics, surgical history, and surgeon preference. Factors that may influence the level of difficulty of hysterectomy include uterine dimensions, patient body mass index, previous abdominal surgery, and surgeon experience with the planned approach. Gynecologic surgeons seem to be able to estimate the level of difficulty of a hysterectomy in a large percentage of cases [2]. Currently, little is known about gynecologists' ability to accurately predict operative time.

When constructing the operating room schedule, many institutions use a variant of block scheduling in which blocks of time are reserved for individual surgeons or groups until a deadline is reached, at which time the remaining time becomes available to other surgeons. However, there does not seem to be a preferred method to accurately use the available block time, and some institutions use commercial scheduling software whereas others use only historical data to generate time estimates [3]. At our institution, which is a tertiary care hospital for women only, the operative time assigned for hysterectomy is based on a computer model that examines the last 10 similar procedures performed by the individual surgeon. After excluding the longest and shortest procedures, the average operating room time of the remaining 8 procedures is used to predict the anticipated duration of the surgery being scheduled, and operating room time is allocated accordingly. However, this model does not take into consideration the various factors that may make the procedure more or less difficult and thus affect the duration. Furthermore, reports in the literature have suggested that relying solely on historical times is likely an ineffective strategy to predict the duration of future procedures [4]. A simple prediction model that combines historical times with surgeons' mean estimates seems to yield a good measure of future operative times [5].

Surgeons' predictions have been taken into consideration in developing prediction models for operating room times [3,6,7]. However, none of the currently available data specifically describe the accuracy of gynecologic surgeons when predicting operative time for hysterectomies. The primary objective of the present study was to determine the accuracy of surgeons' predictions when estimating operative times for hysterectomies compared with the current scheduling algorithm at our institution. The secondary objective was to determine patient and surgery characteristics that may contribute to the accuracy, or lack thereof, of predictions by the surgeons and the scheduling system.

Material and Methods

We conducted a pilot cohort study at a tertiary care hospital for women only in the Northeast United States. Within the hospital, 8 operating rooms are dedicated to gynecologic procedures and breast surgery performed by both gynecologic generalists and subspecialists. After obtaining approval from the institutional review board at Women and Infants Hospital, a 6-question survey was provided daily to all gynecologic surgeons performing a non-emergent hysterectomy that day. Robotic-assisted hysterectomies were excluded from the analysis primarily because this technology was new to our institution at the time. Consequently, the surgeons were still in the steep portion of the learning curve, which would likely render their predictions less accurate, along with potential greater variability in actual procedure duration. In addition, the scheduling software did not include enough cases to make a prediction for robotic operative time in the usual manner.

The survey assessed the surgical approach, indication for the procedure, any additional procedures that would be performed concomitantly, the number of years the provider had been practicing, and the prediction of the duration of the procedure (Fig. 1). The surgeons were asked to estimate operative times in a range of ,1 hour to .3 hours, in 15-minute intervals. The time frame specified was skin incision to wound closure because this is the time used in the scheduling system. The number of surveys per surgeon was limited to 4 to prevent any individual surgeon from skewing the results.

Fig. 1.

Survey administered to participating gynecologic surgeons. LASH 5 laparoscopic supracervical hysterectomy; LAVH 5 laparoscopically assisted vaginal hysterectomy; TAH 5 total abdominal hysterectomy; TLH 5 total laparoscopic hysterectomy; TVH 5 total vaginal hysterectomy.

After each survey was collected, the patient's medical record was reviewed to obtain demographic data, medical and surgical history, body mass index, and uterine size (estimated and actual weight from the pathology report). The operative report and intraoperative nursing record were also reviewed to determine the actual operative time, preoperative and postoperative diagnosis, procedure actually performed, and complications.

For comparison between the surgeons' and the scheduling system prediction we used actual surgery duration in minutes as the gold standard. Minutes were imputed from the surgeons' categorical estimates by taking the upper boundary of the corresponding 15-minute interval. The absolute values of the differences were compared, and for comparability the scheduling system estimate and the actual operative time were rounded to the closest 15-minute interval (,8 minutes difference rounded down, ≥ 8 minutes difference rounded up). To assess factors that could potentially cause the estimates to be inaccurate, a secondary analysis was performed of each procedure and surgeon characteristics and calculated whether and by how much each factor had affected the surgeons' and the scheduling system predictions. Factors evaluated included patient body mass index, previous abdominal surgery, indication for hysterectomy, procedure performed, surgeon experience, and uterine size.

Our hypothesis was that the surgeons' prediction of operative time would be more accurate than the scheduling system estimate. Accuracy was established if the actual procedure time was within the 15-minute increment predicted by either the surgeon or the scheduling system. On the basis of our hypothesis, we designed the study as a superiority trial.

Statistical analysis was performed using commercially available software (STATA version 10; StataCorp., College Station, TX). For our sample size calculation, we set the effect size at an absolute difference of 30 minutes between the surgeons' and the scheduling system prediction. Using a power of 80%, 5% significance level, paired t-test, and within-case standard deviation of 40 minutes, we calculated a sample size of 32 surveys, accounting for approximately 3 surveys per surgeon (design effect of 2). The sample size calculation was based on the number of procedures needed to make comparisons between operative time within each case as opposed to comparisons between or within unique surgeons. That is, the focus for sample size estimation was at the procedure level, with adjustment for dependent errors resulting from within-surgeon correlation. To address this objective, we obtained the sample size required for a paired t-test and inflated this number to account for any potential loss of statistical power due to within-surgeon clustering of estimates. Categorical variables were compared using the Fisher exact test. Continuous variables were compared using the t-test, analysis of variance, Wilcoxon rank-sum test, or Kruskal-Wallis test. Operative time estimates were compared using the paired t-test or McNemar test. A secondary underpowered analysis was performed using multiple linear regression to estimate the effect of characteristics on prediction accuracy (actual minus predicted minutes). Outcome variables and model residuals were checked for deviations in normality. Robust variances were used to assess the influence of within-surgeon correlation when appropriate. Only p , .05 via 2-tailed analysis was considered statistically significant.

Results

Responses were obtained from 30 of 32 surgeons (94%) who received a survey. The group was composed of 23 general gynecologists, 4 gynecologic oncologists, and 3 urogynecologists. Each provider completed 1 to 4 surveys, for a total of 75 surveys. Demographic and patient data are given in Table 1. Surgical approach did not differ according to age, race/ethnicity, body mass index, or surgical history.

Table 1. Baseline patient and procedure characteristics^a.

Variable	Type of hysterectomy
	Total	TAH	TVH	LSC	p value
Total	75	36 (48.0)	18 (24.0)	21 (28.0)
Race/ethnicity
White	66 (88.0)	31 (86.1)	15 (83.3)	20 (95.2)	.46
African American	2 (2.7)	0	1 (5.6)	1 (4.8)
Hispanic	4 (5.3)	2 (5.6)	2 (11.1)	0
Asian	1 (1.3)	1 (2.8)	0	0
Other	2 (2.7)	2 (5.6)	0	0
Age, yr
Mean (SD)	51.7 (11.4)	51.8 (9.7)	54.2 (14.2)	49.3 (11.5)	.42
Median (range)	49 (26–84)	50 (38–81)	56.5 (26–78)	46 (33–84)
Body mass index
<30	42 (56.0)	19 (52.8)	10 (55.6)	13 (61.9)	0.79
≥30	33 (44.0)	17 (47.2)	8 (44.4)	8 (38.1)
Previous abdominal surgery
Yes	42 (56.0)	19 (52.8)	11 (61.1)	12 (57.1)	.87
No	33 (44.0)	17 (47.2)	7 (38.9)	9 (42.9)
Type of previous surgery
Cesarean section	16 (21.3)	8 (22.2)	3 (16.7)	5 (23.8)	.87
Bowel	5 (6.7)	3 (8.3)	1 (5.6)	1 (4.8)	1.00
Adnexal	13 (17.3)	6 (16.7)	3 (16.7)	4 (19.1)	1.00
Other	12 (16.0)	4 (11.1)	5 (27.8)	3 (14.3)	.28
Indication for surgery
Pain	21 (28.0)	12 (33.3)	2 (11.1)	7 (33.3)	.18
Myomas	28 (37.3)	21 (58.3)	3 (16.7)	4 (19.1)	.001
Abnormal uterine bleeding	33 (44.0)	18 (50.0)	4 (22.2)	11 (52.4)	.11
Prolapse	15 (20.0)	0	13 (72.2)	2 (9.5)	<.001
Precancer/cancer	17 (22.7)	10 (27.8)	1 (5.6)	6 (28.6)	.13
Other	10 (13.3)	4 (11.1)	2 (11.1)	4 (19.1)	.68
Additional Procedures
Bilateral salpingo-oophorectomy	34 (45.3)	18 (50.0)	2 (11.1)	14 (66.7)	.001
Unilateral salpingo-oophorectomy	3 (4.0)	2 (5.6)	1 (5.6)	0	.60
Sling	7 (9.3)	0	5 (27.8)	2 (9.5)	.002
Staging/debulking	6 (8.0)	5 (13.9)	0	1 (4.8)	.26
Cystoscopy	31 (41.3)	5 (13.9)	13 (72.2)	13 (61.9)	<.001
Colporrhaphy	11 (14.7)	0	9 (50.0)	2 (9.5)	<.001
Other	10 (13.3)	5 (13.9)	3 (16.7)	2 (9.5)	.91
Uterine weight, mean (SD)	195.1 (188.0)	288 (226.6)	85.5 (61.4)	138.0 (103.7)	.001
Median (range)	130 (27–926)	207 (49–926)	63 (27–237)	103 (33–419)	.001
No. of uteri <250 g	54 (76.1)	19 (57.6)	17 (100)	18 (85.7)
No. of uteri ≥250 g	17 (23.9)	14 (42.4)	0	3 (14.3)
Attending physician experience, yr
0–5	5 (6.7)	1 (2.8)	2 (11.1)	2 (9.5)	<.001
6–10	8 (10.7)	2 (5.6)	4 (22.2)	2 (9.5)
11–20	33 (44.0)	10 (27.8)	7 (38.9)	16 (76.2)
21–30	19 (25.3)	17 (47.2)	1 (5.6)	1 (4.8)
31–40	10 (13.3)	6 (16.7)	4 (22.2)	0

LSC = laparoscopic supracervical hysterectomy; TAH = total abdominal hysterectomy; TVH = total vaginal hysterectomy.

^aData are given as number of patients (%).

Abdominal hysterectomy was the most common surgical approach. For hysterectomies to treat myomas, an overwhelming number were performed via laparotomy (p = .001), whereas most procedures to treat prolapse were performed via the vaginal approach (p < .001). Larger uteri were more likely to be removed abdominally, and most abdominal hysterectomies were performed by attending physicians who had been in practice >20 years.

Surgeons were able to accurately predict the duration of 20 procedures (26.7%) (Table 2). Accuracy was consistent regardless of the route in which the hysterectomy was performed. In comparison, the scheduling system was able to accurately predict the duration of only 7 procedures (9.3%). The difference in accuracy between the surgeons and the scheduling system was statistically significant (p = .01, McNemar test). Although surgeons were more likely to underestimate in their prediction, on average, both the scheduling system and surgeons overestimated the amount of time it would take to complete the procedures included in the study. The surgeons overestimated the duration of the procedure on average by 10.2 minutes, whereas the scheduling system tended to overestimate on average by 34.4 minutes. These differences remained statistically significant after accounting for within-surgeon correlation. The scheduling system significantly overestimated time required to a greater extent than did the surgeons for nearly all data examined (Table 3).

Table 2. Estimated accuracy and time difference between actual and predicted (surgeon and scheduling system) operative time^a.

Variable	Type of hysterectomy
	Total	TAH	TVH	LSC
Actual vs surgeon predicted
Mean (SD)	–10.2 (30.3)	–8.5 (27.4)	–18.7 (34.5)	–5.9 (31.2)
Median (range)	–10 (–91 to 60)	–11 (–66 to 59)	–21.5 (–91 to 33)	–2 (–88 to 60)
Actual vs scheduling system
Mean (SD)	–34.4 (41.5)	–31.1 (34.6)	–36.1 (52.0)	–38.5 (43.9)
Median (range)	–34 (–166 to 63)	–29.5 (–101 to 49)	–31.5 (–166 to 38)	–49 (–114 to 63)
Estimated accuracy
Surgeon
Accurate estimation	20 (26.7)	10 (27.8)	5 (27.8)	5 (23.8)
Underestimation	30 (40.0)	15 (41.7)	6 (33.3)	9 (42.9)
Overestimation	25 (33.3)	11 (30.6)	7 (38.9)	7 (33.3)
Scheduling system
Accurate estimation	7 (9.3)	3 (8.3)	3 (16.7)	1 (4.8)
Underestimation	19 (25.3)	9 (25.0)	6 (33.3)	4 (19.1)
Overestimation	49 (65.3)	24 (66.7)	9 (50.0)	16 (76.2)

LSC = laparoscopic supracervical hysterectomy; TAH = total abdominal hysterectomy; TVH = total vaginal hysterectomy.

^aData are given as number of patients (%).

Table 3. Significant time estimate differences between surgeons and scheduling system according to specific characteristics^a.

Variable	Actual time, min		Mean difference actual estimated		p value
	Mean	Median (range)	OR scheduling system	Surgeons' prediction
Body mass index
<30	104.3	95.5 (39–206)	–40.8	–10.7	<.001
>30	120	127 (49–206)	–26.2	–9.5	.01
Indication for surgery
Cancer	99.2	89 (63–181)	–51.1	–22.8	.009
Multiple	115.1	111 (55–206)	–28.4	–5.7	.001
Type of hysterectomy
TAH	103.6	98.5 (39–185)	–31.1	–8.5	.001
LSC	122	114 (75–181)	–38.5	–5.9	.003
Other procedure
None	98.1	99 (49–180)	–32.3	–1.0	.001
BSO	112.7	106 (39–185)	–35.2	–11.7	.002
USO	114	123 (80–139)	–9.0	29.0	.02
Surgeon experience, yr
11–20	106.2	110 (39–181)	–36.8	–9.2	.003
21–30	105.6	93 (60–206)	–32.5	–7.3	.004
31–40	94.3	100 (51–134)	–36.0	4.3	.002
Uterine weight, g
<250	109.7	102 (39–206)	–38.0	–8.7	<.001
Previous abdominal surgery
None	107.2	100 (51–206)	–45.9	–20.1	<.001
Adnexal	119.5	110 (60–180)	–22.4	3.0	.02
Other procedureb	101.9	89.5 (39–206)	–43.2	–15.6	.01

BSO = bilateral salpingo-oophorectomy; LSC = laparoscopic hysterectomy; OR = operating room; TAH = total abdominal hysterectomy; USO = unilateral salpingo-oo-phorectomy.

^aNegative values and positive values represent minutes of overestimation and underestimation, respectively.

^bOther procedures included cholecystectomy and hernia repair.

Factors that may have contributed to inaccurate estimations are shown in Figure 2. The scheduling system significantly overestimated the time needed for patients who had not undergone previous abdominal surgery compared with those who had previously delivered via cesarean section (–45.9 minutes vs –7.6 minutes; p = .002). The degree of inaccuracy for the scheduling system did not differ significantly by other case characteristics. Surgeons overestimated time by 23 to 32 minutes for prolapse and precancer/cancer indications, and these differences were significantly different from bleeding indications (+16.4 minutes; p < .02). Time to perform a concurrent unilateral salpingo-oophorectomy was significantly underestimated by the surgeons (+29.0 minutes; p = .047), whereas time for concurrent debulking/staging procedures was overestimated (–26.0 minutes; p = .03). Surgeons also overestimated time for patients who had not undergone previous abdominal surgery (–20 minutes), compared with predictions within 5 minutes of the actual time for previous cesarean section, bowel surgery, or adnexal surgery (p < .04). These factors influencing surgeons' predictions were examined via multiple linear regression, and none remained significantly associated with surgeons' accuracy.

Fig. 2.

Prediction accuracy by single factors. Markers indicate mean difference for actual procedure time vs each prediction method, and error bars indicate corresponding 95% confidence intervals. Error bars that do not cross zero (accurate estimation) indicate statistically significant overestimation (negative values) or underestimation (positive values). *The 95% confidence interval for other indication was wide and is not shown. BMI = body mass index; BSO = bilateral salpingo-oophorectomy; LSC = laparoscopic supracervical hysterectomy; TAH = total abdominal hysterectomy; TVH = total vaginal hysterectomy; USO = unilateral salpingo-oophorectomy.

Discussion

The present study shows a suboptimal model in place for operating room scheduling accuracy at our institution, with only 9.3% of hysterectomy procedures scheduled accurately. This translates into an average overestimation of 34.4 minutes, which could result in an additional surgical procedure per day. The currently used scheduling system is limited because it ignores some important factors such as indication for the procedure, patient body mass index, surgical history, and surgeon experience. However, in our secondary analysis none of those factors individually seem to have had a significant effect on prediction accuracy. The literature suggests that even when more procedures are used to make a prediction from historical times, accuracy improves only marginally, by 7 minutes, from using just 1 to 9 previous procedures, at which point the differential between actual and predicted time reaches a plateau [4]. To our knowledge, the present study is the first to specifically examine prediction time for hysterectomies, and it adds to the available evidence that the use of historical operative times is an ineffective way to predict duration of future procedures.

In the secondary analysis, it was observed that the route in which the hysterectomy was performed did not affect accuracy by either the scheduling system or the surgeons. However, when compared with the surgeons' predictions, the scheduling system significantly overestimated the duration of abdominal hysterectomies (31.1 minutes vs 8.5 minutes; p = .001) and laparoscopic hysterectomies (38.5 minutes vs 5.9 minutes; p = .003). In studies that examined operative prediction accuracy, estimated duration of time for laparoscopic procedures has been shown to vary as much as 42% from actual operative time [8]. Because of this wide margin of variability, some authors argue that using predictions to estimate laparoscopic operative time is associated with an inherent degree of inaccuracy [9]. However, in the present study the surgical approach did not seem to affect the scheduling system or the surgeons' estimates.

The higher accuracy of the surgeons was likely secondary to their knowledge and incorporation of relevant patient characteristics and the level of difficulty of the procedure. However, as a pilot study, this trial was powered only to detect differences in operative time within each procedure and not predictive factors of operative time. Therefore, we were not able to find any individual factors significantly associated with surgeon accuracy in our secondary analysis. Yet, our results show that surgeons were significantly better predictors than the scheduling system and had a tendency to overestimate to a lesser extent. This is likely because the factors that may affect accuracy are not constant but are fluid variables that may affect the prediction independently or in combination with one another. This kind of variables make computer models difficult to design and are not taken into account during the scheduling process at our institution, whereas surgeons may be able to account for enough of these variables to be more accurate in their predictions, as our results suggest. Therefore, although the present study did not necessarily have statistical power to examine the secondary outcomes, our results should prove useful for designing future studies adequately powered to examine predictive factors of operative time.

Timeliness in the operating room is important not only for a hospital's running costs and revenues but also to staff, surgeons, and patients. The goal of any scheduling system is to avert underuse and overuse, to improve the efficiency of the operating room. Excluding physician costs, administrators may quote operating room costs at $15 to $20 per minute for a basic procedure [10]. Operating room costs can be as high as $29 per minute for procedures of low complexity and as high as $80 per minute for highly complex procedures [11]. On the basis of our findings and using basic procedure cost estimates, our institution may be losing $516 to $688 per procedure, assuming the overestimated time could be used to schedule at least 1 additional procedure. If we consider only the 75 procedures used in our analysis, the result would be $38 700 to $51 600 in losses. Using the numbers from the surgeons' predictions, there would still be monetary losses. However, making the same assumptions, the losses would be in the range of $153 to $204 per procedure, or $11 475 to $15 300, which is still a savings of $363 to $484 per procedure.

There is a growing body of literature on efficient use of the operating room [3,12–14]. However, to date no studies have addressed primarily gynecologic surgery. A strength of the present study is that it addresses this void in the literature. It also includes a wide variety of cases including benign, urogynecologic, and oncologic indications for hysterectomy. Limitations of the study include that it was performed in a single dedicated women's hospital, in which most surgical procedures performed are gynecologic. Therefore, although many gynecologic services have their own operating rooms and/or operating room teams, our results may not be easily extrapolated to more traditional hospitals that encompass all types of patients and surgical services. In addition, because almost half of the surgical procedures included in our study were performed via laparotomy, it may lack some generalizability given that the rate of minimally invasive hysterectomy is increasing. Furthermore, we recognize that surgeons may be biased toward significantly underestimating operative time to schedule as many procedures as possible into their block time, which may account for the time difference in estimation between the surgeons and the scheduling system. Our results may also have been influenced by the observer effect (Hawthorne effect). Surgeons have control of the duration of the procedure, and it is possible that the hysterectomies under study do not reflect the surgeons' usual practices and differ from historical predictions for that reason.

In conclusion, the objective of the present study was to determine whether there is a more accurate way to predict operative time and improve operating room efficiency. We found that although surgeons' accuracy in predicting operative time was poor, at 27%, it was significantly better than the accuracy of the scheduling system, which was only 9%. Both the surgeons and the scheduling system were likely to overestimate time; however, the scheduling system did so by 34 minutes as opposed to 10 minutes by the surgeons, which translates into a 70% reduction. With use of statistical models that incorporate surgeons' predictions, operative time was underestimated by 12%, and overestimated by 25% [6]. Prediction of operative time is only one of the many ways in which operating room efficiency can be improved; nonetheless, it is the way that surgeons can control and help predict. We were not able to identify any single factor associated with improved accuracy; however, our study was not powered to examine predictive factors. There is substantial complexity associated with accurate prediction of operative time, and accuracy is likely associated not with one but with a combination of multiple factors, only a few of which were analyzed in this pilot study. Further research and multilevel statistical modeling are required to determine which factors and by how much they contribute to accurate time estimation. Ultimately, our goal would be to develop a prediction model that incorporates not only baseline patient and procedure characteristics but also historical times and surgeons' predictions. We believe this model would be more cost-effective for the hospital and would also increase staff, surgeon, and patient satisfaction.