Abstract
Introduction:
Clinical alert systems (CAS) have been used to analyze deviations from hospital standards in the electronic medical record to identify missing documentations and send alerts to the appropriate providers to increase adherence to required elements. To improve compliance, an alert system for documentation of the Immediate Preoperative Assessment (IPOA) was implemented at our institution in August 2018 with the goal of improving documentation compliance rates. We hypothesized that implementation of this alert system would increase the compliance of on-time documentation of the IPOA.
Methods:
An initial data query in our institutional data warehouse was made for all patients who had a completed anesthetic during our study period. This date range corresponded to 6 months before and after August 2nd, 2018, the date when the IPOA alert was implemented and the anesthesiology department. The following analyses were performed: testing the proportion of cases compliant with on-time documentation of the IPOA pre- versus post-implementation for the full cohort and among subsets of interest, testing the time when the IPOA was completed relative to anesthesia end, and testing whether time of day of when surgery occurred had an impact on the time when the IPOA was completed relative to the drapes off/IPOA alert sent time. The proportion of compliance for pre- versus post-implementation was tested by Chi-square test.
Results:
Through retrospective chart review of electronic patient records, 47,417 cases matched our inclusion criteria of patients that had a completed anesthetic between February 2nd, 2018 to February 2nd, 2019. In total, we excluded 5132 cases. The compliance rate of IPOA completion increased from 76% to 88% (P < 0.001) before and after the alert implementation date. In the initial month following alert implementation, the compliance rate immediately increased to 83% and stayed in the high 80’s for the balance of the study period.
Conclusion:
In summary, we demonstrate that automated Clinical Alert Systems operating via a single page notification can improve the compliance rate for documentation of key anesthesia events and that this observation is sustained six months after the implementation date. Furthermore, improvement in compliance is highest shorter cases and cases that occur early in the day. This study shows promising results in the use of automatic CAS system alerts to help hospitals meet the Center for Medicare and Medicaid Services (CMS) and The Joint Commission (TJC) standards.
Keywords: real-time clinical alerts, anesthesia documentation, alert notification, compliance, clinical alert system
1. INTRODUCTION
Institutions must comply with documentation standards set by the Center for Medicare and Medicaid Services (CMS) and The Joint Commission (TJC) to guarantee payor reimbursements for anesthesia services.1,2 Such documentation often involves recordings of clinical events at specific times and attestations affirming that a particular assessment or procedure is performed.3 For instance, all anesthetics require an immediate pre-operative assessment (IPOA) by a qualified anesthesiologist or anesthesia care provider. The importance of the pre-anesthesia evaluation is noted in TJC’s accreditation requirement PC.03.01.03, Element of Performance #8, which states that “The hospital reevaluate the patient immediately before administering moderate or deep sedation or anesthesia.”4 This standard also references the CMS regulation governing anesthesia services.5 Overall, this reevaluation of the patient’s health status includes a final review of the patient’s medical chart, whether the patient has ingested any food or liquid, the type of anesthesia to be delivered, whether the surgery is an emergency, and the patient’s American Society of Anesthesiologist (ASA) physical status score.
However, adherence to the required elements of documentation of the anesthesia electronic record to meet CMS and TJC standards can be especially challenging in the perioperative space.6–8 At our institution, documentation of the IPOA in our electronic medical record must be completed for every case by an anesthesiologist at any time after the patient enters the operating room and before their transfer of care to the post-anesthesia care unit (PACU) (Fig. 1). In figure 1, these two time points are noted as “Anesthesia Start (Patient walks in)” and “Anesthesia End/PACU handoff”. IPOA documentation performed before “Anesthesia Start” or after “Anesthesia End” is considered non-compliant. Given the number of competing clinical, teaching, and administrative obligations anesthesiologists have during procedures, ensuring compliance within this time-frame has proven to be a stubborn problem.
Fig. 1.
Timeline of the intra-operative time frame. An alert is sent if the immediate preoperative assessment (IPOA) has not been documented at the “surgery end/drapes off” time. Documentation done before “Anesthesia start” and after “Anesthesia End/PACU handoff” is non-compliant.
Clinical alert systems have been used to analyze deviations from hospital standards in the electronic medical record to identify missing documentations and send alerts to the appropriate providers to increase adherence to required elements.9–13 To improve compliance, an alert system for documentation of the IPOA was implemented at our institution in August 2018 with the goal of improving documentation compliance rates. If the clinical alert system detected a missing IPOA at the time when the surgery has ended (defined as the time when the drapes were removed from the patient), an alert in the form of a reminder text page was sent to the anesthesiologist (Fig. 1) This time point is denoted as Surgery End/Drapes Off” in Figure 1. The alert was placed at the conclusion of surgery to give the anesthesiologist ample time to complete the IPOA prior to “Anesthesia End/PACU handoff”, without being sent too early in the intraoperative timeline. It was designed to serve as a final reminder before the documentation would be deemed non-compliant. We hypothesized that implementation of this alert system would increase the compliance of on-time documentation of the IPOA. As a secondary analysis, our study examined if the compliance rate was sustained across the 6 months following alert implementation, how the alert affected the timing of IPOA completion, and whether there were differences in compliance rates pre- vs post-implementation across categories of case duration, patient age, and hospital service.
2. METHODS
2.1. Data Collection
Institutional Review Board approval was obtained for this retrospective study (Memorial Sloan Kettering Cancer Center, New York, New York) and written consent was waived because no patient identifiers were collected. This manuscript adheres to the applicable STROBE guidelines. An initial data query in our institutional data warehouse was made for all patients who had a completed anesthetic between February 2nd, 2018 to February 2nd, 2019. This date range corresponded to 6 months before and after August 2nd, 2018, the date when the IPOA alert was implemented and the anesthesiology department was educated on events that would trigger the alert.
The following exclusion criteria was applied. We excluded cases that were duplicates or repeats of existing cases, which were created in our database if an anesthesiologist made an update to the IPOA or if there were multiple attendings starting the case. Additionally, we excluded cases that occurred in OR locations used for pediatric imaging, pediatric ophthalmology surgery, or pediatric procedures, and cases that occurred in a non-OR location. These groups of cases did not have a drapes-off time recorded in the EMR and thus did not have an alert sent time. Last, we excluded cases also missing a proper “drapes-off” notation and cases missing data for time when IPOA was documented. The remaining cases were separated into pre- and post-alert system implementation for analyses.
2.2. Statistical Analysis
The following analyses were performed: testing the proportion of cases compliant with on-time documentation of the IPOA pre- versus post-implementation for the full cohort and among subsets of interest, testing the time when the IPOA was completed relative to anesthesia end, and testing whether time of day of when surgery occurred had an impact on the time when the IPOA was completed relative to the drapes off/IPOA alert sent time. The proportion of compliance for pre- versus post-implementation was tested by Chi-square test. The test for compliance rate was performed first in the full cohort and then among subsets of interest (age of patient (adult vs pediatric), hospital service (Table 6), and case duration (<1 hour, 1–3 hours, 3–6 hours, and >6 hours). In the analysis of secondary endpoints, comparison of time when the IPOA was documented relative to the time of anesthesia end between pre- versus post-implementation periods were conducted using Wilcoxon rank-sum test. To test if time when the IPOA was completed relative to the drapes off/IPOA alert trigger had an association with the time of day when surgery occurred, a Kruskal-Wallis test was performed. All statistical tests were two-sided and P values <0.05 were considered statistically significant. R 3.6.0 (R Core Team, Vienna, Austria) was used for statistical analysis.
Table 6.
Comparison of pre and post implementation compliance rate by service * n/N (%) is compliance rate, which is defined as the number of compliant cases / total number of cases
| Compliance by Service | ||||
|---|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | PRE, N = 22846 | POST, N = 19439 | p-value2 |
| ANES | 837/1011 (83%) | 435/547 (80%) | 402/464 (87%) | 0.004 |
| BRE | 3213/3811 (84%) | 1662/2070 (80%) | 1551/1741 (89%) | <0.001 |
| CRS | 1280/1431 (89%) | 661/761 (87%) | 619/670 (92%) | <0.001 |
| Gl | 5563/7418 (75%) | 2754/4059 (68%) | 2809/3359 (84%) | <0.001 |
| GMT | 1194/1379 (87%) | 625/764 (82%) | 569/615 (93%) | <0.001 |
| GYN | 2065/2327 (89%) | 1071/1256 (85%) | 994/1071 (93%) | <0.001 |
| HEP | 788/886 (89%) | 430/507 (85%) | 358/379 (94%) | <0.001 |
| HNS | 1739/1888 (92%) | 942/1045 (90%) | 797/843 (95%) | <0.001 |
| IR | 6814/9303 (73%) | 3256/4992 (65%) | 3558/4311 (83%) | <0.001 |
| NEURO | 755/847 (89%) | 408/470 (87%) | 347/377 (92%) | 0.020 |
| OPTH | 177/216 (82%) | 100/122 (82%) | 77/94 (82%) | >0.9 |
| ORTHO | 777/881 (88%) | 394/457 (86%) | 383/424 (90%) | 0.074 |
| Other | 116/134 (87%) | 50/59 (85%) | 66/75 (88%) | 0.8 |
| PEDS | 415/515 (81%) | 222/278 (80%) | 193/237 (81%) | 0.7 |
| PLA | 1845/2044 (90%) | 910/1030 (88%) | 935/1014 (92%) | 0.004 |
| PULM | 930/1087 (86%) | 471/583 (81%) | 459/504 (91%) | <0.001 |
| RAD ONC | 512/601 (85%) | 259/316 (82%) | 253/285 (89%) | 0.026 |
| THO | 1969/2257 (87%) | 1032/1246 (83%) | 937/1011 (93%) | <0.001 |
| URO | 3533/4249 (83%) | 1791/2284 (78%) | 1742/1965 (89%) | <0.001 |
Statistics presented: n/N (%)
Statistical tests performed: chi-square test of independence
3. RESULTS
Through retrospective chart review of electronic patient records, 47,417 cases matched our inclusion criteria of patients that had a completed anesthetic between February 2nd, 2018 to February 2nd, 2019. In total, we excluded 5132 cases, which were composed of 1,160 repeats of existing cases, 3885 cases that were performed in an OR or non-OR location that did not record a drapes-off/alert sent time, 86 cases with a missing or incorrectly documented drapes-off/alert sent time, and 1 case with missing data for time when the IPOA was completed relative to anesthesia end. Thus, we included 42,285 cases in both our primary and secondary analysis. 22,846 cases occurred in the pre-implementation period, while 19,439 cases occurred in the post-implementation period (fig. 1).
3.1. Comparison of Case Characteristics in the Pre- and Post-Implementation Periods
The number of pediatric versus adult cases, number of emergency cases, time of day the operation occurred, service type, and case duration in the pre-implementation (6 months before) and post-implementation period (6 months after) were similar. The only characteristic that showed statistically significant difference was whether an alert was necessary in both the pre- and post-implementation period. An alert was deemed as necessary if the IPOA was not completed by the anesthesiologist prior to the recorded “drapes off” time in the EMR. In the pre-implementation period, an alert would have been necessary in 30% of cases, while in the post-implementation period, an alert was sent in 22% of cases (P < 0.001) (Table 1).
Table 1.
Comparison of characteristics between cases in pre- and post-implementation period.
Emergent = whether a case was an emergency case or not.
Alert necessary = whether an alert would have been or was necessary to be sent.
IR = interventional radiology; GI = gastrointestinal; URO = urology; BRE = breast; GYN = gynecology; THO = thoracic; PLA = plastics and reconstructive; HNS = head and neck; CRS = colorectal; GMT = gastric and mixed tumor; PULM = pulmonary; ANES = anesthesiology; HEP = hepatopancreatobiliary; ORTHO = orthopedic; NEURO = neurosurgery; RAD ONC = radiation oncology; PEDS = pediatric; OPTH = ophthalmology; OTHER = includes bone marrow, cardiology, dental, pain
| Table 1: Summary of characteristics at case level | |||
|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | PRE, N = 22846 | POST, N = 19439 |
| Adult vs Peds | |||
| ADULT | 40768 (96%) | 22015 (96%) | 18753 (96%) |
| PEDS | 1517 (3.6%) | 831 (3.6%) | 686 (3.5%) |
| Emergent | |||
| N | 41787 (99%) | 22548 (99%) | 19239 (99%) |
| Y | 498 (1.2%) | 298 (1.3%) | 200 (1.0%) |
| Time of Day | |||
| Morning | 23133 (55%) | 12684 (56%) | 10449 (54%) |
| Afternoon | 16441 (39%) | 8823 (39%) | 7618 (39%) |
| Evening | 2711 (64%) | 1339 (5.9%) | 1372 (7.1%) |
| Service | |||
| IR | 9303 (22%) | 4992 (22%) | 4311 (22%) |
| Gl | 7418 (18%) | 4059 (18%) | 3359 (17%) |
| URO | 4249 (10%) | 2284 (10.0%) | 1965 (10%) |
| BRE | 3811 (9.0%) | 2070 (9.1%) | 1741 (9.0%) |
| GYN | 2327 (5.5%) | 1256 (5.5%) | 1071 (5.5%) |
| THO | 2257 (5.3%) | 1246 (5.5%) | 1011 (5.2%) |
| PLA | 2044 (4.8%) | 1030 (4.5%) | 1014 (5.2%) |
| HNS | 1888 (4.5%) | 1045 (4.6%) | 843 (4.3%) |
| CRS | 1431 (3.4%) | 761 (3.3%) | 670 (3.4%) |
| GMT | 1379 (3.3%) | 764 (3.3%) | 615 (3.2%) |
| PULM | 1087 (2.6%) | 583 (2.6%) | 504 (2.6%) |
| ANES | 1011 (2.4%) | 547 (2.4%) | 464 (2.4%) |
| HEP | 886 (2.1%) | 507 (2.2%) | 379 (1.9%) |
| ORTHO | 881 (2.1%) | 457 (2.0%) | 424 (2.2%) |
| NEURO | 847 (2.0%) | 470 (2.1%) | 377 (1.9%) |
| RAD ONC | 601 (1.4%) | 316 (1.4%) | 285 (1.5%) |
| PEDS | 515 (1.2%) | 278 (1.2%) | 237 (1.2%) |
| OPTH | 216 (0.5%) | 122 (0.5%) | 94 (0.5%) |
| Other | 134 (0.3%) | 59 (0.3%) | 75 (0.4%) |
| OR Duration | |||
| <1 hr | 24739 (59%) | 13353 (58%) | 11386 (59%) |
| 1–3 hrs | 11642 (28%) | 6279 (27%) | 5363 (28%) |
| 3–6 hrs | 4602 (11%) | 2505 (11%) | 2097 (11%) |
| >6 hrs | 1302 (3.1%) | 709 (3.1%) | 593 (3.1%) |
| Alert necessary | 11210 (27%) | 6849 (30%) | 4361 (22%) |
Statistics presented: n (%)
3.2. Primary Analysis: Improved Compliance Rate in Post-Implementation Period
The compliance rate of IPOA completion increased from 76% to 88% (P < 0.001) before and after the alert implementation date (Table 2). Furthermore, the weekly compliance rate in the post-implementation period consistently remained above 80%, while the monthly compliance rate in the post-implementation period remained above 85% (Fig. 3). In the initial month following alert implementation, the compliance rate immediately increased to 83%. It continued to increase in the following months, with the 1st, 2nd, 3rd, 4th, and 5th month post-implementation having a compliance rate of 87%, 89%, 87%, 88%, and 87% respectively (Fig. 3). This sustained improvement suggest that the alert system may have a consistent, continued, positive impact on the anesthesiologists’ documentation compliance with the required IPOA element.
Table 2.
Comparison of compliance rate and time of IPOA completion between the pre- and post-implementation period.
*A negative value represents the number of minutes the IPOA was completed from anesthesia end.
* n/N (%) is compliance rate, which is defined as the number of compliant cases / total number of cases
| Table 2: Compliance among full cohort | ||||
|---|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | PRE, N = 22846 | POST, N = 19439 | p-value2 |
| Compliance | 34522/42285 (82%) | 17473/22846 (76%) | 17049/19439 (88%) | <0.001 |
| Time to IPOA (before anes stop) | −51 (−121, −14) | −48 (−120, −5) | −54 (−122, −20) | <0.001 |
Statistics presented: n/N (%); median (IQR)
Statistical tests performed: chi-square test of independence; Wilcoxon rank-sum test
Fig. 3.
Monthly and weekly compliance rate 6 months before and after alert implementation date
3.3. Secondary Analysis: Earlier IPOA Completion Time in Post-Implementation Period
We also investigated whether the IPOA alert implementation shifted the time that the IPOA was completed during a case. For this metric, we measured the difference in minutes between the time of IPOA completion and the anesthesia end time. In the pre-implementation period, the median time of IPOA completion was −48 minutes (48 minutes earlier than the anesthesia end time). In the post-implementation period, the median time of IPOA completion was −54 minutes (54 minutes earlier than the anesthesia end time) (P < 0.001) (Table 2). A negative value was categorized as compliant, with a more negative value implying an earlier IPOA documentation time. A positive value was categorized as non-compliant, with large positive durations implying longer delays in IPOA documentation. These indicate that since the introduction of the IPOA alert, anesthesiologists were completing the IPOA at an earlier time point in a case.
We investigated whether the time of day (morning, afternoon, evening) of when the surgery started had any impact on the time of when the IPOA was filled out. For this metric, we calculated the number of minutes away from the “drapes off” time (alert trigger). There was a significant difference in the median time when the IPOA was completed in the morning (−38 [−97, 2] minutes) versus afternoon (−30 [84, 3] minutes) versus night (−29 [−63, 0] minutes) (P < 0.001) (Table 3). A negative value referred to how many minutes before “drapes off” time (alert trigger) the IPOA was completed. These results show that IPOA documentation was completed at an earlier time during a case for surgeries occurring earlier in the day.
Table 3.
Comparison of time of IPOA completion between morning, afternoon, and evening cases.
*Drapes off is considered zero timepoint. Negative value represents time (in minutes) before before drapes off. Postiive values represent time (in minutes) after drapes off.
| Time of Day of Surgery and Impact on Time to IPOA Completion | |||||
|---|---|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | Morning, N = 23133 | Afternoon, N = 16441 | Evening, N = 2711 | p-value2 |
| Time to fill out IPOA after alarm (drapes) | −34 (−97, 2) | −38 (−116, 1) | −30 (−84, 3) | −29 (−63, 0) | <0.001 |
Statistics presented: median (IQR)
Statistical tests performed: Kruskal-Wallis test
3.4. Compliance Rate in Various Subgroups (Case Duration, Age, and Service)
In our study, we segmented cases into 4 different time categories based on case duration: < 1 hour, 1–3 hours, 3–6 hours, > 6 hours. Cases lasting <1 hour compromised 59% (N = 24,739) of all cases in our analysis, cases lasting 1–3 hours compromised 28% (N = 11,642) of all cases, cases lasting 3–6 hours compromised 11% (N = 4,602) of all cases, and cases lasting > 6 hours compromised 3% (N = 1,302) of all cases. With these subgroups, we noticed a positive trend when comparing case duration with compliance rate. Case durations of <1 hour had an overall compliance rate of 75%, 1–3 hours had an overall compliance rate of 89%, 3–6 hours had an overall compliance rate of 95%, and >6 hours had an overall compliance rate of 97%. Thus, non-compliant documentation was more likely to occur in shorter cases. We also saw significant differences in compliance rates between the pre- and post-implementation period when accounting for case duration. There was a significant improvement in compliance rate in the post-implementation period for cases lasting <1 hour (68% vs 83%; P < 0.001), 1–3 hours (85% vs 92%; P < 0.001), and 3–6 hours (93% vs 97%; P < 0.001). For cases lasting > 6 hours, there was no significant improvement (96% vs 98%; P = 0.061) (Table 4).
Table 4.
Comparison of pre- and post-implementation compliance rate between cases of varying durations. * n/N (%) is compliance rate, which is defined as the number of compliant cases / total number of cases
| Compliance by OR Duration | ||||
|---|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | PRE, N = 22846 | POST, N = 19439 | p-value2 |
| <1 hr | 18575/24739 (75%) | 9098/13353 (68%) | 9477/11386 (83%) | <0.001 |
| 1–3 hrs | 10312/11642 (89%) | 5355/6279 (85%) | 4957/5363 (92%) | <0.001 |
| 3–6 hrs | 4368/4602 (95%) | 2336/2505 (93%) | 2032/2097 (97%) | <0.001 |
| >6 hrs | 1267/1302 (97%) | 684/709 (96%) | 583/593 (98%) | 0.061 |
Statistics presented: n/N (%)
Statistical tests performed: chi-square test of independence
Furthermore, we investigated whether compliance rates differed by the age of the patient. When considering adult cases (patient age > 18 years old) (N = 40,768), there was a statistically significant increase in compliance between the pre- and post-implementation period (76% vs 88%; P < 0.001). For pediatric cases (patient age < 18 years old) (N = 1,517), there was also a statistically significant improvement in compliance (80% vs 85%; P < 0.004) (Table 5).
Table 5.
Comparison of pre- and post-implementation compliance rate between adult and pediatric cases. * n/N (%) is compliance rate, which is defined as the number of compliant cases / total number of cases
| Compliance by Adult vs Pediatrics | ||||
|---|---|---|---|---|
| Characteristic1 | Overall, N = 42285 | PRE, N = 22846 | POST, N = 19439 | p-value2 |
| ADULT | 33275/40768 (82%) | 16812/22015 (76%) | 16463/18753 (88%) | <0.001 |
| PEDS | 1247/1517 (82%) | 661/831 (80%) | 586/686 (85%) | 0.004 |
Statistics presented: n/N (%)
Statistical tests performed: chi-square test of independence
The pre- and post-implementation compliance rate for different types of surgical services is listed in Table 6. Of the 19 services, 15 demonstrated statistically significant improvement (P < 0.05) in compliance rates between pre- and post-implementation periods. The most notable improvements occurred in the gastrointestinal surgery service, which increased from 68% to 84% (a 16% difference; P < 0.001) and interventional radiology service, which increased from 65% to 83% (an 18% difference; P < 0.001). The other surgical services with statistically significant improvement were anesthesiology, breast, colorectal, gastric and mixed tumor, gynecology, hepatopancreaticobiliary, head and neck, neurosurgery, plastics and reconstructive, pulmonary, radiation oncology, thoracic, and urology. Surgical services where the data did not show statistically significant improvement were ophthalmology, orthopedics, pediatric, and other (which include bone marrow, cardiology, dental, and pain) (Table 6).
4. DISCUSSION
Studies have demonstrated the positive impact of real-time clinical alerts on documentation of clinical events. One study demonstrated improved compliance in arterial catheter placement documentation developed.12 Another study demonstrated that a one-time page reminder within the first 15 minutes of an anesthetic could improve missing allergy documentation.10
Our retrospective study indicates that an automated CDS system alert also proved to have a positive impact on compliance of anesthesia assessment documentation. Compliance was shown to immediately improve within weeks and months following IPOA alert implementation and was sustained over the next 6 months. Furthermore, IPOA documentation was completed at an earlier time for cases during the post-implementation period.
The increase in compliance rate and earlier completion of IPOA documentation can be explained in several ways. First, a simple reminder page to the anesthesiologist may help refocus their attention to completing documentation if they had forgotten. Second, raised awareness that IPOA documentation compliance is low and being monitored by the regulatory affairs department could lead to behavioral modification (Hawthorne Effect) In multiple studies, the Hawthorne effect has been reported to have an immediate positive impact on study outcomes, making it crucial to study behavior rates beyond initial study dates to see if the changes are sustained or become extinct.10,14–16
Through our secondary analysis, we observed significant differences in pre- and post-implementation compliance rates across different subsets of case duration. Cases with a duration of <1 hour had the lowest overall compliance rate and had the greatest improvement in compliance rate. The lower compliance rate for shorter cases is likely because the anesthesiologists have less available aggregate time to document the IPOA before anesthesia end. A longer case would give the anesthesiologist time to attend to competing needs and still complete the IPOA in requisite time.
There were also significant differences in pre- and post-implementation compliance rates across all services except for ophthalmology, orthopedics, pediatric, and other (bone marrow, cardiology, dental, and pain). Gastrointestinal (GI) and interventional radiology (IR) services demonstrated the greatest percentage increase in documentation compliance. One reason may be that their baseline compliance rates had greater room for improvement since they were lower compared to the rates of other services (68% for GI, and 65% for IR). Additionally, GI and IR services typically perform cases with shorter durations, which were shown to have the greatest improvement in compliance rate.
Lastly, we observed a trend when grouping cases by time of day. Mornings were associated with an earlier IPOA completion time. This might be by explained by the idiosyncratic workflow that requires anesthesiologists to supervise more procedures as the day progresss, leading to additional clinical responsibilities that can delay IPOA completion.
Our study differs from previous studies because we investigated a nuanced, extremely time-sensitive, compliance metric, IPOA documentation, that must be completed within a narrow, intraoperative time window. Additionally, the volume of cases analyzed in this study (N = 42,285) was much larger than previous studies, as every anesthetic at our institution requires the completion of IPOA documentation. Furthermore, our study also investigated secondary endpoints that impacted the intraoperative time window and workflow (e.g. time of day, case duration, service, and age) and assessed whether these subgroups revealed any trends in documentation compliance rates that may affect alarm effectiveness. The trends observed give insight into how documentation compliance may differ depending on the characteristics of the case and highlight scenarios where clinical alerts may be most impactful.
There are several limitations to our study. One limitation is the concept of alarm fatigue, where a clinician fails to adequately respond to a clinical alarm because of an excessive amount of alarms. One review article listed that a healthcare professional may be subject to as many as 1000 alarms per shift, with the FDA reporting over 500 alarm-related patient deaths in the past 5 years.17 While we did not account for the total number of alarms received by an anesthesiologist during a case, it is possible these other alarms negatively impacted documentation compliance rate. Second, we cannot exclude the possibility of a training effect influencing the significant increase in compliance rate. However, all anesthesia departments are required to educate staff and the only way to have the anesthesia providers comply with requirements is to make them aware of the requirement. At our institution, general educational email reminders were not found to be sufficient to yield the level of documentation compliance required. Despite educational emails being sent in May 2018, the monthly compliance rates in the pre-implementation period remained below the monthly compliance rates in the post-implementation period. A third limitation is the significantly fewer percentage of cases where an alert was necessary in the pre-implementation period (30%) compared to the post-implementation period (22%). To maintain similarities between the pre- and post-implementation period, we would have expected no statistically significant difference between these two values. However, the alert implementation resulted in an improved compliance rate and earlier time of IPOA completion, resulting in a lower percentage of cases with necessary alerts in the post-implementation period. A fourth limitation is that our results are strongly tied to our institution’s anesthesiology practice pattern of the MD/CRNA care team model. The impact of a clinical alert on anesthesiology documentation standards may differ depending on the institution’s care team models.
In summary, we demonstrate that automated CDS system alerts via a single page notification can improve the compliance rate for documentation of key anesthesia events and that this observation is sustained six months after the implementation date. Furthermore, improvement in compliance is highest in adult cases that have a duration shorter than 6 hours and occur earlier in the day. While this study shows promising results in the use of automatic CDS system alerts to help hospitals meet TJC and CMS standards, future studies over a longer period are needed to corroborate these findings and investigate the sustainability of positive changes derived from alert systems.
Supplementary Material
Fig. 2.
Patient data query, exclusion criteria, and allocation to pre- and post-implementation period.
Highlights.
Alert notification driven by clinical alert system can improve compliance in anesthesia documentation
Alert notification improved compliance in anesthesia documentation for all subgroups of patients studied and this observation was sustained 6 months after implementation.
Clinical alert systems are a powerful tool to improve process measures in anesthesia including compliance with documentation
Financial Disclosures:
The authors’ work was supported and funded in part by NIH/NCI Cancer Center Support Grant P30 CA008748
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Declaration of competing interests
Conflict of Interest: LET is a grant recipient through Merck Investigator Studies Program (MISP) to fund clinical trial at MSKCC (NCT03808077). LET serves a consultancy and advisory role for Merck & Co. Pharmaceutical Company.
Contributor Information
Luis E Tollinche, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
Richard Shi, New York Medical College School of Medicine.
Margaret Hannum, Memorial Sloan Kettering Cancer Center Department of Biostatistics and Epidemiology.
Patrick McCormick, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
Alisa Thorne, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
Kay See Tan, Memorial Sloan Kettering Cancer Center Department of Biostatistics and Epidemiology.
Gloria Yang, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
Meghana Mehta, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
Cindy Yeoh, Memorial Sloan Kettering Cancer Center Department of Anesthesiology and Critical Care Medicine.
CITATIONS:
- 1.CMS’ Program History: Medicare and Medicaid. https://www.cms.gov/About-CMS/Agency-Information/History/index.html. Accessed 7/11/2019.
- 2.Facts about federal deemed status and state recognition. 2018; www.jointcommission.org/facts_about_federal_deemed_status_and_state_recognition/.
- 3.Statement on Documentation of Anesthesia Care. American Society of Anesthesiologists;2018. [Google Scholar]
- 4.Commission TJ. Comprehensive Accreditation Manuals for Hospitals. 2019.
- 5.Services CfMaM. CMS Manual System In. §482.52 Condition of Participation: Anesthesia Services 2011. [Google Scholar]
- 6.Choi CK, Saberito D, Tyagaraj C, Tyagaraj K. Organizational Performance and Regulatory Compliance as Measured by Clinical Pertinence Indicators Before and After Implementation of Anesthesia Information Management System (AIMS). J Med Syst. 2014;38(1). [DOI] [PubMed] [Google Scholar]
- 7.Vigoda MM, Gencorelli F, Lubarsky DA. Changing medical group behaviors: increasing the rate of documentation of quality assurance events using an anesthesia information system. Anesth Analg. 2006;103(2):390–395, table of contents. [DOI] [PubMed] [Google Scholar]
- 8.Spring SF, Sandberg WS, Anupama S, Walsh JL, Driscoll WD, Raines DE. Automated documentation error detection and notification improves anesthesia billing performance. Anesthesiology. 2007;106(1):157–163. [DOI] [PubMed] [Google Scholar]
- 9.Wax DB, Beilin Y, Levin M, Chadha N, Krol M, Reich DL. The effect of an interactive visual reminder in an anesthesia information management system on timeliness of prophylactic antibiotic administration. Anesth Analg. 2007;104(6):1462–1466, table of contents. [DOI] [PubMed] [Google Scholar]
- 10.Sandberg WS, Sandberg EH, Seim AR, et al. Real-time checking of electronic anesthesia records for documentation errors and automatically text messaging clinicians improves quality of documentation. Anesth Analg. 2008;106(1):192–201, table of contents. [DOI] [PubMed] [Google Scholar]
- 11.Wax DB, McCormick PJ, Joseph TT, Levin MA. An Automated Critical Event Screening and Notification System to Facilitate Preanesthesia Record Review. Anesth Analg. 2018;126(2):606–610. [DOI] [PubMed] [Google Scholar]
- 12.Kheterpal S, Gupta R, Blum JM, Tremper KK, O’Reilly M, Kazanjian PE. Electronic reminders improve procedure documentation compliance and professional fee reimbursement. Anesth Analg. 2007;104(3):592–597. [DOI] [PubMed] [Google Scholar]
- 13.Wanderer JP, Sandberg WS, Ehrenfeld JM. Real-time alerts and reminders using information systems. Anesthesiology clinics. 2011;29(3):389–396. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Srigley JA, Furness CD, Baker GR, Gardam M. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf. 2014;23(12):974–980. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67(3):267–277. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Demakis JG, Beauchamp C, Cull WL, et al. Improving residents’ compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000;284(11):1411–1416. [DOI] [PubMed] [Google Scholar]
- 17.Ruskin KJ, Hueske-Kraus D. Alarm fatigue: impacts on patient safety. Curr Opin Anaesthesiol. 2015;28(6):685–690. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.