Reducing Risk with Clinical Decision Support

Summary

Objective

Identify clinical opportunities to intervene to prevent a malpractice event and determine the proportion of malpractice claims potentially preventable by clinical decision support (CDS).

Materials and Methods

Cross-sectional review of closed malpractice claims over seven years from one malpractice insurance company and seven hospitals in the Boston area. For each event, clinical opportunities to intervene to avert the malpractice event and the presence or absence of CDS that might have a role in preventing the event, were assigned by a panel of expert raters. Compensation paid out to resolve a claim (indemnity), was associated with each CDS type.

Results

Of the 477 closed malpractice cases, 359 (75.3%) were categorized as substantiated and 195 (54%) had at least one opportunity to intervene. Common opportunities to intervene related to performance of procedure, diagnosis, and fall prevention. We identified at least one CDS type for 63% of substantiated claims. The 41 CDS types identified included clinically significant test result alerting, diagnostic decision support and electronic tracking of instruments. Cases with at least one associated intervention accounted for $40.3 million (58.9%) of indemnity.

Discussion

CDS systems and other forms of health information technology (HIT) are expected to improve quality of care, but their potential to mitigate risk had not previously been quantified. Our results suggest that, in addition to their known benefits for quality and safety, CDS systems within HIT have a potential role in decreasing malpractice payments.

Conclusion

More than half of malpractice events and over $40 million of indemnity were potentially preventable with CDS.

1. Introduction

Clinical decision support (CDS) systems are electronic tools within health information technology (HIT) systems. CDS tools have principally been developed to improve the quality of patient care and increase patient safety. When effectively used, CDS has been shown to positively impact healthcare cost, diagnosis, quality, patient safety, adherence to guidelines for disease prevention and treatment, and medication errors [1-7]. In addition, CDS may be of value for directly mitigating malpractice risk by promoting communication and preventing errors related to medications, test interpretation, and diagnosis. However, the potential value of CDS for mitigating malpractice risk has been largely unexplored.

The frequency of malpractice claims, their occurrence across all care settings, and the average size of associated payments merit the development of robust systems-based interventions to reduce risk. A recent study of malpractice claims from 1991 to 2005 found that annually 7.4% of physicians experience a malpractice claim and 1.6% made a payment for resolution of the claim [8]. The mean compensation paid out to resolve a claim, whether through settlement, trial, or arbitration (indemnity) was $274,887, with a median payment of $111,749. While high-risk specialties account for a greater proportion of individual physician risk, malpractice events are not confined to complex inpatient settings. In a review of paid claims from 2005 to 2009, the outpatient setting accounted for 43.1% of claims while inpatient claims accounted for 47.6%. The remaining 9.4% of claims involved care delivered in both inpatient and outpatient settings [9]. The principal allegations of malpractice claims are diverse. The majority of issues are clinical, including delayed or missed diagnoses or technical errors in treatment; however, non-clinical allegations, such as breaches of privacy or assault also occur.

The spectrum of CDS types housed in mature hospital HIT are broad, ranging from diagnostic systems to treatment planning algorithms and reminders for screening [10, 11]. We evaluated the relationship between malpractice claims and CDS types and the extent to which claims might be mitigated through the use of HIT. In this study, we applied Osheroff’s broad definition of CDS: “provid[ing] clinicians, staff, patients or other individuals with knowledge and person-specific information, intelligently filtered or presented at appropriate times, to enhance health and health care”[12].

2. Methods

2.1 Claims sample

Partners HealthCare System is an integrated delivery system in the Boston, Massachusetts area. Partners is comprised of hospitals, a home care system and a network of community-based physicians. Most Partners hospitals are insured for professional liability by the Controlled Risk Insurance Company (CRICO). For more than 25 years, CRICO has maintained a database of malpractice claims containing all information related to each claim. We defined a claim as a written demand for compensation for medical injury. Each claim file includes a variety of structured and unstructured elements such as a clinical summary, details of the location of the event (inpatient/outpatient), litigation-related documents (narrative statements from healthcare personnel, peer reviews, and depositions), clinical records deemed pertinent to the case’s defense, and information regarding the patient’s pre- and post-event status. Each claim record also contains data on two types of costs associated with the claim: legal expenses in defending the claim (which are present for all claims regardless of the outcome) and indemnity payments, which are the total compensation paid out to resolve a claim, whether through settlement, trial, or arbitration. For outpatient claims, the allocation to a hospital is assigned based on the primary site of employment of the responsible physician. A claim is classified as closed when it has been dropped, dismissed, paid by settlement or resolved by verdict.

We obtained claim files for all closed malpractice claims (n=477) from seven CRICO-insured Partners hospitals and their associated outpatient practices with an incident date between January 1, 2000 and December 31, 2007 that were asserted by December 31, 2008 and reviewed these claims. Open claims were not used because the full abstraction of the claim completed by CRICO including coding and clinical narrative is not final until the claim is closed. Our sample included 2 academic medical centers (Brigham and Women’s Hospital, Massachusetts General Hospital), 3 community hospitals (Faulkner Hospital, Newton-Wellesley Hospital, North Shore Medical Center), an inpatient rehabilitation hospital (Spaulding Rehabilitation Hospital), and an inpatient psychiatric hospital (McLean Hospital).

CRICO provided secured, de-identified case abstracts for review by the research team. These abstracts included data on the disposition of the case, as well as indemnity loss paid (payment made to the claimant) and legal expenses incurred defending the claim. The study was approved by the Partners Human Research Committee.

2.2 Information System Setting

Partners HealthCare, along with its member hospitals, is well-known for its broad-based implementation of HIT and for the volume and depth of its CDS [13]. According to a 2007 review, Partners has 181 unique types of CDS in place with 7,120 active rules [14]. A recent survey of CDS capabilities at hospital sites and with electronic health record vendors found that across all participating institutions Partners HealthCare utilizes 96.2% of existing CDS types (with two exceptions being diagnostic decision support and computer-assisted treatment planning)[15].

Partners’ sites use a mix of self-developed and vendor HIT systems for inpatient care with varying deployment of decision support capabilities. Each participating institution historically has had substantial autonomy in terms of system selection, deployment and overall HIT strategy. Thus, there is wide variety in CDS deployment across the organization. All sites surveyed in the current study utilize the Longitudinal Medical Record (LMR) for outpatient care. The LMR is a self-developed ONCATCB-certified electronic medical record with embedded CDS as well as integrated computerized provider order entry (CPOE). LMR functionality includes numerous types of CDS including tools such as screening reminders, documentation reminders, drug-drug interaction alerts, drug-allergy alerts, automated flagging of abnormal lab values, and online clinician resources. However, CDS has been implemented on a site-by-site basis allowing for variation within the product as implemented.

2.3 Study Instrument and Claims File Review

The study review team consisted of a board-certified internist and infectious disease specialist with expertise in malpractice and risk informatics (GZ), a medical informatician with expertise in clinical decision support (AW) and a project manager (FM). Case review began with classification of each claim, by consensus of the expert review group, into one of three groups based on the clinical narrative: substantiated, unsubstantiated, or excluded (▶Figure 1). Substantiated claims were those with an identifiable gap in care, a known complication, or an unavoidable outcome evident in the available claim data (narrative and litigation-related documents). Unsubstantiated claims were those characterized by psychosocial factors or non-clinical patient complaints, such as a patient who believed he was held hostage in the hospital. Certain claims that did not meet study criteria were excluded from the study analysis. These included non-clinical claims in which the event of the malpractice claim was not related to clinical care delivery (e.g. HIPAA violations, sexual misconduct or assault). Cases were also excluded if the event occurred at a hospital site other than the seven in the study sample, the loss date was outside of the study period, or if a duplicate of a claim previously reviewed.

Fig. 1.

Flow Diagram

Following categorization, substantiated and unsubstantiated claims (n=403) were reviewed in depth. The panel reviewed the clinical narrative for each case and, where necessary, the case file (either in electronic or paper form). For each case, the panel identified clinical opportunities to intervene: a missed opportunity to take an action that might have averted the loss event. Reviewers considered cognitive-, system-, and patient-related causes related both to the episode of care as a whole and to the particular event. Cases could have no opportunity to intervene, one opportunity to intervene or multiple opportunities to intervene.

If an opportunity to intervene was identified for a claim, reviewers then considered different types of CDS with potential to prevent the event and determined whether CDS might have been able to facilitate the opportunity to intervene and, potentially, to mitigate the risk associated with the case. Reviewers considered a broad definition of CDS [12] that included concepts such as online reference tools (e.g. UpToDate and VisualDx), applications designed to assist in the generation of a differential diagnosis (e.g. DXplain and Isabel) and computer programs that perform automated test result interpretation and diagnosis.

In addition, reviewers considered types of CDS available at the hospital site where the incident occurred, at all other hospital sites in the study sample, at hospitals elsewhere in the United States, or in existing literature on CDS including available CDS taxonomies [15]. They also discussed novel HIT interventions not currently in existence, but realistically viable for development in response to incidents such as those in the claim under consideration. The taxonomy of opportunities to intervene and CDS interventions evolved over the course of the study, and the study team iteratively rereviewed the cases in order to ensure consistent application of the opportunities to intervene and CDS types.

2.4 Role of Funding Source

This work was supported by Partners Siemens Research Council grant number 086712. The funder did not play a role in the design, conduct, or analysis of the study.

3. Results

Of the 477 malpractice claims against CRICO-insured physicians, 74 were excluded: 36 because the event occurred at a non-study site, 36 because the claim related solely to non-clinical events (e.g. sexual misconduct or inappropriate disclosure of protected health information), 1 due to an incorrect loss date and 1 as a duplicate. We reviewed 403 claims. Of these, 44 (10.9%) were judged to be unsubstantiated.

3.1 Opportunities to Intervene and Clinical Decision Support

Of the 359 substantiated claims, 164 had no opportunities to intervene and 195 had at least one opportunity to intervene (131 had one opportunity to intervene, 64 had >1 opportunity to intervene). No opportunities to intervenes were identified in the 44 claims judged to be unsubstantiated. The review team identified 47 unique opportunities to intervene (▶ Supplementary File Table) that were classified into 13 parent categories (▶ Table 1). The most frequent opportunities to intervene related to incorrectly performed procedures (n=50), incorrect diagnoses (n=23), fall prevention (n=16) and retained foreign bodies (n=16). The top ten opportunities to intervene accounted for 65% of all opportunities to intervene (▶ Table 2).

Table 1.

Categories of Opportunities to Intervene

1. Communication

2. Consent

3. Diagnosis

4. Documentation

5. Medication

6. Monitor Lab Results

7. Ordering

8. Patient Care

9. Procedure

10. Radiology

11. Referral Management

12. Results Management

13. Screening

Table 2.

Top 10 Opportunities to Interven

Opportunity to Intervene	Number of Cases (n=272)	Percentage
1. Procedure: perform procedure correctly	50	18.4
2. Diagnosis: diagnosis correctly	23	8.5
3. (tie) Patient Care: fall prevention	16	5.9
3. Procedure: don’t leave foreign body in patient	16	5.9
5. Patient Care: timely	15	5.5
6. Medication: give correct medication	13	4.8
7. (tie) Consent: obtain and document	12	4.4
7. Results Management: test performed, not followed up	12	4.4
9. Documentation: document comprehensively	11	4.0
10. Patient Care: patient monitoring	8	2.9

Among the 195 claims with an opportunity to intervene, CDS interventions with the potential to facilitate the opportunity to intervene and avert the incident were identified for 123 claims. Thus, 30% (123 of 403) of all malpractice events and 63% (123 of 195) of substantiated events with a missed opportunity for action represented by an opportunity to intervene were theoretically preventable by CDS.

There were 41 unique CDS interventions. The most commonly identified interventions were clinically significant test result alerting (n=15), diagnostic decision support (n=13), electronic tracking of instruments (n=12), and a template for procedure specific complications (n=11). The top ten CDS accounted for 61% of all CDS (▶ Table 3).

Table 3.

Top 10 Clinical Decision Support

Clinical Decision Support	Number of Cases (n=164)	Percentage
1. Clinically significant test result alerting	15	9.1
2. Diagnostic decision support	13	7.9
3. Electronic tracking of instruments	12	7.3
4. Template for procedure specific complications	11	6.7
5. (tie) System to enforce pause (timeout)	10	6.1
5. Electronic referral / consult management	10	6.1
7. Medication administration checking – drug, dose, patient verification	9	5.5
8. Clinical alerting	8	4.9
9. (tie) System to enforce timely checks	7	4.3

3.2 Indemnity

The total indemnity paid for all claims was $68,514,395 (▶ Table 4). Substantiated claims with at least one opportunity to intervene (195), accounted for a total of $59,781,978 in potentially preventable indemnity. Substantiated cases with at least one associated CDS (123), accounted for a total of $40,389,079 in potentially preventable indemnity, so 58.9% of all indemnity could potentially be prevented if CDS was consistently employed. ▶ Table 5 and ▶ Table 6 show the top opportunities to intervene and CDS by amount of preventable indemnity. The opportunities to intervene associated with the largest indemnity payments were related to performing procedures correctly ($17,029,545), diagnosing correctly ($11,800,000) and providing timely patient care ($8,256,014). The related CDS associated with the largest potential reduction in indemnity included diagnostic decision support systems ($8,450,000), systems to enforce timely checks ($6,792,482) and multimedia/multi-language informed consent ($5,644,982).

Table 4.

Indemnity Paid

Category (n)	Sum	Average
All cases (435)	$68,514,395	$410,078
Substantiated (359)	$67,668,191	$383,742
Substantiated w/ Opportunity to Intervene (195)	$59,781,978	$306,574
Substantiated w/ Opportunity to Intervene + Clinical Decision Support (123)	$40,389,079	$328,366

*42 cases were excluded because the loss event did not occur at a study site

Table 5.

Top 10 Opportunities to Intervene by Indemnity

Opportunity to Intervene	Indemnity	Percentage of total indemnity with at least one opportunity to intervene ($59,781,978)
1. Procedure: perform procedure correctly	$17,029,545	28.5
2. Diagnosis: diagnose correctly	$11,800,000	19.7
3. Patient Care: timely	$8,256,014	13.8
4. Consent: obtain and document	$5,889,982	9.9
5. Patient Care: patient monitoring	$4,653,750	7.8
6. Results Management: test performed, not followed up	$3,625,410	6.1
7. Communication: staff-staff	$3,541,247	5.9
8. Medication: give correct medication	$3,149,999	5.3
9. Results Management: test ordered, not performed	$3,030,000	5.1
10. Documentation: document comprehensively	$2,999,900	5.0

Table 6.

Top 10 Clinical Decision Support by Indemnity

Clinical Decision Support	Indemnity	Percentage of total indemnity with at least one clinical decision support ($40,389,079)
1. Diagnostic decision support	$8,450,000	20.9
2. System to enforce timely checks	$6,792,482	16.8
3. Multimedia / language informed consent	$5,644,982	14.0
4. Clinically significant test result alerting	$4,825,410	11.9
5. Electronic referral / consult management	$4,387,805	10.9
6. Template for procedure specific complications	$3,719,982	9.2
7. Test reconciliation	$3,030,000	7.5
8. CPOE – legibility	$2,820,000	7.0
9. Procedure modeling / planning	$2,466,664	6.1
10. Screening reminders	$2,300,000	5.7

4. Discussion

Over a third of the substantiated malpractice claims in our sample were potentially amenable to mitigation through CDS and this translated to a potential to save over half of the indemnity paid. These results suggest that, in addition to their known benefits for quality and safety, CDS systems within HIT may have a role decreasing malpractice payments. Without an extensive CDS deployment, electronic medical records are merely a substitute for paper. They simply serve as a tool for viewing results or writing notes and lack sophisticated flagging systems, reminders, or auditing functions that can both support clinician workflow and improve patient care.

Reminder systems, particularly well-designed ones that are seamlessly incorporated into physician workflow, can facilitate test management. The process of care associated with management of a clinical test result is complicated due to multiple steps and a reliance on manual processes without fail-safes. Typically, this process is broken down into test ordering, test performance, test interpretation, transmittal/receipt of results, formation of a follow-up plan, and patient adherence with the plan [16]. Breakdowns at any of these steps can result in a result not being addressed appropriately and can lead to an adverse outcome. In our study, decision support tools for the tracking and management of clinically significant test results were identified most frequently as a potentially beneficial form of CDS. Deployment of just this one type of CDS could potentially have prevented $4.83 million in incurred losses.

Diagnostic decision support was our second most frequently deficient CDS tool and accounted for the greatest proportion of potentially preventable indemnity, $8.45 million. Diagnostic decision support has been shown to be capable of generating accurate diagnoses [17] and modestly increasing clinician diagnostic accuracy [18]. Diagnostic decision support has also been demonstrated to reduce the cost of service for diagnostically challenging cases [19]. Despite some evidence suggesting benefit, diagnostic CDS systems remain underutilized [20].

Another CDS tool for surgical procedures is electronic instrument tracking. Use of an electronic tracking system for instruments, such as system enforced or supported counts, bar coding or radio-frequency identification (RFID) was identified as a potential intervention in 12 claims. RFID and other tracking systems are known to decrease the risk of retained instruments and foreign bodies and are increasingly becoming standard of care nationally [21-24].

Our study has several limitations. First, there is a poor relationship of claims to actual adverse events or negligence [25-27]. So these results do not reflect the relationship between CDS and adverse events in general, only those events which lead to malpractice claims. Second, reviewers were not blinded to the outcome of the patient event nor the malpractice claim. Another review team reviewing the same dataset could develop a different taxonomy or apply it differently. Although all CDS types assigned by the review team were closely related to the critical events in the cases it is possible that, even with the related CDS in place, that some of the cases would still have occurred. Such causes could be due to limitations in the intervention or to the complex, multifactorial nature of the cases, where it is impossible to pinpoint a single cause for the system failure [28]. Although this subjectivity is inherent, the reviewers worked to maximize internal consistency by employing an iterative review and consensus-based coding process. Third, our sample of malpractice claims came from a single academic healthcare system in the Northeast – it is possible that our system’s malpractice experience is not representative, although many patterns in our sample of claims are also present in other nationally representative sample. Fourth, we reviewed a small number of malpractice claims, which often represent the tip of the iceberg of patient-safety-related event. Malpractice claims data generally have several biases, including younger patients and more severe injuries then the general population [29, 30]. Therefore, it is possible that the malpractice claims experience may not be generalizable to entire spectrum of patient safety-related events.

It is important not to conclude from this work that the mere existence or availability of CDS guarantees its effectiveness. To be effective, CDS development and implementation requires a thoughtful examination of overall workflow, as well as consideration of the potential burden on clinician time, alert fatigue and other human factors. An additional challenge to CDS development and implementation is the fact that CDS that demonstrates benefit in a focused research setting may be impractical or excessively costly to implement broadly. CDS also must often be coupled with other preventive measures. Consider the issue of falls prevention: there are many physical preventive measures to help minimize falls (guard rails, human assistants, bathroom alarms, etc), merely providing an alert (CDS) regarding falls risk will do little to mitigate that risk in the absence of programmatic and human intervention. However, in an organization where such programs exist, CDS may do much to achieve consistent and effective application. An additional consideration is the consistency of deployment of CDS – at Partners, different CDS systems are in place at different sites, so we sometimes encountered cases at one site that would have been potentially prevented had they occurred at another site in our system which had related CDS in place. We are now in the process of standardized our clinical information systems, including CDS, across all sites to ensure that consistent logic is applied throughout our system.

These challenges notwithstanding, well-designed CDS that is both seamlessly integrated with clinician workflow and supportive of patient care delivery programs is difficult to achieve. Despite this, we believe, based on our assessment, that there is a potential benefit for such CDS in terms of malpractice risk mitigation and cost savings that has not been previously been documented.

5. Conclusion

Review of closed malpractice claims over a seven-year period allowed definition of an inventory of CDS tools with potential to mitigate malpractice risk. This inventory can be used both locally in our health care system’s HIT and nationally to evaluate the potential for a hospital’s implemented information system to mitigate risk and to inform prioritization of hospital information system enhancements. Based on our work, we believe that organizational risk management activities should include critical self-evaluation of an institution’s current state of HIT and implemented CDS, but that such programs should not be solely HIT (or CDS-driven). Human factors activities (as in communication: staff-staff), education, and less costly technologies are all also necessary to address safety. We recommend that organizations studying adverse events in healthcare, whether manifested as malpractice cases or not, consider the role for CDS in mitigating risk of future reoccurrence.