Implementation of a FHIR-based clinical decision support tool for calculating CHA2DS2-VASc Scores

The CHA2DS2-VASc score guides oral anticoagulant (OAC) treatment decisions in atrial fibrillation (AF), but requires that clinicians gather information on the components of the score, including age, sex, and five clinical conditions.^1,2 Thorough review of a patient’s entire medical history and rote transfer of data to a calculator can be onerous and time-consuming. Here, we discuss the use of SMART on FHIR platform (Substitutable Medical Applications, Reusable Technologies on the Fast Healthcare Interoperability Resources) to address this workflow bottleneck. SMART on FHIR is a platform that allows secure information exchange between electronic medical records (EMR) and third party apps using a standardized data exchange format. The goal is to encourage innovative, clinically useful, EMR-based tools. Our goal was to validate an implementation of the MDCalc app that uses EMR data to calculate a CHA2DS2-VASc score at the point of care, and identify factors that affect app accuracy in clinical practice. This quality improvement project did not fall under the definition of research under 45 CFR part 46 and therefore did not require IRB review.

We prospectively compared automated app scores with clinician scores by identifying consecutive, adult outpatients with a primary visit diagnosis of AF seen at the University of Utah cardiology clinics over a one-month period. Within 24 hours of the clinic visit, one of two reviewers used the beta version of MDCalc on FHIR (MoF) app to automatically calculate a CHA2DS2-VASc score. The MoF app autofills the score as a first step, then allows the clinician to change inputs if needed. We evaluated the first step, automatic calculation prior to any adjustments. For comparison, the reviewer simultaneously identified whether a patient was prescribed an anticoagulant (suggesting a score ≥2), and captured CHA2DS2-VASc scores documented by clinicians in the visit note.

We identified 200 consecutive AF patients seen between 11/5/2018 and 12/7/2018, 111 of whom had a documented CHA2DS2-VASc score. The mean MoF app score was 3.79 (SD 1.86) compared to a mean clinician score of 3.25 (SD 1.63, p=0.02). If the MoF app were used instead of the documented score, 13.5% (n=27) of patients would move into or out of the high risk group (defined as CHA2DS2-VASc score ≥2). Ten percent (n=19) of patients were “up-classified” by the MoF app, meaning they were considered high risk by the app and low risk by the clinician. Four percent (n=8) of patients were “down-classified” by the app. Upon review of these cases, we accounted for documented clinical decisions which lead to patients either being anticoagulated or not regardless of their stroke risk (e.g. patient preference, bleeding, or recent cardioversion). After accounting for these clinical decisions, we found that 3% (n=5) of patients were “up-classified” by the app, and 2% (n=3) of patients were “down-classified” by the app, making the adjusted net reclassification index 4% (n=8).

Among the 111 patients who had a documented clinician score, the exact scores differed in 61% (n=68) of the cases. We identified condition-specific discrepancies for the 60 patients in whom the clinician documented specific components. Overall, we found 70 condition-specific differences (Table 1); heart failure (n=26) and hypertension (n=12) were the most common discrepancies. The app captured a condition that was not noted by the clinician in 57 cases; the clinician noted a condition that was not captured by the app in 13 cases. Two themes emerged. First, the MoF app identified conditions because specific medications were incorrectly attributed to conditions. For example, a prescription for losartan would lead the app to classify the patient as “hypertension present” when that medication may have been used solely for heart failure. Second, the problem list (which is used to generate the MoF score) included conditions that the clinician did not count, or the clinician noted a condition that was not in the problem list (e.g. new patients whose charts had not yet been populated problem list). Notably, the autofill design of the app was intentionally made to be more sensitive than specific, so it would present all potential information to the clinician for consideration.

Table 1.

Condition-specific discrepancies in the CHA2DS2 -VASc score for 60 patients using a MDCalc on FHIR app and clinician documentation

Condition	Captured by App, Not Noted by Clinician	Captured by Clinician, Not Noted by App
Heart Failure	25	1
Hypertension	10	2
Stroke	6	4
Vascular	11	5
Diabetes	5	1
Total	57	13

^*Please note, a single patient may have discrepancies for more than one condition. The table includes 70 condition discrepancies for 60 patients.

Apps and other forms of clinical decision support are increasingly prevalent in healthcare delivery and regulatory guidance is a work in progress.³ Prior studies have found little impact on treatment rates. Chaturvedi et al. found that using automated electronic decision support would have theoretically increased anticoagulation rates by 15%, but similar rates of anticoagulation resulted when put into practice.⁴ Further, evidence from implementation of similar apps for other conditions have shown modest time saving and excellent usability.⁵ The apps could reduce time for data aggregation, which is a measurable endpoint. Still, the current tools still require a “human in the loop” for validation and clinician judgement. As usage increases, healthcare systems need to be mindful of downstream effects on treatment rates and patient outcomes.