Decrease in unnecessary vitamin D testing using clinical decision support tools: making it harder to do the wrong thing

Abstract

Objective:To evaluate the impact of clinical decision support (CDS) tools on rates of vitamin D testing. Screening for vitamin D deficiency has increased in recent years, spurred by studies suggesting vitamin D’s clinical benefits. Such screening, however, is often unsupported by evidence and can incur unnecessary costs.

Materials and Methods:We evaluated how rates of vitamin D screening changed after we implemented 3 CDS tools in the electronic health record (EHR) of a large health plan: (1) a new vitamin D screening guideline, (2) an alert that requires clinician acknowledgement of current guidelines to continue ordering the test (a “hard stop”), and (3) a modification of laboratory ordering preference lists that eliminates shortcuts. We assessed rates of overall vitamin D screening and appropriate vitamin D screening 6 months pre- and post-intervention.

Results:Vitamin D screening rates decreased from 74.0 tests to 24.2 tests per 1000 members (P < .0001). The proportion of appropriate vitamin D screening tests increased from 56.2% to 69.7% (P < .0001), and the proportion of inappropriate screening tests decreased from 43.8% pre-implementation to 30.3% post-implementation (P < .0001).

Discussion:To our knowledge, this is the first demonstration of how CDS can reduce rates of inappropriate vitamin D screening. We used 3 straightforward, inexpensive, and replicable CDS approaches. We know of no previous research on the impact of removing options from a preference list.

Conclusion:Similar approaches could be used to reduce unnecessary care and decrease costs without reducing quality of care.

OBJECTIVE

Our primary objective was to evaluate the impact of clinical decision support tools on rates of inappropriate vitamin D testing.

BACKGROUND AND SIGNIFICANCE

Clinical decision support (CDS) tools embedded in electronic health records (EHRs) can improve care quality and reduce care costs by reducing the rates of clinically unnecessary medical procedure orders; multiple studies have illustrated this potential in diverse settings.^1–3 However, to our knowledge, no prior studies have assessed how CDS impacts rates of inappropriate vitamin D screening. Over the past decade, vitamin D screening has increased dramatically due to studies showing associations between vitamin D and the risk of fractures, falls, mortality, cardiovascular disease, diabetes, hypertension, and other disorders or conditions,^4–12 with a concordant increase in media attention to vitamin D supplementation.^13,14 The rate of outpatient visits in the United States associated with vitamin D deficiency tripled from 2008 to 2010, rising to 1177 visits per 100 000 people¹⁵; half of clinical laboratories surveyed reported that testing for serum 25-hydroxy vitamin D rose by at least 50% between 2008 and 2009.¹⁶

The majority of this vitamin D screening may be unnecessary. Several professional societies recommend against universal screening for vitamin D deficiency. The Endocrine Society does not recommend screening except in patient populations at risk of vitamin D deficiency.¹⁷ The US Preventive Services Task Force found insufficient evidence to support screening for vitamin D deficiency in the general population.^18,19 The American Board of Internal Medicine Foundation’s “Choosing Wisely” program, an initiative to reduce overuse of tests and procedures, recommends avoiding screening for patients at low risk of vitamin D deficiency.²⁰ Though not clinically indicated for the general population, screening for and treating vitamin D deficiency is recommended in specific patient populations, including patients with calcium or parathyroid disorders, malnutrition syndromes, chronic kidney disease, osteoporosis, and those who are on specific medications, especially steroids, specific antiepileptics, or certain HIV medications.^17,21,22

In light of these guidelines, in November 2014, a large (approximately 485 000-member) integrated group-model health care delivery system in the Pacific Northwest (United States) implemented a set of CDS tools designed to address rates of inappropriate vitamin D screening. The tools, described below, included a new organizational guideline on vitamin D screening, an alert triggered by a vitamin D test order requiring acknowledgment by the clinician (a ‘hard stop’), and a modification of physicians’ laboratory ordering preference lists that eliminated shortcuts to ordering the test. This paper presents an evaluation of the impact of implementing these CDS tools on overall rates of vitamin D screening, and on rates of appropriate versus inappropriate vitamin D screening.

MATERIALS AND METHODS

This study was approved by the Kaiser Permanente Northwest (KPNW) Institutional Review Board; patient data were deidentified according to review board regulations. Using data from the KPNW EHR, we identified a cohort of patients by searching for the CPT code for vitamin D testing (see Supplementary Appendix). We compared the rate of vitamin D screening among adult health plan members in the 6 months prior to implementation of CDS tools (May 1 to October 31, 2014) to the rate 6 months following this intervention (November 1, 2014, to April 30, 2015), using a repeated cross-sectional design. To account for potential seasonal variation in vitamin D screening, we also measured rates in the same 6 calendar months of the preintervention year (November 1, 2013, to April 30, 2014).

We searched for patient-specific risk factors for vitamin D deficiency using ICD-9 codes for common diseases associated with vitamin D deficiency (see Supplementary Appendix). We defined appropriate vitamin D screening as testing of patients with 1 or more of the following risk factors: hypo- or hypercalcemia, hypo- or hyperparathyroidism, malnutrition, inflammatory bowel disease, celiac disease, chronic kidney disease, osteoporosis, or long-term steroid, antiepileptic, or specific HIV medication use (see Supplementary Appendix). Disease diagnoses were identified as present if a patient had any of the ICD-9 diagnoses or conditions on the problem list at any time prior to the date of vitamin D screening. A patient’s medications were identified as those dispensed in the 90 days prior to receiving vitamin D screening. We summarized patient characteristics and proportion of patients with appropriate or inappropriate testing by intervention period.

Clinical decision aids

Three CDS tools were developed to address inappropriate vitamin D testing. All 3 CDS tools were implemented on November 1, 2014.

First, the health care delivery system developed a new regionwide guideline on vitamin D testing, supplementation, and treatment based on the most recent evidence. Such guidelines are routinely created and posted on the organization’s website. Clinicians were also notified that this new guideline was available via a informational email.

Second, an alert was created in the EHR that showed the ordering clinician specific information about the guideline any time a vitamin D test was ordered. The alert included a brief summary of the most important points from the new guideline and a link to the complete guideline. The alert was also “hard-stopped” by requiring the clinician to click again on the vitamin D order if he or she wished to continue with it. The order could be canceled by closing the alert.

Finally, vitamin D testing was removed from the laboratory ordering preference list of all clinicians except for endocrinologists, nephrologists, and orthopedists. While it was still possible to order a vitamin D test, removing it from clinicians’ preference lists eliminated the primary shortcut used to order the test. In practice, this meant it now took more clicks to order the test.

RESULTS

Characteristics of health plan members during the pre- and post-intervention periods are shown in Table 1. The mean (SD) age of adult members was 48.6 (17.8) and 48.7 (17.9) in the pre- and post-intervention period, respectively (P < .01). The majority of patients were Caucasian (89.4% pre- vs 89.7% post-intervention, P < .001), and approximately half of the members were women (52.9% in the pre- and post-intervention periods, not significant). Although age and race were statistically different in the 2 time periods given the large sample size, the relative differences are likely not clinically meaningful nor impactful on study results (Table 1).

Table 1.

Characteristics of the entire KPNW patient population by intervention period (from which the population receiving vitamin D screening was identified)

Demographic characteristics	Pre-implementation, n = 371 164	Post-implementation, n = 368 918	P value
Age (years; mean ± SD)	48.6 ± 17.8	48.7 ± 17.9	<.01
Female sex (%)	52.9	52.9	.38
Non-white (%)	10.6	10.3	<.001

In the pre-implementation period, 27 481 members were screened for vitamin D deficiency compared to 8939 members in the post-implementation period. Vitamin D screening rates decreased from 74.0 tests per 1000 members in the pre-implementation period to 24.2 tests per 1000 members in the post-implementation period (Figure 1). We saw no differences in testing rates due to seasonal variation (results not reported).

Figure 1.

Rate of vitamin D testing by intervention period.

Rates of appropriate vitamin D screening (ie, among patients with clinical risk factors for vitamin D deficiency) increased significantly in the post-intervention period compared to the pre-intervention period (Table 2). The proportion of vitamin D screening tests associated with a clinical risk factor increased from 56.2% pre-intervention to 69.7% post-intervention (Figure 2); conversely, the proportion of vitamin D screening tests that were not associated with a clinical risk factor decreased from 43.8% pre-intervention to 30.3% post-intervention (Figure 2).

Table 2.

Demographic and clinical characteristics of patients receiving vitamin D screening by intervention period

Demographic characteristics	Pre-implementation (n = 27 481)	Post-implementation (n = 8939) (%)	Significance, P value
Age (mean ± SD)	54.3 ± 17.3	55.1 ± 17.7	<.0001
Female	18 909 (68.8)	6339 (70.9)	<.001
Non-white race	3050 (11.1)	958 (10.7)	Not significant
Hypo- or hypercalcemia	981 (3.6)	533 (6.0)	<.001
Hypo- or hyperparathyroidism	546 (2.0)	384 (4.3)	<.001
Vitamin D deficiency or hypervitaminosis	10 042 (36.5)	4011 (44.9)	<.001
Malabsorption	1728 (6.3)	1005 (11.2)	<.001
Kidney disease	3151 (11.5)	1387 (15.5)	<.001
Osteoporosis	4139 (15.1)	1682 (18.8)	<.001
Antiepileptics	154 (0.56)	66 (0.74)	Not significant
Steroids	2044 (7.4)	793 (8.9)	<.001
HIV antivirals	144 (0.5)	100 (1.1)	<.001

Figure 2.

Appropriateness of vitamin D testing by intervention period.

We also measured the total number of times the alert fired and, after it fired, whether clinicians either continued with the order or cancelled the order during the intervention period. The rate of vitamin D screening that was still ordered even after the alert fired increased from 90.0% to 96.4% over the 6-month period; conversely, the rate of orders that were cancelled after the alert fired decreased from 10.0% to 3.6% (Table 3).

Table 3.

Continuation and cancellation rates of vitamin D ordering: post-intervention period

Month	Total times alert fired	Continued with order (after alert fired) (%)	Cancelled order (after alert fired) (%)
November 2014	1461	1315 (90.0)	146 (10.0)
December 2014	1584	1452 (91.7)	132 (8.3)
January 2015	1883	1787 (94.9)	96 (5.1)
February 2015	1738	1651 (95.0)	87 (5.0)
March 2015	1725	1661 (96.2)	64 (3.8)
April 2015	1713	1652 (96.4)	61 (3.6)

Of about 1200 clinicians with access to the guideline website, the older version of the internal vitamin D guidelines was accessed only 27.4 times per month in the pre-intervention period (June to October 2014). The revised guidelines were accessed 35.0 times per month in the post-intervention period (June to October 2015). Data in the immediate post-intervention period were not available (November to May 2015).

Finally, the cost of unnecessary testing significantly decreased from the pre-intervention period to the post-intervention period. Based on the average 2015 Medicare reimbursement rate of $40.29 for a vitamin D test, the per-month cost of vitamin D testing decreased from $184 534 to $60 025, an estimated annual cost savings for the health care delivery system of $1.4 M. This estimate is likely higher than the true savings to the health plan, since Medicare reimbursement rates are usually less than published reimbursement rates. Exact cost data were not available for this analysis.

DISCUSSION

Implementation of the set of CDS tools discussed above was associated with significantly reduced overall rates of vitamin D screening and a significant increase in the proportion of ordered vitamin D screening tests that were clinically appropriate. These results support the Institute for Healthcare Improvement’s triple aim of increasing quality, increasing patient-centered care, and decreasing cost.²³ They are also timely given the current “Choosing Wisely” initiative, led by the American Board of Internal Medicine, which seeks to reduce overuse of “low-value” services and health care costs.²⁰

Substantial previous research demonstrates that CDS can improve care processes and quality,^{1–3,24–29} especially when it is provided at optimal moments during clinic workflows and involves specific care recommendations with supporting evidence.^30,31 Much of this evidence involves using CDS to improve the provision of appropriate care, but evidence also shows that CDS can reduce rates of inappropriate care, such as redundant or otherwise unneeded lab tests or procedures unsupported by evidence.^1,32 The results presented here add to this literature as follows. First, to our knowledge, this is the first demonstration of using CDS tools to reduce rates of inappropriate vitamin D screening. Second, we demonstrate how this impact was achieved via application of 3 straightforward, inexpensive, and replicable CDS approaches. Third, while 2 of these CDS strategies, providing links to relevant clinical guidelines or an alert when an order might be inappropriate, have been well demonstrated as effective in prior literature, the third strategy, removing options from a preference list, has been evaluated less often; we know of no prior research on the impact of this strategy.

The 3-part CDS intervention was not implemented in a manner that enabled definitively identifying which aspects drove the observed decreases in vitamin D screening orders. However, some patterns are worth mentioning. The low absolute number of times that the new internal guideline was accessed in the pre- and post-intervention periods makes it unlikely that education alone changed clinician behavior. The percentage of clinicians who accepted the alert, ie, backed out of ordering vitamin D screening after the alert fired, decreased in the post-implementation period from 10% to 4%; some clinicians may have “learned” experientially when vitamin D testing was not indicated and ordered it less. Thus, given the low absolute numbers of clinicians accessing the guidelines and relatively low clinician responses to the alert, we posit that removing the vitamin D test from providers’ preference lists had the greatest effect.

Our study has some important limitations. The tools described here were implemented in an integrated care setting where adherence to organization-wide priorities about standards of care, and changes to the EHR to support these changes, are part of the organizational culture. Our results may not be generalizable to non-integrated care settings, or where similar changes to the EHR system are not feasible.

An interrupted time series design would be optimal for assessing the effectiveness of this intervention. However, in this retrospective, descriptive analysis of an internal quality improvement initiative, only 6 months of post-intervention monthly time points were available, yielding insufficient power for an interrupted time series analysis. Future research should involve more formal assessment using an interrupted time series design with data for at least 12 post-intervention months.

Providers in a private care setting may be wary of refusing to order vitamin D screening if patients request it, and patient requests are a known major driver of inappropriate care. In settings where this is less of a concern, CDS changes like those described here could have an even greater impact on rates of inappropriate care. In addition, data are from patients in the Pacific Northwest, where vitamin D deficiency is endemic and it is recommended that vitamin D be taken by the general population. This geographic difference may also affect the generalizability of our results, as the likelihood of having both vitamin D deficiency and inappropriate screening may differ in areas with more vitamin D exposure.

Given the large absolute decrease in vitamin D testing, there may have been patients who were appropriate for testing but were not tested after the test was removed from providers’ preference lists. We were not able to test this; future studies should specifically address issues related to removing tests from provider preference lists.

Lastly, this is a descriptive study, and so cannot establish a causal link between the CDS changes and the reduction in inappropriate vitamin D testing described here. We do not know definitively which aspects of the 3 CDS changes were the greatest drivers of the impact seen. Further research is needed to identify the types of CDS that are most effective at decreasing unnecessary testing, while not compromising the quality of care delivered.

CONCLUSION

Our findings describe how a set of inexpensive, easily implemented CDS changes greatly reduced rates of inappropriate vitamin D testing in an integrated health plan. Our demonstration that a simple set of tools reduced rates of unnecessary screening has relevance for other delivery systems as they address inappropriate vitamin D screening, and potentially other unnecessary procedures as well.

FUNDING

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector.

COMPETING INTERESTS

The authors have no competing interests to declare.

CONTRIBUTORS

Each author has met the authorship criteria established by the International Committee of Medical Journal Editors, including: (1) made substantial contributions to the conception or design of the work or the acquisition, analysis, or interpretation of data for the work; and (2) drafted the work or revised it critically for important intellectual content; and (3) gave final approval of the version to be published; and (4) agreed to be accountable for all aspects of the work, ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

AHF designed and implemented the intervention, contributed to all analyses, and wrote sections of the manuscript. RG and DMM contributed to analysis design and interpretation and wrote sections of the manuscript. ABS conducted the statistical analyses and wrote sections of the manuscript.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.