Detecting pregnancy use of non-hormonal category X medications in electronic medical records

Abstract

Objectives

To determine whether a rule-based algorithm applied to an outpatient electronic medical record (EMR) can identify patients who are pregnant and prescribed medications proved to cause birth defects.

Design

A descriptive study using the University of Pennsylvania Health System outpatient EMR to simulate a prospective algorithm to identify exposures during pregnancy to category X medications, soon enough to intervene and potentially prevent the exposure. A subsequent post-hoc algorithm was also tested, working backwards from pregnancy endpoints, to search for possible exposures that should have been detected.

Measurements

Category X medications prescribed to pregnant patients.

Results

The alert simulation identified 2201 pregnancies with 16 969 pregnancy months (excluding abortions and ectopic pregnancies). Of these, 30 appeared to have an order for a non-hormone category X medication during pregnancy. However, none of the 30 ‘exposed pregnancies’ were confirmed as true exposures in medical records review. The post-hoc algorithm identified 5841 pregnancies with 64 exposed pregnancies in 52 569 risk months, only one of which was a confirmed case.

Conclusions

Category X medications may indeed be used in pregnancy, although rarely. However, most patients identified by the algorithm as exposed in pregnancy were not truly exposed. Therefore, implementing an electronic warning without evaluation would have inconvenienced prescribers, possibly hurting some patients (leading to non-use of needed drugs), with no benefit. These data demonstrate that computerized physician order entry interventions should be selected and evaluated carefully even before their use, using alert simulations such as that performed here, rather than just taken off the shelf and accepted as credible without formal evaluation.

Under the US Food and Drug Administration's (FDA) classification of pregnancy risk drugs, category X medications have demonstrated fetal abnormalities in animal as well as human studies, and there is evidence of fetal risk based on human experience. Consequently, the risk of using these medications in pregnant women clearly outweighs any possible benefit.¹ As such, category X medications are contraindicated in women who are or may become pregnant. The inappropriate use of category X medications in pregnancy^2–5 would thus constitute a medical error with the potential to inflict harm on the fetus, resulting in the potential for spontaneous abortions or premature delivery and mental and physical disabilities in children exposed to the category X medications in utero.

Background

One approach proposed to address potential drug-related medical errors has been the use of computerized physician order entry (CPOE)^6–9 combined with clinical decision support (CDS). The most commonly discussed CDS intervention is an ‘alert’ or pop-up window that appears in front of the user and often interrupts the task at hand. However, although CDS has been shown to affect physician prescribing patterns and rates of drug-associated medical error significantly,^10–13 the use of unnecessary CPOE interventions may negatively impact clinical practice, potentially facilitating medical errors. Indeed, CPOE systems can produce drug-related errors^{8 14 15} either by failing to prevent the prescribing of contraindicated medications, or because the effect of computer-based reminders declines over time, leading providers to override the computer-based recommendations for drug–drug interactions.^16–18 Furthermore, the less accurate and useful the messages delivered by the CPOE system, the more likely clinicians are to view the entirety of CDS as an annoyance, so that when a truly important warning message does come up, the provider ignores it.¹⁷ This phenomenon has been called ‘alert fatigue’. Also, if an alert intervention fires at an inappropriate time, the provider may actually act on it, resulting in inappropriate care being given to the patient.

For a CPOE intervention to be effective, it must have a reasonable degree of specificity. For example, alerts on category X drugs designed to activate for all women of childbearing age would be sensitive because they would fire for all patients who were truly pregnant. However, the number of false-positive alerts would be extremely high. Physicians viewing these alerts would quickly find them of little value and would begin to ignore them, even when the alerts are valid. In contrast, an alert designed to activate for women with a documented pregnancy diagnosis would be more specific because the pregnancy diagnosis in the record would be highly likely to predict actual pregnancy in the patient. Yet, many pregnancies may be missed if patients get obstetric care elsewhere or if a pregnancy is not recorded in the electronic medical record (EMR). These possibilities need to be evaluated for any alert that is being considered.

As the consequences of inappropriate prescribing of category X medications to pregnant women can be serious for the fetus, we considered the role of a novel CDS intervention within CPOE to prevent this problem. The current investigation was therefore designed as a methodological study to test an EMR algorithm to detect the prescribing of category X medications to pregnant patients.

Research question

The study aimed to address the question of whether a rule-based algorithm can be used in an outpatient EMR to detect the diagnosis of pregnancy and the prescribing of category X medications to patients who are pregnant.

Methods

Data source

In 1998, the University of Pennsylvania Health System (UPHS) began implementing an ambulatory EMR system known as EpicCare.¹⁹ Patient-level information includes demographics, current and past medical histories, a current ‘problem list’, medication list, and allergy detail. Encounter-level documentation includes progress notes, visit orders and diagnoses, results, level of service, discharge data, and vital signs. EpicCare is the most widely used EMR software in the USA. There were relatively few UPHS EpicCare visits before 1999; there has been a marked increase in visits as more practices have started using EpicCare in recent years. The Department of Obstetrics and Gynecology at the Hospital of the University of Pennsylvania did not begin participating in EpicCare until after the completion of our study period. However, all primary care practices were included in the initial UPHS installation, as were many specialty practices. In total, UPHS EpicCare now includes approximately 100 practices, 2.5 million patients, and 600 000 office-visit encounters.

Study subjects

All female patients who sought medical care in EpicCare-utilizing UPHS outpatient offices during 1999–2007 were eligible for inclusion in the study. Operationally, we had pregnancy data (diagnoses, procedures indicating birth, relevant laboratory data, or medications) from systems other than EpicCare for these women. However, if the date associated with one of those indicators for a patient was outside the range of time that a patient was seen in EpicCare, that pregnancy was not counted.

Study design

We designed a study to assess the ability of a rule-based computer algorithm to identify in the EpicCare database women who are pregnant, and identify orders for category X medications during pregnancy. The study was approved by the University of Pennsylvania institutional review board, with waivers of the requirement for informed consent and the Health Insurance Portability and Accountability Act (HIPAA) authorization.

Defining category X medications

Category X medications were determined by FDA risk classification listings in the 2005 edition of Brigg's ‘Drugs in pregnancy and lactation’ and cross-checked in the 2005 Physician's Desk Reference.^{1 20} For ease of presentation, category X medications were grouped into drug classes (3-hydroxy-3-methylglutaryl coenzyme AA reductase inhibitors; ACE inhibitors and angiotensin II receptor blockers; estrogens and progestins; sedatives; anticoagulants; anti-neoplastics; anti-rheumatics; and vitamin A derivatives).

In this investigation we chose to study only non-hormonal category X medications in order to improve the specificity of our algorithm; hormones are often prescribed after abortions or after delivery. If the pregnancy window as determined by the computer algorithm was off by a week or two from the actual pregnancy duration, it could appear that the patient had a category X exposure during pregnancy when in fact it was outside of the pregnancy window. Oral contraceptives may even be prescribed during the last few weeks of pregnancy with instructions to start use after delivery. Again, if we included these drugs, it could appear that the patient had actually been prescribed a category X exposure for use during pregnancy. To avoid this confusion, we studied only non-hormonal category X medications.

Defining pregnancy

Markers for potential pregnancy or plans to become pregnant included: (1) the presence of an International Classification of Diseases Clinical Modification (ICD9 CM) pregnancy diagnosis; (2) billing V codes for prenatal care visits; (3) positive serum or urine human chorionic gonadotropin (hCG >25); (4) orders for ultrasound testing for determination of gestational dates or fetal anatomy; and (5) orders for prenatal vitamins.

Algorithm for defining ‘exposure’ to category X medications during pregnancy

‘Exposure’, for the purposes of this study, is defined as an order for a category X medication during pregnancy. Ideally, pregnancy would be defined as a time window starting backwards from the date of delivery to the date of conception. However, this rule would be too late for a CPOE intervention because of the need to detect pregnancy before delivery. Unfortunately, there was no single source of queryable documentation indicating the estimated date of conception in use in the EpicCare records during the time of our study. A time window that included the 9 months before the pregnancy-defining event and the ensuing 9 months would capture all time at risk for pregnant women to be exposed to category X medications, but would obviously result in large numbers of false-positive findings as half of this temporal window would not be a relevant pregnancy-associated risk period. Therefore, we used two computer algorithms, each seeking to minimize the false-positive time interval before the pregnancy was known, or after the pregnancy was over. One algorithm was a simulation of potential alerts, attempting to predict or anticipate a pregnancy by searching first for an hCG greater than 25 or ICD-9 or V code suggestive of pregnancy and then prospectively searching for a confirmatory marker of pregnancy termination. A second algorithm was a post-hoc analysis based on identifying pregnancy endpoints (post-ectopic, post-abortion, or post-partum marker) and then assigning a start date retrospectively to define a pregnancy window (see table 1 illustrating these two algorithms). In effect, the alert simulation algorithm would identify the patients for whom a proposed alert might fire. In contrast, the post-hoc algorithm would identify patients for whom an earlier alert would have been desirable, as it was based on after-the-fact identification of exposed pregnancies, more complete but not useful to trigger alerts.

Table 1.

Risk exposure windows for the alert simulation algorithm and for the post-hoc algorithm

Alert simulation algorithm (using a prospective strategy)	Post-hoc algorithm (using a retrospective strategy)
If the first observation marker is hCG >25 or ICD-9 or V code indicating pregnancy but without a start date, then	Search for the first pregnancy termination marker (post-partum/abortion/ectopic).
Assume this is the pregnancy start date, and search forward for a pregnancy termination marker (post-partum/abortion/ectopic).	If post-partum/delivery, count backwards 9 months to determine the beginning of pregnancy.
The length of the pregnancy window is the variable duration between date of first observation marker and date of pregnancy termination marker.	If post-abortion, count backwards 5 months to determine the beginning of pregnancy.
If no termination marker available, then add 9 months forward from the first observation marker.	If post-ectopic, count backwards 2 months to determine the beginning of pregnancy.

hCG, human chorionic gonadotrophin; IDC, International Classification of Diseases.

The rules for the alert simulation algorithm specified the following (see also supplementary appendix A, available online only):

1. A pregnancy window was constructed using the date of the first indicator of pregnancy as the start date and the date of the pregnancy outcome as the end date. If the first record for a woman was either hCG greater than 25 (which alone cannot be used to date pregnancies) or an ICD-9 or V code definitely indicating pregnancy, we continued to search forward until we encountered a code for ectopic pregnancy within 2 months after the first indicator of pregnancy, or a code for abortion within 5 months after the first indicator of pregnancy, or a code for post-partum care within 9 months after the first indicator of pregnancy to define the potential ‘risk of exposure period’ during pregnancy. We did not search for delivery codes because those are generally inpatient codes. We looked instead for post-partum codes in the outpatient setting.

2. If either of these three pregnancy outcomes (ectopic, abortion, post-partum) were encountered, we stopped the search and counted this as the first pregnancy. Then, we looked for the next indicator of a new pregnancy that occurred more than 60 days after the first ectopic, abortion, or post-partum code.

3. If none of these three outcomes were found, we searched for other pregnancy markers (either an ultrasound, prenatal vitamins, or a V code viable as an indicator of the earliest date for the pregnancy), and we defined the ‘risk of exposure period’ for this pregnancy as starting with the first date of the marker and ending 9 months forward from that marker. Then we again looked for the next indicator of pregnancy that occurred more than 60 days after the end date assigned to this pregnancy.

4. If we could find none of these three outcomes (ectopic, abortion, post-partum) to indicate the end of a pregnancy, and also could not find any of the other markers as described in (3) above, we discarded the initial marker described in (1) above and continued to search for a subsequent indicator of pregnancy.

Validation of pregnancy

The medical records of a sample of 605 women classified by the algorithm as ‘non-pregnant’ were reviewed electronically to check for possible misclassification of the pregnancy status based on our computer algorithm. This sample size was selected so that with 0 observed events, the 95% one-sided exact CI for the proportion of missed pregnancies would exclude a proportion as high as 0.5%. The electronic charts of the 605 patients were reviewed to examine whether there were indications of pregnancy. The review was performed by two third-year medical students, using guidelines and supervision provided by one of the co-authors (JJ).

Validation of exposure

The medical records of all women classified by the two algorithms as having an exposure to non-hormonal category X medications during pregnancy were reviewed manually by one of the co-authors (JJ) to determine if these were true exposures.

Analysis

The primary outcome of the study was the number of unique orders for category X medications detected by our algorithms as occurring during a defined pregnancy for women seeking medical care in UPHS outpatient offices. This was calculated by dividing the number of pregnancies classified by the algorithms as being exposed to category X medications by the total months at risk of getting a category X medication while pregnant. A Poisson distribution²¹ was used to calculate the incidence and exact 95% CI, separately for non-hormonal category X drugs and for hormonal category X drugs, as well as for individual classes of these drugs.

Because miscarriages and abortions are likely sources of false positives, as patients are likely to meet query-based criteria for pregnancy and are likely to be given category X medications in the process of pregnancy termination, and they are not relevant to the design of alerts, we excluded such potentially false-positive events.

Analyses were conducted using Stata 10.

Results

Using the alert simulation algorithm (ie, the prospective strategy), there were 2201 pregnancies among patients seen in EpicCare practices, with 30 having had an apparent exposure to a non-hormone category X drug during pregnancy. These 2201 pregnancies consisted of 1405 full-term pregnancies and 796 definitely pregnant either by hCG greater than 25 or a pregnancy diagnosis code that we deemed usable as the earliest date. Calculating total pregnancy months yielded 16 969 risk months, with an apparent incidence of exposure to non-hormone category X drugs of 1.77 (95% exact CI 1.19 to 2.52) per 1000 months of pregnancy (see table 2). A review of the medical records of the 30 pregnancies identified by the alert simulation algorithm as having had an order for non-hormonal category X medications showed that none of these were true ‘exposures’. All had a positive hCG, were confirmed to be pregnant, and then had a negative hCG in the system before the category X exposure; there was a missing code for an abortion that is evident in the text of the medical record. This was the source for these false positives detected by the algorithm.

Table 2.

Incidence of apparent exposure to category X medications during pregnancy (excluding abortions and ectopic pregnancies), based on the computer algorithm classifications of exposure and estimates of pregnancy onset and duration

Exposure during pregnancy	Alert simulation algorithm (using prospective strategy)		Post-hoc algorithm (using retrospective strategy)
	No of exposed pregnancies (and 16 969.3 person-months)	Poisson exact (95% CI) incidence per 1000 months of pregnancy	No of exposed pregnancies (and 52 569 person-months)	Poisson exact (95% CI) incidence per 1000 months of pregnancy
All hormonal category X medications*	52	3.06 (2.29 to 4.02)	56	1.07 (0.80 to 1.38)
All non-hormonal category X medications†	30	1.77 (1.19 to 2.52)	64	1.22 (0.94 to 1.55)
Endocrine medications	17	1.00 (0.58 to 1.60)	24	0.46 (0.29 to 0.68)
Anti-ulcer medications (all misoprostol)	9	0.53 (0.24 to 1.01)	0	–
ACE inhibitors	4	0.24 (0.06 to 0.60)	20	0.38 (0.23 to 0.59)
Statins	2	0.12 (0.01 to 0.43)	13	0.25 (0.13 to 0.42)
Anticoagulants	2	0.12 (0.01 to 0.439)	6	0.11 (0.04 to 0.25)
Antineoplastic	1	0.06 (0.00 to 0.33)	2	0.04 (0.0 to 0.14)
Anti-rheumatic	0	–	1	0.02 (0.0 to 0.11)
Retinoids	0	–	2	0.04 (0.0 to 0.14)
Quinine	0	–	2	0.04 (0.0 to 0.14)
Fenofibrate	0	–	1	0.02 (0.0 to 0.11)

Other drug classes examined included sedatives, migraine medications, and the drug ribavirin, but none of the study subjects had been exposed to these.

^*Includes contraceptives, estrogen, progesterone, androgen-anabolic drugs.

^†The result for all non-hormonal category X medications in abortions and ectopic pregnancies only was nine exposures with 558.8 person-months, and an incidence of 16.11 (7.36 to 30.57) per 1000 months of pregnancy.

Using the post-hoc algorithm (ie, the retrospective strategy), there were 5841 full-term pregnancies among patients seen in EpicCare practices, with 64 of these having had a non-hormone category X drug exposure during pregnancy. Based on 52 569 risk months, the apparent incidence of exposure to non-hormone category X drugs using this algorithm was 1.22 (95% exact CI 0.94 to 1.55) per 1000 months of pregnancy (see table 2).

Of the 64 ‘exposed’ pregnancies identified by the post-hoc algorithm, 22 were not confirmed to be pregnant by chart review. These were instances of incorrect coding by the clinician or the algorithm picking up correct coding that did not indicate that a patient was currently pregnant; eg, a patient with an item on her problem list of ‘post-partum pulmonary embolism’ that happened in 1984 rather than when it was recorded post-partum in 2003. One was confirmed to be a true exposure during pregnancy. Of the remaining 41 confirmed pregnancies, the chart review found that the patients were not prescribed category X medications during pregnancy for one of several reasons: (1) For the majority of patients, the category X medication was prescribed before the start of pregnancy for patients who delivered early; recall that the window in the post-hoc algorithm was determined by picking an endpoint and subtracting a specific time period to define the full pregnancy window. Most of these were in fact fertility medications (eg, clomiphene), clearly given before the pregnancy. (2) For several patients, the prescription was entered during the pregnancy window, often as a historical medication for previous existing conditions, but the documentation indicated that the patient was not taking the medication while pregnant (eg, a patient with rheumatoid arthritis who had her arthritis medications changed because she became pregnant). (3) A small number of patients had decided to proceed with an abortion, and opted to continue the category X medication with medical advice.

Only 2117 pregnancies or six ‘exposed pregnancies’ were identified by both algorithms. Also, there were 5454 pregnancies or 144 ‘exposed pregnancies’ identified by the post-hoc algorithm and not by the alert simulation algorithm, the population that might be thought of as false negatives for the alert simulation algorithm.

Finally, to test for possible misclassification of pregnancy (ie, the possibility that based on our computer algorithm we were missing pregnancies in the group that we classified as ‘not pregnant’), we also undertook a review of the records of a sample of 605 cases classified by the algorithms as ‘non-pregnant’. No pregnancy cases were found. This review confirmed 72% (433/605) as truly not pregnant; pregnancy status could not be determined for the remaining 172 cases because they were missing laboratory values or the notes were unclear.

Discussion

Our study has shown that category X medications are indeed used in pregnancy, although rarely. However, our alert simulation algorithm was extremely inefficient in identifying such exposures. One potential solution might be the addition of an accurate delivery date estimate in the EMR. Whether that is feasible, and how well it would work, would need to be evaluated. The medical chart review revealed a weakness in our alert simulation algorithm: patients with a positive hCG, who were confirmed to be pregnant and then had a negative hCG in the system before the category X exposure (there was a missing code for abortion). If we had used the negative hCG as an endpoint, that would have been helpful, and explained all the false positives reviewed.

Several surveys documented similar unwarranted exposures to these contraindicated high-risk drugs during pregnancy. Among women enrolled in Tennessee's TennCare program for Medicaid enrollees and individuals without health insurance during 1995–9, the exposure rate to category X prescriptions during pregnancy was 4.1 per 1000 births (n=391 women), with nearly two-thirds of these women filling prescriptions after clinical signs or pregnancy tests indicating pregnancy.² Andrade et al³ reported a prevalence of exposure to category X drugs of 1.1% (1653 women) after the initial prenatal care visit, based on data from eight health maintenance organizations during 1996–2000. A prevalence of 3.9% (724 women) was reported by Wen et al,²² based on Saskatchewan data during 1997–2000. Using General Practice Research Database data in the UK during 1991–9, Hardy et al⁴ reported that 0.6% (n=501 women) were prescribed category X medications during early pregnancy (defined as a woman's earliest identified pregnancy record plus 70 days). As shown by Raebel et al²³ in the Colorado Kaiser Permanente health maintenance organization, even using a computerized alert to pharmacists when pregnant patients were prescribed category X medications did not completely prevent such dispensings: the incidence was 0.9% (n=54) among the intervention women and 1.2% (n=58) among women receiving usual care. Of note, each of these papers used a different approach from ours and from the others, but each could have resulted in an overestimate of the true rate of category X prescribing in pregnancy, as ours did before we reviewed the medical record in detail. One was very similar to our post-hoc analysis, using claims data and counting backwards 270 days, and without the detailed medical record review.³ Another was similar, but censored the retrospective search for exposures based on the last menstrual period recorded on the birth certificate.² Another was similar, also censoring the backward review, this time based on notations of gestational age.²² Another was different, using computerized medical records and separately examining 90 days before the first notation of pregnancy and also the 70 days thereafter.⁴ Of importance, none went back to medical records to see if the drug in question was intended for use during pregnancy. The only intervention study²³ was stopped early because of multiple false-positive alerts.

There are several limitations to consider in our study. One limitation with the data is that only hCG greater than 25 and pregnancy diagnosis or billing codes could ultimately be used to date the start of pregnancy. Other markers such as ultrasound and prenatal vitamins could suggest a pregnancy but were not sufficient to date the start of pregnancy, so these markers did not contribute to the final classification.

It is important to note that the EpicCare EMR does have a section that allows for the input of obstetric history including pregnancy status (a ‘check box’ that indicates a patient is currently pregnant). This information was not utilized in our algorithm as it is not regularly completed by the departments participating in EpicCare. The Department of Obstetrics and Gynecology did not participate in EpicCare until after the completion of our study period. One topic for further study would be to cross-reference our algorithm with the pregnancy status indicator utilized in EpicCare. However, our hypothesis was that the pregnancy status indicator would serve to augment but not replace the algorithm. Utilizing the ‘currently pregnant’ check box in the absence of a secondary algorithm could lead to two errors: missed pregnancies if the patient gets her obstetric care outside the EpicCare system, and false exposures if the ‘currently pregnant’ check box is not unchecked immediately after delivery. This remains to be evaluated in future work.

Ideally, we would have incorporated text data from sources such as the problem list or progress notes in the algorithm. Specialized methods of text mining, including latent semantic analysis, keyword identification, Bayesian classifiers and networks, and term co-occurrence have been used to extract information from unstructured text. The research that has been done in this area^24–28 indicates that incorporating such data in CDS systems can significantly improve their performance. These approaches should be considered in future work in developing and evaluating CDS-based alerts in domains where clinical conditions and therapeutic exposures are not easily ascertained in structured EMR.

Another limitation is the conservative nature of the cases of category X medication utilization during pregnancy that are identified using the UPHS EpicCare electronic database. It seems likely that women who are pregnant but receiving their obstetric care outside of UPHS may seek ambulatory care within the UPHS system in the form of referred or primary care. The EpicCare database may not be able to capture the pregnancy status for all of these individuals because the pregnancy diagnosis may not be reliably recorded in the EMR. Second, one would assume that the vast majority of category X medications would be prescribed before knowledge of the pregnancy status of the patient. Therefore, pregnancies affected by category X medications may result in miscarriage that remains unidentified by the pregnancy and category X medication queries used by our algorithm. Both of these issues could artificially decrease the estimate of the rate of use of category X medications in pregnancy. On the other hand, category X prescriptions with instructions not to use until after delivery, could artificially increase the estimate of the rate of use of category X medications in pregnancy because these will be identified by the algorithm as ‘exposures’ during pregnancy when in fact there may be no exposure in utero to these drugs.

In addition, the rules used by the post-hoc algorithm to date the start of pregnancy (eg, subtracting 9 months from the date of a post-partum code, subtracting 5 months from the date of a post-abortion code, subtracting 2 months from the date of a post-ectopic date) were crude, and probably contributed to misclassification of some windows of exposure to the study drugs.

Another limitation is that the algorithm is based on expert knowledge of how pregnancy or medication exposure would be detected and the subsequent classification of the patient as pregnant or exposed to a category X medication. This is a commonly used approach in knowledge-based systems and algorithms, but in this approach it is possible that other detectors could have been missed. Future work should address this limitation through the use of machine learning-based algorithms such as naive Bayes classifiers, decision tree induction and rule discovery methods to identify candidate pregnancy and medication exposure features in the EMR data.

Finally, a large limitation of this effort was the low incidence of the use of category X medications in pregnancies. This means we were unable to determine whether our prospective algorithm would have successfully detected such exposures. Therefore, we could quantify falsely positive alerts, but there were few opportunities for falsely negative alerts.

In the context of considering a computerized tool such as an alert to avert the prescribing of high-risk category X medications to pregnant women, there are conceptually two very different situations to note. The first is a retrospectively determined pregnancy, for example, allowing us to say that someone was pregnant within the 9 months before a known delivery. Any exposure during this window represents a true exposure, but if the pregnancy existed before supporting data were available in the EHR, the exposure could not be rendered avoidable using CPOE interventions. In fact, many of the category X exposures in pregnancies observed in this study were related to the pre-existing use of a category X drug in a woman who, during the course of therapy after the initial order, became pregnant. The second is a prospective warning that can only be triggered using data on pregnancy that are already available at the time the category X drug is prescribed. Because this approach misses exposures in the early part of pregnancy, this latter number is likely to be much lower than the first. None of the 30 putative category X exposures during pregnancies detected by our alert simulation algorithm were true exposures. The question is ultimately whether this exposure is low enough to justify not providing the warning based on the presumption that if the rule fires, it is so likely to be a false positive that the noise will lead to alert fatigue.

Conclusion

Non-hormonal category X medications are indeed used in pregnancy, although rarely. Our study demonstrated that identifying pregnancy is difficult, but possible. However, most patients identified as exposed in pregnancy were not truly exposed during the vulnerable periods. As such, had we put in place an electronic warning as originally designed, it would have inconvenienced some physicians (given how common these drugs are), and possibly hurt some patients (denying access to drugs they needed). Also, the potential for increased alert fatigue in the providers might have resulted in inappropriate care being given to the patient. These data demonstrate that CPOE interventions should be selected and evaluated carefully even before their use, using alert simulations such as that performed here, rather than just taken off the shelf and accepted as credible without formal evaluation.