Improving Adherence to Research Protocol Drug Exclusions using a Clinical Alerting System

Abstract

Objective

To develop a general method for using the alerting function of an electronic health record (EHR) system to warn prescribers when a drug order may be in conflict with the restrictions of a patient’s research protocol.

Methods

We examined a sample of clinical research protocols at the National Institutes of Health (NIH) to identify the frequency with which drugs were excluded by protocols. We analyzed two protocols and modeled the exclusions they contained. We then developed a data model to represent the exclusions, expanded the terminology in the NIH’s Biomedical Translational Research Information System (BTRIS) to include relevant drug concepts, and wrote a medical logic module (MLM) for the EHR to match terms for ordered drugs with the drug concepts in the protocol.

Results

We found that 50% of protocols in our sample included drug exclusions. Our model represented exclusion concepts and also concepts related to exemptions from the exclusions. The MLM was deployed in a test environment where it successfully detected orders for excluded drugs and delivered messages to users explaining the exclusion, providing information about the clinical setting and timing where the exclusion applies. BTRIS reports using the same terminology information were able to identify instances where protocol exceptions occurred.

Conclusions

Drug exclusions are frequent components of research protocols; nonadherenece to these exclusions could result in harm to subjects, erroneous study results or inefficiencies due to disqualification of research subjects. Our approach uses an MLM and a simple knowledge base, together with a controlled terminology, to provide a solution to the detection and prevention of possible protocol violations. Further work is needed to model additional aspects of the exclusions, such as timing and co-occurring conditions, to improve MLM accuracy.

Introduction

Prospective clinical research invariably requires a predetermined protocol for recruitment, interventions and data collection. The protocol will specify such things as the method for randomizing subjects to various intervention groups, the intensity and timing of specific treatments, and the types and timing of specific diagnostic procedures.¹ For example, a study might require a member of one group of subjects to receive a study drug on a particular day during the subject’s participation in a drug trial, while requiring a member of another group to receive a placebo on that day; both types of subjects might then have a blood test performed some specified number of days later. Deviations from the prescribed plan in the protocol are an anathema to high-quality clinical research. For example, errors in drug dosing might endanger the health of the subjects, confound the analysis of data subsequently collected, or even render a subject ineligible to continue in the protocol, at potentially great cost to the subject (inconvenience and needless risk), researcher (time and reputation), and society (wasted money and delayed results).

One common type of specification in a research protocol, called an exclusion, restricts the subjects’ activities, such as diet, exercise, additional drugs, or alternative therapies. A simple example is the exclusion of the use of a particular drug from use in a drug trial because it is known to interfere with some action of the study drug. Depending on the protocol, the number of exclusions might be quite large. The dependence on the memory of researchers to always recall all exclusions in a protocol will invariably lead to lapses, resulting in protocol nonadherence, with all of its consequences.

We are studying the use of the clinical alerting component of an electronic health record (EHR) system to improve the quality of research. We have previously created a clinical alert, in the EHR at the NIH Clinical Center, to reduce inconsistencies in recording research data (subject heights and weights recorded by nurses)² and subsequently studied the impact on research workflow (including data quality and the nurses’ responses to true-positive and false-positive alerts).³ We hypothesize that alerts might also be used to prevent researchers from inadvertently ordering protocol-excluded drugs and that this can be accomplished in a generic, easy-to-maintain manner that will nevertheless be powerful enough to provide alerts that are precise in their appropriateness and effect.

Background

The Clinical Research Information System (CRIS) of the NIH Clinical Center

The NIH Clinical Center, a 240-bed inpatient and outpatient medical center located on the NIH main campus in Bethesda, Maryland, is the setting for most of the clinical research that takes place on that campus. In 2004, the Clinical Center installed a Clinical Research Information System (CRIS) that is based on a commercial EHR (Sunrise Clinical Manager, Allscripts, Chicago, IL). CRIS includes an alerting function that uses medical logic modules (MLMs)⁴ that can be tailored to a variety of functions, such as assessing physicians’ orders, warning about abnormal test results, reminding users of necessary actions, and analyzing data entry values for consistency.

One important application of the MLMs is to detect interactions between drugs the subject is currently taking and new drugs that are being prescribed through the order entry system. Rather than creating one MLM for each pair of drug-drug interactions, a single MLM draws on a commercial knowledge base of interactions (Lexicon, Multum, Denver, CO) to identify interactions that involve the ordered drug and each of the subject’s current drugs. However, there is no analogous knowledge base for protocols and their excluded drugs. Although it is technically possible to create an alert for each exclusion (e.g., “If subject is on Protocol XYZ and order is for drug ABC”), the effort required to create, test and manage MLMs needed for the over 1400 active protocols at NIH would be prohibitive.

Based on preliminary investigations into the representation of protocol exclusions (unpublished data), we knew that exclusions would often take the form of drug classes, including classes based on ingredients (e.g., “drugs containing aspirin”), therapeutic intent (e.g., “chemotherapy agents”) and pharmacologic effect (e.g., “cytochrome P450 inhibitors”). CRIS has a controlled terminology of all orderable drugs but has only partial information about their classification. Management of all of the classes used in protocol exclusions, and especially the maintenance of those classes (with an average of ten new drugs added to the Clinical Center formulary weekly) compounds the problem of maintaining drug-exclusion MLMs. In addition, the classification scheme in CRIS is limited to a strict hierarchy so that even if additional classes could be added, they would be necessarily incomplete (e.g., for example, drugs containing ritinovir would need to be added to the classes Antiretroviral Agents and Cytochrome p450 Inhibitors⁵).

We therefore sought an alternative approach that would involve only a small number of MLMs, would require only a minimal amount of knowledge about each exclusion in each protocol, and function independently from changes in the formulary of the Clinical Center’s pharmacy.

The Biomedical Translational Research Information System (BTRIS) and the Research Entities Dictionary (RED)

The Biomedical Translational Research Information System (BTRIS) is a new resource at the NIH that comprises a repository of clinical research data from multiple NIH institutes and centers (including the Clinical Center) and a reporting system that allows NIH researchers to retrieve identified data from their active protocols or search for deidentified data across all protocols.⁶ A key component of BTRIS is the Research Entities Dictionary (RED), which serves as an ontology of the data incorporated into BTRIS, including the names and codes of individual terms from various source systems, knowledge needed to process and store data, and relationships among terms, such as hierarchical classification and semantic associations. The RED currently contains 190,000 concepts, including 12,900 terms for CRIS drug orders, most of which are classified into ingredient-based classes (e.g., Aspirin Preparation), which in turn are classified under therapeutic-intent-based classes (e.g., Anti-inflammatory Agent).

The RED has been designed to handle the mapping between multiple, overlapping high-level concepts (such as drug classes) and the ever-changing detailed instances of classes that occur in clinical data collection. Additional classes can be added to the hierarchy, either as mutually exclusive subclasses of a larger class or as additional classes that subsume concepts from multiple other classes.

Methods

Frequency and Types of Exclusions in Existing Protocols

In order to determine how often exclusions are defined in protocols, we obtained a random sample of active protocols from the Office of Protocol Services. We manually searched these documents for the character strings “exclu” (for “excluded”, “exclusion”, etc.), “prohib” (for “prohibited”, “prohibition”, etc.), and “concomitant”. We reviewed the text associated with each match to determine whether an exclusion was being specified.

The characterization of all types of exclusions in all protocols was beyond the scope of the current project. We sought, instead, to analyze a small number of protocols in depth to obtain sufficient detail for creating the necessary logic for the MLM. We therefore obtained two protocols from a researcher interested in seeing the alert created for those protocols. We searched these protocols (as above) for relevant phrases and then analyzed the related text. In particular, we sought to answer four questions: (1) how were the excluded drugs characterized, (2) were the exclusions limited to specific situations, (3) were there any situations in which a drug could be classified as being among the excluded drugs but was nevertheless exempted from exclusions (that is, it was allowed for some reason), and (4) what would be an appropriate message to deliver to the CRIS user to explain the reason for the alert. We then developed a model for representing the answers to the four questions.

Terminology Modeling

Once the excluded (and exempted) drugs and drug classes were identified for the two protocols, we examined the RED to determine whether it contained sufficient knowledge for matching ordered drugs to protocol concepts. Where necessary, we requested additions (in the form of new drug classes), with suggestions for placement of the classes in the existing hierarchy and the addition of existing concepts as children of the new classes.

Modeling Exclusion Knowledge in CRIS

As previously mentioned, attempting to model the extensive knowledge required for drug exclusions as individual MLMs could not be accomplished in a practical way. However, MLMs are able to access external knowledge (such as the Multum Lexicon) using an external software module that is unconstrained by the MLM execution environment. We modeled the exclusion knowledge using an entity-relation (ER) approach, created tables with the knowledge from the RED, and wrote a query function that could obtain knowledge from the tables.

The Protocol Exclusion MLM

The MLM was written using the basic MLM authoring tool provided by the commercial EHR. The MLM is triggered by any drug order on any subject. The data used by the MLM includes the ordered drug and all protocols in which the subject was participating. It then uses the stored procedure to compare this information to the knowledge base. Positive results from the stored procedure result in an alert.

Retrospective Identification of Protocol Eclusions

Because BTRIS is designed for class-based, cross-subject queries, we explored the feasibility of its use for determining possible protocol nonadherence – i.e., instances in which drugs mentioned as excluded by a protocol were ordered for or administered to a subject on the protocol. There are two intended uses for such a report: (1) to help researchers identify problematic protocols in which the ordering of excluded drugs is occurring with significant frequence and (2) to examine subject data before and after implementation of the alert to determine its impact.

Results

The result of our analysis has been the development of a prototype alerting system that handles protocol drug exclusions. The knowledge base currently contains knowledge for two protocols. The alert has been deployed in a test environment as proof-of-concept. Figure 1 shows the overall architecture and relationships among components.

Figure 1:

Project Components. Research protocol documents are reviewed to extract knowledge needed for the medical logic module (MLM) and stored in the Protocol Knowledge Base. The Research Entities Dictionary (RED) provides classification to relate the more general drug terms from the protocols to the more specific terms for actual drug products. The RED also provides Multum codes to allow mapping of the drug products to drug terms used in the Clinical Research Information System (CRIS). The MLM runs inside CRIS to use protocol knowledge to generate alerts. The Biomedical Translational Research Information System (BTRIS) also uses the RED, along with subject data from CRIS, to produce reports showing instances where excluded drugs have been ordered.

Frequency and Types of Exclusions in Existing Protocols

Of the approximately 1450 protocols that are currently active at the NIH Clinical Center, 158 (11%) were initiated in 2010. We randomly selected 32 protocols (20%) originating in 15 different institutes. Examination of the protocol documents initially turned up matches in 28 protocols that, on review, represented actual drug exclusions in 16 protocols (50%). Table 1 shows the exclusion text that we found in the two protocols we examined in depth; Table 2 shows the results of our analysis of the text to answer our four questions. One aspect that we noted was the great variability of temporal conditions for representing situations in which the exemption was in effect (for example “five days prior to administration of the investigational agent” or “during the first phase of the study”). The formal modeling of temporal reasoning and concomitant conditions seemed beyond the capabilities of the MLM, at least given the great variability of relationships present. We therefore chose to provide this information as part of the message to the user, leaving judgment of the situation to the user. For example, in one instance (ID #5 in Table 1), corticosteroids are allowed if the subject has one of a set of specific conditions. We address this by having the alert message for a prednisone order (for example) state: “Concurrent corticosteroids for myasthenia gravis, or other paraneoplastic syndromes, or other chronic conditions are allowed”.(see Table 2).

Table 1:

Drug Exclusion Text Found in Two Protocols

ID	Text
1	This table lists drugs that may prolong the QTc interval. [Investigational Drug] may be administered after a 5 half-life washout period elapses following discontinuation of prohibited drugs. Drugs labeled “Use discretion” may be co-administered in the absence of other risk factors and with appropriate monitoring. Drugs with a weak association may be administered at usual doses with appropriate monitoring.
2	Any concomitant medication that may cause QTc prolongation, induce Torsades de Pointes, or induce CYP3A4 function (e.g. rifampicin, rifabutin, phenytoin, carbamazepine, phenobarbital and St. John’s Wort).
3	No concomitant use of alternative, complementary therapies or over-the-counter agents will be allowed without approval of the PI.
4	HIV-positive patients receiving combination anti-retroviral therapy are excluded from the study because of possible pharmacokinetic interactions with [Investigational Agent].
5	Corticosteroids should be avoided. Concurrent corticosteroids for myasthenia gravis, or other paraneoplastic syndromes, or other chronic conditions are allowed.
6	Patients may not be receiving any other investigational agents.
7	Prior treatment with drugs of the HDAC [histone deacetylase] inhibitor class.
8	The following table presents a list of drugs that may prolong the QTc. These drugs are prohibited during the study. [Investigational Drug] may be administered after a 5 half-life washout period elapses following the use of these drugs. Washout period is based on roughly 5 half-lives and rounded to a convenient interval.
9	Because there is a potential for interaction of [Investigational Drug] with other concomitantly administered drugs through the cytochrome P450 system, the case report form (CRF) must capture the concurrent use of all other drugs, over-the-counter medications, or alternative therapies. The Principal Investigator should be alerted if the patient is taking any agent known to affect or with the potential to affect selected CYP450 isoenzymes.
10	Avoid concomitant intake of Vitamin E in excess of 100% of the recommended daily dose.

Table 2:

Characterization of Exclusions Found in Two Protocols. Values for ID correspond to those in Table 1.

ID	Excluded	Exempt	Message
1	Drugs with High Risk for Prolonging the QT Interval	None	This drug prolongs the QT interval. It should not be administered with concurrent use of the [Investigational Drug] and should be discontinued 5 half-life washout periods prior to planned use of [Investigational Drug]
	Drugs with Possible Risk for Prolonging the QT Interval	None	This drug prolongs may the QT interval and should be co-administered with [Investigational Drug] only in the absence of other risk factors and with appropriate monitoring.
	Drugs with Conditional Risk for Prolonging the QT Interval	None	This drug may weakly prolong the QT interval and may be administered at usual doses with appropriate monitoring.
2, 8	Drugs that May Cause QTc prolongation	None	This medication may cause QTc prolongation or induce Torsades de Pointes. This medication may be administered if administration of [Investigational Agent] is not planned for 5 half-life washout periods.
	Drugs that Induce CYP3A4 function	None	This medication may induce CYP3A4 function. This medication may be administered if administration of [Investigational Agent] is not planned for 5 half-life washout periods.
3, 9	Alternative and Complementary Agents	None	Because there is a potential for interaction of [Investigational Agent] with other concomitantly administered drugs through the cytochrome P450 system, the case report form (CRF) must capture the concurrent use of all other drugs, including alternative and complementary agents. The Principal Investigator should be alerted if the patient is taking any agent known to affect or with the potential to affect selected CYP450 isoenzymes.
	Over-the-Counter Agents	None	Because there is a potential for interaction of [Investigational Agent] with other concomitantly administered drugs through the cytochrome P450 system, the case report form must capture the concurrent use of all other drugs, including over-the-counter therapies. The Principal Investigator should be alerted if the patient is taking any agent known to affect or with the potential to affect selected CYP450 isoenzymes.
4	Anti-Retroviral Agents	None	HIV-positive patients receiving combination anti-retroviral therapy will be excluded from the study because of possible pharmacokinetic interactions with [Investigational Agent].
5	Corticosteroids	None	Corticosteroids should be avoided. Concurrent corticosteroids for myasthenia gravis, or other paraneoplastic syndromes, or other chronic conditions are allowed.
6	Investigational Agent	[Study Agent]	According to the study protocol, subject may not be receiving other investigational agents.
7	Histone Deacetylase Inhibitors	None	According to the study protocol, subject should not receive prior treatment with drugs of the Histone Deacetylase inhibitor class.
9	Vitamin E	None	Avoid concomitant intake of Vitamin E in excess of 100% of the recommended daily dose.

Terminology Modeling

Analysis of the two protocols showed that terms used for excluded and exempted drugs referred to classes of actual drugs. In some cases, these classes were already in the RED (e.g., Corticosteroids). In cases where they were not in the RED, adding them was a straightforward task: first, create a name for the new class, second, identify a position in the RED hierarchy for the new class, and third, link all appropriate existing classes as subclasses of the new concepts. For the purposes of our prototype, we added the following new concepts to the RED: Drugs that Prolong the QT Interval, Drugs that Induce CYP3A4 Function, Alternative and Complementary Agents, Over-the-Counter Agents, and Histone Deacetylase Inhibitors. In addition, based on a reference included in one of the protocols,⁷ we added three subclasses to Drugs that Prolong the QT Interval: Drugs with High Risk for Prolonging the QT Interval, Drugs with Possible Risk for Prolonging the QT Interval and Drugs with Conditional Risk for Prolonging the QT Interval (Based on Underlying Patient Condition). Figure 2 shows part of the hierarchy in the RED.

Figure 2:

Sample Terminology Hierarchy in the Research Entities Dictionary (RED). The class Pharmaceutical Item Classified by Pharmacologic Effect and its three immediate children were added to the RED for this project. Additional descendants (such as Alfuzosin Preparation already existed in the RED and were given additional parents as needed to include them in this part of the RED hierarchy.

Modeling Exclusion Knowledge in CRIS

We modeled the drug exclusions in a single table with four components: the protocol identifier, the name of the excluded class, the name of the exempted class (if any) and the message to be included in the alert. Table 3 shows the representation of the exclusions and exemptions from Table 2. In order to map the CRIS orders to the drug classes in Table 3, we used the RED to create an Ancestor-Descendant table (not shown) with one set of rows for each excluded or exempted drugs class from Table 2. Each row set contains one row for each orderable drug that is a descendant of the drug class in the RED hierarchy. Table 4 shows some examples of these. The query executed by the MLM joins these tables twice, once to find the excluded drugs and once to find the exempted drugs.

Table 3:

Protocol Exclusion Knowledge Base (contains excluded drug classes and exempted drugs; messages are per Table 2 and are not shown here)

Protocol	Excluded	Exempted
1	Drugs with High Risk for Prolonging the QT Interval	None
1	Drugs with Possible Risk for Prolonging the QT Interval	None
1	Drugs with Conditional Risk for Prolonging the QT Interval	None
2	Drugs that Prolong the QT Interval, Drugs that Induce CYP3A4 Function	None
1	Alternative and Complementary Agent	None
2	Alternative and Complementary Agent	None
2	Over-the-Counter Agents	None
2	Corticosteroids	None
2	Investigational Agent	[Investigational agent in study 2]
2	Histone Deacetylase Inhibitors	None
2	Vitamin E Preparation	None

Table 4:

Ancestor-Descendant Table for Matching Ordered Drug to Excluded and Exempted Drug Classes (partial)

Ancestor	Descendant
Drugs with Conditional Risk for Prolonging the QT Interval	Amantadine (NIAID)
Drugs with Conditional Risk for Prolonging the QT Interval	Amantadine 100 mg Capsule
Drugs with Conditional Risk for Prolonging the QT Interval	Symmetrel (NIAID)
Corticosteroids	PredniSONE 10 mg tablet
Corticosteroids	PredniSONE Oral Solution 5 mg per 5 ml
Corticosteroids	Triamcinolone Injectable 10 mg/mL
Investigational Agent	[Investigational Agent] Infusion
Investigational Agent	Pravastatin or Placebo
Histone Deacetylase Inhibitors	Valproic Acid Syrup 250 mg per 5 ml
Histone Deacetylase Inhibitors	Cisapride (NIAID)
Vitamin E Preparation	Vitamin E 400 International Units Capsule

The Protocol Exclusion MLM

The logic of the MLM compares the ordered drug with the descendants of the excluded drug classes listed for all of the subject’s protocols. If there is a match, the ordered drug is further compared to the descendants of the exempted drug classes for the protocol. If no match is found in this second comparison, an alert is generated, containing the message, and displayed to the user. The user may choose to accept the alert (canceling the order) or may override the alert, providing a reason for doing so. Figure 3 shows an example of the alert.

Figure 3:

Screen Shot of Alert Message in CRIS. User has ordered prednisone (partially hidden frame), which is a drug excluded by one of the subject’s protocols. The message in the alert (topmost frame) explains possible reasons why prednisone might be allowed in this subject.

Retrospective Identification of Protocol Exclusions

BTRIS currently provides a report for drug orders that allows class-based queries, with restriction by protocol and dates (both calendar dates and dates relative to each subject’s date of entry into the protocol). We are therefore already able to retrieve all orders within a protocol for excluded drugs. Figure 4 shows an example of the query page in BTRIS and Figure 5 shows a hypothetical report (using test data) for the query. Further modification of the report is needed to filter out exempted drugs and, like the alert, there is currently no mechanism for modeling the situations in which the exclusion does or does not apply (e.g., a co-occurring condition that requires use of the drug).

Figure 4:

BTRIS Prompt Page for Medication Report (a protocol is selected and the class Corticosteroid Preparation is selected)

Figure 5:

BTRIS Report produced from Query Shown in Figure 4 (test data only; note that report shows orders for all corticosteroid preparations for all subjects in the protocol)

Discussion

Using a commercial EHR and an institutional ontology, we have developed a general approach for generating alerts in response to clinician orders that conflict with the restrictions of clinical research protocols. A medical logic module and a small initial protocol knowledge base demonstrate the feasibility of our approach. A mechanism is in place to determine the rate at which protocol restrictions are being overridden and to monitor the impact of the alerts on increasing protocol adherence. Our approach appears to lend itself well to the modeling of the knowledge in clinical protocols. The effort required to maintain the mapping between the orderable entities and classes of excluded drugs is easily maintained in the RED, often with no effort other than the routine RED maintenance to add new terms into their proper classes.

Clinical alerts for such issues as drug interactions are a relatively mature technology; however, the available pharmacology knowledge bases are not appropriate for the alerts needed in clinical research. There are often no clinical contraindications to the drugs excluded from clinical research protocols; in fact, the drugs may sometimes even be clinically indicated. However, the care of clinical research subjects that takes place over the course of a study must often deviate from the normal standard of care, in part because researchers seek to constrain the variable influences that might affect the result or interpretation of the research, and in part because clinical research intentionally deviates in order to discover new, better standards of care.

We have used clinical alerts to support research documentation,^2,3 to attribute orders to protocols, to notify physicians for returning test results, and to facilitate physician communication (unpublished data). Others have also used alerts to improve clinical documentation and to identify subjects that are eligible for clinical trials.^11,12 We are unaware prior efforts to use clinical alerts to apply biomedical knowledge to improve clinical research.

As with any MLM deployed at our institution, the prospective evaluation of performance will entail monitoring the rate at which alerts are triggered, the percentage of alerts that are overridden and the reasons given for the overrides. The validity of the MLM can be assessed in a manner similar to evaluation of more typical alerts:¹³ by surveying researchers to learn their impressions of the system’s effectiveness, as well as through retrospective analysis of drug orders (using standard BTRIS reports) to compare evidence of nonadherence before and after implementation of the MLM. In addition, we have established mechanisms for alerting other interested parties (in this case, the Pharmacy staff) to monitor for oversensitivity of the alert and to discuss override reasons with clinicians so that the MLM can be “tuned” in a proactive manner.

The work presented here is limited to the actual implementation of a prototype for two protocols, although it is operational and could be deployed, after appropriate evaluation by investigators, with other protocols. Further expansion of the knowledge base will require approval by investigators for each protocol to be included. This approval is likely to depend, in turn, on proof that the alerts are addressing a real problem, evaluation showing that the false-positive rate of the MLM is acceptably low, and demonstration that the alerts are having a beneficial effect.

The present model does not make use of temporal aspects of the data or knowledge, although protocol exclusions are often qualified by situations where the exclusions do and don’t apply. Whether this temporal information can be modeled and used effectively by MLMs is currently unknown, as is an understanding of the capabilities for the MLMs to handle these temporal aspects (something they have been known to struggle with¹⁴). Extending our solution to include temporal reasoning will require a fuller understanding of the types of reasoning required, based on the evaluation of additional research protocols. In the meantime, we believe that the information in the message is sufficient for the user to understand the context of the exclusion in order to perform the temporal reasoning for him- or herself and is better than simply providing the protocol at the point of care (the current status quo).

Ultimately, we believe our approach to using MLMs will improve the quality of research. After all, how many clinicians caring for a research subject will remember, on their own, that one of their patient’s protocols restricts the use of cytochrome p450 inducers, let alone the list of actual inducers? We must stress that the ordering of an excluded drug is not necessarily a protocol violation (due to temporal aspects or other extenuating circumstances that we have not modeled), and may even be indicated for medical reasons that supersede the protocol. Nevertheless, improving adherence to protocols is a worthy immediate goal, since adherence is expected to produce more reliable results while reducing the waste of valuable resources, such as the effort that goes into recruiting and studying subjects that who must later be excluded from analysis due to protocol nonadherence.

Conclusion

We have successfully demonstrated a prototype approach for intervening in clinician orders that are potentially in conflict with restrictions imposed by research protocols. Our method can be used by any EHR system that supports order entry with clinical alerting, together with a controlled terminology that includes orderable terms and classification of those terms. Our method has the potential to improve the quality and efficiency of clinical research.