Developing a Formal Representation for Medication Appropriateness Criteria

Abstract

Inappropriate medications use (IMU) is a serious issue of global concern that leads to a waste of resources and potentially harms the patients. IMU can usually be identified by extracting information about the patient’s conditions and treatments, and comparing them with “medication appropriateness criteria”. To enable automation of these criteria, we developed a formal representation for them, which we called Objective Medication Appropriateness Criteria (OMAC). OMAC represents four aspects of the criteria: trigger, rules, action and metadata. Our evaluation showed that OMAC can completely represent explicitly defined medication appropriateness criteria using links to external knowledge sources. OMAC is the first formal representation for medication appropriateness criteria, and will enable development of structured rules for appropriate use of medications that can be implemented using standards for clinical decision support.

Introduction

Inappropriate medication use (IMU) is a serious issue of global concern that not only leads to a waste of healthcare resources, but also potentially harms patients due to inadvertent side effects ^1–3. Previous studies have shown that several groups of medications are commonly subject to inappropriate use, including antibiotics, antidepressants, antipsychotics, bronchodilators, non-steroidal anti-inflammatory drugs (NSAIDs), proton pump inhibitors (PPIs), and statins ^4–8. Numerous studies developed methods for identifying and reducing IMU, with the focus of their intervention spanning from healthcare professionals and patients, to financial, organizational, and regulatory approaches ^9,10. Typically, these approaches rely on manual identification of IMU. Developing automated methods to reduce IMU is challenging. Inappropriate use of medications is still considered an understudied problem in general ^11,12.

Manual identification of IMU requires extracting information about the patient, the medication, and other treatments, and comparing them with “medication appropriateness criteria”, which are standards that define appropriate use of medications ¹³. Consequently, developing automated solutions to reduce IMU entails two requirements: a framework to represent the medication appropriateness criteria formally, and methods to extract the information needed to compute these criteria. This article focuses on developing a framework for formal representation of medication appropriateness criteria.

Previous researchers have identified several medication appropriateness criteria and metrics through systematic review of the literature ^12,14. These criteria can be categorized into three groups: the first group enumerates the conditions in which the use of medication is appropriate (e.g. see Choudhrey et al.’s criteria for appropriate use of proton pump inhibitors (PPIs)¹⁵), the second group lists conditions in which the use of medication is deemed inappropriate (e.g. see the Beers’ criteria¹⁶), and the third group provides a combination of both (e.g. see Oborne et al.’s criteria on appropriate use of neuroleptics¹⁷). Since older adults are frequently subject to polypharmacy and therefore more likely to experience the negative impacts of IMU (e.g. drug-drug interactions, adverse drug reactions and increased risk of hospitalization) ^18–20 larger collections of medication appropriateness criteria exist for the geriatric population. Examples include the Beers’ criteria¹⁶ and the Screening Tool of Older Persons’ potentially inappropriate Prescriptions (STOPP)²¹ which aim to reduce inappropriate use of medications (overuse), and the Screening Tool to Alert doctors to Right, i.e. appropriate indicated Treatment (START)²² which promotes appropriate use of medications that are omitted (underuse).

Medication appropriateness criteria can be described as a special form of clinical guidelines, although they have distinct features that separate them from the majority of clinical guidelines. Clinical guidelines provide best practices for diagnosis and therapy of diseases, but medication appropriateness criteria are focused on proper utilization of a resource (namely, medications). Clinical guidelines are primarily developed by major medical associations, are organized in a common format and are hosted on repositories such as the National Guideline Clearinghouse ²³; in contrast, medication appropriateness criteria are mostly developed by independent groups of researchers and distributed without using a common format or central repository.

Medication appropriateness criteria are currently only available in narrative form, and transforming them into a computable format is challenging because a formal representation for the components of medication appropriateness criteria does not exist. Different criteria have varying levels of granularity and specificity in defining the medications, diagnoses, and symptoms; in addition, some but not all of the criteria are accompanied by information regarding the level of evidence, target population, or extent of clinical relevance. A framework in which these criteria can be explicitly and comprehensively represented is needed. We developed such a representation framework, which we call the Objective Medication Appropriateness Criteria (OMAC).

Methods

In order to study existing medication appropriateness criteria, we started by identifying these criteria by searching PubMed using the following keywords and their variations to identify published medication appropriateness criteria: inappropriate prescribing, overuse, overtreatment, overutilization, and utilization review. We grouped the relevant studies based on the actual criteria they used to identify IMU. We then curated a collection of published medication appropriateness criteria and used a random subset of those to develop OMAC (for examples, see Table 1). To ensure that we didn’t have any biases in our component selection and semantic aggregation of the concepts, we used another independent set of criteria for evaluation of OMAC.

Table 1.

Examples of medication appropriateness criteria previously published in the literature. These examples are all adopted from Beers’ criteria¹⁶ and STOPP²⁴ and each criteria expresses what medication should be avoided in the provided context. Other examples from other sources were also used in the development and evaluation of OMAC (not shown in the table). The narrative criteria are shown as they appear in the original source. TCA: tricyclic antidepressant; NYHA: New York Health Association; H2 receptor: histamine 2 receptor.

	Source	Narrative criterion
1	2002 Beers’ criteria	Disease: Seizures or epilepsy Drug: Clozapine, chlorpromazine, thioridazine, and thiothixene Concern: May lower seizure thresholds Severity Rating: High
2	STOPP, section A, item 2	Loop diuretic for dependent ankle edema only i.e. no clinical signs of heart failure
2	STOPP, section B, item 3	TCA’s with cardiac conductive abnormalities
3	STOPP, section A, item 6	Beta-blocker in combination with verapamil
4	STOPP, section A, item 7	Use of diltiazem or verapamil with NYHA Class III or IV heart failure
5	STOPP, section B, item 9	Use of aspirin and warfarin in combination without histamine H2 receptor antagonist (except cimetidine because of interaction with warfarin) or proton pump inhibitor

Developing OMAC

We manually analyzed a randomly selected subset of medication appropriateness criteria to identify their components. Each criterion can be described as one or more rules, and the purpose of OMAC was to provide a formal representation for these rules as well as any other aspects of the criteria. We semantically grouped the components we found in the sample criteria to define concepts that comprise the criteria, including high-level elements (such as the general sections of a criterion) and low-level elements (such as modifiers, identifiers, names, etc.) and we also identified the relationships between these concepts and represented them in OMAC.

Different medical concepts are frequently mentioned in the medication appropriateness criteria, such as medications and diseases. Since the purpose of OMAC was to provide a representation for the criteria, and not to enumerate each individual medication or disease, we ensured that OMAC takes advantage of previously developed ontologies and terminologies, by linking to external ontologies and terminologies to the extent possible. We saved OMAC using frames and properties in Protégé version 3.5.

Evaluating OMAC

After the initial design of OMAC was completed, we presented a separate set of 10 medication appropriateness criteria to a group of domain experts (physicians and pharmacists) in form of a questionnaire, asking them to identify and categorize the components of these criteria independently. Each item in the questionnaire consisted of one medication appropriateness criterion statement in its original narrative form, and requested that the participant breaks the statement into basic elements (such as medication names, medication class names, disease names, logical statements, or temporal modifiers). Disagreements in the experts’ responses were identified through qualitative analysis of the responses. Three types of disagreements were considered: differences in the classification of the same word or phrase (e.g. classifying “hypertension” as a disease versus a problem), differences in specification of the elements in the statements (e.g. considering “severe hypertension” as two separate concepts versus one), and classification of terms into concepts that are not explicit (e.g. classifying “long-term use of drug X” as one concept of type “overuse”). In a subsequent questionnaire, we presented the experts with the same narrative criteria but clearly marked these areas of disagreement and asked the experts to translate those terms and phrases into more detailed, explicitly defined concepts. Note that the purpose of this process was not to reach perfect agreement, but rather to identify what “elements” constitute the criteria and also to describe the elements so that they are well-defined, so that we can evaluate OMAC’s coverage for those elements.

Subsequently, we evaluated whether OMAC could represent all of the explicitly defined concepts provided by the experts. We froze the development of OMAC before we started sending out the questionnaires, to ensure that our knowledge of the results of the previous step would not affect our evaluation of OMAC’s completeness. We planned to correct OMAC for any areas of deficiency that would be found throughout this evaluation, only after the evaluation was completed.

Results

We identified 110 medication appropriateness criteria through literature review, and used a random subset of 40 to develop OMAC. We designed OMAC such that each criterion in this subset could be represented using four types of information: ‘trigger’, ‘rules’, ‘action’, and ‘meta-data’. The trigger may consist of one or more medications that are the primary focus of the criterion (when prescribing these medications the criterion would be triggered) or one or more clinical conditions in which the use of a certain medication is desirable (in this case the criterion would focus on underuse). Rules specify the conditions that a patient must meet to be eligible for the criterion (such as age limit, past medical history, medications prescribed, symptoms, or paraclinical findings). Action specifies the recommendation that the criterion makes once the patient meets all the rules; generally, actions are in two forms, either to avoid prescribing a medication or to consider prescribing a medication. Meta-data includes all the additional information that is used to describe the criterion (examples include a name or unique identifier, references to citations, or a justification or concern). As an example, Beers’ criteria not only lists medications or combinations of drugs that should be avoided in the elderly, but also specifies what “concern” exists around using these medications, and also provides a “severity rating” for this concern (low vs. high) to help the clinicians determine the importance of each item in this criteria (Table 1) ¹⁶. We represented the trigger, action and meta-data components using properties for the “criterion” class (Figure 1, right). We used a more complex classification as described below to represent the rules.

Figure 1.

Main concepts in the OMAC (left) and the properties of the criterion class (right).

Each criterion can contain one or more rules, and there are various types of rules in different criteria. These include ‘medication rules’ which specify the medication that is the subject of the criterion as well as co-prescribed medications that need to be considered, and ‘clinical rules’ which specify the diseases, symptoms, laboratory tests results, and demographics that have to be present or that should be absent for the patient to meet the criterion. This can be clarified using the third example shown in Table 1: “TCA’s with cardiac conductive abnormalities”; this item from STOPP criteria states that in elderly patients who have cardiac conductive abnormalities, tricyclic antidepressants (TCAs) should be avoided because of their pro-arrhythmic effects ²⁴. To apply this criterion to a patient, three rules must be satisfied: (i) the patient must belong to the ‘elderly’ demographic group (formally defined as age ≥ 65 years), (ii) the patient must have been diagnosed with a cardiac conductive disorder (including, but not limited to Type I heart block, Type II heart block, or right bundle branch block), and (iii) the patient must have been prescribed a medication that belongs to the TCA class. The first two rules in this example are clinical rules, and the latter is a medication rule.

Clinical and medication rules have different properties: clinical rules may focus on the existence, temporality and duration of a clinical finding or condition, or the value of a measurement, but medication rules may specify the dose, route, frequency and form of a medication. Both clinical and medication rules may include concepts that are externally defined in other ontologies or terminologies (Table 2). In the example provided above, “cardiac conductive abnormalities” can be represented as a clinical rule, which can refer to a pertinent concept in International Classification of Diseases, version 10 (ICD-10) or Systematized Nomenclature of Medicine, Clinical Terms (SNOMED CT), and thus, it is possible to link the concept to a standardized external knowledge source. A link to an external concept consists of four parts: the local name of the concept (e.g. ‘cardiac conductive abnormalities’), the name of the external ontology or terminology, including version number (e.g. ICD-10), URL of the external ontology or terminology (e.g., http://purl.bioontology.org/ontology/ICD10) and the unique identifier of the corresponding concept in that external ontology or terminology (in this case ‘I44’).

Table 2.

Examples of external ontologies and terminologies that can be used to define concepts that are contained in a clinical or medication statement, as part of a medication appropriateness criterion. ATC: Anatomical Therapeutic Classification; NDC: National Drug Code; ICD: International Classification of Diseases; SNOMED CT: Systematized Nomenclature of Medicine, Clinical Terms; CPT: Current Procedural Terminology.

Concept Type	External ontology or terminology
Medication	RxNorm, ATC, NDC
Disease	ICD, SNOMED CT
Symptom	SNOMED CT, Symptom Ontology
Procedure	CPT, ICD, SNOMED CT

Clinical and medication rules can be combined with each other using ‘logical rules’. Each logical rule has a mandatory field which specifies the Boolean operator it is representing (‘AND’, ‘OR’, or ‘NOT’). In the example above, the clinical and medication rules are combined using a logical rule with ‘AND’ logic (i.e. the patient must be among the elderly AND have a cardiac conductive disorder to be eligible for this criterion).

We grouped all the three aforementioned types of rules under a parent class called ‘rules’ (Figure 1, left). To represent complex statements, these rules can be nested to create ‘rule trees’. Clinical and medication rules can only appear as the leaves of the rule tree. Logical rules appear as branches of the tree, and each logical rule references one or more rules of any type. The latter enables nested rules which allow representation of complex logical statements. The last example in Figure 1 (STOPP, item B9) demonstrates a criterion with a complex logic. This complex statement can be encoded through nesting different types of rules, as shown in Figure 2.

Figure 2.

A schematic diagram showing nesting of logical statements representing a complex criterion. For simplicity, logical statements are shown as circles that are labeled based on the Boolean operator associated with them, and medication statements are shown as boxes corresponding to the pertinent medication or medication class. Each of the medication statements may refer to an external ontology or terminology which formally defines the specific medication or drug class; those links are not shown in this figure. The narrative form of this criterion is available in Table 1.

Evaluating OMAC

Eight domain experts collaborated in the first questionnaire. There was no disagreement among experts in their responses for simple and well-defined criteria; for instance, all collaborators described STOPP criteria item A6 (shown in Table 1) using similar components. We observed disagreements in the way more complex criteria were broken down; for example, there was lower agreement on how the terms ‘dependent ankle edema’ and ‘no clinical sign of heart failure’ were categorized by different experts. When experts clarified the areas of vagueness using detailed explicit concepts, we noticed that although they clarified these vague terms using different sets of explicit concepts, they used similar ‘types’ of concepts to describe them. For example, each expert used a different set of ‘signs’, ‘symptoms’ and ‘paraclinical findings’ to describe the phrase ‘clinical signs of heart failure’, but all experts used exactly those three types of information.

All of the types of information that experts used to transform the vague phrases into explicit forms corroborated with the types of information that we had already incorporated into OMAC’s clinical or medication rules. In other words, OMAC had completed coverage for all of the criteria that were coded by the experts, and as a result we did not modify OMAC after this evaluation.

Discussion

Developing a representation format for medication appropriateness criteria is the first step towards developing computable, interchangeable and reusable solutions to prevent inappropriate medications use. OMAC formally defines the structure of explicitly defined medication appropriateness criteria, and allows referencing to external ontologies and terminologies when applicable.

The results of our questionnaire study indicate that at least some of the medication appropriateness criteria are defined using vague terms that were interpreted differently by the experts. These criteria only provide guidelines for appropriate use of medications, and variability in the application of guidelines is a well-established phenomenon in health care practice; however, ideally the guideline itself should be interpreted identically by all of its users so that the variability should be only due to the specific characteristics of the patient or the settings in which the guideline is used, and not due to different interpretations of the appropriate care^25,26. Although our questionnaire study has a small sample size, it signifies the need for well-defined medication appropriateness criteria. OMAC can facilitate this process, as encoding the criteria into OMAC requires translating all terms into explicitly defined medication, clinical or logical rules.

OMAC is designed to be flexible, and allow for multiple ways of defining concepts and their relationships. Through the use of logical rules, it is possible to model the steps that are used to implement medication appropriateness criteria in clinical practice and encode these steps in a computable way. When a clinical or medication concept is in fact referring to a class of diseases or medications, logical steps can be used to internally define these sets instead of referencing external knowledge sources, which is important when defining a concept that does not exist in any external knowledge source. Therefore, the user has the choice of either specifying a medication class by referencing an external entity, or by defining external references to each member of that class and then combining them using an ‘OR’ logic (Figure 3). Each approach has its own advantages: using an external reference for each of the elements in that class makes the local definition of the criteria more explicit, while using an external reference for the class itself reduces the amount of effort needed to encode the criteria in OMAC.

Figure 3.

Classes of medications or diseases can be defined either by creating an external link to the ‘class’ itself (left), or by creating external links to each of the elements of the class and then combining them using ‘OR’ logic locally (right). Boxes with solid borders indicate concepts specified in OMAC, while dotted arrows indicate a link to an external knowledge source, and boxes with dashed borders specify the external knowledge source and the unique identifier of the respective concept in that knowledge source. The links will also include the version of the external ontology as well as a URL linking to that external ontology (not shown in this schematic).

Using an external ontology or terminology to define the concepts in OMAC also has the advantage of reusing knowledge that has been vetted by a group of experts, but a suitable external knowledge source may not be available in all cases, or it may not be as accurate or complete. In addition, not all of the concepts that are found in medication appropriateness criteria can be identically found in external knowledge sources. For instance, one medication appropriateness criteria may specify the severity levels for heart failure using the classification provided by the New York Health Association (Table 1), but this classification may not be already defined in any existing disease ontologies and terminologies. OMAC flexibly supports defining these complex concepts either by external links (when possible) or locally, and the users can choose their preferred method based on the task at hand.

OMAC is different from a guideline representation language. While medication appropriateness criteria can be described as a special form of guidelines, guideline representation languages (e.g. GEM²⁷, GLIF²⁸, EON²⁹, PROforma³⁰, and SAGE³¹) do not enforce the mandate level of detail in their formalism that is needed for representing medication appropriateness criteria. Guideline representation languages provide a structured way to encode the “flow” of decisions in a guideline. However, to ensure that they can support different types of decision and various forms of guidelines, they provide a significant amount of flexibility as to how each decision step is defined. Previous research has shown that these guideline representation models have limitations when applied to medication related guidelines used for chemotherapy, and that representing medication related guidelines as rules can address this limitation.³² OMAC combines this rule-based approach with specific features of guideline representation language (such as the inclusion of meta-data about provenance of the guidelines), to provide a more strict structure to represent the medication appropriateness criteria than guideline representation models, thereby providing a common framework for encoding all such criteria in a similar, interchangeable way. In that sense, OMAC complements the guideline representation languages by providing the formalism that is necessary for a certain type of decisions, namely the decision about appropriateness of medications.

One potential challenge in interchanging OMAC-encoded criteria is that a criterion may be encoded using an external ontology or terminology which may be different from what is desirable for a second user of the criterion. This challenge can be addressed by creating cross-walks between these external knowledge sources; in many cases, this can be easily possible using the Unified Medical Language System (UMLS). Finally, our study is also limited in that we did not conduct a large scale evaluation of the completeness of OMAC. We intend to address this limitation in future research. We also intend to use OMAC to develop structured representations of well-established medication appropriateness criteria and then export them into a format supported by HL7 Clinical Decision Support (CDS) standards. Namely, we intend to use the virtual medical record (vMR) format³³ to represent the patient data, and use the OpenCDS platform³⁴ to integrate the computable medication appropriateness criteria with the medical records and evaluate the accuracy and impact of using this approach to provide decision support regarding appropriateness of medications.

Conclusions

OMAC provides the necessary flexibility for defining concepts using external ontologies or terminologies whenever applicable, and through the use of rules, it enforces the necessary formalism to ensure that all essential concepts of the medication appropriateness criteria are represented using a common structure. OMAC is the first framework that specifies encoding the medication appropriateness criteria into a formal, structured form, which is necessary to incorporate a decision support component aimed at reducing IMU.