Development of a combined system for identification and classification of adverse drug reactions: Alerts Based on ADR Causality and Severity (ABACUS)

Abstract

Currently, adverse drug reaction (ADR) causality and severity are assessed using different systems but there is no standard method to combine the results. In this work, a combined ADR causality and severity assessment system, including an online version, was developed. Logical rules were defined to translate the score obtained from the system into three alert zones: green, amber, and red. The alert zones are useful for triaging ADR cases as they help define the seriousness of the ADR and the urgency of the responses required. This new scoring system may be useful for clinicians, investigators, and regulators seeking information on the likelihood of a drug causing an adverse reaction, and whether an adverse reaction is sufficiently dangerous for the drug to be withheld or undergo further investigation.

Introduction

Causality and severity are the two criteria of fundamental importance for assessing adverse drug reactions (ADRs). Causality is the likelihood of the suspected drug causing the ADR. The causality of ADRs can assessed using clinical judgment alone or in combination with particular algorithms.¹

Based on the definitions provided by World Health Organization–Uppsala Monitoring Centre (WHO–UMC), the term ‘severe’ is used to describe the intensity of a specific event.² This is different from ‘seriousness’, which is based on either outcome or action criteria, and serves as a guide for defining regulatory reporting obligations.² Our recent review found several published studies providing some details of ADR severity (see online supplementary figure 1, available at www.jamia.org).^3–5 The references cited in figure 1 may not be exhaustive but they confirm the observation that varying definitions of severity levels are used by different researchers.⁶

Given the importance of the causality and severity of an ADR in supporting clinical and regulatory decisions, it would be useful to have a single measure to combine these two types of information in order to simplify the assessment of an ADR report. Unfortunately, no combined ADR assessment scale is currently available. In this study, we describe the development of a severity assessment scale and the incorporation of this scale into our previously published ADR causality assessment scale.⁷ Overall assessments of ADR reports are determined and presented as falling into various ‘alert zones’. This approach of ‘alert zone’ assignment was designed to increase the usefulness of the combined causality and severity assessment in helping the end-user determine the appropriate course of action following a particular ADR. This is an attempt to fill an existing gap and provide a decision support tool for the practice of pharmacovigilance.

Case description

In this study, we used nine ADR cases to illustrate how our newly merged causality and severity scoring system can be used. The ADR cases were taken from ADR reports received by the Health Sciences Authority, which is the national body in Singapore handling ADR monitoring. These nine cases were selected based on the drug causalities that were assigned to them by the original clinical reviewer. A variety of causality assignments were chosen with the intention of evaluating the algorithm using a spectrum of test cases to determine its robustness. The case descriptions of the nine chosen ADR cases are presented in table 1.

Table 1.

Actual cases illustrating use of the combined causality and severity scoring system

Case	Case description	Causality*	Severity level	Total score† and alert zone‡
1	A patient developed muscle ache while taking pravastatin 20 mg every night. The muscle ache disappeared when pravastatin was withdrawn.	0.574 (Possible)	S1	0.287 Green
2	A 45-year-old patient presented with light-headedness and raised creatine kinase and aldolase after a single dose of sildenafil citrate. The patient was hospitalized and hydrated, and eventually recovered.	0.620 (Possible)	S3	0.51 Amber
3	A male patient taking a complementary medicine containing ephedrine for many months presented with vomiting, increased drowsiness, and jaundice. Acute liver failure was diagnosed by the attending physician.	0.620 (Possible)	S5	0.710 Red
4	A patient with no known drug allergy developed pruritus after taking clarithromycin 1 g per day for 8 days.	0.639 (Probable)	S1	0.320 Green
5	A patient developed edema of the face after ingesting a single dose of 40 mg omeprazole for gastro-esophageal reflux disease. The patient recovered 2 days after discontinuing omeprazole.	0.741 (Probable)	S2	0.471 Amber
6	A patient developed generalized erythematous, pruritic rash, shortness of breath and involuntary irregular jerking of the upper and lower limbs after receiving 100 mg of intravenous paclitaxel for ovarian carcinoma. The patient recovered after drug was discontinued.	0.741 (Probable)	S3	0.571 Amber
7	A patient developed bronchospasm and collapsed airways after receiving intravenous iohexol for urography. He recovered after intubation and ventilation in the intensive care unit and treatment with epinephrine.	0.741 (Probable)	S4	0.671 Red
8	A 51-year-old male patient presented with an intense urge to fall asleep without warning within 1–2 h after consuming 100 mg of piribedil. This symptom resolved when piribedil was withdrawn but recurred upon rechallenge.	0.843 (Definite)	S1	0.422 Green
9	A patient presented with a low platelet count after taking rofecoxib 25 mg daily for 6 months to manage her osteoarthritic knee. Her platelet count returned to normal following a dechallenge but decreased again when rofecoxib therapy was re-introduced. Therapy with rofecoxib was subsequently discontinued.	0.898 (Definite)	S4	0.749 Red

^*Causality refers to the probability that the adverse drug reaction is caused by the offending drug.

^†Total score is computed by taking the average of the causality and severity scores.

^‡The green zone means that few resources are required to manage the adverse drug reaction; the amber zone means that use of the offending drug should continue to be monitored; the red zone means that use of the offending drug must cease immediately.

Method of implementation

Development of the causality scale

The development of the causality scale has been reported earlier.^{7 8} We surveyed Kramer's algorithm,⁹ which is a comprehensive ADR causality assigning algorithm, and ADR reporting forms from different regulatory agencies. We formulated eight questions that can be answered from the information routinely collected in ADR reporting forms. The scores for the answers to each question were determined with the help of a genetic algorithm, as an exhaustive search of all possible scoring systems was impractical.

Development of the severity scale

For severity assessment, we identified six categories with increasing intensities of severity, denoted as S1 to S6, by consulting existing severity scales.^{5 10 11} S1 to S6 reflect the ascending requirement for more complex measures to manage the ADR. A score was then assigned to the different levels of severity. Conceptually, this score is an arbitrary number as the intention was to develop a relative scale. The arbitrary scores assigned to the different levels ranged from 0 to 1, increasing in steps of 0.2 from mildest intensity to a fatal ADR.

The severity scores were then combined with the scores derived from our ADR causality probability algorithm⁷ to form a combined scoring and assessment system. Theoretically, the causality and severity scores for an ADR would produce a ‘point’ on a two dimensional plane. The two dimensional plane was divided into three different color zones—green, amber, and red. The green zone signifies that the ADR falling into this zone is mild or likely not to be caused by the suspected drug. Hence, the attention or resources committed to handling such an ADR should be limited. An ADR falling into the amber zone has a higher level of severity and higher probability of having been caused by the particular drug. Thus, medical personnel or regulatory authorities should continue to monitor use of the offending drug. An ADR in the red zone is deemed to be so severe that the authorities should suspend or completely withdraw the drug.

Zone assignment

The demarcation of the three zones was determined by eight rules. For the new assessment system to have practical application, these rules should be universally accepted as logical. Thus, we developed these rules based on our previous experience with the development of a quantitative causality ADR scoring system, the outcomes from the various ADR reports from the Health Sciences Authority used for the development of the causality scoring system, as well as inputs from our clinical colleagues. Detailed information about these rules and zone assignment is provided in the online supplementary appendix, available at www.jamia.org.

Assignment of final scores

A final score can be calculated from the severity and causality scores by taking the average of the two scores. The use of the average method underlines the belief that both causality and severity are equally important in the assessment of an ADR. The final score provides a quantitative dimension to help prioritize responses to different ADRs in the same alert zone. For example, if two cases in the amber zone have final scores of 0.471 and 0.571, then the second case should receive more attention since it is closer to the red zone.

Development of the on-line version

An online version of this combined scoring and assessment system (http://padel.nus.edu.sg/software/abacus/) has been developed with the objective of providing the user with an easily accessible and simple to tool for determining the alert zone of ADRs.

Examples and observations

The final version of the combined scoring system comprises four parts and is shown in table 2, available in the online supplementary appendix, available at www.jamia.org. Part 1 displays the algorithm used to identify the probability that a particular ADR is caused by the suspected drug.⁷ Part 2 shows the newly developed severity scale with the corresponding severity score. The final score can be obtained by running a given ADR case or report through part 1 and part 2 and then using the formula provided in part 3. Part 4 is a chart for the user to visually locate the alert zone for the ADR based on the causality probability value and the severity level of the ADR.

Using case 9 as an example, a low platelet count occurred when the patient was 6 months into her therapy with rofecoxib, suggesting the presence of a temporal association between the adverse reaction and the drug. A low platelet count is a known adverse effect of rofecoxib, there was no known causative clinical condition and there was no overdosage of the drug. The patient's condition improved when the offending drug was discontinued. However, the patient experienced another episode of low platelet count when rofecoxib was re-introduced. When this information was fed into part 1 of the algorithm, a score of 49+1+7+0+14+0+1+33=105 was obtained. This gives rise to a probability of (105–8)/108=0.898. A severity category of S4 was assigned to this case as the patient required intensive care as a result of the ADR but did not experience any disability. Thus, her final score based on ABACUS was (0.898+0.6)/2=0.749. Using the color chart in part 4 of table 2 (see online supplementary information, available at www.jamia.org), this case falls into the red alert zone.

Discussion

This new system has several distinct practical advantages over existing independent severity and causality algorithms even if they were to be used in combination.

The first advantage is the reduction in subjectivity and ambiguity in severity assessment. We chose six different severity levels that are based on outcomes and therefore distinct from each other. This is to minimize confusion with terms such as ‘mild’, ‘moderate’, and ‘severe’, which are subjective and relative to a particular starting point. This approach provides well-defined severity categories to avoid any ambiguity in interpreting the severity of the reactions.

With a quantitative approach, this combined scoring system can aid in triage if cases need to be prioritized for management. It can be observed from table 1 that even though drug causality may be the same, dissimilarities in severity levels result in cases being assigned to different alert zones, thus warranting different levels of response. This situation is illustrated in cases 2 and 3, where both cases had exactly the same probability score for ADR causality. However, case 3 had a higher level of severity since the patient required intensive care, than case 2, who only required in-patient care. Therefore, cases 2 and 3 were assigned to the amber and red zones, respectively.

Another way that the combined scoring system can be used quantitatively is exemplified in cases 5 and 6. Although both cases are assigned to the amber zone, a higher score for case 6 indicates that the drug involved should receive higher priority for further investigation or action, since it is closer to the red zone than the drug involved in case 5.

Another advantage provided by this combined system is the assignment of ADRs to different alert zones, which results in better assessment and thus more consistent responses to ADRs. For example, a ‘definite’ ADR may not be as severe as a ‘possible’ or ‘probable’ ADR (see cases 3, 7, and 8 in table 1). When the management of cases has to be prioritized, those in the red alert zone should be dealt with first even if they have lower causality scores than cases in the green or amber zones.

Limitations of the current work

This new amalgamated system has a few limitations. Although the causality assessment system is able to provide a probability value for ADR causality, the authors do not claim that these are true probabilities. However, these values have been shown, at a minimum, to be reliable when used as a ranking system.⁷

Furthermore, it could be argued that the current scores for the severity assessment system and the method for deriving the final score are arbitrary, with equal weights assigned to probability and severity. While such weight assignment may suffice for most cases, the combined scoring system is not able to handle some cases effectively. These include relatively mild ADRs associated with life-style drugs, such as those used for weight-loss and smoking cessation, and severe ADRs associated with chemotherapeutic drugs. In the former situation, the scoring system may place the ADR report in the green zone when it should prompt immediate action, while in the latter, the ADR report may be assigned to the red zone although the episode may not require regulatory measures. Future work will involve the analysis of more ADR reports and the use of more evaluators and experts to further refine the method for computing the final score. The present system of weight assignment for severity and causality can be further fine-tuned to provide a more accurate assignment for special circumstances and patient subgroups.

In summary, this causality and severity scoring method is intended to be a decision support system to help clinicians and health regulators reduce the information available to a manageable amount and enable better decisions and more consistent responses.