Published cases of adverse drug reactions: has the quality of reporting improved over time?

Abstract

Purpose:

A previous study of cases published approximately 10–30 years ago reveals that substantial improvement in the quality of adverse drug reaction (ADR) case reports is needed. Since that evaluation, recommendations are available concerning the content and formatting of case reports.

Objective:

To compare the quality of recently published ADR case reports to a previously published study of the quality of ADR case reports from 10–30 years ago. A secondary objective is to determine the quality of reporting by specialty journal.

Methods:

This was a two-phase study. Phase 1 included an assessment of the 23 patient, drug and ADR variables evaluated in the previous study to allow comparison with recently published case reports in specialty journals. Phase 2 mimicked the methods of Phase 1 with a random selection of available case reports in a 1-year period from a variety of journals.

Results:

For Phase 1, 19 of the 23 variables had significant differences in reporting compared with the previous study. Reporting of active diseases, social history, weight, race, other drugs and dose had frequencies ranging from 25 to 80%, which was an improvement, but affording an opportunity for greater improvement. For Phase 2, 21 of the 23 variables had significant differences compared with the previous evaluation; however illicit drug use, mechanism for ADR and route of administration had significant reductions in reporting.

Conclusion:

Progress has been made in ADR case reporting quality for a variety of journals, but more improvement is required to ensure data are understandable and relatable to patient care.

Introduction

Case reports are used as a source of pharmacovigilance [Wang et al. 2011]. They provide readers with valuable information about unique diseases or treatments that may be too rare to stimulate further investigation or information about a novel occurrence that may stimulate more thorough investigations. An example of a case report that alerted clinicians to a detrimental adverse drug reaction (ADR) is the association between congenital abnormalities and thalidomide [Gagnier et al. 2014]. Over 150,000 case reports of ADRs were indexed in MEDLINE as of mid-2014.

A previous study revealed that substantial improvement in the quality of ADR case reports was needed [Kelly, 2003]. The quality of ADR-related case reports published between the mid-1970s and mid-1990s was evaluated by Kelly who determined that several important patient variables were infrequently reported [Kelly, 2003]. Since this evaluation, recommendations are available concerning the contents and formatting of case reports [Gagnier et al. 2014; Ozçakar et al. 2013; Carleton and Webb 2012]. In addition, there is specific guidance on writing well-structured and informative case reports related to ADRs [Kelly et al. 2007; Cohen, 2006].

The question arises with the additional guidance on publishing case reports: ‘Has the quality of case reporting improved over time?’ Few evaluations assess the quality of case reporting and they are confined to a specific drug type, patient population and proprietary database [Talat et al. 2013; Impicciatore and Mucci, 2010; Impicciatore et al. 2012; Richason et al. 2009].

The purpose of this evaluation is to compare the quality of recently published ADR reports with the quality of case reports published approximately 10–30 years ago. A secondary objective is to determine if the quality of reporting varies by specialty journal.

Methods

This was a two-phase study. Phase 1 included an evaluation of the 23 patient, drug and ADR variables published in the original assessment by Kelly to allow comparison with recently published case reports in specialty journals [Kelly, 2003]. The patient variables include age, gender, recovery status, active diseases, alcohol use, tobacco use, illicit drug use, renal function, allergy history, complete blood count, ADR history, weight, liver function, race and severity of illness. The drug and ADR variables include concomitant administration of other drugs, ADR mechanism, indication, route of administration, duration of therapy, dose, serum drug concentrations, and objective tool used. Phase 2 mimicked the methods in Phase 1 with a random selection of case reports in a 1-year period from a variety of journals. Phase 2 also included an evaluation of these case reports with the 10 components of the Adverse Drug Reaction Probability Scale (also referred to as the Naranjo criteria) since these have also been used to assess the quality of reporting [Talat et al. 2013; Naranjo et al. 1981].

Phase 1: specialty journals for comparison

A MEDLINE search was completed to identify ADR related case reports from journals representing critical care medicine (n = 4), general medicine (n = 6), nursing (n = 2) and pharmacy specialties (n = 6). Journal selection was based on the highest impact factor, in addition to having a clinical focus. This approach to directive journal selection was taken to allow for a specialty journal comparison. Cases were selected based on published dates between 2000 and 2013. Case report selection was based on the 5–10 (depending on availability) most recent publications per journal to reach a total of 30 case reports per specialty, with a total of 100 cases. If a case series was identified then the first case was evaluated for report details. Each case report was evaluated by a reviewer following training using a standardized form containing the 23 variables used in the previous study [Kelly, 2003]. The intent of the variable assessment was to determine if the 23 variables were present or absent within the case report. Figure 1 illustrates the methods for Phase 1.

Figure 1.

Specialty journal assessment.

Phase 2: general journals for comparison

A search was conducted in MEDLINE including dates from March 2013 to March 2014 using the MESH term ‘drug-related side effects and adverse reactions’ and limited to case reports, English language and human. This search yielded 1043 case reports. A random sample of 20% was selected (n = 208) using IBM SPSS version 20.0 (Chicago, IL). This general approach to journal selection was taken to include a representative sample of published case reports. A total of 79 journals were represented in the 208 cases that were assessed. Each report was assessed by a reviewer after training using a standardized form to determine if the patient and drug variables included in the previous study (n = 23 variables) and the criteria of the Adverse Drug Reaction Probability Scale (n = 10) were described (present or not present) [Kelly et al. 2003; Naranjo et al. 1981]. If a case report did not discuss an ADR or the article was not accessible at our institution then a replacement article was randomly selected. A total of 47 articles were reselected with 15 not discussing a drug and 32 not available at our institution. Figure 2 illustrates the methods for Phase 2.

Figure 2.

General journal assessment.

Analysis

A comparison between the specialty journal selection in Phase 1 of our study and the previous published results by Kelly was conducted [Kelly, 2003]. A comparison between the general journal selection in Phase 2 of our study and the previous published results by Kelly was conducted [Kelly, 2003]. Chi-square tests and Fisher’s exact tests were used as appropriate using a web-based program called Social Science Statistics [Stangroom, 2014]. A p value <0.05 was considered to be statistically significant.

Results

Phase 1

Of the 100 cases obtained from the literature, 30 represented critical care medicine, 30 were in general medicine journals, 10 were available in nursing journals and 30 were from pharmacy journals. Of the 23 patient and drug variables, statistically significant differences occurred in 19 variables compared with the Kelly study (p < 0.05) [Kelly, 2003]. The majority of the differences were improvements in reporting, except for a reduction in reporting of the ADR mechanism and route of administration (Figure 3 and Figure 4). The difference for reported drug concentrations (14.2% versus 7%) was trending to a significant decline in our study (p = 0.054). There was not much opportunity for improvement in the age and gender variables in our study because they were already above 90% in the original evaluation [Kelly, 2003]. Reporting of active diseases, social history, weight, race, other drugs and dose had frequencies ranging from 25 to 80% as seen in Figure 3 for the patient variables and Figure 4 for the drug variables. These same variables ranged from 0.3 to 70% in the Kelly study. Use of an objective causality tool increased significantly in our study with use in 25% of cases (p < 0.001) compared with <1% in the previous evaluation [Kelly, 2003].

Figure 3.

Comparison of 15 patient variables reported in case reports.^a,b

^aFor the Phase 1 comparison, statistically significant differences (p < 0.05) were observed between frequencies for the majority of patient characteristics except alcohol use and illicit drug use.

^bFor the Phase 2 comparison, statistically significant differences (p < 0.05) were observed between frequencies for the majority of patient characteristics except alcohol use and tobacco use.

ADR, adverse drug reaction.

Figure 4.

Comparison of eight drug and ADR variables reported in case reports.^a,b,c

^aFor the Phase 1 comparison, statistically significant differences (p < 0.05) were observed between frequencies for most drug characteristics except serum drug concentration.

^bFor the phase 2 comparison, statistically significant differences (p < 0.05) were observed between frequencies for all drug characteristics.

^cUse of an objective tool was significantly (p < 0.05) improve for Phase 1 and Phase 2.

ADR, adverse drug reaction.

A specialty journal comparison is provided in Figure 5. The critical care medicine journals reported serum drug concentrations more frequently. Nursing journals reported the variables tobacco use, severity of illness, ADR mechanism, indication and duration of therapy more frequently. Pharmacy journals reported more for dose, other drugs, patient weight, renal function, liver function and alcohol use. Overall, the pharmacy journals were the most improved of the specialties compared with the Kelly study.

Figure 5.

Comparison of 23 patient, drug and ADR variables for specialty journals in Phase 1.

ADE, adverse drug event; ADR, adverse drug reaction; CBC, complete blood count.

Phase 2

The comparison with the 23 variables evaluated by Kelly indicated that 21 variables had significant differences [Kelly, 2003]. Significant improvements in reporting were not observed for alcohol and tobacco use. The significant differences were improvements in reporting for most variables as seen in Figure 3 and Figure 4 for patient and drug characteristics, respectively. Illicit drug use, mechanism for ADR and route of administration were significant reductions in reporting, not improvements. A causality assessment tool was used in 11.5% of cases for the general journal evaluation.

The following criteria of the Adverse Drug Reaction Probability Scale were discussed in the 208 case reports: 80.3% for previous published cases of this ADR; 59.1% for alternative causes; 99.5% for temporal relationship; 80.3% for dechallenge; 34.1% for rechallenge; 5.8% for drug concentrations; 1.9% for placebo administration; 11.5% for response to dosing adjustment; 15.9% for reaction to similar drug; and 99.0% for signs/symptoms.

Discussion

A comparison of our work with previous data, addressing the quality of case reports approximately two decades earlier, indicates that there is an improvement in the quality of case reporting for many variables, but there is still an opportunity for further advancement. Despite the significant improvement in reporting of active diseases, social history, weight, race, other drugs and dose, there is still an opportunity for more improvement with frequencies in reporting ranging from 25 to 80%. For Phase 2, there were significant reductions in reporting for illicit drug use, mechanism for ADR and route of administration. Guidelines published to assist writers in providing informative data are available; however compliance with these recommendations is not required for publication. In addition, some guidelines could be more specific as related to an ADR. For example, the inclusion of an ADR causality assessment tool or the possible mechanism for the ADR could be included in an ADR specific case report guideline [Gagnier et al. 2014]. Some guidelines indicate ‘when relevant’ to provide patient and drug information; however one’s interpretation of what is relevant is subjective (especially by discipline) and what is relevant may change over time as more is learnt about the ADR.⁷ Instead of the option of reporting data that are relevant, a specific checklist for patient, drug and ADR details should be followed for case reporting. The potential misinterpretation of relevance highlights the importance of providing negative findings, as these may also be deemed pertinent at a later point [Cohen, 2006].

The Adverse Drug Reaction Probability Scale is the ADR causality instrument used most commonly in practice and for published case reports [Kelly, 2003; Kane-Gill and Devlin, 2006]. It consists of 10 questions used to assess the causality of the suspected drug and reaction containing criteria such as alternative causes of the event, dechallenge, and temporal relationship [Naranjo et al. 1981]. The Adverse Drug Reaction Probability Scale and other available causality instruments contain important criteria to consider in making an ADR determination. These patient and drug characteristics are relatively consistent across published instruments [Agbabiaka et al. 2008]. The criteria described in causality instruments should be reported if positive or negative to allow readers sufficient information for interpretation and application in clinical practice. The Adverse Drug Reaction Probability Scale was used to assess the quality of reporting for case reports of ADRs related to psychotropic drugs in a previous evaluation [Talat et al. 2013].

We used the Adverse Drug Reaction Probability Scale to evaluate the quality of case reports and determined a need for improvement in reporting for criteria such as previously published case reports, alternative causes, dechallenge and rechallenge. It is important to note that this expectation is to have these criteria discussed for negative findings (e.g. ‘no dechallenge attempted’ or ‘no alternative causes identified’) to ensure a complete understanding of the case. It is understandable that the use of a placebo is unlikely in case reports but the remaining causality assessment criteria do apply.

Our findings are consistent with the previous evaluation of quality of reporting for psychotropic drugs by Talat and colleagues with 17.5% (7/40) of cases evaluated having insufficient information to confidently apply a causality assessment instrument [Talat et al. 2013]. Use of an objective causality tool increased significantly with use in 25% and 12% of cases for specialty journals and general journals, respectively. There is still a need for greater improvement in the use of a causality instrument and sufficient data to apply a causality instrument by the reader of the case report.

Phases 1 and 2 of this evaluation afford the opportunity for a descriptive comparison of specialty and general interest journals. The specialty journals seemed to have a greater improvement in reporting of patient data than the general interest journals. These findings may be a result of the specialty journals taking a more detailed approach to describing their patient population to the readership.

One of the purposes of case reports is to glean valuable information on rare or unique events when large published trials are not executable or available.¹ In a specific example, Ramasamy and colleagues reviewed over 70 years’ worth of data to better understand the rare occurrence of allopurinol hypersensitivity [Ramasamy et al. 2013]. Their evaluation primarily relied on a substantial number (n = 267) of case reports and cases series. The authors commented: ‘The absence of complete and important data in most case reports continues to hinder our ability to draw more concrete conclusions.’ This assessment highlights the need to publish high quality case reports to allow for possible influences on patient care.

Limitations

This evaluation does have some limitations. The Phase 1 assessment of specialty journals included 10 cases from the literature for nursing journals because fewer ADR case reports were available in the specified time frame. Case reports for Phase 2 may not be a representative sample of the entire body of case reports since the search was limited to one MESH heading. Also, if ADR cases were unavailable then a random replacement was selected. Multiple reviewers were used for case report evaluation after training of the standardized form.

The goal of our study was to see if the data were discussed in the case and not to do any type of clinical assessment, so the use of multiple reviewers was considered an acceptable approach.

Our comparator study by Kelly evaluated case reports available in Clin-Alert abstracts. While Kelly found that Clin-Alert abstracts were 97% accurate and 98% complete when compared with a random sample of original published case reports, it is possible that use of original published case reports would have yielded slightly improved findings. Our study used original published case reports and hence may have slightly better results than Clin-Alert abstracts.

Conclusion

It is noted that progress has been made in ADR case reporting quality, but more improvement is still needed in both specialty journals and general themed journals. The reporting of negative or normal findings should be considered as part of a necessary component within guidelines and not viewed as a subjective contribution of relevance. This evaluation illustrates that there is a need for improvement in the quality of case reports to ensure data are understandable and relatable to patient care. In addition, there is an opportunity to standardize ADR reporting in the literature, beyond case reports. Standardization of ADR characterization and reporting through clinical trials, and institutional event reporting systems could greatly define and prioritize opportunities for improvements in drug safety and the management of ADRs.