Content and Usability Evaluation of Patient Oriented Drug-Drug Interaction Websites

Abstract

Drug-Drug Interactions (DDI) are an important source of preventable adverse drug events and a common reason for hospitalization among patients on multiple drug therapy regimens. DDI information systems are important patient safety tools with the capacity to identify and warn health professionals of clinically significant DDI risk. While substantial research has been completed on DDI information systems in professional settings such as community, hospital, and independent pharmacies; there has been limited research on DDI systems offered through online websites directly for use by ambulatory patients. The focus of this project is to test patient oriented website capacity to correctly identify drug interactions among well established and clinically significant medication combinations and convey clinical risk data to patients. The patient education capability was assessed by evaluating website Information Capacity, Patient Usability and Readability. The study results indicate that the majority of websites identified which met the inclusion and exclusion criteria operated similarly, but vary in risk severity assessment and are not optimally patient oriented to effectively deliver risk information. The limited quality of information and complex medical term content complicate DDI risk data conveyance and the sites may not provide optimal information delivery to allow medication consumers to understand and manage their medication regimens.

Introduction

Adverse drug events, including injury and death, have been reported to affect up to 1.5 million patients annually, according to an Institute of Medicine report on Preventing Medication Errors. Exact numbers are difficult to assess, due to lack of uniform reporting, but adverse drug events are estimated to add 2–4 days to hospital length of stay at a cost of $2500–$5500 per patient^1,2,3. A particular subset of adverse drug events, drug-drug interactions, occur frequently among community and ambulatory care settings with estimates on patient risk of exposures to drug-drug interactions ranging from 2.2% up to 70% of patients, depending on the population studied and the methods used in the study.^4,5

Drug-Drug Interaction (DDI) information systems are essential tools to identify medication combinations which may place patients at risk of adverse drug events (ADE) including the potential risk of major morbidity and mortality⁶. DDI information systems have the capacity to help identify, analyze, and facilitate the avoidance of potentially dangerous or even lethal drug interactions both as stand-alone applications or as integrated decision support in electronic medical record systems. The availability of DDI systems to health professionals includes clinical databases, DDI assessment systems, web-based tools and clinical phone applications that evaluate patient risk via information delivery and clinical decision support^7,8,9,10.

Clinical content on websites provide easy access points for patients or drug consumers to efficiently gather the necessary information to understand medication therapy concerns. However, if patients receive clinical information which lacks important clinical context they may be at risk of potentially unwanted adverse drug events due to the lack of information or inaccurate information. According to the Food and Drug Administration in the year 2012, more than 325,000 serious health outcomes and 54,000 deaths resulted from ADE, providing an important concern on the safe delivery of medications in healthcare¹¹. With the estimated prevalence of DDI mediated ADE varying widely, it is not well established as to what percentage of ADE are directly associated with drug interactions, however, the DDI are a major clinical concern and patient safety opportunity since many of the DDI-mediated ADE are potentially preventable.

Patient oriented DDI Information systems are publically available computer databases available via websites, which have been created to inform patients and drug consumers about relevant drug interactions to help drug consumers stay informed on their medications and assist in ADE prevention. The typical clinical provider-oriented DDI information systems provide information on potential DDI, drug-food interactions, or even drug-alcohol interactions to help the clinician understand potential risk. The purpose of this study is to assess whether free and publically available patient-oriented DDI assessment websites function in similar ways and contain accurate and reliable information concerning DDI risk. Furthermore, the study will assess if these publically available websites contain DDI risk content which is patient oriented in the risk communication information content (multimodal with graphics and risk cues) and the required reading level for the estimated risk information (6–8^th grade reading level).

Background

Current DDI information systems exist primarily in professional settings such as community and hospital pharmacies and electronic prescribing systems for use in clinical decision support by medication prescribers and dispensers. The presence of information systems has been successful in identifying potential DDI exposures at the point of care. In an extended study done by Abarca et al., the study evaluated the performance of DDI screening software in identifying clinically significant DDIs in pharmacy computer systems in community and hospital pharmacies. They used 25 medications with a total of 37 drug pairs and 16 clinically significant drug pairs to create 6 mock patient profiles.¹² Although there are improvements in the DDI screening systems, additional aspects needed to be addressed included the ability to correctly detect clinically significant DDI in hospitals and understand the exact severity of the potential interactions¹³. Later assessment of clinical pharmacy systems also show some concerns still persist in DDI evaluation in the pharmacy setting.¹⁴

DDI information systems are common in pharmacies; however the data from these DDI information systems must also interface with information systems from Pharmacy Benefit Management (PBM) companies and electronic prescribing software¹⁵. In research work focused on the prescription claims data base of a PBM company, Malone et al. performed a retrospective cross-sectional analysis of 46 million participants for 25 clinically potential significant DDIs.¹⁶ The study covered a 25 month time frame and showed that an estimated 374,000 participants may have been exposed to clinically important DDIs. The study also showed higher potential DDIs for individuals based on gender and age differences. These results indicate that even with the presence of available information systems for DDI risk screening, patients may still end up using at-risk medication combinations.

In assessing potential DDI risk, scoring systems have been developed to identify the severity of the potential DDI. Prior work by Oshikoya et al. determined drug-drug interactions and severity ratings for patients with the human immunodeficiency virus. They categorized drug-drug interactions and assigned a score on a scale of A (no known interaction); B (minor/no action needed); C (moderate/monitor therapy); D (major/therapy modification); and X (contraindicated/avoid combination).¹⁷ In addition, their results showed discrepancies between databases used such as Medscape and USA MIMS. Having a means to score the potential DDI risk is needed for clinical decision making to weigh the potential risks and benefits of medication combinations.

Previous efforts have indicated a lack of effective DDI software screening in the pharmacy setting and limited ability to avoid clinically significant DDIs with PBM care delivery. Even with this knowledge, there has been a lack of research focused on the DDI mediated adverse drug effects in the outpatient setting. The research approach is an evaluation of publically available information for patient and/or caregivers concerning DDI. The goal of the study is to analyze individual websites and determine if currently available information is valid and accurate with regards to clinically significant DDI for drug consumers. The approach of the our research follows a similar approach to Oshikoya¹⁷ in classifying variations that may exist between information content concerning severity ratings of important DDI from the project websites. The significance of this research is to determine if the DDI information offered to the general population has plausible clinical risk assessment guidance and provides the necessary information to identify clinically significant DDIs. As previous research has highlighted, discrepancies in DDI screening systems are present in both pharmacies as well as PBMs, this research work aims to assess for variations in the information given to consumers about DDI risk.

Methods

DDI websites

The study focused on identifying available websites which had the capacity to inform medication consumers on potential DDI risk and were freely available to medication consumers. A total of 65 eligible websites were identified with expert selection and the search engine Google using search terms including drug-drug interaction, drug-drug interaction checker and drug-drug interaction assessment search terms. The search engine identified sites were supplemented with additional websites identified by expert informaticists and clinical practitioners. The majority of the sites were identified from the Google search effort. The identified sites were found and confirmed to be online and available during the July to September 2014 time period of primary data collection.

Inclusion Criteria

Websites were included in the study sample if they were freely available to external end-users with direct access to the site or with the use of an input email address for user authentication and potential use for profile tracking. English language sites were included in the study group.

Exclusion Criteria

The necessity for payment information or the lack of DDI checking functionality resulted in study exclusion. The websites must also have each of the clinically significant DDI medications available in their databases at the drug level. If the level of drug matching was only available at the therapeutic class level, the websites were excluded. All non-English language sites were excluded from the study. A total of 44 websites were included in the study after applying the inclusion and exclusion criteria.

DDI medication selection

The drug combinations in this descriptive, cross-sectional analysis of online DDI websites included a total of eight drugs and five DDI pairs which incorporated medications for multiple disease states (e.g. high cholesterol, fungal infection, etc.) to provide relatively broad medication coverage of key therapeutic areas at potential risk of DDI events (Table 1). The clinically significant DDI drug pairs were selected based on the previously published literature¹³ which was developed from expert consensus and large database validation. The selected medication combinations included high risk, high prevalence and potentially preventable drug combinations which could be expected to put patients at risk of adverse events. The presence of alerts for therapeutic class duplication was also measured through the Selegiline-Phenelzine drug pair to identify sites having the ability to assess multiple drug profiles where all the medications in the study set were built into a comprehensive drug profile. Some websites did not have multiple drug assessment (3 or more medication) capacity and could only assess binary medication pairs. Since all the drug pairs have been well established to be clinically significant DDI, it was expected that each should provide an alert when entered into the websites with similar estimated severity risk.

Table 1:

Drug Pairs

Simvastatin & Itraconazole
Warfarin & Gemfibrozil
Warfarin & Levothyroxine
Fluoxetine (SSRI) & Selegiline (MAO-I)
Selegiline & Phenelzine (MAO-I)

For website data entry, the medications were recognized by the generic name as well as brand names and could be recognized as an oral tablet or oral solution in regards to the route of medication. All websites included in the study were able to correctly identify either the generic or brand name of each drug for the interaction assessment.

Data Collection

The DDI website information performance was assessed for clinical content and patient usability. The first method assessed the clinical content of DDI information system severity ratings offered to patients. Each website visited was noted for the presence or absence of a DDI severity rating. Four ratings were identified including contraindicated, major, moderate, or minor interactions. In cases where there was not a specific standard definition on the website, a normalized approximation of the standard score was identified from the DDI data and mapped to the four level standard. The patient oriented data assessment included information capacity scores, patient usability, and readability.

DDI Information Capacity Score

For the information capacity score, each website was quantitatively graded on the criteria including: presence or absence of a drug-drug interaction alert, presence or absence of severity grading, severity rating scales explanation, presence or absence of drug-food interaction, presence and absence of drug-alcohol interaction and therapeutic duplication. Each website that contained the complete information in regards to each of the five drug pairs for each of the categories received points. The websites were graded with five points in three categories; DDI presence, severity rating, and drug-food interaction. Another area graded, Drug-alcohol interaction, received a max of four points (one for each drug pair), however, because the drug pair simvastatin and itraconazole are recognized to have limited interactions with alcohol it was excluded from the scoring. The last category scoring was for therapeutic duplication which received a maximum score of two points. The first point identified whether the sight was able to identify the duplication in drug therapy class and a second point for identifying the drug classes which were responsible. This point scale was used to determine the information capacity of the website and had a maximum total of 21 possible points. A summary of the scoring rubric for the DDI Information Capacity score is noted in Table 2

Table 2:

DDI Information Capacity Score

DDI Identified	Severity Rating listed	Drug-Food Reaction	Drug-Alcohol Reaction	Therapeutic Duplication	Max Score
0–5	0–5	0–5	0–4	0–2	21

Patient Usability

The patient usability score focused on how a consumer would approach a DDI assessment website and how they would assess if the content was patient friendly and contained clinically useful features. Ideally these features were present and represented best practices for drug information management and patient risk communication. One point was received in categories for alert icons, color coding, and severity rating scale presence, medication pick list, and login/profile management for a total of 5 possible points. Alert icons are important as they serve as a more universal image of communication which could help patients understand risk even if language barriers or reading level of the patient was limited. Color coding can help to differentiate severity data, with typical use of the color red being associated with danger and green lights associated with low risk. Severity rating scale data was deemed important in determining how dangerous a DDI exposure is to the patient. Medication pick lists are crucial for the accuracy of navigating and identifying correct drug interactions for the drug consumer. Having login/profile management for multiple drugs allows the user to easily remember the drugs they’ve searched and avoid the inconvenience of having to re-enter a long list of medications. All of these features provided grading parameters for the patient usability scores. Half points were received if the website failed to provide these features for all five drug pairs but had the functionality of some drugs. For most websites, these features were all or none phenomena and would work or not work for all of the drug pairs.

Patient Readability

Patient readability was determined through the Flesch-Kincaid grading model. This grading model was established by Rudolph Flesch and Peter Kincaid based on the understanding that information is more understandable with fewer words and shorter sentences.^18,19,20 The content of information for the simvastatin and itraconazole interaction for each website was the only information content which was graded for readability. The content relevant to the drug consumer was assessed using the Microsoft 2010 Flesch Kincaid scoring assessment. Due to the potential for the commercial or generic drug names to affect this grading model, each of the drug names were replaced with the generic word “drug” instead of simvastatin and itraconazole to decrease the grade levels that would have been achieved with those words in the consumer content. The Flesch Reading Ease score was also assessed for each of the sites to provide additional measures on readability.

Results

Severity Grading

The severity ratings measured from the websites demonstrated substantial variation. Approximately half of the websites did not provide severity rating for any of the drug categories. Although a substantial sub-population of the websites failed to give severity ratings, some indicated severity levels for the five drug pairs. For sites with severity ratings, about 50% indicated a moderate severity for the drug pair Warfarin-Gemfibrozil and the other 50% indicated a major severity for the same drug pair. For the Warfarin-Levothyroxine drug pair, 14 of the 44 drug pairs indicated a moderate risk associated with these two drugs while 5 of the 44 websites indicated that this drug pair had a major risk. For the Fluoxetine-Selegiline drug combinations, 12 websites noted that the interaction was contraindicated while five websites noted that it was a major severity reaction and one website noted that it was a moderate reaction. For the Selegiline-Phenelzine drug combination, nine websites noted that this drug pair was contraindicated; three websites noted that it was a major reaction, and one website noted that it was a moderate reaction. The DDI severity data is summarized in Figure 1.

Figure 1:

Drug-Drug Interaction Severity Frequency

Information Capacity Scoring

Among the 44 websites included in this research, the mean Information Capacity score was 13.36 points with a standard deviation of 3.09 points. The highest scoring website had a score of 21, which was the maximum possible score. Two websites scored 19, which was just below the maximum possible scores. The lowest scoring website had a score of 4 and the second lowest score belonged to three other websites with a total score of 8 each. 42 of the 44 websites lacked the Therapeutic Duplication feature. The results for all the DDI Information Capacity scores are noted in Figure 2.

Figure 2:

DDI Information Capacity Score Distribution

Patient Usability Scoring

The average score for the Patient Usability score for all 44 websites was 2.9 points with a standard deviation of 1.06 points. The distribution of the scores is noted in Figure 3. Only five websites received perfect scores for patient usability scoring. The scores that received half a point for some information features failed to have that specific feature for all drug pairs. The most common features for half points given were alert icons and severity rating scales.

Figure 3:

Patient Usability Scores

Website Readability Scoring

The average Flesch-Kincaid grade level scored was 11.2 with the highest grade level being 16.2, the lowest grade level being 6.7 and a standard deviation of 2.29 with the results summarized in Figure 4. The Flesch Reading Ease score was also measured for the 44 websites was 40.27 with the highest score being 70.9, the lowest score being zero and the standard deviation being 15.11 with the results summarized in Figure 5.

Figure 4:

Reading Level of DDI websites

Figure 5:

Website Reading Ease Measures

Information on Drug-Drug Pairs Identified

Of the five drug pairs included in this study, three drug-drug interactions were not correctly identified by all of the DDI checkers. The two drug pairs, Simvastatin-Itraconazole and fluoxetine-selegiline, were correctly identified for a potential interaction by all of the DDI checkers. All but one website correctly identified the Warfarin-Gemfibrozil drug interaction. Another website failed to identify the Warfarin-Levothyroxine drug pair, but correctly identified all other drug interactions. 20 websites had identical results likely indicating a common DDI assessment engine in use.

Discussion

The study findings indicate that the many of DDI Checkers operate similarly or even identically as would be expected for a series of well-established DDI medications. Of the 44 websites, 20 had identical scores with an Information Capacity score of 13, Patient Usability Score of 2, and Flesch-Kincaid Grade Level of 12.4. The results also show that the majority of these websites may not be optimally patient-oriented with a low average Information Capacity Score and an average Flesch-Kincaid grading level of 11.2. The optimal Flesch-Kincaid reading score for patient oriented content would be lower with grade levels of 6–8 to be easily readable by most patients from a variety of educational backgrounds^18,19,20. The average Flesch Reading Ease was also lower than expected for ease of use by most readers and indicated that the information given to patients may be difficult to interpret.

The research provides important insights on publically available DDI information systems for well-established DDI combinations. The average Information Capacity Score of 13.36 implies that there are many features that are overlooked by the DDI checkers. Scores not received due to lack of a severity rating was the primary reason for the low scoring in Information Capacity. Other features that contributed to a decreased Information Capacity scores included the absence of the Selegiline-Phenelzine drug interaction, lack of presence for drug to food interactions, absence of drug-alcohol interactions, and absence of the therapeutic duplication data. Reasons for the lack of drug-food interaction and drug-alcohol interaction may have been due the fact that the websites focused on strictly drug-drug interactions or may have lacked the content in their database.

Another broader concern was the variability in severity ratings among the websites which provided severity data. The interactions used in the assessment are well known and clinically significant. Although the vast majority of sites indicated the presence of an interaction, the lack of severity data and lack of consistent severity data is potentially problematic. If a patient were to put in a medication combination which has a low risk of adverse clinical effects and gets no cues on severity and at the same time inputs one or more of the combinations included in this study, they may get the message that the low-risk combination is the same risk as the high risk combination. This has the potential to provide inappropriate messaging depending on how the patient perceives and internalizes the lack of risk severity. It is also easy to see a scenario where a patient may have a complicated regimen with multiple interactions and asks their physician or pharmacist about the interactions and gets reassurance on the first couple of interactions and then fails to ask or appreciate the risk for a high risk combination further down the list. Among the medications used in the study, there was also an unexpectedly high variation in severity rankings when it was provided. It would be expected that the severity data would be more homogenous, but in figure one the range went the full range of minor to contraindicated which is concerning from a patient safety standpoint and none of the pairs had full agreement on severity when it was provided. In addition, some of the interactions were not noted as DDIs on some sites.

The patient usability and reading levels could be addressed by insuring the content has multiple modes of risk communication with percentage data, risk symbols and use of color coding or other pictographic elements to convey potential risk when present. For the readability and reading ease, the DDI information system content could use more simplified explanations so as not to overwhelm the drug consumer with the information content and have less use of medical jargon. The high grade level to the content may have been due to the reuse of risk information developed primarily for use by health professionals.

Limitations to the study include a lack of prior studies in DDI information capability to use for scoring and assessment. Without a well-recognized grading criterion for patient oriented DDI information systems, generating one would require a consensus among professionals as to how important each grading factor was in evaluating these information systems. Future work to validate the scoring approach is being planned to assess scoring validity. Another limitation included a small sample of drug pairs as well as limited number of drug classes which were studied which may have affected the study outcomes. There are many more clinically significant drug interactions that were not included in this study and further research could provide additional insight with an evaluation of those drug combinations. Ideally more drug combinations would have complemented the research, but with the time and effort needed to complete the thousands of potential drug interactions it would require a large labor effort. Furthermore, additional DDI combinations are likely to have diminishing returns. Also of note, many of the websites generated identical results and it would be expected that the pattern would continue with testing of additional drug pairs.

An additional limitation was the lack of actual patient feedback on the use of the systems. However, without systematically screening and assessing the quality of the information provided, putting patients with their actual drug regimens was deemed potentially problematic since counter-detailing may have been required based on clinical circumstances and the patient guidance provided by the sites. Assessing patient initiated mock drug profiles may be an alternative approach to get pertinent patient data on website usability.

Conclusion

The results of the research indicate that the content of these websites is similar but a large number of sites lack severity data, advanced drug profile management, and therapeutic class assessment. In addition, there is a broad problem of limited patient readability and limited patient oriented risk communication features. Based on the study results, further research and website development work is needed to improve patient oriented DDI information systems.

Possible approaches to improve patient oriented DDI checking sites could be the incorporation of a standardized severity grading scale to avoid potential confusion for the drug consumers on DDI risk. Improvement of patient oriented DDI information systems should also focus on simplifying drug interaction information provision to consumers with patient oriented content and inclusion of advance drug information management features.

Table 3:

Patient Usability Scoring Method

Alert Icons	Color Coding	Severity Rating Scale	Medication Pick List	Login/Profile Save for Multiple Drugs	Total Score
0–1	0–1	0–1	0–1	0–1	0–5