The impact of drug order complexity on prospective medication order review and verification time

Abstract

Objective

To assess if the amount of time a pharmacist spends verifying medication orders increases as medication orders become more complex.

Materials and Methods

The study was conducted by observing pharmacist verification of adult medication orders in an academic medical center. Drug order complexity was prospectively defined and validated using a classification system derived from 3 factors: the degree of order variability, ISMP high-alert classification, and a pharmacist perception survey. Screen capture software was used to measure pharmacist order review time for each classification. The annualized volume of low complexity drug orders was used to calculate the potential time savings if these were verified using an alternate system that did not require pharmacist review.

Results

The primary study hypothesis was not achieved. Regression results did not show statistical significance for moderate (n = 30, 23.7 seconds, sd = 23.3) or high complexity (n = 30, 18.6 seconds, sd = 23.1) drugs relative to the low complexity drugs (n = 30, 8.0 seconds, sd = 14.4) nor for moderate vs high complexity; (β_{moderate vs low} = 15.6, P = .113), (β_{high vs low} = 10.3, P = .235), (β_{moderate vs high} = 5.3, P = .737). The sensitivity analysis showed statistical significance in the high vs low comparison (β_{high vs low} = 13.8, P = .017).

Discussion

This study showed that verifying pharmacists spent less time than projected to verify medication orders of different complexities, but the time did not correlate with the classifications used in our complexity scale. Several mitigating factors, including operational aspects associated with timing antimicrobial orders, likely influenced order verification time. These factors should be evaluated in future studies which seek to define drug order complexity and optimize pharmacist time spent in medication order verification.

Conclusion

The findings suggest that there may be other factors involved in pharmacist decision-making that should be considered when categorizing drugs by perceived complexity.

INTRODUCTION

There are many patient care demands which require the attention of a pharmacist. A substantial amount of pharmacist time is devoted to prospective medication order review (PMOR) and verification. PMOR activity is a regulatory practice standard of the American Society of Health-System Pharmacists (ASHP), the Joint Commission (TJC), and other agencies, and a legal requirement of most state boards of pharmacy. PMOR has been part of pharmacist workflow for decades. PMOR is a multistep process that requires intentional, prospective examinations of the details of medication orders with regard to their safety, anticipated efficacy, appropriateness for the patient, and proper product selection prior to dispensing the first dose.^1,2 PMOR generally concludes with order verification, which indicates that the order directions are safe and appropriate to carry out, or order clarification, which may result in order cancellation, modification, modifications of other orders, or additional laboratory orders or monitoring parameters. Since 1995, ASHP considers PMOR to be a minimum standard for pharmacies in hospitals. PMOR was described by Flynn in a 2009 commentary where he states “as a profession, we have chosen to mandate that prospective order review by pharmacists is essentially nearly universal—hence the term nearly universal prospective order review (NUPOR)”.³ Of note, Flynn argues that the patient benefits achieved from pharmacist PMOR may be negligible for certain, “low-risk” orders. With proper application of clinical decision support to identify anomalous orders, pharmacist review of these orders might not be necessary, enabling pharmacists to devote more time and attention to complex medication use activities with a greater likelihood for affecting patient care.

Determination of drug order complexity is an abstract and perhaps subjective concept. Some literature indicates that medication regimen complexity can be defined and validated,⁴ but many limitations exist. Much of the complexity work has been done in an outpatient setting with regimens and drugs used in that population and without regard to factors such as severity of consequences and the degree of variability of dosing regimens. The interpretation and understanding of pharmacist clinical decision-making is comparable to models seen in other health disciplines such as the clinical reasoning cycle in nursing practice.^5–7 Pharmacists inherently consider complexity based on intrinsic and personal factors when reviewing medications that stem from their own experiences and biases. These considerations are part of the cognitive effort embedded in the intricacies of decision-making. Cognitive effort has many definitions; 2 authors describe it as “the degree of engagement with demanding tasks” and as “the total amount of cognitive resources—including perception, memory, and judgment—needed to complete a task.”^8,9 Cognitive choices are made by the pharmacist in prioritizing the selection sequence and effort during medication order verification. Factors considered by pharmacists during order verification and dispensing could include the perceived medication priority/urgency (eg, stat), the time the medication is due, the dwell time of the order in the verification queue, the type of medication, and others.¹⁰ Once an order is selected for verification, pharmacists must decide if they have the necessary information to confidently verify the product based on a reconciliation of their perception of complexity with their professional knowledge and judgment.

Perception and cognition are interrelated and perceptual information guides decisions and actions while shaping beliefs.¹¹ A pharmacist’s knowledge and years of clinical experience may influence the way they perceive drug(s) during medication review. This interconnectedness demonstrates how cognitive information can influence perceptual processes, while understanding that cognitive processes also depend on perceptual information.¹¹ When good decisions are made, they should be ethical, evidence-based, measurable, and impactful. One must gather information and evaluate the benefits/consequences before putting the decision into action. Every pharmacist partakes in clinical decision-making when verifying medication orders. During order review and verification, pharmacists must understand patient-specific parameters, refer to reliable evidence before application to the patient, and assess monitoring parameters before verifying or making changes as appropriate. Various patterns and processes of pharmacists’ clinical reasoning and thinking was examined in community pharmacists, demonstrating that these patterns and processes can help pharmacists, in any pharmacy practice environment, make decisions that result in positive contributions to patient care. However, there is limited evidence about the clinical reasoning and decision-making processes pharmacists use when verifying medication orders of varying drug order complexity.⁵ Awareness of the processes that guide clinical decision-making and performance by pharmacists is valuable because they are required to rapidly and accurately make decisions about the safety and appropriateness of medication orders for a large number of patients, often with very diverse health needs.^5,10,12 Despite these complex processes, we are unaware of any evidence that characterizes errors of commission or omission by pharmacists at the point of PMOR and verification. Errors likely occur but, presumably, at a low rate. Similar to any peer review process, judgement is involved and there would likely be considerable subjectivity in designating variances in practice as errors. The literature does support that hospital pharmacists are regarded as major contributors in identifying and reducing medication-related errors, and adverse effects through PMOR.^13–15

A comprehensive literature review of PubMed, OVID Medline, and Google Scholar was performed with these search terms used alone or in combination: “drug order complexity,” “pharmacist order review,” “cognitive effort,” “decision-making approaches,” and “reasoning.” Subsequently, to our knowledge, there are no studies that systematically evaluate the cognitive effort undertaken by inpatient pharmacists related to PMOR. The ability to accurately delineate complexities within a medication order before verification requires both critical thinking and decision-making. These essential skills have a profound impact on patient safety yet remain an unexplored area of research in the pharmacy domain. As an initial step to considering cognitive effort and pharmacist decision-making during PMOR, we aim to assess pharmacist time spent during PMOR and verification to better understand how drug order complexity and time interrelate. This study will provide insight on pharmacist order verification processes and it represents a foundational exploration into Flynn’s hypothesis that the time saved from verifying standardized, low-complexity orders could potentially be re-allocated to other patient-centered activities.³ Enhanced understanding about pharmacist cognitive activities could provide pharmacy managers and supervisors with information that could help them optimize and prioritize pharmacist time devoted to patient-care related activities.

Research Questions. The hypothesis for this study is that the amount of time a pharmacist spends verifying medication orders increases as medication orders become more complex. Two other research questions were described post hoc: 1) How much time is required by inpatient pharmacists to review and verify medication orders of 30 commonly ordered drug-route dyads using a modern electronic medical record (EMR) system? and 2) How much inpatient pharmacist work time would be saved if low complexity medication orders with commonly ordered drug-route dyads were no longer prospectively reviewed and verified by inpatient pharmacists?

MATERIALS AND METHODS

Setting. The study was conducted in an adult pharmacy satellite secluded from patient care at a 1000-bed tertiary care and academic medical center. Pharmacist activities were tracked during an 8-hour shift away from patient care, during which between 700 and 1400 orders could be verified intermixed with a variety of other demand activities, such as responding to nurse questions and product checking and dispensing. The health system uses Epic (Epic Systems, Verona, WI) for its EMR. Pharmacists utilize Willow, the medication module of the EPIC EMR, to access the order verification queue and to prioritize and verify stat and routine patient medication orders placed in the system by providers. This study was reviewed by the Michigan Medicine’s Institutional Review Board and determined that it did not require IRB approval because it does not satisfy the definition of reviewable research, as it falls outside of the Common Rule and FDA definitions of human subject research.

Design. The study was conducted in 3 stages: 1) the development of a drug complexity classification system to prospectively classify drugs into low, moderate, and high complexity categories; 2) implementation and use of the TechSmith Morae (TechSmith Corporation, Okemos, MI) technology to monitor pharmacist behavior and order verification time; and 3) application of the complexity time data to order verification volumes to estimate time commitment for each classification.

Drug-route complexity classification system. A novel, 3-component method was developed to prospectively classify drug orders by complexity in order to quantify the amount of time required to verify medication orders of different complexity classifications. We utilized risk-based scoring components to create a complexity scoring or grouping for the study medications. The 3 components used in the classification system included (1) presence on the Institute for Safe Medication Practices (ISMP) high-alert list, (2) classification of drug variability by number of unique order sentences, and (3) score range from a pharmacist cognitive effort perception survey. These components, and their interrelationship in the categorical assignment of complexity, are described in greater detail below:

1. ISMP high-alert list: If the selected drugs were recognized on the high-alert list, they were noted with a “YES.” “NO” was noted if the drug was not recognized by ISMP as a high-alert drug.

2. Drug Variability/Unique Drug Order Sentences: We utilized the same methodology employed by Woods et al to define atypical orders by identifying the number of unique order sentences greater than 15 (deemed high variability), 6–15 (medium variability), and less than 6 (low variability) for our selected drugs.¹⁶ All verified medications (513 444 orders) in the adult hospital over a 2-month period were extracted from the EMR and ranked according to highest order volume for each medication. The top 30 medications by verified volume were selected; these were comprised of a variety of oral and intravenous medication orders. For every medication order, the generic medication name, route of administration, dose, dose unit of measure, and frequency of administration fields were concatenated into an order sentence.¹⁶ The number of unique order sentences was then counted. Medication orders were grouped into dyads according to the generic medication name and route of administration (eg, aspirin oral).¹⁶ The number of dyads, the total number of orders per dyad, the number of unique order sentences per dyad (eg, “aspirin oral 325 mg daily” is distinguished from “aspirin 650 mg daily” and represents 2 unique order sentences), and the relative incidence of unique orders within each dyad were determined. The dyads were categorized as high, medium, or low variability depending on the number of unique order sentences that characterized at least 80% of all orders in the dyad.¹⁶

3. Pharmacist Cognitive Effort Perception Survey: The 30 most frequently verified dyads above were reviewed by 2 pharmacists on the Pharmacy Management team (1 manager and 1 coordinator) to validate that they are commonly ordered at a high frequency. Next, a 4-question survey was created using Qualtrics, (SAP, Weinheim, Germany). The survey involved 3 demographic information questions that obtained their experience as a pharmacist, duration of employment, and experience with the EMR. These were followed by the final question that was presented as a continuous scale where the pharmacist indicated their perceived cognitive effort for each of the 30 drug-route dyads. For the cognitive effort question, survey participants were asked to respond to the following statement: “Please rate the following medications according to the cognitive effort you use when assessing the appropriateness of a medication order. The scale is continuous, in increments of 1 unit, where No Effort = 0 and Comprehensive Effort = 100. Choose what you believe to be the appropriate score relative to these 2 extremes.” A continuous scale was chosen to enable us to average the scores and ranked order of the drug-route dyads. The survey was tested for accuracy and clarity using face validation by the same 2 pharmacists, who also provide operational oversight of the pharmacy satellite. Subsequently, the cognitive effort perception survey was distributed to 60 inpatient clinical pharmacists working in the main University Hospital, Cardiovascular Center Hospital, and Children’s and Women’s Hospital. Twenty-seven pharmacists (45%) responded to the survey. The mean score for each drug-route dyad was calculated and a distribution plot was created contrasting unique drug order sequences (Y axis) with pharmacist survey mean score (X axis). The majority of responses concentrated in the mean score range of 26–35. Thus, cognitive effort was elected to be classified according to the following tiers: high cognitive effort (average score ≥ 35), moderate cognitive effort (average score 26–34), and low cognitive effort (average score < 26).

Drug-route complexity classification system validation. The pharmacist cognitive effort perception survey, in conjunction with the ISMP high-alert status and drug-route dyad variability, was used to create and subsequently validate a drug complexity scale as depicted in Table 1. Two of 3 factors from the classification system had to be met to be considered a high or a low complexity drug. High complexity drugs were considered to be those listed on the ISMP high-alert list, had a mean score of > 35, and/or have > 15 unique order sentences. Low complexity drugs were not on the ISMP high-alert list, had a mean score of < 26 and/or had < 5 unique order sentences. Moderate complexity drugs were drugs that fit neither classification (ie, they had other combinations of these criteria). The complexity classification system was validated with a group of 5 licensed pharmacists that do not participate in order verification but that work in clinical and/or administrative roles with medication safety programs or committees. The results of the validation are illustrated in Table 2. The validating pharmacists were provided the list of 30 drugs and asked to independently rate the drugs’ complexity as low, moderate, or high based on their personal perception and without foreknowledge of the data used in our classification system. Their perceptions of complexity for each drug-route dyad were noted before the derived complexity scale and its criteria were revealed. There was unanimous agreement between the consultants and the scale on 10 drug-route dyads. The classifications for the remaining twenty drug-route dyads were adjudicated among the consultants via consensus. In order to achieve the needed sample size (see Power analysis) and diversity of complexity, 10 drug-route dyads were selected for study. Four drug-route dyads were selected in the high and low classifications, so that these classifications would be slightly oversampled relative to the 2 drug-route dyads selected for the moderate complexity classification. Of the selected 10 drug-route dyads, 7 were from those unanimously agreed by the authors/validating pharmacists (all 4 in the high complexity, 3 of 4 in the low complexity) while 3 drug-route dyads of high frequency of verification were classified through consensus (2 moderate complexity drug-route dyads and 1 low complexity). The unanimously selected, low complexity drugs were all oral, while the unanimously selected, high complexity and consensus selected classifications were all intravenous.

Table 1.

Drug-route complexity criteria classification

ISMP	RPh Survey	Unique Order Sentences	Drug-Route Dyads
High Complexity
Y	>35	>15	Heparin IV
Y	>35	6<X<15	Lidocaine IV
Y	>35	<5	Fentanyl IV a , Metoprolol IV, Labetalol IV,
			Norepinephrine IV a , Amiodarone IV a ,
			Morphine IV a
Y	26<X<35	>15	None
Y	<26	>15	None
Moderate Complexity
Y	26<X<35	6<X<15	None
Y	26<X<35	<5	Oxycodone PO, Magnesium sulfate IV,
			Dexamethasone IV a
Y	<26	6<X<15	None
N	26<X<35	6<X<15	Prednisone PO
N	>35	6<X<15	Quetiapine PO, Tacrolimus PO
N	26<X<35	>15	None
Low Complexity
N	<26	<5	Senna PO a , Aspirin PO a , Amlodipine PO, Atorvastatin POa, Multivitamins PO, Acetaminophen PO
N	<26	6<X<15	None
N	<26	>15	None
N	26<X<35	<5	Potassium chloride PO, Lisinopril PO, Albumin IV, Gabapentin PO, Ondansetron PO, Furosemide IV
N	>35	<5	Piperacillin-tazobactam IV b , Cefazolin IVa,
			Pantoprazole IV, Lorazepam IV

^a BOLD = selected for study.

^b BOLD = selected for study and reclassified to moderate complexity.

Table 2.

Drug-route complexity classification validation

Drug-Route Dyad	Complexity Score	Consultant Opinion	Validation
Fentanyl IV a			Unanimous a
Amiodarone IV a			Unanimous a
Morphine IV a			Unanimous a
Norepinephrine IV a	High	High	Unanimous a
Heparin IV			Unanimous
Lidocaine IV			Consensus
Labetalol IV			Consensus
Oxycodone PO			Consensus
Lorazepam IV			Consensus
Prednisone PO			Consensus
Tacrolimus PO	Moderate	Moderate	Consensus
Quetiapine PO			Consensus
Magnesium sulfate PO			Consensus
Senna PO a			Unanimous a
Aspirin PO a			Unanimous a
Atorvastatin PO a			Unanimous a
Pantoprazole IV			Unanimous
Multivitamins PO	Low	Low	Unanimous
Cefazolin IV a			Consensus a
Acetaminophen PO			Consensus
Gabapentin PO			Consensus
Lisinopril PO			Consensus
Amlodipine PO			Consensus
Piperacillin/tazobactam IV a	Low	Moderate	Consensus a
Dexamethasone IV a	Low	Moderate	Consensus a
Albumin IV	Low	Moderate	Consensus
Potassium chloride PO	Low	Moderate	Consensus
Ondansetron PO	Low	Moderate	Consensus
Furosemide IV	Low	Moderate	Consensus
Metoprolol IV	High	Moderate	Consensus

^a BOLD = selected for study.

TechSmith Morae technology implementation. Morae Recorder (TechSmith Corporation, Okemos, MI) software was packaged and deployed to designated computer workstations used to record the medication order verification process and time.¹⁷ Based on the survey results and work schedules, the study was limited to experienced (licensed for greater than 1 year and have greater than 6 months of University of Michigan or EMR experience), verifying pharmacists who worked in the central satellite pharmacy. Pharmacists licensed for less than 1 year, all pharmacy residents, work shifts in peripheral pharmacy satellites, outpatient clinics, and clinical shifts where large numbers of order verification do not occur were excluded from the study. All 12 pharmacists were informed of the study prior to their shift and consented to participation for the recording session, but were blinded to the medications being monitored. Both workstation screens were recorded to capture ancillary activities, such as accessing external references and/or other patient-specific clinical data. Each session was saved to a designated folder on the Michigan Medicine shared drive accessible only to study investigators. The Morae Manager component was used to view and time the recorded sessions. The investigators were not present in the pharmacy during order verification.

Power analysis. In order to identify our target sample size, we simulated real world conditions using estimates for order verification times at each complexity level. The input parameters included the number of medications followed, the average number of times a drug was expected to be seen, and the average number of seconds for the high, moderate, and low complexity groups. The estimated average time estimates for low complexity, moderate complexity, and high complexity order verifications were set at 10 seconds, 20 seconds, and 50 seconds, respectively. We found that we would have 82% power with 30 observations each of low, moderate, and high order verification durations. The tests in the simulation used linear mixed model regressions.

Statistical analysis. Mixed model linear regressions were used to compare the average verification times between the high vs low complexity groups, moderate vs low complexity groups, and the moderate vs high complexity groups. Repeated measures analyses were applied to pharmacists’ multiple observations along with random intercepts for each drug to account for any correlated observations within the data. We also analyzed the data in a sensitivity analysis by excluding the 2 antibiotics, piperacillin-tazobactam and cefazolin, because they were both significant outliers within their complexity groups. We analyzed the complexity group variable using an all-pairwise analysis. The post-hoc family-wise error rate (FWER) was preserved at alpha = 0.05 by adjusting the regression P values with the Tukey adjustment method. Analyses were carried out using R version 3.5.1.¹⁸

Data collection and validation. Each medication order verification (ie, observation) was timed using an external stopwatch. Each drug was observed 30 times to achieve a sufficient number of observations to meet the power analysis sample size estimate. Order verifications with mouse inactivity of greater than 20 seconds were excluded to reduce outliers during data analysis as we deemed these to be situations where something may have distracted the pharmacist (eg, phone call, discussions with pharmacy staff, etc.). There were a limited number of orders (10) with mouse inactivity which could have resulted from an order problem that required a pharmacist intervention. These were also excluded because we did not know the nature of the intervention, the extent of time dedicated to the intervention, or if the inactivity was a result of order complexity. Orders excluded due to mouse inactivity and pharmacist intervention were replaced with substitute medication order verifications to achieve our designated sample size.

Once the pharmacist opened the specific medication order from the queue, the timer was measured until the “verify” button was clicked to complete the PMOR process and activate the order. Throughout the pharmacist verification, the number of internal EMR resources (notes, results review, etc.) and external sources (primary literature, Micromedex, Lexicomp, etc.) accessed were documented. To validate the time value, a second investigator timed a sample of 10% (30) of the 300 medication observations. If the second investigator’s times were demonstrated to be within ±10% of original time value, the original time was utilized for final analysis. If significant differences were seen in time (> ±10%), the original timekeeper was responsible for re-timing that verification by performing 3 additional timing efforts for each discrepancy. If the average time from the supplementary observations mirrored the second investigator’s observed time, then the second investigator’s time was used for the final analysis. Twenty percent (6/30 observations) confirmed the second investigator’s verification times and were used for the final analysis. To further ensure accuracy of overall verified times, the original timekeeper retimed 30 additional observations. All of these times were within 10% of the original observation time.

Application of the complexity time data. The amount of time that could potentially be saved and reallocated to other complex patient-related activities was calculated based on order volumes and average verification times. A 2-month sample of verified inpatient drug orders was summed and annualized (multiplied by 6) for the low complexity group in our study (Table 4). The frequency of ordered/verified doses for each study drug was multiplied by the average time to verification in seconds and converted to hours in order to make inferences of the potential time saved in the low complexity category if these orders were to be no longer verified by pharmacists as suggested by Flynn, et. al. The overall amount of time saved is dependent on the annualized number of orders, which varies based on patient volume, population needs, and prescribing behaviors.

Table 4.

Application of time complexity data

Findings Summary
Drug-Route Dyad	Complexity Score	Mean (seconds)	SD (seconds)	Resources Utilized Per Verified Order
Senna PO	low	2.9	6	0
Atorvastatin PO	low	3.1	4.3	0.2
Aspirin PO	low	6.5	9.3	0.43
Cefazolin IV	low	19.4	22.9	1
Dexamethasone IV	moderate	14	14.8	0.3
Piperacillin-tazobactam IV	moderate	33.4	26.2	1.37
Fentanyl IV	high	13.7	16.3	0.17
Amiodarone IV	high	15.3	18.9	0.53
Norepinephrine IV	high	19.8	22.5	0.33
Morphine IV	high	25.5	31.3	1.03
Amount of Time Saved with Low Complexity Group
Drug-Route Dyad	Mean (seconds)	Annualized Volume of Orders	Time Saved per Year (Sec; hours)	Time Saved per Month (Sec; hours)
Senna PO	2.9	37 062	107 480; 29.85	8957; 2.49
Atorvastatin PO	3.1	11 220	34 782; 9.7	2899; 0.81
Aspirin PO	6.5	23 580	153 270; 42.58	12 773; 3.55
Cefazolin IV	19.4	21 132	409 960; 113.88	34 163; 9.49
Totala	31.9	92 994	705 492; 196.01	58 791; 16.33

^aMean-12.5, volume-71 862, Time/year-295 532; 82.1, Time/month-24 627.67; 6.84 if cefazolin totals excluded.

RESULTS

The average number of seconds (and their standard deviation) were calculated by group and by drug. The distribution of the drugs is displayed in a box-and-whisker plot (Figure 1).

Figure 1.

Comparison of completion times between drugs.

The high complexity drugs had a mean of 18.6 seconds (sd = 23.1), the moderate complexity drugs had a mean of 23.7 seconds (sd = 23.3), and the low complexity drugs had a mean of 8.0 seconds (sd = 14.4). The ordering of the means for the 3 groups was different than expected from the primary hypothesis, with the moderate complexity drugs taking more time than the high complexity drugs. The time required by inpatient pharmacists to review and verify medication orders of the 30 commonly ordered drug-route dyads varied by drug-route dyad when measuring the 10 medications utilized in the study. Regression results (Table 3) with the Tukey adjustment method showed that the time required for verification was also not statistically significant for either the moderate or for the high complexity drugs relative to the low complexity drugs (β_{moderatevslow} = 15.6, P = .113), (β_highvslow = 10.3, P = .235), nor for moderate vs high (β_{moderatevshigh} = 5.3, P = .737). The sensitivity analysis excluding piperacillin-tazobactam and cefazolin (Table 3) did show significance in the high vs low comparison (β_highvslow = 13.8, P = .017), while the moderate vs low test (β_{moderatevslow} = 10.3, P = .193) and the moderate vs high test were not statistically significant (β_{moderatevshigh} = −3.5, P = .768).

Table 3.

Statistical analysis results

Regression Analysis			Sensitivity Analysis
Group	Slope	P Value b	Group	Slope	P Value b
Intercept	8.2	.043	Intercept	4.3	.096
High vs Low	10.3	.235	High vs Low	13.8	.017a
Moderate vs Low	15.6	.133	Moderate vs Low	10.3	.193
Moderate vs High	5.3	.737	Moderate vs High	−3.5	.768

^aStatistically significant.

^bTukey adjusted.

The chosen sample size played a role in testing of the probability that the primary study hypothesis was true. It was expected that the high complexity drugs would take longer to verify than what occurred. The results illustrate that moderate complexity drugs in the study had the longest mean verification times in comparison to the low and high complexity drugs. Accordingly, the moderate vs low complexity order comparison had the smaller P value since the moderate data had a higher mean than the high data. Moderate vs high complexity drugs were compared and the time difference was not statistically different. A sensitivity analysis of the data points generated statistical significance for high vs low complexity order at P = .017.

Table 4 illustrates the complexity score, average time until verification, and the number of resources utilized by drug. This table shows that the frequency of resources used for the 2 antibiotics, cefazolin and piperacillin-tazobactam, were among the highest. This was deemed to be an important outlier finding and led us to perform a sensitivity analysis excluding the antibiotics. Cefazolin was classified to be a low complexity medication and averaged 1 resource used for every verified order, while piperacillin-tazobactam was classified as a moderate complexity medication and averaged more than 1 resource used for each observation. Table 4 also demonstrates the amount time that could be potentially saved for each drug-route dyad if alternate procedures were used to verify these orders. When considering the approximate annual volume of orders for the low complexity drugs including cefazolin, the calculated average pharmacist time saved per year would be 196 hours if verification occurred via an alternate method. However, since cefazolin behaved as a more complex medication than our original low complexity categorization, excluding it from the analysis would result in only 4.1 seconds saved per order or 82 hours of pharmacist verification time per year.

DISCUSSION

Since Flynn’s 2009 commentary, which proposed stratification of medication orders based on risk, the concept of how to address the unfulfilled opportunity costs of, or alternatives to, NUPOR has not been robustly studied despite provocation from several influential commenters.^3,19–21 In order to achieve this, we first classified drug-route by complexity, then we determined the average time required to verify orders of these different drug complexity categories, and subsequently we calculated the aggregate amount of time spent by pharmacists verifying low complexity orders in order to provide guidance around the amount of pharmacists’ time that could be reallocated to more complex patient care activities.

Despite mixed but promising results, we did not confirm the study’s primary hypothesis that verification times increase as medication orders become more complex, although, we did denote time differences between the complexity categories. Although the mean order verification time was increased for moderate complexity orders relative to low complexity orders, the order verification time was not increased for moderate complexity orders compared to high complexity orders. Our findings suggest that there may be other factors that should be considered when categorizing drugs by complexity. Although we arbitrarily decided on 3 categories distinguished by logical criteria, there is nothing that deters future research from using more categories or deciding upon different complexity criteria. We identified dispensing logistics as 1 criterion based on the results observed for the antibiotic drugs.

The time required for inpatient pharmacist review and verification of medication orders varied across the drug-route dyads. More research is warranted to help pharmacy leaders and informaticists understand if there can or should be appropriate average time standards established for verifying medication orders of differing complexity, but such standards should be validated by assessing the corresponding quality of work, perhaps by measuring the appropriateness of the verification performed.

Some inpatient pharmacist work time could be saved by automating verification or reallocating verification work to other, less expensive health care personnel for low complexity orders using evidence-based, criteria-driven protocols. Data in our study show that approximately 4.1–7.6 seconds of pharmacist time per order (82–196 hours per year) could be saved in a calendar year if the low complexity orders in our study were no longer verified by pharmacists. Although the impact of selectively verifying only higher complexity orders was assessed for 4 low complexity drugs from the Michigan Medicine drug formulary, broader implementation of the complexity criteria across the institution’s drug formulary could provide more opportunities for time savings which might result in additional resource availability. This time savings could be reallocated to more complex activities that fully utilize the clinical and patient care skills and training that pharmacists have achieved.²² There is substantial evidence that pharmacists’ direct patient care activities (eg, antimicrobial and anticoagulant monitoring and dosing adjustments, facilitating medication access, patient education and counseling, clinical rounding and consultation, patient profile reviews, and other functions) achieve positive patient outcomes.²³ The verification of low complexity orders is a basic and repetitive drug distribution function that pharmacists frequently characterize as rote work due to the extremely low rate of problems. The most serious concerns, such as drug allergies, drug interactions, and dosing errors are generally identified using clinical decision support rules and alerts, while others would need to be identified through anomalous or important patient-specific factors that arise during routine profile review or clinical rounds. Verification of these low complexity orders may actually increase the risk of error due to the lack of vigilance by the pharmacist when reviewing these orders, similar to errors of omission seen with alert fatigue.^24,25 Despite this, we suspect that the majority of the drugs we did not study would be classified as moderate to high complexity drug-route dyad drugs and pharmacists would still need to prospectively review and verify these drugs.

Overall, the results of the study provide valuable insight into order verification times and pharmacist cognitive effort of complex medication orders. The amount of time required to verify the selected antibiotics was unexpected and resulted in findings that varied from our hypothesis. Although many factors contribute to cognitive effort, higher cognitive effort is likely occurring in medication orders which require the use of external resources before completing order verification. It is not clear to what extent specific patient or pharmacist characteristics contribute to cognitive effort, but we believe the estimated time saved from not verifying low complexity drugs in this study provides foundation for further exploration. If this drug-route dyad classification system is accepted and/or revised and extrapolated to a larger sample size of low complexity drugs, the amount of time saved can be significant. The reallocated pharmacist time will benefit patient care by providing more or higher quality clinical services.

Although drug complexity is not a concept mentioned in ASHP or TJC guidelines, its perception continues to shape the PMOR process at individual hospitals and among its personnel. Pharmacists are faced with numerous critical choices each day, and it is imperative that pharmacists consistently use an appropriate decision-making process to ensure that medication-related problems are avoided. For example, during PMOR and verification, pharmacists must refer to reliable evidence, consider patient-specific parameters and characteristics, and assess the safety risk relative to the treatment benefit before verifying orders or making alternate therapeutic recommendations. Martin et al described 5 decision-making approaches covering clinical, ethical, managerial, economic, and legal domains.²⁶ Adapting this problem-solving approach illustrated in pharmacy education literature highlights how each approach plays an important role on patient outcomes during pharmacist order review of medication orders:

• Clinical Intended Outcome: To maximize the therapeutic impact on the patient’s health by emphasizing clinical care quality over drug costs^27–29

• Ethical Intended Outcome: To make the most morally defensible choice by emphasizing moral behavior over drug costs, institutional margin, etc.³⁰

• Managerial Intended Outcome: To maximize the benefit toward achieving the pharmacy organization’s goals and missions by emphasizing pharmacy department support over clinical perfection³¹

• Economic Intended Outcome: To make the most cost-effective choice by emphasizing the relationship between drug cost and clinical outcomes³²

• Legal Intended Outcome: To make the most legally defensible choice by emphasizing adherence to pharmacy laws and policies³³

In 1996, Adamcik et al, assessed pharmacy student critical thinking through a computer assessment program (CAP). Although the CAP did not parallel other instruments of cognitive ability, CAP did reveal similarities in temperament, learning styles, information-processing modes, and problem-solving strategies among the students.³⁴ The instruments discussed by Adamcik, in addition to Martin’s decision-making approaches, are analogous to our research which assessed the impact of drug order complexity. To date, no research within pharmacy has been conducted to validate the cognitive efforts that interns or pharmacists may employ during PMOR. Specifically, each approach described by Martin may affect students’ decision to verify or reject medication orders stratified in the various order complexity categories. While Martin’s study focused on the need to incorporate decision-making approaches in the PharmD curriculum to help students to think more critically, it is relevant to contemporary pharmacy practice as it can help practitioners standardize the decision-making process during PMOR by simplifying the problem through including some information and excluding other information. These 5 decision-making approaches are presented as separate activities; however, in practice, these methods are likely interwoven when medication order verification decisions are made.

A past research study used an evidence-based checklist for PMOR. These incorporated criteria related to order urgency, verification of patients’ identity, therapeutic review, and actionable items.³⁵ Within these 4 categories, pharmacists considered pertinent information such as the type of order, how the order is triaged, a comprehensive therapeutic review, communication with the medical team, and various laboratory test interpretations. Although there is no direct mention of drug order complexity, pharmacists must cogitate this order review checklist, which is imbedded in the cognitive processes used to verify orders. The participating pharmacists in this study believed the checklist demonstrated value “as a way to encourage the development of a systematic and comprehensive approach to medication order review, and as a way to ensure standardized medication order review among pharmacists.”³⁵ The study did not define or measure drug order complexity, but it did describe underlying considerations which affect pharmacist PMOR time. The addition of a complexity classification system could help inform pharmacists as to which drugs require more versus less cognitive effort and time. Our study is a first step that others can consider when developing a complexity classification system that could potentially be applied across a pharmacy department’s drug formulary.³⁶ Our observational study used a pharmacist opinion survey to help create the drug-route complexity scale, and the survey served to amalgamate many hard-to-measure variables used during pharmacist PMOR into a tiered “drug order complexity” concept.

A Medication Regimen Complexity Index (MRCI) score was finalized after expert scrutiny in a 2004 study.⁴ It quantified the complexity of regimens from a patient perspective according to the dosage forms, dosing frequencies, and additional directions. It also involved factors such as the number of medications, the decision-making process (eg, time, skills, knowledge, and ability) necessary in carrying out the regimen, additional directions, and mechanical actions required for administration.⁴ Despite this index, it did not assess the time taken to verify a medication order based on drug complexity. The MRCI study did demonstrate the ability to quantify complexity of prescribed medication regimens in patients with moderate to severe chronic obstructive pulmonary disease who were also participating in a separate randomized controlled research study, but this study is not applicable to inpatient order verification and review of computer-entered orders from prescribers.⁴

Limitations. Our research does not define how long it should take to verify medication orders or set shift-based productivity standards for order verification as there are many nuances imbedded in the decision-making processes that pharmacists consider when verifying a medication. This study demonstrates that observed order verification times do not necessarily correlate with the scale we created. By adjusting future estimates in a power analysis to mirror our study’s results may require the number of observations needed for 80% power to increase. The scale also does not account for the impact of individual pharmacist experiences and biases on either perceived complexity or the appropriate amount of time to properly verify a specific medication. This can be viewed as under-appreciating or over-appreciating pharmacist experience. By looking at the pharmacist assessment of complexity (Table 1), cefazolin and piperacillin-tazobactam had complexity scores of 39 and 46, respectively (above the ≥ 35 threshold used as 1 criterion), perhaps indicating that more emphasis should have been placed on the pharmacist opinion which would have resulted in these antibiotics having a higher complexity classification.

We also did not emphasize the importance of operational logistics as a factor in order verification time as demonstrated by the routine steps taken by pharmacists when verifying antibiotic orders. The operational issue was not proactively anticipated and was underestimated for all medications, in particular for the antibiotics which demonstrated a high use of adjunct resources primarily to appropriately time the next drug administration. In this case, the low and moderate complexity antimicrobials required different workflows (and therefore more time) to check renal function (a clinical consideration) and prior administration times (a distribution-related consideration). The current study would have been more robust had we either not used antibiotics, stratified antibiotics separately into their own complexity category, or more fully considered dispensing logistics associated with medication orders as part of our complexity scale. Had we excluded antibiotics, our primary outcome measure hypothesis would have been more likely to have been proven true. By excluding antibiotics, the mean time difference for low complexity medications would have been an average of 3.8 seconds less, and the moderate complexity medications would have been 9.7 seconds less. This would have resulted in average time separations between the categories more consistent with our hypothesis.

Other limitations to this study include a relatively small sample size, relatively few observations for each medication, utilization of 1 pharmacy satellite in a single health system, and various elements of decision support (lab values, BPAs, etc.) available within the verification queue for specific orders. Results of the study may have been different if we measured complexity times in multiple locations or facilities, if there were different workflows, a different EMR, different classifications or selections of medications, and/or different pharmacist practice models. Another potential influencing confounder of verification times is understanding if the order originated from an order set and if the pharmacist commonly encountered that order/order set. Although this can be monitored with Morae, it was not measured in the study. It is possible that all or a greater number of medication orders came from an order set for 1 category of complexity versus another. For example, this could lead the pharmacist in feeling more comfortable verifying a high complexity medication in a shorter amount of time. Pharmacists’ practice experience is relevant because, with increasing experience, complex medication orders become more familiar and the experienced pharmacist may take less time than is required by a less experienced pharmacist. As with any human-driven process, there is a possibility that faster verification times could lead to more potential errors if the review is not sufficiently thorough. Although this study was developed to measure time as the main objective, we did not measure or correlate the clinical appropriateness of verification relative to the verification time.

Future Research and Direction. Currently, no standards or guidelines exist to define how much time should be devoted to order verification. This leaves it to the individual pharmacist to determine how much cognitive effort should be devoted based on a myriad of factors, including drug order complexity. Several questions emerge: (1) Should the pharmacy profession provide stronger PMOR standards which include key drug or drug complexity specific expectations? (2) Should the pharmacy profession help guide the pharmacist by developing minimum PMOR and verification time standards? (3) Is there a reasonable opportunity to triage orders based on complexity-related factors so that low complexity medication orders can be verified in an alternate manner (eg, automatically if the order passes core clinical decision support tests)? (4) Does the addition of standards and times, combined with greater decision support, increase or decrease the time available for pharmacists in working on other important patient care activities? Answers to these questions, resolved through high-quality research, could help better define the future role and work activities of pharmacists. Furthermore, the answers can contribute to continuous quality improvement programs within pharmacy departments at the point of inpatient PMOR and verification. Our study and other future task–time analyses can begin to establish general standards or metrics for pharmacists to follow regarding medications of differing drug complexity. Finally, the more we know about the cognitive processes used by pharmacists, the more we can better enhance the training provided to the next generation of pharmacy students as they seek to provide the best possible care for the patients with whom they work.

CONCLUSION

Pharmacists are under constant pressure to make quick clinical and drug distribution decisions while providing timely services to patients, nurses, and providers. This is often done in environments that present internal and external distractions. This study details an initial attempt at classifying medications into different levels of complexity and quantification of the time required to review and verify medication orders. We found that there was limited correlation of pharmacist time to medication order complexity when we consolidated 3 factors to define medication order complexity. There are likely many other approaches that could be considered involving various patient, medication, and/or human factors that could contribute to medication order complexity.

FUNDING

This research received no funding from any agency in the public, commercial, or not-for-profit sector. It was supported internally by Michigan Medicine.

AUTHOR CONTRIBUTIONS

DD contributed to data acquisition, and analysis and interpretation, drafting the manuscript, and revising it for important intellectual content. VM substantially contributed to the analysis and interpretation of data, writing the power and statistical analyses, and results sections. BC conceived and designed the study, validated order verification times, and contributed to the writing and critical revisions of the manuscript. All authors provided final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.