Abstract
Purpose
The purpose of the project described here was to use the work outputs identified in part 1 of a 2-part research initiative to build and validate an acute care clinical pharmacist productivity model.
Methods
Following the identification of work outputs in part 1 of the project, relative weighting was assigned to all outputs based on the time intensity and complexity of each task. The number of pharmacists verifying an inpatient medication order each day was selected to represent the labor input. A multivariable linear regression was performed to determine the final work outputs for inclusion in the model. Productivity and productivity index values were calculated for each day from July 1, 2018, through June 30, 2019.
Results
Of the 27 work outputs identified via consensus by the clinical pharmacist working team, 17 work outputs were ultimately included in the productivity model. The average productivity during the period July 2018 through June 2019 was derived from the model and will serve as the baseline productivity for acute care clinical pharmacists.
Conclusion
Validated consensus methodology can be useful for engaging clinical pharmacist in decision-making and developing a clinical productivity model. When thoughtfully designed, the model can replace obsolete measures of productivity that do not account for the responsibilities of clinical pharmacists.
Keywords: benchmarking, clinical pharmacist, consensus, metrics, practice model, productivity
Key Points.
Many current productivity models have limited applicability to acute care clinical pharmacists because the models do not account for clinical responsibilities.
Engaging clinical pharmacists in the development of a novel clinical pharmacist productivity model can improve both buy-in and applicability.
The development of an acute care clinical pharmacist productivity model is achievable using validated consensus methodologies and data analytics.
The evolving role of pharmacists in health systems has made historical methods of productivity measurement either obsolete or inaccurate. A standard definition for productivity, particularly for clinical pharmacists, does not exist.1 This article is the second of a 2-part series that details the development and validation of an acute care clinical pharmacist productivity model at the University of North Carolina Medical Center (UNCMC).
UNCMC is a 933-bed academic medical center. More than 200 pharmacists provide comprehensive clinical, operational, and academic services to patients, students, and residents. Clinical pharmacists are integrated with service-based interdisciplinary teams to provide both clinical services and operational functions (eg, order verification). The pharmacists referenced throughout this publication provide services across 5 service lines (critical care, heart and vascular, internal medicine, oncology, and pediatrics) at the hospital, located on the campus of University of North Carolina at Chapel Hill. Consistent with practices at other health systems, the UNCMC department of pharmacy currently utilizes a productivity metric that does not account for clinical pharmacist responsibilities.
Productivity measures are used by organizations to evaluate efficiency. They vary in complexity, composition, and use.2 There are 2 types of productivity measures. Partial factor productivity relates output to a single input, while total factor productivity relates an index of output to a composite index of all inputs. Labor productivity, the measure of interest in the study described here, is the most common partial factor productivity measure. It is commonly defined as the ratio of the output of goods and services to the labor devoted to the production of that output3:
While it is possible to calculate productivity, productivity analysis typically involves measuring productivity changes over time. A productivity index measures the total percent changes from a base period; in practice, this represents the relationship between budgeted and actual productivity4:
An organization that produces more output per unit of input is considered to be more productive. Productivity measures are inherently subjective and vary by industry. In health-system pharmacy, productivity measures typically relate the number of staff to the amount of work activities.5 Labor inputs are commonly defined as worked hours for a period of time and most often represented by full-time equivalents. Work outputs, in contrast, are more complex. In healthcare, output and value are difficult to measure due to the qualitative nature of service sectors.6 Effective productivity monitoring is critical to maintaining support for continued clinical pharmacy services.7 Specific metrics must be identified to accurately represent the contributions of a clinical pharmacist.8
The purpose of the overall study (parts 1 and 2) was to develop, validate, and implement a clinical pharmacist productivity model. Study objectives included the following: determine comprehensive responsibilities of clinical pharmacists, validate responsibilities through direct observation, weight responsibilities to determine measures, and validate the overall model. The multiple steps and associated methodology used within this research are represented as Figure 1.9
Figure 1.
Overall research methodology.
In part 1, acute care clinical pharmacists identified a list of responsibilities and corresponding measures, or work outputs, using validated consensus methods (Figure 1).9 The consensus process was guided by specific principles to ensure feasibility and sustainability of the model beyond the initial calculation. These decision-making principles included consideration of the following: the model will capture clinical responsibilities, be automated, use available electronic health record data, and be sensitive enough to detect changes in work outputs when labor inputs change. Each responsibility and measure was then assigned a relative weight based on the time intensity and complexity of each task.9 The purpose of part 2 was to use the work outputs identified in part 1 to build and validate an acute care clinical pharmacist productivity model.
Methods
A productivity calculation requires a measure to represent labor. Given that order verification is a ubiquitous function among clinical pharmacists in our organization, the number of pharmacists verifying an inpatient medication order each day was determined to be the most accurate and automated measure to capture labor. Scheduling platforms and time clock applications were not selected due to the erroneous inclusion of pharmacists who were scheduled to complete administrative duties, such as writing guidelines or conducting research, rather than clinical duties.
Pharmacists were included in the labor input if they verified at least 1 order between 7 am and 10 pm, excluding pharmacy residents. This timeframe aligns with the provision of decentralized clinical pharmacy services at UNCMC. Pharmacists were excluded if they were in a primarily operational role (eg, those working in the cancer hospital infusion pharmacy, investigational drug services pharmacy, central inpatient pharmacy, or operating room satellite pharmacy). Pharmacy residents were included if they were filling an evening position typically staffed by a non–pharmacy resident. The model assumes that each pharmacist who is included in the denominator worked an 8-hour shift. It also assumes that their role was primarily clinical rather than administrative or operational. It is important to note that the evening-shift emergency department and pediatric pharmacists included in this model work 10-hour shifts. Additionally, a couple of pediatric pharmacist positions have a mix of clinical and operational duties but were ultimately included in the model on the basis of their significant clinical responsibilities, including pharmacokinetic monitoring and order verification. Labor input methodology was validated by comparing the list of pharmacists who verified an order on a sample of days to each area’s schedule.
Following the identification and weighting of all work outputs, daily counts of each output from July 2018 through June 2019 were obtained from the electronic health record. One year of data was utilized to account for seasonal variations of work such as fluctuations in patient census, vacations and holidays, and pharmacy resident training. Each measure was multiplied by the associated weight identified in part 1 in preparation for a sensitivity analysis. Daily productivity was calculated by multiplying each measure by its associated weight and dividing the sum by the number of included pharmacists for that day. A daily productivity index was calculated by dividing the daily productivity by the average productivity. The resulting productivity calculation represents the total combined acute care clinical services for all adult, oncology, and pediatric clinical pharmacists at UNCMC (Figure 2).
Figure 2.
Productivity calculation.
An initial sensitivity analysis was performed using multivariable linear regression (Stata Version 16, StataCorp LLC, College Station, TX) to determine whether the model could detect a change in productivity as a result of changes in outputs or inputs. A P value of <0.05 was considered significant. Final work outputs included in the model were determined using a combination of statistical significance, practical significance, and relation to other work outputs. A second sensitivity analysis was performed using the final work outputs.
Results
A total of 27 work outputs were included in the initial sensitivity analysis. Of the 27 work outputs included in the initial sensitivity analysis, 8 were found to be statistically significant (Table 1): the respective counts of patients at the acute and step-down levels of care; admissions; new orders verified; orders entered by a pharmacist; admission medication reconciliation; discharge medication reconciliation; and counseling. A total of 17 work outputs were included in the final productivity model (Table 2). Upon conducting the second sensitivity analysis of these 17 work outputs, 10 work outputs were statistically significant: the respective counts of patients at acute and step-down levels of care; time-intensive medications; admissions; new orders verified; discontinued orders verified; orders entered by a pharmacist; admission medication reconciliation; discharge medication reconciliation; and counseling.
Table 1.
Initial Sensitivity Analysis Results
| Work Output | P Value <0.05 |
|---|---|
| Acute census | Y |
| ED census | N |
| ICU census | N |
| Stepdown census | Y |
| Newborn census | N |
| Observation census | N |
| Renal medications | N |
| Time-intensive medications (excluding chemotherapy) | N |
| Chemotherapy medications | N |
| HIV medications | N |
| >8 scheduled medications | N |
| Admissions | Y |
| New orders | Y |
| Needs review orders | N |
| Edit orders | N |
| Discontinued orders | N |
| Orders entered by pharmacist | Y |
| Orders discontinued by pharmacist | N |
| Patient-supplied medications | N |
| Pharmacist dispenses | N |
| Adjust time messages | N |
| Admission medication reconciliation | Y |
| Discharge medication reconciliation | Y |
| Counseling | Y |
| Pharmacist notes | N |
| Patient education handouts | N |
| Interventions | N |
Abbreviations: ED, emergency department; HIV, human immunodeficiency virus; ICU, intensive care unit.
Table 2.
Final Sensitivity Analysis Results
| Work Output | P Value |
|---|---|
| Acute census | 0.002 |
| ED census | 0.904 |
| ICU census | 0.070 |
| Stepdown census | 0.011 |
| Newborn census | 0.284 |
| Observation census | 0.747 |
| Time-intensive medications | 0.042 |
| Admissions | 0.013 |
| New orders | 0.000 |
| Needs review orders | 0.127 |
| Edit orders | 0.954 |
| Discontinued orders | 0.037 |
| Orders entered by pharmacist | 0.036 |
| Orders discontinued by pharmacist | 0.081 |
| Admission medication reconciliation | 0.000 |
| Discharge medication reconciliation | 0.013 |
| Counseling | 0.001 |
Abbreviations: ED, emergency department; HIV, human immunodeficiency virus; ICU, intensive care unit.
There were a number of work outputs that were not statistically significant but that were ultimately recommended for inclusion. Nonsignificant measures were included if they were related to a significant measure and the exclusion of the nonsignificant measure would have resulted in an underrepresentation of workload. For example, level of care describes the volume of patients, which impacts the volume of work outputs; thus, despite level of care being a statistically nonsignificant measure in and of itself, the exclusion of patients categorized as requiring emergency department level of care would underrepresent the work outputs associated with those patients. Additionally, there were order verification types that were believed to be nonsignificant measures solely due to small sample size compared to sample sizes for significant order verification types. While the inclusion of these measures did not substantively impact the productivity index, these measures were ultimately included to be consistent with the decision to include all measures in level of care.
The weighting in Table 3 is the final recommendation of the project team following the results of the sensitivity analysis and review by the clinical pharmacists involved in the project. During the period July 2018 through June 2019, the average productivity was 16. This establishes the baseline productivity for acute care clinical pharmacists at UNCMC and was used as the budgeted productivity for ongoing productivity assessment.
Table 3.
Final Productivity Model Consensus Weighting
|
To illustrate the variations in productivity index by day, a random date generator was used to select 2 sample 10-day periods: August 14, 2018, through August 23, 2018, and February 12, 2019, through February 21, 2019 (Figures 3 and 4). The average productivity index for the first of those periods (August 14-23, 2018) was 0.96; the average productivity index on weekdays was 0.85, while the average productivity index on weekend days was 1.39. This results in a variance of 0.48 and a percent variance of 38%. The average productivity index during the second sample period (February 12-21, 2019) was 0.88; the average productivity index on weekdays was 0.79, while the average productivity index on weekend days was 1.23. This results in a variance of 0.43 and a percent variance of 31%.
Figure 3.
Daily productivity index for sample 10-day period (August 14-23, 2018).
Figure 4.
Daily productivity index for sample 10-day period (February 12-21, 2019).
Discussion
This report describes a novel method to assess clinical pharmacist productivity on a daily basis. While this article focuses on the process by which institutions may build a productivity model, the next step is to implement the model. The authors hypothesize that, when implemented effectively, this model and the associated raw data of clinical pharmacists’ work outputs can be used to illustrate the impact of changes in workload. It provides objective data to departmental leaders, allowing them to monitor the impact of new responsibilities. The value of the model is rooted in the use of a data visualization platform that allows users to interact with and analyze data as close to real time as possible. This information could be used to produce a flexible staffing model and to project staffing needs based on anticipated workload. It may also be used to support the case that additional staff are warranted on certain days of the week or certain times of the year. At the time of writing, the study institution was in the process of building the productivity model into a data visualization platform to allow for the aforementioned monitoring.
It is important to note that this model was designed to assess productivity for all clinical pharmacists at the study institution. The pharmacists involved in the design of the model were asked to consider the responsibilities of any clinical pharmacist rather than responsibilities that may be unique to a subset of clinicians. While this acute care clinical pharmacist productivity model is not intended to redistribute workload according to service line or individual pharmacist, it can be used to broadly trend workload disparities over time and be used in discussions with the hospital leadership regarding justifications for additional clinical pharmacists.
This model would need to be reevaluated should it be applied to a subset of pharmacists, such as oncology clinical pharmacists. This underscores the importance of institutions developing productivity measures that most closely reflect the work being executed and the need for multiple productivity measures to accurately reflect the diverse roles that pharmacists play in a given department. Furthermore, consideration should be given to how this model may be coupled with metrics that capture clinical outcomes and non–patient care responsibilities such as publications and guidelines development. Collectively these characteristics will most comprehensively illustrate the value that clinical pharmacists provide to the healthcare team.
This model required significant ownership by and engagement of clinical pharmacists to ensure that the model was most reflective of clinical practice. Previous models have primarily focused on operational activities, such as order verification, but have failed to capture more modern roles of clinical pharmacists such as transitions of care. The project team is unaware of other published productivity models that were designed with the level of ownership and engagement by clinical pharmacists that this model incorporated.
Consideration must be given to the frequency at which departments should reevaluate the productivity model and budgeted (baseline) productivity. Understanding that workload and clinical practice are ever evolving, the model must be routinely evaluated to ensure that it most accurately reflects clinical pharmacists’ responsibilities. The authors recommend that baseline productivity and work outputs should be assessed at least annually. Based on the authors’ experience, a minimum of 1 year of data is recommended to be used when adjusting the baseline productivity in the future to account for the seasonal fluctuations that are expected to occur. Finally, a sensitivity analysis should be repeated upon making modifications to the model.
Limitations
During the project, conducted over 4 years from initial concept development to first baseline productivity measurement, a number of barriers were overcome. The model required support from not only the senior pharmacy leadership but also frontline staff. It was challenging to maintain a consistent group of participants given the duration of the project, changes in staffing, and competing responsibilities. From the outset, the authors worked diligently to gain the support of the senior pharmacy leadership. The authors also worked closely with the pharmacy service line managers to ensure that appropriate clinical pharmacists were selected to participate and that team members were replaced in a timely manner when appropriate.
Additionally, productivity can be an abstract and uncomfortable topic to discuss with frontline staff. Significant consideration was given to how best to engage staff, such as including clinical pharmacists on the primary project team and throughout various stages of the project. To mitigate concerns from the clinical pharmacists for whom this productivity model was developed, the authors held small group sessions and attended larger team meetings to present the model and seek feedback. The authors believe that transparency and open lines of communication were important attributes of this project.
This productivity model was designed in a way such that it is as automated as possible. However, this poses a challenge when considering clinical pharmacist responsibilities that may not be routinely tracked or captured electronically. Clinical pharmacists’ responsibilities that could not be included in the model for this reason included critical care response, research involvement, committee participation, administrative duties, medication access involvement, and education (including precepting). The ideal clinical pharmacist productivity model for pharmacists who work at large academic medical centers would capture the aforementioned activities. Thus, this model must be coupled with additional tools and reporting methods that tell the story of responsibilities that are not electronically captured, such as publications, guidelines, and precepting volume.
Finally, the external validity of this study must be considered by other institutions. While this exact model should not be universally implemented due to variability in clinical pharmacy practice models, the framework (ie, development and validation) of the model can be applied by other institutions.
Conclusion
The development of an acute care clinical pharmacist productivity model is achievable using validated consensus methodologies and data analytics. Consensus methods were used to identify a list of work outputs that contribute to clinical pharmacist productivity. A sensitivity analysis was performed to determine which work outputs contribute most to productivity and therefore warrant inclusion in the model. If implemented successfully, the model can be used to replace obsolete measures of productivity that do not consider clinical pharmacist responsibilities. This model, coupled with clinical outcomes, helps articulate the value that pharmacists bring to the healthcare team.
Acknowledgments
The authors express their sincere gratitude to Maurice Alexander, PharmD, BCOP; Jessica Auten, PharmD, BCOP; Shawna Beck, PharmD; Paul Dombrower, MS, RPh; Hubert Lahr, PharmD; Kalynn Rohde, PharmD, BCCCP; Edward Sredzienski, MS, RPh; and Kristen Szempruch, PharmD, BCPS, for their generous participation in the project team.
An audio interview that supplements the information in this article is available on AJHP’s website at www.ajhpvoices.org.
Disclosures
This publication was supported by grant number UL1TR002489 from the National Center for Advancing Translational Sciences (NCATS) at the National Institutes of Health (NIH). The other authors have declared no potential conflicts of interest.
References
- 1. Rough SS, McDaniel M, Rinehart JR. Effective use of workload and productivity monitoring tools in health-system pharmacy, part 1. Am J Health-Syst Pharm. 2010;67(4):300-311. [DOI] [PubMed] [Google Scholar]
- 2. Steindel C, Stiroh K.. Productivity: What Is It, and Why Do We Care About It? Federal Reserve Bank of New York; 2001. [Google Scholar]
- 3. Murray A. Partial versus total factor productivity measures: an assessment of their strengths and weaknesses. Int Product Monit. 2016;31:113-126. [Google Scholar]
- 4. US Bureau of Labor Statistics. How is productivity calculated? Accessed June 2, 2020. https://www.bls.gov/k12/productivity-101/content/how-is-productivity-measured/calculating-productivity.htm
- 5. Naseman RW, Lopez BR, Forrey RA, Weber RJ, Kipp KM. Development of an inpatient operational pharmacy productivity model. Am J Health-Syst Pharm. 2015;72(3):206-211. doi: 10.2146/ajhp130803 [DOI] [PubMed] [Google Scholar]
- 6. Sheiner L, Malinovskaya A.. Measuring Productivity in Healthcare: An Analysis of the Literature. Hutchins Center on Fiscal & Monetary Policy at Brookings; 2016. [Google Scholar]
- 7. Pawloski P, Cusick D, Amborn L. Development of clinical pharmacy productivity metrics. Am J Health-Syst Pharm. 2012;69(1):49-54. doi: 10.2146/ajhp110126 [DOI] [PubMed] [Google Scholar]
- 8. Carmichael J, Jassar G, Nguyen PA. Healthcare metrics: where do pharmacists add value? Am J Health-Syst Pharm. 2016;73(19):1537-1547. [DOI] [PubMed] [Google Scholar]
- 9. Vest TA, Simmons A, Morbitzer KA, et al. Decision-making framework for an acute care clinical pharmacist productivity model: part 1. Am J Health-Syst Pharm. 2021;78(x):xxx-xxx [DOI] [PMC free article] [PubMed] [Google Scholar]