School-wide Clinical Intervention System to Document Pharmacy Students' Impact on Patient Care

Abstract

Objectives

To implement and evaluate a school wide, Web-based clinical intervention system to document types and impact of pharmacy students' clinical activities during advanced pharmacy practice experiences (APPEs).

Methods

A clinical intervention form was developed by pharmacy practice faculty consensus and uploaded to a secure Web site. Prior to APPEs, all pharmacy students were trained on the purpose and use of the system as well as strategies to document interventions appropriately.

Results

Over the 3-year period of data collection, 15,393 interventions were documented. Most common intervention types included dosage adjustments, education of patients and providers, and optimization of therapeutic regimens. The majority of the interventions were accepted by the medical team and resulted in positive clinical and economic outcomes.

Conclusions

Our school-wide system allowed students and faculty members to document clinical activities. Reporting can serve a number of purposes, including incorporation into student portfolios and faculty merit and promotion dossiers, and demonstration of the positive impact on patient care.

INTRODUCTION

Preventable adverse drug events lengthen hospital stays, increase healthcare costs, and increase mortality.¹ Pharmacists' ability to reduce errors has been illustrated in a variety of settings including community, ambulatory care, and acute care inpatient practice. Pharmacists have been involved in establishing regimen concordance to promote safe and effective treatment of patients on narrow therapeutic index drugs (eg, warfarin) in the ambulatory setting; decreasing medication discrepancies by as much as 50% via medication reconciliation in the outpatient setting; and managing intravenous to oral antibiotic conversion programs in the acute care setting.^2-4 The addition of clinical pharmacists as part of the healthcare team improves not only the quality of patient care but also clinical outcomes.⁵

The Accreditation Standards and Guidelines 2007 (Standards 2007) from the Accreditation Council for Pharmacy Education, mandates that pharmacy graduates possess basic knowledge, skills, attitudes, and values to practice pharmacy independently. In addition, Standards 2007 states that graduates must be competent to manage medication use systems through medication safety and error reduction programs, among other outcomes.⁶ Therefore, pharmacy programs must provide didactic and clinical education that enables the development of skills for identifying, resolving, and documenting drug therapy problems.

Currently, little data illustrating the clinical impact of pharmacy students exist. Several studies have evaluated the clinical activities of small groups of pharmacy students, estimated associated cost savings, and documented the positive impact of students on health care overall.^7-11One potential challenge for obtaining large samples of student-based interventions is the lack of a universal system to document students' activities during advanced pharmacy practice experiences (APPEs). Many clinical institutions use multiple intervention databases that are homegrown and institution specific. Most databases have been designed for use by clinical staff members and often exclude student access.

In 2007, George MacKinnon emphasized the importance of documenting pharmacy services and interventions in all pharmacy settings.¹² He recommended an initiative to create a universal documentation system to collect student interventions in schools of pharmacy nationwide. Creating such a system would enable students to document outcomes such as patient care plans; assess contributions to ensure safety; record patient encounters that may be needed for licensure, certification, or competency achievement documentation; and serve as a dataset for scholarly endeavors. Directors of experiential education may also find such data useful to demonstrate the contributions of pharmacy students in the creation of new practice collaborations.

In light of the existing literature and these recent practice mandates, the Department of Pharmacy Practice in the School of Pharmacy at Northeastern University developed an electronic intervention system for student and pharmacist use. This paper reports the development, implementation, and utilization of an online clinical intervention system and subsequent analysis of data acquired over a 3-year period.

METHODS

The Northeastern University School of Pharmacy (NUSOP) consists of 2 departments, the Department of Pharmaceutical Sciences and the Department of Pharmacy Practice, that include both practice and social and administrative sciences faculty members. The 210-semester-hour doctor of pharmacy (PharmD) program is organized in a 0-6 year format. The professional curriculum includes 62 semester hours of didactic courses culminating in 36 semester hours of APPEs. The APPE year consists of six 6-week APPEs, 4 of which are required: ambulatory care, community practice, general medicine, and an additional institutional patient-care experience (eg, cardiology, transplant, etc).

This project was initiated in January 2005. An internal needs assessment was conducted to determine current practice of intervention documentation by faculty members and their students. Department of Pharmacy Practice faculty members, representing 25 full-time equivalent (FTE) positions, were invited to participate. Based on the information collected, a draft intervention form was created and circulated for additional input. Three revisions culminated in the intervention form appropriate for use in multiple patient care settings. In May 2005, a Web-based clinical intervention database (wCID) was implemented via the existing APPE management system (Education Management Systems software, ROI solutions, Peoria, Arizona). The protocol for project implementation and data collection was approved by Northeastern University's Institutional Review Board. The wCID was designed to document patient demographics, intervention type, description, drug and disease state involved, acceptance by the medical team, clinical and economic impact, and preceptor involvement.

Concurrently, in 2005, the hard copy of the intervention form was incorporated into 7 courses taught to third year (P3) students: a pharmaceutical care laboratory series and a therapeutics series. The rationale for incorporating this form was to expose students to documentation practices in the classroom that should be applied throughout the APPE year and beyond.

First Year of wCID Implementation (2005-2006)

Students beginning the APPE year participated in an orientation session that focused on the importance of documenting clinical services. A live group demonstration of the wCID was completed using several intervention examples. Emphasis was placed on identifying each intervention on a separate form even if several interventions were related to the same patient. To comply with HIPAA regulations, students were instructed to exclude any protected health information (PHI) during the documentation process. The original Web-based form allowed students to select only 1 intervention category, 1 type of acceptance outcome, and 1 type of clinical impact; however, multiple economic outcomes were allowed. This form also provided a rubric for students to determine the clinical impact of their intervention. At the conclusion of the session, students received a written document describing the intervention documentation process, definitions for outcomes (eg, adverse drug reaction, long-term cost impact, and immediate cost-impact), and example interventions that would result in certain clinical and economic impacts. This document, available on the Web, was distributed to improve consistency and standardize documentation of clinical interventions.

Participation in the documentation effort was encouraged but was left to the discretion of each preceptor. Students were instructed to inquire on day 1 of each APPE whether their preceptor expected them to utilize the wCID for documentation purposes. During the first year of the project, all students were required by the Office of Experiential Education to document their clinical interventions during week 1 of their community APPE. It was left to the discretion of the individual preceptor to mandate documentation for the remaining weeks of the community APPE.

At the end of the first year, students and preceptors were surveyed regarding wCID utilization. In addition, several focus groups were organized to solicit specific feedback to improve the efficiency and reporting functionality of the wCID.

Second Year of Implementation (2006-2007) and Beyond

The wCID was modified in October 2006 based on the feedback received from the survey and focus groups. In the revised online form, existing intervention categories were streamlined and several new categories were added. Dropdown menus for the Top 100 drugs and most common disease states were added. “Acceptance by the medical team” field options were also slightly modified. The most significant improvement was the creation of a reporting function that allowed students to generate intervention reports by each APPE and for the entire year. A function was also added that allowed faculty members to generate a report for each student on their APPE and a yearly report of all student activities at a practice site, which faculty members used in end-year merit and promotion documentation. The process of orienting students to the wCID continued prior to the start of the APPE and use of the wCID by all faculty members and students was encouraged but not required.

Data Analysis

Intervention data for 3 cohorts of students, class of 2006, 2007, and 2008, were analyzed. Two existing databases (original and revised) were merged to report the overall number of interventions documented (Table 1). All other analyses (Tables 2–7) utilized the revised database because of its improved reporting functionality. Further, the detailed analysis of interventions for only 3 required APPEs were reported: general medicine, community, and ambulatory care. Details of the interventions in the specialty institutional areas are beyond the scope of this paper but are available from the corresponding author upon request. Intervention outcomes were compared using Fisher's exact test.

Table 1.

Number of Interventions Documented Based on Required Advanced Pharmacy Practice Experience (APPE) Sites

Table 2.

Five Most Common Intervention Categories by Practice Setting (%)^a

^aRevised database only.

Table 7.

Economic Impact of Intervention^a

^aRevised database only.

^bDefined as decrease in future morbidity and hospitalizations

^cDefined as immediate decrease in drug/laboratory/hospitalization

RESULTS

From May 2005 through April 2008, 15,393 interventions were documented (see Table 1 for breakdown of types of APPEs). During the 2005-2006 academic year, 83 students (100% of the class of 2006) and 58 sites participated (38%). By the end of the data analysis period, student participation remained high, with 94 students (91% of the class of 2008), and site participation increased to 132 sites (80%). During this period of time, 56% of all patient-care APPEs were completed in an institutional setting (general medicine and specialty APPEs), 20% in the community, and 24% in the ambulatory care setting (mostly urban community health centers). The majority of the interventions (65.5%) were documented in an institutional practice setting, 10% in the community setting, and 25% in an ambulatory care setting (Table 1).

The most commonly documented interventions were dose adjustments (17%), education of patients (15%) and providers (7%), recommendations for new drugs for untreated indication (10%), optimization of subtherapeutic regimens (9%), and provision of drug information (7%) (Table 2).

In the community and ambulatory care settings, interventions most often involved chronic disease states such as hypertension, type 2 diabetes mellitus, asthma, and hyperlipidemia. In the general medicine setting, interventions most often involved hypertension, pain, type 2 diabetes mellitus, constipation, and pulmonary infections (Table 3). Sixty-four percent of interventions were initiated by students, 7% by preceptors, 20% by the student and preceptor team, and 6% by other providers (Table 4). Although an intervention could have been initiated by someone other than the student, the student had the responsibility to perform all necessary research and complete the intervention. For example, if a provider requested a student's help with a drug information issue, the student would complete the intervention forms based on the work he or she accomplished, but would record that the intervention was initiated by the provider. Preceptors in the ambulatory care setting had the most involvement in intervention initiation, with 50% of all interventions initiated by, or in conjunction with, the preceptor.

Table 3.

Five Most Common Drugs and Disease States by Practice Setting for Interventions^a

^aRevised database only.

Table 4.

Intervention Initiation by Practice Setting, %^a

^aRevised database only.

Intervention Outcomes

Interventions were well accepted by medical teams. Of all completed interventions with appropriate follow-up, only 7% were rejected and 65% were accepted as is (Table 5). Rejection rates were significantly higher in the general medicine setting (10.3%) compared with the ambulatory care setting (1%) and the community (2.8%), with p < 0.01 for all comparisons.

Table 5.

Intervention Outcome by Practice Setting, %^a

^aRevised database only.

Nearly 88% of all interventions were ranked by students as clinically significant (Table 6). Students in the community setting reported more of their interventions were informational (12.5%) compared to students in general medicine (10.7%; p = 0.14) and ambulatory care settings (5.4%; p < 0.001).

Table 6.

Clinical Impact of Intervention, %^a

^aRevised database only.

^bRecommendation is informational

^cBenefit of the recommendation to the patient could be neutral to slightly beneficial

^dRecommendation would bring care to a more acceptable and appropriate level

^ePotentially serious consequences to patient without this recommendation

From both databases combined, over 5,000 interventions were documented as having long-term impact and nearly 4,000 had immediate cost impact. Over the study period, students documented prevention of 4,222 potential adverse drug reactions and over 1,000 medication errors. When examining these data by practice site (Table 7, revised database only), in the general medicine setting, nearly 36% of all interventions prevented potential adverse drug reactions, compared to 18.5% of ambulatory (p < 0.001) or community setting interventions 31% (p < 0.02). There was a significantly greater number of adverse drug reactions prevented in the community vs. ambulatory care settings (p < 0.001). Four percent of interventions prevented medication errors in an ambulatory care setting, 6% in general medicine, and 8% in community (p < 0.001 for community vs. ambulatory care; p = 0.06 for general medicine vs. community; p < 0.001 for general medicine vs. ambulatory care).

Students reported spending an average of 13 minutes per intervention. This time varied slightly by practice setting: 10 minutes for general medicine, 16 minutes for ambulatory care, and 12 minutes for community practice.

DISCUSSION

Documenting and analyzing the direct effect of pharmacy students in different healthcare settings is a daunting task. Our school successfully developed and implemented a homegrown intervention database that was made available to students via the Internet utilizing the existing APPE management software. Over the 3-year data analysis period, site participation in wCID more than doubled, from 38% to 80% of all available sites. Analysis of over 15,000 student interventions revealed that the clinical impact of the students was substantial as demonstrated by high acceptance rates, clinical significance of interventions, and potential economic impact. The students were actively involved in medication therapy management of disease states such as hypertension, type 2 diabetes mellitus, and hyperlipidemia. Many interventions involved physician and patient education, optimizing standard of care, management and control of disease states, and assisting with appropriate dosing of medications. Data extracted from the database regarding the most commonly observed drugs and conditions proved extremely useful for curricular assessment and improvement. These data are regularly reviewed by the assessment and curriculum committees at our institution to ensure that curricular content aligns with practice trends.

Pharmacists and pharmacy students play an important role in coordination of patient care, helping reduce an estimated 1.5 million preventable medication-related adverse events, with associated annual costs of $3.5 billion.¹³ The quality of patient care was significantly improved by our students through optimization of medication regimens (including dosing), initiation of new drugs to improve adherence to standard of care, and education of patients and providers. Student interventions also provided added value to the practice site, meaning that if a student were not in the particular health-care setting, many of the interventions would not occur. By having direct access to patients and providers, either via medical rounds or by seeing patients in clinics or in the community setting, students were able to gather more information, thereby affording them a better opportunity to identify needed interventions.

The majority of the interventions in the database were documented in the institutional setting (63% of all interventions), which correlates with 56% of all patient care APPEs being completed in this setting. Documentation of clinical services is an integral component of daily practice in many institutions. While 20% of all patient care APPEs are completed in a community setting, only 10% of the interventions from community sites were included in our database. This is the setting where pharmacists and pharmacy students can have the greatest impact on the quality and coordination of patient care. We believe, however, that students and pharmacists face many barriers in documenting their impact in this setting. Many pharmacies prohibit Internet access and students may not have specific time set aside for documentation purposes. In addition, students may not perceive traditional activities in a community setting (eg, patient education, recommendation for nonprescription medications, etc) as “clinical” and therefore fail to document the intervention. The community interventions were predominantly patient education (31.8%), drug information (10.4%), and new drug for untreated disease state (9.4%). Three quarters (74.7%) of the interventions were initiated by the student and 70% of the recommendations were accepted as is. During the 2009-2010 APPE year, we plan to emphasize, and possibly mandate, students to document interventions in the community setting. We are specifically interested in examining student impact on population health and the impact that a school of pharmacy can make on a surrounding community from a public health perspective.

Nearly 25% of all interventions were documented in an ambulatory care setting. These interventions demonstrated that pharmacists were involved in disease state management with a focus on type 2 diabetes mellitus (34%), hyperlipidemia (9.3%), and hypertension (8.1%). These findings were reflective of the overall prevalence of these disease states and again supported the tremendous impact that pharmacy students can make from a public health perspective. High acceptance rates (99% of all interventions with follow-up) clearly demonstrated the strength of the rapport among the students, pharmacists, and prescribers in this setting.

In the institutional setting, pharmacy students improved the quality of patient care by ensuring appropriate patient-specific dosing of medications (18%), optimizing regimens to achieve recommended endpoints (10%), and improving adherence to the standard of care (14%). Fourteen percent of all interventions involved education of providers (10%) and patients (4%).

While it is difficult to assign a dollar value to individual student interventions, the high cost of pharmacy student training may be justified by the impact that students have on improving patient care. To date, economic impact data is not available to calculate the potential cost savings and cost benefits of involving pharmacy students in patient care. Future research will explore the economic, clinical, and educational impact of student interventions.

This study has several strengths. Early and sustained exposure to the intervention form in the didactic P3 courses allowed the students to gain a foundation of knowledge about the importance of documenting interventions, which enabled them to complete the form with greater accuracy. During these didactic courses, faculty members and seminar and laboratory facilitators evaluated students' intervention forms and provided feedback to ensure proper interpretation and documentation. Student performance assessment suggested that these evaluations resulted in more consistent documentation of interventions during the APPE year.

After the first year of wCID implementation and use, extensive surveys and focus groups were organized to solicit feedback regarding the use of the original intervention form, and as a result, numerous improvements were made. Adding and clarifying intervention categories resulted in decreased coding of interventions as “other.” A function to track disease states and drugs documented was also added, which improved the use of wCID to track patient diagnosis and co-morbidities. The addition of the reporting function enables students to generate reports of their interventions, allowing them to demonstrate their work and impact on APPEs to their preceptors, advisors, and potential employers. Reports also allowed faculty members to use intervention data as an assessment tool to document several competencies required during the APPE year. Reporting also allows faculty members to document clinical services that they and their students provide, as required by the merit review process.

Available on the Web, the CID provided easy access and retrieval of information. From an administrative perspective, utilizing the existing APPE management system provided access to all participating students, preceptors, and sites.

Several limitations are important to note. Documenting interventions was not mandatory, therefore, the intervention data may not be representative of actual practice. What caused specific students or preceptors to utilize the wCID has not been determined. Not all adjunct sites may be aware of the wCID, which may have resulted in some selection bias on the types of activities and interventions documented. Going forward, the use of the wCID will become part of preceptor training activities. Also, attention will be focused on collecting information on obstacles students and preceptors report that preclude their use of the wCID. In this study, the decision to make use of the wCID elective was based on 2 factors: (1) the number of sites that asked students to document clinical interventions using their institution-specific documentation system, and (2) the choice to have students avoid duplicating their documentation efforts. Further discussions are pending about making it mandatory for students to use wCID to document their patient care activities as part of demonstrating their achievement of APPE competencies.

While all preceptors are encouraged to review student interventions, this process is not mandatory. Because the preceptor is not required to “approve” or “sign off” on all interventions, it is possible that students may understate or overstate the impact or significance of their intervention. The decision not to mandate this sign-off was based on the potential impact on the workload of adjunct preceptors. In addition, while some measures were implemented to standardize student documentation, additional definitions were needed for the students. For example, no standard definition for medication error was provided. It was assumed that medication error discussions in didactic courses allowed students to categorize accurately the interventions that potentially prevent medication errors. Student training materials will be updated to further clarify these definitions.

While rejection rates for interventions were low (6.5% overall), a field was not included to document the reasons for intervention rejection; therefore, this element was not analyzed further. Providers may have had very appropriate rationales for rejecting some interventions.

Students are encouraged to document their interventions daily. The reality, however, is that many students do not document their interventions routinely, choose to delay documentation because of necessary follow-up, or do not have immediate access to a computer, all of which can result in significant recall bias. Because each intervention must be entered in the system as a separate entry with no specific patient identifiers, evaluating how many patients were positively impacted by the students is not possible. Of particular interest is more specific demographic information (eg, ethnicity), and these documentation fields will be included in the third generation of the database.

A study entitled, “Analysis of Pharmacy Student Interventions Collected via an Internet Based System” discussed student interventions made over a 30-week period and collected through the Internet.⁸ This report most closely resembles the design and the findings of our study, although we collected data from more students and for a longer period of time. Characteristics of student interventions were similar in both studies. One of the main strengths of the NUSOP is that the majority of the ambulatory care APPEs are in urban community health care centers; thus, our students make significant contributions to the care of indigent patients.

The CID will continue to be used to document direct patient care provided by our students, with modifications in the areas where weaknesses have been identified. Capturing demographic information in the database will be expanded so that data required by 2007 ACPE accreditation standards will be collected.

CONCLUSIONS

Our students had significant impact on patient care through optimizing drug regimens, improving adherence to standard of care practices, and educating patients and providers. A Web-based documentation system was created that is user-friendly and can be adopted by other colleges and schools of pharmacy. Expanding class sizes and the constraints placed on the health care system will create the need to provide documentation of evidence and impact on patient care by APPE students. Whether documentation systems are unique or universal, students must be trained to document clinical services and the value that they provide. The economic impact of student interventions requires further research and analysis and may be addressed by a national effort to design a universal documentation system. From a public health perspective, pharmacists and, in our case, pharmacy students, play an important role in coordinating patient care, patient education, and continuous quality improvement of health care. If a universal documentation system were developed and accepted, valuable data about the impact of pharmacy students' on patient care and their surrounding communities could be collected and documented.