Implementation of a School-wide Clinical Intervention Documentation System

Abstract

Objective. To evaluate the effectiveness and impact of a customized Web-based software program implemented in 2006 for school-wide documentation of clinical interventions by pharmacy practice faculty members, pharmacy residents, and student pharmacists.

Methods. The implementation process, directed by a committee of faculty members and school administrators, included preparation and refinement of the software, user training, development of forms and reports, and integration of the documentation process within the curriculum.

Results. Use of the documentation tool consistently increased from May 2007 to December 2010. Over 187,000 interventions were documented with over $6.2 million in associated cost avoidance.

Conclusions. Successful implementation of a school-wide documentation tool required considerable time from the oversight committee and a comprehensive training program for all users, with ongoing monitoring of data collection practices. Data collected proved to be useful to show the impact of faculty members, residents, and student pharmacists at affiliated training sites.

INTRODUCTION

Current practice at most health care institutions is for pharmacists to document the services provided as part of their routine job responsibilities. Clinical interventions made by pharmacists improve patient care and result in cost savings for institutions.^1-12 Intervention data can be useful in providing information to other health care providers or health care system administrators regarding the value of clinical pharmacy services. This documentation also may be used to justify salaries for current pharmacist positions or to validate the need for additional pharmacist positions. Finally, it can be invaluable to administrators in performance evaluations as well as decisions regarding pharmacist workload.

While there are numerous published studies regarding documentation of pharmacists’ clinical services,^1-12 the studies on the impact of clinical services performed by pharmacy students, residents, and faculty members are less robust. Student pharmacists must learn and appropriately apply documentation skills so that doing so will become a professional habit that is continued after graduation. In addition, student documentation can assist experiential directors and preceptors in assessing student abilities, competencies, and professionalism. Students may choose to include these data within a portfolio to highlight their work for residency programs or other future employers. Finally, the clinical services documented by student pharmacists and residents can be used by schools of pharmacy to demonstrate the value of having students and residents involved in patient care within a practice site.

It is equally important for faculty members to document the clinical services they provide as they may be required to justify their position (or shared salary costs) by demonstrating their clinical and financial impact. In addition, faculty members may use intervention data for annual activity reports. These data also may be used as objective evidence in outreach dossiers for promotion and tenure purposes.

Electronic systems are more efficient compared to paper systems for documentation of clinical pharmacy interventions.¹ Numerous products are available for health care institutions that allow users to customize their documentation applications to meet institution-specific needs.^13-28 Even with use of these programs by health care institutions, the majority of colleges and schools of pharmacy do not have a central database to capture interventions performed by students, residents, and faculty members at affiliated practice sites. A single documentation system to capture program-wide intervention data would be ideal for most colleges and schools of pharmacy as each school's students, residents, and faculty members practice in various patient care settings in multiple institutions, many with differing intervention systems or no intervention system at all. Data from multiple intervention systems cannot be easily consolidated to provide meaningful, useful measures regarding the overall impact of a college or school's student pharmacists, residents, and faculty members. Current literature regarding student pharmacist documentation of interventions is limited to small studies that report results from single clinical sites rather than across a class or school.^{6,13,14,29-44} In addition, even fewer publications describe documentation of clinical pharmacy interventions made by pharmacy residents.^13,14,31,44 Only one published study focused on school-wide documentation of clinical interventions by student pharmacists and faculty members.³³

DiVall and colleagues described results from the implementation of a school-wide Web-based clinical intervention system.³³ Twenty-five faculty members and fourth-year (P4) students documented 15,393 interventions over 3 years. Four hundred eighty-four medication errors and 2,555 potential adverse drug reactions also were documented.

While there are limited data regarding documentation of clinical interventions by student pharmacists, pharmacy residents, and faculty members, there are less data published on the financial impact. Taylor and colleagues published findings regarding documentation of clinical interventions by pharmacy faculty members, residents, and students within a community-based hospital. Faculty members (n=2), residents (n=4), and students (n=22) documented more than 2800 clinical interventions over 21 months. The overall estimated cost avoidance resulting from the interventions by faculty members, residents and students was $172,655.³¹

Lee and colleagues found that student pharmacists’ interventions along with interventions by 36 staff pharmacists and 5 pharmacy residents resulted in cost savings and cost avoidance in a veterans medical center.⁶ The authors’ recorded 600 interventions within the medical center over a 1-year period. The estimated mean total cost avoidance over this 1-year period was $420,155.

The existing literature contains a few reports regarding intervention documentation by pharmacy school programs. These reports largely describe documentation at a small number of clinical sites. Existing literature contains no reports of the process of implementing an electronic, Web-based intervention documentation system across all experiential education sites for use by all pharmacy practice faculty members, residents, and student pharmacists. Additionally, no published literature describes the longitudinal clinical and financial impact of an experiential program as measured by a school-wide intervention documentation system.

This paper describes Auburn University Harrison School of Pharmacy's experiences implementing a Web-based software application for school-wide documentation of clinical interventions/activities by pharmacy practice faculty members, pharmacy residents, and student pharmacists. This paper focuses on technological considerations and includes clinical intervention and financial data collected over a 44-month period. The process of selection of the documentation system has been described previously.⁴⁵ This review was approved by the Auburn University Institutional Review Board.

METHODS

Auburn University Harrison School of Pharmacy is the only public school of pharmacy in Alabama and consists of the main campus in Auburn, and a satellite campus in Mobile. Student enrollment in each year of the program is 150 (125 in Auburn and 25 in Mobile). Students are assigned to 1 of 6 regions across the state of Alabama to complete their APPEs. These regions also include sites in west central Georgia; Pensacola, Florida; and Biloxi, Mississippi.

In 2005, the school's dean formed an ad hoc committee to explore school-wide implementation of an intervention documentation program. The committee's process for system selection has been described previously.⁴⁵ The documentation system from Pharmacy OneSource, Quantifi, (Pharmacy OneSource, Bellevue, WA) was selected as it best met the school's needs.

There were several purposes in implementing the documentation system. First, it was important for students to learn to document the direct patient care services provided during practice experiences. We believed that familiarity and experience with the technology of a Web-based documentation system was essential for pharmacy graduates. Second, it was important to document the direct patient care services student pharmacists were providing during APPEs, allowing the school to demonstrate to preceptors the value of teaching student pharmacists. Finally, we believed implementation of intervention documentation as a curricular requirement supported the school in meeting Center of the Advancement of Pharmaceutical Education (CAPE) outcomes related to systems management.⁴⁶

Documentation Tool Preparation

Quantifi is a Web-based software program designed for efficient, structured documentation via the Web. It is designed to interface with hospital information systems, but we did not pursue this functionality due to our large number of practice sites. Preparation efforts focused on creating a streamlined intervention documentation process for all users, while capturing the data that would be important to the school.

Documentation Form.

The intervention documentation form was divided into 4 sections (patient, event, follow-up, and outcome), with each containing fields for information specific to the intervention being documented (Figure 1). While entry of information in the event section was required by Pharmacy OneSource, the other sections were optional and could be hidden (not shown to any users), collapsed (available for use if the user clicks on the section heading to expand it), or expanded (all fields are shown to all users). The event section was expanded automatically.

Figure 1.

Documentation form in Quantifi (http://www.pharmacyonesource.com/applications/quantifi/)

In all 4 sections of the documentation form, several options existed for each documentation field. These options essentially determined which fields were required, optional, or not visible at all. An additional checkbox allowed free text entry in some fields. For our purposes, the most important (and thus required) components of each intervention were where the intervention occurred (service) and documentation of the primary and secondary interventions. Quantifi automatically captured the user's name and time taken for each intervention. Using this approach, a user could quickly document an intervention by completing only 3 fields. A second documentation form within Quantifi allowed users to document multiple occurrences (up to 20) of the same intervention without including patient-specific information. The user also had to select the service, indicating the location of the interventions. Although not visible to the user, the software also determined any monetary savings associated with the interventions.

Form Field Elements.

The majority of fields on the documentation form used customizable drop-down lists, which allowed for capture of consistent, structured information. Without structured information, analysis of aggregate data would be difficult and time consuming. These lists were customizable, allowing for tailoring of the software to meet the school's needs. The service list was edited to reflect the school's IPPE and APPE training sites and was updated when necessary to reflect changes in our program.

A default intervention list was provided that reflected acute care activities. This list required modification to meet the school's needs by adding, deleting, or changing the interventions to reflect IPPE, APPE, and faculty practice activities for inpatient, outpatient, and nontraditional (ie, academic practice) settings. Interventions/activities for outpatient visits were developed to track and document the activities of pharmacy practice faculty members. A secondary intervention also was added to quantitatively track visit type (eg, diabetes, anticoagulation, dyslipidemia, etc).

The intervention list used by the school included a definition for each intervention and activity. The list was updated several times based on feedback from both faculty and student pharmacists to provide clarification and additions/deletions of specific interventions.

One limitation of the documentation form was that the field name was not customizable. This limited the ability to repurpose unused fields to capture important information that could not be gathered elsewhere. Specifically, we wanted to rename an unused field “Preceptor Name” to provide a specific location for students to identify their preceptor. Thus, students had to indirectly identify their preceptor when they selected the service (eg, “site name-preceptor name”). Only full-time faculty members could be identified this way.

Each intervention had several elements: (1) type (intervention name), (2) customizable soft and hard costs (explained below), (3) whether it can be used as a primary, secondary, and/or quick intervention, and (4) customizable time taken. Other elements that were available but not used included intervention class and relative value unit.

The Pharmacy OneSource financial model for cost savings/avoidance was available within the software and was supported by literature from leading institutions. This is the model our oversight committee chose to use. As of December 2010, the system included 20 interventions with a designated soft cost savings/avoidance of $153 and 4 interventions with hard cost savings/avoidance ranging from $17 to $214. Interventions/activities that were added to meet the school's specific needs (n=39) did not have any cost savings/avoidance data assigned to them, so until further research or additional literature became available, only the number of these interventions performed was collected. (The cost savings/avoidance data given represent 2010 figures and may not represent figures used at all institutions.)

User Management

Quantifi provided 3 user profiles: administrator, pharmacist, and manager. Users with administrator accounts maintain and manage the software and user accounts in conjunction with the oversight committee. Manager accounts were assigned to faculty members. Pharmacist accounts were assigned to students.

Being able to establish new user accounts efficiently was important because approximately 150 students needed to be added to the system yearly. This process is completed annually by the vendor to save time. Individuals with administrator access then maintained the user accounts.

Reporting

Reporting is one of the most important features of any documentation system. The site administrators and users with manager accounts can create and save reports that are available to other user accounts. Administrators also can create and save private reports that are not available to other users. Each field on the documentation form is available for selection when creating a new report. Users also are able to define the data range when generating reports, which are viewable on screen in pie, chart, or trend format and are also available as a spreadsheet download. Spreadsheet reports that allow for further filtering and sorting of data within pivot tables also were provided by Pharmacy OneSource. Data could be analyzed by any field data that were documented by individual users. Being able to analyze all secondary interventions required that the data be exported to a spreadsheet and manipulated in a pivot table. The data also can be exported to a database table and then analyzed through the reporting functionality within the database.

Reporting for individual APPEs was handled by the faculty member in accordance with the needs of the experiential site or APPE requirements. Individual faculty members also created reports of their clinical interventions/activities for performance evaluations and to share with practice site administrators. The experiential director managed reporting for school-wide assessment with an annual report provided to the dean.

Training

Pharmacy practice faculty members were expected to use Quantifi to document clinical interventions/activities within their practice sites. Training for new faculty members occurred during faculty orientation each year. In addition, an annual update training for practice faculty members was provided to ensure understanding and consistency of documentation.

Third-year (P3) student pharmacists were expected to document clinical interventions that occurred in the school's IPPE courses. Faculty mentors for the students were expected to assist with this documentation and monitor for accuracy. This requirement helped to prepare student pharmacists for the APPEs. P3 student pharmacist training was completed just prior to the APPEs in one 2-hour session and consisted of an overview of the software and documentation form as well as a review and discussion of the appropriate documentation of several patient care scenarios. Pharmacy practice faculty members serving as APPE regional coordinators assist with training and monitoring of student documentation.

Student pharmacists were encouraged to document on a daily basis to improve accurate documentation of all interventions and minimize recall bias. Delay in documentation could result in less than optimal documentation by some students. The intervention definition list was provided to all faculty members and student pharmacists during training and was posted on the school's intranet. General instruction materials and all training documents also were available on the school's intranet.

Integration in the Curriculum

One of the oversight committee's original objectives was to determine how to integrate intervention documentation into the curriculum. The committee initially focused their efforts on student documentation during APPEs and by pharmacy practice faculty members as part of their normal clinical outreach activities. Documentation by pharmacy practice faculty members began in 2006 and, initially, was not consistent across the department. Through periodic discussions during departmental meetings and with heightened emphasis placed on documentation as part of annual review and promotion/tenure decisions, by December 2010 all practice faculty members were consistently documenting their interventions/activities.

Initially, there was some reluctance by students to document interventions because they did not fully understand the importance of documenting interventions or see any benefit to them directly. Continual efforts were made during training sessions to explain how the data was used by the school administratively and by faculty members during practice experiences to demonstrate and emphasize the importance of documentation.

Intervention documentation was a component of each APPE evaluation and was clearly outlined in the syllabus for each APPE. Students were instructed to document interventions during APPEs with both full-time and affiliate faculty preceptors. Some pharmacy practice faculty members assigned a specific percentage of the student's final grade for appropriate documentation. Student documentation during APPEs was monitored and reinforced by pharmacy practice faculty members precepting the APPE. Regional faculty coordinators also performed periodic reviews of student documentation and addressed any discrepancies. In addition, the experiential director reviewed student documentation during regional meetings with students throughout the P4 year. The director answered any student questions, reinforced the importance of documentation, and clarified any questionable documentation identified on reports.

Intervention documentation also was a component of the IPPE course, as described above. Interventions during IPPEs represented only a small percentage of overall intervention numbers, probably because of the team-based nature of the IPPE program and less emphasis being placed on documentation during IPPEs compared to APPEs.

Sharing of intervention data with affiliate APPE institutions/sites was an important goal of the documentation process. Many sites that documented interventions used different software and/or processes, or did not document interventions at all. We attempted to minimize or eliminate the need for duplicate documentation in multiple systems by faculty members and student pharmacists. Reports of intervention data for affiliate sites can be generated upon request or at specific intervals.

RESULTS

Use of Quantifi for documentation of faculty interventions began in 2006. Fourth-year student pharmacists began documentation in APPEs in May 2007. As of December 2010, 37 pharmacy practice faculty members, 6 pharmacy practice residents, and 489 student pharmacists had used the system to document clinical interventions and activities in practice sites and during the practice experiences.

The oversight committee was dissolved in fall 2009, and ongoing oversight responsibilities were assigned to existing committees. Three individuals were assigned administrator access and performed associated duties within the software. Maintenance of the software required little time or effort and focused on ensuring that the intervention list reflected our needs and that faculty members and students were able to access and appropriately use the software. Intervention documentation has become a routine part of our experiential program.

Impact

The number of student pharmacists using Quantifi doubled over the 3.5 years following implementation. During the evaluation time period (May 2007-December 2010), 187,097 interventions were documented by faculty members and student pharmacists. These interventions (Table 1) accounted for over $6.2 million in cost avoidance (hard and soft dollars). Cost avoidance data do not reflect interventions added to the system by the school because they do not have an associated cost avoidance dollar value.

Table 1.

Summary of Intervention and Activity Data Captured Using a School-wide Electronic Web-based Documentation System (May 2007-Dec 2010)

	2007a	2008	2009	2010
Total number of interventions/activities	21,790	47,821	54,298	63,188
Associated cost avoidance ($)	669,372	1,486,885	1,810,471	2,243,280
Total number of users	138	211	220	254
Total number of faculty users	24	22	27	27
Total number of student users	114	189	193	227
Total student interventions	10,509	35,032	42,312	45,460
Total student cost avoidance ($)	310,198	1,124,832	1,453,895	1,704,079
Total faculty interventions	11,281	12,789	11,986	17,728
Total faculty cost avoidance ($)	359,174	362,053	356,576	539,201
Average number of interventions/activities per user	158	227	247	249
Average cost avoidance per user ($)	4,851	7,047	8,229	8,832
Average number of interventions/activities per faculty member	470	581	444	657
Average cost avoidance per faculty member ($)	14,966	16,457	13,207	19,970
Average number of interventions/activities per student pharmacist	92	185	221	200
Average cost avoidance per student pharmacist ($)	2,721	5,951	7,533	7,507

^a8 months of data

The average number of interventions per student ranged from 92 to 221 per year, and the associated cost avoidance per student ranged from $2,721-$7,533. Intervention data for student pharmacists also included interventions documented by 6 pharmacy residents (490 interventions with $12,102 of associated cost avoidance) during this time period.

The top 5 interventions/activities by number and associated cost avoidance are shown in Table 2. Four of the top 5 interventions/activities by number (except patient counseling) were interventions that we added to the documentation system. Therefore, there are no cost avoidance values for these 4 interventions.

Table 2.

Top 5 Interventions by Number and Cost Avoidance Based on Data From a School-wide Electronic Web-based Documentation System (2007-2010)^a

Interventions by number
Inpatient encounter (chart review/rounding)	42,298
Chart review	16,310
Patient counseling	11,115
Drug therapy adjusted (dose/frequency/etc)	10,807
Outpatient-Pharmacy Care reassess	9,604
Interventions by associated cost avoidance ($)
Patient counseling	1,086,115
Laboratory evaluation	900,602
Antibiotic recommendations	757,346
Renal dose evaluation	512,621
Drug therapy consultation	476,120

^aOnly 8 months of data for 2007.

Sixty-three percent (117,081) of all interventions were documented in inpatient settings, 32% (60,759) in outpatient clinics/primary care settings, 3% (5480) in community pharmacies, 1.6% (3,777) in specialty pharmacy settings (drug information centers, home infusion companies, nuclear pharmacies, poison control centers, administrative, and academic settings), or other unlisted settings. The cost avoidance allocations for each practice setting are similar: 68% ($4,232,319) inpatient, 25% ($1,540,725) outpatient/primary care, 5% community pharmacies ($311,500), and 2.2% ($125,464) specialty/other.

DISCUSSION

Web-based software programs such as Quantifi offer colleges and schools of pharmacy an organized, easy-to-use mechanism for documenting interventions by pharmacy practice faculty members and student pharmacists. The Web-based nature of the application was critical to our success in using it as a school-wide documentation tool. It allowed us to manage hundreds of users in multiple sites to capture structured intervention/activity information in real time. This information can be beneficial in showing affiliated training sites the potential impact that faculty members and student pharmacists can have on patient care. Implementation, training, integration, and reporting are all important steps a school should thoroughly consider when selecting a software program. We spent considerable time modifying the tool to meet our specific needs. We believe this time was well spent because it resulted in a documentation form that meets the needs of a diverse user base.

Documenting of interventions by faculty members and student pharmacists at our institution has increased each year since implementation. Pharmacy practice faculty members most often allow student pharmacists to document collaborative interventions during the practice experiences. Default interventions within Quantifi are typically focused more toward inpatient interventions but through customization options, we were able to edit/add interventions that allowed for documentation in any practice setting and met school needs.

Reporting functionality should allow the end-user to easily sort the data in multiple ways (eg, user type-faculty, resident, student, and type of practice setting). Secondary interventions are not easily tracked in the reporting functionality available to users. Exportation of data into spreadsheets (or those provided by Pharmacy OneSource) is helpful with data analysis, but extracting data in these reports can be tedious and time-consuming. Multiple spreadsheet files may be required for large quantities of data because of maximum line limitations within the spreadsheets.

Continual monitoring and follow-up by faculty members and the experiential director is required to ensure that student pharmacist documentation is occurring and is accurate. In addition to monitoring, establishing student understanding of the importance of documenting interventions during all practice experiences requires continuous communication and training. Affiliate preceptor education and development regarding documentation of interventions by students may help to maximize use of the system in all training sites and help to ensure thorough documentation by the student pharmacists. Monitoring of faculty data is also important to ensure consistency in documentation. Ongoing training for faculty members has been identified as an important component of our intervention documentation efforts.

To this point, intervention documentation in practice experiences at our institution has primarily occurred in the APPEs with limited activity in the IPPEs. Documentation within this portion of the curriculum needs to be reemphasized for more complete documentation of the interventions that are occurring during direct patient care activities of the IPPE program.

Future documentation efforts may include tracking of additional data to assist with assessment of the curriculum and compliance with ACPE accreditation standards. Examples might be to require entry of the primary drug involved in the intervention, patient information such as age and gender, and the significance and outcome of the intervention if known.

CONCLUSION

The Harrison School of Pharmacy successfully implemented a clinical intervention documentation system across APPEs for student pharmacists and across practice sites for faculty members. This initiative involved a broad group of faculty and staff members who spent considerable time refining a commercial tool to meet the unique needs of the educational setting. The most important characteristics of the documentation tool are: (1) flexibility in defining lists, (2) the capturing of structured data, and (3) access from any Web-connected computer. Manipulation of reports was the most challenging aspect of the tool. Data from the first 44 months of use indicated that the school's APPE program was having a substantial impact. Integration and use of the tool during IPPEs has not reached its full potential.