Implementation of an Error-Reporting Module Within a Biorepository IT Application to Enhance Operations

Abstract

The Collaborative (formerly the Cooperative) Human Tissue Network (CHTN) is a federally funded service oriented grant that provides high-quality biospecimens and services to the research community. The CHTN consists of six institutions located throughout the United States to assist investigators in obtaining research specimens required for basic research. The CHTN divisions have similar operating goals: however, each division is responsible for maintaining operations at their local institutions. This requires the divisions to identify ways to maintain and sustain operations in a challenging federally funded environment, especially when the number of investigators requesting services drives the operation. Sustainability plans and goals are often times patched together out of necessity rather than taking a thoughtful approach by clearly defining and aligning activities with business strategy and priorities. The CHTN Western Division at Vanderbilt University Medical Center (CHTN-WD) has responded to this challenge of biospecimen resource sustainability in the face of diminished funding by continually identifying ways to innovate our processes through IT enhancements and requiring that the innovation produce measurable and relevant criteria for credibly reporting our operations progress and performance issues. With these overarching goals in mind, CHTN-WD underwent a Lean Six Sigma (LSS) series to identify operational inefficiencies that could be addressed with redesigning workflow and innovating the processes using IT solutions. The result of this internal collaborative innovation process was the implementation of an error-reporting module (ERM) hosted within our biorepository donor IT application, which allowed staff to report errors immediately; determine the operational area responsible; assess the severity of the error; determine course of action; determine if standard operating procedure (SOPs) revisions were required; and through automated e-mails, alert the area personnel responsible. The module provides a data-reporting feature by date range and area of operation for management and analysis.

Introduction

The CHTN Western Division (CHTN-WD) at Vanderbilt University Medical Center (VUMC) is one of six federally funded prospective procurement resources that have provided academic and commercial investigators with high quality biospecimens since 2001. The scope of collections is aimed at cancer-related resections; however, biospecimens from normal and diseased tissues are collected as requested by investigators. Like many repositories, we face challenges that are driven by reduced funding and increased operational and institutional costs and must find creative ways to neutralize or offset these costs. Identifying advantages, increasing operational efficiency, mitigating risk, and setting credible sustainability goals all bring real value to the biorepository.

Sustainability is perhaps the most confusing concept, especially if managers and directors do not have a firm grasp on business practices. Managers are often left to determine what operational improvement has the greatest benefit with minimal financial expenditure. Strong goal setting and project planning are crucial to the success of the biorepository and to ensuring that a sustainability program is driving real business value while avoiding misallocation of resources. Goals should be achievable and take into account the impact on the available resources, engage the personnel on the ground level of the operation, determine how success or failure will be measured, and how data required for progress statements will be produced.

CHTN-WD took a proactive approach by utilizing the Lean Six Sigma (LSS) philosophy and applying it to our operations with the ultimate goal of becoming a “leaner” operation and determining what core offerings could be most beneficial to our operations and ultimately the CHTN network. LSS goal is to minimize waste and resources, reduce variation and improve profitability by designing and monitoring business activities by streamlining each process and eliminating anything that does not add value for the investigator. In the most basic sense, it means, “doing more with less,” while giving investigators what they want.

Currently, the NCI grant awarded to the CHTN Western Division at Vanderbilt University Medical Center (CHTN-WD) is a 5-year grant that covers approximately 62% of our operating costs.Therefore we must find ways to cut cost and improve efficiency by setting achievable goals with metrics that can be quantified to support our infrastructure. CHTN investigators receiving tissues pay a portion of the procurement cost, which covers the remaining 38% of operations. Building a strong infrastructure, innovating our processes, and utilizing our strongest asset, our IT program, was the most apparent strategy that would have the greatest impact on our success and long-term survival.

Materials and Methods

In 2009, our biorepository began a LSS series¹ to explore ways to improve operations and lower costs, enhance our enterprise through innovative IT improvements, and reduce the complexities associated with our operational requirements to ensure good laboratory practice (GLP) and enhance our total quality management (TQM)² and standard operating procedures (SOP). The structure of the series required personnel “buy-in” and adoption of the LSS philosophy, which is an environment committed to continuous improvement by minimizing waste and the efficient use of resources.^3–5 The series consisted of LSS training sessions and workbooks to manage the process throughout a 6-month implementation schedule. The end result provided repository management with some unique staff perspectives to refining operations, and identified opportunities to improve SOPs and increase overall operational efficiency (Table 1).

Table 1.

Six Sigma Series Outcome

LSS Result	Area of operation	Impact factor (1 lowest, 4 highest)	Success ratio predicted prior to implementation (1 lowest, 5 highest	Cost of implementation (time only)	Cost of implementation (dollars only)	Planned project implementation schedule priority level	Anticipated training problems (1 lowest, 4 highest)
Error Reporting Module	ALL	4 High Impact	5	150 hours-IT 80 hours-Design Version I 2 hours Training 30 hours Beta test 20 hours-Design Version 2 1 hour Training	IT-$9000 Design- $3500 Training- $1200 Beta Testing- $5400 Design Version 2- $900 Training- $300	1	1
Investigator file linked to Donor Portal	Investigator communications	4 High Impact	3	80 hours-IT	IT-$4600	2	3
Consent documents Re-route	Consent and Regulations	1 Low Impact	3	20 hours-Consent staff 2 hours- SOP revisions and training	$3100	3	1

The outcome of the LSS process with the biggest impact on our operations is the development and implementation of an Error-Reporting Module (ERM), which replaces Corrective and Preventative Action (CAPA) forms within our VUMC DQuest IT repository application. The DQuest application, developed by CHTN-WD, is a true end-to-end biorepository IT application, beginning with the identification of the patient and consent to the collection of data and samples to Quality Assurance and Quality Control (QA/QC) and distribution, and ending with invoicing the CHTN approved investigator and request for feedback on services. DQuest repository application and other CHTN products are scheduled to be available to the public in 2015 under a limited licensing agreement.

Many repositories do not have sophisticated strategies for error reporting and analysis and struggle with how to evaluate their operations effectively. Managing day-to-day operations to maintain the repository is a difficult task, and encouraging staff to report errors is equally challenging. It includes the time involved in reporting, the need to stop work and complete cumbersome CAPA documentation, and the fear of poor performance evaluations. Adapting and working with different models for reporting errors became overwhelming and required that we tie together components manually, which was inefficient, time-consuming, and costly, especially when we experienced 40%–50% procurement and distribution increases, and added one to two staff members per year. This significant increase also led to an increase in the amount of time, approximately 18 hours a week, the repository manager dedicated to secondary checking of all repository operational areas due to very few CAPA documentation submissions. The financial burden of the process, approximately $30,000/year coupled with the time commitment was becoming unmanageable and counter-productive to the aims of the repository. Revising the process resulted in improved staff participation, automating the review process and restructuring workload by removing the burden from the repository manager and re-distributing to the laboratory technician III (Fig. 1).

FIG. 1.

Error reporting evolution and associated costs.

The ERM allowed our biorepository to leverage our IT agile development expertise to automate the processes that gather, manage, report, and take action on information necessary to streamline operations and minimize impact on personnel workflow and effort.

The ERM was easy to implement due to our processes being structured to mimic the flow of biospecimens and associated data moving through the system from the identification of the donor to the investigator receipt of the biospecimen and associated data (Fig. 2). This approach allows work to be grouped by function, enables resources to be utilized efficiently, instills ownership, documents the flow of work, and establishes the control points and measurements, and controls process deviations. The DQuest application is integrated with internal hospital systems such as the Operating Room (OR) scheduling software to retrieve patient data, which minimizes errors and eliminates the need for manual entry of patient demographics, procedure data, and medical record numbers. Each operational area requires a lead technician who is responsible for training, SOP revisions for compliance, managing inventory, and optimizing workflow, and reporting to the repository director/manager.

FIG. 2.

VUMC CHTN Tissue Repository workflow and secondary checking process with Biorepository Metrics from year 2013.

As Figure 2 illustrates, the biospecimen and data move through the repository and are processed (workflow), reviewed by the subsequent operational area (secondary check), and assigned a status. A brief description of work associated with each operational area and the associated metric for year 2013 is provided to show the scope and size of our repository.

SOPs are structured so that boundaries delineate the input and output sides of the work flow domain. Internal to the process are interfaces between activities that demarcate the point at which work flows between process elements and secondary checks are applied to report and reduce errors (Fig. 3).

FIG. 3.

An example of shipping input/output boundary and workflow.

The biospecimen and data must pass through these input and output boundaries, passing secondary checks in order to move to the next status. If the biospecimen or associated data does not meet the quality standard, a determination is made to reject and destroy the biospecimen, report an error associated with the biospecimen or data, or move the biospecimen to the next level with a descriptor attached to the case for review by the next secondary checker. Errors can occur at any point in the process. Capturing errors to obtain a true repository error rate has been challenging in the past due to staff correcting the error without reporting the event.

The ERM form consists of a series of drop downs and auto-populated fields to minimize work flow interruptions and provide a mechanism for staff to share feedback regarding the potential source and impact of the error. The time involved to complete the ERM is dependent upon the type of error being reported. On average, the time to report an error and capture error details, takes between 15 seconds and 1 minute. There are challenges to defining what an error is and is not and how to attach a value (severity and impact) to the error. The general concept of an “error” implies that there is a mistake or an inaccurate result, which can be misleading. CHTN-WD defines an error as a deviation from normal processes (SOPs), increased workload in any operational area or transaction, or a deviation from the normal process that is out of our control. The approach we took to defining errors used a combination of the scientific approach and the LSS approach, which provided the framework for how we determined the types of errors and their severity and impact.

In the scientific approach, systemic or determinate errors can be avoided or corrected and the severity and impact can be determined, such as ensuring that equipment is properly calibrated. Random or indeterminate errors are those in which there is no control over the error, such as the patients inability to provide a blood sample, due to poor vein quality.

The LSS approach focuses on eight kinds of waste: defects, overproduction, waiting, non-utilized talent, transportation, inventory, motion, and extra processing. Each operational area utilized the LSS-DMAIC method to define, measure, analyze, improve and control functions in their work area.⁶ The processes in each area were broken down into “task-transactions” containing potential for error and waste. Once processes were defined, we combined these data with known errors reported over the course of 3 years. Detailed process flow charts were developed outlining each transaction in each operational area in which errors have occurred and the data used to create a Risk Analysis and Mitigation (RAM) document to work in tandem with internal SOPs. These were critical for identifying opportunities for improvement in both training and SOP development. Table 2A and 2B provides examples of each type of error that has occurred in our operations.

Table 2A.

Systematic Error Examples

Systematic or determinate errors	Examples of errors
Operational	Wrong concentration of antibiotic added to media
Personal	Newly trained staff unable to grossly detect fatty breast from breast tissue
Instrumental	Scales incorrectly calibrated
Reagent	Contaminated media
Error of method	Obtaining viable tumor cells from diseases which require neo-adjuvant therapy as standard of case
Additive or Proportional errors	Impurities in RNA later™, which may render biospecimen unusable
Random or indeterminate errors	Errors in which there is no control

Table 2B.

Lean Six Sigma Error Examples

Lean Six Sigma Errors or waste	Examples of errors
Overproduction	Ordering or making large quantities of reagents with short shelf life
Transportation	Failure to prepare for collections (wrong blood tubes in collection bags)
Rework	Pathology reports contain medical device manufacturer and serial number
Overprocessing	Reviewing or performing the same task multiple times
Motion	Errors in pulling biospecimens for orders
Inventory	Procurement of unrequested samples
Waiting	Unresponsive staff

The combined data resulted in the creation of the first prototype of the ERM, which was beta-tested for approximately 4 months by personnel. Adjustments were made using feedback from the testing phase to create the final ERM product.

The ERM product is embedded within the DQuest application as a menu item and can be accessed by any staff member, with login credentials, during any stage in the biospecimen lifecycle. Some targeted operations, such as changing the status of a biospecimen to “pending to destroy,” will trigger the ERM and require the user to complete the form, as the result of the new status of the biospecimen (Fig. 4).

FIG. 4.

Automated ERM based on status (QAQC decision) of biospecimen.

The ERM allows the user to link a case ID with an error report or manually input a case ID to report the error. Ninety percent of staff is cross-trained in at least two operational areas, in addition to their primary role: one downstream area and one upstream area. For example, consent staff is trained to approach and consent patients as their primary role, and chart review data entry and bio-fluids processing as secondary roles. This cross training ensures accurate reporting of the “most probable” operational area in which the error occurred.

Personnel are required to make assessments on the severity of the error by selecting one of the following criteria:

• • Minor errors that can be corrected within a short period of time and will not affect operations and there are no HIPAA or security violations.

• • Major errors, which are required to be corrected as soon as possible because operations will be affected long-term and there are no HIPAA or security violations.

• • Critical errors, which halt operations until they are reconciled. These errors may involve a HIPAA or security violation.

Personnel are also required to describe the error, how the error was corrected or reconciled, assess the operational impact, and determine if SOPs need revisions. Operational SOPs are embedded in the DQuest application, which provides personnel with the opportunity to review the SOPs while reporting the error to improve decision-making or expose vulnerabilities within the SOPs (Fig. 5).

FIG. 5.

VUMC CHTN Standard Operating Procedures hosted on DQuest IT application.

Once errors are saved to the system, an automated e-mail notification is sent to the person responsible for the area in which the error occurred, informing them of the error and the details. The repository manager also receives the e-mail, which provides real-time visibility. Error reports can be viewed and downloaded by area of operation and date range or error ID, in a variety of formats. The error reports are provided to staff and discussed at weekly lab meetings. Areas experiencing a high volume of errors are evaluated to determine root cause and a plan is developed to improve system performance.

Results

CHTN-WD at VUMC implemented an ERM within our existing DQuest application in an effort to obtain an accurate biorepository error rate and to monitor and manage performance.

Repository managers bear the load of day-to-day operations, which may include staff operating in many different physical locations and different work times throughout the enterprise. To manage the workload in such a demanding environment, the manager must find ways to improve performance, lower costs, and increase responsiveness and accelerate information flow. We found that turning to IT solutions was the most cost-effective way to achieve measurable success with minimal financial input and minimal budget restructuring for our operation. The biggest challenge was communicating to the staff that the error-reporting philosophies were to help achieve operational excellence, determine error root cause, and not to assign blame or single out any one department or person.

There are 20 different functional areas identified in our operations in which errors could potentially arise. At any given time, there are approximately seven to ten employees reporting errors throughout the lifecycle of the specimen in our operations. Table 3 illustrates the reported errors in our operations from June 2013 to May 2014, the operational area responsible and the number of critical, minor, and major events associated with the errors. Examining the error report in detail provides information into business processes that could be enhanced or streamlined and also provides a way for employees to improve their critical thinking skills and judgment as they learn to incorporate their feedback in meaningful and constructive ways. For instance, there were 108 errors reported in the pathology report area, of which 56 were deemed critical and potential patient privacy violations. The secondary checker identified the errors either during the QAQC step or the data review steps before the shipment left our repository. Of the 56 errors, 29 contained a surgical pathology number, 16 errors contained the primary clinician or surgeons name, 4 contained a date of service, 6 contained the institution rendering the services, and 1 contained the serial number and model number of a medical device.

Table 3.

Error Report: June 2013–May 2014 Example

Operational Area	# Errors reported	# Critical	# Major	# Minor	# Employees reporting	# Employees fixing	# Requiring SOP revision
Chart review	8	3	0	5	1	1	0
Consent	33	8	6	19	6	5	4
Core Histology Service	35	4	2	29	6	5	0
Fluids Processing	95	1	1	93	4	4	1
Information Technology	2	0	2	0	2	1	0
Lab Functions	5	0	5	0	4	2	1
Pathology Report	108	56	13	37	9	7	0
QAQC	2	0	1	1	2	1	0
QAQC-result entry	32	0	0	32	3	1	0
QAQC-Slide setup	25	9	11	5	3	2	0
Shipping	3	0	0	3	2	3	0
Shipping-Fresh	6	1	3	2	3	4	1
Shipping-Frozen	1	1	0	0	1	1	1
Specimen Notification	2	0	0	2	1	0	0
Tissue Procurement	211	1	6	204	8	7	1
Block Processing (FFPE)	0
Investigator File Update	0
Investigator Relations	0
Invoicing	0
QAQC Pathologist Review	1			1	1	1
Total Errors Found	569	84	50	433

Having access to this information to analyze and act upon creates a competitive advantage and enables sustainable management through business process improvements and productivity gains while enhancing our reputation for providing accurate data with high quality specimens.

The repository manager is responsible for managing resource load, adjusting operations to level and balance the load, and to ensure that there is minimal impact when errors or problems arise or when implementing new initiatives. The ERM has shifted the burden from the manager “hunting” for errors within the operation, to the staff actively participating and supporting the error reporting process, thus enabling a true evidence-based decision making process supported by intelligent analytics. Performance and impact of the ERM, demonstrated in Table 4, provides nine examples that have led to an increase in our operational efficiency.

Table 4.

Performance and Benefits

Performance statement	Impact and/or benefit	Performance statement	Impact and/or benefit
Decrease cycle time between error notification and correction	Decreased backlogged error review process performed by repository manager	Empower staff to react immediately and record improvement opportunities	Increased: *Staff development *Critical and logical thinking skills *Subject matter expertise
Increase operational transparency, accountability and productivity (proactive monitoring)	Real-time analytics	Audit compliance	Increases the efficiency by addressing the logs in weekly lab meetings to quickly determine negative impact
Reduce incomplete data	Higher quality specimens and data	Assist internal TQM operational area certification for staff	Known or new errors can be introduced into the system to test and/or validate compliance with reporting
Reduce complexity of reporting	Increased staff participation in reporting and identifying new areas of concern	Assist internal TQM operational area certification for staff	Known or new errors can be introduced into the system to test and/or validate compliance with reporting
Promote enterprise-wide harmonized process and centralize reporting	Significantly reduced repository managers effort for collating, organizing, tracking and responding to errors and can be accessed by all staff members

Previously there was a performance gap between the realized and the potential value of the information that could be gleaned from errors. Staff corrected errors without informing the manager or the operational area responsible, thereby bypassing the valuable information that could have improved processes and reduced the likelihood of the errors being repeated. The ERM has allowed our operations to view data obtained from the ERM as a true information asset to our operations. Labeling, binning, and analyzing data from each operational error has enabled our program to view the data as either nonfinancial or a financial asset, which in terms of sustainability is crucial to our enterprise.

Conclusion

Sustainability, by definition, is to endure and remain diverse and productive. The ability to capture an event-driven processing model (ERM) to deliver real-time information to both the users and the repository director is invaluable in determining quickly any operational areas that are in jeopardy of exceeding an allowable error rate. The ultimate benefits are in the productivity, lower operational cost, reduced workload for managers, and most importantly, auditing and accountability as well as capturing data required to develop metrics that can drive efficiency.

The cost of the development, implementation, and initial training documentation was approximately $20,300. This is considered a one-time investment with occasional drop down values being added by the IT personnel, which accounts for less than 0.01% effort per year. The LSS predecessor activities that occurred and resulted in the ERM are independent of the IT development and training costs, since they have been spread across multiple operational enhancements. The only current cost associated with the production ERM is the effort spent by the personnel assigned to review, organize, and lead problem-solving discussions on errors that occurred between scheduled lab meetings, which is approximately $7000 per year.

The ERM responsibility shifted to a senior lab technician who has been trained in 90% of the critical and functional areas, which has taken the burden off the manager and provides a birds-eye view of the operations, and if needed, can drill down quickly to areas that do not meet the expected quality standards.

The development and implementation costs were returned quickly as shown in Table 5, which provides a breakdown of associated costs. It is important to note that the institution provided both the production and test server environment, database administrator and architect, as part of the indirect cost associated with our award. The LSS series CHTN-WD implemented was intended to familiarize staff with the philosophy and principles and is an iterative process; to enhance and diversify operations; to identify problems and provide workable solutions that are lean; and to produce a lasting culture of continuous improvement to sustain our repository both financially and operationally. These are important criteria to maintain and secure additional funding and for preparing for a repository certification program.

Table 5.

Cost Table

	CHTN ERM Development cost	VUMC provided**	ERM utilization & enhancements per year
Software Development (ERM)	$9000	0	<$100.00
Server cost (Production and Test)**	0	100%	0
Database Administrator and IT Architect**	0	100%	0
LSS Design V1	$3500	0	0
Training V1 beta testing	$6600	0	0
Design V2	$900	0	0
Training V2 Production	$300	0	0
Total cost of development and implementation (one time cost)	$20,300	N/A**
ERM Auditing, reporting results and re-training (Lab tech effort) 2011 to present			$7500.00
TOTAL COST (per year)			$7100.00
2003–2009 Costs	Cost (per year)
CAPA-type reporting (repository manager effort)	∼$30000

Acknowledgments

This study was supported by the National Institutes of Health, National Cancer Institutes, Grant 1UM1CA183727-01. The authors would also like to acknowledge Ms. Linda Sircy and Ms. Kiley Wease for their contributions to the figures, diagrams, and tables.

Author Disclosure Statement

No competing financial interests exist.