Total integrated performance excellence system (TIPES): A true north direction for a clinical trial support center

Abstract

This paper presents the quality journey taken by a Federal organization over more than 20 years. These efforts have resulted in the implementation of a Total Integrated Performance Excellence System (TIPES) that combines key principles and practices of established quality systems. The Center has progressively integrated quality system frameworks including the Malcom Baldrige National Quality Award (MBNQA) Framework and Criteria for Performance Excellence, ISO 9001, and the Organizational Project Management Maturity Model (OPM3), as well as supplemental quality systems of ISO 15378 (packaging for medicinal products) and ISO 21500 (guide to project management) to systematically improve all areas of operations. These frameworks were selected for applicability to Center processes and systems, consistency and reinforcement of complimentary approaches, and international acceptance. External validations include the MBNQA, the highest quality award in the US, continued registration and conformance to ISO standards and guidelines, and multiple VA and state awards. With a focus on a holistic approach to quality involving processes, systems and personnel, this paper presents activities and lessons that were critical to building TIPES and establishing the quality environment for conducting clinical research in support of Veterans and national health care.

1. Introduction

This paper presents the evolution of an entrepreneurial and efficient Federal Government entity, the VA Cooperative Studies Program (CSP) Clinical Research Pharmacy Coordinating Center (the Center). The CSP was formally organized in 1972 [1] and currently consists of five biostatistical and data management coordinating centers (BDMCCs), the pharmacy center (Center), five epidemiological centers, a pharmacogenomics laboratory, and a network of dedicated enrollment sites or NODES [2]. One of the premier research programs worldwide, CSP conducts multicenter clinical trials on the efficacy or comparative effectiveness of treatments for a range of areas including cardiovascular disease, endocrine disorders, neurological conditions, cancer, psychiatric disorders, and infectious diseases that impact Veterans and the nation. Several of these studies have been considered landmark trials that have influenced clinical practice and formed the evidence base for treatment guidelines.

The Center provides pharmaceutical/device, regulatory and patient safety support while the BDMCCs provide design/analysis, data collection, data management, and overall project management to CSP's cooperative clinical trials. Along with the Center, they conduct clinical trials using CSP's established program guidelines. The Center and the BDMCCs have a symbiotic relationship which provides the foundation for a robust common key work system and key work processes [3,4]. The Center's process documentation and core competencies that underlie the performance excellence system are shown in Fig. 1.

Fig. 1.

The figure illustrates the number of Center sections, standard operating procedures (SOPs), and approved methods and procedures (AMPs) related to the key work system and key work processes along with core competencies.

The mission of the Center, in supporting multicenter clinical trial research, is to improve the health of the nation's Veterans by providing creative pharmaceutical solutions to global clinical research. Particular effort has been made to support a culture of engagement, continuous improvement, principles-based action, and accountability. Employee surveys indicate success in creating such an environment. The approximately 100 people working at the Center, labor to help fulfill President Lincoln's promise, “To care for him who shall have borne the battle, and for his widow, and his orphan” by serving and honoring America's Veterans.

Within this context, the Center has progressively integrated several management systems and complementary quality frameworks to achieve a robust operational and performance excellence system. The subsequent paragraphs describe the assimilation, implementation, and synergistic benefits of these frameworks and systems.

2. Methods

2.1. Total Integrated Performance Excellence System (TIPES)

An integrated quality system provides value to an organization by promoting and sustaining a culture of quality and performance excellence, mitigating the risks associated with multicenter clinical trials and protecting the research participants [5]. Tsiakals [6] adapted Maslow's hierarchy of needs to create a quality hierarchy (QH) founded on ISO 9001:2000. As in Maslow's model and the QH model, lower order needs or requirements must be met before higher order ones can be addressed. The Center augmented this model to develop a more comprehensive QH that goes far beyond Tsiakals' model. The Center's current QH represents the culmination of three cycles of improvement and now integrates the Center's core competencies into management and quality systems, a construction known as TIPES (Fig. 2). This entailed the formation of a steering group consisting of senior leaders, Baldrige category leads and subject matter experts to deploy the TIPES concepts and operationalize daily work within this framework. This group meets on a periodic basis to ensure the understanding of TIPES concepts and track progress towards integrating Center processes in alignment with quality system principles.

Fig. 2.

The quality hierarchy illustrates four management systems that represent the core competencies along with integrated quality systems, strategic elements and goals. These foundations represent TIPES.

2.2. Core competency management systems

Baldrige [7] defines core competencies as, “Your organization's areas of greatest expertise; those strategically important capabilities that are central to fulfilling your mission or that provide an advantage in your market or service environment.” The core competencies, incorporated in its management systems, are fundamental Center assets which are integral to how activities are developed, organized, and delivered under a quality framework.

2.2.1. Regulatory and patient safety management (RPSM)

RPSM (Fig. 2) constitutes the Center's QH base, ensuring compliance with the multitude of regulations governing clinical trial organizations' operational and research activities. Performance excellence in this field is essential to the inherent function of the Center and its role within CSP as a Program. Importantly, RPSM includes active monitoring of clinical trial participants' safety. The quality system (QS) underlying RPSM includes ISO 9001:2008 (ISO 9001), ISO 15378:2011 (ISO 15378), and the ISO-required internal and external (third party) audit system.

2.2.2. Clinical materials management (CMM)

CMM (Fig. 2) is responsible for the Center's tangible products: clinical materials manufacturing, packaging, labeling, central distribution, and inventory management, ensuring quality at every step. CMM also includes the Clinical Trials Pharmacy Support Center (CTPSC) consisting of three software modules: An Inventory Tracking System, Randomization and Treatment Assignment System, and Participating Site Pharmacy System. CMM represents a crucial support system on which Pharmaceutical Project Management relies to achieve the Center's mission. As above, the QS for CMM includes the ISO 9001 Framework and additionally ISO 15378, a packaging standard for pharmaceuticals.

2.2.3. Pharmaceutical Project Management (PPM)

PPM (Fig. 2) coordinates and oversees the pharmaceutical, regulatory, and patient safety aspects of clinical trials through the Center's matrix management system comprised of 12 functional groups. PPM provides the Center's external interface as it manages Center product and service delivery. In addition, PPM collaborates with the BDMCCs, study chair's offices, local site investigators, and study coordinators throughout the U.S. and other countries. The QS for PPM begins with the ISO 9001 Framework, adds the Organizational Project Management Maturity Model (OPM3) Framework and employs the ISO 21500:2012 (ISO 21500) Guidance on Project Management. ISO 9001 and 21500 are critical, as they require the internal and external audits that ensure process adherence and quality.

2.2.4. Performance excellence management (PEM)

PEM (Fig. 2) provides a holistic approach to organizational management that promotes a culture of excellence, continuous improvement, and innovation, resulting in competitive advantages. PEM fosters a systems approach to deliver ever-increasing value to customers and stakeholders via organizational products and services. A systems approach accomplishes work through the skills of the workforce, interlinking work processes and resource utilization to create value and capacity; systems-related decisions are inherently strategic [7,8]. W. Edwards Deming said, “six percent of the problems we experience can be traced back to people. Ninety-four percent are inside the system.” PEM continuously improves the organization and its employees through value-added products and services, delivered with a focus on customers, operations, and financial and market place results. The QS for PEM adds OPM3 and ISO 9001 to the Baldrige Framework.

2.3. Quality system frameworks

The three overarching frameworks of the QS supporting TIPES are Baldrige, OPM3, and ISO 9001. These frameworks were selected for applicability to Center processes and systems, consistency and reinforcement of complimentary approaches, and international acceptance. Other ISO standards, including 15378 and 21500 are supplemental to these three frameworks. The collective QS is the backbone of TIPES. When fully deployed and interwoven throughout the four management systems (RPSM, CMM, PPM, and PEM), the TIPES QS frameworks combine synergistically, moving the Center toward total performance excellence, thereby pointing the organization in what the Center refers to as the “true north” direction.

2.3.1. Baldrige excellence framework [7].

The Baldrige Performance Excellence Program was created in the mid-1980s by the National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce, as a response to the need to increase global competitiveness. Initially focused primarily on businesses, the Baldrige Framework and Criteria now encompass government, health care, education, and nonprofit sectors and are designed for application by any organization regardless of size. The Leadership 500 Excellence Awards regularly rank the Baldrige Performance Excellence Program in the top 10 in the government and military category; for the past two years, it has been awarded first place [9]. In their evaluation of performance excellence models, Marsh, et al. [10], indicate that the Baldrige Criteria does the best job of identifying the core values and concepts that lead to organizational excellence. It is, therefore, a reputable and resilient approach for any organization. Rigorous utilization of the Baldrige Framework helps to focus an organization to reach its goals, improve results, and become more competitive.

The Criteria include seven major categories and prompt organizations to evaluate their effectiveness through a series of probing questions (Table 1). To formally participate in the program, organizations are asked to document the answers to over 200 questions. A notable benefit of the Baldrige process is the feedback provided by trained examiners. The feedback report contains detailed information on strengths and opportunities for improvement, and maturity of approaches and results. The Center has used Baldrige feedback reports as a key input to its operational and strategic planning. Achieving recognition as a Baldrige Award recipient signals to the outside world that an organization is a role model.

Table 1.

Quality framework category descriptions and alignments.

Baldrige category & description [7]	OPM3 Enabler category alignment [15]	ISO 9001:2008 & ISO 15378:2011 Category alignment [11]
1.0 Leadership Describe the organization's governance system. Describe how senior leaders' personal actions guide and sustain your organization and how they fulfill the organizations legal, ethical, and societal responsibilities	3 Governance 6 Management Systems 7 Organizational Project Management Communities 8 Organizational Project Management Methodology 9 Organizational Project Management Policy and Vision 10 Organizational Project Management Practices 11 Organizational Project Management Techniques 12 Organizational Structures 16 Resource Allocation 17 Sponsorship	5.1 Management Commitment 5.2 Customer Focus 5.3 Quality Policy 5.5 Responsibility, Authority, and Communication 5.6 Management Review 6.1 Provision of Resources 6.3 Infrastructure 8.4 Analysis of Data
2.0 Strategy Describe how the organization develops strategic objectives and action plans, implements them, changes them if circumstances require, and measures progress.	8 Organizational Project Management Methodology 9 Organizational Project Management Methodology 11 Organizational Project Management Techniques 15 Project Success Criteria 18 Strategic Alignment	5.4 Planning 8.4 Analysis of Data
3.0 Customers Describe how the organization engages its customers for long-term marketplace success, including how the organization listens to the voice of the customer, builds customer relationships, and uses customer information to improve and to identify opportunities for innovation.	2 Competency Management 5 Knowledge Management and PMIS 11 Organizational Project Management Techniques 17 Sponsorship	7.2 Customer-Related Processes 7.3 Design and Development 8.2 Monitoring and Measurement
4.0 Measurement, Analysis, and Knowledge Management Describe how the organization selects, gathers, analyzes, manages, and improves its data, information, and knowledge assets; how it learns, manages information technology, and uses review findings to improve performance.	1 Benchmarking 4 Individual Performance Appraisals 5 Knowledge Management and PMIS 11 Organizational Project Management Techniques 13 Project Management Metrics	8.1 Measurement, Analysis, and Improvement – General 8.2 Monitoring and Measurement 8.4 Analysis of Data 8.5 Improvement
5.0 Workforce Describe how the organization assesses workforce capability and capacity needs, builds a workforce environment conducive to high performance, and engages, manages, and develops the workforce to utilize its full potential in alignment with the organization's overall business needs.	2 Competency Management 8 Organizational Project Management Methodology 10 Organizational Project Management Practices 14 Project Management Training 16 Resource Allocation	6.2 Human Resources 6.4 Work Environment
6.0 Operations Describe how the organization designs, manages, improves, and innovates its products and work processes and improves operational effectiveness to deliver customer value and achieve ongoing organizational success.	5 Knowledge Management and PMIS 6 Management Systems 7 Organizational Project Management Communities 8 Organizational Project Management Communities 11 Organizational Project Management Techniques	4.1 QMS – General Requirements 4.2 Documentation Requirements 6.5 Maintenance Activities (15378 Only) 7.1 Planning or Product Realization 7.2 Customer-Related Processes 7.3 Design and Development 7.4 Purchasing 7.5 Product and Service Provision 7.6 Control of Monitoring and Measuring Equipment 8.2 Monitoring and Measurement 8.3 Control of Nonconforming Product 8.4 Analysis of Data 8.5 Improvement
7.0 Results Describe the organization's performance and improvement in all key areas – product and process, customer focused, workforce focused, leadership and governance, and financial and market results. Describe the performance levels relative to those of competitors and other organization with similar product offerings.	No direct alignment	8.2 Monitoring and measurement 8.4 Analysis of data

2.3.2. ISO 9001 framework (quality management systems) [11,12].

The International Organization for Standardization was founded in 1946 and based in Geneva, Switzerland. In 2014, ISO certification bodies issued 1.1 million certifications to organizations in 162 countries worldwide [13]. ISO standards and guidelines represent a wide variety of industries, products, services, management systems, and other topical areas such as global social responsibility, environmental protection and ethics. The ISO 9001 standard is the most comprehensive quality management standard in the ISO suite.

ISO 9001 focuses on enhancing customer satisfaction by meeting customer requirements. Organizations must identify and manage linked activities, creating a system of processes that together produce a desired outcome. The systems approach provides ongoing control of processes, resulting in a high level of consistency or effectiveness. ISO 9001 is organized around eight quality management principles: (1) Customer focus, (2) Leadership, (3) Involvement of people, (4) Process approach, (5) System approach to management, (6) Continual improvement, (7) Factual approach to decision making, and (8) Mutually beneficial supplier relationships.

Like the Baldrige Framework, ISO uses third-party assessments to assess the organization's QMS, providing confidence to the organization and its customers that the QMS is documented, demonstrable, effective, and maintained. Like the Baldrige process, ISO 9001 is voluntary; however, ISO creates urgency through required annual audits for continual certification and importantly, mandates an ongoing comprehensive system of internal QMS audits. In contrast, Baldrige award recipients cannot reapply for the award for five years. In the Center's experience, this five-year interval can present a challenge in maintaining an organization's performance excellence momentum; however, the assimilation of ISO 9001 and OPM3 into the organization supports forward progress.

2.3.3. Organizational Project Management Maturity Model (OPM3) framework [14,15].

OPM3 measures an organization's maturity, described by this model as “the robustness of its Organizational Project Management (OPM) Infrastructure” [16]. This maturity-based approach, founded on concepts developed by the Software Engineering Institute (SEI) with the Capability Maturity Model Integration (CMMI) [17], begins with a baseline assessment of an organization and maps a desired future state. The Project Management Institute describes OPM as “… a strategy execution framework that utilizes portfolio, program, project management, and organizational-enabling practices to consistently and predictably deliver organizational strategy and results and a sustainable competitive advantage” [15]. In the OPM3 Framework Portfolio management provides a process to select projects/programs that align with the organization's mission and strategy. OPM3 provides a method for visualizing relationships and interactions between portfolios, programs and individual projects, ultimately creating a framework to assure that activities are consistent with an organization's strategic challenges, strategic opportunities, innovation, strategic advantages, intelligent risk and mission, vision and values (Fig. 3).

Fig. 3.

The figure represents the linkages between each of OPM3's domains of portfolio and program/project management along with an organization's strategic plan and operations. It also represents the relationship of strategic challenges, strategic opportunities, and innovation with strategic advantages, and how they trigger intelligent risk taking and their contribution to the realization of the mission-vision-values. This culminates in the development of strategic goals and objectives and consequently the strategic plan.

OPM3 measures an organization's maturity level for strategy execution by assessing the three domains of project, program and portfolio. Organizational maturity levels, from lowest to highest, are: (1) Standardize, (2) Measure, (3) Control, and (4) Continuously Improve. OPM3 assesses the level of maturity that is desired by an individual organization. Each of these maturity levels has a set of requirements or capabilities that are required to accomplish a best practice in the domain and to achieve a desired level of maturity.

2.3.4. Supplemental quality systems

ISO 15378 [18] (Primary packaging materials for medicinal products) contains requirements for the application of ISO 9001 to activities governed by the regulatory requirements of current Good Manufacturing Practice (21 CFR Parts 210 and 211), including manufacturing, packaging, labeling, and distribution of medicinal products. This standard lays out requirements for a QMS equipped to handle the specialized needs of these highly-regulated activities.

The release of ISO 21500 [19] (Guidance on project management) allows organizations to demonstrate project management process conformance through the internationally recognized ISO system. The guidance provides direction on concepts and processes considered good practice in project management and is directly aligned with the requirements/capabilities of OPM3 in the Project domain and Standardize level of improvement (Table 2). While ISO 21500 does not require the organization to identify their desired maturity level, it does provide a bridge for ISO-based organizations to use when aligning their ISO 9001 Quality Management System with their Organizational Project Management (OPM) Framework.

Table 2.

Alignment of the baldrige framework with OPM3 best practices framework at the project domain and standardize level and ISO 21500:2012 [7,15,19].

Baldrige	OPM3 Best practice	ISO 21500
1.0	SeeTable 1– OPM3 Organizational Enabler Category	4.3.17 Define project organization
3.0	1195 Project identify stakeholders process	4.3.9 Identify stakeholders
	2035 Project manage stakeholder engagement process	4.3.10 Manage stakeholders
	7540 Project control stakeholder engagement process	4.3.10 Manage stakeholders (repeat is intentional)
5.0	1090 Project plan human resource management process	4.3.15 Establish project team (repeat is intentional)
	1115 Project estimate activity resources process	4.3.16 Estimate resources
	1150 Acquire project team process	4.3.15 Establish project team
	1155 Manage project team process	4.3.20 Manage project team
	1200 Project plan risk responses process	4.3.30 Treat risks
6.0	1005 Develop project charter process	4.3.2 Develop project charter
	1020 Develop project management plan process	4.3.3 Develop project plans
	1030 Project collect requirements process	4.3.11 Define scope
	1035 Monitor and control project work process	4.3.5 Control project work
	1040 Project define scope process	4.3.11 Define scope
	1050 Project define activities process	4.3.13 Define activities
	1060 Project sequence activities process	4.3.21 Sequence activities
	1070 Project estimate activity durations process	4.3.22 Estimate activity durations
	1075 Project create WBS process	4.3.12 Create WBS
	1080 Project development schedule process	4.3.23 Develop schedule
	1100 Project estimate costs process	4.3.25 Estimate costs
	1110 Project determine budget process	4.3.26 Develop budget
	1120 Project plan risk management process	N/A
	1130 Project plan quality management process	4.3.32 Plan quality
	1160 Project plan communications management process	4.3.38 Plan communications
	1170 Project identity risks process	4.3.28 Identify risks
	1180 Project perform qualitative risk analysis process	4.3.29 Assess risks
	1190 Project perform quantitative risk analysis process	4.3.29 Assess risks (repeat is intentional)
	1210 Project plan procurement management process	4.3.35 Plan procurements
	1230 Direct and manage project work process	4.3.4 Direct project work
	1240 Project perform quality assurance process	4.3.33 Perform quality assurance
	1250 Develop project team process	4.3.18 Develop project team
	1260 Project manage communications process	4.3.39 Distribute information 4.3.40 Manage communications
	1270 Project conduct procurements process	4.3.36 Select suppliers
	1290 Project control procurements process	4.3.37 Administer procurements
	1300 Project control communications process	4.3.40 Manage communications
	1310 Project perform integrated change control process	4.3.6 Control changes
	1320 Project validate scope process	N/A
	1330 Project control scope process	4.3.14 Control scope
	1340 Project control schedule process	4.3.24 Control schedule
	1350 Project control costs process	4.3.19 Control resources 4.3.27 Control costs
	1360 Project control quality process	4.3.34 Perform quality control
	1370 Project control risks process	4.3.31 Control risks
	1380 Project close procurements process	4.3.37 Administer procurements
	1390 Close project or phase process	4.3.7 Close project phase or project 4.3.8 Collect lessons learned
	7500 Project plan scope management process	N/A
	7510 Project plan schedule management process	N/A
	7520 Project plan cost management process	N/A
	7530 Project plan stakeholder management process	N/A

2.4. Alignment of frameworks and systems

2.4.1. Baldrige Framework as the holistic foundation for all quality systems [7].

In medicine, the practice of treating the whole person, including physical, emotional and spiritual health is often referred to as a holistic approach seeking optimal health and wellness [20]. We can apply this same principle to managing an organization. Organizations are made up of many different types of people and components. Intuitively, taking a holistic approach to managing an organization (linking processes into systems) will enhance its ability to fulfill its mission and move closer to accomplishing its vision. The holistic approach to management is an integral component of the Baldrige Framework and its Criteria (Table 1). The Criteria also address core values and concepts that are embedded beliefs and behaviors and comprise the foundation of the Criteria. Baldrige Performance Criteria are inter-related and provide an alignment of all critical organizational components [21], a whole-body approach to the management of a research organization.

2.4.2. Alignment of the Baldrige Framework with the ISO 9001 framework and ISO 15378

Because Baldrige is more comprehensive and less prescriptive than ISO or OPM3, the Center uses the Baldrige Criteria as its foundational standard into which the other quality management systems are assimilated. Schulingkamp [22] states, “The Baldrige Framework provides a holistic systems-based business model that builds alignment across the organization by making connections between and reinforcing organizational systems, process, strategy and results.” ISO 9001 is process focused, linking processes into systems, which enhances Baldrige's systems perspective. ISO 15378 includes ISO 9001 requirements and adds one new section that is packaging-specific [18]. Like ISO 9001, the 15378 standards primarily support Baldrige Category 6 (Operations), with other categories supported to a lesser degree.

ISO is focused on documents, records, audits, and corrective and preventive action, while Baldrige criteria are strategic and results focused; both are focused on customer, process, and continuous improvement. Table 1 demonstrates the alignment of ISO 9001 with the Baldrige Categories and the extent to which ISO standards improve an organization's ability to conform to the Criteria. Additionally, Table 1 shows that although the ISO 9001 standards support all the Baldrige process categories to some degree, a significant portion of the ISO standard is most directly aligned with the Baldrige Category 6 (Operations).

When comparing ISO standards and Baldrige Criteria, there is rarely a one-to-one correlation, but frequently similar content or intent. One area where Baldrige and ISO diverge is internal audits. While Baldrige stresses the use of valid and accurate data for measurement and analysis, it does not require internal audits. ISO's rigorous internal audit requirement as a mechanism for analysis and improvement provides a valuable supplement to the Baldrige criteria.

Focusing the Center's internal audit program with respect to the five key work processes (Fig. 1), facilitated the streamlining of work processes and identified gaps as performance improvements (Table 3 Example). For instance, the Center initiated improvements in clinical trial budgeting based on internal audit findings of redundancies and unclear responsibilities between two Center sections. A rigorous internal audit schedule has yielded zero non-conformances in the last five external ISO audits. In addition, the 2017 ISO external audit assessed the effectiveness of the eight management principles (refer to 2.3.2) with all being rated as highly effective or effective, with additional critical ISO elements of management review and internal audits also rated as highly effective. Furthermore, over the past five years the overall ISO QMS was rated highly effective by the ISO certifying body. A 2009 Baldrige Feedback Report [23] Opportunity for Improvement (Table 3 Example), “Key work process requirements defined by the Center do not consistently reflect process – based requirements,” propelled the Center to define the key work processes from study planning through study analysis and unique process requirements as shown in Fig. 1. As stated in the 2009 Baldrige Feedback Report [23] a key strength was noted as, “The Center's leaders have developed systems and a culture that focusses on the future to ensure the creation of strategies, systems, and methods for achieving performance excellence, stimulating innovation, building knowledge and capabilities, and ensuring the Center's sustainability” (Table 3 Example).

Table 3.

Alignment of examples with quality system elements.

Baldrige	OPM3 organizational enabler category	OPM3 best practices	ISO 9001/ISO 15378	ISO 21500	Examples (section, example)
1.0 Leadership	3, 6, 7, 8, 9, 10, 11, 12, 16, 17	No OPM3 Best Practices in Project Domain/Standardize Level aligned with Baldrige Leadership Category	5.1–5.3, 5.5, 5.6, 6.1, 6.3, 8.4	4.3.17	2.4.4 Interlocking committee structure 4.1.2 Connecting employees with Mission 4.1.2 Evolving organizational structure 4.1.2 Having right people in management positions 4.1.3 Leaders interacting with all employees 4.1.4 Valuing employees
2.0 Strategy	8, 9, 11, 15, 18,	No OPM3 Best Practices in Project Domain/Standardize Level aligned with Baldrige Strategy Category	5.4, 8.4	N/A	2.4.2 Defining key work processes 2.4.2 Systems' culture focus on future 2.4.3 Project management as Core Competency 3.1.4 Managing intelligent risks/Establishing new capabilities 4.1.4 Providing environment, tools, learning opportunities
3.0 Customers	2, 5, 11, 17	1195, 2035, 7540	7.2, 7.3, 8.2	4.3.9, 4.3.10	3.1.2 Cycle of improvement to improve site inventories and reduce shipping errors 3.1.3 Customer engagement 3.1.4 Managing intelligent risks/Establishing new capabilities
4.0 Measurement, Analysis, and Knowledge Management (MAKM)	1, 4, 5, 11, 13	No OPM3 Best Practices in Project Domain/Standardize Level aligned with Baldrige MAKM Category	8.1, 8.2, 8.4, 8.5	N/A	2.4.2 Internal audits (Audits and records) 2.4.2 Improving clinical trial budgeting 3.1.2 Cycle of improvement to improve site inventories and reduce shipping errors 3.1.4 Innovation Management/Technological innovations 4.1.4 Providing environment, tools, learning opportunities
5.0 Workforce	2, 8, 10, 14, 16	1090, 1115, 1150, 1155, 1200	6.2, 6.4	4.3.15, 4.3.16, 4.3.20, 4.3.30	2.4.3 Evaluating SDM Team 2.4.3 Develop competencies for PPMs and PMs 2.4.4 Interlocking committee structure 3.1.3 Employee engagement 4.1.2 Connecting employees with Mission 4.1.2 Having right people in management positions 4.1.3 Employees knowing expectations 4.1.4 Providing environment, tools, learning opportunities 4.1.4 Maintaining low turnover rate
6.0 Operations	5, 6, 7, 8, 11	1005, 1020, 1030, 1035, 1040, 1050, 1060, 1070, 1075, 1080, 1100, 1110, 1120, 1130, 1160, 1170, 1180 1190, 1210, 1230, 1240, 1250, 1260, 1270, 1290, 1300, 1310, 1320, 1330, 1340, 1350, 1360, 1370, 1380, 1390, 7500, 7510, 7520, 7530	4.1–4.2, 6.5 (15378), 7.1–7.6, 8.2–8.5	4.3.2–4.3.8, 4.3.11–4.3.14, 4.3.18, 4.3.19, 4.3.21–4.3.29, 4.3.31–4.3.40	2.4.2 Streamlining work processes for performance improvement 2.4.2 Defining key work processes 2.4.3 Deploying project management 2.4.3 Project management as Core Competency 2.4.3 CTPP cycle of improvement 2.4.3 Integrated risk and change control 2.4.3 Streamlining process documentation, aligning to key work processes 3.1.4 Innovation management/Technological innovations 4.2 Emphasis on Lean 4.2 Reducing waste through batch pilot runs
7.0 Results	N/A	N/A	8.2, 8.4	N/A	2.4.2 Records, audits 4.1.4 Maintaining low turnover rate

2.4.3. Alignment of the Baldrige Framework with the OPM3 best practices framework and ISO 21500

Table 1 demonstrates the alignment of OPM3's Organizational Enabler Best Practices with the Baldrige Categories. The Table shows the extent to which OPM3 supports an organization's ability to develop in each of the Baldrige Categories. As cited in the Center's 2009 Baldrige Feedback Report [23], “The deployment of project management throughout the workforce has elevated this skill nearly to the level of a core competency” (Table 3 Example); building upon strengths, the Center formalized Pharmaceutical Project Management as a core competency in 2014. In addition, the Center completed a cycle of improvement with the Clinical Trial Project Plan tool (Table 3 Example) as a Center-specific method to better manage clinical trials through centralized and accessible documentation of requirements. The Center also rolled out integrated risk and change control policies (Table 3 Example) to improve communication across the team and better manage stakeholder's expectations.

An OPM3 assessment indicated a key improvement area related to evaluating performance of the study design and management team; the Center then developed a competency framework for the Pharmaceutical Project Managers and Project Directors to assure understanding of expectations and provide a clear path for development (Table 3 Example).

Table 2 shows how the OPM3 best practice framework primarily aligns with Baldrige Category 6 (Operations). Additionally, Table 2 displays the alignment of the project domain with ISO 21500, providing an initial baseline to plan and assess improvement activities [24]. By conforming to these standards, the Center clarified, streamlined, and reduced redundancies in its process documentation of the five key work processes (Table 3 Example) shown in Fig. 1.

2.4.4. Interlocking committee infrastructure and Baldrige

Effective communication is essential to daily operations of any organization, but is difficult to accomplish. The Center's Committee Hierarchy (CH) (Fig. 4) provides a structure formalizing subject matter ownership, knowledge sharing and performance improvement throughout every level of the organization. According to Marsh et al. [10], “… interlinked councils (or committees) assure consistency of approach and transfer of lessons learned.” Each key committee maintains responsibility for specific areas and provides representatives to interact with leadership committees.

Fig. 4.

The figure illustrates the committee hierarchy including leadership, reporting structure and Baldrige Category assignments.

Because committee leadership is critical to the success of the CH, committee chairs are rotating positions (every 1–3 years), allowing many employees to gain knowledge and experience at several levels of the committee structure. The efficiency and effectiveness of the CH requires periodic review of committee structures and corresponding charters for continuous improvement. A recent TIPES innovation made each committee responsible for evaluating and maintaining compliance with a Baldrige Category (Fig. 4). This approach helps ensure that the Baldrige process is institutionalized as a routine process and involves a broad spectrum of employees throughout the organization. The interlocking committee structure (Table 3 Example) is a key pillar of the TIPES infrastructure (Fig. 1) and is synergistic with the matrix management structure and the Center's documentation tree system of standard operating procedures (SOPs) and supporting work instructions or approved methods and procedures (AMPs). The CH was listed as a Center strength and best practice in the 2009 Baldrige Feedback Report [23].

3. Results

3.1. Key integration achievements

3.1.1. External recognition and validation

After 18 performance excellence applications and 12 site visits in a 14-year period from Quality New Mexico (Performance Excellence), VA Robert W. Carey Award (Organizational Excellence), and MBNQA (Performance Excellence) the Center achieved the highest award in each of these programs. The Center is the first VA and second Federal Government entity to receive the MBNQA. Having standards in place is useful, but external validation through audits, registrations and certifications are important milestones to measure success and progress. The Center was registered in ISO 9001 in 2003 and received certificates of conformance in ISO 15378 in 2011 and ISO 21500 in 2013. As a Federal governmental entity, the Center has provided a model for government and service-based organizations seeking to achieve quality standards. While there is no one pathway to quality achievement, validation by external bodies aids in promoting quality driven principles throughout the organization.

3.1.2. Organizational Project Management (OPM3) results

Since 2008, the Center has conducted multiple internal OPM3 assessments. The baseline results in terms of capabilities and best practices achieved or gaps revealed have been used to prioritize improvements designed to increase its organizational maturity level and support the Center's performance excellence culture. The outcomes are reflected in the re-organization of the Center's process and procedures structure (Fig. 1) as well as the organizational structure supporting the portfolio domain (Fig. 3). Since 2012, cycles of improvement are reflected in measures such as fewer inadequate supplies in the field and reduced shipping errors (Table 3 Example).

3.1.3. Workforce and customer engagement and productivity

Baldrige [7] defines workforce engagement as, “The extent of workforce members' emotional and intellectual commitment to accomplishing your organization's work, mission, and vision.” Customer engagement is defined as “… customers' investment in or commitment to your brand and product offerings.” Human Sigma [25], the sum of employee engagement and customer engagement, is about managing an organization's complex human systems of the employee-customer encounter. Engaged employees generate greater output with higher quality, with direct cost efficiencies, and generate stronger customer connections, which result in exceptional levels of customer retention, profitability, and growth [26]. Over the last several years, the Center has shown a steady trend of high employee engagement. Employee engagement (Table 3 Example) achieved an average 92nd percentile rank over the last five surveys, based on Gallup's Q¹² U S. Government Workgroup-Level Database. Customer engagement (Table 3 Example) remains high with an overall average score of 4.6 on a 5.0 scale over the last three years. Furthermore, over 80% of the Center's non-VA collaborations is from repeat customers, whom the Center defines as “engaged.”

3.1.4. Holistic strategy

Integrating the quality frameworks provides a solid foundation for Center activities through documentation of processes and periodic performance check-ups with internal and external audits. The holistic framework also supports the Center's ability to seek opportunities for a sustainable future. As the Baldrige Criteria continue to evolve with best practices found in industry, the Center is challenged to also evolve, to be responsive to new approaches, and to innovate. Baldrige's requirement that organizations identify and use their strategic advantages has propelled the Center, as a government entity, to closely examine its unique position and to develop strategies capitalizing on its advantages and opportunities while addressing challenges. This in turn has created opportunities for innovation and intelligent risk taking in its pursuit of strategic objectives. For instance, technological innovations (Table 3 Example) have resulted in improved web-based software systems for the Center including site inventory control, drug distribution, and patient randomization and treatment assignments. Through the Center's innovation management processes and by taking and managing a series of intelligent risks (Table 3 Example), the Center established a biorepository for bio-specimen storage and a method for direct to patient dispensing. These new capabilities expand the Center's offerings and provide the foundation for future growth. Baldrige examiners as well as VA visitors have frequently noted that the Center does not operate like a government bureaucracy, but more closely resembles the private sector in its entrepreneurial approaches.

3.2. Bottom – line outcomes

The success of the Center's quality and entrepreneurial approach is demonstrated through the Center's employee engagement, customer engagement, ability to develop new and renew partnerships and collaborations, increased capability to provide new products and services, and increased capability and capacity to support more ongoing multicenter clinical trials (increased from 26 to 38 over 8 years), all without an appreciable increase in staffing. Employees being the Center's greatest asset believe (four year averages) that Center staff are committed to producing top quality work (91.5%), their colleagues are loyal to their matrix work teams (94.75%), they understand how their jobs help the organization achieve success (97.5%), and they are proud to work for the organization (96.25%.).

4. Discussion

4.1. Lessons learned

4.1.1. Remaining relevant through performance excellence

Every organization must be concerned with remaining “relevant.” This is true in both the private sector with its financial imperatives and the government sector, particularly in a research environment. Research activities and programs are inherently competitive and must be particularly productive to remain viable. In an environment with limited resources and potentially high risk, it requires a solid foundation for assessing and improving all aspects of its operations – a holistic approach to assure the relevancy and quality of products and services.

4.1.2. The approach to relevancy had to be compelling and urgent

In approaching the idea of becoming a “Center of Excellence,” the Center needed to convince management and staff that the journey was necessary to ensure future viability. From the outset, the Center knew that the journey would be demanding and risky, in that it required an intense focus on performance excellence. The Center's mission to serve Veterans created a sense of urgency to succeed, all within the constraints of budget. Employees strongly connect to the Center's mission (Table 3 Example), scoring an average of 4.67 out of 5.0 over the last six years on a mission survey question, “The mission or purpose of my organization makes me feel my job is important.” The evolution of the Center organizational structure (Table 3 Example) enabled it to position the right people in the right positions; those individuals had to be disciplined in their thought, decisions and actions. Having the right people in management also enhanced the motivation and inspiration of the workforce [27].

An important question to be answered was – what does this mean for employees? The answer was apparent: employees felt a tremendous sense of accomplishment in being part of a recognized world-class organization that is workforce and customer-centric. The journey contributed to job satisfaction, engagement and security, and a sense of accomplishment and personal and professional growth. Comments from employees made it clear that their ability to contribute to the well-being and spirit of the organization, to its important mission, and to the accomplishment of a lofty goal through the achievement of Presidential recognition (the MBNQA) imparted a feeling of “importance” and meaning. The employees felt that they played a role in national history, thereby reaching the pinnacle of self-actualization on Maslow's Hierarchy of Needs.

4.1.3. Leadership is critical at all levels to transform a typical government culture

Leadership is critical at all levels of the organization to transform a typical government culture into one in which employees are highly engaged, high performing, and forward thinking. The hierarchy of leadership at the Center includes four tiers made up of varying levels of supervision and responsibility within the organization: senior, executive, management, and staff leadership. These groups provided the leadership necessary to move the organization forward in an effective, efficient and productive manner that exceeded expectations. To inspire others within the organization, leaders at these levels needed to be passionate about the mission, vision, and values assimilated with the organization's ethical and behavioral expectations, comprising the heart of a culture, that epitomizes “Riding for the Brand” (Fig. 5). Employees know what is expected of them at work (Table 3 Example) as shown on a survey question, scoring an average of 4.6 out of 5.0 over the last six surveys. Further, leaders harmoniously interacted with all employees (Table 3 Example) by fully integrating them into the committee hierarchy, strategic planning, process development, and improvement activities. A strength finding from the 2009 Baldrige Feedback Report [23] states, “Leaders have created a compelling vision through shared stories, and they reinforce that vision by serving as role models for core values and ethics.”

Fig. 5.

The figure illustrates the heart of organizational culture that is constructed around the passion and commitment for the organization's Mission, Vision, Values and Ethical Expectations. The figure also shows the importance of strategic challenges, advantages, opportunities and objectives in realizing an organizational vision and the significance of innovation and intelligent risk taking in that process.

4.1.4. Transformation

Changing the Center's culture included developing a devoted passion not only for the mission, vision, and values, but for the QS frameworks and systems that needed to be implemented and assimilated into daily operations. The charge went beyond revision, remodeling or reconstruction, instead required a refocusing on the culture in addition to overhauling the operating infrastructure. In government, one could consider this a radical or revolutionary change. The transformation involved a focus on many elements including (1) the culture of engagement (employee and customer), (2) capability and capacity, (3) efficiency, effectiveness and productivity, (4) identification of organizational strategic advantages while taking intelligent risks to address strategic challenges and to capitalize on strategic opportunities, (5) a vigorous, honest, and fact-based dialogue, (6) continuous change and improvement, (7) optimism and faith that the transformation was a “true north” direction, and above all (8) valuing employees and their contributions (Table 3 Example), which is essential for inspiring commitment.

Transformation is always difficult, particularly when done on top of existing workload. The Center provided the environment, tools, and learning opportunities (Table 3 Example) that allowed for transformation. All the above systems (Baldrige, ISO, OPM3) facilitated employee understanding of the Center's work processes, leading to greater levels of performance excellence. Successful transformation requires both understanding (getting it) and practice (doing it). Ken Miller [8] states, “The constant interplay between getting it and doing it is what leads to mastery.” He believes that if we get it, we will have the discipline to do it and live it, and if you are living it you can't imagine going back to the old way.

Transformation efforts, if properly communicated and understood, are a form of community service to the organization to make it better. Part of the communication strategy involves helping everyone understand that successful transformation is a way of keeping the organization relevant and making it a better place to work, thereby securing employees' livelihood and increasing their engagement. Furthermore, voluntary employee turnover remains low (Average < 4%) (Table 3 Example), outperforming the federal government and other benchmarks. Over the last four years, this has been reflected in employee responses to a survey question asking if they are proud to work for the Center (Average 96.25%).

4.2. Future focus

The Center operates within a larger research program context that exists within the largest national integrated healthcare system in the U.S. Therefore, opportunities and challenges will continue to arise for further development of the quality efforts. Some are more research-specific, while others are in context of organizational goals more broadly focused on earning the public's trust and confidence. To these ends, future activities will emphasize the foundational principles of continued improvement and a holistic approach to management under the TIPES framework.

ISO 9001:2015 – The Center will continue ISO certification under the newly revised ISO 9001:2015 standard, which will create even greater synergy between Baldrige, ISO and OPM3 Frameworks. One of the key changes to the ISO 9001:2015 standard brings it into closer alignment with the Baldrige emphasis on results, which supports the Center's continuous development and use of leading indicators.

Program and Center Emphasis on Lean – Lean (Table 3 Example) is a set of improvement and alignment tools to augment the Center's quality frameworks for operational excellence. Perhaps more importantly, the most successful Lean approaches are founded on a culture of leadership commitment, communication, employee engagement and empowerment and teamwork [28]. As the Center's parent organization (CSP) seeks to increase efficiency and value in all aspects of its operations, the Center has developed a strategic objective to reinforce its Lean culture foundation. The objective includes training all employees in Lean concepts and methods, using actual Center projects to teach Lean principles while incorporating the system's applications in our environment.

As discussed by Trewn et al. [29], “Lean is a never ending, systematic approach for identifying and eliminating waste and improving flow of a process while engaging employees.” Moreover, Lean tools support the organization of the workforce to deliver more value to customers [28,29] in a way that Baldrige and ISO do not specifically address. Lean does, however, support and enhance the continuous improvement quality concepts and the workforce focus inherent in the ISO 9001 and Baldrige frameworks. Using a structured problem-solving methodology of Define-Measure-Analyze-Improve-Control (DMAIC) along with cultural enablers that focus on the participation and empowerment of employees at all levels, Lean's approach complements the Center's culture and is becoming an integral part of the Center's QH's quality systems. One early example is the implementation of batch pilot runs (Table 3 Example), significantly reducing waste, in the Center's manufacturing process.

4.2.1. Further development of portfolio and program domains using OPM3

The Center continues to use OPM3 as an internal assessment tool to further analyze the relationships within Portfolio, Program and Project domains and to take advantage of improvements identified in this assessment process. Additionally, the Center will continue to re-assess domains periodically to identify opportunities for improvement. This supports the Center's holistic approach that uses multiple standards as components of its innovation engine and performance excellence pursuit.

5. Conclusion

Since the inception of the Center's quality journey, a question was often posed to its leadership: “What do you want me to do, this quality stuff or my job?” The answer was always, “We want you to do both, but we want you to change the way you do your job.” This recognition is the ultimate transformation at the employee level that enabled the Center to make progress and excel. As the Center's quality efforts gained momentum, employees became increasingly self-regulated, as evidenced by high employee engagement scores.

It is our sincere hope that this paper will inspire other local, state, and Federal government entities, no matter how small or large, to pursue a performance excellence journey for the sake of customers, employees, stakeholders and the country. Government organizations can start and progress along the “road to excellence” despite the many challenges inherent in government. Leadership must have the will and fortitude to begin and stay the course.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the development, authorship, and/or publication of this manuscript.

Funding

Development of this manuscript was supported in part by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Cooperative Studies Program using resources and facilities at the VA Cooperative Studies Program Clinical Research Pharmacy Coordinating Center, Raymond G. Murphy VA Medical Center, and the Biomedical Research Institute of New Mexico. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.