Building Psychiatric Quality Programs and Defining Quality Leadership Roles at Four Academic Medical Centers

Quality of care has become an important concept in healthcare, following the Institute of Medicine report “To Err is Human,” which suggested that there were as many as 98,000 medical errors per year in 1999 [1]. According to the US Health and Human Services Agency, Quality improvement (QI) “consists of systematic and continuous actions that lead to measurable improvement in health care services and the health status of targeted patient groups” [2].

Academic departments have a multifaceted mission to educate future healthcare leaders, provide high-quality care and value for patients, and advance research and scholarship innovation [3]. Many academic medical centers, under which clinical departments function, are tasked with addressing growing demands of value (quality and safety and experience divided by cost), as well as with supporting effective QI activities [4]. However, several barriers have been described in implementing quality programs within academic departments, including lack of departmental support, limited engagement by faculty and trainees, and limited expertise in QI [4]. Prior publications have described faculty development in QI and patient safety expertise and have documented the successful development of quality programs in other specialties such as internal medicine [5, 6].

In psychiatry, QI projects are well described in the literature, including innovative projects to reduce antipsychotic polypharmacy, integrate behavioral health and primary care, implement interdisciplinary care rounding, and structure handoffs to improve communication [7–9]. The development of QI programs using a maturity matrix in psychiatry has also been previously described [10]. Within clinical training, QI and patient safety are explicitly described in the practice-based learning and improvement and system-based based practice core competencies and are now requirements as part of the Accreditation Council on Graduate Medical Education Milestone assessments [11]. Despite the need for QI programs within academic settings, descriptions of psychiatric QI programs in academic medical centers are not well characterized in the literature. In this column, the authors aim to describe four academic psychiatry quality programs; highlight the roles, responsibilities, and critical skills of QI leaders; and share project examples.

The Four Programs

The four psychiatric quality programs described in this column span the West and Northeast coastal regions and are each affiliated with Schools of Medicine. The programs include the Yale, Oregon Health & Science University (OHSU), University of California San Francisco (UCSF), and University of California Los Angeles (UCLA). The four centers feature both public and private universities and health centers. A variety of psychiatric services are represented, including inpatient, outpatient, intensive outpatient, and partial hospitalization services. With regard to the direct clinical roles of quality leaders themselves, two work primarily in outpatient clinical roles, one in an inpatient clinical role, and one works clinically in both inpatient and outpatient consultation roles. Descriptions of the QI physician leadership roles and programs are provided in Tables 1 and 2, respectively.

Table 1.

Psychiatry quality program leaders roles, job funding, organizational quality structure, and scope of responsibilities

Variables	Oregon Health & Science University (OHSU)	University of California, Los Angeles (UCLA)	University of California, San Francisco (UCSF)	Yale
QI title	Quality Medical Director	Department of Psychiatry, Chief Quality Officer	Executive Director for Informatics & Quality	Associate Medical Director of Quality Improvement
Educational training in QI	Localized, institutional training	Localized, institutional training	2-year localized, intensive training during residency	Chief Resident year in QI, External program (Intermountain Advanced Training Program)
Years in psychiatry-specific quality role	4	5	2	2
Is job description standardized in the academic medical center?	Yes	Yes	No, in development	Yes
Time and funding	Partial FTE	Stipend only	Partial FTE and stipend	Partial FTE and stipend
Funding mechanism	Health system	Health system	Health system	Psychiatry hospital
Clinical role	Outpatient	Inpatient	Outpatient	Inpatient and outpatient (Consultation in a Specialty Clinic)
Organizational quality structure	Reports to Department Chair and Chief Medical Officer. Departmental QI infrastructure reviewed annually. Departmental QI plans submitted to organizational QI oversight committee for review and approval annually. Leadership (Chair, administration, nursing) are all supportive and engaged in quality efforts Support via organizational QI specialists and departmental administrative staff. No designated analyst. Administrative support available on as needed basis	Direct report to Department Chair and Hospital CEO, related report to UCLA Health Chief Quality officer. Provides annual presentation to health system leadership in quality. Health System establishes overall structure of QI goals, then adapted by every department, including psychiatry No data analyst provided. Performance improvement specialists available at health system level.	Reports to Psychiatric Hospital President (who is also the UCSF Dept. of Psychiatry Chair) and has dotted line reporting structure with Chief Quality Officer of the university. Chairs the psychiatric hospital’s Quality Council and reports out to Medical Staff Executive Committee. Serves as Lean coach alongside Executive Director for Operations for all improvement efforts. Also reports to Chief Medical Informatics Officer QI Specialist assists with lean/QI efforts and performs data analysis. Senior Quality Analyst focus on regulatory compliance	Reports to the Medical Director who also functions as Vice Chief of Psychiatry. Co-chairs quality/safety council with dyadic clinical outcomes nurse leader. Leadership (Vice president of service line, nursing lead, and medical director) are all supportive in quality efforts. Represents psychiatry in health system quality/safety meetings, including System Quality Council and Quality Vice Chairs/Medical Director meetings Data analysis provided on an ad-hoc basis for psychiatric QI projects. Shared administrative assistance provided through psychiatric hospital
Scope
Quality improvement	Lead	Lead	Lead	Lead
Quality assurance/regulatory compliance	Support	Support	Support	Co-lead a
QI education	Resident/fellow education and mentorship. Ad-hoc faculty and staff QI training	Resident/fellow mentorship and supervision in hospital QI projects. Faculty and staff development	Resident/fellow education and mentorship. Medical Student mentorship. Ad-hoc staff/faculty training	Trainings for staff/faculty; collaborate with designated education lead for residency on QI for residents. Resident mentorship
Informatics	General informaticist available through institution	Dedicated departmental physician informaticist available	Lead as part of role	Psychiatric informaticist available through institution
Analytics	Available through institution	Available through institution	Dedicated Analyst	Available through institution

^aCo-lead refers to working collaboratively with health system-employed quality assurance professionals who centrally report behavioral health and other quality measures directly to payers and regulators

Table 2.

Description psychiatric quality program structure and project examples

Variables	Oregon Health & Science University (OHSU)	University of California, Los Angeles (UCLA)	University of California, San Francisco (UCSF)	Yale
Description of clinical operations	No inpatient psychiatric beds/partial hospitalization/intensive outpatient; Department Of Psychiatry outpatient encounter volume in 2019: 34,735—all ages	Inpatient psychiatric hospital: 74 beds—all ages; PHP/IOPa encounter volume in 2019: 7182—all ages; Outpatient encounter volume in 2019: 61,077—all ages	Inpatient psychiatric hospital: 22 beds—adult only; PHP/IOP in 2019 encounter volume: 4149—adult, Outpatient encounter volume in 2019: 26,342—all ages	Inpatient psychiatric hospital: 118 beds—adolescent, adult, and geriatric; IOP and Outpatient encounter volume in 2018: 31,127
Facility funding	Publicly funded	Publicly funded	Publicly funded	Private non-profit
Regional location	Pacific Northwest	West Coast	West Coast	East Coast
Primary QI methodology	Lean/Institutional Performance Excellence	Lean	Lean	Institute for Healthcare Improvement
Key project examples
Current efforts	1) Implementation of measurement-based care via measurement feedback system in outpatient clinics, 2) Revision of Suicide Risk Screening and Safety Interventions Policy in the non-psychiatric setting, 3) Development of organizational wide educational modules on suicide prevention and treatment. 4) Implementation of technology to improve Morbidity & Mortality Conferences and Peer Review process	1) Reduce Avoidable emergency department visits, 2) Establish Patient/Family Advisory Council 3) Reduce excess length of stay for inpatient psychiatry, 4) Post-discharge suicide crisis follow up, 5) Universal suicide risk screening for med/surgical hospital and emergency departments, 6) Standardized screening for agitation in emergency department, 7) Optimization of restraint and seclusion electronic orders and documentation	1) Outpatient metabolic monitoring for patients on atypical antipsychotics. 2) Enhanced suicide screening and assessment. 3) Enhanced measurement-based care with automation. 4) Improving depression treatment outcomes in PHP/IOP. 5) Productivity capture/Coding accuracy improvement effort	1) Administrative leadership rounds on high length of stay patient and patients with difficult disposition, 2) Utilization management presence on daily rounds, 3) Implement a "huddle board" to visually display metrics of length of stay, reported patient safety events, patient experience, and other provider metrics, 4) Admissions redesign to improve efficiency of care, 5) electronic suicide safety planning pilot project
Upcoming efforts/priorities	1) Implementation of Psychiatry E-consult through EHR, 2) Develop QI dashboards, metrics, benchmarks, 3) Enhanced capture of patient experience data through updated electronic surveys, 4) Capture treatment outcomes with services delivered via telehealth	1) Improve inpatient psychiatry patient satisfaction, 2) Develop physician-level QI data and report, 3) Enhance psychiatry suicide risk assessment documentation template	1) Digital In-Basket management safety improvement, 2) Development of Patient Advisory Counsel for enhanced patient experience, 3) Reduced unnecessary readmissions, 4) Improved documentation timeliness, 5) Improved treatment planning workflows and compliance	1) Implement optimization efforts patient care plan, 2) Strengthen service line education and communication on quality/safety (monthly seminar series, newsletter), 3) Improve documentation of diagnoses and discharge dates in order to improve workflows, 4) Enhance and measure suicide risk assessment documentation in compliance with the Joint Commission

^aPHP/IOP refers to partial hospitalization program and intensive outpatient program

Notably, the four programs were developed or reformulated recently, with the earliest program starting more than 10 years ago, re-started in 2015 with new leadership, and the most recent program started in 2018. The program priorities aligned on QI efforts related to federal regulatory changes. Over the last year, these efforts included enhanced suicide risk screening and assessment, as part of the Joint Commission’s National Patient Safety Goals on suicide prevention in 2019 [12], as well as the Centers for Medicare and Medicaid Services’ Inpatient Psychiatry Quality Reporting Program [13]. Although the four programs have worked on distinct clinical quality projects, all programs incorporated QI projects that demonstrate improvement of care, such as measurement-based care in the outpatient setting, length of stay reductions in the inpatient setting, and metrics of patient experience. Table 2 provides examples of specific improvement efforts by the four programs.

Descriptions of Psychiatric Quality Leadership Roles

Each program is led by a psychiatrist, with variable titles including “Associate Medical Director,” “Physician Lead,” “Executive Director”, “Quality Medical Director,” and “Chief Quality Officer.” The leadership role for each program is formalized and designated by the Department of Psychiatry and/or psychiatric hospital with oversight of the scope of quality work. Examples of each psychiatric quality lead’s scope and funding are listed in Table 1. Consistent among the core roles of each QI leader is the responsibility to teach and disseminate QI education throughout the psychiatry department. Although institutions differed on the specific quality model and theory (Lean for three institutions and Institute for Healthcare Improvement for one institution), each program emphasizes the need for the QI leader to teach residents and to promote QI-related faculty development, including with affiliated institutions with psychiatric quality leaders such as the Veterans Administration). Generally, the QI leader is also responsible for dissemination of QI efforts through committee participation and direct reporting relationships to departmental, hospital, and institutional quality leaders. At one institution, the QI leader also co-leads quality assurance. While significant overlap may exist between QI and quality assurance, distinguishing features of quality assurance include a specific focus on regulatory compliance, mandated federal and state regulations, and oversight of clinician and staff performance.

Five Key Skills for Academic Medical Center Leaders Developing Specialized Expertise in Psychiatric Quality

Although developed independently, each program includes individuals and resources that address five common skills related to quality expertise. These skills include [1] education and specialized training in QI, [2] interdisciplinary team collaboration, [3] QI project cycles and project management, [4] metric management, and [5] data acquisition and information technology (IT) competence. Although these themes are not exhaustive, they provide a common foundation to help guide skills development for individuals leading psychiatric quality programs. Examples of other key activities (which are beyond the scope of this paper) include collaboration on QI initiatives in other clinical departments, patient/family stakeholder groups, and other institutions.

Education

First, the psychiatric quality leaders have benefitted from specialized training in QI theory and methods, whether internal or external to their institution. Some institutions have quality course or fellowship programs through the institution or in collaboration with the Veterans Administration [14]. External trainings commonly used in healthcare include the Institute for Health Improvement Open School and Lean training programs. Intermountain Healthcare Advanced Training Program and Mini-Advanced Training Program are external quality training options [15]. Education in and understanding of quality tools and processes is critical for quality leaders and teams to communicate using a shared language, focus on targeted initiatives, identify metrics, and carry projects through successful cycles. In addition, understanding tools to graphically display QI efforts, such as run charts and control charts, is necessary in order to accurately document and track progress.

After learning quality science, QI leaders are expected to educate other faculty, staff, and trainees, while also disseminating this knowledge through academic work. Several authors have written about QI and education to date [16, 17]. Through this active engagement, psychiatric QI leaders may build capacity and collaborate with other leaders and stakeholders to further the development of future quality infrastructure and leadership in psychiatry. While formal QI education for trainees is not necessarily a primary responsibility of the QI leader, there are a range of roles related to trainees, including invitations to participate in QI projects and interfacing with residency leaders on these activities. For example, at one institution, the QI lead mentored a chief resident in a project evaluating factors that contribute to increased efficiency or worsened stress and burnout in a busy emergency psychiatry service [11].

Interdisciplinary Collaboration

Another skill common to the QI leaders of the four institutions is the ability to facilitate interdisciplinary collaboration. For example, the psychiatry quality leader from each of the four institutions has a cross-cutting role to represent psychiatry in multispecialty meetings and interface with the quality personnel of other departments. Deeper relationships are also established at two institutions, where a dyadic relationship between a nursing clinical outcomes leader and the psychiatric quality leader promotes collaboration with nursing. Similarly, interdisciplinary team projects are common. This requires the need to maintain strong relationships among the different disciplines of psychiatry, nursing, advanced practice professionals, social workers, occupational therapy, and milieu counselors, among others. In many circumstances, interdisciplinary teams are formed to work on QI efforts and patient safety. Collaborative relationships across medical and surgical specialties are also important. At one institution, the quality leader served on the steering committee of the medical-surgical hospital committee that developed and implemented universal screening for suicide risk and developed workflows for more efficient psychiatric consultation in the emergency department [18]. Additional functions of the quality leaders include facilitating morbidity and mortality conferences and identifying root causes to suboptimal outcomes such as serious safety events, death by suicide soon after discharge, unwarranted falls, infection outbreaks, and poor transitions of care. These efforts have led to multispecialty and interdisciplinary quality conferences and care reviews at several institutions.

The QI leader may also help to enhance relationships with broader system resources, such as with clinical informatics, data infrastructure, and programming resources that enable and facilitate QI projects. The success of QI projects often depends on collaboration with other specialties, departments, and programs, as well as health services researchers. Skilled psychiatry quality leaders are able to facilitate these collaborations within institutions and in the larger community health system. One example of collaboration between academic- and community-based centers is demonstrated by an effort to expand education and resources for the management of depression in community settings [19].

Project Cycles and Project Management

Quality leaders are also typically tasked with managing complex QI projects. A project management skillset is critical to improvement processes. Therefore, quality leaders should actively learn specialized management skills that are experiential in nature and beyond the scope of a traditional QI didactic curriculum. These skills may be obtained through direct mentorship or through specific training programs (as previously outlined). Skills necessary to manage complex QI projects include the creation of a team, identification of leadership, project planning, goal setting, assessment of current state, knowledge about organizational structure, and facilitation skills with a focus on learning. Multiple Plan-Do-Study-Act cycles are often required to sustain QI efforts and make improvement work both efficient and meaningful [20]. Ideally, quality work is integrated into the daily tasks and modeled by quality leaders. Finally, access to quality experts, including quality specialists, is essential to assist quality leaders in navigating the complex factors and variable factors affecting quality efforts.

Metric Management

With healthcare moving towards value-based care, data have become another valuable tool to drive change. Harnessing data to transparently offer information on provider practice patterns and patient outcomes can be useful in identifying trends and ultimately changing behavior. Quality leaders are often asked to identify the metrics needed to support, audit, and provide feedback on a given QI project. Tracking metrics over time allows for analysis to determine the impact of QI efforts on the desired outcome. Identifying goals and key metrics can be challenging. A variety of measures are available including structural, process, balancing, and outcome metrics, the latter being the most clinically valuable, yet the most difficult to obtain.

Once identified, metrics can be managed through tools such as online dashboards, scorecards, and visualization boards. For example, one institution uses online dashboards that can trend measures of patient restraints, oral and IM medications, length of stay, and patient experience. This same institution uses visual boards known as “huddle boards” during rounds to discuss safety events that have occurred and provider-specific metrics. Another institution uses the “True North Scorecard” as an additional management tool. The scorecard is a double-sided document containing key quality data highlighting the areas of patient experience, quality and safety, our people (staff and faculty experience), strategic growth, and financial strength. Each sub-item has measurements of baseline, year to date, most recent value, trend lines, benchmarks (when available), and goals, as well as a primary owner who is held responsible for meeting the goal each fiscal year.

Data Acquisition and IT Competence

Beyond education and identifying the appropriate metrics for a given project, another skill needed for quality leadership is the ability to acquire the data necessary for projects. This makes data acquisition and IT competence crucial to a quality leader’s skill set. Data are typically stored in the electronic medical record or another repository, such as a clinical dashboard or as a part of a metric-based report. The institutions described in this column use the Epic electronic medical record as a primary data source for data collection and extraction. Quality leaders must possess the skills necessary to extract the data in an organized and analyzable manner or have access to a competent data analyst to assist with these tasks. This type of work may require the use of various software programs to validate data. For example, data may not have been consistently or accurately collected within the medical record. The existence of a discrete data field does not guarantee that the data has been accurately entered even if collected. In these situations, the QI leader can work with other IT team members to determine the best approach to locating the desired information. This will often result in a project to improve clinician compliance with data entry and to ensure the consistency of data for future efforts. Another example includes partnerships with established health services research teams in psychiatry within an institution to inform modeling and quality program design.

Challenges and Future Directions

Challenges encountered by QI leaders include workforce development, academic promotion pathways for quality leaders in psychiatry, and a need for collaboration and networking opportunities. In workforce development, one important challenge is the need for a QI training pipeline. For example, none of the four academic institutions currently have a psychiatric fellowship program for specialized training in quality science, although there are post-residency programs in health services research that include a focus on QI methods (e.g., National Clinician Scholars Program). Within residency training, the ACGME requires QI and patient safety to be part of system-based practice and practice-based learning and improvement core competencies. However, only a few programs have described QI educational curricula [21–23]. The curricula are primarily didactic based, with less emphasis on project-based experiential learning. The skills needed for QI leadership require experiential learning and navigation of complex systems and reporting structures. In the future, academic institutions will need to determine how best to accelerate psychiatric training to produce a robust and capable QI workforce.

Further, quality work and leadership are not readily calculated into traditional academic promotion criteria, which commonly emphasize scholarship such as publications, committee participation, and presentations. Examples of QI projects and frameworks led by academic QI leaders that have led to publication include projects to develop a QI maturity matrix, to improve workflows and reduce of physician burnout in emergency psychiatry consultation service, to create an academic psychiatry QI curriculum, and to implement measurement-based care in an academic setting [10, 17, 24, 25]. Although QI can be meaningfully incorporated into academic work products, future directions should clarify the standards for QI-related academic work and the development of mentorship programs with QI-knowledgeable senior faculty, as well as potential community collaborations for broader models of care and QI. For this reason, some of the programs have an ongoing affiliation with a psychiatric health services research center that helps facilitate publications and projects such as pilot QI trials or collection of qualitative data. Until psychiatric quality programs are more robust within individual institutions, a mechanism to systematically identify extramural mentors and complementary methods resources may be useful to assist departments in the development of psychiatric QI leaders. In addition, evaluating the effectiveness and cost-benefits of psychiatry quality programs will be an important area of future research.

Finally, we advocate for the development of a consortium of clinical psychiatry QI leaders, who both practice clinically and work on the frontlines of QI leadership. This type of collaboration would seek to expand beyond the offerings by trade organizations and national regulatory agencies, which function more to guide quality policy and metric development rather than the practice of implementation and dissemination of quality work in psychiatry with knowledge of the clinical issues in the context of the systems being evaluated. A national effort is needed to both define and implement QI changes in psychiatric practice as well as support expansion of the psychiatric QI program infrastructure that may have a broader impact on health systems and public health through consistent collaboration.

In conclusion, quality programs within academic psychiatry departments are critical investments in light of regulatory and healthcare system changes aimed to enhance patient safety and deliver high-value care to patients (and even respond to and evaluate system responses to events such as the COVID-19 pandemic). This column summarizes perspectives from four existing quality programs within psychiatry departments at academic medical centers. It identifies the common roles, skills, and specialized training required for developing psychiatric QI leaders and quality programs and opportunities and key limitations, for example, promotion support for QI in our field. Key changes needed include resources for QI, linkages to broader data and analysis support infrastructures, broadening education in and uses of QI findings in care, and collaboration and sharing of perspectives across institutions. The key skills and program information provided in this column can help inform other academic departments with interests in developing a QI program and building capacity to address patient safety and quality in psychiatry.

Compliance with Ethical Standards

Disclosures

Dr. Black has a significant financial interest in Owl Insights, a company that may have a commercial interest in the results of this research and technology. This potential conflict of interest has been reviewed and managed by the Oregon Health & Science University. On behalf of all the other authors, the corresponding author states that there are no conflicts of interest.