The Academic Community Early Psychosis Intervention Network: Toward building a novel learning health system across six US states

Abstract

Introduction

Compared to usual care, specialty services for first‐episode psychosis (FES) have superior patient outcomes. The Early Psychosis Intervention Network (EPINET), comprised of eight U.S. regional clinical networks, aims to advance the quality of FES care within the ethos of learning healthcare systems (LHS). Among these, the Academic Community (AC) EPINET was established to provide FES care, collect common data elements, leverage informatics, foster a culture of continuous learning and quality improvement, and engage in practice‐based research.

Methods

We designed and implemented a novel LHS of university‐affiliated FES programs within a hub (academic leadership team) and spoke (FES clinics) model. A series of site implementation meetings engaged stakeholders, setting the stage for a culture that values data collection and shared learning. We built clinical workflows to collect common data elements at enrollment and at consecutive 6‐month intervals in parallel to an informatics workflow to deliver outcome visualizations and drive quality improvement efforts.

Results

All six clinical sites successfully implemented data capture workflows and engaged in the process of designing the informatics platform. Upon developing the structure, processes, and initial culture of the LHS, a total of 614 patients enrolled in AC‐EPINET, with the most common primary diagnoses of schizophrenia (32.1%) and unspecified psychotic disorders (23.6%). Visualized outcomes were delivered to clinical teams who began to consider locally relevant quality improvement projects.

Conclusions

AC‐EPINET is a novel LHS, with a simultaneous focus on science, informatics, incentives, and culture. The work of developing AC‐EPINET thus far has highlighted the need for future LHS’ to be mindful of the complexities of data security issues, develop more automated informatic workflows, resource quality assurance efforts, and attend to building the cultural infrastructure with the input of all stakeholders.

1. BACKGROUND

Primary psychotic disorders generally develop in adolescence or early adulthood and often deleteriously impact the personal, social, and occupational growth of individuals who develop such illnesses. ¹ , ² , ³ Delayed or prolonged time between psychosis onset and engagement in treatment (Duration of Untreated Psychosis, DUP) results in poorer clinical, psychosocial, and functional outcomes. ⁴ , ⁵ As such, there has been robust movement toward early intervention and treatment beyond what can be achieved within traditional mental health care services.

Specialized teams that integrate evidence‐based treatments for delivery to younger patients and their families have demonstrated improvements over usual services in studies examining a variety of models, across international settings. ⁶ , ⁷ In the US, two randomized controlled trials published in 2015 reported reductions in psychiatric hospitalization ⁷ and improvement in vocational engagement ⁸ and quality of life. ⁸ , ⁹ These trials represented community care settings and reported positive health economic evaluations. ⁵ , ¹⁰ The model of care, subsequently termed coordinated specialty care (CSC), which emphasizes an intensive recovery‐oriented approach to promoting early detection and intervention through a multidisciplinary approach that includes vital services such as case management, psychotherapy, supported employment and education services, family education and support, and medication management ¹¹ , ¹² was supported by a third implementation study within a broader NIMH initiative ¹³ that produced manuals and training programs. Since 2016, the Substance Abuse and Mental Health Services Administration has required states to “set‐aside” at least 10% of their Mental Health Block Grant funding for such services. This combination of scientific evidence and federal funding has led to the growth of CSCs across the US, with now more than 100 clinics. ¹⁴ , ¹⁵

The unusually rapid proliferation of CSCs has been recognized as an important response to a longstanding knowledge‐to‐practice gap. ¹¹ However, this highlighted two challenges. First, ensuring adequate standards of care across new, and often innovative, adaptations of this model. Second, a need for research to understand the impact of CSC on the lives of patients and caregivers in real‐world clinical settings. ¹² , ¹⁷ , ¹⁸ This culminated in the recognition that CSC, while better than usual care, did not fully address all the needs of individuals with first‐episode psychosis (FES) and required further research and refinement. ¹⁶ After consultation with stakeholders, the Division of Services and Interventions Research at the NIMH initiated the Early Psychosis Intervention Network (EPINET) in 2019 ¹⁹ to advance a framework based on a Learning Health Systems approach. ²⁰ This approach seeks to integrate best‐practice care, quality improvement, and discovery‐oriented research. It thus has salience for psychosis spectrum disorder treatment, wherein important gaps in both implementation (“doing what we know”) and knowledge (“learning from what to do”) are evident in current systems of care. ²¹ A learning health system occurs when “science, informatics, incentives, and culture are aligned for continuous improvement and innovation…and new knowledge is captured as an integral by‐product of the care experience.” ²² A key characteristic of such systems is deliberate and simultaneous attention to four interrelated but distinct components or activities that include:

1. Science—both application of knowledge derived from prior research to practice, and articulating questions that arise from practice to drive research;

2. Informatics—a robust and user‐friendly workflow that facilitates the collection, secure storage, and analysis of data that serves the data analytic needs of both clinicians and researchers, and feedback progress and outcomes to stakeholders;

3. Incentives—or facilitation of both external (e.g., accountability to funders, stakeholders) and internal (clinician desire for autonomy, feedback and mastery; patient's data used to promote better care processes for themselves and others) motivators of excellent healthcare provision and knowledge generation

4. Culture—that values measurement to support non‐punitive ethos of performance improvement, research and transparent accountability to all stakeholders.

Supported by a socio‐technical infrastructure consisting of people, processes, technology, and policy, the learning healthcare systems (LHS) has the potential to rapidly produce improvements in care processes and health outcomes by enabling health systems to coordinate stakeholders, embed improvement work into clinical workflows, and manage the proliferation of relevant clinical research. ²⁰ , ²¹ , ²² , ²³ , ²⁴ This work is accomplished through the development of a multi‐stakeholder learning community that brings together the relevant roles for generating scientific and improvement ideas, tailoring health system infrastructure, and conducting the work of the LHS (e.g., leadership, governance, and sustainability). ²⁵ , ²⁶ LHSs have been successfully utilized in other chronic illnesses, ²⁷ , ²⁸ , ²⁹ , ³⁰ but outside of prototype implementations in schizophrenia, ¹⁹ they have remained largely aspirational for mental illnesses. ³¹ , ³² , ³³ The focus of this article is the Academic Community EPINET (AC‐EPINET), which as an aspiring LHS has progressed toward implementation of the core components of learning health systems. Development and implementation of AC‐EPINET infrastructure, processes and efforts toward a continuous learning and improvement culture are detailed from initiation in April 2021 through December 2023.

2. METHODS

2.1. Context

EPINET includes eight regional hub‐and‐spoke networks, consisting of over 100 clinics with over 5000 patients all using a patient‐level core assessment battery (CAB) focused on the care of early course psychosis across the US. The CAB is a consensus battery that consists of standardized measures and items that assess key domains relevant to early psychosis psychopathology and intervention (https://nationalepinet.org/core-assessment-battery-cab/). It was developed by the EPINET Steering Committee composed of principal investigators from each of the regional hubs and the National Data Coordinating Center. ³⁴ The AC‐EPINET, which joined in 2020, consists of six university‐affiliated FES that follow the CSC care model. Unlike earlier established EPINET hubs, ³⁵ , ³⁶ AC‐EPINET spanned across six separate states in the East, Midwest, and Southern United States. AC‐EPINET was informed by the principles of the Institute of Medicine's LHS model, which envisions full implementation of the four core components of learning health systems: science, informatics, incentives, and culture. ³⁴ , ³⁷ , ³⁸ To adequately describe the design and implementation of AC‐EPINET, we focus on the structure, processes, and culture of building the LHS infrastructure.

2.2. LHS structure

Development and coordination of this LHS was a non‐trivial undertaking, requiring significant effort from all stakeholders. ³⁴ , ³⁵ , ³⁶ , ³⁷ , ³⁸ Pragmatically, this involved establishing a new set of best practices and clinical workflow protocols, which were flexible and acceptable to a diverse set of stakeholders, including clinicians, administrators, investigators, and institutional data privacy officers. AC‐EPINET included a “hub” (leadership and clinical spoke at Indiana University and informatics team at Yale University) and five additional clinical “spokes” (Ohio State University, Tulane University, University of Michigan, University of Rochester, and Vanderbilt University). See Figure 1. Sites were selected based on their affiliation with an academic institution and early psychosis programs that shared the following characteristics: (1) established early intervention services informed by the CSC model; (2) community‐based, “real‐world” early psychosis clinics that serve a range of underserved populations, including urban poor and rural patients; (3) deep expertise in assessment and data collection; and (4) research experience.

FIGURE 1.

Academic Community‐Early Psychosis Intervention Network is geographically diverse and organized around coordinated specialty care clinics that are affiliated with academic institutions. Pink circles represent clinical sites, while stars represent the hub (overlapping five‐point administrative, and four‐point informatics, components).

Four core groups were formed to build the AC‐EPINET LHS, including an executive committee, assessment and implementation core, informatics core, and data management and analytics core. The executive committee was comprised of site PIs and relevant stakeholders and was responsible for alignment across clinical and scientific goals, with careful consideration for all network stakeholders' needs. The executive committee guided AC‐EPINET toward the goal of facilitating meaningful innovations in clinical assessment, data sharing, and visualization. The “assessment and implementation” core was comprised of both hub and site‐level leadership. This core's primary responsibilities included the work of operationalizing data collection within the real‐world clinical workflows of each clinic. The informatics core was led by members from Yale University's Program for Specialized Treatment Early in Psychosis (STEP), the developers of Health Outcomes, Network and Education (HONE), an informatics platform designed to integrate with the workflow of an LHS and facilitate real‐time visualization of preidentified prioritized clinical outcomes. ³⁹ , ⁴⁰ , ⁴¹ , ⁴² The data management and analytics core were established to ensure data quality and manage and analyze LHS data, including preparing data for dissemination.

All data collection procedures were reviewed by a centralized Institutional Review Board at the hub (Indiana University) and determined to be exempt from further review after classification as Quality Improvement. The IRBs of other AC‐EPINET institutions also reviewed and agreed with the conclusion of the central IRB. All data were collected within the standard clinical care provided by AC‐EPINET CSC programs. All LHS patients were informed of the purpose of the LHS, and that the data collected would be used to improve clinical outcomes and promote understanding of recovery in early psychosis.

Patients were invited into AC‐EPINET if they were: (1) Engaged in care at one of the six clinical sites; (2) between 14 and 35 years of age; (3) had psychosis onset within 5 years; (4) had a diagnosis of primary affective or non‐affective psychotic disorders confirmed at each site by clinical interview and all other available sources of information (e.g., medical records, family interviews); (5) had outpatient status; (6) had no primary substance‐induced psychosis disorders; and (7) IQ >70 (determined by medical record review and clinical impressions).

2.3. Processes of LHS development

Developing the workflow of AC‐EPINET involved three core aspects of the LHS model: to actively align with stakeholders, incentivize high‐value care, and encourage continuous quality improvement. To begin, we held a startup meeting via teleconferencing technology, which was recorded and made available to those members not in attendance, as well as those who joined AC‐EPINET later. Great care was taken to understand the unique perspectives of all stakeholders, particularly around the development of procedures for modified clinical workflows with data capture, outcome selection and visualization, and potential quality improvement projects. Hub staff then met with clinical teams at each site to further consider and discuss goals for the AC‐EPINET LHS. The primary purpose of these meetings was to engage the clinical teams, promote a culture with shared ownership of the LHS (toward QI culture), and communicate a commitment to providing benefit to all stakeholders (incentive). For instance, some discussions focused on the benefits of AC‐EPINET's informatics platform, which was designed to provide clinical teams with ongoing visual analysis of each clinic's performance on benchmarked patient outcomes and reveal valued areas for targeted improvement. To encourage continuous quality improvement, we discussed strategies to use the network level data in quality improvement efforts. Some sites began to develop local QI efforts, such as decreasing clinic‐level DUP. These efforts were ongoing and coupled with transparency supported via bi‐weekly meetings with site PIs, who could then disseminate and solicit relevant information to/from the clinical team members.

Data were collected by trained site clinical team members, supported by research assistants. Patient‐level common data elements were captured via self‐report or clinician administration. Clinical staff at each of the network sites reviewed the EPINET CAB administration manual and were trained by the director of the hub's Assessment and Implementation core [JLV], who also remained available to network site raters for questions or concerns. The training was completed via prerecorded training visits and asynchronous virtual learning, which enabled busy clinicians to complete the training sessions on their own timelines. Select clinician‐administered measures (those that captured symptoms and functioning) were also practiced via experiential exercises. At the end of the initial training on clinician measures, clinicians were asked to score either video or written vignettes within a PI‐driven gold standard to balance assessment rigor with real‐world clinical practicality. If a rater's scores did not fall within a gold standard window, they were asked to meet with the assessment core, discuss differences and provide additional vignettes to score. All raters participated and were required to remain “certified” on these symptom and functional measures. This protocol was in place during initial training for all raters and confirmed annually thereafter via training vignettes and response submission to the Hub. This process was periodically adjusted to ensure high quality, efficient data collection that worked within each network site's clinical flow and documentation networks.

Regarding science and informatics, the LHS development process involved two core tasks: digital capture of the care experience and access to knowledge of the clinic performance to guide patient care improvements. Clinician‐centered implementation addressed Informatics, Science, Culture, and Incentives to integrate LHS activities as deeply as possible into daily clinical workflows. Hub staff hosted a series of site engagement meetings to explore similarities and differences between the clinics, identifying core common values and objectives for the network. For instance, all sites valued CSC programming but there were notable differences in the CSC model and how particular components were resourced. Common benchmarked outcomes, grounded in the literature and later programmed into our informatics platform, were also largely a product of such engagement meetings. Following this engagement process, a pragmatic data capture pipeline was then designed and launched, capitalizing on legacy systems wherever possible to minimize measurement burden and regulatory barriers.

For this initial phase of the LHS, the workflow included data capture via Research Data Capture (REDCap) ⁴³ and data integration and visualization through a purpose‐built informatics platform HONE. REDCap is a secure, web‐based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture; (2) audit trails for tracking data; (3) automated export procedures for seamless data downloads; and (4) procedures for data integration with external sources. Clinician‐administered scale data were either manually entered into the platform by research assistants (n = 5 sites) or exported directly from electronic health records using automated data extraction algorithms (n = 1 site). Self‐report data were captured directly via REDCap surveys. Data were then extracted and connected to a central hub using the HONE informatics platform. AC‐EPINET HONE was adapted from a core set of select data elements with research‐derived benchmarks, developed at STEP, ³⁹ , ⁴⁰ , ⁴¹ and launched to both academic and clinical users at the sites. Figure 2 illustrates the data pipeline infrastructure, detailing elements from data capture to submission to the EPINET National Data Coordinating Center. Several methods were used to engage the sites to utilize HONE and embed its use into their standard clinic workflow. The aim of this process was to support AC‐EPINET sites to leverage relevant healthcare data and identify performance gaps that can drive continuous improvement in care delivery and quality, while also enabling practice‐based research. Clinician‐friendly data visualization dashboards were designed to maximize access, engagement, and outcome parameters at the patient, clinic, and network levels to spur quality improvement. This was a reciprocal process with ongoing engagement from clinical teams on how to design and display visualizations to maximize efficiency and use.

FIGURE 2.

Data use agreements (DUAs) were in place between each site and the hub and the EPINET National Data Coordinating Center. A contributor agreement was also executed for data sharing to the National Data Archive. Data shared beyond the site was de‐identified and HIPPA compliant. Users are credentialled and managed at each site and use dual‐factor authentication, with SMS as standard. ^denotes a progressing aspirational goal, with two AC‐EPINET sites currently using their EMR for primary data capture; ⁺denotes universal step. AC‐EPINET, Academic Community‐Early Psychosis Intervention Network; CSC, coordinated specialty care; REDCap, Research Data Capture.

In parallel to the work to establish the processes of building the LHS, another important task of AC‐EPINET was to participate in discovery‐oriented research. This component of our work involved conducting a comparative pilot clinical trial of telehealth versus in‐person CSC service delivery. The pilot study was designed specifically to address key questions about the value of telehealth and technology‐assisted service delivery as a primary mode of intervention for the FES population. This ongoing randomized pilot trial could inform subsequent, definitive studies. If CSC telehealth proves effective, it could gain widespread use as an attractive adjunct to the CSC model, thereby making it more accessible and effective for a larger proportion of young people with early‐stage psychotic disorders. Though not the focus of this article, it is notable that the clinical teams in real‐world settings contributed to this research effort.

2.4. Developing an LHS culture

As described above, a site‐inclusive implementation strategy was used to build LHS culture and minimize disruption of data collection on clinical workflows. Building a culture of organizational learning and continuous quality improvement is a monumental task even within a singular organization due to stakeholder group misalignment, let alone across six states and distinct sites with independent regulatory oversight. Creating a learning culture is essential for high‐quality care ⁴⁴ but must occur across multiple levels. Shein ⁴⁵ described three distinct cultures that can develop within organizations that reflect the tacit assumptions of “operators”, “engineers” and “executives.” Like in our previous work, ⁴⁶ , ⁴⁷ these were exemplified respectively by the workflow and the need of frontline clinicians to be appreciated by those who were planning and implementing standardized collection of data elements across diverse clinical systems. This, in turn, needed to be placed within institutional frameworks for data privacy and sharing. Several presentations on how to embed data collection within an ethos of quality improvement with concrete examples were reviewed at investigator and program staff meetings (executives and engineers) and were followed by visits from the IU and Yale sites to each of the clinics to foster a shared understanding of LHS goals. One product of these discussions was a regular clinician led, multi‐site videoconference that was offered to clinicians from various disciplines and representing diverse theoretical orientations.

Consistent with the LHS model, each level of leadership, including the executive committee, hub, core work groups, and site PIs, was intentional about fostering a culture of collaboration and adaptability in aid of continuous learning. This process involved bi‐weekly meetings of site PIs, a site‐inclusive assessment and implementation work group, and regular virtual site visits by hub staff. This work was designed to collaboratively address concerns ranging from data security to the impact of data collection procedures on clinical workflows. The informatics team also met with clinical and research staff at each of the sites to co‐develop visualizations of their data on the shared platform, HONE, to be used for continuous quality improvement. Site PIs also contributed to this process at the site level as they engaged in leadership that induced a culture of teamwork, collaboration and adaptation to the additional task associated with being a member of an LHS. Finally, an important component of this work involved regular meetings among site leadership and hub leadership to review data quality and completeness within an ethos of performance improvement. Taken together, the vital ingredient of these component efforts to address culture was ongoing bi‐directional communication regarding refining clinical workflow, data capture and use for continuous improvement.

AC‐EPINET also supported a culture of continuous learning via the organization of an interprofessional case conference series for member clinicians. Interprofessional education and workplace learning activities have been shown to have a positive impact on care delivery and disease markers in chronic conditions such as diabetes. ⁴⁸ Beyond learning from single clinical cases, interprofessional case conferences can increase awareness of other professions in patient care and facilitate collaboration. ⁴⁹ Coordinated by an author and clinical psychologist [AL], the AC‐EPINET case conference was offered to clinicians in various disciplines, including psychology, psychiatry, social work, and occupational therapy, and represented diverse theoretical orientations, with styles of practice informed by cognitive behavioral therapy, acceptance and commitment therapy, and metacognitive reflection and insight therapy. The group met quarterly via videoconference for 1 h on rotating days and times to maximize clinician availability. Attendance ranged from 5 to 14 clinicians present, with representation from each member institution. The initial series of case conferences included brief introductions, followed by a 10–15‐min case presentation by a clinician. An open discussion, facilitated by the coordinating clinical psychologist, focused on a range of topics, including psychotherapy, medication management, and rehabilitative support. The facilitator encouraged participants to share perspectives from their own discipline, theoretical orientation, and ethics code. Later in the series, a new format was adopted, in which an expert guest speaker would facilitate the case discussion after presenting a brief “lightning talk” describing their approach (e.g., metacognitive reflection and insight therapy, acceptance and commitment therapy, mindfulness‐based psychotherapy for psychosis, etc.). This enabled interprofessional discussions through a novel lens, while simultaneously providing didactic information about a particular approach which clinicians could then pursue in future professional development endeavors.

3. RESULTS

All six clinical sites successfully implemented data capture and engaged in building the informatics platform. Between 4/1/2021 and 11/1/2023, a total of 614 patients were enrolled into AC‐EPINET, with the primary diagnoses of schizophrenia (32.1%) and unspecified psychotic disorders (23.6%). Table 1 details the basic demographic characteristics of baseline CABs from individuals enrolled in AC‐EPINET as of 11.1.2023. In terms of data capture, AC‐EPINET successfully implemented EPINET's consensus CAB across the six sites. Many measures of the CAB were captured via self‐report and others were clinician administered. The CAB data details patients' pathway to care, personal demographics, psychiatric and medical history, legal history, social and occupational functioning, symptoms, and quality of life. Psychiatric, medication, and functional outcomes were also captured at each follow‐up visit while patients remained enrolled.

TABLE 1.

Academic Community‐Early Psychosis Intervention Network (AC‐EPINET) baseline demographic data as of 11.1.23.

	N (%)	M (SD)
Age	614	21.81 (4.01)
Sex at birth	610
Male	417 (68.36)
Female	193 (31.64)
Race	609
Black	238 (39.08)
White	282 (46.32)
Other	53 (8.70)
No answer	36 (5.90)
Ethnicity	573
Hispanic	56 (9.77)
Non‐hispanic	517 (90.23)
Primary diagnosis	567
Schizophrenia	182 (32.10)
Unspecified schizophrenia	134 (23.63)
Other specified psychotic disorder	57 (10.05)
Schizophreniform	43 (7.58)
Schizoaffective	69 (12.17)
Major depression with psychotic features	26 (4.59)
Bipolar disorder with psychotic features	50 (8.82)
Highest level of education	490
Less than high school	47 (9.59)
High school	241 (49.18)
Some college and/or vocational training	139 (28.37)
Completed college	38 (7.76)
Advance degree training	25 (5.10)
Marital status	551
Never married	525 (95.3)
Married	15 (2.72)
Separated	5 (0.91)
Divorced	1 (0.16)
No answer	5 (0.91)

		Median (IQR)
Duration of untreated psychosis	482
First symptom to enrolling in EPINET		7.05 (17.4)
First symptom to mental health care		1.10 (4.9)
First symptom to medication		1.45 (6.2)

Note: Data collected for common assessment battery of EPINET (https://nationalepinet.org/core‐assessment‐battery‐cab/); baseline (captures events 6 months prior to enrollment in AC‐EPINET). Sample sizes vary among variables due to omitted items by clients or clinicians.

The informatics platform, HONE, was successfully adapted and made operable for each site and the hub. AC‐EPINET HONE established a core set of select data elements with aspirational goals and launched to both academic and clinical users. The initial objectives were to create visualizations to display outcomes, such as DUP, hospitalizations, symptoms, cardiovascular risk, and suicide attempts. In addition, our sites expressed a special interest and desire to be able to explore outcomes related to equity in access (such as birth sex and race), vocational engagement and successful transitions of care at discharge. Inclusion of error bars and proportion of missing data supported pragmatic, real‐time interpretation. Figure 3 shows a snapshot of the AC‐EPINET HONE display. This dashboard was provided to clinical teams, accessible via any device connected to the internet, to ultimately be used in designing valued stakeholder‐informed (clinicians, patients, and families) QI projects.

FIGURE 3.

Academic Community‐Early Psychosis Intervention Network (AC‐EPINET) coordinated specialty care Health Outcomes Network, Education (HONE) dashboard, designed for clinical end users with significant stakeholder input. The dashboard displays a core set of outcomes (“Objectives” on figure) with aspirational (blue) and achievable (green) benchmarks, displayed in aggregate with interactive overlay options: For the entire network, by individual site, as well and by sub‐groups (race, birth sex, and medication status). The dashboard enables clinical teams to readily compare their outcomes (across and within sites over time) and to identify targets for and monitor the success of quality improvement projects.

The Science component of an LHS involves both “doing what we know” and “learning what to do.” In the first domain, AC‐EPINET supported evidence based, quality care delivery via workplace‐based education and discussion of QI projects. In the second domain, the network involved all clinicians, research staff, and patients in efforts to improve data quality (both self‐report and clinician administered). Also, ongoing work continues in recruitment for and analysis of research projects and dissemination via presentations at national and international conferences, and manuscripts. The Hub has focused this process on the inclusion of both trainees and junior faculty involved in AC‐EPINET.

4. DISCUSSION

This report describes the implementation of a learning health system across six academically affiliated FES services. AC‐EPINET was informed by an approach that seeks integration across four key domains of science, informatics, incentives, and culture to align activities toward both delivery of best‐practice care while also facilitating research. ¹⁹ This report outlined the development of the necessary structure, processes, and cultural elements, and some preliminary outcomes. Several emergent challenges in AC‐EPINET provide important lessons of more general relevance to stakeholders who are already involved or wish to build learning health systems.

4.1. Data privacy and security

Clinical sites varied in their standard approaches to obtain and extract patient information, and the dual use of patient data for both quality improvement and research required that consent procedures that are traditionally distinct, become integrated into a common workflow. For data transfer and dissemination efforts, data use agreements were executed for de‐identified datasets between each clinical site and the hub, and then between the hub and the EPINET National Data Coordinating Center. Considerable effort was spent on discovery around each clinical institution's practices around data collection, legacy informatics systems and interpretations of HIPAA statutes. The process was prolonged and required extensive engagement by our hub informatics core. Lessons learned from these experiences included understanding that this process must be started as soon as possible given its duration; the need to offer a range of flexible options to account for differing levels of comfort with data sharing; and the need to address varying attitudes and levels of understanding toward the nature of Global Unique Identifiers (which was resolved here by using pseudoguids). Global Unique Identifiers are Universal participant identification numbers frequently used in projects supported by the US NIH that are based on the personal information of a specific study participant, without exposing personally identifiable information. Pseudoguids provide similar capacity as a unique identifier but are not based on complete personally identifying information. These unique identifiers enable the matching of a participant across research data repositories. The limitations on data sharing imposed by institutions also prohibited full implementation of HONE and required instead the construction of a new data pipeline from REDCap databases (already approved by all institutions) into HONE via an API. This considerably delayed the delivery of HONE‐based dashboards to the sites.

4.2. Integrating clinical and data capture workflows

An important task of AC‐EPINET was to provide clinical stakeholders with useful feedback from their data that could be used in decision making and program improvement. We soon discovered that varied levels of interaction between clinical and research activities across sites presented a particularly difficult challenge. While our sites were selected based on their research experience and established infrastructure, perhaps at the expense of generalizability to other community‐facing programs, we found that many challenges mirrored those found in non‐academically oriented settings. Indeed, we noted challenges related to competing demands on clinician time, difficulty with patient engagement in the data collection processes, and data quality. This was particularly obvious for our teams where established research and clinical teams had not previously collaborated closely. Several factors contributed to this challenge, including a limited understanding about outcome‐oriented care, a lack of familiarity with how to integrate population‐level outcomes with individualized care, and limited time to implement new or modified procedures.

Some AC‐EPINET sites had previously integrated clinical and research teams, while others had a clear division between them. While the CAB was being collected as part of routine clinical care, some sites (particularly those with less integration among clinical and research teams) did not have a previous history of extensive data capture and often struggled to see the benefit of building this work into already busy workflows. Thus, to engage sites, we allowed a great deal of flexibility around data capture processes. In retrospect, more uniformity may have been useful. For instance, some sites relied very heavily on research coordinators to enter data while other sites chose to create forms within their electronic medical record to capture data within existing documentation workflows. This created challenges for the hub to adequately monitor and provide feedback to improve data quality. Moreover, incomplete and missing data were issues at some sites. Once this was identified, the hub engaged each site and discussed challenges of data collection and worked with site PIs to use resources available to emphasize benefits to the clinicians of a more complete data set. Future iterations of FEP LHSs will benefit from a more consistent, network‐wide approach to data quality assurance. It could also be beneficial to expand the oversight model to include a dedicated individual or group to coordinate across the entire network and ensure that clinical teams are engaged early in the use of their own data. In sum, our experience points to the importance of careful consideration of the need for continuous refinement of data capture, completeness, and quality processes, as well as determination of the visualizations deemed most useful to clinical teams and how such visualizations will be used in their ongoing quality improvement efforts.

4.3. Cultural infrastructure and continuous QI

While the initial phase of building the AC‐EPINET LHS resulted in a solid infrastructure for data capture, informatic workflows, and engaged clinical stakeholders, we recognize that less focus was placed upon developing a broader LHS cultural infrastructure, which would require the involvement of additional stakeholders and consideration of LHS culture as systems of meaning, social relations, and expression. ²⁰ In moving toward a culture of CQI, our experience suggests that clinical sites should be actively engaged, as early as possible, in the development of quality improvement projects based on the network's outcomes. Further, focused attention should be paid to expanding non‐clinical stakeholder involvement, incorporating more input from patients, caretakers, educators, and other community partners into QI project idea generation and work, and vetting findings from the LHS. By expanding focus upon the cultural infrastructure during the design phase, future developing LHS' can be explicit about the culture they would like to foster and hence create an ethos of performance improvement, research, and transparent accountability to all stakeholders.

5. CONCLUSIONS

AC‐EPINET has implemented an LHS, embracing a culture of shared learning and incorporating the EPINET common assessment battery within the workflows of six academically affiliated CSCs. Lessons from this experience can serve as a model for other FEP networks seeking to build the LHS framework to improve client outcomes. The work of developing AC‐EPINET thus far has highlighted the need for future LHS' to be mindful of the complexities of data security issues and allow for ample time and resources to gain institutional approvals and data use agreements, develop more automated informatic workflows, resource quality assurance efforts as an important component for LHS success, and attend more to building the cultural infrastructure with the input of all stakeholders.

CONFLICT OF INTEREST STATEMENT

Drs. Srihari and Cahill are co‐founders of STEP‐Forward, LLC which consults with agencies to build learning health systems. No other authors have conflicts of interest to declare.

Vohs JL, Srihari V, Vinson AH, et al. The Academic Community Early Psychosis Intervention Network: Toward building a novel learning health system across six US states. Learn Health Sys. 2025;9(2):e10471. doi: 10.1002/lrh2.10471