Transforming an Autism Pediatric Research Network into a Learning Health System: Lessons Learned

Abstract

Introduction:

The Autism Speaks Autism Treatment Network that serves as the Autism Intervention and Research Network on Physical Health (ATN/AIR-P) has a mission to improve the health and well-being of children with Autism Spectrum Disorder and determine the best practices that lead to improved outcomes and expedite the translation of findings to practice. To better achieve this mission, the ATN/AIR-P is engaging in a design process to transition to a Learning Network (LN), the Autism Learning Health Network. The purpose of this paper is to: (1) make the medical and patient communities aware of an Autism LN that is based on the Institute of Medicine’s definition of a Learning Health System; (2) describe how and why the ATN/AIR-P transformed to an LN; and (3) share lessons learned that might inform the transition of future existing networks surrounding other conditions.

Methods:

Design methods included: an in-person design session with various stakeholders, the development of a Key Driver Diagram and redesign of organizational processes, network governance, and data collection and analytics.

Results:

We realized many benefits in making the transition to an LN along with many lessons that can inform the design and implementation of the LN model when transforming existing networks to learning health systems.

Conclusions:

Transitioning a well-established research network requires a complex redesign of existing processes, data infrastructure, and cultural shifts compared with developing a new LN. We identified factors that may inform the transition of future established networks to expedite the process.

INTRODUCTION

Autism Spectrum Disorder (ASD) is a heterogeneous developmental disorder impacting 1 in 59 children in the United States.¹ The prevalence of ASD has outpaced available trained providers and placed a strain on the medical and educational system. Medical and mental health conditions, such as anxiety, sleep disturbances, gastrointestinal (GI) issues, and seizures commonly co-occur in children with ASD.^2,3 These co-occurring conditions, in combination with core symptoms of ASD, negatively impact the health and well-being of children with this diagnosis.^4,5 Widely accepted evidence-based practice standards for treating ASD and these co-occurring conditions are lacking. The heterogeneity of presenting features and symptoms across the autism spectrum, the pervasive nature of deficits across numerous domains, interventions spanning numerous systems of care, and the lack of outcome measures designed for this population make it difficult for clinicians to: (1) determine the best course of treatment to recommend to a specific patient and (2) evaluate the impact of any single implemented intervention.

The Autism Speaks Autism Treatment Network (ATN)/Autism Intervention Research Network on Physical Health (AIR-P), initiated in 2008, is a joint network that currently consists of 12 academic medical centers across North America. These institutions work together to improve diagnosis and medical care for children with ASD by developing a multidisciplinary model of care, identifying and describing co-occurring medical conditions, developing and disseminating best practice standards of care, and providing a platform for research. The ATN/AIR-P network provides clinical services to over 35,000 children with ASD annually. The research focus of the AIR-P is to improve the physical health of children with ASD through intervention research. It builds on the clinical focus of the ATN and further increases the network’s impact on patients and families. The joint network has historical strengths in research, building consensus guidelines, family engagement, and creating toolkits for patients, families, and clinicians.^6–8

In 2011, as part of network activities, the ATN/AIR-P began implementing quality improvement (QI) projects to evaluate clinical improvements such as screening for insomnia and constipation and improving monitoring of antipsychotic medication. These activities were focused and short term. Building on this base, the network leadership wanted to push care improvement and create a system for measuring change, both to document what is/is not working and to accelerate improvements in care and the well-being of children and their families. The network also wanted to address critical gaps in ASD research such as the inclusion of underserved populations and the translation of these findings into clinical practice. The network saw a need for increased focus on reducing variation and assuring the use of effective practices with a high degree of reliability. Hence, the ATN/AIR-P leadership envisioned functioning as a network-based Learning Health System commonly referred to as a “Learning Network” (LN), to more quickly acquire and test new knowledge and translate best practices into standard care.

Although a QI network is usually a short term (12–18 months) and often focuses on process change, an LN maintains a continuous focus on outcomes. This focus may include some intermediate process measures but continues the work long term, changing systems to achieve the desired outcomes. An LN applies rigorous QI science methodology to identify gaps in services to be targeted for improvement and identify successful practices to replicate across the network site. LNs endeavor to achieve population health outcomes at scale. An LN platform does this by aligning clinical care, informatics, and culture to focus on continuous improvements, innovation, and research. Data are continually generated and collected as part of the clinical process and are linked across participating sites (while maintaining patient privacy and rights). Data are analyzed on a regular schedule and are shared across the network, catalyzing discovery and interventions. An essential element of an LN model is the active partnering with patients and families in all aspects of the network.^9–11

METHODS

Participants

Human Resources Services Administration’s funding of the existing ATN/AIR-P Network provided the foundational backing for this effort. All network sites received the option to join the LN transition initiative. At the time of launch, 11 of the 12 sites agreed to participate. Currently, all 12 network sites are participating. Network leadership engaged the expertise of the Anderson Center for Health Systems Excellence (Anderson Center) at Cincinnati Children’s Hospital Medical Center to support transition activities.

Framework

The Anderson Center team facilitated the network transition using a design approach developed by Associates in Process Improvement following the Model for Improvement. Development of the LN followed an actor-oriented organizational structure, which includes: (1) aligning around a common goal; (2) development of standards, processes, policies, and infrastructure to support multicenter collaboration; and (3) sharing of information, knowledge, and resources to achieve the goal. “Actor-oriented” describes an organizational structure that is designed to be flexible and adaptive. This type of organizational structure allows for more distributed decision-making to enable large groups of people to self-organize to problem solve issues within the healthcare system that are important to them.^12–14

Although the ATN/AIR-P possessed many of the important elements, the network lacked several critical components required to function as an LN successfully. Addressing these gaps were essential first actions steps in preparing for the network transition (see Table 1). Network leadership, Anderson Center QI consultants, and family partners completed the Maturity Model tool. This tool, in development by the Anderson Center, rates 6 network capabilities: governance and policies, network management, QI, research facilitation, engagement, and information technology. These capabilities are used to measure progression to a mature LN. Results were reviewed in detail and provided a baseline for systems changes to advance the transition.

Table 1.

Identified Skills Needed by ATN/AIR-P for Transition to an LN

Typically, an LN is formed by sites coming together with a common desire to create a network under a Learning Health Systems model. As such, stakeholders develop the network with a mindset toward improvement. The mission, vision, and governance are developed toward this end. In this case, the ATN/AIR-P was a well-established network with an existing research mission, vision, and governance structure. The critical first step in transitioning the ATN/AIR-P to function as an LN involved obtaining buy-in from Network stakeholders. Although there was a general understanding of the benefits of QI methodology in healthcare, there was a limited appreciation of how an LN has the potential to create a superior platform for evidence gathering and research.^15,16 There was concern that the shift would jeopardize rather than complement a rigorous research environment. Education and frequent communications around the future vision of the Network with stakeholders led to better understanding and support of the transformation to an LN. Once buy-in was established, we initiated the next steps for LN development.

Procedure

We assembled a design team to establish the overall mission and goal of the Autism Learning Health Network. The design team included ATN/AIR-P leadership, Anderson Center team, parents, clinicians, researchers, and data analysts/biostatisticians. The design process included an in-person 2-day design session followed by weekly collaborative leadership calls. Because healthcare systems traditionally hold to an established hierarchy, it was important to establish equality in team participation and input. Thus, the team established a value that all team members’ input, including parents, was equally considered. At the in-person design session (described in Table 2), families were asked to speak about what they thought an optimal/ideal outcome for persons with ASD would look like, clinical team members shared their perspectives, and the group discussed a shared vision and current system gaps. Participants then identified a focus area to target which was a priority for families and fell within the scope of the network. Two primary aims were selected: (1) to improve the overall health and quality of life (QoL) of children with ASD and (2) to increase the percentage of children with ASD who receive all recommended routine health services (physical examinations, dental services, and vision and hearing screenings) per American Academy of Pediatrics/Bright Futures recommendations. The primary focus on improving physical health and QoL is well aligned with the established mission and priorities of the network funders and allows for consideration of other factors that impact care and outcomes for children with ASD, such as co-occurring conditions and timely access to diagnostic and treatment services. In the first version of the Key Driver Diagram (KDD) (Fig. 1), the aims focused solely on improving the QoL of children with ASD and receipt of recommended physical healthcare. We identified challenging and interfering behaviors as a primary key driver. Behavior challenges have high prevalence and significant impact in the ASD population. It became clear in the iterative design process that because behaviors are drivers of physical health and QoL, we would also need to focus on improving early identification and effective management of challenging behaviors to improve outcomes for individuals with ASD at a population level. Therefore, reducing the proportion of children with moderate-to-severe concerns about behavior became the third aim.

Table 2.

Activities for Transition to an LN

Fig. 1.

Key Driver Diagram. Definitions: Recommended health services: Annual physical exam/well visit, dental visit, and vision and hearing screening per American Academy of Pediatrics/Bright Futures recommendations. Highlighted sections indicate current work in progress. Arrows indicate directional support.

To measure the 3 outcomes, we partnered with Duet Health to develop a web portal and free app operating on Android or iOS platforms that patients can use to enter data. The app allows more specific detail than typical phone messages and the convenience of completing surveys anywhere, at any time, including between clinic visits. The web portal and app meet Health Insurance Portability and Accountability Act security requirements for electronic Personal Health Information transfer, assuring confidentiality.

Over the ensuing months, leadership and decision-making structures were reconsidered to better incorporate collaborative leadership input. We reviewed existing network activities, and those determined to be outside the priority areas of the transformation to an LN or impact the priority aims from the KDD, were phased out or modified to better align with the LN focus. Measures and data collection methods were revised to reflect the newly developed KDD.

RESULTS

Progress of Transition Activities

Several positive Network changes have resulted from the initiation of the LN transition.

Adoption of the LN Philosophy

Perhaps most importantly, the commitment to a Network aim has provided a shared cross-Network focus for families, leadership, and site teams. This vision has facilitated work toward developing a cohesive system for evidence gathering, implementation of change, and dissemination of findings.¹⁷

QI teams are now active in all 12 Network sites. Monthly QI webinars are held to promote collaboration among QI teams and provide a forum to share ideas, challenges, successes, and testing strategies for improvement. The LN provides formal opportunities for on-going training in QI methodology. At the initiation of the transition process, 9 site teams participated in an online training course on fundamentals in QI methodology to improve site-level QI capabilities. We expanded the data coordination team to include expertise in QI analytics.

Further Integration of Family Partners

The Network engaged leaders in patient and family partnering to assist in moving from “family engagement” to a model of “co-production.” Family engagement is creating a partnership between families and clinicians, schools, community, etc. These partnerships are important for promoting patient/student well-being and success. Co-production goes a step beyond. The partnership is reciprocal allowing families and clinicians to educate each other. In co-production, patient and family members have an opportunity to participate side by side with the clinicians and cultivate a deeper understanding of each other’s expertise and values.¹⁸ In this model, families are actively involved in decision-making across all aspects of the LN. Family representatives were active participants in the design day and continue to serve on leadership calls and receive QI training as part of site QI teams and monthly webinars. They are involved in the development of the app, measures, and KDD. Patient and parent partners attend remote and in-person meetings and are frequent presenters and leaders in active Network workgroups.

Maturing of the Network

The Maturity Model tool, used to track growth in several different domains of the LN, indicated growth toward functioning as an LN. A date comparison between January 2016 and January 2017 reveals growth in 5 of 6 measured domains. “Systems of Leadership” was slightly reduced, but this may be due to the elimination of 3 items within the “Systems of Leadership” domain during tool revision. Subsequent Maturity Model ratings will assist in tracking network progress toward a functioning LN (Fig. 2).

Fig. 2.

Learning Network Maturity Model.

An independent identification for the LN was established (Autism Learning Health Network) to provide opportunities for network expansion beyond the current ATN/AIR-P grant while allowing those partner sites or network to join ALHN while maintaining independent network identities and scope of work

Collection of Parent-reported Outcomes

Activation and uptake of the web portal and/or app to report data have been slow but continue to improve. In January 2017, the cumulative enrollment for the Network in the app was 70 patients; as of September 2018, the cumulative enrollment was 731 patients. Although we have realized the modest improvement in enrollment, interventions to test expanding options for data collection are underway. Preliminary baseline data on the 3 outcome measures validate the need to monitor routine medical care in ASD as diversity in the LN expands.

Additionally, the data validate the need to address concerns of QoL and demonstrate a significant need to screen and better manage challenging and interfering behaviors. Over 70% of the parents report moderate to extremely severe behavior. These data support families’ strong voice that improving challenging behaviors, including medical management, should be a priority focus for the improvement of the population. In future reports, we will present details of the aim and specific efforts toward improvements on the aim.

DISCUSSION

The purpose of this paper is 2-fold. First is to make the medical and patient community aware of the development of an Autism LN.¹¹ The second purpose is to share the strategies used, including benefits and challenges, and to transform an existing research focused network into a functioning LN. We anticipate that these lessons will inform future networks considering this transition.

Significant Network advances have been realized in developing a cohesive focus, maintaining buy-in from stakeholders, engaging parents and caregivers as co-producers, and aligning activities with the LN focus. A commitment has emerged from Network site leaders to expand the effort, both regarding expanding enrollment to ultimately include all patients with ASD at each site and in developing an expanded model that includes additional LN partner sites. Valuable lessons continue to be learned that may be applied to improve the speed and success of future transitions of existing networks to an LN.

There are unique challenges in transforming an existing network into an LN rather than starting de novo. Before launch, it may have been beneficial to engage in a more detailed self-evaluation process with the expanded collaborative team. This would allow for a shared understanding of current network mission, activities and timelines, governance structure and decision making, data management infrastructure as it relates to QI analytics, and network QI capability and resources. Consideration of these elements is essential for the development of an efficient design model. Conceptualization of the LN may begin with efforts that very closely align with a network’s existing mission and strengths, rather than engaging in a more open-ended design idealization process. This approach may allow existing resources to be better utilized, leading to more rapid transition and some early successes for network members. The LN work can, and should, expand as the network evolves. During the transition process, ATN/AIR-P leadership underestimated the complexity of redesigning the current processes and data infrastructure. Even slight shifts in network mission required a great deal of communication and consideration. The design team failed to appreciate fully the level of commitment from site leaders to the existing research mission and structure. In retrospect, it would have been beneficial to implement formal change approaches including: (1) education and communication; (2) broader participation and involvement; and (3) negotiation and agreement,¹⁹ as a precursor to, and integrated within, the development approaches.

Another area of challenge involved the shift from collecting clinical data to parent-reported outcome (PRO) data electronically through an app or web portal. PRO data are extremely important to the success of our LN as outcomes such as general health and well-being of children with ASD are best measured through parent report. Additionally, PRO data can be used to collect and evaluate status between clinical visits. However, there may have been benefits in optimizing clinical data collection first, given the Network’s track record of success in this area, before expanding to PRO data, given the complexity of collecting these data.

Although we underestimated challenges the transition process presented, the realized and anticipated benefits outweigh the challenges encountered. Although the experiences of ATN/AIR-P transitioning into an LN may not be fully generalizable to other situations, the learnings do provide valuable considerations for the future transformation of other established networks. The ability to integrate large data collection within clinical processes to inform strategies to changes in clinical care and support research that can be translated and disseminated more quickly is essential in improving care. As more existing research networks undertake the shift to an LN model and additional experiences are shared, and tools are refined to monitor progress, the design process can be refined to eliminate or mitigate many of the barriers and challenges experienced by previous networks and accelerate future transitions.

CONCLUSIONS

The transition to the Autism Learning Health Network has allowed the ATN/AIR-P to develop a shared vision for a community of patients, families, clinicians, and scientists who will use data for clinical care, improvement, research, and innovation. The transition process has assisted the ATN/AIR-P in fostering infrastructure to integrate improvement and research, create a systematic approach for gathering and analyzing clinical data, develop and test interventions, and build a formal means for dissemination of findings. The important work to broaden enrollment will add rich diversity to the data and yield greater success for improving healthcare and outcomes for individuals with ASD. As more networks shift from traditional research to an LHS model, we anticipate that the experiences of the ATN/AIR-P will provide important guidance to inform successful future transitions.

DISCLOSURE

Dr. Lannon is a senior faculty advisor for the American Board of Pediatrics. The other authors have no financial interest to declare in relation to the content of this article.

ACKNOWLEDGMENTS

This project is/was supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under cooperative agreement UA3 MC11054 - Autism Intervention Research Network on Physical Health and Autism Speaks. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, the U.S. Government, or Autism Speaks. This work was conducted through the Autism Speaks Autism Treatment Network serving as the Autism Intervention Research Network on Physical Health.