Navigating the Digital Health Ecosystem to Bridge the Gap from Innovation to Transformation: A NODE. Health Perspective on Digital Evidence

Abstract

This article examines the challenges of broad adoption of digital solutions within the healthcare industry and why evidence is so critical for advancement of digital health technologies. It then illustrates how emerging nonprofits are positioned to help the ecosystem overcome these challenges and to catalyze the process of matching the right evidence-based solution to the right clinical challenge.

Introduction

In 2017 the healthcare venture capital industry invested 5.8 billion USD into digital medicine in the hopes of propelling the transformation of the industry and making some money while doing it [1]. While some of these investment dollars have indeed contributed to improving healthcare, many have not generated the desired returns clinically or financially. Investing in digital solutions within healthcare is a tricky business. Investors must evaluate not only the typical attributes of the company, i.e., the team, the market, and the technology, but also multiple additional healthcare specific factors that can play dramatic roles in its success. For example, solutions that require electronic medical record integration often demand heavy involvement of the local information technology (IT) teams, which may have different priorities and timelines. Security requirements related to compliance with the Health Insurance Portability and Accountability Act (HIPAA) and now Europe's General Data Protection Regulation (GDPR) enforce that digital solutions undergo extra scrutiny related to the ways they collect, analyze, manage, transmit, and store patient data. All of these factors make pilot efforts very time consuming and complicated. So, when a healthcare system begins down the road of investigating the use of a digital health solution for one of its needs, efficiency is paramount. The more prepared a digital health company is, the more established its track record is, the more evidence that company has of success, and, ideally, the more credible the evidence behind the company's technology, the smoother that road may be.

While traditional measures of evidence, including the randomized controlled trial, still bear relevance to digital health solutions, increasingly such technologies are being called to present real-world data and real-world evidence [2]. Proponents of real-world evidence recognize that patient selection and engagement in the environment of a controlled trial is not the same as what may be seen when digital technologies are implemented in non-trial settings. As digital health companies present themselves to potential customers, the roles that credible real-world data and evidence play are increasingly important.

What is more challenging, however, when it comes to digital integration and adoption is understanding which solutions have an impact on clinical and financial outcomes. And yet, as critical as this is, digital solutions often lack quality clinical evidence showing their impacts relative to current care [3]. Many of these digital solutions fail in their abilities to show improvements in clinical outcomes due to a lack of understanding of how clinical validation works. Other common reasons for the failure of digital solutions include limited durable patient engagement and their inability to position themselves as valuable, if not necessary, tools in the clinical workflow.

Filling a Need

Many hospitals and healthcare systems have developed digital innovation departments whose charge it is to identify and vet potential digital health solutions for their systems' needs. These groups, each within their own institutions, tend to reinvent the same spokes of very similar wheels. In a truly learning healthcare system, however, this duplicative effort is inefficient. Just as there is common evidence from studies and clinical trials available to payers as they consider adding new medications to their formularies, so too should there be common evidence available to healthcare systems in the digital health marketplace. Although the concept of a “digital health formulary” is not new, there has been little success to date in aggregating the rigorous evidence about such tools to make healthcare systems, providers, and patients confident in their selection. It is the mission of the nonprofit Network of Digital Evidence in Health (NODE. Health; nodehealth.org), as part of a growing ecosystem that includes organizations such as the Digital Therapeutics Alliance (DTA; dtxalliance.org) and others, to develop strategies to overcome these challenges, to guide digital health companies toward generating evidence, to enable healthcare systems to share with one another, and to more quickly identify the right evidence-based digital solutions to their clinical needs.

The concept of evidence curation is not new. The National Guidelines Clearinghouse, for example, whose funding was cut in July of 2018, served for 20 years as a main source of evidence-based clinical practice guidelines [4], and promoted their dissemination, implementation, and use. Organizations such as the International Consortium for Health Outcomes Measurement [5], serve as brokers of evidence around the use of patient-reported outcome measures for different disease states. Similarly, the ecosystem around digital health evidence (Fig. 1), as well as digital tools intended to support validation, is growing, but the number of nonprofit organizations in this space remains limited [6, 7]. As the Food and Drug Administration continues to unfurl its Digital Health Software Precertification (Pre-Cert) Program [8], we can expect to see more structure around how digital evidence will be viewed and used in the future.

Fig. 1.

Sample landscape of organizations generating and curating digital evidence.

Why Is Evidence So Much More Important in Healthcare than in Other Industries?

The focus on evidence in healthcare is not unique to the industry. All industries exploring new ideas naturally require evidence that the time, energy, and money put into the venture are justified. The consumer products industry looks to customer feedback and willingness to pay, and the transportation industry pays for comprehensive studies to assess the economic impact of investments, while the food industry looks for validation in competitions or customer taste tests. But healthcare's thirst and need for evidence trump most other industries for some obvious and not so obvious reasons [9].

People's Lives Are at Stake

The easy answer to why healthcare requires increased rigor related to evidence for new innovations is related to what is at stake. For a drug to be approved for prescription, it is supposed to have generated appropriate evidence of both safety and efficacy. The principle of do no harm, related to safe practice of medicine, is deeply embedded in physician practice, and rightfully so. Moreover, as pertains to efficacy, any prescribed treatments, therapies, or clinical protocols must be highly scrutinized to ensure they contribute to better outcomes, or are at least non-inferior to the current standards of care.

Healthcare Is a Slowly Changing Industry

While the industry of consumer apps, wearable fitness trackers, and wellness moves at an incredibly rapid pace, traditional healthcare is a historically slowly changing industry that tends, out of regulatory oversight, to be risk averse. So, unless the evidence for a new intervention or digital health application is extremely high, or unless there are strong financial drivers (e.g., the Health Information Technology for Economic and Clinical Health [HITECH] Act, or value-based care), there is little incentive for healthcare decision makers to take unnecessary risks.

Clinical Evidence Alone Is Not Sufficient

While some digital health solutions may be able to credibly demonstrate clinical efficacy, they must also demonstrate cost-effectiveness. If a digital health solution improves outcomes, but only does so at an increased cost, it is unlikely to get adopted. However, few digital health startups recognize the importance of, or are capable of credibly demonstrating both clinical efficacy and cost-effectiveness.

Many Physicians Are Trained with an Academic Rather than Pragmatic Mindset

Many physicians exploring the digital technology or solution are ensconced in research. They expect data to underlie the digital health claims, particularly if they also intend to publish findings in the peer-reviewed literature. Many have spent years understanding the importance of rigorous study design and require the highest of standards when evaluating a new digital solution. These expectations of rigor are a high bar for many digital health solutions. However, clinical trials, due to strict patient inclusion/exclusion criteria, are not necessarily representative of the results that might be expected with real-world digital health implementations, a topic that has been described as a challenge for the industry [2]. Clinical champions who have experience with formal clinical trials, but who are also rooted in implementation science coupled with pragmatic evidence, are assets to digital health as an industry.

Digital Solutions Are Overcoming “Snake Oil” Labeling

Ever since the American Medical Association Chief Executive Officer and Executive Vice President James L. Madara, used the phrase “digital snake oil” during his address to the American Medical Association's House of Delegates at the 2016 Annual Meeting [10], digital medicine has been under scrutiny to prove its worth. In the early days of digital medicine, monitoring devices and app-based solutions were adopted eagerly by the “quantified selfers,” i.e., those patients who were generally healthy but highly conscious of and interested in their activity levels, caloric intake, and energy expenditure. Many of these solutions reported, to various degrees of unvalidated accuracy, heart rate, activity, and blood pressure. The medical industry sometimes referred to these as “toys.” Since then, digital solutions have come a long way, and increasingly have demonstrated their impact on key medical quality indicators such as readmission rates [11]. However, the scrutiny rightfully remains as the industry tries to separate the snake oil solutions from those that have a demonstrable impact on the cost and quality of care that patients receive.

So What Is the Problem in Finding Evidence-Based Digital Solutions?

Despite the necessity for evidence-based claims, developers of digital solutions are struggling to navigate the healthcare industry to obtain evidence of their impact on clinical outcomes. Typically, a startup will reach out to healthcare systems through their networks to ask if the system is interested in solving a particular problem using a new solution. The startup may claim to produce clinical or administrative benefits or increase patient satisfaction in a vague way, but it is also highly interested in clinical evidence to validate these claims. However, there are several issues related to this exploration in the early stages of trying to obtain clinical evidence.

Navigating the Ecosystem

The evolution of the industry and the adoption of digital technologies in healthcare are slowed by complex bureaucracy, laborious administrative approval processes, risk assessment, burdened health information technology departments, and hesitance to insert new applications into the clinical workflow. The healthcare ecosystem is incredibly complex to navigate. Healthcare companies must not only identify and collaborate with a clinical champion at the site, i.e., someone who is passionate about the technology and what it can do for his/her department, but also design their solutions to concurrently satisfy pain points for the customer (the healthcare system decision maker who is often the one who writes a check or signs off on the pilot), the clinical users (physicians, physician assistants, nurses, medical assistants, and others), administrative users such as practice managers, patients, health IT departments, and security/privacy officers. Identifying just a clinical champion who is empowered and interested in exploring the technology can take months, but having a finger on the pulse of all of the above stakeholders can be daunting for fledgling companies. Often, ideas are just passed from one office to another until they disappear. Companies lucky enough to have navigated these processes often suffer from poorly designed pilots, unnecessary complexity, and lengthy and inconclusive studies. The term “death by pilot” has pervaded the industry and great solutions have often succumbed to poorly supported pilot implementations or integrations.

Different Partnership Approaches Impact Validity

There are incredibly differing viewpoints in how these partnerships or “studies” should be approached. In some cases, the digital solution claims that it is solving a problem for the healthcare system and should be compensated for the exploratory use of its technology. In other cases, the startup is interested in the use case alone and so might offer a free “pilot.” Either way, understanding when to partner, pay, or engage in a free pilot often impacts the validity of the results itself. The different approaches create a confusing array of studies that range from fundamentally flawed to overly scientific that won't necessarily reflect real-world evidence [12].

Resourcing the Study

Obtaining clinical evidence takes appropriate resources. These resources range from the principal investigator, or the clinical champion that sponsors the study, to the administrative sponsor, the research analyst(s), the biostatistician, and, if it is considered human subject research, the institutional review board. In many instances, the business side of the healthcare system is driving the decision to pilot a digital health solution and attempts to take the lead on driving the study. Unfortunately, even though the success of a pilot hinges on what seems like a simple question, i.e., “Does the solution work?”, the business stakeholders do not typically have the background or knowledge of how to design a clinical study or a study on clinical cost-effectiveness. Furthermore, many digital health startups are small enough that they themselves are not resourced with individuals who have the appropriate expertise to design and evaluate a pilot study that has sufficient scientific rigor to pass peer review. Similarly, many healthcare systems, particularly those of nonacademic medical centers, may not have all of these resources either. Moreover, while many academic medical centers do have physician-scientists, digital health companies should recognize the difference between a physician (one who does not typically have any formal study design or scientific training) and a physician-scientist. Only those who have conducted formal research or clinical trials will tend to have the expertise needed. All of these are resources that are typically paid on a per-enrollment basis to manage and conduct a study. Understanding this process, and indeed paying for it, is usually new territory for startup companies and may be financially out of reach for many.

To Integrate or Not?

Full integration into the clinical workflow or the electronic medical record is often required to achieve full benefits of a clinician-facing digital tool. Unfortunately, the complexity related to the integration process may delay the pilot implementation and require significant upfront financial commitment for both the startup company and the implementation site. While some solutions can be validated without integration, others may require partial or full integration for functionality. Startup companies should focus on an absolute minimum set of integration needs necessary to realize clinical or administrative benefits of the digital health solution, and should recognize that the start-up costs for a healthcare system to integrate into their electronic medical records are often substantial enough that there is not sufficient justification for doing so for pilot projects.

Time

Conducting studies with scientific rigor takes time, which is not often a luxury that small companies have. Designing and agreeing on the study, its outcome metrics, and how to measure and capture those metrics alone takes time. Additionally, enrollment to attain a sufficient sample size, and then running the study accounting for attrition, can also be time consuming. Finally, procuring the data and analyzing them involves a considerable investment. Many young digital health companies will not be around long enough to see their studies through, and those that do may not be able to afford the (often gratis) time of their own personnel's involvement in the study execution and analysis.

What Does the Digital Ecosystem Need and How Does NODE. Health Help?

The healthcare industry ecosystem is highly reliant on clinical evidence, especially for new solutions in their infancy. However, this process can be complicated and is often overwhelming both for companies with new and untested digital health solutions and for healthcare systems trying to separate the signal from the noise. The nonprofit NODE. Health supports both parties by providing guidance for digital health solution companies seeking to generate evidence, and it serves as a reputable source of existing digital solutions for healthcare systems seeking to rapidly land on technologies to address their needs.

NODE. Health

NODE. Health, a nonprofit, 501(c)(3), is the home for evidence-based digital medicine that brings together a network of societies, organizations, and innovators dedicated to digital transformation in health. NODE. Health serves as a coordinated network of diverse healthcare systems – societies that can help startups navigate the process, advise clinical study and pilot design, and ultimately support clinical validation and publication of evidence gathered across multiple healthcare sites. NODE. Health members represent a diverse group of academic innovation centers, industry, investors, and entrepreneurs. NODE. Health's mission is to promote the rigor of evidence-based medicine in the emerging area of digital healthcare technologies to help create evidence-based digital medicine. Combining the spirit of innovation with scientific rigor, NODE. Health adds to and builds upon the digital health knowledge base increasingly demanded by leading healthcare systems and members of industry alike. In short, NODE. Health acts as a clinical validation network of the digital health industry in the rapidly growing field of digital evidence-based medicine.

NODE. Health membership includes 20+ healthcare systems across the country, with affiliates around the world comprised of a diverse set of academic, research, community, specialty, and ambulatory health systems serving diverse populations. Because of the size of NODE. Health's network, the nonprofit is positioned to design studies and clinical trials that can accommodate or target multiple health settings, payer mixes, geographies, patient groups, and medical circumstances. Core to the NODE. Health mission is to help bring all the key stakeholders (healthcare systems, hospital and clinicians, as well as investors, startups, payers, and digital partners) together to understand what is needed when it comes to clinical evidence within digital medicine. NODE. Health helps in a few specific ways:

• Sharing information about ongoing single site digital medicine pilots to prevent duplication

• Standardizing existing governance and regulatory policies to fast track pilot adoption and evaluation

• Adopting new research designs and clinical trial endpoints that are relevant to digital health

• Launching strategic initiatives to support multisite digital medicine pilots

• Supporting clinical validation for digital solutions through multisite trials

• Publishing and disseminating digital health results, and helping to launch the journal of digital medicine evidence.

NODE. Health is intended to support a critical gap in the industry, connecting digital solutions with healthcare systems that can adopt solutions and demonstrate efficacy. However, not all solutions require extensive clinical validation. For example, those that offer a benefit and have a limited potential to do harm, such as enabling patients to access their own records, may not require a randomized controlled trial to demonstrate a clinical outcome. By analogy to the pharmaceutical industry term GRAS (generally recognized as safe), some solutions pose benefits with little safety risk and do not require extensive evidence before adoption. However, regardless of whether or not a digital solution requires evidence for clinical efficacy, nearly all will require credibility with regard to claims of cost-effectiveness. If the cost of the new technology exceeds the cost savings of the outcome desired by the healthcare system, that technology will struggle for adoption. A concept of GRAS for digital solutions means that formal trials need not necessarily slow down the adoption of digital health solutions. Moreover, while the pharmaceutical industry is committing more and more deeply to using digital health technologies (from eDiaries to wearable trackers) in novel ways, many in the pharma space are still working on finding their way, as evidenced by their participation in programs like the FDA Pre-Cert program and by the report by the Clinical Trials Transformation Initiative on how to use mobile technologies in clinical trials [7, 8]. Understanding those digital solutions that do require clinical validation, along with implementing the appropriate pilot design, can save the industry millions of dollars and streamline healthcare innovation.

Conclusion

Medicine and healthcare are complex, and a lot is on the line. For decades the industry has tried to balance innovation with safety. New pharmaceuticals and medical devices can require decades of research and development, costing hundreds of millions of dollars before reaching the healthcare end users. While the co-development, iteration, and evaluation of digital health applications often fall in part to the healthcare end users themselves, and can be perceived in their operational frameworks as resource intensive processes, those healthcare systems with investigators and innovation centers that know how to operate nimbly much like their industry startup partners, can dramatically reduce the cycle time to validation. As the number of digital health companies and digital health applications burgeon, those that partner with such nimble innovation centers and organizations committed to demonstrating evidence in digital health stand to distinguish themselves in the demanding healthcare marketplace with high-quality digital evidence.

Disclosure Statement

Brian Van Winkle is the executive director of NODE. Health and he is on the executive board of the organization. Yauheni Solad also serves on the NODE. Health executive board. Nitin Vaswani serves as the executive program director of the organization. Benjamin Rosner reports equity interest in Infermedica, Inc.