TABLE 3.

Participant Observations Regarding Types of Evidence Desired for Evaluating DTx and Determining Value

Type of Evidence	Desired Details and Questions to Consider
Required for market entry
Information security	Compliance with HIPAA data security requirements
Usability	Is there a level of health/digital literacy that is required to receive benefit from the DTx? Does the DTx operate as intended? For example, do all components of the software function as designed?
Required for evaluation of value
Clinical effectiveness	Premarket: Must demonstrate safety and efficacy using standard endpoints prior to market authorization by regulatory authority What effect does the DTx have on clinically accepted, standard endpoints for the disease based on a measurable set of data? What is a clinically meaningful benefit/result? How do outcomes in the real world compare with those used for regulatory approval? What is the effect on patient satisfaction and quality of life? What level of evidence (e.g., RCT) is appropriate? Will depend on the health condition/medical claim
Engagement (adherence)	Do patients use the DTx as intended in the real world? What “dosage” (level of sustained use over time) is required to achieve desired outcomes? Patient acceptance of the user interface/satisfaction of using the therapeutic Patient-reported outcomes can be used for assessment Potential for product updates to alter the user interface and affect engagement
Safety	What are the adverse events in clinical trials? What are the adverse events in real-world use? How do adverse events compare to standard of care? What is the potential for harm? For example, what is the effect if the patient discontinues another therapy as a result of using the DTx?
Comparative effectiveness	How does the DTx compare with other available treatments for the condition? The level of rigor required will depend on the potential for harm, availability of other therapies, and whether the DTx is considered an adjunct or a replacement (i.e., a standalone treatment)
Cost impact	Can cost avoidance be demonstrated? How does the DTx affect total cost of care?
Data access	Who owns the data? Who has access to the data? How are the data used?
Ongoing evaluations	How will product updates be assessed to provide ongoing assurances of efficacy, safety, and usability?

DTx = digital therapeutic; HIPAA = Health Insurance Portability and Accountability Act; RCT = randomized controlled trial.