An Economic Framework to Measure Value of Pediatric Telehealth

Abstract

Background: Health care is experiencing significant disruptive innovation with the use of technology, including telemedicine and virtual modalities to deliver care. These new models can dramatically improve access to care and reduce health care disparities for patients, especially in underserved and vulnerable populations like children.

Problem: To assure diffusions and retention of new approaches they must be assessed for economic value. However, measuring telehealth programs using only an institutional financial viewpoint fails to convey all the societal value of such interventions, so economic studies of telehealth are needed but complex to design.

Methods: We reviewed economic study guidelines, details from telehealth study research protocols, IRB submission questions, and telehealth study design literature. We then used an iterative consensus process to develop a framework for measuring value of virtual care at the societal level.

Results: The work produced a set of tools that are useful for designing studies for measuring value of virtual care at the societal level. The framework and tools are presented in this report.

Conclusions: The conceptual framework for economic evaluation was feasible for use when applied to measure pediatric telehealth value.

Background

Economic evaluation is a robust and evidence-based scientific field that seeks to measure overall societal value of specific interventions.^1,2 This method of evaluation is fundamentally different from the financial and business models that are used primarily by organizations to determine overall cost: although financial modeling can assess an organization's costs and revenues, an economic evaluation takes a broad perspective to assess the impact of an intervention on all stakeholders. Value is commonly defined as outcomes achieved per cost, and economic evaluation is a method of measuring value that captures each of the domains of the triple aim of health care: population health, cost of care, and patient experience of care (including quality and satisfaction).^3,4

Telehealth and Economic Evaluation

The field of telehealth has a unique and critical need for this method of value measurement. Health care is currently experiencing a digital revolution, developing promising new care models with unprecedented opportunity for movement toward population health and value-based care. However, if telehealth initiatives are measured using the narrow lens of organizational costs alone, the societal value of these new care models will be overlooked. This period of industry disruption has created a critical climate for showing value to inform policy development. The methods of measurement must take a societal viewpoint to include impact on the entire system. Economic evaluation is the bridge between organizational financial measurements and clinical and socioeconomic outcomes for patients.

Economic evaluation in pediatric telehealth has an even more critical need due to the vulnerable nature of our patient population. Inequitable access to pediatric care creates significant health care disparities for children since much of pediatric care is regionalized and centered in urban settings.^5–7 This fundamental disparity leads to worse clinical outcomes and is exacerbated for children with special health care needs.^8,9 Telehealth has an enormous impact on access to care, but many of the measurable benefits are delayed. In addition, since children are comparatively healthy and only account for ∼8.4% of health care costs in a fee-for-service system,¹⁰ funding for pediatric initiatives is often secondary to the care of adult patients with high volume and high reimbursement diagnoses that can be developed at scale. Furthermore, many children are covered by Medicaid insurance, whose smaller profit margin further disincentivizes health systems to prioritize pediatrics over a privately insured or Medicare population.

If economic evaluation is not incorporated into measurement techniques, pediatric telehealth programs risk loss of funding on an organizational level since traditional financial models do not capture the total contribution of telehealth. From a population health viewpoint, chronic disease in pediatrics is on the rise. Investing in the health of children today is an investment in healthy adults of the future, which has a downstream financial impact on society, as demonstrated by economic analysis of childhood vaccinations.¹¹ Vaccines are measured as prevention, whereas telehealth should be analyzed as a treatment intervention. The time scale must be considered, and one should recognize that access to care may have a longstanding economic effect. Access to care is integral for future value-based models of care that align systemic incentives to keep patients healthy and out of the hospital, and the exclusion of pediatrics from value-based care models would be unethical.

To further the critically important economic evaluation of pediatric telehealth interventions, this publication proposes a framework for measuring value of virtual care at the population level.

The Pediatric Telehealth Economic Framework

A framework has been developed as a set of tools to help guide organizations, investigators, and telehealth leaders step by step through the process of successfully measuring value of telehealth interventions. This framework includes five components:

• (1)Formulating a research question
• (2)Describing an intervention
• (3)Data sources for economic analysis
• (4)Methods for institutional review board approval
• (5)Protocol for statistical analysis.

Further application and development for specific use cases are under development and are described in Table 1.

Table 1.

Step-by-Step Guide for Economic Evaluation Using Supporting Pediatric Research on Outcomes and Utilization of Telehealth-CTSA Economic Evaluation Framework

PROCESS STEPS	EXAMPLE
Formulating a research question	Q1—Effect: Is there a decrease in ED visits, admissions, and overall cost for technology-dependent children after initiation of a team-based virtual care program, compared with events and costs before the program?
	Q2—Examination of bias: Is the trend observed in Q1 different for a matched group of kids who did not get these services?
Describing an intervention	Conceptual model of intervention (or logic model), patient eligibility, recruitment methods, enrollment, baseline data collection, care team composition, virtual system, intensity of services, prevention of dropout
Data sources for economic analysis	24+ months of state-wide ED and hospital billing data “tagged” with an indicator of program use, matched by propensity-score methods to data from children who have not received services.12
Institutional review board application facilitation	Ask for informed consent for use of archival billing data on enrollment. Optimize privacy by using “not-easily-identifiable” data. Do not ask for any more data than what is needed to measure your outcomes. Sample size calculations and power analysis on the cost comparisons are a key component; request help from a biostatistician if needed.
Protocol components for statistical analysis	Difference-in-differences design with patients matched by propensity scoring. Multivariable generalized linear models with appropriate link functions (e.g., logit, identity, and log) will be used. If any covariates are imbalanced between groups, they will be included in the multivariable models. Transformation (gamma) if needed for the cost data analyses.

ED, emergency department; CTSA, clinical and translational science awards.

Summary and Conclusions

Measurement of value using an economic framework from the societal perspective is critical to the success of new and innovative technologically supported care models and shifting toward patient-centric value-based care.

There are several challenges to this methodology that should be addressed. First, economic evaluation of nonrandomized studies must acknowledge the presence of selection bias. By definition, preintervention data are biased. It is a challenge to demonstrate future cost of care reduction when the patient undergoes a significant health intervention. Second, the lack of a comparison group for an intervention requires additional analytical methods to demonstrate true effects. Third, the time course of measuring an intervention cannot be underestimated. In many cases, the cost of care after an intervention increases initially before demonstrating a sustained decrease over time. Finally, there is a significant delay in the availability of data due to the reporting structure of these data. Data from public data sources can be delayed by 9–18 months due to claims processing, but institutional cost and claims data are limited to their own system and cannot capture full patient financial data. Timely analytics and evaluation are essential in modern medicine, and a robust health information exchange can provide health data in a near real-time manner. Efforts to evaluate telehealth initiatives should include pediatric patient health data. The recent COVID-19 pandemic or other future emergent crises demonstrate the importance of timely data collection and analytics reflecting the impact of telehealth. These challenges due not preclude rigorous economic evaluation but must be accounted for when assessing the value of interventions.

The use of standardized measures and methods for economic evaluation of individual telehealth programs also opens the door for impactful evidence-based collaborative research studies through networks such as the Supporting Pediatric Research on Outcomes and Utilization of Telehealth (SPROUT) network. The creation of a strong evidence base in telehealth, particularly pediatric telehealth, is critical to advocacy efforts to inform policy makers of the true impact of new models of care on the population, the crucial need for equitable funding of these care delivery models, and the importance of transitioning the health system toward value-based care. In tandem with this economic framework, SPROUT has developed the broader SPROUT Telehealth Evaluation and Measurement (STEM) framework, which provides guidance on selection of appropriate metrics and measurement approaches for specific telehealth programs and services, geared toward the targeted value domain and stakeholder audience. This economic framework synergizes across multiple STEM value domains as a means of defining the approach to measuring the economic impact of improved outcomes, access, quality, and costs.¹³

In conclusion, the comprehensive measurement of value proposed in this economic framework incorporates a population-level perspective to account for gains that would otherwise not be included in financial analysis. Use of this framework can lead to publication of evidence of the true value of telehealth and a strong evidence base to lead to policies that can support continued funding of these innovations, with an overall goal to provide high-quality patient-centric cost-effective care for patients no matter where they live.

Internal Review Board Approval

This is a perspective article that does not include human subjects and, therefore, does not require IRB approval.

Disclaimer

The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Disclosure Statement

The authors have no competing interests, personal financial interests, employment by an organization that may gain or lose financially from the publication of the article, or personal relationship that may inappropriately affect the integrity of the research reported.

Funding Information

This publication was supported, in part, by NIH/NCATS SPROUT-CTSA Collaborative Telehealth Network Grant No. U01TR002626.