Systematic evaluation of patient-reported outcomes in clinical trials of digital health in cardiovascular diseases

Abstract

The integration of patient-reported outcomes (PROs) in digital health (DH) trials for cardiovascular diseases (CVDs) remains unknown. We searched ClinicalTrials.gov for randomized trials that tested DH interventions in CVDs from 2004 to 2024. The search identified 8037 trials, with 673 eligible trials included in the analysis. Among these, 321 trials (48%) incorporated PROs. The number of DH trials and the use of PROs have shown a significant upward trend. Phone-based interventions predominated (38%), mostly targeting hypertension (38%) and heart failure (27%). Behavioral interventions showed higher prevalence of PROs’ usage (1.24 [1.04–1.48]), while trials for diagnostic or screening purpose (0.39 [0.20–0.77]) utilized PROs less frequently. Only 15% of trials reported results on ClinicalTrials.gov, while 58% were published in PubMed after completion. Despite DH trial expansion, PRO integration remains insufficient, especially in trials where clinical and patient perspectives are important in informing treatment decisions. Timely results dissemination is critical to improving transparency.

Introduction

Cardiovascular diseases (CVDs) remain the leading cause of death worldwide, imposing significant burdens on healthcare systems and exacerbating patient suffering^1,2. Digital health (DH) has expanded exponentially over the past decades, leveraging digital, mobile, and wireless technologies to enhance personalized care and optimize healthcare delivery^3–8. Compared with traditional medical devices and pharmaceuticals, DH tools greatly enhance care accessibility, driven by patient adoption and utilization, which in turn shapes research quality and improves quality of care⁹. With the shift toward patient-centered care, there is an increasing recognition of the importance of patient-reported outcomes (PROs) that assess patient experience, quality of life, and symptom burden using data that comes directly from the patient (i.e., without the interpretation of the patient’s responses by a physician or anyone else)¹⁰. Generally, patient-reported health status can be categorized into symptoms or symptom burden, functional status (physical, mental/emotional, or social functional limitations), and health-related quality of life^11,12. PROs are considered important metrics to measure disease experience, treatment efficacy, and quality of care^13,14. Additionally, PROs play a crucial role in CVD management due to the dramatic variability of patient health status and the inherent limitations of clinician-reported outcomes^15–17.

The landscape of clinical evidence underlying DH interventions in CVDs over the past two decades has not been well characterized, and we are unaware of any up-to-date research on the overall use of PROs in these trials registered on ClinicalTrials.gov. Previous literatures have predominantly focused on the use of PROs in oncology trials, and most studies evaluating trends of DH trials focused on a particular registration element within ClinicalTrials.gov (e.g., specific funders)^5,7,18,19. Integrating PROs into clinical trials of DH techniques is essential for capturing a comprehensive view of patient health conditions and enhancing their acceptability, informing optimal shared decision-making, and providing information for regulators on the efficacy and patient tolerability of treatment^20–22. In addition, a thorough appraisal of the dissemination practices for DH trials in CVDs is warranted to ascertain the extent of trial transparency and accessibility.

In this context, we sought to conduct a comprehensive analysis of randomized trials of digital health registered on ClinicalTrials.gov from January 2004 to August 2024, with a particular focus on PROs. The aims of the study are to: 1) demonstrate the characteristics and trends of randomized DH trials in CVDs registered on ClinicalTrials.gov; 2) explore the current use of PROs in DH trials in CVDs and identify the percentage of trials that used PROs as outcomes; and 3) analyze the dissemination of trial results on ClinicalTrials.gov and PubMed over the past two decades.

Results

We obtained 8037 trial records from the ClinicalTrials.gov search, where 705 of them were randomized trials in cardiovascular diseases. 32 trials were excluded after screening the use of digital health techniques in interventions, and 673 trials remained for final analysis, where 321 of them reported at least one PRO measure in outcomes (Fig. 1).

Fig. 1. Trial selection.

The flowchart illustrates the process of identifying and screening studies, from ClinicalTrials.gov searches to the final selection of studies meeting the eligibility criteria. DH digital health, PRO patient-reported outcome.

Use of digital health techniques in trials of cardiovascular diseases

The total number of DH trials in CVDs demonstrated an overall upward trend per year, from zero trial in 2004 to 106 trials in 2023 (Fig. 2a). The number of DH trials in CVDs that used at least one PRO as an outcome endpoint increased from one trial in 2012 to 56 trials in 2023. The lower number of trials in 2024 than in 2023 is due to the search strategy being limited to 31 August, 2024.

Fig. 2. Yearly use of PROs and different digital health techniques in trials of digital health in cardiovascular diseases.

a Bar charts for number of DH trials in CVDs using versus not using PROs as outcomes per year. The digits on the bar chart represent the percentage of trials including PROs as outcomes among all trials per year. The lower number of trials in 2024 than in 2023 is due to the search strategy being limited to 31 August, 2024. b Stacked area chart for number of trials using different DH techniques. Among the DH techniques investigated, the most commonly used were mobile app and telehealth. PROs patient-reported outcomes, DH digital health, CVDs cardiovascular diseases.

Among the digital health techniques investigated, the most commonly used in trials of CVDs were mobile app (40%), telehealth (38%), smartphone (30%) and texting (22%), highlighting the widespread use and convenience of smartphones in disease management (Fig. 2b). In terms of the primary purpose of digital health interventions in these trials, 217 (32%) trials focused on treatment, 137 (20%) trials on prevention, 105 (16%) trials on supportive care, and 42 (6%) trials on diagnosis or screening (Supplementary Fig. 2). The ranking of PRO inclusion rates among these purposes was similar, with 32% of treatment trials, 20% of trials for prevention trials, and only 6% of diagnosis or screening trials incorporating PRO measures.

Trials of digital health techniques in cardiovascular diseases per country

304 (45%) of 673 trials of DH techniques in CVDs were conducted in the USA, followed by 54 (8%) in China, 41 (6%) in Canada, 29 (4%) in Spain, 25 (4%) in Germany, and 20 (3%) in Brazil. The geographic distribution of DH trials that incorporate PROs is different from that of all DH trials. Among the six countries with the highest number of DH trials in CVDs, the PRO inclusion rate was relatively balanced, ranging from 46% of trials in US to 60% of trials in Brazil (Supplementary Fig. 3 and Supplementary Table 4)²³.

Trials of digital health techniques per sub-category of cardiovascular diseases

Among the 673 trials of DH techniques in CVDs, the five most common subcategories of diseases were hypertension (258 [38%]), heart failure (182 [27%]), coronary artery disease (161 [24%]), arrhythmias (105 [16%]), and valvular heart disease (13 [2%]). The inclusion rates of PROs within these subcategories were 98 trials (38%) for hypertension, 115 (63%) for heart failure, 89 (55%) for coronary artery disease, 38 (36%) for arrhythmias, and 8 (62%) for valvular heart disease (Fig. 3).

Fig. 3. Use of PROs in digital health trials per subcategory of cardiovascular diseases.

The bar chart displays the number of DH trials using versus not using PROs as outcomes per subcategory of CVDs. The five most common subcategories of diseases were hypertension, heart failure, coronary artery disease, arrhythmias and valvular heart disease. PRO patient-reported outcome, DH digital health, CVD cardiovascular disease.

Of the 321 trials of DH techniques that included at least one PRO, a total of 203 PROs were identified and 25 of them were CVD-specific (Table 1). Among 120 trials using CVD-specific PROs as outcomes, the most commonly used were Kansas City Cardiomyopathy Questionnaire²⁴ (48 [40%]), Minnesota Living with Heart Failure Questionnaire²⁵ (19 [16%]) and Seattle Angina Questionnaire²⁶ (12 [10%]). Among 272 trials using non-CVD specific PROs as outcomes, the most frequently used were International Physical Activity Questionnaire²⁷ (46 [17%]), Patient Health Questionnaire²⁸ (42 [15%]) and Perceived Stress Scale²⁹ (28 [10%]) .

Table 1.

Endpoint position of PROs, and top CVD/non-CVD specific PROs in digital health trials of cardiovascular diseases which used at least one PRO

Endpoint of PRO	Number of trials (N = 321)
Primary	46 (14.3%)
Secondary	201 (62.6%)
Both primary and secondary	55 (17.1%)
Other	19 (5.9%)
CVD specific PRO (n = 25, listed top 10)	Number of trials (N = 120)
Kansas City Cardiomyopathy Questionnaire	48 (40.0%)
Minnesota Living with Heart Failure Questionnaire	19 (15.8%)
Seattle Angina Questionnaire	12 (10.0%)
Atrial Fibrillation Effect on QualiTy-of-Life	11 (9.2%)
Atrial Fibrillation Severity Scale	8 (6.7%)
HeartQoL questionnaire	6 (5.0%)
Duke Activity Status Index	5 (4.2%)
American Heart Association’s Life’s Simple 7	3 (2.5%)
Chronic Heart Failure Questionnaire	3 (2.5%)
Non-CVD specific PRO (n = 178, listed top 10)	Number of trials (N = 272)
International Physical Activity Questionnaire	46 (16.9%)
Patient Health Questionnaire	42 (15.4%)
Perceived Stress Scale	28 (10.3%)
Visual Analogue Scale	26 (9.6%)
Hospital Anxiety and Depression Scale	23 (8.5%)
Generalized Anxiety Disorder - 7	23 (8.5%)
Patient Activation Measure	20 (7.4%)
EuroQoL 5-Dimension 5-Level	18 (6.6%)
Patient-Reported Outcomes Measurement Information System	17 (6.2%)
Morisky Medication Adherence Scale	16 (5.9%)

Associated characteristics and prevalence ratios of digital health trials in cardiovascular diseases with use or no use of PROs

Among 673 DH trials in CVDs, 239 (36%) were completed and 241 trials (36%) are still recruiting. 166 trials (25%) were sponsored by NIH, followed by 89 trials (13%) sponsored by federal or other governments and 77 trials (11%) sponsored by industry. In terms of trial design, 601 trials (89%) applied an parallel assignment, and 31 trials (5%) applied an crossover design. Behavioral interventions were employed in 288 trials (43%), while 101 trials (15%) utilized device-based interventions. 189 trials (28%) applied single masking, and 91 trials (14%) used double masking.

PROs were used as a primary endpoint in 46 trials (14%), as a secondary endpoint in 201 trials (63%), and as both primary and secondary endpoints in 55 trials (17%). A comparison of trial characteristics with the use or no use of PROs is presented in Table 2. We found that trial status, intervention type, primary purpose, age group, and enrollment size were significantly associated with the use of PROs. Trials that were still recruiting (OR 1.50 [1.06–2.11]), included participants aged over 65 years (OR 1.79 [1.02–3.14]), employed behavioral interventions (OR 1.24 [1.04–1.48]), or utilized combination products (e.g., drug-device or drug-biological combinations) (OR 1.99 [1.42–2.77]) were more likely to include PROs. In contrast, trials with a primary purpose of diagnosis or screening (OR 0.39 [0.20–0.77]) and those enrolling more than 1000 participants (OR 0.57 [0.38–0.85]) were less likely to incorporate PROs. There is a significant trend for improved use of PROs over time. Other factors such as type of sponsor, sex group, trial phase, number of collaborators, intervention model, masking, results reporting, publications, number of recruitment centers, and location were not significantly associated with the use of PROs.

Table 2.

Trial characteristics associated with use or no use of PROs

		PROQOPID PRO used (N = 321)	PROQOPID PRO not used (N = 352)	Relevance ratio (95% CI)	p-value (chi-square)
Status					0.0113
	Completed	102 (31.8%)	137 (38.9%)	1.13 (0.79–1.61)
	Recruiting	136 (42.4%)	105 (29.8%)	1.50 (1.06–2.11)
	Not recruiting	45 (14.0%)	50 (14.2%)	1.26 (0.85–1.85)
	Suspended, terminated or withdrawn	15 (4.7%)	22 (6.2%)	1.08 (0.65–1.78)
	Unknown	23 (7.2%)	38 (10.8%)	Reference
Funder					0.2863
	NIH	80 (24.9%)	86 (24.4%)	0.99 (0.82–1.20)
	Federal/Other government	45 (14.0%)	44 (12.5%)	1.04 (0.82–1.31)
	Industry	30 (9.3%)	47 (13.4%)	0.80 (0.59–1.08)
	Other	166 (51.7%)	175 (49.7%)	Reference
Numbers of collaborators					0.5053
	1	162 (50.5%)	177 (50.3%)	Reference
	2	82 (25.5%)	91 (25.9%)	0.99 (0.82–1.20)
	3	37 (11.5%)	29 (8.2%)	1.17 (0.92–1.49)
	4	16 (5.0%)	25 (7.1%)	0.82 (0.55–1.22)
	>4	24 (7.5%)	30 (8.5%)	0.93 (0.68–1.28)
Sex					0.8395
	All	305 (95.0%)	331 (94.0%)	0.96 (0.24–3.84)
	Female	15 (4.7%)	20 (5.7%)	0.86 (0.20–3.61)
	Male	1 (0.3%)	1 (0.3%)	Reference
Phase					0.4962
	Early phase1	1 (0.3%)	2 (0.6%)	0.71 (0.14–3.51)
	Phase2	6 (1.9%)	4 (1.1%)	1.27 (0.76–2.13)
	Phase3	5 (1.6%)	2 (0.6%)	1.52 (0.94–2.44)
	Phase4	4 (1.2%)	2 (0.6%)	1.41 (0.80–2.50)
	Unknown	305 (95.0%)	342 (97.2%)	Reference
Intervention model					0.4955
	Sequential	3 (0.9%)	8 (2.3%)	Reference
	Parallel	290 (90.3%)	311 (88.4%)	1.77 (0.67–4.66)
	Factorial	15 (4.7%)	15 (4.3%)	1.83 (0.66–5.13)
	Crossover	13 (4.0%)	18 (5.1%)	1.54 (0.54–4.39)
Intervention type					0.0194
	Behavioral	154 (48.0%)	134 (38.1%)	1.24 (1.04–1.48)
	Device	44 (13.7%)	57 (16.2%)	1.01 (0.78–1.31)
	Diagnostic test	2 (0.6%)	5 (1.4%)	0.66 (0.20–2.15)
	Combination product	6 (1.9%)	1 (0.3%)	1.99 (1.42–2.77)
	Procedure	2 (0.6%)	6 (1.7%)	0.58 (0.17–1.94)
	Other or Mixture	113 (35.2%)	149 (42.3%)	Reference
Primary purpose					0.0002
	Prevention	60 (18.7%)	77 (21.9%)	0.90 (0.66–1.23)
	Diagnostic/Screening	8 (2.5%)	34 (9.7%)	0.39 (0.20–0.77)
	Health services research	43 (13.4%)	63 (17.9%)	0.84 (0.60–1.17)
	Treatment	117 (36.4%)	100 (28.4%)	1.11 (0.84–1.47)
	Supportive care	61 (19.0%)	44 (12.5%)	1.20 (0.89–1.61)
	Other	32 (10.0%)	34 (9.7%)	Reference
Masking					0.5480
	Single	97 (30.2%)	92 (26.1%)	1.15 (0.96–1.39)
	Double	47 (14.6%)	44 (12.5%)	1.16 (0.92–1.46)
	Triple	18 (5.6%)	19 (5.4%)	1.09 (0.77–1.56)
	Quadruple	7 (2.2%)	7 (2.0%)	1.12 (0.66–1.92)
	None	152 (47.4%)	190 (54.0%)	Reference
Masking person					0.4315
	Investigator	10 (3.1%)	15 (4.3%)	0.90 (0.55–1.48)
	Outcomes assessor	65 (20.2%)	54 (15.3%)	1.23 (1.00–1.50)
	Participant	20 (6.2%)	21 (6.0%)	1.10 (0.78–1.53)
	Care provider	2 (0.6%)	2 (0.6%)	1.12 (0.42–3.02)
	Mixture	72 (22.4%)	70 (19.9%)	1.14 (0.93–1.39)
	Unknown	152 (47.4%)	190 (54.0%)	Reference
Reported results in ClinicalTrials.gov					0.2515
	Yes	19 (5.9%)	17 (4.8%)	1.29 (0.91–1.83)
	No	83 (25.9%)	120 (34.1%)	Reference
Has publications in PubMed after primary completion					0.7118
	Yes	57 (17.8%)	81 (23.0%)	0.93 (0.69–1.24)
	No	45 (14.0%)	56 (15.9%)	Reference
Age group					0.0257
	≥65 years	312 (97.2%)	328 (93.2%)	1.79 (1.02–3.14)
	<65 years	9 (2.8%)	24 (6.8%)	Reference
Study size					0.0072
	≤100	134 (41.7%)	131 (37.2%)	Reference
	101–500	139 (43.3%)	135 (38.4%)	1.00 (0.85–1.19)
	501–1000	29 (9.0%)	39 (11.1%)	0.84 (0.62–1.14)
	>1000	19 (5.9%)	47 (13.4%)	0.57 (0.38–0.85)
Number of centers					0.8825
	1	240 (74.8%)	261 (74.1%)	Reference
	2–5	58 (18.1%)	63 (17.9%)	1.00 (0.81–1.23)
	6–10	13 (4.0%)	13 (3.7%)	1.04 (0.70–1.55)
	>10	10 (3.1%)	15 (4.3%)	0.84 (0.51–1.36)
Country of recruitment					0.7182
	HIC, UMIC or mixture	310 (96.6%)	337 (95.7%)	1.13 (0.72–1.79)
	LMIC or LIC only	11 (3.4%)	15 (4.3%)	Reference
Region of recruitment					0.9275
	North America only	162 (50.5%)	179 (50.9%)	1.07 (0.79–1.44)
	Europe and Central Asia only	90 (28.0%)	97 (27.6%)	1.08 (0.79–1.48)
	East and Pacific only	41 (12.8%)	41 (11.6%)	1.12 (0.79–1.60)
	Other or mixture	28 (8.7%)	35 (9.9%)	Reference
Year					0.0003
	2012–2014	5 (1.6%)	9 (2.6%)	Reference
	2015–2017	31 (9.7%)	62 (17.6%)	0.93 (0.44–1.99)
	2018–2020	102 (31.8%)	120 (34.1%)	1.29 (0.63–2.64)
	2021–2024	183 (57.0%)	159 (45.2%)	1.50 (0.74–3.05)

Dissemination of registered trial results

Among the 239 completed trials, 36 (15%) have submitted results to ClinicalTrials.gov, with 19 including PROs. Of the 138 trials that published manuscripts on PubMed, 57 reported PROs when registered on ClinicalTrials.gov. Among these, consistent PRO reporting between ClinicalTrials.gov and the corresponding PubMed articles was observed in 12 trials (21%). In 27 trials (47%), not only were all PROs as listed in ClinicalTrials.gov reported, but also additional PROs were documented. One trial (2%) did not report any PROs in its PubMed manuscript; 14 trials (25%) reported only a subset of PROs that were stated in ClinicalTrials.gov, and 3 trials (5%) reported a subset of the PROs listed on ClinicalTrials.gov while also including other unprespecified PROs in their PubMed articles. There was no significant association between the use of PROs and results submission to ClinicalTrials.gov (OR 1.29 [0.91–1.83]), nor between PRO use and publications available on PubMed (OR 0.93 [0.69–1.24]). The median interval from the primary completion date to the results submission date was 764 days. Figure 4a shows the submission interval according to the use of PROs. The difference is more apparent after the median survival time, where trials not using PROs were more likely to remain non-submitted. Figure 4b compares submission timelines by sponsor type, indicating that trials sponsored by NIH tend to submit results faster than those sponsored by federal or other government agencies, industry, or other entities.

Fig. 4. Kaplan–Meier curves showing days to results or manuscripts submission from primary completion date for trials.

a Kaplan–Meier curves for days to results or manuscripts submission by use of PROs. b Kaplan–Meier curves for days to results or manuscripts submission by type of sponsors, including federal or other governments, industry, NIH and other sponsors. PROs patient-reported outcomes.

Discussion

The present study comprehensively summarizes for the first time the characteristics of DH trials in CVDs and provides novel insights into the role of PROs over the past twenty years. We found rapid growth in the number of DH trials in CVDs. Notably, there has been a marked rise in the incorporation of PROs in these trials from 2004 to 2023, with over one-third of trials that included PROs designating them as primary or co-primary outcomes. PROs were most commonly used in heart failure and hypertension, for which assessment of health-related quality of life and symptom burden is particularly important. Furthermore, the use of PROs was more pronounced in trials aimed at digital health interventions for treatment and supportive care, while it was less used in trials centered on diagnostic or screening purposes. Among the completed trials, the submission rate of results to ClinicalTrials.gov was low, and the interval times from the primary completion date to the results submission date were likely to be shorter in trials using PROs than those not using PROs as outcomes.

To our knowledge, this study is the first to investigate the use of PROs and their associated characteristics specifically within DH trials targeting CVDs. Previous research has examined either digital health interventions or the use of PROs in trials related to other diseases or broader contexts. The existing literature indicates a general increase in the application of PROs in clinical trials^30,31; though some exceptions have been noted, such as a reported decline in PRO usage in atrial fibrillation trials from 1999 to 2018³². A scoping review of 96,736 trials registered on ClinicalTrials.gov between 2007 and 2013 revealed that 27% of trials incorporated PROs, predominantly in oncology-related studies³³, highlighted that adherence to reporting guidelines for PROs in clinical trials was consistently suboptimal. Dr. Vanderhout and colleagues³⁴ reported that 35% of pragmatic trials from a narrow window of 2014 to 2019 included PROs as primary or coprimary outcomes—a proportion comparable to the 34% observed in our analysis of DH trials in cardiovascular diseases. A similar proportion of funding sources was also observed, showing that trials using PROs were more likely to be sponsored by governments and universities, followed by industry. Given the relatively lower prevalence of PRO use in industry-sponsored trials, there is a pressing need for more patient-centered methodologies in these studies, underscoring the necessity for an updated and comprehensive analysis of DH trials in CVDs.

The increasing trend in the use of PROs in DH trials for CVDs aligns with existing evidence supporting the positive impact of digital health interventions on cardiovascular risk factor management³⁵. The growing adoption of PROs may stem from increased awareness of their role in clinical decision-making, patient-centered disease self-management, and quality-of-life improvements^20,36–38. Some researchers advocate for including patient-reported symptoms as a component of high-quality care^39,40. This trend is further reinforced by the release of guidelines and scientific statements on PROs^41,42, as well as methodological frameworks such as the SPIRIT-PRO⁴³ and the CONSORT PRO⁴⁴ extensions, which enhance the rigor of PRO design and reporting in alignment with global patient-centered care standards^42,45,46.

The application of DH interventions varies across different geographical regions. The United States, European countries, and Canada have traditionally been the leading countries conducting clinical trials^23,47. However, our findings reveal that China has now emerged as the second-leading country in DH trials focused on CVDs, with Canada ranking third. Brazil has entered the list of the top six countries for the first time, with a high proportion of PRO usage. This shift indicates a growing recognition of the importance of DH techniques^48,49 and PROs^50–52 in these countries.

PRO measures have been predominantly utilized in digital health technologies aimed at managing chronic conditions such as hypertension, as well as in heart failure and coronary artery disease, where the evaluation of health-related quality of life and symptom burden is critical. The extensive application of PRO measures in these domains underscores the increasing acknowledgment of PRO data in clinical decision-making processes, facilitating diagnosis and monitoring of symptom alleviation. Numerous clinical trials addressing conditions like hypertension have been structured to encourage self-management and lifestyle changes, thereby enhancing quality of life and mitigating symptom burden^53–56. Our findings corroborate this trend, revealing that PRO measures were most frequently employed in trials incorporating behavioral interventions, thereby highlighting their significant role in clinical practice.

In this study, PRO measures were most frequently used in trials of digital health aimed for treatment and supportive care, while their use was the lowest in trials focused on diagnostics or screening-focused trials. These findings align with previous studies on other types of interventions and suggest that engaging patients in the co-design of DH trials is valuable in ensuring that studies are tailored to their needs^20,34. Our systematic evaluation identified very few trials specifically designed to assess the acceptability or usability of the digital health technologies being tested. We believe that prioritizing patient perspectives is essential for making trials more ethical and beneficial.

Considering the dissemination of trial results, the proportion of trials with use of PROs submitting results on ClinicalTrials.gov was poor (only 5%), which was also an issue in early-phase oncology trials with only 3% reporting the results in other reviews⁴⁷. The interval times from primary completion date to results submission date were much longer than that in other studies^57,58, and only 36 trials had submitted results in compliance with the 1-year deadline⁵⁹.

The study has important public health implications. Although DH trials in CVDs are encouraged to incorporate patient perspectives, the overall use of PROs remains limited, and their implementation presents significant challenges. Barriers exist at multiple levels, including patient-related factors such as patient’s inability to complete PRO measures, survey fatigue, privacy concerns, and perceived lack of relevance or value^40,60,61. Physician-level barriers include limited time to discuss PROs with patients during clinics, insufficient knowledge of the interpretation and integration of PROs into clinical practice, and skepticism about their usefulness. At service level, the lack of integration of PROs into clinical workflows and inadequate IT infrastructure to enable easy data collection further hinder adoption. Addressing these challenges through targeted strategies is essential to optimizing PRO implementation and maximizing their impact on digital health research and clinical practice.

Our study found that most DH trials in CVDs used PROs as secondary outcomes, which might reduce the chance of identifying statistically sufficient differences between groups⁴. It is advisable for trialists and methodologists to prioritize the selection of PROs as either primary or coprimary outcomes, while also making appropriate adjustments for multiplicity, particularly in studies where both clinical and patient perspectives are critical for informing treatment decisions, thereby enhancing the robustness of the evidence generated³⁴. Healthcare professionals should empower patients and their caregivers with resources that facilitate their understanding and involvement in the design of trials utilizing PROs, such as web-based tools that support the collection of PRO measures. We found trials with a large number of participants enrolled, for example over 1000 participants, were less likely to use PROs in outcomes, suggesting that more attention might be needed for the use, monitoring, collection and management of PROs across multiple centers⁶² in such DH trials for CVDs. Furthermore, transparency in the reporting of trial protocols, including comprehensive descriptions of the interventions and the demographics of the enrolled patients, is essential for DH trials. Our review identified that numerous trials failed to provide detailed information regarding the digital health interventions, which is crucial for evaluating the validity of the trials. Therefore, registrants should be encouraged to report protocols in accordance with the guidelines set forth by the SPIRIT-PRO Extension⁴³ and CONSORT PRO Extension⁴⁴.

Over 15 years have elapsed since the introduction of the reporting requirements established by the U.S. Food and Drug Administration (FDA) Amendments Act of 2007 which broadened the regulations governing ClinicalTrials.gov and mandated the reporting of trial results within 12 months of the completion of trials for all medical products regulated by the FDA⁵⁹. However, our review indicates that the percentage of submitted trial results remains low, with many submissions significantly exceeding the one-year deadline. A critical factor contributing to this situation—irrespective of disease category or type of intervention—appears to be the absence of enforcement actions or explicit penalties imposed by regulatory authorities^58,63,64. To address this, regulatory agencies, along with journals, institutions, and sponsors, should strengthen oversight and provide clearer guidance on trial reporting and results submission. It is noteworthy that trial registration rates experienced a significant increase following the implementation of policies by the International Committee of Medical Journal Editors (ICMJE)^65,66. Building on this precedent, one potential solution is for ICMJE-affiliated journal editors to mandate results submission to ClinicalTrials.gov as a prerequisite for article publication⁶⁷. Furthermore, a stricter adherence to CONSORT guidelines, which would include the obligatory incorporation of trial registration numbers in published articles, would enhance the accuracy of automated linkages between registry records and their corresponding publications, thereby improving the quality of reporting^34,44,68. Given the substantial increase in trial registrations following previous policy changes, such initiatives are likely to enhance the transparency and dissemination of ongoing clinical research.

One strength of this work is the depth and detail of the analysis of associated characteristics with use of PROs in DH trials in the cardiovascular disease field, as well as the dissemination of trial results. Some characteristics were analyzed for the first time, including method of intervention assignment, masking and role of blinded participants which might be associated with quality and transparency of trial design, and type and number of collaborator sponsors who did not take the lead responsibilities of trial. Another strength is that we applied natural language processing methods to assist data collection and extraction procedures, including batch searching and downloading trials from ClinicalTrials.gov, identification and normalization of cardiovascular diseases, identification of PROs in outcome descriptions, retrieving and selection of completed trials to associated PubMed publications, and crawling manuscripts’ data for submission dates. These approaches reduce workflow substantially, and the extracted data were easier to be normalized and categorized into different groups simultaneously.

Our work has several limitations. First, we selected trials from ClinicalTrials.gov maintained by National Library of Medicine in the United States, which may exclude some trials registered in other websites such as EU Clinical Trials Register⁶⁹, Chinese Clinical Trial Registry⁷⁰, etc. Similarly, in the analysis of dissemination of completed trials, we focused on manuscripts submitted to PubMed and did not consider those published in other databases, which would lead to an underestimation of publication rates. Second, the insufficient enforcement and regulation of reporting standards on ClinicalTrials.gov often results in incomplete or outdated trial registry entries, which may lead to inaccuracies in data extraction and analysis. To minimize misclassification, two independent reviewers screened all trials on digital health interventions for cardiovascular diseases, and any discrepancies were independently resolved through discussion with a senior third reviewer. Third, although PROs are valid, reliable, and sensitive measures of patients’ experiences in clinical trials and especially useful in chronic diseases or trials of digital health, we acknowledge that PROs might not be informative or feasible in some cases depending on the trial design and research aims, such as trials involving population with cognitive impairment or studies where objective clinical endpoints (e.g., laboratory values) are the primary focus. Finally, our automatic pipeline of PRO identification was developed based on the vocabulary of 5517 PROs from Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID)⁷¹, which might miss PROs outside the database. However, it is the current largest PRO database and we did the analysis in accordance with previous literature and methodologies^20,72.

In conclusion, this systematic analysis of the ClinicalTrials.gov registry provides valuable insights into the significance of PRO measures in the evaluation of digital health technologies within clinical trials for cardiovascular diseases. PROs are instrumental in capturing the patient perspective, underscoring the importance of prioritizing patient-centered outcomes, even within the most technologically sophisticated healthcare systems. Furthermore, the prompt dissemination of trial results is crucial for improving data transparency, and initiatives are necessary to foster a culture of systematic reporting and practice.

Methods

The trial selection process is shown in Fig. 1, and the semi-automated pipeline of data processing and analysis is shown in the Supplementary Fig. 1. This study is reported as per the extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (PRISMA-S) guideline (see Supplementary Table 1).

Data collection and screening

We systematically searched ClinicalTrials.gov for interventional trials started between January 1, 2004 and August 31, 2024, with any digital health technologies applied in the intervention or treatment procedures. The search terms used the Medical Subject Heading concepts including “digital health”, “mobile health”, “mHealth”, “telehealth”, “telemedicine”, “remote monitoring”, “mobile application”, “text messaging”, “web-based”, “wearable device”, “sensor”, “wireless technology”, etc. We did not include the patient-reported outcomes and synonyms in the search strategy to avoid missing any relevant trials. The complete search strategy is in the Supplementary Table 2. We wrote a Python script to automatically search and download the candidate trials of digital health in XML format. An initial search was performed on January 14, 2024 and an updated search was followed on September 2, 2024.

We excluded observational studies, registries, and expanded access reports. Diseases or conditions were extracted to filter the trials of cardiovascular diseases as defined in Medical Subject Headings (MeSH). We applied the biomedical entity extraction tool BERN2⁷³ from the “Condition” field from each trial, and obtain the trials with extracted disease entities belonging to the pre-selected 154 cardiovascular disease terms (see Supplementary Table 3). We further checked the trial summary, participation eligibility criteria and design details to remove those trials that did not involve any cardiovascular disease in the trial settings but still recorded cardiovascular diseases in the “Condition” field.

We extracted the descriptions of interventions, titles, brief and detailed summaries from the potential digital health trials of cardiovascular diseases in XML format and converted them into an Excel table. After a pilot process aiming to improve the process, two reviewers (QW and XH) independently screened all studies, excluding trials not assessing digital health techniques and trials if digital health techniques were used in a surgery treatment procedure, or were not used for patient intervention (for example, some trials used digital health for clinician education). Discrepancies were resolved through discussion by a third senior reviewer (JC).

Data extraction

Trial characteristics were extracted and generated automatically: status (completed, recruiting, not recruiting, suspended/terminated/withdrawn, unknown), phase (1, 2, 3, 4 or not applicable), strategy for assigning interventions to participants (sequential, parallel, factorial, crossover), masking (single, double, triple, quadruple, other), type of intervention (behavioral, device, diagnostic test, combination product, procedure, other or mixture), primary purpose of the intervention (treatment, prevention, diagnostic/screening, health services research, supportive care, other), sponsors, collaborators, number of collaborators, parties who are prevented from having knowledge of interventions assignment (investigator, outcomes assessor, participant, care provider, mixture, unknown), participants age group (≥65 years or <65 years), sex, number of participants enrolled, trial locations, start year of trial, whether the trial reported results in ClinicalTrials.gov, economic group of recruitment country (high income country/upper middle and high income country/mixture, lower middle income country or low income country only), and region of trial recruitment (North America, Europe and Central Asia, East and Pacific, mixture or other). The Economic group and region information was generated from countries of recruitment and then mapped into country classification in the World Bank Group⁷⁴.

We combined two fields (lead sponsor class and collaborator sponsor class) from ClinicalTrials.gov to categorize trials by type of sponsors⁷⁵. Specifically, if NIH was listed either as the lead sponsor of a trial or a collaborator, we classified the study as NIH sponsored; if NIH was not involved, but either the lead sponsor or a collaborator was from federal or other governments, we classified the study as sponsored by federal/other governments; if NIH or federal/other governments were not involved, but the lead sponsor or a collaborator was from industry, we classified the study as sponsored by industry; remaining trials were assigned “other” as the sponsor type.

We also extracted whether a trial has published results in PubMed database after trial primary completion. We wrote a Python script to automatically search each trial’s NCT ID in PubMed, and crawled the metadata of retrieval results to obtain the earliest submission date for associated manuscripts. We used a validated PubMed Clinical Queries filter⁷⁶ (setting as broad “therapy”) for clinical queries to exclude protocols, commentaries and other non-relevant publication types. For trials that reported PROs on ClinicalTrials.gov and have published manuscripts available on PubMed, we reviewed the PROs reporting within the corresponding PubMed manuscripts to evaluate the consistency of PROs’ reporting. Type of digital health techniques was extracted by retrieving keywords in intervention descriptions.

For trials that featured PROs as outcomes, we specifically extracted whether the related PRO was assigned as a primary or secondary endpoint. We collected 5517 PRO terms and their abbreviations from Patient-Reported Outcome and Quality of Life Instruments Database (PROQOLID)⁷¹, excluding those that belong to the clinician-reported outcomes, observer-reported outcomes, performance outcomes or health-care use measures. The collected PRO vocabulary was used to identify PROs from outcome descriptions for each trial: we first split outcome descriptions into noun chunks by the spaCy library⁷⁷ in Python; we then applied fuzzy matching between potential PRO chunks and the full names in PRO vocabulary, and computed similarity score based on Levenshtein distance⁷⁸; we applied exact matching for abbreviation in PRO vocabulary. We manually reviewed the matching terms if the similarity score was no less than 80 or PROs were included in at least three trials, and all abbreviation terms if the number of term characters was less than four. We revised or merged any incorrect results. We collected the type of Clinical Outcome Assessment in the PROQOLID database and manually categorized whether a PRO is a cardiovascular disease-specific or generic measure.

Data analysis

Characteristics of studies reporting PROs were compared to those that did not report PROs as outcome measures. The number of digital health trials in cardiovascular diseases by region, type of digital health techniques by year and primary purpose, and percentage of sub-categories of cardiovascular disease were demonstrated in figures; the prevalence of PROs were tabulated using frequency distributions. Other categorical variables were described as number (percentage), and associations were checked by chi-square tests with a significant level of 0.05. The strength of the association with each categorical characteristic was described via risk ratios and 95% confidence intervals. Year of trials was grouped into 3-year intervals and the Cochran-Armitage test⁷⁹ was performed for trend. The Kaplan–Meier method⁸⁰ was used to model the primary completion date to result submission date. All analyses were conducted in Python version 3.10.9, except for the Cochran-Armitage test and Kaplan–Meier curves, which were performed in R version 4.2.2.

Author contributions

Q.W. and X.H. contributed equally as co-first authors. X.H., D.S. and J.Z. take full responsibility for the integrity of the data and the accuracy of the data analysis as co-corresponding authors. Q.W. and X.H. conceptualized the study and prepared the original draft of the manuscript. Q.W., X.H. and X.B. did the data collection, annotation and curation. Q.W., X.B. and C.W. implemented R/Python codes and did the formal analysis. D.S. and J.Z. supervised the data collection and analyses. J.L. supervised the visualization. J.C. and Q.Z. did project administration. All authors provided comments on preliminary versions and approved the final manuscript.

Data availability

The datasets used in the study are publicly available from ClinicalTrials.gov and PubMed. Questions regarding analyzed data access should be addressed to the corresponding author.

Code availability

The underlying code for analysis in the study is not publicly available but may be made available to qualified researchers on reasonable request from the corresponding author.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at 10.1038/s41746-025-01637-8.