Potential clinical impact of predictive modeling of heterogeneous treatment effects: scoping review of the impact of the PATH Statement

Abstract

Background:

The PATH Statement (2020) proposed predictive modeling for examining heterogeneity in treatment effects (HTE) in randomized clinical trials (RCTs). It distinguished risk modeling, which develops a multivariable model predicting individual baseline risk of study outcomes and examines treatment effects across risk strata, from effect modeling, which directly estimates individual treatment effects from models that include treatment, multiple patient characteristics and interactions of treatment with selected characteristics.

Purpose:

To identify, describe and evaluate findings from reports that cite the Statement and present predictive modeling of HTE in RCTs.

Data Extraction

We identified reports using PubMed, Google Scholar, Web of Science, SCOPUS through July 5, 2024. Using double review with adjudication, we assessed consistency with Statement recommendations, credibility of HTE findings (applying criteria adapted from the Instrument to assess Credibility of Effect Modification Analyses (ICEMAN)), and clinical importance of credible findings.

Results:

We identified 65 reports (presenting 31 risk models, 41 effect models). Contrary to Statement recommendations, only 25 of 48 studies with positive overall findings included a risk model; most effect models included multiple predictors with little prior evidence for HTE. Claims of HTE were noted in 23 risk modeling and 31 effect modeling reports, but risk modeling met credibility criteria more frequently (87 vs 32 percent). For effect models, external validation of HTE findings was critical in establishing credibility. Credible HTE from either approach was usually judged clinically important (24 of 30). In 19 reports from trials suggesting overall treatment benefits, modeling identified subgroups of 5–67% of patients predicted to experience no benefit or net treatment harm. In five that found no overall benefit, subgroups of 25–60% of patients were nevertheless predicted to benefit.

Conclusions:

Multivariable predictive modeling identified credible, clinically important HTE in one third of 65 reports. Risk modeling found credible HTE more frequently; effect modeling analyses were usually exploratory, but external validation served to increase credibility.

INTRODUCTION

Overall, or average, treatment effects from randomized clinical trials (RCTs) provide limited information to patients making personal treatment decisions. (1–6) Even in strongly positive RCTs, some patients do not benefit from the favored treatment and yet may experience adverse effects. Patients and clinicians would benefit greatly if more individualized evidence could be generated and reported from RCTs.

Most publications of RCT results continue to limit examination for possible heterogeneity of treatment effects (HTE) to one-at-a-time comparisons between numerous patient subgroups, e.g. men vs. women, persons with vs. without diabetes, even though guidelines for identifying HTE in RCTs (1,7–11) have long emphasized that such analyses are at great risk for both false positive and false negative findings. An additional limitation of this approach is that individuals simultaneously belong to multiple subgroups that may vary in whether or how they appear to benefit. Thus, guidelines consistently recommend limiting the number of subgroups studied to those with prior evidence or strong biologic or clinical rationale for HTE and using caution in interpreting or applying findings to clinical practice.

The emergence of precision medicine (12) and patient-centered outcomes research (13) heightens interest in identifying important HTE. In 2020, an expert panel funded by the Patient-Centered Outcomes Research Institute (PCORI) published The Predictive Approaches to Treatment Heterogeneity (PATH) Statement (14,15), which described predictive modeling approaches that incorporate multiple patient attributes simultaneously to identify HTE and predict individualized treatment effects. The Statement pointed out that HTE, sometimes referred to as treatment effect modification and tested as a statistical interaction, should be sought on both the absolute scale (e.g., as risk differences) and on the relative scale (e.g., as ratios of risks, odds or hazards) and emphasized that heterogeneity in absolute treatment effects matters more to individual patients and clinicians making treatment decisions.

The Statement distinguished two approaches to predictive modeling. “Risk modeling” incorporates multiple baseline patient characteristics into a model predicting risk for the RCT’s outcome (usually the primary outcome). In a second step, both absolute and relative treatment effects are examined across pre-specified strata (e.g., quarters) of predicted risk. (16) In the second approach, “effect modeling”, a model is developed within the RCT data to directly estimate individual treatment effects by including treatment, multiple covariates and interactions of treatment with one or more covariates. Both regression methods and more flexible, non-parametric, data-driven machine-learning algorithms (e.g., 17–20) have been used in effect modeling.

The Statement recommended risk modeling whenever an RCT demonstrates an overall treatment effect. Risk of study outcomes varies substantially in most RCT populations when participants are stratified using a multivariable model. Assuming that the relative treatment effect is homogeneous in the population, the absolute benefit would be expected to increase as predicted risk increases, (21–22) This mathematical relationship has been called “risk magnification.” (23) and is often referenced in the evidence-based medicine literature (10,24,25) and is implicit in clinical guidelines that reserve treatments with high costs or potential adverse effects for those with higher baseline risk (26,27) The Statement encouraged use of validated external models for predicting risk if available, but indicated that in their absence, models can be developed within the RCT population, using baseline covariates and observed study outcomes and including both study arms.

Although effect modeling theoretically permits a more robust examination of possible HTE, the Statement emphasized the vulnerability of these approaches to overfitting, (28) and recommended that effect modeling be used only when there are a small number of previously established effect modifiers. It also encouraged use of methods to reduce risks of “over-fitting” of data as well as validation of effect model findings in external datasets when possible.

We conducted a scoping review (29) to assess the impact of the PATH Statement in terms of the frequency with which predictive modeling analyses of RCT data have appeared and cited the PATH Statement since its publication, the consistency of analyses with Statement consensus criteria (Supplement Table 2), and the credibility and clinical importance of claimed HTE. We applied criteria adapted from the Instrument to assess Credibility of Effect Modification Analyses (ICEMAN) (11) to assess credibility, and when HTE was found to be credible, we assessed “clinical importance,” using the Statement definition of “variation across patients in the absolute treatment effect sufficient to span clinically-defined decision thresholds, supporting differing treatment recommendations for patient subgroups.”

METHODS:

Identification of Reports for Inclusion.

Using the Cited By functions in PubMed, Google Scholar, Web of Science and the SCOPUS database (Supplement Table 1A), we identified reports that appeared between January 7, 2020 and July 24, 2024, cited the Statement and presented multivariable predictive modeling in RCT data to identify HTE. We included non-peer-reviewed reports from pre-print archives and dissertations on institutional websites.

Among 312 citations identified (Figure 1), 83 (30–112) involved analyses of data from RCTs. Sixteen (30–45) were excluded because they did not examine HTE using multivariable predictive models. Supplement Table 1B presents reasons for exclusion. Three analyses were represented by two reports each (46–51) and one report (110) presented predictive models from two distinct trials in different clinical areas, leaving 65 reports analyzing data from 162 RCTs.

Figure 1.

Flow Diagram for identification and screening of all reports citing the PATH Statement and for exclusion of reports not meeting study criteria for presenting a predictive model of individual treatment effects from RCT data. Abbreviations: RCT: randomized controlled trial; IPDMA: independent patient data meta-analysis.

*Details of these 16 studies are presented in Supplement Table 1b

Review of Predictive Model Reports.

Variables collected and coding instructions are presented in Supplement Tables 3 and 4, respectively. All details of analytic strategies and findings of HTE were doubly reviewed by the first author and one co-author. A “learning” set of six reports was reviewed and discussed by all co-authors. Thereafter, co-reviewers discussed and resolved initial disagreements.

Review classified each report as risk modeling, effect modeling or both and further classified effect models into those based primarily on regression methods (e.g., ordinary least squares, logistic, proportional hazards, Bayesian regression methods) and those using more flexible data-driven machine-learning algorithms. For both risk and effect modeling analyses, we noted whether authors reported having found HTE on either absolute or relative scales and whether results of statistical testing for HTE were presented. We included statistical tests for treatment-covariate interactions from regression models, direct contrasts of treatment effects across subgroups, reporting of confidence intervals for subgroup treatment effect estimates, and overall tests of the null hypothesis of homogeneity of treatment effect in machine-learning algorithms.

We determined whether the performance of final models for predicting individual or subgroup treatment effects was validated in datasets external to the derivation population, including validations conducted in entirely distinct RCTs, those conducted in pre-specified, non-random subsets of the original RCT population (e.g., subsets selected on bases of geography (trial sites) or time of enrollment), and those conducted in large observational cohorts.

Assessment of Credible and Clinically Important HTE.

To assess credibility of claimed HTE, on either absolute or relative scales, we adapted four of the five ICEMAN criteria for RCTs (11,113). Detailed description and scoring guidance for each criterion and the overall credibility score are detailed in Supplement Tables 7A and 7B. Although ICEMAN criteria were originally developed for evaluating treatment effect modification by single covariates, four apply readily to predictive modeling with multiple covariates. These include 1) Did the authors test only a small number of interactions; 2) Was possible effect modification by each covariate supported by prior evidence; 3) If the covariate is a continuous variable, were arbitrary, data-driven cut points avoided; and 4) Does a statistical test for interaction suggest that chance is an unlikely explanation of the apparent HTE? The fifth criterion, whether the direction of interaction was hypothesized in advance, is not applicable to predictive modeling, given that multiple covariates and potentially complex interactions are evaluated simultaneously. No single criterion, including that of statistical testing, is treated as either sufficient or necessary for establishing overall credibility. Overall credibility scores range from 1 to 4 (very low, low, moderate, or high credibility).

Because risk models involve a single effect modifier (the baseline risk score) with strong prior theoretical and empirical support (21) for HTE, at least on the absolute scale; and because risk modeling either reports pre-specified risk score cut-points or treats risk as a continuous variable, risk models can be expected to score well when ICEMAN criteria are applied. Most effect models tested multiple potential treatment-covariate interactions, often with little prior evidence and therefore tended to score poorly with application of adapted ICEMAN criteria. However, we gave considerable weight to external validation of effect model performance in another population. External validation essentially tests for HTE across a single vector, the “effect score” or predicted individual treatment effect, much as risk modeling tests for HTE across the risk score. (114) Overall credibility usually rose to “moderate” if models performed well in external validation, even if credibility of derivation analyses would have been scored as very low.

We classified all reports scored as at least moderate overall credibility as “credible” and assessed findings for clinical importance. Per the PATH Statement, clinical importance is based exclusively on the size and direction of observed differences in absolute treatment effects between subgroups and whether these differences appear sufficient to support differing treatment recommendations. An additional consideration was whether findings for all outcomes studied, including adverse effects of treatment, were consistent in supporting the same treatment choice.

Results

General Description.

Predictive models of HTE appeared with increasing frequency each year following publication of the PATH Statement (Table 1). Among the 65 reports (46–112), we identified 31 risk modeling and 41 effect modeling analyses. Seven reports (60,84,86,91,105,109,111) presented both risk and effect modeling analyses. Most effect modeling reports examined large numbers of potential interactions, and the majority employed data-driven, non-parametric analytic methods.

Table 1.

Characteristics of the 70 analyses (65^* reports) of predictive models for possible heterogeneity of treatment effect

	Risk Modeling Analyses (n=31)	Effect Modeling Analyses (n=41)
Publication Year
2020	2	3
2021	4	8
2022	7	7
2023	9	15
2024 (through July 5)	9	8
Initial Publication Status
Peer-reviewed	24	34
Pre-print archive†	3	7
Dissertation	4	1
Data Source
Re-analyses of Single RCT	16	26
Initial Analyses of RCT(s)	2	0
IPDMA of 2 or more RCTs	13	15
Comparative Effectiveness Research? ‡
Yes	13	25
No	18	16
Overall RCT Results §
Null (no overall difference)	6	12
Modest effect	7	13
Strong effect	18	16
Sample Size - HTE Analyses
Range (min - max)	574 – 330,460	200 – 26,877
Median	1,907	2,294
Interquartile Range (25th – 75th)	999 – 3,740	1,250 – 8,828
For Risk Models Only (n=30) Type of Risk Model Used
External model applied	14
Internal model developed	17	16
For Effect Models Only (n=41) Number of Potential Interactions Tested
Range (min - max)	-	2 – 58
Median	-	17
Interquartile Range (25th – 75th)	-	10 – 22
Type(s) of Effect Models Employed Regression Methods (n=13) ‖		-
“Conventional” Regression¶	-	6
Regression with penalization to avoid overfitting**	-	7
Machine-Learning Algorithms (n=28) ††
Causal Forests	-	13
Other tree-based Methods‡‡	-	10
Meta-Learner Methods	-	9

^*Seven reports (refs ⁶⁰,⁸⁴,⁸⁶,⁹¹,¹⁰⁵,¹⁰⁹,¹¹¹) presented both risk and effect modeling analyses.

^†Seven of 10 reports (refs ⁴⁹,⁷⁰,⁷³,⁷⁶,⁸¹,⁸⁸,¹¹¹) originally identified through pre-print archives have subsequently been published in peer-reviewed journals. Three (refs ⁵⁸,⁸⁹,¹⁰²) have not appeared as peer-reviewed publications.

^‡Comparative effectiveness was defined as comparison of two or more alternative, active interventions. (for more detail, see coding instructions Supplement Table 4).

^§Null: overall treatment effect does not differ significantly from zero; Modest effect: estimated relative risk reduction is ≤ 20%; Strong effect: estimated relative risk reduction >20%. For continuous outcomes, a standardized mean difference significantly >0, but ≤ 0.8 was considered moderate; and a standardized mean difference >0.8 was considered strong. In one report (ref ⁷⁸), an overall trial effect could not be determined.

^‖These include only reports based solely on regression models without additional analyses employing more flexible machine-learning algorithms.

^¶“Conventional regression” includes linear, logistic and Cox proportional hazards regression models that did not employ either penalization / regularization methods (e.g., LASSO, penalized ridge regression, elastic net) or cross-validation methods explicitly intended to reduce over-fitting.

^**These include regression models that incorporated penalization /regularization methods or cross-validation methods explicitly intended to reduce over-fitting (or both)

^††Because some reports featured more than one machine-learning algorithm, the rows below are not mutually exclusive

^‡‡Includes examples of model-based recursive partitioning, gradient-boosted regression trees, random forest, and Bayesian additive regression trees (BART)

Reviewer Agreement.

Excluding six reports (presenting six effect models and one rik model) used for training reviewers, initial between-reviewer disagreement rates for 19 doubly-reviewed items ranged from 0 to 47%, with an overall average of 10.1% (details, Supplement Table 5). Initial disagreement was greater for assessments of the credibility and clinical importance of claimed HTE. Possibly because assessment of credibility and clinical importance were added near the end of data collection and without additional training, initial disagreement was more common for these items, although generally this was easily resolved upon discussion.

Risk Models.

Concordance with eight Statement criteria related to risk modeling was above 60% for 5 of 8 (Supplement Table 3). Only 52% (25) of the 48 reports with positive overall findings included a risk model. Fourteen of 31 risk modeling analyses used an external prediction model. Half of reports presented risk model scores by treatment arm. All but one presented absolute treatment effects by level of risk and most reported relative treatment effects as well.

Study authors claimed findings of HTE in 23 of 31 risk modeling analyses (Figure 2, Supplement Table 8). For 13 reports, HTE was found on the absolute but not the relative scale (i.e., risk magnification). For the remaining 10, relative treatment effects also appeared to vary across levels of baseline risk. In five, (49,61,79,91,96) relative treatment effects were greater in individuals at higher risk for experiencing trial outcomes. Relative treatment benefit was confined to individuals in the middle of the risk distribution in three reports (47,65,69) or to those at lowest risk in two (59,84).

Figure 2.

Adjudicated results of review of all eligible reports for type of predictive modeling (risk or effect), for claims by authors of heterogeneity of treatment effects (HTE), for credibility of HTE (using adapted ICEMAN criteria), and for clinical importance of HTE found to be credible.

Effect Models.

Concordance was low for 3 of the 6 Statement criteria among effect modeling reports (Supplement Table 4). Only 6 of 41 (52,60,84,86,108,109) restricted analyses to small numbers of covariates with strong prior evidence for effect modification. Most explored many candidate effect modifiers with little prior evidence. Only nine (51,52,53,55,70,80,92,93,108) applied effect model findings to external datasets for validation. Authors claimed HTE in 31 of the 41 effect model reports (Figure 2, Supplement Table 8). Thirty presented evidence for absolute treatment effect differences across subgroups; nine presented evidence for relative effect differences. In 14 of the 31, authors heeded recommendations to report model performance metrics that evaluate prediction of individual treatment effects rather than prediction of risk for the outcome.

Assessment for Credibility of HTE.

Most reports, whether of risk or effect modeling, claimed to have identified HTE (Figure 2), but as expected, risk models were more likely to be scored as credible when ICEMAN criteria were applied. Detailed scores for the 51 reports claiming HTE are given in Supplement Table 8. Twelve of 13 risk modeling analyses claiming risk magnification and eight of ten that found HTE on a relative scale were scored as credible. By contrast, findings of HTE in 31 effect modeling reports were judged credible in only 10. Most effect modeling reports explored many variables with little prior evidence for effect modification. Those employing data-driven machine-learning algorithms also allowed the data to guide cut-point selection for continuous covariates. Among the 10 effect modeling reports judged to present credible HTE, nine (51,52,55,70,80,92,93,108,111) validated model predictions of individual treatment effects in independent cohorts, usually another RCT. Three of the ten (52,99,108) also met ICEMAN criteria (and PATH recommendations) by restricting analyses to a very small number of candidate effect modifiers with strong prior evidence.

Assessment for Clinically Important HTE.

Reviewers judged findings from 24 of 30 reports with credible HTE to be clinically important (Figure 2). Details for these are 24, including rationales for classification as clinically important, are summarized in Table 2. In 19, RCT findings had suggested a moderate (n=6) or strong (n=13) benefit of one treatment vs. another. Yet, predictive modeling identified subgroups representing 5 to 67% of the trial population for whom no benefit or possible net harm would be expected from that treatment. In five, overall results suggested no benefit, but predictive modeling identified subgroups representing 25–67% of participants who did appear to benefit from one treatment vs. another. Findings of six reports with credible HTE were judged not clinically important because the heterogeneity, though credible, did not span a threshold suggesting differing treatment choices; (95,112) or because of conflicting findings across outcomes, (75,88) failure to add clinical value to previous risk-based selection strategies, (79) or concurrence with authors on the need for additional investigation, possibly testing additional effect modifiers. (55)

Table 2.

Studies found to have credible and clinically important heterogeneity of treatment effects (HTE)

Discussion

Evidence-based medicine has historically encouraged clinicians and patients to rely on average treatment effects from RCTs to support individual decision-making, (23,115) despite recognizing limitations with this approach. In the four and a half years following publication of the PATH Statement, a steadily growing number of publications across a wide range of clinical areas has employed predictive modeling to examine possible HTE in RCT results. Among the 65 reports we identified, more than a third found HTE that was both credible and clinically important, suggesting that patients and clinicians could often do better than relying solely on average effects.

Consistent with the PATH Statement rationale and ICEMAN criteria, risk modeling was more likely than effect modeling to produce findings of credible HTE because of its relative simplicity. Importantly, HTE was not always confined to the absolute scale (risk magnification). In nine reports, credible and important HTE was identified for relative as well as absolute treatment effects across baseline risk. In five, (49,54,61,79,96) relative as well as absolute effects were greater for persons at higher predicted risk. In two, (59,84) persons at low risk benefited while high-risk patients may have been harmed by the same treatment; and in two, (47,69) maximal benefit was found for those in the mid-range of risk. This U-shaped, or “sweet spot,” pattern (47) is clinically intuitive and has also been observed elsewhere. (116)

These findings demonstrate that simple assumptions of risk magnification are not well-founded and illustrate the potential value of routine risk modeling of RCT results. They may offer clinical insights about specific effect modifiers. Traits incorporated into risk scores because they are strong predictors of study outcomes may sometimes also be treatment effect modifiers, either directly or as proxies for unmeasured attributes. In an RCT comparing therapeutic-dose heparin with usual thromboprophylaxis for patients hospitalized with COVID-19, (84) respiratory status at baseline was the most potent predictor of clinical outcomes but was also found to be a strong modifier of heparin treatment effect in initial subgroup analyses. Only patients with better baseline respiratory status benefited from heparin. When this trait was incorporated into a risk model, only patients with lower risk scores appeared to benefit. In three RCTs (47,59,69) where incidence of study outcomes was particularly high (range 27–61%), no benefit was observed in the highest stratum of predicted risk. For such extremely high-risk individuals, risk prediction models likely included attributes reflecting irreversible disease or competing causes of the outcome that would make treatment futile.

The increasing use of effect modeling, particularly exploratory machine-learning approaches, suggests enthusiasm for moving beyond risk stratification to more flexible estimation of individualized treatment effects. Several authors (58,76,88) expressed concerns that although risk scores can create patient subgroups well-matched on risk, subgroup members may still be heterogenous for the specific characteristics that contributed to their risk scores and therefore potentially heterogeneous in their responses to treatment. Comparative advantages and disadvantages of risk versus effect modeling remain incompletely understood. Seven reports (57,81,83,91,105, 109,111) presented both risk and effect modeling of the same RCT data. Effect modeling added new insights to risk modeling in only one instance. (111) In this IPDMA of eight RCTs of corticosteroids for community-acquired pneumonia, effect modeling identified a single powerful relative treatment effect modifier, c-reactive protein, a variable that was not included in the external risk model. Although both models found credible HTE, the effect model performed better in external validation. More generally, the existence of strong effect modifiers, whether known in advance or not, is a likely pre-requisite for finding that an effect model improves on risk modeling.

Nearly all effect modeling reports followed Statement suggestions to use shrinkage methods and internal validation strategies to reduce over-fitting (Supplement Table 4). Nevertheless, inconsistencies in several reports illustrate the persistent challenges of false positive signals of HTE in exploratory effect modeling and underscore the need for external validation. For example, two reports (55,82) applied causal forest algorithms to data from the SPRINT and Action to Control Cardiovascular Risk in Diabetes (ACCORD) trials evaluating intensive systolic blood pressure control. One (82) found evidence of HTE, the other did not. In two reports (60,78) from a trial of dabigatran vs. warfarin for stroke prevention in atrial fibrillation, one (60) suggested interactions of three covariates with treatment choice and significant HTE; the second, using four machine-learning algorithms applied to the same RCT data, found no evidence for HTE. Two reports (62,110) compared results of multiple machine-learning algorithms, finding inconsistent evidence for HTE between algorithms and even within algorithms when random initiation seeds were altered.(62) Two reports (92,111) compared regression-based methods with machine-learning algorithms in effect modeling, both finding that regression models performed better in external validations.

The inconsistency of machine-learning approaches to effect modeling has also been addressed by others (117). Most effect models reviewed here explored large numbers of candidate treatment interactions despite relatively modest numbers of outcome events. Specification of best practices in this emerging area is beyond the scope of the present review, and much remains to be learned concerning sample size needs and optimal approaches to internal validation.

Meanwhile, the value of external validation illustrated here highlights the importance of making data from completed RCTs available and of creating large, well-characterized real-world cohorts with treatment and covariate data for validation and extending HTE findings to populations with differing patterns of risk and treatment. (118) The same cohorts could support development of new, more representative risk prediction models.

Given the frequent utility of risk modeling, sponsors of new RCTs should consider in advance whether appropriate external risk models exist and plan for collection of baseline data needed for estimating individual risk. Within the past decade, editorial guidelines for reporting positive RCTs findings have come to require presentation of absolute as well as relative measures of overall treatment effect because of their greater relevance in clinical decision-making. (119–121) We suggest that reports of positive RCTs could be further enhanced by requiring that treatment effects, in both relative and absolute terms, be presented in relation to baseline risk. Ultimately, it will remain critical to demonstrate the safety and effectiveness of any predictive model when employed for personalizing treatment choices in real world populations.

Limitations

The search strategy would not have captured predictive modeling reports that did not cite the PATH Statement. We conducted a broad search for such reports using title/abstract words “randomized” plus “heterogeneity of treatment effects” for the same time period. This strategy yielded 60 publications, but only seven were predictive models in RCTs, 3 of which were included in our review. That this search found so few of the 65 reports we reviewed indicates that it is not a parallel approach for finding other predictive models. Nevertheless, we believe that the reports citing the PATH Statement offer a highly relevant population for assessing its influence. Even with the ICEMAN and PATH criteria for assessing credibility and clinical importance of HTE, some subjectivity remains. The close association of two authors (DK, JS) with production of the PATH Statement should be kept in mind.

Conclusions

The PATH Statement appears to be influencing research practice. Although effect modeling holds promise for predicting individualized treatment effects, the need for external validation is a constraint. Risk modeling provides a more straightforward initial approach when overall trial findings are positive and often identifies clinically important HTE.

Funding Statement

Drs. Selby and Maas and Mr. Fireman report no funding related to work performed on this publication. Dr. Kent was funded by a National Institutes of Health (NIH)/National Center for Advancing Translational Sciences (NCATS) grant (UM1TR004398-01). Dr. Selby previously served as the Executive Director of the Patient-Centered Outcomes Research Institute (PCORI). The views and findings presented in this publication are solely the responsibility of the authors and are not presented on behalf of or as the views of PCORI.