Patient‐Centered Rocket Science: Accurate but Imprecise

The randomized clinical trial has served as a long‐standing foundation of medical innovation. The premise is simple: does treatment “A” work better than “B” in a similar group of patients? In the early era of cardiovascular clinical trials, it was sufficient, if incomplete, to compare only the changes to the number of terminal events in each arm of the trial. As cardiovascular outcomes have improved through the development of better treatment strategies, determining appropriate end points for clinical trials that properly incorporate a variety of clinical outcomes has become increasingly complex. In general, the overall reduction in death events has created the impetus to look at other clinical outcomes. These outcomes provide better use of available information, sometimes at the expense of interpretability for the patient and provider.

Many trials, including ROCKET AF (Rivaroxaban Once Daily Oral Direct Factor Xa Inhibition Compared With Vitamin K Antagonism for Prevention of Stroke and Embolism Trial in Atrial Fibrillation), ¹ have adopted a variety of clinical events and proxy outcomes, such as biomarkers, to increase the event capture and simultaneously reduce the number of patients required to demonstrate a difference. These are commonly combined into a time‐to‐first event composite, or in some pairwise comparison. It is important to understand that the components of a composite event will have differences in the asperity of the included outcomes, especially as the number of outcomes is increased. ²

With multiple types of events included, there are differences in the timing of events that may inherently cause the analysis to favor less severe but earlier‐occurring events. There is also no expectation of all event types to respond to a treatment uniformly. To further complicate the interpretation, there are no “standards” available, so many studies have differing operational definitions of outcomes including what components come into major adverse cardiovascular events. ³

The result of the analysis of a composite outcome can be difficult to interpret for the clinician and the patient. For example, suppose a new treatment shows improvement on the composite but has a numerically higher death rate. There have also been a number of trials that use proxy measures and show no pragmatic benefit to the patients. ⁴ Measures that use the best available information, are patient centered, and allow direct comparison of treatments are important in the pursuit of translational medicine.

Understanding the balance between a variety of possible clinical and safety events for a new treatment can be difficult for the seasoned clinician, let alone the inexperienced patient. ⁵ As such, making informed decisions on treatments while balancing relative risk improvements in outcomes with low absolute risk, is a big challenge. Simple, meaningful metrics that are more interpretable and beneficial to all should be the goal. As patients and treatments become more complicated, so will the set of highly specific outcomes and analyses.

In contrast, we can look at measurements like the Charlson comorbidity index, which was developed some time ago and has become ubiquitous, as while it is messy, it captures the overall disease burden reasonably well by being accurate but imprecise. The computation assigns an overall score to underlying complexity. The different comorbidities are captured and summarized into a single, interpretable value; someone with a Charlson score of 6 is more complicated than someone with a Charlson score of 3, even if we don't know the details of precisely why. Despite the lack of detail, it allows a quick and accurate summary of complexity.

The reanalysis of the original ROCKET AF study by Harrington et al in this issue of the Journal of the American Heart Association (JAHA )⁶ proposes the use of a similar simple, accurate, but in many ways imprecise metric as an outcome measure, days alive and out of hospital (DAOH), for atrial fibrillation (AF) trials. ⁷ The use of this metric negates the complexity of differentiating end point choices and metrics and concentrates on patients being alive and not in acute care. The advantage of this is that more severe nonfatal outcomes are associated with longer hospitalization. This is of interest to the patient, provider, and health system. Similarly, by monitoring over an extended time, multiple events and rehospitalizations are accounted for as well.

The DAOH metric forgoes some of the detailed precision in exchange for accuracy. A treatment with an average DAOH of 300 is intuitively worse than one with a DAOH of 350, even without the specific reasons. This difference could be driven by any number of factors, including better survival, shorter stay in the hospital, and fewer repeat hospitalizations, all of which fall toward patient preference. Combining these various sources of information to a single measurement with an interpretable effect size eludes many of the time‐to‐event methods. Given the relative simplicity of the metric, it becomes one that can be derived or evaluated from most routinely collected and even health services (administrative) data. This allows comparison of the trial data with other sources. With a view toward value‐based medicine, this metric also provides direct comparison between multiple disease states, allowing comparison between survival and resource usage between diverse patient populations, in a way that an economic evaluation cannot.

On the other hand, in terms of patient goals and perceptions, these measures remain crude and are insufficiently granular to help guide therapeutic decisions for the individual. For example, following a specific treatment or intervention, a patient may be at home (not hospitalized or in a nursing home or other nonhospital facility requiring medical supervision) but more likely to experience significant and persistent declines in self‐care, level of independent daily activities, and cognitive function. For many patients, the risk of such potential outcomes may be crucial in their decision.

For other AF‐related treatment decisions, such as catheter ablation versus medical therapy, attempts to incorporate the patient perspective have focused on administration of standardized structured multi‐item quality‐of‐life instruments, both general and AF specific. ⁸ In clinical trials of AF ablation, treatment group comparisons indicate statistically significant superiority for ablation ⁹ , ¹⁰ and often correspond to changes in other outcome end points (symptom frequency and severity, ECG‐documented atrial arrhythmia recurrence). However, the appropriate cut points to declare a significant change in quality‐of‐life scores (either individually or in aggregate) remains problematic. It is unclear which items most closely reflect patient satisfaction with outcomes relative to baseline status before intervention. Quality‐of‐life end points based on structured instruments also depend on the specific instrument employed, limiting generalizability. Better integrated metrics of patient perceived outcomes and treatment satisfaction need to be developed, tested, and validated in large AF populations and diverse treatment strategies.

When end points such as death or stroke occur infrequently during the specified follow‐up period, as in the reanalysis of ROCKET AF, DAOH did not provide new or useful insights. In contrast, when adverse events are more frequent, either because of the inherent risk of the intervention or the greater susceptibility of subpopulations to adverse events, this approach might be more useful, particularly if combined with patient perceived outcome measures and satisfaction. The concept of frailty has been introduced recently as an integrated measure of one such vulnerable population. Frailty has been identified in 30% of the AF population and has been associated with a greater risk of death, stroke, and bleeding. ¹¹ It is also associated with greater risk of adverse events early after catheter ablation, ¹² though its longer‐term impact is unknown. Focus on frail or other vulnerable subgroups, and the development and refinement of tools to address “at home” disability could be incorporated into the design of future clinical trials and observational registries.

An important component of the work of Harrington et al⁶ is the review of an existing study with another method to see if the conclusions hold if analyzed in a different but patient‐oriented way. Of course, it is not definitive as it is a secondary analysis, but it should also give the reader pause if there is only 1 specific lens through which “A” outperforms “B.” Harrington et al suggest that the DAOH metric may be insensitive, given that some outcome differences were seen in the original trial. Perhaps the original metrics were not sufficiently meaningful to capture the impact on the patient.

Disclosures

None.