PROBAST-2019 was used as the starting point.19 20 It included 20 signalling questions across four domains: participants, predictors, outcome, and analysis. We anticipated that these domains and signalling questions would already largely apply to AI/machine learning based prediction model studies as well as regression based approaches.

In October 2020, we conducted a survey among more than 50 authors who had participated in an earlier living review on prediction models for covid-19.5 Each of these authors had applied PROBAST-2019 on multiple occasions, including on papers with AI/machine learning based prediction models. The survey asked about their experiences of PROBAST-2019, its applicability to AI/machine learning based prediction model studies, suggestions for improvement or changes in wording of existing domains and signalling questions, and whether domains or signalling questions needed to be deleted or added. All suggestions and recommendations were harmonised into an initial list of 26 candidate signalling questions distributed over four risk of bias domains, then labelled as participants and data sources, predictors, outcomes, and analysis. The domains of participants and data sources, predictors, and outcomes each included an applicability subdomain. This initial list served as the basis for the first round of the online large scale Delphi survey.

The PROBAST+AI working group identified participants from authors of relevant publications, through social media (eg, X, formerly Twitter), and based on personal recommendations of Delphi participants (snowballing recruitment). Participants were recruited covering all key stakeholder groups (table 1) from a range of settings (eg, university, primary care, hospital, biomedical journal, patient and non-profit organisations, and for-profit organisations), and bearing in mind geographical and other aspects of diversity.

We designed and shared the Delphi surveys electronically using the REDCap online platform (www.projectredcap.org), and later the Castor online platform (www.castoredc.com). Owing to a change within University Medical Centre Utrecht’s system, we needed to switch data capturing programs after the first round.

In the first round of the Delphi process, we asked for each of the initial 26 candidate items to be rated. For each signalling question, we asked participants whether they strongly disagreed, disagreed, neither agreed nor disagreed, agreed, or strongly agreed to its inclusion in the development of PROBAST+AI. Participants were also invited to comment on any domain or signalling question, and to suggest new items. For an item to be included, a level of agreement (response to strongly agree or agree) of ≥80% had to be achieved. For items that did not surpass the 80% threshold but were identified as essential, we checked whether a reformulation of the signalling question—that is, based on received comments and expert opinion—led to a higher result in the next round. CAN, JAAD, and TK analysed the narrative responses, with agreement from KGMM, MvS, JBR, LH, GSC, RDR, and PD. After each round, we presented the aggregated quantitative results for agreement to the participants of the next round. Responses were anonymous.

A hybrid consensus meeting chaired by KGMM, LH, JAAD, PD, RDR, and MvS was held on 17 October 2023. Of 29 participants invited to the consensus meeting, 26 attended (see supplementary figure 1). Participants were identified to ensure a balanced representation of the key stakeholder groups and geographical and other aspects of diversity. In preparation for the final consensus meeting, KGMM, JAAD, TK, CAN, PD, LH, JBR, RDR, GSC, and MvS developed a pre-final PROBAST+AI based on input from the Delphi survey rounds in several hybrid meetings. Ten days before the consensus meeting, participants were emailed a document containing a brief overview of PROBAST+AI, the format of and instructions for the consensus meeting, a summary of the aggregated responses from the last Delphi survey round, and the draft PROBAST+AI. The tool shared with the consensus meeting participants included four domains and 16 signalling questions for quality assessment of model development, and the same four domains and 18 signalling questions for risk of bias assessment of the model evaluation. Given the high endorsement for many items in the third round of the Delphi survey, we selected only a subset of 10 signalling questions for plenary discussion and voting during the consensus meeting. The 10 items had either undergone rewording after the third round or were new items introduced after that round.

In accordance with PROBAST-2019,19 20 when using PROBAST+AI we advise specifying the purpose of the assessed prediction model. For this we recommend defining the PICOTS (Population, Index model, Comparator model, Outcome, Timing, Setting, and intended use of the prediction model) criteria as provided by the guidance of the Cochrane Prognosis Methods group (https://methods.cochrane.org/prognosis/) and described in CHARMS (checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies61). Defining the PICOTS directly indicates the aim of the assessment or review of the prediction models.

Prediction model studies can include model development or model evaluation, or both.19 20 38 PROBAST+AI includes different signalling questions depending on the type of prediction model study. Therefore, we recommend that assessors and reviewers state whether they address model development only or model evaluation only, or both. Furthermore, model evaluation distinguishes between estimation of the model’s apparent performance, the internal validation performance, and the external validation performance (see box 1 for descriptions). When a publication focuses on updating a previously developed model, such as adding one or more new predictors, the model development part of PROBAST+AI should (also) be used. When a publication focuses on evaluating the performance of an existing model in other (external) participant data, only the model evaluation part should be used.

This step aims to identify areas where concerns about quality and risk of bias might be introduced in the prediction model study, or where concerns about applicability might exist.19 20 For each domain the quality (for model development) and risk of bias (for model evaluation) assessment comprises four sections (see table 2, also see supplementary tables 3 and 4 for detailed guidance on use): Section 1—general information from the study or model to support answering the signalling questions of that domain; section 2—answering the signalling questions; section 3—a judgment of concerns about quality (for model development) or risk of bias (for model evaluation) per domain; and section 4—rationale for the overall quality judgment (separately for the development) or risk of bias judgment (separately for the evaluation) of the prediction model.

As with PROBAST-2019,19 20 assessors can record any additional information used to answer the signalling questions in the box related to rationale for any judgment. Signalling questions are answered as yes, probably yes, no, probably no, no information, or, when appropriate, not applicable. Quality concerns (for model development) are judged as low, high, or unclear, and risk of bias (for model evaluation) is judged as low, high, or unclear. All signalling questions are phrased such that yes answers or probably yes answers indicate low concern for quality (ie, high quality) or low risk of bias. Any signalling question answered as no or probably no flags the potential for quality concerns or bias. Subsequently, assessors need to use their judgment to determine based on these answers whether the entire domain should be rated as low, high, or unclear quality concern (for the model development) or low, high, or unclear risk of bias (for the model evaluation). If a signalling question is answered with no, it does not automatically result in a high concern for quality or a risk of bias rating of the entire domain. The no information category should be used only when reported information is insufficient to permit a judgment. The not applicable category may be available for items that are not applicable for certain types of prediction models or situations. When the rationale for the overall domain judgment is recorded separately for the model development and for the model evaluation, the domain’s quality or risk of bias rating will be more transparent. This can also facilitate discussion among different reviewers or assessors who complete assessments independently.

The first three domains, in accordance with PROBAST-2019,19 20 also include assessment of concerns about the applicability of the prediction model (low, high, unclear) to the review question or to the assessors’ intended use of the assessed model. Applicability is defined as any concern that the included data of the participants and setting (domain 1); or the definition, preprocessing, assessment, or timing of assessment of the predictors (domain 2); or the outcome definition, assessment, or timing of assessment (domain 3), do not match the prediction model review question or the assessor’s intended use of the model. Accordingly, applicability refers to either applicability of a study to the question of the reviewer (for example, when one conducts a systematic review of prediction model studies) or whether a particular developed or evaluated model is indeed useful for the intended use or purpose of the assessor. For example, a model can have a low concern for quality if data and participant selection were appropriate for the modeller’s intended purpose, but a high concern for applicability if either does not match how the reviewer intends to use that model.

The final step of the PROBAST+AI tool is similar to that in PROBAST-2019,19 20 in which the four domain ratings are combined into an overall judgment on the quality and applicability of the model for model development and separately on the risk of bias and applicability for model evaluation. This overall judgment is scored as either low, high, or unclear concern of the quality; low, high, or unclear risk of bias; and low, high, or unclear concern of the applicability. Step 4 in supplementary table 3 provides guidance on how to make an overall judgment on quality, risk of bias, and applicability, as well as the original PROBAST-2019 guidance.19 20

For example, a high overall concern of applicability (for both model development and model evaluation) indicates a limited or poor applicability of the scored model for the review question or the assessor’s intended use of the model, whereas a low overall concern of applicability indicates a good applicability of the scored model.19 20 Similarly, for the quality judgment of model development, an overall high concern indicates a low quality of the model development (production) process, whereas an overall low concern indicates a high quality of the model development process. And similarly for the risk of bias judgment of the model evaluation, a low risk of bias indicates that the reported estimates for model performance are valid (unbiased), whereas a high risk of bias indicates the performance estimates might systematically differ from the true model performance. For all three (ie, judgment of applicability, quality, or risk of bias), an unclear overall judgment indicates that reported information was insufficient to make an adequate judgment.

These overall judgments may sometimes involve changing or reclassifying an overall high concern to low concern (for quality of the model development process) or a high risk of bias to low risk of bias (for model performance estimates).19 20 For example, for the model evaluation part, if a model performance was evaluated without any external performance evaluation, domain 4 might have been scored as high risk of bias. If the model was (typically in that same study) developed (ie, fitted) on a large dataset and evaluated with some form of internal model performance validation, however, this high risk of bias might be changed to an overall low risk of bias rating, provided that the other three domains had low concern about quality and risk of bias.

We anticipate that PROBAST+AI will help all stakeholders (eg, prediction model developers and companies, researchers, editors, reviewers, healthcare professionals, patients, ethical review boards, guideline developers, and health policy organisations) who encounter prediction models in the healthcare sector to understand and appraise the quality, risk of bias, and applicability of prediction models and prediction model studies. Using PROBAST+AI to guide the design and analysis of a prediction model development study or evaluation (validation) study, or both, should help to reduce research waste while improving the accuracy, effectiveness, generalisability, and appropriate use and fairness of prediction models in any healthcare setting or domain where prediction or classification plays a role, and regardless of the data analytical modelling (ie, prevailing statistical or AI/machine learning) technique used.

This post hoc study focuses on orthostatic hypertension, but the search strategy of our original systematic review, focused on orthostatic hypotension, was described elsewhere.5 7 In brief, our review was registered in the PROSPERO registry on 28 April 2020 (CRD42020153753) and initial searches included MEDLINE/PubMed), Embase, and the Cochrane Central Register of Controlled Trials databases without language restrictions. A research librarian (CM) prepared our search strategy, which focused on hypertension, blood pressure treatment, standing blood pressure (particularly orthostatic hypotension), and randomized trials. Duplicate records were removed in EndNote, and two independent investigators (SPJ and JLC) screened abstracts with Covidence, with discrepancies adjudicated by consensus. This search was updated to include citations through 13 November 2023 (supplementary methods 1; supplementary figure A). The search ultimately entailed 1127 unique abstracts and 70 unique trials. Only one of the eligible trials was excluded owing to inability to share individual participant data. Because our original search included all trials with standing blood pressure measurements, we were able to use the outcomes of this search to examine orthostatic hypertension as a post hoc analysis.

The original systematic review was guided by the following PICO (population, intervention, comparison, outcomes) criteria.8 Population: trials of at least 500 adults (ages 18 years or older) with elevated blood pressure or hypertension (based on seated measurements). Intervention: at least six months of randomized antihypertensive drug treatment (blood pressure goal or active agent). Comparison: at least two blood pressure goals (one less than the other) or active therapy versus placebo. Outcome: orthostatic hypotension measured after randomization. Although orthostatic hypotension was the outcome of the original search, all these trials also had the relevant data for calculation of orthostatic hypertension. We excluded trials of pregnant women or children, animal experiments (non-human trials), reviews, observational studies, and studies without direct measures of orthostatic hypotension (for example, based on self-report or claims). We pooled trials together overall and by type—that is, those comparing two treatment goals (a lower versus a higher goal) or placebo controlled trials.

In addition to our systematic review above, we also attempted to contact investigators of trials of antihypertensive drug treatment in adults with elevated blood pressure or hypertension included in a recent meta-analysis focused on cardiovascular disease,9 asking about the availability of standing blood pressure measurements. This process led to the inclusion of one trial not identified through our original search.10 One trial was not able to provide us with data owing to data sharing restrictions.11 All trials identified had both pre-randomization and post-randomization orthostatic blood pressure assessments, which could be used to derive orthostatic hypertension. Risk of bias characterization was updated to reflect orthostatic hypertension as the primary outcome of this systematic review (supplementary table A).12

Similarly to our previous work, we chose a priori to examine pooled effects by categories of trial design: trials of blood pressure treatment goal (that is, one goal lower than the other goal) and trials of an active antihypertensive agent versus placebo. More intensive treatment included patients assigned a lower blood pressure treatment goal and those assigned to active antihypertensive treatment, and less intensive treatment included those assigned a higher blood pressure treatment goal and those assigned to placebo.

We determined the difference between standing minus seated blood pressure for each trial at all available visits (a visit being a clinical session whereby a participant interacted with a study team and blood pressure was measured). We defined orthostatic hypertension as standing minus seated systolic blood pressure of ≥20 mm Hg or diastolic blood pressure of ≥10 mm Hg, the definition used in SPRINT and our previous work.13 14 15 16 17 Seated blood pressure varied by trial protocol—for example, based on one measurement or based on the average of two or three measurements (sometimes with the first measurement excluded). Standing blood pressure similarly varied according to trial protocols but often included only a single measurement (see table 1). We defined standing systolic hypertension as a standing systolic blood pressure ≥140 mm Hg. This was incorporated into a recently updated definition of orthostatic hypertension—that is, a change in systolic blood pressure of ≥20 mm Hg and a standing systolic blood pressure of ≥140 mm Hg (the new consensus definition).18 19 Orthostatic hypotension was defined as standing minus seated systolic blood pressure of ≤−20 mm Hg or diastolic blood pressure of ≤−10 mm Hg.20 Baseline orthostatic hypertension or standing systolic hypertension was based on the seated and standing blood pressures measured in the visit in closest proximity and before or during the randomization visit.

We obtained the following covariate information from each trial: age, sex (women, men), race (black, non-black; this was not universally available), pre-randomization seated and standing systolic and diastolic blood pressure, baseline creatinine or estimated glomerular filtration rate or chronic kidney disease status, body mass index, diabetes status, previous stroke, and history of cardiovascular disease. We defined obesity as body mass index ≥30 and stage 3 chronic kidney disease as estimated glomerular filtration rate <60 mL/min/1.73 m² on the basis of the 2021 CKD-EPI race-free, creatinine equation21 or self-reported history of kidney disease (SHEP trial only). Differences in the definitions of diabetes, stroke, and cardiovascular disease between studies were described elsewhere.5 7

Pre-randomization and post-randomization visit data from all trials were appended into a single analytic dataset before pooled analyses. Analyses were restricted to blood pressure, body mass index, and estimated glomerular filtration rate measures between the 0.01st and 99.99th centiles of all measurements (baseline and follow-up) from all nine trials to account for biologically implausible outliers (particularly relevant for stratified analyses; see supplementary table B for values corresponding to these thresholds and supplementary table C for values corresponding to 0.1st and 99.9th centiles). We summarized population characteristics via means and proportions overall, by orthostatic hypertension status, by trial type, and according to each study. We used kernel density plots (bandwidth 5) to visually examine the distribution of systolic and diastolic blood pressure in seated and standing positions and the difference between positions (standing minus seated) according to the pre-randomization visit and follow-up visits among participants assigned to a lower blood pressure treatment goal or active therapy and among those assigned to a higher blood pressure treatment goal or placebo across all studies. We compared characteristics between participants with and without a baseline orthostatic hypertension assessment.

We plotted the proportion of orthostatic hypertension detected during study visits grouped according to a series of time intervals: month 0/pre-randomization, after randomization to ≤1 month, >1 to ≤6 months, >6 to ≤12 months, >12 to ≤24 months, >24 to ≤36 months, >36 to ≤48 months, and >48 months. We plotted proportions overall according to assignment (that is, a lower treatment goal/active therapy or a higher treatment goal/placebo) via generalized estimating equations (Poisson family, log link, robust variance estimator, exchangeable correlation matrix) without adjustment. We tabulated the number of measurements and number of individual participants at risk, determining the proportion with orthostatic hypertension at any time during each time period. We also examined the proportions, changes in proportion, and odds of orthostatic hypertension over time, using generalized estimating equations (binomial family, logit link, robust variance estimator, exchangeable correlation matrix) adjusted for study. We used generalized estimating equations to account for repeated measurements within participants as they are able to generate valid variance estimates even when the within group correlation structure is mis-specified. Models included interaction terms to assess for differences at different time points. We also examined the relation between baseline orthostatic hypertension and follow-up orthostatic hypertension via generalized estimating equations (binomial family, logit link, robust variance estimator, exchangeable correlation matrix), using an interaction term with randomized treatment assignment to assess whether this relation differed by treatment.

In addition, we compared the effect of more intensive treatment (that is, a lower treatment goal or active therapy) versus higher treatment goals or placebo on the odds of orthostatic hypertension during follow-up visits, using generalized estimating equations (binomial family, logit link, robust variance estimator, exchangeable correlation matrix). We did these analyses for individual trials and pooled by trial type (that is, the five blood pressure treatment goal trials and the four placebo controlled trials) and overall. We repeated this as a sensitivity analysis using a Poisson family log link.

We repeated models using alternate definitions of orthostatic hypertension (described above) both for 0.01st to 99.99th centiles of blood pressure values and with truncation at 0.1st and 99.9th centiles. We also determined mean systolic or diastolic blood pressure before and after randomization and treated orthostatic change in systolic or diastolic blood pressure as a continuous outcome (these models used a normal family, identity link; by contrast, all models with alternate definitions of orthostatic hypertension as dichotomous outcome variables used a binomial family logit link).

Moreover, we did subgroup analyses examining orthostatic hypertension in the following pre-specified strata: age (≤75 or >75 years), sex (men or women), race (non-black or black), pre-randomization seated blood pressure (systolic blood pressure ≥130 mm Hg or diastolic blood pressure ≥80 mm Hg, no or yes), diabetes (no or yes), previous stroke (no or yes), stage 3 chronic kidney disease (<60 or ≥60 mL/min per 1.73 m²; in SHEP, kidney disease was self-reported), body mass index (<30 or ≥30), history of cardiovascular disease (no or yes), standing systolic blood pressure before randomization (<140 or ≥140 mm Hg), and pre-randomization orthostatic hypertension (no or yes). We used interaction terms to compare effects across strata. We repeated these analyses using the recent consensus definition for orthostatic hypertension,18 19 as well as with truncation at the 0.1st and 99.9th centiles of continuous covariates.

We did all analyses for the nine trials as well as by trial type—that is, the five trials that compared two blood pressure treatment goals or the four placebo controlled trials. We used a two stage meta-analysis with a random effects model weighted by the inverse variance in sensitivity analyses and evaluated heterogeneity between studies via the I² statistic.22 We examined heterogeneity by trial design and overall. Although our a priori intention was to pool studies, this plan was subject to evaluation of heterogeneity (both its magnitude and direction of effect).23 Small study effects were assessed via Egger’s test and funnel plots.24 We used Stata 15.1 for all statistical analyses. We considered a two tailed P value of <0.05 without adjustment for multiple comparisons to be statistically significant. A dummy dataset and analytic codes are available at https://doi.org/10.7910/DVN/RHUF9F; the code is also available in supplementary methods 2.

The original systematic review was initiated without patient or public involvement. However, a patient of SPJ with a history of orthostatic hypertension was a motivation for this work and reviewed this manuscript at the time of the revision request. This patient’s feedback was incorporated into the manuscript.

Of the 31 124 participants with 315 497 measurements contributing to this individual participant data meta-analysis, the mean age was 67.6 (standard deviation (SD) 10.4) years with 25.1% over the age of 75 years; 47.4% of participants were women (table 2). Differences between participants with and without orthostatic assessments at baseline are found in supplementary table D. Before randomization, the mean seated systolic blood pressure was 152.6 (SD 21.3) mm Hg and the mean seated diastolic blood pressure was 80.9 (11.5) mm Hg. After standing, systolic blood pressure was 152.3 (SD 21.2) mm Hg and diastolic blood pressure was 83.9 (12.1) mm Hg, with a mean postural change in systolic blood pressure of −1.9 (SD 11.4) mm Hg and in diastolic blood pressure of 2.4 (7.4) mm Hg (see supplementary table E for similar results based on all available pre-randomization visits). Before randomization, 8.7% of participants had orthostatic hypotension, 16.7% had orthostatic hypertension, and 1.9% had standing hypertension. The distribution of systolic and diastolic blood pressure became narrower with treatment and shifted to the left (supplementary figures B and C). Little change occurred in the distribution of orthostatic changes before and after treatment, regardless of assignment.

The proportion of participants with orthostatic hypertension increased initially but then decreased over time in both arms, with a greater reduction in the more intensive treatment group (fig 1). We observed a similar pattern with respect to the relative odds of orthostatic hypertension in that a significant increase occurred within the first month in the lower goal/active therapy group compared with the higher goal/placebo group (supplementary table F). However, these initial increases did not persist over time in the lower goal/active therapy group, whereas the odds in the standard group remained elevated compared with pre-randomization. Having orthostatic hypertension before randomization was associated with having orthostatic hypertension during follow-up (odds ratio 2.53, 95% confidence interval (CI) 2.39 to 2.68), and this relation did not differ by randomized treatment assignment (P for interaction=0.61).

Although all trials, except AASK, showed a lower odds of orthostatic hypertension in either the lower goal or active arms, only SHEP and SPRINT had statistically significant results (fig 2). Pooling the five trials comparing blood pressure treatment goals showed that a lower (more intensive) treatment goal was associated with lower odds of orthostatic hypertension (odds ratio 0.95, 95% CI 0.92 to 0.99). Similarly, pooling together the placebo controlled trials showed that active therapy lowered the odds of orthostatic hypertension (odds ratio 0.87, 95% CI 0.83 to 0.93). We found moderate heterogeneity across the nine trials (I²=38.0%). When we pooled the nine trials together, a lower goal or active therapy reduced the odds of orthostatic hypertension compared with a higher goal or placebo (odds ratio 0.93, 95% CI 0.90 to 0.96). A sensitivity analysis using Poisson regression did not meaningfully change our findings (prevalence ratio 0.94, 95% CI 0.91 to 0.97). In sensitivity analyses using a two stage analysis, results were similar, both overall and by trial design (supplementary figures D-F).

We examined alternative definitions of orthostatic hypertension. With nearly all definitions examined, a lower (more intensive) blood pressure treatment goal or active therapy was associated with a lower odds ratio of orthostatic hypertension compared with a higher blood pressure treatment goal or placebo (table 3). We also examined the effect of treatment on orthostatic change as a continuous outcome variable (supplementary table G). Whereas trials of blood pressure treatment goal tended to increase orthostatic change in systolic blood pressure (that is, a trend toward an increase in systolic blood pressure with standing), placebo controlled trials tended to decrease orthostatic change in systolic or diastolic blood pressure. We also repeated the principal analyses using the recent consensus definition and the systolic change alone with similar results (see supplementary figures G-I). Finally, we examined alternate definitions of orthostatic hypertension with truncated centile ranges with similar results (supplementary table H).

More intensive treatment (that is, a lower blood pressure treatment goal or active therapy) was associated with a lower odds of orthostatic hypertension than a higher blood pressure treatment goal or placebo in strata of age, sex, seated systolic blood pressure of ≥130 mm Hg or diastolic blood pressure of ≥80 mm Hg, estimated glomerular filtration rate <60 mL/min/1.73 m², body mass index, history of cardiovascular disease, pre-randomization standing systolic blood pressure of ≥140 mm Hg, and pre-randomization orthostatic hypertension (table 4). We found evidence for greater reduction in the occurrence of orthostatic hypertension among non-black participants and participants without diabetes. Results were similar when we defined orthostatic hypertension by using the consensus definition (supplementary table I) and with truncated centiles (supplementary table J).

Blood pressure is highly regulated by the autonomic nervous system in healthy adults such that blood pressure remains relatively constant across body positions. Whereas substantial focus has been directed toward falls in blood pressure on standing (that is, orthostatic hypotension), little attention has been given to orthostatic hypertension.25 26 Recent epidemiological evidence has identified increases in blood pressure on standing as potentially pathologic, linking orthostatic hypertension with a range of adverse events, including cardiovascular disease, stroke, kidney disease, and cognitive impairment.1 2 3 These long term associations with adverse health outcomes contribute to a growing belief that orthostatic hypertension may represent a form of unrecognized or masked hypertension that may require monitoring of adults in the standing position and adjusting drug treatment accordingly.27

This study confirms that orthostatic hypertension is common among adults with hypertension, but it also shows that more intensive blood pressure treatment might attenuate orthostatic hypertension over time. This is an important observation. Our previous work showed that more intensive treatment caused a net increase in the difference in blood pressure in response to standing.5 However, this study suggests that this effect may be short term and dissipate with chronic treatment. This observation, if replicated, may be important for treating clinicians, who might be dissuaded from treating hypertension because of short term orthostatic hypertension. Physiologic mechanisms for this observation are unclear. The short term increase may be secondary to autonomic over-response to valsalva, cardioacceleration after leg muscle contraction, or mobilization of excess lower extremity fluid.28 29 In the long term, we speculate that the resolution of orthostatic hypertension may be related to healthy remodeling of the vasculature with tighter blood pressure control.30 Further study of mechanisms should be evaluated in future work, particularly with repeat standing measurements. This may also be related to measurement error and additionally to more controlled blood pressure in general, such that blood pressure and fluctuation in blood pressure measurement is also lower,31 32 reducing risk for orthostatic hypertension.

Our study used a definition of orthostatic hypertension that mirrored the one for orthostatic hypotension, used in previous work.13 14 15 16 17 However, discussion is ongoing as to how orthostatic hypertension should be defined.25 Unlike with orthostatic hypotension, which focuses on changes in blood pressure alone, recent guidelines have advocated for a definition that includes both an increase in systolic blood pressure on standing and an elevated standing threshold (systolic blood pressure ≥140 mm Hg).18 This definition was proposed for a general population, not necessarily a hypertensive population. When we used this definition, the prevalence of orthostatic hypertension was substantially lower in our population. Nevertheless, the effects of treatment were even more pronounced. This is due in part to our observation that antihypertensive agents, particularly the longer acting agents used in many of the trials in our study, lower blood pressure in all body positions.5 33 As a result, standing hypertension would also be reduced with more intensive treatment. From the perspective of studying mechanisms of injury related to orthostatic hypertension, we caution against the use of this joint definition as it may make identifying and evaluating treatment response to the rise in blood pressure, which itself may be pathologic and occur below the 140 mm Hg threshold, more difficult.17 Some authors have also questioned whether diastolic blood pressure should be included in definitions of orthostatic hypertension, as diastolic blood pressure usually increases with standing.34 However, given that systolic and diastolic blood pressure are known to be correlated in both seated and standing positions, their change would be expected to correlate as well, and thus some patients with a rise in diastolic blood pressure would also have a rise in systolic blood pressure. Whether thresholds of change in systolic blood pressure or diastolic blood pressure are optimal for identifying risk with cardiovascular disease should be the focus of subsequent work. Nevertheless, the effects of treatment on orthostatic increases in systolic and diastolic blood pressure as defined in this study, using thresholds that mirrored those for orthostatic hypotension, were quite consistent.

We did not identify compelling evidence that the effects of treatment differed by demographic or medical characteristics. Although a strong interaction between black and non-black populations was apparent, this information was not uniformly collected by trials outside of the US, reducing our sample for this analysis. We also observed that effects were attenuated among adults with diabetes. Mechanisms are beyond the scope of this study, but we speculate that this lack of effects among black adults and those with diabetes may reflect known challenges in achieving blood pressure control. Additional research should probe these associations further.

Our study has limitations. Firstly, we identified only nine trials, which differed with respect to their interventions, frequency of follow-up, duration, blood pressure measurement procedures, and study populations. These differences might have influenced our results. Despite these differences, our findings were relatively consistent and our sample was sufficiently large that additional trials are not likely to alter our pooled observation. Secondly, generalizability to clinical practice may be limited owing to the strict entry criteria used by these trials, differences in prescribing regimens that might not reflect real world drug choices, and careful monitoring and drug titration protocols that may affect titration patterns. Thirdly, subgroup analyses relied on covariate definitions that were based on self-report and at times differed in definition across studies. Any resulting misclassification could weaken contrasts across subgroups, reducing our ability to detect differences. Moreover, we pre-specified our subgroups. Whether associations might differ across different categories (for example, younger age) should be examined in dedicated studies. Fourthly, we did not examine antihypertensive drug class in this study, which should be a focus of subsequent work. Fifthly, assessments of orthostatic hypertension were based on seated-to-standing protocols, which may not be interchangeable with supine-to-standing maneuvers.29 33 35 In the case of orthostatic hypotension, some authors have proposed modified thresholds for identifying adults with orthostatic hypotension (that is, a drop in systolic blood pressure of 15 mm Hg or diastolic blood pressure of 7 mm Hg).36 In our own work, we have observed a net increase in blood pressure with standing from the seated position.29 Thus, whether a seated-to-standing protocol should have a higher or lower threshold compared with supine-to-standing to establish orthostatic hypertension remains unclear. Moreover, aside from SYST-EUR, none of the major outcome trials examined in our study measured supine blood pressure.33 The manner by which starting position might underestimate or overestimate orthostatic hypertension should be the focus of future work. Sixthly, although both lower treatment goals and active therapy lowered the occurrence of orthostatic hypertension, the effect of active therapy was greater in magnitude. Whether different goals might alter the observed effect is beyond the scope of this study. Seventhly, temporal effects of treatment on orthostatic hypertension should be interpreted cautiously, as different populations contributed to visits at different time points. We attempted to include all available data to preserve the trials’ randomized contrasts. However, as some trials (for example, ACCORD) assessed orthostatic blood pressure after starting, some of these participants did not have a pre-randomization assessment, which could influence estimates of orthostatic hypertension at baseline. This would not affect the pooled contrast overall but could affect the proportion with orthostatic hypertension over time. Finally, our analysis did not examine the effects of treatment on clinical events among adults with orthostatic hypertension. This has been questioned in previous work and represents an important focus for subsequent research.14

This study has notable strengths. Firstly, this systematic review and meta-analysis represents one of the largest data collections of orthostatic hypertension in the context of drug treatment for hypertension. Secondly, doing an individual participant meta-analysis allowed for greater harmonization of data and examination of under-represented subgroups that were not feasible within individual trials. Thirdly, whereas data on orthostatic hypertension have been presented from various trials with respect to outcomes, to our knowledge this is the only patient level meta-analysis of the risk of orthostatic hypertension in treated hypertensive patients. Finally, the associations between treatment and orthostatic hypertension across trials were similar, suggesting that the effects of more intensive hypertension treatment on orthostatic hypertension are quite reproducible.

Our study potentially has clinical implications. Orthostatic hypertension has received increasing attention as a novel and distinct presentation of hypertension. This carries the suggestion that distinct pharmacologic strategies are needed for its treatment. Some authors have suggested that β blockers may be more effective for treating orthostatic hypertension by blunting the β adrenergic response to standing, and evidence also shows that peripheral α blockers may be effective.6 However, neither of these classes is preferred for initial treatment of hypertension on the basis of the experience from hypertension outcome trials.37 Nevertheless, if either is superior to other classes in reducing orthostatic hypertension, it might be of value as add-on therapy in the presence of residual orthostatic hypertension. Although more work is needed to evaluate specific drug classes with respect to orthostatic hypertension, our data provide reassurance that focusing on seated blood pressure control and treating seated hypertension among adults with hypertension can modestly reduce orthostatic hypertension. At this time, no trials are assessing whether treating standing blood pressure levels to some goal provides additional benefit to the traditional seated approach.

In this large, individual participant data meta-analysis of blood pressure treatment trials, more intensive blood pressure treatment, especially with active treatment (versus placebo), modestly reduced the occurrence of orthostatic hypertension regardless of its definition or baseline demographic and medical characteristics. Future research should examine orthostatic hypertension in relation to clinical outcomes as well as whether specific classes of antihypertensive drugs or lower treatment goals might better prevent orthostatic hypertension and its sequelae.

The new framework uses a broad definition of natural experiments. We use the term to refer to events or processes outside the control of a researcher that divide a population into exposed or unexposed groups, or groups with differing degrees of exposure. A natural experimental evaluation uses data emerging from the introduction, delivery, or withdrawal of a natural experiment to evaluate the impact of the intervention on an outcome or outcomes. Other authors have proposed narrower definitions, such as those based on a list of study designs or on methods that can address unobserved confounding,29 or that require intervention assignment to be “as-if randomised.”30 We prefer a broader definition for two reasons. Firstly, study design labels do not necessarily indicate study quality, and lists of methods tend to be arbitrary and rapidly become outdated. Secondly, “as-if randomisation” can be difficult to define precisely.31 “As-if randomisation” captures an important feature of natural experiments, but represents one end of a spectrum of possible natural experiments rather than characterising the whole range of opportunities that can usefully be exploited with the right choice of methods.24 The focus on specific events or processes (known as assignment or allocation processes) that determine exposure distinguishes natural experimental evaluations from the broader range of observational studies.

We recommend an approach to planning natural experimental evaluations adapted from the MRC/NIHR framework for the development and evaluation of complex interventions.32 That framework highlights the value of a complex systems perspective for evaluation because considering a natural experiment event as a disruption within a system can help identify the breadth of potential intended and unintended impacts, as well as the role of context in shaping the effects of the intervention. Figure 1 presents the stages of identifying and appraising opportunities for a natural experimental evaluation and working out a feasible and appropriate design. Three important phases in the scoping and planning of natural experimental evaluations are identifying and theorising natural experiments, assessing their evaluability, and conducting feasibility studies for a future evaluation.

Identifying natural experiment—A variety of circumstances can provide opportunities for a natural experimental evaluation.24 We outline five kinds of opportunity that have been widely used. One is where there is a clear division in the presence or type of exposure between otherwise similar subpopulations by time or place of implementation; for instance, when policies such as state level gun control laws are implemented in some areas but not others.33 Individual level allocation mechanisms such as eligibility criteria embedded within a policy—for example, means tests that define entitlement to social security benefits34 35—provide a second kind of opportunity. A third is the phased implementation of a policy across a population, such as implementation of the UK social security benefit Universal Credit.36 37 A fourth is when randomisation is built into the policy, as in the case of a lottery used to allocate housing places.38 A fifth is when flaws or shortcomings emerge in policy delivery, such as database errors39 40 or false negative test results in the UK’s Test and Trace programme for covid-19.

Assessing evaluability of natural experiment and feasibility of evaluation—A formal evaluability assessment is one way of ensuring that natural experimental evaluations are well designed and address relevant questions. Evaluability assessment is a systematic, collaborative approach to evaluation planning that is increasingly used in public health research.41 The assessment helps to identify key uncertainties that the evaluation should address, develop a theory of how the intervention works,32 and reach consensus about the plausible overall and distributional (ie, equity) effects the intervention could produce, the potential influence of the evaluation on future policy decisions, and how the results might contribute to the wider evidence base.42 43 Assessing the evaluability of the natural experiment can help to ensure a shared understanding with stakeholders of what an evaluation can and cannot deliver. The process enables information to be gathered on intervention delivery and the availability of, and access to, monitoring data, and establishes a clear understanding of the assignment process for the intervention. A thorough assessment of the feasibility of the evaluation should be conducted to assess the practicalities of implementing the evaluation design, such as whether routinely collected data adequately capture differences in exposure and outcomes, that the data will enable sufficient statistical power in analyses, and whether alternative methods can be used if the preferred option is not feasible.43 As with randomised trials, the funding for natural experimental evaluations might have to incorporate contingency in case the proposed evaluation is unviable; for example, by including an explicit breakpoint in the award when a formal decision of whether to continue would be made.

Protocols and preregistration—It is best practice to develop a protocol, or some other form of prior study plan appropriate to the methods being used, and to place it in the public domain before analysis commences. Natural experimental evaluations commonly use several datasets and methods of analysis, and are often retrospective. Publishing analysis plans before data analysis begins enables users to see which analyses reflect previous hypotheses and which have been informed by emerging findings. Protocols can of course be adapted, if need be, provided that amendments are systematically recorded to maintain a transparent record of how the study design has evolved.

Engaging stakeholders—In evaluations of natural experiments, there will be a diverse range of stakeholders involved at different stages of the intervention and the evaluation. Relevant stakeholders might include legislators, policy makers, organisations, and individuals responsible for implementation, institutions enabling access to necessary datasets, advocacy groups, representatives of communities affected by the intervention, and the evaluation funder.44 Involvement of such stakeholders throughout the evaluation maximises the likelihood of findings being relevant, understood, taken up, and used for decision making. To avoid conflicts of interest, clear boundaries should be agreed for stakeholder involvement and the protocol or evaluation plan made publicly available.45

Taking complex systems perspective—Population health and health system interventions typically have several components and their impacts are moderated by interactions with elements of the wider system in which they are implemented. When evaluating a natural experiment, considering such interactions can help researchers understand why the intervention succeeds or fails to achieve its intended impact,46 or why impacts vary from one setting to another.47 For example, if we want to evaluate the introduction of a tobacco tax, we might consider how smokers, retailers, producers, smugglers, the mass media, think tanks, tobacco control advocates, and the cigarette taxation system might react to the introduction of the tax in ways that could dampen or amplify its effects. Taking a systems perspective involves including processes in the evaluation to build an understanding of how the intervention interacts with its context to produce impact. This can include using system mapping48 49 (eg, developing a causal loop diagram) to create a robust theory of change for the intervention, identifying outcomes to measure (intended and unintended), or using a system dynamics model to simulate the evolution of the system over time.50

Evaluation designs that use both qualitative and quantitative methods are needed to provide an understanding of how the intervention effects were achieved given interactions between the intervention and elements of the wider system. Use of mixed methods can strengthen the estimation of effect sizes by providing a detailed understanding of the assignment process and how far it can be expected to generate otherwise comparable groups of exposed and unexposed units. Single or multiple qualitative and quantitative methods can be combined within an evaluation in several ways, including sequential exploratory, sequential explanatory, parallel convergent (triangulation), and integrated approaches.51 52 The value of a mixed methods approach is greater if planned in advance,53 for instance, by including an integration work package in the project plan and explicitly earmarking resources for it.54 For example, in research to increase understanding of sugar-sweetened beverage taxation policies, simultaneous consideration of several types of data could be used to investigate whether the tax reduces the number of sugar-sweetened beverage consumers, and whether this reduction leads to an increase in the political acceptability of further taxes on sugar-sweetened beverages.55

The research question and the nature of the assignment process will determine what methods can be used to obtain quantitative effect estimates in a natural experimental evaluation.56 A useful planning tool is the target trial framework, which matches elements of study design to the components of a hypothetical randomised trial.57 A variety of study designs and quantitative analytical methods are available. Each method has strengths and limitations, and will be more or less applicable in specific circumstances. An overview is provided in table 1. When selecting methods, it is therefore best to avoid thinking in terms of a hierarchy. Instead, the choice will be determined by the research questions, the circumstances of the evaluation, and the availability of data. Often, the ideal data for a natural experimental evaluation will not be available. Because the assignment processes used in natural experimental evaluations are rarely random, threats to internal validity caused by selective exposure to the intervention are always a concern. Use of a combination of methods (each with differing strengths and limitations) might help address the threats. When reporting the analysis, it is important to state the treatment effect being evaluated by specifying the causal contrast or estimand(s);56 and to explain for users whether the main interest is in the average effect of the intervention on an individual or the average effect of the intervention on the population.58

Economic evaluations should ideally be conducted in conjunction with evaluations of effectiveness of the natural experiment because there are resource constraints on the implementation of policies. Designing, conducting, and reporting economic evaluations of natural experiments generate specific challenges. Because natural experimental evaluations often assess effects that are by-products rather than directly intended outcomes of the intervention, economic evaluation will often require a broad, “societal,” perspective rather than a sector specific perspective. For example, in an evaluation of the health impacts of Universal Credit, a UK social security programme intended to prevent poverty and provide incentives to work, the economic evaluation examines health and wellbeing, as well as income, employment, and economic productivity outcomes.59 Such a broad perspective will require data on multisectoral costs and outcomes, which could be hard to obtain. Routinely collected data can enable a long time horizon over which outcomes are calculated. If suitable data are available, methods such as distributional cost effectiveness analysis60 and extended cost effectiveness analysis can be used to investigate the equity impact of an intervention. Challenges involved in designing and conducting an economic evaluation alongside natural experimental evaluation are outlined by Deidda and colleagues,61 and guidance has recently been developed for identifying appropriate use and application of complex system models in economic evaluations.62

Qualitative methods can strengthen natural experimental evaluations in several ways. They can help researchers to characterise the intervention, understand assignment processes, explore mechanisms, identify threats to validity, help define parameters, and interpret and strengthen causal claims. Evaluations should therefore be planned and conducted in an integrated way to ensure that the qualitative components are incorporated throughout to achieve maximum use of the qualitative data. Below we highlight the components of an evaluation to which qualitative methods contribute beyond their use in process evaluations.63 Table 2 provides examples of using qualitative methods. Note that the framework focuses on the use of qualitative methods within evaluations whose primary aim is to estimate effect sizes of interventions, rather than studies whose main goal is to address questions such as the contexts in which interventions work.47 64

A key use of qualitative methods is to characterise the intervention in order to understand its rationale and the organisational, historical, political, and social and policy context, including co-occurring interventions, in which it is implemented.65 66 Qualitative system mapping techniques (eg, group model building67) can be used to identify the elements of contexts that are important to include in the evaluation68 and can help to identify important preconditions, assumptions, and potential mechanisms of effect, given the structure of the system.69 Qualitative methods can also inform the selection of exposed and unexposed populations, and choice of appropriate outcomes and indicators.70 This helps ensure that the chosen quantitative indicators capture what is intended, such as outcomes that are important for stakeholders, and that their strengths and limitations are well understood. In some natural experimental evaluations, qualitative methods are used to generate data on outcomes, to enable triangulation, or to identify secondary outcomes not captured in quantitative datasets or unanticipated at the outset.71 Qualitative data might also serve as primary evidence on changes in knowledge, understanding, or practices that are associated with an intervention when quantitative data on such outcomes are not available (table 2).

Most importantly, qualitative methods can help us to understand mechanisms and mediators, and help explain change. Analysis of qualitative data can explore why the intervention did, or did not, lead to anticipated outcomes through making inferences about causal processes from comparisons within the case,72 drawing on approaches such as process tracing73 or analytic induction.74 Appropriate qualitative analysis, in the light of theories of change, strengthens causal inferences and claims about transferability.

Reporting—Accurate, clear, and comprehensive reporting of the natural experiment and the evaluation is crucial for the best use and understanding of the evaluation. Details of reporting guidance likely to be useful to researchers conducting natural experimental evaluations are provided in supplementary file 2.

Critical appraisal—A systematic assessment of the design, conduct, and analysis of a study might be required to understand the rigour of an individual study or undertaken as part of evidence synthesis. No single critical appraisal tool can fully assess the risk of bias of all natural experimental evaluation study designs.75 76 77 78 For some types of natural experimental evaluations, and some systematic reviews, an appraisal framework more like those used for qualitative research might be more appropriate.79 80

Evidence synthesis—Synthesising evidence from natural experimental evaluations requires consideration of how to manage the expected diversity in study design and characteristics. For some review topics, it might be more valuable to examine whether there is any evidence for an effect or to explore intervention mechanisms, for example, rather than to estimate an overall effect size within a meta-analysis. Approaches for synthesis without meta-analysis will often be useful; for example, guidance provided by Cochrane81 82 and the RAMESES (Realist and Meta-narrative Evidence Syntheses: Evolving Standards) group.83 The specifics of the evidence synthesis questions and studies involved will determine the appropriate methods to use, with a mixed methods design often useful.84 85

Certainty of evidence—If the aim of evidence synthesis is to estimate an effect size, then it is usually appropriate to summarise confidence in the overall effect estimate; that is, to formally assess the overall certainty of the findings. The framework most often used is GRADE (Grading of Recommendations, Assessment, Development, and Evaluations).86 Challenges in using GRADE with natural experimental evaluations—including the difficulty of classifying risk of bias for study designs not typically used in epidemiology, selecting outcomes for synthesis, and the lack of differentiation in certainty assessments for population health and health system interventions—are being addressed by the GRADE Public Health Group by developing further guidance and provision of training.87

Natural experimental evaluations often use data collected for other purposes. Such sources include administrative or commercial datasets, population surveys, and data collected from point of sale, traffic sensors, fitness apps, and so on. Secondary analysis of such datasets enables interventions to be evaluated retrospectively using data whose unit cost is a small fraction of the cost of collecting primary data. However, negotiating access to such datasets can often be a time consuming, expensive, and uncertain process, especially if the research involves combining data from several sources. An alternative is to use secure research platforms known as Trusted Research Environments, which are designed to curate data securely and to provide researchers with efficient access. For example, the Brazilian Centre for Data and Knowledge Integration for Health (CIDACS) stores, processes, and links identified data and has secure procedures to provide access to deidentified or anonymised data linking social benefit programmes with deaths, births, and infectious diseases (https://cidacs.bahia.fiocruz.br/en/).

Good practice considerations, derived from the content of the updated framework, are provided for different users. Box 1 presents a condensed form of these recommendations. The main messages for planning, commissioning, conducting, reporting, and using evidence from natural experimental evaluations have been grouped according to the key audience, concentrating on messages that are practical and implementable.

The REC-CAGEFREE II trial was an investigator initiated, multicentre, randomised, open label, non-inferiority trial conducted in 41 sites across China. The rationale and design of the trial have been described previously.27 The trial was conducted in accordance with the Declaration of Helsinki and Good Clinical Practice guidelines, and the protocol was approved by the ethics committee of Xijing Hospital (ID: KY20212080-F-1) and responsible ethics committees in all participating centres. Written informed consent was obtained from all patients. The study protocol and statistical analysis plan are provided in the appendix. An independent data and safety monitoring board provided external oversight to ensure the safety of trial participants. Committee members and participating investigators are listed in the appendix and table S1. This trial is registered at ClinicalTrials.gov, NCT04971356.

Participants who had a clinical presentation of acute coronary syndrome (ST/non-ST elevation myocardial infarction or unstable angina) and underwent percutaneous coronary intervention with paclitaxel-coated balloons with no stent implantation were eligible for enrolment. The selection of suitable patients and lesions for DCB treatment and subsequent procedural techniques followed the recommendations of the German Consensus Group on DCB interventions and the third report of the International DCB Consensus Group,28 13 as detailed in the methods section of the appendix. No restrictions were placed on the type of lesion (de novo or in-stent restenosis), treated vessel diameter, or the specific brand of paclitaxel coated balloon that was used (brands and features of DCBs used are summarised in table S2). Key exclusion criteria included people younger than 18 years and older than 80 years, prior haemorrhagic stroke, need for long term oral anticoagulant therapy, cardiogenic shock, or treatment for in-stent thrombosis. A full list of the inclusion and exclusion criteria are listed in table S3. Data for patient sex, race, and ethnic group were collected from medical records.

Immediately after percutaneous coronary intervention, patients were randomly assigned in a 1:1 ratio using a web based centralised system to receive either stepwise DAPT de-escalation or standard 12 months DAPT.3 Randomisation sequences were computer generated with the dynamic permuted block method, with block sizes of two or four, and stratified by site and the type of lesion being treated (de novo or in-stent restenosis). Patients and the investigators were not masked to treatment allocation; however, members of the independent clinical event committee who adjudicated the endpoints and statisticians who developed the statistical programmes were masked to treatment allocation.

Follow-up visits were scheduled to occur at months 1 (±14 days), 3, 6, and 12 (±30 days) after randomisation. Visits were preferably conducted on site; however, if patients were unable or unwilling to visit the outpatient clinic, the scheduled visit could be replaced by a telephone call, except for the 30 day and one year visits. A mobile application operating on the WeChat platform was developed to facilitate adherence to the allocated medications; participants were contacted monthly through this application to assess their health status and medication compliance.

Participants who had been randomly assigned to the stepwise DAPT de-escalation group received aspirin plus ticagrelor for one month after the procedure, followed by ticagrelor monotherapy for five months, and then aspirin monotherapy for six months. Participants who had been randomly assigned to the standard DAPT group received aspirin plus ticagrelor for 12 months (figure S1). For maintenance, aspirin was prescribed at 100 mg daily and ticagrelor was prescribed at 90 mg twice daily. Ticagrelor was replaced with clopidogrel in patients who had dyspnoea or who were unable to continue taking ticagrelor. Loading doses of aspirin (300 mg) and ticagrelor (180 mg) were administered in patients who had no history of any antiplatelet medications at the time of percutaneous coronary intervention.29 Patients prescribed clopidogrel before percutaneous coronary intervention were switched to ticagrelor as soon as possible after randomisation.30 To maximise adherence to treatment allocation, participants were given all antiplatelet medication free of charge during their follow-up visits. Other medical treatments were left to the physician's discretion, but guideline directed medical treatment was strongly recommended.29

The primary efficacy endpoint was net adverse clinical events (a non-hierarchical composite of all cause death, stroke, myocardial infarction, revascularisation, and Bleeding Academic Research Consortium (BARC) type 3 or 5 bleeding) assessed at 12 months. If non-inferiority was met for the primary efficacy endpoint, then the prespecified secondary efficacy endpoints were assessed in a fixed sequence in the following order: clinically relevant ischaemic or bleeding event, BARC type 2, 3 or 5 bleeding, BARC type 3 or 5 bleeding, BARC type 2 bleeding (table S4). A clinically relevant ischaemic or bleeding event was predefined as a hierarchical composite of all cause death, stroke, myocardial infarction, BARC type 3 bleeding, revascularisation, and BARC type 2 bleeding events with the individual components treated as having different clinical importance by using the win ratio method.31 This hierarchy was established based on previous studies.31 32 The safety endpoints include the patient oriented composite endpoint (a non-hierarchical composite of all cause death, stroke, myocardial infarction, and revascularisation), device oriented composite endpoint (a non-hierarchical composite of cardiovascular death, target vessel myocardial infarction, and clinically and physiologically indicated target lesion revascularisation), target vessel failure (a non-hierarchical composite of cardiovascular death, target vessel myocardial infarction, and clinically and physiologically indicated target vessel revascularisation), their individual components, and definite or probable stent (device) thrombosis.

Outcome events were adjudicated by an independent clinical event committee, according to definitions of the Academic Research Consortium-2,33 the fourth universal definition of myocardial infarction for spontaneous myocardial infarction,34 and BARC (detailed definitions in appendix methods).35 Adverse events were centrally collected, and any document that could lead to unblinding of treatment assignment was redacted before submission to the clinical event committee.

Sample size and power calculations were based on a non-inferiority assumption for the primary outcome. According to data from previous trials,21 36 we assumed that 8% of patients in the standard DAPT group would reach the primary endpoint at one year. The non-inferiority margin of 3.2%, which corresponded to 40% of the estimated event rate, was chosen based on clinically acceptable thresholds of difference,37 (summarised in table S5) based on previous non-inferiority trials comparing antiplatelet regimens after stent implantation.18 19 22 Considering an anticipated 5% patient attrition rate, 1908 patients were required for the study to have 80% power to show non-inferiority with a 5% one sided type I error rate. Although a one sided type I error rate of 2.5% is considered more robust for a non-inferiority assessment, we opted for a one sided type I error rate of 5% because this rate has been used previously for evaluating the de-escalation of DAPT.18 22 23 24 Nevertheless, the assessment of non-inferiority based on a one sided 97.5% confidence interval (CI) of the primary endpoint was also reported as a sensitivity analysis.

The primary analysis was based on a covariate-adjusted analysis of treatment difference in the cumulative event rate of the primary endpoint in the intention-to-treat population. The prespecified covariates were age, sex, hypertension, hyperlipidaemia, diabetes, smoking status, history of cardiovascular disease, stroke, clinical presentation, lesion characteristics (de novo or in-stent restenosis), and centre effect (with details in appendix methods). The treatment difference was defined as the stepwise DAPT de-escalation group minus standard DAPT group. The cumulative event rate was estimated at 360 days by the Kaplan-Meier method, with the standard error of difference calculated using Greenwood's method and P value calculated using an approximate z test. Non-inferiority was concluded when the upper limit of a one sided 95% CI in the treatment difference of the primary endpoint was less than 3.2%. Additionally, an unadjusted measurement of the treatment difference (crude analysis) was conducted as a sensitivity analysis.

If non-inferiority was met with the primary endpoint, a predefined hierarchical testing structure for the secondary endpoints was implemented with the fixed sequence outlined in the appendix methods. The secondary endpoint of clinically relevant ischaemic or bleeding events was analysed using the win ratio method. For other secondary endpoints, the difference in cumulative event rates and their two sided 95% CIs are reported. As post hoc sensitivity analyses of the secondary outcome, we used the cumulative incidence function (Aalen-Johansen estimator) to account for the competing risk of death. The main results are presented in the intention-to-treat population. The analyses of the primary and secondary outcomes were also performed in the per protocol population.

Detailed information regarding the multiplicity considerations, covariate adjusted analysis, and win ratio analysis is provided in the appendix methods. The analysis was done using R statistical software version 4.2.1 (R Project for statistical computing).

No funding was allocated for involvement of patients or the public in the design, conduct, reporting, or dissemination plans of our research. Nevertheless, we spoke to the patients about the concept of the study during study designing and collected their opinions, and asked a member of the public to read our manuscript after submission.