Signal Detection and Methodological Limitations in a Real-World Registry: Learnings from the Evaluation of Long-Term Safety Analyses in PSOLAR

Abstract

Introduction

Psoriasis Longitudinal Assessment and Registry (PSOLAR) was designed in 2007 as the first disease-based registry for patients with psoriasis.

Objective

The aim of this study was to discuss methodological limitations and post hoc analyses in long-term safety registries using learnings from analyses of a potential safety risk for major adverse cardiovascular events (MACE) in PSOLAR.

Methods

PSOLAR is an international observational study of over 12,000 psoriasis patients that was conducted to meet postmarketing safety commitments for infliximab and ustekinumab. A recent annual review of registry data indicated a potential MACE risk for ustekinumab vs. non-biologics based on prespecified COX model regression analyses, which yielded an adjusted hazard ratio (HR) of 1.533 (95% confidence interval [CI] 1.103–2.131). Therefore, we conducted a comprehensive review of key statistical methodology and implemented post hoc analytical methods to address specific limitations.

Results

The following limiting factors were identified: (1) inclusion of both prevalent and incident (new) users of biologics; (2) unanticipated imbalances in patient characteristics between treatment cohorts at baseline; (3) limited availability of relevant clinical data after enrollment; and (4) divergence of characteristics associated with outcomes among comparator groups over time. The analysis was modified to include only incident users, propensity scores were used to weight HRs, and adalimumab was deemed a more clinically appropriate comparator. The revised HR was 0.820 (95% CI 0.532–1.265), indicating no meaningful increase in MACE risk for ustekinumab.

Conclusion

Our results, which do not support a causal association between ustekinumab exposure and MACE risk, underscore the need for ongoing assessment of analytical methods in long-term observational studies.

Key Points

The Psoriasis Longitudinal Assessment and Registry (PSOLAR) is a large observational study designed in 2007 to meet postmarketing safety commitments for infliximab and ustekinumab for the treatment of psoriasis.

Key limitations in analysis methodology identified in a comprehensive review of a potential safety risk were the inclusion of patients who used biologics before registry entry, unanticipated imbalances in patient characteristics across cohorts, and limited availability of all relevant clinical data.

Fundamental modifications to predefined analyses confirm no increased risk of major adverse cardiovascular events for ustekinumab, and emphasize the need for periodic assessment of study design and analytical methods to maintain validity in long-term observational studies.

Introduction

Long-term treatment with systemic and biologic agents is needed due to the chronic and debilitating nature of psoriasis. Safety profiles for psoriasis therapies derived from clinical trials designed to support regulatory approval are limited by strict inclusion/exclusion criteria and a relatively short duration of treatment and follow-up. To fully understand the benefit–risk profiles of both novel and established therapies, safety surveillance reflecting actual clinical use through observational studies is valuable [1–4]. Patient registries are especially useful for detecting uncommon safety events that become apparent only in larger populations and for longer periods of time than that evaluated in randomized clinical trials.

The Psoriasis Longitudinal Assessment and Registry (PSOLAR) is a multicenter, prospective, disease-based registry for assessing the long-term safety experience of patients with psoriasis. PSOLAR was designed to detect potential safety signals as part of postmarketing commitments to health authorities for two biologic therapies for psoriasis: infliximab (REMICADE^®, Janssen Biotech, Inc., Horsham, PA, USA) and ustekinumab (STELARA^®, Janssen Biotech, Inc., Horsham, PA, USA) [5, 6]. The registry was initially launched in June 2007 to evaluate infliximab-treated patients with psoriasis; the first patient receiving ustekinumab for psoriasis was enrolled in March 2009. As of the annual data cut-off in 2019, a total of 12,090 patients had been enrolled in 16 countries and followed for a median of 7.54 years (73,133.6 patient-years [PY] of follow-up).

PSOLAR was designed as a signal-detection registry and, using prespecified methodology based on guidance from health authorities, data are analyzed by the sponsor (Janssen Scientific Affairs, LLC, Horsham, PA, USA) on an annual basis. Results are provided to regulatory agencies as Interval Safety Registry Reports and presented to the PSOLAR Steering Committee composed of dermatologists with expertise in psoriasis. These annual reports focus on cumulative rates of reported adverse events, with specific attention given to adverse events of special interest (such as all-cause mortality, serious infections, malignancy, and major adverse cardiovascular events [MACE]). As the main purpose of the registry is to evaluate safety data, the study protocol stipulates that any safety finding identified using the prespecified statistical methodology should be further evaluated based on appropriate post hoc analyses to determine if the finding represents a true safety risk.

A recent annual review of registry data indicated a meaningful difference in adjusted hazard ratios (HRs) for MACE with ustekinumab treatment versus the non-biologics cohort, indicating a potential risk. Further investigation revealed key methodological limitations that had occurred over the long-term course of the registry. It became apparent that initial design features of the registry may not support adaptation to the evolving treatment landscape or account for changing patient characteristics due to switching therapies. To mitigate these longitudinal weaknesses, a series of post hoc analyses were performed. In this study, we discuss the design of PSOLAR in light of MACE risk by detailing both prespecified and post hoc analytical methods and outcomes. These lessons learned from PSOLAR highlight the importance of re-evaluating methodology over time and may provide insight for other real-world registries.

Methods

Objective and Design of the Study

The design of the PSOLAR registry (ClinicalTrials.gov: NCT00508547) has been reported in detail [5, 6]. Briefly, patients were adults (≥ 18 years of age) who were eligible to receive, or were receiving, any systemic therapy for psoriasis. Each patient was to be followed for 8 years from the date of enrollment, with data collected approximately every 6 months after the initial visit. Patients were assigned to one of four treatment cohorts, including those treated with ustekinumab, other sponsor biologics (i.e. infliximab [Remicade^®]), non-sponsor biologics (i.e. other tumor necrosis factor-α inhibitors, interleukin (IL)-17 inhibitors, IL-23 inhibitors), or non-biologics, depending on treatment at registry entry or changes in treatment made during the course of participation in the registry.

Prespecified Statistical Methodology

The predefined analysis plan for MACE included the calculation of cumulative unadjusted incidence rates per 100 PY across four treatment cohorts and modeled analyses utilizing Cox proportional HR methodology. The biologic cohorts considered in the prespecified analyses for annual safety reports are defined based on the manufacturer of the biologic (i.e. ‘Other Sponsor’ or ‘Non-Sponsor’) compared with ustekinumab and non-biologic agents. Starting in 2019, an expanded search strategy was used to identify MACE in the registry [7–9].

Treatment differences in the time to MACE outcomes were estimated using a Cox proportional hazards regression model adjusting for baseline covariates and time-varying effects of biologic and immunomodulator therapies. Adjusted HRs, 95% confidence intervals (CIs), and corresponding p-values based on a Wald Chi-square test were calculated for the ustekinumab cohort compared with the non-biologics cohort. To adjust for confounding related to imbalances between treatments within the Cox model, covariates deemed relevant to the development of MACE or the selection of treatment were included in the multivariable analyses. Demographic and disease characteristics and treatment exposures included in the model were age at baseline, sex, ethnicity, body mass index at baseline, calendar year, duration of disease, Physician’s Global Assessment score at baseline, psoriatic arthritis diagnosis at baseline, history of immunomodulator use, history of biologic use, history of diabetes, history of cardiovascular disease (e.g. angina, atherosclerotic disease, congestive heart failure, coronary artery disease, myocardial infarction, transient ischemic attack/cerebrovascular accident/stroke, hypertension, hyperlipidemia), smoking status, and time-dependent use of ustekinumab, other biologics, and immunomodulators.

Primary time-to-event analyses were performed for all patients ever exposed to ustekinumab, including those who started treatment before, at the time of, or after registry entry (regardless of the timing of the most recent ustekinumab exposure relative to the event). To evaluate potential risk related to more proximal exposure, additional analyses were performed based on counting events only within the 91-day window after ustekinumab exposure.

The cumulative unadjusted incidence rates of MACE in a recent annual data cut (Table 1) were similar to those determined in previous years in all treatment cohorts. However, in the ever-exposed analysis for MACE, the HR showed a meaningful difference for the ustekinumab cohort versus the non-biologics cohort (HR 1.533, 95% CI 1.103–2.131) (Table 2). The secondary analysis of patients experiencing a MACE within 91 days of the previous exposure also yielded an elevated HR (but to a lesser extent and not statistically significant) for the comparison between the ustekinumab and non-biologics cohorts (HR 1.193, 95% CI 0.917–1.553).

Table 1.

Cumulative unadjusted incidence rates of MACE across treatment cohorts; ever-exposed and 91-day-exposed populations

	Ustekinumab	Other sponsor biologicsa	Non-sponsor biologicsb	Non-biologicsc
Ever-exposed analysisd
Years of follow-up	27,938 PY	8653 PY	25,849 PY	10,693 PY
Rates/100 PY (n)	0.71 (197)	0.80 (69)	0.58 (151)	0.64 (68)
91-day-exposed analysise
Years of follow-up	16,351 PY	5269 PY	21,407 PY	30,107 PY
Rates/100 PY (n)	0.65 (107)	0.68 (36)	0.61 (130)	0.70 (212)

MACE major adverse cardiovascular events, n number of events, PY patient-years (number of years from baseline until discontinuation or 23 August 2019), NSAIDs non-steroidal anti-inflammatory drugs

^aThe ‘Other Sponsor Biologics’ cohort includes infliximab and golimumab

^bThe ‘Non-Sponsor Biologics’ cohort includes etanercept, adalimumab, efalizumab, and alefacept

^cThe ‘Non-Biologics’ cohort includes immunomodulators, retinoids, NSAIDS, steroids, and phototherapy.

^dDetermination of incidence rates in the ever-exposed analysis is subject to attribution rules based on the following hierarchy: Ustekinumab, Other Sponsor Biologics, Non-Sponsor Biologics, and Non-Biologics. Therefore, events and PY accrued in therapies lower on the hierarchy are attributed to the highest therapy

^eDetermination of incidence rates in the 91-day-exposure window for biologics was based on the number of events (and total PY of exposure) reported during or within the 91-day window following the most recent exposure (i.e. duration of treatment or ≤ 91 days after biologic discontinuation/last exposure). Events and exposure occurring > 91 days after discontinuation of a biologic are counted in the non-biologics cohort. In cases of exposure to two biologics within 91 days, attribution rules apply based on the hierarchy described above

Table 2.

Cox proportional hazard regression models for MACE

	HR	95% CI	p value
Ever-exposed analysis
Prespecifieda	1.533	1.103–2.131	0.011
Propensity-score adjustedb	1.176	0.929–1.490	0.177
Compared with ADAc	0.820	0.532–1.265	0.370
91-day-exposed analysis
Prespecifieda	1.193	0.917–1.553	0.189
Propensity-score adjusted	NA	NA	NA
Compared with ADAc	0.592	0.366–0.956	0.032

ADA adalimumab, CI confidence interval, HR hazard ratio, IPTW inverse probability of treatment weighting, MACE major adverse cardiovascular events, NA not applicable

^aThe prespecified analyses used Cox proportional hazards regression methodology adjusted for baseline covariates and time-varying effects of biologic therapies using non-biologics as the comparator for ustekinumab

^bPropensity-score adjustments with IPTW were used to mitigate the effect of residual imbalances in baseline characteristics between the ustekinumab and non-biologics cohorts

^cEvaluation of methodological limitations of previous analyses indicate that adalimumab is the most appropriate comparator for ustekinumab and that only incident users should be included in the analyses

Post Hoc Analyses

Given the signal-detection design of the registry, a cross-functional team of clinical experts representing cardiology, epidemiology, and biostatistics conducted a comprehensive assessment of the methodologies for analysis of MACE risk; the review identified several methodological limitations. Treatment paradigms for psoriasis have undergone significant shifts with the introduction of new therapies; at the initial design stage of this long-term registry (i.e. up to 8 years of patient follow-up), such changes could not have been anticipated. In some cases, patient characteristics diverged between treatment cohorts over time as treatments switched (and consequently cohorts changed), otherwise known as population drift. The following were considered pertinent to influencing the outcomes of MACE and potentially other adverse events of special interest in the prespecified analyses: (1) inclusion of prevalent users of biologics (those with ongoing use of biologics with initiation of treatment pre-dating registry entry) in addition to incident users (those who started treatment at enrollment or during registry participation); (2) unanticipated imbalances in patient characteristics between treatment cohorts at baseline resulting from the non-randomized, observational nature of the registry; (3) limited availability of relevant clinical data after registry entry; and (4) divergence of characteristics associated with outcomes among the comparator groups over time on registry.

1. Prevalent versus incident users

The registry includes both prevalent and incident users of biologics. While inclusion of prevalent biologic users (i.e. those who received treatment at any time, even before enrollment) in the registry allowed for efficient and timely recruitment and was necessary to achieve a sample size required by regulatory agencies, it introduced bias into the prespecified analyses as there were no true disease severity data at baseline to assess pretreatment (i.e. prebiologic) status and patient characteristics. Because two-thirds (66%) of patients in the ustekinumab cohort are prevalent users, our analysis could not adequately measure disease severity between treatment groups at baseline. As biologic treatment for psoriasis typically follows a stepwise paradigm with a biologic introduced after ineffective non-biologic therapy, we hypothesized that ustekinumab was used to treat more severe disease that could be associated with increased risk for MACE [10, 11].

Analyses focusing on incident users, i.e. those who started biologic treatment on-registry, provide more pertinent information than those using both prevalent and incident users. The former ensures that baseline data collected at the time of treatment initiation are accurate and that all time on treatment is captured in the analysis [12, 13]. Balance and consistency between cohorts is more likely to be achieved when enrollment occurs at inception of treatment (incident use). Furthermore, analyses from observational studies that included only incident users reportedly produce results that are more consistent with findings from randomized trials compared with those that included both prevalent and incident users [14].

• (2)Imbalances between treatment cohorts

Due to the non-randomized nature of the registry, baseline differences in several demographic and disease characteristics are expected between the treatment cohorts (Table 3). As some baseline features that differed are also risk factors for MACE (e.g. age, obesity), it was necessary to conduct adjusted HR modeling analyses to estimate risk. To assess the robustness of these analyses and further discern residual imbalances between the ustekinumab cohort and the prespecified non-biologics comparator cohort that could bias the resulting HR estimate, additional methods of covariate adjustment based on epidemiologic principles were considered.

Table 3.

Baseline characteristics across treatment cohorts; prevalent users ever-exposed to treatment

		Ustekinumab [N = 4832]	Other sponsor biologics [N = 1300]	Non-sponsor biologics [N = 4007]	Non-biologics [N = 1951]
Age, years		47.3 ± 12.98	49.5 ± 13.55	48.6 ± 13.96	51.5 ± 15.68
Age category, years
18–24		213 (4.4)	34 (2.6)	163 (4.1)	93 (4.8)
25–34		661 (13.7)	170 (13.1)	547 (13.7)	251 (12.9)
35–44		1124 (23.3)	263 (20.2)	830 (20.7)	314 (16.1)
45–54		1338 (27.7)	349 (26.8)	1037 (25.9)	403 (20.7)
55–64		1063 (22.0)	301 (23.2)	927 (23.1)	435 (22.3)
≥ 65		433 (9.0)	183 (14.1)	503 (12.6)	455 (23.3)
Median duration of PsO, years		17.2	16.5	14.3	9.7
Psoriatic arthritis		1704 (35.3)	740 (56.9)	1563 (39.0)	345 (17.7)
Confirmed by specialist		817 (16.9)	358 (27.5)	600 (15.0)	125 (6.4)
Obesity class II/III		1245 (26.0)	354 (27.5)	861 (21.9)	378 (19.5)
Historical peak
%BSA with PsO		32.2 ± 24.98	34.1 ± 26.53	28.8 ± 24.12	22.6 ± 22.29
Systemic therapy, n		3238	916	2891	1282
		1186 (36.6)	408 (44.5)	944 (32.7)	271 (21.1)
PGA score of 4/5, n		2629	739	2405	1102
		1052 (40.0)	260 (35.2)	735 (30.6)	295 (26.8)

BSA body surface area, PGA Physician Global Assessment, PsO psoriasis, SD standard deviation

Data are expressed as n (%) or mean ± SD unless otherwise specified

Propensity scoring is an accepted method to control for measured confounding in non-interventional studies. The propensity score summarizes all information from explanatory variables, such as disease severity and comorbidities, and estimates the probability of a patient receiving the intervention of interest (i.e. ustekinumab) [15]. Individual patients may have the same or similar propensity score, yet only some will have received the intervention of interest. If baseline risk factors and disease characteristics are comparable, the estimated propensity score distributions would appear to be similar for all patients irrespective of treatment (as would be expected in a 1:1 randomized clinical trial). The propensity score is often estimated for each patient from a logistic regression model in which treatment assignment is the outcome variable.

Different propensity score implementation methods may be used to model the estimation of effect, including inverse probability of treatment weighting (IPTW). The IPTW approach uses the propensity score to weight each treatment group to have a distribution of covariates similar to that of the overall study population, which balances these characteristics in each treatment group. While propensity scores are generally useful for controlling confounding, they can sometimes introduce bias (e.g. if the propensity score model includes incorrect variables). It is also important to consider ‘trimming’ both ends of the propensity score distribution to eliminate patients who are treated contrary to prediction, as this may reduce the impact of bias due to unmeasured confounding caused by frailty [15]. Critical inspection of the distribution of weights is essential, particularly for assessing the impact of excessively large weights that could have an unduly high influence on results [16, 17].

In PSOLAR, propensity scores were estimated to generate IPTW balancing in the ustekinumab and non-biologics cohorts using the same baseline risk factors that were prespecified for the primary Cox regression model. While neither approach showed statistical significance, the IPTW approach gave an HR closer to the null value (1.176, 95% CI 0.929–1.490), compared with that calculated without IPTW (propensity-score adjusted). However, the lack of overlap between score distributions on the propensity score histogram indicated that there was residual covariate imbalance between patients receiving ustekinumab and those receiving non-biologic treatments (Fig. 1). This indicated that an active comparator for ustekinumab may be more appropriate to address this limitation when evaluating certain adverse events of interest, such as MACE.

• (3)Limited availability of relevant clinical data after registry entry

Fig. 1.

Histogram of propensity scores for the ustekinumab and non-biologics cohorts; prevalent and incident users in the ever-exposed population (prespecified analyses)

Understanding how treatment cohorts change over time and adjusting for these differences are essential for ensuring unbiased and valid results. While PSOLAR provides a robust dataset regarding clinical characteristics of patients with psoriasis collected at baseline and over time, some health-related parameters that may be relevant to cardiovascular outcomes are not captured after enrollment. For example, the occurrence of new or worsened comorbidities is not collected beyond baseline. In addition, use of concomitant non-psoriasis medications (e.g. statins or aspirin) is not collected at any time during the registry. The collection of these parameters over time was not implemented to reduce operational burden associated with data capture at each study site. Therefore, the lack of relevant data after baseline is a study design issue that cannot be mitigated by modifying analytical methods.

• (4)Divergent comparator groups over time on the registry

Another consideration related to the evolution of the registry over time is the difference in the duration of registry participation or treatment period between cohorts. For example, the time to discontinuation or treatment switch was longer for the ustekinumab cohort compared with the non-biologics cohort (median 8 vs. 4 years) (Fig. 2). While the Cox model regression analysis accounts for differing time on registry, it does not account for potentially important differences in how the clinical condition of patients may evolve over time, especially at the time of registry discontinuation or treatment switch. In PSOLAR, patients who began treatment with a non-biologic may have stepped up to more effective treatment with a biologic agent, whereas patients treated with a biologic were not likely to switch to a non-biologic. This is in keeping with ‘healthy adherer’ bias, wherein patients who remain in a study and adhere to an existing therapy are likely to be healthier than those who discontinue or switch therapies [13]. Therefore, the comparison between the ustekinumab and non-biologics cohorts may be further biased by the retention of healthier patients in the non-biologics cohort or those who have limited access, which may reduce the utility of the non-biologics cohort for the primary comparison and support the use of an active comparator in the analysis. Furthermore, differences between treatment cohorts at baseline are likely to have diverged further over time, particularly in light of the long-term nature of the registry.

Fig. 2.

Kaplan–Meier analysis of time to discontinuation of treatment switch in the ustekinumab and non-biologics cohorts; ever-exposed population

Results

To address the identified methodological flaws, the prespecified methodology was modified in several ways in post hoc analyses (Table 4). First, to align observation of the treatment period, only incident users were included in the multivariable Cox model regression analysis for MACE. In addition, the model was weighted with propensity scores to address residual imbalances in baseline characteristics between treatment cohorts beyond the prespecified HR model. Finally, due to the disparities identified both at baseline and over the course of the registry, a comparator other than non-biologics was deemed more clinically appropriate for the evaluation of potential MACE risk associated with ustekinumab.

Table 4.

Methodological limitations in prespecified analyses and analytical solutions based on a post hoc approach

Study limitation	Analytical solution
1. Inclusion of prevalent and incident usersa	Incident user analysis
2. Imbalances in baseline characteristics across cohorts	Incident user analysis Modified Cox analysis/PS adjustment More comparable active comparator
3. Limited availability of relevant data after baselineb	Not able to mitigate
4. Divergence of characteristics across cohorts over time	Incident user analysis More comparable active comparator

PS propensity scoring

^aPrevalent users include those with ongoing use of biologics (with initiation of treatment pre-dating registry entry in some patients), while incident users include only those who started treatment at enrollment or during registry participation

^bRelevant data not collected after baseline include new or worsened comorbidities and concomitant non-psoriasis medication; inconsistently collected after baseline includes body weight and smoking status

Review of the most important considerations for choosing a potential comparator indicated that adalimumab (Humira^®, AbbVie Inc., Chicago, IL, USA) was the most appropriate candidate for comparison with ustekinumab. The adalimumab cohort included the most patients (and the most PY of follow-up) in PSOLAR after the ustekinumab cohort; therefore, using the adalimumab cohort provided sufficient statistical power for subsequent analyses. Furthermore, clinical characteristics at baseline were generally comparable between patients receiving ustekinumab and those receiving adalimumab in the registry (Table 5). Furthermore, ustekinumab and adalimumab were approved by regulatory authorities within a similar timeframe in the US (25 September 2009 and 22 January 2008, respectively), and the clinical efficacy profiles from their psoriasis registrational trials are similar [18, 19]. Finally, adalimumab has no known risk of MACE [19].

Table 5.

Baseline characteristics by treatment; incident users ever exposed to treatment

	Ustekinumab [N = 1005]	Adalimumab [N = 759]
Age, years	46.2 ± 13.58	46.8 ± 13.94
Age category, years
18–24	60 (6.0)	36 (4.7)
25–34	155 (15.4)	128 (16.9)
35–44	235 (23.4)	160 (21.1)
45–54	271 (27.0)	204 (26.9)
55–64	196 (19.5)	154 (20.3)
≥ 65	88 (8.8)	77 (10.1)
Median duration of PsO, years	16.8	10.4
Psoriatic arthritis	311 (30.9)	233 (30.7)
Confirmed by specialist	162 (16.1)	85 (11.2)
Obesity class II/III	238 (24.0)	196 (26.1)
Historical peak
%BSA with PsO	33.8 ± 25.28	28.2 ± 24.12
Systemic therapy, n	682	573
	223 (32.7)	147 (25.7)
PGA score of 4/5, n	554	462
	215 (38.8)	165 (35.7)

BSA body surface area, PGA Physician Global Assessment, PsO psoriasis, SD standard deviation

Data are expressed as n (%) or mean ± SD, unless otherwise specified

In the post hoc analysis, the revised histogram following propensity scoring for incident users demonstrated smaller differences between the ustekinumab and adalimumab cohorts compared with those between the ustekinumab and non-biologics cohorts, indicating that the ustekinumab and adalimumab cohorts are more comparable (Fig. 3). Using this more robust analysis for the ever-exposed population, the updated HR was 0.820 (95% CI 0.532–1.265), indicating no meaningful difference in the risk of MACE for ustekinumab- vs. adalimumab-treated patients with psoriasis. When the analysis was based on incidence of MACE within 91 days of the previous exposure, similar results were observed (HR 0.592, 95% CI 0.366–0.956) (Table 2).

Fig. 3.

Histogram of propensity scores for the ustekinumab and adalimumab cohorts based on revised analyses; incident users in the ever-exposed population (post hoc analyses)

Discussion

An evaluation of the analytical methodologies of the long-term ongoing PSOLAR program was triggered by a potential safety signal for MACE in ustekinumab-treated patients compared with non-biologics (HR 1.533, 95% CI 1.103–2.131). Following the application of more methodologically appropriate analyses, no meaningful difference in the risk of MACE with ustekinumab was observed (HR 0.820, 95% CI 0.532–1.265). Our comprehensive review highlighted important limitations that should be considered when designing studies and interpreting results from other long-term patient registries. Notable findings included the challenge of evaluating patients starting treatment prior to enrollment (prevalent users), the potential impact of baseline imbalances between treatment cohorts, the limited availability of some relevant data after enrollment, and the divergence in the health status of the comparator groups as the registry matured. Additionally, the longitudinal nature of PSOLAR (>10 years) could not account for the evolving availability of new treatment options that contributed to the opportunity for switching therapy. Post hoc analyses were performed to mitigate the issues around prevalent user data, imbalances between cohorts, evolving standards of care over time, survival bias, and unequal discontinuation rates. Although these limitations are presented in the context of the MACE analyses, the lessons learned may apply to all safety assessments in PSOLAR and shed light on analytical approaches for future disease-based registries and observational studies.

Our results confirm the power of PSOLAR to evaluate safety events and document an effective evaluation of a potential signal. Although the prespecified analyses showed a potential risk of MACE for ustekinumab compared with non-biologics during registry follow-up in the ever-exposed population, critical review and appropriate modification of the original analytical procedures did not confirm this as a true risk. This example highlights that careful consideration of the dynamic nature of registries is necessary to generate clinically meaningful data.

Some of the limitations described here are inherent in the design of all observational studies, such as the lack of randomization resulting in substantial imbalances between treatment cohorts at baseline and the potential impact on long-term outcomes. Many are commonly encountered in observational studies (e.g. patient participation and treatment selection bias, inability to adjust for potentially relevant covariables, and the complex effects of switching treatments during enrollment) and have been documented in reports of PSOLAR and other psoriasis registries [20–30]. Careful planning of study design, particularly data collection and statistical methodology, during protocol development, can abate many of these issues.

Conclusion

Our findings confirm that, although they are critically important in the overall evaluation of pharmacologic products, observational studies require ongoing evaluation and possible adaptation to ensure that study design and analytical methods continue to be appropriate in order to yield valid results through the lifecycle of the study.

Declarations

Funding

PSOLAR was supported by Janssen Scientific Affairs, LLC.

Conflicts of interest

Robert Bissonnette is an Advisory Board Member, Consultant, Speaker, Investigator for, and/or received honoraria or grants from AbbVie, Almirall, Anaptys Bio, Arcutis, Arista, Bausch Health/Valeant, Boehringer Ingelheim, Boston Pharma, Bristol-Myers Squibb, Dermavant, Eli Lilly, Escalier, Janssen, Kineta, Kyowa Kirin, LEO Pharma, Pfizer, Regeneron, Sienna and UCB. Alice B. Gottlieb has received honoraria as an advisory board member and consultant for Avotres Therapeutics, Belersdorf, Boehringer Ingelheim, Bristol-Meyers Squibb Co., Incyte, Janssen, Leo Pharma, Eli Lilly, Novartis, Sun Pharmaceutical Industries, Inc., UCB, and Xbiotech (only stock options that she has not used); and has received research/educational grants from Boehringer Ingelheim, Incyte, Janssen, Novartis, UCB, Xbiotech, and Sun Pharma. Richard G. Langley has received honoraria or grants as principal investigator, scientific advisory board member, and lecturer for AbbVie, Amgen, Celgene, Leo, Merck, Novartis, and Pfizer; and as principal investigator and scientific advisory board member for Boehringer Ingelheim, Lilly, and UCB. Craig L. Leonardi has received honoraria or grants as an consultant/advisory board member for AbbVie, Amgen, Boehringer-Ingelheim, Dermira, Eli Lilly, Janssen, Leo, Pfizer, Sandoz, UCB and Vitae; as an investigator for Actavis, AbbVie, Allergan, Amgen, Boehringer-Ingelheim, Celgene, Coherus, Cellceutix, Corrona, Dermira, Eli Lilly, Galderma, Glenmark, Janssen, Leo Pharma, Merck, Novartis, Novella, Pfizer, Sandoz, Sienna, Stiefel, UCB and Wyeth; and as a speaker for AbbVie, Amgen, Celgene, Eli Lilly, Janssen, Novartis, Ortho Dermatologics, Sun Pharmaceuticals, and UCB. Kim A. Papp has received honoraria and/or grants from AbbVie, Akros, Amgen, Arcutis, Astellas, AstraZeneca, Avilion, Baxalta, Boehringer Ingelheim, Bristol-Myers Squibb, Canfite, Celgene, Centocor, Coherus, Dermira, Dow Pharma, Eli Lilly and Company, Forward Pharma, Galderma, Genentech, Gilead, GlaxoSmithKline, Janssen, Kyowa Hakko Kirin, LEO, MedImmune, Meiji Seika Pharma, Merck (MSD), Merck-Serono, Mitsubishi Pharma, Moberg Pharma, Novartis, Pfizer, PRCL Research, Regeneron, Roche, Sanofi Aventis/Genzyme, Samsung Bioepsis, Sun Pharma, Takeda, UCB Pharma, and Valeant/Bausch Health. David M. Pariser has received honoraria or grants/research funding as an investigator at Abbott Laboratories, Almirall, Amgen, AOBiome LLC, Asana Biosciences LLC, Bickel Biotechnology, Celgene Corporation, Dermavent Sciences, Dermira, Eli Lilly and Co., Leo Pharma, Menlo Therapuetics, Merck & Co., Novartis Pharmaceuticals, Novo Nordisk A/S, Ortho Dermatologics, Pfizer Inc., Regeneron, Stiefel (a GSK company), and Valeant Pharmaceuticals International; as an consultant at Atacama Therapeutics, Bickel Biotechnology, Biofrontera AG, BMS, Celgene, Dermira, Leo Pharma, Novartis, Pfizer, Regeneron, Sanofi, TDM SurgiTech, TheraVida, and Valeant Pharmaceuticals International; as an advisory board member at Pfizer; and as a Data Safety Monitoring Board member at BMS. Jonathan Uy, Kim Parnell Lafferty, Wayne Langholff, Steven Fakharzadeh, Emily S. Brouwer, and Andrew J. Greenspan are employees of Janssen, of which Johnson & Johnson is the parent company, and own stock or stock options in Johnson & Johnson. Jesse A. Berlin is an employee of Johnson & Johnson and owns stock or stock options in Johnson & Johnson. Bruce E. Strober has received honoraria or research grants as a consultant for AbbVie, Almirall, Amgen, Arcutis, Arena, Aristea, Boehringer Ingelheim, Bristol-Myers-Squibb, Celgene, Dermavant, Dermira, Equillium, Janssen, Leo, Eli Lilly, Meiji Seika Pharma, Mindera, Novartis, Pfizer, GlaxoSmithKline, UCB Pharma, Sun Pharma, Ortho Dermatologics, Regeneron, and Sanofi-Genzyme; as a speaker for AbbVie, Amgen, Eli Lilly, Janssen, and Ortho Dermatologics; as scientific director for the Corrona Psoriasis Registry; as an investigator for Dermavant, AbbVie, Corrona Psoriasis Registry, Dermira, Cara, and Novartis; and as Editor-in-Chief at the Journal of Psoriasis and Psoriatic Arthritis.

Ethics approval

The registry protocol was approved by an Institutional Review Board or Ethics Committee at all study sites.

Consent to participate

Written informed consent was provided by all patients before participating in the study.

Consent for publication

All authors approved the final version of the manuscript and agreed to submit for publication.

Data availability statement

The data sharing policy of Janssen Pharmaceutical Companies of Johnson & Johnson is available at https://www.janssen.com/clinical-trials/transparency.

Code availability

Not applicable.

Author contributions

RB, RGL, CLL, KAP, DMP, WL, and BS contributed to the study conception and design of the registry. KAP and WL led the data collection and analysis. All authors drafted the text and/or provided critical review of drafts and approved the final manuscript.