Use of Data from Multiple Registries in Studying Biologic Discontinuation: Challenges and Opportunities

Abstract

Many biologic disease-modifying antirheumatic drug (DMARD) discontinuation studies have been conducted, but mainly in trial settings which result in limited generalizability. Registry studies can complement the current literature of biologic DMARD discontinuation by providing more generalizable information. However, registries may need to be combined to increase power and provide a more diverse patient population. This increased power could provide us information about risk and benefits of discontinuing biologic DMARD in typical clinical practice. However, use of multiple registries is not without challenges. In this review, we discuss the challenges to combining data across multiple registries, focusing on biologic discontinuation as an example. Challenges include: 1) generalizability of each registry; 2) new versus prevalent users designs; 3) outcome definitions; 4) different health care systems; 5) different follow up intervals; and 6) data harmonization. The first three apply to each registry, and the last three apply to combining multiple registries. This review describes these challenges, corresponding solutions, and potential future opportunities.

Prior biologic discontinuation studies

Many biologic disease-modifying antirheumatic drug (DMARD) discontinuation studies in rheumatoid arthritis (RA) patients have been conducted to date [1, 2]. In our previous review summarizing 14 such studies [1], we categorized them into three groups: (a) randomized controlled trials, in which discontinuation and continuation strategies were randomly assigned; (b) single arm prospective studies of discontinuation, in which patients were prospectively recruited for biologic discontinuation; and (c) long-term extension of efficacy trials, in which patients who discontinued biologic DMARDs were observed. Many of these studies were conducted in rather specialized settings that may not be fully representative of typical clinical practice. In addition, patients from clinical trials can differ in important ways from general clinic populations, for example disease activity and presence of comorbidities that may impact the success of discontinuation of therapy. The current evidence would be supplemented with information gained from more generalizable sources, such as registries.

Definition of registries

One paper [3] stated that the term registry is often loosely used to mean “any database storing clinical information collected as a byproduct of patient care”, and defined a medical data registry as “system functioning in patient management or research, in which a standardized and complete dataset including associated follow-up is prospectively and systematically collected for a group of patients with a common disease or therapeutic intervention”.

In the “User’s Guide” published by Agency for Healthcare Research and Quality (AHRQ) [4], registry was defined as “an organized system that uses observational study methods to collect uniform data (clinical and other) to evaluate specified outcomes for a population defined by a particular disease, condition, or exposure, and that serves one or more predetermined scientific, clinical, or policy purposes”. Others have defined registries as “longitudinal observational cohorts, typically prospective, which enroll patients with a specific purpose; it could either be drug- or disease-based, or both” [5].

For practical purposes, we define a registry as a longitudinal follow-up database consisting of clinical data collected as a byproduct of usual care. By “usual care”, we mean typical clinical practice where treatment decisions are made by patients and physicians rather than predefined study protocols.

Registries enroll subjects based on a particular disease, condition, or exposure [4], Product registries, health services registries, disease or condition registries, and combinations of these are examples. In the case of biologic discontinuation studies, both biologic DMARD registries (product registries) and RA registries (disease registries) can be utilized.

Studies combining multiple registries

Particularly after the introduction of biologic DMARDs, there has been increased interest in use of registries in studying real-life long-term effectiveness and safety of these agents [5], since randomized controlled efficacy trials do not provide sufficient answers to these questions due to the restrictive nature of their inclusion criteria and follow-up [6–8]. Combining multiple databases together can improve power and has been used in studying rare diseases, rare exposures, and rare outcomes; for example, a rare neurodevelopmental disorder [9] and rare environmental exposures, such as infrequently applied pesticides can be well studied in combined registries [10]. In rheumatology, the European Collaborative Registries for the Evaluation of Rituximab in rheumatoid arthritis (CERERRA) initiative for rituximab use in daily practice in Europe is an example [11]. This study addressed the effectiveness of rituximab using 10 European cohorts, resulting in a large patient sample (n = 2019), which would not have been possible in any one of these registries or countries alone. Comparing across registries may also be used to reveal regional or national differences in diseases and treatment practice. Similarly, the increased power from multiple registries is useful for biologic DMARD discontinuation studies because the numbers of eligible patients, i.e., those who have discontinued biologic DMARDs in good disease control, are expected to be few in typical practice.

Nevertheless, when using data from combined registries, we are faced with several challenges; some of them are challenges to all registries (challenges 1–3 below) and some are methodological complexities specific to combining registries (challenges 4–6 below).

Challenge 1) Generalizability of each registry

Generalizability as a particular strength of registry studies is dependent on the source population from which the registry enrolls subjects and how these subjects are enrolled. If the source population is not the typical RA patient on a biologic DMARD results will not be generalizable. The representativeness of the biologic DMARD users in a given registry is dependent on how these subjects compare to the population of biologic DMARD users in the country. Some registries contain (almost) all biologic DMARD users in a given country, for example the British Society for Rheumatology Biologics Register (BSRBR) [12]. Registries that are not directly required by the health care system usually enroll patients from one or several participating institutions (or practices) and may capture patients associated with rheumatologists involved with research, not representative of all rheumatology practice. Unless the sample of patients is truly random, there is the potential for bias in the acquisition of patients that could impact the results. These points need to be examined before claiming the generalizability of information obtained from the registry. To ensure generalizability, nationally (or internationally) representative registries that enroll wide range of patients at multiple centers are preferable [13–15].

Challenge 2) New users versus prevalent users designs

When studying comparative effectiveness of two active agents, choosing new users of both agents is important for ensuring exchangeability [16]. Biologic DMARD registries are usually comprised of new users of biologic DMARDs, as the U.K.’s BSRBR [12] or postmarketing surveillance registries in Japan [17–20]. In contrast, disease-based RA registries may enroll prevalent RA cases already using biologic DMARDs. If the enrollment date of patients is after the initiation of a biologic DMARD, information prior to initiation is often incomplete.

However, this is less problematic for discontinuation studies where the study index date is typically defined as the time at discontinuation of therapy. Sensitivity analysis comparing new users only design to both new users and prevalent users design is recommended if there is a suspicion that prevalent users and new users may have had different baselines before use of biologic DMARDs.

Challenge 3) Outcome definition

Studying outcomes that are not directly related to the primary reason for which the registry was started can present challenges, as endpoints may not be collected in a direct manner. Biologic DMARD discontinuation study is usually not the primary reason for registries and thus the outcome determination may not be ideal. The definition of “failure of discontinuation” (the outcome of interest in biologic DMARD discontinuation studies) has not been standardized in previous non-registry studies. In our previous review, we examined how “failure of biologic DMARD discontinuation” was defined across various studies [1]: all studies used increase in disease activity, and many included reuse of biologic DMARDs for the definition of failure in discontinuing biological DMARDs. Moreover, the thresholds of increase in disease activity varied, and there was no consensus on whether intensification of non-biologic DMARDs or glucocorticoids should constitute failure.

In a registry study, long intervals between study visits might obscure an increase of disease activity in between visits, thus, “failure of discontinuation” could be missed by criteria that only use disease activity and biologic DMARD reuse (Figure). This is primarily why intensification in non-biologic DMARDs and glucocorticoids should be regarded as a sign of failure. The thresholds for failure should be determined in such a way that they are comparable across registries.

Figure.

In a registry study, not all clinical visits are necessarily captured as part of the study protocol. In this example, biologic discontinuation is detected at study visit 2. The definition with disease activity threshold violation and biologic DMARD reuse at study visits misses the failure that occurred between study visits 2 and 3. By including non-biologic treatment changes detected at study visit 3 as a failure criterion, the outcome of interest will be indirectly captured.

Challenge 4) Health care system differences

Due to the rapid development of new biologic DMARDs and their high cost, different countries have different biologic DMARDs approved (drug lag) and also have different policies regarding biologic DMARDs use and reimbursement, resulting in varying access to biologic DMARDs [21]. In some countries, biologic DMARDs are prescribed at the discretion of physicians and commonly used, for example, in the U.S., 43% of RA patients received biologic DMARDs in one study [22]. Biologic prescription practice is more restricted by practice guidelines that are required by health insurance providers, in some European and Asian countries for example [15, 21, 23]. In these settings, the users of biologic DMARDs are expected to differ. In more restrictive prescription setting, there may be fewer early RA patients compared to long-standing RA patients. Such patients may have different patterns of treatment response both before and after discontinuation of biologic DMARDs. Also, in some countries, including the USA, patients may pay directly for a portion of their drug costs (i.e., co-payment). This could impact their decision on whether and when to stop particular therapies. Finally, in the not-too-distant future, discontinuation of biologics will likely become incorporated into treatment recommendations and individual country guidelines, which will also have an effect on the data.

This could potentially cause a problem if pooling data, but it may also be possible to take advantage of these differences to compare different treatment strategies. Thus, if registries from different health care systems are to be studied, the guidelines regarding biologic DMARD prescription should be assessed for expected prescription pattern differences. If the difference is substantial enough to make biologic DMARD users in these registries very different, direct pooling of data should be avoided. The focus should be a cross-registry comparison, which can potentially provide interesting “natural experiments” in different treatment strategies. Modified meta-analysis using primary data is another possibility in such a situation [24].

Challenge 5) Different follow up intervals

The intervals of follow-ups can be different among registries, for example some registries may have information on every physician visits while the other may collect information on a less frequent basis (i.e., every 6 months, annually, etc). Therefore, when combining data across multiple registries, it is very likely that assessment timepoints vary. This can be further complicated by missing values, giving rise to unbalanced data even within each registry.

To overcome this, one can use the “least common denominator” approach by simply focusing on the longest of all available intervals, but much data would be thrown away through such an approach. A more appropriate approach may be to use analytical methods that can accommodate different intervals for individual patients, such as (generalized) linear mixed effect models for repeatedly measured binary outcomes [25] or extended Cox models for time-to-event outcomes [26], which can accommodate time-varying variables.

Challenge 6) Data harmonization

One purpose of a biologic discontinuation study is to identify variables that can predict continued disease control after cessation of biologic DMARD. If we can predict patients that can successfully discontinue biologic DMARDs, it can reduce drug exposure and its associated risks and costs. To develop such prediction model across registries it would require variables measured differently to be matched. This process is often called “data harmonization”, and there is debate about requirements for data harmonization [27].

The most robust way of harmonizing variables is to prospectively design multiple registries with harmonization in mind. This so called “stringent approach to harmonization” [27], requires collaboration before registries are started and would be very time-consuming. This will result in higher quality data, but may cancel out one benefit of registry studies, namely prompt access to data that can be utilized quickly.

The “flexible approach to harmonization” [27], on the other hand, is an effort to match variables in previously collected data. For example, in the case of biologic DMARD discontinuation studies, one element of the composite outcome (see Challenge 3) is the disease activity. Different disease activity measures have been used in different registries, for example, Disease Activity Score 28 with erythrocyte sedimentation rate (DAS28-ESR) [14], DAS28 with C-reactive protein (DAS28-CRP) [28], and Clinical Disease Activity Index (CDAI) [13]. These measures correlate well in biologic DMARD users [29], but the thresholds for remission and low disease activity have different characteristics depending on the measures used [30, 31]. To overcome this challenge, the collection of each component of the composite scores (such as joint counts) might be useful to recalculate a desired composite score. If harmonizing scores are difficult, one could also consider harmonizing the disease activity categories (remission, low disease activity, etc) or treatment response categories [32].

Discussion and future direction

Combining data from multiple registries may be useful to study outcomes as biological DMARD discontinuation. Nevertheless several potential challenges need to be addressed, as we discussed above (summarized in Table). Registry studies can give us insights into biologic DMARD discontinuation patterns and outcome in real-life practice settings, which could supplement currently available evidence.

Table.

Challenges and solutions for (multiple) registry studies of biologic DMARD discontinuation.

Challenge	Solution
1) Generalizability of registries	Check if the source population for the registry is typical population of biologic DMARD users.
2) New vs prevalent users of biologic DMARDs	Prevalent users can be used as long as they are new to discontinuation. Sensitivity analysis is recommended.
3) Outcome definition	Changes in non-biologic DMARDs should be incorporated in a composite “failure of discontinuation” definition.
4) Different health care system	If registries are from very different health care systems with different utilization patterns of biologic DMARDs, comparison rather than pooling is preferred.
5) Different follow up intervals	Analysis methods that can accommodate “unbalanced” longitudinal data with varying follow up intervals should be used.
6) Data harmonization	Variables should be matched as raw variables (swollen joint count, etc) rather than composite variables such as disease activity scores if possible.

The first three are challenges to all registries (challenges 1–3), and the last three are methodological complexities specific to combining registries (challenges 4–6).

This approach has strengths compared to ordinary meta-analysis (result-level combination of published study results) in some aspects. Firstly, it enables more careful examination of the heterogeneity of the subjects by using the individual-level data. Secondly, it allows better adjustment for these baseline differences. Thirdly, some variable heterogeneity can be fixed by redefining variables using raw individual-level data. However, when the sample sizes of data sources are very different, small data sources may be overshadowed by larger ones, thus, modified random-effects meta-analysis using primary data may be preferable [24].

The use of multiple registries, is not limited to biologic DMARD discontinuation studies. Research questions that require generalizable clinical information and large sample sizes can potentially benefit from combining datasets. Potential examples include studies of rare exposures such as very newly introduced medications or rare outcomes such as some toxicities. In addition, cross-national comparisons using multiple registries can answer interesting health services questions, as well as providing natural experiments through treatment variation.

In conclusion, the use of multiple registry data studies could offer precious opportunities for studying biologic DMARD discontinuation and beyond.