Bayesian comparative effectiveness study of four consensus treatment plans for initial management of systemic juvenile idiopathic arthritis: FiRst Line Options for Systemic juvenile idiopathic arthritis Treatment (FROST)

Abstract

Background

Systemic juvenile idiopathic arthritis is a rare febrile arthritis of childhood characterized by a potentially severe course, including prolonged glucocorticoid exposure, growth failure, destructive arthritis, and life-threatening macrophage activation syndrome. Early cytokine-blocking biologic therapy may improve long-term outcomes, although some systemic juvenile idiopathic arthritis patients respond well to non-biologic treatment, leaving optimal management undefined. Consequently, treatment of new-onset systemic juvenile idiopathic arthritis by expert clinicians varies widely.

Purpose

To describe a pragmatic, observational comparative effectiveness study that takes advantage of diversity in the management of a rare disease: FiRst-Line Options for Systemic juvenile idiopathic arthritis Treatment (FROST), comparing non-biologic and biologic consensus treatment plans for new-onset systemic juvenile idiopathic arthritis within the 60-center Childhood Arthritis and Rheumatology Research Alliance Registry.

Methods

FROST is a multicenter, prospective, non-randomized study that compares four Childhood Arthritis and Rheumatology Research Alliance consensus treatment plans for new-onset systemic juvenile idiopathic arthritis: 1) glucocorticoids alone, 2) methotrexate, 3) IL-1 blockade, 4) IL-6 blockade. Patients consenting to participation in the Childhood Arthritis and Rheumatology Research Alliance Registry are started on one of four Consensus Treatment Plans at the discretion of the treating physician. The outcome of primary interest is clinically inactive disease off glucocorticoids at 9 months, comparing non-biologic (Consensus Treatment Plan 1+2) vs. biologic (Consensus Treatment Plan 3+4) strategies. Bayesian analytic methods will be employed to evaluate response rates, using propensity scoring to balance treatment groups for potential confounding. With 200 patients in a 2:1 ratio of biologic to non-biologic, there is a >90% probability of finding biologic consensus treatment plans more effective if the rate of clinically inactive disease is 30% higher than for non-biologic therapy. Additional outcomes include Patient-Reported Outcomes Measurement Information System measures and other parent/patient reported outcomes reported in real time using smartphone technology. Routine operation of the Childhood Arthritis and Rheumatology Research Alliance Registry will allow assessment of outcomes over at least 10 years.

Results

FROST began enrollment in November 2016.

Limitations

The observational design may not provide balance in measured and unmeasured confounders. Use of Consensus Treatment Plan strategies at frequencies more unbalanced than predicted could reduce the chance of finding differences in efficacy.

Conclusions

FROST will provide the first prospective comparison of Childhood Arthritis and Rheumatology Research Alliance’s consensus-derived non-biologic vs. biologic management strategies in systemic juvenile idiopathic arthritis, performed in a real-world setting wherein each patient receives standard-of-care treatment selected by the treating physician. Outcomes include clinician- and patient/family-reported outcomes, empowering both physician and patient decision making in new-onset systemic juvenile idiopathic arthritis.

TRIAL REGISTRATION

NCT02418442

Introduction

Juvenile idiopathic arthritis is the most common rheumatic disease of childhood. Although multiple therapeutic options are available, in many clinical scenarios the optimal treatment remains unknown. This knowledge gap is particularly acute for the most severe form of childhood arthritis, systemic juvenile idiopathic arthritis. Encompassing approximately 10% of all juvenile idiopathic arthritis in North America, systemic juvenile idiopathic arthritis is characterized by high spiking fevers and rash in addition to arthritis.¹ Additional manifestations include lymphadenopathy, hepatosplenomegaly, marked systemic inflammation, and in some patients a life-threatening “cytokine storm” termed macrophage activation syndrome. Approximately half of patients follow a chronic course, during which fever, rash and other systemic symptoms fade but chronic destructive arthritis persists. Such patients are at high risk of long-term disability, both from joint destruction and from growth restriction due to systemic inflammation and high-dose glucocorticoid therapy.^2,3 The term “adult onset Still’s disease” is employed to describe the same syndrome when it arises in adulthood.⁴

Over the last decade, advances in the therapy of systemic juvenile idiopathic arthritis have improved the prognosis for many patients. Small clinical series showed that interleukin 1 (IL-1) blockade can abrogate fever in the majority of patients, while also contributing to amelioration of joint inflammation.^5–8 IL-6 has also been strongly implicated in disease pathogenesis.⁹ The utility of antagonists for these cytokines was confirmed in prospective randomized clinical trials (RCTs), leading to United States Food and Drug Administration approval for canakinumab (anti-IL-1β) and tocilizumab (anti-IL-6 receptor).^10–12 Smaller RCTs confirmed more modest benefit for the recombinant IL-1 receptor antagonist anakinra as well as the IL-1 “trap” rilonacept, although neither has yet obtained approval in the United States for this indication.^13,14

Despite these advances, substantial uncertainty surrounds the optimal use of biologic agents in systemic juvenile idiopathic arthritis. Approximately 10–40% of systemic juvenile idiopathic arthritis patients manifest a monophasic course, entering sustained remission within 12–24 months.^2,15 These patients may exhibit acceptable outcomes without biologic therapy. However, first-line biologic therapy may minimize harmful glucocorticoid exposure, and has been proposed to take advantage of a “window of opportunity” during which cytokine blockade prevents the development pathogenic T lymphocytes responsible for chronic arthritis.^16,17 Nevertheless, the risk/benefit ratio of non-biologic vs. biologic therapy is unknown. Safety data from long-term extension studies of the RCTs of tocilizumab and canakinumab will be limited by small numbers, absence of controls receiving non-biologic alternatives, and a patient population with established rather than new-onset disease. For example, infections and macrophage activation syndrome have been observed in systemic juvenile idiopathic arthritis patients receiving either non-biologic or biologic therapy, as has life-threatening lung disease including pulmonary hypertension.^10,11,18,19 Further, biologic therapies are expensive and must be administered parenterally, in the case of anakinra by daily subcutaneous injection, negatively affecting patient quality of life. Thus it remains unclear whether the balance of safety and efficacy favors biologic or non-biologic management for new-onset systemic juvenile idiopathic arthritis.

This paucity of evidence translates into marked practice variation within the pediatric rheumatology community. A 2010 survey conducted by the Childhood Arthritis and Rheumatology Research Alliance (CARRA), a 140+ center research network that encompasses the large majority of pediatric rheumatologists in the United States and Canada, found that depending on presenting characteristics and disease severity, between 5–40% of respondents chose biologic therapy first for new-onset systemic juvenile idiopathic arthritis.²⁰ Correspondingly, of 56 systemic juvenile idiopathic arthritis patients with disease duration less than 6 months enrolled in the CARRA “Legacy Registry” from 2010–2013, 57% received glucocorticoids alone; 37% a disease modifying drug, usually methotrexate; 25% IL-1 blockade; and 5% IL-6 blockade.²¹ Both biologic therapy and non-biologic alternatives are considered standard-of-care first-line options by the American College of Rheumatology.^22,23

This therapeutic variation renders systemic juvenile idiopathic arthritis particularly suitable for comparative effectiveness research. If the treatment received depends primarily on who provides care, rather than clinical presentation, then practice heterogeneity provides the opportunity to observe how similar patients fare when treated differently.^24,25 The present study, entitled FROST (FiRst-line Options for Systemic juvenile idiopathic arthritis Treatment), employs this approach to study new-onset systemic juvenile idiopathic arthritis within the CARRA Registry, an observational registry launched in 2015 in 60 pediatric rheumatology centers across the United States and Canada.²⁶

To facilitate FROST, the CARRA juvenile idiopathic arthritis workgroup first sought to develop standardized consensus treatment plans that reflect standard-of-care practice while facilitating comparison of different approaches to treatment. Greater than 80% consensus was achieved within the full CARRA membership on four treatment approaches to new-onset systemic juvenile idiopathic arthritis without active macrophage activation syndrome for whom nonsteroidal anti-inflammatory drugs are deemed insufficient initial therapy: 1) glucocorticoids alone; 2) methotrexate, with or without glucocorticoids; 3) IL-1 inhibition, with or without glucocorticoids; 4) IL-6 inhibition, with or without glucocorticoids (Table 1).²⁰ In a pilot study of CARRA systemic juvenile idiopathic arthritis consensus treatment plan implementation across 13 CARRA centers between 2011 and 2014, 27% of patients (8/30) were allocated to non-biologic therapy (2 glucocorticoids alone, 6 methotrexate: 6/6 + glucocorticoids) while 73% (22/30) received first-line biologic therapy (12 IL-1 inhibition: 7/12 + glucocorticoids, 10 IL-6 inhibition: 2/10 + glucocorticoids). The main predictor of treatment was the center in which the patient received treatment, rather than clinical features, supporting the assumption that physician choice can be employed to “pseudo block randomize” new-onset systemic juvenile idiopathic arthritis patients among treatment arms.²⁷

Table 1.

FROST Consensus Treatment Plans

Consensus Treatment Plan 1: Glucocorticoid Prednisone 1mg/kg (max 60mg) daily Optional IV methylprednisolone pulse 30mg/kg (max 1g) IV daily for 3 days

Consensus Treatment Plan 2: Methotrexate Methotrexate 0.5mg/kg (max 15mg) PO or SQ weekly Optional glucocorticoids: prednisone 1mg/kg (60mg max) +/− methylprednisolone pulse 30mg/kg (max 1g) IV daily for 3 days

Consensus Treatment Plan 3: IL-1 inhibitor Anakinra 2–5 mg/kg (max 100mg) SQ daily, OR Canakinumab 4mg/kg (max 300mg) every 4 weeks Optional glucocorticoids: prednisone 1mg/kg (60mg max) +/− IV methylprednisolone pulse 30mg/kg (max 1g) IV daily for 3 days

Consensus Treatment Plan 4: IL-6 inhibitor Tocilizumab 8mg/kg (if >30kg) or 12 mg/kg (if <30kg) IV every 2 weeks Optional glucocorticoids: prednisone 1mg/kg (60mg max) +/− IV methylprednisolone pulse 30mg/kg (max 1g) IV daily for 3 days

Recruitment for FROST began in November 2016. The design, implementation, and Bayesian analysis plan of FROST are described here, illustrating an approach for the multicenter investigation of a rare disease characterized by substantial diversity in management.

Hypotheses

The hypothesis of FROST is that first-line biologic therapy (IL-1 or IL-6 blockade) will more effectively achieve clinically inactive disease than non-biologic therapy (glucocorticoids or methotrexate), reducing overall glucocorticoid exposure and improving clinical outcomes and patient quality of life as assessed by patient/caregiver- and physician-reported outcomes, with an acceptable risk profile. FROST thereby addresses the questions of greatest relevance to patients and families: What is the most effective therapy for new-onset systemic juvenile idiopathic arthritis, and is it safe for my child?

An additional goal of FROST is to confirm the feasibility of consensus treatment plan-based observational comparative effectiveness research embedded in the CARRA Registry to define optimal management of rare rheumatologic diseases of children, including efficacy, tolerability and safety.

Study design

The CARRA Registry

CARRA is a research network of pediatric rheumatology centers that encompasses >95% of practicing pediatric rheumatologists in the United States and Canada. In 2010, the CARRA created a pediatric rheumatic diseases registry that enrolled almost 9500 subjects from 62 sites, 528 of whom had systemic juvenile idiopathic arthritis.²¹ This “Legacy Registry” was replaced in July 2015 with the new CARRA Registry, an expanded program compatible with collection of prospective, long-term observational data in a format that fulfills requirements for United States Food and Drug Administration-mandated post-marketing (Phase IV) surveillance.^26,28 The data coordinating center is the Duke Clinical Research Institute. Written informed consent for Registry participation is provided by patients themselves if age 18 or older, and otherwise by parents/guardians, with assent from the child participant depending on age (typically 9 years and older). Participants consent to longitudinal data gathering from the medical record, as well as to providing Patient-Reported Outcome Measure (Table 2) and to telephone surveys into adulthood for longer-term monitoring of clinical status at least every 6 months for 10 years. Clinicians document medication start and stop dates, clinical status and characteristics, and adverse events that are either serious or of specific interest. Biosample collection associated with the Registry, including blood, urine, buccal cells, oral flora and stool, is permitted in patients who provide additional consent. The Registry protocol was developed to encompass multiple rheumatic diseases in children, as well as scalable data collection, to enable its use for observational sub-studies such as FROST.²⁵

Table 2.

FROST Patient Reported Outcome Measures

Item	Notes	Collection visits (months)
Collected at Visit
CHAQa	ACRb; 34 items; Past 7 days	0, 3, 6, 9, 12, 18, 24
Pain Intensity	3 items; Current to past 14 days	0, 0.5, 1, 3, 6, 9, 12, 15, 18, 21, 24
Pain Interference	PROMISc; 8 items; past 7 days	0, 1, 3, 6, 9, 12, 18, 24
Pain 2/2 Rheum Disease	1 item; Past 7 days	0, 0.5, 1, 3, 6, 9, 12, 15, 18, 21, 24
Patient/Parent Global Well-being	ACR; 1 item; Current	0, 0.5, 1, 3, 6, 9, 12, 15, 18, 21, 24
Parent Disease Activity	1 item; Past 7 days	0, 0.5, 1, 3, 6, 9, 12, 15, 18, 21, 24
Pediatric Global Health 7	PROMIS; 7 items; past 7 days	0, 1, 3, 6, 9, 12, 18, 24
Physical Function Mobility	PROMIS; 8 items; past 7 days	0, 3, 6, 9, 12, 18, 24
Physical Function Upper Extremity	PROMIS; 8 items; past 7 days	0, 3, 6, 9, 12, 18, 24
Fatigue	PROMIS; 10 items; past 7 days	0, 3, 9, 12, 18, 24
Depressive Symptoms	PROMIS; 6 items; past 7 days	0, 9, 24
Anxiety	PROMIS; 8 items; past 7 days	0, 9, 24
Medication Adverse Events	JAMAR; 1 item; current	0, 0.5, 1, 3, 6, 9, 12, 15, 18, 21, 24
PedsQL Family Impact	36 items; past 1 month	0, 9, 24
Home Collection
Fever	1 item	Every 2 days for 2 weeks; then every week for weeks 3–12
Rash	1 item	Every 2 days for 2 weeks; then every week for weeks 3–12
Pain	1 item	Every 2 days for 2 weeks; then every week for weeks 3–12
Oral steroid dose	1 item	Every week for months 0–9

^aCHAQ: Child Health Assessment Questionnaire

^bACR: American College of Rheumatology Juvenile Idiopathic Arthritis Core Set

^cPROMIS^®: Patient-Reported Outcomes Measurement Information System

Ethical considerations

The CARRA Registry is approved by the Institutional Review Board or equivalent authority at each institution, either directly or via a reliant review process through the Duke University Health System Institutional Review Board. A notable feature of the study design is that patients need consent only to Registry participation, since FROST patients receive standard of care therapy selected by their treating physician with input from the patient and family, with data collection is within the scope of the CARRA Registry. This markedly simplifies the enrolment process. For patients who agree to contribute biosamples, the Registry biosample consent is sufficient.

Inclusion and exclusion criteria

Entry criteria for FROST are specified formally in the Supplemental Material. Brief, patients between the ages of 6 months to 18 years meeting the CARRA definition of systemic juvenile idiopathic arthritis²⁰ are eligible if enrolled in the CARRA registry and previously untreated for systemic juvenile idiopathic arthritis except nonsteroidal anti-inflammatory drugs and/or short-course (<2 weeks) systemic glucocorticoids. Patients are excluded for any relative or absolute contraindication to biologic therapy. Since the onset of systemic juvenile idiopathic arthritis is sometimes difficult to pinpoint, disease duration is not an exclusion criterion.

Intervention

Patients receive standard-of-care therapy at the discretion of their primary pediatric rheumatologist. Clinicians are requested to adhere to the consensus treatment plan compatible with their care plan, but may deviate as clinically indicated, with deviations documented in the study record. Clinical and laboratory data from studies performed in the course of usual care are collected at routine clinical evaluation, as close as possible to the following schedule: baseline, 1–2 weeks, and months 1, 3, 6, 9, 12, 15, 18, 21 and 24. Patients are followed for at least an additional 8 years in the CARRA Registry. At each visit, clinicians complete relevant electronic case report forms, including a specific assessment of disease activity, medication regimen and adverse events. Patient-reported outcomes are recorded on a computer tablet at visits, and home patient-reported outcomes (presence of fever, rash, pain level and prednisone dose) recorded via smartphone or tablet-based surveys (Medidata, Inc.), every 2 days for 2 weeks after starting the consensus treatment plan and then once every week for 3 months. Glucocorticoid dose is reported weekly for a total of 9 months. Patients without access to a smartphone are provided with an Apple mini iPad tablet.

Outcomes

Given the planned Bayesian analysis, a single primary outcome is not defined. However, the outcome of principal interest used to determinate sample size is attainment of clinically inactive disease off glucocorticoids at 9 months. Clinically inactive disease is defined as no active arthritis, a physician’s global assessment of disease activity score of 0, erythrocyte sedimentation rate and/or C-reactive protein in the normal range, no extra-articular features of systemic juvenile idiopathic arthritis (fever, rash, serositis, splenomegaly, or generalized lymphadenopathy), no uveitis, and duration of morning stiffness <15 minutes.²⁹ Data required to assess this composite outcome are routinely collected within the Registry. Discontinuation of glucocorticoids was included in the outcome because of the high morbidity associated with sustained treatment in children, including with respect to linear growth, bone mineralization, and other consequences. The main comparison is between non-biologic (Consensus Treatment Plans 1+2) and biologic (Consensus Treatment Plans 3+4) strategies. Additional outcomes and analyses are specified in the Supplemental Material.

Statistical analysis

Bayesian methods will be used to estimate response rates, allowing (a) continual updating of estimates of treatment effects as data accrue, without concerns for multiple comparisons; (b) direct probability statements about which consensus treatment plan has the better likelihood of success; and (c) the results to be assessed from the point of view of those who have different viewpoints (prior beliefs) about the relative effectiveness of the different consensus treatment plans.

Since this is an observational study, patient characteristics may differ between groups treated with biologic and non-biologic consensus treatment plans. Unbiased comparison of response rates will require adjustment through statistical models accomplished using binary logistic regression to construct a propensity score from baseline covariates.³⁰ The propensity score will be used in a stratified analysis to generate adjusted comparisons of response rates between groups. All analyses will use a Bayesian framework. Sensitivity analysis will examine the robustness of the results and rankings to informative, uninformative and skeptical prior distributions on the response rates, as well as alternative methods of generating (pre-specification, Bayesian model averaging) and incorporating propensity score (matching and regression).³¹ The main propensity score analyses will be carried out once there are sufficient children with the clinically inactive disease outcome at 9 months to allow for the propensity score models to include important potential confounders, recognizing that to date no clinical predictors have been identified that can distinguish good-prognosis from poor-prognosis children with new-onset systemic juvenile idiopathic arthritis.

The observational nature of FROST allows for patients to switch to a different consensus treatment plan at the discretion of the treating clinician. Further, the IL-1 blockade consensus treatment plan encompasses both anakinra and canakinumab, such that some patients may switch anti-IL-1 therapy within this consensus treatment plan, for example for inadequate response or intolerance of the drug or of its method of administration. In the real-world clinical setting of this study, patients may switch consensus treatment plans, not receive the intended treatment, or receive treatments outside of the intended consensus treatment plan for other reasons, such as failure to obtain insurance approval. Further, since the primary analysis compares non-biologic vs. biologic treatment, patients may switch consensus treatment plans within a treatment group. For example, methotrexate may be added to the regimen of a patient treated initially with glucocorticoids alone, or a patient on IL-1 blockade may be switched to IL-6 blockade. Planned management of these therapeutic changes is described in the Supplemental Material.

Descriptive analyses will characterize patient groups, utilization of different consensus treatment plans, and the responses to each consensus treatment plan. A Bayesian monitoring strategy will be employed as data accumulate to update the estimates of the response rates in each of the 4 consensus treatment plan groups. This approach will allow the 4 consensus treatment plans to be ranked as to the probability that a consensus treatment plan has the best or worst response rate. If there is a high probability that one consensus treatment plan has the worst response, a recommendation may be made to cease enrolling children to that consensus treatment plan.

To address unmeasured confounding, we will also consider instrumental variable analysis to compare response rates across consensus treatment plans using physicians’ stated frequency of initiating biologic agents at the time of diagnosis as the instrument. The instrumental variable estimates of treatment effectiveness will be compared to the Bayesian propensity score estimates but they will not allow for computing the probability that a consensus treatment plan is the best or worst, so are less useful for decisions around dropping a plan from further consideration.

Sample size considerations

The findings of the completed systemic juvenile idiopathic arthritis consensus treatment plan pilot study suggest that the frequency of strategy usage will be non-biologic (Consensus Treatment Plans 1+2) 33%; biologic (Consensus Treatment Plans 3+4) 67%.²⁷ Based on published data, summarized in the Introduction, the probability of achieving clinically inactive disease off glucocorticoids at 9 months was estimated at 0.3 for non-biologic consensus treatment plans and 0.6 for biologic consensus treatment plans. In a stratified propensity score analysis, the chance of showing a difference in effectiveness of the two strategies for a given sample size depends on the degree of imbalance between groups on the calculated propensity score (unknown until patients are enrolled), the percentages in each propensity score stratum (which we can fix at 20% in each of 5 strata), and the actual clinically inactive disease probabilities for the two consensus treatment plan groups in each propensity score stratum. We carried out a simulation study with 200 patients, divided into 66 non-biologic and 134 biologic recipients, to calculate the Bayesian power for the propensity-stratified comparison of clinically inactive disease off glucocorticoids at 9 months analyzed with a Bayesian logistic regression model. We defined a “statistically significant” benefit for the biologic consensus treatment plans as a posterior probability of at least 95% that the primary endpoint is achieved more frequently with biologic consensus treatment plans and “Bayesian power” as the proportion of 4000 simulated datasets with a “statistically significant” benefit. With moderate imbalance in the propensity score (the ratio of non-biologics to biologics ranging from 20%:80% to 46%:54% across propensity score strata), 200 patients provide a Bayesian power of 99% when the true probabilities of clinically inactive disease are 0.6 and 0.3, and 82% when these probabilities are 0.6 and 0.4. Table 3 shows estimated Bayesian power for a range of clinically inactive diseases for biologics and non-biologics for sample sizes of 150, 200 and 250. With more, but still plausible, imbalance in propensity score, there is a generally a 3–5% reduction in power (results not shown). In a set of simulations having probabilities of clinically inactive disease ranging from 0.3 to 0.5, and equal for biologics and non-biologics, the estimated frequentist type I error rate was 5%.

Table 3. Sample size considerations for binary outcome clinically inactive disease.

This table lists Bayesian power estimated from the simulation study for three sample sizes and 11 combinations of the overall probabilities of clinically inactive disease (CID) in biologic and non-biologic groups. All values in the table are percentages.

Prob(CID)		Difference in Prob(CID)	Probability of showing benefit
Biologics	Non-biologics		n=150	n=200	n=250
40	30	10	29	36	41
40	25	15	52	62	73
40	20	20	75	85	93
50	40	10	29	32	40
50	35	15	49	59	69
50	30	20	72	83	90
50	20	30	96	99	>99
60	50	10	26	34	42
60	45	15	48	59	69
60	40	20	70	82	89
60	30	30	95	99	>99

Adverse event reporting and documentation

Adverse events are collected within the CARRA Registry for pharmacosurveillance purposes on all systemic juvenile idiopathic arthritis patients, including but not limited to those enrolled in FROST. Serious adverse events as defined by the US Food and Drug Administration will be monitored and reported by CARRA in compliance with safety reporting requirements, as well as events of particular interest, including macrophage activation syndrome, opportunistic infections, pulmonary hypertension, and interstitial lung disease.

Biospecimen collection

Plasma, DNA, serum, peripheral blood mononuclear cells, and whole blood RNA will be collected at baseline and 6 months from FROST patients providing appropriate consent, as detailed in the Supplemental Materials. Samples and other research materials can be accessed through CARRA (www.carragroup.org).

Discussion

Research into rare diseases poses numerous logistical, financial and ethical challenges, especially in children. Systemic juvenile idiopathic arthritis exemplifies these hurdles. Despite compelling data for the efficacy of IL-1 and IL-6 antagonists in systemic juvenile idiopathic arthritis, the optimal role for these biologic agents in the management of new-onset disease remains unclear with respect to both efficacy and risk. This knowledge gap compels physicians and caregivers to make a “high stakes” therapeutic choice in the absence of essential information.

An uncommon subtype of childhood arthritis, newly-diagnosed systemic juvenile idiopathic arthritis is encountered at most a few times a year even in large referral centers. In this context, conducting a prospective RCT across enough centers to achieve statistical power is a daunting challenge. Recruitment is further complicated if physicians possess strongly-held beliefs about the superiority of one therapy over another. Although the community as a whole remains at equipoise with respect to optimal treatment of new-onset systemic juvenile idiopathic arthritis, individual physicians may be reluctant to expose their patients to therapies that they consider therapeutically inferior or unjustifiably hazardous.

Under such circumstances, the observational comparative effectiveness study design provides a feasible path forward.²⁵ Leveraging the existing CARRA Registry infrastructure and protocol, FROST enables recruitment across many sites with a minimum of site-specific start-up logistics and costs. As an observational study wherein treatment remains at the discretion of the physician and the patient/family, both recruitment and consent are markedly simplified. Physicians contribute to the collection of meaningful scientific data while providing patients with therapy that they consider optimal. Further advantages include the real-world patient population and clinical setting, as compared to the highly-selected subjects and artificially-controlled environment that typify the traditional RCT, as well as the opportunity for 10-year follow-up that is organizationally and financially impracticable in a stand-alone RCT. Thus, FROST addresses a significant and controversial clinical issue in the most severe form of juvenile idiopathic arthritis with a relatively modest investment of time, effort and resources, representing an innovative model that can be applied to the study of many rare diseases in both children and adults. FROST also provides the first opportunity to study canakinumab and tocilizumab systematically as first-line agents for systemic juvenile idiopathic arthritis, and to evaluate prospectively whether there is indeed a window of opportunity to alter long-term outcomes in new-onset systemic juvenile idiopathic arthritis patients.

The efficiencies of FROST are balanced against the limitations of the design. Patients are not randomized to different treatments but rather receive the consensus treatment plan that their physicians believe to be best, raising obvious issues of confounding by indication, even if (as pilot trial data show) choice is driven in practice more by physician preference than by patient phenotype.²⁷ In FROST, we address this issue through post hoc statistical correction, including propensity stratification. These corrective methods are unlikely to achieve the balance of measured and unmeasured confounders achieved by randomization. In systemic juvenile idiopathic arthritis, this problem is mitigated by the fact that, despite diligent effort, investigators have been unable to define markers at disease onset that predict course, such that even highly experienced physicians cannot identify patients at risk for poor outcomes.^2,32–36

A further limitation is that neither patient nor physician is blinded to treatment. Accordingly, the demonstrably greater placebo effect of parenteral versus oral therapies could favor biologic therapy.³⁷ However, in FROST each physician will implement the therapy in which he or she has greatest confidence, potentially minimizing differences in placebo effect as well as any attempt to “game the system” by accelerating glucocorticoid discontinuation in order make the chosen consensus treatment plan appear successful. Further, many endpoints are objective (e.g. absence of fever, discontinuation of glucocorticoids, overt macrophage activation syndrome, and laboratory markers). Thus, while lack of blinding remains an important limitation, we anticipate that its effect on the interpretability of the results will be modest.

Other limitations result from the real-world observational context. Without a rigid RCT approach, the potential absence of critical clinical or laboratory assessments may affect the measurement of outcomes. This concern is mitigated through programmed validity and consistency checks at the point of data entry and through specific follow-up data queries to the clinical sites, helping to minimize missing data.²⁶ Some patients may switch therapy, including within a therapeutic class (e.g. anakinra to canakinumab, both IL-1 antagonists) or within an arm (e.g. canakinumab to tocilizumab, both biologics). This poses a challenge to analysis, but one reflective of real-world practice. The strategy for analysis is defined in Table S1, wherein switching is considered an outcome of initial therapeutic choice to enable us to address both whether a biologic-first approach is superior to a non-biologic approach as well as to assess each consensus treatment plan individually. Marginal structural modeling may enable us to develop estimates of efficacy that account for treatment switching by balancing patient factors, but is not part of our primary analysis strategy.

If successful, FROST will pave the way for similar studies in other rare diseases. CARRA has developed consensus treatment plans not only in systemic juvenile idiopathic arthritis but also in the more common polyarticular form of juvenile idiopathic arthritis, for which a similar study funded by the Patient Centered Outcomes Research Institute (PCORI) is underway (NCT02593006), as well as in pediatric lupus nephritis, juvenile dermatomyositis, and localized scleroderma.^20,38–40 Growing facility with observational comparative effectiveness studies using published consensus treatment plans will accelerate advances in the understanding and therapy of multiple conditions included in the CARRA Registry. If successful, these studies will serve as a model and promote the use of registry-embedded comparative effectiveness studies for rare diseases outside of pediatric rheumatology, helping to facilitate a new more viable approach to clinical research. Such studies will not replace RCTs as the gold standard of evidence, but can enable rigorous investigation and therapeutic progress where RCTs are impracticable.