Key analytic considerations in design and analysis of randomized controlled trials in osteoarthritis

Abstract

Objective

To highlight methodologic challenges pertinent to design, analysis, and reporting of results of randomized clinical trials in OA and offer practical suggestions to overcome these challenges.

Design

The topics covered in this paper include subject selection, randomization, approaches to handling missing data, subgroup analysis, sample size, and issues related to changing design mid-way through the study. Special attention is given to standardizing the reporting of results and economic analyses.

Results

Key findings include the importance of blinding and concealment, the distinction between superiority and non-inferiority trials, the need to minimize missing data, and appropriate analysis and interpretation of subgroup effects.

Conclusion

Investigators may use the findings and recommendations advanced in this paper to guide design and conduct of randomized controlled trials of interventions for osteoarthritis.

Introduction

Researchers who design and conduct randomized controlled trials to establish the efficacy of treatments for OA should pay special attention to several design features highlighted in this paper: changes in trial design, blinding, defining placebo, choice of primary outcome and optimal time of outcome assessment, and prevention of informative censoring. We provide recommendations to address these issues in the design and analytic stages (Table I). We also highlight standardized reporting, economic evaluation alongside clinical trials, and value of information analysis for prioritization of research.

Table I.

Recommendations to address methodologic challenges in osteoarthritis clinical trials.

Domain	Recommendation
	Design	Analysis
Selection of primary endpoint	Clearly define primary outcome (endpoint), means of measuring, timing of assessment. Ensure that selected outcome is acceptable by regulatory organizations and scientific community. Do not select outcomes that did not undergo a thorough evaluation regarding validity and reliability	Plan and execute primary analysis focused on selected a priori primary outcome, which should be clearly and unambiguously defined. Always report results on primary outcome.
Choice of study design	Decision between superiority and non-inferiority designs should be governed by the novelty of treatment modality and choice of control group. If the treatment modality under investigation is compared to placebo – superiority design is the design of choice. In situations where the treatment under investigation is compared to another active treatment non- inferiority design are chosen if new treatment is likely to have similar efficacy but offers better tolerability and safety profile	Reporting should be consistent with chosen study design. The results of superiority and, in particular, non-inferiority trials are best presented using two-sided 95% confidence intervals.
Blinding and Allocation Concealment	When possible, both participants and the research team should be blinded to treatment assignment. When this is not possible, those ascertaining the outcome and analyzing the data should be blinded. Design features such as a varying block randomization scheme and sequentially numbered, opaque, sealed envelopes should be utilized to ensure allocation concealment.	Unblinding of data should not take place until the trial is terminated and data are cleaned to an acceptable level of quality. If an interim analysis is performed, this must be a completely confidential process by independent statisticians and conducted with blinded data.
Randomization	Randomization should be performed using a computer random number generator, incorporating strategies to ensure concealment (e.g. varying random blocks, sequentially numbered, opaque sealed envelopes).	Primary analysis should use intention to treat principal, at least in superiority trials. The intention-to-treat population plays a slightly different role in non-inferiority trials. It is usually a good recommendation to present results for both the intention-to-treat population and the per-protocol population.
Sample size	Careful sample size calculations should be made before the start of the RCT, and should take into account the hypothesized difference in primary outcome between the treatment and control groups and the variability in outcome. It is important that investigators consider an effect size that is realistic, but also clinically important. Sample size calculations should also take into account secondary outcomes and subgroup analyses.	In some cases analytic sample size calculations cannot be easily performed because of methodological complexity. It may then be a good alternative to use sample size simulation instead.
Missing data	Strategies should be put in place at the design phase to minimize missing data. These include limited burden of data collection on study participants, incentives, and clear protocols for study staff to follow for contacting participants. Detailed reason for dropout should be recorded, and participants wishing to discontinue their assigned intervention should be given the opportunity to continue study assessments.	Last observation carried forward and complete case analyses require strong assumptions and should be avoided unless there is strong justification. Likelihood based approaches such as mixed-effects models and multiple imputation are valid approaches when the missing data is MAR. Sensitivity analyses should be utilized to assess the robustness of the results to this assumption.
When to consider design changes	Design changes should be kept to a minimum and be pre-specified in the study protocol. Such pre-specified design changes are usually initiated by the results of an interim analysis, especially regarding the sample size and termination of a trial.	Interim analyses may be logistically complicated as they usually require independent statistical analysis. Avoid interim analyses, especially if they are not pre-specified.
Reporting	The study protocol can be published in a journal, and a short version of it should be registered at a public trial registry accepted by the ICMJE and WHO prior to the first patient’s randomization.	The statistical analysis and results reporting should comply with the CONSORT statement’s requirements and methodological recommendations provided by regulatory authorities. When submitting a manuscript to a scientific journal, include a copy of the study protocol and a completed CONSORT statement checklist and flow chart.

Osteoarthritis trials

Pharmacological treatment for OA consists both of symptom modifying and structure modifying drugs. Non-pharmacologic treatments generally focus on symptomatic relief and functional restoration, but can also be evaluated for structural modification, using biochemical or imaging markers. For example, the IDEA trial focused on efficacy of weight management and exercise for pain reduction, but also showed impact on inflammatory and MRI biomarkers¹.

For structure modifying interventions, clinical endpoints are also important. For example, an endpoint of the structure modifying intervention may be time to joint replacement. The bona fide clinical endpoint (e.g. reduction in joint space) may be substituted by a biomarker if the treatment’s effect on this biomarker correlates with the treatment’s effect on the bona fide endpoint. This correlation, the surrogacy value of the biomarker, must be shown before the biomarker is used as a primary endpoint².

Trials of symptom and structure modifying treatments typically have a randomized, double blind, parallel group design. These trials are recommended to be placebo controlled. Symptom modifying treatments are also recommended to have an established active comparator as a ‘usual care’ control³. Due to ethical considerations the comparison with placebo in the presence of another active treatment that has been shown to be more efficacious than placebo is not advisable. The goals of the equivalence or non-inferiority trials are to show that the new treatment is no worse than accepted active comparator, or, in the case where new treatment is compared with placebo, that all treatment effect is just due to placebo. The planning and conducting of such trials may include discussions about interim analyses and design changes. To ensure success of the trial, it is critical to define the key clinical hypothesis of interest and design the trial to provide conclusive results regarding this hypothesis.

Allocation concealment and blinding

The purpose of concealed allocation is to avoid selection bias. Investigators should provide sufficient detail on how allocation concealment was achieved so the reader can determine the likelihood of success. The assessment of treatment concealment is especially relevant in RCTs with subjective patient reported outcomes. Allocation concealment means that the person implementing randomization does not know or can’t predict the next treatment allocation. It reinforces the value of randomization in reducing selection bias.

Blinding refers to the fact that the study participant does not know what treatment he/she is getting. Double blinding means that participants and investigators are unaware of treatment assignment; this may be challenging to achieve in studies of physical therapy, behavioral therapy, weight management, and exercise. When it is hard to blind study participants or intervention providers to the treatment assignments, several steps can be taken: 1) those who ascertain outcomes should be blinded to treatment arm allocation; 2) study participants should be advised not to discuss treatment details with outcome assessors; 3) providers delivering the intervention should not participate in outcome assessment; and 4) in studies with active and control interventions, both arms should be portrayed to participants as ‘active’ and intended to reduce symptoms or improve function.

We recommend that a plan for allocation concealment should be an integral part of the manual of operating procedures (MOOP). The minimum methodological standards of ensuring allocation concealment include randomization schema using varying block size implemented centrally; and, sequentially numbered, opaque, sealed envelopes (SNOSE). Additional means of ensuring allocation concealment include: protocols that clearly state that the informed consent and baseline assessment are done prior to randomization, inclusion of reports describing means of allocation concealment in the manuscripts and report of baseline comparisons between arms on the established prognostic factors.

Randomization and stratification

Another rationale for randomization is to avoid possible dependencies between consecutive patients and obtain valid variance estimates. Randomization prevents systematic imbalance of measured and unmeasured baseline characteristics. To avoid random imbalance in important prognostic factors, the randomization should be stratified on these factors. This design issue should then also be taken into account in the statistical analysis. We recommend, depending on sample size, to stratify the randomization on a few known prognostic factors, taking into consideration that stratification without blocking is ineffective.

Placebo effect vs. trial effect

Often, individuals seek trial participation when their symptoms flare. Baseline flare can affect interpretation of study results since OA pain often is episodic. Building in an observation period of 1–2 weeks could help to distinguish the ‘trial’ effect from placebo or attention control effect. During this initial observation period, all participants could be exposed to attention control.

Regression to the mean occurs when the inclusion criteria are broad and some participants have an extreme initial value of a variable that is imperfectly correlated with subsequent measurements due to measurement error or biological variation. If these participants are measured again we may observe that the mean of this “extreme” subgroup is closer to the mean of the original sample. For example, a first high blood pressure measurement on a subject will tend to be closer to the average at the next measurement of that subject. The same can occur with a serum biomarker. The more the value deviates from the population mean and the lower the correlation between the initial and subsequent measurement, the larger the regression⁴. As a consequence, within-group change could be exaggerated and difficult to interpret. We recommend that the evaluation of treatment effects should thus not be based on within-group change but on between-group comparisons. In addition, the pre-treatment observation period can help distinguish true treatment effects from regression to the mean.

It is imperative to distinguish achieving symptomatic relief vs. sustaining pain relief. The initial observation period is helpful to reduce the impact of flare on efficacy in achieving relief. There is presently no accepted standard for minimal duration of effect required to establish sustainability; this issue is worthy of discussion in the clinical trial community. We recommend that trial planning activity should include careful collection of preliminary data or secondary analysis of prior studies to establish a reliable pattern of pain trajectories over time, in studies where pain is the primary outcome.

Choice of outcomes

It is important to center the study on outcomes that are meaningful to the patient and lead to appreciable improvements in quality of life. One lively debate regarding the choice of primary outcome centers on observed measures of functional performance vs. self-reported measures of pain, function, and other domains. We summarize key elements of performance-based and self-report outcomes to help guide such deliberations.

Performance-based or objective outcomes

These measures involve an observer-based assessment of specific tasks such as walking a set time or distance, arising from a chair, and the like. A recent meta-analysis showed that several of these measures including 40m self-paced walking test and timed up and go tests had the best psychometric properties among unidimensional performance based measures⁵.

Patient-oriented or subjective outcomes

More recently, greater attention has been given to ascertainment of outcomes that are meaningful and important to patients with OA. In contrast to performance-based outcomes, the patient reported measures assess patient perceptions of functional status, pain, and other domains. Several established self report measures used frequently in OA research include Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) for lower extremity arthritis, the Knee Injury and Osteoarthritis Score (KOOS) and Hip Injury and Osteoarthritis Score (HOOS), and more recently the PROMIS-29 (and other aggregations of PROMIS measures)^6–8. PROMIS-29 is a composite score that encompasses overarching domains that OA is likely to affect, including functional limitations, pain interference and intensity, ability to fulfill desired social roles, anxiety and depression, sleep disturbance and fatigue. It is noteworthy that the PROMIS initiative also includes a suite of assessments conducted using computer adaptive testing and invoking item response theory. Each participant is given a limited number of items, with the degree of difficulty of the item based upon responses to prior items. We recommend that the rationale for primary endpoint be clearly delineated in study protocol alongside with the validated means of measuring the primary endpoint.

Planning trials/sample size calculations

Sample size refers to the minimal number of participants required to establish efficacy with a specified level of confidence that the observed difference between intervention and control cannot be ascribed to chance. Careful sample size considerations require clear understanding of the magnitude of the effect that is perceived as clinically important. This ensures that an unnecessarily large sample will not lead investigators to overstate the importance of small effects that are unlikely to be clinically meaningful, even with p-values below a critical threshold (e.g. 0.05)^{9, 10}. Conversely, ensuring that the trial is appropriately powered will allow investigators to distinguish between a negative trial, meaning that the data did not suggest that the outcome differs between the control and intervention arms, and an underpowered trial, in which clinically meaningful differences were observed but did not achieve statistical significance. That is, “Absence of evidence is not evidence of absence.” The adequacy of the sample size has of course also important economical and ethical implications.

Technically, the assessment of sample size is usually based on a two-sided hypothesis test of the primary endpoint (or on a two-sided confidence interval). If a one-sided test (or confidence interval) will be used in the analysis, the ICH recommendation is to use a significance level of half the one that would have been used for a two-sided test (and equivalently for a one-sided confidence interval). The sample size is then calculated from the defined Type I error rate (false positive; usually 5%) and type 2 rate (false negative; usually 10–20%), an estimate of the endpoint’s variability, and the smallest treatment effect to be detected in the trial.

As several different patient number scenarios are required to evaluate the consequences of the uncertainty in the used estimates, to achieve a robust study design, a computer program for sample size calculation can be recommended for the calculation. Such programs are included in most statistical software packages.

Feasibility assessment

Sample size estimation should be accompanied by parallel assessment of study feasibility. The feasibility assessment should take into consideration the number of eligible patients seen in a particular clinic or other setting over the course of a specified unit of time (e.g. week, month, year), the intensity of the study (time commitment, intensity of the intervention), and the willingness of eligible participants to take part in an RCT^{11, 12}. The feasibility assessment offers critical insight into the anticipated duration of enrollment and necessity to recruit additional centers.

Subgroup analysis and multiplicity issues

A subgroup analysis addresses the hypothesis that the efficacy of the regimen under study may be greater (or worse) in particular subgroups. Such a hypothesis implies that treatment success is greater in the presence of certain factors than in the absence of those factors. Examples might include a specific trait such as severe malalignment or obesity in persons with knee OA or number of affected joints in persons with hand OA^13–15.

As recommended by the CONSORT guidelines (Consolidated Standards of Reporting Trials checklist to improve the quality of reporting of RCTs)¹⁶, only pre-planned subgroup analyses should be undertaken. Multiple testing increases the chances of a false positive result (Type I error), but the Type I error rate can be controlled by adjusting the statistical significance level of each analysis according to the number of tests conducted. It is necessary to have a pre-specified description of confirmatory analyses documented in the study protocol or the statistical analysis plan (SAP). A simple Bonferroni correction for multiple testing requires that the threshold for significance be divided by the number of tests. For example, a study with 5 pre-planned subgroup analyses would adjust the significance threshold to 0.05/5 = 0.01 for each analysis in order to maintain an overall Type I error of 0.05¹⁷. The Bonferroni-Holm procedure is a less conservative option for multiple testing adjustment¹⁸. All confirmatory analyses should be stated in the original registration of the RCT on ClinicalTrials.gov or other trial registries.

Planning for subgroup analyses alongside an RCT often requires a larger sample size. Studies powered to detect a main effect are unlikely to have sufficient sample size to detect interaction effects or establish the presence of a prognostic factor with sufficient levels of statistical certainty. Caution should be exercised in interpreting and reporting the results of subgroup analyses when the study has not been powered to detect an interaction effect. In these circumstances, lack of statistical significance due to low statistical power can be misinterpreted as evidence of the absence of the differential efficacy among subgroups¹⁹. Additional non-pre-specified subgroup analyses can also be performed, but should be clearly reported as exploratory.

We recommend that the decision to conduct a pre-specified subgroup analysis in RCTs should be justified by prior data suggesting that a potential prognostic factor either promotes or inhibits the hypothesized mechanism of effect of the intervention. Confirmation that a potential prognostic factor indeed is associated with differential efficacy should be supported by a formal statistical test for interaction²⁰.

Missing data prevention and handling

To minimize dropouts, investigators may discuss during the design phase the minimum follow-up duration that would be clinically meaningful without straining both study participants and personnel. Ways to minimize study dropouts include offering study participants the opportunity to complete visits in person or by phone, accommodating participant schedules, offering reimbursement for travel and/or parking, and modest incentive stipends for participation. It is also important to establish protocols to ensure that research staff exert the maximal effort permitted by the governing ethics body to obtain follow up data from participants. Clear protocols for research staff to follow in contacting study participants, such as the number of contact attempts, should be outlined at the beginning of the study²¹.

Detailed reasons for dropout should be recorded if a participant requests withdrawal. Reasons may include a participant’s assessment that he/she is too busy to continue participating, lack of efficacy, side effects (e.g., injury sustained during exercise), etc. This level of detail is especially important if reasons for dropout differ between treatment arms. Collecting detailed reasons for dropout will allow investigators to better understand the dropout mechanism and assess whether or not dropout is related to outcome²². Participants wishing to discontinue their assigned intervention should be offered the opportunity to continue study assessments, facilitating intention-to-treat inferences. Intention-to-treat implies that subjects are analyzed according to the arm randomized, irrespective of the actual treatment they received. The number of participants dropping out and reasons for dropout should be reported for each arm.

Missing data could lead to reduction of the sample size which in turn may affect statistical power of the study. Further, missing observations, if available, could have more extreme values that would, in turn affect the estimation of the variability of the effect and could artificially narrow the confidence interval for the treatment effect.

While last observation carried forward (LOCF) and complete case (CC) analytic approaches are frequently used in the analysis of RCTs, both require strong, and often unrealistic, assumptions about the missing data mechanism. In particular, both methods require that the missing data be missing completely at random (MCAR) -- that is, that the missing data are completely independent of observed or unobserved measurements (dependent or independent variables). This implies that that study completers are a random sample of the original study cohort. The less restrictive missing at random (MAR) assumption is often more reasonable. MAR assumes that dropout may depend on observed outcomes or covariates, but does not depend on unobserved data. For example, in a study of physical function in patients with knee OA, older participants may be more likely to miss follow-up visits. Older patients may also have more functional limitations. Thus, simply examining observed function could lead to a biased estimate – observed function would be an overestimate of physical function in the cohort, since older patients, with lower function, are more likely to be missing. However, if given a participant’s age, missingness is independent of function, the data are said to be missing at random. That is, the missingness depends on observed data (age) but not the missing data (function).

Under the MAR assumption, likelihood-based approaches such as mixed-effects models will produce valid inferences. Multiple imputation (MI) is another strategy that is valid under MAR. In this case, a range of plausible values is imputed for the missing outcomes using a prediction equation. This equation can incorporate ancillary information about missing data, including observed outcomes up until time of dropout, and reason for dropout. One of the reasons MI is superior compared to simple imputation methods, such as Last Observation Carried Forward, is that LOCF leads to underestimated variance and inflated statistical significance.

While there is no statistical test to verify the MAR assumption, its appropriateness should be investigated by reviewing the dropout reasons^22.22. It is possible that data will be missing not at random (MNAR), meaning that missingness is related to unobserved outcomes. This is also termed informative censoring. Sensitivity analyses should be utilized to assess the robustness of the results to this assumption.

Since there is no simple statistical solution to handling data missing not at random, we underscore the importance of minimizing missing data at the design and implementation phases of RCTs.

Interim analysis

Interim analyses may compromise the scientific value of an RCT by inflating the Type I error rate. Also, conditioning the effect estimation on interim observations may bias both the effect estimate and its variance if the trial is terminated early. Interim analyses should therefore generally be avoided. If an interim analysis is planned, it should be described in the study protocol and performed in a way that protects the Type I error rate.

We recommend that the performance of the interim analysis must be a confidential process authorized and conducted by an independent statistician and decision making committee, such as Data Safety and Monitoring Board. It also means that multiplicity adjustments are likely to be necessary.

Stopping a trial early for reasons of efficacy or safety concerns

Interim analyses are used to interpret accumulating information during a trial, often to investigate whether to terminate the trial at an early stage. The analysis can reveal whether the treatments already have convincingly established different - or similar - treatment effects, or that they have too severe side effects.

The group sequential trial design allows repeated testing of treatment effect, but trials designed with an interim analysis as part of a conventional design are often planned with only one interim analysis. The decision to stop a trial early should focus both on the primary endpoint as well as consistency among secondary endpoints and safety data.

Some participants may have incomplete follow up at the time of the interim analysis and thus not be included in the interim analysis. When their follow up is completed, and they are included in a new analysis, the outcome of the trial may differ. When an interim analysis leads to early stopping, the final results that include these additional subjects are considered more important than the results from the interim analysis, which excluded these subjects.

Futility stopping

An interim analysis may also include an evaluation of whether or not the trial can achieve its objectives. If the interim results indicate that the trial is unlikely to achieve statistical significance, stopping it can save important resources²³. Such futility testing is often performed on the primary or a suitable intermediate endpoint and based on forecasting the outcome of a reference test using conditional or predictive power. It is important to recognize that Type I error usually is not protected if a trial is continued when the interim analysis suggests stopping the trial for futility²⁴.

Change in Trial Design

Changes that occur after the first patient is randomized can compromise the scientific integrity of a trial. In some cases, however, such changes can lead to better use of resources or shorter study time. An adaptive trial is based on a study protocol that includes detailed description of one or more pre-planned interim analyses and corresponding design modifications that can be made with full control of the Type I error rate²⁴. Adaptive trials can increase the effectiveness and reduce the costs of new treatments, but they have limitations². When considering modifying an ongoing trial the complete trial recommendations of the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) recommendations should be taken into account².

Reassessing sample size

When an interim analysis neither shows superiority with regard to efficacy, nor suggests that a continuation of the trial would be futile, a sample size reassessment may be useful. Sample size calculation performed with an empirical based variance estimate is likely to be more informative than one performed with a hypothetical variance. However, a sample size reassessment that is based on the result of an ongoing trial may inflate the Type I error rate. Adjustment to account for the inflated Type I error will increase the necessary sample size. Thus, a procedure that does not affect the Type I error rate should be used, such as one that does not require unblinding of the data. Sample size re-estimation should be done by a study statistician, with the rest of the investigative team blinded

Since unblinding trial data in an interim analysis has several disadvantages, we recommend that it should be avoided when reassessing sample size. Several methods exist, the simplest one just ignoring treatment allocation and using a one-sample variance estimate^25–27. This variance estimate may be biased when a treatment effect exists, but a bias adjustment can be made with the treatment effect assumed (under alternative hypothesis) in the sample size calculation when calculating the type 2 error rate²⁷.

Superiority and non-inferiority

Statistically nonsignificant outcomes in trials designed to show that one treatment is superior to another are often wrongly interpreted as a proof of no difference between treatments. Equality of treatment effects can never be proven with statistical methods. Instead, equivalence and non-inferiority trials are designed to show that two treatments are not “too different”, or that a new treatment is not “unacceptably worse” than a standard one²⁸. While the null hypothesis in a superiority trial is that treatment effects are identical, the null hypothesis in an equivalence or non-inferiority trial is defined with reference to a specific difference in treatment effects, the equivalence, or non-inferiority, margin. The thresholds for not “too different” or not “unacceptably worse” are defined using clinical criteria. When the research team envisions that both superiority and non-inferiority approaches might be appropriate, the trial should be powered as a non-inferiority trial, and the switching from non-inferiority to superiority should be planned and described in the protocol a priori. The superiority and non-inferiority approaches also differ with regard to the role of the study population. As the ITT population tends to be less conservative than the per-protocol population in non-inferiority trials, the latter is often chosen as the primary study population.

We recommend that if, after a successful interim analysis, a non-inferiority trial is continued in order to show superiority, the final conclusion of the trial should be based on all collected data, even if this result happens to be less supportive than the one from the interim analysis.

Changing primary endpoint

If important scientific knowledge has been gained after the initiation of a trial, it may be appropriate to incorporate this knowledge into the trial²⁹. For example, consider a pharmaceutical trial in OA patients in which the primary outcome is the change in a serum biomarker associated with radiographic progression. If a valid, and more responsive biomarker became available during the course of the trial, it might be reasonable to consider adding this new biomarker to the trial endpoints.

The main principle when considering changing the primary endpoint is whether the change is independent of the data collected in the trial. In some large trials with long duration, it may be appropriate to change an endpoint even if the change depends on data. The trial would then have to be divided into two stages, the first being hypothesis generating (endpoint identifying), the second hypothesis testing (confirmatory), and the outcome of the trial relying entirely on the second stage. The design change, like other major changes, should be documented in an amendment to the study protocol.

Other major design changes

Other changes, such as treatment duration, co-medications, changes of inclusion or exclusion criteria, may be necessary in an ongoing trial. In some cases, the changes affect the sample size. If such changes are made, a formal protocol amendment should be made, and the primary analysis should be stratified according to whether the patients were randomized before or after the change²⁴. Homogeneity of the results should be carefully investigated, but a combination of the results would still require careful argumentation.

Analyzing and reporting RCT data

Analysis and reporting of results from a RCT should be conducted according to the guidelines proposed by the CONSORT group for pharmacologic studies and CLEAR NPT (CheckList to Evaluate A Report of a NonPharmacological Trial) group for non-pharmacologic studies using the intention-to-treat (ITT) analysis as the primary analysis^{12, 16, 30–38}. In the intention- to- treat analysis, all study participants are analyzed according to treatment arm to which they were randomized, taking advantage of the balance in measured and unmeasured factors.

Study teams have an ethical obligation to report the results of their research activities³⁹. The manner in which study data are reported has evolved significantly over the past 20 years, and there is general consensus today of the basic requirements. All studies should be prospectively entered into an appropriate clinical trials registry. There are numerous country and region-specific registries, and the WHO maintains the International Clinical Trials Registry Platform (ICTRP), whose main aim is “to facilitate the prospective registration of the WHO Trial Registration Data Set on all clinical trials, and the public accessibility of that information.”²⁶ In the United States, registration is required for trials that meet the Food and Drug Administration Amendments Act (FDAAA) definition of an “applicable clinical trial”⁴⁰. “Applicable Clinical Trials” include trials of drugs, biologics and devices, other than phase 1 studies, of agents that are subject to FDA regulation and that have one or more sites in the United States, are conducted under an FDA Investigational New Drug (IND) or Investigational Device Exemption (IDE) or involves a drug, biologic or device manufactured in the United States and exported for research. Trials must be registered within 21 days of the first participant being enrolled. Clinicaltrials.gov is a web-based registry resource²⁵ developed as a consequence of the Food and Drug Administration Modernization Act of 1997 (FDAMA)⁴¹ and maintained by the National Library of Medicine, a part of the National Institutes of Health. The data entered are publicly available. The scope of clinical trials that must be registered and the information provided, including study results, has been significantly expanded since its initiation. Timely registration of clinical trials has become a requirement from the majority of scientific journals for acceptance of manuscripts reporting study results and is a long-standing recommendation of the International Committee of Medical Journal Editors (ICMJE)⁴².

Less consensus exists at this time regarding what specific clinical trials information should be made publicly available. Internationally, the WHO Registry Platform Working Group on the Reporting of Findings of Clinical Trials supports that “the findings of all clinical trials must be made publicly available” but has provided no specific template or recommendations⁴³. In the United States, Section 801 of the FDAAA requires the submission of “basic results” for certain clinical trials, generally not later than one year after study completion⁴⁴. As a consequence, the clinicaltrials.gov results database was implemented and allows the submission of data in a standard, tabular format consisting of sections dealing with participant flow, baseline characteristics, outcome measures and statistical analyses and adverse events. The sponsor or principal investigator is responsible for submitting these data. Details are provided at http://prsinfo.clinicaltrials.gov/ and the article by Tse et al⁴⁴.

The information content to be reported in publications continues to elicit discussion, with the CONSORT statement⁴⁵ currently serving as a widely accepted standard for the reporting of clinical trials. The statement includes an evidence-based set of recommendations, presented as a 25-item checklist and flow diagram. An accompanying “explanation and elaboration document”⁴⁶ provides the principles upon which the recommendations were generated and offers valuable information to assist in ensuring the quality of reporting of clinical trials in the literature. Extensions of the CONSORT statement to randomized controlled trials that differ from the standard parallel-group design have recently been published⁶. Many scientific journals now strongly suggest or require that submitted manuscripts adhere to the CONSORT principles, checklist and flow diagram. Details regarding the CONSORT group and its activities are available from its website⁴⁷. Guidelines for data content and the structure of reports required for filings to regulatory authorities are beyond the scope of this update but can be accessed by those interested (guideline document of ICH⁴⁸ and guidance document of FDA¹⁸).

Economic evaluations alongside RCTs in osteoarthritis

When trial results indicate that an intervention is efficacious, policy makers must decide whether the treatment offers good value for the additional resources spent^49–53. To address this question, investigators may consider a formal cost-effectiveness or cost-utility analysis alongside clinical trials or beyond the timeframe of the trial^{54, 55}.

OA treatments generally affect quality of life, not longevity. Therefore, investigators considering economic evaluation alongside an RCT should include appropriate measures of quality of life in the data collection instruments. In economic evaluations, quality of life is quantified by utilities ranging from 0 to 1, with 1 corresponding to perfect health and 0 to death^{56, 57}. Investigators should decide whether assessment of utilities will be done directly, using the Standard Gamble or Time-Trade-Off method⁵⁸, or derived indirectly using instruments such as EuroQol or short form health surveys (SF36 or SF12), WOMAC, with cross-mapping of health states assessed with these measures to population-based repositories of preference-based utilities^59–63.

Investigators should collect data related to subjects’ health care utilization and select a time frame over which patients can reliably recall their utilization of health care. It is of paramount importance to decide on the perspective (societal, government, payer, provider) that informs the collection of the cost data. Questions related to health care utilization should encompass visits to primary doctors and specialists, medication use, hospitalizations, and emergency department visits. It is important to account for direct medical and indirect costs. Direct medical costs include pharmacological and non-pharmacological regimens, ambulatory and emergency department visits and inpatient stays. Indirect costs capture productivity losses due to a medical condition.

The Panel on Cost-Effectiveness in Health and Medicine, recommends that cost-effectiveness analyses (CEA) be done over the long term, potentially the lifetime, to capture long-term consequences of treatment strategies. For example in the case of OA, a key economic and quality of life outcome is total knee replacement. Often, to conduct a CEA over a long time duration, data from an RCT need to be augmented by data from other sources. Decision analysis modeling should be employed in addition to commonly used statistical methods to conduct a CEA over the lifetime horizon.

Careful attention should be given to the investigation of uncertainty in parameter estimates. Depending on the type of economic evaluation (stochastic evaluation alongside of RCT or model based evaluation, beyond the time frame of the RCT) different types of uncertainty should be considered. For the evaluation alongside of the RCT, there are four main types of uncertainty: methodological, sampling variation, extrapolation, and generalizability/transferability. The main methods to handle methodological and generalizability/transferability concerns include the one- and multi-way sensitivity analyses. Parameter uncertainty is more relevant to model-based evaluations. The reduction of uncertainty through additional research may improve decisions but comes at a cost. Probabilistic sensitivity analyses followed by formal value of information (VOI) analyses, a formal methodology that is designed to facilitate establishing research priorities, evaluate the impact of uncertainty, and the value of future research to reduce uncertainty⁶⁴.

The traditional cost-effectiveness analysis is designed to address whether a treatment offers good value but does not provide insight into affordability. This issue can be addressed with budget impact analysis (BIA). The goal of BIA is to quantify the financial consequences of adopting the specific treatment strategy (such as a weight loss program) by various payers, including insurance organizations, health care systems, and government, given real life resource constraints. The BIA permits projection of how adoption of the treatment could impact spending for specific payers. The results of BIA are often used for budget planning and changes in health insurance premiums.