Design and analysis of crossover trials for investigating high-risk medical devices: A review

Abstract

During evaluation of new investigational medical devices, the FDA recommends that investigators design a crossover clinical trial, in which the patients are arranged to cross over from one treatment arm to another. The FDA annually receives the premarket applications of investigational medical devices, in which sponsors design and conduct crossover trials as their confirmatory clinical trials for evaluating safety and effectiveness of the devices. This article reviews the use of crossover clinical trial in pivotal clinical trials for high-risk medical devices regulated and approved by the United States Food and Drug Administration (FDA) from 2005 to 2018. As an example, in which a crossover trial was implemented during the regulatory approval process of a new device, the article additionally presents the FDA-approved premarket approval (PMA) submission of the Medtronic MiniMed 530G system, an artificial pancreas device system that mimic a normally functioning pancreas by monitoring glucose levels and administering insulin at a certain threshold maintained by sensing blood glucose levels. The article also reiterates the critical regulatory considerations that the FDA made regarding the design and analysis of crossover trials in its guidance for industry.

1. Introduction

In a randomized controlled trial, each study subject is randomized to receive an investigational treatment. Such designs of clinical trials are fundamental in clinical research [1]. A crossover clinical trial, however, is designed such that each subject receives a sequence of investigational treatments, typically with the aim to compare the effects of these treatments, not the sequences of the treatments [2]. The main feature distinguishing crossover trials from randomized controlled trials and other types of clinical trials is that each subject has the opportunity to receive different, and potentially all, investigational treatments. The advantages are that the subjects serve as their own comparison controls and a smaller sample size is required. The disadvantages of crossover trials include longer study period and potential carryover effects, period effects, and treatment-by-period interactions, each of which can deteriorate the precision of treatment effect estimation [2]. Yet, sophisticated and appropriate design and analysis strategies in a crossover trial can substantially reduce or even eliminate these negative impacts [2].

In the regulatory practice of the Center for Devices and Radiological Health (CDRH) in the U.S. Food and Drug Administration (FDA), crossover trials appear in a large number of applications seeking approval of a new investigational medical device [3,4]. In these applications, the sponsors design crossover clinical trials for the purpose of demonstrating the safety and effectiveness of an investigational device. In this article, we reviewed the crossover clinical trials reported in the premarket approvals (PMA) [5,6] of high-risk medical devices in a period of 14 years from 2005 to 2018. The public database of PMAs maintained by the CDRH was searched for all PMAs in this period. A total number of 5 PMAs were identified as approved applications that included at least one crossover clinical trial. Furthermore, one of the PMAs, for the MiniMed 530G artificial pancreas device system (APDS) [7] with enhanced digital diabetes management, was taken as an example for detailed discussion. The goal of this review is to promote consistent understanding among FDA regulatory reviewers and industrial sponsors regarding design and analysis of crossover trials for new investigational medical devices, including new APDSs, and consequently to facilitate their regulatory applications and review processes.

2. An overview of crossover clinical trials in the premarket approvals of high-risk medical devices from 2005 to 2018

The FDA requires industrial sponsors to submit a PMA application for each of the Class III high-risk investigational medical devices as a scientific review process to evaluate the safety and effectiveness of the device [5,6]. Once the PMA is approved, a private license is then granted to the applicant for marketing the new device. The public database of PMAs (database link: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpma/pma.cfm) is a database that holds, for the CDRH's releasable approved PMAs, the information and files that elucidate: the PMA and supplement numbers, applicant name and address, generic and trade names of the device, advisory committee, date received and decision date, product code, FDA approval order and statement, PMA summary of safety and effectiveness (SSED), and device labeling. Particularly, the SSED of a PMA is mandated by 520(h)(1)(A) of the Federal Food, Drug, and Cosmetic Act [5]. The document is intended to present an objective and balanced critique of the scientific evidence on the safety and effectiveness of a device and to present to the public the basis of the decision to approve or deny the PMA. As such, the SSEDs are summaries of both positive and negative aspects and both potential risks and possible benefits of the PMAs. As required by U.S. government publication 21 CFR 814.20(b)(3) [5,6], SSEDs must contain sections for general information, indications for use, device description, contraindications, warnings, precautions, alternative practices and procedures, marketing history, potential adverse effects of the device on health, summary of preclinical studies, summary of clinical studies, conclusions drawn from the studies, panel recommendations, CDRH decision, and approval specifications. We conducted an exhaustive search of the public database of PMAs and collected the SSEDs of all releasable and approved PMAs with a decision date between September 1, 2005 and December 31, 2018. The searching criterion was set to limit the “Decision Date” from September 1, 2005 to December 31, 2018 and included only original submissions in “Supplement Type”. The search was not restricted on devices in any particular medical field. The goal of this search was to utilize the information within the collected SSDEs to identify those FDA-approved PMAs, in which the sponsor demonstrated the safety and effectiveness of the proposed device by conducting a crossover trial as the pivotal clinical trial.

A total number of 414 PMAs approved within the time range by the FDA were returned after the search. The keywords “crossover” and “cross-over” were searched in the SSEDs of the 414 PMAs. A total of 20 PMA SSEDs were found to contain the keyword “crossover” and a total of 14 PMA SSEDs were found to contain the keyword “cross-over”. Two SSEDs had both the keyword “crossover” and the keyword “cross-over”. Thus, the SSEDs from 32 PMAs were selected for final review. During the final review process, we carefully examined and studied the content of each SSED to discern whether the sponsor designed and conducted a crossover clinical trial in the pivotal clinical studies in the PMA. Five PMAs with a crossover trial were finally identified. The trade names of the corresponding five products are: VISTAKON® Contact Lens, Clear and Visibility Tinted with UV Blocker (PMA number: P040045) [8]; FC2 Female Condom (PMA number: P080002) [9]; Artificial Pancreas Device MiniMed 530G System (PMA number: P120010) [7]; PowerLook® Tomo Detection Software (PMA number: P160009) [10]; and t:slim X2 Insulin Pump with Basal-IQ Technology (PMA number: P180008) [11]. Among the remaining 27 PMAs, 19 of them were irrelevant and 8 PMAs reported a clinical study allowed the patients in the control group to crossover to receive treatment after the parallel randomized controlled period. The search result showed that 5 out of 414 PMAs, or equivalently 1.21% of the PMAs, within the period of approximately 14 years from 2005 to 2018 designed and conducted a crossover trial to demonstrate safety and effectiveness of the investigational Class-III medical devices. Table 1 lists the product names, PMA numbers, FDA approval dates, description of medical device, and description of crossover trials of five FDA-approved PMAs with crossover pivotal trials. In Section 3, we present details of the design and analysis of the pivotal clinical in the PMA submission of the Artificial Pancreas Device MiniMed 530G System (PMA number: P120010) [7]. We present this PMA because (i) this is most consequential investigational device among the five PMAs, (ii) crossover trials are common when evaluating artificial pancreas device systems in the FDA-regulated submissions, and (iii) the SSED of P120010 was complete and complied with all necessary details.

Table 1.

Product names, premarket approval (PMA) numbers, FDA approval dates, description of medical device, and description of crossover trials of FDA-approved PMAs with at least one crossover pivotal trial from September 2005 to December 2018.

Product name (PMA number; FDA approval date)	Descript of medical device	Description of crossover pivotal trial
VISTAKON® Contact Lens, Clear and Visibility Tinted with UV Blocker (P040045; December 20, 2005)	The device is a spherical soft contact lens developed for optical correction of refractive ametropia, including myopia and hyperopia, among phakic or aphakic patients [8].	The crossover trial, Corneal Swell Study, was a randomized, controlled, patient-blinded, bilateral crossover study to evaluate the corneal swelling produced by the new contact lens and by hydrophilic contact lens as a control device in daily and overnight wear [8].
FC2 Female Condom (P080002; March 10, 2009)	The device is a female-use condom for reducing the risks of pregnancy and sexually transmitted infections [9].	The crossover trail was a prospective, randomized, controlled, crossover clinical study. The active control device is the FC1 Female Condom, and the investigational device is the FC2 Female Condom. The study compared 4 condom failure modes in failure rates [9].
Artificial Pancreas Device MiniMed 530G System (P120010; September 26, 2013)	The MiniMed 530G system is a medical device that provides a patient (sixteen years of age and older) continuous delivery of basal insulin and administration of insulin boluses for the purpose of managing diabetes mellitus [7].	One of the two pivotal clinical trials was a multi-center, randomized, crossover study to evaluate the efficacy of the threshold suspend feature in reducing exercise induced hypoglycemia. In Group A, subjects wore the MiniMed System with an active threshold suspend feature (denoted by “TS-ON”) during the exercise-induced hypoglycemia visits in Period 1 of the study. In Group B, the subjects had a deactivated threshold suspend feature in the system (denoted by “TS-OFF”) in Period 1 of the study. In Period 2, from Visits 9–12, the two groups crossed over their threshold suspend tool function (TS-ON or TS-OFF) [7].
PowerLook® Tomo Detection Software (P160009; March 24, 2017)	The device is a software device to be used by radiologists to detect soft tissue densities, including masses, architectural distortions, and asymmetries in the 3-dimensional GE Senoclaire breast tomosynthesis images [10].	The clinical study, Pivotal Reader Study, was a multi-reader, multi-case study to demonstrate safety and effectiveness of the software device. The study had a crossover design, reading with or without the assistance of the software. In this study, each patient image served as its own control [10].
t:slim X2 Insulin Pump with Basal-IQ Technology (P180008; June 21, 2018)	The device is an insulin pump with new Basal-IQ technology for insulin delivery suspension [11].	The clinical trial was a randomized, controlled, crossover study to demonstrate the effectiveness of the Basal-IQ feature in the FDA-approved t:slim X2 Insulin Pump. The study consisted of two 3-week periods with and without enabled Basal-IQ feature, respectively [11].

3. A typical application of crossover clinical trial in the premarket approvals of high-risk medical devices: MiniMed 530G artificial pancreas device system

The MiniMed 530G system is a first-generation [12,13] APDS manufactured by Medtronic Diabetes that was approved by the FDA on September 26, 2013. The MiniMed 530G system is a medical device that can provide a patient (16 years of age and older) continuous delivery of basal insulin and administration of insulin boluses for the purpose of managing diabetes mellitus. Both the rate of basal insulin delivery and the amount of insulin bolus administration are adjustable by the users. The user can also program the system so that, when the glucose level detected by the sensor is below a prespecified threshold value, the delivery of insulin will be automatically suspended [7]. The MiniMed 530G system is comprised of the following devices and software: (1) MiniMed 530G Insulin Pump for delivering insulin; (2) Enlite Sensor, a sensor inserted into a patient's subcutaneous tissue and connected to MiniLink Real-Time System for continuously monitoring glucose levels; (3) Enlite Serter for inserting the Enlite Sensor; (4) the MiniLink Real-Time System, a transmitter device for providing power to the sensor and measuring real-time glucose values; (5) CareLink® Professional Therapy Management Software for Diabetes for enhancing health care provider management of diabetic patients; (6) a network-based CareLink® Personal Therapy Management Software for Diabetes for managing patient data from the MiniMed 530G system; and (7) the Bayer Contour NextLink glucose meter interacting with the MiniMed 530G system to measure glucose levels in capillary whole blood [7]. Two embedded software systems in the MiniMed 530G system, CareLink® Professional and CareLink® Personal, provide state-of-the-art digital management of diabetes.

The industrial sponsor of the PMA of the MiniMed Paradigm X54 System was Medtronic PLC. They designed and performed a pivotal clinical study to demonstrate the safety and effectiveness of the device to the regulatory agency for premarket approval [7]. The study was a multi-center randomized controlled crossover trial to demonstrate the effectiveness of the threshold suspend feature in reducing exercise induced hypoglycemia among type I diabetes patients [7]. The trial was performed using the Veo MMT-X54 insulin pump, as well as Sof-Sensor MMT-7003, as the sponsor and the FDA agreed that the MiniMed Paradigm X54 System was comparable to the MiniMed 530G system in calibration algorithm and threshold suspend feature. A crossover design and an active control device were considered in this clinical trial due to the nature of diabetes as a chronic disease and ethical considerations. A total of 50 subjects were enrolled in five sites and completed the study within two age cohorts: 42 subjects in the adult cohort ($\ge$ 22 years) and 8 subjects in the pediatric cohort (18–20 years). All of these subjects were type I diabetes patients, and all were accepting insulin pump therapy prior to the enrollment of the pivotal study. Each patient underwent 12 study visits before randomization. Visits 1–6 occurred within a two-week run-in period. Subjects were randomized into two groups. In Group A, the subjects wore the MiniMed System with an active threshold suspend feature (denoted by “TS-ON”) during the exercise-induced hypoglycemia visits in Period 1 of the study. In Group B, the subjects had a deactivated threshold suspend feature (denoted by “TS-OFF”) in Period 1 of the study. In Period 2, from Visits 9–12, the two groups crossed over their threshold suspend tool function (TS-ON or TS-OFF). A total of 134 hypoglycemic induction experiments were performed on all but two subjects, of which 98 experiments were successful. There were two pre-determined co-primary effectiveness endpoints: the duration of induced hypoglycemia and the severity of induced hypoglycemia. Multiple secondary effectiveness endpoints were also considered as part of effectiveness evaluation. Safety of the system was evaluated by device-induced adverse events: all, serious and unanticipated. Among the 134 hypoglycemic induction experiments, 69 were threshold-suspension active and 65 of them were inactive. During these experiments, the hypoglycemic inductions were stopped nine times in the 69 threshold-suspension active experiments and eight times in the 65 threshold-suspension inactive experiments. Regarding the severity of induced hypoglycemia, the system functioned effectively and successfully suspended basal insulin delivery as it was designed. This feature did not create any significant difference in hypoglycemia inductions between the two groups. Regarding the duration of induced hypoglycemia, no significant difference was reported in the nadir glucose between threshold-suspension active and inactive experiments, with a mean nadir glucose as 59.5 $\pm$ 5.72 and 57.6$\pm$ 5.69 (p-value = 0.015), respectively. A total of 29 adverse events were reported in the trial, but 20 of them were not device-related or procedure-related. While 21 subjects faced at least one adverse event categorized as mild or moderate severity, no serious or unanticipated adverse events occurred [7]. The FDA concluded that “The results of the pivotal clinical study performed to support this submission establish a reasonable assurance of safety and effectiveness that the MiniMed 530G System can detect trends and track patterns and temporarily suspend the delivery of insulin when used as intended, as an adjuvant to blood glucose testing in subjects with diabetes mellitus. The sofeware for the Threshold Suspend tool is the same for the Veo pump and the 530G System” [7].

4. Regulatory considerations and discussions

FDA's E6 Good Clinical Practice guidance for industry [14] was developed by the expert working group within the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use. The E6 guidance provides a unified standard for conduct of clinical trials to demonstrate the efficacy and safety of medical products. The E6 guidance highlights the critical role of design and statistical analysis in conducting the clinical trials and statistical analyses. In the E9 Statistical Principles for Clinical Trials guidance for industry [15], the FDA listed crossover design as one of three recommended clinical trial designs (along with parallel group design and factorial design) in clinical trial configuration. The E9 guidance recommends that, in all three types of clinical trial designs, the data analysis sets of subjects (intention-to-treat set, per-protocol set, or other sets) and data analysis are prespecified in the study protocol and the issues of missing values, multiplicity, subgroup effects, interactions, and influence of covariates are properly planned and addressed [15]. Study design and statistical analysis of crossover trials for investigational medical devices are to follow these two guidance documents.

To date, the FDA has not distributed guidance specifically for crossover trials. Yet, the crossover trials are extensively adopted in establishing bioequivalence between a test drug and a reference drug. FDA's guidance for industry on Statistical Approaches to Establishing Bioequivalence [16] recommends conducting a clinical trial with either nonreplicated crossover design, such as two-period two-sequence design, or replicated crossover design with more than two periods and two or more sequences. Our exhaustive review shows that the crossover design with two periods and two sequences dominates the crossover clinical trials for investigational medical devices. The replicated crossover design with more than two periods and two or more sequences is evidently superior to a simple two-by-two design in estimating interactions and carryover effects, but it elevates the study duration and cost. The guidance on establishing bioequivalence recommends general linear modeling procedures available in PROC GLM in SAS [16] for analysis of nonreplicated crossover trials and linear mixed-effects modeling procedures with either the methods of moments or the restricted maximum likelihood methods in PROC MIXED in SAS for analysis of replicated crossover trials. These approaches also apply to the statistical analysis for the crossover trials for medical devices. Sophisticated statistical analysis methods for clustered data [17] are required for analyzing the data collected from crossover trials. Details on these data analysis methods were given by Jones and Kenward [2] and can be adopted in the crossover clinical trials for investigational medical devices. However, it is recommended that the industrial sponsors obtain the consent from the CDRH during regulatory review processes [18].

The FDA's guidance for industry on Design Considerations for Pivotal Clinical Investigations for Medical Devices elaborates principles for the design of pre-market pivotal clinical studies for establishing the safety and effectiveness of an investigational medical device [18]. In the guidance, two broad types of medical devices, therapeutic (and aesthetic) devices and diagnostic devices are distinguished by their intended use. The guidance explicitly lists parallel group design, paired design, and crossover design as three recommended pivotal study designs for comparing two or more treatment. The FDA recommends the use of a crossover design when the effects of treatments or diagnostic tests do not carry over from one period to the next and when a paired design is not applicable. Although Section 2 shows that a crossover trial was conducted in the pivotal studies of five therapeutic devices, the guidance states that a crossover design can be applied in both therapeutic (aesthetic) and diagnostic studies. All general and specific study design considerations in the guidance apply to the crossover designs. General design considerations include bias and variability, study objectives to support intended use and labeling, subject selection and stratification, and site selection. Specific study design considerations for clinical outcome studies (the studies for assessing clinical outcome parameters of safety and effectiveness) include subject endpoints, randomization, blinding, placebo effect, study controls, sources of bias and bias minimization. Specific study design considerations for diagnostic clinical performance studies (the studies for assessing diagnostic performance of a device with a clinical reference standard) include specification of intended use, choice of clinical reference standard, study population, study planning and specimen collection, blinding, skill and behavior of device interaction, and resources and control of bias. For statistical analysis, the guidance states very little. However, it emphasizes that inappropriately performed and unplanned post-hoc analyses may undermine the usefulness of a pivotal study. Therefore, the study protocol for a crossover trial must have a detailed Statistical Analysis Plan (SAP) that elaborates, for instance, assessment of carryover effects and interactions.

Our review showed that, as the randomized controlled trials are still considered as the ‘gold standard’, crossover trials and design are certainly underutilized for evaluating the safety and effectiveness of the Class III high-risk medical devices. In fact, there are advantages to design and implement a crossover trial [19], including (i) requiring a small sample size compared with the parallel-group trials, (ii) diminished influence by other confounding factors given that the study subjects serve as their own controls, and (iii) decreased variability in study endpoints. In addition, compared with the parallel-group randomized controlled trials, crossover trials are considered having their ethical advantages because, during these trials, study subjects in the control group also receive treatment in one or more the study periods. This is preferable for the subjects with chronic diseases or progressive cancers. However, cautions were raised by some researchers arguing that, in oncology trials, it may create biases and can be an obstacle to evaluate the treatment effect carefully and rigorously, if investigators design the study in the way that patients in the control arm are designed to cross over to the treatment arm right after disease progression. When disease progression during the trials is a concern, then the stepped wedge design may be considered [20,21]. As an alternative of crossover design, the stepped wedge design for randomized controlled trials is new and increasingly popular [20,21]. In the stepped wedge design, individuals or clusters are randomized to cross from receiving control to investigational treatment at sequential periods, after an initial period of control for all study subjects. The crossover process continues until all individuals or clusters receive the treatment [20,21]. The industrial sponsors may consider the stepped wedge design in their future pivotal clinical trials for investigational medical devices.

However, we should also note that there are limitations to perform a crossover trial; that is, the crossover design may be inappropriate if certain conditions are not fulfilled [19]. First, crossover trials are appropriate for evaluating the devices that treat persistent or long-lasting diseases. For the medical devices to treat some short-term illnesses or acute conditions, it is usually not likely to perform crossover trials. Second, there are always risks for aliasing between the treatment effect and the carry-over effect or period effect or sequence effect. Even with a designed wash-out period, it is hard to conclude whether the effect in the previous treatment period can be completed eliminated after the wash-out period. As a result, the treatment effect might be confounded with other effects. Lastly, crossover trials include two or more study periods and are more time-consuming compared with the parallel-group randomized controlled trials. Thus, if a sponsor cannot afford a longer study duration, then the crossover trials are not an option.

5. Conclusion

In the FDA regulatory process of PMA submissions of new investigational medical devices and APDSs, a crossover trial is recommended as one of the study design strategies to demonstrate safety and effectiveness of the devices. Both industrial sponsors and FDA scientific reviewers shall abide by the FDA's guidance documents for industry regarding the design and analysis of crossover trials.

Funding

Dr. Hui Zhang's research was in part supported by the Northwestern Brain Tumor SPORE (National Institutes of Health Grant No. P50CA221747), Mesulam Center for Cognitive Neurology and Alzheimers Disease (National Institutes of Health Grant No. P30AG013854), and Robert H. Lurie Comprehensive Cancer Center (National Institutes of Health Grant No. P30CA060553), all of which were affiliated with Northwestern University Feinberg School of Medicine in Chicago, IL, USA.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.