“Open mesh” or “strictly selected population” recruitment? The experience of the randomized controlled MeMeMe trial

Abstract

Among randomized controlled trials (RCTs), trials for primary prevention require large samples and long follow-up to obtain a high-quality outcome; therefore the recruitment process and the drop-out rates largely dictate the adequacy of the results. We are conducting a Phase III trial on persons with metabolic syndrome to test the hypothesis that comprehensive lifestyle changes and/or metformin treatment prevents age-related chronic diseases (the MeMeMe trial, EudraCT number: 2012-005427-32, also registered on ClinicalTrials.gov [NCT02960711]). Here, we briefly analyze and discuss the reasons which may lead to participants dropping out from trials. In our experience, participants may back out of a trial for different reasons. Drug-induced side effects are certainly the most compelling reason. But what are the other reasons, relating to the participants’ perception of the progress of the trial which led them to withdraw after randomization? What about the time-dependent drop-out rate in primary prevention trials? The primary outcome of this analysis is the point of drop-out from trial, defined as the time from the randomization date to the withdrawal date. Survival functions were non-parametrically estimated using the product-limit estimator. The curves were statistically compared using the log-rank test (P=0.64, not significant). Researchers involved in primary prevention RCTs seem to have to deal with the paradox of the proverbial “short blanket syndrome”. Recruiting only highly motivated candidates might be useful for the smooth progress of the trial but it may lead to a very low enrollment rate. On the other hand, what about enrolling all the eligible subjects without considering their motivation? This might boost the enrollment rate, but it can lead to biased results on account of large proportions of drop-outs. Our experience suggests that participants do not change their mind depending on the allocation group (intervention or control). There is no single answer to sort out the short blanket syndrome.

Introduction

Several problems affect data management and the research quality of randomized controlled trials (RCTs). Recruitment issues, sampling bias, ethics, patient preferences, and treatment comparisons can, in practice, limit the application of the RCT’s design.¹ Approximately 50% of trials recruit to target or on time but part of the endeavor thins down as a result of drop-out rate.² Another important point is that follow-up should be attempted for all patients who are randomized.³ Complete follow-up allows an assessment of the treatment effect under real conditions (effectiveness) as recommended by the European Medicines Agency in its “Guideline on Missing Values in Confirmatory Clinical Trials”.⁴

Among RCTs, trials for primary prevention of diseases require large samples and long follow-up to obtain a high-quality outcome; therefore the recruitment process and the drop-out rates largely dictate the adequacy of the results. Large interventional cardiovascular prevention trials are paradigmatic promising examples. The Systolic blood PRessure INtervention Trial (SPRINT)⁵ started in 2010 and aimed to recruit approximately 9,250 people to be randomized in the intensive blood pressure arm (goal of systolic blood pressure <120 mmHg) or in the standard control pressure arm (goal of systolic blood pressure <140 mmHg). SPRINT achieved recruitment of 9,361 persons,⁶ reaching the initial recruitment goal (986 dropped out, 10.5%). Such trials require enormous economic and logistic investments: SPRINT was conducted at 102 clinical sites (organized into five clinical center networks) in the United States, including Puerto Rico. Only big consortiums are able to provide such an effort: SPRINT was sponsored by the National Heart, Lung, and Blood Institute, cosponsored by the National Institute of Diabetes and Digestive and Kidney Diseases, the National Institute of Neurological Disorders and Stroke, and the National Institute on Aging.

On the other hand, for profit trials are often able to perform a high level primary prevention trial with large samples and long follow-up. The Avoiding Cardiovascular Events Through Combination Therapy in Patients Living With Systolic Hypertension (ACCOMPLISH)⁵ trial was able to assign 11,506 patients with hypertension, who were at high risk for cardiovascular events, to receive treatment with either benazepril plus amlodipine or benazepril plus hydrochlorothiazide. They managed a mean follow-up of 36 months and only 143 persons were lost to follow-up.⁷

We are conducting a Phase III RCT on persons with metabolic syndrome (MS) to test the hypothesis that comprehensive lifestyle changes and/or metformin (MET) treatment prevents age-related chronic diseases (the MeMeMe trial).⁸ The MET/placebo component of the study is double-blind. It is a single-center trial conducted at the Fondazione IRCCS Istituto Nazionale dei Tumori di Milano (Foundation IRCCS National Cancer Institute). The trial is ongoing and recruitment is under way.

Here, we briefly analyze and discuss the reasons which may lead to participants dropping out from our trial. Good Clinical Practice (GCP) clearly indicate that a subject is not obliged to give his/her reason(s) for withdrawing prematurely from a trial⁹ and this point is clearly specified in the MeMeMe informed consent form. However, in case of a withdrawal, we make every reasonable effort to ascertain the reason(s), while fully respecting the subject’s rights in accordance with GCP.

To date, 670 people have given written informed consent to participate in the MeMeMe trial (some of them are still waiting for the baseline clinical visit and blood sample). Baseline characteristics of the 420 volunteers randomized to MET/placebo within the whole population enrolled in MeMeMe trial are summarized in Table 1. Among the 670 people, 187 have left the study (~28%). There were 134 screening failures, who were found not to have MS or who were diabetic at baseline, so they did not enter the randomization. Then, 23 people changed their mind before randomization. The main reason for changing their mind was the different advice on participation in a primary prevention RCT given by their general practitioners or other physicians.

Table 1.

Baseline characteristics of the 420 volunteers randomized to metformin/placebo within the whole population enrolled in MeMeMe trial⁸

Characteristic	Placebo (207) (mean ± SD)	Metformin (213) (mean ± SD)	P-value*
Age (years)	63.32±3.68	64.29±5.13	0.10
Sex (n, %)	Male (75, 36%)	Male (71, 33%)
	Female (132, 64%)	Female (142, 67%)
Weight (kg)	87.25±16.98	84.39±14.76	0.094
Waist circumference (cm)	100.04±12.29	98.74±10.79	0.29
Diastolic pressure (mmHg)	88.77±9.39	89.42±9.08	0.51
Systolic pressure (mmHg)	144.57±16.1	150.18±34.7	0.053
Glycemia (mg/dL)	103.58±10.21	103.59±10.0	0.99
Total cholesterol (mg/dL)	215.44±36.80	215.94±38.56	0.90
HDL (mg/dL)	52.87±13.13	58.13±15.09	<0.01
LDL (mg/dL)	135.36±33.11	133.43±35.2	0.60
Triglycerides (mg/dL)	135.94±62.11	131.18±55.67	0.08
AST (IU/L)	22.16±18.20	20.61±6.50	0.29
ALT (IU/L)	26.12±13.27	23.12±10.53	0.02
GammaGT (IU/L)	33.11±25.37	31.24±20.67	0.45
Education (n, %)
Junior high school	(46, 22%)	(49, 23%)	0.20
High school	(120, 58%)	(132, 62%)
University	(41, 20%)	(32, 15%)
Working/retired (n, %)
Working	(139, 67%)	(149, 70%)	0.98
Retired	(68, 33%)	(64, 30%)

Notes:

^*Student’s t-test P-value for continuous variables, chi-squared test P-value for qualitative variables. Statistically significant values are shown in bold.

Abbreviations: HDL, high-density lipoprotein; LDL, low-density lipoprotein.

At the time of writing, 420 volunteers were randomized to MET or placebo group. Only 30 participants have dropped out of the trial after randomization (~7%). Five of them developed the primary outcome. In our experience, participants may back out of a trial for different reasons. Drug-induced side effects are certainly the most compelling reason. However, in our trial only six volunteers dropped out for this reason and for four of them it arose during the initial 30-day treatment with 500 mg/day MET (trial period) before randomization. Only two participants experienced treatment-related adverse events after randomization. Curiously, on unblinding we saw that one of them had been allocated to the placebo group, as if indicating a “nocebo” effect.

But what are the other reasons, relating to the participants’ perception of the progress of the trial which led them to withdraw after randomization? What about the time-dependent drop-out rate in our primary prevention RCT? Are there differences between the MET and placebo arms not related to the drug’s side-effects?

People who decide to join a trial like this may have different reasons. They may share the aim of the study, being aware of the high-level scientific proposal of chemoprevention as primary outcome. But some people may only be interested in a secondary outcome such as weight loss. Therefore, they may drop out if they see no metabolic/anthropometric changes from baseline.

From the literature it is clear that investigators closely analyze the drop-out ratios between interventional and control arms, but rarely consider this issue in a time-dependent way. Did participants withdraw after weeks, months or years from baseline? Did the withdrawal times of volunteers in the placebo arm overlap with those in the intervention arm?

Methods and results

We are not dealing with an “intention to treat” analysis or an “ad interim” analysis according to the study protocol. We did a time-dependent analysis only on participants who withdrew for “other reasons” after randomization. The total number of events is 23 (MET 13, placebo ten). The primary outcome of this analysis is the point of drop-out from the trial, defined as the time from the randomization date to the withdrawal date. This means that all subjects included in this analysis developed the outcome. Survival functions were non-parametrically estimated using the product-limit estimator, also known as the Kaplan–Meier estimator (Figure 1). The curves were statistically compared using the non-parametric log-rank test (P=0.64, not significant).

Figure 1.

Kaplan–Meier estimator of participants who withdrew from MeMeMe trial.⁸

Note: The red line refers to the metformin group, green to placebo.

Although there were no reasons to suppose any violation of proportional hazard rule (the basic assumption that permits one to apply the semi-parametric Cox regression model, as outlined by Harrel),¹⁰ our analysis showed the curves crossing. Therefore we did not apply a Cox regression model to estimate the HR¹¹ and could not adjust for important covariates such as age, sex, or education which might affect the time-dependent decision to withdraw.

Baseline covariate distributions were summarized using descriptive statistics (mean and SD for continuous variables, and absolute and percentage frequencies for categorical variables). Table 1 shows the baseline features of the 420 volunteers randomized to MET/placebo within the whole population enrolled in MeMeMe trial. Otherwise, Table 2 shows the baseline covariate distribution of the 23 people taken into account for this peculiar survival statistical analysis. In Table 1 it is shown that people randomized to MET group were older and thinner than volunteers in control group, although this difference is not statistically significant. The placebo group had a better pressure profile (not significant). The unique significant difference between the two randomized arms is high-density lipoprotein cholesterol (P<0.01), but we are confident that enlarging the sample size during the randomization process would smooth out this gap.

Table 2.

Baseline characteristics of the 23 people who dropped out after randomization by group

Characteristic	Placebo (10) (mean ± SD)	Metformin (13) (mean ± SD)	P-value*
Age (years)	54.67±5.43	56.78±7.0	0.52
Sex (n, %)	Male (2, 20%)	Male (1, 8%)
	Female (8, 80%)	Female (12, 92%)
Weight (kg)	85.63±17.35	85.86±16.95	0.91
Waist circumference (cm)	96.68±13.01	99.18±12.36	0.10
Diastolic pressure (mmHg)	90.83±10.63	86.33±14.43	0.50
Systolic pressure (mmHg)	144.67±19.53	135.56±31.44	0.50
Glycemia (mg/dL)	97.5±7.56	98.44±7.0	0.81
Total cholesterol (mg/dL)	196.5±28.87	217±25.56	0.19
HDL (mg/dL)	56.0±15.62	57.4±15.94	0.46
LDL (mg/dL)	121.83±30.45	136.89±27.02	0.35
Triglycerides (mg/dL)	99.50±42.64	102.55±45.46	0.90
AST (IU/L)	23.67±6.81	20.78±6.83	0.44
ALT (IU/L)	28.5±14.31	27.67±18.92	0.92
GammaGT (IU/L)	30.83±15.52	31.88±17.78	0.91
Education (n, %)
Junior high school	(2, 20%)	(3, 23%)	0.17
High school	(6, 60%)	(9, 69%)
University	(2, 20%)	(1, 8%)
Working/retired (n, %)
Worker	(7, 70%)	(10, 69%)	0.99
Retired	(3, 30%)	(3, 31%)

Note:

^*Student’s t-test P-value for continuous variables, chi-squared test P-value for qualitative variables.

Abbreviations: HDL, high-density lipoprotein; LDL, low-density lipoprotein.

The small sample (23 persons) taken into account in Table 2 may lead to clinical difference: waist circumference is bigger in MET group, on the contrary, diastolic pressure and systolic pressure is worse in placebo group. Nevertheless, the differences are not significant.

Comparing Tables 1 and 2, we can see that the proportion of men and women changes: Table 2 shows a larger proportion of women who have left the study (from 64% to 80% in placebo group and from 67% to 92% in treatment group). This phenomenon can be explained in different ways: are men less interested in losing weight and therefore remain in primary prevention study even in case of no changes in anthropometric features? The small sample size does not permit such considerations. However, this point can be a subject of debate among investigators who are planning primary prevention clinical trials. Statistical analysis was done using R software, version 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria).¹²

These very preliminary results do not suggest any significant difference between the time-dependent drop-outs in the MET and placebo arms (Figure 1).

Discussion

Researchers involved in primary prevention RCTs seem to have to deal with the paradox of the proverbial short blanket syndrome. Recruiting only highly motivated candidates might be useful for the smooth progress of the trial, but it may also lead to a very low enrollment rate. On the other hand, what about enrolling all the eligible subjects without considering their motivation for taking part in the trial? This might boost the enrollment rate, but it can also lead to biased results on account of large proportions of drop-outs.

In this complex scenario, the preliminary interview before candidates sign the written informed consent form plays an important part in the subsequent management of the clinical trial. The principal investigator (PI)’s experience is the key to guaranteeing a correct balance between “open mesh” and “strictly selected population” recruitment.

Ethical and methodological aspects must also be taken into account. Clinical research should be open to all subjects who express willingness to be involved in a trial and who fulfill the inclusion criteria. There are important government portals which provide information about clinical research studies to patients, their families, and health care professionals. ClinicalTrials.gov provides information about study locations and specific contact information to assist with enrollment.⁵ Another powerful tool is ResearchMatch,¹³ a free and secure registry developed by major academic institutions across the United States. All Americans have access to the portal and could agree to be contacted by research groups involved in clinical trials for the specific features required. The website explains clearly that “ResearchMatch can help ‘match’ you with any type of research study, ranging from surveys to clinical trials, always giving you the choice to decide what studies may interest you.”¹³ This statement makes it quite clear that the patient is free to choose whether a given clinical trial is suitable for her/him. Why refuse a candidate who may be eligible, only because the PI suspects that she/he might eventually withdraw for trivial reasons? Here we are again up against the short blanket syndrome. It is a chronic issue that primary prevention research must deal with.

There is a growing opinion that clinical research must, as much as possible, adhere to a pragmatic point of view.^14–16 Pragmatic trials are RCTs which compare interventions in clinical settings and look at a range of effectiveness outcomes and impacts. Reducing the follow-up losses is mandatory.

Our experience suggests that participants do not change their mind depending on the allocation group (intervention or control). There is no single answer to sort out the short blanket syndrome. An investigator’s experience is essential for clinical research.

The UK Supreme Court in 2015 declared that “Patients are now widely regarded as persons holding rights, rather than as the passive recipients of the care of the medical profession”, thus affirming the patient as a subject and not simply the object of medical care.¹⁷ In accordance with the Declaration of Helsinki¹⁸ and with GCP, the PI must complete a number of tasks before a participant gives informed consent.⁹ This is only the first ring in a long chain of contact with participants. Once more, every single step in conducting an RCT is fundamental for the success of the entire study.

Ethics approval and consent to participate

Ethical approval for this research was obtained from the ethics committee of the Milan National Cancer Institute (EudraCT number 2012-005427-32). All the participants have provided written informed consent. The written informed consent form had been previously approved by the ethics committee and Italian Competent Authority (AIFA, Agenzia Italiana del FArmaco).