Barriers and facilitators to the participation of subjects in clinical trials: An overview of reviews

Abstract

Background

The demand for clinical trial participants is today one of the highest it has ever been and continues to increase. At the same time, subject recruitment continues to be problematic and the major reason for clinical trial premature terminations. The literature on clinical trial recruitment, which spans several decades and includes hundreds of studies, has an abundance of findings that can be synthesized by way of an overview to provide a well-informed and complete picture of the factors that determine subject participation.

Objectives

An overview of the systematic reviews that report barriers and facilitators to clinical trial participation was conducted. The extracted data were synthesized, and a thematic framework of the factors that affect subject participation in clinical trials was developed. The overview extended across medical subjects and demographics.

Methods

Thirty reviews that complied with the inclusion criteria were included. These reviews covered 753 relevant primary studies and reported 881 barriers and facilitators. The barriers and facilitators were thematically synthesized and a thematic framework of 20 themes was developed. The quality of the included reviews was assessed and reported.

Main results

Several opportunities to increase clinical trial participation, by developing interventions and changing the trial design, derived from an analysis of the thematic framework. That analysis also showed that most of the 20 themes operate mainly as a barrier or as a facilitator, and that most have an effect across medical subjects. As to the quality elements assessed, some reviews complied almost fully but most only partially.

1. Introduction

Several developments contributed to the large demand for clinical trial participants that started in the second half of the last century and to the corresponding research interest in the recruitment of those participants. One was the significant scientific developments of the 19th and 20th centuries that produced a multitude of new medicinal products that needed to be tested, particularly after WWII [[1], [2], [3]]. A second was the emergence in the 1940s and 1950s of clinical trials incorporating all the essential elements, common today, like control groups, randomization, and blinding [[3], [4], [5], [6]]. A third was the development of a drug regulatory environment [2,3,6]. In the United States, starting with the basic regulations of 1906 and evolving into the much stricter 1962 Kefauver-Harris Amendment that required not only proof of safety for new drugs, but substantial evidence of efficacy based on adequate and well-controlled studies [2,3]. A fourth was the development of strict regulations for the ethical conduct of clinical research and the protection of study participants [2,4,5].

These developments resulted in the creation of a clinical trials industry that grew quickly, even exponentially, and demanded a large and ever-increasing number of study participants [[2], [3], [4]]. The challenge of recruiting so many participants contributed to a new line of research inquiry that started, according to our research, in the second half of the last century, but most notably in the 1970s and 1980s [[7], [8], [9], [10], [11], [12], [13], [14], [15], [16]]. That literature on participant recruitment grew quickly during the 1990s, accelerated in part by the need for study participants in the HIV vaccine trials and by the requirements of the NIH Revitalization Act of 1993 for the inclusion of women and minorities in clinical research. After the 1990s the literature continued to develop in response to a continually increasing demand for study participants and problematic recruiting activities. As of this writing, clinicaltrials.gov lists 362,558 research studies, with 55,565 actively recruiting [17]. And, still today, recruitment is problematic and contributes to longer trials, the need for additional study sites, and smaller sample sizes; and it is the major reason for clinical trial premature terminations [[18], [19], [20], [21], [22], [23]].

The literature on participant recruitment is by now well developed and includes hundreds of primary studies and a good number of reviews. It includes studies from many countries, has covered clinical trials for many medical subjects, and has covered many of the demographics and ethnicities of the participant groups. It has also covered, via commentaries and empirical studies, many of the ethical and other concerning topics associated with participant recruitment. The research methods employed go from the simple to the sophisticated and include qualitative, quantitative, and mixed-methods studies, among others.

Our objective was to systematically summarize this extensive literature, with such an abundance of findings, to better understand the factors that affect clinical trial participation. We conducted an overview of the systematic reviews that report barriers and facilitators to clinical trial participation. The extracted data were synthesized, and a thematic framework of the factors that affect clinical trial participation was developed.

2. Methods

2.1. Criteria for selecting reviews for inclusion

Before presenting the inclusion and exclusion criteria, two clarifications about the scope of the overview are discussed. First, the clinical trials recruitment literature has investigated both the enrollment of study subjects and the enrollment of the physicians that either conduct the clinical trials or perform subject recruitment for those trials. In the current study, only the enrollment of study subjects is covered.

Second, Ford et al. [24] proposed a model that classifies into three categories the factors that affect subject recruitment: Awareness, Opportunity, and Acceptance/Refusal. As Ford et al. [24] explain, “to accept or refuse participation in a clinical trial, an individual must be aware that the study is being conducted and must have an opportunity to participate” (p. 229). In the current study, only factors affecting Acceptance/Refusal are covered. Of the three categories, the most complex is Acceptance/Refusal, and most of the factors reported in the literature belong to this category.

As to inclusion criteria, the overview included publications that satisfied all the following criteria: (a) were systematic reviews, or other types of literature reviews, that systematically reviewed primary studies with the main objective of identifying barriers and facilitators to clinical trial participation; (b) were publications that reported barriers and facilitators that corresponded, mainly or exclusively, to the Acceptance/Refusal category of Ford et al. [24]; (c) were written in English; (d) were focused on clinical trials as the type research study where potential participants were invited to participate; and (e) were focused, mainly or exclusively, on adults.

As to exclusion criteria, the overview excluded publications that satisfied any of the following criteria: (a) were publications in which the literature review was not the sole purpose of the study - in addition to the literature review the publication reported the results of a qualitative or a quantitative empirical study; (b) were publications that reported barriers and facilitators that corresponded, mainly or exclusively, to the Awareness and Opportunity categories of Ford et al. [24]; (c) were publications that reported, mainly or exclusively, barriers and facilitators to the participation of physicians in clinical trials; (d) were written in languages other than English; (e) were focused not on clinical trials but on another type of research study; (f) were focused exclusively on children; (g) were focused on the study of interventions to increase clinical trial recruitment; (h) were overviews of reviews; or (i) were commentaries or viewpoints, rather than formal systematic reviews.

Two of the authors (ERT, CDP) independently applied the inclusion/exclusion criteria. Differences were resolved by discussion and consensus. All final decisions were documented.

One additional clarification is that in this study we adopted the definition of systematic review presented by Gough et al. [25]. According to them, for a review to be systematic, it must include four key elements: (a) the review question, (b) the identification and description of the relevant research covered, (c) the critical and systematic appraisal of the research reports covered and the synthesis of findings, and (d) the presentation of statements related to the review question that can be justified based on the research evidence reviewed.

2.2. Search methods for identification of reviews

Two of the authors (ERT, MGP) conducted the search activities. One of the authors (MGP) is an experienced School of Medicine research librarian. The search activities were conducted independently, but the same searches were not duplicated - each person conducted different searches.

The following electronic databases were included in the search activities: Cochrane Database of Systematic Reviews, PubMed/Medline, CINAHL, PsycInfo, Scopus, Epistemonikos, DoPHER, DARE, HTA (Health Technology Assessment Database), Wiley, and Google Scholar. The search of these databases was not limited to specific dates.

In addition to searching the databases, one of the authors (ERT) examined the references section of all the relevant reviews identified and of all the relevant primary studies identified in a scoping review conducted at the start of this research project to investigate the nature and extent of the participation in clinical trials literature.

The following search terms were employed in the search strategy, individually and in combinations: clinical trial; clinical trials, trial, participation, recruitment, enrollment, barriers, facilitators, systematic reviews, and reviews. The search strategies for two of the databases are presented in the supplemental files.

The initial search activities were conducted during December of 2017 and during 2018. The bulk of the search activities, however, took place during March and November of 2019. The search conducted during November of 2019 is the catch-up search, recommended in the literature, that takes place just before starting the synthesis process [25].

The search activities concentrated mainly on published studies. They also covered, however, reports considered grey literature that are published by non-commercial and non-academic organizations.

2.3. Data collection and primary outcome measures

The first column of Table 1 identifies the specific data items extracted from the reviews included in this overview study. Among them, the three primary outcome measures of interest: Barriers Reported, Facilitators Reported, and Unclassified Factors Reported. Unclassified Factors are those factors that affect clinical trial participation as a barrier and/or as a facilitator but were not classified as a barrier or as a facilitator in the review. In the rest of this document, the word factor will be used at times to facilitate the reference to these three types of factors that affect clinical trial participation: barriers, facilitators, and unclassified.

Table 1.

Data extracted and data requirements for assessing the quality of the systematic reviews.

Data items extracted from the systematic reviews	Required by:
	AMSTAR [26]	PRISMA [27]	IOM [28]
General:
Title
Year published
Authors
Number primary studies included
Total sample size - all primary studies combined
Research question	X	X	X
Population covered
Barriers reported
Facilitators reported
Unclassified factors reported (affect the participation decision but are not classified as barriers or facilitators)
Source of barriers, facilitators, and unclassified factors reported (synthesis process or primary studies?)
For assessing the quality of the systematic reviews:
Criteria for including or excluding primary studies	X	X	X
Number of independent reviewers who applied inclusion/exclusion criteria for primary studies	X		X
Databases searched for identifying primary studies	X	X	X
Other sources consulted for identifying primary studies	X	X	X
Were the search keywords identified? (yes/no)	X	X	X
Were the search dates identified? (yes/no)		X	X
Were the primary studies included identified? (yes/no)	X	X	X
Were the primary studies included described (table of characteristics)? (yes or no)	X	X	X
Were the primary studies excluded identified? (yes/no)	X		X
Method for assessing quality of primary studies	X		X
Were the implications of the quality assessment of primary studies covered in the discussion and/or conclusions? (yes/no)	X
Was the homogeneity/heterogeneity of the primary studies assessed and/or discussed? (yes/no)	X
Was publication bias assessed, discussed, or considered? (yes/no)	X	X
Number of independent reviewers who extracted the data from primary studies	X	X	X
Was a judge or consensus process used to resolve differences among reviewers who extracted the data from the primary studies? (yes/no)	X		X
Was a comprehensive process to synthesize the findings followed and reported? (yes/no)		X	X
Which was the synthesis method followed?		X	X
Was conflict of interest addressed? (yes/no)	X		X

Two of the authors (ERT, MGP) independently extracted the data from each of the reviews included in this overview and independently entered the data into an Access database. After the data entry was completed, these two authors met and compared their independent data entries for each of the reviews to identify, discuss, and reconciled differences by consensus. Another author (CDP) served as a judge to resolve differences that could not be reconciled by consensus.

The authors of some of the reviews included were contacted to clarify information or to request missing information. All responded.

2.4. Quality of included reviews

AMSTAR [26] was the main instrument used to assess the quality of the reviews included in the overview. The requirements of two other instruments were also incorporated to complement AMSTAR: (a) PRISMA [27], and (b) the standards for systematic reviews published by the Institute of Medicine of the National Academies (National Academy of Medicine) [28]. All applicable items from these three instruments were incorporated into the quality review. Table 1 identifies the data items extracted for the quality review together with an indication of the quality assessment instruments that specified their use.

2.5. Synthesis of results and thematic framework development

The Thematic Synthesis method [[29], [30], [31]] was followed to synthesize the data on barriers, facilitators, and unclassified factors extracted from the reviews and to produce a thematic framework of the factors that affect clinical trial participation. This synthesis method is used for encoding qualitative information using themes that describe, organize, and interpret the qualitative information [29]. According to Barnett-Page and Thomas [32], the epistemological assumption of this synthesis method is critical realism, where “knowledge of reality is mediated by our perceptions and beliefs.”

Before starting the synthesis process, described next, a workshop was conducted to discuss with the authors the specifics of the Thematic Synthesis process and to train them in the specific activities they were about to conduct.

The following are the specific activities followed in the synthesis process, in the order followed: (1) All the three authors independently studied the barriers, facilitators, and unclassified factors extracted from the reviews (raw data) and independently defined an initial list of no more than 20 themes. These initial lists of themes were generated inductively from the raw information, which is one of the methods recommended in the Thematic Synthesis literature [29]. (2) The three authors met to discuss their individual lists of synthesis themes and to integrate them into a single list. In Thematic Synthesis this integrated list is referred to as The Code [29], and it is the list of themes used in the synthesis process to codify the raw data (in this study, the individual factors reported in the reviews). As part of this step, a Codebook was prepared that included the following for each of the 20 themes in the code: a label, a definition, indicators of when the theme occurs (inclusions), indicators of when the theme does not occur (exclusions), and examples of factors belonging to the theme. The codebook provided general guidance during the encoding process of the next two steps where the code's themes were assigned, or applied, to the raw data. (3) Two of the authors (ERT, MGP), to validate the code, independently assigned the themes in the code to a sample of 105, or about 12 %, of the factors in the extracted raw data. The computed interrater reliability was 0.88. Based on the experience using the code, several clarifications were incorporated into the codebook. No additional themes, however, were necessary. (4) The same two authors (ERT, MGP) independently assigned the themes in the code to the rest of the factors in the extracted raw data. The final interrater reliability for the whole process (all factors) was 0.76, which is above the 70 % minimum recommended [29]. (5) The two authors (ERT, MGP) who assigned the code's themes to the factors in the extracted raw data met to discuss and reconcile their coding disagreements by way of consensus. An agreement was reached on a final assignment of themes to all the factors in the extracted raw data and that final version was used in the rest of the analysis. (6) The three authors met and decided against defining a lower level of subthemes for the 20 themes in the code – the 20 themes were already detailed enough.

The interrater reliability was computed as the percentage of agreement between the two raters. Specifically, the number of times both coders agreed divided by the number of times coding was possible. This method was selected based on the nature of the coding in this study and on the literature's recommendations [29]. Because of the nature of the computation, both raters were given the same weight. Also, both raters had about the same level of coding experience.

2.6. Data management

A database was designed and implemented using Microsoft Access to store and manage the data collected and analyzed in this overview study. Microsoft EXCEL and Power BI were also used in the analysis of data exported from the Access database. The bibliographic data were stored and managed using Mendeley and RefWorks-ProQuest.

2.7. Other analysis

CCA (Corrected Covered Area) [33] was computed as a measure of the degree of overlap in the primary studies covered in the reviews included in the overview. This computation and assessment are recommended by the Cochrane Handbook for Systematic Reviews of Interventions in their Overviews of Reviews chapter [34].

3. Results

3.1. Search results and description of included reviews

The PRISMA Flow Diagram is presented in Fig. 1. As indicated in the diagram, about 21,086 records were identified, of which 4957 were screened by looking at the title and, if necessary, by looking at the abstract. Abstracts were consulted when a title seemed relevant but additional information was needed to make a final call. Of the 4957 records screened, only 61 were found relevant and were assessed for eligibility by examining the full text. Of the 61 assessed for eligibility, 30 were included and are described in Table 2 and 31 were excluded and are described in Table 3. The main reasons for exclusion were: (a) not following a systematic review process, (b) having a focus on interventions rather than on barriers and facilitators to clinical trial participation, (c) not reporting the barriers and facilitators of interest, and (d) reporting same data as another review already included.

Fig. 1.

Prisma flow diagram – records identified, screened, assess for eligibility, and included.

Table 2.

Characteristics of included publications.

First Author, Publication Year, Title	Research Question	Population	Medical Subject	Demographics	Primary Studies	Factors Reported
Bell, Jennifer A. H.; 2015; Cancer patient decision making related to clinical trial participation: an integrative review with implications for patients' relational autonomy [44]	The objective of this integrative review of the literature is to summarize the factors and contexts that influence cancer patient decision-making related to CT participation. A secondary objective is to analyze how sociopolitical influences impact cancer patients' relational autonomy within the context of CT decisions.	Cancer patients who are potential participants in clinical trials.	Cancer	No Restrictions	51	From synthesis
Biedrzycki, Barbara A.; 2010; Decision making for cancer clinical trial participation: A systematic review [45]	The purpose of this systematic review is to describe the current state of the science regarding patient decision making for cancer clinical trial participation.	Cancer patients (adults).	Cancer	No Restrictions	16	From synthesis
Cox, K.; 2003; Why patients don't take part in cancer clinical trials: An overview of the literature [46]	Through such a review it is hoped that insights will be gained into some of the barriers to trial participation and subsequently into ways of overcoming them. [Initially restricted to cancer clinical trials but later expanded to include clinical trials outside cancer.]	Initially restricted to potential participants in cancer clinical trials but later expanded to include clinical trials outside cancer.	No Restrictions	No Restrictions	35	From synthesis
Detoc, M.; 2017; Barriers and motivations to volunteers' participation in preventive vaccine trials: a systematic review [47]	Our aim, without focusing on HIV vaccine trials though, is to identify the common and specific barriers as well as the motivations which influence potential volunteers whether to take part or not in PVT (Preventive Vaccine Trials).	Potential volunteers for Preventive Vaccine Trials.	Vaccines	No Restrictions	17	From synthesis
Dhalla, Shayesta; 2011a; Barriers of enrolment in HIV vaccine trials: A review of HIV vaccine preparedness studies [48]	In this article, we categorize and examine barriers identified in research of this kind for participation in phase III HIV vaccine trials identified in various populations in HIV VPS, and also compare these barriers between Organization for Economic Co-operation and Development countries (OECD) and non-OECD countries.	Potential participants in HIV vaccine trials	HIV	No Restrictions	53	From synthesis
Dhalla, Shayesta; 2011b; Motivators of enrolment in HIV vaccine trials: A review of HIV vaccine preparedness studies [49]	The present article reviews both social and personal motivators of WTP (Willingness to Participate) in HIV vaccine trials in both the OECD and the non-OECD countries, specifically in the context of HIV VPS (Vaccine Preparedness Studies).	Potential participants in HIV vaccine trials in both the OECD and the non-OECD countries.	HIV	No Restrictions	35	From synthesis
Ellis, Peter; 2000; Attitudes towards and participation in randomised clinical trials in oncology: A review of the literature [50]	This paper broadly reviews the issues concerning patient and physician participation in randomised clinical trials.	Patient and physician participants in randomised clinical trials.	No Restrictions	No Restrictions	22	From synthesis
Fayter, Debra; 2007; A systematic review highlights threats to validity in studies of barriers to cancer trial participation [51]	To investigate the barriers, modifiers, and benefits involved in participating in randomized controlled trials of cancer therapies as perceived by health care providers and patients.	Health care providers & patients who are potential participants of randomized controlled trials of cancer therapies.	Cancer	No Restrictions	56	From synthesis
Forcina, Victoria; 2018; Perceptions and attitudes toward clinical trials in adolescent and young adults with cancer: a systematic review [52]	We aimed to conduct a systematic review of studies limited to AYA patients which assessed attitudes and beliefs that influence cancer CT enrollment to prioritize areas for future study and intervention.	Adolescent and young adults with cancer (AYA) who are potential participants in clinical trials	Cancer	Adolescents and Young Adults (AYA)	6	From synthesis
Ford, Jean G.; 2008; Barriers to recruiting underrepresented populations to cancer clinical trials: A systematic reviewa [24]	To determine the barriers to participation of underrepresented populations in cancer-related trials.	Underrepresented populations - cancer clinical trialsb	Cancer	Under-represented Populationsb	65	From synthesis
Grand, Melissa M.; 2012; Obstacles to participation in randomised cancer clinical trials: A systematic review of the literature [53]	This review examines the relationship between the obstacles to participation in cancer clinical trials and accrual, focusing wherever possible on clinical trials in Radiation Oncology.	Potential participants in cancer clinical trials (clinicians and patients) (focusing wherever possible on clinical trials in Radiation Oncology).	Cancer	No Restrictions	24	From primary studies
Gregersen, Trine A.; 2019; What matters in clinical trial decision-making: a systematic review of interviews exploring cancer patients' experiences [54]	To systematically review and thematically synthesize the experiences of patients and relatives when they have to decide whether or not to participate in a clinical oncology trial and to provide knowledge about the decision-making process.	Patients with advanced cancer who are potential participants in clinical oncology trials	Cancer	No Restrictions	11	From synthesis
Hurley-Rosenblatt, Arlene; 2011; Barriers to volunteer enrollment in HIV preventive vaccine clinical research trials: A review of the literature [55]	The purpose of this article is to explore factors that deter recruitment of volunteers into HIV preventive vaccine trials.	Potential volunteers for HIV preventive vaccine trials.	HIV	No Restrictions	4	From primary studies
Limkakeng, Alexander; 2013; Willingness to Participate in Clinical Trials among Patients of Chinese Heritage: A Meta-Synthesis [56]	We carried out a systematic review of literature published between 1985 and 2009 to understand Chinese patients' motivations and concerns to participate in clinical trials.	Patients of Chinese heritage who are potential participants in clinical trials.	No Restrictions	Chinese	5	From synthesis
Luschin, Gero; 2012; Reasons for and against participation in studies of medicinal therapies for women with breast cancer: A debate [57]	We compiled this systematic review to identify reasons why women with, or at high risk of, breast cancer do or do not participate in medicinal studies of breast cancer.	Women with, or at high risk of, breast cancer who are potential participants in medicinal studies of breast cancer.	Cancer	Women	9	From synthesis
Mills, Edward J.; 2004; Barriers to participating in an HIV vaccine trial: A systematic review [58]	Researchers have attempted to identify the barriers to enrolment by questioning individuals potentially eligible for or participating in (HIV) preventative vaccine studies. We sought to synthesize the information from these studies by conducting a systematic review of this literature using content analysis techniques, particularly focusing on the currently existing qualitative data.	Individuals potentially eligible for or participating in (HIV) preventative vaccine studies.	HIV	No Restrictions	26	From synthesis
Mills, Edward J.; 2006a; Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors [59]	We aimed to identify the concerns of patients with cancer about, and the barriers to, participation in clinical trials. We did a systematic review to assess studies of barriers to participation in experimental trials and randomised trials for validity and content.	Patients with cancer who are potential participants in clinical trials.	Cancer	No Restrictions	33	From synthesis
Mills, Edward J.; 2006b; Barriers to participation in HIV drug trials: a systematic review [60]	We systematically reviewed the literature to identify barriers and concerns amongst HIV patients to participation in HIV clinical drug trials.	HIV patients who are potential participants in HIV clinical drug trials.	HIV	No Restrictions	14	From synthesis
Nielsen, Zandra Engelbak; 2019; Cancer patients' perceptions of factors influencing their decisions on participation in clinical drug trials: A qualitative meta‐synthesis [61]	The aim of this study was to examine cancer patients' perceptions of factors that may influence their decisions on participation in phase I–III clinical drug trials.	Cancer patients who are potential participants in phase I–III clinical drug trials	Cancer	No Restrictions	9	From synthesis
Rivers, Desiree; 2013; A systematic review of the factors influencing African Americans' participation in cancer clinical trials [62]	This systematic review was conducted to synthesize the existing evidence regarding key considerations influencing African Americans' participation in cancer clinical trials (CCTs).	African Americans who are potential participants in cancer clinical trials.	Cancer	African Americans	31	From synthesis
Ross, Sue; 1999; Barriers to participation in randomised controlled trials: A systematic review [63]	We report a systematic literature review of barriers to clinician and patient participation in randomised trials, and make recommendations for improving the conduct of trials based on the findings.	Clinicians and patients who are potential participants in randomised controlled trials.	No Restrictions	No Restrictions	78	From synthesis
Salman, Ali; 2016; A review of barriers to minorities' participation in cancer clinical trials: Implications for future cancer research [64]	This paper aims to describe common barriers to the participation of ethnic and racial minorities in cancer clinical trials and discuss the facilitators and possible strategies that could improve the recruitment rate of racial/ethnic minorities in future cancer clinical trials.	Ethnic and racial minorities who are potential participants in cancer clinical trials.	Cancer	Minorities	28	From synthesis
Schmotzer, Geri L.; 2012; Barriers and facilitators to participation of minorities in clinical trials [65]	The purpose of this review is to investigate barriers and facilitators that provide possible explanations for the low participation rate of women and minorities in clinical trials with a specific focus on the field of cancer research.	Women and minorities who are potential participants in clinical trials (with a specific focus on the field of cancer research).	Cancer	Women and Minorities	22	From synthesis
Shah, Jatin Y.; 2010; What Leads Indians to Participate in Clinical Trials? A Meta-Analysis of Qualitative Studies [66]	Our study addressed an important research question as what are the factors, from the perspective of potential Indian participants, that contribute to their participation in clinical trials.	Indians who are potential participants in clinical trials	No Restrictions	Indians	7	From synthesis
Swanson, G. Marie; 1995; Recruiting minorities into clinical trials: Toward a participant-friendly system [67]	The purpose of this review is to describe the state of the art in recruiting participants for clinical trials designed to test new methods of treatment or disease prevention. The ultimate objective of this review is to provide a summary of key issues in recruiting diverse populations into clinical trials, particularly ethnic and racial minorities.	Potential participants for clinical trials (with a focus on diverse populations, particularly ethnic and racial minorities)	No Restrictions	Minorities	39	From synthesis
Todd, Anne M.H.; 2009; A systematic review examining the literature on attitudes of patients with advanced cancer toward research [68]	This systematic review examines the literature on attitudes of patients with advanced cancer toward research and aims to define common themes.	Patients with advanced cancer	Cancer	No Restrictions	11	From synthesis
Tournoux, Caroline; 2006; Factors influencing inclusion of patients with malignancies in clinical trials [69]	We, therefore, sought to review articles about recruitment and willingness to participate in clinical trials in oncohematology to understand why patients may or may not be included.	Potential participants in clinical trials in oncohematology.	Cancer	No Restrictions	75	From synthesis
Townsley, Carol A.; 2005; Systematic review of barriers to the recruitment of older patients with cancer onto clinical trials [70]	Older patients are significantly underrepresented in cancer clinical trials. A literature review was undertaken to identify the barriers that impede the accrual of this vulnerable population onto clinical trials and to determine what specific strategies are needed to improve the representation of older patients in research studies.	Older patients with cancer (>65 years) who are potential participants in clinical trials	Cancer	Older Patients	9	From synthesis
Walsh, Elaine; 2016; Factors affecting patient participation in clinical trials in Ireland: A narrative review [71]	Our objective was to identify the key factors pertaining to patient participation in clinical trials, to better understand the identified low participation rate of patients in one clinical research facility within Ireland.	Patients who are potential participants in clinical trials	No Restrictions	No Restrictions	61	From synthesis
White, Clare; 2010; What do palliative care patients and their relatives think about research in palliative care? —a systematic review [72]	This systematic review aims to identify the views of palliative care patients and their families towards research, the factors that are important when considering participation, and the types of research trial they would support or reject.	Palliative care patients, and their relatives, who are potential participants in research in palliative care.	Palliative	No Restrictions	8	From synthesis

^aFord et al. [73] report the same study and data as Ford et al. [24] in this table. The data on facilitators and the research question were extracted from Ford et al. [73].

^bUnderrepresented populations in Ford et al. [24] included adolescents, older adults (age≥65 years), individuals of low socioeconomic status, individuals who resided in rural areas, African Americans, Latinos/Hispanics, Asian Americans and Pacific Islanders, and American Indians/Alaska Natives.

Table 3.

Characteristics of excluded publications and reasons for exclusion.

First Author	Year	Title	Reasons for Exclusion
Brown, G [74].	2014	Barriers to recruiting ethnic minorities to mental health research: A systematic review	The main focus is not on CT (as the type study where potential participants were invited to participate). Also, some of the PS focused on the study of interventions to increase CT recruitment.
Bugeja L [75].	2018	Barriers and enablers to patient recruitment for randomised controlled trials on treatment of chronic wounds: A systematic review	The main focus is on the study of interventions to increase CT recruitment. Just a few barriers are reported and no facilitators.
Cox, K [76].	1996	Ethical and practical problems of early anti‐cancer drug trials: a review of the literature	The main focus is not on the identification of the barriers and facilitators of interest in this overview.
Dainesi S.M [77].	2014	Reasons behind the participation in biomedical research: a brief review	It is a brief review, a “preliminary recognition” incorporating elements of a critical review. It is not a systematic review – important elements of a systematic review process are missing.
Dawson, S [78].	2018	Black and minority ethnic group involvement in health and social care research: A systematic review	The main focus is not on CT (as the type study where potential participants were invited to participate). The barriers and facilitators of interest in this overview are covered only briefly, and were not reported in most of the PS.
Denson, A.C [79].	2014	Participation of the Elderly Population in Clinical Trials: Barriers and Solutions	It is a literature review, not a systematic review. A significant part of the paper is focused on physician barriers, opportunity barriers, and solutions (interventions).
Dhalla, S [80].	2013	Motivators to participation in medical trials: The application of social and personal categorization	It is an overview of eight reviews (of motivators to participation in actual cancer trials).
Domecq, J. P [81].	2014	Patient engagement in research: A systematic review	The focus is not on the identification of the barriers and facilitators of interest in this overview. The focus is on patient involvement in the design, conduct, and dissemination of research.
Dunleavy L [82].	2018	Using the ‘Social Marketing Mix Framework’ to explore recruitment barriers and facilitators in palliative care randomised controlled trials? A narrative synthesis review	Most of the PS, 34 out of 48, focused on CT design and on interventions to increase CT recruitment.
Fayter, D [83].	2006	Systematic review of barriers, modifiers and benefits involved in participation in cancer clinical trials	The data and results reported are the same as the ones reported in Fayter et al. [51], a systematic review already included in the current overview.
Gaston, C [84].	2005	Information giving and decision-making in patients with advanced cancer: a systematic review	The focus is on the study of interventions to improve information giving and participation in decision-making. The barriers and facilitators of interest in this overview are not discussed.
George, S [85].	2014	A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders	The focus is on health research in general, rather than on CT exclusively (as the type study where potential participants were invited to participate).
Gorini, A [86].	2015	Decision-Making Process Related to Participation in Phase I Clinical Trials: A Nonsystematic Review of the Existing Evidence	It is a nonsystematic review and just three barriers are mentioned, and no facilitators. Focus is on proposing an intervention to improve CT participation decision-making.
Gotay, C. C [87].	1991	Accrual to cancer clinical trials: Directions from the research literature	It is a critical review, not a systematic review – important elements of a systematic review process are missing. Also, some emphasis on the review of interventions and on physician related variables.
Heller, C [88].	2014	Strategies addressing barriers to clinical trial enrollment of underrepresented populations: A systematic review	The focus is on the study of interventions to improve CT enrollment of underrepresented populations - it is not on the identification of the barriers and facilitators of interest in this overview.
Hunninghake, D. B [16].	1987	Recruitment experience in clinical trials: literature summary and annotated bibliography	The main focus is not the identification of the barriers and facilitators of interest in this overview. It is a literature review, not a systematic review - important elements of a systematic review process are missing.
Hussain‐Gambles, M [89].	2004	Why ethnic minority groups are under‐represented in clinical trials: A review of the literature	Many of the barriers reported are Awareness and Opportunity barriers. It is a narrative review, not a systematic review - important elements of a systematic review process are missing.
Lovato, L. C [90].	1997	Recruitment for controlled clinical trials: Literature summary and annotated bibliography	Main focus is on recruitment plans and strategies for CT. It is a literature summary, not a systematic review.
McMahon, V. A [91].	2011	Understanding decision and enabling factors influencing clinical trial participation in Australia: A viewpoint	It is a commentary, a viewpoint, it is not a systematic review.
Newman, P. A [92].	2013	HPV vaccine acceptability among men: a systematic review and meta-analysis	The main focus is on rates and correlates of HPV vaccine acceptability among men. The barriers and facilitators of interest in this overview are not discussed.
Pierce, R [93].	2003	Prostate cancer and psychosocial concerns in African American men: literature synthesis and recommendations	The main focus is on prostate cancer and psychosocial concerns in African American men. The barriers and facilitators of interest in this overview are not discussed.
Reifenstein, K [94].	2018	A commentary: Will we ever get enough? strategies to enhance minority participation in research	It is a commentary– important elements of a systematic review process are missing. Also, it has a significant focus on interventions to increase CT recruitment.
Ridda, I [95].	2010	Difficulties in recruiting older people in clinical trials: An examination of barriers and solutions	It is a literature review, not a systematic review. A significant part of the paper is focused on solutions (interventions) and opportunity barriers.
Stunkel, L.a [96]	2011	More than the money: A review of the literature examining healthy volunteer motivations	The studies reported in some of the PS are not clinical trials.
Tishler, C. L [97].	2002	The recruitment of normal healthy volunteers: A review of the literature on the use of financial incentives	It is a literature review, not a systematic review. The main focus is not the identification of the barriers and facilitators of interest in this overview.
Treweek, S [98].	2013	Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis	The focus is on the study of interventions to improve recruitment to randomized controlled trials - it is not on the identification of the barriers and facilitators of interest in this overview.
Unger J.M [99].	2019	Systematic Review and Meta-Analysis of the Magnitude of Structural, Clinical, and Physician and Patient Barriers to Cancer Clinical Trial Participation	A significant part of the study is focused on Awareness and Opportunity barriers. The focus is not on the identification of the barriers and facilitators of interest in this overview.
UyBico, S. J [100].	2007	Recruiting vulnerable populations into research: a systematic review of recruitment interventions	The focus is on the study of interventions to enhance the enrollment of vulnerable populations into health research studies. The focus is not on the identification of the barriers and facilitators of interest in this overview.
Ward, L. C [101].	1999	A systematic review reporting doctors' and patients' attitudes toward participation in clinical research trials: Final report	Only a draft version of this publication was made available to the authors of the current overview. The authors were not able to get a copy of the final report.
Wilder, J [102].	2016	A systematic review of race and ethnicity in hepatitis C clinical trial enrollment	It is a study of African American participation rates in North American HCV clinical trials. It does not cover the barriers and facilitators of interest in this overview.
Zhang, T [103].	2013	Reporting and representation of ethnic minorities in cardiovascular trials: A systematic review	The focus is on the reporting and representation (% of enrollment) of ethnic minorities in cardiovascular trials. The barriers and facilitators of interest in this overview are not covered.

Note. CT = clinical trial or clinical trials; PS = primary studies.

^aAn additional consideration of Stunkel and Grady [96], when preparing this table, persuaded the authors to recommend that this publication be considered for inclusion in future similar overview studies.

Of the 21,086 records identified, only 4957 were screened because in some of the electronic databases consulted only the first results of some of the searches were relevant for initial screening.

Some notable characteristics of the reviews included, described in Table 2, include the publication year. Only two were published in the 1990s, one in 1995 and the other in 1999. Most were published after 2005 (n = 24), with at least one published in most years thereafter. Also notable are the medical subjects and demographics covered. Several medical subjects are covered, but the majority covered Cancer (n = 16) and HIV (n = 5) exclusively. As to demographics, several are also covered, but only two are covered in more than one review, women (n = 2) and minorities (n = 5). Two of the reviews were dedicated to specific ethnicities and most reviews were not restricted to a particular demographic. Another notable characteristic is the number of primary studies covered. A grand total of 860 primary studies were covered, ranging from 4 to 78 per review.

The sample of reviews included in this overview study is a heterogeneous sample. In this study, homogeneity was not required because no aggregations or computations requiring homogeneity were included [25]. If anything, we were expecting heterogeneity; and it was welcomed. It was expected because in an overview of an extensive literature, as the one conducted, the reviews included will naturally cover different population characteristics. And it was welcomed because in a thematic synthesis like the one conducted the variability resulting from heterogeneity contributes to better results by showing how the outcomes of interest vary across population characteristics [30,35].

As recommended in the literature, immediately prior to submitting the manuscript for publication, a top-up search was conducted to identify systematic reviews that comply with the inclusion criteria and were published after our last search [36]. Five reviews were identified and are listed in references [[37], [38], [39], [40], [41]]. In addition, after finalizing the synthesis activities, two additional reviews that had not been identified, and complied with the inclusion criteria, came to our attention. These two reviews are also identified in references [42,43]. All these newly identified reviews were examined, and it was determined that the factors reported in these studies are not different from the factors already covered in the synthesis and that neither the thematic framework nor the conclusions presented in this overview will change if these new studies were to be included.

The search for studies in a qualitative thematic synthesis like the one conducted in this overview study does not have to be exhaustive [25,36]. Once a particular concept is identified, not much is gain by adding studies that identify the same concept [30,35]. Conceptual saturation, according to the literature, has been achieved [25].

In this study, the search strategy was linear and was planned at the beginning [25]. And although the objective was not to be exhaustive, it was to identify as much of the relevant existing reviews as it was possible and practical. Because an entire literature was being examined, the aim was to achieve, in the words of Gough et al. [25], “sufficient breadth and representation” (p. 100) of what has been published. The search, however, was constrained by the limitations and difficulties that are natural to these activities, that make exhaustive searches impractical and improvable, and that are well documented in the literature [25].

3.2. Methodological quality of included reviews

The information extracted for the assessment of the quality of the included reviews is presented in Table 4.

Table 4.

Quality elements of reviews included in overview.

	Bell	Biedrzycki	Cox	Detoc	Dhalla	Dhalla	Ellis	Fayter
	2015 [44]	2010 [45]	2003 [46]	2017 [47]	2011a [48]	2011b [49]	2000 [50]	2007 [51]
No. Primary Studies (PS)	51	16	35	17	53	35	Yes/Noj	56
Total Sample Size (all PS)	Yesa	Not reported	Not reported	Not reported	Yesa	Not reported	Not reported	Not reported
Research Question Included?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Methods Description	Detailed	Detailed	Some detail	Detailed	Detailed	Detailed	General	Detailed
Databases Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Non-Database Sources?	No	Yes	Yes	Yes	Yes	Yes	No	Yes
Search Dates Identified?	No	No	No	Yes	Yes	Yes	No	Yes
Search Keywords Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Inclusion/Exclusion Criteria Identified?	Yes	Yes	No	Yes	Yes	Yes	No	Yes
No. Reviewers Applied I/E Criteria	2	1	Not reported	Not reported	Not reported	Not reported	Not reported	2
PS Included Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes/Noc	Yes
PS Included Described? i	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
PS Excluded Identified?	No	No	No	No	No	No	No	No
Quality of PS Assessed?	Yes	Yes	Yes	No	No	Yes	No	Yes
Implications of PS Quality Discussed?	Very briefly	Yes	Very limited	No	No	Yes	No	Yes
PS Homogeneity Covered?	No	Yes	Yes	Yes	No	No	No	Yes
Publication Bias Covered?	No	No	No	Yes	No	No	No	Yes
No. Reviewers Extracted PS Data	2b	Not reported	Not reported	2	Not reported	Not reported	Not reported	2
Judge/Consensus in Data Extraction?	Yes	Not reported	No	Not reported	Not reported	No	Not reported	Yes
Comprehensive Synthesis Process?	Yes	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok	Yes
Conflict of Interest Addressed?	Yes	Yes	No	Yes	No	No	No	Yes
	Forcina	Ford	Grand	Gregersen	Hurley-Rosenblatt	Limkakeng	Luschin	Mills
	2018 [52]	2008 [24]	2012 [53]	2019 [54]	2011 [55]	2013 [56]	2012 [57]	2004 [58]
No. Primary Studies (PS)	6	65	Yes/Nod	11	4	5	9	26
Total Sample Size (all PS)	Yesa	>627,044	Yesa	Yesa	Not reported	Yesa	Yesa	Yesa
Research Question Included?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Methods Description	Detailed	Detailed	Detailed	Detailed	Some detail	Detailed	Detailed	Detailed
Databases Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Non-Database Sources?	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
Search Dates Identified?	No	Yes	No	No	No	No	Yes	No
Search Keywords Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Inclusion/Exclusion Criteria Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
No. Reviewers Applied I/E Criteria	Not reported	2	2	1	Not reported	Not Reported	3	2
PS Included Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes	Yes
PS Included Described? i	Yes	Yes	Yes	Yes	No	Yes	Yes	Yes
PS Excluded Identified?	No	No	No	No	No	No	No	No
Quality of PS Assessed?	Yes	Yes	No	Yes	No	Yes	Yes	Yes
Implications of PS Quality Discussed?	Yes	Yes	No	No	No	Yes	Yes	Partially
PS Homogeneity Covered?	No	Yes	No	No	No	No	No	No
Publication Bias Covered?	No	No	No	No	No	No	No	Yes
No. Reviewers Extracted PS Data	2	2	Not reported	Not reported	Not reported	2	3	2
Judge/Consensus in Data Extraction?	Yes	Yes	Not reported	Not reported	No	Yes	Not reported	Yes
Comprehensive Synthesis Process?	Yes	Yes	No	Yes	No	Yes	Yes/Nok	Yes
Conflict of Interest Addressed?	Yes	Yes	Yes	No	Yes	Yes	Yes	Yes

	Mills	Mills	Nielsen	Rivers	Ross	Salman	Schmotzer
	2006a [59]	2006b [60]	2019 [61]	2013 [62]	1999 [63]	2016 [64]	2012 [65]
No. Primary Studies (PS)	33	14	9	31	78	28	22
Total Sample Size (all PS)	6174	Yesa	236	Not reported	Not reported	Not reported	Not reported
Research Question Included?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Methods Description	Detailed	Detailed	Detailed	Detailed	Some detaile	Some detail	Detailed
Databases Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Non-Database Sources?	Yes	Yes	No	No	No	No	No
Search Dates Identified?	No	No	No	No	No	No	No
Search Keywords Identified?	Yes	Yes	Yes	Yes	No	Yes	Yes
Inclusion/Exclusion Criteria Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
No. Reviewers Applied I/E Criteria	2	2	Not reported	2	Not reported	Not reported	Not reported
PS Included Identified?	Yes	Yes	Yes	Yes	Yes	Yes/Noc	Yes/Noc
PS Included Described? i	Yes	Yes	Yes	Yes	Yes	No	No
PS Excluded Identified?	No	No	No	No	No	No	No
Quality of PS Assessed?	Yes	Yes	Yes	Yes	No	No	No
Implications of PS Quality Discussed?	No	No	Partially	No	No	No	No
PS Homogeneity Covered?	Yes	No	No	No	No	No	No
Publication Bias Covered?	Yes	Yes	No	Yes	Yes	No	No
No. Reviewers Extracted PS Data	2	2	Not reported	3	2	Not reported	Not reported
Judge/Consensus in Data Extraction?	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
Comprehensive Synthesis Process?	Yes	Yes	Yes	Yes/Nok	Yes	Yes	Yes/Nok
Conflict of Interest Addressed?	Yes	Yes	Yes	No	Yes	No	No
	Shah	Swanson	Todd	Tournoux	Townsley	Walsh	White
	2010 [66]	1995 [67]	2009 [68]	2006 [69]	2005 [70]	2016 [71]	2010 [72]
No. Primary Studies (PS)	7	Yes/Nof	11	75	9	61	8
Total Sample Size (all PS)	Yesa	Not reported	Yesa	Yesg	Not reported	Not reported	Yesa
Research Question Included?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Methods Description	Detailed	Some detail	Detailed	Detailed	Detailed	Detailed	Detailed
Databases Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Non-Database Sources?	Yes	Yes	Yes	Yes	Yes	No	Yes
Search Dates Identified?	No	No	Partially	No	No	No	No
Search Keywords Identified?	Yes	Yes	Yes	Yes	Yes	Yes	Yes
Inclusion/Exclusion Criteria Identified?	Yes	No	Yes	Yes	Yes	Yes	Yes
No. Reviewers Applied I/E Criteria	2	Not reported	2	2	2/3	1	Not reported
PS Included Identified?	Yes	Yes/Nof	Yes	Yes	Yes	Yes/Noh	Yes
PS Included Described? i	Yes	No	Yes	Yes	Yes	No	Yes
PS Excluded Identified?	No	No	Yes	No	Yes	No	No
Quality of PS Assessed?	No	No	Yes	No	No	No	No
Implications of PS Quality Discussed?	No	No	No	No	No	No	No
PS Homogeneity Covered?	No	No	No	No	No	No	Yes
Publication Bias Covered?	No	No	No	Yes	No	No	No
No. Reviewers Extracted PS Data	3	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
Judge/Consensus in Data Extraction?	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported	Not reported
Comprehensive Synthesis Process?	Yes	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok	Yes/Nok
Conflict of Interest Addressed?	Yes	No	No	Yes	Yes	No	Yes

Note. PS = primary studies; I/E = inclusion/exclusion; SR = systematic review.

^aTotal sample size (all PS) was not reported but sample sizes for the individual PS were reported.

^bOne person extracted the data into a matrix, and another independently reviewed the extracted data.

^cThe PS are not identified, but in the results, findings, and/or discussion sections some PS are identified in the citations.

^dTotal number of PS is not indicated but they are identified in the tables.

^eSome elements of the search strategy were not reported but were provided by the authors of the SR.

^fPS are identified in the citations in the discussion and in the tables, but no comprehensive list is presented, nor a total number reported.

^gTotals reported by subject type.

^hSome PS are identified in the citations in a table. The rest were identified by the authors when contacted.

ⁱTable of characteristics.

^jPS are identified in the citations in the text, but no comprehensive list is presented, nor a total number reported.

^kSynthesis performed but synthesis method not identified (reported).

Before making some general observations, it is important to note that nine of the reviews assessed for quality were published before the first of the quality assessment instruments used in this overview was published in 2007 [26].

Another important clarification is that numerical scores or ratings are not provided as part of the quality assessment. The assessment presented is qualitative and is based on a very comprehensive consideration of the elements that according to the assessment instruments determined methodological quality. In addition to the observations made by the authors, the readers will also be able to participate in the assessment by looking at the table and considering not only the overall quality, but the quality of the individual reviews included in the overview.

A look at Table 4 reveals that most of the reviews comply only partially with the quality elements assessed. Only a few of the reviews comply with most of these quality elements. This inconsistency and lack of compliance with some of the methodological quality requirements have been present in other published systematic reviews and has been reported in the literature [104].

Interestingly, there are some quality elements with which most of the reviews complied. These include some of the first nine elements listed in the table, several of which are related to the search strategy followed. These nine quality elements are also some of the ones that have received more attention in the systematic review literature.

By contrast, there are some quality elements, mainly in the lower half of the table, with which most of the reviews did not comply. For instance, most do not identify the primary studies excluded, and most do not report the number of independent reviewers who extracted the data from the primary studies and the mechanism used to resolve differences among these independent reviewers.

3.3. Synthesis of results and thematic framework developed

Table 5 presents the thematic framework that resulted from the thematic synthesis process. To facilitate the presentation, the 20 themes generated were classified into five general categories: About the Trial, About the Potential Participant, About Information, About Others, and About Other Costs and Benefits. For each theme, the table presents the label selected, a definition, and several examples of actual factors reported in the included reviews.

Table 5.

Thematic framework that resulted from the thematic synthesis process.

Synthesis Themes	Definition	Examples of Actual Factors Reported in the Included Reviews
About the Trial:
General Attitude Towards Research, the Healthcare System, and the Pharmaceutical Industry	Attitudes, perceptions, distrusts, and fears towards research at a general level; and towards the Healthcare System and the Pharmaceutical Industry, also at a general level.	distrust of pharmaceutical companies; distrust of the medical profession; uncomfortable with experimentation
Attitude to Trial Specific Individuals and Organizations, Including Trusts and Mistrusts	Attitudes toward the individuals or the organizations of the specific trial where the potential participant is asked to participate.	trust in the physician and the medical institution; belief that (the) investigator is more interested in the research than in (the) patient wellbeing
Characteristics of Trial – Medical and Procedural	Medical and procedural characteristics of the trial and their effect on the trial participation decision. Also, the potential participant's fears, concerns, or perceived risks about the consequences of trial participation.	fear of randomization; concerns about side effects; too much time required to participate; the potential to be followed more closely by their doctor or nurse; quality of life might be reduced
Other Fears, Concerns, and Perceived Risks Not About Trial Characteristics	Fears, concerns, perceived risks, or uncertainties about matters, other than trial characteristics, that affect the trial participation decision. Also, concerns about issues of privacy, confidentiality, or the handling of personal information.	need to switch physicians; family issues/considerations; concern of not receiving appropriate therapy for oneself; fears about confidentiality; anxious about the possibility of detection of something new and unpleasant
Obstacles to Participation	Obstacles to the participation of otherwise willing participants. Obstacles prevent participation because what the participant requires in order to participate is unavailable. If removed, a barrier to participation is removed.	transport and travel difficulties; problems with work schedules and other commitments; loss of income; insurance concerns; childcare/family responsibilities
About the Potential Participant:
Individual Characteristics	The following individual characteristics: socioeconomic, ethnic, demographic, personality, and psychological.	black or Asian respondents, or between 18 and 24 years, lower willingness to participate; younger patients were more favorably disposed towards both survey and therapeutic research
Cultural background	Cultural background of the potential participant.	cultural beliefs or myths about specific diseases or illness in general
Decision Making Style and Preferences, Including Risk Preferences	Decision-making style of the potential participants. Also, decision-making preferences concerning risk-taking, level of participation of the physician in the decision-making process, and others. Additionally, considerations regarding decision making options, including the availability or unavailability of other options in addition to the trial.	belief that (the) doctor should make (the) decisions; do not want to lose control of decision-making; feeling coerced to join
Beliefs in God, Spirituality, or Religion; and Other Beliefs	Beliefs and attitudes toward God, spirituality, and religion. Also, other beliefs.	conflict with religious beliefs; believed their fate was in the hands of God
Dispositions Including Willingness, Preferences, and Dislikes	A predisposition, or lack thereof, that affects the decision to participate. May include willingness, preferences, and dislikes.	discomfort from medical procedures; dislike of needles or injections; want to be drug free
Health, Disease, or Psychological State; and Timing of Request to Participate	Disease state, health state, or psychological state of the potential participant. Also, time -within the course of the disease- when the request to participate is received.	too unwell to participate; recent cessation of injecting drug; decision to enroll at time of diagnosis
Personal experiences	Personal experiences of the potential participant, including previous participation in clinical trials, that affect the trial participation decision.	memory of close person(s) with breast cancer; already decided once to participate in a medicinal study
Hope	Having hope in a general sense, or for a specific trial result.	hope for reduction in tumor size; hoped to prolong life or to manage difficult symptoms
About Information:
Need for Information about the Clinical Trial	Information needs of potential participants for being able to decide on clinical trial participation. Also, how that information is communicated and presented.	lack of knowledge of what is required of trial participants; information about the trial is too technical and too complex to be easily understood
Misconceptions and Misunderstandings	Understandings by the potential participant regarding the clinical trial that are incorrect or have been misunderstood.	reusing disposable syringes; vaccines are lethal
About Others:
Altruism and Other Selfless Motivations	A desire or intention to contribute to the benefit of others who are not related to the potential participant.	desire to help others; protect other people
Contributions to Research	A desire or intention to contribute to research, science, or the advancement of knowledge.	advancing medical knowledge; contributing to scientific knowledge
Influence from Others	Influences from others in the decision to participate. Others may be relatives, friends, physicians, other individuals, or even institutions or the media.	recommendation from family or friends; negative media attention surrounding the intervention
About Other Costs and Benefits:
Costs of Participation	Direct and indirect costs of participation not covered in the other themes. May include actual or perceived costs, and financial or non-financial costs.	costs of participating (direct and indirect)
Benefits of Participation and Incentives	Benefits from participation and incentives not covered in the other themes, including psychological benefits and indirect benefits to those close to the participant.	seeking a personal benefit; incentives (free meals, financial, other)

The labels selected for the themes incorporate, as much as possible, terminology that is common in the clinical trials participation literature. This was done for consistency, to facilitate understanding, and to link the current study with the rest of the literature.

A total of 20 themes resulted from the thematic synthesis process. Although we carefully considered having a smaller number of themes, in the end, those 20 were necessary to cover the breadth and diversity of the factors extracted and synthesized. A total of 881 factors (barriers, facilitators, and unclassified) were extracted from the included reviews and they covered an extensive range of very diverse topics. Having 20 themes was also necessary for minimizing exclusions, maximizing differentiation, and capturing the qualitative richness of the subject; three essential elements of well-defined themes [29].

Another benefit of having more themes is that the presentation of the main factors that affect the participation decision is more specific and clearer, facilitating understanding, analysis, and the identification of areas of opportunity for interventions.

The drawback of having 20 themes was that the synthesis process took more time and was somewhat more complex. Our judgment, however, is that in this case the benefits of having more themes outweighed any potential complications.

One additional observation regarding the results of the thematic synthesis is that no analytical themes were developed. According to Thomas and Harden [30], when the studies being reviewed cover the review question directly, there is no need to go over what is covered in the reviewed studies. In our case, all the included reviews directly covered the review question explored in this overview study. In our next study, already underway, we investigate a research question that requires the definition of higher order concepts that will be based on the thematic framework proposed in the current overview study.

Table 6, Table 7 present additional analysis of the thematic framework developed. Before discussing them, it is important to restate that in this overview study we extracted from the included reviews three types of factors that affect the decision to participate in a clinical trial: barriers, facilitators, and factors unclassified in the reviews as either barriers or facilitators.

Table 6.

Total barriers, facilitators, and unclassified factors per synthesis themes.

Synthesis Themes	Barriers (n = 548)	Facilitators (n = 291)	Unclassified Factorsa (n = 45)
About the Trial:
General Attitude Towards Research, the Healthcare System, and the Pharmaceutical Industry	64		1
Attitude to Trial Specific Individuals and Organizations, Including Trusts and Mistrusts	8	6	1
Characteristics of Trial – Medical and Procedural	191	49	7
Other Fears, Concerns, and Perceived Risks Not About Trial Characteristics	32	4
Obstacles to Participation	67	7	2
About the Potential Participant:
Individual Characteristics	13	8	1
Cultural background	11
Decision Making Style and Preferences, Including Risk Preferences	13	12	6
Beliefs in God, Spirituality, or Religion; and Other Beliefs	7	3	2
Dispositions Including Willingness, Preferences, and Dislikes	32	8	1
Health, Disease, or Psychological State; and Timing of Request to Participate	18	5	4
Personal experiences	2	5
Hope	1	14	1
About Information:
Need for Information about the Clinical Trial	34	19	8
Misconceptions and Misunderstandings	18
About Others:
Altruism and Other Selfless Motivations	1	43	1
Contributions to Research		20	1
Influence from Others	26	27	6
About Other Costs and Benefits:
Costs of Participation	2	1	1
Benefits of Participation and Incentives	8	60	2

Note. The number of factors reported in this table is higher than the number of factors extracted because some extracted factors included more than one concept and were assigned to more than one theme.

^aReported factors that affect the participation decision but were not classified as barriers or facilitators.

Table 7.

Number of Times Synthesis Themes are Present in Each of the Medical Subjects.

Synthesis Themes	Medical Subjects
	Cancer (n = 16)b	HIV (n = 5)b	Vaccines (n = 1)b	Palliative (n = 1)b	No Restrictionsa (n = 7)b
About the Trial:
General Attitude Towards Research, the Healthcare System, and the Pharmaceutical Industry	24 (1.5)	19 (3.8)	6 (6)		16 (2.3)
Attitude to Trial Specific Individuals and Organizations, Including Trusts and Mistrusts	6 (.4)	1 (.2)		1 (1)	7 (1)
Characteristics of Trial – Medical and Procedural	96 (6)	75 (15)	18 (18)	13 (13)	45 (6.4)
Other Fears, Concerns, and Perceived Risks Not About Trial Characteristics	15 (.9)	9 (1.8)	5 (5)	1 (1)	6 (.9)
Obstacles to Participation	36 (2.3)	15 (3)	8 (8)	2 (2)	15 (2.1)
About the Potential Participant:
Individual Characteristics	18 (1.1)			1 (1)	3 (.4)
Cultural background	7 (.4)	1 (.2)			3 (.4)
Decision Making Style and Preferences, Including Risk Preferences	25 (1.6)	1 (.2)		2 (2)	3 (.4)
Beliefs in God, Spirituality, or Religion; and Other Beliefs	9 (.6)	1 (.2)			2 (.3)
Dispositions Including Willingness, Preferences, and Dislikes	26 (1.6)	4 (.8)	3 (3)	4 (4)	4 (.6)
Health, Disease, or Psychological State; and Timing of Request to Participate	20 (1.3)		1 (1)	3 (3)	3 (.4)
Personal experiences	3 (.2)	3 (.6)	1 (1)
Hope	14 (.9)			1 (1)	1 (.1)
About Information:
Need for Information about the Clinical Trial	35 (2.2)	2 (.4)	5 (5)	1 (1)	18 (2.6)
Misconceptions and Misunderstandings	1 (.1)	13 (2.6)	3 (3)		1 (.1)
About Others:
Altruism and Other Selfless Motivations	16 (1)	11 (2.2)	3 (3)	8 (8)	7 (1)
Contributions to Research	11 (.7)	4 (.8)	1 (1)	1 (1)	4 (.6)
Influence from Others	35 (2.2)	7 (1.4)	2 (2)	2 (2)	13 (1.9)
About Other Costs and Benefits:
Costs of Participation	2 (.1)	1 (.2)			1 (.1)
Benefits of Participation and Incentives	36 (2.3)	15 (3)	9 (9)	2 (2)	8 (1.1)

Note. The first of the two numbers presented for each row/column intersection is the number of times that the synthesis theme is present in the reviews of the medical subject. The second number, in parenthesis, is the average of the first number by the number of reviews in the medical subject and is presented as a measure of comparison among the columns.

^aNot restricted to a particular medical subject.

^bNumber of reviews in each medical subject.

Table 6 documents how many of the extracted factors corresponding to each theme are barriers, facilitators, and unclassified factors. This table can be used to assess if the theme operates more as a barrier, as a facilitator, or if it cannot be classified as one or the other. For example, Characteristics of Trials – Medical and Procedural operates more as a barrier than a facilitator. Same for Health, Disease, or Psychological State; and Timing of Request to Participate. Hope, on the contrary, and as expected, operates more as a facilitator.

Table 7 documents, for each theme, how many times it is present in each of the medical subjects covered in the included reviews. This table can be used to assess how common is each theme in each of the medical subjects. Each column in the table incorporates all the reviews that covered the column's medical subject. For each row/column intersection, two numbers are presented: the total number of factors corresponding to the theme in the row and the reviews in the column and, in parenthesis, the total number of factors divided by the number of reviews in the column. This second number, an average per review, is included to facilitate the comparison among the columns. Three caveats must be mentioned. First, the number of reviews per medical subject is very uneven. This is the reason why the averages were included. Second, the sample of reviews for most of the medical subjects is very small; in two cases only one observation. The third caveat is a clarification. This table shows the distribution of factors among the medical subjects in the reviews included in the overview. No generalization to other populations can be made. Finally, some examples of how to use this table. Looking at the table one can see that Hope is common in Cancer reviews but non-existent in HIV and Vaccines reviews. A result that speaks about the differences between being a patient participant versus a voluntary healthy participant. Another theme that operates similarly is Health, Disease, or Psychological State; and Timing of Request to Participate - common in Cancer, but non-existent or less common in HIV and Vaccines. Another interesting one is Characteristics of Trial – Medical and Procedural. This one is more prevalent in HIV than in Cancer (higher average).

In the online supplemental materials for this article another table covering the thematic framework is included. Table A1 is a matrix of the thematic framework themes and the reviews included in the overview. The row/column intersections display the number of factors extracted. This table can be used to assess the impact of the individual reviews on the thematic framework. It also shows how the factors corresponding to each theme are distributed among the included reviews.

3.4. Additional analysis

An additional analysis was conducted to assess the degree of overlapping in the primary studies covered in the included reviews. This is a required analysis for overviews [34] and is presented to provide a better description of the nature of the evidence reviewed and synthesized.

Table 8 shows, for each of the included reviews, how many primary studies are unique (included only in that review) and how many are repeated (included in more than one review). These figures are also totalized at the bottom of Table 8. The analysis in this table is intended to be basic and is complemented by a more comprehensive analysis presented below. Still, according to this basic analysis, there is some degree of overlapping in the primary studies of the included of reviews.

Table 8.

Unique, repeated, and total primary studies per included reviews.

Included Reviews	Primary Studies per Included Reviews
(First Author, Year Published)	Unique	Repeated	Total
Bell, Jennifer A. H. (2015) [44]	13	38	51
Biedrzycki, Barbara A. (2010) [45]	12	4	16
Cox, K. (2003) [46]	19	16	35
Detoc, M. (2017) [47]	15	2	17
Dhalla, Shayesta (2011a) [48]	17	36	53
Dhalla, Shayesta (2011b) [49]	7	28	35
Ellis, Peter (2000) [50]	9	13	22
Fayter, Debra (2007) [51]	17	20	37
Forcina, Victoria (2018) [52]	6	0	6
Ford, Jean G. (2008) [24]	21	24	45
Grand, Melissa M. (2012) [53]	7	17	24
Gregersen, Trine A. (2019) [54]	3	8	11
Hurley-Rosenblatt, Arlene (2011) [55]	1	3	4
Limkakeng, Alexander (2013) [56]	1	4	5
Luschin, Gero (2012) [57]	6	3	9
Mills, Edward J. (2004) [58]	7	19	26
Mills, Edward J. (2006a) [59]	5	28	33
Mills, Edward J. (2006b) [60]	12	2	14
Nielsen, Zandra Engelbak (2019) [61]	3	6	9
Rivers, Desiree (2013) [62]	23	8	31
Ross, Sue (1999) [63]	43	19	62
Salman, Ali (2016) [64]	10	8	18
Schmotzer, Geri L. (2012) [65]	5	14	19
Shah, Jatin Y. (2010) [66]	3	4	7
Swanson, G. Marie (1995) [67]	37	2	39
Todd, Anne M.H. (2009) [68]	4	7	11
Tournoux, Caroline (2006) [69]	9	27	36
Townsley, Carol A. (2005) [70]	3	6	9
Walsh, Elaine (2016) [71]	42	19	61
White, Clare (2010) [72]	5	3	8
Grand Total:	365	388	753

Note. Unique primary studies are those included in only one review. Repeated primary studies are those included in more than one review.

As previously indicated, a total of 860 primary studies were covered in the reviews included in this overview. The total in Table 8 of 753 is lower than 860 because some of the reviews included in this overview study had primary studies that were irrelevant to the overview question and their corresponding data were excluded. For example, primary studies covering barriers or facilitators to the enrollment of physicians in clinical trials were outside the scope of this overview and were excluded. This kind of exclusion of primary studies that are outside the overview's scope is contemplated and recommended in the literature on overviews of reviews methodology [34]. Of the 753 primary studies covered in the overview, 511 were distinct publications and the rest were repetitions.

To better analyze the overlapping of primary studies in overviews, the literature recommends calculating and interpreting the Corrected Covered Area (CCA) [34]. The computation and interpretation of CCA is described in Pieper at al [33]. The computed CCA for this overview study is 0.02, or 2 %. According to the interpretation proposed in Pieper at al. it is a slight overlap. Out of the total possible combinations of primary studies (511) and reviews (30), only 242 repetitions of the primary studies were present (in the Pieper at al. computation, 511 is also subtracted from the denominator).

4. Discussion

4.1. Summary of Main Results

The objective of this overview study was to systematically summarize the literature on subject recruitment to clinical trials to better understand the factors that affect the participation decision. The barriers, facilitators, and other factors reported in the included reviews were thematically synthesized and a thematic framework was developed and presented in Table 5, Table 6, Table 7, and A1.

The thematic framework presents a complete and somewhat detailed picture of the factors that according to the research literature examined affect the participation of subjects in clinical trials. The factors are diverse and have to do with the trial, the potential participant, the clinical trial information, other persons or entities that influence or benefit, and other costs and benefits of participation. Most of the factors in the thematic framework come as no surprise because they have been investigated and discussed in the literature, in some cases quite extensively. Some of the factors, however, have received less attention, and some may even represent new topics for investigation. For instance, the theme Dispositions Including Willingness, Preferences, and Dislikes, if not new, is certainly less discussed, and represents an important and interesting window into why subjects accept or decline an invitation to participate – it is also, of all the factors About the Potential Participant, the one mentioned the most in the included reviews.

As evident in Table 6, most of the factors operate mainly as a barrier or as a facilitator. All the factors About the Trial operate mainly as barriers, as do the factors About Information and several of the factors About the Potential Participant. The factors About the Trial are of special interest because they have the highest level of occurrence in the included reviews, as attested by the number of barriers, facilitators, and other factors extracted. In a different direction, two of the factors About Others operate mainly as facilitators, while the other factor in this category, Influence from Others, can be as much a barrier as a facilitator.

Regarding the medical subjects, Table 7, the results show that most factors operate across subjects. One notable and somewhat expected exception is Hope, which is a factor in cancer clinical trials but not in HIV and vaccines trials where most participants are healthy volunteers rather than patients.

Another important observation about the factors in the thematic framework is that some may be modified, by way of interventions or changes in trial design, to be less of a barrier or more of a facilitator. Two of the factors in the About the Trial category, Characteristics of Trial – Medical and Procedural and Obstacles to Participation, may be modified by changing the trial design. Targeting these two factors is important because they are two of the factors most mentioned in the included reviews; and changing the trial design to facilitate recruitment has been advocated in the literature, even by FDA [[105], [106], [107]]. The other factors in this category are attitudes, which are more difficult to change; and fears, concerns, and perceived risks that may be modifiable to a certain extent by, for example, providing better information [108]. The two factors in the About Information category are also modifiable by providing better information and by communicating that information effectively. Most of the factors About the Potential Participant, however, are either difficult to change or cannot be changed. Still, some factors in this category, like Hope and Cultural Background, may be targeted by interventions to increase or modify the factor's effect. Interventions can also be employed to increase or modify the effect of the factors in the About Others category.

Some of the factor modifications alluded above represent important areas of opportunity. For example, many potential participants fail to enroll because of obstacles to participation. These obstacles are the same from trial to trial and include expected practical difficulties like transportation, parking, lunch money, childcare issues, and working hours conflicts. If obstacles are removed, these potential participants will agree to participate. The two factors in the About Information category are also important areas of opportunity that may not be too difficult to address. Many who fail to enroll complain about lack of information about the clinical trial – questions and concerns that were not addressed. Others do not enroll because of misconceptions and misunderstandings that tend to repeat from trial to trial and can be anticipated. More attention to these information related difficulties in the recruitment process will also contribute to additional enrollments. Another critical area of opportunity, more difficult to address but with a higher potential payoff, is related to the Characteristics of Trial – Medical and Procedural factor. There are elements of the trial design that can be modified to reduce participation barriers. As indicated, this has been addressed in the literature and include things like broadening eligibility criteria, using electronic communication to reduce site visits, having more flexible hours for the participants’ visits, reducing the number of visits required, using mobile medical professionals to visit participants at their location, reducing the trial duration, and doing less intensive testing [[105], [106], [107]]. Virtual Clinical Trials (VCT), where most activities are performed remotely using digital technologies and resources closer to the participants, are a good example of how clinical trials may be modified to facilitate the recruitment and retention of participants [[109], [110], [111]]. As an alternative to traditional trial designs the VCT approach has been getting increased attention, accelerated in part by the COVID-19 Pandemic, by the successful application of the concept in several trials, and by the realization that the current approach to clinical trials design, centered on the investigators rather than on the participants, is not sustainable [[109], [110], [111], [112]].

4.2. Overall completeness of evidence

A total of 30 reviews were included in this overview study, representing 511 distinct primary studies, and including 881 barriers, facilitators, and unclassified factors. These 30 reviews cover the main medical subjects and demographics covered in the primary studies of this literature.

Also, as indicated, a top-up search was conducted right before submitting the manuscript for publication to identify newly published systematic reviews not included in this overview study. After examining the newly identified systematic reviews it was concluded that the results reported in these reviews are consistent with the factor data extracted and synthesized in this overview study and do not change the thematic framework and conclusions reported.

4.3. Quality of evidence

A few of the reviews included complied with almost all the quality elements assessed. Most of the reviews included, however, complied with some but not all of the quality elements evaluated. As was noted, about a third of the reviews included were published before the development and publication of the quality assessment instruments used in this overview study. As was also noted, most of the reviews included complied with the quality elements that have received more attention in the systematic review methodology literature but failed to comply with the quality elements less emphasized in that literature. Also important to keep in mind, and also previously indicated, is that the lack of compliance with all quality elements is a problem that affects systematic reviews in general [104].

Another important observation is that if the few reviews with the lowest quality compliance were excluded, there will still be significant support for the themes included in the thematic framework and for the conclusions of this overview study, as evidenced in Table A1 in the supplementary files.

The diversity in review quality in this overview study is an important result that gives the reader a picture of the state of the quality of the reviews in the literature examined.

4.4. Potential biases in the overview process

One potential bias is the possibility that the search activities conducted failed to identify compliant reviews that may change the results and conclusions reported in this overview study. This is a potential bias in overview studies and is in part a result of the limitations and difficulties in the search activities for systematic reviews [25].

Another potential bias in the overview process results because some of the reviews included did not examine the quality of the primary studies. Quality problems in the primary studies may have biased the results presented in the reviews and, consequently, the results reported in this overview study.

4.5. Agreements and disagreements with other primary studies and/or reviews

The results reported in this overview study do not disagree with the results reported in the literature reviewed. On the contrary, because this study thematically synthesized that literature it agrees with what is reported in terms of barriers and facilitators that affect clinical trial participation. To enhance that level of agreement, as previously indicated, the labels used in the thematic framework incorporate, as much as possible, the terminology used in the literature reviewed. What is different in this overview study is that it integrates into a single consolidated presentation what other primary studies and reviews covered only partially.

Authors’ conclusions

The literature examined reports hundreds of barriers, facilitators, and other factors that affect the participation of subjects in clinical trials. In this overview study these factors were synthesized into a thematic framework of 20 themes, facilitating the analysis and application of the findings and recommendations of this extensive literature.

Implications for practice

Based on an analysis of the thematic framework, the discussion section identifies a number of opportunities for interventions and trial design changes that may decrease the barriers and increase the facilitators to clinical trial participation. The opportunities identified are only some of the actions that may be taken to address the 20 factors in the thematic framework. All of the 20 factors represent areas of opportunity for the development of interventions, and all shall be considered when defining actions for increasing clinical trial recruitment. One benefit of this overview study is the integration of results from hundreds of publications from the literature examined, giving the reader a holistic and complete view of the factors to address and of the areas of opportunity to consider when aiming to increase the participation of subjects in clinical trials.

Implications for research

The literature on subject recruitment to clinical trials examined in this overview study spans several decades and includes hundreds of primary studies and a good number of reviews. Among the primary studies, the literature includes qualitative studies that identify and explore themes of interest, quantitative and mixed methods studies that test some of the relevant variables, longitudinal studies, and studies that present analyses or commentaries on the subjects of interest. The empirical studies have covered many of the medical subjects pertinent to clinical trials and most of the relevant demographics, including specific countries and even specific states within the United States. One interesting observation about the empirical studies is that they generate results that are somewhat similar in terms of the themes and variables of interest identified, even when considering very different medical subjects and demographics. This last point is reflected in the reviews included in this overview study and is evidenced by how wide-ranging among the included reviews is the support for the 20 themes of the thematic framework (see Table A1 in the supplementary materials). One related observation when reflecting on this literature is that by now most of the themes and variables of interest may have already been identified, as may have several of the relationships among those themes and variables. What we have found to be less common in this literature are theoretical propositions identifying constructs, moderators, mediators, and the relationships between them. There have been some contributions to this effect, including the proposition of models or conceptual frameworks [24,113,114]. However, these models or conceptual frameworks seem to be used almost exclusively by the original publication or authors and are not employed in any manner by other studies. The literature on subject recruitment to clinical trials will benefit from additional theoretical work. Identifying the relevant constructs will make possible the testing of relationships and effects in order to learn about how the constructs operate, which have a stronger effect, and how are those effects moderated and mediated. These results, in turn, will make possible the design of more effective interventions and the measurement and assessment of the interventions’ effects [115].

Of all the notable observations that resulted from this overview study, one that will benefit from additional research is the one about the differences between patient and healthy-volunteer participants. These two groups have very different motivations that translate into different considerations when deciding about participation. Some studies have covered this line of inquiry, but questions remain about how these two different decision processes operate [116,117].

Finally, we conclude this section by presenting a couple of methodological implications for systematic reviews. Given the results of the quality assessment conducted in this overview study, one implication for future systematic reviews conducted in this literature is the need to follow the recommendations of quality instruments like AMSTAR [26]. Also, to facilitate the search activities, the title and abstract of these future systematic reviews should include the words systematic review.

Contributions of the authors

Edgardo Rodríguez-Torres: Conceptualization, Methodology, Software, Verification, Formal Analysis, Investigation, Data Curation, Writing - Original Draft, Visualization, Supervision, Project Administration. Margarita M. González-Pérez: Verification, Formal Analysis, Investigation, Data Curation, Writing – Review. Clemente Díaz-Pérez: Conceptualization, Formal Analysis, Investigation, Resources, Writing – Review.

Sources of support

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article: