Increasing Racial and Ethnic Equity, Diversity, and Inclusion in Cancer Treatment Trials: Evaluation of an ASCO-Association of Community Cancer Centers Site Self-Assessment

Abstract

Clinical trial participants do not reflect the racial and ethnic diversity of people with cancer. ASCO and the Association of Community Cancer Centers collaborated on a quality improvement study to enhance racial and ethnic equity, diversity, and inclusion (EDI) in cancer clinical trials. The groups conducted a pilot study to examine the feasibility, utility, and face validity of a two-part clinical trial site self-assessment to enable diverse types of research sites in the United States to (1) review internal data to assess racial and ethnic disparities in screening and enrollment and (2) review their policies, programs, procedures to identify opportunities and strategies to improve EDI. Overall, 81% of 62 participating sites were satisfied with the assessment; 82% identified opportunities for improvement; and 63% identified specific strategies and 74% thought the assessment had potential to help their site increase EDI. The assessment increased awareness about performance (82%) and helped identify specific strategies (63%) to increase EDI in trials. Although most sites (65%) were able to provide some data on the number of patients that consented, only two sites were able to provide all requested trial screening, offering, and enrollment data by race and ethnicity. Documenting and evaluating such data are critical steps toward improving EDI and are key to identifying and addressing disparities more broadly. ASCO and Association of Community Cancer Centers will partner with sites to better understand their processes and the feasibility of collecting screening, offering, and enrollment data in systematic and automated ways.

BACKGROUND

Clinical trials are vital for advancing cancer discoveries, yet fewer than 5% of patients with cancer participate.¹ Racial/ethnic diversity among participants is important for generalizability of study results.^2,3 Participants are much less diverse than the population of people with cancer,⁴ and patients who are Black or Latinx remain consistently underrepresented.^1-3,5,6

Although barriers to trial participation occur at multiple patient, provider, payer, organizational, and system levels, investigators failing to consistently offer trials to patients remains a critical barrier.⁷ A recent systematic review and meta-analysis reported that 55% of patients eligible and offered trial participation accept with similar acceptance rates between Black and White patients.⁸ Low participation rates of underrepresented racial/ethnic populations are further compounded by low reporting of race/ethnicity data.³ A 2022 analysis of 2 decades of US-based trials reported that fewer than 44% report any race/ethnicity data⁹—demonstrating the pressing need to address this issue at the sponsor level, as well.

Several initiatives have developed tools to address aspects of recruiting patients from underrepresented racial/ethnic populations for trials.^10-16 Such tools are often specific to a therapeutic area or type of research program¹⁷; address patient-based barriers only¹⁸; and/or are designed to raise awareness and educate patients, providers, and/or communities.^12,19-22 To our knowledge, existing initiatives do not provide tools that enable different types of trial sites to evaluate their programs, policies, and procedures for equity, diversity, and inclusion (EDI) and/or assess their performance in EDI participation metrics collection, evaluation, and monitoring along the continuum of clinical trial screening, offering, and enrolling patients on trials.

ASCO and Association of Community Cancer Centers (ACCC) collaborated on a multipart initiative to improve trial site performance regarding enrolling patients from underrepresented racial/ethnic populations and enhancing diversity among clinical trial participants.^23,24 The aim of the ASCO-ACCC Site Self-Assessment Pilot Study was to evaluate the feasibility, utility, and face validity of the Assessment across different types of cancer clinical trial sites.

METHODS

ASCO-ACCC Site Self-Assessment Development

The Assessment was developed on the basis of quality improvement (QI)²⁵ principles with a goal for sites to ultimately employ a rapid cycle Plan-Do-Study-Act²⁶ strategy. A research design team, including ASCO and ACCC members, cancer equity experts, statisticians, and research methodologists developed the methodology for the pilot study and evaluation of the Assessment. The study was overseen by a working group of the ASCO-ACCC Steering Group and informed by the ASCO-ACCC Patient Partners Advisory Group, expert consultants, a previous QI project in oncology,²⁷ and a landscape analysis that identified factors that influence performance outcomes related to trial enrollment of historically underrepresented racial/ethnic populations. The Assessment was designed to help clinical trial sites record the total number of patients, by race/ethnicity, who were screened for, offered, and enrolled in clinical trials. It was also designed to help sites complete step 1 (plan) of the rapid cycle Plan-Do-Study-Act by facilitating an internal assessment to gain insights and identify strategies for improving EDI across the trial screening and enrollment continuum.

Outcomes

The Assessment was evaluated for its utility (ie, its effectiveness to measure performance, detect variability, accurately discriminate between low- and high-performing sites, and identify opportunities and strategies for improvement), feasibility (ie, whether research sites could and would use the Assessment outside of the study, across different domains, and a variety of site types), and face validity (ie, appropriateness and comprehensiveness of the Assessment components to improve EDI in enrollment for underrepresented populations).

Study Population

ASCO-ACCC issued an open call to sites to apply to participate in a Pilot Project that included two pilot studies: (1) this Assessment pilot study and (2) an Implicit Bias Training Program pilot study. Eligible organizations were based in the United States and provided statements of interest and support to increasing EDI in trial enrollment related to patients from underrepresented racial/ethnic populations. Seventy-five sites completed applications, and all were selected for the pilot studies: 65 assigned to the Assessment and 50 to the Training Program, with 40 assigned to both arms.

For each site, a primary point of contact was responsible for obtaining insights from team members to complete the Assessment and coordinating data collection and entry. Sites received a modest stipend for study completion. The Pilot Project protocol was reviewed by WIRB-Copernicus Group institutional review board and deemed research exempt.

Data Collection

Sites provided data (via REDCap) about site and community characteristics and completed the Assessment, which included two components (noted below). Additionally, sites completed feedback surveys immediately after completion of each section and an overall feedback survey within one week after they completed the Assessment (via REDCap). The focus of the pilot study was on patients who were Black and/or Hispanic/Latinx.

Part 1: Performance assessment of EDI in clinical trials.

The performance assessment (Fig 1) enabled sites to identify deficiencies across the trial screening and enrollment continuum. Sites were asked to enter data on the proportion of newly diagnosed patients who were (1) screened for trials (ie, general screen for suitable trials, screen for a specific trial, and eligibility determination for a specific trial), (2) offered trials (onsite and/or offsite), and (3) enrolled into trials. For each step along the continuum, sites were asked to enter aggregate data for newly diagnosed patients by select races/ethnicities (Black, Hispanic/Latinx, and White) and for their overall patient population. It was anticipated that the COVID-19 pandemic would have affected screening and enrollment; therefore, sites were asked to provide aggregate 2019 and 2020 data.

FIG 1.

Components of ASCO-Association of Community Cancer Centers Site Self-Assessment for EDI in Clinical Trials (Pilot Version). ^aParticipating sites were asked to provide the number of patients for each item for patients overall, and those who were identified as Black, Hispanic/Latinx, or White. ^bIt was not specified whether data should be for patients enrolled in trials onsite and/or offsite; sites may not have access to whether patients consented to an offsite trial. EDI, equity, diversity, and inclusion; SOP, standard operating procedure.

Part 2: Assessment of opportunities to improve EDI performance.

Part 2 (Fig 1) was designed to assess potential causes of low enrollment of patients from racial/ethnic populations and identify opportunities for improvement. It assessed factors that affect clinical trial screening, offering, and enrollment, including institutional and/or site policies, programs, and procedures; workforce; and availability of suitable trials. It included seven domains (described in Fig 1), including whether community populations have access to site, site mission and leadership, availability of trials, screening for trials, offering trials, supporting patient participation in trials, and supporting patient retention in trials. Thirty-six questions explored the extent to which respondents agreed their site had a particular strategy in place that addressed the needs of patients who are Black and/or Hispanic/Latinx (Likert scale response options were strongly disagree, somewhat disagree, neutral, somewhat agree, and strongly agree). Respondents were asked to provide specific examples of policies, programs, and procedures in place to document and clarify their responses.

Feedback surveys.

Survey questions asked about satisfaction (eg, ease of navigation, instructions, and accuracy of findings), time spent, identification of new insights (eg, gaps and opportunities for improvement, specific strategies for improvement), desire to use or recommend the Assessment, perceived barriers/limitations, and whether the Assessment included components that would enhance enrollment of patients from Black and/or Hispanic/Latinx populations. Responses generally consisted of five-point Likert scales (ie, satisfaction, agreement, or likelihood) and space for comments and suggested revisions.

Data Analysis

Feasibility and utility were assessed through feedback survey questions about satisfaction (eg, overall impressions, ease of navigation, instructions, accuracy of findings, and time to complete), identification of new insights (eg, gaps and opportunities for improvement and specific strategies for improvement), desire to use or recommend use of the Assessment in the future, and perceived barriers and limitations. Additionally, the availability of data was considered in the evaluation of feasibility and utility of the Assessment. Face validity was assessed with survey questions about whether the Assessment included the right components to lead to better enrollment of patients from historically underrepresented racial/ethnic populations. Descriptive analyses, including frequency distributions and cross-tabulations, were used to summarize responses. Likert scale responses were coded as ordinal (1-5), and means were estimated with 95% CIs. Stacked bar charts were used to show the racial/ethnic diversity of patient populations across sites. Open-text feedback was summarized through an informal qualitative analysis to identify common themes regarding opportunities to improve the Assessment.

RESULTS

Participating Site Characteristics

Sixty-two sites completed the study (95% response rate), representing a range of settings and practice types (Table 1). Sites had medians of 46 clinical investigators (range, 1-292), and 165 patients enrolled in trials annually (range, 2-3,436), and two-thirds (68%) had more than 20 years research experience. Half of sites had ≥ 31 National Cancer Institute–funded (53%) or industry-funded (50%) trials open for accrual.

TABLE 1.

Participating Research Site Characteristics

The Data Supplement (online only) summarizes the racial/ethnic diversity of patient populations reported. The Data Supplement reports number of sites with screening and enrollment measures and policies in place, including setting goals for screening (55%) and enrollment (73%), systematic screening process (53%), and collection of objective performance data (61%).

Overall Impressions of the Site Self-Assessment

Most sites (81%) were satisfied with using the Assessment. A majority (82%) reported that the Assessment identified opportunities for improvements, 63% were able to use it to identify a new strategy or change, and 74% thought it had potential to help increase EDI in trials (Table 2). Most sites reported they would use the Assessment if publicly available and would recommend its use to other research sites (69% and 74%, respectively).

TABLE 2.

Site Feedback and Impressions About the Site Self-Assessment (Pilot Version)

Part 1: Performance Assessment of EDI in Clinical Trials

Most sites (63%) reported satisfaction with Part 1, and 63% agreed that collecting performance measure data would lead to more diverse enrollment (Table 2). The median time from beginning to completion of this portion was 120 minutes (range, 7-870).

Most sites were unable to provide requested data on trial screening, offering, and enrollment by race/ethnicity (Data Supplement). Only two sites (3%) were able to provide all the data requested at each step in the Assessment. Sites that collected data did not do so routinely, and most (79%) had challenges with data collection and had to compile through multiple sources and/or manual extraction (40%-100% across steps). Sites with missing data did not collect data at all (36%-64% across steps), did not collect data in a systematic way (0%-29% across steps), or stated it would be too burdensome to manually review charts to extract data (12%-29% across steps).²⁸ Some sites with available data reported that their data were incompatible with the Assessment's requirements (eg, their screening and enrollment processes, terms, and/or definitions were different). Patterns of data availability were similar across type and size of site (Data Supplement) and for both 2019 and 2020.

Part 2: Assessment of Opportunities to Improve EDI Performance

Most sites (76%) were satisfied with Part 2, and 63% agreed that collecting the data would lead to more diverse enrollment (Table 2). One-third (34%) reported challenges with data collection. The median time to complete this part of the Assessment was 120 minutes (range, 25-600).

Means were similar in hospital/health system and academic practices across domains. Means for independent practices differed for some questions, but sample size was small, and we did not test for differences. Practices tended to disagree that they have an organizational policy to just ask eligible patients to participate, site-specific goals/strategies to increase participation, prescreening standard operating procedures (SOPs) to match patients to trials, workforce training policies for prescreening patients, accountability systems for matching, and consenting patients to trials. Approximately equal numbers of practices agreed/disagreed that they have sufficient workforce to prescreen every patient for trials and that implicit bias training is required for team members. Data Supplement demonstrates mean responses (on the basis of the ordinal values of one to five for the five response options, with 95% CIs) for questions in each domain, by practice type. Across all domains, there were few questions that generally showed high levels of agreement.

DISCUSSION

The frequency of reporting race/ethnicity in trial manuscripts and reports and proportional racial and/or ethnic representation in trials remain low.³ Documenting data about the number and characteristics of patients screened for, offered, and enrolled in trials is an important step toward evaluating and improving EDI in trial participation. This pilot study examined the feasibility, utility, and face validity of a site self-assessment to enable diverse types of cancer trial sites to review their policies, programs, procedures, and their internal data on trial screening, offering, and enrollment with the goal of enhancing EDI in trials. Participating sites responded favorably to the feasibility, utility, and face validity of the Assessment and agreed that it provided new insights and increased awareness about their performance. Participating sites reported they would recommend the Assessment to other research sites if it were publicly available.

Consistent with current estimations of the low frequency of reporting on the number of participants by race/ethnicity,^3,9 most sites were unable to provide the Part 1 enrollment performance data because of access constraints, compatibility challenges, and/or, lack of systematic data collection. Participants provided contextualizing comments about challenges in access to or compatibility of data such as, “some data collected does not currently include race/ethnicity,” or “the order of the steps doesn't correspond to the way we operate at our cancer center.” The data were also burdensome to compile manually for some sites and required a considerable time commitment, suggesting a prospective and systematic data collection approach is needed to avoid manual, retrospective chart review. Academic or larger sites were marginally more likely than private practices or hospital/health system practices to have access to data, whereas smaller sites were less likely to have data for any steps in the enrollment process (Data Supplement).

Participants generally agreed that the Assessment enabled them to identify specific strategies to increase racial/ethnic diversity in trials. Sites reported a range of existing strategies for improvement through Part 2 (Assessment of Opportunities to Improve EDI Performance), such as developing internal diversity goals and metrics, and investing in dedicated staff and programs to ensure diversity. Participants commented on the value of the internal assessment, such as “It pushed our center to stop and carefully review current protocols…we learned more about our strengths and weaknesses.” The Assessment also enabled sites to identify areas for improvement including the need for SOPs for data collection on screening, enrollment, and race/ethnicity and automating and centralizing collection of such data and metrics. Part 2 also enabled sites to identify their own need to develop targeted strategies for engagement with Black and/or Hispanic/Latinx communities and to enhance their SOPs, policies, and processes to establish accountability for increasing racial/ethnic EDI in trials. Participants suggested a version of the Assessment could be completed annually as part of the Commission on Cancer clinical research standard or as a QI study for cancer programs.

There are a variety of reasons for underrepresentation of certain racial/ethnic populations in clinical trials. These factors are complex and may occur at multiple patient, provider, payer, sponsor, and organizational, and system levels. It is important to consider these complexities, across stakeholder groups, to understand and address barriers to EDI in clinical trials.²³ Institutional leaders promoting EDI policies (including screening every patient for a trial) may have the greatest impact.^23,29

Interventions to address barriers to EDI in trial participation are required at multiple levels. Such interventions include resources to enhance community engagement, training to reduce provider implicit bias, and strategies to address site-based barriers such as protocols, policies, SOPs, and processes for collecting, reviewing, and documenting trial screening, invitation, and enrollment metrics.^7,23,30 Collecting standardized patient demographic data is a prerequisite to developing tailored interventions to address site-based barriers to enrollment¹⁶ and yet these standards are lacking (not only race/ethnicity but also sexual orientation and gender identity).³¹ Although tools and resources have been developed to address EDI across the trial enrollment continuum,^21,32 few interventions have been designed to enable different types of trial sites and programs to collect and review their own internal data as a foundation for identifying and addressing deficits in the screening and enrollment process.^33,34 Part 2 of the Assessment was successful in this regard and was able to detect variability between site responses, items, and domains.

The challenges in data collection present limitations to these findings. Only eight private practices participated in the study, which limits generalizability of findings to this setting. Because most sites were not able to provide data on screening, offering, and enrolling patients, we were unable to perform intended analyses on feasibility, utility, and face validity for Part 1 (Performance Assessment of EDI in Clinical Trials). Given the lack of data for both 2019 and 2020 and feedback from participants regarding data availability, it is reasonable to conclude that the COVID-19 pandemic was not a key factor influencing the sites' ability to obtain data. The overall lack of available data strongly points to a need for systematic and automated ways to capture data at each step in the pathway to enrollment. There is high variability across sites regarding processes for screening, offering, and enrolling; how data are (or are not) collected; and terms and definitions. Future research will involve consulting with the study sites that captured data along the clinical trial enrollment continuum to learn and share best practices. Establishing the reliability and validity of these outcomes as measures for EDI in clinical trials is also an important next step.

ASCO and ACCC made a revised version of the Assessment publicly available as a QI tool to help sites identify and address opportunities for improvement.³⁵ This new assessment will help to advance the ASCO-ACCC Research Statement's recommendation to screen every patient for clinical trial participation as part of high-quality cancer care.²³ Although psychometric properties have not been established, the Assessment provides an incremental step for sites to help improve EDI in trials.

In conclusion, an important step toward assessing and enhancing EDI in trial participation is documenting data about which patients are screened for, offered, and enrolled into clinical trials. Without routine data collection, research sites are unable to evaluate and monitor whether their patients have equitable access to trials, assess barriers, or establish benchmarks and measure effectiveness of strategies to address disparities. The ASCO-ACCC Site Self-Assessment provided new insights and increased awareness about site performance, and enabled sites to identify specific strategies to increase racial/ethnic diversity in oncology trials. ASCO and ACCC are conducting additional research with pilot study sites to better understand the feasibility of collecting clinical trial screening and enrollment data in systematic and automated ways, such as through electronic health record systems. ASCO and ACCC remain committed to helping oncology sites improve their processes and achieving EDI in clinical trials.

AUTHOR CONTRIBUTIONS

Conception and design: Carmen Guerra, Alice Pressman, Patricia Hurley, Elizabeth Garrett-Mayer, Suanna S. Bruinooge, Leigh Boehmer, Lea Ann Bernick, Marjory Charlot, Jennie Crews, Lola Fashoyin-Aje, Worta McCaskill-Stevens, Janette Merrill, Greg Nowakowski, Manali I. Patel, Amelie Ramirez, Randall A. Oyer, Lori J. Pierce

Administrative support: Patricia Hurley, Jen Hanley Williams

Provision of study materials or patients: Patricia Hurley, Jen Hanley Williams, Melinda Kaltenbaugh

Collection and assembly of data: Patricia Hurley, Melinda Kaltenbaugh, Jen Hanley Williams

Data analysis and interpretation: All Authors

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Increasing Racial and Ethnic Equity, Diversity, and Inclusion in Cancer Treatment Trials: Evaluation of an ASCO-Association of Community Cancer Centers Site Self-Assessment

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/op/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).