Count me in: using a patient portal to minimize implicit bias in clinical research recruitment

Abstract

Objective

Determine whether women and men differ in volunteering to join a Research Recruitment Registry when invited to participate via an electronic patient portal without human bias.

Materials and Methods

Under-representation of women and other demographic groups in clinical research studies could be due either to invitation bias (explicit or implicit) during screening and recruitment or by lower rates of deciding to participate when offered. By making an invitation to participate in a Research Recruitment Registry available to all patients accessing our patient portal, regardless of demographics, we sought to remove implicit bias in offering participation and thus independently assess agreement rates.

Results

Women were represented in the Research Recruitment Registry slightly more than their proportion of all portal users (n = 194 775). Controlling for age, race, ethnicity, portal use, chronic disease burden, and other questionnaire use, women were statistically more likely to agree to join the Registry than men (odds ratio 1.17, 95% CI, 1.12–1.21). In contrast, Black males, Hispanics (of both sexes), and particularly Asians (both sexes) had low participation-to-population ratios; this under-representation persisted in the multivariable regression model.

Discussion

This supports the view that historical under-representation of women in clinical studies is likely due, at least in part, to implicit bias in offering participation. Distinguishing the mechanism for under-representation could help in designing strategies to improve study representation, leading to more effective evidence-based recommendations.

Conclusion

Patient portals offer an attractive option for minimizing bias and encouraging broader, more representative participation in clinical research.

INTRODUCTION

Overview

When someone’s absence at an event strikes you, you might wonder “did they decide not to attend or were they not invited?” Women are historically under-represented in clinical research studies relative to their percentage of the population and even more so relative to their percentage of interactions with the health care system. Why is this? Do women decide not to participate in clinical research more frequently than men, or are they invited to participate less frequently through explicit or implicit bias?^1,2 Under-representation of women in clinical research affects the accuracy of clinical guideline development and evidence-based care decisions.^3,4 In this study, we set out to determine whether women accept offers to participate in research differently than men thereby contributing to the observed disparity in clinical research enrollment.

Under-representation of women in clinical studies

When examining enrollment in clinical trials by sex, under-representation of women is frequently found. In 2000, Harris and Douglas reviewed enrollment by sex in studies funded by the National Heart, Lung, and Blood Institute (NHLBI) between 1965 and 1998.⁵ Excluding single-sex studies, women constituted 38% of enrolled patients with no significant improvement over time. A review of 135 randomized clinical trials cited by the 2007 American Heart Association Guidelines for Prevention of CVD in Women found that the percentage of women enrolled increased from 18% in 1970 to 34% in 2006 but still far short of the population distribution.⁶ A third review of 89 studies performed between 2000 and 2007 for Food and Drug Administration (FDA) approval of high-risk cardiovascular devices found that only 33% of participants were women.⁷

A 2018 review of studies used for FDA approval between 2005 and 2015 calculated female study participation-to-population ratios (PPR) for several cardiovascular conditions.⁸ Women were represented nearly equally in studies of hypertension (PPR 0.9) and atrial fibrillation (PPR 0.8–1.1), over-represented in studies of pulmonary hypertension (PPR 1.4), but still significantly underrepresented in studies of congestive heart failure and coronary artery disease (PPR 0.6). Outside of cardiovascular disease, other studies generally have shown under-representation of women in randomized clinical trials (eg, in the fields of HIV care, surgery, and cancer).^9–11

Under-representation of women in clinical trials matters to clinicians and patients as they seek to apply the results of clinical trials in individual patient treatment decisions.¹² Sex proves a powerful biological variable affecting patterns of disease and response to treatment in a wide range of clinical situations.^3,13–17 For instance, in the aforementioned cardiovascular device study, 12 of 47 (26%) studies found an effect of female versus male sex in the safety and/or efficacy of the device. If women are under-represented, the power of a study to detect sex-modified outcome benefits or risks is correspondingly reduced.

Potential causes of under-representation of women in clinical studies

Why, then, are women commonly under-represented in clinical trials? A possible cause is that women are invited to participate less frequently than men due to bias—either explicit or implicit. Explicit bias operates when a conscious attitude based on immaterial characteristics (such as race or sex) directly influences one’s behavior or decisions relating to a person or group. In implicit bias, on the other hand, the attitude is subconscious but still affects one’s behavior. With implicit bias then, a disconnect exists between a person’s conscious thoughts and their actions.

Study enrollment bias could be explicit in the form of study inclusion/exclusion criteria. Exclusion of women from randomized clinical trials was at one time commonplace. In 1994, new NIH guidelines on including women and minorities as subjects in clinical research went into effect as required by the 1993 NIH Revitalization Act. Since then, although progress has been made, much remains to be done.¹⁸ Until recently, explicit exclusion of pregnant women still often occurred. Heyrana et al. note these women should not be considered “vulnerable” (implying lack of self-determination) but instead as “scientifically complex” with full cognitive capability to make informed decisions for themselves.¹⁹

Implicit bias of study investigators leading to lower female enrollment rates is more difficult to discern but could also explain differences in observed rates of participation by sex, race, or other categories.^20,21 When looked for, implicit bias has been found to affect human activities including hiring and promotion,²² housing opportunities,²³ prison sentencing,²⁴ and health care treatment.¹ In health care, implicit (unconscious) bias among clinicians has been shown to affect rates of diagnosing illness, testing, and offering medication or surgical treatment.^20,21 Accordingly, the possibility of implicit bias in research participant selection seems plausible.²⁵

However, another possible cause of the observed lower female enrollment rate is that women decline to participate in clinical trials more commonly than men even when offered equal opportunity to do so. To paraphrase Scott, in the absence of explicit exclusion, the available data suggest that either women are less likely than men to be considered for screening in trials (implicit bias), and/or they are less likely than men to consider participation in trials.⁸ Existing studies to shed light on this distinction are sparse.

The present study

To empirically evaluate these 2 possible causes in this study, we sought to eliminate the first in order to provide a pure evaluation of the second. Specifically, our Health System set out to provide all patients the opportunity to volunteer for a clinical research registry. Agreeing to be on this registry allows future contact about studies for which they may qualify. We provided this invitation through an online questionnaire on the electronic health record (EHR) patient portal made available to all patients, which would prevent the opportunity for human implicit bias in offering participation. Patient portals, a requirement of certified EHRs as part of the Health Information Technology for Economic and Clinical Health Act, are now widely available.²⁶ The possibility of leveraging patient portals for clinical research recruitment has been recognized, and early successes and lessons learned have recently been published.²⁷

Among patients presented with the questionnaire, we set out to compare the percentage of females vs males who volunteered to be contacted about clinical research controlling for level of patient portal use, demographic features (age, race, ethnicity), and degree of comorbidities.

MATERIALS AND METHODS

Study design, setting, time period, and participants

The study design is an observational cross-sectional study of patients who use the patient portal offered by the University of Texas Southwestern Health System (UT Southwestern). UT Southwestern is an academic medical center with 2 tertiary/quaternary care hospitals and a large ambulatory group practice with over 1000 faculty physicians and advanced practice providers.

In May 2016, UT Southwestern began presenting patients using its patient portal with the opportunity to volunteer for inclusion in a Research Recruitment Registry. This project, including the language used in the online invitation questionnaire, was approved by the UT Southwestern Institutional Review Board. The study period for this report covers the first 30 months of registry enrollment, from May 2016 through Oct 2018. An invitation to join the Research Recruitment Registry was placed on the initial home screen of the patient portal, so patients were included as “invited” if they had any portal activity during this 30-month study period. Since some patients may have first visited the portal during the final days of the study period, we included 2 months following the study period (Nov–Dec 2018) when evaluating for any response to the invitation questionnaire (though anyone first accessing portal during this time was not included in the study).

Initial study screening included patients 18 and older who had any active interaction with UT Southwestern Health System during the 30-month study period (n = 426 510 patients, see Figure 1). Active interaction was defined as any of the following: office visit, procedural or surgical encounter, hospital encounter (including ED and imaging encounters), telephone encounters, and/or patient portal activity. Study-eligible patients included those who were alive (not marked as deceased in the EHR) at the time of final data extraction, and who had sex demographic information in the EHR (418 053 patients, Box 3 in Figure 1). The final study population included active portal users defined as patients with any patient portal activity during the 30-month study period (194 775 patients, Box 5 in Figure 1).

Figure 1.

Study patient inclusion flow diagram.

Software

UT Southwestern employs Epic (Epic Systems) as its EHR, with a single instance for its hospitals and clinics. The EHR’s patient portal (MyChart) is offered to both ambulatory patients and inpatients and is accessible via a web-based interface and mobile device. The EHR software enables local sites to create and deploy patient-reported outcomes (PRO) questionnaires via the patient portal. Chronic disease registries (n = 104) have been configured within the EHR using a rapid-cycle method previously reported with patients assigned to registries by automated rules commonly based on SNOMED CT.^28,29 We used the EHR’s reporting database (Clarity) as the data source for extracting the study’s data sets, and R (version R-3.5.2) and R Studio for data analysis.³⁰

Procedures

Developing the research recruitment registry and portal questionnaire

The Volunteer Research Recruitment Registry project was initiated by the UT Southwestern Center for Translational Medicine, part of the Clinical and Translational Science Award network, with the goal of linking UT Southwestern patients with clinical research opportunities. Through a focus group of experienced research coordinators and with patient input, themes that are important to patients in deciding to participate in clinical research were elicited. Similar registries at other institutions were reviewed. With input from stakeholders, a patient consent form and communication language were drafted.

Health System approval was received to make enrollment available to all patients via a patient portal questionnaire. After reading the registry’s description, the patient could indicate their decision by answering a single question: “Do you agree to be added to the Research Recruitment Registry?” The response options were “I agree” or “Not at this time.” The patient’s most recent answer was stored as a discrete patient-level data element in the EHR. This data element could be applied as a search criterion when querying for potential study participants and was made available in i2b2 and the EHR’s self-service reporting tool (SlicerDicer) for use by investigators performing study feasibility analyses.

Real-time registry participant lists were made available employing the EHR’s chronic disease registry capability.²⁸ But in this case, registry inclusion was based on a patient’s answer to the registry agreement question rather than the presence of a specific chronic condition.

Presenting the opportunity to volunteer via the patient portal

Using the questionnaire, patients were presented the option to provide electronic consent to opt-in or opt-out of the registry at any time via the patient portal. The initial screen prominently featured a logo with the project slogan “Count me in” (Figure 2), hyperlinked to the Research Registry consent questionnaire. A Helpful Links section of the home page provided another hyperlink to the Research Registry consent. A central recruitment office was available to be contacted by telephone or email with any questions. The questionnaire was also available on the patient portal mobile app (Figure 3). Response via either the web-based questionnaire or the mobile questionnaire updated the same patient-specific data element.

Figure 2.

Patient portal home screen, with option to “Participate in clinical research” presented on initial login.

Figure 3.

From the Questionnaires activity in either the patient portal mobile app (shown) or web client, the option to volunteer to be added to the Research Recruitment Registry is offered.

Data management

Data sets were extracted from the EHR’s reporting database (Clarity) using structured query language (SQL) queries. During query development, a patient identifier (medical record number) was included for query validation and quality assurance purposes. Once the queries were validated as returning data elements from the correct fields in the EHR, all patient identifying information was removed from the SQL query, and the data sets extracted for study analyses contained no patient-identifying information. The data sets were stored in Excel and comma-separated value format and imported into R Studio for logistic regression analyses.³¹

Data analysis

Primary and secondary outcome variables

The primary outcome variable was Agreement to Be Included in the Volunteer Research Participant Registry, stored as 1 = True and 0 = False, which will also be referred to as Agreement.

A secondary outcome variable calculated for patient demographic subgroups was the PPR.⁸ This is calculated as the ratio of the percent of a subgroup participating in a study divided by the percentage of that subgroup in the comparison population. For instance, the PPR for females agreeing to participate in the registry = [(# of females agreeing) / (# total persons agreeing)] / [(# of females in study population) / (# total persons in study population)].

Independent variables

Independent variables used as factors in univariable and multivariable analyses included demographic characteristics and variables for potentially confounding factors such as intensity of patient portal use and presence of comorbid conditions.

Demographic variables were all derived from the EHR’s demographics section, and included:

1. Age at Study Onset. Defined as age as of first patient portal access if initial access occurred during the 30-month study period; for all patients with pre-existing portal access, defined as their age as of May 1, 2016.

2. Sex. Defined using the field “Sex” in the EHR’s demographic section, as self-reported by the patient. (Note: separate fields became available in the EHR during the study period for “Sex defined at birth” and “Gender Identity”. These currently have far fewer entries than the main “Sex” field – see Discussion section).

3. Race, using the standard options listed by the National Institutes of Health (NIH).³²

4. Ethnicity, also using the standard options listed by the NIH.

Intensity of patient portal use was defined primarily as a quantitative continuous variable for use in logistic regression analyses. The source of patient portal use data was a logging table with 1 row per patient portal access event. Some examples of patient portal access events include a patient reading or sending a secure message, viewing test results, or viewing their upcoming appointment details. Patient portal access events were not categorized; all were weighted equally.

• Study Months: Defined as the number of months inclusive between the month of the patient’s study onset date (previously defined as for Age at Study Onset) and the end of the study, with a possible range of 1–30.

• Monthly Portal Use Rate: Defined as the total number of patient portal access events divided by the number of Study Months. This variable was used in regression analyses.

• Monthly Portal Use Category: Defined as “Low”, “Medium”, or “High” based on cutoffs representing approximately the 33rd and 67th percentiles of Monthly Portal Use Rates. The cutoff values used were: Low <= 5.4, Medium > 5.4 and <= 23.5, and High > 23.5 (in units of Patient Portal Access Events per month). This variable was used in comparisons of patient demographics by Monthly Portal Use Category (see online supplemental material Appendix A.

• Ever Answered Patient Reported Outcome (PRO) Questionnaire: This patient-level True/False variable was stored as 1 = True if the patient had ever answered a PRO Questionnaire question and 0 = False if not. Because the Research Recruitment Registry questionnaire was accessed on the same screen as PRO Questionnaires, we considered that patients who had answered a PRO Questionnaire may be more likely to have also seen and responded to the Research Recruitment Questionnaire, and so we wanted to control for this in the multivariable model.

As a marker of the presence of comorbid conditions, we leveraged our use of Chronic Disease Registries within the EHR (see Methods) covering both common and specialized conditions.

• # Disease Registries: We defined this quantitative variable as a simple count of the number of EHR-based chronic disease registries to which the patient was currently assigned (as of the data extraction date). Patient registries for other purposes were excluded (eg, wellness registries for health maintenance and accountable care organization membership registries).

Study population characteristics

Demographics and other characteristics of the included study population (active portal users) were compared with the non-portal-using members of the overall eligible patient population (see Figure 1), with the NULL hypothesis of no difference. Comparisons of proportions were performed with a two-proportion z-test; comparisons of means of continuous variables were performed with Student’s t test.

Additionally, within the study population of portal users, demographic characteristics were determined for 3 subgroups based on portal usage rates during the study period (low, medium, high).

Research recruitment registry member characteristics

The age-sex distribution of those who agreed to participate in the Research Recruitment Registry was plotted (using Excel), along with the same distribution for the study population of all active patient portal users. Data for the group answering “I Agree” to be included in the Research Recruitment Registry was also binned into 10-year age bands for display in a histogram broken down by sex.

Logistic regression analyses were then performed, using Agreement as the binary outcome variable. Single-variable analyses were performed for Sex, Race, Ethnicity, and Age. Multivariable regression was performed also adding in Monthly Access Rate, Ever Answered PRO Questionnaire, and # Disease Registries. All regression analyses were performed using the following R functions and associated library packages:

• logistic regression: glm function (library: stats)

• collinearity assessment: vix function (library: car)

• odds ratio calculations: or_glm function (library: oddsratio)

Output was saved to an R Notebook with R code and statistical results knitted together in an HTML document (see online supplemental material Appendix B).³³

RESULTS

Study population characteristics

Characteristics of the study population (patients with any patient portal activity during the study period) are shown below compared with the remaining eligible population of active Health System patients without current patient portal use (Table 1). Also shown are characteristics of subgroups of the study population by portal use intensity.

Table 1.

Characteristics of the study population (active portal users) compared with non-portal users. The ratio of representation of a subgroup among active portal users vs among all eligible patients is shown as a Portal Representation Ratio

	Study Population: Active portal users (Box 5 in Figure 1)	Non-portal users (Box 4 in Figure 1)	P value (Study Population vs non-portal users)	Portal Representation Ratio (vs all eligible patients, Box 3)
Patients (n)	194 775	223 278	N/A	N/A
Average Age (yrs)	50.5 +/− 17	51.7 +/− 17	< .0001	N/A
Sex:
Female (n, %)	122 521	140 364	.79	1.00
	(62.9%)	(62.9%)	(NS)
Male (n, %)	72 254	82 914	.79	1.00
	(37.1%)	(37.1%)	(NS)
Race:
American Indian or Alaska Native	668	515	< .0001	1.21
	0.3%)	(0.2%)
Asian	11 354	6 721	< .0001	1.35
	(5.8%)	(3.0%)
Black or African American	19 976	35 308	< .0001	0.78
	(10.3%)	(15.8%)
White	127 756	108 026	< .0001	1.16
	(65.6%)	(48.4%)
Native Hawaiian or other Pacific Islander	265	245	.015	1.12
	(0.14%)	(0.11%)
Other	7795	9368	.0016	0.97
	(4.0%)	(4.2%)
Unavailable	26 961	63 095	< .0001	0.64
	(13.8%)	(28.3%)
Ethnicity:
Hispanic/Latino	18 167	38 671	< .0001	0.69
	(9.3%)	(17.3%)
Non-Hispanic/-Latino	152 226	124 992	< .0001	1.18
	(78.2%)	(56.0%)
Unknown	24 382	59 615	< .0001	0.62
	(12.5%)	(26.7%)
Other Variables:
Ever Answered PRO Questionnaire (%)	5231	N/A	N/A	N/A
	(2.7%)
Average # of Disease Registries per Patient	2.2	0.2	< .0001	N/A
	(range 0–24)

The sex breakdown of the study population of active portal users matches non-portal users almost exactly (62.9% females in both). The average ages were comparable with active portal users slightly younger (mean 50.5 vs 51.7 years). Compared with non-portal users, study population members were less likely to have missing race and ethnicity information (13.8% vs 28.3% for race = “Unavailable” and 12.5% vs 26.7% for Ethnicity = “Unknown”). The study population contained lower proportions of Black persons (10.3% vs 15.8%) and self-identified Hispanic/Latino persons (9.3% vs 17.3%). The ratio of representation of these subgroups among active portal users vs among all eligible patients was thus also reduced (0.78 for Blacks, 0.69 for Hispanics/Latino). On the other hand, Asians were represented more frequently among active portal users than among all eligible patients (Portal Representation Ratio 1.35). Among active portal users, 5231 (2.7%) had ever answered a PRO questionnaire delivered via the patient portal. The study population qualified for more chronic disease registries on average than the non-portal-using group (2.2 vs 0.2) possibly due to more complete clinical information in their EHR.

Research recruitment registry member characteristics

The age distribution of the study population of all active patient portal users shows a marked (and bimodal) female predominance below age 70 after which the female and male counts are superimposable (Figure 4). The distribution for those who agreed to participate in the Research Recruitment Registry shows a similarly striking female predominance below age 70, with nearly equal membership above 70. Figure 5 displays the age-sex breakdown data for Recruitment Registry members by 10-year age bins.

Figure 4.

Age-Sex distribution of (left) the entire study population of all active MyChart portal users and (right) those agreeing to be Research Recruitment Registry members.

Figure 5.

Histogram by 10-year age bands of age-sex distribution of Research Recruitment Registry members.

Agreement to join research recruitment registry

Agreement rates by sex/race/ethnicity with participant-to-population ratios (PPRs)

Agreement numbers for inclusion in the registry by demographic subgroup are shown in Table 2. The overall study population to which the invitation was passively displayed on the patient portal is also shown by subgroup. PPR values were calculated as described in the Methods section—generally values < 0.8 or > 1.2 are considered meaningful under-representation or over-representation, respectively.⁸

Table 2.

Agreement (invitation acceptance) rates and Participant-to-Population Ratios (PPR)

	PPR (0.8-1.2)	Accepted Registry Invitation (n)	Accepted Registry Invitation (%)	In Overall Study Population (n)
Grand Total	N/A	13 660	7.0%	194 775
Sex
Female	1.03	8828	7.2%	122 521
Male	0.95	4832	6.7%	72 254
Race
American Indian or Alaska Native	1.30	61	9.1%	668
Female	1.31	40	9.2%	435
Male	1.29	21	9.0%	233
Asian	0.46	365	3.2%	11 354
Female	0.43	217	3.0%	7217
Male	0.51	148	3.6%	4137
Black or African American	0.88	1227	6.1%	19 976
Female	0.92	959	6.5%	14 794
Male	0.74	268	5.2%	5182
Native Hawaiian or other Pacific Islander	0.75	14	5.3%	265
White	1.18	10 549	8.3%	127 756
Female	1.22	6631	8.5%	77 670
Male	1.12	3918	7.8%	50 086
Some other race	0.70	322	4.9%	6573
Declined	0.84	72	5.9%	1222
Unavailable/Unknown	0.56	1050	3.9%	26 961
Ethnicity
Hispanic or Latino	0.76	967	5.3%	18 167
Female	0.77	679	5.4%	12 501
Male	0.72	288	5.1%	5666
Non-Hispanic/-Latino	1.10	11 740	7.7%	152 226
Female	1.13	7558	8.0%	95 049
Male	1.04	4182	7.3%	57 177
Unknown	0.56	953	3.9%	24 382

Among subgroups with 1000 or more members in the overall study population, PPR values showed under-representation of Black males, Asians (both sexes), and persons of Hispanic or Latino ethnicity (both sexes).

Univariable logistic regression models with odds ratio for women vs men and race, ethnicity

Univariable logistic regression was performed using the following variables:

• Sex: Women were statistically significantly more likely to agree to participate in the Research Recruitment Registry than men (P < .001).

• The odds ratio for women (vs. men) was 1.083 (95% CI, 1.045–1.124).

• Race: Asian or Black/African-American persons were less likely to agree to participate in the Registry than White persons (P < .001 for both).

• The odds ratio for Asian persons was lowest at 0.369 (95% CI, 0.331–0.410), and for Black persons was 0.727 (95% CI, 0.684–0.773).

• Ethnicity: Hispanic/Latino persons were less likely to agree to participate in the Registry than Non-Hispanic/-Latino persons (P < 0.001).

• The odds ratio for Hispanic/Latino persons was 0.673 (95% CI, 0.629–0.719).

Full information from the univariable logistic regression model is included in the online supplemental material (Appendix B).

Multivariable logistic regression model

In a multivariable logistic regression model performed using 4 categorical variables (Sex, Race, Ethnicity, Ever Answered PRO Questionnaire) plus 3 quantitative variables (Age, Monthly Portal Use Rate, # Disease Registries), all 7 variables were found to be significant predictors of agreeing to be in the Research Recruitment Registry (see Table 3). A check for multicollinearity among these variables by calculating generalized variance inflation factors showed no significant collinearity with GVIF < 2 and GVIF ^ (1/(2*Df)) < 2 for each of the 7 variables in the regression model (see online supplemental material Appendix C).

Table 3.

Predictors of Agreement in multivariable regression model, with odds ratios and confidence intervals (CIs)

Predictor	Estimate	Std Error	z value	Pr (>|z|)	Odds Ratio	CI low (2.5%)	CI high (97.5%)
Intercept	−2.690	0.0363	−74.05	< 2e-16
Sex = Female (reference = Male)	0.155	0.0194	8.00	1.20e-15	1.168	1.125	1.214
Race = American Indian or Alaskan Native (ref = White)	0.132	0.1379	0.964	0.335	1.142	0.863	1.483
Race = Asian (ref = White)	−0.932	0.0557	−16.74	< 2e-16	0.394	0.352	0.438
Race = Black (ref = White)	−0.427	0.0325	−13.15	< 2e-16	0.652	0.612	0.695
Race = Hawaiian or other Pacific Islander (ref = White)	−0.445	0.2810	−1.58	0.113	0.641	0.353	1.177
Race = Unavailable/Unknown (ref = White)	−0.442	0.0409	−10.81	< 2e-16	0.643	0.593	0.696
Ethnicity = Hispanic or Latino (ref = Non-Hispanic/-Latino)	−0.368	0.0368	−9.98	< 2e-16	0.692	0.644	0.744
Ethnicity = Unknown (ref = Non-Hispanic/-Latino)	−0.227	0.0431	−5.28	1.29e-7	0.797	0.732	0.866
Age at Study Onset (increment = 5 years)	−0.0069	0.0006	−11.00	< 2e-16	0.966	0.960	0.972
# Disease Registries (increment = 1)	0.106	0.0032	32.78	< 2e-16	1.112	1.105	1.119
Monthly Portal Access Rate (increment = 10)	0.0049	0.0001	42.40	< 2e-16	1.050	1.048	1.053
Ever Answered PRO Questionnaire	1.271	0.0343	37.09	< 2e-16	3.566	3.334	3.813

In this multivariable logistic regression including 6 other variables besides Sex, the odds ratio for women (vs men) in the multivariable model remained significant at 1.168 (95% CI, 1.125–1.214), as shown in Table 3.

DISCUSSION

Principal findings

Under-representation of women and other demographic groups in clinical research studies could be due either to bias (explicit or implicit) during screening and recruitment, or by lower rates of deciding to participate when offered. By making an invitation to participate in a Research Recruitment Registry available to all patients accessing our patient portal, regardless of demographics, we sought to remove implicit bias in offering participation and assess agreement rates independently.

Our study population of portal users had nearly identical age and sex proportions as our overall group of active Health System patients. Portal users were more likely to be White and non-Hispanic, however.

Among all portal users, women were appropriately represented in the Research Recruitment Registry (PPR 1.03). Controlling for age, race, ethnicity, portal use, ever answering a PRO questionnaire, and chronic disease burden, women were statistically more likely to accept the invitation to join the Registry than men (odds ratio 1.17, 95% CI, 1.12–1.21). This supports the view that historical under-representation of women in clinical studies is likely due, at least in part, to implicit bias in offering participation.

In contrast, Black males, Hispanics and Asians (both sexes) had low PPRs. This under-representation persisted in the multivariable regression model.

Practical implications

Under-representation in clinical research—distinguishing implicit bias versus willingness to participate

Why is distinguishing the nature of under-representation important? Knowing the root cause can help guide actions one might take to reduce under-representation. If due to implicit bias, then proactive efforts to address this within the study team and procedures could improve representativeness of enrolled participants. Alternatively, if lower willingness to participate drives under-representation, researchers might study any potentially modifiable factors and/or develop novel recruitment approaches that result in higher participation rates.³⁴

Although women in our study were fully represented when implicit bias in invitation was eliminated, the same was not true of all racial and ethnic subgroups. Despite being above-average users of the patient portal, Asians as a group had the lowest agreement rate for joining the Research Recruitment Registry (PPR = 0.46). Other groups with a Registry PPR below the equivalence range of 0.8–1.2 included Hispanics/Latinos (0.76) and Black males (0.74) but not Black females (0.92). Among all race/ethnicity groups except Asians, females agreed to participate more frequently than their male counterparts. In contrast, Asian males agreed to participate more frequently than Asian females (PPR 0.51 vs 0.43). This interaction between race/ethnicity and sex could well reflect myriad factors including family structure, sex-specifc roles, religious beliefs, cultural history, logistical barriers, and trust in institutions.^35,36 When lower agreement rates rather than implicit bias drive reduced equity in clinical research, improvement is likely to require deeper understanding of the barriers specific to a given group. Working with group members to co-design culturally-aware approaches more likely to convey the benefits of being represented in clinical research has been shown helpful.^34,37,38

Data-driven personalized, precision medicine

In the coming era of increasingly data-driven precision medicine tailored more specifically to individual patients, patient sex is 1 biological variable known to affect the benefit and/or risk of treatment options. Accordingly, we need to have high-quality clinical research data on women as well as men for optimal personalization of treatment strategies—enrolling representative patient groups is crucial for getting relevant inputs into modern data science approaches.

EHR patient portals for research recruitment

Recruitment into clinical trials remains an enduring challenge. EHR patient portals are now ubiquitous, with many including a mobile app, and investigators have begun exploring patient portal use for research recruitment with promising results.²⁷ Leveraging the female preponderance among patient portal users (Figure 4) may be 1 way to encourage enrollment by women in clinical studies.

Limitations

Other opportunities for implicit bias

We believe we minimized implicit bias by automating display of the recruitment invitation to all patients on the portal home page. However, Blacks and Hispanics are under-represented among our portal patients. Although the Health System protocol is to encourage all patients to have a portal account, our study design doesn’t allow for distinguishing implicit bias at the step of offering patient portal access versus different rates of deciding to enroll when offered.

Passive versus active invitation to join

In this study, patients were shown the invitation to participate in the Research Recruitment Registry passively through display on the portal home page. Accordingly, overall enrollment as a percentage of the study population is low (7%). We are moving toward actively inviting patients by sending them an electronic message via their portal account. A cautionary note in Pfaff’s article is that the study enrollment rate following patient portal message recruitment was 4.2%. Examining the increase in enrollment in our registry following a messaging campaign should provide additional information on patients’ response to direct portal messaging.

Definition of sex/gender used

During the study period, our EHR’s configuration was updated to enable collection of additional sexual orientation and gender identity (SOGI) information.³⁹ For the period of this study, the fields have relatively few entries, and their inclusion would not have changed the study’s findings. With additional collection of SOGI information, it should become feasible to also evaluate for implicit bias for those subgroups.⁴⁰

CONCLUSION

Among all patient portal users offered the opportunity to enroll in our Research Recruitment Registry, women proved more likely to accept the invitation to join than men by a small but statistically significant margin even when controlling for age, race, ethnicity, portal use, and degree of comorbidities. This supports the view that historical under-representation of women in clinical studies is likely due, at least in part, to implicit bias in offering participation. Patient portals offer an attractive option for minimizing bias and encouraging broader, more representative participation in clinical research.

FUNDING

Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award Number UL1TR001105. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

CONTRIBUTORS

RDT, KEW, MEBH, MV, VK, SLM, DLW, TAB, SIG were responsible for the conception and design of the registry and study, and KEW, MV, VK, SLM, SIG, TAB created and operated the registry.

Data extraction, data de-identification, and data cleaning were completed by MV, VK, DLW. Analysis and/or interpretation of data was the responsibility of VK, KEW, SLM, MV, DLW, LC, SCR, RDT. Logistic regression using R was completed by MMW. DLW (primary writer), VK, KEW, SLM wrote the first draft of the manuscript, and VK, KEW, DLW, LC, MMW, SLM, SCR, RDT made the revisions to it.

All authors, VK, KEW, MV, SLM, DLW, TAB, LC, SIG, MEBH, MMW, SCR, and RDT Approved the version of the manuscript to be published, and agreed to be accountable for all aspects of the work.

CONFLICT OF INTEREST STATEMENT

None declared.