Geography, generalisability, and susceptibility in clinical trials

Randomised clinical trials (RCTs) are generally considered the highest standard of evidence in medical research, as randomised treatment allocation promotes homogeneity in baseline characteristics between treatment groups, maximising internal validity and reducing both bias and confounding. RCTs, however, often enrol a convenience clinical sample, and can face challenges of external validity if that sample does not represent the full population at risk, or the full range of co-exposures and susceptibility factors likely to be encountered in clinical practice.1, 2, 3 Many such biases can be geographical in nature; for example, proximity to clinical sites can influence recruitment and retention,⁴ which is important because neighbourhoods differ in socioeconomic status and environmental exposures (ie, air pollution), both shown to affect respiratory health⁵ and therefore potentially influencing observed treatment efficacy. In this moment, when clinical trials for COVID-19 vaccines are being run with unprecedented expediency to mitigate a virus that has disproportionately impacted minority populations and those with lower socioeconomic status,⁶ thoughtful attention to representativeness, generalisability, and spatial co-exposures in RCT populations is of paramount importance.

Despite this discrepancy in attention to internal versus external validity, clinical guidelines prioritise RCT results in making treatment recommendations, even when available RCT data might represent a very different population. RCTs have not traditionally recruited cohorts that are unbiased representations of the population at risk, nor reported adequate information on cohort characteristics, including demographics and co-exposures, to support thorough assessment of a trial's applicability to another population.⁷ External validity can be further limited by factors influencing an individual's decision to participate; some evidence suggests that asthma RCTs have been disproportionately comprised of individuals with lower socioeconomic status who lack access to high-quality medical care;⁸ other evidence suggests lesser access to clinical trials for rural communities and those with lower socioeconomic status.⁹ Clinical trials have been criticised for these potential challenges to external validity, and some improvements have been made: best practices for pragmatic RCTs have been developed,¹⁰ including more complex randomisation strategies to minimise bias, and a larger number of RCTs are now reporting more thorough information on patient selection, eligibility, and enrolment—although it is still only a minority of RCTs that fully comply with these standards.7, 11, 12

Socioeconomic status both directly and indirectly influences health and treatment outcomes through a complex array of social, environmental, and medical factors, including health-care access. The greater severity of asthma among children of lower socioeconomic status in the USA is well established, and clinical outcomes vary substantially by socioeconomic status, in part because participants with lower socioeconomic status often reside in areas with greater pollution, chronic stressors (eg, violence), poorer-quality housing, or fewer healthy dietary options. Given the limited range of clinical data to capture these complex social and environmental co-exposures in an RCT population, however, it is challenging to determine how each factor might influence observed treatment efficacy or a given trial's generalisability. While several observational studies and RCTs have focused on inner-city populations (eg, the Inner-City Asthma study),¹³ none of them, to our knowledge, have examined whether treatment efficacy differs for participants in relatively high-pollution versus low-pollution areas within the urban setting (eg, closer vs further from a major roadway), despite well established associations between air pollution and asthma. Even very well conducted major RCTs, including large National Institutes of Health (NIH)-sponsored clinical trials, have not yet explicitly incorporated social and environmental factors into study design and implementation.⁷

To assess representativeness in socioeconomic status and environmental exposures among participants in some exemplar asthma RCTs, we developed geographic information system (GIS)-based metrics to characterise the residential census tracts of 874 adults and children in RCTs run by AsthmaNet, an NIH-sponsored clinical research network. Data were extracted from baseline questionnaires of three trials—Best African-American Response to Asthma Drugs (BARD),¹⁴ Steroids In Eosinophil-Negative Asthma (SIENA),¹⁵ and Step-up Yellow Zone Inhaled Corticosteroids to Prevent Exacerbations (STICS)¹⁶—conducted using the same protocols in 17 cities distributed across the USA. We geocoded participant residences and linked these in GIS to census tracts and national roadmaps (using StreetMap Premium for ArcGIS 2016) to create commonly used indicators of tract-level socioeconomic status, including median household income and percentage of population living below the US federal poverty level. As an indicator of near-roadway pollution exposure, we calculated weighted roadway density¹⁷ within multiple distances of each home.

A majority of participants (71·4% for BARD, 54·5% for SIENA, and 55·3% for STICS) lived in tracts with median household incomes below the 2016 US average (US$59 039), and with a greater-than-average percentage of residents living in poverty (figure ). After merging baseline data across all three trials, greater roadway density near the home and tract-level poverty were separately associated with lower baseline lung function (percentage predicted FEV₁), after adjusting for age, sex, race, and ethnicity. These results reveal that participants in these three multicentre RCTs disproportionately lived in areas of lower socioeconomic status (which have higher air pollution exposures than other areas, on average, in the USA),²⁰ and that greater roadway densities (a proxy for traffic-related pollution) conferred lower baseline lung function. Such social or environmental co-exposures should be considered in clinical trials broadly, where possible, as they might plausibly alter observed treatment efficacy and, if extreme, could affect generalisability. Further research is needed to determine whether and how spatially-distributed co-exposures influence treatment response during study interventions.

Figure.

Percentage of participants in each of three AsthmaNet clinical trials living in high-poverty census tracts compared with the US average, by race

Horizontal lines indicate US averages, which are calculated from the total population, by racial group, living in high-poverty tracts for all tracts in the USA in 2016. The poverty rate among AsthmaNet participants is calculated as the percentage of participants living in census tracts where the proportion of residents living in poverty¹⁸ is greater than the overall poverty rate for the US population in 2016 (ie, >12·7%).¹⁹ The race variable is self-defined primary race from the AsthmaNet baseline registry form. STICS enrolled children (5–11 years of age), SIENA enrolled adults and adolescents (≥12 years of age), and BARD enrolled adults, adolescents, and children (≥5 years of age). BARD=Best African-American Response to Asthma Drugs. SIENA=Steroids In Eosinophil-Negative Asthma. STICS=Step-up Yellow Zone Inhaled Corticosteroids to Prevent Exacerbations.

Advances in spatial analysis and GIS have driven a rapid increase in the use of geographical analysis in epidemiology internationally.⁵ We propose that GIS can also be a powerful tool to refine the interpretability and applicability of RCT data—both in better defining the generalisability of any given RCT, and in more clearly identifying subpopulations for whom a given intervention might be most beneficial. Characterising the geographical context of RCT cohorts, ideally at the outset of any trial, can help to identify potentially influential social or environmental co-exposures (eg, living in high-pollution areas, or in sub-standard housing), and could inform on spatial patterning and clustering in recruitment and retention. In resource-limited settings (eg, lower-income and middle-income countries), geographical analysis could help to target RCT recruitment more cost-effectively, by more precisely matching participant characteristics and spatial co-exposures to those of the intended treatment population. Ultimately, using spatial analysis and GIS to better understand the lived context of RCT participants, thus better accounting for socioeconomic and environmental co-exposures, can help to improve the interpretability of RCT results and to better identify subpopulations for whom a given intervention might be particularly effective, and will inform on the true generalisability of a given RCT's results, all with the aim of improving patient care.

© 2021 Martyn F. Chillmaid/Science Photo Library