Small Is Essential: Importance of Subpopulation Research in Cancer Control

The ability to harness the benefits of “big data” has had a revolutionary impact on science, with its focus on the volume and variety of data sources, and application of both traditional and innovative analytic methods appropriate for large, aggregated data sets. We are concerned, however, about the opposite: “small data,” for which the size, dispersion, or accessibility of the population of interest makes it difficult to obtain adequate sample sizes to test specific research questions. Examples include racial or ethnic subpopulations (e.g., Honduran Latin Americans), populations occurring in specific geographic areas (e.g., reservations), and populations that have relatively rare characteristics (e.g., transgender persons). A great challenge is determining when a small group is of practical or theoretical interest (Figure 1). We define “practical and theoretical interest” broadly to include issues involving social justice, biological or geographic factors, and disease burden.¹ Ultimately, it is critical to ensure that all segments of the US population benefit from this research and from the latest technologic advances in cancer care services and delivery.

FIGURE 1—

Research with small data: identifying challenges.

INCLUDING UNDERREPRESENTED GROUPS IN RESEARCH

An example of the potential negative ramifications of not including underrepresented groups in research—or inappropriately aggregating them across groups—comes from the study of racial and ethnic health disparities and issues of equity in the United States. Intervention research often does not include a wide range of racial/ethnic subgroups; so it is not feasible to test whether an intervention created specifically for the majority group is also efficacious for the subgroups. Likewise, the ability to test whether an intervention can be altered for a particular subgroup is also often not possible. Epidemiological and surveillance research usually involves the inclusion of “minority or underserved populations” in addition to White or non-Hispanic White (NHW) groups. While this has allowed for a better understanding of these smaller populations and provides some progress toward addressing health inequities, there remain pockets of communities that are severely underrepresented within the broader “minority and underserved populations.”^2–6

As a further example, although Asian Americans as a whole have high incomes and good health outcomes overall when compared with NHWs, Hispanics, African Americans, and American Indian/Alaska Natives, this generalized statistic masks the fact that subgroups of Asian Americans, such as the Cambodians and Hmong, lag severely behind other Asian Americans.^3,4,7,8 Even within the NHW population there are communities that have long been disadvantaged (such as those living in Appalachian states), with low levels of income, literacy, and health outcomes.^9–11 These subgroups have generally been omitted or excluded from the research process because of challenges with identification and recruitment. Through this commentary, we hope to encourage research in subpopulations; we recommend both the development of new methods and the innovative use of existing methodological and analytic strategies across both intervention and epidemiological research.

ALTERNATIVE STUDY DESIGNS AND ANALYSIS PROCEDURES

There is growing recognition that to implement interventions in small populations, it may be necessary to consider alternative study designs, such as the use of single-case designs attributing propensity scores, and randomized group designs. In 2013, many of the studies submitted to the Division of Cancer Control and Population Sciences at the National Cancer Institute (NCI) on subpopulation research that did not score well in peer review received comments that the randomized clinical trial design was not appropriate because the sample size was insufficient to detect changes in the effect of the intervention. This criticism raises the question of whether these studies would be better suited for alternatives to the standard randomized control trial design, such as single-case, within-subject controls, and a variety of quasi-experimental designs.

One solution for testing interventions in small samples is to focus on within-rather than between-group designs. Because a within-group design uses the sample as its own control, there is no need for a separate control group, reducing by up to half the sample size required for accurate statistical comparisons. Among group designs, there are a number of quasi-experimental approaches that could be considered, including interrupted time series^12–15 and stepped wedge designs, the latter being particularly useful for studies in which there are distinct and dispersed cohorts or communities in which the intervention can be rolled out in a staggered manner.^16,17 Single-case studies involving a series of N-of-1 trials could be used to test intervention adaptations in an iterative manner, and Bayesian estimates can be produced from this series of trials to evaluate the potential generalizability of the findings to the subpopulation.¹⁸ Within-subject designs require more intensive longitudinal data than typically obtained through between-subject designs, but the advent of technologies for capturing temporally dense data, such as ecological momentary assessment and passive sensor technologies, makes these approaches more viable. Such data could also be used in conjunction with multilevel analyses of behavior across different spatial areas. This kind of study design can be statistically powerful, even with modest numbers of samples per geographic unit.¹⁹

For epidemiological research, innovative recruitment methods may be very useful. For example, respondent-driven sampling^20,21 has been successfully employed to identify and recruit groups for studies in which there is no existing sample frame, such as drug addicts or ethnic subgroups. Innovative analytic approaches, such as integrative data analysis,²² could be employed where independent data sets are combined together and analyzed as a whole to produce adequate representation and sample sizes. Integrative data analysis can also be used to combine data across multiple iterations of the same national survey where any one sample does not constitute an adequate sample size.

ADDRESSING THE CHALLENGE OF SMALL DATA

The National Institutes of Health (NIH)—and by extension the NCI—has an obligation to conduct research to improve the health of all Americans, not just the health of the majority population or those who are easy to identify. We therefore recommend the development and the use of methodological and analytic procedures to allow subpopulations to be meaningfully included in research. Figure 1 illustrates a model for determining when a “small” group is of research interest. However, it is also clear that other entities need to be involved in identifying populations of interest and in developing initiatives to address these groups, not just those who are responsible for grant funding decisions. For example, at the NIH, training for peer reviewers in study sections may be needed to ensure that they are knowledgeable about these innovative methods so that sound, rigorous scientific applications that employ them are understood and scored appropriately.

In addressing the above issues, NCI is planning a workshop to address three areas related to small populations:

• (1) identification, recruitment, and retention strategies;

• (2) epidemiological design and analytic approaches for small samples; and

• (3) intervention design and analytic approaches for subpopulations.

Based on the products of this workshop and responses to this editorial, the NCI will explore next steps to strengthen subpopulation research.