Risks and Opportunities to Ensure Equity in the Application of Big Data Research in Public Health

Abstract

The “big data” revolution presents an exciting frontier to expand public health research, broadening the scope of research, and increasing the precision of answers. Despite these advances, scientists must be vigilant against also advancing potential harms towards marginalized communities. In this review we provide examples in which big data applications have (unintentionally) perpetuated discriminatory practices, while also highlighting opportunities for big data applications to advance equity in public health. Here, big data is framed in the context of the five Vs (Volume, Velocity, Veracity, Variety and Value), and we propose a sixth V, Virtuosity, which incorporates equity and justice frameworks. Analytic approaches to improve equity are presented using social computational big data, fairness in machine learning algorithms, medical claims data, and data augmentation approaches as illustrations. Throughout, we emphasize the biasing influence of data absenteeism and positionality and conclude with recommendations to incorporate an equity lens into big data research.

INTRODUCTION

In the context of health, big data is considered “health data from multiple sources at scale”(84) and is considered ‘big’ when, in fact, it is complex. Big data encompasses: data from biospecimens; health records; medical and other imaging; individual, community, or satellite sensors; administrative records; policies, laws, and human rights records; environmental indicators; behavioral data; internet, social media, and mobile phone data; stock market trends, public opinion and more. A commonality for big data research in public health is that these data are almost always evaluated using observational rather than experimental methodologies.

Since a 2018 review of big data and public health (99), the risks to misinterpretations of big data findings for vulnerable populations that may exacerbate health inequity have become more apparent¹. Equity-focused critiques suggest that incomplete conceptualizations of big data may solidify biases, which further marginalize vulnerable populations (37). For example, concerns about racial bias in algorithms (112), a digital divide that only includes data for certain populations (84), and the absence of big data in health and social systems in several lower income countries (36), are widely expressed.

There are also incredible opportunities to leverage big data for health equity, through bringing together structural determinants data previously not included in epidemiologic research, integrating new forms of data, and bridging policy analysis with health planning -- all of which can aid in directing public health action to ensure more equitable outcomes (28,78) (Table 1). Advocates highlight the importance of big data for the uptake of evidence-based interventions (the purview of implementation science) as well as for, “delivery of the right intervention to the right population at the right time, and includes consideration of social and environmental determinants” (78). Research linking health data with new forms of structural determinants data, for example with measures of racially directed violence or discriminatory housing policies (such as redlining) (104), can help expand the scope of public health (28). By de-silo-ing and amalgamating data from seemingly unconnected sources (“mash ups”) advocates hope to create new actionable knowledge (34).

Table 1:

Examples of Recent Uses of Big Data for Selected Core Public Health Activities and Topics

PUBLIC HEALTH AREA	EXAMPLES
1. Descriptive Epidemiology Focus: Description, exploratory analysis, prediction	Analysis of an algorithm based on multiple data sources can applied to predict families at risk for lead poisoning, compared with a validated sources (119)	Examines associations between Twitter-derived sentiments related to racial or ethnic minorities and birth outcomes, building on construct development for ‘sentiments’ as an area-level measure of social context (108–109)	Explores relationship between highly public anti-Black violence, responses to public violence based on Internet activity following events, and patterns of mental health distress among Black Americans in the US (26)	Two-sample capture-recapture study building on profiles for MSM (men who have sex with men) through a social networking app to estimate the total number of MSM in mainland China and migration (66)
2. Social Behavior Monitoring (often in conjunction with sentinel events) Focus: Detection of events or patterns, monitoring of behavior signals, often with hypothesis testing	Application of multi-level models to study the relationship between searches for ”stress” and “스트레스”; “drinking” and “음주”; “exercise” and “운동”; and “suicide” and “자살”in South Korea at a primary level (month) and Korea’s suicide rate, unemployment rate, and after reports of a celebrity suicide or not at a secondary level (year). Results showed that stress-related searches increase suicide search volume on Google (134)	Multiple Systems Estimation (MSE) to estimate the number of extra-judicial killings committed by the Guatemalan government during a massive counter-insurgency campaign from 1978–1996. Stratified analyses revealed that, in some regions, indigenous people were killed at a rate 5–8 times greater than nonindigenous people (86)	Application of Multiple Systems Estimation (MSE) methods, using a combination of log-linear and Bayesian model averaging, to estimate human rights abuses focusing on estimating the total number of civilian killings and disappearances in El Salvador during 1980–1992 (44)	Exploration of the extent to which the phrases, “COVID-19” and “Chinese virus” were associated with anti-Asian sentiments in the context of WHO recommendations to avoid discriminatory language (e.g. location, ethnicity) and tweets by the U.S. President Trump containing racially discriminatory ant-Asian language (61)
3.Disease Surveillance Focus: Detection of disease/symptom status changes (infectious diseases); modelling trends in behaviors and outcomes	Regression-based, multi-sample stratified capture-recapture analysis to estimate the prevalence of opioid use disorder in British Columbia, Canada over time (94)	Exploration of Zika-related Twitter conversations around the time of North and South American outbreak, finding showed geographic rates of Zika-related discussion are moderately correlated with Zika incidence (123)	Infectious-disease modeling for predicting new COVID-19 infections, with inclusion of human-mobility data based on mobile-phone records in the US, presenting limitations of the method when data were absent from key sectors of society, such as those in prison and in nursing homes and helped explain disparities in mobility patters based on economic drivers of mobility (2)	Development of a local geography tool for identification of different types of COVID-19 clusters, combining multiple sources of data (27)
4. Evaluate Impact of Programs/Interventions/Policies Focus: Impact of programs and policies on health outcomes, behavior, care processes, systems of care/public health	Assessment of a national health system risk prediction tool/algorithm for population health management which identified inherent biases towards Black patients when cost (instead of illness) was evaluated – (112)	Evaluation of capture-recapture methods to estimate undetected COVID-19 cases, as compared with independent prevalence estimation using epidemiologic techniques, reported similar findings and less uncertainty (10)	Evaluation of black-market pricing of buprenorphine to better understand supply and demand for opioid addiction treatment using and substance use program needs (62).	Oregon Medicaid and commercial colorectal screening clustered by County socioeconomic and health statistics found that low-income areas and those with limited providers had the lowest rates of screening (29)

In this paper we: (1) summarize concerns about big data and health equity (2) present a series of analytic approaches to explore and address inequities using big data, and (3) discuss data augmentation methods to embed a health equity lens into big data research. We use the 5 V’s framing (6) encompassing: Volume, Velocity, Veracity, Variety and Value ( Box 1, Figure 1) adding a 6^th V, for Virtuosity, to re-conceptualize big data research to explicitly focus on equity.

BOX 1: THE 6 Vs.

Volume:

The breadth and depth of data.

Velocity:

The speed that data is accumulated - often close to the time of data collection.

Variety:

Types of data that can include unstructured (e.g. video, images, free text) as well as structured formats (rows and columns of data).

Veracity:

Refers to the trustworthiness of data for the questions being posed.

Value:

The decision-making potential derived from big data analyses.

Virtuosity:

The obligation of scientists to: (1) incorporate equity and justice frameworks into big data, for each of the V’s and (2) develop solutions for dismantling the -isms and -phobias that perpetuate biases in the data and interpretation.

Figure 1:

The 6 Vs of Big Data Adapted for conceptualizing Big Data with Foregrounding of Health Equity Considerations

1. DATA DO NOT MAKE THEMSELVES: RISKS FROM BIG DATA CONCEPTUALIZATION

Many critiques of big data concern the veracity and variety of data and risks to health equity resulting from how data are created. Precision public health, for example, is criticized for relying on datasets that don’t reflect underlying structural relationships, missing upstream socio-economic determinants (77). For example, early in the COVID-19 pandemic, cell phone-based mobility data was misinterpreted to suggest that non-compliance with restrictions was widespread in low-income communities in the U.S (compared to higher income communities) (20,106). A fuller exploration demonstrated that the privileges of working from home were not available to essential workers who had to continue commuting for work (20). Without a deeper understanding of upstream drivers, there is the risk that big data-based policies will further exacerbate inequities.

The potential to address health equity through big data research rests in large part, on the inclusiveness and accuracy of data for all types of individuals and communities (veracity and variety). Although there are numerous national population-based surveys in the US and other countries which have engaged methods to include more diverse samples, declining survey participation rates over time have contributed to challenges in using these national surveys for some equity-focused research (8). Widespread concerns have emerged about how little attention to inclusion and representation exist in big data formation, such that those who identify as racial ethnic or gender minorities, individuals from resource poor settings with limited cyberinfrastructures to capture data, and from environments for which providing personal data can result in discrimination and underrepresentation (30). One of the most important challenges associated with gaps in data for vulnerable populations can be thought of as ‘data absenteeism’ (84) whereby some groups are ‘absent’ from data. Paramount to any scientific endeavor is a need to understand how data were collected (or generated) and the gap between the study population (study participant data) and the target population. Barring a true census in which everyone is included, people will be missed; and it is incumbent on the scientist to assess if certain types of people or communities are systematically missed or misrepresented. Equity in who benefits from scientific knowledge and resource allocation fundamentally cannot be achieved if groups are not represented in the scientific studies that generate the knowledge that informs the distribution of resources.

Additionally, data are not reflective of the real world but instead represent protocols designed to collect or produce data, and the conscious or unconscious biases of investigators which may systematically miss or undercount certain communities (30). The United States census, for example, tasked with counting every resident of the United States every ten years, has systematically undercounted racial and ethnic minorities (and minority language groups) in communities because census workers did not feel comfortable visiting homes in low-income neighborhoods, where racial/ethnic minorities disproportionately live (4). As a result, these communities do not benefit from resources that are allocated according to population size.

There is a well-recognized digital divide in which many people do not have access to digital tools used to collect data (e.g., smartphones, internet portals). Data absenteeism is particularly relevant to global health as many data do not exist for populations who do not have widespread internet use. Assumptions that individuals are the sole owners of cell phones or computer devices and that data captured reflect these same unique individuals also generates biases in data, as demonstrated in a study of Ebola transmission risk in which multiple individuals shared a phone (36). Conceptualizing virtuosity in the formation of big data requires asking: Who is included and not included here and why (variety)? Can we address possible data gaps with supplementation methods or datasets (variety)? How could discrimination through lack of inclusion bias interpretation, and have a detrimental impact (value)? Are there theories related to marginalization we can apply to the research questions that enable a more thorough understanding of the data inputs and their meaning (veracity)? What assumptions are we making about historical, upstream or socioecological factors, racialization or discrimination risk in regard to the questions we are exploring (veracity)? How are we making assumptions about variables in our data and who they speak for, and how can we address them, if biases may be present (veracity)?

Data absenteeism has been linked to inaccurate algorithms, as when limited diversity among individuals included in machine learning-based risk models, results in model preferences that unfairly characterize what is ‘normal’, deviating from ‘normal’ or excluded altogether (9,24,30,112,118). Big data critiques also refer to uneven power structures embedded in the data’s origins (referred to as the positionality of data and its creators), resulting in calls for the democratization of data and data augmentation (37). Theory can play a role in how one approaches big data and equity (39), not only a socioecological theory to help understand different layers of information, but with theories presenting a perspective on What shapes inequities and for whom? and Who’s in the position of power in the data structure, development of data, analysis?

Machine learning programs are intended to improve health outcomes, reduce expenditures, and improve service delivery in clinical and public health programs. Machine learning involves developing algorithms and applying them to synthesized data (41). Recent studies show how discrimination can occur when algorithms are applied in the absence of complete data, in the presence of biased data and/or when biased assumptions are used to characterize data inputs (35, 41, 112, 117, 133). In a highly publicized study, electronic health record algorithms that determined which patients were referred for extra care, Obermeyer et al identified that the algorithm excluded Black patients who were equally as sick as White patients (112). The algorithm developers had inappropriately applied a variable capturing total health care expenditures as a proxy for unmet health needs in establishing risk scores. This example illustrates the inverse prevention law, in which those with greatest need, are least likely to receive resources (14) because the cost variable did not capture the unmet needs of Black patients (value). In the following sections, we present various approaches to enmesh equity approaches into big data research to mitigate these challenges.

2. ANALYTIC APPROACHES TO IMPROVE EQUITY FOR BIG DATA

Equity in Social Computational Big Data

Social computational big data includes data from technology companies such as Google, Facebook and Twitter (139) which are collected and stored from application programing interfaces (APIs), and contain data restrictions. (126). Moment by moment oscillations in behavior that are depicted on these online platforms leave a digital footprint which can be aggregated to uncover emerging population trends across a wide range of health topics (54,56,57,65,69–70, 82–83,111,114,120–122,,127,136,139,148). Social computational data is frequently validated against gold standard metrics, such as governmental sources, to corroborate it’s use for predicting real world activities (11,55,56,62,87,128). However, ethical challenges exist and this section examines the sixth “V” of Virtuosity in the context of using social computational big data focusing on: 1) data access and availability, 2) inclusion and representativeness 3) advantages, 4) methods of analyses, and 5) future goals for the field as related to equity.

As of 2021, on average 500 million tweets are tweeted on Twitter, daily, with 1.88 billion daily active users on Facebook, 500 million on Instagram, and 3.5 billion searches on Google (17,38,97). This massive amount of data volume and velocity provides an archive of human behavior. Data must be captured through the application programming interface (APIs) that technology giants provide. However, the API has access limitations based on industry controls (124), which reduce the accessibility for scientists to capturing the full volume and velocity of big data on these platforms. Originally, Facebook provided a public API to collect their data for research related purposes (51,85), but it is no longer publicly accessible. For Twitter, it is estimated that Twitter’s free streaming API enables the data collection of 1% of all tweets (100–101) significantly less than the full dataset of tweets from “Twitter’s Firehose,” a prohibitively expensive service (72). Consequently, social computational big data is highly volatile as it is based on the ever-changing nature of the industry and public opinion, such as personal privacy and security demands (2,68). Academic scientists are not the gatekeepers of this data but must work with the industry’s positionality as to what and how much data can be retrieved.

Data absenteeism is an important concern in social computational big data research (59,81). These online data sources do not bridge the digital divide but may in fact widen it and cannot be considered a replacement for methods that capture the needs of digitally excluded populations, such as those who cannot afford access, rural areas with limited connectivity, and resource constrained nations with political and economic barriers to access (138). Social computational big data does not undergo population-based sampling and thus, is not representative of the general population (100). Social media platforms are skewed towards younger users (18–29 years old) and those in urban areas. (32,95). Both reduce the generalizability of findings to estimate populations trends (32,45,93). Finally, the ability to identify the demographics of online users is a challenge, and even though new computational methods can detect user demographics on social media (18–19), data anonymity and privacy set forth by the industry often limit the collection of meta-data from their users profiles.

Social computational big data can provide a richer picture of experiences, compared with traditional data capturing methods, which has important equity implications for public health research. The origins of these platforms were not intended for research but recall and social desirability bias may be less present. Internet search queries and discussions on social platforms are organic, making them more authentic, unfiltered and genuine compared to data collected explicitly for research related purposes 12,33, 42,88). Lastly, without the barriers of social desirability bias these data can present more truthful and accurate view about beliefs and behaviors (15,22,53, 60, 64,89,131,140,147) For instance, people tend to lie when it comes to racially charged topics. An empirical example explored how racist searches on Google were a robust negative predictor of Obama’s voting share while national survey estimates about being racist were not (135).

A validation method for social computational big data compares results to governmental sources (gold standard) as with the example of Yelp reviews being validated for capturing foodborne illness through correlating Yelp results with data from Centers for Disease Control foodborne outbreak reports (110). This research led to the development of a supplemental foodborne illness surveillance system combining Twitter and Yelp data for public health departments’ foodborne illness tracking.(50) Such an approach is more sensitive and responsive to real-time signals compared to federal surveys of health conditions and behaviors, which can be limited in data lag and representativeness. (8)

Another method of validation is to establish a relationship between online beliefs and behaviors and offline consequences. An example of this online-to-offline relationship is how online racism can be a signal of the perpetuation of real-world hate crimes (58,105). A strong time series lag time correlation has been found between Anti-Islam related hashtags on Twitter and anti-Muslim hate crimes (102) and evidence of a 1-week earlier lag time between the negative sentiment towards Mexicans and Hispanics on Twitter and worse mental health outcomes in this population (58) suggests the connection between the online and offline world. Therefore, data from online sources can be used to reveal subversive feelings and beliefs that traditional epidemiologic methods may not fully contend with.

Finally, the racialization of diseases and the long history of abuse conducted in medical research (40) has prevented scientists from capturing data on stigmatized and marginalized populations (5,12,43,91). For instance, a review of government funded cancer studies found that all racial and ethnic minorities are considerably underrepresented in cancer clinical trials with fewer than 2% of studies focusing on minority health needs (21,80). Even in studies of environmentally-related diseases that disproportionally affect minority communities, Black, Brown, and Indigenous populations are less likely than their White counterparts to be represented.(16,80). Data from online social media platforms may help reduce this gap in recruitment, as it has been shown that a greater proportion of racial and ethnic minorities use social networks (32,45).

Fairness in Machine Learning

Technically, the machine learning community approaches equity through the concept of fairness. In recent years there have been substantial efforts to formalize this concept in machine learning algorithms. Three formal definitions have gained recognition: 1) anti-classification, 2) classification parity and 3) calibration (23). In the anti-classification approach, protected attributes like race, gender or age, and their proxies, are not explicitly included in analyses. The idea is that these protected attributes should not be considered in the final model, which by definition would make predictions unequally across all protected groups. Therefore, by excluding them the model is forced to predict the expected value of the outcome by using variables that are not considered to lead to unfair models. Here, no guarantees about the accuracy of the model for different groups are made. With classification parity, common definitions of model accuracy or performance (e.g. false negative rate) are forced to be equal across different groups defined by the protected variables. As such, in the case the model is used to redistribute limited resources (i.e. access to health care), and then protected groups are equally considered. Although this guarantees the model performs the same for all groups, it usually comes at the expense of a minimum common denominator where at least one group performs worse than it otherwise would. In calibration, it is required that conditional on the risk-estimate provided by the model, outcomes are independent of the protected attributes. Although this last definition is always desirable, in practice it falls short of protecting all groups.

The definitions of fairness seem intuitive to an extent. For instance, it is natural to think that if one is expected to build a fair model, protected variables should not be used to decrease or increase the probability of certain prediction (anti-classification). Equally, if the final model does not have the same accuracy across all groups defined by the protected variables one might think that the model is not fair towards that group (classification parity). Finally, it is logical to assume that if a model is on average representing the true underlying risk of the specific group this model is fair (calibration). More importantly, given that a formal definition of fairness is agreed upon, algorithms can be designed to satisfy it (3). For instance, a recent method constructs decision rules that ensure true positive rates and false positive rates are equal among pre-specified groups (48). Equally, techniques that use pre- and post-processing of the input variables and regularization to lead to parsimonious models have been devised to guarantee parity across different demographic variables (103,150). Unfortunately, no notion of fairness comes for free without affecting model performance for some or all groups (96).

Let us take the concept of anti-classification. Sometimes, the variables left out have highly predictive values and not including them results in a less accurate model for one of the groups. For example, measurements of oxygen saturation level has been shown to be less accurate for Black patients than White patients (133). As such, an algorithm that leaves out protected variables would not correct for the inherent bias of the technique. Similarly, although male patients with breast cancer have higher mortality as compared to female patients (while the latter have substantially higher incidence) (143), any model trying to predict mortality of breast cancer patients or incidence must consider sex to accurately predict risk levels. Additionally, identifying all proxy variables is extremely difficult and if they are left in the model the influence of the protected variables on the outcome can be ‘learned’ even when they are not present. For instance, the difference in intensity of the pixels for computer tomography images is enough for an algorithm to ‘learn’ that the images come from different hospital systems (149). If hospital systems see patients with different underlying risks, then the model would use this information without accurately learning the true underlying features that would lead to a correct interpretation of the image.

Expanding on the male breast cancer example, due to the stark differences, any dataset built to predict mortality of breast cancer patients will be comprised of many more female than male patients, and requiring a false negative rate to be the same across all protected groups (classification parity) will be detrimental for female patients (143). Finally, requiring proper calibration (outcomes need to be independent of the protected attributes conditional on the risk estimate) can be extremely misleading if the outcome is not well explained by the collected variables or a proxy outcome is analyzed. In the Obermeyer study described earlier, total health care expenditures was used as a proxy for unmet health needs in establishing risk scores (112). Requiring calibration of a model built with these data would have been problematic for Black patients as it would be reinforce the root problem; the fact that the outcomes collected (expenditures) did not represent the true need for medical attention.

Although debiasing machine learning models have gained a lot of attention in the recent years, solutions to apply all scenarios have not yet been found, underscoring the difficult nature of this problem. Therefore, depending on the specific problem at hand, scientists should investigate the three current accepted definitions of fairness and evaluate which are most relevant to the specific problem being tackled.

Equity and Clustered Big (claims) Data

Claims data from the Centers for Medicare & Medicaid or private health insurers are rich sources of big data that are used to collect habitually unchanging patient health information and other demographics. Claims data can be considered as high in Volume given how much data are collected during healthcare visits, of which there are a Variety of data, such as health conditions, diagnostic codes, billing, and payment and the data are produced at a high Velocity—frequently updating given how many health services are provided daily. But, what about the Veracity or the trustworthiness of the data? Incorporating the 6^th V—Virtuosity—supports an equity lens in which to interrogate the data to properly understand the patterning of health outcomes uncovered in claims data. In this section, we consider large administrative data sets that reflect routinely collected patient information.

To analyze claims data, scientists use AI to develop machine learning algorithms that “learn” to detect patterns in the data. These data are being used to examine relationships of health outcomes, for example mortality among Veterans Health Administration patients and predicting Type II diabetes (125,144). However, elucidating health patterns using claims data is challenging given the initial purpose for collecting the data are for billing rather than monitoring of patient and population health. There are numerous challenges with the use of algorithms to detect patterns in claims data.

First, many patterns detected are correlational, and are limited in their ability to explain why certain patterns are uncovered (137,146). Moreover, the validity of methods and algorithm transparency need to be addressed. One of the main biases in algorithms is omitted variable bias, which arises from having limited information on other potentially relevant factors that influence the outcome (146). The lack of information is particularly true for datapoints that measure the structural and social determinants of health. For example, in a study that predicted Type II diabetes, having limited information about the patient’s diet or access to healthy and nutritious food creates omitted variable bias. Another challenge is data absenteeism. When states did not expand Medicaid entire swaths of populations were missing within claim’s records. Given the various challenges with AI to analyze big (claims) data, a reorientation toward ensuring Veracity and Virtuosity are evaluated is paramount. Attention to biases including measurement error, sampling bias, and ascertainment bias, can mitigate equity risks from claims data (75). Equally important, such methods may best be viewed as hypothesis generating, rather than hypothesis testing.

To help explain “why” the patterns are seen in claims data, analysis approaches would benefit from explicitly leveraging the inherent data structures. Clustering of populations within different environments (e.g. counties) are intrinsic data structures. One statistical technique that supports incorporating health equity lens to big data analysis are multi-level (hierarchical) models. Multi-level models (MLM) concern themself with data that are nested within clusters of higher orders of influence (e.g., years, months, classrooms, countries, etc.) (52). In typical MLM analyses, the units of analysis are individuals (at a lower level) that are grouped within contextual (higher level) units. The use of MLM, and other similarly situated analyses has been growing, particularly in research aimed at understanding how policy and other structural determinants may shape the outcomes of interest (49,79,92,115–116). For example, in a 2011 study by Klawitter the results indicated that gay men who lived in states with anti-discrimination policy protections in employment earned 8% more per year, compared to gay men in states with no anti-discrimination legislation-- portraying the importance of leveraging the clustered structure of the data to explain the patterns (79).

In health insurance claims, the data points (patient data and outcomes) are clustered within geographic regions (e.g., Census tracts, Counties, States) with diverse policy and structural environments. The diverse geographies the data are clustered within have varied policies, such as minimum wage, labor rights, anti-discrimination protections, and poverty alleviation measures. How might the policy and environmental contexts in which the patients live shape the patterning of the health outcomes that the AI analyses might find in the claims data? Scholarship is beginning to grapple with these questions as applied to big data. A study by Davis et al., found that 42% of Oregon Medicaid and commercial insurance patients that were eligible for colorectal screenings had a completed screening over a four year period (29). Interestingly, with the use of MLM, the authors found that the percentage varied by county, such that counties with higher rates of socioeconomic deprivation (e.g., lower high school graduation rates, higher unemployment rates) and lower rates of endoscopy specialists had the lowest colorectal screening rates (29). Studies that intentionally leverage the data structure can enhance equity approaches to claims data analysis, and help ensure that proper algorithm “learning” is taking place to support accurate interpretation of the results.

3. DATA AUGMENTATION TO IMPROVE BIG DATA EQUITY

Much of the appeal of ‘big data’ comes from its high Volume attribute, which carries the aspiration of statistical flexibility to support complex analyses and protect against Type I and II error. High volume, referring to the number of participants, may even bring the aspiration of generalizability, in which identified statistical associations are broadly applicable to other populations. However, a high volume of participants cannot replace the representativeness of participants, and no amount of data can confer generalizability if representativeness is lacking (71).

Data augmentation methods such as multiple systems estimation (MSE), also known as capture-recapture, can be used to assess and correct undercounting of populations, thereby improving representativeness and moving closer to equity (Virtuosity). With origins in wildlife biology, MSE has common applications in public health to estimate the size of “hidden” and “hard-to-reach” populations as well as evaluating the completeness of surveillance systems and identifying patterns of differential underreporting (47,145). MSE estimates the total size of a population based on the degree of overlap of two or more incomplete lists (or samples) of that population (i.e., how many unique individuals are observed on multiple lists). The greater the overlap, the smaller the unobserved population (i.e., the number of people not already observed on any of the lists); conversely, the smaller the overlap, the greater the unobserved population. Several assumptions are necessary for valid estimation, yet the ‘list independence’ assumption often receives the most attention in public health applications (67). This assumption states that the lists used for MSE must be statistically independent from one another; being on one list does not increase or decrease one’s probability of being on another list. A variety of design and statistical approaches are available to satisfy or relax this assumption, most notably regression modeling to account for list dependency (145).

MSE has been applied in countless settings, using various types of lists, to provide context and scope for populations of public health interest (10, 47,66,107,113,145). For example, Hu et al. implemented a two-sample MSE of men who have sex with men (MSM) in mainland China using high volume data from social networking sites (66). Simulating mobile phone positions covering the country, the authors recorded profile IDs from Blued, a popular MSM social networking app, over two 2-week periods seven months apart, as two independent samples for the MSE study. Each sample included nearly 2.5 million profiles, with an overlap of 1.3 million profiles. They estimated 8,288,536 MSM in mainland China. Subnational estimates by cities and provinces served as denominators to calculate the burden of HIV and sexually transmitted infections among MSM in different areas.

Incorporating at least three lists provides additional statistical flexibility for MSE. In an example of incorporating high variety data, Min et al. linked six data sets covering the spectrum of medical touchpoints to estimate the prevalence of and trends in opioid use disorder (OUD) in British Columbia, Canada from 1996 to 2017 (94). Stratified analyses identified the greatest increase in OUD prevalence among males 12–30 years and 31–44 years, from 2013 to 2017. Stratified analyses, as illustrated in the example, are especially important for the advancement of equity and highlight whether subgroups are systematically underrepresented in data (and differentially deprived of resources) or, in the case of human rights abuses, systematically targeted in instances of mass killings (44,86). As noted by Barocas, commenting on the need to include stratified estimates of race/ethnicity in applications of MSE, “inaccurate counting is another form of structural racism and leads to widening disparities.” (7).

Potential Challenges and Opportunities in Record Linkage

Lists and data sources are almost always incomplete, and oftentimes systematically incomplete. Failing to acknowledge this fact risks perpetuating inequities by continuing to underrepresent vulnerable populations in scientific studies. Populations that are of key interest due to their unique vulnerabilities to health outcomes are often not represented in high Volume data sets. These populations may be missed because they only comprise a small proportion of the overall population, or because of the stigma and discrimination that often follows from disclosing their identity.

Beyond the MSE applications, record linkage of overlapping and complementary data sets can be used for other data augmentation purposes. Just as administrative records may be linked for MSE, administrative records, as well as federal population-based surveys (8), are increasingly being linked to generate complex data sets mapping individuals’ contacts with institutional touchpoints. Some of these data sources may come from institutions that explicitly serve specific marginalized populations (e.g., housing services, social welfare programs, health clinics providing services to specific groups such as female sex workers or transgender women), ensuring that these populations are included (by design) in the creation of the data. Other data sources may come from hospital records, arrest records, and social services (e.g., housing, child welfare, substance use treatment). Each data source collects overlapping information with a different focus and, in aggregate, provides more context through which to view the lived experience of the individual, albeit keeping in mind the limitations of data positionality described earlier. In order to not perpetuate inequities and discriminatory practices, scientists must exercise caution in which data sets are selected for data augmentation and be vigilant against any biases that may be in place in the construction of those data sets (37).

Record linkage, which was not an aim when creating these siloed data systems, poses additional challenges. Investigators must think creatively about how to accurately match the same individual across multiple systems based on limited personal identifiers while protecting everyone’s privacy. A potential solution may be found in hashing (31) and biometric scanning (e.g., fingerprint scanners, iris scanners) (1,130,141); emerging, low-cost methods, in which algorithms generate complex alphanumeric IDs based on identifiable information. The ID uniquely identifies the client’s record but cannot be reverse coded to identify the client. The algorithm can be applied internally within each institution to preserve client privacy while facilitating the linkage of records across institutional touch points, clinics, and other data sources. However, in relation to biometric scanning, fingerprint scanners may be associated with the criminal justice system. Even though the code generated from the scan cannot be linked back to criminal or immigration databases, this approach to record linkage may not be considered acceptable to certain groups or be met with skepticism (1,76).

DISCUSSION

Our paper examines important equity and justice considerations in the conceptualization and application of big data methods. For users of big data, it is paramount to challenge our assumptions and develop new or improved frameworks to ensure equitable big data research practices. Williams et al., states “In particular, creators of algorithmic systems have three general classes of approaches to prevent discrimination: they can make the data less biased beforehand, build fairness criteria into the algorithm (discussed earlier), or alter the application of the rules after the algorithm runs.”(146). However, it would be more ideal to prevent discrimination in all three classes. One strategic way to enshrine Virtuosity within big data practice is to foreground the experiences of particular groups that experience marginalization. Incorporating frameworks such as intersectionality theory, which recognizes that when social identity categories intersect (reflecting interlocking systems of privilege and oppression and they may result in unique and intensified forms of discrimination), can improve the way we approach uses of big data in public health (25). At a minimum, this requires practitioners of big data to recognize and challenge their positionality to the data itself and its interpretations. For example, use of big data sources through a health equity framework mandates the grappling of issues, such as racism, sexism, homophobia, transphobia, classism, etc. The explicit application of theoretical and analytic frameworks that contend with the structural and contextual factors that shape our lives is paramount in research using big data.

To improve equity in big data research, we provide several suggestions to critically incorporate the 6^th V of Virtuosity. The first is to include social epidemiologists in the research and prioritize social epidemiology training beyond programs in epidemiology and other public health disciplines. Scientists who study social epidemiology have a deeper knowledge of the structural and systemic forces that have generated a distribution of advantages and disadvantages in society (73). The second suggestion is to increase the level of diversity in researchers across disciplines pursuing big data and equity. Discriminatory biases can be prevented through the addition of a wide range of perspectives as this can reduce the likelihood of generating biases based on singular viewpoints (46). Thirdly, generate parterships between industry and academia (128). Big tech should work with social epidemiologists to generate more ethical and virtuous research. Fourthly, federal and state policies are needed to safeguard against biased and discriminatory production of big data. Fifth, as scientists it is important for us to evaluate our own biases and understand that we do not have the breadth of experience to know what is fully needed to improve equity.

Coupling these recommendations with a focus on training, it is critical that schools of public health, departments of epidemiology and biostatistics, and other data science training programs have diverse representation among students, staff, and faculty. These programs should emphasize core competencies in sampling theory and designs, participatory engagement and working with data and collaborators spanning diverse disciplines (e.g., social welfare, criminal justice), as well as understanding the role of historical and structural racism and inequality. When teaching focuses on biases, especially as they relate to study design and analysis, competencies should include a historical lens highlighting structural inequalities and interrogating its influence on data, data absenteeism, and the positionality of data. We also encourage the discussion of methodological approaches to reduce the impact of such biases. As an example, the Department of Epidemiology and Biostatistics at the University of California, San Francisco, in partnership with the Bakar Computational Health Sciences Institute and the Center for Health and Community, has developed a pre-doctoral training program that integrates the theoretical frameworks and methodological tools of behavioral and social scientiests with the rapidly evolving technical repertoire of computational health scientists. Graduates of the Data Science Training to Advance Behavioral and Social Science Expertise for Health Research (DaTABASE) program (NIH/NIMHD T32MD015070) are trained to apply analytic tools to novel data sets for research on behavioral and social processes underlying health disparities. Finally, public health practitioners must honor data augmentation and community involvement in it, and as such must listen and acknowledge the lived experience of often ‘absent’ communities and ensure that they are at the forefront of the design and development of equitable research (15, 22, 60, 89, 131,140, 147).