Applying a Community-Engaged Participatory Machine Learning Model

Abstract

Although predictive algorithms have been described as the definitive solution to bias in health care, machine learning techniques may also propagate existing health inequities within the community context. However, there may be ways in which machine learning techniques can help community psychologists, public health researchers and practitioners identify patterns in data in a way that empowers improved outcomes. Incorporating community insight in all stages of machine learning research mitigates bias by positioning members of underrepresented communities as the experts of their lived experiences. As community psychologists already prioritize community-based participatory practices, we propose three core guiding principles for a community-engaged participatory model for research using machine learning techniques: shared decision-making, reflexivity and structural humility, and flexibility and adaptability. Guided by these three principles, we emphasize grounding priority setting, problem formation, model assumptions, and interpretation of the resulting algorithmic patterns in the truths born from the lived experiences of people closest to the problem. We also suggest opportunities for bi-directional and mutually empowering partnerships between algorithmic scientists and the communities to which their algorithms will be applied. Inclusion of community stakeholders in all stages of machine learning for health research provides an opportunity to develop algorithms that are both highly effective and ethically grounded in the lived experiences of target populations.

Machine learning is an emerging method in medicine and public health. In theory, by developing predictive algorithms, scientists can identify patterns in big data that can inform novel health solutions to improve health outcomes. Although predictive algorithms have been touted as a definitive solution to bias in health care, machine learning techniques may also propagate and amplify existing health inequities within community contexts (Lett & La Cava, 2023; Robinson et al., 2019; Sveen et al., 2022).

Unfortunately, machine learning algorithms are often trained on data that do not represent the populations they will be applied to (Wen et al., 2022). Additionally, machine learning algorithms often fail to include sufficient input from members of the historically marginalized communities who are most affected by health inequities (Prabhakaran & Martin, 2020). For example, an artificial intelligence-based facial recognition system was found to be unreliable in identifying dark-skinned women, reflecting racial bias with implications for dermatologic equity (Buolamwini & Gebru, 2018). In another example, an algorithm developed by UnitedHealth discriminated against chronically ill Black health plan members (Obermeyer et al., 2019). The UnitedHealth algorithm used historical health spending to determine which patients would most benefit from a specific preventive program. Since Black patients spent less than White patients who were comparably ill, using prior healthcare spending to predict current healthcare needs systematically disadvantaged Black patients. The UnitedHealth algorithm intentionally excluded race in an attempt to avoid bias (Obermeyer et al., 2019). However, scientists cannot simply ignore race due to systemic oppression that shapes the experience of racial minorities (Lett et al., 2022; Robinson et al., 2019). Scientists must actively acknowledge the influence of racism, intersectional stigma, and other forms of oppression by thoughtfully incorporating these factors into algorithm development (Lett et al., 2022; Lett & La Cava, 2023).

This commentary aims to provide a framework for conducting community-engaged machine-learning research that emphasizes community members as experts in their lived experiences. Meaningful inclusion of historically marginalized communities might have mitigated bias in prior instances of discriminatory machine learning applications. We propose the inclusion of stakeholders who are representative of the population to which an algorithm will be applied in all stages of machine learning for community health (pre-data collection to results interpretation and implementation). Community-engaged research provides an opportunity to create algorithms that are grounded in the lived experiences of structurally marginalized communities (Asabor et al., 2022). This facilitates consideration of structural factors that cannot be measured and may not otherwise be captured in algorithmic development.

Community-based participatory research is one form of community-engaged research that allows for equitable partnership among researchers and community members (Israel et al., 2001). Researchers acknowledge that community members are experts of their lived realities while fostering co-learning throughout the research process. Community-based system dynamics is a method of algorithmic development that relies on community participation (Prabhakaran & Martin, 2020). Community-based system dynamics is a participatory approach to solving complex problems that employ community engagement and systemic modeling, such as computational method (Videira et al. 2017) It involves engaging relevant community stakeholders. Using this method, community stakeholders and researchers develop a shared understanding of the problem and map out models for solving identified problems (Prabhakaran & Martin, 2020). By mapping out these models with the involvement of community stakeholders, implicitly held causal assumptions are made explicit, thus providing an opportunity to identify biased assumptions that should be addressed in the development of an algorithm (Prabhakaran & Martin, 2020).

There are existing frameworks that have identified specific methods for engaging community members in artificial intelligence research. The Artificial Intelligence (AI) in Community Citizen Science is a framework that integrates algorithmic development and community engagement to create social impact through empowering the targeted community (Hsu et al., 2022). Community Citizen Science underscores the importance of co-creating or co-adapting algorithms with local community members, collecting data in partnership with community members and understanding and interpreting data with community members. Lett and Cava (2022) provide an intersectional framework for addressing fairness in health science artificial intelligence models by emphasizing the relevance of multiple intersecting factors to health inequity among multiply marginalized communities. The authors acknowledge the importance of community involvement and providing guidance on the application of intersectionality to mitigate bias in different stages of the algorithmic development process (data pre-processing, feature engineering, model training, validation, deployment, and updating). They enumerate six dimensions of intersectionality (social inequalities, intersecting power relations, social context, relationality, complexity, and social justice) and detail their specific implications for machine learning fairness in the health sciences. Bergman et al (2024) provide a community-centered methodology: SocioTEchnical Language agent Alignment (STELA), for developing artificial intelligence models that align with the values and goals of the community for which the artificial intelligence model is developed. In this approach, researchers work with community members to identify their perspectives that are salient for developing rules employed during the algorithmic development process.

Our proposed framework builds upon the existing evidenced-based literature by adding essential elements that underscore the spirit of developing and co-creating algorithms which should occur when engaging communities.

A model for community-engaged participatory machine learning

Using three guiding principles, we emphasize incorporating community insights throughout the machine learning research process (Figure 1).

Figure 1.

Model for Community-Engaged Participatory Machine Learning

1. Shared decision-making. The shared decision-making principle emphasizes soliciting community participation at every stage of the research process to inform collaborative decision-making.

2. Reflexivity and structural humility. The reflexivity and structural humility principle emphasizes recognition of the value that community participation brings to the quality of algorithmic development. Community members bring their lived experiences influenced by structural systems that may have fostered negative health outcomes, and lack of trust in research endeavors. This principle encourages researchers reflect on their biases, their relationship to community members and the project, and how researchers’ experiences may differ from those of community members. This intentional exercise may continuously remind researchers of the importance of the contribution of community members’ lived experiences that may otherwise be absent from the research team and the need to consider these experiences in algorithmic development.

3. Flexibility and adaptability. The flexibility and adaptability principle emphasizes a willingness to respond to community feedback after intentional solicitation and valuing of community contributions, even when such a response increases the complexity of algorithmic development.

These principles feed into each other and can be necessarily mutually constitutive. Below, we enumerate specific, overlapping ways to apply these principles from the start of algorithmic development to the study’s conclusion.

Project conception and problem formation

In participatory machine learning, research should begin with community participation in the priority setting and problem formation stage because community members are the experts of their lived experiences and, in the case of medicine, their health challenges (Lett et al., 2022). Qualitative research methods such as focus groups facilitated by community partners, co-working sessions of people from affected communities, or individual interviews can inform algorithmic problem formation (Opara et al., 2020). Scientists may choose to follow a model like Community-based System Dynamics or Artificial Intelligence in Community Citizen Science while incorporating other forms of community-centered and community-valued collaborative idea-generation practices. Community participation in problem formation represents an opportunity to practice shared decision-making. Next, scientists should flexibly experiment with algorithmic problem selection in collaboration with community partners. This allows for discovering whether the initial co-selected algorithmic problem is a valid proxy for the health or social challenge of interest. Such an approach mitigates the bias that can lead to an unintentionally discriminatory algorithm application in healthcare. Experimentation with problem formation is an opportunity to apply reflexivity and structural humility, adaptability, and flexibility.

During project: Algorithmic development

In machine learning, the training data is the data used to train an algorithmic model. Sharing training data and underlying algorithmic assumptions in an accessible way with community members is a practical opportunity to practice shared decision-making, reflexivity, and structural humility. Reflexivity involves actively examining ourselves as scientists and acknowledging our inherent biases and how they may impact the conclusions we draw from our work. Reflexivity is multifaceted and involves collaborative practices that challenge researchers’ subjectivity. For instance, a researcher who is being reflexive may ask, “Do racist stereotypes shape my beliefs about this population? Engaging in this reflexive work is critical since machine learning is inherently subjective due to the reliance on assumptions by the scientists who shape problem selection and identify training data for the algorithm (Cummings & Li, 2021). Cummings argues that biases in machine learning arise through the series of small choices scientists make when developing and choosing algorithms (Cummings & Li, 2021). Specifically, external biases may arise in the design and execution of a study. When these external biases interact with existing biases held by the researcher who is at the forefront of choosing algorithms to apply to a particular dataset, this can result in flawed conclusions (Cummings & Li, 2021). Thus, reflexivity is a critical component of community participatory machine learning. Structural humility builds upon the concept of structural competence (i.e., the ability to determine how existing systems of power have contributed to adverse health outcomes for marginalized communities) (Metzl & Hansen, 2014). Structural humility represents an evolution from the concepts of cultural competence (i.e., awareness of how one’s cultural values differ from those of others) and cultural humility (i.e., the ability to be curious and respectful about someone else’s cultural values and lived experience) (Samarasekera, 2022). In addition, structural humility acknowledges that there are systems in place that systematically disadvantage groups of people, which can lead to negative health outcomes (e.g., having access to purchase a home only in areas that have higher crime and pollution even when you can afford to live in a better neighborhood). It emphasizes that it is not merely cultural differences that need to be navigated when working with community members but rather truths of lived experience born from the structural positioning of various stakeholders with different relationships to oppression and power. Humility acknowledges that complete competence is likely unattainable due to differences in lived experiences and that engagement should be characterized primarily by curiosity (Samarasekera, 2022). This is especially critical in algorithmic development since algorithms can perpetuate existing structural racism (Obermeyer et al., 2019). For example, a researcher who embraces structural humility may ask participants or community members questions such as, “This algorithm suggests this conclusion. Does this seem true based on your experience? What might be contributing to this inconsistency found in the algorithm?”

Training data typically represents a fixed percentage of the overall data and the initial input used to teach a machine learning algorithm. This allows for the recognition of the patterns that the algorithm will eventually predict. Training data is regarded as one of the most essential aspects of machine learning algorithm development because low-quality data can negatively impact the algorithm’s accuracy. Scientists might employ “member checking” as part of their accessible sharing of training data and resulting intermediate algorithmic outputs. Member checking is a strategy commonly employed in qualitative research that allows researchers to share emerging patterns and preliminary results while still collecting and analyzing data (Candela, 2019). Member checking strengthens the validity of one’s findings (Candela, 2019). Summarizing community responses and qualitative findings to intermediate algorithm outputs from training data represents an opportunity to apply principles two and three. By incorporating reflexivity, structural humility, flexibility, and adaptability, member checking in machine learning ensures that the “truth” described by the algorithm resonates with community perspectives.

Project conclusion

At the conclusion of the research, developers should test the algorithm in a trial setting before widespread deployment to check for racial and other biases. The ideal trial setting would be one where the results of algorithm application do not yet have a binding effect on any health decision-making. Within this stage, reflexivity should be employed again to consider the possible harmful effects of how the algorithm may be applied. Essential questions about possible bias include: Would members of a structurally marginalized group be systematically disadvantaged if the algorithm was applied as is? Because much of the data that will inform algorithm development is inherently flawed, anticipating racism and other forms of bias may allow researchers to recalibrate before widespread application. Recalibration might entail re-training the algorithm with more ethno-racially diverse data, for example. Finally, embracing structural humility, scientists should request feedback from community members, such as in public community forums (e.g., town halls) and community advisory boards, to inform meaningful additional potential applications of the developed algorithm, follow-up studies, or related opportunities for further community engagement.

Application of the community-engaged participatory machine learning model: How this might work using a hypothetical example focused on addressing adolescent substance use.

Imagine that a team of researchers has identified that substance use is a significant problem in a particular community and that the application of machine learning to a combination of data sources (publicly existing data, geometric data) may provide a potential solution to identify adolescents at risk for substance use to connect them to treatment.

Stage 1: Project conception and problem formation

As detailed in our model, the first step would be to engage community stakeholders, including parents, adolescents, community leaders, and researchers, to clarify if this is also a problem that the community identifies and wants to address. At this stage, researchers must be open to the possibility that community members have other problems that they prefer to address or that community members may be interested in a different aspect of substance use (e.g., not the identification of adolescents at risk but perhaps areas in the community where substance use may be more prevalent). This collaboration with the community reflects shared decision-making critical to community engagement in problem-solving. Researchers being open to the possibility of the problem being modified reflects flexibility and adaptability and exercises structural humility as it acknowledges that community members are experts in their lived experiences and, therefore, know their most salient health issues.

Stage 2: Algorithm development

In this hypothetical scenario, the next step would involve researchers collaborating with community members to identify suitable data sources for training and testing the algorithms. Throughout this phase of data preparation, it is crucial for researchers to engage and inform community members at every stage of the process. Community members may possess valuable insights into potential biases in collecting certain data that researchers should address when selecting appropriate models.

For instance, community members may be aware that one of the data sources being considered includes only adolescents with private health insurance. This information will be crucial for researchers to be aware of when finalizing data sources to use. Furthermore, researchers must determine which variables or features from existing data sources should be included in algorithm testing. This decision should be informed by existing theoretical frameworks and careful consideration of potential biases. The selection of variables for a machine learning algorithm can introduce bias, even if researchers believe their choices are grounded in empirical evidence. For example, including family history of incarceration in the model may inadvertently skew results towards identifying more Black adolescents in an urban community. This is due to systemic racial discrimination and over-policing in urban cities, leading to higher incarceration rates among Black individuals (Jackson et al., 2023). While this inclusion may hold scientific relevance, researchers must remain aware of its potential impact on bias within the model and be prepared to address it.

Without input from the community, researchers risk overlooking the historical and structural implications associated with certain variable selections. Therefore, it is essential to review chosen variables and features alongside community stakeholders. Researchers should explain the rationale behind their selections and be receptive to questions and concerns raised by community members. This enables researchers to gain insight into historical contexts that may influence these perspectives ultimately leading to higher quality, more reliable algorithmic models.

Additionally, researchers should articulate why they have opted for specific machine learning algorithms to analyze the data. This transparency fosters trust and understanding between researchers and community stakeholders, while also encouraging researchers to critically evaluate their own assumptions and biases, thus ensuring structural humility.

Stage 3: Project conclusion

After developing the algorithm, researchers need to scrutinize which variables/features most significantly influenced its performance and assess whether the model performs consistently across different racial/ethnic groups within the community. For instance, if the algorithm shows a disparity in identifying at-risk adolescents between White and Black populations, researchers must investigate the underlying reasons and modify the model to ensure health equity.

Following a comprehensive review and consensus among researchers and community members, and incorporating input from community stakeholders, the model should undergo retesting with new data sources to ensure its consistency and alignment with the community’s experiences. Although this process may pose challenges due to the time involved, it represents a genuine commitment to community engagement throughout all stages of algorithm development and testing, thereby minimizing potential harm to the community that researchers aim to serve. By implementing a community advisory board, for example, community members can have the opportunity to refine, expand, and customize the model according to their evolving needs. For example, advanced artificial intelligence tools like ChatGPT undergo continuous updates and revisions to cater to various scales of applications. This highlights the potential for community-developed models to undergo similar adaptations to cater to diverse needs (Ali et al., 2023; Ray 2023).

Limitations

Active involvement of community members in the development and implementation of machine learning models can be perceived as challenging by researchers due to concerns about potential difficulties in engaging the community. This may lead to a reluctance to commit to such involvement, especially considering the time constraints faced by community members due to other obligations. However, historical context teaches us that excluding community members from research projects that directly affect them leads to a breakdown of trust. This, in turn, decreases the likelihood of community members utilizing the resulting programs.

Moreover, programs that exclude community participation may inadvertently or intentionally harm the very individuals they aim to serve. Therefore, it is imperative to reconsider our approaches to fostering community engagement in machine learning research.

Utilizing a community-engaged approach in algorithm development may present challenges in terms of scalability, as it may hinder the generalizability of certain details. However, it’s crucial to weigh this limitation against the benefits of addressing community-specific needs in a way that fosters trust and active community involvement. Each community possesses its own unique needs and demographics, which may pose challenges for the widespread application of a community-developed machine learning model. Nevertheless, a significant advantage of the community-engaged process lies in its ability to tailor and personalize algorithms to suit the specific requirements of each community.

Conclusion

Including stakeholders who represent the population targeted by an algorithm throughout all stages of machine learning for community health and psychology research offers a chance to develop algorithms that are not only highly effective but also ethically sound, reflecting the real experiences of the communities involved. We emphasize the importance of grounding priority setting, problem formation, model assumptions, and interpretation of algorithmic patterns in the insights derived from the lived experiences of community members directly affected by the issues. By fostering meaningful community participation, we can mitigate algorithmic bias by empowering members of structurally marginalized and historically underrepresented communities to act as authorities on their own experiences.