Incorporating end-user perspectives into the development of a machine learning algorithm for first time perinatal depression prediction

Abstract

Objective

Machine learning algorithms can advance clinical care, including identifying mental health conditions. These algorithms are often developed without considering the perspectives of the affected populations. This study describes the process of incorporating end-user perspectives into the development and implementation planning of a prediction algorithm for new perinatal depression onset.

Materials and Methods

A focus group (N = 12 providers) and four virtual community engagement studios (N = 21 patients) were conducted. The project team presented on the initial development of a novel prediction algorithm used to detect first time perinatal depression. Rapid qualitative analysis coded the prediction algorithm’s completeness, interpretability, and acceptability to stakeholders, with the goal of informing clinical implementation of a patient-facing screener produced from the prediction algorithm.

Results

Providers and patients showed consensus on the interpretability of the prediction algorithm’s variables and discussed additional variables believed to be predictive of depression to ensure its completeness. In terms of acceptability, patients expressed a desire to discuss predictive risk screening results with their provider, while providers voiced concerns about limited bandwidth for these discussions. Both groups identified the need for post-screening resource connection but raised concerns over the availability of depression prevention specific resources. Providers and patients reported positively about their engagement in the sessions.

Discussion

Qualitative findings were incorporated into iterative algorithm development and informed an implementation pilot plan.

Conclusion

This study demonstrates how the expertise of the end-users of a risk prediction algorithm can be incorporated into its development, which may ultimately increase clinical adoption.

Background and significance

Prediction algorithms that use machine learning (ML) continue to advance in health-related fields, including the effective prediction and/or detection of physical and mental health conditions.¹ As these prediction algorithms are tested and implemented in real-world healthcare settings, careful attention to privacy, transparency, generalizability, interpretability, and equity is required.^2–4 Prediction algorithms developed without these guiding constructs risk perpetuating inequities in healthcare^5,6 or low adoption, as healthcare providers, patients and/or systems may find the algorithm output irrelevant.⁷

Over the past decade, several frameworks, policies, and checklists have been created with the goal of defining artificial intelligence/ML ethics,^3,8 with increased emphasis on engaging end-users during algorithm validation testing and implementation.^9,10 Further, some professional organizations, such as the American Medical Association (AMA),¹¹ have outlined policies/principles for the development and use of such algorithms in healthcare. For example, the AMA encourages healthcare AI design and implementation to be transparent and “in keeping with best practice in user-centered design.”¹² These guidelines aim to ensure that the algorithms being implemented are safe and effective, designed in conjunction with end-users, and protect end-users from undue harm.

While there have been calls to engage multidisciplinary teams and perspectives in ML research, more broadly, and holistic frameworks exist that take theory and ethical principles into account,^2,10,13 little guidance has been given on how to practically incorporate end-user (eg, provider and/or patient) perspectives into prediction algorithm development and implementation planning. The incorporation of these perspectives is essential for responsible algorithm development and for assessing algorithmic relevance to healthcare providers and their patients.

Research and evaluation methods from social science disciplines (eg, qualitative, community-participatory, and human-centered design activities) have been used to assess organizational needs and capacity for change,¹⁴ healthcare outcome priorities,¹⁵ patient and provider motivations,¹⁶ shared decision making,¹⁷ barriers/facilitators to efficient clinical workflows,^16,18 and to amplify the voices of marginalized communities in the development of health interventions.¹⁹ Moreover, these methods often form the foundation for developing, implementing, and scaling healthcare-related interventions.^14,20,21 There is also precedent for using such methods to offer necessary context for algorithm validity and implementation. For example, qualitative interviews have been utilized to explore algorithmic fairness, accountability, and acceptability.^22–24 Furthermore, human-centered design strategies are increasingly being used, across a variety of disciplines, to support the development and implementation of psychometric measures and clinical algorithms.^14,25–27

For this work, we define perinatal depression as depression occurring during pregnancy or up to six months after birth.²⁸ Perinatal depression is a common pregnancy-related complication and is the leading cause of maternal mortality in the US.²⁹ As a result, there is a need for screening tools that can be administered early in pregnancy—ideally at one timepoint—to identify those who are likely to develop depressive symptoms at a future stage of pregnancy. Established perinatal depression screening tools^30,31 are designed to identify individuals with active symptoms of depression,^32,33 and do not capture underlying risk factors that may be associated with future depression/symptom onset.^34,35 In order to effectively deliver targeted preventive care, professional organizations have called for the development of new tools that can identify individuals at risk of depression among those who are not currently symptomatic.^36,37 The success of those tools will ultimately be defined by their adoption into clinical workflow and their acceptability to both clinicians and the patients they serve.

Objective

Using patient-reported data from early pregnancy, we designed a prediction algorithm to identify patients at risk of developing first time moderate-to-severe depression in their second, third or fourth trimesters of pregnancy.³⁸ Details of the data collection process and the technical aspects of developing the prediction algorithm have been published elsewhere.³⁹

Here, we describe the approach used to incorporate provider and patient feedback into the prediction algorithm’s development to enhance its relevance as well as its usability prior to implementation in routine clinical practice. Figure 1 outlines the entire process of the predictive algorithms development process, from the technical build to translating it into a clinical screener for future depression risk.

Figure 1.

This figure shows the stakeholder engaged process of developing and implementing a perinatal mental health risk prediction algorithm into clinical care.

The prediction algorithm was developed using self-report patient data collected through an evidence-based prenatal support smartphone app, MyHealthyPregnancy. The MyHealthyPregnancy app was designed to identify and intervene on pregnancy-related risk factors in between routine prenatal care visits.^39–45 The app was deployed to approximately 12,000 UPMC prenatal patients from 2019 to 2022. Through the app's electronic consent process, users agreed to share identifiable data with their healthcare providers for both care and research purposes, as well as anonymized aggregate data for scientific development.

The apps data capture features were guided by a human-centered design process with clinicians, perinatal individuals, and community organizations.⁴⁰ The app’s features, including types of data captured, were intentionally designed to identify a range of potential pregnancy-related risks, including preeclampsia, interpersonal violence, preterm labor, and depression (Figure 1, phase 1). App data predominantly fell into three categories: medical history (eg, prior miscarriage, diabetes, first pregnancy), psychosocial/behavioral (eg, partner support, current drug use, alcohol use), and demographics (eg, relationship status, age, education level).

Using app data, the PC-KCI algorithm⁴⁶ identified the strongest, likely causal pathways that predict new perinatal depression, from a comparison of multiple ML prediction approaches (Figure 1, phase 2). PC-KCI combines two methods: the Stable PC-Algorithm⁴⁷ and Kernel-based Conditional Independence Tests (KCI⁴⁸). The PC-KCI algorithm considers both accuracy and simplicity in its design and is nonparametric in nature. As such, it is particularly well-suited for producing simple, interpretable models for clinical risk identification. The identified risk factors are then used as covariates in a generalized additive regression model,⁴⁹ generating a predictive algorithm grounded in causal discovery methods.

Here, we document the qualitative end-user feedback that was collected in tandem with the prediction algorithm’s technical development (Figure 1, phase 3) to produce a clinically implementable predictive perinatal depression risk screener (Figure 1, phase 4). The goal of gathering and integrating end-user feedback was to proactively identify and address potential barriers to real-world implementation and engagement. Understanding patient and provider perspectives on the role and meaning of each screener item is central for engagement and clinical practice. Further, patient comfort in responding to risk screener questions will likely affect their likelihood of providing a response. Therefore, in anticipation of implementing an algorithm-derived screener for early identification of first-time perinatal depression, we focused our qualitative feedback on addressing providers’ and patients’ perceptions of the algorithm’s completeness (whether the existing variables are adequate), interpretability (whether the existing variables reflect end-user understanding), and its general acceptability as a predictive measure of depression risk.

Materials and methods

Provider focus groups

Providers were engaged first, attending a 90-minute focus group in May 2023. A focus group approach, where participants can share their perspectives as well as hear and discuss other participants’ perspectives, was selected due to a high likelihood of divergent experiences which would benefit from group dialogue. Providers were recruited through word-of-mouth from the project team’s existing network. A purposeful sampling approach was employed to ensure involvement of obstetric, behavioral health, and family practice providers who are most likely to monitor and treat perinatal individuals for depression.

At the beginning of the focus group, the project team presented on how patient self-report data was originally collected and the output from the ML prediction algorithm (Figure 2A and B). First, providers learned about the development of the prediction algorithm, including data collection (Figure 2A). Subsequently, the project team explained the two highest performing prediction algorithms along with their corresponding sets of variables and prediction performances (Figure 2B). Next, the project team facilitated a discussion, using a semi-structured guide (Supplementary Material S1), focusing on the prediction algorithm which constituted the more parsimonious set of variables, inclusive of health-related social needs (ie, food insecurity—Figure 2B, bottom). Providers discussed (1) the comprehensiveness of the variables for depression prediction (completeness); (2) whether the existing variables were consistent with their clinical understanding of depression (interpretability); and (3) implementation challenges and possible solutions for embedding the prediction algorithm, in the form of a risk screener, into routine clinical care (acceptability).

Figure 2.

This figure presents how (A) patient self-report data was originally collected, (B) the output from the ML prediction algorithm, and (C) the prediction algorithm translated into a risk screener.

Expertise shared was deemed part of professional clinical service and not human subjects research by the University of Pittsburgh’s Institutional Review Board. As a result, no consent process was required, and monetary compensation was not provided for time spent on this professional activity.

Virtual community engagement studios with perinatal patients

Following the analysis of the provider focus group, the project team engaged a sample of patients to participate in one of four 90-minute Virtual Community Engagement Studios (VCES) held in July through September 2023. A VCES approach allows researchers to gather direct input and recommendations from members of a community.⁵⁰ Unlike traditional focus groups, VCES are an “interactive and consultative model” in which participants with lived experience (eg, patients) actively support the development of various phases of the research process.^51,52 Patients were recruited through the University of Pittsburgh’s national research registry as well as flyers and tabling at obstetrics and gynecology (OBGYN) offices in Pittsburgh, PA. Patients with and without a history of depression were sampled to offer insight into the experience of depression and to learn from those for whom the prediction algorithm might serve as a means of identifying first time depression risk.

Similar to the provider focus group, each VCES included a brief presentation of the project, an overview of the risk screener variables (Figure 2C), and a facilitated semi-structured discussion (Supplementary Material S2) about the risk screener and how best to implement its use in clinical settings. Participants were shown the translation of the prediction algorithm—a set of patient-facing risk screener questions (Figure 2C). Patients provided feedback on (1) the wording of the screener questions (interpretability); (2) how well screener questions fit with their own experience (completeness); and (3) on how the risk screener could best be implemented in a routine prenatal healthcare setting (acceptability).

Verbal participant consent was provided prior to engaging in the VCES. VCES participants were compensated $100 for their time.

Analysis

A project manager took detailed notes during the provider focus group and each VCES, and verbatim transcripts were generated via Microsoft TEAMs. The provider focus group transcript and notes were reviewed by two members of the project team (one researcher with qualitative analysis expertise and a project manager trained in qualitative data analysis). Guided by the research questions, the notes and transcript were summarized into discussion themes. The qualitative findings were discussed and interpreted by the full project team.⁵³

To generate timely and actionable implementation results, a rapid qualitative analysis approach was used to summarize VCES data.⁵⁴ A summary template was developed based on the main topics asked about during the VCES. To assess the usability and relevance of the summary template, two project team members (one researcher with qualitative analysis expertise and a project manager trained in rapid qualitative analysis) independently tested the template on the same transcript, summarizing relevant content and noting representative quotes. The transcript summaries were then compared, inconsistencies discussed, and the template modified as needed. After consistency was established (ie, project team members summarized similar content), each project team member was assigned two transcripts to complete the summary template. Summaries were viewed side-by-side and synthesized by project team members for unique and recurring opinions and to identify representative quotes across VCES.

Provider and VCES participant satisfaction surveys

At the completion of the provider focus group and each VCES, an anonymous survey was sent to participants, via MS Forms, to rate their satisfaction with their respective group. The survey included Likert-scale questions that were developed by the project team and were adapted from the Vanderbilt Institute for Clinical and Translational Research Community Engagement Studio Community Expert Evaluation Survey.⁵⁵

Study procedures were approved by the University of Pittsburgh’s Institutional Review Board (STUDY23010085). All research activities were funded by a grant from the National Institute of Mental Health (5R34MH130950).

Results

Provider focus group participants (N = 12) included physicians (n = 3); social workers (n = 2); OBGYNs (n = 2); a registered nurse (n = 1); a certified nurse midwife (n = 1); a perinatal psychiatrist (n = 1); a maternity department administrator (n = 1); and a behavioral health therapist (n = 1). Focus group participants were primarily female (64%) and white (82%). Two (16%) were Asian.

VCES patient participants (N = 21) included individuals who were currently pregnant (n = 12) or within 6 months postpartum (n = 9). Of participants who provided a history of depression (n = 14), 64% reported having a history of depression (perinatal or otherwise). See Table S1 for VCES participant demographics.

Results from both the provider focus group and the patient VCES pertain to the variables from the preferred prediction model shown to providers in its nine-variable graphical form (Figure 2B, bottom) and shown to VCES participants in the form of a set of nine depression risk screener questions (Figure 2C). Table 1 shows representative quotes across data collection techniques.

Table 1.

Exemplar quotes from Provider Focus Groups and VCES

Theme: Algorithm Interpretability
“Trauma background, and then also…emotional coping skills and problem-solving skills are implicated in suicide and part of the Zero Suicide Prevention models. So, you know, one wonders, aside from just the stressors, the sort of intrinsic traits that women might have that help them respond to their environment”	Behavioral Health Provider
“They’re all aspects of your life like your relationship and money, and taken as parts […] may not make you stressed out, or each piece may make you stressed out, […] which might add to your depression, and I think that’s why they’re all mentioned. […] They’re all key pieces of the puzzle that makes you depressed.”	VCES 1 Participant
Theme: Algorithm Completeness
“I will say, ‘unusually,’ did strike me as a maybe not the right word, but…I don’t know what the right one would be. Maybe ‘more stressed than usual’, that’s all I could come up with, too. But, I just feel it’s hard because we all have different baselines, and I get that. ‘Unusually,’ just for some reason it doesn’t resonate with me either.”	VCES 3 Participant
“I’m actually in recovery right now and I’m gonna speak from experience. Whenever I was pregnant, I was using and I was afraid to get clean, but I was also afraid to harm my baby. So I went on a maintenance program and did like a Subutex conversion. […]I just got done with the conversion and three days later I gave birth, so everything was still in my system. So obviously CYS has gotten involved, but yeah, that was a huge contributor to stress and my mental health, because I’m always constantly thinking in the back of my head, ’Are they gonna take my child? Am I gonna get better?’”	VCES 4 Participant
Theme: Algorithm Acceptability
“A couple extra minutes of questions, that this is literally going to increase the amount of screening questions that an MA is asking maybe by 50. It’s a lot [of] extra work. And so, my argument would be if we could ask about ‘feeling blue,’ ‘financial stress,’ ‘health issues,’ and you get an AUC of .89 that’s pragmatic. And you might have to take away [concerns about] ‘physical appearance’…Prioritize. Be even more parsimonious.”	Obstetric Provider
“You know, if someone’s afraid that CPS is going to come and take their kid, if they answer that they’re financially unstable or they don’t have access to resources, but instead said, ’would you like to learn more about […] this program that can provide food or this program that can provide rent assistance?’”	VCES 2 Participant
“So, with them [OB/midwife] I would have been fine with being just interviewed. I think definitely if it was like an iPad or app or questionnaire…that would be a lot easier to answer it that way because I wouldn’t feel there would be the potential to feel judged to where with a couple of the doctors, I did sometimes feel judged.”	VCES 4 Participant
“Prenatal and postnatal depression- it can get really scary and its[the] kind of things that people don’t feel comfortable talking about. So, I think it’s a great way to set up the conversation around, you know, ‘it’s OK to feel this way. It’s normal.’ I can see how some people might be like, ‘oh, God, that’s just one more thing that I’m now going to be stressed and anxious about is that I might get this.’ But I think it’s important to have the conversations that women don’t try to go through that alone…”	VCES 3 Participant
“‘We’re not seeing signs of you [being] depressed yet, but you [have] the markers. They could possibly mean you could get depressed in the coming weeks/months with the added stress of having a baby. So, I just wanna make sure you’re OK and you have the help you need.’ [You] may want to start looking into this now, So that way if you do have to wait months to get help, you know it’s help is coming and you’re at least [getting] your foot in the door to talk to people you need to talk to.’”	VCES 1 Participant

Interpretability

Providers and patients considered all nine prediction algorithm variables as relevant to identifying risk of perinatal depression, citing literature and/or a scientific foundation to support the inclusion of each variable. However, patients focused their discussion primarily on seven of the nine variables which they felt to be most relevant to identifying perinatal depression risk, identifying that “feeling blue” and “anxiety,” while important to measure, were manifestations of the experiences of the other seven variables. Table 2 shows a thematic summary of variables from the prediction algorithm that were discussed in-depth.

Table 2.

Summary of Virtual Community Engagement Studio (VCES) feedback by algorithm variables discussed in-depth.

Variable construct(s)	Feedback summary
Labor & Delivery	Perceived to be related to negative experiences with a past pregnancy (eg, a complex pregnancy or postpartum depression).
Money	Perceived (across all VCES) as being “magnified” during and after pregnancy due to personal stress about one’s own health/wellbeing (ie, the pregnant individual) as well as concern over how these challenges could impact the baby/their ability to provide for the baby.
Food Security
Health Issues
Growing Up (ie, resilience developed in childhood)	Perceived (by at least one person in three VCES) as only being relevant if people hold onto the past. The impact of a person’s childhood could influence personal expectations/concerns going into parenthood, which could be a contributing factor for depression development.
Relationships	Perceived as extremely relevant (across three VCES) as babies can put unexpected strains on relationships. Several participants noted that while relationship stressors were not personally relevant, they could be for others.
Physical Appearance	Perceived as a variable that’s connection to depression is not immediately obvious for all individual group members (across two VCES) but, after group discussion, there was agreement that it could be a contributing factor to depression for others. Note: In the same VCES, other participants discussed how this variable negatively impacted their wellbeing, especially considering unexpected/unwanted body changes that resulted from pregnancy.

Completeness

Providers and VCES participants also raised potential depression predictors not represented in the prediction algorithm. For example, multiple providers identified first pregnancy and other social determinants of health variables (eg, transportation) as known risk factors for depression. Other risk factors raised by providers included: (1) history of, or current, intimate partner violence (IPV); (2) history of or current substance use disorder (SUD); and (3) coping skills.

VCES participants similarly raised SUD and/or other mental health conditions as potential contributing factors to developing depression, among several other factors not discussed by providers: (1) social support; (2) emotional support; (3) having other children and related experiences, such as breastfeeding complications; (4) history of miscarriage; (5) concern over current events; (6) challenges faced during pregnancy (eg, having to take off work) due to pregnancy symptoms (eg, morning sickness); and (7) baby’s health prior to delivery.

VCES participants additionally offered feedback on how to phrase the risk screening questions to ensure appropriate inclusion of the variable being measured. For example, each VCES organically discussed the use of the word “unusually” as a modifier in four of the nine variables (ie, Relationships; Money; Physical Appearance; Labor/Delivery). This term was viewed as being too subjective—posing a challenge in accurately responding to the questions. Specifically, participants noted everyone has a different baseline, making it challenging to systematically define what “usual” is. VCES participants suggested several ways to modify these questions: (1) remove the word “unusually”; (2) create a rating scale; (3) ask “how are you feeling about…”; (4) provide a free text box for response elaboration; and (5) rephrase questions in terms of frequency—eg, “how often do you feel stressed about financial stability?.”

VCES participants also believed it would be easier to complete the risk screener if variables that they perceived as capturing the same construct were asked about together (eg, having the Money question immediately followed by the Food Security question). See Table 3 for VCES participant feedback on how to adapt each screening question for clarity.

Table 3.

Summary of VCES participant feedback on clarity of risk screening questions.

Variable Constructs	Feedback Summary
Feeling ‘Blue’	Unclear why the words “diagnosed” and/or “depression” are not included since they are included in the ‘Anxiety’ question. Unclear if the individual answering the question should respond based on how they feel when taking the screener or based on other points in time.
Anxiety	Unclear as to what the word “history” means in the context of this variable question—some participants wondered if it referred to past experiences with anxiety or if it was a past official diagnosis of anxiety. The word “diagnosis” was viewed by some participants as prohibitive to individuals who had not received an official diagnosis of anxiety. Suggestions to improve clarity included: (1) remove the word “history”; (2) remove the word “diagnosis”; (3) rephrase to “having current feelings of anxiety.”
Money	Participants believed this question lacked specificity and therefore might be difficult to answer. It was noted that having specific examples (or checkboxes) embedded in the question would be preferable. Examples of specificity include (1) worrying over debt; (2) not being able to provide for the family; (3) not being able to pay for childcare; and (4) not being able to pay for doctor visits/hospital stays. It was suggested to combine Food Security with Money.
Food Security	Unclear if this variable question encompasses breastfeeding/milk supply concerns and/or formula concerns such as national shortages.
Growing Up	Suggestions to improve clarity included: (1) ask directly if there is a history of childhood trauma; (2) ask the questions in reverse—for example, ask if they had someone to support them when they were growing up; and (3) ask one of the questions, but not all as most people will likely answer the questions in the same way they answer the first.
Health Issues Stress; Physical Appearance; Labor/Deliver	Unclear if the Health Issues Stress variable question is asking about new or ongoing health issues that coincided with pregnancy. Further, it was not apparent to some participants what health issues would be relevant—ones that could impact the pregnancy/baby or ones that impact the mom but not necessarily the pregnancy/baby (ie, arthritis). Suggestion to clarify by making the question less specific by focusing on Health Issues Stress experienced “while you are pregnant”. Unclear as to how the Physical Appearance variable question is different from the Health Issues Stress variable question—seeing both in a screener may confuse people in terms of how to respond. Several participants interpreted the Physical Appearance variable question to be asking about issues that would happen during pregnancy including stretch marks or gestational diabetes. One participant shared they liked the room for interpretation in both the Physical Appearance and the Labor/Delivery variable questions.
Relationships	Unclear who counted as a “relationship,” with some participants inferring that it was about the baby’s father and/or a romantic partner, and others believing it included family and/or other individuals.

Acceptability

Providers discussed concerns around the feasibility of adding additional data collection in the form of a risk prediction screener into their existing workflow, primarily due to perinatal visit time constraints. Providers raised questions on how data collection would fit into existing clinical processes, specifically how screening for future depression risk might aid or conflict with existing screening workflows (eg, routine depression screening or administering the health-related social needs questionnaire, in which there is some topical overlap with the depression risk screening). To reduce data collection burden, providers proposed using routinely collected data as proxies for a subset of the algorithm variables. Providers stated that they would be willing to sacrifice some prediction accuracy in favor of reducing the number of patient-report screening questions.

VCES participants explored three key topics related to screening implementation: (1) the importance of knowing your risk prediction score; (2) comfort levels when disclosing information asked in the screener; and (3) preferred modes of administration. Almost all VCES participants conveyed the importance of informing pregnant individuals about their risk for developing depression. Participants believed this knowledge would facilitate: (1) the normalization of depression as a healthcare topic; (2) the provision of education on depression symptoms and/or “warning signs”; and (3) the opportunity to take preventative measures.

On the other hand, VCES participants also noted that risk results could induce patient anxiety and fear; especially without clear resources being discussed and/or without education around postpartum depression. VCES participants further voiced concerns that risk screener results may cause a provider to have a negative perception of the patient and/or their parenting ability. They also discussed several reasons they may be uncomfortable with or unwilling to share information solicited by the screener: (1) not trusting their provider; (2) not feeling listened to; (3) not feeling the provider has an interest in their mental health or general wellbeing; or (4) the provider being new to them.

To partially alleviate the above concerns, VCES participants discussed several strategies that could increase patient engagement with the risk screener. These strategies include pre- and post-screening talking points providers could incorporate into their communication with patients (provider-patient communication recommendations are illustrated in Figure 3). Several participants also explained that administering the screener via a tablet/paper packet in-office or completing the screener electronically, prior to a visit, could decrease the risk of patients feeling stigma or judgment as they would not need to engage in a direct conversation to provide their responses. Alternatively, other participants noted that it would be helpful to talk through the questions/responses with the provider so the provider could learn more about their specific situation and provide tailored care based on the individual’s unique experiences.

Figure 3.

This figure presents a summary of qualitative feedback translated into pre- and post-screening talking points providers could incorporate into their communication with patients.

Post-screening resources

Both providers and VCES participants expressed concerns about screening for depression risk without having a clear and defined patient follow-up and treatment strategy. Providers discussed how the follow-up process should be embedded within current clinical workflows, thereby increasing the likelihood of sustainability. At the same time, providers expressed concerns about inundating patients with too much information, which could make following-up more difficult. Table 4 shows representative quotes.

Table 4.

Additional exemplar quotes from Provider Focus Groups and VCES.

Theme: Post-Screening Resources
“Ideally, in my opinion, having an integrative practice with a behavioral health specialist or a care coordinator, social worker, embedded in the office, is the way to go… the person screening, the physician seeing the patient could do a warm handoff. You know, having that access to that level of professional at the time in which is needed really engages the patient far better, creates less stigma, and does a variety of things to get the patient more embedded into the treatment that they need for their anxiety or depression. I mean, it may be just education about, postpartum depression in the future or what to look for signs, symptoms, that kind of stuff done by a behavioral health [provider].”	Obstetric Provider
“Like what is your OB going to do about the fact that you might have financial stress, but you know, I was thinking about the ones where maybe you could take baby steps. Physical appearance—OK, if that's a big stressor for you, we have this list of cookbooks that are great for pregnant women that we recommend. Like ‘recommended things’ down to Instagram accounts.”	VCES 3 Participant
“You know if the goal here is to facilitate pregnant women getting the supports they need to prevent postpartum depression, those doctors and midwives having some type of link to therapists who are trained in postpartum depression and are available to take on the person quickly—rather than, you know, you having to find somebody that has an opening. That can take a while, and a lot of people might decide I try and I couldn't find anybody.”	VCES 4 Participant
“I think it's aways great to offer support groups because sometimes it's just easier to hear from other people that have gone through it than it is a professional sometimes, who may or may not have experienced it themselves.”	VCES 1 Participant

For any resource offered, VCES participants strongly voiced that providers should ensure the resources offered are available, accessible, and useful. Participants suggested that provider offices could reduce patient burden by providing contact information for local resources, directly connecting the patient to the recommended resource, and supporting the patient in finding out if a recommended resource is covered by insurance and/or can be accessed during non-traditional work hours. Providers and VCES participants identified several resources that may be helpful to those who screen positive for being at risk for first-time perinatal depression as shown in Figure 4.

Figure 4.

This figure presents a Venn Diagram of provider and VCES identified resources for those at risk for perinatal depression.

Provider and VCES participant satisfaction surveys

All but one provider (n = 11, 92%) chose to complete an anonymous online evaluation survey. All 11 providers (100%) reported feeling that they were qualified to contribute to the discussion, had the chance to share their perspective, and that their contribution was taken into consideration during the focus group. 10 of the 11 providers (91%) reported feeling that the research team sufficiently explained the goal of the focus group, the technical development process of the prediction algorithm, and how their input would be used in the iterative modeling work. When asked about the interpretability and potential bias of the algorithm, seven of the 11 providers (64%) felt that it did not take a lot of effort for them to understand the prediction algorithm. Nine of the 11 providers (82%) believed that the prediction algorithm adequately considered the needs of patients facing systemic bias in healthcare. All 11 providers (100%) agreed that the use of the prediction algorithm has the potential to improve quality of care.

VCES participants were provided with an online evaluation survey upon completion of each VCES to measure their experience and engagement in the process and 20 participants (95%) completed it. All 20 participants reported feeling the scheduling process was efficient, that they were sufficiently prepared for the VCES, and that the VCES facilitators allotted sufficient time to adequately address questions/concerns. One participant felt that there was not enough time allocated for the VCES. When asked about specific contributions to the research project, participants felt they adequately provided ideas on (1) research procedures (n = 11; 55%); (2) informing the community (n = 7; 35%); and (3) how to use this research to benefit the community (n = 10; 50%). Additionally, participants felt they provided feedback on project feasibility (n = 9; 45%) and increased the researchers’ sensitivity to the community (n = 7; 35%).

Triangulation of feedback for implementation planning

The project team reviewed the focus group and VCES analyses to assess what potential algorithm changes may be warranted based on the end-user feedback. A systematic approach to model re-assessment was taken, in which each suggested variable, or potential proxy for a suggested variable, was explored for its presence or absence in the data set. Suggested variables missing from the model were added, and those already present in the model were assessed at their point of model drop-out. There was consensus across providers and patients on three variables (eg, miscarriage history, current or previous SUD, and IPV) that they perceived to be relevant to onset of depression and that could be added to the prediction algorithm because they were measured in the original data set.

First, when IPV was included in the model, it was uniquely associated with depression risk. However, retaining IPV in the model did not improve the algorithm’s overall ability to predict depression (as measured by AUC, sensitivity, and specificity). Retaining the IPV variable also increased the total number of necessary variables for prediction. Next, miscarriage history was assessed. This variable was marginally independent, meaning it was not directly associated with depression risk and therefore was not retained by the algorithm. When added to the algorithm, current substance use was also not retained as predictive of depression. However, previous substance use, or SUD, was not explicitly measured in this data set. Since these variables did not enhance the prediction ability of the algorithm nor reduce patient burden, and providers expressed preference for fewer screener questions/data collection points (even at the expense of prediction accuracy), the project team decided not to update the existing risk-screener to include IPV or the two other variables (see Figure 1, phase 3).

Next, using results from the focus group and VCES, the project team developed a concept poster, a human-centered design method,⁵⁶ that summarizes the purpose and importance of a prediction algorithm for first time perinatal depression and outlines how such a tool could be integrated into clinical workflows. Figure 5 shows the final concept poster illustrating the stakeholder-informed workflow for implementing the screening tool in clinical practice. This tool will be used to elicit feedback from health system leaders and implementation staff, and to gain buy-in for wide-spread implementation.⁵⁷

Figure 5.

This figure shows a concept poster of a workflow, informed by end-users, for implementing the screening tool in clinical practice.

Discussion

This study demonstrates the use of qualitative and human-centered design methods to collect and incorporate end-user perspectives in the development of a clinical prediction algorithm that will ultimately be implemented as a first-time depression risk screener in routine prenatal care among those with no active depression symptoms. Providers and patients believed the variables in the risk prediction algorithm (and algorithm-derived risk screener) were relevant to identifying risk of first-time perinatal depression. Both providers and patients discussed other variables they believed to be relevant to include in the prediction algorithm, including miscarriage history, history of SUD, and exposure to IPV. Subsequently the project team assessed the potential of including those three variables in the prediction algorithm. Ultimately, the variables were not included in the final model/risk screener as they did not (1) improve model accuracy, and (2) would increase patient and/or provider data collection burden. Even though explicit changes to the model did not occur, the process outlined in this paper for working with providers and patients to design and refine a prediction algorithm is replicable and essential to ensure relevance (completeness), acceptability, and utility.

Providers and patients also discussed several variables (eg, social support, emotional support, post-delivery complications, impacts of current events, and work-related constraints on healthcare access) they believed to be relevant to identifying risk of perinatal depression that were not collected through the original data sets used to create the algorithm. The project team plans to work with health system partners to collect this data and once collected, reassess the algorithm. Additionally, the project team will compare the original screener to a patient-centered revised version, which incorporates phrasing suggested by VCES participants. By randomizing pregnant individuals in their first trimester to receive the original screener or the patient-centered revision, the team will be able to measure differences in uptake and acceptability of the screeners, as well as evaluate the depression prediction ability of the patient-centered revision.

Some providers queried whether data already collected through the electronic health record (EHR) could serve as a proxy for variables included in the screener. However, it should be noted that no sufficient proxy variables for these specific items are currently available in the EHR.

Both providers and patients offered feedback on how to best integrate and implement the risk screener into routine care. While acknowledging the potential benefit of early screening for first time perinatal depression risk, providers discussed the importance of ensuring that screener implementation does not interfere with existing pregnancy health data collection efforts and voiced concerns with how an additional screener would impact their already time-constrained visits. These concerns are well documented barriers to screening tool implementation in perinatal care settings.⁵⁸ Further, a multitude of screening implementation solutions/best practices have been previously reported,^58,59 including the incorporation of the screening tool into the existing workflow. Considering both provider workflow integration concerns and patient willingness to complete risk screener questions electronically, further exploration into how to optimally deploy the screener is warranted. Currently, there is limited literature evaluating mental health screening completion by administration type (ie, in-person, electronic, before or during a perinatal clinical visit).^38,60 Ongoing work by our team is testing the feasibility of delivering both the screening questions and a responsive set of resources directly to patients through our electronic health portal.

Conversely, patients believed the risk screener would facilitate increased dialogue around perinatal depression, with several voicing concerns of stigma. To that end, they discussed the importance of pre- and post-screening conversations between providers and patients. As preferences for level of communication around the risk screener is disparate between providers and patients, a deeper understanding and testing of the risk screener within a clinical workflow, including a focus on provider-patient communication is needed.

Lastly, both providers and patients believe that the risk screener is only valuable if there is a clear, and practical follow-up and/or treatment strategy; another common barrier to implementing mental health screening tools.^38,58 It would be unethical to routinely screen individuals at risk for mental health conditions if no resources are available to address that risk. Next steps in our research include understanding what resources are available (and which are not) within our healthcare system to provide perinatal patients who are at risk for developing depression. Our goal is to assess a suite of tailored preventive care resources that would meet the specific needs of a patient, as identified by the risk prediction algorithm. While there is an already existing strain on the current US behavioral health system to meet the needs of those who are actively depressed, we anticipate intervening prior to depression onset may ultimately decrease the stress on system resources.

Limitations

The generalizability of end-user feedback may be limited due to the nature of our provider sample (ie, one focus group restricted to individuals from within our healthcare setting) and the VCES method (ie, patients were advisors informing on development and implementation verses research subjects). As we scale the implementation of the risk prediction algorithm/risk screener, we will continue to employ standardization methods for measure development (eg, cognitive interviews, administration, tests of dimensionality).⁶¹ Relatedly, it is likely that our single provider focus group did not reach thematic saturation. To gain further insights into the challenges of integrating the screener into a specific clinical workflow and ideate solutions, even within our own healthcare system, it would be beneficial to complete additional provider focus groups and observation visits with specific provider offices or team members (eg, doulas and care managers).

Our use of existing data (collected with the MyHealthyPregnancy app) to develop our prediction algorithm limited our ability to test the influence of certain variables identified by providers and VCES participants on the predictability of perinatal depression.

Conclusion

The use of social science methodologies to inform all stages of algorithm development and implementation can provide critical insights into the design and integration of prediction algorithms in healthcare settings—ultimately increasing clinical adoption. Our approach to incorporating end-user perspectives provides a tangible example for others who may be engaging in clinical algorithm development work. We offer details of a process that considers end-user driven revisions to prediction algorithm inputs and informs the design of an implementation strategy for testing the algorithm-produced risk screener in clinical practice.

Author contributions

Conceptualization: Kelly Williams, Cara Nikolajski, Tamar Krishnamurti. Methodology: All authors. Formal analysis: Kelly Williams, Samantha Rodriguez, Elaine Kwok. Project administration: Samantha Rodriguez, Elaine Kwok. Writing - original draft: Kelly Williams, Tamar Krishnamurti. Writing - review & editing: All authors. Acquisition of funding: Tamar Krishnamurti.

Supplementary material

Supplementary material is available at Journal of the American Medical Informatics Association online.

Funding

This study was supported by a grant from the National Institute of Mental Health [R34 MH130950].

Conflicts of interest

Authors T.K. and H.S. hold equity ownership in Naima Health LLC. The remaining authors have no relevant financial or non-financial interests to disclose.

Data availability

Deidentified data may be made available for academic research purposes upon request.