Abstract
Early life adversity can significantly impact child development and health outcomes throughout the life course. With the COVID-19 pandemic exacerbating preexisting and introducing new sources of toxic stress, social programs that foster resilience are more necessary now than ever. The Helping Us Grow Stronger (HUGS/Abrazos) program fills a crucial need for protective buffers during the COVID-19 pandemic, which has escalated toxic stressors affecting pregnant women and families with young children. HUGS/Abrazos combines patient navigation, behavioral health support, and innovative tools to ameliorate these heightened toxic stressors. We used a mixed-methods approach, guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework, to evaluate the implementation of the HUGS/Abrazos program at Massachusetts General Hospital from 6/30/2020–8/31/2021. Results of the quality improvement evaluation revealed that the program was widely adopted across the hospital and 392 unique families were referred to the program. The referred patients were representative of the communities in Massachusetts disproportionately affected by the COVID-19 pandemic. Furthermore, 79% of referred patients followed up with the initial referral, with sustained high participation rates throughout the program course; and they were provided with an average of four community resource referrals. Adoption and implementation of the key components in HUGS/Abrazos were found to be appropriate and acceptable. Furthermore, the implemented program remained consistent to the original design. Overall, HUGS/Abrazos was well adopted as an emergency relief program with strong post-COVID-19 applicability to ameliorate continuing toxic stressors while decreasing burden on the health system.
Keywords: toxic stress, patient navigation, resilience, RE-AIM, COVID-19 pandemic, early childhood
Introduction
Early life adversity, defined as recurrent stressful events that occur during sensitive periods of development, can have profound impact on child development and health outcomes throughout the life course (1–4). Pregnancy and the early childhood years are examples of critical periods of development during which the parent-child dyad is more vulnerable to toxic stressors (5). Adversity during these critical periods of development influences neurodevelopmental processes at the cellular level (6, 7), disrupts normal immunoregulatory scaffolding (8), and results in cumulatively increased risk for disease in adulthood (2, 8, 9). Furthermore, the early life environment can exert intergenerational impact on risk for chronic disease throughout the life course via epigenetic mechanisms (10). Reassuringly, protective buffers can curb the negative impact of toxic stress and build resilience among children and families experiencing adversity (11–14). Therefore, social programs that foster resilience are necessary.
The COVID-19 pandemic exacerbated preexisting and introduced new sources of toxic stress for families with young children. Specifically, the newfound challenges of at-home parenting (15), financial insecurity (16, 17), racial disparities in health outcomes (18–20), and behavioral health burdens have all escalated to critical levels (21–23). This is especially true for marginalized communities such as racially and ethnically diverse populations, immigrants, and families in poverty (18, 19, 24–26). The Helping Us Grow Stronger (HUGS/Abrazos) program fills a crucial need for protective buffers during the COVID-19 pandemic, which has escalated toxic stressors affecting pregnant women and families with young children. The multimodal strategy utilized by HUGS/Abrazos to support patients from communities hardest hit by the COVID-19 pandemic has been previously described (27). Specifically, HUGS/Abrazos supports patients served by Massachusetts General Hospital (MGH), including MGH community health centers in Chelsea and Revere, two communities most severely impacted by the COVID-19 pandemic in Massachusetts.
Given the quickly evolving public health crisis, analytical methods that can assess public health interventions without delaying implementation are crucial. RE-AIM – Reach, Effectiveness, Adoption, Implementation, and Maintenance – provides an evaluation framework that assesses the delivery of public health interventions while bridging the gap between practice and research (28–30). The RE-AIM framework has been especially helpful when used to inform adaptations and dissemination of interventions in low resource settings (31–33). We used a mixed-methods approach, guided by the RE-AIM framework, to evaluate the implementation of the HUGS/Abrazos program to inform future program adaptations, dissemination, and sustainability.
Methods
Overview of HUGS/Abrazos Program
The design and implementation of the HUGS/Abrazos program has been previously described (27). HUGS/Abrazos aimed to (1) use targeted patient navigation to address unmet health-related social needs; (2) provide short-term, immediate behavioral health support; and (3) create cross-systems linkages among community partners using centralized resource repository and an integrated referral system. The targeted patient population included communities in and surrounding Boston, MA that were heavily impacted by the COVID-19 pandemic. These communities had higher number of immigrants, families living in poverty, and residents of racially and ethnically minoritized groups compared to state average. Eligibility criteria included (1) pregnant women and families with children under 6 years old; (2) demonstration of an unmet socioeconomic or behavioral health need; and (3) had an established provider within MGH system. In a cross-departmental collaboration, providers in Pediatrics, Obstetrics & Gynecology (OB/Gyn), Family Medicine, and other specialties made initial referrals to HUGS/Abrazos. Referrals were triaged and assigned to a community health worker (CHW), who navigated patients toward community resources, or to the behavioral health team, who provided behavioral health support, or to both. During the first encounter, a screening questionnaire was used to assess for specific unmet socioeconomic or behavioral health needs. Referred patients connected with the CHW for up to three touchpoints, and with the behavioral health team for up to four touchpoints. We utilized a centralized resource repository in Aunt Bertha and the Integrated Referral and Intake system (IRIS) to streamline communication and workflow. All patients received a care package that included a $50 gift card for groceries, age-appropriate activity kits, and supplies to encourage healthy practices during the pandemic. At the conclusion of the program, patients were referred to long term services if necessary and available. HUGS/Abrazos conception and design began in April 2020, and program launch occurred in July 2020. Evaluation of implementation included patients referred to the program between 6/30/2020 and 8/31/2021. The Mass General Brigham Institutional Review Board determined the evaluation of the HUGS/Abrazos program to be local program evaluation intended for quality improvement purposes and did not require Institutional Review Board oversight.
Overview of Mixed Methods Evaluation Using the RE-AIM Framework
We used a mixed methods approach, guided by the RE-AIM framework, to evaluate the implementation and delivery of HUGS/Abrazos (30). Quantitative data related to reach, effectiveness, and adoption (R,E,A) were obtained from the electronic health record (EHR) and administrative data. Qualitative data related to implementation and maintenance (I, M) were obtained through focus group sessions, which subsequently underwent rapid qualitative analysis described below. See Table 1 for the specific measures used to assess each domain of the RE-AIM framework.
Table 1.
Evaluation of the HUGS/Abrazos program using the RE-AIM framework.
| RE-AIM Component | Measure | Data source |
|---|---|---|
| Reach | Total number of patients seen in practices referring to HUGS | Electronic Health Record (EHR) |
| Number of unique referrals made to HUGS | ||
| Number of families who completed ≥ 1 touchpoint | ||
| Socio-demographics of referred patients | ||
| Effectiveness | Number of touchpoints with community health, behavioral health, and community health + behavioral health combined | Electronic Health Record (EHR) |
| Average number of referrals provided to community resources | ||
| Reason for referrals to community resources | ||
| Adoption | Characteristics of practices referring to HUGS | Administrative and EHR data |
| Characteristics of providers making referrals to HUGS | ||
| Implementation | Appropriateness of HUGS | Qualitative focus groups with community health, behavioral health, and physician champions |
| Acceptability of HUGS | ||
| Penetration of HUGS | ||
| Fidelity to the program and adaptations made | ||
| Maintenance | Sustainability of HUGS | Qualitative focus groups with community health, behavioral health, and physician champions |
RE-AIM, Reach, Effectiveness, Adoption, Implementation, Maintenance.
Quantitative Evaluation Methods
To evaluate adoption, defined as the representativeness of settings that implement a new program (29), we utilized the EHR and administrative data to determine the characteristics of practices and providers who made referrals to HUGS/Abrazos.
To evaluate reach, defined as the participation rate and characteristics of the program-eligible population (29), we utilized the EHR to determine the number of patients seen in practices referring to HUGS/Abrazos and the number of unique referrals made to HUGS/Abrazos. Due to the possibility in which multiple referrals were made for the same family or individual for different reasons, we tracked the number of unique families referred. After we identified families, we then selected the first referral made and used that information when reporting. When possible, we linked child and parent data. Of the referrals made, we determined the number of families who completed at least one touchpoint during their HUGS/Abrazos participation. We summarized the socio-demographics of referred patients, which included parent age, child age, sex, race and ethnicity, language, birth country, insurance status, education level, marital status, and employment status, and stratified by those who completed touchpoints and those who did not.
To evaluate the effectiveness, defined as the impact of the program for the participating population (29), we utilized the EHR to determine the number touchpoints patients completed with the CHW, the behavioral health team, or both. Additionally, we determined the reasons for which referrals to community resources were made and the average number of referrals provided to patients for community resources.
We calculated descriptive statistics for the number of referred patients, unique referrals to HUGS/Abrazos, touchpoints completed, and the referrals to community resources. We performed statistical analyses using RStudio 1.4.1717 (R Core Team) (34).
Qualitative Evaluation Methods
We designed and facilitated three focus group sessions with stakeholder groups to elucidate their perspectives on the implementation and maintenance of HUGS/Abrazos. The first focus group session involved all the CHWs who provided patient navigation of community resources appropriate to each patient's health related social needs (n = 6). The second focus group session involved all members of the behavioral health team, who provided stress reduction strategies, mindfulness techniques, and cognitive behavioral therapy to alleviate acute behavioral health needs (n = 3). Finally, the third focus group session involved primary care physicians (PCP), who were part of the program's initial conception and design team and served as clinician champions heralding the program's launch (n = 2). The focus group interview guide was developed according to the sustainability-enhanced RE-AIM framework with questions tailored to evaluate implementation outcomes, which included appropriateness, acceptability, penetration, fidelity to program design, and sustainability (30, 35, 36).
We used rapid qualitative analysis methods that have been successfully used in prior studies to inform implementation when results are needed in a timely manner (37, 38). We recorded and transcribed the focus group sessions for rapid qualitative analysis to determine key findings related to implementation and maintenance of HUGS/Abrazos. First, the evaluation team created a summary table that outlined (1) each implementation outcome with its associated focus group questions; (2) key findings; and (3) related exemplar quotes in the transcript. Next, the analytic team extracted data from one focus group transcript to populate the summary table. The evaluation team then reviewed and modified the summary table based on the initial analysis of one transcript. The analytic team subsequently extracted data from remaining focus group transcripts to populate the newly modified summary tables. A second review of the summary tables by the evaluation team was performed to ensure accuracy and consistency in data extraction. Finally, the summary tables were used to populate a matrix in Microsoft Excel to identify themes and subthemes consistent across stakeholder groups. The evaluation team reviewed and discussed the matrix to finalize the themes. The conception and design team of HUGS/Abrazos performed a final review of the identified themes and subthemes.
Results
Adoption
A multidisciplinary cohort of providers from 31 different MGH site specific departments, grouped into 12 overall department categories, referred patients to the HUGS/Abrazos program. Most referrals originated from Pediatrics/Adolescent Health (38.0%), Obstetrics (31.1%), Family Medicine (11.7%), and Behavioral Health/Psychiatry (6.6%) which correlates well with the HUGS/Abrazos intended patient population of pregnant women and families with children under 6 years old. Remaining referrals to HUGS/Abrazos originated from department categories that include but are not limited to emergency medicine, care coordination, social services, and more. Referring providers included physicians, midwives, psychologists, social workers, nurse practitioners, and others.
The MGH HealthCare Centers in Chelsea and Revere were the most common referral sites, making up 42.6 and 23.5%, respectively, of parent referrals and 62.4 and 22.0%, respectively, of child referrals. This correlates well with Chelsea and Revere being the primary sites of HUGS/Abrazos' initial design and implementation. However, 33.9 and 15.6% for parent and child referrals, respectively, originated from MGH main hospital and other MGH affiliated community-based health centers, suggesting successful widespread multi-site adoption of HUGS/Abrazos in the MGH hospital system.
Reach
A total of 6,267 women and 8,055 children under 6 years old were seen for obstetric and pediatric care, respectively, at practices participating in HUGS/Abrazos since its implementation. A total of 551 referrals were made for HUGS/Abrazos, and of this, 392 referrals were made for unique families during the evaluation period (ex. one family may be referred for both food insecurity and baby supplies). The racial and ethnic demographics of referred patients were comparable to the racial and ethnic demographics of the communities in which we targeted our outreach efforts. For example, the racial and ethnic demographics of parents in the HUGS/Abrazos program included 64.2, 22.9, and 8.5% of Hispanic, non-Hispanic Whites, and non-Hispanic Blacks in comparison to 67.0, 20.6, and 5.4% of the same racial and ethnic groups, respectively, in Chelsea, MA (39). A greater percentage of parents referred to HUGS/Abrazos were foreign-born compared to the percentage of foreign-born residents in Chelsea, MA (67.6 vs. 45.4%) (39). Meanwhile, most children referred to HUGS/Abrazos were born in the United States (93.4%). A majority of referred patients, 41.3% for parent referrals and 61.1% of parents of referred children, spoke a language other than English as their primary language. We also obtained the demographic data for patients who were referred to HUGS/Abrazos but did not participate in any touchpoints for a full scope of our reach. Please see Table 2 for detailed demographics data of the HUGS/Abrazos patient population.
Table 2.
Characteristics of families referred to the HUGS/Abrazos program (N = 392).
| All referred parents | Parents who engaged in HUGS | Parents who did not engage in HUGS | All referred children | Children whose parents engaged in HUGS | Children whose parents did not engage in HUGS | |
|---|---|---|---|---|---|---|
| N = 206 | N = 164 | N = 42 | N = 186 | N = 146 | N = 40 | |
| Age at referral | ||||||
| Mean (SD) | 29.5 (6.7) | 29.7 (6.8) | 28.6 (6.2) | 1.8 (1.8) | 1.9 (1.8) | 1.6 (1.8) |
| Range | (16.4, 56.0) | (16.4, 56.0) | (18.7, 42.9) | (0.0, 7.0) | (0.0, 6.6) | (0.0, 7.0) |
| Age categories of children at initial referral | ||||||
| 0–5.9 mo. | N/A | N/A | N/A | 55 (29.6) | 42 (28.8) | 13 (32.5) |
| 6.0–11.9 mo. | N/A | N/A | N/A | 28 (15.1) | 21 (14.4) | 7 (17.5) |
| 1.0–1.9 yrs. | N/A | N/A | N/A | 30 (16.1) | 21 (14.4) | 9 (22.5) |
| 2.0–3.9 yrs. | N/A | N/A | N/A | 41 (22.0) | 35 (24.0) | 6 (15.0) |
| ≥ 4.0 yrs. | N/A | N/A | N/A | 32 (17.2) | 27 (18.5) | 5 (12.5) |
| Sex, n (%) | ||||||
| Male | 2 (1.0) | 2 (1.2) | 0 (0) | 90 (48.4) | 69 (47.3) | 21 (52.5) |
| Female | 204 (99.0) | 162 (98.8) | 42 (100.0) | 96 (51.6) | 77 (52.7) | 19 (47.5) |
| Race/Ethnicity, n (%) | N = 201 | N = 135 | ||||
| White | 46 (22.9) | 28 (17.5) | 18 (43.9) | 8 (5.9) | 5 (4.7) | 3 (10.3) |
| Hispanic or Latino | 129 (64.2) | 114 (71.2) | 15 (36.6) | 108 (80.0) | 87 (82.1) | 21 (72.4) |
| Black or African American | 17 (8.5) | 12 (7.5) | 5 (12.2) | 12 (8.9) | 9 (8.5) | 3 (10.3) |
| Asian or Multiracial | 9 (4.5) | 6 (3.8) | 3 (7.3) | 7 (5.2) | 5 (4.7) | 2 (6.9) |
| Language, n (%) | N = 206 | N = 185 | ||||
| English | 121 (58.7) | 89 (54.3) | 32 (76.2) | 72 (38.9) | 49 (33.8) | 23 (57.5) |
| Spanish | 75 (36.4) | 69 (42.1) | 6 (14.3) | 100 (54.1) | 85 (58.6) | 15 (37.5) |
| Other | 10 (4.9) | 6 (3.7) | 4 (9.5) | 13 (7.0) | 11 (7.6) | 2 (5.0) |
| Birth Country, n (%) | N = 176 | N = 182 | ||||
| Foreign Born | 119 (67.6) | 103 (73.0) | 16 (45.7) | 12 (6.6) | 12 (8.4) | 0 (0) |
| Insurance, n (%) | ||||||
| Public | 163 (79.1) | 132 (80.5) | 31 (73.8) | 172 (92.5) | 136 (93.2) | 36 (90.0) |
| Private | 43 (20.9) | 32 (19.5) | 11 (26.2) | 14 (7.5) | 10 (6.8) | 4 (10.0) |
| Education, n (%) | N = 199 | |||||
| Some high school or less | 59 (29.6) | 52 (32.9) | 7 (17.1) | N/A | N/A | N/A |
| High school graduate | 73 (36.7) | 59 (37.3) | 14 (34.1) | N/A | N/A | N/A |
| More than high school or other | 67 (33.7) | 47 (29.7) | 20 (48.8) | N/A | N/A | N/A |
| Marital Status, n (%) | N = 204 | |||||
| Unmarried | 123 (60.3) | 100 (61.7) | 23 (54.8) | N/A | N/A | N/A |
| Employment, n (%) | N = 198 | |||||
| Unemployed | 122 (61.6) | 99 (62.7) | 23 (57.5) | N/A | N/A | N/A |
Effectiveness
A total of 392 unique families were referred to HUGS/Abrazos (see Figure 1). Of these families, 310 or 79% of them completed at least one touchpoint with either a CHW or the behavioral health team. Although we did not collect data on reasons for non-participation, possible explanations may include newfound access to another resource, inability to participate due to time constraints and other stressors, access to technology for virtual visits, language and cultural barriers despite availability of interpreters, fear of social stigma, and anxiety around immigration status. Participating families maintained longitudinal relationships through multiple touchpoints with either the CHW, the behavioral health team, or both. Of the 220 families who completed CHW only services, 194 completed at least two touchpoints, and 83 completed three touchpoints. Of the 33 families who completed behavioral health only services, 28 completed at least two touchpoints, 20 completed at least three touchpoints, and 15 completed the maximum four touchpoints. A total of 57 families received both CHW and behavioral health services. Participation rates in either CHW or behavioral health team touchpoints for these families trended similarly to the data for CHW only and behavioral health only families. Families who engaged in patient navigation services with the CHW were referred to an average number of 4.4 community resources and had an average of 3.71 reasons for these referrals (Supplementary Table 1). The most common reasons for community resource referrals included but are not limited to infant supplies (60.2%), food security (52.3%), and support with housing and related legal issues (48.5%) (Supplementary Table 1).
Figure 1.
Flowchart of referrals to the HUGS/Abrazos Program (6/30/2020–8/31/2021). Patients can complete up to three touchpoints with the community health workers or up to four touchpoints with the behavioral health team.
Implementation and Maintenance
Table 3 summarizes the themes and subthemes related to the implementation and maintenance of HUGS/Abrazos, as well as exemplar quotes, that emerged from rapid qualitative analysis of focus group sessions with CHWs, the behavioral health team, and PCP champions. All focus group participants were females except for two male participants. Professional roles included: CHW, social worker, psychologist, and pediatricians. When assessing appropriateness, or the perceived fit of the program for its intended audience and setting (35), three key subthemes emerged: (1) HUGS/Abrazos successfully targeted its patient outreach toward peripartum women and families with unmet socioeconomic and behavioral health needs, (2) provided them with emergency relief of the most acute issues, and (3) did so in a manner that maximized equity and accessibility. When assessing acceptability, or the perception among stakeholders that the program is agreeable (35), HUGS/Abrazos was perceived as a successful multidisciplinary collaboration that simultaneously reduced the burden on an already overwhelmed behavioral health system and led to positive impacts on patients' lives. When assessing fidelity, or the degree to which the program was implemented as it was designed (35), most stakeholders found that the core components of HUGS/Abrazos – patient navigation, acute behavioral health support, and utilization of a centralized resource repository and integrated referral system – were implemented with high fidelity though flexibility was necessary during individual interactions to fulfill differing needs. In terms of program penetration (35), or the degree to which the program has integrated into the existing infrastructure, HUGS/Abrazos successfully closed the gap among previously siloed resources and care teams to forge new coalitions and relationships within the hospital system and with community partners. From the perspective of the HUGS/Abrazos PCP champions, behavioral health teams, and CHWs, implementation barriers and program limitations included technological challenges in setting of virtual visits, lack of interpreter services at partner organizations, and sensitivity of conversations around immigration status, all of which may directly influence the effectiveness of the program for participants. Finally, stakeholders believe there is value in maintaining HUGS/Abrazos beyond the pandemic as the need for social programs that can foster protective buffers against toxic stressors, such as acute socioeconomic and behavioral health needs, will remain.
Table 3.
Emergent themes and exemplar quotes of the implementation of HUGS.
| Themes and Subthemes | Exemplar Quotes |
|---|---|
| Appropriateness | |
| HUGS appropriately reaches pregnant women and families with young children who have unmet social and behavioral health needs |
Most of the folks have been impacted by the pandemic. There's been a lot of different losses, [ranging] from loss in families, loss of jobs, loss of financial income…[HUGS] has been helpful and useful. I feel like short-term works. They might be referred back later on, but as a quick intervention…it is beneficial. |
| HUGS provides the appropriate short-term supports, including patient navigation to connect patients to community resources, and behavioral health to provide time-sensitive relief | |
| HUGS is an accessible and equitable program by offering phone and video visits, services in English and Spanish, and not billing health insurance, but these features are not without known drawbacks and barriers | |
| Acceptability | |
| The services provided and connections formed through HUGS have led to positive impacts on patients' lives | HUGS is able to tie this all in a bow, put everything together so that people can talk to each other and deliver the best care for the patients. |
| HUGS implements effective collaborations among providers (e.g., PCP, community health workers, social workers) and streamlines connections to care | |
| HUGS provides immediate access to behavioral health supports thereby reducing the time for patients to receive care and the burden on the system | |
| Fidelity and Adaptations | |
| The core components of HUGS have remained the same and only minor modifications have been made | Overall, everything has stayed the same: …the gift cards, the books, the community health worker involvement. |
| Flexibility within the program is important because patients have differing needs | |
| Penetration | |
| HUGS has brought together multiple hospital departments to develop new resources to better serve the patient population |
Another strength of [HUGS] is that, in medicine … we work in siloes a lot. This forced a deliberate communication with each other …That's one of the strengths of this interdisciplinary collaboration that was very deliberate and eye opening. |
| HUGS has joined existing coalitions and has formed relationships with community organizations and should continue forging these relationships | |
| Referrals to HUGS are dependent on providers and their knowledge of the program, talking to families about the program, and other competing demands during the visit | |
| Sustainability | |
| HUGS was initially developed as a COVID-19 program to provide time-sensitive relief, but the program should be sustained as socioeconomic and health challenges will remain | I think it has potential to continue because a lot of families are benefitting from it. |
| Developing a plan to financially sustain HUGS is important to maintain the program | |
Discussion
Several key findings emerged when assessing HUGS/Abrazos program using the RE-AIM framework. First, patients referred to HUGS/Abrazos were representative – racially, ethnically, and socioeconomically – of the communities that were disproportionately affected by the COVID-19 pandemic, demonstrating that we are effectively reaching families in need of support. Second, there were high participation rates by referred families throughout the program, illustrating that HUGS/Abrazos filled a crucial need for patient navigation of resources and acute behavioral health support. Third, the adoption and implementation of the key components in HUGS/Abrazos were appropriate and acceptable, and they remained faithful to its original design. Finally, there is overwhelming stakeholder support in maintaining HUGS/Abrazos beyond the COVID-19 pandemic, as it has proved to be an effective delivery model in mitigating acute exacerbations of toxic stress. Taken together, these results suggest that the implementation of HUGS/Abrazos was effective in providing emergency relief to help decrease the burden on the health system.
Short-term behavioral health interventions have been shown to be effective in mitigating mental health needs during the perinatal period and among adolescents (40–44). More recently, the availability of telepsychiatry in the ED setting de-escalated mental health crises and limited the burden on an overwhelmed system during the COVID-19 pandemic (45). HUGS/Abrazos utilized similar principles in an emergency relief program to effectively address behavioral health concerns escalated by the pandemic, which were previously not addressed due to limited resources in the mental healthcare system. In terms of our patient population, HUGS/Abrazos served an age group that encompassed the perinatal period to early childhood, which is a particularly sensitive period to external adversity (5), yet few behavioral health programs address. Additionally, HUGS/Abrazos relied on an integrated structure that combined behavioral health support, patient navigation services, and direct relief. Patient navigation has previously been proven to be an effective strategy in addressing the socioeconomic factors underlying complex health needs (46–48). We combined patient navigation strategies with with resource platforms, Aunt Bertha and IRIS, to enhance centralization of resources and closed loop communication among all involve parties.
Several factors contributed to successful reach, adoption, and implementation of the HUGS/Abrazos program. Little is known about factors that support program uptake. By using the RE-AIM framework, we can begin to elucidate these factors and thereby improve the sustainability and diffusion of this innovative program and provide a roadmap for other public health innovations (49, 50). Based on a scoping review that examined factors that influenced implementation, we identified several factors that supported the implementation of HUGS/ Abrazos (49). Essential to this program was early stakeholder input from a multidisciplinary team including CHWs, the behavioral health team, and cross-departmental providers on the specific operational processes in HUGS/Abrazos allowing the program to be efficiently implemented across multiple MGH-affiliated sites. This led to the development of strong relationships within the HUGS/ Abrazos team, as well as partnering departments and organizations. We anticipate the stakeholder engagement and relationships will be critical in the maintenance of this program as has been demonstrated in the literature (51). During the inception of HUGS/ Abrazos, the team had a clear understanding of who the target population was which allowed for focused development and implementation (52). Additionally, effective recruitment of CHWs who were already familiar with the community resources and our targeted patient population ensured readiness to deliver patient navigation services. HUGS/ Abrazos program also had adequate resources to provide the necessary services. The resources included financial, personnel, and dedicated time and were a result of external funding, prioritization from the hospital system reflective of the importance of this program, and the strong relationships developed.
The strengths and limitations of program design has been discussed in detail in a prior publication (27). For our evaluation process, the use of the RE-AIM framework is a key strength that allows us to broadly assess implementation and identify areas for adaptation without disrupting intervention delivery and plan for maintenance and dissemination. One limitation is the scope of data collection, limited to EHR, administrative data, and qualitative data from those delivering the program. We used a pragmatic approach to evaluation to reduce burden and therefore we do not have data from referring providers or from families who were served by the program. As a result, we are unable to make conclusions on direct effectiveness, such as improvement in mental health or alleviation of socioeconomic needs after program participation. Additionally, there remains a perception of HUGS/Abrazos as a pandemic-specific relief program. However, the socioeconomic and behavioral health needs that HUGS/Abrazos address will outlive the pandemic, thus ensuring the program's post-COVID applicability.
In conclusion, the HUGS/Abrazos program is an emergency relief program that provides patient navigation of resources and acute behavioral health services to support vulnerable patient populations while reducing burden on an overwhelmed health system. HUGS/Abrazos serves as a protective buffer for vulnerable pregnant women and families with young children against toxic stressors exacerbated by the COVID-19 pandemic while also fostering resilience. Our evaluation of this quality improvement program, guided by the RE-AIM Framework, demonstrates that HUGS/ Abrazos was successfully adopted, reached its intended population, was effective in sustaining high participation rates and providing needed services, was acceptable, and maintained high fidelity. Next steps should focus on objective assessments of program efficacy, such as usage of validated mental health assessment instruments. Integration of social and behavioral health supports, multidisciplinary collaboration, and use of innovative tools that streamlined workflow are the basic principles that empowered the rapid implementation and effectiveness of HUGS/Abrazos, making the program an exemplary delivery model for future similar programs.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.
Author Contributions
MLi, MSi, MPe, and ET contributed to the conceptualization and design of the implementation evaluation process. MLu performed the quantitative statistical analysis and assisted with the interpretation of the data. MLi, MSi, MPe, SP, VC, and S-YTC conducted the qualitative analysis and/or assisted with the interpretation of the data. MSe, AE, CT, GK, AB, MH, MPa, WL, and ET assisted with interpretation of all data and critically reviewed the manuscript for important intellectual content. MLi and MSi drafted the first version of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.
Funding
Program funding was provided by the Boston Foundation to the Kraft Center for Community Health at MGH and through support by Grant no. R01HD100022 from Eunice Kennedy Shriver National Institute of Child health and Human Development awarded to Andrea G. Edlow. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funders. The sponsors had no role in the evaluation, collection, analysis, and interpretation of data, writing of report, or decision to submit for publication.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
We thank the referring MGH practices and MGH staff for their dedication to the HUGS/Abrazos Program.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.862388/full#supplementary-material
References
- 1.Felitti VJ, Anda RF, Nordenberg D, Williamson DF, Spitz AM, Edwards V, et al. Relationship of childhood abuse and household dysfunction to many of the leading causes of death in adults. The adverse childhood experiences (ACE) Study. Am J Prev Med. (1998) 14:245–58. 10.1016/S0749-3797(98)00017-8 [DOI] [PubMed] [Google Scholar]
- 2.Shonkoff JP, Garner AS, The committee on psychosocial aspects of child and family health C on EC. Siegel BS, Dobbins MI, Earls MF, et al. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. (2012) 129:e232–46. 10.1542/peds.2011-2663 [DOI] [PubMed] [Google Scholar]
- 3.Committee Committee on Psychosocial Aspects of Child and Family Health C on EC. Garner AS, Shonkoff JP, Siegel BS, Dobbins MI, Earls MF, et al. Early childhood adversity, toxic stress, and the role of the pediatrician: translating developmental science into lifelong health. Pediatrics. (2012) 129:e224–31. 10.1542/peds.2011-2662 [DOI] [PubMed] [Google Scholar]
- 4.Nelson CA, Gabard-Durnam LJ. Early adversity and critical periods: Neurodevelopmental consequences of violating the expectable environment. Trends Neurosci. (2020) 43:133–43. 10.1016/j.tins.2020.01.002 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Knudsen EI. Sensitive periods in the development of the brain and behavior. J Cogn Neurosci. (2004) 16:1412–25. 10.1162/0898929042304796 [DOI] [PubMed] [Google Scholar]
- 6.Reh RK, Dias BG, Nelson CA, Kaufer D, Werker JF, Kolb B, et al. Critical period regulation across multiple timescales. Proc Natl Acad Sci. (2020) 117:23242–51. 10.1073/pnas.1820836117 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Fox SE, Levitt P, Nelson III CA. How the timing and quality of early experiences influence the development of brain architecture. Child Dev. (2010) 81:28–40. 10.1111/j.1467-8624.2009.01380.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Elwenspoek MMC, Kuehn A, Muller CP, Turner JD. The effects of early life adversity on the immune system. Psychoneuroendocrinology. (2017) 82:140–54. 10.1016/j.psyneuen.2017.05.012 [DOI] [PubMed] [Google Scholar]
- 9.Berens AE, Jensen SKG, Nelson CA. Biological embedding of childhood adversity: from physiological mechanisms to clinical implications. BMC Med. (2017) 15:135. 10.1186/s12916-017-0895-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Bianco-Miotto T, Craig JM, Gasser YP, van Dijk SJ, Ozanne SE. Epigenetics and DOHaD: from basics to birth and beyond. J Dev Orig Health Dis. (2017) 8:513–9. 10.1017/S2040174417000733 [DOI] [PubMed] [Google Scholar]
- 11.Savage-McGlynn E, Redshaw M, Heron J, Stein A, Quigley MA, Evans J, et al. Mechanisms of resilience in children of mothers who self-report with depressive symptoms in the first postnatal year. PLoS ONE. (2015) 10:e0142898. 10.1371/journal.pone.0142898 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Collishaw S, Pickles A, Messer J, Rutter M, Shearer C, Maughan B. Resilience to adult psychopathology following childhood maltreatment: evidence from a community sample. Child Abuse Negl. (2007) 31:211–29. 10.1016/j.chiabu.2007.02.004 [DOI] [PubMed] [Google Scholar]
- 13.Blair C, Granger DA, Willoughby M, Mills-Koonce R, Cox M, Greenberg MT, et al. Salivary cortisol mediates effects of poverty and parenting on executive functions in early childhood. Child Dev. (2011) 82:1970–84. 10.1111/j.1467-8624.2011.01643.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Traub F, Boynton-Jarrett R. Modifiable resilience factors to childhood adversity for clinical pediatric practice. Pediatrics. (2017) 139:e20162569. 10.1542/peds.2016-2569 [DOI] [PubMed] [Google Scholar]
- 15.Brown SM, Doom JR, Lechuga-Peña S, Watamura SE, Koppels T. Stress and parenting during the global COVID-19 pandemic. Child Abuse Negl. (2020) 110:104699. 10.1016/j.chiabu.2020.104699 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Van Lancker W, Parolin Z. COVID-19, school closures, and child poverty: a social crisis in the making. Lancet Public Health. (2020) 5:e243–4. 10.1016/S2468-2667(20)30084-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Dunn CG, Kenney E, Fleischhacker SE, Bleich SN. Feeding low-income children during the Covid-19 pandemic. N Engl J Med. (2020) 382:e40. 10.1056/NEJMp2005638 [DOI] [PubMed] [Google Scholar]
- 18.Millett GA, Jones AT, Benkeser D, Baral S, Mercer L, Beyrer C, et al. Assessing differential impacts of COVID-19 on black communities. Ann Epidemiol. (2020) 47:37–44. 10.1101/2020.05.04.20090274 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Hooper MW, Nápoles AM, Pérez-Stable EJ. COVID-19 and racial/ethnic disparities. JAMA. (2020) 323:2466–7. 10.1001/jama.2020.8598 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Price-Haywood EG, Burton J, Fort D, Seoane L. Hospitalization and mortality among black patients and white patients with Covid-19. N Engl J Med. (2020) 382:2534–43. 10.1056/NEJMsa2011686 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Fegert JM, Vitiello B, Plener PL, Clemens V. Challenges and burden of the Coronavirus 2019 (COVID-19) pandemic for child and adolescent mental health: a narrative review to highlight clinical and research needs in the acute phase and the long return to normality. Child Adolesc Psychiatry Ment Health. (2020) 14:20. 10.1186/s13034-020-00329-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet Lond Engl. (2020) 395:912–20. 10.1016/S0140-6736(20)30460-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Patrick SW, Henkhaus LE, Zickafoose JS, Lovell K, Halvorson A, Loch S, et al. Well-being of parents and children during the COVID-19 pandemic: a national survey. Pediatrics. (2020). 10.1542/peds.2020-016824 [DOI] [PubMed] [Google Scholar]
- 24.Cholera R, Falusi OO, Linton JM. Sheltering in place in a xenophobic climate: COVID-19 and children in immigrant families. Pediatrics. (2020) 146:e20201094. 10.1542/peds.2020-1094 [DOI] [PubMed] [Google Scholar]
- 25.Azar KMJ, Shen Z, Romanelli RJ, Lockhart SH, Smits K, Robinson S, et al. Disparities in outcomes among COVID-19 patients in a large health care system in California. Health Aff Proj Hope. (2020) 39:1253–62. 10.1377/hlthaff.2020.00598 [DOI] [PubMed] [Google Scholar]
- 26.Page KR, Venkataramani M, Beyrer C, Polk S. Undocumented US immigrants and Covid-19. N Engl J Med. (2020) 382:e62. 10.1056/NEJMp2005953 [DOI] [PubMed] [Google Scholar]
- 27.Liu M, Neri Mini F, Torres C, Kwete GM, Boudreau AA, Hunter ML, et al. Fostering resilience in pregnancy and early childhood during the COVID-19 pandemic: the HUGS/Abrazos program design and implementation. Front Public Health. (2021) 9:422. 10.3389/fpubh.2021.633285 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Glasgow RE, Estabrooks PE. Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Prev Chronic Dis. (2018) 15:E02. 10.5888/pcd15.170271 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. (1999) 89:1322–7. 10.2105/AJPH.89.9.1322 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Shelton RC, Chambers DA, Glasgow RE. An extension of RE-AIM to enhance sustainability: addressing dynamic context and promoting health equity over time. Front Public Health. (2020) 8:134. 10.3389/fpubh.2020.00134 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Bottorff JL, Huisken A, Hopkins M, Nesmith C. A RE-AIM evaluation of Healthy Together: a family-centred program to support children's healthy weights. BMC Public Health. (2020) 20:1754. 10.1186/s12889-020-09737-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Quinn AK, Neta G, Sturke R, Olopade CO, Pollard SL, Sherr K, et al. Adapting and operationalizing the RE-AIM framework for implementation science in environmental health: Clean fuel cooking programs in low resource countries. Front Public Health. (2019) 7:389. 10.3389/fpubh.2019.00389 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Whooten RC, Horan C, Cordes J, Dartley AN, Aguirre A, Taveras EM. Evaluating the implementation of a before-school physical activity program: A mixed-methods approach in Massachusetts, 2018. Prev Chronic Dis. (2020) 17:E116. 10.5888/pcd17.190445 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.RStudio Team . Boston, MA: RStudio: Integrated Development Environment for R. (2021). Available online at: http://www.rstudio.com/ [Google Scholar]
- 35.Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. (2011) 38:65–76. 10.1007/s10488-010-0319-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Mettert K, Lewis C, Dorsey C, Halko H, Weiner B. Measuring implementation outcomes: An updated systematic review of measures' psychometric properties. Implement Res Pract. (2020) 1:2633489520936644. 10.1177/2633489520936644 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Keith RE, Crosson JC, O'Malley AS, Cromp D, Taylor EF. Using the consolidated framework for implementation research (CFIR) to produce actionable findings: a rapid-cycle evaluation approach to improving implementation. Implement Sci IS. (2017) 12:15. 10.1186/s13012-017-0550-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs. in-depth qualitative analytic methods from a process evaluation of academic detailing in the veterans health administration. Implement Sci. (2019) 14:11. 10.1186/s13012-019-0853-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.U.S. Census Bureau (2021). 2015-2019 American Community Survey 5-Year Estimates Data Profiles. Available online at: https://data.census.gov/cedsci/table?g=1600000US2513205&tid=ACSDP5Y2019.DP05 (accessed December 6, 2021).
- 40.Rahman A, Malik A, Sikander S, Roberts C, Creed F. Cognitive behaviour therapy-based intervention by community health workers for mothers with depression and their infants in rural Pakistan: a cluster-randomised controlled trial. Lancet Lond Engl. (2008) 372:902–9. 10.1016/S0140-6736(08)61400-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Rahman A, Waqas A, Nisar A, Nazir H, Sikander S, Atif N. Improving access to psychosocial interventions for perinatal depression in low- and middle-income countries: lessons from the field. Int Rev Psychiatry. (2021) 33:198–201. 10.1080/09540261.2020.1772551 [DOI] [PubMed] [Google Scholar]
- 42.Chowdhary N, Sikander S, Atif N, Singh N, Ahmad I, Fuhr DC, et al. The content and delivery of psychological interventions for perinatal depression by non-specialist health workers in low and middle income countries: a systematic review. Best Pract Res Clin Obstet Gynaecol. (2014) 28:113–33. 10.1016/j.bpobgyn.2013.08.013 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Bu H, Duan W. A single-session positive cognitive intervention on first-year students' mental health: Short-term effectiveness and the mediating role of strengths knowledge. J Am Coll Health J ACH. (2019) 67:515–22. 10.1080/07448481.2018.1497639 [DOI] [PubMed] [Google Scholar]
- 44.Winzer R, Brucefors AB. Does a short-term intervention promote mental and general health among young adults? - An evaluation of counselling. BMC Public Health. (2007) 7:319. 10.1186/1471-2458-7-319 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Reinhardt I, Gouzoulis-Mayfrank E, Zielasek J. Use of telepsychiatry in emergency and crisis intervention: current evidence. Curr Psychiatry Rep. (2019) 21:63. 10.1007/s11920-019-1054-8 [DOI] [PubMed] [Google Scholar]
- 46.Blake-Lamb T, Boudreau AA, Matathia S, Tiburcio E, Perkins ME, Roche B, et al. Strengthening integration of clinical and public health systems to prevent maternal-child obesity in the First 1,000 Days: a collective impact approach. Contemp Clin Trials. (2018) 65:46–52. 10.1016/j.cct.2017.12.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.McKenney KM, Martinez NG, Yee LM. Patient Navigation Across the Spectrum of Women's Health Care in the US. Am J Obstet Gynecol. (2019) 218:280–6. 10.1016/j.ajog.2017.08.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Yee LM, Martinez NG, Nguyen AT, Hajjar N, Chen MJ, Simon MA. Using a patient navigator to improve postpartum care in an urban women's health clinic. Obstet Gynecol. (2017) 129:925–33. 10.1097/AOG.0000000000001977 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Valaitis RK, Carter N, Lam A, Nicholl J, Feather J, Cleghorn L. Implementation and maintenance of patient navigation programs linking primary care with community-based health and social services: a scoping literature review. BMC Health Serv Res. (2017) 17:116. 10.1186/s12913-017-2046-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-Year review. Front Public Health. (2019) 7:64. 10.3389/fpubh.2019.00064 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare. Implement Sci. (2018) 13:27. 10.1186/s13012-017-0707-4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Ferrante JM, Cohen DJ, Crosson JC. Translating the patient navigator approach to meet the needs of primary care. J Am Board Fam Med. (2010) 23:736–44. 10.3122/jabfm.2010.06.100085 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.