Abstract
Background
This qualitative study aims to identify facilitators of and barriers to the implementation of Meaning-Centered Psychotherapy (MCP) by providers of mental health services to Latinos in the US and Latin America using the practical, robust implementation and sustainability model (PRISM). This information will be used to increase usability and acceptability of MCP for Latino patients with cancer and their providers in Latin America and the US.
Methods
A total of 14 Latino cancer patient mental health providers completed in-depth semi-structured interviews. Participants were recruited from 9 countries and 12 different sites. They provided feedback about barriers to and facilitators of implementation of MCP at the patient, provider, and clinic levels in their clinical setting. The qualitative data from the interviews was coded according to PRISM domains. Three analysts independently coded the transcripts; discrepancies between analysts were resolved through discussion and consensus.
Results
Based on PRISM, themes were: clinic environment (protected time for training and supervision), intervention characteristics (adapt the intervention using more simple language, include more visual aids, include more family-oriented content), patient (develop materials for the identification and screening of patients, provide educational materials, increase motivation and knowledge about psychotherapy, assess commitment to psychotherapy, adapt for the inpatient vs. outpatient setting), provider (receive interactive/participatory training, educational materials, ongoing supervision, have flexibility of delivering the intervention in a less structured manner, theoretical framework of the provider) and external environment (work at policy level to integrate services for oncology patients).
Conclusion
These qualitative data revealed potential facilitators and barriers of this intervention (MCP) on an international scale. Identified cultural, contextual, and healthcare systems factors illustrated the importance of examining pre-implementation needs prior to implementing a trial. We will design and plan a future RCT using the PRISM framework and these pre-implementation data.
Plain Language Summary
This study integrates frameworks of implementation science and cultural adaptation through the examination of pre-implementation contextual issues at the preparation phase of a cultural adaptation of a psychotherapeutic intervention, Meaning-Centered Psychotherapy (MCP), for Latinos with advanced cancer. By examining implementation needs early in the implementation process, during preparation, the intervention can be adapted in a way that attends to and addresses the providers’ most cited challenges in implementation: having a rigid protocol/structure, complexity of the intervention, disease burden preventing adherence to the intervention, transportation, and competing demands, and limited clinic space to offer the intervention.
Keywords: Latin America, language, delivery of mental health, qualitative research, psychotherapy, neoplasms, hispanic Americans, cultural adaptation, implementation science, cultural competence
Introduction
The 2020 worldwide estimated incidence of new cancers was 19.3 million and of new deaths was 10 million. Approximately 7.6% of new cancers and 7.1 new deaths occurred in Latin America (Caribbean, Central America, and South America) (GLOBOCAN, 2020). Worldwide in many low-resource countries, the majority of cancer patients are diagnosed with advanced-stage disease (American Cancer Society, 2018). Many patients with advanced cancer suffer from emotional and spiritual distress and symptoms, end-of-life despair, and hopelessness (McClain et al., 2003; Smith et al., 2003), and a number of psychotherapeutic interventions have been designed to address this suffering; in particular, Meaning-Centered Psychotherapy (MCP) has shown great promise. MCP was designed to target the specific psycho-spiritual needs of patients with advanced cancer by enhancing a sense of meaning, peace, and purpose as they face an advanced diagnosis (Breitbart et al., 2018; Breitbart et al., 2012; Breitbart et al., 2010).
Although several psychotherapeutic interventions have been designed for advanced cancer (Breitbart et al., 2018; Breitbart et al., 2012; Breitbart et al., 2010; Edmonds et al., 1999; Kissane et al., 2007; Lorenz et al., 2008; Luckett et al., 2011; Okuyama et al., 2017; Uitterhoeve et al., 2004), none of them have been developed or adapted for Latinos. Further, previous research has shown that Latinos diagnosed with cancer have less access to psychosocial and psychotherapeutic services (Castro-Figueroa et al., 2019; Costas-Muñiz et al., 2020; Costas-Muniz et al., 2017; Luckett et al., 2011) and that multilevel and contextual factors (e.g., individual, provider, health system, and social) might influence the access to and uptake of psychosocial services for Latinos diagnosed with cancer (Costas-Muñiz et al., 2020). Primary barriers to the access and use of psychosocial services in the general cancer patient population include having no perceived need for psychosocial services and support, lack of information or wanting more information about services (Dilworth et al., 2014), uncertainty about the need for specialized mental health care, and the cost of such treatment (Greenberg, 2004), receipt of support from other sources outside the cancer clinic, lack of awareness of services, and lack of provider referral (Eakin & Strycker, 2001). Among Latinos diagnosed with cancer, barriers include lack of information or wanting more information about services, lack of awareness of where to receive services, uncertainty about the need for specialized mental health care, and availability of culturally competent services (Costas-Muñiz et al., 2020).
Culturally adapted interventions are far more effective than non-adapted interventions for ethnic minority patients (Benish et al., 2011; Chowdhary et al., 2014; Griner & Smith, 2006; Hall et al., 2016). Mental health interventions with participants from only one cultural group are 4 times more effective than interventions provided to multicultural groups, and those conducted in the participants’ native languages are twice as effective as interventions conducted in English (Griner & Smith, 2006). Cultural adaptation is the systematic modification of an intervention protocol to consider language, culture, and context so that the intervention is compatible with the client’s cultural patterns, meanings, and values (Bernal, 2009).
Implementation science refers to the scientific study of methods to promote the integration of research findings and evidence-based interventions into health care policy and practice. While a cultural adaptation model or framework aims to facilitate the modification and study of an intervention to make it compatible with a different/new culture, implementation models aim to facilitate the study and integration of interventions into practice (Cabassa & Baumann, 2013). According to Cabassa and Baumann (2013), cultural adaptation models usually fall short in informing the intervention implementation process because they tend to ignore important contextual factors that impact how evidence-based treatments are used and sustained in organizations and community settings. Many cultural adaptation models propose that adaptations have to take place early on in the implementation process, particularly when deciding which interventions to implement (exploration phase) and when preparing for the adoption of the new treatment (preparation phase) in the new settings and populations. However, they provide little guidance for deciding when adaptations to other important implementation elements (e.g., service context, organizational characteristics) should be considered and how to incorporate these adaptations throughout the implementation process.
The Practical, Robust Implementation and Sustainability Model (PRISM) (Feldstein & Glasgow, 2008; Woodbridge et al., 2014) is a useful implementation health care delivery model for contexts in which there is a principal concern for intervention sustainability, such as in low-resource contexts. Culturally and linguistically adapting interventions that consider implementation and contextual factors and studying implementation needs and processes early in the preparation phase of cultural adaptation are of critical importance. PRISM is a practical and comprehensive implementation framework that integrates aspects of diffusion of innovation, models for quality improvement, and reach, effectiveness, adoption, implementation, maintenance (RE-AIM) (Feldstein & Glasgow, 2008; Woodbridge et al., 2014). PRISM’s dimensions include (1) organizational perspectives on an intervention, (2) external environment, (3) recipients’ characteristics, (4) implementation and sustainability infrastructure (training and supervision supports), and (5) intervention design (Feldstein & Glasgow, 2008; Woodbridge et al., 2014). PRISM is a framework that specifies constructs that may influence implementation and has been used for global adaptations of interventions because it guides the examination of the intervention interactions with the characteristics of multilevel contexts/factors that may influence uptake, implementation, integration, and outcomes (McKay et al., 2020). This approach is critical for cultural adaptations in a global health context.
In previous mixed-methods studies, information was gathered to culturally adapt MCP Costas-Muniz et al. (2020b) with attention to the dimensions of the ecological validity model (EVM) of cultural adaptation: language, context, persons, metaphors, concepts, goals, and methods (Bernal, 2009). Although the EVM states the need to adapt to the socioeconomic, political, and multisystem context of the recipients, it does not provide clear direction on how to do it. Therefore, in prior studies, we have used the PRISM model to analyze facilitators of and barriers to adaptation in context, considering the domains of external environment, intervention design, implementation infrastructure, the adopting organization, and intervention recipient’s characteristics. This study builds on the findings of these prior studies by examining pre-implementation factors that might influence the uptake and delivery of MCP on an international scale with Latinos. Specifically, this qualitative study aims to identify multilevel factors that might facilitate or hinder the future implementation of the culturally adapted intervention of MCP for Latinos (MCP-L) in oncological contexts. This study explores perceived facilitators of and barriers to the implementation of MCP-L with providers of mental health services to Latino patients in the United States and in Latin American countries.
Guided by PRISM (Feldstein & Glasgow, 2008; Woodbridge et al., 2014), we used a qualitative approach with stakeholders (service chiefs, mental health providers), to identify contextual and site-specific MCP-L barriers and facilitators prior to implementation. We selected PRISM to explore issues of implementation and sustainability, considering how the external environment, intervention design, implementation infrastructure, adopting organizations (with particular emphasis on health care teams and providers), and patients could influence program implementation and success. This paper aims to contribute to implementation science using PRISM to integrate cultural adaptation and implementation at the intervention preparation phase.
Methods
Description of intervention
MCP for Latinos: MCP is a seven-session intervention based on the work of Viktor Frankl and his concepts of logotherapy (Frankl, 1984, 1988, 2010). Frankl’s basic concepts include (1) meaning of life (the need to find meaning in our existence is a basic primary motivating force shaping human behavior), (2) life has meaning (life has meaning and never ceases to have meaning, or the potential for meaning, from the first moments of life up to the last, although what is meaningful may sometimes change over time as people’s circumstances change), (3) freedom of will (the “freedom” to find meaning in our existence and to choose our attitude toward suffering), (4) creativity, experience, and attitude as the 3 main sources of meaning in life, and (5) meaning existing in a historical context. MCP is designed to help patients with advanced cancer to improve their spiritual well-being by sustaining or enhancing a sense of meaning, peace, and purpose as they approach the end of life. The goals of MCP are to enhance hope, make sense of the cancer experience, and help to attain a sense of legacy. The primary desired outcome of MCP-L is to improve the spiritual well-being of Latino cancer patients, as measured by the Spanish language version of the Functional Assessment of Chronic Illness Therapy—Spiritual Well-being Scale (FACIT-Sp), which assesses a sense of meaning in life, harmony, peacefulness, and a sense of strength and comfort from one’s faith (Peterman et al., 2002). Secondary desired outcomes are to decrease psychological symptoms and improve quality of life. MCP has proven efficacy and acceptability in predominantly English-speaking White patients with advanced cancer in trials conducted in the United States (Breitbart et al., 2018; Breitbart et al., 2012; Breitbart et al., 2010). Evidence supports the efficacy of individual and group formats of MCP in enhancing meaning (as measured by the Meaning/Peace subscale of the FACIT-Sp scale) (Peterman et al., 2002) and reducing symptoms of anxiety, depression, and hopelessness and improving the quality of life in advanced cancer patients (Breitbart et al., 2018; Breitbart et al., 2012; Breitbart et al., 2010). An adaptation targeting the unique meaning-related needs of Latinos with advanced cancer has not yet been tested.
Research framework
The PRISM model has guided the data collection and inquiries of the interviews, and deductive analysis was guided by this theoretical framework. The PRISM model (Feldstein & Glasgow, 2008; Woodbridge et al., 2014) considers how the domains of external environment (elements such as payors, policy, or competition, which influence implementation), intervention or program design (intervention elements from the perspective of the organization and the patients), implementation infrastructure (infrastructure within a given context, which influences implementation and sustainability of an intervention), the adopting organization, and intervention recipients (organization and patient characteristics, which influence the intervention’s ability to be implemented) influence program implementation and success. PRISM is ideal for guiding this MCP-L multilevel implementation effort because it builds on several implementation science frameworks and can guide activities at the early pre-implementation/preparation phase. This study describes the perspectives of providers who work at the patient, provider, and clinic/administration levels regarding the implementation of MCP for use with Latino cancer patients.
Participants/sample
Recruitment of providers was conducted through the FIPOL (Formación de Investigación Psicosocial Oncológica Latinoamericana [Training on Psychosocial Oncology Research for Latin America]) network’s listserv (see www.fipol.info). The FIPOL network includes clinicians and researchers interested in psychosocial oncology for Latinos. Through the FIPOL network, contact information of eligible providers was identified. Providers were approached by email and completed an eligibility screener. All approached providers agreed to participate. Using purposive sampling, 15 providers were selected because of their ability to provide complementary information based on their background, country, and type of role and provided written informed consent online. The interviews were conducted in Spanish via videoconference by the principal investigator. Only 14 completed the interview; one could not complete it due to a lack of technological/internet access. This selected sample included 14 mental health providers of Latino cancer patients from 9 countries and 12 different clinical sites, with varying roles and positions in their institutions. A sample of 14 mental health providers from 9 countries participated in this qualitative study (e.g., United States, Spain, Chile, Colombia, Argentina, Puerto Rico, Mexico, Peru, and Cuba). All had an active clinical practice with Latino cancer patients and had >5 years of experience and practice. Ten worked at major cancer centers, three in cancer clinics embedded in hospitals, and one in a private community cancer clinic. Ten worked in the public sector (major public cancer institutes) and four in the private sector. Seven of the providers were service chiefs or coordinated the psycho-oncology service or mental health unit of their clinics/hospitals. All provided consultancy and psychotherapeutic services to adults diagnosed with cancer. Thirteen of them also reported having active research programs.
Interviews were conducted from May to July 2019 by the principal investigator who is bicultural and bilingual and has over 10 years of experience in psycho-oncology and qualitative research. The interviews were 31–95 min long. The research received institutional review board approval on January 25, 2019 (45 CFR 46.111 and/or 21 CFR 56.111; protocol number: 15-076).
Measures
The mental health providers completed in-depth semi-structured interviews. The interviews were divided into two parts. This manuscript presents the findings of the second part of the interview. In the first part, the MCP manual content was shared with the mental health providers and they were asked questions about comprehension and acceptability of the goals, themes, concepts, and strategies of MCP and acceptability and feasibility of MCP strategies. The questions of the first part were guided by the EVM (cultural adaptation framework) (Costas-Muniz et al., 2017; Costas-Muniz, Torres-Blasco, et al., 2021). The second part included questions about implementation barriers and facilitators. Two main questions (What would be facilitators of the implementation of MCP-L? What would be barriers of the implementation of MCP-L?) with three probes (e.g., Facilitators and barriers at the patient, provider, and clinic/administration level) assessed the potential for future implementation of MCP-L in their settings. The sample provided feedback about future contextual (barriers and facilitators) implementation of MCP at the individual-patient, provider, and clinic levels in their clinical setting.
Analyses
This study uses a deductive content analysis qualitative framework. Deductive content analysis is used when the structure of analysis is operationalized on the basis of previous knowledge and/or is guided by a theoretical framework (Elo & Kyngas, 2008; Mayring, 2015). A deductive approach that is based on a specific theory or model moves from the general information provided to specific themes (Elo & Kyngas, 2008; Mayring, 2015). All qualitative data were managed and coded with Atlas.ti software using the PRISM domains and their categories. The interviews were recorded and transcribed for analysis. The transcription was conducted by a bilingual research assistant who was previously trained by a qualitative research specialist. The categories from the PRISM model were selected a priori; and themes were classified using the PRISM categories. Three analysts (from a team of four analysts including the investigator, research assistant, and graduate students) independently coded the same transcripts. The author with expertise in qualitative methods and implementation science (RCM) trained all the analysts prior to the coding activities, which included didactic learning about deductive analysis and the PRISM framework and discussion of the PRISM concepts and meanings. Intercoder reliability was conducted through team-based consensus-building by discussing points of divergence and convergence in team meetings. When there was divergence in the use of a code, the team reviewed the narrative and proposed codes and their definitions. Each analyst provided a rationale for the use of the proposed code/s. These discussions continued until the group reached a consensus on the codes selected and their application. The author (RCM) moderated these discussions.
Results
The PRISM model (Feldstein & Glasgow, 2008; Woodbridge et al., 2014) guided the qualitative analysis, and we coded the themes according to the six PRISM domains: intervention or program design (includes organizational perspective and patient perspective), external environment, implementation infrastructure, and intervention recipients (includes organization and patients characteristics) and its 39 categories. Table 1 shows the data domains, categories, and quotes
Table 1.
Qualitative domains, categories, and quotations, guided by Practical, Robust Implementation and Sustainability Model (PRISM).
| Qualitative domain | Categories | Providers endorsing No. (%) | 
|---|---|---|
| Intervention: Organizational perspective | Readiness | 3 (21.4) | 
| Strength of the evidence base | 1 (7.1) | |
| Addresses barriers of frontline staff | 6 (42.9) | |
| Coordination across departments and specialties | 2 (14.3) | |
| Burden (complexity and cost) | 3 (21.4) | |
| Usability and adaptability | 8 (57.1) | |
| Trialability and reversibility | 0 | |
| Ability to observe results | 0 | |
| Intervention: Patient perspective | Patient centeredness | 8 (57.1) | 
| Provides patient choices | 2 (14.3) | |
| Addresses patient barriers | 8 (57.1) | |
| Seamlessness transition between program elements | 1 (7.1) | |
| Service and access | 3 (21.4) | |
| Burden (complexity and cost) | 4 (28.6) | |
| Feedback results | 0 | |
| External Environment | Payor satisfaction | 0 | 
| Competition | 0 | |
| Regulatory environment | 4 (28.6) | |
| Reimbursement | 0 | |
| Community resources | 0 | |
| Implementation and Sustainability Infrastructure | Performance data | 0 | 
| Dedicated team | 0 | |
| Adopter training support | 10 (71.4) | |
| Relationship and communication with adopters (bridge researchers) | 3 (21.4) | |
| Adaptable protocols and procedures | 13 (92.9) | |
| Facilitation of sharing best practices | 6 (42.9) | |
| Plan for sustainability | 0 | |
| Recipients: Organizational characteristics | Organizational health and culture | 1 (7.1) | 
| Management support and communication | 2 (14.3) | |
| Shared goals and cooperation | 2 (14.3) | |
| Clinical leadership | 0 | |
| Systems and training | 0 | |
| Data and decision support | 1 (7.1) | |
| Staffing and incentives | 0 | |
| Expectation of sustainability | 0 | |
| Recipients: Patient characteristics | Demographics | 2 (14.3) | 
| Disease burden | 6 (42.9) | |
| Competing demands | 1 (7.1) | |
| Knowledge and beliefs | 3 (21.4) | 
Intervention or program design
Categories under the Organizational Perspective of the intervention included readiness (n = 3), strength of the evidence base (n = 1), addresses barriers of frontline staff (n = 6), coordination across departments and specialties (n = 2), burden (complexity and cost) (n = 3), usability and adaptability (n = 8), trialability and reversibility, and ability to observe results. Eight providers shared themes related to usability and adaptability of the intervention, and six shared themes related to addresses barriers of frontline staff. Examples of themes about usability and adaptability of the intervention included flexibility of the intervention to incorporate topics brought by patients; adaptability of the language used in the intervention for individuals with low literacy; flexibility to skip or reorder sessions, as needed or clinically relevant; and flexibility to deliver the intervention accommodating the patient’s schedule and clinic space availability. Regarding usability and adaptability of the intervention a provider commented: “Yes, I think it’s useful, in the sense that I think it’s a really good guide to use, again, with flexibility.” Examples of themes about addresses barriers of frontline staff were having staff, such as nurses and assistants, able to identify the need for this intervention and promote it; having more mental health professionals to deliver the intervention; and having secure space available (private office space) to deliver the intervention. Other themes mentioned were (1) readiness of the providers to discuss end-of-life topics with patients and readiness of the intervention itself to be implemented in the context of practitioners’ settings; (2) sharing information about the evidence base of the intervention for its use with Latinos; (3) efficient coordination with oncology departments and providers to receive referrals for the intervention; (4) reducing the complexity of the language and concepts of the intervention for use with the patient population; and (5) considering the cost of providing outpatient services across different settings and scenarios.
Themes related to the Patient Perspective domain of the intervention included patient centeredness (n = 8), provides patient choices (n = 2), addresses patient barriers (n = 8), seamlessness transition between program elements (n = 1), service and access (n = 3), burden (complexity and cost) (n = 4), and feedback results. Eight providers commented about both the importance of patient centeredness and addresses patient barriers. The importance of patient centeredness was reflected in comments related to (1) considering the needs of each patient (when choosing them, when adapting the intervention); (2) recommending having a strong therapeutic relationship before delivering the intervention, due to the sensitive nature of the topics; (3) incorporating patients’ needs and desires, such as literacy or family involvement; (4) getting an orientation or endorsement from the oncologist to be well informed; and (5) “focusing on improving meaning at an advanced stage [. . .] will reduce suffering.” See Table 2 for exemplary quotations. For example, regarding patient centeredness, a provider stated: “I think that there should be a therapeutic relationship first, trust with the therapist and that you can make a space where you meet beforehand or at least do introductory sessions where empathy is built because I feel they include very valuable topics of life.”
Table 2.
Quotation examples by category and domain.
| Domain and categories | Quotations | 
|---|---|
| Intervention: Organizational Perspective | |
| Readiness | “So I think we have to be very careful… with the topic of death and trainees. Because depending on what practices and how much exposure they’ve had, it could be awkward. Merely because of ignorance and because they really do not feel comfortable (with the topic of death).” | 
| Strength of the evidence base | “we don’t have evidence-based interventions that allow us to be able to relieve these patients [with advanced cancer]. So we definitely are going to use it [the intervention].” | 
| Addresses barriers of frontline staff | “I’m just thinking cost [of training]. And so you know something that’s affordable/feasible for most people [providers].” | 
| Coordination across departments and specialties | “…that a nurse, like a secretary, like a doctor, can detect the need in patients, a matter that if they do not necessarily have the psychological services integrated within the clinic, they can make the appropriate referral, and be able to benefit from an intervention like this.” | 
| Burden (complexity and cost) | “…the subject of death that could be intimidating or complex, but no less necessary…the extent to which the provider/therapist feels comfortable with the subject, it will be easier to work it.” | 
| Usability and adaptability | “Yes, I think it’s useful, in the sense that I think it’s a really good guide to use, again, with flexibility.” | 
| Intervention: Patient Perspective | |
| Patient centeredness | “I think that there should be a therapeutic relationship first, trust with the therapist and that you can make a space where you meet beforehand or at least do introductory sessions where empathy is built because I feel they include very valuable topics of life.” | 
| Provides patient choices | “I think that also, giving the importance that family members recognize the importance of also providing emotional support to their family members…give them the opportunity to participate [family members] if [the patients] they prefer it….” | 
| Addresses patient barriers | “I think the most obvious topic is death. There are many people who are reluctant to work or talk about it, but that will be known quickly. …the issue of death could be threatening, but no less necessary. That is, you have to work it well, basically…In other words, present it well…” | 
| Seamlessness transition between program elements | “I think there must be a prior therapeutic relationship when faced with doing [Meaning Centered] therapy.” | 
| Service and access | “We should find a [physical] space so that patients can come and also that the mobility of the patients allows them to participate…sometimes the schedules…the traffic…the distance…” | 
| Burden (complexity and cost) | “And the person does not have the plan, if s/he has the money, s/he agrees, and there are people who do not have money and are left without access.” | 
| External Environment | |
| Regulatory Environment | “There are few doctors who are perhaps integrated in what is psychological care in the environment of cancer care and so I think that perhaps make more public policies in the area.” | 
| Implementation and Sustainability Infrastructure | |
| Adopter training support | “My biggest comment is that we can be given training to be able implement this spectacular program.” | 
| Relationship and communication with adopters (bridge researchers) | “I think too that there could be, apart from the training, to be some way that therapists when they are stuck or they have questions, again whether they from the community, through the training, or that they have a go to person or people that they can go to when they’re stuck or they’re sort of needing or where something kind of….They got a reaction that they have never gotten and they don’t know how to proceed with it, someone to go to…” | 
| Adaptable protocols and procedures | “As long as the structure maintains flexibility, I think it’s very useful. And obviously what I mean by flexibility is not that you don’t meet the goals, but you well know that one can come with an “x” goal for patients that are inconsistent or with group patients that one sees regularly, and when you get there they change everything.” | 
| Facilitation of sharing best practices | “Yes, definitely, I think that a discussion between peers or supervisors and supervised, could help a lot.” | 
| Recipients: Organization Characteristics | |
| Organizational health & culture | “Here (cancer institute) the psychological (care) is well integrated, it is seen as another component of health (care)” | 
| Management support and communication | “as a (national) public hospital/institution, it supports this type of interventional research” | 
| Shared goals and cooperation | “the nurse, assistant or physician could be able to detect the need…even if they do not have it (mental health care) integrated but could offer and adequate referral” | 
| Data and decision support | “Help the [medical] team…to identify patients who could benefit more out of it.” | 
| Recipients: Patient Characteristics | |
| Demographics | “…I think you can get people to see that in Spanish too, but again the language has to be more colloquial…” | 
| Disease burden | “And also an impediment if people are at the end of life, maybe mobility…or that they’re patients and by sitting in a chair due to pain or discomfort will not let them be comfortable for a long time.” | 
| Competing demands | “Many times patients cannot come for economic reasons…those from the province…only come for cancer treatment, but not for psychology. What we do is that, if they come from the province…once their treatment appointments are over…we place the psychology appointments.” | 
| Knowledge and beliefs | “We do this with patients who are going to do radiotherapy and chemotherapy in different ways, but in psycho-prophylaxis surgery within the program, the doctor gives him a paper, orders…And when it’s over everything goes back to the doctor and one of those is to go through mental health services and the one who has a rejection says: ‘But I’m not crazy, why do I have to go to the psychologist?’ So it seems to me that a facilitator would be to do something with psychoeducation.” | 
Comments related to addresses patient barriers included (1) the need for the intervention to be adapted for patients with low literacy and low education; (2) reducing the stigma of using mental health care by inviting family members (to help normalize the process) and offering mental health care as integrated oncological care; (3) patient’s own suffering being a barrier to introspection and discussion of transcendental topics; (4) patients’ financial and insurance coverage being an impediment if they have to pay for the sessions, have a high deductible or only a certain number of services or sessions are approved; (5) having transportation barriers (lack of transportation or having to pay for transportation); (6) having geographical barriers (patients who reside in distant or remote areas and can only travel sporadically to receive treatments); and (7) patient’s willingness to adhere to a psychotherapeutic intervention; and 8) having physical limitations, such as speech problems or mobility restrictions.
External environment
Regarding the External Environment, providers commented about the regulatory environment. They (n = 4) shared the importance of optimal insurance coverage for mental health care of cancer patients; the importance of recognizing psycho-oncology care as integral care at the public policy level; and, in countries with a universal health care system, facilitating the implementation of these types of interventions. See Table 2 for exemplary quotations, including: “There are few doctors who are perhaps integrated in what is psychological care in the environment of cancer care and so I think that perhaps make more public policies in the area.”
Implementation and sustainability infrastructure
Providers shared comments about Implementation and Sustainability Infrastructure in the following categories: adopter training support (n = 10), relationship and communication with adopters (bridge researchers) (n = 3), adaptable protocols and procedures (n = 13), and facilitation of sharing best practices (n = 6). Most providers reported the need for adequate training support that is (1) remote or virtual, (2) ongoing, (3) with adequate supervision of cases, (4) affordable or free, (5) with emphasis on improving confidence and comfort in discussing end-of-life topics, (6) group supervision format (to learn from the trainer and peers), (7) training in Spanish, (8) training at different levels (basic, intermediate, and advanced training), (9) building a virtual learning community to support continued education, and (10) facilitating contact with other therapists who are implementing the intervention. A quote illustrating the need for training support was: “My biggest comment is that we can be given training to be able implement this spectacular program.”
Almost all the providers reported the need to have an adaptable protocol and procedure, which included (1) a flexible structure that allows flexibility to address topics or issues that might arise during the cancer trajectory; (2) an adapted protocol that uses simple language for patients with low literacy or low education; (3) ability to alter the sequence of sessions or topics if clinically relevant; and (4) adaptability for patients with both recent or long-term diagnoses; and 5) ability to combine content, if needed, for patients who might have difficulty committing to the 7 sessions.
Recipients
Other organizational (organizational health and culture: n = 1, management support and communication: n = 2, shared goals and cooperation: n = 2, and data and decision support: n = 1) and patient characteristics (demographics: n = 2, disease burden: n = 6, competing demands: n = 1, knowledge and beliefs: n = 3) that might facilitate the implementation of MCP are were also discussed by participants. Notably, providers reported that the disease burden (e.g., patients who are not too sick), competing demands (e.g., patients who are not too busy with treatments), patient demographics (e.g., education or literacy level), and knowledge and beliefs (e.g., patient belief/outlook on mental health), are important considerations when recommending and implementing these types of interventions. For example, one provider commented: “…I think you can get people to see that in Spanish too, but again the language has to be more colloquial…” See Table 2 for exemplary quotations.
Discussion
PRISM themes highlighted potential barriers to and facilitators of implementation of MCP-L in diverse settings in Latin America and the United States and identified thematic areas to improve mental health care for Latino cancer patients.
Culturally adapted interventions are far more effective than non-adapted interventions for ethnic minority patients (Benish et al., 2011; Chowdhary et al., 2014; Griner & Smith, 2006; Hall et al., 2016) and consideration of key contextual factors of implementation should be a critical part of any cultural adaptation program. Our previous research informed the cultural adaptation and translation of MCP with attention to the dimensions of the EVM: language, context, persons, metaphors, concepts, goals, and methods (Bernal, 2009). For the EVM, context refers to the consideration of the client’s broader social, economic, health care, and political contexts. Context, also includes, the socioeconomic stressors that influence adaptation, access to care (i.e., insurance status), as well as, issues related to the health care system context where the patient receives care. As such addressing barriers to care should be seen as a key element of a comprehensive cultural adaptation. In this study, we used the PRISM model to focus on contextual facilitators and barriers guided by the PRISM model (dimensions of external environment, intervention design, implementation infrastructure, and intervention recipients’ characteristics). The EVM model highlights the need to address context, but the PRISM provides a more detailed framework of the multilevel dimensions and characteristics that need to be considered. In general, the themes (categories of the PRISM model) that were identified as potential barriers and facilitators by four or more providers (out of 14) were (1) addresses barriers of frontline staff, (2) usability and adaptability (of the intervention), (3) patient centeredness (of the intervention), (4) addresses patient barriers, (5) burden (complexity and cost) (of the intervention from the patient’s perspective), (6) regulatory environment (public policy regarding mental health care for cancer patients), (7) adopter training support, (8) adaptable protocols and procedures (flexibility in the protocol), (9) facilitation of sharing best practices (information for ongoing implementation), and (10) patient’s disease burden.
Using PRISM (Feldstein & Glasgow, 2008; Woodbridge et al., 2014) to assess possible future implementation yielded important insights into potential barriers to and facilitators of MCP-L implementation in diverse settings and helped identify crucially important areas for potential adaptations. Potential challenges, such as having a rigid or inflexible protocol/structure, complexity of the intervention, disease burden, transportation, and competing demands, and limited clinic space to offer the intervention, training needs, were among the most cited challenges. Providers also reported that the patients’ competing demands (e.g., too many appointments), patient demographics (e.g., education or literacy level), and knowledge and beliefs (e.g., patient belief/outlook on mental health), are important patient-level considerations when implementing these types of interventions. Many of these will be addressed with three general approaches: a rigorous cultural adaptation and transcreation of the intervention manual to make it responsive to the needs of patients and providers (increase flexibility, decrease complexity of content to address literacy, addressing mental health stigma in the intervention), intervention implementation through telehealth (remotely) or through the use of a hybrid model of presential and telehealth implementation (a more feasible approach for patients with greater disease burden, transportation issues, and competing demands and limited clinic space), and interventionist training support and supervision using in the same manner resources online (remote supervision, webinars, website, blog, email communication, etc.) and/or hybrid approach of presential and online training.
Notably, an important facilitator highlighted by the providers was training support and supervision. Providers stated that different levels of training are needed: beginning, intermediate, and advanced. A previous study with psychology graduate students from Puerto Rico demonstrated a high level of interest in clinical practice and research in psycho-oncology (80.2%, n = 81), but there was poor general knowledge about and few opportunities for specialized training (13.6%, n = 14), highlighting the need to develop training opportunities (Pérez-Ramírez, 2017). Further, at the basic level and with early stage professionals, it is important to address the comfort of the provider who is working with patients with advanced illness and engaging in therapeutic conversations about disease progression and end-of-life experiences. Previous research has shown that mental health providers who counsel Latinos with advanced cancer perceive end-of-life conversations as challenging because patients may find it culturally unacceptable to talk about death; patients may avoid end-of-life conversations to maintain hope for improved health, they may be fearful or in denial, or they may have a narrow focus on hope and survival (Costas-Muniz, Garduno-Ortega, et al., 2020). Training is needed that addresses these difficult conversations to improve competency and comfort in maintaining them.
Training should also include ongoing supervision with different educational modalities, such as videoconferencing, web-based education, printed material, and virtual community support. Implementing a comprehensive model of training, including supervision, access to remote learning activities, and enhanced peer-based collaborative learning, are approaches that could address this implementation need. Coordinating with the oncological team to detect patients who might benefit from MCP and providing adequate education about the intervention and mental health services in general, can also be important approaches that might facilitate the intervention’s implementation. Developing materials for the identification and screening of patients, and developing promotion and advertising strategies about mental health treatment in general and MCP specifically could improve patient and provider intervention acceptability and improve referrals and self-referrals.
Regarding the external environment, providers commented that recognizing at the public policy level psycho-oncology as integral to oncological care (e.g., including psychosocial care in national plans, public support for insurance coverage) facilitates the implementation of these types of psychosocial interventions. It is critically important the support not only of the integration of psychosocial oncology care in the national cancer care plans, but also the development, dissemination, and implementation of practice guidelines based on expert consensus and/or scientific evidence sponsored or developed by professional organizations, institutes, and other public health entities to support the development and implementation of evidence-based practice. Further, previous research has highlighted the importance of studying the practice of psycho-oncology with underserved populations and in low-resource settings to address disparities in access to psychosocial oncology services and support the use of evidence-based practice (Costas-Muniz et al., 2021). Improving the acceptance and use of general psychosocial care in cancer will facilitate the implementation and sustainability of interventions, such as MCP-L. The work of professional organizations, such as the International Psycho-oncology Society (www.ipos-society.org/), FIPOL network (Costas-Muniz et al., 2022) (www.fipol.info), and the Latin American Network of Psycho-Oncology (www.facebook.com/SomosRELPO/), is critically needed to bring awareness and acceptance of psycho-oncology, promote it as being integral to oncological care, and promote the integration of psychosocial care into Latin American national cancer programs or cancer plans, so it can receive the support needed (Grassi et al., 2012).
Considering these findings, the cultural adaptation plan of MCP-L includes these modifications:
Increase flexibility: The intervention manual includes cultural adaptation notes with instructions based on previous findings on how to modify or tailor the intervention to address low literacy, death stigma, linguistic and national diversity, etc. For example, some exercises, and readings are optional, assignments can be completed during the session, etc.
Decrease complexity of content to address literacy: All the manual content and language were revised, simplified, and we added visual aids (e.g., images, icons, and graphics) to address low literacy.
Addressing mental health stigma in the intervention: The first session includes information (and a video link) about what psychotherapy/counseling is, who can benefit from it, the importance of receiving it during the cancer experience, and the benefits of it.
Intervention implementation through telehealth: Interventionists receive training pertinent to the use of telehealth as well as the Guidelines of the Use of Telehealth in Psycho-Oncology for Latin America (developed by the FIPOL network). As stated, telehealth is particularly useful for patients with greater disease burden, transportation issues, and competing demands/multiple appointments.
Interventionist training support and supervision: The first author has developed an interactive virtual interventionist training program that includes information about the implementation of MCP-L based on prior research, information about cultural adaptations, remote supervision, and email communication. In the next phase of this research, we will develop a website to include training materials (videos, readings) and a blog that will facilitate peer communication.
Using PRISM (Feldstein & Glasgow, 2008; Woodbridge et al., 2014) to understand the pre-implementation needs of a culturally adapted intervention provided the opportunity to identify barriers that are generalizable to the implementation of culturally adapted behavioral or psychotherapeutic interventions across sites with different characteristics and contexts. Our study shows the importance of the integration of cultural adaptation and implementation and dissemination models to address important contextual factors that impact how evidence-based interventions are used and sustained in organizations and low-resource community settings. By adopting and examining implementation needs early in the implementation process, during preparation, the intervention can be adapted in a way that includes important elements of implementation (e.g., service context, organizational characteristics) (Cabassa & Baumann, 2013). Our findings demonstrate the need to identify challenges to and facilitators of implementation during the preparation phase of the intervention. Our main findings of challenges to and facilitators of implementing this intervention in low resource settings in the United States and Latin America (having a rigid protocol/structure, complexity of the intervention, disease burden preventing adherence to the intervention, transportation, and competing demands, limited clinic space to offer the intervention, training support) can also inform the preparation/formative phase of future cultural adaptations of other interventions aimed at Latino cancer patients.
Limitations and next steps
This evaluation was conducted by a qualitative method that may be difficult to replicate in other contexts or interventions. Our approach considered only the perspectives of mental health providers and did not include the perspectives of patients, oncological health care providers, or administrators and did not use other means of data collection, such as review of printed materials and reports, interviews with other stakeholders, and questionnaires and other validated assessments used in implementation science. Consideration of patient and stakeholder perspectives (health care providers including oncologists, nurses, etc.) is a key domain of PRISM, and future work should include these perspectives along with those of other providers in addition to using other research approaches. This would yield a more comprehensive view of implementation needs at an early stage in the cultural adaptation process. This study was conducted with providers from the United States and Latin America, and some of the contextual factors, facilitators, and barriers identified may be unique to different countries. Future studies should determine which contextual elements are present by country of practice and which contextual elements influence the implementation of psychosocial and psychotherapeutic interventions in diverse practice settings, including academic medical centers, national institutes, hospital-embedded cancer clinics, and community clinics.
Conclusion
Our qualitative data demonstrated barriers and potential facilitators of this psychotherapeutic intervention on an international scale. Cultural, contextual, as well as health care systems factors, illustrate the necessity of examining pre-implementation needs prior to conducting a trial. These pre-implementation data will be used to design and implement the intervention RCT guided by the PRISM framework.
Acknowledgments
The authors would like to thank Sonya J. Smyk, Memorial Sloan Kettering, for editorial support.
Footnotes
Author contributions: Contributed to the design of the analyses and data collection: RCM, NTB, MC.
Conceived and designed the analyses: RCM, NTB, MC.
Collected the data: RCM, NTB, ECF, MC, BN, OGV, MFM, and JCS.
Contributed data or analysis tools: RCM, NTB, MC.
Performed the analyses: RCM, NTB, MC.
Wrote the paper: RCM.
Drafted the paper: RCM, NTB, ECF, MC, OGV, MFM, JCS, FG.
Critical revision of the paper: RCM, NTB, ECF, MC, OGV, MFM, JCS, FG.
Final approval of the version to be published: RCM, NTB, ECF, MC, OGV, MFM, JCS, FG.
Availability of data and materials: The datasets generated and/or analyzed during the current study are not publicly available due to the institutional data sharing policy, but, are available from the corresponding author on reasonable request.
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethics approval and consent to participate: The research was approved by the Memorial Sloan Kettering Cancer Center Institutional Review Board on January 25, 2019 (45 CFR 46.111 and/or 21 CFR 56.111; protocol number: 15-076).
Consent for publication: Not applicable.
ORCID iDs: Rosario Costas-Muñiz https://orcid.org/0000-0003-0942-5657
Jose C. Sanchez https://orcid.org/0000-0002-5713-4566
References
- American Cancer Society (2018). Global cancer facts & figures 4th edition. American Cancer Society. [Google Scholar]
 - Benish S. G., Quintana S., Wampold B. E. (2011). Culturally adapted psychotherapy and the legitimacy of myth: A direct-comparison meta-analysis. Journal of Counseling Psychology, 58(3), 279–289. 10.1037/a0023626 [DOI] [PubMed] [Google Scholar]
 - Bernal G., Jiménez-Chafey M. I., Domenech Rodríguez M. M. (2009). Cultural adaptation of treatments: A resource for considering culture in evidence-based practice. Professional Psychology: Research and Practice, 40(4), 361–368. 10.1037/a0016401 [DOI] [Google Scholar]
 - Breitbart W., Pessin H., Rosenfeld B., Applebaum A. J., Lichtenthal W. G., Li Y., Saracino R. M., Marziliano A. M., Masterson M., Tobias K., Fenn N. (2018). Individual Meaning-Centered Psychotherapy for the treatment of psychological and existential distress: A randomized controlled trial in patients with advanced cancer. Cancer, 124(15), 3231–3239. 10.1002/cncr.31539 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Breitbart W., Poppito S., Rosenfeld B., Vickers A. J., Li Y., Abbey J., Olden M., Pessin H., Lichtenthal W., Sjoberg D., Cassileth B. R. (2012). Pilot randomized controlled trial of individual Meaning-Centered Psychotherapy for patients with advanced cancer. Journal of Clinical Oncology, 30(12), 1304–1309. https://ascopubs.org/doi/10.1200/JCO.2011.36.2517 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Breitbart W., Rosenfeld B., Gibson C., Pessin H., Poppito S., Nelson C., Tomarken A., Timm A. K., Berg A., Jacobson C., Sorger B., Abbey J., Olden M. (2010). Meaning-centered group psychotherapy for patients with advanced cancer: A pilot randomized controlled trial. Psycho-Oncology, 19(1), 21–28. 10.1002/pon.1556 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Cabassa L., Baumann A. (2013). A two-way street: bridging implementation science and cultural adaptations of mental health treatments. Implementation Science, 8(1), 90. 10.1186/1748-5908-8-90 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Castro-Figueroa E. M., Torres-Blasco N., Rosal M. C., Castro W. P., Gonzalez-Lorenzo M. E., Velez H., Costas-Muniz R., Jimenez J. C. (2019). Preferences, use of and satisfaction with mental health services among a sample of Puerto Rican cancer patients. PloS One, 14(7), e0216127. Article e0216127. 10.1371/journal.pone.0216127 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Chowdhary N., Jotheeswaran A. T., Nadkarni A., Hollon S. D., King M., Jordans M. J., Rahman A., Verdeli H., Araya R., Patel V. (2014). The methods and outcomes of cultural adaptations of psychological treatments for depressive disorders: A systematic review. Psychological Medicine, 44(6), 1131–1146. 10.1017/S0033291713001785 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muniz R., Castro-Figueroa E., Torres N., Claros M., Galindo-Vazquez O., Narang B., Gany F. M. (2021). Practice of psycho-oncology with Latino patients: an international study. Psycho-Oncology, 30(1), 127–133. 10.1002/pon.5552 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muniz R., Galindo Vázquez O., Montana M. F., Castro E., Torres N., Mg L. R., Mg J. C. S., Esenarro L., Bergerot C. D., Castro J., Gany F., Breitbart W. (2022). FIPOL: An initiative to support research collaborations, resource development, scientific communication, and educational opportunities in psychosocial oncology for Latin America. Psycho-Oncology. 10.1002/pon.5922 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muñiz R., Garduño-Ortega O., González C. J., Rocha-Cadman X., Breitbart W., G F. (2017). Cultural and linguistic adaptation of meaning-centered psychotherapy for Spanish-speaking Latino cancer patients. In W. Breitbart (Ed), Meaning-centered psychotherapy in the cancer setting: Finding meaning and hope in the face of suffering (pp. 134–144). Oxford University Press. [Google Scholar]
 - Costas-Muniz R., Garduno-Ortega O., Torres-Blasco N., Castro-Figueroa E., Gany F. (2020a). “Maintaining hope:” challenges in counseling Latino patients with advanced cancer. J Psychosoc Oncol Res Pract, 2(3), e028. 10.1097/OR9.0000000000000028 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muniz R., Hunter-Hernandez M., Garduno-Ortega O., Morales-Cruz J., Gany F. (2017). Ethnic differences in psychosocial service use among non-Latina White and Latina breast cancer survivors. Journal of Psychosocial Oncology, 35(4), 424–437. 10.1080/07347332.2017.1310167 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muniz R., Torres-Blasco N., Castro-Figueroa E. M., Gonzalez C. J., Breitbart W., Gany F. (2020b). Meaning-Centered Psychotherapy for Latino patients with advanced cancer: cultural adaptation process [article]. Journal of Palliative Medicine, 23(4), 489–497. 10.1089/jpm.2019.0423 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Costas-Muñiz R., Garduño-Ortega O., Hunter-Hernández M., Morales J., Castro-Figueroa E. M., Gany F. (2020). Barriers to psychosocial services use for Latina versus non-Latina White breast cancer survivors. American Journal of Psychotherapy, 74(1), 13–21. 10.1176/appi.psychotherapy.20190036 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Dilworth S., Higgins I., Parker V., Kelly B., Turner J. (2014). Patient and health professional's perceived barriers to the delivery of psychosocial care to adults with cancer: A systematic review [review]. Psycho-Oncology, 23(6), 601–612. 10.1002/pon.3474 [DOI] [PubMed] [Google Scholar]
 - Eakin E. G., Strycker L. A. (2001). Awareness and barriers to use of cancer support and information resources by HMO patients with breast, prostate, or colon cancer: patient and provider perspectives. Psycho-Oncology, 10(2), 103–113. http://www.ncbi.nlm.nih.gov/pubmed/11268137. 10.1002/pon.500 [DOI] [PubMed] [Google Scholar]
 - Edmonds C. V., Lockwood G. A., Cunningham A. J. (1999). Psychological response to long-term group therapy: A randomized trial with metastatic breast cancer patients. Psycho-Oncology, 8(1), 74–91. [DOI] [PubMed] [Google Scholar]
 - Elo S., Kyngas H. (2008). The qualitative content analysis process. Journal of Advanced Nursing, 62(1), 107–115. 10.1111/j.1365-2648.2007.04569.x [DOI] [PubMed] [Google Scholar]
 - Feldstein A. C., Glasgow R. E. (2008). A Practical, Robust Implementation and Sustainability Model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety, 34(4), 228–243. 10.1016/S1553-7250(08)34030-6 [DOI] [PubMed] [Google Scholar]
 - Frankl V. E. (1984). Man's search for meaning. Washington Square Press. [Google Scholar]
 - Frankl V. E. (1988). The Will to Meaning: Foundations and Applications of Logotherapy. Meridian. http://books.google.com/books?id = 5gImYTIc7BcC.
 - Frankl V. E. (2010). The Doctor and the Soul: From Psychotherapy to Logotherapy. Knopf Doubleday Publishing Group. http://books.google.com/books?id = v_qiURM2r2MC.
 - GLOBOCAN (2020). Estimated number of new cancers and deaths in 2020, worldwide, both sexes, all ages https://gco.iarc.fr/today/online-analysis-table?v = 2020&mode = cancer&mode_population = continents&population = 900&populations = 900&key = asr&sex = 0&cancer = 39&type = 1&statistic = 5&prevalence = 0&population_group = 0&ages_group%5B%5D = 0&ages_group%5B%5D = 17&group_cancer = 1&include_nmsc = 1&include_nmsc_other = 1#collapse-group-1-13-2.
 - Grassi L., Watson M., co-authors I. F. o. P.-O. S. (2012). Psychosocial care in cancer: An overview of psychosocial programmes and national cancer plans of countries within the international federation of psycho-oncology societies [review]. Psycho-Oncology, 21(10), 1027–1033. 10.1002/pon.3154 [DOI] [PubMed] [Google Scholar]
 - Greenberg D. B. (2004). Barriers to the treatment of depression in cancer patients. Journal of the National Cancer Institute, 2004(32), 127–135. 10.1093/jncimonographs/lgh019 [DOI] [PubMed] [Google Scholar]
 - Griner D., Smith T. B. (2006). Culturally adapted mental health intervention: A meta-analytic review. Psychotherapy, 43(4), 531–548. 10.1037/0033-3204.43.4.531 [DOI] [PubMed] [Google Scholar]
 - Hall G. C. N., Ibaraki A. Y., Huang E. R., Marti C. N., Stice E. (2016). A meta-analysis of cultural adaptations of psychological interventions. Behavior Therapy, 47(6), 993–1014. 10.1016/j.beth.2016.09.005 [DOI] [PubMed] [Google Scholar]
 - Kissane D. W., Grabsch B., Clarke D. M., Smith G. C., Love A. W., Bloch S., Snyder R. D., Li Y. (2007). Supportive-expressive group therapy for women with metastatic breast cancer: survival and psychosocial outcome from a randomized controlled trial. Psycho-Oncology, 16(4), 277–286. 10.1002/pon.1185 [DOI] [PubMed] [Google Scholar]
 - Lorenz K. A., Lynn J., Dy S. M., Shugarman L. R., Wilkinson A., Mularski R. A., Morton S. C., Hughes R. G., Hilton L. K., Maglione M. (2008). Evidence for improving palliative care at the end of life: A systematic review. Annals of Internal Medicine, 148(2), 147–159. 10.7326/0003-4819-148-2-200801150-00010 [DOI] [PubMed] [Google Scholar]
 - Luckett T., Goldstein D., Butow P. N., Gebski V., Aldridge L. J., McGrane J., Ng W., King M. T. (2011). Psychological morbidity and quality of life of ethnic minority patients with cancer: A systematic review and meta-analysis. Lancet Oncology, 12(13), 1240–1248. 10.1016/S1470-2045(11)70212-1 [DOI] [PubMed] [Google Scholar]
 - Mayring P. (2015). Qualitative content analysis: theoretical background and procedures. In A. Bikner-Ahsbahs, C. Knipping & N. Presmeg (Eds), Approaches to qualitative research in mathematics education. Advances in Mathematics Education. Springer. 10.1007/978-94-017-9181-6_13 [DOI] [Google Scholar]
 - McClain C. S., Rosenfeld B., Breitbart W. (2003). Effect of spiritual well-being on end-of-life despair in terminally-ill cancer patients. Lancet, 361(9369), 1603–1607. 10.1016/s0140-6736(03)13310-7 [DOI] [PubMed] [Google Scholar]
 - McKay M. M., Sensoy Bahar O., Ssewamala F. M. (2020). Implementation science in global health settings: collaborating with governmental & community partners in Uganda. Psychiatry Research, 283, 112585. 10.1016/j.psychres.2019.112585 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Okuyama T., Akechi T., Mackenzie L., Furukawa T. A. (2017). Psychotherapy for depression among advanced, incurable cancer patients: A systematic review and meta-analysis. Cancer Treatment Reviews, 56, 16–27. 10.1016/j.ctrv.2017.03.012 [DOI] [PubMed] [Google Scholar]
 - Peterman A. H., Fitchett G., Brady M. J., Hernandez L., Cella D. (2002). Measuring spiritual well-being in people with cancer: The Functional Assessment of Chronic Illness Therapy–Spiritual Well-being Scale (FACIT-Sp). Annals of Behavioral Medicine, 24(1), 49–58. 10.1207/S15324796ABM2401_06 [DOI] [PubMed] [Google Scholar]
 - Ramírez E. P., Blasco N. T., Ortega O. G., Castro E., Muñiz R. C. (2017). Necesidades de Adiestramiento de Investigación en Medicina Conductual, Psicología de la Salud y Psico-oncología en Estudiantes Graduados en Puerto Rico [Behavioral medicine, health psychology and psycho-oncology puerto rican graduated students research training needs]. Revista Puertorriquena de Psicologia, 28(2), 296–313. Spanish. [PMC free article] [PubMed] [Google Scholar]
 - Smith E. M., Gomm S. A., Dickens C. M. (2003). Assessing the independent contribution to quality of life from anxiety and depression in patients with advanced cancer. Palliative Medicine, 17(6), 509–513. 10.1191/0269216303pm781oa [DOI] [PubMed] [Google Scholar]
 - Uitterhoeve R. J., Vernooy M., Litjens M., Potting K., Bensing J., De Mulder P., van Achterberg T. (2004). Psychosocial interventions for patients with advanced cancer: A systematic review of the literature. British Journal of Cancer, 91(6), 1050–1062. 10.1038/sj.bjc.6602103 [DOI] [PMC free article] [PubMed] [Google Scholar]
 - Woodbridge M. W., Sumi W. C., Yu J., Rouspil K., Javitz H. S., Seeley J. R., Walker H. M. (2014). Implementation and sustainability of an evidence-based program: lessons learned from the PRISM applied to first step to success [article]. J of Emot and Behav Disord, 22(2), 95–106. 10.1177/1063426613520456 [DOI] [Google Scholar]
 
