Adapting care approaches for patients with diabetes and depressive symptoms in India improved patient-provider communication and patient satisfaction and engagement with care.
Keywords: Adaptation, Scale-up, Integrated care, Diabetes management
Abstract
Integrated care models offer one approach to treat patients with chronic multimorbidity. However, because these models consist of multiple components designed to provide individualized care, they should be adapted to best meet the needs of patients in diverse settings. This paper presents qualitative and quantitative data from a realist process evaluation embedded in the INtegrating DEPrEssioN and Diabetes treatmENT (INDEPENDENT) study to uncover what worked, for whom, how, and in what circumstances. We aimed to examine adaptations made to a care coordinator-led behavioral intervention, and how patients responded to adaptations, through a secondary analysis of these data. A purposive sample of 62 patients and 3 care coordinators were recruited based on their involvement with the INDEPENDENT care model across two clinics. Patients were interviewed and surveyed about their experiences in the care model and care coordinators were interviewed about their experiences implementing intervention components. Interview data were coded for reported modifications in intervention content, tools, and delivery; then, these adaptations were categorized by how they served to enhance implementation in the Indian context. Adaptations made in the delivery of this care model served two functions: (a) to improve health promotion communication between care coordinators and patients and (b) to improve patient engagement. Patients’ expressed needs were consistent with what care coordinators perceived. Patients were satisfied with adaptations designed to improve intervention fit for local contexts. Study findings demonstrate that adapting components of an integrated care model can enhance patient satisfaction and engagement with behavioral interventions.
Implications.
Practice: The adaptation of behavioral intervention components of the INDEPENDENT care model provides examples of how health practitioners in India can improve communication and engagement with patients in chronic care settings.
Policy: Integrated depression and diabetes care models can be implemented in diverse clinical settings if adaptations are made to align intervention components with patient-centered values and take into consideration social and physical determinants of health within the local context.
Research: Further research is needed on how variations in context impact the implementation of adaptation strategies within and across clinical settings.
Introduction
The bidirectional link between diabetes and depression is well documented and it is estimated that approximately 15.1% of the roughly 72.9 million people with diabetes in India are affected by depression [1,2]. Depression is associated with poor health behaviors (e.g., sedentariness, smoking, and high-calorie diet) and an increased risk of Type 2 diabetes and associated complications [3,4]. Patients with diabetes and comorbid depression are less likely to attend medical appointments or adhere to medical treatment plans because they often experience reductions in emotional and physical energy levels that reduce their ability to carry out self-management practices or attend appointments [5–7]. Additionally, access to mental health services in India is limited by the availability of trained mental health professionals (0.75 psychiatrists per 100,000 people) [8]. In order to improve the delivery of physical health care among patients with comorbid depression, health services interventions have focused on coordinating care for patients through referral systems and the delivery of psychotherapy in the primary care setting [9,10]. Alternative care models that combine physical and mental health care services in a specialized care setting are emerging [11,12]. However, the vast majority of these health systems innovations are developed in high-income countries, which contributes to a widening gap in health service needs since the global burden of depression and diabetes remains highest in low- and middle-income countries.
Merging and adapting models of integrated care delivery from the USA (TEAMcare [13]) and India (CARRS Translational Trial [14]), a study team of investigators from the USA and India developed the INtegrating DEPrEssioN and Diabetes treatmENT (INDEPENDENT) care model [15]. The INDEPENDENT care model was designed to improve access to mental health care, lower depressive symptoms, and improve cardiometabolic disease outcomes among patients with diabetes and comorbid depressive symptoms in India. The utilization of nonspecialized staff to treat mental health problems and the integration of mental health care services into other care settings are approaches advocated in low- and middle-income countries as a means to broaden the base of mental health services in low-resource settings [16,17]. In line with these recommendations, the INDEPENDENT model uses care coordinators, nonspecialized support staff members, to provide depression and diabetes self-care support, monitor patient health outcomes with the use of a decision support and electronic health record system, and keep clinicians accountable for providing and intensifying therapies (both behavioral change counseling and medications) when patient health outcomes are not improving. Care coordinators serve as a link between the patients, the patients’ usual care diabetes physicians who provide routine care for patients, and specialists (i.e., a psychiatrist and endocrinologist) involved in population health management (through case reviews). Diabetes poses a large and growing burden in India [18]; therefore, integrating depression treatment into the diabetes care platform may increase the reach of mental health service provision.
The adaptation and evaluation of integrated care models are complicated by the care models’ sensitivity to local context and the inclusion of individualized care approaches. Despite these challenges, there is evidence that psychological treatment using nonspecialist health workers is more effective in low- and middle-income countries when intervention components are adapted for contextual and cultural relevance [19,20]. In India, Patel et al. have culturally adapted psychological treatments for depressive disorders [21], with a focus on integrating mental health services at the primary care level [22,23]. In these studies, adaptations were made to language, therapist selection, materials, and the treatment approach in order to ensure that the treatment was accessible and acceptable to patients [24]. Diabetes prevention trials have also been adapted for Indian populations living in the UK and India [25,26]. Culturally sensitive approaches for promoting combined diabetes and depression management have yet to be developed and tested.
Formative work for the INDEPENDENT study identified strategies for how to adapt integrated depression and diabetes for the Indian context [27]. Of relevance to the behavioral intervention components, patients and health care professionals indicated that to be effective and culturally appropriate, the intervention would need to engage family members in the treatment process, use simple educational materials and patient-friendly language, and provide strategies for coping with stigma. With evidence that culturally tailoring behavioral health interventions improves intervention effectiveness and participant satisfaction [28,29], this formative data were used to guide initial adaptations to the intervention. However, in addition to this, there were adaptations in delivery that were implemented by the care coordinators on an individualized basis.
In order to promote the adoption, sustainability, and scale-up of evidence-based interventions for diabetes and depression management in India, the purpose of this mixed-methods qualitative and quantitative study is to explore what adaptations were made during delivery to culturally tailor care coordinator-led behavioral intervention components of the INDEPENDENT care model. For the purposes of this study, intervention adaptation was defined as any modifications in intervention content, tools, and delivery based on the local implementation context [30,31]. These adaptations include modifications as reported being made by the care coordinators following the initial adoption and implementation of the integrated care model across the clinic sites. This study seeks to document how contextual factors relevant to the Indian population, and subcultural groups, that impact health behavior and health care utilization inform when and why adaptation processes where made in a chronic care quality improvement intervention and how patients responded to those adaptations.
METHODS
A secondary analysis of qualitative data collected as a part of a realist process evaluation of the INDEPENDENT trial was conducted to highlight adaptations to the behavioral intervention components and assess patients’ satisfaction, alongside a brief questionnaire. The study was approved by the Institutional Review Boards of Emory University, USA, and the All India Institute of Medical Sciences and Madras Diabetes Research Foundation, India.
Evaluation framework
A type of theory-driven evaluation, the realist evaluation framework seeks to explain why and how intervention outcomes are achieved. The philosophical underpinnings of this approach propose that individuals are agents of change independent of their environment [32]. Creators Pawson and Tilley propose that, in order to determine what works, for whom, in what circumstances, and why, researchers must identify and examine the intervention’s underlying mechanisms, the contexts under which the mechanisms operate, and the patterns of outcomes triggered by interactions between actors and their environment [32–34]. In sum, the realist evaluation framework operates through three key elements: (a) mechanisms, (b) contexts, and (c) outcome patterns [35]. Mechanisms describe the processes of how implementers and users interpret and act in response to intervention components. Context describes the conditions that support or hinder intervention mechanisms. Outcome patterns are the results achieved through the activation of different combinations of mechanism and context. For this process evaluation, a realist framework was used to explain under what conditions and why care coordinator-led behavioral intervention components were adapted to fit the context of urban diabetes clinics in India. The framework is method neutral, though often one or more quantitative and qualitative methods are used to develop, test, and refine program theories on how interventions operate [35]. Qualitative interview data from two groups of intervention stakeholders (i.e., patients and care coordinators) were examined to identify and explore adaptations made to the behavioral components of the INDEPENDENT care model in addition to using a questionnaire to quantitatively assess patient satisfaction with the care patients received in the study.
Study settings
This case study was conducted in two urban diabetes clinics in India (i.e., Delhi and Chennai) implementing the INDEPENDENT care model. Since context is one of the three elements analyzed in the realist evaluation framework, these clinics were selected out of four implementation sites because they offered the greatest contrast in terms of several key characteristics detailed in Table 1. The variation in context allows for a comprehensive understanding of how and why intervention adaptations occurred by taking into account alternative circumstances that may have influenced the care coordinators’ implementation reasoning or the patients’ responses to the care model.
Table 1.
Characteristic | Site 1 | Site 2 |
---|---|---|
Geographic location | North India | South India |
Institution type | Government | Private |
Integration of psychiatric care | Collaboration with in-house psychiatry department | Collaboration with off-site psychiatrist |
Form of medication assistance available | Free or low-cost medication available at in-house pharmacy | In-house pharmacy or referral to off-site pharmacies offering free or low-cost medication |
The INDEPENDENT care model
The INDEPENDENT care model is a multicomponent intervention that focuses on reducing depressive symptoms and cardiovascular disease risk factors (≥50% reduction in symptoms checklist [SCL]-20 scores, ≥0.5% point reduction in glycated hemoglobin (HbA1c), ≥5 mmHg reduction in systolic blood pressure, or ≥10 mg/dL reduction in low-density lipoprotein cholesterol [LDL-c]) among outpatients with diabetes and moderate-to-severe depressive symptoms (patient health questionnaire [PHQ]-9 ≥10). This care model sought to improve patient health outcomes and quality of life via four components: (a) health worker-led participant-activation; (b) care coordinator-led individualization of care plans; (c) web-enhanced decision support software to prompt clinicians to be attentive to and modify therapies regularly; and (d) multidisciplinary team treat-to-goal case reviews designed to review and adjust treatment for patients whose depressive symptoms and cardiometabolic risk factors are not improving. Care delivery was supported with decision support software and patient case reviews with psychiatrists, endocrinologists, and a trained care coordinator. See Kowalski et al. [15] for the trial protocol and descriptions of all intervention components.
Behavioral interventions used by the care coordinators included: motivational interviewing, behavioral activation, and problem-solving treatment strategies. Motivational interviewing was used to strengthen patients’ commitment to behavior change through the use of reflective listening, reinforcement of positive statements, the assessment of patients’ readiness to change, and the affirmation of patients’ self-direction throughout the process of behavior change [36,37]. Behavioral activation strategies were used to engage patients in self-management by reinforcing positive behavior [38,39]. Lastly, structured procedures for problem-solving were used to address patients’ difficulties systematically [40,41].
Participants and procedure
Participants for this study were purposively sampled from the care coordinators and trial participants, aiming to represent patients varying by age, sex, educational background, household income levels, and health trajectories.
Care coordinators informed patients receiving treatment in the INDEPENDENT study about the opportunity to participate in an interview by telephone or in person during clinic visits. To minimize participation burden, patients were asked to participate during their current or upcoming clinic visits. Only a subsample of four to six patients per site was identified from clinic records as having a positive or negative mental health trajectory (increasing or decreasing PHQ-9 scores since study enrollment) in order to capture diversity in patient experiences based on mental health outcomes. Variation in other key demographics (e.g., age and sex) among patients occurred using this recruitment procedure without the need to target patients with those characteristics. Patient recruitment stopped when data saturation had been met at each site, meaning no new information was produced with additional interviews [42]. Only two patients at the Chennai site declined to participate because of time constraints.
Participants provided informed consent to participate in the interview and to complete a brief questionnaire. L.C.M.J. and a trained bilingual interviewer at each site conducted a total of 62 semistructured patient interviews from September 2017 to May 2018. Interviews were conducted at the end of the 1 year active intervention period for the majority of patients, with some patients having engaged in the INDEPENDENT care model for 6–11 months. Patients were asked about their motivation to participate, care experiences, and interactions with the care coordinators and physicians. Interviews were audio-recorded and ranged from 40 to 60 min in length. Due to diversity in patient medical histories, self-management practices, and the cultural lenses through which diabetes, depression, and self-care were understood, no new themes (i.e., patterns of meaning within the textual data) were identified during data monitoring and analysis procedures as of Interviews 27 (Site 1) and 35 (Site 2) at each clinic.
In September 2017, L.C.M.J. recruited the care coordinators at both study sites to participate in a one-on-one interview regarding the implementation of the INDEPENDENT care model. At this point in time, care coordinators had implemented the intervention with some patients for the full 1 year of active intervention and were able to reflect on the initial and ongoing implementation processes. Therefore, care coordinators were providing retrospective reflections on the adaptation process. L.C.M.J. obtained written consent prior to the interviews. Interviews were audio-recorded and lasted approximately 75 min each. Care coordinators were asked about each intervention component and how it functioned within their respective clinic sites. For example, they were asked what their roles and responsibilities were in the study, how confident they were in carrying out those responsibilities, and what resources were put into place in order to support them in successfully engaging patients in this model of care. Interview questions focused on the role of contextual factors and how they influenced the implementation, functioning, and experience of the integrated care model.
Qualitative data preparation and analysis
L.C.M.J. transcribed verbatim and deidentified all care coordinator interviews, while all patient interviews were translated and transcribed by the bilingual interviewer. Patient’s trial identification numbers were used to anonymize patient transcripts. A third-party staff member conducted quality control checks to ensure the accuracy of the meaning-based translations. Data were analyzed using a realist evaluation analytic approach [32,43], where contexts and mechanisms linked to desired implementation outcomes were coded within and across care coordinator transcripts. This study focused on data pertaining to the inputs (i.e., care coordinator training, decision support, and electronic health record system), mechanism (i.e., care coordinators’ confidence to carry out their patient counseling and support responsibilities), contextual factors (e.g., stigma, education level of patients, and clinic resources), and implementation outcomes related to the care coordinator-led behavioral intervention components (i.e., counsel patients regarding depression and diabetes self-care and provide responsive and individualized care).
Two team members independently coded all care coordinator transcripts for references to behavioral interventions (e.g., goal setting, motivational interviewing, and nutritional education), including data related to intervention resources related to the implementation of that intervention component, the individual’s response to the situation, and the context in which the event occurred. For example, in a narrative segment where a care coordinator described how they helped a patient come up with a way to improve medication adherence, the availability of clinical support systems and patients’ circumstances (e.g., lack of transportation) would be coded as context, care coordinators’ attitudes and beliefs about their ability to effectively counsel patients would be coded as a mechanism, and the coded outcome would be that the care coordinator provided counseling on patient self-management for their chronic conditions. All instances of behavioral interventions being carried out by care coordinators were coded for context, mechanism, and outcome within each transcript. Once all behavioral intervention references were identified within the transcripts, those segments were reviewed and sorted based on whether a modification to the behavioral intervention component, its content, tools, or delivery was described. Then, all data segments corresponding to adaptation features were thematically categorized according to the function of the adaptation.
Patient transcripts were subsequently coded for references to intervention adaptations (e.g., use of visual aids and family engagement) identified in the care coordinator transcripts in order to describe patients’ responses to the adapted intervention components. Care coordinator and patient data were triangulated, converging data on both patient and care coordinator descriptions of the identified adapted behavioral intervention components, to assess patient acceptability and satisfaction with the adaptation feature.
To ensure intercoder reliability, the coders met to compare the coded transcripts and discuss discrepancies in coded segments of textual data, as well as any discrepancies in the identification and categorization of adaption functions. Differences in coding and interpretations were resolved through discussions and, when a mutual agreement could not be achieved, a third-party member helped us reach consensus. Microsoft EXCEL was used to manage the data.
Questionnaire procedure and analysis
At the end of each patient interview, participants were asked to provide general feedback regarding their satisfaction with the treatment they had received in the INDEPENDENT trial by completing the eight-item Client Satisfaction Questionnaire [44]. This standardized satisfaction measure has been used widely across mental health service studies and exhibits very good internal reliability (α = .88) and construct validity (rs = .8) [45]. Questions, such as “To what extent has our program met your needs?” were scored on a scale of 1 to 4. The total score, ranging from 8 to 32, is calculated by summing up each item’s score, where higher scores indicate higher satisfaction. Additionally, there is evidence that the Client Satisfaction Questionnaire is correlated with client-reported symptoms, including depressive symptoms, where greater satisfaction is associated with fewer reported symptoms [46,47].
Because the majority of patients reported being illiterate or visually impaired, participants were administered the questionnaire orally in English, Hindi, or Tamil. The interviewer read each question and its corresponding response options aloud, marked the participant’s response, and then repeated the selected response option back to the participant to ensure the correct response was recorded. This questionnaire took approximately 5 min to administer. Survey data were entered into EXCEL where descriptive statistics were calculated.
RESULTS
Participants
All three care coordinators (aged 26–39 years) were female with an educational background in nutrition sciences. Table 2 presents the demographic characteristics of the patients interviewed. The participating patients were similar in terms of age and sex compared to the full trial sample. Participants represented all education levels, household income brackets, and types of insurance coverage.
Table 2.
Site 1 (n = 27) | Site 2 (n = 35) | Total sample (N = 62) | |
---|---|---|---|
M (SD) | |||
Age | 50.7 (7.7) | 53.2 (7.7) | 52.1 (7.7) |
Total number of household members | 5.5 (2.8) | 3.7 (1.2) | 4.5 (2.2) |
n (%) | |||
Sex (female) | 14 (51.8) | 20 (57.1) | 34 (54.8) |
Marital status | |||
Married | 26 | 28 | 54 |
Widow/widower | 1 | 4 | 5 |
Separated/divorced | – | 3 | 3 |
Education level | |||
Professional degree | 2 | 1 | 3 |
Graduate degree | 5 | 5 | 10 |
Secondary school | 14 | 24 | 38 |
Primary school | 5 | 4 | 9 |
>Primary (literate) | – | 1 | 1 |
>Primary (illiterate) | 1 | – | 1 |
Monthly household income (INR) | |||
3,000–10,000 | 1 | 9 | 10 |
10,001–20,000 | 11 | 12 | 23 |
20,001–30,000 | 6 | 9 | 15 |
30,001–40,000 | 2 | – | 2 |
40,001–50,000 | 3 | 1 | 4 |
>50,000 | 4 | 4 | 8 |
Health insurance | |||
No | 21 | 32 | 53 |
Yes, employer | 5 | 1 | 6 |
Yes, private | 1 | 2 | 3 |
SD standard deviation.
Patient satisfaction
Patients at the two sites were highly satisfied with the care they received in the INDEPENDENT care model. The average combined Client Satisfaction Questionnaire score was 29.7 (standard deviation [SD]: 2.16), with Site 1 (mean: 30.03, SD: 1.87) demonstrating a similar mean to Site 2 (mean: 29.35, SD: 2.35). Patients also expressed satisfaction with the counseling and support they received as part of the intervention during the evaluation interviews. For example, one patient captured the sentiments of many patients when he stated:
Coming here, there is happiness. We can gain understanding on certain things like how this happens, why it happens… rather being ignorant of diabetes and get admitted in hospital in critical stage. It is better to know things and avoid what to do and what not to do to keep ourselves healthy. Now I know which fruit will do what, and vegetables. They counselled me everything in the beginning. I am very satisfied here.
Even patients who chose not to consistently manage their conditions according to what they were counseled expressed satisfaction with their choice of lifestyle modifications and the guidance and support offered by the care coordinators. Patients reported positive views toward the intervention, expressing particular appreciation for the flexibility and cultural sensitivity of the care coordinators in their approach to counseling and follow-up self-care support.
Adaptations
Two functions of adapting behavioral intervention approaches were identified: (a) to improve health promotion communication between care coordinators and patients and (b) to improve patient engagement. The adaptations enacted to achieve these goals are listed in Table 3. All adaptation features were developed independently across sites. Only one adaptation feature, repurposing provider-facing electronic decision support tools (i.e., showing the patient decision support software generated graphics), was not implemented across both sites.
Table 3.
Adaptation function | Adaptation feature |
---|---|
Improve health promotion communication | • Integrate cultural values • Use colloquial terms to discuss depression • Emphasize visual aids within the prescribed intervention tools • *Repurpose provider-facing electronic decision support tools |
Improve patient engagement | • Maintain flexibility in how appointments are scheduled • Maximize privacy by using empty rooms or low-traffic areas to counsel patients • Engage family members when patients are receptive |
*Adaptation feature only occurred in Site 1.
Patient-care coordinator communication
Care coordinators worked with patients to educate them regarding diabetes and depression self-care (e.g., self-monitoring, adherence to medication, diet, exercise, and smoking cessation) and engaged them in behavioral activation techniques. Care coordinators reported received initial and ongoing training from investigators who had previous experience with TEAMcare; however, participants indicated that these trainings did not do enough to emphasize how the behavioral intervention components could be adapted to integrate Indian culture. For example, one care coordinator summarized the need to avoid using the term depression when discussing patient’s mental health by stating, “a lot of people will not take it in a very positive manner if you tell them ‘oh you are depressed’... because of the stigma that most people have attached with the word depression.” The main strategy used to address the mental health components of the intervention involved using colloquial terms and idioms of distress (e.g., “tension”) to discuss the occurrence of depressive symptoms with patients. As the trial progressed, care coordinators felt equipped to discuss sensitive topics with patients but still avoided using the term “depression” unless a patient referred to themselves as “depressed.”
Patients described feeling comfortable meeting with the care coordinators because they provided simple and clear explanations when communicating with patients. When listing examples of how their care coordinator helped them understand or respond to their health conditions, one patient shared, “she told me my tension has increased because of family issues,” while another shared the following guidance she received in a counseling session, “she guides me and tell me that it is all in the hands of God and we as human being cannot do anything in that. So, there is nothing to be worried about. Talking to her makes me very much relaxed and gives calm to my mind.” Care coordinators reported framing their sessions according to patient-centered values, and most patients described their approach to self-management in relationship to family commitments and religious beliefs. Patients frequently attributed a lack of motivation to their inability to overcome their “fate,” “destiny,” or “karma” as determined by their God. Religion was also recognized as having a protective effect by one patient who explains: “This disease motivates you to commit suicide but as I am from Jain community, I am not allowed to commit suicide. So, my caste and my medication together help me to live my life without taking any wrong decisions.” Care coordinators used these cultural values to contextualize the implementation of behavioral intervention components.
The care coordinators provided patients with written and visual educational materials at the onset of the trial. Though a minority of patients did not remember receiving these resources, those that did named the dietary tools with images and eating schedules as the most valuable resource they received during the intervention. As one patient described, “They have given a diet chart. It is a schedule regarding what and when I have to eat. [The care coordinators] have suggested me to eat pulses, gram, salad, etc. They have told me to eat less roti and more vegetables. All this has benefitted me a lot.” Patients reported receiving versions of these materials in either English or either of the two predominant local languages (i.e., Hindi or Tamil). One care coordinator highlighted the diverse linguistic landscape of India as a potential counseling barrier, stating “knowing the local language is very important. India is so vast and there are so many languages. So, I think that could be the barrier.” However, patients also reported the need for more language options when receiving pamphlets and educational resources with written text. Recognizing this limitation and the fact that some patients are illiterate, the care coordinators emphasized the visual aids incorporated into some of the intervention tools. Participants responded positively to the visual aids, as expressed by one patient, “[the care coordinator] advised that book for me because even if I can’t read it, I can understand from images, right!” Another patient highlighted the need for the use of visual aids over textual educational resources, stating: “I do not know how to read then what I will do of books?” Patients often linked the use of the dietary picture book with food diaries, where patients recorded their food intake so that they could more easily discuss dietary modifications with the care coordinators at their next counseling session. Though not all patients were prescribed insulin, only one person noted that the care coordinator provided a guide with visuals on what part of the body to inject insulin following an in-person educational demonstration.
The care coordinators found the visual elements of the electronic decision support software beneficial in tracking patients at the individual and clinic levels but, at Site 1, they also used the visual elements of the software to motivate patients. At the individual level, the software produces line graphs mapping patients’ medical trajectories across measures such as HbA1c, blood pressure, cholesterol levels, and the nine-item patient health questionnaire measuring depressive symptomology. At the clinic level, a dashboard lists all patients and assigns them a color, using traffic light color coding, based on the most up-to-date patient health information. The colors helped care coordinators prioritize patients by providing an at-a-glance summary of patients with uncontrolled diabetes and/or depressive symptoms. The care coordinators at this site reported using the dashboard and deidentified graphs of patients who were successful in managing their diabetes and depressive symptoms to motivate patients with uncontrolled cardiometabolic risk factors or depressive symptoms, who believed lifestyle interventions and medication adherence would not benefit them given the severity of their medical conditions. One care coordinator explained:
We show them the dashboard and show them how their values are in red. [It] really helps. Seeing the red, it does motivate them. We’ve done that with a couple of patients who aren’t doing well. Showing the graphs and the dashboard as well. Showing them as well how other patients are doing. A lot of times they feel that it’s something they have to live with throughout their life. So, when we tell them that this particular patient, who also came in at the same time, they came in you see how well they are doing because they are motivated and they are working on themselves. So, when they see, yes, somebody else’s A1C has improved they feel motivated.
Patients who had experienced this motivational technique appreciated that the visuals helped them understand their condition, both the current status relative to other patients and the potential progress they could make in improving their health.
Patient engagement
From taking an individual to her/his physician appointments to buying her/his medication, during the implementation of the INDEPENDENT care model, the family played an important role in supporting patient self-management across both sites. Care coordinators echoed the importance of family support when working with patients to set health goals in counseling sessions. One care coordinator explained that “by involving the family members, the patients feel better,” and another elaborated why this approach is better suited for the Indian context, stating:
I think in India we are just too family oriented. Of course, it’s disintegrating. When there were joint families it was so [much] stronger and now in India you find more nuclear families. However, I think socially we are the kind of people who will reach out to a family member when there is a crisis. We always knew that involving a family member is important or the results are much better when we include a family member.
To what extent and when to involve family was a decision each coordinator made on a case-by-case basis. After sharing two different incidents, one where a patient did not receive family assistance to buy medications and another where a patient needed help accessing emergency care, a care coordinator shared how engaging family members looks different for each patient, stating: “It depends on what the participant is dealing with, and then you set the short-term goals, thinking what is critical at that point of time.” Many patients reported being asked to invite one or more family members to attend a counseling session with them, with patients not in favor of this option making it clear that they did not want to burden their family members or that they felt the attendant would later hold personal information shared in the session against them.
Care coordinators also developed ways to better engage patients in the INDEPENDENT care model throughout the implementation process, recognizing that health was often not patients’ top priority. One care coordinator captured the relationship between socioeconomic status, family, and health among their clinic’s patient population when she stated, “people from lower socioeconomic background do not want to leave their jobs also because they are daily wagers. They do not give much importance to their health. They are just busy earning for their family.” Both sites accommodated patient’s evolving work and family commitments by maintaining flexible scheduling procedures. One individual captured the willingness of patients to engage with care coordinates because of this approach, stating, “I like their approach very much. They would not force me to visit on their convenient day but ask for my dates. They would adjust their tasks for me. This is what I like. This is the main reason. Only here we can see staff who would adjust things for us so I cooperate with them.” Similarly, care coordinators used phone appointments to check-in on patients and conduct counseling sessions when patients were unable to attend the clinic. In extreme cases, where patients were nonresponsive for an extended period of time, a home visit was conducted.
The behavioral interventions were designed to be implemented in private spaces; however, limitations in clinic space required care coordinators to adapt to their physical work environment in order to engage patients. Since the utilization of clinic space varied throughout the week, care coordinators rotated where they met with patients—selecting whichever available location offered the greatest amount of privacy. As one care coordinator elaborated: “You at least try and find some corner where you can sit and talk to the participant. There needs to be some amount of privacy so the patient opens up.” Private rooms were not always available, however, when patients came in for an appointment. In those cases, patients largely felt they “could not tell the remaining truth just because others could listen,” as stated by one patient. Only one individual expressed that being counseled in the presence of other patients was acceptable because she felt others in similar situations could learn from her situation.
Discussion
This mixed-methods study examined the adaptation processes of a quality improvement intervention incorporating collaborative care approaches developed in the USA for implementation in the Indian health care setting. Consistent with Resnicow et al.’s [48] model for cultural sensitivity, which posits that there are two dimensions (i.e., surface and deep structures) that need to be adapted in order for interventions to fit and be accepted within a specific culture, we found that both surface structure (i.e., the visual intervention characteristics and use of local terminology) and deep structure (i.e., integration of cultural elements into intervention strategies and messages) intervention components needed to be targeted in the adaptation process to satisfy patients and address sociocultural factors that influence health behavior change in the Indian population. Findings from this study demonstrate that when adaptations created alignment between perceived and actual needs of patients, patients expressed satisfaction with the care and support they received and a willingness to continue engaging with the care model.
Adaptations made to the behavioral intervention components in the INDEPENDENT care model aligned with cultural adaptations suggested by our formative work [27] and made to other diabetes or depression prevention programs targeting Indian communities [21,25,26], with family centeredness and patient education level as the underlying reasons for the modified approaches. Similar to these trials, presentation strategies (i.e., bilingual materials and staff and references to local lifestyle elements) were employed in this intervention [49]. While formative work highlighted that the intervention should seek to engage family members, employ patient-friendly language, provide simple education materials, and help patients cope with stigma, implementation strategies for these approaches were not specified or monitored across study sites. This provided flexibility for each site to implement these strategies; however, it best suited the needs of their local patient population. In this study, we also documented adaptation strategies that were not brought up during the formative research phase, such as integrating cultural values and providing flexibility in scheduling procedures. This is likely because several adaptations were motivated by barriers to implementing the care model that could not be anticipated and by resources offered through the intervention software, which had yet to be introduced to the care coordinators.
Drawing from the trial’s process evaluation data, we found that adaptation features, such as integrating cultural values, produced opposing effects regarding patient engagement in self-management practices. For example, care coordinators might use religion framing to educate patients on the cause of their depressive symptoms, while efforts to behaviorally activate patients could be set back as a result of a patient’s religious belief in predestination. Similarities and differences among adaptation features across the two sites highlight the need to assess how variations in context impact the implementation of different adaptation strategies.
The use of data from a realist evaluation of the INDEPENDENT trial allowed us to identify why and under what circumstances adaptations to the care coordinator-led behavioral intervention components and the content, tools, and delivery of those components occurred by exploring combinations of linked mechanisms and contexts that corresponded to how care coordinators implemented behavioral intervention components. Qualitative data from both implementation actors and service users stressed that each contextual factor existed on a continuum and that, over time, norms and attitudes of individuals, communities, and society change. As such, adaptations to behavioral interventions should be flexible and reassessed over time. Additionally, the contextual factors identified all meet the Healthy People 2020 definition of social and physical determinants of health [50], highlighting the need for health care policies centered on the framework of the human right to health [51]. Further research is required to develop and test intervention monitoring and evaluation measures that assess how social and physical determinants of health are factored into adaptation processes.
Strengths and limitations
This study’s use of a mixed qualitative and quantitative approach across service users and implementation actors allowed for the examination of intervention adaptations made by care coordinators in light of patients’ social-cultural beliefs from multiple data sources. This study is one of the first to use a realist evaluation approach to assess when, why, and under what circumstances intervention adaptations were made to culturally tailor a behavioral health intervention component. Adaptations to the function of intervention tools and clinic spaces reflect strategies to overcome implementation barriers characteristic of the model of care and institution types, both of which engage with patients from low socioeconomic backgrounds who may need visual education aids and flexibility in patient engagement to meet competing family and work needs. The contrast between the two selected trial sites highlighted variations in the content of linguistic strategies, where locally specific idiomatic phrases for distress were identified as means through which care coordinators could breach the topic of depression with patients. The study is limited by the fact that patients were largely recruited upon the basis of the clinic’s schedule, thus, making it more likely that patients with upcoming appointments would be recruited. The difference in languages spoken across clinic sites meant that patients had to be interviewed by different bilingual research assistants, which may have resulted in differences in data quality across sites. Both interviewers participated in an intensive qualitative training prior to the study in order to standardize interview techniques, as well as translation and transcription practices.
It is possible that implementation actors may have felt pressured to participate and underreport adaptations to trial components because the study was ongoing and adaptations may have been viewed as trial deviations to study leadership. As such, the informed consent process stressed voluntary participation. Additionally, L.C.M.J. is an American public health researcher whose presence as an outsider to the participants’ communities may have influenced their responses. For each interview, L.C.M.J. altered the power relations between the interviewer and the interviewee by giving participants the chair in each room reserved for the physician and then introduced herself, stating her role in the study so that participants understood how their responses would be used (i.e., for the purpose of evaluating the intervention and not auditing employees or providing medical advice or medications to patients).
The Client Satisfaction Questionnaire captured patients’ satisfaction with their overall health service experience participating in the INDEPENDENT trial. Since the care coordinator-led support and follow-up were the only added care elements visible to patients, it is likely that most patients reflected on their experience with the site care coordinator(s) when responding to the questionnaire, but interactions with site physicians may have also influenced satisfaction ratings. This questionnaire was administered orally due to the low literacy levels and visual impairments across the patients interviewed and, therefore, resulted in a 100% response rate. This procedure may have positively skewed the responses, however, as patients may find it difficult to openly express discontent with clinical services. Previous research in community mental health settings has shown that oral administration of the client satisfaction questionnaire, as opposed to a written self-reported mode of administration, resulted in 10% higher satisfaction ratings [52]. This study is strengthened by the use of quantitative and qualitative methods to explore patient’s perspectives and satisfaction with care as the patient interviews contextualize the results of this questionnaire with qualitative data reflecting why patients were or were not satisfied with the care they received.
Conclusions
This study demonstrates that the cultural tailoring of an intervention can occur iteratively following a formative research phase to adapt the intervention for local context. Therefore, public health practitioners seeking to implement behavioral health interventions in new settings may benefit from active monitoring and evaluation to assess the effectiveness of different strategies for different intervention components. These findings demonstrate that intervention implementation strategies, and not just intervention resources, need to be adapted. The rationale for selecting adaptation strategies reported here will be useful when scaling up this care model across diverse contexts, with contextual factors drawing attention to the need to consider how social and physical determinants influence implementation of behavioral health interventions. Public health practitioners seeking to implement similar models of integrated care in high-income countries may also draw from these findings to understand mechanisms by which health interventions are culturally adapted to suit the needs of diverse patient population in low-resource settings.
Acknowledgments:
The authors would like to acknowledge and thank a number of individuals without whom this study would not have been possible. We would like to thank the patients for their participation and time and the investigators and study staff at each of the clinic sites listed below.
Emory University, USA
Drs. V. Narayan and N. Mehta
University of Washington
Dr. Jurgen Unutzer
Madras Diabetes Research Foundation (MDRF), Chennai
Site PI: Dr. V. Mohan
Co-I: Drs. R. Shankar, R.M. Anjana, S. Jagdish, Parthasarathy, and S. Kasthuri
Lead Study Coordinator: Dr. S. Poongothai
Study Coordinator: Mr. P. Nandakumar
Recruiter/Screener: Mr. Viswanthan
Medical Officer: Dr. Ranjani
Care Coordinator (CC): Ms. B.B. Sundari
Outcomes Assessor (OA): Ms. K. Karkuzhali, Dr. Prathiba, and Dr. Rajalakshmi
Data Manager: Mrs. J. Rani
All India Institute of Medical Sciences (AIIMS), New Delhi
Site PI: Dr. N. Tandon
Co-I: Drs. R. Khadgawat and R. Sagar
Care Coordinator (CC): Ms. C. Chopra and Ms. B. Grover
Care Coordinator Support: Dr. D. Khakha
Outcomes Assessor (OA): Dr. R. Tandon and Mr. J. Joseph
Endocrine and Diabetes Centre (EDC), Vishakhapatnam
Site PI: Dr. G.R. Sridhar, Prof. K. Madhu
Care Coordinator (CC): Ms. S. Aruna Sri
Outcomes Assessor (OA): Dr. Ch. Venkateswarlu
Diacon Hospital, Diabetes Care and Research Center, Bangalore
Site PI: Dr. S.R. Aravind
Co-I: Dr. B. Sosale and Dr. P. Rai
Study Manager: Dr. Sunita
Care Coordinator (CC): R. Karthik
Outcomes Assessor (OA): Usha and R. Geethanjali
Funding: This study was funded by the National Institutes of Health through the Fogarty Global Health Fellows and Scholars Program (VECD Consortium, D43 TW009337) and the National Institute for Mental Health.
COMPLIANCE WITH ETHICAL STANDARDS
Conflicts of Interest: Leslie Johnson, Lydia Chwastiak, Subramani Poongothai, Nikhil Tandon, Ranjit Mohan Anjana, Sosale Aravind, Gumpeny Ramachandra Sridhar, Deepa Rao, Viswanathan Mohan, and Mohammed K. Ali declare that they have no conflicts of interest.
Authors’ Contributions: L.C.M.J conceived the project and planned it with L.C., S.P., N.T., R.M.A., S.A., G.R.S., D.R., V.M. and M.K.A. L.C.M.J collected and analyzed the data with support from S.P., N.T., R.M.A. and V.M. All authors contributed to the final manuscript.
Ethical Approval: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Ethical approval for this study was received from the Institutional Review Boards of Emory University, USA, and the All India Institute of Medical Sciences and Madras Diabetes Research Foundation, India. This article does not contain any studies with animals performed by any of the authors.
Informed Consent: Informed consent was obtained from all individual participants included in the study.
References
- 1. Mohan V, Sandeep S, Deepa R, Shah B, Varghese C. Epidemiology of type 2 diabetes: Indian scenario. Indian J Med Res. 2007;125(3):217–230. [PubMed] [Google Scholar]
- 2. International Diabetes Federation. IDF Diabetes Atlas. Brussels, Belgium: International Diabetes Federation; 2012. [Google Scholar]
- 3. Mezuk B, Eaton WW, Albrecht S, Golden SH. Depression and type 2 diabetes over the lifespan: A meta-analysis. Diabetes Care. 2008;31(12):2383–2390. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. de Groot M, Anderson R, Freedland KE, Clouse RE, Lustman PJ. Association of depression and diabetes complications: A meta-analysis. Psychosom Med. 2001;63(4):619–630. [DOI] [PubMed] [Google Scholar]
- 5. Gonzalez JS, Peyrot M, McCarl LA, et al. . Depression and diabetes treatment nonadherence: A meta-analysis. Diabetes Care. 2008;31(12):2398–2403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Bowser DM, Utz S, Glick D, Harmon R. A systematic review of the relationship of diabetes mellitus, depression, and missed appointments in a low-income uninsured population. Arch Psychiatr Nurs. 2010;24(5):317–329. [DOI] [PubMed] [Google Scholar]
- 7. Egede LE, Ellis C. The effects of depression on diabetes knowledge, diabetes self-management, and perceived control in indigent patients with type 2 diabetes. Diabetes Technol Ther. 2008;10(3):213–219. [DOI] [PubMed] [Google Scholar]
- 8. Garg K, Kumar CN, Chandra PS. Number of psychiatrists in India: Baby steps forward, but a long way to go. Indian J Psychiatry. 2019;61(1):104–105. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Butler M, Kane RL, McAlpine D, et al. . Does integrated care improve treatment for depression? A systematic review. J Ambul Care Manage. 2011;34(2):113–125. [DOI] [PubMed] [Google Scholar]
- 10. Egede LE. Disease-focused or integrated treatment: Diabetes and depression. Med Clin North Am. 2006;90(4):627–646. [DOI] [PubMed] [Google Scholar]
- 11. Happell B, Stanton R, Scott D. Utilization of a cardiometabolic health nurse—A novel strategy to manage comorbid physical and mental illness. J Comorb. 2014;4:22–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Katon W, Lin EH, Von Korff M, et al. . Integrating depression and chronic disease care among patients with diabetes and/or coronary heart disease: The design of the TEAMcare study. Contemp Clin Trials. 2010;31(4):312–322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Katon WJ, Lin EH, Von Korff M, et al. . Collaborative care for patients with depression and chronic illnesses. N Engl J Med. 2010;363(27):2611–2620. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Ali MK, Singh K, Kondal D, et al. ; CARRS Trial Group Effectiveness of a multicomponent quality improvement strategy to improve achievement of diabetes care goals: A randomized, controlled trial. Ann Intern Med. 2016;165(6):399–408. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Kowalski AJ, Poongothai S, Chwastiak L, et al. . The INtegrating DEPrEssioN and Diabetes treatmENT (INDEPENDENT) study: Design and methods to address mental healthcare gaps in India. Contemp Clin Trials. 2017;60:113–124. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Beck A, Nadkarni A, Calam R, Naeem F, Husain N. Increasing access to cognitive behaviour therapy in low and middle income countries: A strategic framework. Asian J Psychiatr. 2016;22:190–195. [DOI] [PubMed] [Google Scholar]
- 17. Rathod S, Pinninti N, Irfan M, et al. . Mental health service provision in low- and middle-income countries. Health Serv Insights. 2017;10: doi: 10.1177/1178632917694350. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Tripathy JP. Burden and risk factors of diabetes and hyperglycemia in India: Findings from the global burden of disease study 2016. Diabetes Metab Syndr Obes. 2018;11:381–387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Singla DR, Kohrt BA, Murray LK, Anand A, Chorpita BF, Patel V. Psychological treatments for the world: Lessons from low- and middle-income countries. Annu Rev Clin Psychol. 2017;13:149–181. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Chowdhary N, Sikander S, Atif N, et al. . The content and delivery of psychological interventions for perinatal depression by non-specialist health workers in low and middle income countries: a systematic review. Best Pract Res Clin Obstet Gynaecol. 2014;28(1):113–133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Chatterjee S, Chowdhary N, Pednekar S, et al. . Integrating evidence-based treatments for common mental disorders in routine primary care: Feasibility and acceptability of the MANAS intervention in Goa, India. World Psychiatry. 2008;7(1):39–46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Patel V, Chisholm D, Rabe-Hesketh S, Dias-Saxena F, Andrew G, Mann A. Efficacy and cost-effectiveness of drug and psychological treatments for common mental disorders in general health care in Goa, India: A randomised, controlled trial. Lancet. 2003;361(9351):33–39. [DOI] [PubMed] [Google Scholar]
- 23. Patel V, Weiss HA, Chowdhary N, et al. . Lay health worker led intervention for depressive and anxiety disorders in India: impact on clinical and disability outcomes over 12 months. Br J Psychiatry. 2011;199(6):459–466. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Chowdhary N, Jotheeswaran AT, Nadkarni A, et al. . The methods and outcomes of cultural adaptations of psychological treatments for depressive disorders: A systematic review. Psychol Med. 2014;44(6):1131–1146. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Wallia S, Bhopal RS, Douglas A, et al. . Culturally adapting the prevention of diabetes and obesity in South Asians (PODOSA) trial. Health Promot Int. 2014;29(4):768–779. [DOI] [PubMed] [Google Scholar]
- 26. Mathews E, Thomas E, Absetz P, et al. . Cultural adaptation of a peer-led lifestyle intervention program for diabetes prevention in India: The Kerala diabetes prevention program (K-DPP). BMC Public Health. 2018;17(1):974. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Rao D, Lipira L, Kumar S, et al. . Input of stakeholders on reducing depressive symptoms and improving diabetes outcomes in India: Formative work for the INDEPENDENT Study. Int J Noncommun Dis. 2016;1(2):65–75. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Huang YC, Garcia AA.. Culturally-tailored interventions for chronic disease self-management among Chinese Americans: A systematic review [published online January 31, 2018]. Ethn Health. 2018;1–20. doi:10.1080/13557858.2018.1432752. [DOI] [PubMed] [Google Scholar]
- 29. Barrera M Jr, Castro FG, Strycker LA, Toobert DJ. Cultural adaptations of behavioral health interventions: A progress report. J Consult Clin Psychol. 2013;81(2):196–205. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Smith E, Caldwell L. Adapting evidence-based programs to new contexts: what needs to be changed? J Rural Health. 2007;23(Suppl:37–41. [DOI] [PubMed] [Google Scholar]
- 31. Backer T. Finding the Balance: Program Fidelity and Adaptation in Substance Abuse Prevention: A State-of-the-Art Review. Rockville, MD: Center for Substance Abuse Prevention; 2002. [Google Scholar]
- 32. Pawson R, Tilley N.. Realistic Evaluation. London, UK: Sage; 1997. [Google Scholar]
- 33. Mark MM, Julnes G. A realist theory of evaluation practice. In: Henry GTJ, Mark MM, eds. New Directions for Program Evaluation. San Francisco, CA: Jossey Bass; 1998:3–32. [Google Scholar]
- 34. Hewitt G, Sims S, Harris R. The realist approach to evaluation research: An introduction. Int J Ther Rehabil 2012;19:250–259. [Google Scholar]
- 35. Pawson R, Tilley N. Realist evaluation. In: Matthieson S, ed. Encyclopaedia of Evaluation. Newbury Park, CA: Sage; 2004. [Google Scholar]
- 36. Miller WR, Rollnick S. Ten things that motivational interviewing is not. Behav Cogn Psychother. 2009;37(2):129–140. [DOI] [PubMed] [Google Scholar]
- 37. Rollnick S, Miller W. What is motivational interviewing? Behav Cogn Psychother. 1995;23(4):325–334. [DOI] [PubMed] [Google Scholar]
- 38. Martell C, Addis M, Jacobson N.. Depression in Context: Strategies for Guided Action. New York, NY: W W Norton & Co; 2001. [Google Scholar]
- 39. Lewinsohn PM, Graf M. Pleasant activities and depression. J Consult Clin Psychol. 1973;41(2):261–268. [DOI] [PubMed] [Google Scholar]
- 40. Arean P, Hegel M, Vannoy S, Fan MY, Unuzter J. Effectiveness of problem-solving therapy for older, primary care patients with depression: Results from the IMPACT project. Gerontologist. 2008;48(3):311–323. [DOI] [PubMed] [Google Scholar]
- 41. Harpole LH, Williams JW Jr, Olsen MK, et al. . Improving depression outcomes in older adults with comorbid medical illness. Gen Hosp Psychiatry. 2005;27(1):4–12. [DOI] [PubMed] [Google Scholar]
- 42. Hennink MM, Hutter I, Bailey A.. Qualitative Research Methods. London, UK: Sage; 2011. [Google Scholar]
- 43. Pawson R. The Science of Evaluation: A Realist Manifesto. London, UK: Sage; 2013. [Google Scholar]
- 44. Nguyen TD, Attkisson CC, Stegner BL. Assessment of patient satisfaction: development and refinement of a service evaluation questionnaire. Eval Program Plann. 1983;6(3–4):299–313. [DOI] [PubMed] [Google Scholar]
- 45. Attkisson CC, Greenfield TK. Client Satisfaction Questionnaire-8 and Service Satisfaction Scale-30. In: Maruish ME, ed. The Use of Psychological Testing for Treatment Planning and Outcome Assessment. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc; 1994:402–420. [Google Scholar]
- 46. Attkisson CC, Zwick R. The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Eval Program Plann. 1982;5(3):233–237. [DOI] [PubMed] [Google Scholar]
- 47. Kelly PJ, Kyngdon F, Ingram I, Deane FP, Baker AL, Osborne BA. The client satisfaction questionnaire-8: Psychometric properties in a cross-sectional survey of people attending residential substance abuse treatment. Drug Alcohol Rev. 2018;37(1):79–86. [DOI] [PubMed] [Google Scholar]
- 48. Resnicow K, Baranowski T, Ahluwalia JS, Braithwaite RL. Cultural sensitivity in public health: Defined and demystified. Ethn Dis. 1999;9(1):10–21. [PubMed] [Google Scholar]
- 49. Mier N, Ory MG, Medina AA. Anatomy of culturally sensitive interventions promoting nutrition and exercise in hispanics: A critical examination of existing literature. Health Promot Pract. 2010;11(4):541–554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Secretary’s Advisory Committee on Health Promotion and Disease Prevention Objectives for 2020. Healthy People 2020: An opportunity to address the societal determinants of health in the United States.2010: http://www.healthypeople.gov/2010/hp2020/advisory/SocietalDeterminantsHealth.htm. Accessibility verified May 20, 2019.
- 51. World Health Organization. Social Justice and Human Rights as a Framework for Addressing Social Determinants of Health. Final Report of the Task group on Equity, Equality and Human Rights. Stronks K, Toebes B, Hendriks A, Ikram U, Venkatapuram S, eds. Copenhagen, Denmark: World Health Organization; 2016.http://www.euro.who.int/__data/assets/pdf_file/0006/334356/HR-task-report.pdf?ua=1. Accessibility verified June 5, 2019. [Google Scholar]
- 52. LeVois M, Nguyen TD, Attkisson CC. Artifact in client satisfaction assessment: Experience in community mental health settings. Eval Program Plann. 1981;4(2):139–150. [DOI] [PubMed] [Google Scholar]