Mediational Intervention for Sensitizing Caregivers to improve mental health outcomes in orphaned and vulnerable children

Abstract

Objective.

There is an urgent need to equip community-based careworkers with the skills to address the mental health needs of orphans and vulnerable children (OVC) as an essential response to shortages in human resources for mental health in Sub-Saharan Africa. We conducted a quasi-experimental feasibility trial in South Africa to adapt and evaluate an established year-long semi-structured, manualized video-feedback caregiver intervention (the Mediational Intervention for Sensitizing Caregivers; MISC) for community-based organizations (CBOs).

Methods.

Following a year-long iterative cross-cultural adaptation of MISC, we recruited 88 OVC (ages 7–11; 45.5% girls) and their CBO careworkers (N = 18; 94.4% female). Two CBOs (45 children; 9 CBO careworkers) received 12 months of MISC, and two CBOs (43 children; 9 CBO careworkers) received treatment as usual. Child mental health and quality of caregiving were assessed at 6 months into the intervention and at completion through multi-informant questionnaires and video-recordings of careworker-child interactions. Qualitative interviews were conducted to evaluate feasibility and acceptability.

Results.

MISC-CBO was acceptable and feasible in terms of attendance and post-intervention interviews. MISC improved child mental health, as well as the quality of careworker caregiving in terms of interactive effects for affective and cognitive (Expanding) components of MISC, and main effects for the cognitive components of Rewarding and Provision of meaning. MISC components did not mediate the effects of the intervention.

Conclusions.

The current study shows that laypersons with no tertiary education and virtually no prior training who undergo MISC training can improve caregiving quality and the mental health of OVCs.

There are approximately 26 million people living with HIV/AIDS in sub-Saharan Africa (SSA) (United Nations, 2016). The extensive prevalence and associated mortality rates mean that children are at continued risk of HIV infection, living with a parent with chronic illness, orphanhood, and a host of related vulnerability factors, including poverty, lack of basic resources and access to services, and impaired caregiving. These interconnecting factors related to the effects of HIV/AIDS on children has led to the operational term “orphans and vulnerable children” (OVC; Foster, 2006). Globally, there are over 16 million OVC, the vast majority in sub-Saharan Africa (United Nations, 2016). In South Africa (SA), estimations are between 1.9 million (UNAIDS, 2012) to 3.7 million (UNICEF, 2013). OVC suffer many negative consequences, including malnutrition, school dropout, poor psychosocial well-being, early sexual debut (younger than 14 years of age), lack of family support, homelessness, child labor, increased violence or sexual abuse exposure, and heightened stress (Cluver, 2011; Foster, 2006), conferring significant risk for short and long-term mental health problems (Betancourt et al., 2014; Cluver, 2011).

Despite the urgent need to address the negative impact of HIV/AIDS on children’s mental health, there is a global crisis in scarcity of mental health workers in the developing world (Kakuma et al., 2011). One solution is to build caregiver and community capacity by enhancing traditional community-based support systems (Schenk & Michaelis, 2010; Schenk, 2009; Skeen et al., 2017), which are more cost-effective than care provided in clinical settings. Community-based care is also more consistent with African cultural values of shared care for the community (Eze, 2010). Community-based organizations (CBOs) offer strategic points of intervention in this regard (Marais et al., 2018; Marais et al., 2014), and are attended by children who have higher likelihood of living in overcrowded households, orphaned, and with greater exposure to community violence (Yakubovich et al., 2016). CBOs are grassroots level organizations that are developed by members of the community with the help of non-governmental organizations and some government support. They are community-led and community driven, and provide various forms of support to OVC including meals, counselling services, financial assistance, and healthcare counseling (Marais et al., 2018; Richter et al., 2009). OVC make use of CBOs, because legal guardians are not always able to provide meals, emotional and other material support at home. OVC typically visit the CBO after school and when assistance is needed. CBO careworkers work with children individually and in groups.

While CBOs provide critical community support, CBO careworkers rarely receive specialized training in caregiving or child mental health. A community development approach (Christens, 2012) recommends an emphasis on human care (Jordans & Tola, 2013) and task shifting (“task sharing”), defined as delegating tasks to existing but untrained cadres - an essential response to shortages in human resources for mental health (Kakuma et al., 2011). Careworkers themselves have expressed a need for training in effective ways to improve the mental health of OVC (Marais et al., 2014). As yet, few evidence-based mental health interventions exists for CBOs (Skeen et al., 2017) with not a single intervention designed to improve the quality of caregiving provided by CBO careworkers. Training CBO careworkers in the principles of quality caregiving has significant scale-up potential as this knowledge can be transferred and generalized to legal guardians of OVC in the child’s home.

Against this background, and based on the well-established premise that higher quality caregiving capacity improves mental health outcomes in children also in the context of OVC (e.g. Cluver, 2011; Richter et al., 2009), the current study aimed to adapt and evaluate an established OVC parenting intervention for the CBO context in Sesotho speaking Black South Africans. The Mediational Intervention for Sensitizing Caregivers (MISC; Klein, 1996) is a year-long semi-structured, manualized video-feedback caregiver intervention that targets the “serve and return” in the here-and-now (Shonkoff et al., 2012) between caregiver and child. It is theoretically grounded in both Bowlby’s attachment theory (Bowlby, 1973) and Feuerstein’s theory of cognitive modifiability (Feuerstein, 1979). Empirical data (Klein, 1984, 1991; Klein & Alony, 1993; Klein, Wieder, & Greenspan, 1987) suggest that specific characteristics of interactions with children constitute mediational behaviors that affect children’s predisposition to learn from new experiences. These include a set of affective components, including eye contact, smiles, vocalization, touch, physical closeness, turn-taking, sharing of joy, expression of positive affect, synchrony, length of communication chains, and excitement expressed toward things, people and experiences in the environment. It also includes a set of cognitive (mediational) components of learning, including focusing, providing meaning, expanding, regulating, and rewarding. Together, these components form the foundation of MISC and are considered the basic determinants of quality mediation to children. For instance, using an everyday interaction like making a sandwich, a caregiver may stand close to a child and smile while she talks (affective components). She may then say “Look here” (focusing). “This is the bread we will cut for the sandwich” (providing meaning). “Have you ever cut bread before?” (expanding). “First, we hold the bread like this. Then we carefully cut with the knife like this” (Regulating). “Good work! we will try another slice!” (Rewarding). Studies have shown that these components better predicted child cognitive outcomes four years later than children’s own cognitive test scores (Klein et al., 1987). Similar findings were reported for low socioeconomic status (SES) American (Klein et al., 1987) and Israeli (Klein, 1984) mother–infant samples. A study on psychosocial outcomes (Shuper Engelhard, Klein, & Yablon, 2013) found that providing children with an explanation regarding behavior in social situations (“regulation of behavior”) was significantly related to prosocial behavior at home and in daycare. This study also found that when mothers frequently commanded their children to do something without explaining why, higher frequencies of aggressive behavior were noted. In the MISC intervention, through video feedback, the MISC trainer sensitizes the caregiver to these affective and cognitive components in her everyday interactions with the child, constituting the mechanisms of change promoting cognitive and/or socio-emotional learning.

Four randomized controlled trials support the effectiveness of MISC. The first was conducted in 68 mother–infant dyads of a low-SES, urban, high-crime-rate community in Israel. Results at post-intervention and up to six years later showed increased and sustained maternal mediation, in addition to positive child outcomes. Similar results were demonstrated in an RCT with 120 rural Ugandan child–caregiver dyads with HIV/AIDS (Boivin et al., 2013a) and an RCT of 119 uninfected HIV-exposed children and their caregivers (Boivin et al., 2013b). These results were replicated in larger RCTs of HIV-exposed, but uninfected (Boivin et al., 2017) and HIV/AIDS-affected (Bass et al., 2017) children in Uganda where children showed significant cognitive gains, especially in language development, which has clear implications for mental health problems in OVC. For instance, St. Clair et al. (2019) demonstrated in a semi-urban impoverished sample in South African children that poor language ability was related to a range of concurrent adverse difficulties, such as attention deficits, self-esteem problems, social withdrawal, and depressive symptoms. Conversely, increased levels of language ability were related to better psychosocial profiles, underscoring the close link between cognitive and socio-emotional developmental outcomes in children.

The value of MISC is its cultural and developmental transportability, and relevance for both cognitive and socio-emotional outcomes. MISC is a strengths-based intervention that does not impose a model of parenting on the caregiver. Instead, it ascribes to an implicit change model that facilitates reflective capacity so that the caregiver enhances her caregiving capacity independently and consistent with her own culture (Sharp et al., 2020). Because MISC does not make use of any materials or worksheets, but utilizes the everyday interactions between caregiver and child (cleaning up, doing homework, discussing daily events, etc.), MISC is highly suitable for low-resource settings (Klein, 1996; Sharp, Shohet, Givon, Marais, & Boivin, 2018). It teaches a set of principles common to high quality caregiving that can be applied to any developmental age, culture or setting of the child, making MISC highly adaptable and scalable cross-culturally and cross-developmentally (Klein, 1996; Sharp et al., 2018).

The current study

Despite being developmentally transportable, we chose to conduct MISC for children in the 7–11 age range based on the importance of building mental health resilience against the adolescent onset of psychiatric problems, often coinciding with sexually risky behaviors in OVC (Cluver et al., 2011). MISC has not yet been explicitly evaluated for use in 7–11 year olds. The current study is also the first application of MISC in OVC with mental health as primary outcome and the CBO careworker (instead of home caregiver/parent) as unit of intervention. It is also the first adaptation of MISC for the SA Sesotho cultural context.

Based on the fact that MISC has been successfully adapted in various countries (Klein, Shohet, & Givon, 2017) and the iterative nature of our intervention adaptation approach (Matthews & Hudson, 2001), we expected MISC to be adaptable, feasible and acceptable to SA Sesotho culture and the CBO context. Based on the scientific premise that quality caregiving enhances mental health outcomes in children, we expected that MISC vs. Treatment as Usual (TAU) would improve child mental health at post-intervention (12 months after the start of the intervention), controlling for age, gender, the quality of the child’s home environment, and socio-economic status. Finally, consistent with our model suggesting affective and cognitive components to be the active mechanisms accounting for the positive effects of MISC, we expected that MISC would improve the quality of caregiving in careworkers as indicated by increased use of mediational affective and cognitive components assessed through a standardized observational coding system of video-taped interactions. Lastly, we hypothesized that improvement in mental health outcomes would be mediated by improved quality caregiving assessed at six months into the intervention.

Methods

Participants and setting

Relatively few RCTs are conducted in OVC communities (Skeen et al., 2017), because of the ethical and practical challenges associated with randomization in low resource settings (Schenk, 2009). A quasi-experimental design that includes a control group and process evaluation with strong foci on qualitative and community participatory methods is the next recommended option (Schenk, 2009). Accordingly, we followed intervention adaptation approaches (Matthews & Hudson, 2001) to include an initial formative research phase (qualitative interviews and focus groups) with community stakeholders, a Community Advisory Board (CAB), CBO careworkers and service users to adapt MISC to its unique context. The formative phase was followed by a quasi-experimental study to compare MISC with Treatment as Usual (TAU). In reporting our methods and results, we follow the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) criteria (Des Jarlais, Lyles, Crepaz, & TREND Group, 2004). IRB approval was obtained from the ethics committees of the University of the Free State and the University of Houston. Trial registration: University of Houston ClinicalTrials.gov Identifier: NCT04359043, MISC-CBO in Children Affected by HIV/AIDS.

The study was conducted in the Free State, with an HIV prevalence rate of 15% (compared to 13.2% for SA). The Free State is the third largest province and the third most urbanized province in SA, with the Mangaung Metropolitan Municipality the most densely populated. Of the 752,906 people living in Mangaung, 618,408 (82%) are Black (mostly Sesotho), 32,071 (4%) are Coloured, 1,257 (0.2%) are Indian/Asian, and 101,170 (13%) are White (the SA term, “Coloured” is used by the SA census and widely accepted to categorize individuals of mixed race). Values relevant to the current study that are consistent with the Sesotho culture include a focus on authoritarian caregiving. The society is mostly hierarchical and children are expected to obey adults. Corporal punishment is common (although illegal). CBO careworkers commonly referred to the ideal child as “one who listens”. Additional cultural values relevant to the current study are beliefs that ancestors play a role in mental health problems, physical expression of psychological problems, and gender based norms in terms of managing emotional problems (Sharp, Skinner, Serekoane, & Ross, 2011). Currently, 31% of children in the Mangaung Metropolitan Municipality are orphaned. The population from which the sample was drawn is poor: 38.2% did not have adequate clothing, more than 60% of household income emanated from grants. The mean average household monthly expenditure was 1,518 rand/R (approx. $115 based on R13.2 to $1 exchange rate at the time of survey), with 11.9% of total expenditure spent on clothing and 75.6% on food.

We recruited 88 OVC and their CBO careworkers (N = 18) across four CBOs. Two CBOs (45 children; 9 careworkers) received MISC, and two CBOs (43 children; 9 careworkers) received TAU. Recruitment flow, retention and baseline characteristics of the sample are discussed in the Results section.

Interventions

Intervention description.

The MISC intervention refers to work with the careworker as “training”, which is carried out in three basic modes. These modes are complementary and concurrent, each scaffolding the effects of the others. The first mode is through bi-weekly individual video feedback over 12 months (24 sessions) that took place at the CBO. During the session, the MISC trainer and careworker watched a videotaped interaction between that careworker and the child and analyzed the interaction according to the MISC affective and cognitive components. For example: “I want to show you something nice you did with the child… You see, here you looked at the child and smiled. What do you think made the child smile?” Or, “I would like to share with you some parts of the video that I have observed and that I have questions about. We will observe them together and will share our thoughts regarding certain behaviors.” After the identified part of the video has been shown: “Look here at his behavior. What do you think he was trying to signal? Did you pay attention to it while you were interacting with him?” Or, “Here you said to the child… Do you think that you could expand on it? What else could you say?” Through this iterative process of shared reflection on the interactions between careworker and MISC trainer, the careworker’s own reflective capacity begins to increase. The careworker starts to “stop to think and reflect” on her behavior, thereby becoming sensitized to whether the child is learning from the interaction.

The second mode of training was through “in-service training” that took place during a different visit with the careworker and child, also bi-weekly. It does not involve video feedback, but real-time feedback during an in-vivo interaction. The MISC trainer would identify teachable moments, helping the careworker implement concepts and ideas that were discussed during prior video guidance sessions to enhance the quality of the interaction in the “here and now”.

The final mode of training consisted of bi-monthly group meetings, facilitated by MISC trainers during which careworkers had the opportunity for sharing and elaborating on issues related to implementation MISC, consolidating their learning by expanding their individual experiences with their peers. Discussion revolved around challenges and misunderstandings that careworkers encountered as they applied MISC concepts; for instance, if a child did not respond well to a MISC intervention.

Intervention adaptation.

Formative work was guided by a standard intervention adaptation approach (Matthews & Hudson, 2001) and Schenk and Michaelis’s (2010) recommendations for community-based service development for OVC to include the importance of locally tailored interventions, community and child participation, working directly with women, multifaceted targeting strategies, actively addressing stigma, and careful oversight and monitoring of program implementation. To report the adaptation process, we make use of the Framework for Reporting Adaptations and Modifications-Expanded (FRAME; Wiltsey Stirman, Baumann, & Miller, 2019), which includes eight components. Space does not allow a full discussion of these steps and we refer readers to the Supplemental online material. Here, we highlight the adaptations that were considered with regard to culture, CBO context and the mental health focus. The authoritarian nature of Sesotho culture, specifically the lack of eye contact with children, and the incompatibility of cognitive expansion of topics with Sesotho culture were discussed. It was decided not to modify these components in order to preserve the core elements of the intervention that are needed for the intervention to be effective, but to evaluate their relevance through implementation. Regarding context (CBO) adaptations, the study team had to build in stronger managerial support for careworkers. Individual video-feedback sessions were to be alternated with in-service training on a bi-weekly basis to accommodate CBO schedules. Mental health adaptations required the incorporation of socio-emotional examples during MISC training sessions with careworkers. These adaptations resulted in approval from the CBO to proceed with the adapted version, thereafter named MISC-CBO.

MISC training and supervision.

Two Sesotho speaking MISC trainers underwent a 3-day workshop by the MISC co-developers, followed by a period of observation and supervision using videos and roleplay until the MISC trainers were deemed adherent. The MISC trainers were supervised bi-weekly, and as needed, via Skype by two MISC developers (CS and DG) and a clinical psychologist (CS). MISC trainers maintained log diaries on intervention progress and session attendance.

Treatment as usual.

The two CBOs assigned to TAU received standard CBO services consisted of assisting OVC with the identification process of being an OVC, helping them obtain birth certificates, helping them obtain and administer medication if they were sick, a recreational program and assisting children with their homework. Child attendance in the MISC and TAU CBOs were equivalent in terms of time spent at the CBO.

Measures

Mental health outcomes.

The Strengths and Difficulties Questionnaire (SDQ; Goodman, 2001) is a multi-informant 25-item measure of emotional and behavioral disorders for children aged 3–17 widely used across the world. Both informant and youth versions of the SDQ utilize a three-point Likert scale ranging from ‘not true (0)’, ‘somewhat true’ (1), to ‘certainly true’ (2). The SDQ includes five subscales each comprising of five DSM-derived items: Conduct Problems, Inattention-Hyperactivity, Emotional Symptoms, Peer Problems, and Prosocial Behavior, and the use of a total difficulties score is common as a general index of mental health problems (Goodman, 2001). The SDQ has been validated for use Sesotho-speaking populations in SA (Sharp et al., 2014). Keeping with best practice of including multiple informants in assessing mental health in children (De Los Reyes, Thomas, Goodman, & Kundey, 2013), and consistent with a formative approach to handling multiple indicators to index a construct (Diamantopoulos & Siguaw, 2006), the SDQ total difficulties scores across informants (child, legal guardian, and careworker) were summed to create a composite SDQ outcomes score (the relatively low sample size precluded computation of latent variables across informants). Further details concerning a computation of composite scores are provided in the data analytic section. The Cronbach’s alphas for the SDQ composite difficulties scores were .81 (baseline), .79 (6 months follow-up) and .70 (12 months follow up).

Caregiving quality.

The Observing Mediational Interaction (OMI; Klein, 2014) is an observational coding scheme specifically developed to evaluate the extent to which a caregiver’s interactions show evidence of MISC affective and cognitive components. The OMI was used in three prior MISC RCTs (Boivin et al., 2013a, 2013b; Boivin et al., 2017). Following their approach, a standard 5-minute interaction task between careworker and child was video-recorded and the same task was repeated at 6- and 12 months post-baseline. Specifically, the careworker interacted with the child in a colouring exercise. The affective components were each rated on a 5-point scale (1 – none; 2 – rarely; 3 – sometimes; 4 - most of the time; 5 - all of the time) and summed to derive a total score of competence in affective components across the full 5-minute interaction task. The cognitive (mediational) components were tallied in terms of the frequency that they occurred during the 5-minute task interaction. These are (Klein, 1996): (1) Focusing: Any adult act or sequence of acts that is directed toward achieving a change in the child’s perception, or response (e.g., “Let’s focus here for a moment”). (2) Providing meaning: Through provision of meaning, the adult names, describes, and gives meaning to the child’s experience. (“You are colouring the apple using the red colour”; “Wow! You are colouring fast!”). (3) Expanding: An adult’s behavior directed toward broadening the child’s cognitive awareness by explaining, clarifying, comparing, or adding new experiences that go beyond the immediate interaction. This is the metacognitive component of MISC, referred to as “stretching” the child, and is crucial to learning, e.g. “Can you remember the last time you ate a juicy apple? Can you tell me about that?” (4) Rewarding: Any verbal or nonverbal behavior of an adult that expresses satisfaction with a child’s behavior without explanation (“Good”) or with explanation (“Very good; you really explained this very well to me”). Through the process of rewarding, children learn to reflect on how to achieve success and, in time, will generalize this autonomously to new problems. (5) Regulating: The caregiver brings to the child’s awareness the possibility of “thinking” before doing, of planning steps of behavior toward attaining a goal. By modelling, demonstrating, or scheduling objects or events in time and space, the adult introduces a pattern (plan) of activities for the child, thereby regulating the pace and reducing the child’s impulsiveness in perception, elaboration, and expression (e.g. “First we colour here; then it easier to colour the difficult part”).

Two Sesotho coders were trained by the OMI co-developers and brought to coding reliability through standard reliability procedures (Haidet, Tate, Divirgilio-Thomas, Kolanowski, & Happ, 2009). These procedures included (1) the use of a predetermined coding or rating scheme (in this case the OMI), which allows for manualized procedures and decision rules to assign codes to emotional and cognitive components, (2) 3-day training in OMI coding, (3) practice until deemed competent, (4) competency test and approval as trained coder by the OMI developers (CS and DG), and (5) quarterly supervision and consensus meetings with the OMI co-developers (CS and DG) to maintain fidelity in coding, including calibration reliability checks evaluating whether the coder was still reliable based on supervisor assessment.

Child’s home environment.

The Caldwell Home Observation for the Measurement of the Environment (HOME; Bradley et al., 1992) assesses the quality and quantity of stimulation and support available in a child’s home environment through observation and interview with the legal guardian in the child’s home. It evaluates evidence for acceptance of the child, learning materials, parental involvement, parental responsivity, variety in experience, and organization of the environment. The HOME has been validated in SA (Richter & Grieve, 1991), and was completed by trained fieldworkers. The fieldworkers were trained by one of the senior investigators (MB) who is qualified to administer the HOME as was carried out in prior RCTs (Boivin et al., 2013b; 2017). The Cronbach’s alpha in the current study was .87.

Socio-economic status.

Following recommendations to use a variety of indicators that include household assets (Booysen et al., 2008), we gathered information on the dwelling type, access to water and sanitation, use of electricity, and eight forms of assets such as household appliances (television, radio, stove, phone, car) and productive manufacturing or farming assets to derive a composite asset score (counting how many of the listed assets are present) indicating socio-economic status. A questionnaire was completed by trained fieldworkers during a home visit.

Acceptability and feasibility.

Acceptability and feasibility were assessed through frequency of session attendance by careworkers and children, and Penchansky and Thomas (1981) model of “access” to care that covered Acceptability, Accessibility, Affordability, Availability and Accommodation of MISC in the CBO context. This framework assesses the fit between intervention characteristics and its context and therefore offers a helpful model to evaluate feasibility and acceptability. CBO careworkers were interviewed post-intervention face-to-face by a female Sesotho research team member at the CBO. The interviewer did not participate in the intervention, and had prior experience in conducting qualitative research. Interviews lasted 30 minutes. Because the interview schedule was based on an existing framework, a directed content analysis (Hsieh & Shannon, 2005) was used to identify themes of acceptability and feasibility consistent with the Penchansky and Thomas (1981) framework. Full details of the qualitative analytic approach are provided in the Supplementary online materials following the Consolidated Criteria for Reporting Qualitative Studies (COREQ; Booth et al., 2014) criteria.

Intervention fidelity and adherence.

Following prior MISC research (Bass et al., 2017; Boivin et al., 2013a, 2013b; Boivin et al., 2017), the OMI (videotaped interactions between CBO careworkers and children) were used to document fidelity of MISC training while serving as one of the outcome measures. As such, evidence of treatment effects on OMI outcomes was interpreted as an index of fidelity.

Procedures

Recruitment and consent.

CBOs were recruited by a Sesotho project coordinator, leveraging long-standing community partnerships between the University of the Free State, Centre for Development Support and NGOs in the Mangaung Township. CBOs were assigned to MISC depending on interest to participate. CBOs facilitated contact with children’s legal guardians to provide informed consent. Consent and child assent were obtained in person. All consent and assent procedures were witnessed and documented by a study staff member. Children in both treatment arms were offered referrals for additional mental health services as needed, although this offer was not taken up and children did not receive mental health services during the study period. CBOs in both arms received a monthly amount of R2800 for 18 months and covering the 12 month intervention period for participation. Grocery vouchers were given to legal guardians and children for completing the baseline, 6- and 12 month follow ups.

Outcomes assessment.

Trained research staff conducted baseline, 6 months and 12 months (post-intervention) follow-up assessments via in-person interviews using tablets with CSPro (Census and Survey Processing System), a public domain data processing software package developed by the U.S. Census Bureau and ICF International.

Data analytic strategy to evaluate outcomes

We formulated separate composites for baseline and post-baseline timepoints (6 and 12 months follow-ups). Each composite score was computed by summing SDQ total difficulty scores across informants (child, legal guardian, and careworker). Because preliminary analyses suggested that obtained results did not differ when using composites separate informants’ scores as outcomes, we focused in subsequent sections on reporting findings pertaining to composites.

The effects of MISC on SDQ outcomes.

To evaluate the effects of MISC vs TAU on SDQ outcomes, we used multilevel models with participants (level 1) nested within careworkers (level 2) to account for clustering of data. In other words, we had a 2-level clustering structure, with participants nested within caseworkers. We did not nest careworkers within CBOs as the total number of CBOs was too small (n = 4). The data analytic approach involved computation of two models: (1) unconditional model without predictors to estimate variance components and calculate intraclass correlation coefficients, and (2) conditional model to estimate the MISC effects, controlling for the SDQ at the baseline, post-baseline timepoints (6 and 12 months follow-ups), and other covariates (age at the baseline, gender, home environment, orphan status, and socio-economic status). In the conditional model, an interaction of MISC and time was tested to examine whether the MISC effects were similar or different for SDQ outcomes at follow-ups. A first-order autoregressive structure was used to account for correlations between post-baseline timepoints (6 and 12 months follow-ups). The models were estimated using the maximum likelihood under the PROC MIXED procedure in SAS software, Version 9.4 of the SAS System for Windows. The maximum likelihood estimation addressed attrition in the study.

The effects of the MISC on OMI outcomes.

Similar steps to the ones outlined for the SDQ outcome were undertaken to evaluate the effects of MISC on OMI outcomes. Separate sets of analyses were computed for affective and cognitive components. These models included a 2-level clustering structure, with participants nested within caseworkers. Importantly, the interaction of MISC and time was examined to determine whether the MISC effects were similar or different for OMI outcomes at 6 and 12 months follow-ups. The false discovery rate of Benjamini and Hochberg (1995) was used as a correction method to account for multiple testing. CBO effects were not examined in conditional models for SDQ and OMI outcomes because (by design) two out of four CBOs were assigned to MISC, while the other two were assigned to TAU. This created a redundancy between MISC and CBO variables.

Mediation analyses.

Figure 1 provides a schematic representation of the panel model that was used for testing multiple mediation effects. We used the panel model to fully take advantage of the longitudinal nature of our data and appropriately capture the process across timepoints. Separate sets of analyses were computed for affective and cognitive components and examined the mediating effects of OMI scores at 6 and 12 month follow-ups on SDQ at 6 and 12 month follow-ups, controlling for OMI and SDQ scores at baseline. We were particularly interested in testing two indirect effects, namely (1) intervention → OMI 6m → SDQ 6m, and (2) intervention → OMI 12m → OMI 12m. To clarify, each model simultaneously tested these indirect effects. As such, each model included all three timepoints (baseline and two follow-ups); stated differently, we did not test indirect effects concerning 6 and 12 months follow-ups in separate models. Child age at the baseline, gender, home environment, orphan status, and socio-economic status were added as covariates in the models. The Preacher and Hayes (2004) bootstrapping approach was used to test indirect effects, with 1,000 bootstraps computed. Additionally, steps developed by Baron and Kenny (1986) were followed to ensure that mediation conditions are met. Step 1 was evaluated using the PROC GLM procedure in SAS software, Version 9.4 of the SAS System for Windows, whereas Steps 2 – 4 were tested with the path analysis using Mplus, Version 7.2. A multilevel framework was not applied in mediation models as intraclass correlation coefficients were small suggesting a negligible variability due to careworker. Careworkers were not nested within CBOs as the total number of CBOs was too small (n = 4).

Figure 1.

A schematic representation of the panel model that was used for testing mediation effects.

Notes. OMI baseline, 6- and 12 months denote OMI scales in a general sense. Separate models were computed for each affective and cognitive component of OMI.

Results

Baseline characteristics

Figure 2 provides a flow diagram of progress through the phases of the quasi- experimental trial. Descriptive statistics are displayed in Table 1. The sample included 35.1% non-orphans who were “HIV/AIDS affected” (i.e. due to poverty of living with a parent with HIV/AIDS; Foster, 2006), 48.1% single orphans who lost a parent and 16.9% double orphans (who lost both parents).

Figure 2.

Flow diagram of the progress through the phases of the quasi-experimental trial

Notes. Children and careworkers are included in analyses if they have any one of three timepoints, ending up in total n = 45, and n = 43 for final analyses. Missing data were accounting for in data analytic strategy.

Table 1.

Sociodemographic variables and outcomes by condition

CBO careworkers	Total n = 18	MISC n = 9	TAU n = 9	Chi 2 or t, p
Female, No (%)	17 (94.4%)	8 (88.9%)	9 (100%)	0.0, 1.0
# months working at CBO	88.79 (74.22)	113.29 (77.83)	64.29 (71.75)	−1.113, .283
	0 (0)	0 (0)	0 (0)
# years prior training in child caregiving	29.4%	22.2%	37.5%
Prior experience in working with children (%)	100%	100%	100%	.025, .875
Educational level: completed highschool (%)	6.89 (1.15)	8 (0)	5.78 (2.15)	n.a.
Children	Total n = 88	MISC n = 45	TAU n = 43
Female, No (%)	40 (45.5%)	17 (37.8%)	23 (53.5%)	2.19, .139
Age at intake, Mean (SD)	9.38 (1.49)	9.06 (1.70)	9.70 (1.20)	1.94, .056
SES, Mean (SD)	2.90 (1.13)	2.76 (1.36)	3.03 (0.84)	1.02, .312
Home: Emotional Climate		3.68 (2.07)	2.70 (1.45)	−2.37, .020 *
Orphan status				8.18, .043 *
Both parents alive	27 (35.1%)	20 (52.6%)	7 (17.9%)
One parent deceased	37 (48.1%)	14 (36.8%)	23 (58.9%)
Both parents deceased	13 (16.9%)	4 (10.5%)	9 (23.1%)
Living with				4.018, .134
Extended family, No (%)	33 (42.9%)	12 (31.6%)	21 (53.8%)
Non-family, No (%)	3 (7.7%)	7 (18.4%)	4 (10.3%)
Home caregiver (who completed the SDQ)				9.04, .060
Father, No (%)	4 (4.5%)	0 (0%)	4 (9.3%)
Mother, No (%)	35 (39.8%)	21 (46.7%)	14 (32.6%)
Grandparent, No (%)	29 (33%)	11 (24.4%)	18 (41.9%)
Aunt/Uncle, No (%)	5 (5.7%)	4 (8.9%)	1 (2.3%)
Other, No (%)	5 (5.7%)	2 (4.4%)	3 (7%)
SDQ composite score across multiple informants	M (SD), % above cutoff	M (SD), % above cutoff	M (SD), % above cutoff	t, p
Baseline	37.08 (13.69) 25%	40.15 (14.48) 30%	34.00 (12.28) 20%	− 2.05, .044 *
6 months	33.33 (13.60) 20.5%	36.32 (15.84) 26%	30.25 (10.15) 15%	−1.95, .056
12 months	36.12 (10.33) 12.5%	35.79 (11.04) 13%	36.44 (9.72) 12%	0.26, .798
OMI affective components
Mean (SD) baseline	35.96 (5.47)	36.11 (6.29)	35.83 (4.67)	−0.23, .822
Mean (SD) 6 months	32.60 (4.04)	34.53 (3.11)	30.04 (3.74)	−5.20, <.001 *
Mean (SD) 12 months	35.71(5.74)	40.28 (3.59)	31.27 (3.48)	−10.27, <.001 *
OMI cognitive components
Focusing
Mean (SD) baseline	2.22 (2.50)	2.67 (3.16)	1.78 (1.54)	−1.60, .113
Mean (SD) 6 months	0.94 (1.09)	1.25 (1.27)	0.62 (0.74)	−2.52, .014 *
Mean (SD) 12 months	1.23 (1.85)	1.41 (1.90)	1.06 (1.81)	−0.76, .449
Provision of meaning
Mean (SD) baseline	47.99 (21.77)	42.00 (19.16)	53.83 (22.79)	2.49, .015 *
Mean (SD) 6 months	50.00 (18.27)	53.61 (17.72)	46.18 (18.32)	−1.73, .089
Mean (SD) 12 months	47.94 (17.74)	57.63 (11.29)	38.82 (17.99)	−5.05, <.001 *
Expanding
Mean (SD) baseline	4.32 (4.78)	4.72 (5.58)	3.93 (3.88)	−0.74, .464
Mean (SD) 6 months	2.87 (3.90)	4.83 (4.55)	0.79 (1.10)	−5.04, <.001 *
Mean (SD) 12 months	7.59 (8.14)	13.06 (7.95)	2.44 (3.85)	−6.97, <.001 *
Rewarding
Mean (SD) baseline	1.84 (2.30)	2.41 (2.69)	1.28 (1.69)	−2.25, .027 *
Mean (SD) 6 months	3.01 (4.56)	5.33 (5.40)	0.56 (0.70)	−5.11, <.001 *
Mean (SD) 12 months	1.73 (2.22)	3.09 (2.47)	0.44 (0.66)	−6.05, <.001 *
Regulating
Mean (SD) baseline	9.16 (6.55)	11.41 (6.79)	6.98 (5.57)	−3.18, .002 *
Mean (SD) 6 months	8.16 (5.07)	9.53 (4.91)	6.71 (4.88)	−2.41, .019 *
Mean (SD) 12 months	7.88 (6.32)	10.00 (7.34)	5.88 (4.44)	−2.78, .007 *

Note.

^*p-value <.05.

Attendance and attrition

Attendance and attrition were used as quantitative indices of feasibility and acceptability. As shown in Figure 2, one CBO (20 children and five careworkers) withdrew shortly after baseline as they felt they could not meet the requirements of participation. A replacement CBO was recruited and baseline data collected. All CBOs were visited for the intended 24 sessions over a 12–14 month period. Sessions were attended by all careworkers and children who were enrolled in the study. As shown in Figure 2, 80 children were recruited for the baseline, 20 were lost to follow-up (16 before the six month follow-up and four before the 12 month follow-up), and eight were recruited as replacement. The data for all 88 children were included in the analysis with missing datapoints accounted for. Fifteen careworkers were part of the baseline (after CBO replacement), with an additional three careworkers recruited over the course of the study (two before the 6 month follow-up and one before the 12 month follow-up), and nine careworkers were lost to follow-up. The data of all 19 careworkers were included in the analysis with missing data accounted for in analyses. Child and careworker attrition was due to children and careworkers moving out of the area, or careworkers finding paid employment, which is typical of low-resource transient communities.

Results of post-intervention qualitative feasibility and acceptability interviews

Directed content analysis of post-intervention interviews revealed several themes in support of the feasibility and acceptability of MISC.

Cultural fit with MISC.

The first theme to index acceptability was that cultural fit was obtained despite initial discomfort with expected cultural-incompatible MISC components, most notably hierarchical adult-child relationships in Sesotho culture, which discourages eye contact. One careworker stated “In the beginning, the eye contact part of the programme gave me problems because I had never done it before. Even when you look them in the eyes, indeed there’s no longer that thing. We’re not ashamed anymore to look them in the eyes”, and another offered this insight: “Eye contact is not part of our culture, but I now believe that eye contact is essential to see whether the child is listening to you.” Sesotho culture also endorses an authoritarian child-rearing approach, which may have resulted in low acceptability of MISC. However, careworkers expressed appreciation for breaking from a more authoritarian tradition (e.g. “Mhm. All I can say is yes, the MISC has worked for me, right? I was harsh on children and I didn’t know that like.... okay sometimes we have to... to know a child by looking at them and see how different they are, whether they are happy, has problems, and so on.”).

Overall satisfaction with MISC.

Careworkers expressed overall satisfaction with MISC in terms of their professional development (“I feel great after this training because I have learnt a lot”) and the satisfaction they derive from using MISC with the children in their care (“MISC has brought joy for both the children and myself”). This satisfaction appeared to extend to the broader community further indexing acceptability of MISC. Careworkers reported children being cleaner which they read as a sign of greater care being taken at home (“The children would show up being dirty, but after MISC they come to the center clean. You see that the MISC has an impact at home.”), and that the community’s view on the role of the CBO in the community has changed (“It has changed to a point now they... they know that we’re not only an organisation who’s focusing on maybe assisting only children on homeworks, educational activities but we go beyond.”).

Understanding and implementaion of MISC components.

Careworkers expressed a full understanding of MISC components, and articulated the impact of its implementation on children. Here, a careworker expresses understanding of the cognitive component of rewarding and the affective component of touch and closeness (“I feel that I truly now know how to reward and touch. I make sure that the children do not get bored during our interactions. I give them affect. I also use eye contact to see if they understand”). Here, a careworker expresses the cognitive component of affecting/giving meaning (“With the MISC training, I know I have to be sensitive in how I do things with the children. I have to do things intentionally. If I want to make a child focus, I need to do this intentionally”). In implementing the cognitive and affective components, caregivers expressed an improved awareness of children’s perspectives (“It taught me being in the children’s shoes or a person who can act on behalf of a child”) and the impact it has on children (“We are now more aware of the children’s emotions. The children feel more welcome at the centre”).

MISC feasibility.

MISC was deemed feasible with regard to several characteristics. First, because it did not require additional tasks or activities and utilized the everyday interactions between caregivers and children, MISC was deemed feasible and easily accommodated in careworkers’ daily work routines (“It was easy because MISC was part of our daily work”). Similarly, careworkers supported the affordability of MISC because MISC does not require additional materials (“So we didn’t have extra costs as an organization to say we’re gonna spend over, on this. We did not need anything extra for the MISC”). While initial discomfort with the video-feedback format may have impeded feasibility, careworkers reported that they got used to video-feedback and integrated it into their way of work (“And even these videos, you get scared you understand... but as time went on, I ended up seeing myself being okay feeling free to act naturally”).

The function of the MISC trainer.

Careworkers articulated various functions performed by the MISC trainers that enhanced the availability of MISC. Trainers were important in trouble shooting during the implementation of MISC (“The trainer let you talk when there was a problem, or if you did not quite understand”). The use of MISC components by the MISC trainer in her work with careworkers was also noticed. Here, a careworker comments on the use of the cognitive component of focusing (“I would ask, may I please speak about something that challenges me in this and that. She was able to listen attentively.”). The importance of emotional components were also commented on (“When we talked I would be free. She supported me and cared”.).

The effects of MISC on child mental health outcomes

Table 2 presents fixed effects estimates based on the conditional model for the SDQ composite outcomes across multi-informants (child self-, legal guardian-, careworker report). As depicted in Figure 3 – Panel A, the interactive effect of MISC and post-baseline timepoints was statistically significant for the SDQ outcome (F(1, 86) = 13.43, p < .001, ${\eta }^2$ = 0.04), over and above other terms in the model. MISC participants had lower SDQ at 12 months follow-up (M = 34.02, SD = 2.69) relative to TAU participants (M = 39.21, SD = 3.06). However, MISC participants had higher SDQ at 6 months follow-up (M = 37.96, SD = 2.58) relative to TAU participants (M = 32.20, SD = 2.93). There was a statistically significant effect of SDQ at the baseline (F(86) = 7.82, p = .006, ${\eta }^2$ = 0.09), with higher SDQ baseline scores related to higher SDQ scores at follow-ups.

Table 2.

Fixed Effects Estimates based on Conditional Models for SDQ and OMI Outcomes

	SDQ		OMI Components
			Affective		Provision of meaning		Rewarding		Expanding		Focusing		Regulating
	F	p	F	p	F	p	F	p	F	p	F	p	F	p
MISC	0.01	0.941	21.50	< .001	4.02	0.048	6.37	0.014	13.08	< .001	1.14	0.289	0.82	0.369
Time	1.05	0.309	30.82	< .001	2.67	0.106	2.30	0.133	43.60	< .001	0.37	0.544	1.63	0.206
MISC*Time	13.43	<0.001	11.15	0.001	1.31	0.256	2.84	0.096	12.24	< .001	0.42	0.518	1.92	0.170
Baseline outcome	7.82	0.006	< .01	0.961	3.60	0.061	0.08	0.785	1.43	0.235	1.55	0.217	1.73	0.192
Baseline age	1.99	0.162	1.67	0.200	0.05	0.825	1.87	0.176	0.46	0.498	1.63	0.205	2.14	0.147
Gender	0.21	0.649	0.61	0.438	0.98	0.325	2.21	0.141	0.16	0.694	0.67	0.416	6.16	0.015
Home environment	0.35	0.555	0.93	0.338	0.60	0.441	0.01	0.931	2.23	0.139	1.68	0.199	7.78	0.007
Orphan status	0.43	0.654	0.45	0.638	0.67	0.517	< .01	0.996	0.02	0.979	<. 01	0.996	0.26	0.775
Socio-economic status	2.14	0.147	0.53	0.470	0.01	0.877	<. 01	0.984	< .10	0.974	0.10	0.758	0.34	0.562

Note. Bolded= statistically significant results.

Figure 3.

Effect of MISC vs. TAU on child mental health and caregiving quality

Note. Panels A through C represent line plots depicting interactions of MISC and post-baseline timepoints for (A) SDQ outcome, (B) Affective components of OMI outcome, and (C) Expanding component of OMI outcome. Please note that the 6m and 12m means are adjusted (conditional on covariates in the estimated models) while the baseline mean is unadjusted. Error bars for the baseline means are therefore not included as they inaccurately present variability in the data due to their unadjusted nature.

The effects of MISC on the quality of caregiving

Table 2 presents fixed effects estimates based on the conditional models for affective and cognitive OMI outcomes. The interactive effect of MISC and post-baseline timepoints was statistically significant for the MISC affective components (F(1, 73) = 11.15, p = .001, ${\eta }^2$ = 0.08) and the Expanding cognitive component (F(1, 81) = 12.24, p < .001, ${\eta }^2$ = 0.06), controlling for other terms in the model. As depicted in Figure 3 – Panels B and C, the interactive effects were dominated by the main effects of MISC, such that MISC had higher affective and Expanding scores, regardless of time (6 and 12 months follow-ups).

Although the interactive effect of MISC and post-baseline timepoints was not statistically significant for the Rewarding component of OMI, there was a statistically significant main effect of MISC (F(1, 81) = 6.37, p = .014, ${\eta }^2$ = 0.15), with higher Rewarding scores (M = 4.07, SD = 0.84) for MISC participants relative to TAU participants (M = 0.73, SD = 0.99) across post-baseline timepoints. There was also a statistically significant main effect of MISC on Provision of meaning (F(1, 81) = 4.02, p = .048, ${\eta }^2$ = 0.09), with MISC participants had scores (M = 59.91, SD = 5.71) relative to TAU participants (M = 44.93, SD = 4.70) across post-baseline timepoints.

The effect of child gender (F(1, 81) = 6.13, p = .015, ${\eta }^2$ = 0.03) was statistically significant for the OMI Regulating component (boys: M = 10.00, SD = 1.15; girls: M = 7.79, SD = 1.06). There was also a statistically significant effect of the quality and quantity of stimulation and support in the home environment (F(1, 81) = 7.78, p = .007, ${\eta }^2$ = 0.01) on the Regulating component, with higher HOME scores related to higher OMI Regulating scores.

Mediation analyses

In the interest of space only statistically significant findings are discussed. The direct effects of the MISC on the affective components ($\beta$ = 0.56, t(62) = 5.16, p < .001), and Provision of meaning ($\beta$ = 0.23, t(62) = 2.00, p = .046), Expanding ($\beta$ = 0.84, t(62) = 6.92, p < .001), Focusing ($\beta$ = 0.35, t(62) = 2.50, p = .012), Rewarding ($\beta$ = 0.54, t(62) = 5.00, p < .001), and Regulating ($\beta$ = 0.34, t(62) = 3.33, p = .001) at 6 months follow-up were statistically significant. The direct effects of the MISC on the affective components ($\beta$ = 0.86, t(57) = 9.04, p < .001), and Provision of meaning ($\beta$ = 0.52, t(57) = 5.36, p < .001), Expanding ($\beta$ = 0.37, t(57) = 4.50, p < .001), and Rewarding ($\beta$ = 0.51, t(57) = 3.74, p < .001) at 12 months were statistically significant. None of indirect effects were statistically significant, suggesting that mediation hypothesis was not supported for the outcomes. Table 3 summarizes direct and indirect effects for the SDQ and OMI at 6 and 12 months follow-up.

Table 3.

Direct and Indirect Effects for SDQ and OMI and 6 and 12 Months Follow-up

	SDQ 6m			SDQ 12m
	Estimate	Lower (2.5%)	Upper (97.5%)	Estimate	Lower (2.5%)	Upper (97.5%)
OMI affective components	0.02	−0.12	0.16	−0.10	−0.44	0.25
OMI Provision of meaning	0.04	−0.02	0.10	0.01	−0.13	0.15
OMI Rewarding	−0.01	−0.08	0.06	0.00	−0.04	0.04
OMI Expanding	0.14	−0.06	0.34	0.04	−0.05	0.12
OMI Focusing	0.02	−0.05	0.10	−0.06	−0.17	0.06
OMI Regulating	0.09	−0.05	0.22	−0.11	−0.29	0.07

Note. Estimates and confidence intervals based on 1,000 bootstrap samples for indirect effects; the SDQ 6m column presents the indirect effect for OMI 6m (intervention → OMI 6m →SDQ 6m); the SDQ 12m column presents the indirect effect for OMI 12m (intervention → OMI 12m →SDQ 12m).

Discussion

We conducted a quasi-experimental feasibility trial to adapt and evaluate an established OVC parenting intervention for the Sesotho CBO context. Results showed that MISC-CBO was deemed acceptable and feasible in terms of attendance and post-intervention qualitative interviews. Furthermore, quantitative outcomes suggested improvement of child mental health (multi-informant SDQ total difficulties scores) in MISC relative to TAU by post-intervention, as well improved quality of caregiving at post-intervention for MISC affective components and the cognitive components of Expanding. Main effect for the cognitive component of Provision of Meaning and Rewarding were also demonstrated. However, OMI scores at 6 months into the intervention did not appear to mediate SDQ outcomes at post-intervention (12 months follow-up).

This is the first study specifically designed for improving the caregiving quality of CBO careworkers to improve the mental health needs of OVC. As such, MISC adaptation and training explicitly included a focus on children’s mental health (socio-emotional leaning moments) resulting in positive outcomes for children in this study, compared with other MISC trials where the focus was on improving cognitive developmental outcomes (Boivin et al., 2013a, 2013b; Boivin et al., 2017). The latter trials in Uganda with preschoolers and their legal guardians did not show equivalent outcomes for child mental health, underscoring the importance of an explicit focus on mental health content in MISC training if the improvement of mental health is the primary outcome. We wish to highlight that MISC showed improvement in child mental health outcomes despite significant turnover in CBO careworkers (with three careworkers receiving only six months of MISC training). This speaks to the agility of MISC to affect outcomes in real-world low-resource settings where it is sometimes impractical and potentially unethical to impose university-laboratory style intervention designs on communities. Such intervention designs often include the use of master’s level (not layperson) interventionists, resulting in interventionists who do not speak the local language and who are not acceptable to the community; or study designs that due to extensive inclusion and exclusion criteria do not match the heterogeneity of the population, or may deny treatment to children who are not randomized. While cluster randomized trials may overcome some of these difficulties, they are expensive and require high numbers of units (e.g. CBOs) to allow randomization at a higher level. While we recognize the effect of MISC despite high staff turnover, we consider ongoing training and supervision essential to maintain consistent use of MISC-CBO at an organization in the face of high staff turnover. Because MISC is easily learnable/teachable by laypersons (Sharp et al., 2020), also evidenced in the current study, we consider this a feasible and sustainable proposition that future implementation research can further evaluate.

Second, while MISC has been previously used with parents and in school- and daycare settings with teachers, this is the first application of MISC to a CBO context. High attendance combined with positive feedback during post-intervention interviews point to the acceptability and feasibility of MISC in the CBO context and underscore the importance of bringing the intervention to the community. Although only speculative, the possibility that MISC training and implementation may have enhanced CBO volunteer retention compared to TAU further underscores this point. The current study showed that laypersons who are as affected by the impact of HIV/AIDS through exposure to chronic illness and poverty as the children they care for, with limited tertiary education and virtually no training in working with children, could improve the quality of their caregiving as well as the mental health of the children they work with. As such, the current study adds to growing evidence showing the value for community-based care and support services for OVC (Richter et al., 2009; Sikkema et al., 2015; Skeen et al., 2017).

A third innovation in the current study relates to the fact that caregiving outcomes and mechanisms of change were evaluated through observable (video-taped) components directly tied to the intervention (OMI affective and cognitive/learning components) addressing concerns over the generality and vagueness of mechanisms of change in child mental health intervention research (Kazdin, 2007). Contrary to expectations, the cognitive component of Expansion showed significant increase at post-intervention. Expansion is seen as the component of MISC responsible for most increase in cognitive modifiability (flexible thinking) in children (Klein, 1996). The implications of cognitive flexibility for mental health and developmental outcomes in children are well-known (e.g. Dajani & Uddin, 2015). MISC provides a cross-diagnostic approach to targeting cognitive flexibility through stimulating reflective and meta-cognitive processing (Sharp et al., 2020). While interactive and main effects of the intervention on affective and cognitive/mediational components are promising, we expected the OMI components to mediate SDQ outcomes at post-intervention. Failure to do so is hard to interpret in an underpowered study. The post-hoc power analysis suggested that we had on average power equal to 0.10 to detect statistically significant indirect effects. An alternative interpretation to be assessed in a larger study with a longer follow-up period is that a mediational effect may be detectable later than six months into the intervention. It is also possible that additional mediators may interact with OMI components. For instance, careworker age, stress, depression or trauma exposure may affect the impact of OMI components on SDQ outcomes; or child characteristics, such as child headed households or child cognitive ability may show similar interactive effects. Future work, powered to include such moderators would shed further light on the mechanistic impact of MISC components.

The current study followed best practices in including formative (qualitative) and quantitative approaches in cross-cultural intervention adaptation and evaluation of feasibility, acceptability and outcomes. With feasibility, acceptability and promising preliminary outcomes, MISC-CBO is now poised for further evaluation using a larger scale randomized control design, powered to include additional moderators and mediators, as well as a longer follow-up period post-intervention. Other limitations that should be addressed in future work include the need for more careful monitoring of MISC adherence. While intervention effects on OMI outcomes suggest careworker adherence to MISC, we did not systematically code MISC trainer interactions with careworkers to quantify fidelity. Year-long training in MISC prior to the intervention and bi-weekly supervision somewhat mitigates concerns, but future studies would do well to include more careful monitoring of MISC trainer-careworker interactions in addition to careworker-child interactions. Our study furthermore did not include an active control condition and randomization, taking into account the ethical concerns of not providing a service to some children in dire need (Schenck, 2009). We could therefore not adequately account for variability in the baseline measures. Although we used adjusted means (conditional on covariates) for the 6 months and 12 months outcomes in computed models, we had to use unadjusted means for the baseline measures since they were entered as covariates rather than outcomes in the computed models. Consequently, the computation of the reliable change index in the current project would not be meaningful since not all the means were on “the same metric” (unadjusted versus adjusted). In addition, our study was necessarily focused on Sesotho speaking Black South Africans, and findings are generalizable only to this population. In analyzing our qualitative data, in-language coding was not attempted due to lack of access to Sesotho speaking qualitative researchers. In-language coding is often preferable because it keeps the data rooted in the participant’s own language and should be attempted in any future research in this population. Moreover, the directed approach to content analyses we used also has limitations, most notably the fact that by using an existing framework like the Penchansky and Thomas model, the qualitative data is approached with inherent bias. In answering probed questions, research participants may be cued to answer in certain ways. Given that our purpose was to evaluate feasibility and acceptability, we feel that we were justified in using a directed approach as our goal was not to develop a new construct. Finally, while we ensured fidelity of coding of OMIs through following a rigorous OMI coding training schedule until the two coders were trained to reliability, in addition to ongoing supervision with OMI developers/coders (CS and DG), we did not double code interviews to calculate inter-rater reliability. While our approach followed prior approaches to MISC intervention studies, future MISC studies should include an evaluation of inter-rater reliability.

Despite these limitations, the current study introduces a culturally-appropriate, developmentally transportable, sustainable and scalable evidence-based CBO intervention that can be readily and effectively implemented globally in low-resource settings. While replication in other countries is needed, the current study adds to the existing Ugandan and Israeli evidence that MISC is a promising intervention with children generally at risk from disease, malnutrition and neglect.