A Direct Observational Measure of Family Functioning for a Low-Resource Setting: Adaptation and Feasibility in a Kenyan Sample

Abstract

Family interactions are recognized as highly influential for youth development of psychopathology. Key challenges for assessing family functioning include cross-cultural variability in functioning and self-report measurement challenges. Observational measures—adapted to cultural context—provide an approach to addressing challenges. This study aimed to adapt a direct observational tool for assessing family interaction patterns in Kenya, to outline a replicable adaptation process, and to explore tool feasibility and acceptability. We reviewed existing tools to assess their adaptability based on compatibility with context-specific data. After initial modifications, the measure was iteratively adapted through pilot testing and collaborative discussions between U.S. and Kenyan collaborators that drove changes and further piloting. The measure was administered to 26 families. The Family Problem Solving Code was chosen for adaptation. The tool’s activity structure was feasible to administer, but activity content showed low acceptability, requiring new content. Final activities included (a) a hands-on problem-solving task, (b) a discussion of marital conflict with couples, and (c) a structured discussion of family hopes. Codes were adapted to reflect culturally congruent descriptions of behavior, expressions, and interactions, including an emphasis on nonverbal interactions. The scoring system was modified to facilitate training and consistent rating among trainees with limited experience. Observational tool findings were consistent with those of an interview assessing family functioning, rated by clinical and non-clinical raters. Adaptation resulted in a culturally relevant tool assessing family functioning that proved feasible and acceptable. The adaptation process also proved feasible and efficient in a low-resource setting, suggesting its utility for other contexts.

The family environment can have a powerful impact on a range of child and adolescent outcomes, including mental health, physical health, engagement in sexual risk and substance use, academic achievement, and likelihood of experiencing future relationship violence (Betancourt et al., 2014; Lawson, 2008; Norman et al., 2012; Perrino, González-Soldevilla, Pantin, & Szapocznik, 2000). The family environment can be characterized by a range of factors—for example, concrete characteristics such as number of family members, household income, or physical home and neighborhood conditions. It can also be described in terms of nuanced dynamics that are often more difficult to observe and measure. These dynamics include the quality of relationships among family members, in part determined by communication; how members parent and co-parent, including disciplining and advising; emotional closeness or distance; role and responsibility organization; and family alliances (Conger, Rueter, & Conger, 2000; Olson, Russell, & Sprenkle, 2014). Taken together, interaction patterns determine how a family functions and impacts caregiver and children’s day-to-day interactions.

Each of these aspects of family functioning can disrupt or strengthen family cohesion and long-term child emotional and behavioral outcomes. High-quality communication, monitoring, balanced supportive parenting, consistent discipline, and a warm climate characterize protective environments (Knerr, Gardner, & Cluver, 2013; Masten, 2014). Conversely, poor family functioning is often characterized by poor communication, harsh or neglectful parenting, disconnectedness, and family conflict, which are in turn associated with poor mental health among family members (Wiggins, Mitchell, Hyde, & Monk, 2015; Yap, Pilkington, Ryan, & Jorm, 2014). Further, environmental adversities, such as poverty or violence, often exacerbate risk for poor youth adjustment and exert barriers to creating and maintaining protective family interaction patterns (Masten & Wright, 1998). These stressful circumstances can be common conditions in low and middle-income countries (LMICs; Tol, Song, & Jordans, 2013). In such environments, family patterns may play an even more important role, at times buffering against potential ecological risks to child outcomes, or adding to cumulative risks for youth (Paradis et al., 2009; Repetti et al., 2002). The need to intervene at the family level to promote child well-being is recognized by global mental health leaders (Collins et al., 2011). Yet, a key obstacle to developing and rigorously studying family-level interventions is that family functioning is challenging to measure (Aspland & Gardner, 2003; Patel, Flisher, Nikapota, & Malhotra, 2008).

Family processes are often difficult to assess using uniform methods across contexts, in part because they can be strongly influenced by culture and context. Even within the United States, parenting strategies and relationship dynamics associated with positive youth outcomes have been shown to vary by cultural group (Domènech Rodriguez, Donovick, & Crowley, 2009; Kim, Wang, Orozco-Lapray, Shen, & Murtuza, 2013; White, Liu, Gonzales, Knight, & Tein, 2016). Some evidence suggests that parenting styles characterized by high warmth and elevated demandingness—often termed “no-nonsense” parenting—among Mexican-American parents are predictive of positive child outcomes, which differs from typically cited U.S. constructs (i.e., high warmth, low harshness, high monitoring; Carlo, White, Streit, Knight, & Zeiders, 2017). Another relevant example comes from findings showing the diversity of ways that beliefs and norms about gender influence family member interactions, with some cultures emphasizing traditional gender roles and others deemphasizing strict delineation by gender (Harrington et al., 2016; Lorenzo-Blanco, Unger, Baezconde-Garbanati, Ritt-Olson, & Soto, 2012). Understanding such differences, and the impact on relationship quality, has important implications for informing interventions and the ways we measure their effects.

Further, current measures of family functioning often consist of self-report assessments. These provide valuable information, but they are limited by social desirability bias, hindering their ability to provide an “objective” assessment. Further, they often reflect indicators of functioning and terminology that is most relevant for families in the country of measure development (Lund, Stansfeld, & De Silva, 2014). As such, there is a critical need for other sources of data on family functioning.

Direct observational assessments provide less biased and, arguably, more in-depth assessment, address some of the limitations inherent to survey tools, and can provide a window into culturally nuanced interactions (Aspland & Gardner, 2003). Observational tools typically include an activity or prompt that elicits family interactions, a codebook and scoring system for codifying interactions, and procedures for rating interactions. The family is often videotaped completing the activities to enable later coding. Such assessments of family functioning have primarily been developed and studied in high-income countries (HICs; Sperry, 2012) and have yet to be developed or adapted for LMICs. While there are clear advantages to using direct observation, there are complexities involved in their use that render them difficult to use widely even in HICs. For instance, they require some access to technology, time, and human resources to complete training, administration, and coding. These types of barriers and challenges may become especially complex in LMICs: there is often a lack of training mechanisms, overarching scarcity of financial and human resources, and, from a family and clinician perspective, less familiarity with assessment procedures involving videotaping and structured coding.

Direct observational measures also present a unique challenge for cultural and contextual adaptation. While there is emerging clarity regarding the importance and processes for adapting self-report surveys across cultures and contexts (Jayawickreme, Jayawickreme, Atanasov, Goonasekera, & Foa, 2012; Kaiser, Kohrt, Keys, Khoury, & Brewster, 2013; Kohrt & Hruschka, 2010; Van Ommeren et al.,1999),there are no such guidelines for developing or adapting observational measures for contexts that differ from the sites where tools were developed. To adapt an observational measure, one must address how to identify and account for subtle differences, such as how to best elicit nonverbal and verbal behaviors in assessment and then how to categorize those into meaningful indicators of quality of relationship functioning. Existing measures cannot be simply translated, and the insights related to differences in language that guide adaptations to survey measures are not sufficient for this task. As such, a process by which a direct observational measure can be developed or adapted in ways that capture cultural differences and have the capacity for administration and scoring with few resources is needed. To begin to fill these gaps, this study aimed to adapt a direct observational tool for assessing family functioning in Kenya for clinical and research use, outline a replicable process for adaptation that uses multi-step iterative adaptation, and explore feasibility and acceptability.

Methods

SETTING

Study activities were conducted in peri-urban communities outside of Eldoret, Kenya. Eldoret is the third largest town in Kenya and is located on a main transportation route that runs from a costal port through Nairobi and into Uganda. The town lies within the Rift Valley and is home to multiple ethnic groups including Kalenjin, Luhya, Kikuyu, and Luo populations.

PARTICIPANTS

Participants included families with at least one caregiver over age 18 and at least one child between the ages of 10 and 17. We used a broad definition of caregivers to capture both biological and nonbiological parents, as well as multigenerational caregivers to ensure results were relevant to a range of families. A total of 26 families were recruited from two sources. The first group of families (N = 17) participated as part of a larger measures validity study to develop and establish culturally appropriate assessment tools for measuring family functioning and youth mental health in Kenya (Puffer, Green, Giusto, et al., 2018). These families were referred by community leaders who were instructed to refer families representing a range of functioning levels, including both high-functioning families with adaptive interactions and low functioning families with dysfunctional patterns. This sample included families with diverse family structures. Of the 17, six were single-parent homes and two of the caregivers were adoptive parents (e.g., uncle, older sister). Tribal affiliations also ranged as follows: Luhya (9), Kalenjin (4), Luo (2), Kisii (1), and Kikuyu (1). After adapting and piloting the tool with these families, our coding system stabilized, and no substantial conceptual modifications were made.

In order to assess the tool’s performance in a clinical sample, a second sample of participants (N = 9) was recruited as part of a family-based intervention study for which the observational measure was included as part of a pre-post intervention battery of measures (Puffer, Healy, Green, Giusto, & Ayuku, 2017).These families also were recruited by community leaders referring them to the intervention because of concerns related to poor family functioning. Families were eligible for the intervention study if they were (a) experiencing some type of family distress as reported by a community leader and/or self (e.g., marital conflict, separation, poor problem solving) and (b) had a child between the ages of 10 and 17 exhibiting emotional or behavioral problems, again reported by leader—and/or parent—or self. As such, this group of families typically presented with complex interaction patterns and marked difficulties related to family functioning, which provided data regarding the tool itself, as well its potential clinical use. Of the nine, tribal affiliations were as follows: Kalenjin (5), Luhya (3), Borana (1).

CODERS AND ENUMERATORS

Four enumerators completed tool administration training and participated in rating procedures described in results. Each was from the study area near Eldoret and had worked with the research team assisting with one or both of the related studies for at least 3 months at the time of training. Raters all had post secondary education ranging from diploma level (1–2 year degree) to one who had completed a bachelor’s degree in psychology.

PROCEDURES

Adaptation

The process of adaptation followed six broad steps outlined in Figure 1. These consisted of (1) analysis of context specific data on family functioning, (2) identification of existing tool(s) for potential adaptation through a comparative literature review, (3) initial adaptation, (4) pilot-testing, (5) collaborative team discussion, and (6) iterative adaptation. Pilot-testing, discussion, and adaptation functioned in a reciprocal feedback loop.

FIGURE 1.

Macro-level Adaptation Process

Adaptations of the tool itself occurred at three levels: (a) the activity prompts that elicited the observed behaviors, (b) the codebook definitions and scoring system to account for cultural and contextual differences related to which behaviors were most salient for assessing quality of interactions, and (c) rating procedures, focusing on changes to promote feasibility of reliable and efficient coding within this setting. For each of these three, collaborative discussion and review were guided by key questions shown in Figure 2. The institutional review boards at Duke University and the Kenya Medical Research Institute approved all procedures.

FIGURE 2.

Decisional Adaptation Process of Observational Tool Components

Step 1: Analysis of context-specific qualitative findings

Exploratory qualitative data informed the instrument’s construct validity. Formative qualitative work helps ensure the target construct is measured in locally and ethnographically valid ways (Bass, Bolton, & Murray, 2007; De Jong & Van Ommeren, 2002; Kaiser et al., 2013; Kohrt et al., 2011; Van Ommeren, 2003). In our case, qualitative work centered on exploring both locally defined meanings of family functioning, as well as the degree of overlap with definitions that underlie existing tools, an approach previously used in cross-cultural instrument development for this purpose (Betancourt, Speelman, Onyango, & Bolton, 2009; Miller et al., 2006).

For this study, qualitative data were collected previously in Eldoret to inform the choice and adaptation of the measure. Data from the previous study included 9 interviews with local mental health experts and 14 focus group discussions (FGDs) with caregivers and adolescents (ages 10–17)— including orphans, defined as an adolescent who had lost one or both parents—about family functioning and mental health. These were conducted in Swahili by trained local research assistants and were then transcribed directly to English from recordings. Results elucidated key aspects of family functioning associated with mental health outcomes from community members’ perspectives (Puffer, Healy, Giusto, et al., in preparation). Specific to the development of an observational measure, FGDs included video-recorded role-plays during which caregivers and adolescents were asked to demonstrate positive and negative family interactions to capture observable behavioral indicators of functioning. FGD participants were divided into two groups and provided an introduction. They were asked to role-play one of the common family problems mentioned earlier during the course of the FGD. A group was prompted as follows: “Pretend you are a family who is doing well. Act out what the family would say and do [to address the problem],’ or ‘Pretend you are a family who is NOT doing well. Act out what the family would say and do.” Groups were given 5 minutes to prepare and encouraged to make the role-play as realistic as possible. Prompts for adolescents differed slightly; groups were prompted to pretend to be a family who is “happy” or a family who is “not happy.” For both adults and adolescents, role-plays resulted in verbal and nonverbal markers of functioning, including tone, facial expression, as well as interaction patterns and conversation content that drove initial adaptations.

Step 2: Identification of existing measures to adapt

We reviewed existing observational tools assessing family functioning. We evaluated tools based on multiple factors, including theoretical model congruency with context-specific findings, how they have been used in empirical research, and feasibility. In considering feasibility for use in a low-resource setting, we paid specific attention to administration and coding that was feasible in terms of its ability to be rated accurately and efficiently without extensive background knowledge or specialized training. We also prioritized tools with some procedural structure to promote standardized administration by individuals with limited experience conducting family assessments. Through this process, we identified the existing tool that was the best fit to provide a foundation from which to adapt.

Step 3: Initial Adaptation

We made initial adaptations to the tool activities and corresponding codebook before beginning pilot testing, as some needed changes were clear. We first modified the existing activities based on material and human resource feasibility and sustainability (e.g., inexpensive activity materials) and then trained the local team in activity administration. At this point, we did not exclude any activities prior to piloting. Next, we compared the existing codebook to the context-specific indicators identified in Step 1. This was done to confirm which aspects were congruent with the domains already rated in the existing measure and to identify domains, or aspects of domains, that were missing. In our case, these included domains such as positive behavior and discrimination and favoritism, which are described later. Within congruent categories, such as positive behavior, we adapted the rating definitions (e.g., behavior valence in terms of what extent a behavior was positive, negative, or neutral such as expected parent-child eye contact) based on qualitative findings, including the videotaped role-plays. We also added new rating domains, such as discrimination and favoritism, to the codebook, but at this point did not eliminate any existing rating categories prior to piloting. Suggested changes were discussed, and consensus was reached on all modifications.

Step 4: Iterative Adaptation Process

We then began pilot testing and the iterative process of discussing and adapting components to refine the measure into a culturally and contextually meaningful and acceptable tool. At the beginning of the process, the tool was reassessed and revised after every administration through a series of collaborative discussions including members of the U.S.- and Kenya-based teams. Discussions were guided by the component-specific questions outlined in Figure 2. Assessments were conducted in Swahili and rated in Swahili for local team members and transcribed directly into English for U.S. raters.

To adapt the activities themselves, we were guided by the following questions: Is administration feasible? Are activities acceptable to the family? Does the activity elicit adequate interactions? For instance, we discussed the family’s observed reactions, comfort, length of engagement, and behavioral variability. If elicited behaviors were sufficient, we examined the codebook and rating procedures. For codebook definitions and the scoring system, component two, we asked the following: Are behaviors able to be categorized within existing domains? Are interactions able to be rated? Are the ratings of the behaviors conceptually and culturally meaningful? Regarding the rating procedures, component three, we were guided by the following: Was rating feasible with resources? Are procedures acceptable to raters? Are interactions able to be rated consistently across raters? Rating system procedures evolved until no substantial conceptual modifications were suggested. While these guiding questions focus on separate components, adaptations to one often informed the modifications in another. For instance, any alterations to the activities necessitated reevaluation of the effectiveness of the rating procedures as applied to the interactions elicited by the new activity.

In-Depth Interview

We then compared the observational measure to an in-depth interview of family functioning, used to identify families with high levels of distress requiring counseling. This was a preliminary step for considering criterion validity. These preliminary data came from families who were part of a larger study assessing the validity of multiple measures of family functioning in Kenya. Four families from the first round sample completed both the observational assessment and an in-depth semistructured interview. The semistructured interview assessed relational functioning and “caseness”—whether family therapy was indicated or not (Yes/No). The Global Assessment of Relational Functioning (GARF) Scale was used as the model for the overall interview format (Group for the Advancement of Psychiatry Committee on the Family, 1996). Interview scoring consisted of a multistep process involving multiple U.S. and Kenyan raters—different from the observational measure raters—who independently reviewed interview transcripts, interviewer notes, and recordings. Teams then reached consensus on ratings through collaborative discussion. Final ratings ranged from 0 (critically dysfunctional) to 100 (most functional or adaptive). We compared observational ratings to interview ratings to explore instrument agreement.

Results

CONTEXT-SPECIFIC QUALITATIVE FINDINGS

Interview, FGD, and video-recorded role-play data resulted in context-specific indicators of family functioning. Core themes included the importance of structure and hierarchies; communication, emotional trust and respect; problem solving and planning, particularly regarding managing finances in the face of poverty; gendered family processes; and risks of discrimination and favoritism. Related to behaviors likely to emerge in direct observation, the strong emphasis on hierarchy and boundaries by both age and gender (males holding more power) was important to notice. Norms for parent-child communication often emerged with more communication described as occurring between same gender pairs (e.g., father-son communication more expected than mother-son communication). These patterns were delineated by age, such that interactions between a mother and son would have different cultural expectations in childhood versus adolescence. Further, favoritism and discrimination of certain family members was described as a clear marker of distance and dysfunction. Conversely, emotional closeness and trust emerged as important indictors of positive functioning.

Video-recorded role-plays provided more specific depictions of well- and poor-functioning families. Examples of clearly positive indicators included: smiling at one another, affectionate handshake, looking to an individual when speaking, leaning in, and initiating cooperation. Behaviors that were clearly negative were child disobedience or defying a caregiver request, throwing arms in the air, or using a pressured tone. We also identified ambiguous behaviors that were common but not as obviously positive or negative. For example, we noticed among both children and women instances of hesitation, looking down, looking at the head of household before speaking, or reticence to speak. It was unclear at first whether these were displays of fear (negative) versus respect (positive), but role-play context and description in the FGDs helped to elucidate the differences. For instance, role-plays consistently demonstrated respectful interactions between parents and youth marked by child reticence and minimal eye contact. This was slightly different than role-plays developed to show poorly functioning families in which children were equally quiet but also displayed fear of the parent marked by heightened arousal and almost no eye contact. Role-plays also informed interpretation of verbal interaction content; participants described in FGDs that talking “well” and “being one thing,” as contrasted with members “going their own way,” were indicators of adaptive functioning. The role-plays provided examples of the types of language people use with one another when “talking well,” such as the male and female caregiver planning how to acquire school fees and explaining the plan to the child. Lastly, related to emotional closeness, the behavioral presentation of this only rarely included effusive displays of affection, such as physical contact. Rather, they were more often characterized by subtle behaviors such as smiling, comfort speaking, and gentle encouragement to participate. These results laid the foundation for choosing a measure to adapt, designing activities to elicit relevant behaviors, and developing a coding system that would capture interactions that emerged as most salient and reflective of functioning.

IDENTIFIED MEASURE FOR ADAPTATION

We then identified and reviewed 23 candidate direct observation tools used in clinical and/or research settings and selected the Family Problem Solving Code (FAMPROS; Forbes, Vuchinich, & Kneedler, 2001). FAMPROS is an observational tool assessing global family functioning and family problem-solving characteristics. FAMPROS activities consist of three discussion-based family activities: planning a fun activity (5-minute; not coded) and two discussions of current problems, with one problem chosen by the parent and one chosen by the child, which are recorded and coded by an observer. FAMPROS was selected in part because of its use of macro-level codes, as opposed to micro-codes that are often narrow in scope. The macro-level codes in FAMPROS include: Positive Behavior, Negative Behavior, Participation, Coalition, and Relationship Quality, as well as Problem Solving Process codes that assess the mechanics of problem-solving (e.g., whether families can clearly define a problem) and dynamics (e.g., relationship quality). Overall, these categories were a good match with what emerged as most important in our data, and they were broad enough to be flexibly adapted into culturally congruent code descriptions. The tool’s codes also are based in common and broad theories of family functioning that draw on behavioral and family system approaches and family problem-solving research (Forbes et al., 2001). As such, the tool does not rely on a complex diagnostic coding system but is more focused on concrete, straightforward behavioral coding and is designed to optimize efficiency, time, and accuracy. Together, these characteristics led us to select FAMPROS due to anticipated feasibility and acceptability for training a range of potential raters in administration and coding, potential for adaptation, and alignment with contextual findings that included problem solving as a central component.

ADAPTED OBSERVATIONAL TOOL

Here we outline the newly adapted observational tool that resulted from the process described above, including the activities, the codebook, and the rating procedures.

Activities

Guided by iterative piloting, adaptations resulted in three activities: one new hands-on problem-solving task involving all participating family members, one discussion of a problem between members of a couple that was highly modified from the original problem discussion prompts, and one new whole-family discussion of hopes and plans for the future. We moved away from the original FAMPROS activities, asking families to discuss problems together because they demonstrated low acceptability among families, carried risk of social desirability bias, and provided limited opportunities for observing family interactions that included children. When piloted, families exhibited significant discomfort, such as looking around for what to say, confusion, verbal expressions of misunderstanding, and long periods of silence. This seemed driven by being asked to discuss a family problem in front of new people and, perhaps even more so, with children present. This discomfort seemed to influence the family’s problem selection, with initial families typically choosing to discuss an issue that had already been solved or that was extremely minor. Family discussions of problems also typically resulted in the exclusion of children, as it is customary for caregivers to discuss issues first among themselves; when there are follow-up conversations with children, these are often formal and parent-led, focused on delivering information or decisions to the child. As such, the original prompts were not yielding accurate representations of natural interactions in this context.

After identifying these problems, the task was to generate activities that were more acceptable and retained the original purpose of eliciting individual behaviors and dyadic and whole family interactions that fell under the macro-level domains of functioning and captured the global problem-solving process. The new activities were designed to elicit analogous constructs consistent with the locally derived indicators in a culturally acceptable manner.

Activity 1: “House Building”

First, we developed a hands-on “House Building” activity to assess effective problem solving in a less direct manner that encouraged whole family processes and emphasized nonverbal behaviors. This was done to promote more comfort among family members, including children. In this activity, the family is given 10 minutes to work together to build a house “strong enough to withstand a storm” using four of five locally available materials that they choose. These materials include tape, clay, tinfoil, cardboard, and paper, which were abstract enough for building creatively but familiar enough to be acceptable and easy to use. After 5 minutes, the facilitator destroys the house and says, “The storm has come. You will need to start again.” The family is then given the remaining time to rebuild. The activity prompts the family to interact together to solve a concrete “problem” in the face of a stressor that is less personal than a chosen problem and is standardized across families. This activity elicited interactions from all family members, including children, provided rich nonverbal information alongside valuable verbalizations, and allowed families to become more comfortable with videotaping before being asked to disclose personal information.

Activity 2: Marital Problem Discussion

The second activity aligns with the original FAMPROS prompts that ask participants to discuss an actual problem, but the adapted version consists of a facilitated discussion of a problem between a couple rather than the whole family given the lack of acceptability of involving children in these discussions. In the first attempt at adaptation, couples were asked to “think about a problem, then discuss how they would solve that problem” yet, this led to short exchanges, and couples often chose a problem they had already solved. After stating the problem, there was typically little to no further discussion. Potential reasons for this raised by the Kenyan team were difficulty truly understanding the expectations for this unfamiliar task, as well as discomfort discussing in front of people the family did not know. They wanted to assist the families who were struggling by providing reminders and examples to make them feel more comfortable. Therefore, the next iteration included a prompt to remind the couple of the goal— to think of an issue they currently disagree on that they have not yet solved. After the couple discusses for 1–2 minutes, the facilitator then asks what issue they plan to discuss and if it is a current issue that has not been solved. If the family has not chosen an issue at that point, then the facilitator offers to help them decide by presenting common issues couples face, reminding them of the prompt, then leaving the room to allow discussion. For instances in which the couple finished their conversation in under 2 minutes, the facilitator thanks the couple for the effort thus far, asks them to continue talking, and, if they are unable, again reiterates the activity prompts. The prompts were designed to be minimal to clarify the task and increase family comfort without biasing the discussion or forcing unnatural interactions. In addition to facilitating conversations and yielding more useful data, administrators also preferred this approach because it gave them a means to guide participants (i.e., provide examples of issues if the couple was confused) without deviating from the protocol.

Activity 3: Family Hopes Discussion

The final activity is designed to elicit whole family verbal communication about an important issue—a family-level process most similar to that evoked by the original FAMPROS. Given initial difficulties with prompting families to discuss a problem, families are instead asked to describe their hopes for the future, to choose one hope to discuss, and to plan how to make that hope a reality. Team members and initial families provided feedback that sharing hopes and planning for the future would be acceptable to discuss with the whole family, including children. After initial piloting, an introductory story was also added to promote levity, encourage more involvement of children, and help provide more guidance to the family about what the “planning” process should entail. The final version begins with a brief story about a “giraffe family” making a plan to reach their “hope”— delicious fruit at the top of a tree. After the story of the giraffes piling up sticks and stones to reach the treetop, the enumerator asks the family to identify what the giraffes hoped for and what their plan was to ensure an understanding of “hope” and “plan.” The enumerator then asks the child, “What is a hope that you have for you and your family in this year?” followed by asking each caregiver; they are then instructed to “choose one hope and talk about your plan for the ways you work together to reach that hope.” With this final version, families were better able to understand the goal of identifying a hope, attempt discussion, and, to some extent, involve the child in the process. In this case, the example seemed to provide a helpful model but also was enough removed from reality that families would not repeat the story’s same problem or plan. Instead, as intended, hopes and plans varied in type, extent, and depth across families; as examples, hopes included going to market to buy cloth, going to school, saving money, or building a rental house.

Codebook Scoring System and Definitions Adaptations

The FAMPROS codebook definitions and scoring system were modified to reflect activity adaptations and context-specific findings to ensure that behaviors were interpreted (i.e., coded) correctly and to capture the variability within all of the most salient interaction patterns. Modifications also were made to calibrate scoring and improve rating consistency across coders.

The final coding system included six global code domains. Five domains were rated across all activities, including: Positive Behavior, Negative Behavior, Relationship Quality, Participation, and Discrimination/Favoritism. All of the code labels remained the same except for one. An original FAMPROS domain, Coalition, was replaced with the new “Discrimination and Favoritism” domain. See Table 1 for a description of the codes and modifications from the FAMPROS. The sixth domain is Problem-Solving and Planning Process; this domain was rated based on four codes specific to each activity (see Table 2). These were adapted to be more specific to new activities, and given the change from problem-focused discussion activities to new team-oriented activities, operationalizations of codes became more reflective of increased opportunities for teamwork and unity.

Table 1.

Global Codes of Family Functioning and Adaptations From FAMPROS

Global Code	Level of Adaptation from FAMPROS	Rationale for adapting in Kenyan sample	Specific Adaptation Description
Positive Behavior	Minimal	Caregiver-Child hierarchy and roles important: Obedience/respect emphasized more than warmth	Caregiver-child eye contact deemphasized (unless in fear) Addition of respect and obedience
Negative Behavior	Minimal	Caregiver-Child hierarchy and roles important: Disobedience/ lack of respect emphasized more than lack of warmth	Caregiver-child eye contact deemphasized Include disobedience Include fear of parent
Participation	Moderate	Hierarchies, boundaries, and role important by age and gender: Level of expected participation lower for youth, particularly female youth; More command verbalizations and compliance between parent and child versus child initiation expected	Increased expectation of caregiver participation and initiation Higher for male caregiver expected to be in provider role Emphasize nonverbal participation/engagement
Relationship Quality	Moderate	Expected communication patterns vary by gender and role; Direct communication more expected High levels of overall verbal communication not as expected; Non-fearful respect more desired than warmth	No expectation of equal participation Deemphasized amount of expected verbal communication Deemphasized displays of physical affection
Discrimination and Favoritism	New	Discrimination/Favoritism is a more culturally salient indicator of functioning in comparison to the original FAMPROS Coalition code; Coalition required multiple individuals “teaming up” whereas Discrimination/Favoritism allows for individuals discriminating or favoring one another	The original FAMPROS “Coalition” code replaced by Discrimination and Favoritism, which consists of the amount of negative behavior towards one person compared to others (discrimination) and the amount of positive behavior directed at one person compared to others (favoritism); these fall on opposite ends of the same scale–high discrimination (−7), high favoritism (+7), None (0)

Table 2.

Activity-Specific Problem-Solving Process Codes

New Activity	Participants	Activity Specific Problem-Solving and Planning Process Codes
House Building	Whole Family	• Extent of family understanding of the activity • Extent of family teamwork and cooperation • Quality of house built/ability to complete house pre-post storm • Quality of family’s overall house building process
Problem Solving Discussion	Couple	• Extent to which couple’s problem is defined/clear and understood • Extent of problem resolution • Quality of proposed resolution/solution • Quality of couple’s overall problem solving process
Hopes and Plans Discussion	Whole Family	• Extent to which family’s hope and plan are defined/clear • Extent of family discussion and planning • Quality of the proposed plan • Quality of the overall discussion and planning process

Across code definitions, a common change was to add or emphasize nonverbal indicators of functioning throughout score descriptions, given that the original emphasis on verbal communication was insufficient in this setting. This was especially important given the way the FAMPROS is scored between specific dyads; the emphasis on nonverbal behaviors allows for more variation in dyads, such as father-daughter dyads where less verbal communication is customary and not necessarily negative. In the following paragraphs, we describe the final global code domains rated across activities.

The Positive Behavior code assesses the amount of positive behavior an individual directs to another, as well as the overall amount of positive behaviors at the family level. Negative behavior refers to the amount and intensity of evident harshness, anger, dominance, disagreement, and complaining. Although domains of positive and negative behavior were minimally changed, one notable adaption included altering a child’s reticence and lack of eye contact—coded as a negative indicator in FAMPROS—to a potentially positive indicator reflecting obedience/respect. Coders were trained to determine this distinction, as well as that of other ambiuous behaviors, through team discussions during joint video ratings. In this case, the team explicitly discussed distinguishing the nature of eye contact by considering other verbal and nonverbal indicators, such as those related to child affect, especially fear/comfort, as markers to help determine when a lack of eye contact was negative or positive.

The Participation code rates how active a family member is during interactions and activities. This is rated by individual, not dyad. For instance, a mother who does not participate in the interactions with her daughter and partner, shows little interest, and appears withdrawn, would receive a rating of Not at All Active, while a mother who appears engaged throughout activities, initiates some ideas, and participates throughout may be rated as Extremely Active. Differing levels of participation across members can provide a helpful indicator of family hierarchies, as well as unbalanced dynamics. The construct of participation remained the same, and moderate changes were made to code definitions.

Relationship Quality assesses how well a family dyad and/or the whole family relates with one another, as well as interpersonal closeness. In contrast to the previously described domains, relationship quality is bidirectional, accounting for reciprocal interactions. For example, between a father and son, this might consist of displays of support, reciprocal positive behavior, open communication, and providing opportunities for the child to speak.

Discrimination and Favoritism refers to the amount of negative behavior or positive behavior towards one person when compared to others in the family; this is rated by dyads (e.g., father to child). Discrimination and favoritism were conceptualized as two ends of the same bidirectional scale (i.e., high discrimination [−7] on one end, and high favoritism [+7] on the other, and zero [0] equaling no discrimination or favoritism). Discrimination includes behaviors such as never taking the side of one person while siding with others; not agreeing/supporting one individual while doing so for another; ignoring a certain individual; or isolating one person in any way. Favoritism refers to speaking more to a certain person; protecting one more than others; giving special treatment; telling others to only listen to one person, and so on.

Regarding scoring, we maintained the original FAMPROS 7-point and 5-point scales across coding domains, but we made significant modifications to scoring descriptions. To increase ease of scoring, we added anchoring descriptions, examples, and questions at the endpoints and midpoints of the scales. Additionally, we created four composite scores: (a) Quality of Family Interactions– Relational: the average of family-level Positive Behavior, Negative Behavior (reverse scored), and Relationship Quality ratings; (b) Quality of Family Interactions–Process: the average of one rating from each activity reflective of problem-solving and planning processes (Quality of Problem Solving in activities 1 and 2; Quality of Discussion/Planning in activity 3); (c) Quality of Couple Interactions– Relational: average of couple Positive Behavior, Negative Behavior (reverse scored), and Relationship Quality; (d) Quality of Parent-Child Interactions–Relational: average of parent-child Positive Behavior, Negative Behavior (reverse scored), and Relationship Quality.

Rating Procedures

The final rating procedure for our adapted measure allows for one coder to complete ratings while watching the videotaped interactions—a process that requires approximately 30 minutes. This is likely a reasonable goal for future uses of this measure. However, the several steps we followed to reach this final procedure may be useful for future adaptations for new contexts in order to facilitate the adaptation process and to reach reliability across raters, especially raters who are from different cultures and contexts, and often also geographically separated. Here we describe each phase.

The rating procedures during the early adaptation phases began with group meetings including both American and Kenyan members of the research team. This consisted of individuals watching videos and taking structured notes followed by in-depth discussion, and then agreement on consensus ratings. This process served the dual purpose of identifying aspects of activities or the codebook that needed to be tailored and becoming familiar with rating. This was a lengthy process that served as an initial step to codify interactions and calibrate ratings, taking an estimated 90 minutes per family. This was done for four families, at which point we had a viable and fully manualized codebook and scoring system. The second phase was a more structured process in which three raters (1 American, 2 Kenyan) first rated the families independently. Ratings were then compared and discussed, and differences were reconciled to come to 100% agreement across all codes. This was done for four additional families until few or no adaptations were being made to the activities or codebook. Rating typically took 75 minutes.

After an essentially fully adapted tool was available, we continued consensus coding but with only two raters—one from the U.S. and one from Kenya. Raters scored independently using the final codebook and were able to score while watching the videos for the first time, which took 30 minutes. They then discussed discrepant ratings in person or via Skype, which required approximately 15 minutes. When the codebook failed to clarify a rating, the local rater’s score is prioritized and given as the final score. We continued rating until 80% agreement was reached among independent raters for four families prior to any consensus discussion (range: 83.1%–93%). This met our predetermined criterion for allowing local coders to complete scoring independently. In using this measure in the future, therefore, we can follow these third phase procedures of two-rater consensus coding to reach reliability and shift to one-rater coding. (For this study, however, we did continue to conduct final consensus rating in order to develop gold-standard videos with full ratings to use in future trainings.)

AGREEMENT WITH INTERVIEW ASSESSMENT

Among the four families with complete observational and interview ratings, there was 100% agreement on whether counseling was recommended. Here we present descriptions of the comparison across measures for two of those families. For Family A, consisting of a mother, father, and a 13-year-old son, overall family relationship quality ratings as assessed by the observational measure and interview both independently fell in the middle range, with a score of 4 on a 7-point scale from the observational assessment and a 45 out of 100 from the interview. Across both instruments, the couple’s marital relationship emerged as the most strained and the mother-son relationship as the strongest. Within the couple’s relationship, the observational assessment noted the mother rolling her eyes at the father, few interactions, and the mother remaining expressionless to father’s attempts at directions or jokes; they were, however, observed to have the ability to speak relatively calmly throughout activities during their limited interactions. Interview ratings demonstrated some love in the relationship, but also described the mother’s fear of the father, emotional distance, and a history of poor communication.

For Family B, consisting of a single mother and 16-year-old daughter, ratings across instruments were commensurate, demonstrating similar assessments of both positive and negative interaction patterns. Regarding mother-daughter relationship quality, the observational assessment yielded a rating of 4 (“Moderately Close”) on a 7-point scale, while interview ratings of their relationship functioning fell at 54 out of 100, a rating in the middle of the scale also reflecting moderate levels of positive indicators, such as communication and emotional closeness. Observational results further demonstrated difficulty problem-solving during the house-building activity and planning discussion, with the daughter taking the lead during both activities and the mother having difficulty engaging and, at times, hindering the progress of building or planning. From the house-building activity, one note reads as follows: “Even though the daughter makes progress on the house, the lack of effort and frustration with her mother eventually led her to tear down the house and stop working.” Interview results complement and corroborate these observational results. From the interview, the daughter indicates she is unhappy with her family role, as she has too much responsibility earning income for the family, and interviewers noted that the mother and daughter roles seemed reversed.

Discussion

Through a multistep process, we adapted an observational measure of family functioning that proved feasible and acceptable to administer and rate in a low-resource context. Results yielded a direct observational tool for assessing family functioning that could be adapted across other low-resource settings. FAMPROS was chosen as the anchoring measure for adaptation and was initially modified based on context-specific family interaction data. Through iterative pilot testing, collaborative discussion, and adaptations, tool components were further adapted to elicit behaviors analogous to the original tool via contextually relevant and acceptable procedures. The final measure retained many original FAMPROS constructs and the same foundational scoring system with slight modifications to domains of functioning and significant modifications to how domains were operationalized. The prompts and activities were almost completely changed in response to problems with acceptability in the local context. Additionally, initial comparison of the observational instrument with an alternative assessment of family functioning suggest promise for the measure’s criterion validity. Other authors have used alternatives to gold-standard clinician diagnosis for establishing criterion validity in resource-limited settings (Bass et al. 2008; Betancourt et al. 2009). In this instance, we used an in-depth interview modeled on a clinical assessment, with consensus ratings assigned by Kenyan and American clinicians and nonclinicians. In addition to demonstrating agreement between methods, the comparison helped demonstrate the tool’s potential for identifying families’ clinical problems to help determine treatment and where to begin treatment.

Next we discuss findings specific to this Kenyan setting that were influential from a cultural perspective in terms of how they influenced the coding system and how the measure is likely to continue to perform as-is in this context. Then we discuss ways in which the measure and methodology could be useful for adapting this tool or other observational tools for new contexts.

KENYA-SPECIFIC FINDINGS

An interesting conceptual finding about family functioning that informed adaptation was the relative lower emphasis on quantity of verbal communication in the context, especially between parents and children. Verbal communication was deemed important, especially between caregivers, but was not described as occurring to the extent emphasized in the context where FAMPROS was originally developed. In other words, the expected extent of communication differed, but not the expected quality or functional utility. This aligns with findings demonstrating communication saliency, type, and amount differ across cultures (Halford, Hahlweg, & Dunne, 1990; Rehman & Holtzworth-Munroe, 2007).

Similarly, the culturally driven emphasis on respect, roles, and hierarchy influenced the expected amount of engagement and type of participation by gender and age. This was particularly evident in the parent-child relationship, in which parental expectations of respect and obedience were highly valued, whereas the original FAMPROS gave equal or even more emphasis to expressions of affection. This complements literature suggesting cultural and socioeconomic contexts affect the type and quality of care provided (i.e., goals, practices) and the child’s reception of care (e.g., compliance; Grusec, 2011). This also aligns with parenting studies in the U.S. within low-resource communities that suggest more authoritarian parenting styles focused on safety (e.g., high monitoring, high obedience) may be more beneficial in higher risk contexts (Ceballo, Kennedy, Brgman, & Epstein-Ngo, 2012; Dixon, Graber, & Brooks-Gunn, 2008). If we had not modified our code system, the low verbal discussion across domains and parent-child interactions with few instances of affection would have been categorized as “poorer” functioning, which our qualitative data suggested would have been an invalid finding. By adapting our criteria to align more closely with cultural norms, we increased our confidence that the tool is assessing interactions in a more culturally valid way.

POTENTIAL APPLICATIONS FOR OTHER SETTINGS AND MEASURES

One central contribution of this study is an observational tool that could be adapted for other low-resource settings. For those interested in employing this specific observational measure of family functioning for a different low-resource setting, the tool’s codebook, scoring system, structure, and rating procedures are likely to apply given these revisions were driven by resource considerations. Codebook definitions may require some changes, however, based on context-specific data on behavioral indicators of functioning. In such cases, researchers and/or clinicians can apply a similar, though likely abbreviated, process of adaption used in this study. The rest of the process can be replicated within teams of local and international collaborators.

The second main contribution of this study is a structured methodology for developing and adapting observational measures for new contexts that can be applied to other measures regardless of specific content. The methods present a process addressing challenges in establishing cross-cultural assessments by presenting a feasible methodological process for adaptation. Although significant work has been done to advance the adaptation of survey tools (Jayawickreme et al., 2012; Kohrt et al., 2011; Rasmussen et al., 2015) and the development of novel, local tools (Kaiser et al., 2013; Kohrt et al., 2015), this work has largely been limited to survey instruments. This study helps to fill these gaps by providing an approach to culturally and contextually adapt direct observation tools. Here we demonstrate the process applied to family functioning in Kenya, though this methodology could be applied to measures of other constructs such as social communication or child developmental and behavioral disorders (Bayley, 2006; Kim-Cohen et al., 2005; Pelham, Fabiano, & Massetti, 2005). A process to efficiently modify tools in a culturallymeaningful way creates a starting point for individuals to capture important psychosocial outcomes with some level of cultural or contextual calibration.

LIMITATIONS AND FUTURE DIRECTIONS

This study is limited primarily by its small sample and by its lack of comparison to other measures of family functioning that could allow for more formal analyses of construct validity. Rather, the results presented here are preliminary steps that promote reliability and validity in ways that could be tested in future studies. For this, a variety of methods employed for validating survey tools in previous global mental health work would be useful when adapted for assessing family-level constructs (Bolton & Tang, 2002; Bolton, Wilk, & Ndogoni, 2004; Kaiser et al., 2013). Further, while the variability in functioning of the small number of recruited families is a strength of this study at this stage of measure development, future, larger studies will be valuable. Larger studies would be helpful for better exploring potentially interesting comparisons across families by child age or tribal affiliation as well as for identifying more nuanced patterns of interactions that could potentially reveal subtypes or profiles of family functioning. For instance, families able to effectively solve problems in highly stressful environments may exhibit interaction patterns consistent with a “resilient” family profile. We began to see some patterns emerge that could be hypothesis-generating, but gathering adequate data to support these was well beyond the scope of this study. Such work could inform the use of the tool as a clinical screener for families in need of family treatment or those with potentially resilient profiles who may benefit from a different form of intervention (e.g., economic). Future work will also be needed to determine whether this measure and process is indeed useful across other contexts, as well as the extent of adaptation needed in different cultural settings.

CONCLUSION

It proved feasible and acceptable to adapt, administer, and rate a direct observational measure of family functioning in a low-resource setting. The tool elicited variable behaviors and interactions across families, was able to be scored consistently by trained lay coders with high agreement, and showed promising similarities to a separate and distinct assessment of family functioning. Results point to its potential for use in other settings with limited access to resources. An observational tool useful across contexts could improve the quality of measurement of family relationships in LMICs, or low-resource areas in HICs, for research and clinical purposes. The collaborative process of adaptation also proved feasible and efficient, suggesting potential utility for culturally adapting observational tools for other contexts and outcomes of interest. Specifically within the Kenyan setting, the resulting measure provides a tool for conducting more rigorous research on family-based intervention effectiveness. Better understanding the impact of interventions has the potential to identify optimal strategies for improving family functioning and promoting child well-being.