Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jun 13.
Published in final edited form as: J Emot Behav Disord. 2011 Jun 2;20(4):260–274. doi: 10.1177/1063426611398876

The Role of Quality Service Systems in Involving Families in Mental Health Treatment for Children with Severe Emotional Disturbances

Lindsay Satterwhite Mayberry 1, Craig Anne Heflinger 1
PMCID: PMC4905723  NIHMSID: NIHMS730209  PMID: 27307690

Abstract

Family involvement in the planning and execution of mental health treatment has been shown to positively influence child outcomes; however, there is wide variability in the levels of involvement by families. The current study investigated the influence of child, family caregiver, service system, and community factors on the level of family involvement in the mental health treatment of children with severe emotional disturbances (SED). While prior work in this area has primarily focused on family and child characteristics, the current study adds to the knowledge base by also exploring factors associated with the quality of service delivery that may influence levels of family involvement. The current study is a secondary analysis of existing data from the Impact Study of Medicaid in Mississippi and Tennessee (Brannan & Heflinger, 2006) that used in-depth interviews with parents and case review ratings. One hundred thirty-six children and adolescents who received Medicaid coverage for mental health treatment in Mississippi and Tennessee were the focus of this study that assessed child and caregiver characteristics at the beginning of the study and rated service system factors six-month after the study began. Results of the multiple regression analysis revealed the quality of the service delivery system had the most influence on level of family involvement. Results suggest that levels of family involvement may have less to do with caregiver health, or other family characteristics, and more to do with the quality of the service delivery system.

Keywords: Family Involvement, Family Participation, Mental Health, Children with Severe Emotional Behavioral Disturbances, SED

Children and adolescents with severe emotional disturbances (SED) may require care that goes beyond the services provided by the mental health system alone; children with SED typically have challenges that require services from the schools, the community, and occasionally the juvenile justice system (Stroul & Friedman, 1986). Current best practice models suggest that each of these service systems be coordinated and include the parent and child as an equal decision maker. The focus on family involvement and comprehensive services is reflective of a paradigm shift in the way mental health systems view the family -- over the past twenty years, families have come to be viewed as partners and assets in the treatment process. Contrary to earlier views of family members as being blamed for their children's mental or emotional health problems (Grunebaum, 1984; Hatfield, 1982; Spaniol, Jung, Zipple, & Fitzgerald, 1987), current federal priorities emphasize the inclusion of family members in all aspects of the treatment process (Osher & Osher, 2002; Pullmann, 2009). Policy initiatives to involve families in the planning and treatment process have been extensive, and include the development of System of Care (SOC) principles as a best practice (Stroul & Friedman, 1986). The concept of service providers collaborating with each other, and with the family, to develop a relevant and comprehensive treatment plan is appealing and should not be limited to just SAMHSA-funded System of Care communities but rather adopted as a best practice in all communities. Nonetheless, reviews have found that even service systems that have been promoted as SOC models may not be effectively involving families (Cook & Kilmer, 2004; Koroloff & Friesen, 1990) and other communities with SOC funding have been criticized for failing to fully implement effective strategies associated with involvement of family members (Measelle, Weinsten, & Martinez, 1998; Pullmann, 2009).

Prior to the development of the SOC philosophy, and its implementation in numerous states, children with SED were often in overly restrictive settings, and many children were not receiving the services they needed (Stroul & Friedman, 1986). Thus, the goal of the SOC approach is twofold: improve outcomes for children with SED and keep children in the least-restrictive environments required for their care. Although often poorly realized, family involvement is a critically important principle to the goals of the SOC approach (Brannan, 2003). Therefore, it is crucial to understand what influences family involvement in the mental health treatment process in order to more fully contribute to best practices and improve treatment outcomes.

Family Involvement

Research has established the importance of family factors in improving treatment outcomes for this population (Bickman, Foster & Lambert, 1996; Brannan & Heflinger, 2006; Foster, Saunders, & Summerfelt, 1996; Richards, Bowers, Lazicki, et al., 2008). Specifically, the levels of strain of primary family caregivers of children with SED have been shown to profoundly affect treatment and outcomes. Increased caregiver strain significantly increases children's use of mental health services (Brannan & Heflinger, 2006). Family and caregiver characteristics, such as education (low levels), resources (low levels), and caregiver strain (high levels) have consistently been shown to have effects on receiving more restrictive services (Bickman, Foster & Lambert, 1996; Brannan & Heflinger, 2005; Foster, Saunders, & Summerfelt, 1996), having longer lengths of stay (Foster, 1998), incurring higher costs of care (Brannan, Heflinger, & Foster, 2003), higher severity of child symptoms, and lower levels of coordination with service providers (Yatchmenoff, Koren, Friesen et al., 1998). Parental involvement in child and adolescent treatment has also been shown to improve treatment outcomes for both internalizing (Clarke, Hops, Lewinsohn et al., 1992) and externalizing disorders (Erhardt & Baker, 1990).

Despite these findings that family factors influence children's mental health treatment, research on the factors that influence family involvement is not as well developed. Research is needed on which factors support or impede family involvement in the planning and execution of children's mental health services and the treatment process. Contributing to the lack of research on the impact of various factors on family involvement is the absence of a consistent definition. For the purpose of this study, which aims to understand what factors affect family involvement in the planning and execution of mental health treatment, family involvement will be conceptualized using Curtis and Singh's (1996) four part definition: “involvement of families in all aspects of services at all times, education of the family about the nature of services and how the service system works, involvement of the family in decision-making, and keeping the family informed on the process and progress of treatment” (p. 505).

The construct of family involvement in the treatment process is complex, consisting of multiple constructs and processes including family empowerment, family support, family engagement, and family choice (Hoagwood, 2005). Family involvement, as defined by Curtis and Singh (1996) is distinct from each of these constructs. For instance, this definition of family involvement is conceptually narrower than definitions that include family advocacy or family empowerment, which have broader implications for psychosocial development (e.g. a belief that one can work to bring about change, a feeling of personal control) and socio-political involvement (e.g. participation in advocacy activities) (Curtis & Singh, 1996; DeChillo, Koren, & Schultze, 1994). In addition, family involvement is a separate construct from family collaboration in mental health treatment, as increased family collaboration has been shown to positively impact family involvement and satisfaction (DeChillo, 1993). Thus, we believe that an adequate level of family involvement in the treatment process, including communication between the family and treatment providers, is a necessary (though not sufficient) condition for family advocacy, family empowerment, collaboration, and family satisfaction.

Family engagement has been typically defined in research as the instigation and maintenance of service use over time (McKay & Brannon, 2004), or attendance and adherence to treatment regimens (Nock & Ferriter, 2005). There is evidence the factors that influence family engagement are related to parent characteristics alone (Pekarik & Stephenson, 1988), as guardians are solely responsible for instigating service use and bringing children to services. Characteristics most influential in family engagement include parent perceptions of the child's problems and the effectiveness of treatment (Morrissey-Kane & Prinz, 1999; Nock & Ferriter, 2005). A recent systematic review of family involvement in the treatment process highlights the growing body of work regarding prolonged engagement in treatment (Hoagwood, 2005). Studies on prolonged engagement have highlighted the importance of family perceptions of the mental health system (i.e. believing that mental health treatment is helpful, believing that mental health problems are capable of changing) in sustaining treatment for children (Kazdin and colleagues, 1997, 1999, 2000, 2003, 2004). The same factors that engage parents in treatment may also influence their level of sustained involvement in the treatment process. Finally, the involvement of families in children's mental health treatment planning, where family members are treated as equal decision-makers, is a separate construct from the engagement in family therapy, in which the family system is regarded as the unit of treatment.

Most attempts to measure family or parental involvement have been based on reports by service providers (Baker, Blacher, & Pfeiffer, 1993), parents/caregivers, their children (Hawley & Weisz, 2005; Israel, Thomsen, Langeveld, & Stormack, 2007; Robinson, Kruzich, Friesen, Jivanjee, & Pullmann, 2005), or on a set of observed behaviors (such as attendance at treatment planning meetings, completions of daily monitoring sheets, and extra communications with treatment providers) (Richards et al., 2008). There is one published psychometrically sound scale to rate family involvement, the Family Involvement Scale, which assesses both the family's perception of their involvement in mental health services (Singh, Curtis, Cohen et al., 1996a) and the professionals’ perception of the family's involvement (Singh, Curtis, Cohen et al., 1996b).

It should be noted that each of these methods for measuring family involvement in treatment may be biased in specific ways. For instance, providers may be likely to report family involvement with unconscious bias against specific demographic or cultural characteristics of the family (Baker et al., 1993), or to conflate family involvement with treatment outcomes (i.e. blaming family members when a child's treatment is not proceeding as expected). Further, measures of family involvement that focus only on observed behaviors, such as caregiver attendance at meetings or completion of requested tasks (Richards et al., 2008), may overlook an important aspect of the definition of family involvement, that is, that families are kept informed and updated by the service system. Likewise, self-reported accounts of family involvement, taken from caregivers, have demonstrated bias in previous studies. Curtis and Singh (1996) found that caregivers with more years of education reported having lower levels of family involvement. The authors suggested that respondents with higher levels of education had higher expectations for their level of involvement and so under-appreciated and underreported their level of involvement as compared to respondents with lower levels of education. Thus, the negative relationship between caregiver education and levels of family involvement may be unique to caregiver-report measures, as other studies have found socioeconomic status to have a positive relationship with family involvement when using other measures (Baker et al., 1993). This summary of the different methods that have been used to capture the levels of family involvement highlight the importance of obtaining reports of family involvement from numerous sources: the family members, the child, and the providers.

Conceptual Models Linking Family Involvement, Service Use, and Treatment Outcomes

This study builds on Bronfenbrenner's (1977, 1996) work that describes human development within a social-ecological framework, acknowledging that children with SED live in families who reside in communities that are also influenced by broader societal factors. This model can be adapted to depict determinates of family involvement. That is, level of family involvement in the mental health treatment of their child in the current study is considered to be determined by four levels of factors: (a) child, (b) family, (c) service system, and (d) community factors, with measures described in detail below.

Previous studies on family involvement have primarily focused on child and family specific socio-demographic or treatment related characteristics. For example, more severe mental health symptoms, Caucasian race, male gender, and younger age have all been associated with higher levels of family involvement (Baker et al., 1993; Richards et al., 2008). Caregiver characteristics such as marital status and lower socioeconomic status (i.e. lower levels of educational attainment and lower family income) (Baker et al., 1993) have been associated with lower levels of family involvement in mental health treatment. Some service system characteristics have been examined as well. For example, Robinson and colleagues (2005) examined the influence of treatment setting (residential as compared to outpatient) and found decreased levels of family involvement for youth in residential treatment facilities for several reasons, including distance between family home and treatment setting, and restrictions placed on family communication with the youth. In addition, Hawley and Weisz (2005) found that increased perceptions of therapeutic alliance between the therapist and the primary caregiver was associated with increased family involvement and sustained engagement in treatment, whereas therapeutic alliance with the child was not associated with increased child or family involvement in the treatment process. It should be noted, however, that neither the quality of the service system nor community characteristics have been included in previous studies as influencing family involvement.

Study Purpose

Adapting Bronfenbrenner's social ecological model, the present study examines the influence of child, family, service system, and community characteristics on the level of family involvement in the mental health services delivery system of their child. We predict that child and family factors found important in previous studies will continue to predict family involvement to varying degrees. Specifically, we expect younger, white children, with more severe internalizing and externalizing symptoms to have higher levels of family involvement (Baker et al., 1993; Richards et al., 2008). Children with co-occurring disabilities or chronic illnesses may have lower levels of family involvement (Baker et al., 1993). Further, we hypothesize that children who live with parents (as opposed to guardians who are often state appointed) that are married or partnered, with higher socioeconomic status (monthly household income and level of education) will have higher levels of family involvement (Baker et al., 1993). Additionally, we predict that caregivers with higher levels of strain and poor health will be associated with decreased involvement (Brannan & Heflinger, 2006).

We further hypothesize that factors assessing the quality of the service delivery system and community level factors will also be important in predicting levels of family involvement in the mental health treatment of children with SED. Service system factors, based on SOC principles, include the accessibility (or availability) of behavioral health services, the array of various types of services, the coordination of services, the degree to which treatment was individualized to the needs of the child and family, and the degree to which the treatment plan focused on both short and long-term goals and transitions (versus only focused on acute problems) (Hernandez et al., 2001; Stroul & Friedman, 1986). We hypothesize that children in more rural areas will have lower levels of family involvement, based on evidence that rural families may be less likely to receive adequate mental health treatment (Hartley, 2004; Hartley, Bird, & Dempsey, 1999) and that increased distance from treatment facility is associated with a decrease in family involvement (Baker et al., 1993; Robinson et al., 2005). Finally, we predict that children residing in the state of Mississippi will have increased levels of family involvement, because the fee for service system has been linked to increased access to behavioral health services, as compared to the managed care system in Tennessee (Brannan & Heflinger, 2005; Cook et al., 2004).

Methods

The current study is a secondary analysis of existing data from the Impact Study of Medicaid in Tennessee and Mississippi (Impact Study) (see Brannan & Heflinger, 2006). As part of this larger multi-state study (Cook et al., 2004), in-depth standardized face-to-face interviews were conducted with caregivers of children enrolled in Medicaid who had a serious emotional disorder (SED) and the identified child (if age 12 or older) at first contact in the study (baseline) and again six-months after the initial interview (follow-up). Baseline interviews occurred as families agreed to participate in the study, which could have been up to several years after the initial Medicaid-funded mental health service that identified the sample for recruitment. All of the children in this study were receiving ongoing mental health treatment paid by Medicaid at baseline, thus this study was not a program evaluation or intervention study but, rather, a longitudinal study of children with SED (see also Cooke et al., 2004).

In the Tennessee/Mississippi sites only, study staff were also trained to conduct interviews with service providers, review the clinical records of children in the study, and implement service testing methods (described below). These activities were completed at the time of follow-up, six months after the baseline interviews with children and caregivers. The current study combines baseline and follow-up data, including information from the service testing, to test the conceptual model examining child, family, service system, and community factors and their influence on level of family involvement in their child's treatment during the six-month study period.

Service testing

In the Impact Study, the level of family involvement and the other service system factors, described below, were captured using a “service testing methodology.” Service testing methodologies primarily using interviews and record reviews, such as those used in the present study, have also been employed to evaluate how well SOC principles are implemented in practice and have been successfully used in SOC sites (Hernandez et al., 2001; Stephens, Holden, & Hernandez, 2004). In addition, they have been used to monitor the quality of care to children in several states, including federal lawsuits involving the mental health or child welfare systems (Bazelon Center for Mental Health Law, 1998; Tennessee Commission on Children and Youth, 2004).

Service system ratings were part of a standardized service testing process that included clinical file reviews and interviews with the child, the primary family caregiver (note that these interviews were part of the Impact Study), and at least two service providers who had been involved over the past six months (added for the service testing). In addition to the CBCL and other child and family measures described below, the standardized interviews at follow-up included a detailed listing of the type and amounts of services received, and parent and child report of their experiences with the service providers including satisfaction and their participation in a variety of family support and treatment planning activities. Clinical record review also included examining the treatment plan(s) for caregiver involvement.

Sample

Participants in the current study included 136 children who (a) met the twofold criteria for SED at baseline (diagnosis and impaired functioning1), (b) had a history of intensive levels of mental health service use or great quantities of community-based mental health service use in the past, and (c) for whom study staff were able to conduct a case review rating for level of family involvement. The participants were enrolled in and had historically received mental health services through their respective state's Medicaid program. The children's ages ranged from 6 to 17 years with a mean of 11.8 and a standard deviation of 2.8 years. Around 40% were adolescents, age 13 to 17 years old. Three quarters of the sample were male, which is representative of the proportion of males in the SED population. The sample was approximately evenly divided between white and black children, and children and their caregivers reported the same racial identity in 88% of the cases.

Measures

The dependent variable was the level of family involvement in treatment, and the four types of independent variables were: (a) child, (b) family/caregiver, (c) service system, and (d) community factors. Each is described below.

Family involvement

The performance of the service system with regards to family involvement was rated by study staff for the six-month time period between baseline and follow-up interviews using a standardized rating scale in a service testing protocol. Study staff who conducted the interviews and reviewed the children's clinical records used a standardized 6-point scale (ranging from 1 “completely unacceptable” to 6 “optimal) to rate all service system variables, including the dependent variable, family involvement (see Table 1). An “unacceptable” rating (falling between one and three) for family involvement essentially reflected that family members were involved on a limited basis or not at all and/or that treatment-planning and other meetings had not been scheduled to accommodate family schedules. Table 1 provides examples of the anchors used to provide ratings on the degree of family involvement for a family in the study

Table 1.

Example of criteria used to rate levels of five service system characteristics

Service System Characteristics
Score Family Involvement Access & Array Coordination Individualized Long-term View
Optimal Rating = 6 Key Family members and/or child's substitute caregiver(s) are full, effective, and on-going participants in all aspects of assessment, planning services, making service arrangements, selecting providers, monitoring, and evaluating services and results. Special accommodations or supports are offered as needed to assist participation. A full array of behavioral health supports and services is available to reach optimal levels of functioning. There is a single point of coordination for the child/family services, and the service coordinator works with all members of the service team and the child/family. All necessary supports and services are assembled into a holistic and coherent process, with child/family preferences reflected. The child has an explicitly written and up to date LTV that is consistently articulated among service members. Transitions are anticipated, and the child's strengths and needs are included.
Substantially Acceptable Rating = 5 Key family members are regular participants in most aspects of services. Special accommodations to facilitate participation may be made on some occasions, if requested by the family. An array of behavioral health supports and services is available to help the child reach favorable levels of functioning. Formal and informal supports and services are generally satisfactory to the child/family. There is a generally effective single point of coordination and accountability for the child/family's services and results. Essential behavioral health supports and services are assembled into a holistic and sensible service proves with a workable fit between the child/family situation and the services. The child has a clearly written LTV that anticipates the next major transition, and considers the strengths and needs of the child/family.
Sufficiently Acceptable Rating = 4 Key family members are regular participants in most aspects of services. Special accommodations to facilitate participation may be made on some occasions, if requested by the family. An array of services and supports is available to help the child reach minimally acceptable levels of functioning. Services are available, although they may require a lot of effort to obtain, and are seen as marginally acceptable to the child/family. There is minimally adequate coordination and accountability for the child/family's services and results. Services are minimally integrated across settings. Basic behavioral health supports and services are assembled into a sensible process having a minimally acceptable fit between child/family situation and the service mix. Some child/family preferences are reflected, but there is some conflict between family situation and services. The child has a written set of strategic goals that creates and implicit LTV that is accepted and sued by service team members. The next major transition is anticipated in these goals.
Partially Acceptable Rating = 3 Key family members are notified of the service planning meetings. Meetings are held at the convenience of the service providers and family participation is limited to planning and annual evaluations. A limited set of supports and service may be available and used, but may be seen as partially unsatisfactory by the child/family. The array provides few options and access requires a high level of effort. There is limited coordination of services with little accountability for service delivery and results. Services are fragmented and breakdowns may occur occasionally. Basic behavioral health supports and services are not assembled into a sensible service process. The fit between child/family situations is inadequate to meet identified needs. The child has several goals set by one or more agencies that create a common planning direction used by service team members. The goals address the next major decision, but there is no formal plan in place.
Substantially Unacceptable Rating = 2 Key family members are notified late about service planning members. Meetings are held at the convenience of service providers. Plans are made before the meetings and parents are expected to accept what is offered. Few supports and services may be available and used. They may be seen as generally unsatisfactory. There are very few options for services, substantially limiting the use of professional judgment and family choice. Access is significantly limited. There is limited coordination of services with little accountability for service delivery and results. Services are fragmented and breakdowns may occur occasionally. Basic behavioral health supports and services are not assembled into a sensible service process. The fit between child/family situations is inadequate to meet identified needs. The child has several goals set by one or more agencies that create a common planning direction used by service team members. The goals address the next major decision, but there is no formal plan in place.
Completely Unacceptable Rating = 1 Service planning and decision-making activities are conducted at times and places that prevent effective consumer participation. Decisions are made without the knowledge or consent of parents, caregivers, or the youth. Services are denied because of failure to show or comply. Important information is withheld from parents. Few if any supports and services are provided. They may not fit the actual needs of the child and may not be dependable over time. There is no coordination and accountability for services and results. Needed services may be absent or fragmented. The family may “get lost in the system” for periods of time. Basic behavioral health supports and services may not be provided or may not be assembled into a unified process. The fit is unacceptable or services are woefully inadequate to meet identified needs. Child/family preferences are not reflected and they report undependable or conflicting service strategies. There is no common future planning direction that is accepted and used by service team members. Goals do not address required future transitions.
Open in a new tab

© Human Systems and Outcomes, 1998

According to the service testing protocol used (Human Systems and Outcomes, Inc., 1998), “the central concern of this exam was that the family members are active participants in shaping and directing behavioral health service arrangements that impact on their lives. Emphasis was placed on direct and ongoing involvement in all phases of service: assessment, planning, selection of providers, monitoring modifications, and evaluation (p. 34)”. Study staff also assessed the level of education about mental health symptoms and treatment possibilities that family members had received from their service system with interview questions such as “How often did [agency] explain things in a way you/your child could understand?” and “Did anyone from [agency's] office tell you about the different kinds of treatments that are available for your child's condition?” The level of family involvement in decision-making was assessed through information collected from interviews with providers and caregivers, plus caregivers’ response to the specific question “How often were you involved as much as you wanted in decisions about your child's services?” Thus, this assessment focused on all four aspects of Curtis & Singh's (1996) definition of family involvement: (a) family involvement in all aspects of treatment from assessment to evaluation, (b) education of the family about the services and service system, (c) involvement of the family in decision making (regarding meeting times, treatment goals, interventions, etc.), and (d) open communication from service providers to family members about treatment and scheduled team meetings.

To insure the reliability and validity of service system ratings, staff who provided ratings were supervised by one of the two principal investigators to insure that there was adherence to rating guidelines. These meetings between study staff who provided ratings and their supervisors included jointly reviewing the following information: (a) the interview data from caregiver, child, and providers; (b) clinical records and other documents obtained, and (c) a written synopsis of the case. During supervision, the rating for family involvement was either approved as originally submitted or revised following the discussion and the final family involvement rating was used in this study.

Study staff rated family involvement at follow-up, the same time that they rated the other service system factors. The average family involvement rating was 4.0 (SD =1.5). Overall, 71% of the sample had an acceptable rating for family involvement.

Child factors

All child variables were obtained from the baseline interviews. Demographic variables of age, race, and gender, mental and physical health variables were included in this factor. The Child Behavior Checklist (CBCL: Achenbach, 1991) assessed the children's level of psychological symptoms. The CBCL is a widely used parent-report assessment in child psychotherapy with established reliability and validity. Specifically, the analyses focused on the Internalizing and Externalizing scores, as higher levels of family involvement and caregiver strain have been linked to more severe externalizing symptoms (Brennan & Brannan, 2005; Corcoran & Dattalo, 2006; Corcoran & Pillai, 2006). Overall, 100% of the sample had an Externalizing t-score that surpassed the clinical cut off, and 83.82% of the sample had an Internalizing t-score that surpassed the clinical cut off point. In other words, this sample had severe externalizing and internalizing symptoms.

A single item was used to measure child health, assessing the incidence of co-occurring disabilities or chronic illness. This item, asked of parents, was “Have you ever been told by a doctor, nurse, or other health professional that [child name] has any disability or chronic illness? (If yes, what type?)” Nearly 60% of the sample had been diagnosed with a disability or chronic illness, co-morbid with SED.

Family factors

All family variables were obtained from the baseline interview. Family measures included caregiver relationship to child, caregiver marital and education status, monthly household income, level of caregiver strain, and caregiver health. Caregiver relationship to child, marital status, and education, were each collapsed into two or three categories for reasons related to sample size and statistical power. For the caregiver relationship to child, biological, adoptive, or foster parents were included in one group (n=105), while “other legal guardians,” primarily grandmothers and aunts/uncles, composed the second group2 (n=36). Self-reported marital status was collapsed into three categories: married or living as married/partnered, divorced/separated/widowed, and never married. Sixty percent of the sampled caregivers were not married or partnered. Self reported educational attainment was also divided into three groups: less than high school degree, graduated high school, and more than high school degree. Two-thirds of the sampled primary caregivers (68.4%) reported having completed high school.

Information on total household income was gathered through a series of questions inquiring about a variety of sources of income. These were summed to report a total monthly household income, as an alternative to asking caregivers to estimate their total household income and report a single number. Total monthly household income ranged from $235 to $8030, and 73% reported total monthly incomes under $2000.

The total score from the Caregiver Strain Questionnaire (CGSQ) (Brannan, Heflinger, & Bickman, 1997) was used to assess overall levels of family caregiver strain experienced by the caregiver. The CGSQ is a 21 item survey (with responses on a 5-point scale from 1 “not at all” to 5 “very much”) that assesses the number and frequency of symptoms of strain the caregiver is experiencing from the burdens of providing daily care. It is a reliable, valid, and frequently used scale for measuring caregiver strain among families of children with emotional or behavioral disorders (Brannan & Heflinger, 2005). Caregivers reported moderate levels of strain (M = 3.1, SD = 1.6) (see Table 1).

Caregivers were also asked to rate their own health on a scale from 1 (poor health) to 5 (excellent health) using an item from the National Health Interview Survey (CDC, 2007). Overall, caregivers reported “fair” to “good” health (M = 2.6, SD = 1.1) with 52% of the sample reporting “poor” or “fair” heath. This represents rates that are worse than national averages (Salomon, Nordhagen, Oza, et al., 2009).

Service system factors

The performance of the service system was rated by study staff on the quality of services that had been provided to the child in the 6-month period since the baseline interview in the following four areas: (a) behavioral health service access and array, (b) service coordination, (c) individualized service, and (d) long-term view in treatment planning (see Table 1). The procedures, 6-point scale, and methods used to insure reliability and validity were the same as those described above for family involvement. Study staff, who had conducted the interviews and reviewed the children's clinical records, used a standardized 6-point scale (ranging from 1 “completely unacceptable” to 6 “optimal) to rate each of four service system variables. A rating between one and three on any of the service factors was considered unacceptable, and a rating between four and six, acceptable. “Unacceptable” ratings in any of the service system variables essentially reflected that the array of services available did not meet the basic needs of the child, that services were not adequately individualized to the needs of the child, that service coordination among the child's providers and school was fragmented or absent, and/or that treatment goals and services focused only on acute needs without taking the child's long term and developmental needs into account. As with the family involvement ratings described above, all case review ratings were reviewed in supervision to insure the reliability and validity of service system ratings and the service system variables used in this study were these approved ratings. The average ratings for each of the service system areas were array and access M =3.7, SD = 1.5, 60% acceptable; service coordination M = 3.6, SD = 1.7, 58% acceptable; individualized service M = 3.6, SD = 1.6, 57% acceptable; and long-term view M = 3.3, SD = 2.3, 30% acceptable. All four service system ratings had an average rating that was barely above the acceptable cutoff of 3.0. Due to the high multicollinearity between the four service system variables, an index was created (Cronbach's alpha = .87) that calculated an average for these four variables. This index, labeled the “Quality of Service System,” ranged from one to six with higher scores indicating higher levels of acceptable service delivery and was used in the multiple regression analyses.

Community factors

Community variables included the child's state of residence and rurality. The state of residence, either Mississippi or Tennessee, determined the type of Medicaid funding, fee for service or managed care (respectively). Rurality was based on the county in which the children resided. The Rural-Urban Commuting Area (RUCA) Codes (Version 2.0) were used to classify counties in Tennessee and Mississippi (Washington State Department of Health, 2006), and then dichotomized (1 = rural, 0 = not rural), corresponding with RUCA Categorization C (WWAMI Rural Health Research Center, 2006). The sample was almost evenly divided between residents of Tennessee and Mississippi and a little more than one third resided in rural communities.

Analyses

In addition to descriptive analyses of all factors and bivariate analyses to examine differences between children with high levels of family involvement versus those with low levels, we used multiple regression to predict levels of family involvement. In order to develop the most parsimonious model, we conducted a series of four regression analyses, examining each set of factors (child, family, service system, and community) separately, including those variables that were significant at p < .15 in the bivariate analyses. All significant predictors (at p < .05) from each equation were included in a final model.

Results

Bivariate Analyses

First, bivariate logistic regression analyses (for continuous predictors) and cross-tabulations with chi-squared statistics (for categorical predictors) were conducted to determine which predictor variables influenced the probability of having an acceptable level of family involvement. Several variables significantly (at p < .05) predicted the probability of having acceptable levels of family involvement when examined independently: child age, race, caregiver's self-reported health, behavioral health system array and access, service coordination, individualized services, long-term view, and state of residence/type of Medicaid system (see Table 2). In general, white, younger children who had healthier parents or a positive rating on each area of service system quality were more likely to have more involved families. Also, children living in Tennessee with a managed care Medicaid system had higher probability of acceptable family involvement than those in Mississippi with a fee for service system in these bivariate analyses.

Table 2.

Sample Summary Statistics: Subdivided into Unacceptable and Acceptable Ratings of Family Involvement

Family Involvement
Characteristics Total Unacceptable Acceptable
n 136 39 97
% (CI) or Mean (SD) % (CI) or Mean (SD) % (CI) or Mean (SD)
Child Factors
    Gender (%)
        Male 74.2 (66, 82) 82.1 (69, 95) 71.1 (62, 80)
        Female 25.8 (18, 34) 17.9 (5, 31) 28.9 (20, 38)
    Race (%)
        White** 48.5 (40, 57) 35.9 (21, 52) 53.6 (44, 74)
        Black 44.1 (36, 53) 51.3 (35, 68) 41.2 (31, 51)
        Other 7.4 (2, 12) 12.8 (2, 24) 5.2 (0, 10)
    Age (years, mean)** 11.8 (2.8) 12.2 (2.7) 11.6 (2.8)
    Comorbid Disability/Chronic Illness (%) 58.5 (51, 67) 64.1 (48, 80) 56.3 (46, 66)
    CBCL (t scores, mean)
        Externalizing 79.7 (7.7) 79.7 (9.9) 79.7 (6.7)
        Internalizing 75.3 (9.7) 75.1 (10.7) 75.4 (9.4)
Family Factors
    Caregiver Relationship to Child (%)
        Biological/Adoptive/Foster parent 77.8 (71, 85) 76.9 (63, 91) 78.1 (70, 87)
        Other Legal Guardian 22.2 (15, 29) 23.1 (9, 37) 21.9 (13, 30)
    Caregiver Marital Status (%)
        Never Married 19.1 (12, 26) 23.1 (9, 37) 17.5 (10, 25)
        Married/Partnered 41.2 (33, 50) 35.9 (20, 52) 43.3 (33, 53)
    Divorced/Widowed 39.7 (31, 48) 41.0 (25, 57) 39.2 (29, 49)
    Caregiver Education Status (%)
        Less than High School 31.6 (24, 40) 35.9 (20, 52) 29.9 (21, 39)
        High School Degree 28.7 (21, 36) 30.8 (16, 46) 27.8 (19, 37)
        More than High School 39.7 (31, 48) 33.3 (18,49) 42.2 (32, 52)
    Monthly Household Income ($, mean)* 1,650 (1,218) 1,448 (875) 1,731 (1,327)
    Caregiver Strain Total Score (mean) 3.1 (1.6) 3.3 (1.8) 3.1 (1.5)
    Caregiver Health (mean)*** 2.6 (1.1) 2.2 (1.1) 2.7 (1.0)
Service System Factors (%)
    Array & Access****
        Unacceptable 39.6 (31, 48) 73.0 (58, 88) 26.9 (18, 36)
        Acceptable 60.4 (52, 69) 27.0 (12, 42) 73.1 (64, 82)
    Service Coordination****
        Unacceptable 42.2 (34,51) 74.3 (59, 90) 30.1 (21, 40)
        Acceptable 57.8 (49, 66) 25.7 (10, 41) 69.9 (60, 79)
    Individualized Services****
        Unacceptable 42.7 (34, 51) 77.1 (63, 92) 30.2 (21, 40)
        Acceptable 57.3 (49, 66) 22.9 (8, 37) 69.8 (60, 79)
    Long Term View****
        Unacceptable 69.9 (61, 79) 87.5 (73, 100) 64.6 (54, 75)
        Acceptable 30.1 (21, 39) 12.5 (0, 27) 35.4 (25, 46)
Community Factors
    Rural (%)* 36.0 (28, 44) 43.6 (27, 60) 33.0 (23, 43)
    State of Residence (%)**
        MS/Fee for Service 53.4 (45, 62) 69.2 (54, 84) 47.4 (37, 58)
        TN/Managed Care 46.3 (38, 55) 30.8 (16, 46) 52.6 (42, 63)
Open in a new tab
*

p < .15

**

p<.05

***

p < .01

****

p < .001

Note: CBCL= Child Behavior Checklist (Achenbach, 1991); Caregiver Strain Questionnaire (Brannan, Heflinger, & Bickman, 1997) scored 1 (low) to 5 (high). Caregiver health similarly rated 1 (poor) to 5 (excellent).

Multiple Regression Analyses

Model 1: Child factors

The first model included the predictor factors pertaining to the individual child that were significant at p < .15 in the bivariate analyses. Child age and race were the only statistically significant predictors of family involvement in the bivariate analyses, and both variables remained significant in the multiple regression Model 1 (at p < .05). Younger children and white children had significantly higher rates of family involvement (see Table 3).

Table 3.

Multiple Regression Models of Child, Family, Service System, and Community Level Factors on Family Involvement (1=poor to 6=optimal)

Model 1 Child Factors Model 2 Family Factors Model 3 Service System Factors Model 4 Community Factors Model 5 All Predictors
Child Level Factors β (se) p β (se) p β (se) p β (se) p β (se) p
    Race: White (referent Non-white) 0.593 (0.227) .018 0.283 (0.215) .190
    Age in years −0.113 (.040) .005 −0.028 (0.036) .434
Family Level Factors
    Monthly Household Income in $1,000 increments 0.158 (0.082) .056
    Caregiver Health 0.350 (0.113) .002 0.195 (0.098) .048
Service System Level Factors
    Quality of Service System 0.663 (0.090) <.001 0.629 (0.096) <.001
Community Level Factors
    Rural −0.206 (0.288) 0.476
    State of Residence: TN/Managed Care (referent MS/Fee for Service) 0.477 (0.261) .070
(Adjusted) R2 0.08 0.09 0.37 0.04 0.41
df 2, 133 2, 131 1, 123 2, 133 4, 118
F score 6.29** 7.22** 54.55*** 2.55 15.60***
Open in a new tab

* p<.05

**

p<.01

***

p<.001

Note: Robust Standard Errors

Model 2: Family factors

The second model included all predictor factors pertaining to the family system or caregiver that were significant at p < .15 in the bivariate analyses. In this regression model, caregiver's self-rated health was the only statistically significant predictor of family involvement at p < .05. Monthly household income was no longer significant in Model 2, when controlling for caregiver health.

Model 3: Service system factors

The quality of the service system had a significant impact on the level of family involvement. In bivariate analyses, all four service system characteristics significantly influenced family involvement and in the regression Model 3 the Quality of Service System Index was significantly associated with family involvement (p < .001). Also, the service system model had a much higher adjusted R-squared statistic (.37 as compared with .08 and .09 for Models 1 and 2) and F statistic (54.55 as compared with 6.29 and 7.22) than the previous two models, indicating that the service system ratings explain four times as much variance in family involvement ratings as the previous two models.

Model 4: Community predictors

The fourth model included both community variables, as they were both significant at p < .15 in the bivariate analyses. In the bivariate analyses Mississippi children were significantly more likely to receive an unacceptable rating for family involvement, however, this finding did not hold in the multiple regression after controlling for rurality. In Model 4, neither community variable was predictive at p < .05.

Model 5: All factors

When all significant factors from Models 1 through 4 were included in a final model, only caregiver's health and quality of the service system remained significant predictors of family involvement. When controlling for other variables in the model, for every one-unit increase in reported caregiver health, there was an increase of .195 in the family involvement rating (p = . 048), and for every one-unit increase in the overall rating of the quality of the service system, there was an increase of .629 in the rating of family involvement (p < .001). Child race and age fell out of the model for interesting reasons. White children were significantly more likely to have healthy caregivers (β = .092, p = .021), but were not more likely to have a higher quality service system. Younger children were more likely to have a higher quality service system (β = −.567, p = .001), but were not more likely to have healthy caregivers. Implications of these results are discussed below.

Post-hoc logistic regression analyses were used to aid in substantive interpretation of these results. For these analyses, both family involvement and the quality of service system index were dichotomized into acceptable or unacceptable. Caregiver health and the dichotomized service system index rating were the only variables included in this pot-hoc logistic regression model. Having an acceptable service system increased the odds of having acceptable family involvement over nine times (OR=9.29, CI=3.3, 26.0). The relationship between caregiver health, quality of the service system, and family involvement can be seen more clearly in Figure 1.

Figure 1.

Figure 1

Open in a new tab

The Probability of Having Acceptable Family Involvement: Effect of Caregiver Health by Quality of Service System

As reflected in Figure 1, the effects of caregiver health on family involvement are less important if the child has an acceptable service system. Most strikingly, children who have an acceptable service system and a caregiver with poor health are more likely to have acceptable family involvement than children who have an unacceptable service system and a caregiver with excellent health. Levels of family involvement seem to have less to do with caregiver health and more to do with the combined quality of a service system's array and access to services, service coordination, individualized services, and long-term planning for the needs of the child/family.

In Model 5, race (white versus non-white) no longer significantly predicted family involvement. Previous studies have found significant relationships between race and level of family involvement, however, the present findings indicate that the effects of race appear to work primarily through its association with caregiver health.

Discussion

This study is the first to examine family involvement in mental health treatment by including child, family, service system, and community factors. By far, the most influential predictor of adequate levels of family involvement in mental health treatment for children with SED was the quality of the service system. The second most influential predictor was caregiver health. Children with caregivers in poor health had higher levels of family involvement in acceptable service systems than children with caregivers in excellent health in unacceptable service systems. Thus, service system quality is an important predictor of acceptable family involvement in treatment.

The finding that caregiver's health predicted family involvement was based on existing theory and previous research findings. The hypothesis that white children would have higher levels of family involvement was supported as well in initial analyses, but when adult health was controlled race effects were no longer influential. Thus, white children had higher levels of family involvement when they had healthier caregivers. Race appears as a variable in many studies for the numerous social inequities that it represents, and this study has demonstrated that one way race may be affecting children's mental health is through the higher probability of better health among white caregivers as compared to non-white caregivers, reflecting well known health disparities (LaViest, 2002).

Several other hypotheses based on previous research were not supported in this study. Reasons for these inconsistencies may have to do with limitations of the sample. For instance, it is highly probable that increased severity on the CBCL scores of Internalizing and Externalizing symptoms would influence levels of family involvement in a sample with a range of scores; however, the sample in this study all had extremely severe scores. This lack of variability may have led to the finding of insignificant effects on family involvement. Similar limitations may explain the absence of education or income effects. Baker and colleagues (1993) found a strong relationship between socioeconomic status and family involvement, but the same findings were not replicated here. The current study sample, where the target child in the families was enrolled in Medicaid and the average monthly income was $1,650, is not representative of higher socioeconomic families. Thus, these findings may represent a restriction of range limitation, and SES effects may have been found with a wider range of incomes.

Similarly, caregiver strain did not demonstrate the hypothesized relationship with family involvement. Further exploration revealed that increased caregiver strain was significantly related to more severe child symptomatology. Caregivers who reported higher levels of strain were significantly more likely to have children with more severe Internalizing scores (β = .034, p = .007) and more severe Externalizing scores (β = .053, p < .001) on the CBCL. The lack of association between caregiver strain and family involvement may be connected to the severity of symptoms in this sample. Caregiver strain may still be a factor that influences family involvement, and the noted lack of family involvement in treatment for severely emotionally disturbed children could be a sign of high levels of caregiver strain associated with the high levels of externalizing and internalizing behaviors of SED children.

Other limitations of the study, in addition to a lack of variability on some variables, include methodological and measurement concerns. Primarily, it is important to note that the children sampled were not receiving services from a self-identified “system of care”, but rather a typical community service system. Services received through the existing state Medicaid systems were rated according to SOC principles, which indicate best practice, in order to examine quality of care. Second, the study staff who rated the level of family involvement also rated the service system factors, and rated all these factors at the six-month follow-up. Therefore, it is possible that the study staff tended to rate family involvement and other service system factors similarly as a result of the proximity of the ratings in time. The study staff were all trained and supervised, and hopefully this possible contamination of data was acknowledged and avoided, but there is no quantified inter-rater reliability measure on this sample to check for bias. Third, caregiver health was self-rated, which is subject to bias and error. We do not have an objective measure of caregiver's medical history or medical service use, but rather a measure of the caregiver's perceived health.

However, these findings remain important for theory on family involvement in mental health services. This study offers strong preliminary evidence of the importance of the quality of the service system in involving families in treatment of children with SED. When children were served in ways that demonstrated higher access to an array of services, higher levels of service coordination, more individualized service delivery, and with a long-term perspective on treatment needs and goals, their families were more likely to be involved in treatment. This finding corresponds with families acting as rational consumers of behavioral health services, who seek to maximize their payoff (in the form of competent providers and effective treatment results) for the investment of their valuable time and energy. This finding also could be interpreted as families acting as emotional, stressed human beings whose goal is to be heard and feel cared for by service providers and service systems. Regardless of the interpretation, this finding supports the recent inclusion of family perspectives and beliefs about the service system in studies of prolonged family engagement in children's treatment (as emphasized by Hoagwood, 2005) and the conclusion that family perceptions may be an influential barrier to treatment adherence (MacNaughton & Rodriguez, 2001). If families who have negative beliefs about the efficacy of the system, and families with poorly operating service systems, all have decreased participation in treatment, we are lead to the conclusion that families may be accurate in their perceptions and beliefs about the mental health system. The problem has been that service providers and researchers have been guilty of assuming that all services are good and helpful, and blaming the family when they were not involved in the treatment process. On the contrary, in this study trained study staff rated only 60% of the sampled children's service delivery systems as acceptable. Nonetheless, 71% of the children had families who were adequately involved in treatment. Future research on family involvement should include ratings of the quality of the services being provided to the child and family, instead of only child and family demographic and diagnostic variables.

The most important next step is to demonstrate that family involvement and children's outcomes can be improved by increasing the quality of service systems. Family involvement may play a mediating role in the relationship between children's outcomes and quality of the service system. If service system quality predicts family involvement (as shown here), and service system quality also improves child outcomes, does family involvement mediate the effects of service system quality on outcomes? Is it possible that children in a high quality service system will have better outcomes based on family involvement? Or are both family involvement and child outcomes related to system quality independently? Future work should examine the role that family involvement plays in the relationship between quality service systems and child outcomes to understand the best ways to improve treatment.

Footnotes

1

To meet criteria for SED at baseline, the children had to meet the two-part federal definition (SAMSHA, 1993) by having both clinical scores on the Child Behavior Checklist and clinical level of impairment on a measure of psychosocial functioning (see Combs-Orme, Heflinger, & Simpkins, 2002, for details). There were no statistically significant differences between TN and MS children's scores on the Internalizing or Externalizing scores.

2

Including foster parents in the second group, “other legal guardians”, did not significantly alter any findings.

References

  1. Achenbach TM. Manual for Child Behavior Checklist Ages 4-18 and 1991 Profile. University of Vermont, Department of Psychiatry; Burlington, VT: 1991. [Google Scholar]
  2. Baker BL, Blacher J, Pfeiffer S. Family involvement in residential treatment of children with psychiatric disorder and mental retardation. Hospital and Community Psychiatry. 1993;44:561–566. doi: 10.1176/ps.44.6.561. [DOI] [PubMed] [Google Scholar]
  3. Bazelon Center for Mental Health Law . Making child welfare work: How the R.C. lawsuit forged new partnerships to protect children and sustain families. Washington, D.C.: 1998. [Google Scholar]
  4. Bickman L, Foster EM, Lambert W. Who gets hospitalized in a continuum of care? Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35:74–80. doi: 10.1097/00004583-199601000-00015. [DOI] [PubMed] [Google Scholar]
  5. Brannan AM. Ensuring effective mental health treatment in real-world settings and the critical role of families. Journal of Child and Family Studies. 2003;12:1–10. [Google Scholar]
  6. Brannan AM, Heflinger CA. Child behavioral health service use and caregiver strain: Comparison of managed care and fee-for-service Medicaid systems. Mental Health Services Research. 2005;7:197–211. doi: 10.1007/s11020-005-7452-z. [DOI] [PubMed] [Google Scholar]
  7. Brannan AM, Heflinger CA. Caregiver, child, family, and service system contributors to caregiver strain in two child mental health service systems. Journal of Behavioral Health Services and Research. 2006;33(4):408–422. doi: 10.1007/s11414-006-9035-1. [DOI] [PubMed] [Google Scholar]
  8. Brannan AM, Heflinger CA, Bickman L. The Caregiver Strain Questionnaire: Measuring the impact on the family of living with a child with serious emotional disturbance. Journal of Emotional and Behavioral Disorders. 1997;5:212–222. [Google Scholar]
  9. Brannan AM, Heflinger CA, Foster EM. The role of caregiver strain and other family variables in determining children's use of mental health services. Journal of Emotional and Behavioral Disorders. 2003;11:78–92. [Google Scholar]
  10. Brennan EM, Brannan AM. Participation in the paid labor force by caregivers of children with emotional and behavioral disorders. Journal of Emotional and Behavioral Disorders. 2005;13:237–246. [Google Scholar]
  11. Bronfenbrenner U. Toward an experimental ecology of human development. American Psychologist. 1977;32:513–530. [Google Scholar]
  12. Bronfenbrenner U. The ecology of human development: Experiments by nature and design. Harvard University Press; Cambridge, MA: 1996. [Google Scholar]
  13. Centers for Disease Control and Prevention [March 2010];National Health Interview Survey: Celebrating the first 50 years 1957-2007. 2007 from http://www.cdc.gov/nchs/nhis/about_nhis.htm.
  14. Clarke G, Hops H, Lewinsohn P, Andrews J, Steeley J, Williams J. Cognitive-behavioral group treatment of adolescent depression: Prediction of outcome. Behavioral Therapy. 1992;23:342–354. [Google Scholar]
  15. Combs-Orme T, Heflinger CA, Simpkins CG. Comorbidity of mental health problems and chronic health conditions in children. Journal of Emotional and Behavioral Disorders. 2002;10:116–125. [Google Scholar]
  16. Cook JA, Heflinger CA, Hoven CW, Kelleher KJ, Mulkern V, Paulson RI, Stein-Seroussi A, Fitzgibbon G, Burke J, Williams M, Kim JB. A multi-site study of Medicaid-funded managed care versus fee-for-service plans' effects on mental health service utilization of children with severe emotional disturbance. The Journal of Behavioral Health Services & Research. 2004;31:384–402. doi: 10.1007/BF02287691. [DOI] [PubMed] [Google Scholar]
  17. Cook JR, Kilmer RP. Evaluating systems of care: Missing links in children's mental health research. Journal of Community Psychology. 2004;32:655–674. [Google Scholar]
  18. Corcoran J, Dattalo P. Parent involvement in treatment for ADHD: A meta-analysis of the published studies. Research on Social Work Practice. 2006;16:561–570. [Google Scholar]
  19. Corcoran J, Pillai V. A meta-analysis of parent-involved treatment for child sexual abuse. Research on Social Work Practice. 2006;18:453–464. [Google Scholar]
  20. Curtis WJ, Singh NN. Family involvement and empowerment in mental health service provision for children with emotional and behavioral disorders. Journal of Child and Family Studies. 1996;5:503–517. [Google Scholar]
  21. DeChillo N. Collaboration between social workers and families of patients with mental illness. Families in Society. 1993;74:104–115. [Google Scholar]
  22. DeChillo N, Koren PE, Schultze KH. From paternalism to partnership: Family and professional collaboration in children's mental health. American Journal of Orthopsychiatry. 1994;64:564–576. doi: 10.1037/h0079572. [DOI] [PubMed] [Google Scholar]
  23. Erhardt D, Baker BL. The effects of behavioral parent training on families with young hyperactive children. Journal of Behavioral Therapy and Experimental Psychiatry. 1990;21:121–132. doi: 10.1016/0005-7916(90)90017-f. [DOI] [PubMed] [Google Scholar]
  24. Foster EM. Does the continuum of care improve the timing of follow-up services? Journal of the American Academy of Child and Adolescent Psychiatry. 1998;37:805–814. doi: 10.1097/00004583-199808000-00010. [DOI] [PubMed] [Google Scholar]
  25. Foster EM, Saunders RC, Summerfelt WT. Predicting level of care in mental health services under a continuum of care. Evaluation and Program Planning. 1996;19:143–153. [Google Scholar]
  26. Grunebaum H. Comments on Terkelson's “Schizophrenia and the family: II. Adverse effects of family therapy. Family Process. 1984;23:421–428. doi: 10.1111/j.1545-5300.1984.00421.x. [DOI] [PubMed] [Google Scholar]
  27. Hartley D. Rural health disparities, population health, and rural culture. American Journal of Public Health. 2004;94:1675–1678. doi: 10.2105/ajph.94.10.1675. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Hartley D, Bird DC, Dempsey P. Rural mental health and substance abuse. In: Ricketts TC, editor. Rural health in the United States. Oxford University Press; New York, NY: 1999. pp. 159–178. [Google Scholar]
  29. Hatfield AB. Commentary: Therapists and families: Worlds apart. Hospital Community Psychiatry. 1982;33:513. doi: 10.1176/ps.33.7.513. [DOI] [PubMed] [Google Scholar]
  30. Hawley KM, Weisz JR. Youth versus parent working alliance in usual clinical care: Distinctive associations with retention, satisfaction, and treatment outcome. Journal of Clinical Child and Adolescent Psychology. 2005;34:117–128. doi: 10.1207/s15374424jccp3401_11. [DOI] [PubMed] [Google Scholar]
  31. Hernandez M, Gomez A, Lipien L, Greenbaum PE, Kathleen HA, Gonzalez P. Use of the system-of-care Practice Review in the National Evaluation: Evaluation the fidelity of practice to system-of-care principles. Journal of Emotional and Behavioral Disorders. 2001;9:43–52. [Google Scholar]
  32. Hoagwood KE. Family-based services in children's mental health: A research review and synthesis. Journal of Child Psychology and Psychiatry. 2005;46:690–713. doi: 10.1111/j.1469-7610.2005.01451.x. [DOI] [PubMed] [Google Scholar]
  33. Human Systems and Outcomes, Inc . Case review protocol: For examination of the impact of managed care on children with serious emotional disorders and adolescents with substance abuse problems (Version 1.4) Author; Tallahassee, FL: 1998. [Google Scholar]
  34. Israel P, Thomsen PH, Langeveld JH, Stormark KM. Parent-youth discrepancy in the assessment and treatment of youth in usual clinical care settings: Consequences to parent involvement. European Child & Adolescent Psychiatry. 2007;16:138–148. doi: 10.1007/s00787-006-0583-y. [DOI] [PubMed] [Google Scholar]
  35. Kazdin AE. Perceived barriers to treatment participation and treatment acceptability among anti-social children and their families. Journal of Child and Family Studies. 2000;9:157–174. [Google Scholar]
  36. Kazdin AE. Psychotherapy for children and adolescents. Annual Review of Psychology. 2003;54:253–276. doi: 10.1146/annurev.psych.54.101601.145105. [DOI] [PubMed] [Google Scholar]
  37. Kazdin AE. Evidence based treatment: Challenges and priorities for practice and research. Child and Adolescent Psychiatric Clinics of North America. 2004;13:923–940. doi: 10.1016/j.chc.2004.04.002. [DOI] [PubMed] [Google Scholar]
  38. Kazdin AE, Holland L, Crowley M. Family experiences of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology. 1997;65:453–463. doi: 10.1037//0022-006x.65.3.453. [DOI] [PubMed] [Google Scholar]
  39. Kazdin AE, Wassell G. Predictors of barriers to treatment and therapeutic change in outpatient therapy for antisocial children and their families. Mental Health Services Research. 2000;2:27–34. doi: 10.1023/a:1010191807861. [DOI] [PubMed] [Google Scholar]
  40. Kazdin AE, Wassell G. Barriers to treatment participation and therapeutic change among children referred for conduct disorder. Journal of Clinical Child and Psychology. 1999;23:160–172. doi: 10.1207/s15374424jccp2802_4. [DOI] [PubMed] [Google Scholar]
  41. Koroloff NM, Friesen BJ. Family-centered services: Implications for mental health administration and research. Journal of Health Administration. 1990;17:13–25. doi: 10.1007/BF02518576. [DOI] [PubMed] [Google Scholar]
  42. LaViest TA, editor. Race, ethnicity, and health: A public health reader. Jossey-Bass; San Francisco: 2002. [Google Scholar]
  43. MacNaughton KL, Rogriquez JR. Predicting adherence to recommendations by parents of clinic-referred children. Journal of Consulting and Clinical Psychology. 2001;69:262–270. doi: 10.1037//0022-006x.69.2.262. [DOI] [PubMed] [Google Scholar]
  44. McKay MM, Brannon WM. Engaging families in child mental health services. Child and Adolescent Psychiatric Clinics of North America. 2004;13:905–921. doi: 10.1016/j.chc.2004.04.001. [DOI] [PubMed] [Google Scholar]
  45. McKay MM, Pennington J, Lynn CJ, McCadam M. Understanding urban child mental health service use: Two studies of child, family, and environmental correlates. The Journal of Behavioral Health Services and Research. 2001;28:475–483. doi: 10.1007/BF02287777. [DOI] [PubMed] [Google Scholar]
  46. Morrissey-Kane E, Prinz RJ. Engagement in child and adolescent treatment; The role of parental cognitions and attributions. Clinical Child and Family Psychology Review. 1999;2:183–198. doi: 10.1023/a:1021807106455. [DOI] [PubMed] [Google Scholar]
  47. Measelle JR, Weinsten RS, Martinez M. Parent satisfaction with case managed systems of care for children and youth with severe emotional disturbance. Journal of Child and Family Studies. 1998;7:451–467. [Google Scholar]
  48. Nock MK, Ferriter N. Parent management of attendance and adherence in child and adolescent therapy: A conceptual and empirical review. Clinical Child and Family Psychology Review. 2005;8:149–166. doi: 10.1007/s10567-005-4753-0. [DOI] [PubMed] [Google Scholar]
  49. Osher TW, Osher DM. The paradigm shift to true collaboration with families. Journal of Child and Family Studies. 2002;11:47–60. [Google Scholar]
  50. Osher TW, deFur E, Nava C, Spencer S, Toth-Dennis D. Systems of care: Promising practices in children's mental health, 1998 Series. Center for Effective Collaboration & Practice, American Institutes for Research; Washington, DC: 1998. New roles for families in systems of care. pp. 68–70. [Google Scholar]
  51. Pekarik G, Stephenson LA. Adult and child client differences in therapy dropout research. Journal of Clinical Child Psychology. 1988;17:316–321. [Google Scholar]
  52. Pullmann MD. Participatory research in systems of care for children's mental health. American Journal of Community Psychology. 2009;44:43–53. doi: 10.1007/s10464-009-9245-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Richards MM, Bowers MJ, Lazicki T, Krall D, Jacobs AK. Caregiver involvement in the intensive mental health program: Influence on changes in child functioning. Journal of Child & Family Studies. 2008;17:241–252. [Google Scholar]
  54. Robinson AD, Kruzich JM, Friesen BJ, Jivanjee P, Pullmann MD. Preserving family bonds: Examining parent perspectives in the light of practice standards for out-of-home treatment. Journal of Orthopsychiatry. 2005;75:632–643. doi: 10.1037/0002-9432.75.4.632. [DOI] [PubMed] [Google Scholar]
  55. Salomon JA, Nordhagen S, Oza S, Murray CJ. Are Americans feeling less healthy? The puzzle of trends in self-rated health. American Journal of Epidemiology. 2009;170:343–351. doi: 10.1093/aje/kwp144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Singh NN, Curtis WJ, Cohen R, Nicholson MW, Villani TM, Wechsler HA. Measuring perceptions of family involvement. I: The Family Involvement Scale—Family Version. Commonwealth Institute for Child and Family Studies; Richmond, VA: 1996a. [Google Scholar]
  57. Singh NN, Curtis WJ, Cohen R, Nicholson MW, Villani TM, Wechsler HA. Measuring perceptions of family involvement. II: The Family Involvement Scale—Professional Version. Commonwealth Institute for Child and Family Studies; Richmond, VA: 1996b. [Google Scholar]
  58. Spaniol L, Jung H, Zipple A, Fitzgerald S. Families as a resource in the rehabilitation of the severely psychiatrically disabled. In: Hatfield AB, Lefley HP, editors. Families of the mentally ill: Coping and adaptation. Guilford Press; New York: 1987. pp. 167–190. [Google Scholar]
  59. Stephens RL, Holden EW, Hernandez M. System-of-care Practice Review scores as predictors of behavioral symptomatology and functional impairment. Journal of Child and Family Studies. 2004;13:179–191. [Google Scholar]
  60. Stroul BA, Friedman RM. A system of care for severely emotionally disturbed children and youth. Georgetown University Child Development Center, CASSP Technical Assistance Center; Washington, DC: 1986. [Google Scholar]
  61. Tennessee Commission on Children and Youth . Children's program outcome review team 2004 evaluation results. Author; Nashville, TN: 2004. [Google Scholar]
  62. Yatchmenoff MS, Koren PE, Friesen BJ, Gordon LJ, Kinney RF. Enrichment and stress in families caring for a child with a serious emotional disorder. Journal of Child and Family Studies. 1998;7:129–145. [Google Scholar]
  63. Washington State Department of Health . Guidelines for using rural-urban classification systems for public health assessment. Washington State Department of Health; Tumwater, WA: 2006. [Google Scholar]
  64. WWAMI Rural Health Research Center . Rural Urban Commuting Codes (Version 2.0) University of Washington; Seattle, WA: 2006. [March 2010]. from http://depts.washington.edu/uwruca/ruca-news.php. [Google Scholar]

RESOURCES