CONGRUENT PERCEPTIONS IN THE EVALUATION OF COMMUNITY-CARE FACILITIES

Abstract

The purpose of this research was to determine whether differential perceptions of the environment of a sheltered-care facility by residents and operators of that facility would be of use in predicting a primary outcome variable — a resident’s social integration. Differences between resident and operator assessments in a variety of areas were examined as were the conditions contributing to agreement or disagreement, such as resident satisfaction or the extent of a family-like atmosphere at the facility. The implications for using perceptual congruence as an evaluation strategy were discussed.

For those faced with the prospect of evaluating community-oriented treatment environments, the task is not a simple one. Not only must the problem of assessing outcome be addressed, but an equally important and difficult task involves identifying the dimensions of an environment that affect outcome. If the creation of an effective program is to be a rational process, then the means by which positive outcomes are produced (or inhibited) must be understood.

With respect to the problem of identifying environmental characteristics, a major contribution has been made by Moos and his colleagues (Moos, 1974a, 1974b, 1975; Moos & Otto, 1972) in construction and dissemination of the COPES scales. These scales permit an objective and replicatable description of treatment situations. The COPES focuses description on relevant program characteristics, such as treatment orientation or social climate, rather than on the more traditional ones, such as facility size or staff composition (Moos, 1972).

Given a vehicle for description in program-relevant terms, it would seem intuitively compelling that agreement between staff and residents of treatment facilities as to the specific characteristics of their shared environment might be a significant indicator of the effectiveness of the treatment program. If staff and residents held differing views as to such basics as program goals, social climate, or behavior of staff and residents, could the program be regarded as successful? But, on the other hand, does disagreement on a descriptive level actually reflect disagreement (and perhaps conflict) on the everyday behavioral level, and, if so, does this disagreement in turn negatively affect outcome?

Moos in his consideration of residential treatment environments suggests that agreement between clients and staff as to the nature of the program may in fact be important for effective program operation and thus has significant evaluative implications (Moos, 1972). Moos has based much of his analyses on a clinical or rational evaluation of differences between patterns of client/staff agreement (Moos, 1974a, 1974b, 1975). It should be pointed out, however, that little is known empirically concerning the correlates of agreement in these settings. Indirect evidence from research on psychotherapy had indicated that when clients and therapists share similar value systems more positive therapeutic results are obtained (Cook, 1966; Burdock, Cheek, & Zubin, 1960; Welkowitz, Cohen, & Ortmeyer, 1967). Along similar lines, Rogers, Gendlin, Kiesler, and Traux (1967) reported that the greater the similarity between assessments of their therapeutic relationship made by clients and therapists, the more successful the results of the therapy.

That more should be known concerning the correlates of agreement is apparent, given the sparseness of the research literature and the fact that evaluators of community programs are often faced with the “Rashomon-like” situation where grossly different perceptions of the same environment are held. What accounts for these differences; are there contextual factors indigenous to a program that promote differing perceptions; and perhaps most important, how do differential perceptions affect the fulfillment of program goals?

It is the purpose of this research to investigate the relationship between perceptual congruence as evidenced by agreement or disagreement on COPES assessments, and a major outcome variable, the residents’ social integration. Staff and resident describe the same shared environment and differences are related to the social integration of the resident. In addition, to determine how perceptual congruence is related to other program characteristics (e.g., facility treatment orientation and consumer response) the correlates of congruence are also investigated.

Method

Participants.

A probability sample of all nonretarded, mentally ill residents, aged 18–65, who were living in California sheltered-care facilities was surveyed as part of a larger study of sheltered care (Segal & Aviram, 1978). A sample of operators of the same facilities was also surveyed. The total sample consisted of 499 residents and 234 operators. Complete details of the probability base for the sampling, the interview schedule and associated methodology, along with other data descriptive of the sample (age, sex, education, psychopathology, etc.,) can be found in Segal and Aviram (1978).

Measures.

All members of both samples responded to an extensive interview schedule that included the Community Oriented Programs Environment Scale (COPES) (Moos, 1974a, 1974b). Utilizing the COPES items as one way of describing a facility, the congruence between descriptions of the same facility as given by residents and by operators forms the basis of this research.

The COPES scales were chosen for this study because of their broad descriptive relevance to the environment of sheltered care and because of their increasing use as a descriptor and possible evaluative index of treatment and rehabilitation settings.

The COPES consists of 102 forced-choice items appropriate to the description of community-based programs (Moos, 1974a). Responses to individual items can be reduced to scores on ten relatively independent subscales. These in turn can be further reduced to three scales descriptive of facility function in three general areas. The first area includes items that describe the Personal Relationships within a facility. These items relate to a resident’s involvement in the program, the extent to which a supportive environment is maintained, and the degree to which spontaneity and the expression of feeling is encouraged. The second area, Treatment Program, includes items relating to the extent to which autonomy is encouraged and the degree to which the resident is oriented to leaving the program. It also includes items that refer to the extent to which personal problems are focused on and the extent to which residents are encouraged to express anger and aggression. The third area, System Maintenance, refers to the extent to which the facility stresses order and organization, the extent to which program clarity is a goal, and the degree to which the staff uses methods to keep residents “under control.” Further rationale for the construction and use of the COPES scales can be found in Moos’s Evaluating Treatment Environments (Moos, 1974b). COPES scores utilized in this research were simple-sum scores for each of the three major COPES areas. Scores in the staff control subscale were not included in the System Maintenance scores. In order to compare the perception of residents and operators, absolute difference scores were calculated in each of the three areas.

Social Integration.

A major criterion variable in the work of Segal and Aviram has been the degree of social integration achieved by the sheltered-care resident. They have maintained that the level of functioning in community life provides an identifiable, objective indicator of the social well-being of the ex-mental patient and thus can provide 2 valuable criterion measures for program evaluation.

In conceptualizing this variable, two types of social integration were identified: External and Internal. For each type, five levels of social involvement are assessed relative to social actions or objects within the resident’s surrounding environment. External Integration refers to the degree of involvement outside of the facility and levels are defined in terms of time spent (presence), availability of goods and services and potential community contacts (access), frequency of engagement in family and community social activities (participation), contribution to the community through work or study (production), and use of the community’s goods and services (consumption). Internal Integration, on the other hand, refers to a similar assessment of involvement, but relative to social acts within or mediated by the sheltered-care facility (Segal & Aviram, 1978).

Procedure.

To assess the relationship between the congruence of assessments and social integration, resident/operator difference scores in each of the three COPES areas — Personal Relationships, Treatment Program, and System Maintenance — were correlated with the residents’ External and Internal Integration scores. Also, to assess the possible effects of the direction of the congruence scores, the sample was divided into two groups for each COPES area. One group contained residents who scored equal to or higher than the operator of their facility, and the other group who scored lower than the operator. Correlations were obtained with External and Internal Integration for each group. In order to investigate the possible value of using COPES scores alone, residents’ and operators’ area scores were correlated separately with residents’ External and Internal Integration scores.

Predictors of congruence.

Irrespective of the value of agreement or disagreement as a possible predictor of social integration, it would remain of interest to investigate, within the same context, environmental conditions under which differences arise. Thus, the predictors of congruence were also investigated.

In the larger study, Segal and Aviram (1978) examined a large number of variables as potential predictors of a variety of outcome behaviors including social integration. In the present study, a subset of these variables was chosen on the basis of their previous performance as predictors and to the extent that they were representative of major groupings of variables; that is, variables that refer to characteristics of residents, those that refer to characteristics of facilities, and those descriptive of the surrounding community. Their value in predicting difference scores was explored using regression techniques.

Results

The congruence (mean difference) scores for the three COPES areas are 7.2 for Personal Relationships (range = 32), 8.4 for Treatment Programs (range = 70), and 2.5 for System Maintenance (range = 20). Congruence scores were relatively independent across the three areas. Although the correlations between congruence scores in the area of Personal Relationships and scores in the Treatment Program (r = .17) and System Maintenance (r = .13) areas were significant (p < .05), only a small proportion of variance is accounted for.

Social Integration

Client/Staff Agreement and Outcome.

Congruence scores in each of the COPES facility function areas were correlated with both External and Internal Integration. Table 1 gives the correlations within each area.

TABLE 1.

Correlation of Resident/Operator COPES Difference Scores with Resident’s Social Integration

Correlations of Directional Difference Scores by COPES Area
Social	Personal	Treatment	System
Integration	Relationships	Program	Maintenance
External	−.08	−.04	.06
Internal	−.17*	−.11*	−.10

Correlations of Directional Difference Scores by COPES Area
Social Integration	Personal Relationships		Treatment Program		System Maintenance
	Res > Op	Res < Op	Res > Op	Res < Op	Res > Op	Res < Op
External	−.05	−.06	.12*	−.06	.01	.10
Internal	−.04	−.15*	.001	−.08	.05	−.20*

^*Significant at p < .05.

Only the correlations between congruence scores calculated as absolute difference scores for Personal Relationships and Internal Integration (r = −.17) and for Treatment Program scores and Internal Integration (r = −.11) were significant at the .05 level. In this case, a small negative correlation indicated that congruence on these two COPES scores (i.e., low difference scores) was weakly related to greater social integration within the sheltered-care facilities.

When the direction of the differences (Directional Difference Scores) was taken into account (see Table 1), it had the effect of only slightly changing the magnitude and extent of associations. The association between congruence and Internal Integration held up only when the resident had a score which was less positive (i.e., lower) than the operator’s and then only for Personal Relationships. For System Maintenance, a significant correlation appeared with Internal Integration (r = −20) when the residents’ scores were lower than the operator’s. The relationship between congruence of Treatment Program assessments and Internal Integration was no longer significant, but a weak reversal occurred with noncongruence correlating (r = .12, p < .05) with External Integration when the resident’s score was equal to or greater than the operator’s.

Outcome and independent client and staff assessments.

Table 2 lists the significant correlations with social integration when resident and operator COPES scores were taken separately and not as difference scores. All three COPES resident scores were significantly related (p < .05) to Internal Integration. Two resident COPES scores were significantly related to External Integration. For operators the only significant association was between their Personal Relationships scores and resident’s Internal Integration.

TABLE 2.

Correlation of Residents’ and Operators’ COPES Area Scores with Resident’s Social Integration

Social integration	Personal Relationships		Treatment Program		S)stem Maintenance
	Resident	Operator	Resident	Operator	Resident	Operator
External	.20*	.06	.34*	.11	.06	.01
internal	.44*	.18*	.24*	.01	.21*	.06

^*Significant at p < .05.

Outcome as a function of resident COPES scores, client/staff and the interaction of these predictors.

Because of the strong association between resident COPES scores and social integration, a regression analysis was performed on social integration to look at the simultaneous contribution of difference scores, residents’ COPES assessments, and the interaction of these two variables. The interaction between resident’s COPES assessments and the agreement/disagreement scores of residents and operators provided an index of how much the “objective” assessment of the resident is influenced by a context of agreement or disagreement. It is often observed that agreement between instructor and student assessment of a course, especially with respect to high grades, leads the student to assess the course more favorably.

In all cases our results indicated that the residents’ score was significant and the strongest predictor when considered with difference scores alone (see Table 3). However, when the interaction of these two variables was put into the model it replaced residents’ scores in the system maintenance area as a significant predictor of internal integration. Agreement as to the poor quality of the facility (i.e., the low end of the COPES scale) interacting with a poor resident assessment of the facility on system maintenance criteria led to a reduction in internal integration. Disagreement as to the quality of the facility interacting with a very positive assessment by residents (perhaps an overgenerous assessment) on system maintenance criteria led to increased resident internal integration. The latter was perhaps dependency based (see Table 3). The interaction between congruence scores and resident assessments was also significant in the relationship area. High levels of disagreement interacting with high resident ratings of the facility on relationship criteria led to decreased external integration. This may be another reflection of the residents’ dependency on the facility environment. Agreement between operators and residents and poor resident assessments led to increased external integration. This finding seems to result from an avoidance of a bad situation.

TABLE 3.

The Relative Contribution of Resident COPES Scores and Resident/Operator Difference Scores to the Prediction of Social Integration

	Social Integration
	External			Internal
COPES Scores	r	βa	R2	r	β	R2
Relationship			5%			21%
1. Difference Score	−.08	.23*		−.17	.15*
2. Resident’s Assessment	.20	.36**		.44	.53**
3. Interaction of 1 × 2	.05	−.13*		.17	.02
Treatment			12%			6%
1. Difference Score	−.04	.004		−.11	−.21*
2. Resident’s Assessment	.34	.34**		.24	.10
3. Interaction of 1 × 2	.20	.01		.09	.18
System Maintenance			1%			6%
1. Difference Score	.06	.12		−.10	−.37**
2. Resident’s Assessment	−.07	.002		.21	−.01
3. Interaction of 1 × 2	.01	−.08		.08	.36**

^*Significant at p < .05.

^**Significant at p < .01.

^aStandardized partial regression coefficients.

Predictors of congruence.

Table 4 outlines the relative importance of predictors of agreement for the three COPES facility function areas as reflected by their partial beta weights. Given a large number of predictors, only a small proportion of the behavioral differences related to agreement was explainable. The regression models for each of the three areas explain approximately a fifth of the differences in behavior related to agreement between residents and operators. For Personal Relationships, nine predictors were significant at least at the .05 level and accounted for 22% of the variance in difference scores. Nine predictors were significant for Treatment Program and accounted for 16% of the variance. For System Maintenance, seven predictors were significant and accounted for 18% of the variance. Only four of the predictors were significant for more than one scale; none were common to all three.

TABLE 4.

Predictors of Agreement between Resident/Operator COPES Assessments in Three Areas^a

		Relationship (R)
Predictor	Standardized Partial β	Predictor
R-l	.22	Good Consumer Response.
R-2	.17	Isolation of Resident Group.
R-3	.17	Fewer Residents in Facility Group with a History of Mental Illness.
R-4	.13	Operator Seeks Help for Resident Problems Less Frequently or Readily.
R-5	.10	Operator does not Eat with Residents.
R-6	.10	Resident would Like to Stay Indefinitely.
R-7	.10	Purpose of Facility not seen as Treatment by Operator.
R-8	.08	Resident is Satisfied with His or Her Life.
R-9	.06	Resident Chose House Because He or She Liked the Living Arrangements, Neighborhood, Operator, or Other Residents.
		Treatment (T)
T-l	.16	Services Seen as more Harmful to Resident.
T-2	.13	Operator Does Not Belong to a State Operator’s Association.
T-3	.11	House Does Not have a Therapy or Rehabilitation Program.
T-4	.11	None of the Residents Keep their own Medication.
T-5	.09	Operator Describes Facility as a Permanent Home for Residents.
T-6	.08	Resident would Like to Stay Indefinitely.
T-l	.08	Population of Facility is Older and Stable.
T-8	.07	Purpose of the Facility Described by the Operator is not Treatment.
T-9	.06	Atmosphere of the Facility is a Family or Home.
		System Maintenance (SM)
SM-1	.18	Good Consumer Response.
SM-2	.17	Resident Group has more Residents who have been in a State Hospital for more than Five Years.
SM-3	.16	Operator shows Higher Concern for Social Disability.
SM-4	.12	Resident and Operator are Fairly Close (within 10 points) on Reiss’s Socioeconomic Index (SEI).
SM-5	.11	Operator has no Problems Recruiting new Residents.
SM-6	.10	Operator has Formal Medication Procedure for all Residents.
SM-7	.07	Operator is the Facility Owner.

^aPredictors are all significant at p < .05. They are also ordered, within COPES areas, according to their relative importance as expressed by their standardized partial beta weights (β).

Although few predictors of agreement were found that were common to more than one COPES area, the majority fell into one or another of two broad content categories (see Table 4). One category appears to reflect the resident’s satisfaction with his present situation, while a second category can be characterized as descriptive of a stable and self-contained facility. This two-way categorization of predictors was most relevant to the Personal Relationship and Treatment Program predictors. The System Maintenance predictors were less easily accommodated within this two-dimensional framework.

Predictors of agreement that fell into the first category of resident satisfaction include: positive consumer response (R-1, S-1), the resident wishes to stay on indefinitely at his/her facility (R-6, T-6), the resident is satisfied with his/her life (R-8), and the resident has chosen the facility because of its positive attributes (R-9). To wish to remain in a living situation indefinitely was a predictor of agreement in both the Relationship and Treatment areas. It not only indicated residents’ satisfaction with their current situation, but also related to their desire for stability.

Predictors of agreement falling into the second category of the stable and self-contained facility included: the non-treatment-oriented facility (R-7, T-8), an isolated resident group (R-2), the failure of the operator to seek help outside the facility to cope with resident problems (R-4), outside services are seen by the operator as more harmful than helpful (T-1), the facility does not have a therapy or rehabilitation program (T-3), the facility function is perceived by the operator to be a permanent home (T-5), the atmosphere of the facility is family or home-like (T-9), the resident population is older and more stable (T-7), the resident group includes residents who have been in state mental hospitals more than five years, automatically making them an older, more mature group, (SM-2), the operator and the residents share similar socioeconomic status (SM-4), the operator has no problem recruiting residents (SM-5), and the operator is the owner of the facility (SM-7).

For this second category, the predictor appearing within both Personal Relationship and Treatment Program areas was one that described the facility as a non-treatment-oriented facility (R-7, T-8). With respect to this finding, it is important to note that most individuals leaving mental hospitals and going into community care facilities are seeking a long-term, stable living environment and a treatment atmosphere is often one imposed upon them rather than one they seek. It is perhaps the reaction to this imposition which accounts for the fact that agreement was related to non-treatment-oriented facilities. It has been noted by others that treatment-oriented environments tend to be change oriented and by necessity, disruptive and disagreement producing. Street, Vinter, and Perrow (1966) pointed out that, “In a true treatment situation there must be substantial tolerance for disruption of routine, ambiguities in criteria for staff and inmate performance, and conflict among staff and inmates” (p. 281) and, “By definition, the treatment institution seeks a high degree of change …” (p. 153). In view of this, it is likely that a nontreatment, more maintenance-oriented facility would value stability, attempt to maintain a family-like atmosphere and, within the organization, press for accommodation and agreement.

Conclusion

Although various types of “difference scores” have been used by a number of investigators, (cf. Moos’s review, 1974b), few studies have examined the significance of differential perceptions of participants in a shared setting. In the case of this study, one might have assumed that not sharing the point of view of those who operate the setting would be indicative of some level of conflict and this, in turn, would be reflected in a lower level of desired behavior — social integration. Conversely, it might be assumed that congruence would result in the opposite behavior — increased social integration. This was not the case.

Only a weak relationship was observed between perceptual congruence and social integration. In addition, when the relative contribution of difference scores and residents’ COPES assessments in each of the three COPES facility function areas was assessed, residents’ COPES assessments were clearly the superior predictors. It is clear that if one were to choose, it is the residents’ direct assessment of the environment that one should take as pertinent in predicting the extent of social integration a resident demonstrates in that environment.

Thus, as an evaluative tool, one can raise questions as to the value in this context of the concept of perceptual congruence as a means of assessing environments. We think that our data indicate that clinicians or evaluators of community-care facilities might well focus assessment on the facility resident alone if they are concerned with the goal of predicting social integration, particularly within the facility. While agreement between operators and residents regarding the assessment of their facility is related to the positive evaluation of the facility by the resident, it is perhaps too far removed from the actual social participation of the resident to be a good outcome indicator.

With respect to agreement per se, it appears to be obtained under conditions of a stable, satisfying environment that does not emphasize change. If one values agreement per se, then our findings might be construed to support Wolfensberger’s (1972) argument for separating the domicilary function of residential care from the change-oriented treatment function. It seems likely that the more one presses for change, the greater the potential for generating disagreement. We are inclined to agree with Wolfensberger that, in order to avoid the complications of the total institution (Goffman, 1961), there should be a separation of treatment from the domiciliary function. We believe, however, that there is also a need to explore the utility of disagreement and conflict in the residential treatment setting. Such explorations will better enable us to maximize the potential of treatment and to understand in a more detailed way the inconsistencies in current treatment ideologies. Perhaps the most glaring illustration of such an inconsistency applies to the therapeutic community. In the therapeutic community there is an emphasis on mutual participation and on obtaining congruence of perceptions between staff and clients. Yet there is also an emphasis placed on change. This latter emphasis would seem to increase the potential for staff/client disagreement and thus for conflict within this treatment setting, but perhaps it is within a system of conflict resolution that treatment becomes meaningful.

In conclusion, we would suggest that the empirical correlates of differences continue to be explored to search for other meanings of discrepancies. Such discrepancies may offer further clues as to the impact of the environment on the participants.