Quality of care in psychiatric emergency services

Introduction

Managed mental health care organisations primarily (MMCOs) make profits from savings on costly services like psychiatric hospitalisation (Winegar, 1992; Dorwart, 1990; Fishel, et al., 1993).

Clinical decisions about the continued need to provide expensive inpatient services should be better understood to address serious concerns about the possibility of deterioration (Lazarus, 1993). The need is greatest in the psychiatric emergency service (PES) of general hospitals. This is where most civil commitment evaluations are done and where significant numbers of inpatient stays are approved. Little empirical work either conceptualises or validates quality of care indicators in clinical decision making (Tischler, et al., 1974; McGlynn, et al., 1988; Wolford, 1988). The lack of validated quality of care indicators increases the risk that the pressures for cost cutting overwhelm the ability to provide good care. We offer a clinical decision-making model for the assessment of the quality of care in the PES.

There is no consensus on defining quality of care in PES. It means different things to different stakeholders in the treatment process. Following Rundall and Gardner (1991), we observed that defining quality of care depends on the frame of reference of three significant stakeholders; patients, providers and administrators. The art of care – engaging the patient in the assessment and related interventions – addresses the patient’s perspective. Conformity with professional standards is defined as technical quality, the provider’s perspective. Administrators, are likely to aim for efficiency in the delivery of quality care. The proposed model of clinical decision making relies on these three dimensions. Measures used to define the model were chosen on the basis of their empirical utility in predicting evaluation outcomes. The selected measures have shown utility in predicting variations in patient disposition and functional status (Segal et al., 1995), medication regimine (Segal, et al., 1996a), and use of less restrictive alternatives (Segal et al., 1996b). Davis (1991) notes that assessments of quality of care must involve measures of input or structure (e.g. physical characteristics, staff-patient ratios), process (the adequacy of services) and outcomes.

Methods

Sample

A total of 683 cases were assessed in nine California PES’s. Seven facilities were in the San Francisco Bay area, one in the Central Valley, and another in Los Angeles.

Evaluations were observed on all days of the week. Any incoming patient was sampled who had not yet been seen by a clinician and was available when a researcher and staff clinician were free.

Data was gathered by research clinicians using structured instruments for the independent assessment of each case. These were social workers and psychologists, experienced in assessing severely mentally ill adults, who accompanied a PES staff clinician and client through evaluation from the time of the client’s arrival until a decision about disposition was made. The clinician simply went about his or her business keeping the observer informed as to any information received by phone. The observer reviewed charts and other written material. The PES clinician was not aware of the instruments coded by the researcher.

Quality of Care Measures

The technical quality of care was assessed with The Quality of Care Index (Johnson, et al., 1985). This includes 27 items selected and weighted to measure the quality of care provided to PES patients. It reflects the medical-model professional standard (the providers perspective). It was developed through panels of psychiatrists or physicians experienced in psychiatric assessment identifying evaluation components translating to item format, and weighing according to relative importance for assessment using nominal group process techniques. This was designed to generate index criteria for evaluation from the judgement of experts in the field (Delbecq, et al., 1975). The process through which Johnson et al., selected items gives reasonable assurance that the index includes the relevant professional concepts and procedures (content validity). Each item was given likelihood ratio scores by two different panels whose results were cross validated. The correlation between the average quality ratings was .89. The scale score for each individual is the sum of the item scores relevant to the patient’s case divided by the number of relevant items.

The Art of Care scale consists of four items about the clinician’s engagement with the patient:

• engage in collaborative interaction

• elicit information

• include in planning at a level appropriate to his/her functioning

• attend to feelings and respond empathically.

The items are coded 1 if present and 0 if absent. The score, an average of the four scores, ranges from 0 to 1. Interrater agreement averaged .75 and the internal consistency Alpha = .69. The Art of Care scale addresses the patient perspective. It operationalises the concept introduced by Brook and Avery (1976, 1977), who suggests that

any definition of quality of care must include the dimension of the “art of care” which is defined as “the manner of physician care relative to the patient as an individual, as measured by its sensitivity, openness, and non-authoritarian nature” (1976, p3).

Efficiency, the optimum investment of time required to complete quality evaluation (the administrative perspective), was measured to match the complexity of the patients’ clinical needs and presentation with the amount of time allocated. All the items in the Technical Quality of Care Scale and the Art of Care Scale were recoded as dummy variables with values of 1 or 0, where one reflected an action that was negative or neutral vis á vis a quality practice. For example, the first scale item in the technical quality scale has three possible values:

1. “Evidence of attempt to contact significant others” reflects an action that enhances quality – positive score on the item;

2. “no contact needed”, is neutral;

3. “no evidence of attempt to contact”, which is negative.

We created two dummy variables to indicate “negative” and “neutral” actions. A person registering a neutral on the scale item receives a score of zero on the negative dummy variable and a score of one on the neutral dummy variable. A person registering a negative on the item receives one on the “negative” dummy variable and zero on the neutral dummy variable. A person registering a positive receives a zero on both the neutral and negative dummy variables.

All dummy variables created from the two scales were regressed on the total amount of time taken to complete the evaluation (T). The obtained model was significant (p<.000), as was its intercept, ßo(p<.03). Given that tasks enhancing quality of care were the residual category, the coefficient on any dummy variable is an estimate of the difference between the average time it took to perform the scale item and the time taken to perform with quality, the intercept of the derived equation, ßo, estimates the average amount of time it took to complete a quality evaluation.

Total or actual time for the evaluation can be thought of as having three components:

1. non-clinical time (e.g. time waiting),

2. time taken by the performance of actual clinical tasks, and

3. time taken to complete a quality evaluation. The latter is estimated by βo.

Actual evaluation time, T, minus ßo, is the deviation of actual evaluation time from the average time taken for a quality evaluation. T−ßo may be partitioned into two components such that: $\text{T}-\text{ßo}=(\text{T}-\widehat{\tau })+(\widehat{\tau }-\text{ßo})$. $\widehat{\tau }$ is the estimated average time required for the performance of various clinical tasks in the manner performed, i.e. with their scale quality coding. $\text{T}-\widehat{\tau }$ is the difference between the actual time taken to complete the evaluation and the average time it took to perform the tasks at whatever level of quality. It is the amount of actual evaluation time not explained by the quality with which clinical tasks were performed (our estimate of non-clinical time). $\widehat{\tau }-\text{ßo}$ is the time taken by clinical tasks and the average time to complete a quality evaluation (ßo).

Exploration of the factors associated with $\text{T}-\widehat{\tau }$ (non-clinical time) showed that these differences are essentially random. $\widehat{\tau }-\text{ßo}$, the difference between average time accounted for by performance of clinical tasks and the estimated average time to complete a quality evaluation, is taken as our measure of the optimum investment of time-efficiency.

Analysis

First, we describe the observed quality of care on each dimension: technical quality, art of care, and optimum investment of time. Second, we compare case examples (see four case summaries following Conclusion section) of “good” and “poor” quality of care. Third, we discuss the empirical relationship of the quality of care scales to each other and to actual evaluation time.

Results

Characteristics of Patients and Clinicians

The modal patient was white (65%), male (56%), age 27, and spoke fluent English (93.9%). Minorities were, well represented in the sample, including 18.4% Black, 10.7% Spanish Surname, 2.0% Asian and 3.5% other minorities. Females comprised 44% of the sample. The mean age was 35.6 years. Only 2.3% spoke no English. Although nearly 90% were considered to have some mental disorder, only 61.7% had a major mental disorder. Client functioning at entry was rated on the Global Assessment Scale (GAS) (X = 35.6, s.d 13.6, median 35); 74% were of a severity appropriate to receive acute treatment (GAS score 40 and below) (Endicott, el at., 1976). These patients had functioning levels varying from “major impairment” to needing “constant supervision.”

Evaluating clinicians were primarily psychiatrists or other physicians (50%), but also included registered nurses (16.4%), master’s-level psychologists and social workers (6.8% each), licensed psychiatric technicians (6.2%), other trainees (4,3%), Ph.D. psychologists (2.5%), and others. (7.4%). Most non-psychiatrists had a psychiatrist available for consultation. Evaluators were 85% white, with 4.9% Spanish Surname, 3.7% black, 4.3% Asian, and 1.2% other. Minority clinicians saw about 50% more than their proportionate share of cases, but an ethnic match was not available for every client. The evaluators had an average of 5.5 years’ experience in the psychiatric emergency room.

Technical Quality of Care

The Technical Quality of Care scores were distributed approximately normally with a mean of .540, a standard deviation of .392, and a median of .579. The maximum possible value was 1.83 and the minimum −2.61. The observed range was 1.35 to −1.12. This restricted range represents the fact that in any given case a large number of items from the technical quality scale may not apply and so were entered with the score of zero.

To describe the technical quality variables in five categories, we started at the scales theoretical and empirically defined neutral point of zero. One standard deviation (.392) on either side of the neutral was considered an evaluation of neutral quality. Between the first and second standard deviations (.78), those scoring above zero were labelled evaluations of “good” quality. Those scoring above the second standard deviation were labeled evaluations of “very good” quality. Those scoring between the first and second standard deviations below zero were labelled “bad” quality. Those scoring below the second deviation were labelled “very bad”. Some 31.1% of the cases had very good quality and 0.5% (3 cases) had very bad quality (see Table 1). Just over two-thirds of the sample received good or very good technical quality of care. No significant differences were observed in the Technical Quality of Care Scales scores by age or race. Male gender was related to the receipt of higher technical quality of care (p<.05).

Table 1.

DISTRIBUTION ON QUALITY OF CARE AND TIME INDICATORS (N-670)

	QUALITY OF CARE
	Very Bad	Bad	Neutral	Good	Very Good
Technical Quality	0.5%	1.1%	28.8%	37.1%	31.1%
Art of Care	4.1%	18.7%	14.5%	23.3%	39.5%
	DEVIATION FROM AVERAGE QUALITY PRACTICE TIME $\left(\widehat{\tau }-{\beta }_0\right)$
	Most Time Saved	Much Time Save	Average Time Saving	Some Time Saved	Extra Time Alloted
% Deviating	1.8	15.1	47.6	30.8*	4.8
	TOTAL EVALUATION TIME
**	Shortest Evaluation Time/Bottom 5th of Range	2nd 5th of Range	Mid-Range	4th 5th of Range	Longest Evaluation Time/Top 5th of Range
% By Category	59.3	25.6	12.0	2.5	0.6

^*For .3% of cases in this category $\widehat{\tau }-{\beta }_0=0$

^**When the range of actual evaluation time was split into fifths most cases fell into the first quintile

A few case examples (see case naratives following Conclusion section), disguised for purposes of confidentiality, illustrate the levels of technical quality of care observed. The first case presented here scored “very good” in the technical quality score range, with a score of 1.04.

This interview received technical quality points for: less than 30 minutes waiting prior to first interview; history of current problem and past psychiatric history taken; current medication recorded; contacts made with significant others; a mental status exam, an evaluation of dangerousness, and a psychiatric consultation completed; the patient seen by a M.D; vital signs and a medical workup completed; a psychiatric diagnosis made compatible with the history, a reference to social support during the interview made, the supportive and reassuring manner of the evaluators; patient not given a lethal supply of medication; hospitalisation, referral and specific follow-up arrangements were made. Questions focussed on detoxification, prescribing medications, lithium level, patient elopement, restraints and child abuse did not apply. The clinicians failed only to seek a history of abused substances and allergy to medications.

Case 2 represents the “neutral” category of the technical quality scale’s range, with a specific score of 0.36.

This lost points on technical quality because the clinician did not act supportively, did not attempt to contact significant others nor a current therapist, did not perform a mental status exam, did not assess substance abuse history, did not do a medical work up, and did not refer to social support. The score gained points from waiting period prior to the exam of under 30 minutes, taking history of current problem, making a psychiatric diagnosis compatible with the history, attention to dangerousness and hospitalisation since dangerousness was found, vital signs, clinican being an M.D. (psychiatrist), psychiatric history obtained, medications recorded and lithium level obtained. Questions concerning the nurse’s support, medication prescribed, depression, drug overdose, restraints, elopement, drug allergy and child abuse did not apply. Some vital aspects of technical quality were not performed. However, enough information was provided to ensure appropriate disposition. This combination resulted in a score at the middle of the scale.

Case 3 scored in the “very poor” category of technical quality with a specific score of −.81. This evaluation lost points for no attempt to contact significant others, no reference to social support, no psychiatric consultation, not supportive during the evaluation, no mental status exam, reason for restraints (shackles) not evident on chart, no vital signs, no attempt to contact prior counsellor, no past psychiatric history, no statement recorded on current medications, no substance abuse history, no assessment for drug allergy, no medical work up and no history of current problem. Positive points were obtained for less than 30 minutes wait, evaluating dangerousness, evidence of follow-up.

Questions relating to hospitalisation, substance abuse detox or overdose, prescribing medication, elopement and child abuse did not apply. It is perhaps true that some routine practices would not have revealed information leading to a different result in this case. However, failure to take routine steps caused the case to score low.

Art of Care

Since the Art of Care scale had no previous empirically defined neutral value, its total possible range was divided into fifths to obtain a descriptive distribution. Table 1, line 2, shows the distribution of artful care by descriptive category. Over one-third (39.5%) of the cases scored “very good”, and two-thirds of cases scored “good” or “very good”. Fifteen percent were neutral, and almost 23% were considered bad or very bad. No significant differences were observed in Art of Scale scores by age, race or gender.

Case 1, described previously, obtained a perfect score on Art of Care. Every item was covered. The clinician worked through the patient’s intitial resistance to elicit information, was considerate or tangential statements, recognised the patient’s feelings and asked the patient how she felt about hospitalisation.

An additional case, Case 4, scored in the neutral category, with a score of .50. The clinician did elicit information from the patient and attend to patient’s feelings. The clincian did NOT attempt to engage in collaborative interaction nor include the patient in planning. It was not clear whether the clincian was empathic.

Case 2 had received no points for the art of care, since the clinician did not engage the patient, elicit information, attend to her feelings or involve the patient in treatment planning. The observer found it unclear whether the doctor was empathic. This patient seemed very difficult to involve, which may have influenced the evaluator. However, the lack of scale points arises from the evaluator’s failure to make efforts to include the patient, not from the patient’s behaviour.

Optimum Investment of Time

The estimated average time to complete a quality evaluation (bo) was two hours and 22 minutes. Adjusted R-square for predicting time from the performance of clinical tasks was .16 (d.f.+46,634, p<.0000). 16 percent of the variance in total time (T) spent on evaluation was explained by the actual handling of clincal tasks bearing on technical quality of care and artful care (i.e. predicting time, $\widehat{\tau }$). Actual time for evaluations ranged from 15 minutes to ten hours. The distribution was strongly skewed to the right, with a mean of 1.416 hours, a median of 1.00 hour and a standard deviation of 1.221 hours.

For 94.9% of cases, predicted time $(\widehat{\tau })$ was less than the average taken for a complete quality evaluation (ßo). Deviations of estimated practice time from estimated average quality practice time $\left(\widehat{\tau }-\text{ßo}\right)$ occurred because a procedure was not applicable to the situation or the clinician failed to perform a required clinical activity i.e. the clinician saved time at the expense of good practice.

The difference betwen estimated average practice time, and average quality time, $\widehat{\tau }-\text{ßo}$, varied from 41 minutes more than the average to two hours and fifty-two minutes less than the average. Splitting the deviation range into five equal segments showed that only 1.8% of the sample fell into the first quintile, describing evaluations where the most time was saved, 47.6% were in the third quintile where the average amount of time was saved, and 4.8% were in the fifth quintile where extra time was alloted i.e. from 1 to 41 minutes above the average time for quality work (see Table 1). While no differences in the optimum amount of time alloted to each case were observed by age or gender, clinicians spent significantly less time evaluating African Americans (−.74 versus −.58; p<.02).

Table 2 shows a summary of each case example on quality measures, the amount of time given, and the deviation of practice time from average quality practice time.

Table 2.

SUMMARY OF CASE SCORES ON QUALITY MEASURES

	Technical Quality	Art of Care	Actual Time in Minutes	Estimated Deviation from Quality Practice
Case 1	Very good	Very good	45	Average time saved
Case 2	Neutral	Very bad	20	Much time saved
Case 3	Very bad	Bad	90	Much time saved
Case 4	Very good	Neutral	195	Some time saved

The cases roughly validate the deviation measure by showing that the “some” to “average” deviations from estimated average quality practice time occur on the higher quality cases. The more extreme deviations occur in the lesser-quality cases (indicating time savings at the expense of required practice tasks).

Relating Quality Indexes

The validity of the deviation measure is further confirmed by the significance of the relationship between the categorical technical quality score and the categorial presentation of the deviation between average practice and average quality time, $\widehat{\tau }-\text{ßo}$ (X²= 288.97, df 16, p<.00000). The cross tabulation showed that while only 5% of the “good” and “very good” cases on the technical quality measure fell in the “much” and “most” time saved categories on the deviation between average practice and average quality time dimensions, $\widehat{\tau }-\text{ßo}$, 91% of the “bad” and “very bad” cases did. the relationship between the categorical representation of the Art of Care and categories of $\widehat{\tau }-\text{ßo}$ was also significant (X²=38.5, df 16, p<.00128). It supports the conclusion that time was saved at the expense of engaging the client. However, relationships between categories of good/very good art of care and much/most time saved are not as discriminating as those observed for the technical quality of care scale.

A correlation matrix (Table 3) shows that the two dimensions of quality of care measured by the Technical Quality Scale and the Art of Care Scale are independent (r=.058,n.s).

Table 3.

CORRELATION AMONG QUALITY OF CARE SCALES AND TIME MEASURES (N=670)

	Technical Quality	Art of Care	$\widehat{\tau }-{\mathrm{\beta }}_0$	Actual time
Technical Quality scale		.058	.524	.251
		(n.s.)	(p≤.01)	(p≤.01)
Art of Care Scale			.120	.046
			(p≤.01)	(n.s.)
$\widehat{\tau }-{\mathrm{\beta }}_0$				.464
Actual time				(p≤.01)

Both are positively and significantly related to the deviation of estimated average time spent on clinical tasks from average time for quality evaluations $\left(\widehat{\tau }-\text{ßo}\right)$. Both are associated with not saving time at the expense of quality. The Art of Care was associated with less time saved at the expense of quality, indicating that some time is required for engaging the patient. However, it was not significantly associated with total time, probably because, above a certain minimum of time, additional time is less important for engaging the patient. The time for engagement can be interwoven with other activities. On the other hand, technical quality is moderately related to not saving time below the optimum and is related to total time. Many items of technical quality require seperate activities which add time.

Conclusion

On each quality scale, almost two thirds of the patients received very good or good care. This appears very satisfying, a sign that quality care can be provided in PES evaluations.

The tendency to provide higher technical quality of care to males may be associated with the tendency of males to be perceived as more dangerous than females.

The tendency to save practice time at the expense of African American patients is a concern, although no differences in the technical quality or the art of care were observed.

Our scales provide precise definitions and numerical representations of quality. A full third did not receive good quality care. Perhaps the most important lesson in a time of cost cutting is that time saved is most often at the expense of quality care. Many clinicians view current cost containment efforts as having deleterious consequences for the quality of care in admission evaluations (Rodriguez, 1989). These results offer some support for their belief.

Case Example 1.

A 55-year old, black female was referred to a private hospital by relatives who said they were unable to control her and she had thrown away her medications a few days ago. After seeing her twice in two days and giving 5 mg haloperidol p.o. with no effect, the psychiatrist sent her, under an involuntary hold with police transport, to the PES for formal evaluation. The referral letter stated she was “mildly delusional and has not been sleeping for 24 hours.”

Asked why she was in the PES, she replied, “Because I’m sick with schizophrenia,” She also complained of not being able to sleep. Although initially argumentative and unwilling to give information, she was able with support and much tangential conversation to provide some history. “During one ‘breakdown’, I was hearing things, speaking in tongues I had never heard and seeing colours of the room change. The police came and handcuffed me – I don’t know why, I wasn’t doing anything. My husband just sat there. He was the one who called the police when I told him to get some help.” Four years later she had thought of killing herself. Five years after that, she had been hospitalised. Restoril in hospital helped her sleep. After 4 to 5 days she had refused Restoril and left. She had been given Haldol, Cogentin and Elavil for several years.

During a collateral contact, relatives reported she had been “hyper” and had thrown medication away. The clinician’s diagnosis was Axis I: schizoaffective; Axis II; Deferred; Axis III.High blood pressure (taken in the PES); Axis IV and V: (no information) zero. She denied current hallucinations, thinking problems or suicidal ideation. She was loosely oriented to time but clear on place and person.

Asked how she felt about being admitted to the hospital, she said. “Fine, I just want to get the right help.” She was unable to define what the right help would be. She was admitted forthwith as gravely disabled and dangerous to self. The evaluator implied she was not in significant danger, but was gravely disabled due to her clinical condition and could benefit from restabilising on medications. Haldol p.o. was prescribed.

Case Example 2.

A 22-year old, Caucasian female was transferred from an open ward on an involuntary hold for danger to others and danger to self after striking a staff member and threatening fellow patients. When escorted to an isolation room, she had apparently struck her head intentionally on the wall. The referral letter also stated she had originally been sent to an open ward after having struck a resident in a supported housing residence. She had become increasingly irritable, with decreased sleeping and eating. She did not evidence suicidal ideation or auditory hallucinations, but her speech was pressured and she seemed unduly concerned about the evaluator’s German accent. Patient had a three-year history. Apparently the onset of her psychiatric problems was concurrent with the death of the patient’s brother in a motorcycle accident. She had multiple hospitalisations and several diagnoses (mainly schizophrenia) and had previously been maintained on 2400 mg lithium daily. Two weeks previouly, this dose had been reduced to 1200 mg for unknown reasons. Despite the fact that she was on lithium, she had never received a bipolar diagnosis.

In the PES, the patient wandered about, seemingly agitated, but in fair control. She was overweight and somewhat unkempt, with much red blotching in her face. In a brief interview, she sat facing the window in a defiant posture. She denied knowing she had struck a staff person on an open ward. She did not protest when told she would be staying in PES for some time. Following the interview, the evaluator phoned the open ward to ask about Thorazine given prior to transport and to get the patient’s recent litihum levels. On the phone, the evaluator was suprised the lithium level was so low (.065). The patient was retained involuntary, apparently based on previous behaviour on the open ward, low lithium levels and uncooperativeness in the evaluation. Minimal effort was made to engage the patient or gain collateral information. There was no indication anyone ever attempted to work with the death of her brother as a key issue and no mention of family history.

Case Example 3.

PES received a phone call about a patient who had gone into a rage at a juvenile detention centre. PES staff advised the juvenile be isolated for an hour. An hour and a half later the patient was sent in to PES. Patient appeared to be wearing leg shackles. The counsellor escorting the patient said that prior to admission to detention the patient had been beaten about the head in a fight. The juvenile who had done this had recently been admitted to the detention facility and when a patient accidently was permitted into an area where he saw the other teen, the patient began yelling and swore he would kill him. It was unclear whether isolation had calmed the patient.

Clinician: Do you know why you are here?

Patient: I have a bad temper.

C: What happened?

P: This guy stabbed me 3 times so when I saw him, I lit up.

C: Do you know where you are?

P: (Named the hospital correctly)

C: Do you want to go to the hospital?

P: I’m O.K. where I’m at.

C: What happens the next time you see this guy?

P: We’d quarrel, they said I won’t see him again. I was doing o.k. for a couple of days

C: Is it o.k. if your temper is out of control?

P: No.

C: Where you are going now, will you be seperate from him?

P: Yeah.

C: Will you maintain?

P: Yeah.

After being assured by the counsellor that the teenagers would be kept seperate. the evaluator returned the patient to juvenile detention and. after the patient and escort had left, complained about the referral.

Case Example 4.

A 39-year old white male was escorted by police under an involuntary hold for danger to others. Patient had not been taking prescribed lithium and been drinking heavily when police arrived. In PES, patient was loose, tangential and frequently barely comprehensible. Placed in a cooling off room, handcuffed, the patient struggled constantly with the cuffs. The pressured, hostile aspect of the patient’s current presentation was unusual in his history of more than 15 past hospitalisations. The clinician commented on the difficulty of filling out a global dangerous rating because of the patient’s gross disorientation and incoherence.