Table 3.

Published descriptions and properties of unique performance indicators for public mental health care

Study	Performance indicator	Related criterion	Result
Simpson & Lloyd [17]	• Methadone maintenance (MM) drug abuse treatment client evaluation score (composite measure from 7 items) • Therapeutic communities (TC) drug abuse treatment client evaluation score (composite measure from 7 items) • Out-patient drug-free (DF) drug abuse treatment client evaluation score (composite measure from 7 items) • outpatient detoxification (DT) drug abuse treatment client evaluation score (composite measure from 7 items)	1 year post treatment (high scores more favorable): • opoid use • nonopoid use • marijuana use • alcohol use • employment • jail • return to treatment within 1 year • composite score (all above mentioned criteria)	• More positive MM treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, return to treatment, and the composite score. • More positive TC treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, employment, jail, and the composite score. • More positive DF treatment client evaluation was sig. pos. related to 1-year post treatment opoid use, nonopoiduse, marijuana use, and the composite score. • More positive DT treatment client evaluation was sig. pos. related to 1-year post treatment return to treatment
Srebnik et al. [22]	• Satisfaction domain: client satisfaction questionnaire; involvement in treatment; treatment appropriateness; safety at mental health center • Functioning domain: physical; mental; social and leisure; skills for handling stress and symptoms • Quality of life domain: safety; concerns about living condition; goal attainment; victimization • Clinical status domain: four-dimensional classification scale	• Percentage of clients with any meaningful activity • Percentage of clients with a independent living situation • Percentage of clients with no out-of-community (hospital/jail admission) episode	• No sig. associations of satisfaction measures with any of the criteria • Sig. pos. association of functioning measures with living situation • Sig. neg. association of functioning measures with out-of-community episode • Sig. pos. association of quality of life measures with meaningful activity • Sig. pos. association of clinical status measure with meaningful activity, living situation, and out-of-community episode
Druss et al. [30]	• Promptness and continuity of outpatient follow-up after discharge • Any outpatient follow-up after discharge • Length of stay • Readmission within 30 days • Readmission within in 180 days and total days readmitted within 180 days	Individual level and hospital level measures of satisfaction with: • General service delivery • Alliance with inpatient staff	• Sig. pos. association of promptness and continuity of outpatient follow-up and alliance with inpatient staff at individual level and at hospital level • Sig. pos. association of any outpatient follow-up and alliance with inpatient staff at individual level • Sig. pos. association of length of stay and alliance with inpatient staff at individual level • No sig. association of early readmission with any of the criteria • Sig. neg. association of readmission intensity and general service delivery at individual level
Macias et al. [28]	• International Center for Clubhouse Development Certification status	• 3 organizational resource variables • 7 survey variables reflective of clubhouse model fidelity	• No sig. association of resource variables and clubhouse certification status • Sig. pos. association with 6 of the 7 fidelity variables.
Huff [32]	• Crisis service utilization within 30 days after discharge • Median index episode length of stay • Median number of service contacts within 30 days after discharge • Number of providers contact within 30 days after discharge • Diagnostic evaluation services within 30 days after discharge • Early ambulatory contact (within 5 days) • Medication management service within 30 days after discharge • Psychotherapy service within 30 days after discharge	• Readmission for an acute episode of care to any acute mental health provider within a 30-day period after being discharged	• Sig. pos. association of crisis service utilization and 30-day acute relapse risk • No sig. association of length of stay and 30-day acute relapse risk • Sig. pos. association of service contacts and 30-day acute relapse risk • Sig. pos. association of provider contacts and 30-day acute relapse risk • Sig. neg. association of diagnostic evaluation services and 30-day acute relapse risk • Sig. neg. association early ambulatory contact and 30-day acute relapse risk • Sig. neg. association of medication management services and 30-day acute relapse risk • Sig. neg. association of psychotherapy service and 30-day acute relapse risk
Shipley et al. [91]	• Mean patient satisfaction score (4-item questionnaire) • Mean clinician satisfaction score (4-item questionnaire) • Mean referrer satisfaction score (4-item questionnaire) • Mean time form referral to first appointment • Proportion of patients in which referrer is notified of contact within 6 weeks of referral • Proportion of patient referred and offered appointments who attended	• Five clinical teams of a psychiatric service, one of which was regarded as seriously deficient by an independent review. All teams had important differences in type of referrals and diagnostic group	• Sig. differences in patient satisfaction between teams with which the poorly performing team could be identified • No sig. differences in clinician satisfaction between teams • No sig. differences in referrer satisfaction between teams • Sig. differences in time to first appointment PI that failed to identify deficient team. • Sig. differences referrer notification that failed to identify deficient team. • No sig. differences of patient attending appointments PI between teams.
Chinman et al. [41]	• Service-need index, incorporating ratings of drug and alcohol use, patient's average acuity score, and GAF-m score	• Average caseload of treatment team or program. • Average number of outpatient service hours provided to patients	• Sig. neg. correlation between service-need index and caseload • Sig. pos. correlation between service-need index and service hours provided
Dausey et al. [44]	• Preadmission care (binary variable, continuous variable, spline variable)	• Duration of index admission (length of stay). • Use of post discharge aftercare at 30 days. • Readmission at 14, 30 and 180 days.	• Sig. neg. association between preadmission care and length of stay. • Sig. pos. association between preadmission care and aftercare at 30 days • Preadmission care is associated with a slight increase in probability of readmission
Davis & Lowell [42,43]	• Percentage of funds allocated to state hospitals to community-based services is at the theoretical optimum proportions (43% to 57%)	• Suicide rate per 100,000 population • Cost per capita for mental health care	• Suicide rate lower in states in which funds allocation proportion are close to the optimum. • Relation between expenditure and cost per capita is only found when states that differ more than 12% from the ideal funding partition are excluded
Beaulieu et al. [50]	• NPHPSP Local public health system performance measurement instrument	• Documentary evidence • External judge rating of performance	• Documentation to support agencies' responses to the local instrument validated their responses • External judge ratings were unreliable due to lack of knowledge of local systems
Edlund et al. [59]	• Percentage of persons with any alcohol, drug, or mental disorder that received at least 4 visits with a mental health specialist or 4 visits with a primary care provider that included counseling for mental health problems and/or that received medication that was efficacious for the individual's disorder and used at a dosage exceeding the minimum recommended dosage for an adequate duration • Active treatment after assessment: use of inpatient, day treatment, or residential care; use of prescribed psychotropic medications daily for a month or more; or a period of potentially therapeutic outpatient treatment for alcohol, drug or mental conditions	• Overall satisfaction with the mental health care available for personal or emotional problems during the past 12 months	• Sig. pos. association of appropriate counseling/appropriate pharmacotherapy, and satisfaction with available mental health services • Sig. pos. association of active treatment, and satisfaction with available mental health services
Charbonneau et al. [62]	• Dosage adequacy: antidepressant average daily dosage during 3-month profiling period meets guideline-recommended minimum daily dosage • Duration adequacy: inadequate duration defined as > 21% of the profiling period without antidepressants • Follow-up visit adequacy: at least 3 visits to primary care or psychiatry clinics within 3 months of the initial depression encounter; at least 2 visits in addition to the initial one within 3 months of diagnosis	• Inpatient overall, and psychiatric hospitalizations during the 12 months after the depression care period	• No sig. association between dosage adequacy and any criteria • Sig. neg. association between duration adequacy and subsequent overall or psychiatric hospitalizations • No sig. association between follow-up visit adequacy and any criteria
Druss et al. [64]	• Medication during at least 3 follow-up mental health care visits in the 3 months after a new depressive episode • Ongoing medication treatment in the 3-month period after a new depressive episode • Ongoing medication treatment in the 6 months after a new depressive episode • Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 30 days of hospital discharge • Percentage of members hospitalized for a mental disorder who had an ambulatory visit with a mental health care provider within 7 days of hospital discharge	• Volume of ambulatory mental health use • Volume of inpatient mental health discharges Volume of inpatient mental health days.	• Sig. pos. association between volume of ambulatory mental health use, and medication management PI, and outpatient follow-up PI • Sig. pos. association of volume of inpatient mental health discharges, and medication management PI, and outpatient follow-up PI • Sig. pos. association of volume of inpatient mental health, and medication management PI, and outpatient follow-up PI
Leff et al. [66]	• Service fit: the congruence between services prescribed or needed and services received	• Mortality: natural deaths, medico-legal deaths, suicides	• The relationship between service fit and mortality is more apparent in models based on medico-legal deaths and suicides than in the model based on natural deaths
Nieuwen-huijsen et al. [110]	• Assessment of symptoms (2 criteria), one of both criteria not met within 2 consultations. • Correct diagnosis (3 criteria), one of more criteria not met within 2 consultations. • Evaluation curative care (2 criteria), one of both criteria not met within 2 consultations. • Assessment of work-related causes (2 criteria), one of both criteria not met within 2 consultations. • Evaluation of work disabilities (2 criteria), one of both criteria not met within 2 consultations. • Interventions targeted at the individual (1 criterion), criterion not met within 3 consultations. • Interventions targeted at organization (1 criterion), criterion not met within 3 consultations. • Interventions targeted at providers of care in curative sector (2 criteria), one or both criteria not met within 3 consultations. • Advice on return to work (2 criteria), one or both criteria not met at each consultation. • Timing of consultations (2 criteria), criterion 1 not met at first consultation or criterion 2 not met at consultation 2 or 3. • Summed score over 9 indicators with sufficient content validity and variability.	• Time to return to work • Change in level of fatigue • Patient satisfaction	• No sig. association assessment of symptoms and any criteria • No sig. association correct diagnosis and any criteria • Sig. pos. association evaluation curative care and satisfaction. • No sig. association assessment of work-related causes and any criteria • Sig. neg. association evaluation work disabilities and return to work. • No sig. association organizational interventions and any criteria • Sig. pos. association interventions curative sector and return to work. • No sig. association advice to return to work and any criteria • Sig. neg. association timing of consultations and return to work. • Sig. pos. association overall quality of care and return to work, and satisfaction.
Rost et al. [70]	• A prescription for an antidepressant medication was noted from up to 30 days before to 14 days after index episode start date; dosage sufficient to take medication for 84 out of 114 days following first prescription; 3 non-emergency visits room visits to a primary care or mental health provider at least one of them had to be with the prescribing provider • 4 or more specialty depression care counseling visits in the 6 months following the index visit	• Absenteeism: lost work hours in the past 4 weeks due to illness or doctor visits	• No sig. association appropriate medication and change in absenteeism over 1 year • Sig. association appropriate psychotherapy and change in absenteeism over 1 year
Wierdsma et al. [111]	• Clients receive any psychiatric care in the year before involuntary admission	• Length of stay (less than 3 weeks; more than 6 months; mean number of days) • Ambulatory follow-up • Readmission (within 3 months; within 1 year) • Continued care 12 months after involuntary admission	• Sig. pos. association preadmission care and length of stay, and continued care after 12 months. • No sig. association preadmission care and ambulatory follow-up, and readmission