Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Jul 1.
Published in final edited form as: J Behav Health Serv Res. 2013 Jul;40(3):294–305. doi: 10.1007/s11414-013-9331-5

The Impact of System Level Factors on Treatment Timeliness: Utilizing the Toyota Production System to Implement Direct Intake Scheduling in a Semi-Rural Community Mental Health Clinic

A Weaver 1, CG Greeno 2, DH Goughler 3, K Yarzebinski 3, T Zimmerman 3, C Anderson 4
PMCID: PMC3732800  NIHMSID: NIHMS466815  PMID: 23576137

Abstract

This study examined the effect of using the Toyota Production System (TPS) to change intake procedures on treatment timeliness within a semi-rural community mental health clinic. One hundred randomly selected cases opened the year before the change and one hundred randomly selected cases opened the year after the change were reviewed. An analysis of covariance (ANCOVA) demonstrated that changing intake procedures significantly decreased the number of days consumers waited for appointments (F(1,160)=4.9; p=.03) from an average of 11 days to 8 days. The pattern of difference on treatment timeliness was significantly different between adult and child programs (F(1,160)=4.2; p=.04), with children waiting an average of 4 days longer than adults for appointments. Findings suggest that small system level changes may elicit important changes and that TPS offers a valuable model to improve processes within community mental health settings. Results also indicate that different factors drive adult and children’s treatment timeliness.

Keywords: community mental health, treatment timeliness, engagement, system level factors, intake, Toyota Production System

Introduction

Missed appointments for mental health treatment negatively impact both consumers, who do not receive needed care, and providers, whose resources are not used efficiently.1 Consumers’ first appointments are frequently missed, less often rescheduled, and require more staff time than appointments for on-going treatment.2 Studies suggest that both consumer characteristics 3,4,5,6,7 and system level factors are associated with non-attendance. 5,6,8

As community mental health providers face increased pressure to offer efficient and effective services with diminishing financial resources, it is critical for agencies to address system level factors that may impact consumer engagement, as well as the quality and cost effectiveness of care. While existing work examines a variety of interventions to improve non-attendance at first appointments, including letters, phone calls, and wait list contingencies, limited work explores agency processes and procedures that likely influence consumer engagement.9,10

Addressing system level factors is of particular importance to community mental health providers in rural communities. Individuals in rural areas are as likely to experience psychiatric distress, yet less likely to receive treatment than their urban counterparts.11,12 Individuals in rural areas who do initiate care are more likely to delay help-seeking until they experience significant functional impairments.13 Community mental health providers are often the only treatment option for rural Americans, as 75% of rural counties lack a psychiatrist14 and 60% of rural counties are designated mental health professional shortage areas.15,16 As a result of these significant access issues, ensuring rural consumers’ ability to obtain timely mental health services is crucial. However, there has been little examination of treatment-conferred access to care, such as initial entry into treatment, within the rural mental health service setting.17,18

Research consistently demonstrates that treatment timeliness, or the amount of time between consumers’ initial contact with mental health service providers and their first appointment, is an important system level predictor of attendance.5,6,8 Evidence shows that delays in scheduling appointments may substantially decrease the rate of kept first appointments. The odds of a consumer cancelling or “no showing” increase by 12% for every day of delay between their initial contact with a provider and their first appointment, controlling for sex, age, and program type. 5 Additionally, consumers seen for an assessment within one week of their request for services are significantly more likely to return.6 Stasiewicz and Stalker 8 report that 72% of consumers whose first appointments were scheduled within 48 hours of their initial contact kept their appointments. Gallucci and colleagues 5 found appointment delay was greater for child and adolescent consumers than for adult consumers.

The Institute of Medicine 19 identified timeliness as a primary aim for improving the health care system in the 21st Century. Additionally, there has been an increased focus on improving access to specialty outpatient care.20 In response to these concerns, health care systems have successfully adopted quality improvement methods that advanced safety and reliability in private industries 21; though the use of these approaches within mental health settings has been limited.22 One of these models, the Toyota Production System (TPS), has led to significant gains across a variety of settings and has potential utility for improving treatment timeliness in mental health settings as well.23

The Toyota Production System (TPS)

The Toyota Production System (TPS) is a consumer focused management philosophy and practice “designed to provide the tools for people to continually improve their work.”24 The underlying principles of TPS are continuous improvement and respect for people. The model seeks to reduce inconsistency and waste within organizational processes by increasing value-added steps and eliminating non-value added steps.25 Spear and Bowen 26 note that in order to suitably apply TPS, organizations must create simple, direct connections between consumers and suppliers. In order to achieve these goals, the TPS asserts that (1) the people who do the work must be a part of the change, (2) organizational processes must be examined from the consumer perspective, (3) staff and administration must engage in group problem solving to drive organizational learning and reflection, and (4) success is based on the team, not individuals.24 Principles of the Toyota Production System have been successfully applied in health care settings, contributing to reductions in central line infections and ventilator-associated pneumonias, as well as to improved resolution of patient safety concerns.27,28,29 However, limited work examines utilizing the TPS to improve processes within mental health service settings. In the only identified study, Young and Wachter 22 found that applying TPS principles to patient transfers from inpatient and outpatient units of a psychiatric hospital to outpatient medication management clinics enhanced access, reducing the time between the transfer and first appointment by 87%.

The current study examined the effect of using the Toyota Production System (TPS) to evaluate and change intake procedures on treatment timeliness at a semi-rural community mental health clinic in southwestern Pennsylvania. The work also explored patterns of difference in treatment timeliness across program type (child v. adult).

Methods

Setting

This work took place at a large community mental health clinic located in a semi-rural area of southwestern Pennsylvania. The clinic is located in a low-income, predominately White community with high levels of unemployment. The clinic offers outpatient treatment to adults and children, providing approximately 9300 hours of therapy per year to more than 1600 individuals. The clinic employs eight full-time clinicians, two full-time intake workers, and two support staff. On average, clinicians offer 240 hours per week of direct patient contact, including 43 hours per week reserved for new consumers’ first appointments. The agency utilizes an electronic case management system for scheduling, charting and billing.

The clinic has participated in a long-standing community-university partnership designed to address the substantial gap between academic research and community practice.30,31 The organizational culture of this clinic placed high value on quality improvement and took pride in improving consumer outcomes. The university partnership provided the infrastructure and support to develop and test agency driven intervention efforts. Much of the collaborative work focused on improving consumer engagement with special attention placed on developing innovative engagement strategies tailored to both consumer and clinician needs. As part of this effort, the clinic’s Director of Behavioral Health, with active support from the Executive Director, participated in a regional effort to train social service organizations in the Toyota Production System. This provided the agency with a new tool kit for evaluating internal processes and procedures that may impact consumer engagement.

Evaluating the Existing Intake Process

Agency administrators, clinicians, and staff collectively decided to use TPS principles to assess their intake process. Intake represents a consumers’ first contact with the agency, is a critical point for engagement, and directly impacts treatment timeliness. The TPS model required the examination of the intake process and procedures from the consumer perspective, as well as the identification of any non-value added steps.23

The existing intake process was a call back system (see Figure 1.). When consumers initiated a new call for service, they first spoke to support staff who took their insurance information, necessary for establishing their eligibility for services. The support staff would then ask for the consumers’ contact information and tell them they would receive a call from an intake worker who would collect additional information. A message was then placed in an intake worker’s office mailbox. The intake worker would call consumers back and gather the information necessary to determine appropriate services, including the presenting problem, services requested by the consumer, treatment history, and a risk assessment. At the end of that call, the intake worker would tell the consumer to wait for another call from support staff who would schedule the first appointment. A message was then placed in a support staff person’s mailbox. The support staff would call the consumer back and schedule a first appointment. It should be noted that if the intake worker had concerns about a consumers’ risk assessment, immediate action was taken to provide same day services.

Figure 1.

Figure 1

Open in a new tab

Process Map of Call Back System

A team of support staff, clinicians, and administrators was assembled to evaluate the clinic’s intake process utilizing TPS principles. The team developed a detailed chart mapping the existing intake process from the consumer perspective. The chart was analyzed for sources of potential error and delay. This process illuminated key issues that were likely increasing consumer wait time for a first appointment and therefore negatively impacting consumer engagement.

After mapping the intake process the team realized that in order to schedule a first appointment, consumers were required to complete three separate phone calls. The team became concerned that the call back system did not adequately meet consumer or agency needs. The following potentially non-value added elements were identified.

Disconnect between consumer and agency perceptions of the intake process

Agency procedures often develop based on what works for staff, clinicians, and administrators; though inefficiencies inadvertently develop. In order to manage very complicated scheduling, support staff were needed at the front end of the intake process in order to establish consumers’ eligibility for services and insurance status and then at the back end of the intake process in order to match the consumer with a therapist who can provide the needed services and is credentialed for their particular insurance plan. However, after considering the intake process from the consumer perspective, it became apparent that there was disconnect between the clinic’s view of the call back system and the consumers’ perception. In fact, the team realized that the call back system may have unintentionally communicated inattention or ambivalence to consumers’ identified needs. When consumers called for services and were asked to wait for two additional phone calls before making an appointment, it may not have seemed like the clinic viewed their needs as important or viewed them as a priority. This may have discouraged consumers from ultimately following through with treatment. Further, by the time the intake process was completed, the consumers’ might have felt better and no longer wanted to seek treatment. The team acknowledged the need for the support staff and intake workers to perform two separate roles but had not realized they were essentially functioning separately.

Potential for error

The mapping process led the team to identify non-value added steps that also presented the potential for error. While the call back system procedures were detailed and staff responsibilities were delegated, the steps themselves created considerable opportunities to lose contact with a consumer. Low-income, vulnerable populations often face housing instability and their contact information can quickly change. There are no guarantees that the agency will be able to reach the consumer after their initial call for service. Additionally, it can be difficult to leave messages for consumers due to confidentiality concerns. Support staff and intake workers also noted that the call back system often resulted in “phone tag”. As a result of the difficulties associated with re-contacting consumers, it is likely that many consumers fell through the cracks and did not receive needed care. Further, the reliance on hand written messages placed in staff mailboxes was identified as a potential source of error. Messages can easily get lost or overlooked getting from support staff to intake workers and back again. The team realized that the call back system could negatively impact both agency efficiency and quality of care.

Underutilizing important agency human resources

The call back system resulted in a lot of staff time spent making phone calls, and the team identified this as a key area of waste. As previously mentioned, it often took a series of phone calls to reconnect with consumers to gather the information necessary for intake and another series of phone calls to schedule the first appointment. As community mental health clinics are challenged to offer higher quality care with fewer financial resources, every minute of staff time is valuable. When support staff and intake workers spend time trying to reach clients, the agency is losing human resources that could be spent doing other important work. It should be noted that this was a sensitive topic, as staff had legitimate concerns about job security and wondered if there would be enough work to justify two full time intake workers and additional support staff if the intake process changed.

Designing a New Intake Process

After using TPS principles to assess their intake process from the consumer perspective and identify aspects that were non-value added and potential sources of error, the team worked to design a new intake process that would eliminate the waste and add value. Utilizing Spear’s 23 distillation of the Toyota Production System, the team focused on specifying the process in detail, establishing unambiguous connections between each step, and designing a simple, direct pathway. Further, the team was guided by their awareness of the consumer perspective.

The consensus was that all aspects of the intake process must happen when a consumer made an initial call for service. While this system, referred to as direct intake scheduling, may seem simple, it is a pro-active, innovative design (see Figure 2.). The TPS encourages agencies to find and observe other providers utilizing similar interventions. With help from the regional TPS training organization, the clinic was only able to identify one other agency, a large Children’s Hospital, in the region that used direct intake scheduling. As encouraged by TPS, administrators and staff observed direct intake scheduling at the Children’s Hospital to learn from their effort.

Figure 2.

Figure 2

Open in a new tab

Process Map of Direct Intake Scheduling

The team collaboratively determined co-location of support staff and intake workers was necessary before direct intake scheduling could begin. It was clear that support staff and intake workers would need to work closely in order to facilitate this process. It was noted that their work spaces were far apart, which would make coordinating live calls challenging. In order to provide direct scheduling, it was determined that support staff and intake workers needed to be co-located. Sharing an office space would provide the physical proximity necessary as well as encourage a collaborative work environment. When a new call for service came in, it could simply be transferred between support staff and intake workers in order to complete three necessary components: obtain the eligibility and insurance information, complete the actual intake, and schedule the first appointment, in one phone call.

Though all of the staffing stayed the same, the agency had to gradually prepare for taking live calls. It was essential to understand the average number of new calls for service per day, heavy call times throughout the day, as well as months with higher call volumes. This information allowed the team to determine if and when back up was needed as well as identify the amount of time that intake workers and support staff may be able to spend on other duties. Once intake workers and support staff were co-located and the team evaluated typical call volumes, the team set a date for taking live calls. While there had been concern among staff that direct intake scheduling might be so efficient that their jobs may be in jeopardy, the number of cases opened the year after this important procedural change increased by 33%. Even with the increased number of intakes completed, the intake staff had down time and was given other duties, such as making engagement phone calls.

Sample

New calls for services between September 1, 2006 and August 31, 2007 (before the change in intake procedures) and September 1, 2007 and August 31, 2008 (after the change in intake procedures) served as the sampling frame for this study. The samples consisted of 1218 and 1226 cases, respectively. A power analysis using G-Power v. 3 32 revealed that 191 cases were needed to detect a relatively small effect of .20 at the .05 level with power equal to .80 for a two-tailed distribution.

De-identified electronic case records of 100 new calls for services initiated the year before the change in intake procedures and electronic case records of 100 new calls for services initiated the year after the change in intake procedures were randomly selected and analyzed for this study. The electronic case records include the initial call date for all consumers who contact the agency, the date and outcome of scheduled appointments, as well as demographic and insurance information.

The demographic characteristics for the samples were comparable. The comparison group was 61% female, 75% white, and had a mean age of 31.21 (SD=16.21), while the intervention group was 66% female, 76% white, and had a mean age of 31.78 (SD=16.44). Twenty-seven percent of the comparison group and 31% of the intervention group were insured by Medicaid or Medicare.

Measures

Treatment Timeliness

Treatment timeliness refers to the amount of time, in days, between new consumers’ initial call for services and their first kept appointment. Treatment timeliness was assessed through electronic case records.

Gender

Gender was dichotomized (0 = male; 1 = female).

Race

Race was categorized (0 = white; 1 = Black/African American; 2 = Latino/a/Hispanic; 3 = Asian/Pacific Islander; 4 = unknown).

Age

Age was measured as a continuous variable.

Program Type

The age variable was used to determine program type, adult or child. Cases where consumers were under 18 at the time of their initial call for services were coded as child, while cases where consumers were 18 or older at the time of their initial call for services were coded as adult (0=child; 1=adult).

Insurance Status

Insurance status was categorized (0 = private insurance; 1 = fee for service; 2 = Medicaid/Medicare; 3 = Managed Care; 4 = no insurance; 5 = unknown).

Analytic Plan

Descriptive statistics were calculated for each sample. In addition, group differences between consumers who attended a first appointment and consumers who never attended an appointment after an initial call for service were explored. Independent samples t-test results assessed the mean difference in treatment timeliness.

An Analysis of Covariance (ANCOVA) was performed on treatment timeliness as a function of the intake intervention and program type, controlling for sex, race, and type of insurance. The pattern of difference on treatment timeliness between adult and child programs, controlling for sex, race, and type of insurance was also assessed using ANCOVA. An analysis of outliers and influential points revealed that four cases exceeded the extreme value identified as 45 days between an initial call and a first kept appointment. These four cases, with values of 101 days and 106 days in the comparison group and 49 days and 82 days in the intervention group, were not included in the ANCOVA analysis. These cases inflated the variances to such a degree that the assumptions of ANCOVA were violated, making it impossible to appropriately assess the intake intervention. The analytic sample for the ANCOVA included 160 cases.

Results

Eighty-two percent of consumers in each group attended a first appointment after an initial call for service, while 18% of consumers in each group never attended an appointment. There were no significant differences on sex, race, or type of insurance between consumers who kept a first appointment and consumers who never attended an appointment.

Bivariate analysis of treatment timeliness revealed a mean difference between groups. As Levene’s test of Equality of Variances was significant (F(1,160)=6.46, p=.01), an independent samples t-test with equal variances not assumed found a significant difference in treatment timeliness (t=2.04; p=.04). In addition, a mean difference on treatment timeliness between consumers in the adult program and consumers in the children’s program was also found (t=2.45; p=.02).

The ANCOVA controlling for sex, race, and type of insurance demonstrated that direct intake scheduling, developed using TPS principles, significantly decreased the number of days consumers waited for their first appointment (F(1,160)=4.9; p=.03). Before this important change, consumers waited an average of 11 days for an appointment while after the intervention the wait was an average of 8 days. The pattern of difference on treatment timeliness was also significantly different between program types (F(1,160)=4.2; p=.04), controlling for sex, race, and type of insurance, with children waiting an average of 12 days and adults waiting an average of 8 days for an appointment. None of the demographic variables were significant.

Discussion

Missed first appointments, common in community mental health settings, negatively impact providers’ ability to offer efficient, cost-effective care. As community mental health providers balance the demand for quality, effective care with the financial realities that require cost-effectiveness, addressing system level factors that have been shown to improve attendance is of increasing importance. This is of particular relevance to rural and small community mental health service settings where there are a limited number of providers, and consumers tend to seek care when symptomatology is severe. Quality improvement models developed in private industry, such as the Toyota Production System (TPS), offer strategies for evaluating system-level factors and have been successfully applied in healthcare settings. However, little work examines the utility of such approaches within mental health settings.22

Findings from this study suggest that using TPS principles to evaluate and change intake procedures from a call back system to a direct scheduling system improved treatment timeliness and increased agency efficiency at a semi-rural community mental health clinic. On average, consumers initiating service after this important procedural change experienced a 27% decrease in wait time for a first appointment, attending a first appointment three days sooner than consumers who initiated services prior to the intervention. Additionally, the number of cases opened the year after this important procedural change increased by 33%. While it may be that more individuals sought treatment, it is also likely that direct intake scheduling was more efficient, allowing a greater number of consumers to complete the intake process and initiate care.

It should be noted that the change in intake procedures did not impact attendance rates at first appointments. Eighteen percent of consumers did not attend first appointments both before and after intervention. This is a much lower no show rate than the 39% to 50% typically found in outpatient mental health settings 33,34 and likely reflects the agency’s long standing university partnership that focused on increasing consumer engagement. Furthermore, literature suggests that consumers seen within one week of their initial service call are significantly more likely to attend their first appointment.6 While this important change in intake procedures significantly improved treatment timeliness, the average consumer still waited eight days for an appointment. The increase in treatment timeliness may not have been enough to impact attendance. Finally, future work should examine the relationship between treatment timeliness and consumers’ treatment trajectories, as the change in intake procedures and decreased wait time may impact attendance over the course of treatment.

These results support the utility of the TPS in mental health settings. Not only did the TPS approach lead to quality improvement, but also it encouraged agency administrators, clinicians, and staff to collaboratively evaluate and change the intake process. Through this process, the agency increased its awareness of the consumer perspective and better understood the importance of identifying what processes and procedures may communicate to consumers. Further, this collaborative approach generated practice-based wisdom that may help other community mental health agencies identify and change important system level characteristics likely to influence quality of care, engagement, and attendance.

Additionally, findings suggest that there may be different factors driving children’s and adult’s treatment timeliness. On average, children waited 12 days for an appointment, while adults waited 8 days. This is likely the result of other system level factors that delay children’s ability to obtain a first appointment. Before scheduling a first appointment, children must be matched with a clinician who treats children, has the skills necessary to meet the children’s presenting needs, and is credentialed by their insurance plan. Not all clinicians at this clinic serve children, so the number of potential appointment options is less than for adults. Further, almost all children rely on others, usually parents, to initiate mental health treatment and transport them to services.35,36 Therefore, it may be more difficult for support staff to find open appointments at times that are convenient for both the child (after school hours) and the parent or caregiver. Community mental health providers must be aware of this potential barrier to timely service and examine the potential impact of offering evening and weekend appointments for children’s services.

Limitations

This study is not without limitations. There is an inherent time difference in this quasi-experimental design. Changes in addition to the intake intervention, such as staff turnover or the flow of referrals from other service providers, may have occurred during the study period, and those changes may have impacted treatment timeliness.

Additionally, treatment timeliness was operationalized as the number of days between the initial call for services and the first kept appointment. This was chosen as a kept appointment represents consumer engagement. However, in order to more comprehensively assess the changes in intake procedures, it may also have been useful to compare the initial call for services to the first offered appointment. This would assess the agency’s ability to provide a timely appointment and may better measure the impact of this important change on the system.

Finally, this work did not control for diagnosis or previous treatment. Both of these factors have been associated with attendance at first appointments.

Implications for Behavioral Health

Community mental health clinics face a lot of pressure to provide quality, efficient care with fewer resources. Results of this study suggest that examining system level factors and making small procedural change can illicit important effects, improving services for consumers without utilizing a lot of resources. This work also demonstrates that the TPS offers a useful model for addressing system level factors and guiding the change process. The practice-based wisdom gained from this study may inform other community mental health agencies and encourage them to consider incorporating quality assurance models, such as the TPS, in order to assess processes and procedures that may inhibit consumers’ access to quality, efficient care.

Table 1.

1.
Descriptive Statistics
Group 1: Before Intervention (N=100) Group 2: After Intervention (N=100)
% %
Sex (% female) 61 66
Race (% white) 75 76
Type of Insurance
 % Private 26 28
 % Fee for Service 6 5
 % Medicare/Medicaid 27 31
 % Managed Care 32 29
 % Uninsured 0 2
 % Unknown 9 5
Program Type (% Adult) 70 74
Ever Attended First Appointment (% yes) 82 82
Mean SD Mean SD
Age 31.21 16.21 31.78 16.44
Open in a new tab

Table 2.

2.
Group Comparisons (N=200)
DV: Ever Attended First Appointment
Test Statistic df p
Sex χ2 = 0.548 1 0.459
Race χ2 = 6.381 4 0.172
Type of Insurance χ2 = 3.185 4 0.527
Open in a new tab

Table 3.

3.
Analysis of Covariance (N=160)
DV: Treatment Timeliness
F p
Sex 0.005 0.943
Race 0.001 0.979
Type of Insurance 1.056 0.306
Intervention 4.886 0.029
Program Type (child v. adult) 4.203 0.042
Open in a new tab

Acknowledgments

This work was supported by NIMH R24 MH 0066872: Collaboration to promote engagement of low-income clients, Carol Anderson, PI.

Footnotes

Conflict of Interest Statement

None of the authors of the above manuscript have declared any conflict of interest that may arise from being named as an author on this manuscript.

References

  • 1.Shoffner J, Staudt M, Marcus S, et al. Using telephone reminders to increase attendance at psychiatric appointments: Findings of a pilot study in rural Appalachia. Psychiatric Services. 2007;58(6):872–875. doi: 10.1176/ps.2007.58.6.872. [DOI] [PubMed] [Google Scholar]
  • 2.Sparr LF, Moffitt MC, Ward MF. Missed psychiatric appointments: Who returns and who stays away? American Journal of Psychiatry. 1993;150(5):801–805. doi: 10.1176/ajp.150.5.801. [DOI] [PubMed] [Google Scholar]
  • 3.Carpenter PJ, Morrow GR, DelGandio AC, et al. Who keeps the first outpatient appointment? American Journal of Psychiatry. 1981;138:102–105. doi: 10.1176/ajp.138.1.102. [DOI] [PubMed] [Google Scholar]
  • 4.Chen A. Noncompliance in community psychiatry: A review of clinical interventions. Hospital and Community Psychiatry. 199;42(3):282–287. doi: 10.1176/ps.42.3.282. [DOI] [PubMed] [Google Scholar]
  • 5.Gallucci G, Swartz W, Hackerman F. Brief reports: Impact of the wait for an initial appointment on the rate of kept appointments at a mental health center. Psychiatric Services. 2005;56(3):344–346. doi: 10.1176/appi.ps.56.3.344. [DOI] [PubMed] [Google Scholar]
  • 6.Greeno CG, Anderson CM, Shear MK, et al. Initial treatment engagement in a rural community mental health center. Psychiatric Services. 1999;50(12):1634–1636. doi: 10.1176/ps.50.12.1634. [DOI] [PubMed] [Google Scholar]
  • 7.Kruse GR, Rohland BM, Wu X. Factors associated with missed first appointments at a psychiatric clinic. Psychiatric Services. 2002;53(9):1173–1176. doi: 10.1176/appi.ps.53.9.1173. [DOI] [PubMed] [Google Scholar]
  • 8.Stasiewicz PR, Stalker R. Brief report: A comparison of three “interventions” on pretreatment dropout rates in an outpatient substance abuse clinic. Addictive Behaviors. 1999;24(4):579–582. doi: 10.1016/s0306-4603(98)00082-3. [DOI] [PubMed] [Google Scholar]
  • 9.Benway CP, Hamrin V, McMahon TJ. Initial appointment nonattendance in child and family mental health clinics. American Journal of Orthopsychiatry. 2003;73(4):419–428. doi: 10.1037/0002-9432.73.4.419. [DOI] [PubMed] [Google Scholar]
  • 10.Williams ME, Latta J, Conversano P. Eliminating the wait time for mental health services. Journal of Behavioral Health Services & Research. 2008;35(1):107–114. doi: 10.1007/s11414-007-9091-1. [DOI] [PubMed] [Google Scholar]
  • 11.Hauenstein EJ, Petterson S, Merwin E, et al. Rurality, gender, and mental health treatment. Family & Community Health. 2006;29(3):169–185. doi: 10.1097/00003727-200607000-00004. [DOI] [PubMed] [Google Scholar]
  • 12.Wang PS, Lane M, Olfson M, et al. Twelve-month use of mental health services in the United States: Results from the National Comorbidity Survey Replication. Archives of General Psychiatry. 2005;62(6):629–639. doi: 10.1001/archpsyc.62.6.629. [DOI] [PubMed] [Google Scholar]
  • 13.Browning D, Andrews C, Niemczura C. Cultural influences on care seeking by depressed women in rural Appalachia. The American Journal for Nurse Practitioners. 2000;4(5):22–32. [Google Scholar]
  • 14.Hartley D, Bird D, Dempsey P. Mental health and substance abuse. In: Ricketts T, editor. Rural Health in the United Stated. New York: Oxford University Press; 1999. [Google Scholar]
  • 15.Gamm LG, Stone S, Pittman S. Rural Healthy People 2010: A companion document to Health People 2010. Vol. 1. College Station, TX: The Texas A & M University System Health Science Center, School of Rural Public Health, Southwest Rural Health Research Center; 2003. Mental health and mental disorders – A rural challenge. [Google Scholar]
  • 16.Mulder PL, Shellenberger S, Kenkel M, et al. The behavioral health care needs of rural women. Washington, DC: American Psychological Association, Committee of Rural Health; 2001. [Google Scholar]
  • 17.Brekke J, Long JD, Nesbitt N, et al. The impact of service characteristics on functional outcomes from community support programs for persons with schizophrenia: A growth curve analysis. Journal of Consulting and Clinical Psychology. 1997;65(3):464–475. doi: 10.1037//0022-006x.65.3.464. [DOI] [PubMed] [Google Scholar]
  • 18.Fortney J, Sullivan G, Williams K, et al. Measuring continuity of care for clients of public mental health systems. Health Services Research. 2003;38(4):1156–1175. doi: 10.1111/1475-6773.00168. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Institute of Medicine. Crossing the quality chasm: A new health system for the 21st Century. Washington, DC: National Academy Press; 2001. [Google Scholar]
  • 20.Hankinson MT, Faraone D, Blumenfrucht M. Sustained improvement for specialty clinic access. Joint Commission Journal of Quality and Patient Safety. 2006;32(3):142–151. doi: 10.1016/s1553-7250(06)32019-3. [DOI] [PubMed] [Google Scholar]
  • 21.Resar RK. Making noncatastrophic health care processes reliable: Learning to walk before running in creating high-reliabiity organization. Health Services Research. 2006;41(4):1677–1689. doi: 10.1111/j.1475-6773.2006.00571.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Young JQ, Wachter RM. Applying Toyota Production System principles to a psychiatric hospital: Making transfers safer and more timely. Joint Commission Journal on Quality and Patient Safety. 2009;35(9):439–448. doi: 10.1016/s1553-7250(09)35061-8. [DOI] [PubMed] [Google Scholar]
  • 23.Spear SJ. Fixing health care from the inside, today. Harvard Business Review. 2005;83:78–91. [PubMed] [Google Scholar]
  • 24.Liker JK. The Toyota Way: 14 Management Principles from the World’s Greatest Manufacturer. Madison, WI: CWL Publishing Enterprises, Inc; 2004. [Google Scholar]
  • 25.Institute for Healthcare Improvement (IHI) IHI Innovation Series white paper. Cambridge, MA: IHI; 2005. Going Lean in Health Care. [Google Scholar]
  • 26.Spear S, Bowen HK. Decoding the DNA of the Toyota Production System. Harvard Business Review. 77:96–106. [Google Scholar]
  • 27.Shannon RP, Frndak D, Grunden N, et al. Using real-time problem solving to eliminate central line infections. Joint Commission Journal on Quality and Patient Safety. 2006;32(9):479–487. doi: 10.1016/s1553-7250(06)32063-6. [DOI] [PubMed] [Google Scholar]
  • 28.Resar R, Pronovost P, Haraden C, et al. Using a bundle approach to improve ventilator care processes and reduce ventilator-associated pneumonia. Joint Commission Journal on Quality and Patient Safety. 2005;31(5):243–248. doi: 10.1016/s1553-7250(05)31031-2. [DOI] [PubMed] [Google Scholar]
  • 29.Furman C, Caplan R. Applying the Toyota Production System: Using a patient safety alert system to reduce error. Joint Commission Journal on Quality and Patient Safety. 2007;33(7):376–386. doi: 10.1016/s1553-7250(07)33043-2. [DOI] [PubMed] [Google Scholar]
  • 30.Proctor EK. Leverage points for the implementation of evidence-based practice. Brief Treatment and Crisis Intervention. 2004;4(3):227–242. [Google Scholar]
  • 31.Gonzales JJ, Ringeisen HL, Chambers DA. The tangled and thorny path of science to practice: Tensions in interpreting and applying “evidence. Clinical Psychology: Science and Practice. 2006;9(2):204–209. [Google Scholar]
  • 32.Faul F, Erdfelder E, Lang A, et al. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007;39(2):175–191. doi: 10.3758/bf03193146. [DOI] [PubMed] [Google Scholar]
  • 33.May RJ. Effects of waiting for clinical services on attrition, problem resolution, satisfaction, attitudes toward psychotherapy, and treatment outcome: A review of the literature. Professional Psychology: Research and Practice. 1991;72(3):209–214. [Google Scholar]
  • 34.Staudt MM. Helping children access and use services: A review. Journal of Child and Family Studies. 2003;12(2):49–60. [Google Scholar]
  • 35.Broadhurst K. Engaging parents and carers with family support services: What can be learned from research on help-seeking? Child & Family Social Work. 2003;8(4):341–350. [Google Scholar]
  • 36.Logan DE, King CA. Parental facilitation of adolescent mental health service utilization: A conceptual and empirical review. Clinical Psychology. 2001;8(3):319–333. [Google Scholar]

RESOURCES