Abstract
Objective
To explore staff perceptions about sustainability, commitment to change, participation in change process, and information received about the change project within the Veterans Administration Primary Care and Mental Health Integration (PC-MHI) initiative and to examine differences from the Veterans Health Administration Mental Health Systems Redesign (MHSR) initiative.
Data Sources
Surveys of change team members involved in the Veterans Affairs PC-MHI and MHSR initiatives.
Study Design
One-way analysis of variance examined the relationship between commitment, participation and information, and sustainability. Differences in PC-MHI sustainability were explored by location and job classification. Staff sustainability perceptions were compared with MHSR results.
Principal Findings
Sustainability differed by staff discipline. Difference between MHSR and PC-MHI existed by job function and perceptions about the change benefits. Participation in the change process and information received about the change process were positively correlated with sustainability. Staff commitment to change was positively associated with staff perceptions about the benefits of change and staff attitudes toward change.
Conclusions
Sustainability is an important part of organizational change efforts. Change complexity seems to influence perception about sustainability and impacts staff perceptions about the benefits of change. These perceptions seem to be driven by the information received and opportunities to participate in the change process. Further research is needed to understand how information and participation influence sustainability and affect employee commitment to change.
Keywords: integration, mental health, primary care, sustainability, Veterans Health Administration
Veterans Health Administration (VHA) has made a major effort to meet very high standards in the delivery of health care, in general, and mental health care, in particular. The VHA aspires to a system of care that is population-focused, evidence-based, team-delivered, and veteran-centric. Its goals for excellence in the delivery of mental health care are clearly delineated in the Uniform Mental Health Services Handbook.1 Leaders in VHA recognized early on that merely identifying appropriate practices for the facilities to use would not be sufficient. They would also need to develop the skills and motivation necessary to implement and sustain of these practices. In rolling out goals and practices, one tool available to Veterans Integrated Service Network (VISN) and Mental Health leaders was the Mental Health Systems Redesign (MHSR) project (based on the Institute for Healthcare Improvement model), which was designed to help the VISN and facility mental health leaders learn skills to make the necessary and sustained changes in their systems of care. Staff from all Veterans Affairs (VA) medical centers created a team to select and implement a process improvement project focusing on topics such as managing patient flow, clinical care, and improving access.
Our group previously investigated the characteristics of the change processes in the MHSR project. We explored whether the changes pursued had characteristics of sustainability per the perceptions of the members of the change team.3 The British National Health Service's Sustainability Index (SI) measured staff perceptions. In brief, we found significant heterogeneity in the scores achieved on the SI across VISNs, facilities, and types of staff members replying. Four of the 21 VISNs (19%) with survey responses had scores in the range that would predict lack of sustainability in the British system. Employee perceptions about their involvement, as well as the perceived benefits and credibility of the change differed by job classification, such that administrators rated aspects of change more highly than clinicians. Thus, despite mental health change leaders' involvement in nationwide learning collaborative and coaching efforts at the VISN level, the characteristics thought to be linked to sustainability were rated very differently across projects, VISNs, and types of employees.
The Uniform Mental Health Services Handbook mandates the implementation of Primary Care and Mental Health Integration (PC-MHI).4-8 Integration of primary care and mental health services was required in all VA medical centers and large community-based outpatient clinics. Such efforts require a cross-departmental collaboration to design and implement a new system of care to serve all of Veterans' physical and behavioral health needs. Goals of PC-MHI are to (1) promote effective treatment of mental health and substance abuse disorders in primary care settings and (2) increase access to improve the quality of care by addressing unmet treatment needs.4,8
Recently, the VHA Office of Mental Health Operations has promulgated the first report (with many to come) on the progress of VHA facilities and VISNs in implementing these programs and on their effectiveness. Despite the fact that PC-MHI has been rigorously and repeatedly determined to be effective in decreasing depressive symptoms, improving mortality rates, and decreasing suicidality of depressed veterans and other subjects in primary care clinics,9–11 the Office of Mental Health Operations report noted that the implementation of integrated mental health services in primary care (using a blended model of collaborative, collocated care, and disease care management) was lagging behind other implementations of evidence-based practices.12 The report indicated that there is marked variation in the penetrance rate despite steady growth during the past 3 years. In the fiscal year (FY) 2010, 4.7% of all veterans seen in a primary care setting received integrated treatment, with 5.3% in FY2011, and 5.6% in the first quarter of 2012. The variation across VISNs ranges from 3% to 10% nationally, based on data reported in the VHA PC-MHI Dashboard for the first quarter of 2012, and the variation in penetrance within VISNs is even greater. For example, the rate in an upper Midwestern network composed of 7 medical centers and 30 community outpatient clinics ranged from 3% to 14%.
The PC-MHI intervention differed from the MHSR initiative because of its complexity and mandated top-down changes. Thus, we wondered whether the complexity of the PC-MHI protocols and procedures, and changes driven from the top-down versus participant-developed change projects, would be perceived to be particularly difficult to implement. The context of this study is also important in that we studied PC-MHI versus MHSR implementation in 2 VISNs. The first VISN is an upper Midwestern network composed of 7 medical centers and more than 30 community outpatient clinics. The second VISN is a northeast network composed of 6 medical centers and more than 30 community outpatient clinics. Thus, our study of heterogeneity of implementation and perceived sustainability in these VISNs could inform future efforts to implement other complex changes across the VA.
In addition to British National Health Service's SI measures, we chose to assess affective commitment to the changes specified in the PC-MHI projects, as we hypothesized that the differences we saw in previous studies between clinicians and administrators might reflect a different level of commitment. Employees (including clinicians) play an important role in the implementation and sustainability of quality improvement efforts.13 Their role translates into their commitment to the change process. As organizations adapt and transform, it is important that efforts to manage the introduction of new processes or ideas influence employees' affective commitment toward change. Building on organizational commitment research, Herscovitch and Meyer14 developed a model to assess employee commitment to organizational change. The employee commitment to organizational change model examines employee commitment to a specific change within the organization. It defines commitment as “a force (mind-set) that binds an individual to a course of action deemed necessary for the successful implementation of a change initiative.”14(p475) Recent studies have examined how employee commitment to a change process is shaped by an employee's perceived locus of control and organizational identification15,16 and influenced by trust, belief in organizational leadership, and leadership style.17-19
Some of the attributes related to information about the change and participation in the change process that influence employee commitment to change also facilitate or impede change sustainability. For example, a locally generated change idea based on staff needs and supported by an internal champion helps create an intrinsic staff motivation to participate in the change process. Involvement of staff in quality improvement efforts adapts the change process to the local environment. Parker and colleagues20 classified it as a local participatory approach to quality improvement. Through involvement in the change process, staff attitudes toward change increase the quality and likelihood of sustaining that change.21-27 Thus, we hypothesized that some deficits inherent in VA-wide mandated changes in terms of affective employee commitment might exist but that these might vary across types of programs and types of employees.
Thus, we surveyed staff involved in PC-MHI implementation efforts from 2 VISNs with ongoing effort to integrate primary care and mental health services. We assessed their responses to the SI and compared them with the responses of participants in other VA mental health implementations, as well as with the standards for these instruments. Because of the complexity of the intervention and that top-down changes were mandated, we hypothesized that PC-MHI perceived sustainability would be lower than for other MH implementations. In other words, staff perceptions of sustainability would differ when they had input in the change process versus having the planned changes being more directive. We also assessed staff perceptions about their commitment to change, participation in and information received about the change process. We predicted that PC-MHI Affective Commitment would be lower than for usual implementations. We predicted that there would be significant differences across classes of employees.
Methods
Study population
We worked with the leadership of the 2 VISNs to get a list of individuals, including physicians, nurses, social workers, and support staff (n = 106) involved in PC-MHI initiatives in the 2 VISNs. We sent e-mail invitations asking respondents to voluntarily complete a brief 29-item online survey. Follow-up e-mails were sent as reminders to increase the overall survey response rate. University of Wisconsin Madison researchers functioned as outside evaluators and received and analyzed survey results. Anonymity was promised to all persons completing the survey. No identifiable data or information was shared with any VHA employees or officials. The University of Wisconsin Madison Health Sciences Institutional Review Board approved the research as exempt.
Outcome measures
The online survey assessed staff perceptions related to sustainability, affective commitment to the change, participation in the change process, and information received about the change project.
Sustainability propensity was measured using the British National Health Service's SI.22,28 Affective commitment to change assesses the desire of employees to provide support for the change based on a belief in its inherent benefits, not in the correctness of the change.14 It is measured using the 6-item scale (α = .92). Sample items include “This change was a good strategy for this organization” and “This change was not necessary (reverse scored).” In this pilot study, employees were asked to assess their commitment to PC-MHI.
Change Participation is a 4-question scale that was developed to assess an employee's perceived level of participation during an organizational change.29 It was designed to measure employee perceptions regarding their input into the change process. Items focus on employee awareness of opportunities to help introduce the change, ask questions about the change, and participate during the implementation of the change within the organization. For example, “I was able to participate in the implementation of this change.” Using employees in the public housing and community development sector, the scale was tested and validated (α = .72).
Quality of Change Information is a 4-question scale designed to measure the usefulness and value of the information about the change.30 Sample items include “The information received about this change was useful” and “I was able to ask questions about this change.” It was validated in studies (α = .87) assessing employee openness to organizational change and acceptance of the organizational change.29,30
In this pilot study, employees responded to each of these scales using a 5-point scale ranging from 1 (“strongly disagree”) to 5 (“strongly agree”).
Demographic questions included gender, race, ethnicity, and information about job function (eg, administration/management, clinical, clerical/reception); discipline (physician, psychiatrist, nurse, nurse practitioner, licensed practical nurse, psychologist, and other), employment status (full time, part time, contract, and other); and facility and VA tenure.
Analysis
We conducted the analysis in 2 phases. The first part of the analysis explored the survey responses for the PC-MHI initiative in the 2 VISNs. Specifically, we looked at respondent demographics. We employed a 1-way analysis of variance to examine differences in sustainability, employee commitment, and participation and information scores across facilities, by job classification and staff discipline (eg, psychiatrist, social worker). We also explored the correlations between employee commitment, participation and information, and sustainability scores.
The second part of the analysis explored staff perceptions about sustainability when they participated in the identification, selection, and implementation of the change project (MHSR) versus when the project changes were driven using a top-down approach (PC-MHI). In other words, do staff perceptions about sustainability differ on the basis of the complexity of the change and whether the change is driven from the bottom-up as compared with top-down change? We explored differences in respondent demographics and sustainability. The data set for analysis included those staff from 2 VISNs who participated in the PC-MHI (n = 64) and included only staff in the 2 VISNs who participated in the MHSR initiative (n = 50). The analysis for the overall SI score and the subscales of Process, Staff, and Organization was limited to those respondents who answered all questions within the subscale, Process (n = 4), Staff (n = 4), or Organization (n = 2), or all 10 questions for the overall SI.
Key findings
Participant characteristics
Surveys were sent to 106 VA personnel involved in PC-MHI in the 2 VISNs. A total of 64 (60.4%) responses were received; however, only 51 of the surveys had useable data. Staff demographics are in given in Table 1. Respondent disciplines were nurse, licensed practical nurse, or nurse practitioner (n = 23, 35.9%), social workers (n = 19, 29.7%), physician or psychiatrist (n = 11, 17.2%), or psychologist (n = 9, 14.1%). Most respondents were clinicians (79.7%) who worked full time (87.3%) and were females (68.8%). Demographics of this sample differed from those staff from the same 2 VISNs who were surveyed in the MHSR initiative.3 Although the results (Table 1) showed a significant difference in the distribution of respondents by job classification (χ2 = 24.25, P = .000), it is primarily due to how the PC-MHI sample was derived. Differences in tenure also existed between the 2 initiatives. Respondents in the PC-MHI had been employed in the VA on average 5 years less than those persons in the MHSR project (9.1 years vs 13.9 years, P = .010) and at their respective facilities, approximately 5.6 years less than the MHSR respondents from the same 2 VISNs.
Table 1. Respondent Demographics: PC-MHI Versus MHSR.
| % (n) | ||||
|---|---|---|---|---|
|
|
||||
| Employee Demographics | PC-MHI | MHSRa | χ2 | P |
| Gender | 5.63 | .060 | ||
| Male | 29.7 (19) | 22.5 (9) | ||
| Female | 68.8 (44) | 65.0 (26) | ||
| Refused/no response | 1.6 (1) | 12.5 (5) | ||
| Discipline within VA | ||||
| Nurse, LPN, or nurse practitioner | 35.9 (23) | |||
| Social worker | 29.7 (19) | |||
| Physician or psychiatrist | 17.2 (11) | |||
| Psychologist | 14.1 (9) | |||
| Clerical | 1.5 (1) | |||
| No response | 1.5 (1) | |||
| Job classification | 22.60 | .000 | ||
| Administration/management | 14.1 (9) | 42 (21) | ||
| Clinician | 79.7 (51) | 36.0 (18) | ||
| Clinical administrator/management | 4.8 (3) | 14.0 (7) | ||
| Clerical/reception | 1.6 (1) | 8.0 (4) | ||
| Mean (SD) | Mean (SD) | F | ||
|
|
||||
| Average tenure, years | ||||
| Facility tenure | 7.9 (8.2) | 13.5 (10.0) | 10.42 | .001 |
| VA tenure (staff survey) | 9.1 (9.3) | 14.2 (10.2) | 7.57 | .006 |
Abbreviations: LPN, licensed practical nurse; MHSR, Mental Health Systems Redesign; PC-MHI, Primary Care and Mental Health Integration; VA, Veterans Affairs.
Ten respondents for the 2 VISNs for the MHSR study did not respond to the gender question. Discipline data were not collected as part of the MHSR initiative.
Table 2 shows the average employee responses for the questions related to sustainability propensity, affective commitment to change, participation in the change process, and information received about the change. The average overall sustainability score across all respondents was 55.50, a score at a level indicating that change may not be sustained within the organization. The average employee response on the other 3 scales was at or below the median scale response. With the exception of the effectiveness question (P = .048), the small number of responses did not indicate that significant differences existed across responding facilities. Staff members who identified themselves as physicians, psychiatrists, or psychologists differed in their perceptions about the benefits of the change as compared with social workers (μ = 1.86 vs μ = 4.78, P = .038). In other words, the social workers were more likely to perceive that the changes would improve efficiency and make their jobs easier. No differences existed by job classification in the PC-MHI initiative.
Table 2. PC-MHI Average Respondent Scores.
| n | Mean (SD) | |
|---|---|---|
| Sustainability questions | ||
| Benefits | 51 | 3.60 (3.29) |
| Credibility | 51 | 4.69 (2.97) |
| Adaptability | 51 | 4.51 (2.46) |
| Effectiveness | 49 | 4.15 (2.32) |
| Staff involvement | 48 | 6.26 (4.02) |
| Staff attitudes | 50 | 4.49 (4.47) |
| Senior leadership engagement | 50 | 8.40 (5.09) |
| Clinical leadership engagement | 48 | 10.59 (5.32) |
| Organization | 49 | 4.16 (2.64) |
| Infrastructure | 49 | 4.26 (3.64) |
| Process subcategory score | 49 | 17.03 (8.83) |
| Staff subcategory score | 46 | 29.88 (14.89) |
| Organization subcategory score | 49 | 8.42 (5.64) |
| Total sustainability score | 46 | 55.50 (27.38) |
| Affective commitment to change | 49 | 1.83 (0.77) |
| Participation in the change process | 49 | 2.84 (1.23) |
| Information about the change process | 49 | 2.44 (1.04) |
Abbreviation: PC-MHI, Primary Care and Mental Health Integration.
Relationship of commitment, participation, and information to sustainability
We explored the relationship between staff perceptions about sustainability and their affective commitment to the change (ie, Do they believe in the value of the change and actively support it?), their ability to participate in the change process, and their belief that they received adequate information about the change. The results indicate a significant positive correlation between the Participation scale and all parts of the SI (see Table 3). In this pilot study, employee perceptions about the amount of information received about the change process were positively correlated with most parts of the SI except for the questions related to adaptability and senior leadership. Results from the analysis also found significant correlations between an employee's affective commitment to the change and Benefits of the Change and the Affective Commitment to Change Scale (r = 0.369, P = .009) and staff attitudes toward the change (r = 0.355, P = .012). A significant positive correlation also exists between the Process subcategory and the Affective Commitment to Change Scale (r = 0.290, P = .045). To further explore the relationship, we used the median overall sustainability score to create a high-sustain group versus low-sustain group. We compared the difference in the affective commitment to Change, Participation, and Information scale scores in these 2 groups. Although no difference for affective commitment existed, respondents in the high-sustain group had significantly higher Participation (3.71 vs 2.53, P = .001) and Information (4.02 vs 3.06, P = .001) scale scores. In other words, staff members who perceived a higher likelihood that the changes would be sustained believed that they had more opportunities to participate in the change process and received more information about the change.
Table 3. Correlations among Survey Items.
| Affective Commitment to Change | Participation in the Change Process | Information About the Change | |
|---|---|---|---|
| Affective commitment to change | |||
| Participation in the change process | 0.372a | ||
| Information about the change | 0.301b | 0.733a | |
| Sustainability: Benefits of the change | 0.369a | 0.580a | 0.521a |
| Sustainability: Credibility of the benefits | 0.096 | 0.343b | 0.352b |
| Sustainability: Adaptability of the improved process | 0.234 | 0.443a | 0.273 |
| Sustainability: Effectiveness of the system to monitor progress | 0.203 | 0.490a | 0.297b |
| Sustainability: Staff involvement | 0.015 | 0.392a | 0.370b |
| Sustainability: Staff attitudes | 0.355b | 0.605a | 0.532a |
| Sustainability: Senior leadership | 0.116 | 0.473a | 0.214 |
| Sustainability: Clinical leadership | 0.161 | 0.531a | 0.583a |
| Sustainability: Fit within the organization | 0.163 | 0.545a | 0.351b |
| Sustainability: Infrastructure | 0.223 | 0.398a | 0.366b |
| Sustainability: Organization category | 0.220 | 0.511a | 0.400a |
| Sustainability: staff category | 0.212 | 0.662a | 0.582a |
| Sustainability: Process category | 0.290b | 0.588a | 0.469a |
| Total sustainability score | 0.254 | 0.671a | 0.560a |
Correlation is significant at the .01 level (2-tailed).
Correlation is significant at the .05 level (2-tailed).
Differences in sustainability perceptions: MHSR versus PC-MHI
We explored how staff perceptions about sustainability differed by comparing how staff members within the 2 VISNs differed in their perceptions about sustainability between the PC-MHI and MHSR initiatives. The analysis focused first on responses to the 10 individual questions in the SI and then explored the differences across the 3 subcategories in the SI: Process, Staff, and Organization. The PC-MHI respondents perceived their processes as significantly less likely to be consistent with sustainability than MHSR respondents. The results found significant differences between staff perceptions of sustainability for 1 question (Table 4). Staff involved in the PC-MHI initiative had a lower perception of the Benefits of the Change (PC-MHI = 3.60 vs MHSR = 5.38, P = .010). We looked closer at the response distribution for this question. The Figure shows that more MHSR respondents thought that the change improved efficiencies and made jobs easier whereas respondents in the PC-MHI initiative thought that the change did not improve efficiencies or make jobs easier. The overall difference in the distribution was significant (χ2 = 9.02, P = .029). The results also indicated that staff perceptions about the overall process of sustaining change (eg, the Process subcategory) were also significantly less for staff who participated in the PC-MHI initiative versus the MHSR initiative in the 2 VISNs (P = .49).
Table 4. Staff Sustainability Perceptions: A Comparison of Results from the PC-MHI Versus MHSR Initiatives.
| Average Response | Number of Respondents | |||||
|---|---|---|---|---|---|---|
|
|
|
|||||
| SI Category | SI Question | PC-MHI | MHSRD | PC-MHI | MHSRD | P |
| Process | Benefitsa | 3.60 | 5.38 | 51 | 44 | .010 |
| Credibilityb | 4.69 | 5.66 | 51 | 44 | .081 | |
| Adaptability | 4.51 | 5.18 | 51 | 44 | .174 | |
| Effectiveness | 4.15 | 4.29 | 49 | 44 | .781 | |
| Staff | Staff involvement | 6.26 | 6.58 | 48 | 42 | .702 |
| Staff attitudes | 4.49 | 5.53 | 50 | 44 | .229 | |
| Senior leadership | 8.40 | 9.91 | 50 | 44 | .180 | |
| Clinical leadership | 10.59 | 10.96 | 48 | 44 | .735 | |
| Organization | Organization | 4.16 | 4.57 | 49 | 43 | .449 |
| Infrastructure SI category | 4.26 | 4.90 | 49 | 41 | .392 | |
| Organization level | 8.42 | 9.53 | 49 | 41 | .322 | |
| Staff level | 29.88 | 33.00 | 46 | 42 | .321 | |
| Process levela | 17.03 | 20.50 | 49 | 44 | .049 | |
| Total sustainability | 55.50 | 63.79 | 46 | 39 | .151 | |
Abbreviations: MSHR, Mental Health System Redesign; PC-MHI, Primary Care and Mental Health Integration; SI, Sustainability Index.
Significant at P > .050.
Significant at P > .10.
Figure.
Staff response distribution for benefits to change question. MSHR indicates Mental Health System Redesign; PC-MHI, Primary Care and Mental Health Integration.
Although the results indicated no difference within the PC-MHI initiative by job classification, we also explored staff perceptions about sustainability across the 2 initiatives. Because of the small sample of clerical and other support staff (n = 4) and clinician administrators (n = 10), we excluded these groups from the analysis. The results found that administrators in the PC-MHI initiative have a significantly lower perception of whether the benefits of the change would make jobs easier and improve efficiency (μ = 3.92 vs μ = 6.35, P = .029). This same group was also less likely to believe that the process could be adapted to other organizational changes and that a system was in place to support continuous improvement of the PC-MHI efforts (μ = 3.91 vs μ = 5.82, P = .039). For the clinician-only group, the results yielded no significant differences between the 2 studies. The P values for each sustainability question were greater than .60 except for staff involvement (P = .21). In this case, the PC-MHI clinicians rated their involvement higher (6.53 vs 4.83) than their peers in the MHSR initiative within the 2 VISNs.
Discussion
Efforts to sustain changes associated with PC-MHI are challenging.4 As hypothesized, staff members in the PC-MHI initiative did not perceive their projects as having a strong potential for sustainability. They rated all 10 items in the SI as lower than did MHSR respondents. PC-MHI staff ratings of the process factor were significantly lower than those of MHSR staff. Also as hypothesized, staff members differed in their sustainability ratings, with psychiatrists and psychologists rating the benefits to patients and staff as less credible than social workers. We did find as hypothesized that some deficits existed in terms of affective commitment to change, but there was not a difference across programs and types of employees. Staff perceptions about participation in the change and information received about the change were positively correlated with sustainability and with affective commitment to change.
Prior to this study, we speculated that the reason for reported lags in PC-MHI implementation in the VA was due to the complexity of this change. Changes in scheduling, veteran behavior, primary care provider behavior, and mental health care provider behavior were all required. Complexity of the change process seems to affect staff perceptions regarding the likelihood that the changes will be sustained successfully. Staff members in the PC-MHI initiative were less likely than their colleagues in the MHSR initiative to believe that the changes will improve efficiency and make their jobs easier. The difference may be influenced in how the change projects were selected. Staff members in the MHSR initiative had more input into the selection of the change project and as a result, they, through participation, may have better understood how the change would improve overall efficiency. However, it is not clear how participation in the change process influenced staff perceptions about sustainability. Future research should explore this relationship.
Previous research indicated that staff (clinicians vs managers) had different opinions about whether the benefits of the change were immediately obvious and supported by evidence.3 The small sample size in this pilot study, combined with the fact that the majority of the respondents were clinicians, influenced the results. A comparison across the 2 initiatives within the 2 VISNs was a little more informative despite the small sample size. The results seem to show that intervention complexity and the top-down approach to change influence administrator perceptions about the benefits associated with and the adaptability of the change. From a clinician's perspective, the results may indicate that whether the intervention is complex and top down or simpler and bottom up has limited their belief in the sustainability of a new program or practice. It may also speak to the nature of the change efforts, as clinicians in the PC-MHI believed that they were more actively involved. Future research should explore how the directional implementation and change complexity individually and in conjunction influence staff perceptions of sustainability both within and across job classifications.
The results also indicate that the discipline of the respondent (eg, nurse vs social worker vs physician) may influence their perceptions about sustainability for several questions, including benefits beyond helping patients, effectiveness of the monitoring system, senior leadership support, and the infrastructure to support change. The small sample size limits the generalizability of these results. Future studies should evaluate whether any formative differences across disciplines exist with the VHA.
This is one of the first studies to explore the relationship between sustainability and employee commitment to change. Although limited by the sample size, results seem to imply that staff beliefs about the benefits of the change and their attitudes toward sustaining the change may affect their affective commitment to the change process. The literature on employee commitment to change indicates that commitment is a target of employee behavior and is influenced by antecedents associated with the change.14,31 Similarly, staff perceptions about sustainability could be considered a targeted behavior that an organization might want to influence. Exploratory regression analyses examined predictors of overall sustainability propensity and affective commitment to change. In each case, staff demographic variables (tenure, job, and discipline) as well as perceptions about their ability to participate in the change process and information received about the change process were independent variables. Participation in the change process was the only significant predictor of sustainability propensity (F = 38.6, P = .000) and affective commitment to change (F = 6.8, P = .012).
The relationship between higher staff perceptions of sustainability and higher participation and information scores is also important. It provides some evidence supporting the findings that successful efforts to the integration of primary care and mental health services are prevalent in the programs with high-functioning teams, a strong passion for service integration, and a solid primary care physician and behavioral health partnership.4 These results also provide evidence of the importance for organizations to adapt strategies to increase staff involvement in the change process and effectively share information about the impact of the change.23-27,32-35 However, the applicability of these findings is limited. Questions remain about the role of participation and information within a quality improvement effort. For example, it has been suggested that the source of information is critical and competing demands may influence quality improvement participation.20 Future research needs to explore how information types and sources as well as how levels of participation and competing demands influence commitment to change and employee perceptions of sustainability propensity.
Limitations
The small sample size is a limitation of this study (nonrandom/convenience). The sampling was limited only to staff who participated in the PC-MHI initiative within the 2 VISNs. As a result, the sampling process seems to have only solicited responses from clinicians and not clerical or administrative staff. The absence of these staff in the survey may have limited the discovery of differences in perceptions about sustainability exist across job categories. Since staff from only 2 VISNs were included in the sample, a future study involving a larger sample across multiple VISNs is needed to better generalize the findings within the VHA. The results also reflect staff perceptions of organizational change only within the VHA. Further research is needed to evaluate whether similar results are present in other health care delivery systems.
Conclusion
Sustainability is an important part of organizational change efforts. Results from this study highlight that efforts to sustain more complex changes such as PC-MHI are more challenging than projects targeting systems redesign, which often focus on one specific outcome such as wait times. It also indicates that employees' perception about the change process and their commitment to change are related to staff's perception of sustainability of organizational change. The role of change in the organization in improving efficiency and making jobs easier is a crucial component associated with staff perceptions of its benefits. These perceptions seem to be driven by the type of information that is received about change and opportunities to participate in the change process. In turn, information and participation may lead to increased employee commitment to change. Despite the promising findings, organizations will still need to explore how they can use information and participation to influence the likelihood that change will be sustained. The SI and employee commitment to change surveys are useful tools for this endeavor.
Acknowledgments
This material is based on work supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development Mental Health QUERI (RRP 09-161). Dr Ford's work was also supported in part by grant 5 R01 DA020832 from the National Institute of Drug Abuse. The authors thank Maureen Fitzgerald for her editorial assistance. They also appreciate the many employees of the Veterans Health Administration who took the time to participate in this study.
The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.
Dr Ford provides consulting services for the NIATx Foundation.
Dr Kirchner consults with Arkansas State University and has a grant pending with the University of Arkansas Health Sciences. For the remaining authors, none were declared.
References
- 1.Department of Veterans Affairs, Veterans Health Administration. VHA Handbook 1160.01. Washington, DC: Department of Veterans Affairs, Veterans Health Administration; 2008. [Google Scholar]
- 2.Ford JH, II, Krahn D, Wise M, Oliver KA. Measuring sustainability within the Veterans Administration Mental Health System Redesign initiative. Qual Manag Health Care. 2011;20(4):263–279. doi: 10.1097/QMH.0b013e3182314b20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Post EP, Metzger M, Dumas P, Lehmann L. Integrating mental health into primary care within the veterans health administration. Fam Syst Health. 2010;28(2):83–90. doi: 10.1037/a0020130. [DOI] [PubMed] [Google Scholar]
- 4.Pomerantz AS, Sayers SL. Primary Care-Mental Health Integration in healthcare in the Department of Veterans Affairs. Fam Syst Health. 2010;28(2):78–82. doi: 10.1037/a0020341. [DOI] [PubMed] [Google Scholar]
- 5.Pomerantz AS, Shiner B, Watts BV, et al. The White River model of colocated collaborative care: a platform for mental and behavioral health care in the medical home. Fam Syst Health. 2010;28(2):114–129. doi: 10.1037/a0020261. [DOI] [PubMed] [Google Scholar]
- 6.Wray LO, Szymanski BR, Kearney LK, McCarthy JF. Implementation of Primary Care-Mental Health Integration services in the Veterans Health Administration: program activity and associations with engagement in specialty mental health services. J Clin Psychol Med Settings. 2012;19:105–116. doi: 10.1007/s10880-011-9285-9. [DOI] [PubMed] [Google Scholar]
- 7.Zivin K, Pfeiffer PN, Szymanski BR, et al. Initiation of Primary Care-Mental Health Integration programs in the VA Health System: associations with psychiatric diagnoses in primary care. Med Care. 2010;48(9):843–851. doi: 10.1097/MLR.0b013e3181e5792b. [DOI] [PubMed] [Google Scholar]
- 8.United States Department of Veterans Affairs VHA Support Service Center (VSSC). Washington, DC. http://powerpointfree.net/vha-support-service-center-6856.ppt.
- 9.Oslin D, Ross J, Sayers S, Murphy J, Kane V, Katz I. Screening, assessment, and management of depression in VA primary care clinics. The Behavioral Health Laboratory. J Gen Inter Med. 2006;21:46–50. doi: 10.1111/j.1525-1497.2005.0267.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Zanjani F, Miller B, Turiano N, Ross J, Oslin D. Effectiveness of telephone based referral-care management, a brief intervention designed to improve engagement in depression and substance treatment. Psychiatr Serv. 2008;59(7):776–781. doi: 10.1176/ps.2008.59.7.776. [DOI] [PubMed] [Google Scholar]
- 11.Pomerantz A. Improving treatment engagement and integrated care for veterans. The primary mental health care clinic at the White River Junction VA Medical Center, Vermont 2005 APA Gold Award. Psychiatr Serv. 2005;56:1306–1308. [Google Scholar]
- 12.Greenberg GG, Pilver L, Desai R, editors. VAMC Dashboard Report: Status of Uniform Mental Health Services Handbook Implementation Primary Care-Mental Health Integration Dashboard. Washington, DC: Department of Veterans Affairs; Sep 12, 2011. Updated June 20 2011. [Google Scholar]
- 13.Kirchner JE, Parker LE, Bonner LM, Fickel JJ, Yano EM, Ritchie MJ. Roles of managers, frontline staff and local champions, in implementing quality improvement: stakeholders' perspectives. J Eval Clin Pract. 2012;18:63–69. doi: 10.1111/j.1365-2753.2010.01518.x. [DOI] [PubMed] [Google Scholar]
- 14.Herscovitch L, Meyer JP. Commitment to organizational change: extension of a three-component model. J Appl Psychol. 2002;87(3):474–487. doi: 10.1037/0021-9010.87.3.474. [DOI] [PubMed] [Google Scholar]
- 15.Chen J, Wang L. Locus of control and the three components of commitment to change. Pers Indiv Differ. 2007;42(3):503–512. [Google Scholar]
- 16.Michel A, Stegmaier R, Sonntag K. I scratch your back—you scratch mine. Do procedural justice and organizational identification matter for employees' cooperation during change? J Change Manage. 2010;10(1):41–59. [Google Scholar]
- 17.Conway E, Monks K. HR practices and commitment to change: an employee level analysis. Hum Resour Manage J. 2008;18(1):72–89. [Google Scholar]
- 18.Herold DM, Fedor DB, Caldwell S. The effects of transformational and change leadership on employees' commitment to a change: a multilevel study. J Appl Psychol. 2008;93(2):346. doi: 10.1037/0021-9010.93.2.346. [DOI] [PubMed] [Google Scholar]
- 19.Michaelis DB, Stegmaier R, Sonntag K. Affective commitment to change and innovation implementation behavior: the role of charismatic leadership and employees' trust in top management. J Change Manage. 2009;9(4):399–417. [Google Scholar]
- 20.Parker LE, Kirchner JAE, Bonner LM, et al. Creating a quality-improvement dialogue: utilizing knowledge from frontline staff, managers, and experts to foster health care quality improvement. Qual Health Res. 2009;19(2):229–242. doi: 10.1177/1049732308329481. [DOI] [PubMed] [Google Scholar]
- 21.Davies B, Edwards N. The action cycle: sustain knowledge use. In: Strauss S, Tetroe J, Graham ID, editors. Knowledge Translation in Health Care: Moving From Evidence to Practice. Oxford, England: Wiley-Blackwell; 2009. pp. 165–173. [Google Scholar]
- 22.Davies B, Tremblay D, Edwards N. Sustaining evidence based practice systems and measuring the impacts. In: Bick D, Graham ID, editors. Evaluating the Impact of Implementing Evidence Based Practice. Oxford, England: Wiley-Blackwell; 2010. pp. 165–188. [Google Scholar]
- 23.Johnson K, Hays C, Center H, Daley C. Building capacity and sustainable prevention innovations: a sustainability planning model. Eval Program Plann. 2004;27(2):135–149. [Google Scholar]
- 24.O'Loughlin J, Renaud L, Richard L, Sanchez Gomez L, Paradis G. Correlates of the sustainability of community-based heart health promotion interventions. Prev Med. 1998;27(05):702–712. doi: 10.1006/pmed.1998.0348. [DOI] [PubMed] [Google Scholar]
- 25.Parker LE, de Pillis E, Altschuler A, Rubenstein LV, Meredith LS. Balancing participation and expertise: a comparison of locally and centrally managed health care quality improvement within primary care practices. Qual Health Res. 2007;17(9):1268–1279. doi: 10.1177/1049732307307447. [DOI] [PubMed] [Google Scholar]
- 26.Savaya R, Spiro S, Elran-Barak R. Sustainability of social programs. Am J Eval. 2008;29(4):478–493. [Google Scholar]
- 27.Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. 2005;26(3):320–347. [Google Scholar]
- 28.Maher L, Gustafson D, Evans A. Sustainability. Leicester, England: British National Health Service Modernization Agency; 2004. [Google Scholar]
- 29.Wanberg CR, Banas JT. Predictors and outcomes of openness to changes in a reorganizing workplace. J Appl Psychol. 2000;85(1):132–142. doi: 10.1037/0021-9010.85.1.132. [DOI] [PubMed] [Google Scholar]
- 30.Miller VD, Johnson JR, Grau J. Antecedents to willingness to participate in a planned organizational change. J Appl Commun Res. 1994;22:59–80. [Google Scholar]
- 31.Meyer JP, Herscovitch L. Commitment in the workplace: toward a general model. Hum Resour Manag Rev. 2001;11:299–326. [Google Scholar]
- 32.Bateman N. Sustainability: the elusive element of process improvement. Int J Oper Product Manage. 2005;25(3):261–276. [Google Scholar]
- 33.Bray P, Cummings DM, Wolf M, Massing MW, Reaves J. After the collaborative is over: what sustains quality improvement initiatives in primary care practices? Jt Comm J Qual Patient Saf. 2009;35(10):502–508. doi: 10.1016/s1553-7250(09)35069-2. [DOI] [PubMed] [Google Scholar]
- 34.Jacobs RL. Institutionalizing organizational change through cascade training. J Eur Ind Train. 2002;26:177–182. [Google Scholar]
- 35.Solberg L. Lessons for non-VA care delivery systems from the US Department of Veterans Affairs Quality Enhancement Research Initiative: QUERI series. Implement Sci. 2009;4:9. doi: 10.1186/1748-5908-4-9. [DOI] [PMC free article] [PubMed] [Google Scholar]