Comparing Safety Climate between Two Populations of Hospitals in the United States

Abstract

Objective

To compare safety climate between diverse U.S. hospitals and Veterans Health Administration (VA) hospitals, and to explore the factors influencing climate in each setting.

Data Sources

Primary data from surveys of hospital personnel; secondary data from the American Hospital Association's 2004 Annual Survey of Hospitals.

Study Design

Cross-sectional study of 69 U.S. and 30 VA hospitals.

Data Collection

For each sample, hierarchical linear models used safety-climate scores as the dependent variable and respondent and facility characteristics as independent variables. Regression-based Oaxaca–Blinder decomposition examined differences in effects of model characteristics on safety climate between the U.S. and VA samples.

Principal Findings

The range in safety climate among U.S. and VA hospitals overlapped substantially. Characteristics of individuals influenced safety climate consistently across settings. Working in southern and urban facilities corresponded with worse safety climate among VA employees and better safety climate in the U.S. sample. Decomposition results predicted 1.4 percentage points better safety climate in U.S. than in VA hospitals: −0.77 attributable to sample-characteristic differences and 2.2 due to differential effects of sample characteristics.

Conclusions

Results suggest that safety climate is linked more to efforts of individual hospitals than to participation in a nationally integrated system or measured characteristics of workers and facilities.

Based on mounting evidence that better safety climate is related to lower incidence (Naveh, Katz-Navon, and Stern 2005; Hofmann and Mark 2006; Neal and Griffin 2006; Vogus and Sutcliffe 2007; Singer et al. 2008b;) and greater reporting (Cohen et al. 2004; Weingart et al. 2004; Gandhi et al. 2005;) of adverse events and to increased communication among managers and staff (Hofmann and Morgeson 1999), considerable effort among hospitals is being focused on improving safety climate. Along with hospitals' own efforts, several voluntary, collaborative initiatives that could improve safety climate (e.g., Leapfrog Group's patient safety leaps, and Institute for Healthcare Improvement's 5 Million Lives campaign) have garnered substantial participation among both public and private hospitals. “Benchmarking” of safety-climate survey results through participation in such collaborative initiatives is an effective way for hospitals to target quality improvement efforts. Benchmarking enables a hospital to compare its survey results with those of other hospitals, thereby facilitating identification of relative strength and weakness. It is being encouraged by numerous organizations. Since 2002, the Joint Commission's performance improvement standard (PI.01.01.01) has encouraged hospitals to collect data on staff perceptions of safety risks and improvement opportunities and to compare data with external sources (Joint Commission on Accreditation of Health Care Organizations 2002). The Agency for Healthcare Research and Quality (AHRQ) established the Hospital Survey on Patient Safety Culture Comparative Database for this purpose in 2006. Its 2009 database included safety-climate results from 622 hospitals (Sorra et al. 2009). Independent investigators engaged in benchmarking safety climate have also identified systematic differences in safety climate within and among hospitals, which provide clues to improving safety climate more generally (Singer et al. 2003; Thomas, Sexton, and Helmreich 2003; Makary et al. 2006; Sexton et al. 2006a, b, c; Vogus and Sutcliffe 2007; Hartmann et al. 2008; Singer et al. 2008a, 2009). Benchmarking safety-climate survey results across health care systems is difficult due to coordination challenges.

Our own effort to measure and benchmark safety climate is unique in that we used essentially the same survey instrument and sampling and administration procedures at approximately the same time in two discrete populations of hospitals in the United States: the Veterans Health Administration (VA) health care system and a national sample, excluding VA hospitals. This provided a novel opportunity to compare safety climate across two populations encompassing great diversity in both individual hospital characteristics and overall organizational structure, and potentially to identify any hospital features systematically related to safer care.

Differences in safety climate between VA and other U.S. hospitals may inherently exist because the VA is a nationally integrated network, while few U.S. hospitals come from large, integrated systems and none come from nationally integrated systems. As a system, the VA enjoys distinct advantages in broadly implementing and enforcing compliance with standardized safety activities. The VA conducts several initiatives with potential to improve safety climate. For example, the VA National Center for Patient Safety was established specifically to promote a systems approach to preventing and reducing harm to patients and to encourage hospitals to conduct root cause analyses after safety incidents (http://www.va.gov/ncps/vision.html, accessed on August 26, 2008, for NCPS).

In this paper, we examine differences in safety climate between 69 diverse U.S. hospitals and 30 VA hospitals using cross-sectional employee surveys. Given potential advantages in promoting strong safety climate in a nationally integrated network, we hypothesized that safety climate among VA hospitals would be stronger than among U.S. hospitals.

Differences in safety climate between U.S. and VA samples may arise from multiple sources. First, there may be variation between the two samples in measured characteristics associated with safety climate—for instance, one sample may contain more large hospitals than the other. The residual difference in safety climate between the two samples would be attributable to differential effects of the sample characteristics between the two health care systems (e.g., hospital size may impact VA hospitals differently than U.S. hospitals). We performed comparisons of safety climate in these two settings that allowed us to discern the relative impact of these potential sources of difference. We hypothesized that variance in observed sample characteristics would explain more of the difference in safety climate between U.S. and VA hospitals than would differential effects of sample characteristics on the two groups.

METHODS

Data Sources

We used the Patient Safety Climate in Healthcare Organizations (PSCHO) survey to collect data on employees' perceptions of safety climate. While various instruments exist to measure hospital safety climate (Colla et al. 2005; Flin et al. 2006;), the PSCHO instrument is the only one with established reliability and validity in both U.S. and VA hospital settings (Singer et al. 2007; Hartmann et al. 2008;). PSCHO survey items use a five-point, Likert scale ranging from “strongly agree” to “strongly disagree,” with a neutral midpoint. Items reflect 12 dimensions that capture various aspects of safety climate. We divided these dimensions into three categories, based on the extent to which they described hospital (e.g., “Organizational Resources for Safety”), work-unit (e.g., “Unit Safety Norms”), and interpersonal (e.g., “Fear of Blame and Punishment”) contributions to safety climate (Singer et al. 2007).

Because of modifications resulting from psychometric testing, two slightly different versions of the PSCHO survey were used in this study. In U.S. hospitals we used a 45-item instrument, while in the VA we used a 42-item instrument. The two versions have 41 common items, 39 of which map onto the 12 safety-climate dimensions. Both versions of the PSCHO also contained six close-ended demographic items.

Because the development of a strong safety climate necessitates a homogenous focus on preventing safety failures, the PSCHO instrument is scored to highlight responses opposed to safety, which we refer to as “problematic responses.” We generated scores for items, dimensions, and safety climate overall. First, we calculated the mean percent problematic response (“PPR”) for a given item across all respondents. We then calculated the mean of all item means in a dimension and the mean of all item means in the survey. A lower mean indicates a better perception of safety climate. This method of scoring identifies areas of nonuniformity in safety focus that are of potential concern and that might benefit from interventions to improve the safety climate.

Data for characteristics of respondents' jobs were obtained from the 2004 American Hospital Association (AHA) Annual Survey of Hospitals. Using these data, we determined hospitals' nurse staffing ratios, bed size, teaching status, national census region, and urban or nonurban location (see Table 1).

Table 1.

Respondents' Individual and Facility Characteristics

	U.S. (N=13,841)		VA (N=4,581)
Variables	N	%	N	%
Individual characteristics
Age category (years)
18–30	1,249	9.3	180	4.1
31–50	6,973	52.0	2,066	46.7
>50	5,197	38.7	2,179	49.3
Time at facility
≤10 years	7,817	57.6	2,319	51.8
>10 years	5,749	42.4	2,154	48.2
Male
Yes	4,708	35.8	2,150	49.8
No	8,451	64.2	2,170	50.2
Nurse
Yes	3,444	25.5	1,016	22.7
No	10,050	74.5	3,461	77.3
Senior manager
Yes	2,267	17.8	544	12.6
No	10,455	82.2	3,778	87.4
Employment in high hazard unit (OR, ER, ICU, PACU)
Yes	2,716	22.6	731	17.5
No	9,308	77.4	3,454	82.5
Facility characteristics*
Region
East	3,891	28.1	1,344	29.3
Midwest	2,628	19.0	1,234	27.0
South	3,663	26.5	1,201	26.2
West	3,659	26.4	802	17.5
Teaching status†
Major teaching	4,900	35.4	3,155	68.9
Minor teaching	3,176	23.0	1,045	22.8
Non-teaching	5,765	41.6	381	8.3
Urban‡
Yes	11,643	84.1	4,452	97.2
No	2,198	15.9	129	2.8
Bedsize
Small (≤99)	1,483	10.7	525	11.5
Medium (100–249)	2,166	15.7	981	21.4
Large (≥250)	10,192	73.6	3,075	67.1
Nurse staffing ratio§
Mean (SD)	12.1	(4.2)	9.7	(4.1)

Note. T-test (nursing staffing ratio) or χ² tests (all other variables) were conducted to test for differences between non-VA and VA respondents. p-values were significant (<.001) for all comparisons.

^*Facility characteristic variables were created using American Hospital Association Annual Survey Database FY'04.

^†“Major teaching” hospital category includes members of the Council of Teaching Hospitals of the Association of American Medical Colleges; “minor teaching” hospitals have residency training programs approved by the Accreditation Council for Graduate Medical Education or by the American Osteopathic Association; and “non-teaching” hospitals are neither of the above.

^‡AHA CBSA (Core-Based Statistical Area) type is “division” or “metropolitan.”

^§Full-time equivalent registered nurse hours per total facility inpatient days.

ER, emergency room; ICU, intensive care unit; OR, operating room; PACU, postanesthesia care unit; SD, standard deviation; VA, Veterans Health Administration.

Approval from relevant Institutional Review Boards was granted before conducting the studies.

Samples

We used stratified random sampling strategies in both populations. The U.S. hospital sample represented non-VA public and private acute-care hospitals, approximately equally divided among U.S. census regions and size categories. The VA sample represented a balanced geographic distribution of VA hospitals in four performance strata based on AHRQ's Patient Safety Indicators (PSIs) (low, medium, high, and other), to minimize selection bias. Details of the sampling strategies have been summarized elsewhere (Hartmann et al. 2008; Singer et al. 2009;).

Although we did not stratify the U.S. sample based on performance, the sample included 69 hospitals whose PSI rates were similar to those of all U.S. hospitals. Our recruitment strategy, however, dictated that average size and related characteristics would differ from the U.S. average (Singer et al. 2009). In addition, despite recruitment efforts, hospitals from the Midwest were underrepresented in our sample compared with the U.S. average.

The VA sample included 30 hospitals, including eight facilities each from the high, medium, and other PSI rate strata, and six from the low stratum. The VA facilities represented a balanced geographic distribution within each PSI stratum, with the exception of no low PSI hospitals in the West (Rosen et al. 2008).

Administration of Surveys

U.S. hospital survey administration took place from July 2006 to May 2007; the VA administration was conducted from December 2005 to May 2006. In both groups we sampled 100 percent of senior managers, defined as department head or above; 100 percent of active hospital-based physicians; and a random 10 percent of all other employees. Senior managers and physicians were over-sampled because of their relatively small numbers and their potentially low response rates, respectively.

For U.S. hospitals, we also sampled 100 percent of employees in three work areas in 12 larger hospitals with relatively high response rates in a 2004 survey administration so as to permit work-area-level analyses while maintaining respondent confidentiality. In these hospitals, we over-sampled employees in work areas that in 2004 were least likely to meet our 10-respondent minimum reporting requirement: laboratories (lab), operating rooms (ORs), and intensive care units (ICUs). Budget constraints drove this selection approach.

In the VA, to allow for analysis of work areas in which employees conduct work of intrinsically greater hazard, we also sampled 100 percent of employees in certain work areas in 10 randomly selected hospitals. The specific work areas were the OR, postanesthesia care unit, ICU, and emergency department. In this paper, we refer to these as “high hazard units” (HHUs).

The sampling frames in U.S. and VA hospitals consisted of 36,375 and 9,309 personnel, respectively. Both samples excluded individuals who no longer worked at the facility and those who used a survey response postcard to indicate that they did not wish to participate.

Analysis of Data

Weighting of Data

Two U.S. hospitals were excluded from analysis because AHA data suggested improbably high nurse staffing ratios. One VA hospital was dropped because it returned data for physicians only. For the remaining hospitals, we employed weighting techniques to reflect the two sampling frames accurately (Singer et al. 2003; Hartmann et al. 2008;). Identical weighting calculations were performed for each sample. First, we determined separate sampling and nonresponse weights. Regarding the latter, for the U.S. hospital sample we calculated a nonresponse weight for each workgroup (senior managers, physicians, and other employees) within each hospital. In VA hospitals, we calculated four nonresponse weights: for senior managers, physicians, HHU employees, and regular staff for each hospital. Then, in both samples, we multiplied the nonresponse and sampling weights and used the resulting “combined weight” to calculate a proportional weight that accounted for hospital size differences.

Statistical Analysis

For all analyses, the unit of analysis was the individual. Initially, we compared sample characteristics of respondents in U.S. and VA hospitals. We compared overall mean PPR in each hospital, graphically distinguishing hospitals from the U.S. and VA samples. We assessed internal consistency reliability for the 12 dimensions of the PSCHO instrument by calculating Cronbach's α coefficients for proposed dimensions for each sample. We compared average PPR among U.S. and VA hospitals for each item and dimension.

The dependent variable for all statistical models was PPR for each individual across all 39 PSCHO survey items, a summary measure we call “safety climate overall.” All models included variables describing individual respondents (i.e., gender, age, length of time at institution, job type, management category, and employment in HHU) and the facilities in which they worked (i.e., geographic region, hospital size, urban location, and nurse staffing ratio). Teaching status was not included in the models because major teaching status was correlated with large hospital size (r=0.5, p<.001) in both samples.

We examined the relationship between PPR and respondent characteristics in the United States and VA by estimating a separate hierarchical linear model (HLM) for each sample. To test the appropriateness of using two-level HLM to account for nesting of individuals within hospitals, we first ran random effects ANOVA “empty” models that included no independent variables (Snijders and Bosker 1999). Comparison of the two-level models with the linear regressions revealed significant differences at the hospital level in both samples (χ²=449 and 21 for U.S. and VA hospitals, respectively; both p<.001), indicating that there were meaningful differences in PPR among staff from different hospitals and that two-level random intercept models were preferred. The models did not assume that PPR was uniformly represented within a facility; rather, they allowed for variation within and across facilities at the individual level. We did not use three-level HLMs to account for work-area variance due to limitations of the work-area data.

To examine variance in observed sample characteristics between the U.S. and VA hospitals and differential effects of sample characteristics on PPR in the two groups, we conducted a regression-based decomposition approach developed by Oaxaca and Blinder (Blinder 1973; Oaxaca 1973;). Oaxaca–Blinder decomposition has been broadly applied in the economics literature and more recently in health services research (Kirby, Taliaferro, and Zuvekas 2006; Shen and Long 2006; Hudson, Miller, and Kirby 2007;). We refer to systematic variance between the two samples in characteristics associated with safety climate as the “sample-characteristics component” because it is explained by observable variation in sample characteristics. We estimated the sample-characteristics component by using the U.S. hospital model estimates as the reference model. The residual difference in PPR between the two samples is called the “unexplained component” and includes (a) differential effects of the sample characteristics in the model between the two health care systems, and (b) differences in unobserved factors such as differences in characteristics of patients. Thus, the sample-characteristics component in our Oaxaca–Blinder decomposition measures the expected difference in PPR assuming that the same model is applicable to both systems, and the unexplained component measures the extent to which the effects of observed and unobserved characteristics in the models differ between U.S. and VA hospitals. In other words, the unexplained component indicates how the U.S. and VA samples would differ if the distribution of the sample characteristics were exactly the same.

Analyses were conducted using Stata (version 9.2), including the Oaxaca module (Jann 2008) for the Oaxaca–Blinder decomposition.

RESULTS

Survey Response

Among the 67 U.S. hospitals studied, 13,841 individuals responded to the survey (41 percent). Response rates for individual hospitals ranged from 13 to 100 percent. Response also varied by type of personnel, with 62 percent of senior managers, 20 percent of physicians, and 50 percent of frontline employees responding.

For the 29 VA hospitals, we obtained an overall response rate of 50 percent (4,581 respondents). Response rates varied among hospitals (26–73 percent) and among personnel (69 percent for senior managers, 38 percent physicians, 38 percent HHU personnel, and 60 percent other staff).

Comparison of Sample Characteristics

All comparisons between the demographic characteristics of U.S. and VA samples revealed statistically significant differences (p<.001; Table 1). Respondents in U.S. hospitals were considerably younger than those in VA hospitals. U.S. hospital personnel also worked less time at their facility and were less likely to be male. They were more likely than VA personnel to work in HHUs and to be nurses and senior managers. In the U.S. sample, respondents more often worked in hospitals from the West, categorized as large, and with higher nurse staffing ratios. They were less often in major teaching hospitals and urban areas.

Safety-Climate Perceptions by Hospital

The graph displays the overall PPR and 95 percent confidence interval for each of the 96 hospitals in our study, displayed from lowest PPR (best safety climate) to highest (worst safety climate), differentiating between U.S. and VA hospitals. The range in safety climate among U.S. hospitals is larger than among VA hospitals, based on point estimates. In U.S. hospitals PPR varied from 7.7 to 24.5 percent, and in VA hospitals the range was 11.6–23.3 percent. More than twice as many VA hospitals fell in the bottom half of the distribution (n=21) than in the top (n=8). The results, however, place individual VA hospitals among both the top 10 and bottom 10 hospitals surveyed, and uncertainty in the point estimates suggests few meaningful differences between U.S. and VA hospitals (Figure 1).

Figure 1.

Overall Percent Problematic Response (PPR) by Hospital, U.S. and VA Hospitals 2006

Comparison of Safety-Climate Perceptions

Table 2 presents survey results by item and dimension. Cronbach's α's for all dimensions were within an acceptable range (0.6–0.8) except for “Fear of Shame” (0.4) and “Fear of Blame and Punishment” (0.5). The low reliabilities for the latter two scales reflect the reduced number of common items remaining after dropping those items that were not phrased identically in the U.S. and VA surveys. Results for these dimensions are presented because the domains represent potentially important aspects of safety climate; however, they should be regarded as tentative and interpreted with caution.

Table 2.

Mean Percent Problematic Response (PPR) among All Respondents by Item and Dimension: U.S. and VA^*

	Problematic Response Rate (Cronbach's α Coefficient)
Dimensions† and Text of Item	U.S.	VA
Hospital contributions to safety climate
Senior managers' engagement	14.7(0.80)	18.6(0.82)
Senior management has a clear picture of the risks associated with patient care.	14.2	17.7
Senior management has a good idea of the kinds of mistakes that actually occur in this facility.	17.6	20.4
Senior management supports a climate that promotes patient safety.	9.6	12.2
Senior management considers patient safety when program changes are discussed.	10.8	14.6
Patient safety decisions are made by the most qualified people, regardless of rank or hierarchy.	22.1	29.7
Good communication flow exists up and down the chain of command regarding patient safety issues.	13.8	17.2
Organizational resources for safety	13.6(0.65)	16.4(0.63)
I have enough time to complete patient care tasks safely.	18.1	20.5
I am provided with adequate resources (personnel, budget, and equipment) to provide safe patient care.	17.9	24.4
I have received sufficient training to enable me to address patient safety problems.	6.9	8.1
This facility devotes sufficient resources to follow up on identified safety problems.	11.4	12.5
Overall emphasis on patient safety	9.0(0.57)	9.7(0.57)
Compared with other facilities in the area, this facility cares more about the quality of patient care it provides.	10.6	11.0
Overall, the level of patient safety at this facility is improving.	7.4	8.4
Work-unit contributions to safety climate
Unit managers' support	18.8(0.59)	21.7(0.59)
Management in my unit helps me overcome problems that make it hard for me to provide safe patient care.	19.4	25.1
In my unit, management puts safety at the same level of importance as meeting the schedule and productivity.	19.8	19.6
Whenever pressure builds up, management in my unit wants us to work faster, even if it means taking shortcuts that might negatively affect patient safety.	17.2	20.5
Unit safety norms	9.9(0.57)	10.6(0.61)
My unit takes the time to identify and assess risks to ensure patient safety.	6.7	7.9
My unit does a good job managing risks to ensure patient safety.	5.7	6.9
In my unit, there is significant peer pressure to discourage unsafe patient care.	23.2	20.6
In my unit, anyone found to intentionally violate standards or safety rules is corrected.	7.2	10.0
Deliberate violations of standard operating procedures are rare in my unit.	6.6	7.5
Unit recognition and support for safety efforts	28.7(0.63)	31.0(0.64)
My unit recognizes safety achievement through rewards and incentives.	47.8	48.3
I am rewarded for taking quick action to identify a serious mistake.	34.8	39.1
My unit provides training on teamwork in order to improve patient care performance and safety.	22.0	24.4
My performance is evaluated against defined safety standards.	10.8	12.0
Collective learning	8.9(0.69)	10.3(0.70)
Mistakes have led to positive changes in my unit.	9.8	11.7
On my unit, we identify and fix safety problems before an incident actually occurs.	9.4	11.3
Our process of accident and incident investigation is effective at identifying root causes.	9.6	10.7
In my unit, patient safety problems and errors are communicated to the right people so that the problem can be corrected.	6.9	7.4
Psychological safety	12.2(0.63)	14.4(0.65)
Staff feel comfortable questioning the actions of those with more authority when patient safety is at risk.	19.0	22.1
Staff freely speak up if they see something that may negatively affect patient care.	10.3	11.2
I am comfortable reporting safety concerns without fear of being punished by management.	7.3	9.7
Problem responsiveness	12.5(0.69)	15.2(0.70)
Bringing patient safety concerns to management's attention usually results in the problem being addressed.	13.8	17.4
When I take the time to communicate about patient safety problems there is appropriate follow-up.	11.1	13.0
Interpersonal contributions to safety climate
Fear of shame	4.8(0.44)	5.3(0.41)
Asking for help is a sign of incompetence.	5.9	6.1
If I make a mistake that has significant consequences and nobody notices, I do not tell anyone about it.	3.6	4.4
Fear of blame and punishment	32.2(0.54)	23.1(0.53)
If people find out that I made a mistake, I will be disciplined.	35.6	25.9
Clinicians who make serious mistakes are usually punished.	28.8	20.4
Other aspects of safety climate
Provision of safe care	36.4(0.67)	36.0(0.63)
In the last year, I have witnessed a coworker do something that appeared to me to be unsafe for the patient.	31.1	29.7
I have never witnessed a coworker do something that appeared to me to be unsafe patient care.	41.7	42.2
Overall average‡	15.9	17.2

^*All means were calculated using weights.

^†Mean of all items in dimension averaged to calculate dimension mean.

^‡Overall means for U.S. and VA not significantly different from each other (t-test, p=.83).

VA, Veterans Health Administration.

The overall average PPR (i.e., mean of individual item means) was not significantly different in U.S. (mean=15.9, SD=1.61) and VA hospitals (mean=17.2, SD=1.56; p=.55). For 10 of the 12 individual dimensions, mean PPR was lower in U.S. than in VA hospitals. However, a smaller percentage of VA than U.S. respondents indicated fear of blame or punishment and that they witnessed or participated in unsafe care.

Relationship of Safety Climate and Sample Characteristics

All respondent characteristics, with the exception of time worked in the hospital, related significantly and in the same direction to safety climate overall in both samples (Table 3a). Being male and a nurse were positively related to PPR (worse safety climate), while age >50, being a senior manager and working in an HHU were negatively correlated with PPR (better safety climate). The magnitude of these correlations, however, differed somewhat by sample. PPR among men was higher than among women by 0.8 and 2.6 percentage points in U.S. and VA hospitals, respectively. PPR among senior managers was lower than among nonsenior managers by 4.6 percentage points in U.S. hospitals and by 7.3 percentage points in the VA.

Table 3a.

Association of Individual Mean Percent Problematic Response (PPR) with Individual and Facility Characteristics

	Hierarchical Linear Models Coefficient (SE)
Variable	U.S.		VA
Individual characteristics
Male	0.843*	−0.37	2.628**	−0.63
Age >50 years	−1.990**	−0.34	−1.724**	−0.6
Time at facility >10 years	0.253	−0.33	1.115	−0.6
Being nurse	3.717**	−0.34	4.616**	−0.66
Being senior manager	−4.596**	−0.87	−7.250**	−1.82
Employment in HHU	3.078**	−0.38	2.739**	−1.01
Facility characteristics
Region South	−0.947*	−0.42	2.970**	−0.93
Region Midwest	−2.596**	−0.47	−1.401	−0.91
Region East	−1.547**	−0.4	−0.919	−0.88
Bedsize (large)	1.731**	−0.36	1.068	−0.72
Urban location	−0.381	−0.4	2.860**	−1.08
Nurse staffing ratio	−0.119**	−0.04	0.051	−0.1
Intercept	15.71**	−0.58	10.88**	−1.94

^**p<.01

^*p<.05.

HHU, high hazard unit; SE, standard error; VA, Veterans Health Administration.

In contrast, characteristics of the facilities in which respondents worked related to safety climate in considerably different ways in the two samples. For U.S. hospitals, all facility characteristics with the exception of urban location related significantly to safety climate. For example, larger size was associated with higher PPR while a higher nurse staffing ratio was associated with lower PPR. In the VA, only working in the South and in an urban location were significantly correlated with safety climate, and in both instances the direction of correlation was opposite that of U.S. hospitals. VA employees working in the South had higher PPR than VA employees in the West. However, employees in Western U.S. hospitals had higher PPR than all other regions. VA employees working in urban hospitals had higher PPR than employees working in nonurban locations; the opposite was true for U.S. hospital employees.

Oaxaca–Blinder Decomposition of Safety-Climate Results

The Oaxaca–Blinder decomposition analysis allowed us to quantify the extent to which the predicted difference in PPR between the U.S. and VA hospitals was due to (a) variation in the two systems' distributions of sample characteristics (sample characteristics component) and (b) differences in the U.S. and VA model characteristics as expressed by the values of the coefficients (unexplained component). The model calculated the difference between sample characteristics and unexplained components based on the coefficients from the U.S. hospital model for each variable and the difference between the distributions of each characteristic for each sample. That is, gives the effect of the difference in characteristic X between U.S. and VA hospitals on the predicted PPR, using the U.S. hospital model (i.e., ) as the reference.

The net difference in safety climate, based on the predicted means of the U.S. and VA models, was 1.39 higher predicted PPR for the VA (Table 3b). Some of this difference was attributable to observed sample characteristics. For example, male respondents had higher PPR in VA than U.S. hospitals (Table 3a). Because the VA also had more male respondents, the result was a higher predicted PPR for the VA sample—on average by 0.108 percentage points (Table 3b). On the other hand, the impact of the VA sample having more respondents older than 50 than the U.S. sample was that the VA's predicted PPR for this factor was −0.298, because this characteristic was associated with lower PPR. The aggregate impact of variation between U.S. and VA hospitals in the distribution of sample characteristics was −0.766, suggesting that the predicted VA PPR should be 0.77 percentage points lower than that for U.S. hospitals (based on the U.S. hospital sample as the reference). However, the unexplained component accounted for a larger portion of the difference in safety climate between U.S. and VA hospitals than the sample characteristics component. The differential effects of observed characteristics (i.e., differences in model coefficients) plus differences in unobserved characteristics predicted average PPR in the VA to be 2.160 percentage points higher than in U.S. hospitals.

Differences in U.S. and VA Safety Climate, Decomposition Results

	Mean	[95% CI]/(SE)
Predicted mean PPR for
Non-VA	15.3	[16.1–17.2]
VA	16.7	[14.9–15.6]
Total difference between U.S. and VA (based on predicted means)	1.39**	−0.33
Difference attributable to variation in†
Male	0.108*	−0.048
Age >50 years	−0.298**	−0.054
Time at facility >10 years	0.036	−0.046
Being nurse	−0.115**	−0.034
Being senior manager	0.032*	−0.015
Employment in HHU	−0.373**	−0.05
Region South	0.041*	−0.02
Region Midwest	−0.194**	−0.04
Region East	−0.147**	−0.041
Bedsize (large)	−0.028	−0.017
Urban location	−0.066	−0.07
Nurse staffing ratio	0.237**	−0.07
Difference attributable to variation in sample characteristics	−0.766**	−0.16
Difference attributable to differences in model coefficients (differential effects of sample characteristics)	2.160**	−0.35

^**p<.01

^*p<.05.

^†These use U.S. model estimates from Table 3a as reference.

CI, confidence interval; HHU, high hazard unit; PPR, percent problematic response; SE, standard error; VA, Veterans Health Administration.

DISCUSSION

This study is the first to compare hospital safety climate between two fundamentally different sets of hospitals: one a nationally integrated hospital network, the other predominantly independent general acute-care hospitals. The study summarizes safety climate in the VA and other U.S. hospitals and factors influencing safety climate in each setting. Results also show how sample characteristics contribute to differences in safety climate between settings.

Overall, we found no difference in safety climate between U.S. and VA hospitals on average, based on descriptive statistics. Differences with respect to specific dimensions were significant, generally favoring U.S. hospitals. However, the range in safety-climate results among U.S. hospitals substantially overlapped, suggesting that neither population has achieved superior safety climate. In addition, relative to high reliability organizations, such as naval aviation, which serve as the “gold standard” for safety achievement despite hazardous and demanding conditions, safety climate in both U.S. and VA settings was considerably worse (Gaba et al. 2003). This finding does not support our first hypothesis, that participating in a nationally integrated hospital network would be associated with stronger safety climate. It appears that potential advantages associated with the system's intense focus on safety improvement and its ability to implement uniformly its improvement program may have been outweighed by local considerations. While institutional programs may facilitate the ability of local managers to improve safety, they may not be targeted closely enough to the actual challenges of the workplace to make a difference alone.

We also found that characteristics of individuals influenced safety climate consistently across settings when controlling for other factors. Older age and more seniority corresponded to more positive perceptions of safety climate, while working as a nurse or in an HHU were associated with more negative perceptions. These findings are consistent with studies showing perceptions of safety climate differ by workgroup and management level (Pronovost et al. 2003; Sexton et al. 2006c; Singer et al. 2008a, 2009). In contrast, facility characteristics influenced safety climate differently in U.S. and VA samples. Working in southern and urban facilities corresponded with higher PPR among VA employees and lower PPR in the U.S. sample. Other studies have found similarly mixed results regarding effects of geographical and structural characteristics within non-VA hospitals (Baldwin et al. 2004; Coburn et al. 2004; Loux, Payne, and Knott 2005; Longo et al. 2007;). Also consistent with prior studies (Aiken et al. 2002; Stone et al. 2007; Weissman et al. 2007;), we found that higher nurse staffing ratios were associated with lower PPR in U.S. hospitals.

Decomposition analysis examined the influence of (1) variation in the distribution of observed sample characteristics among personnel in an integrated network compared with other U.S. hospitals and (2) differential effects of sample characteristics in each group. The overall difference between the samples, that is, the influence of (1) and (2) together, was a 1.4 percentage point higher PPR for the VA. We hypothesized that variations in sample characteristics between settings would explain more of this difference in safety climate than would differences in effects of those sample characteristics. Our results do not support this hypothesis. Instead, it was the differential effects of sample characteristics that explained more of the difference in safety climate between U.S. and VA hospitals. The difference based on the distribution of all the VA sample characteristics compared with U.S. characteristics was negative, indicating that the VA would be expected to have a 0.77 percentage point lower PPR based on observed sample characteristics alone. The unexplained difference, indicating the differential effect of sample characteristics, was 2.2 percentage points higher PPR in VA than in U.S. hospitals. This second difference was driven primarily by two factors: region and location, both of which act in opposite directions on PPR in the U.S. and VA models, and by unobserved characteristics. Decomposition of the residual suggests that our model explained just 5.9 percent of the variation in the outcome measure. Future research should explore additional characteristics of hospitals and factors driving the effects of region and location in order to determine whether some modifiable factors may be involved that could provide leverage for change.

Our results suggest that characteristics of respondents and their work facilities influence safety-climate scores. Thus, in comparing safety climate among hospitals or over time in hospitals whose respondent characteristics may have changed, it is important to include known characteristics in analyses. Such longitudinal studies would also provide opportunity for research on how the effects of respondent characteristics on PPR change over time.

Results should be interpreted within the context of several limitations. This was a cross-sectional study; thus, we cannot make assertions about causality. We cannot explain the mechanisms underlying effects of various factors on safety climate. Nor can we differentiate the effect on safety climate of observed from unobserved characteristics in the unexplained component of the difference between samples. We cannot rule out nonresponse bias as a factor in our results. The methodology in both settings aimed to maximize response rates while maintaining the voluntary and anonymous nature of the surveys. While the VA sample achieved a response rate that is similar to that of other studies of this type (Asch, Jedrziewski, and Christakis 1997; Jepson et al. 2005;), the overall response rate in the U.S. sample was lower. We adjusted for nonresponse and sampling bias through the use of weights in our analysis; however, it is possible that results do not accurately represent the facilities or populations intended. A related issue is the representativeness of the hospitals in each sample. We conducted a stratified random sampling strategy in both settings, but since participation was voluntary, sampled facilities may differ from facilities in their respective populations in unanticipated ways. As noted, administration dates and recruitment and sampling strategies also differed slightly between U.S. and VA samples. Although recruited on the basis of size and region rather than PSI rates, those rates among the U.S. hospital sample did not differ from those of U.S. hospitals overall. In addition, within the U.S. hospital sample we found no difference when we compared overall mean PPR between over-sampled hospitals and the other hospitals in that sample. Finally, while our models included variables associated with safety climate in the literature, we were limited by variables available in our datasets.

Nevertheless, the methodology employed in our study represents an advance over prior research. In particular, the decomposition analysis provides information about systematic differences in sample characteristics and the effects of specific characteristics on safety climate in different settings. By achieving a more thorough understanding of what is driving apparent differences in safety-climate survey results among hospitals we can proceed more clearly toward developing effective improvement interventions.

The results presented suggest that continued efforts are needed to improve safety climate in hospitals. While participation in systems can provide some advantages in this regard, the large unexplained component of safety climate from the regression estimates suggests that other factors, such as hospitals' emphasis on creativity and innovation and their leaders' abilities to motivate, implement, and sustain improvement, may matter more.

Supporting Information

Additional supporting information may be found in the online version of this article:

Appendix SA1: Author Matrix.

Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.