Abstract
Aim
To explore the factor structure, reliability, and potential usefulness of a patient safety climate questionnaire in UK health care.
Setting
Four acute hospital trusts and nine primary care trusts in England.
Methods
The questionnaire used was the 27 item Teamwork and Safety Climate Survey. Thirty three healthcare staff commented on the wording and relevance. The questionnaire was then sent to 3650 staff within the 13 NHS trusts, seeking to achieve at least 600 responses as the basis for the factor analysis. 1307 questionnaires were returned (36% response). Factor analyses and reliability analyses were carried out on 897 responses from staff involved in direct patient care, to explore how consistently the questions measured the underlying constructs of safety climate and teamwork.
Results
Some questionnaire items related to multiple factors or did not relate strongly to any factor. Five items were discarded. Two teamwork factors were derived from the remaining 11 teamwork items and three safety climate factors were derived from the remaining 11 safety items. Internal consistency reliabilities were satisfactory to good (Cronbach's alpha ⩾0.69 for all five factors).
Conclusions
This is one of the few studies to undertake a detailed evaluation of a patient safety climate questionnaire in UK health care and possibly the first to do so in primary as well as secondary care. The results indicate that a 22 item version of this safety climate questionnaire is useable as a research instrument in both settings, but also demonstrates a more general need for thorough validation of safety climate questionnaires before widespread usage.
Keywords: patient safety, safety climate questionnaire, safety culture, teamwork
As health care comes to be seen as a potentially high risk environment, there is increasing pressure to assess the safety culture of healthcare organisations. Some authors promote the use of semi‐structured or qualitative approaches to assessing culture,1 while others suggest that culture can be assessed using a questionnaire approach.2,3
There is a real debate over how effectively culture can be assessed using climate questionnaires. As described by Denison,4 culture refers to the “deep structure of organisations” (values, beliefs and behaviours) and is traditionally assessed through interviews or observation, whereas climate mainly concerns individuals' perceptions of their work environment (policies, practices and procedures) at a particular point in time5,6,7,8 and is amenable to measurement by questionnaire.9 Despite this debate, safety climate questionnaires have been used to assess safety culture in many safety critical industries such as nuclear power, aviation10 and petrochemicals.11 Some questionnaires for use in health care have been derived from work in other industries such as aviation12 or have included several items validated in other settings—for example, the armed forces.13 Most of the available healthcare safety climate questionnaires have been developed in the United States.
It has been suggested that safety climate questionnaire data may be used as an indicator of aspects of the underlying safety culture,14 while others have promoted use of these data to evaluate safety programmes or track changes over time in a healthcare setting.15,16 Now a number of US developed safety climate questionnaires can be accessed via the internet1,16,17 and are beginning to be used by US and UK healthcare organisations.
Although there is a slowly building research literature on the scientific basis for measuring safety culture in health care, there has been a warning that “the enthusiasm for measuring culture may be outpacing the science”.2 This may particularly be the case where there has been limited testing of how consistently the questionnaires measure specific domains of safety, and also of whether questionnaires developed in one environment are valid for use in another healthcare setting. Such environmental differences might be at the national level—for example, whether a US developed questionnaire has the same meaning in a UK setting—or at the organisational level—for example, whether a questionnaire designed for acute hospital care can also be used in primary care/general practice.
The aim of this study was to explore these questions by investigating the factor structure, internal reliability, and usefulness of a US developed safety climate questionnaire in a UK healthcare setting. An additional aim was to explore the relevance of the same questionnaire in both primary and secondary care. The study sought to achieve enough responses from each of the two settings to be able to undertake an analysis of the underlying factor structure of the questionnaire items (rather than attempting to assess the actual safety culture of the individual organisations which would require a response rate of at least 50–60%).
Methods
Selection of instrument
A review of available patient safety climate questionnaires was undertaken in late 2003. Selection criteria were that (1) there was some public domain evidence base concerning development, (2) the instrument measured safety climate at the clinical team or directorate level, and (3) the instrument was short enough for use by busy health professionals. Using these criteria there was a limited selection from which to choose, some still being in the final development stage13,15,17 and others having limited peer reviewed data available, including the “family” of the approximately 65 item Safety Attitude Questionnaire (SAQ),2,12,18 the 19 item Safety Climate Survey,3,16,19 and the 27 item Teamwork and Safety Climate Survey.18
Using our selection criteria, we identified two of the shorter instruments (the Stanford University Patient Safety Climate in Healthcare Organizations questionnaire13 and the Teamwork and Safety Climate Survey18) as possibly useful among frontline NHS staff.
Thirty three healthcare professionals, 16 from primary care and 17 from acute hospital care, were asked to complete these two questionnaires and comment on their understanding of each item in the manner of a “thinking aloud” protocol.20,21 On the basis of this work, we concluded that the 27 item Teamwork and Safety Climate Survey contained a greater number of items that were applicable to frontline clinical teams; this instrument was therefore taken forward for further study. The original one page format was retained and UK related demographic questions added.
The sample
The purpose of using the questionnaire in a single round study was to provide sufficient responses on which to carry out factor analysis and reliability analysis of the questionnaire structure. At least 300 responses were required for the factor analysis of each of the healthcare staff populations (acute care and primary care), a minimum of 600 in all. The sample size for the study was calculated accordingly, with the anticipation that the response rate may be as low as 20%.
Staff from 13 healthcare organisations were invited to take part. In the four acute hospital trusts a random sample of 1900 recipients was drawn from staff involved in direct patient care, management, clinical support services, or patient contact administration roles. The Teamwork and Safety Climate questionnaires were distributed through the hospital postal systems and returned in a reply paid envelope direct to the study team.
In the nine primary care trusts the questionnaire was sent to 1750 staff including general practitioners, practice nurses and practice managers, and a sample of other staff (community nurses, health visitors, school nurses and allied health professionals).
Analysis of data
Factor analysis explores the extent to which individual items in a questionnaire can be grouped together according to the correlations between the responses to them, hence reducing the dimensionality of the data. If a questionnaire is to have construct validity, the items should measure key underlying concepts (or factors) in a coherent way; items successfully measuring the same underlying factor should consistently generate similar responses to each other. The resulting groups of items can then be examined to interpret the meaning of the factors.
The questionnaire contained two sections: “teamwork” and “safety climate”.18,25 An initial exploratory factor analysis on all 27 items showed that the teamwork questions factored out separately from the safety climate questions, but that there were multiple factors within each section. Exploratory factor analysis was therefore undertaken separately on each of the two domains of the questionnaire: teamwork (14 items) and safety climate (13 items). This was carried out on a random 50% sample of respondents (the “construction” half of the data) using principal components extraction. The number of factors to be extracted was determined using the Kaiser criterion (eigenvalues >1) in conjunction with assessment of scree plots (the former method has a tendency to over‐extract but in this case the two methods suggested identical solutions). Oblique rotation was used to aid interpretation of the resulting factor loadings. An optimal factor structure was derived and the internal consistency reliabilities of the resulting factors were assessed using Cronbach's alpha. Analysis was performed in Mplus using maximum likelihood estimation.
Confirmatory factor analysis was then undertaken to assess the fit of the proposed factor structure to the remaining 50% of the dataset. This was examined using a number of fit indices—CFI (comparative fit index), TLI (Tucker‐Lewis index), RMSEA (root mean square error of approximation) and SRMR (standardised root mean square residual)—in addition to the model χ2 statistic. Missing items were listwise deleted and items were treated as continuous variables.
To assess how well the factor model separately fitted each of the primary care and hospital datasets, and also whether there was a difference in the level of reported teamwork and safety climate between the two groups, a test of factorial invariance and population homogeneity was carried out. This consisted of performing a series of confirmatory factor analyses within which successive model parameters (interfactor correlations, scale means, variances and factor loadings) were allowed to vary between the primary and secondary care groups. At each stage the results were examined to determine whether allowing the parameters to differ between the groups had improved the fit of the model (which would suggest a difference between the groups either in the factor structure or in the level of teamwork/safety climate). In addition, a confirmatory factor analysis was carried out on each of the primary and secondary care groups separately.
Research governance
External scientific review was provided by the Sheffield Health and Social Research Consortium. Ethical review was provided by North Sheffield Research Ethics Committee and research governance approval sought from each participating organisation.
Results
Face validity
As a result of the “thinking aloud” exercise, minor adaptations were made to the questionnaire wording before it was used in the survey. For example, “institution” was changed to “organisation” and “physicians” to “doctors”. However, care was taken not to alter the underlying meaning of the items and, for this reason, some wording was left unchanged—for example, the term “briefings” (which was unfamiliar to a number of respondents) and “medical error” (which several respondents associated only with doctors/medical interventions).
Survey response rates
Since the aim of the study was primarily to examine the factor structure, a single round posting was used (with no follow up requests to non‐responders). 1307 responses were received, the overall response rate being 36% (33% for primary care trusts and 38% for acute hospital trusts). The number of responses and associated response rates were felt to be sufficient for factor analysis since non‐responders were unlikely to differ from respondents in terms of the pattern of relationship between their responses to different questions (that is, the factor structure).
Applicability of items to staff groups
A number of responses were obtained from staff not involved in direct patient care, many of whom answered “not applicable” to several questions. Factor analysis was therefore carried out only on responses from the 897 staff involved in direct patient care, which included 237 hospital nurses, 187 primary care nurses, 51 hospital doctors, and 152 GPs.
Many questions, particularly in the teamwork domain, exhibited very weak discrimination with a high proportion of respondents answering “agree” or “strongly agree”. Item responses are shown in tables 1–3 and the skew of the distributions in table 4.
Table 1 Teamwork factors and % responses to items.
| Disagree strongly* | Disagree slightly* | Neutral* | Agree slightly* | Agree strongly* | Not applicable† | |
|---|---|---|---|---|---|---|
| Teamwork factor 1: Input into decisions and collaboration with other staff (Cronbach's α = 0.84) | ||||||
| Nurse input is well received where I work. | 1 | 3 | 6 | 23 | 66 | 4 |
| Decision making where I work uses input from relevant staff | 3 | 4 | 9 | 29 | 54 | 2 |
| The doctors and nurses here work together as a well coordinated team | 3 | 8 | 9 | 33 | 47 | 4 |
| Disagreements where I work are resolved appropriately (i.e. not who is right, but what is best for the patient) | 4 | 7 | 16 | 35 | 38 | 2 |
| It is easy for staff here to ask questions when there is something that they do not understand | 1 | 3 | 5 | 24 | 67 | 1 |
| I have the support I need from other staff to care for patients | 1 | 5 | 5 | 27 | 62 | 1 |
| I am satisfied with the quality of collaboration that I experience with senior doctors where I work | 5 | 9 | 12 | 35 | 40 | 11 |
| Teamwork factor 2: Information handover (Cronbach's α = 0.69) | ||||||
| I know the first and last names of all the staff I worked with during my last shift/period of work | 4 | 5 | 3 | 14 | 74 | 3 |
| Important issues are well communicated at shift changes/between periods of work | 3 | 7 | 11 | 37 | 42 | 11 |
| Briefings are common where I work | 4 | 7 | 13 | 34 | 41 | 8 |
| I am satisfied with the quality of collaboration that I experience with nurses where I work | 2 | 4 | 5 | 32 | 57 | 4 |
*Percentage of valid responses (not including “not applicable”) from direct patient care staff.
†Percentage of total responses from direct patient care staff.
Table 2 Safety climate factors and % responses to items.
| Disagree strongly* | Disagree slightly* | Neutral* | Agree slightly* | Agree strongly* | Not applicable† | |
|---|---|---|---|---|---|---|
| Safety climate factor 1: Attitudes to safety within own team; capacity to learn from errors (Cronbach's α = 0.73) | ||||||
| I am encouraged by my colleagues to report any patient safety concerns I may have | 1 | 4 | 7 | 28 | 60 | 1 |
| The culture where I work makes it easy to learn from the errors of others | 4 | 7 | 21 | 35 | 32 | 3 |
| I receive appropriate feedback about my performance | 8 | 13 | 17 | 34 | 28 | 1 |
| Medical errors are handled appropriately here | 1 | 4 | 18 | 27 | 50 | 5 |
| I know the proper channels to which I should direct questions regarding patient safety | 1 | 4 | 6 | 30 | 59 | 1 |
| Safety climate factor 2: Overall confidence in safety of organisation (Cronbach's α = 0.70) | ||||||
| The levels of staffing where I work are sufficient to handle the number of patients | 22 | 23 | 11 | 25 | 19 | 2 |
| I would feel safe being treated as a patient in this service | 5 | 10 | 11 | 27 | 47 | 1 |
| Management does not knowingly compromise the safety of patients | 6 | 6 | 14 | 22 | 51 | 2 |
| Safety climate factor 3: Perceptions of management's attitudes to safety (Cronbach's α = 0.78) | ||||||
| This organisation is doing more for patient safety now than it did one year ago | 4 | 5 | 36 | 28 | 27 | 5 |
| Leadership is driving us to be a safety centred organisation | 4 | 7 | 32 | 34 | 23 | 3 |
| My suggestions about safety would be acted upon if I expressed them to management | 5 | 6 | 16 | 38 | 35 | 2 |
*Percentage of valid responses (not including “not applicable”) from direct patient care staff.
†Percentage of total responses from direct patient care staff.
Table 3 Responses (%) to items omitted from final factor analysis.
| Disagree strongly* | Disagree slightly* | Neutral* | Agree slightly* | Agree strongly* | Not applicable† | |
|---|---|---|---|---|---|---|
| Teamwork items | ||||||
| Where I work, it is difficult to speak up if I perceive a problem with patient care‡ | 6 | 12 | 4 | 23 | 55 | 2 |
| I am frequently unable to express disagreement with the senior clinical staff here‡ | 9 | 13 | 14 | 27 | 37 | 6 |
| Briefing staff on handovers between shifts/periods of work (i.e. to plan for possible contingencies) is important for patient safety. | 0.4 | 0.3 | 3 | 12 | 84 | 15 |
| Safety climate items | ||||||
| Staff frequently disregard rules or guidelines (e.g. hand‐washing, treatment protocols/clinical pathways, etc) that are established for the area where I work‡ | 7 | 10 | 8 | 22 | 53 | 2 |
| Where I work, it is difficult to discuss errors‡ | 4 | 11 | 9 | 30 | 46 | 1 |
*Percentage of valid responses (not including “not applicable”) from direct patient care staff.
†Percentage of total responses from direct patient care staff.
‡These items are negatively worded.
Table 4 Mean score for all items in each factor (scale).
| Factor | Mean score | 95% CI |
|---|---|---|
| Teamwork factor 1 | ||
| Input into decisions and collaboration with other staff | 4.25 | 4.21 to 4.30 |
| Teamwork factor 2 | ||
| Information handover | 4.24 | 4.19 to 4.29 |
| Safety climate factor 1 | ||
| Attitudes to safety within own team; capacity to learn from errors | 4.10 | 4.05 to 4.14 |
| Safety climate factor 2 | ||
| Overall confidence in safety of organisation | 3.68 | 3.61 to 3.75 |
| Safety climate factor 3 | ||
| Perceptions of management's attitudes to safety | 3.75 | 3.69 to 3.81 |
Exploratory factor analysis
Exploratory factor analysis initially suggested a three factor solution for the teamwork domain and a three‐factor solution for the safety climate domain. However, the four negatively worded items consistently formed a separate factor. This occurred when analysing the teamwork and safety climate domains separately and when analysing all 27 items together (see table S1 available online at http://www.qshc.com/supplemental). Other authors22,23 have suggested that artificial factors can occur as a result of respondents misreading negative items. Some of the staff we spoke to also commented that they had almost misread these items, especially since only a small proportion of items were reverse worded. These items were therefore removed from the analysis at this stage (table 3).
Exploratory factor analysis was then carried out on the remaining items in each of the teamwork and safety climate domains. The optimal solution for the teamwork domain contained two factors which together explained 50% of the variance of the 11 items (table 1). This suggested two underlying themes which were interpreted as representing: (1) input into decisions and collaboration with other staff, and (2) information handover.
The best solution for the safety climate domain gave three factors explaining 61% of the variance (table 2). From their item loadings, these were interpreted as representing: (1) attitudes to safety within own team and capacity to learn from errors, (2) overall confidence in safety of the organisation, and (3) perceptions of management's attitudes to safety. The factor loadings (a measure of how strongly each item relates to each factor) are shown in tables 5 and 6.
Table 5 Optimum factor loadings of teamwork items*†.
| Teamwork items | Factor loadings | |
|---|---|---|
| Teamwork factor 1 | Teamwork factor 2 | |
| Input into decisions and collaboration with other staff | Information handover | |
| Teamwork factor 1 | ||
| Nurse input is well received where I work | 0.697 | |
| Decision making where I work uses input from relevant staff | 0.675 | |
| The doctors and nurses here work together as a well coordinated team | 0.830 | |
| Disagreements where I work are resolved appropriately (i.e. not who is right, but what is best for the patient) | 0.716 | |
| It is easy for staff here to ask questions when there is something that they do not understand | 0.439 | 0.365 |
| I have the support I need from other staff to care for patients | 0.510 | 0.420 |
| I am satisfied with the quality of collaboration that I experience with senior doctors where I work | 0.779 | |
| Teamwork factor 2 | ||
| I know the first and last names of all the staff I worked with during my last shift/period of work | 0.615 | |
| Important issues are well communicated at shift changes/between periods of work | 0.654 | |
| Briefings are common where I work | 0.456 | |
| I am satisfied with the quality of collaboration that I experience with nurses where I work | 0.532 | |
*Pattern matrix generated using principal components extraction and oblique rotation.
†Factor loadings under 0.300 omitted for clarity.
Table 6 Optimum factor loadings of safety climate items*†.
| Safety climate items | Factor loadings | ||
|---|---|---|---|
| Safety climate factor 1 | Safety climate factor 2 | Safety climate factor 3 | |
| Attitudes to safety within own team; capacity to learn from errors | Overall confidence in safety of organisation | Perceptions of management's attitudes to safety | |
| Safety climate factor 1 | |||
| I am encouraged by my colleagues to report any patient safety concerns I may have | 0.707 | ||
| The culture where I work makes it easy to learn from the errors of others | 0.623 | ||
| I receive appropriate feedback about my performance | 0.663 | ||
| Medical errors are handled appropriately here | 0.807 | ||
| I know the proper channels to which I should direct questions regarding patient safety | 0.592 | ||
| Safety climate factor 2 | |||
| The levels of staffing where I work are sufficient to handle the number of patients | 0.905 | ||
| I would feel safe being treated as a patient in this service | 0.445 | 0.554 | |
| Management does not knowingly compromise the safety of patients | 0.519 | −0.411 | |
| Safety climate factor 3 | |||
| This organisation is doing more for patient safety now than it did one year ago | −0.903 | ||
| Leadership is driving us to be a safety‐centred organisation | −0.786 | ||
| My suggestions about safety would be acted upon if I expressed them to management | 0.383 | −0.442 | |
*Pattern matrix generated using principal components extraction and oblique rotation.
†Factor loadings under 0.300 omitted for clarity.
Internal consistency reliabilities
Internal consistency reliabilities (how clearly a set of items measure a single theme) were satisfactory to good, with Cronbach's alpha 0.69 or above in all five factors (tables 1 and 2). Removing a further item from the initial five items forming teamwork factor 2 improved the internal consistency reliability of this factor. This item (“Briefing staff on handovers between shifts is important for patient safety”) appears to relate to opinion rather than what actually happens.
Confirmatory factor analysis
Confirmatory factor analysis on the remaining 50% (the “validation half”) of the dataset indicated an almost adequate fit of the model to the data under the widely applied fit indices criteria.24 The CFI and RMSEA took values of 0.93 and 0.08 for teamwork, and 0.94 and 0.07 for safety climate, respectively (see table S2 available online at http://www.qshc.com/supplemental). These can be compared with the cut off values for a good model, estimated as a CFI >0.95 and an RMSEA <0.06.24
Use of the questionnaire in different care settings
To assess how well the factor model fitted each of the primary care and hospital datasets and also to explore whether there was a difference in teamwork or safety climate between the two groups, a series of confirmatory factor analyses were performed, allowing successive model parameters to vary between the groups. Separate confirmatory factor analyses were also carried out on each of the primary and secondary care groups (see table S3 available online at http://www.qshc.com/supplemental).
There was some difference in reported teamwork climate between the groups; the factor model showed a slight but significant improvement when scale means were allowed to vary, and mean scores on both teamwork factors were lower for secondary care than for primary care. There also appeared to be some difference in optimal factor structure between the groups, since allowing the factor loadings to vary resulted in a significant improvement in fit (difference in χ2 = 39 on 9 df, p<0.05). Factor loadings for two items (“It is easy for staff here to ask questions when there is something that they do not understand” and “I have the support I need from other staff to care for patients”) were relatively low for the primary care sample (0.713 and 0.562), reflected in the low percentage of variance explained for these items for the primary care sample (R2 = 0.249 and R2 = 0.195 compared with R2 = 0.411 and R2 = 0.512 for secondary care). Separate confirmatory factor analyses on the two datasets indicated that the model fitted the secondary care data substantially better than the primary care data (CFI = 0.938 and 0.858, respectively).
For safety climate the results of the multigroup analyses suggested that the optimal factor model was similar across both primary and secondary care, since improvement in fit was not statistically significant when factor loadings were allowed to vary between the groups. The separate confirmatory factor analyses on the two datasets indicated that the model offered a better fit to the hospital data, although the difference was smaller than for the teamwork climate model. However, there was evidence of a difference in level of safety climate between the groups, with the model fit greatly improved by allowing variation in the factor means (difference in χ2 = 43 on 3 df, p<0.05) and variances (difference in χ2 = 15 on 3 df, p<0.05). The mean scores on each of the three safety climate factors were significantly lower for the secondary care subsample, and their variances were greater.
Discussion
The purpose of this study was twofold: (1) to test whether the questionnaire met conventional scientific criteria for internal reliability and factor structure, and (2) to determine whether there was any difference in the factor model when compared between primary care and secondary care responses.
Factor structure
One key consideration when using questionnaires to assess safety climate is the need for a well grounded item content (reflecting the topics to be covered by the questionnaire), together with a clearly defined factor structure relating groups of items (questions) to specific themes.
Interpretation of the factor structure of this questionnaire was not clearcut since some items were found to relate to more than one factor. This may cause difficulties in interpreting results if the questionnaire was to be used to evaluate safety programmes or track changes over time.15 Conversely, the factors within a topic such as safety climate are likely to be interrelated to some extent, which may partially account for the cross‐loading of several items to more than one factor (tables 5 and 6).
Removing five items from the questionnaire improved the internal reliabilities of both domains. The three factor safety climate solution was the more satisfactory, explaining 61% of the variance of the 11 items, whereas the two factor teamwork scale only explained 50%. Furthermore, our teamwork factor 1 and safety climate factor 1 correspond reasonably well with the “teamwork climate” and “safety climate” factors in the 60‐item Safety Attitudes Questionnaire from which the Teamwork and Safety Climate Survey questionnaire was originally developed25 and which has recently been proposed for hospital‐wide use.2 Interestingly, our three safety climate factors also agree well with three key safety climate dimensions identified in reviews by Flin et al14 and Wiegmann et al.26 These relate to (1) employees' own attitudes to risk and safety, (2) organisational commitment and safety system, and (3) management's attitudes to safety.
Negatively worded items
In common with many other survey instruments, the Teamwork and Safety Climate Survey questionnaire contained some items that were negatively worded (table 3). Although suggested as a means of reducing response bias,27 negatively worded items have often been found to factor out separately.22,28,29 Schmitt and Stults23 showed that an artificial factor can be produced when as few as 10% of respondents fail to recognise the reversal of the wording. Since the four negatively worded items exhibited this clustering behaviour, they were removed from the final stage of the factor analysis.
Use of the questionnaire in primary and secondary care
We have also been able to show, we believe for the first time, that a safety climate questionnaire can be used across a whole health community, both in primary and in secondary care. The final overall factor model appeared to fit both groups reasonably well, although the fit was slightly better for the hospital data than for primary care, which is not surprising given that the questionnaire was designed for a hospital setting. This suggests that, after further refinement (our interview data suggest that some questions may not be as relevant for primary care), comparisons using this type of questionnaire could be made between primary and secondary care.
However, a further cautionary note is that staff whose main role was not direct patient care answered “not applicable” to many questions which has implications for the use of this type of questionnaire across whole healthcare organisations.
Limitations of the study
The aim of this study was to evaluate the factor structure—and hence usefulness—of this safety climate questionnaire in UK health care. However, we did not aim to assess from first principles the key dimensions of safety climate in an NHS setting. This would have been the best strategy had we been setting out to create a new questionnaire, but it would have been much more resource intensive.
At this stage of the analysis we did not set out to undertake a multilevel confirmatory factor analysis to explore the influence that caregiver type (for example, doctor or nurse) might have had on the results.
Implications
The refined 22 item questionnaire provides a measure of safety climate in both primary and secondary UK health care, meeting some of the criteria on factor structure and internal reliability. It might, for example, prove useful in research studies that sought associations between safety culture and health outcomes, seeking also to determine the predictive validity of the instrument. However, there are enough cautionary points arising from the item content and factor analysis of this questionnaire to suggest that there is more to be done in exploring the properties of safety climate instruments, even those recently released,17 before proceeding wholesale into measuring safety climate across health services, at least those in the UK.
As Pronovost and Sexton2 have recently pointed out, there is still much work required before we are able to understand the full value of using climate questionnaires in health care, including the meaningfulness of the resulting data. Until it is possible to derive evidence of predictive validity, such as whether positive culture data predict measurably safer health care, the costs of routinely using safety climate questionnaires may not be justified.
Further data are available in tables S1–3 available online at http:/www.qshc.com/supplemental.
Acknowledgements
The authors thank the members of staff in all 13 sites who assisted with the organisation of the surveys. Particular thanks are due to Eric Thomas and Bryan Sexton for their support in using the Teamwork and Safety Climate questionnaire. Karen Beck acted as project administrator and her support was invaluable.
Abbreviations
CFI - comparative fit index
RMSEA - root mean square error of approximation
SRMR - standardised root mean square residual
TLI - Tucker‐Lewis index
Footnotes
Funding was provided by the Sheffield Health and Social Care Research Consortium and by the devolved funds programme at the University of Sheffield.
Competing interests: none declared.
Further data are available in tables S1–3 available online at http:/www.qshc.com/supplemental.
References
- 1.National Patient Safety Agency Safer health care. Available at http://www.saferhealthcare.org.uk (accessed 25 August 2005)
- 2.Pronovost P, Sexton B. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care 200514231–233. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kho M E, Carbone J M, Lucas J.et al Safety Climate Survey: reliability of results from a multicenter ICU survey. Qual Saf Health Care 200514273–278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Denison D R. What is the difference between organizational culture and organizational climate? A native's point of view on a decade of paradigm wars. Acad Manage Rev 199621619–654. [Google Scholar]
- 5.Schneider B. The psychological life of organizations. In: Ashkanasy NM, Wilderon CPM, Peterson MF, eds Handbook of organizational culture and climate. Thousand Oaks, CA: Sage, 2000xvii–xxi.
- 6.Barling J, Loughlin C, Kelloway E K. Development and test of a model linking safety‐specific transformational leadership and occupational safety. J Appl Psychol 200287488–496. [DOI] [PubMed] [Google Scholar]
- 7.Griffin M A, Neal A. Perceptions of safety at work: a framework for linking safety climate to safety performance, knowledge, and motivation. J Occup Health Psychol 20005347–358. [DOI] [PubMed] [Google Scholar]
- 8.Zohar D. Safety climate: conceptual and measurement issues. In: Quick JC, Tetrick LE, eds. Handbook of occupational health psychology. Washington, DC: American Psychological Association, 2003123–142.
- 9.Schein E H. Sense and nonsense about climate and culture. In: Ashkanasy NM, Wilderon CPM, Peterson MF, eds. Handbook of organizational culture and climate. Thousand Oaks, CA: Sage, 2000xxiii–xxx.
- 10.Helmreich R, Merritt A.Culture at work in aviation and medicine. Aldershot: Ashgate, 1998
- 11.Mearns K, Flin R, Gordon R.et al Measuring safety climate on offshore installations. Work Stress 199812238–254. [Google Scholar]
- 12.Sexton J B, Thomas E J, Helmreich R L. Error, stress and teamwork in medicine and aviation: cross sectional surveys. BMJ 2000320745–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Singer S J, Gaba D M, Geppert J J.et al The culture of safety: results of an organization‐wide survey in 15 California hospitals. Qual Saf Health Care 200312112–118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Flin R, Mearns K, O'Connor P.et al Measuring safety climate: identifying the common features. Saf Sci 200034177–192. [Google Scholar]
- 15.Nieva V F, Sorra J. Safety culture assessment: a tool for improving patient safety in healthcare organizations. Qual Saf Health Care 200312(Suppl II)ii17–ii23. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Institute for Healthcare Improvement Available at http://www.ihi.org/IHI/Topics/PatientSafety/SafetyGeneral/Tools/Safety+Climate+Survey+%28IHI+Tool%29.htm (accessed 25 August 2005)
- 17.Agency for Healthcare Research and Quality Hospital survey on patient safety culture. Rockville, MD: Agency for Healthcare Research and Quality, 2005, Available at http://www.ahrq.gov/qual/hospculture/ (accessed 25 August 2005)
- 18.The University of Texas Center of Excellence for Patient Safety Research and Practice Available at http://www.uth.tmc.edu/schools/med/imed/patient_safety/survey&tools.htm (accessed 25 August 2005)
- 19.Sexton J B, Helmreich R L, Pronovost P.et alSafety climate survey. University of Texas 2003
- 20.Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs 20034257–63. [DOI] [PubMed] [Google Scholar]
- 21.Ericsson K A, Simon H A.Protocol analysis: verbal reports as data (revised edition). Cambridge, MA: MIT Press, 1993
- 22.Roberts R E, Lewinsohn P M, Seeley J R. A brief measure of loneliness suitable for use with adolescents. Psychol Rep 1993721379–1391. [DOI] [PubMed] [Google Scholar]
- 23.Schmitt N, Stults D M. Factors defined by negatively keyed items: the result of careless respondents? Appl Psychol Measure 19859367–373. [Google Scholar]
- 24.Hu L, Bentler P M. Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling 199961–55. [Google Scholar]
- 25.Sexton J B, Thomas E J, Helmreich R L.et alFrontline assessments of healthcare culture: Safety Attitudes Questionnaire norms and psychometric properties. Technical Report 04‐01. Houston: The University of Texas Center of Excellence for Patient Safety Research and Practice, 2003, Available at http://www.uth.tmc.edu/schools/med/imed/patient_28 safety/survey&tools.htm (accessed 25 August 2005)
- 26.Weigmann D A, Zhang H, von Thaden T.et alA synthesis of safety culture and safety climate research. Savoy, Illinois: University of Illinois at Urbana‐Champaign, 2002
- 27.Nunnally J C, Bernstein I.Psychometric theory. New York: McGraw‐Hill, 1994
- 28.Cordery J L, Sevastos P P. Responses to the original and revised Job Diagnostic Survey: is education a factor in responses to negatively worded items? J Appl Psychol 199378141–143. [Google Scholar]
- 29.Spector P E, Van Katwyk P T, Brannick M T.et al When two factors don't reflect two constructs: how item characteristics can produce artifactual factors. J Manage 199723659–677. [Google Scholar]