Psychometric properties of the Georgian version of the Safety Attitudes Questionnaire: a cross-sectional study

Abstract

Objective

To study the psychometric properties of the Georgian version of the Safety Attitudes Questionnaire short version.

Design

Cross-sectional study.

Setting

Three Georgian hospitals.

Participants

Personnel of participating hospitals (n=305 responses, estimated response rate 30%).

Interventions

None.

Primary and secondary outcome measures

Psychometric properties (model fit, internal consistency, construct validity, convergent and discriminant validity) of the instrument, factor structure derived from the data.

Results

The Georgian version of Safety Attitudes Questionnaire demonstrated acceptable construct validity and internal consistency (Cronbach’s alpha 0.61–0.91). Three factors, Teamwork Climate, Safety Climate and Working Conditions, had limited convergent and discriminant validity. Confirmatory factor analysis with the original six-factor model resulted in limited model fit (χ²/df=2.14, root mean square error of approximation (RMSEA)=0.06, goodness of fit index (GFI)=0.83, CFI=0.88, TLI=0.86). Exploratory factor analysis resulted in a modified four-factor model with satisfactory model fit (χ²/df=2.09, RMSEA=0.06, GFI=0.88, CFI=0.93, TLI=0.91).

Conclusions

The Georgian version of the Safety Attitudes Questionnaire (short version) demonstrated acceptable psychometric properties, with acceptable to good internal consistency and construct validity. While the whole model had limited fit to the data, a modified factor model resulted in good model fit. Our findings suggest the dimension Working Conditions has questionable psychometric properties and should be interpreted with caution. Other two correlated dimensions Teamwork Climate and Safety Climate share considerable variance and may be merged. Overall, the instrument can provide valuable information relevant for advancement of patient safety culture in Georgian hospitals.

Strengths and limitations of this study.

• First study to validate a Georgian version of the Safety Attitudes Questionnaire (short version).

• A comprehensive analysis of the survey instrument’s performance, including exploratory and confirmatory factor analysis.

• Multiprofessional sample from multiple hospitals.

• Study findings are limited by the study sample, which included three general hospitals.

Introduction

Over the past decades, overwhelming evidence has been accumulated suggesting that patient safety is an ongoing challenge for modern healthcare systems. Cultivating the culture of safety in healthcare organisations has been recommended to enable better communication and open exchange, to learn from errors, eventually leading to better patient outcomes.¹ The recent report by the Organisation for Economic Cooperation and Development (OECD), the Economics of Patient Safety² analysed the state of the research and its implications on a larger, national level. Among the key messages and conclusions, the report underlines the importance of placing patient safety among national priorities, and establishing positive patient safety culture at the organisation level. Ensuring high quality and safe healthcare services for all citizens, in line with these recommendations, is also among the current priorities of the Georgian government.^{2 3} The healthcare services in the country are largely provided by private organisations which are increasingly required by state regulatory organisations and funding bodies to establish processes and systems to ensure improvements in patient safety.⁴ Thus, in order to analyse and develop the culture of safety, Georgian healthcare organisations require valid instruments to measure safety culture in local environments.

The Safety Attitudes Questionnaire short version (SAQ-S),⁵ originally adapted from the aviation industry to be used in US hospitals, is among the most frequently used instruments for measuring safety culture internationally.⁶ It has been translated into several different languages and validated in many countries.^6–14 Overall, validation studies using SAQ-S have reported acceptable psychometric properties, indicating that the instrument may be relevant for measuring and promoting patient safety culture in different healthcare settings.

In this study, we aimed to evaluate psychometric properties of a Georgian version of the Safety Attitudes Questionnaire short version (SAQ-S-GE), including internal consistency, convergent and discriminant validity, fit to original factor model and possible alternative factor structure.

Methods

Setting and data collection

The data for this cross-sectional study were collected as part of the study project Patient Safety Culture in Georgian Healthcare (PaSCu.Ge), which aimed to establish a baseline evaluation of patient safety culture in local hospitals using translated and adapted versions of internationally well-studied instruments, the Hospital Survey on Patient Safety Culture¹⁵ and Safety Attitudes Questionnaire.⁵ The psychometric properties of the former are presented elsewhere.¹⁶ Using the SAQ-S-GE, data were collected in three Georgian hospitals in two cities between June and August 2017. Hospital employees could complete the survey electronically or on paper. Hospital representatives were trained to act as local study coordinators and facilitate employee participation.

Patient and public involvement

Representatives of patient and public groups were not involved in the study design and implementation. Main findings of the study will be made publicly available online (in both Georgian and English).

Sample

Two of the three participating hospitals have 100–150 hospital beds and the third has <50 hospital beds. All three are for-profit multiprofile hospitals, with an estimated total of 1000 employees who met the inclusion criteria. All personnel of the three participating general hospitals, employed for more than 1 month, were invited to participate. Before completing the survey, all participants were informed that participation was voluntary and anonymous, and provided informed consent.

Measure

Safety Attitudes Questionnaire short version (SAQ-S)^{5 17} consists of 36 items, 31 of which are grouped into six dimensions. All 36 items of the instrument measure participants’ agreement (from 1=strongly disagree to 5=strongly agree) to various patient safety-related statements on a five-point Likert scale. In this study, we also included the outcome item Patient Safety Grade from another widely used instrument on patient safety culture, the Hospital Survey on Patient Safety Culture.¹⁵ The item asks for an employee evaluation of patient safety on a five-point quality scale (from 1=failing to 5=excellent). In addition, the questionnaire included the demographic information on study participants (ie, department, profession, gender, tenure).

The original version of the SAQ-S was translated from English to Georgian by a native speaker with experience of working in Georgian healthcare. The translated version was adapted to the Georgian healthcare context without changing the overall structure of the instrument. Next, the Georgian version was back-translated to English by a professional translator. The discrepancies with the original version were discussed by the research team (NG, AH and TM) and necessary revisions were made. We asked five Georgian professionals (healthcare researchers and managers, physicians and nurses) who were not otherwise associated with the study to do a cognitive pretest of the revised version and to provide feedback on the content and language. Based on the results of the pretest, we were able to establish the final Georgian version of the questionnaire used in this study (SAQ-S-GE). In order to support comparability of the results, we made sure to maintain the overall composition and item wording of the original SAQ-S. The SAQ-S-GE is available on request from the corresponding author.

Analysis

Data processing and preliminary analysis

Before the analysis, negatively coded items were reverse coded, so that higher scores correspond to more positive safety culture. Descriptive analyses, as well as analyses of acceptability, were conducted using the complete sample. It has been shown that by means of imputing the missing answers, a considerable part of the sample may be made available for the analysis sensitive to missing values.¹⁸ However, in order to maintain high data quality, we excluded cases with more than 10% missing answers before imputation. The remaining missing values were imputed using the expectation maximisation algorithm.

Descriptive statistics

Mean scores for SAQ-S-GE dimensions were calculated by averaging the corresponding items. We calculated means and SD and the percentage of positive responses (scores 4 and 5) for each item and dimension.⁵

Acceptability

To evaluate the acceptability of the questionnaire, we calculated the percentage of missing answers on individual items and complete dimensions. We considered floor and ceiling effects (ie, the percentage of lowest and highest available answers, respectively), as an indication of the instrument’s performance at the extremes of the measured construct. For dimensions, we considered a floor or ceiling effect of <15% acceptable.¹⁷

Internal consistency, construct validity, convergent and discriminant validity

As an indication of internal consistency of the instrument, we calculated Cronbach’s alpha for each dimension. Cronbach’s alpha ≥0.7 was considered good.^{19 20} We evaluated Spearman’s correlations between the SAQ-S-GE dimensions, as well as correlation with the additional single-item outcome variable Patient Safety Grade, as preliminary analysis of construct validity. Because all dimensions are considered to be measuring constructs related to patient safety, we expected to find low to moderate positive correlations. However, excessive correlation between dimensions (>0.85) could indicate possible collinearity.^{15 19}

Additionally, we evaluated the convergent and discriminant validity of the SAQ-S-GE.²¹ As an indication of convergent validity of a dimension, we calculated the average variance extracted (AVE) and expected it to be >0.5. For divergent validity, we used the Fornell-Larcker criterion²¹ and expected a higher square root of AVE (√AVE), compared with the highest correlation with other factors.

Exploratory factor analysis

Before conducting the factor analysis, we evaluated if the data were suitable for the analysis. We used Kaiser-Meyer-Olkin (KMO) and measure of sampling adequacy (MSA) (>0.7 desired, >0.9 perfect) to evaluate if the sample of items and each individual item were adequate for factor analysis.¹⁹ A significant p value (<0.05) of Bartlett’s test of sampling adequacy would indicate that it is possible to extract more than one factor.¹⁹

To explore a possible alternative factor structure based on our data, we conducted exploratory factor analysis (EFA) using the 31 items of the original six dimensions of SAQ-S. We used maximum likelihood algorithm for factor extraction with Varimax orthogonal prerotation and Promax oblique rotation to aid with interpretation of the factor model.²⁰ The number of factors extracted was guided by scree plot inspection and the Kaiser criterion (eigenvalues>1). We considered factor loadings ≥0.4 significant and cross-loading <0.4 acceptable.^{19 20} We evaluated the similarities and differences between the EFA-based modified factor structure and the original model.

Confirmatory factor analysis

Confirmatory factor analysis (CFA) was used to evaluate how well the data fit the original factor model. The hypothesised model of SAQ-S-GE is presented in online supplementary appendix 1. We used the following indices and benchmarks: normed χ² (χ²/df ≤2.5), root mean square error of approximation (RMSEA≤0.07), goodness of fit index (GFI>0.90), comparative fit index (CFI≥0.90) and Tucker-Lewis Index/non-normed fit index (TLI>0.90).^{5 19 20} We analysed the fit of complete model, as well as that of each of the six original dimensions. Finally, we evaluated the fit of the EFA-based modified model. All analyses were done using SAS V.9.4.

Results

Study sample and descriptive statistics

A total of 305 questionnaires were collected from three participating hospitals, resulting in an estimated response rate of 30.5%. Twenty-one participants (6.9%), all from one hospital, used the online questionnaire. Most participants indicated having direct contact with patients (83.3%) and no managerial functions (61.0%). Descriptive characteristics of the sample are presented in table 1.

Table 1.

Descriptive characteristics of study sample

Characteristics	N	%
Total sample	305	100.0
Gender
Female	219	71.8
Male	66	21.6
Missing	20	6.6
Profession
Nurse	79	25.9
Physician	128	42.0
Other	79	25.9
Missing	19	6.2
Patient contact
Yes	254	83.3
No	28	9.2
Missing	23	7.5
Managerial functions
Yes	77	25.2
No	186	61.0
Missing	42	13.8
Hours per week
Less than 20 hours	11	3.6
20–39 hours	76	24.9
40–59 hours	135	44.3
60 hours or more	61	20.0
Missing	22	7.2
Years in the department
Less than 1 year	28	9.2
1–5 years	49	16.1
6–10 years	70	23.0
11–15 years	38	12.5
16–20 years	20	6.6
21 years or more	73	23.9
Missing	27	8.9
Years in the field
Less than 1 year	4	1.3
1– 5 years	37	12.1
6–10 years	51	16.7
11–15 years	31	10.2
16–20 years	36	11.8
21 years or more	117	38.4
Missing	29	9.5

Percentage of missing answers per item was 6.8% on average, with a minimum of 2.3% (“Working here is like being part of a large family”) and a maximum of 13.1% (“I receive appropriate feedback about my performance”). The dimension Stress Recognition had a floor effect >15%, while all other dimensions demonstrated a ceiling effect of >15%. Mean values and percentages of positive responses, as well as corresponding standard errors for all SAQ-S-GE dimensions, each individual item and the single item Patient Safety Grade are presented in table 2. The table presents all original 36 items and the corresponding six factors according to the guidelines of the Centre for Healthcare Quality and Safety of the University of Texas (available at https://med.uth.edu/chqs/survey).

Table 2.

SAQ-S-GE dimensions and items; percentage of missing values, floor and ceiling effects, mean scores and percentage of positive responses with corresponding standard errors

Dimensions/Items	Missing %	Floor %	Ceiling %	Mean scores (SE)	Percent of positive responses (SE)
Teamwork Climate	0.7	0.7	29.8	4.41 (0.04)	85.1 (1.2)
1. Nurse input is well received in this clinical area.	6.6	6.6	44.3	4.01 (0.07)	74.4 (2.6)
2. In this clinical area, it is difficult to speak up if I perceive a problem with patient care. (N)	9.2	5.9	57.4	4.11 (0.08)	72.9 (2.7)
3. Disagreements in this clinical area are resolved appropriately (ie, not who is right, but what is best for the patient).	6.2	2.3	68.5	4.55 (0.05)	90.2 (1.8)
4. I have the support I need from other personnel to care for patients.	8.5	1.3	70.5	4.66 (0.05)	94.3 (1.4)
5. It is easy for personnel here to ask questions when there is something that they do not understand.	6.6	3.3	69.5	4.50 (0.06)	88.1 (1.9)
6. The physicians and nurses here work together as a well-coordinated team.	3.6	2.3	76.1	4.61 (0.05)	90.8 (1.7)
Safety Climate	3.6	0.0	16.7	4.27 (0.04)	81.6 (1.2)
7. I would feel safe being treated here as a patient.	7.5	2.0	63.3	4.50 (0.05)	89.7 (1.8)
8. Medical errors are handled appropriately in this clinical area.	7.2	0.7	63.9	4.53 (0.05)	89.8 (1.8)
9. I know the proper channels to direct questions regarding patient safety in this clinical area.	6.6	1.3	73.4	4.65 (0.05)	91.6 (1.6)
10. I receive appropriate feedback about my performance.	13.1	12.8	32.8	3.61 (0.09)	62.6 (3.0)
11. In this clinical area, it is difficult to discuss errors. (N)	8.2	9.8	52.8	3.94 (0.09)	70.7 (2.7)
12. I am encouraged by my colleagues to report any patient safety concerns I may have.	6.9	3.0	62.0	4.43 (0.06)	86.3 (2.0)
13. The culture in this clinical area makes it easy to learn from the errors of others.	6.6	3.0	49.5	4.22 (0.06)	81.4 (2.3)
Job Satisfaction	0.3	1.0	57.4	4.61 (0.04)	91.5 (1.3)
15. I like my job.	3.6	1.6	76.4	4.66 (0.05)	92.9 (1.5)
16. Working here is like being part of a large family.	2.3	1.3	76.4	4.68 (0.04)	94.6 (1.3)
17. This is a good place to work.	4.3	1.6	65.2	4.53 (0.05)	90.4 (1.7)
18. I am proud to work in this clinical area.	3.6	1.0	74.8	4.64 (0.04)	90.5 (1.7)
19. Morale in this clinical area is high.	3.9	2.3	73.1	4.59 (0.05)	90.1 (1.7)
Stress Recognition	5.6	15.4	5.2	2.82 (0.08)	46.6 (2.3)
20. When my workload becomes excessive, my performance is impaired.	6.9	37.4	14.8	2.71 (0.10)	45.8 (3.0)
21. I am less effective at work when fatigued.	6.2	28.5	16.4	2.94 (0.09)	51.0 (3.0)
22. I am more likely to make errors in tense or hostile situations.	7.2	27.9	28.2	3.16 (0.10)	55.1 (3.0)
23. Fatigue impairs my performance during emergency situations (eg, emergency resuscitation, seizure).	9.8	42.0	13.1	2.46 (0.09)	34.2 (2.9)
Perceptions of Hospital Management	3.3	0.3	43.3	4.31 (0.06)	82.5 (1.8)
24. Hospital management supports my daily efforts.	6.2	3.6	59.0	4.31 (0.07)	83.2 (2.2)
25. Hospital management doesn’t knowingly compromise patient safety.	7.9	0.7	74.4	4.64 (0.05)	90.0 (1.8)
26. Hospital management is doing a good job.	7.9	4.9	49.5	4.09 (0.07)	75.8 (2.6)
27. Problem personnel are dealt with constructively by our hospital management.	8.5	3.3	57.7	4.34 (0.06)	82.8 (2.3)
28. I get adequate, timely information about events that might affect my work, from hospital management.	7.5	7.5	53.8	4.13 (0.08)	78.4 (2.5)
Working Conditions	4.6	0.0	30.2	4.25 (0.05)	80.8 (1.6)
29. The levels of staffing in this clinical area are sufficient to handle the number of patients.	4.9	3.9	64.9	4.37 (0.07)	85.5 (2.1)
30. This hospital does a good job of training new personnel.	7.5	5.2	46.6	4.07 (0.07)	77.7 (2.5)
31. All the necessary information for diagnostic and therapeutic decisions is routinely available to me.	10.2	1.0	57.0	4.33 (0.06)	81.8 (2.3)
32. Trainees in my discipline are adequately supervised.	10.2	5.2	53.1	4.19 (0.07)	76.6 (2.6)
Items not belonging to any dimension
14. My suggestions about safety would be acted on if I expressed them to management.	7.2	4.3	47.2	4.13 (0.07)	76.3 (2.5)
33. I experience good collaboration with nurses in this clinical area.	3.6	1.6	88.9	4.85 (0.04)	95.9 (1.2)
34. I experience good collaboration with staff physicians in this clinical area.	3.6	1.6	87.9	4.83 (0.04)	96.3 (1.1)
35. I experience good collaboration with pharmacists in this clinical area.	8.2	0.7	75.1	4.69 (0.04)	88.6 (1.9)
36. Communication breakdowns that lead to delays in delivery of care are common. (N)	8.2	12.5	39.0	3.48 (0.09)	55.7 (3.0)
Added single item Patient Safety Grade	7.5	0.3	6.2	3.69 (0.04)	62.8 (2.9)

n=305. Five items, namely, numbers 14 and 33–36, are not part of any scale and were not used in the factor analysis. These items are part of the instrument because they provide additional information relevant to the Patient Safety Culture.

N, Negatively worded items; SAQ-S-GE, Georgian version of Safety Attitudes Questionnaire short version.

Internal consistency and construct validity, convergent and discriminant validity

After removing 42 cases with more than 10% missing answers on any of 31 SAQ-S-GE items and imputing the remaining missing values, 263 questionnaires were available for further analyses. The dimensions of the SAQ-S-GE demonstrated good internal consistency with Cronbach’s alpha ≥0.7, with the exception of the dimension Working Conditions (α=0.61). All interfactor correlations were statistically significant, except for the correlation between Stress Recognition and Working Conditions. Most dimensions correlated positively with each other, except for Stress Recognition, which had a negative correlation with all other dimensions. Convergent validity of the three out of six dimensions, Teamwork Climate, Safety Climate and Working Conditions, failed to reach the required benchmark of 0.5. Also, √AVE of these three dimensions was much lower than the highest correlation with other factors, demonstrating limited discriminant validity. Another three dimensions, Job Satisfaction, Stress Recognition and Perceptions of Hospital Management, showed good reliability with good internal consistency and convergent validity, and acceptable divergent validity, with Job Satisfaction having slightly less √AVE compared with the highest correlation. Table 3 presents the results of validity analyses of the original six dimensions.

Table 3.

Internal consistency, indicators of convergent and divergent validity, and factor correlations of the original six-factor model

	α	AVE	√AVE	Factor correlation matrix
				TC	SC	JS	SR	HM
TC—Teamwork Climate	0.71	0.34	0.59
SC—Safety Climate	0.72	0.31	0.56	0.83**
JS—Job Satisfaction	0.90	0.65	0.81	0.70**	0.82**
SR—Stress Recognition	0.83	0.56	0.75	−0.22*	−0.19*	−0.16*
HM—Perceptions of Hospital Management	0.89	0.64	0.80	0.47**	0.61**	0.64**	−0.23**
WC—Working Conditions	0.61	0.30	0.55	0.68**	0.76**	0.75**	−0.13	0.75**

Note: Analyses conducted after imputing missing values; n=263.

*p<0.05, **p<0.001.

α, Cronbach’s alpha; AVE, average variance extracted; √AVE, square root of average variance extracted.

Evaluating fit of the data to original model, and exploring an alternative model

KMO test returned 0.89, while the average MSA for the individual items was 0.86 and varied between 0.59 and 0.95. Bartlett’s test was highly significant (p<0.0001), further indicating that the sample was adequate for factor analysis.

EFA resulted in a modified four-factor model with 21 items. Ten items were removed from this model because of either low factor loadings (<0.4) or high cross-loadings (>0.4). Two dimensions, Teamwork Climate and Safety Climate, were merged to form a combined dimension of Teamwork and Safety Climate. Three original dimensions, Job Satisfaction, Stress Recognition and Perceptions of Hospital Management, remained in the model, retaining all or most of the original items. The items from the dimension Working Conditions were mostly removed from the model. The EFA-based four-factor model is presented in table 4.

Table 4.

Rotated factor pattern based on the exploratory factor analysis (EFA)

EFA-based four-factor model	Original six dimensions and corresponding items	Factor loading
Factor 1: Teamwork and Safety Climate	Teamwork Climate
	1. Nurse input is well received in this clinical area.	RM
	2. In this clinical area, it is difficult to speak up if I perceive a problem with patient care.(N)	RM
	3. Disagreements in this clinical area are resolved appropriately (ie, not who is right, but what is best for the patient).	0.609
	4. I have the support I need from other personnel to care for patients.	0.685
	5. It is easy for personnel here to ask questions when there is something that they do not understand.	0.564
	6. The physicians and nurses here work together as a well-coordinated team.	0.720
	Safety Climate
	7. I would feel safe being treated here as a patient.	0.564
	8. Medical errors are handled appropriately in this clinical area.	0.637
	9. I know the proper channels to direct questions regarding patient safety in this clinical area.	0.515
	10. I receive appropriate feedback about my performance.	RM
	11. In this clinical area, it is difficult to discuss errors. (N)	RM
	12. I am encouraged by my colleagues to report any patient safety concerns I may have.	0.420
	13. The culture in this clinical area makes it easy to learn from the errors of others.	RM
Factor 2: Job Satisfaction	Job Satisfaction
	15. I like my job.	0.606
	16. Working here is like being part of a large family.	RM
	17. This is a good place to work.	0.669
	18. I am proud to work in this clinical area.	0.775
	19. Morale in this clinical area is high.	RM
Factor 3: Stress Recognition	Stress Recognition
	20. When my workload becomes excessive, my performance is impaired.	0.757
	21. I am less effective at work when fatigued.	0.757
	22. I am more likely to make errors in tense or hostile situations.	0.697
	23. Fatigue impairs my performance during emergency situations (eg, emergency resuscitation, seizure).	0.756
Factor 4: Perceptions of Hospital Management	Perceptions of Hospital Management
	24. Hospital management supports my daily efforts.	0.766
	25. Hospital management doesn’t knowingly compromise patient safety.	0.482
	26. Hospital management is doing a good job.	0.844
	27. Problem personnel are dealt with constructively by our hospital management.	0.691
	28. I get adequate, timely information about events that might affect my work, from hospital management.	0.851
	Working Conditions
	29. The levels of staffing in this clinical area are sufficient to handle the number of patients.	RM
	30. This hospital does a good job of training new personnel.	RM
	31. All the necessary information for diagnostic and therapeutic decisions is routinely available to me.	RM
	32. Trainees in my discipline are adequately supervised.	0.417

N, Negatively worded item; RM, removed from the model.

In CFA, we evaluated the fit of our data to the original six-factor model.⁵ While the whole model hardly satisfied set criteria for a good fit, individual dimensions had better fit indices. Finally, we checked the fit of the EFA-based four-factor model to the data using CFA, which resulted in acceptable fit indices. Results of all CFA, together with considered thresholds for acceptable fit, are presented in table 5.

Table 5.

Results of confirmatory factor analyses using different factor models

Model for analysis	χ²/df	GFI	CFI	TLI	RMSEA
Thresholds for acceptable fit	≤2.5	>0.90	>0.90	>0.90	≤0.07
Original six-factor model	2.10	0.83	0.88	0.86	0.06
Dimension Teamwork Climate	3.45	0.96	0.93	0.89	0.10
Dimension Safety Climate	2.83	0.96	0.93	0.89	0.08
Dimension Job Satisfaction	3.01	0.98	0.99	0.97	0.09
Dimension Stress Recognition	4.44	0.98	0.98	0.95	0.11
Dimension Perceptions of Hospital Management	5.00	0.97	0.97	0.95	0.12
Dimension Working Conditions	2.88	0.99	0.97	0.90	0.08
EFA-based four-factor model	2.09	0.88	0.93	0.91	0.06

N=263. Underline indicates values that do not reach corresponding thresholds

CFI, comparative fit index; χ²/df, normed χ²; GFI, goodness of fit index; RMSEA, root mean square error of approximation; TLI, Tucker-Lewis Index/non-normed fit index.

Discussion

In this study, we evaluated the psychometric properties of SAQ-S-GE by analysing its acceptability, internal consistency, convergent and discriminant validity, as well as the fit to the original model and an alternative factor structure. We were able to identify dimensions with satisfactory properties and dimensions that may need further improvements and/or study.

Overall, the SAQ-S-GE was well accepted, with acceptable percentages of missing answers. Some studies using SAQ reported much lower (<2%) missing rates,^{5 7 14} while our findings are more in line with European studies, reporting up to about 10% missing.9 10 22–24 Further, we observed a significant ceiling effect in most dimensions and items, indicating that the instrument may not be effectively distinguishing the measurements at the higher end. A recent study using Rasch analysis similarly found considerable ceiling effect in all dimensions of the SAQ-S.²⁵ Moreover, in the recent study conducted in Georgian hospitals with another safety climate instrument, Georgian version of Hospital Survey on Patient Safety Culture, we have also found considerable ceiling effect in most dimensions.¹⁶ This effect may be explained at least partially by social desirability bias pushing responses towards the positive end. Interestingly, the only dimension in our study demonstrating a floor effect was Stress Recognition. This dimension stood out in further analyses as well.

Our analysis of internal consistency obtained acceptable to good results, with only the dimension Working Conditions showing a low Cronbach’s alpha. This is in line with other studies using the SAQ that reported lower internal consistency for this dimension.12 13 23 24 Most items from this dimension had relatively high rates of missing values and were eventually removed from the alternative factor model, further indicating possible problems with validity or stability of the dimension. At this stage, the scores from this dimension should be interpreted with caution.

All dimensions except Stress Recognition were positively associated with each other, and with the outcome item Patient Safety Grade, reinforcing the validity of the instrument to measure a common underlying construct—Patient Safety Culture. The dimension Stress Recognition, however, was negatively associated with most other dimensions, which is well in line with findings from other studies using the SAQ.5 9 10 23 24 26 Taylor and Pandian²⁶ called this dimension ‘a dissonant scale’ and recommended separating it from the instrument. Indeed, as many authors have pointed out, this scale asks participants for a self-evaluation while all remaining items refer to behaviours of others (ie, team, management or organisation). In spite of the described unexpected performance, items included in this scale, representing recognition of the effect of stress on performance, are without a doubt important for establishing a better culture of safety. Thus, stress recognition should be further measured and developed in healthcare organisations.

Our analysis of convergent and discriminant validity revealed problems with three dimensions—Teamwork Culture, Safety Culture and Working Conditions. The former two seem to have limited validity because of high intercorrelation, which was reaffirmed in EFA by merging these two dimensions together. Similarly, in most studies using SAQ-S, the correlation between these two dimensions was moderate to high, including the original study by Sexton et al ⁵ reporting within-area correlation of 0.94. This could indicate overall association between these two dimensions, not specific to our study. The third dimension, Working Conditions, not only demonstrated poor validity but was also mostly removed from the alternative model. Similarly, EFA conducted by Smits et al, ¹¹ using an ambulatory version of the instrument, resulted in a five-factor solution, with Working Conditions not being presented in the model.⁵ Overall, except for the items associated with the dimension Work Conditions, the SAQ-S-GE adequately measures the underlying constructs.

Limitations

Our sample was limited to three general hospitals from two cities in Georgia while there are many other similar hospitals in the country. Thus, the findings should not be directly generalised to other healthcare settings in Georgia. Another limitation of the study was the relatively small sample size, due to which we were not able to conduct a split-sample validation. Thus, our findings should be tested in future research using a larger independent sample. Further research could also establish the ability of the instrument to measure change over time, which was not allowed by the cross-sectional study design.

Conclusions

The SAQ-S-GE demonstrated adequate psychometric properties, with acceptable to good internal consistency and construct validity. While the original six-factor model had poor fit to the data, we demonstrated an alternative factor model with acceptable model fit, indicating one problematic dimension and most dimensions being relatively stable and thus suitable for further studies. Until these findings are cross-validated in future studies with larger sample size, we argue for using the instrument in its full form, but recommend caution while interpreting the data on Working Conditions. As in most other studies, the dimension Stress Recognition was dissociated from the remaining instrument, while simultaneously demonstrating good psychometric properties. This dimension may require further investigation. The SAQ-S-GE can provide valid and useful information to further patient safety culture in Georgian hospitals.