Measuring Safety Culture in the Ambulatory Setting: The Safety Attitudes Questionnaire—Ambulatory Version

Abstract

Background

Provider attitudes about issues pertinent to patient safety may be related to errors and adverse events. We know of no instruments that measure safety-related attitudes in the outpatient setting.

Objective

To adapt the safety attitudes questionnaire (SAQ) to the outpatient setting and compare attitudes among different types of providers in the outpatient setting.

Methods

We modified the SAQ to create a 62-item SAQ—ambulatory version (SAQ-A). Patient care staff in a multispecialty, academic practice rated their agreement with the items using a 5-point Likert scale. Cronbach’s alpha was calculated to determine reliability of scale scores. Differences in SAQ-A scores between providers were assessed using ANOVA.

Results

Of the 409 staff, 282 (69%) returned surveys. One hundred ninety (46%) surveys were included in the analyses. Cronbach’s alpha ranged from 0.68 to 0.86 for the scales: teamwork climate, safety climate, perceptions of management, job satisfaction, working conditions, and stress recognition. Physicians had the least favorable attitudes about perceptions of management while managers had the most favorable attitudes (mean scores: 50.4 ± 22.5 vs 72.5 ± 19.6, P < 0.05; percent with positive attitudes 18% vs 70%, respectively). Nurses had the most positive stress recognition scores (mean score 66.0 ± 24.0). All providers had similar attitudes toward teamwork climate, safety climate, job satisfaction, and working conditions.

Conclusion

The SAQ-A is a reliable tool for eliciting provider attitudes about the ambulatory work setting. Attitudes relevant to medical error may differ among provider types and reflect behavior and clinic operations that could be improved.

Introduction

Errors and adverse events are common in the outpatient setting.^1–6 Because most health care is delivered in the ambulatory setting, providers and researchers are interested in improving patient safety for outpatients. Research in healthcare and other safety-critical environments, such as the aviation industry, indicates that understanding the systems involved in the delivery of services is important for developing interventions to prevent errors. Attitudes of individuals within organizations may provide insight into how systems function and how they may be improved. For example, Helmreich and colleagues developed the flight management attitudes questionnaire (FMAQ) to measure commercial aviation crewmember attitudes about communication, teamwork, and organizational culture.⁷ Based in part on the FMAQ, team and human factors training interventions were developed to improve cockpit crew performance and to prevent errors.⁸

The FMAQ has been adapted for use in hospital inpatient settings.^9,10 The new survey, called the safety attitudes questionnaire (SAQ),¹¹ has been used to compare attitudes between clinical areas, to prompt patient safety improvement activities such as executive walkrounds,¹² and to measure change in safety-related attitudes after these interventions were implemented.¹³

We are aware of several other instruments designed to measure safety attitudes of hospital providers^14–17; however, there are no ambulatory versions of these surveys. This is despite evidence regarding the epidemiology of outpatient medical errors and the need to improve safety.^1–6 Our objectives were to adapt the SAQ to the outpatient setting and compare attitudes among different types of providers and among different specialties in the outpatient setting.

Methods

Adaptation of Survey

We reviewed all the SAQ items and made minor changes in wording to make items applicable to the outpatient setting. For example, the item “In this intensive care unit, it is difficult to discuss errors” was changed to “In this office, it is difficult to discuss errors.” The resulting SAQ—ambulatory version (SAQ-A) that was administered consisted of 62 items (see Appendix available online). We added 5 items pertaining to the process of ambulatory care based on literature review of outpatient medical errors, quality improvement, and patient safety. Respondents rated their agreement with each item using a 5-point Likert scale ranging from 1 (disagree strongly) to 5 (agree strongly).

Administration of Survey

The SAQ-A was administered to all 409 outpatient providers (156 physicians, 253 staff) at an academic, urban, outpatient practice in Texas during February and March 2002. Outpatient providers were defined as any employee having direct patient contact including physicians, nurses, managers, medical assistants, receptionists, and technicians. Specialties surveyed included general medicine, pediatrics, Obstetrics and Gynecology, adult cardiology and pulmonary medicine, neurology, orthopedics, family practice, general Surgery, otolaryngology, dermatology, and neurosurgery. Surveys were mailed to participants’ work addresses with a stamped and addressed return envelope. Participants who completed the survey received a $5 movie certificate. Nonresponders were sent a second survey after 2 weeks, a reminder card 2 weeks later, and a third and final survey 2 weeks after the reminder card. The institutional review board approved the study.

Survey Analysis

Two hundred eighty two surveys were returned for an overall response rate of 69%. Twenty-four surveys were excluded because the respondent’s position within the practice could not be identified. Providers classified as support staff (radiology technician, referral coordinator, ambulatory office representative, or receptionist) responded “not applicable” to more than 25% of the items. Therefore, they were removed from the data set (n = 68). Final analyses were limited to the following provider groups: physicians, nurses, managers, and medical assistants, and included 190 surveys (46% of all mailed and 67% of all returned surveys).

For the SAQ-A, we tested the 6-factor structure of the SAQ that was previously developed after extensive exploratory and confirmatory factor analyses (CFA).¹¹ Here we evaluated the 6-factor structure using CFA and the following measures of model fit¹⁸: the comparative fit index (CFI), the Tucker–Lewis index (TLI), and the root mean square error of approximation (RMSEA) using the recommended cutoff values of greater than 0.90 for the CFI and TLI, and less than 0.08 for the RMSEA using AMOS 4.01 (Chicago, IL, USA), and SPSS 13.0 (Chicago, IL, USA). The 5 ambulatory-process-of-care items were not included in the factor analysis so the factor structure of the SAQ-A might remain similar to prior versions of the survey.¹¹ Thirty items comprised 6 scales: job satisfaction, perceptions of management, safety climate, working conditions, stress recognition, and teamwork climate. Twenty-seven items were eliminated from the original inpatient version of the scales¹¹ but are reported here because they may increase understanding of a given work environment.

We scored each scale by first converting the 5-point Likert scale to a 100-point scale as follows: 1 = 0, 2 = 25, 3 = 50, 4 = 75, and 5 = 100. Items were reversed scored when necessary so the higher the score, the more positive the attitude. Responses to each item in a scale were summed then divided by the number of items in that scale to create a scale score that ranged from 0 to 100. Scores are reported as the percentage of respondents who have positive attitudes toward each factor (score ≥ 75). These percentages are equivalent to scores 4 or 5 on the Likert scale (agree or strongly agree). Internal consistency, a type of reliability, was measured using Cronbach’s alpha.

The means and standard deviations of scale scores were calculated for the 4 types of providers: physicians, nurses, managers, and medical assistants. Mean scores for provider types were compared using analysis of variance (ANOVA) with statistical significance defined as P < 0.05. When ANOVA indicated a significant difference, follow-up comparisons were done using Duncan’s multiple range test. We did not test for differences among providers’ clinical specialties due to small sample sizes. We performed statistical analysis using SAS version 8.2.

Results

The majority of the 251 respondents were physicians (58%) and female (70%) (Table 1). On average, all the groups had at least 6 years of experience. A confirmatory factor analysis tested the validity of the 6-factor structure of the SAQ-A. The 6-factor model fit the data well (CFI = 0.973, TLI = 0.977, RMSEA = 0.067). Internal consistency was good for the 6 scales: job satisfaction (alpha = 0.86), perceptions of management (alpha = 0.72), safety climate (alpha = 0.76), working conditions (alpha = 0.68), stress recognition (alpha = 0.72), and teamwork climate (alpha = 0.82). Appendix (available online) shows the items in each scale, and Table 2 shows the mean scores and standard deviations for each scale by provider type.

Table 1.

Characteristics of the 251 providers

	Physicians (n = 111)	Nurses (n = 30)	Managers (n = 10)	Medical assistants (n = 39)	Support staff* (n = 61)
Mean age (SD)	44.7 (10.6)	39.2 (9.3)	35.1 (8.4)	34.0 (9.0)	32.9 (9.7)
Mean years in current position (SD)	14.9 (10.3)	11.2 (9.3)	9.7 (7.1)	6.7 (4.7)	7.3 (5.8)
Mean years at organization (SD)	8.5 (7.3)	4.3 (5.2)	3.9 (4.6)	2.4 (1.8)	2.1 (2.3)
Female, n (%)	43 (39)	30 (80)	10 (100)	34 (89)	58 (95)

Some subjects did not responded to all questions, so the number of respondents may vary by 1 to 6.

*Support staff were not included in final analyses.

Table 2.

Percentage of providers with positive safety attitudes and mean scale scores by provider type

	Physicians (n = 111)	Nurses (n = 30)	Managers (n = 10)	Medical assistants (n = 39)
Teamwork climate
% with positive attitude	52	58	70	45
Mean score (SD)	73.7 (21.5)	76.0 (21.3)	79.6 (19.9)	71.8 (24.4)
Safety climate
% with positive attitude	39	65	60	48
Mean score (SD)	68.5 (19.5)	75.5 (15.7)	76.3 (21.9)	68.8 (17.8)
Perception of management*
% with positive attitude	18	32	70	33
Mean score (SD)	50.43 (22.5)	58.0 (27.7)	72.5 (19.6)	58.0 (26.5)
Job satisfaction
% with positive attitude	48	47	80	51
Mean score (SD)	69.9 (21.3)	70.8 (22.3)	83.0 (16.5)	71.9 (24.4)
Working conditions
% with positive attitude	29	36	60	35
Mean score (SD)	60.1 (22.5)	58.7 (28.55)	66.7 (25.0)	60.1 (23.3)
Stress recognition†
% with positive attitude	37	45	40	20
Mean score (SD)	59.1 (24.4)	66.0 (24.0)	61.7 (20.5)	47.2 (27.3)

Positive attitudes were defined as having scale scores >75, the equivalent of agree or strongly agree on the Likert scale used for the response options. Statistical comparisons are based on the mean scores.

*P < 0.05 based on ANOVA. Duncan’s multiple range test indicated that the means scores for physicians and managers differed significantly.

†P < 0.05 based on ANOVA. Duncan’s multiple range test indicated that the means scores for physicians and medical assistants differed significantly.

Only 18% of physicians had positive perceptions of management, compared to 32% of nurses, 33% of medical assistants, and 70% of managers. There were statistically significant differences in mean scores between managers and physicians (72.5 ± 19.6 vs 50.4 ± 22.5, respectively, P < 0.05) (Table 2). Less than half of all provider groups had positive stress recognition scores (positive scores indicate greater acknowledgement of the effects of stress). The percentage of nurses (45%) with positive stress recognition was more than double the percentage of medical assistants (20%) reporting stress recognition and the difference in their mean scores was significant (66.0 ± 24.0 vs 47.2 ± 27.3, respectively, P < 0.05).

Other differences among providers were not statistically significant, but still concerning for the low scores. Only 39% of physicians had a positive attitude towards safety climate and less than half of the physicians and nurses were satisfied with their jobs (47% and 45%, respectively). Physicians, nurses, and medical assistants had relatively similar, and low, perceptions of working conditions, as compared to the managers (29%, 36%, and 35%, respectively). Finally, the 4 provider types had similar teamwork scores ranging from 45% of medical assistants to 70% of managers with favorable responses.

Ambulatory Process of Care Items

Few providers felt there was “adequate and timely transfer of patient information between primary care physician and specialist” (Table 3). Only 31% of physicians were satisfied with the current referral process in their offices. Sixty-six percent of all provider types felt that “abnormal test results are frequently lost or overlooked.” Conversely, the majority of providers felt “medications are filled correctly.”

Table 3.

Percentage of providers with favorable attitudes toward the ambulatory process of care items

	Physicians (n = 111)	Nurses (n = 30)	Managers (n = 10)	Medical assistants (n = 40)	Overall
I am satisfied with the current referral process in this office	31	36	56	54	38
There is adequate and timely transfer of patient information between primary care physician and specialist	27	36	25	47	32
Medications are refilled in a timely manner	53	80	33	66	60
Medications are filled correctly	72	87	50	74	74
Abnormal test results are frequently lost or overlooked	62	67	83	74	66

Discussion

Provider attitudes about topics relevant to patient safety have been measured in acute care settings, but little is known about these attitudes in ambulatory settings. We created the SAQ-A that measures provider attitudes about job satisfaction, perceptions of management, teamwork, safety climate, working conditions, and stress recognition. Our confirmatory factor analysis found that the ambulatory version of the SAQ had the same factor structure as the original SAQ. In addition, all 6 SAQ-A scales have good internal reliability and can be used in future efforts to measure the safety climate or safety “culture” of ambulatory practices.

There were some notable differences in scores among types of providers. Very few physicians had positive assessments of management, in stark contrast to the percentage of managers having a positive perception of themselves. The other statistically significant finding was that nurses had the highest stress recognition scores and medical assistants the lowest. Higher stress recognition scores indicate more recognition of the effects of stress on the ability of a provider to perform optimally in delivering safe care. Although not statistically significant, managers and nurses had the highest safety attitude scores. This is consistent with other research that consistently reports better attitudes from those at the top of the hierarchy in organizations. While it can be helpful for leaders to have positive attitudes, it may be a problem if their attitudes reflect an unrealistic view of the practice or if their attitudes are markedly different from those of others.¹⁰

Our study has several limitations. First, support staff such as referral coordinators, radiology technicians, and receptionists were excluded because approximately 25% of the items did not apply to them. The SAQ-A should be adapted for support staff that have direct patient contact. Second, our data are from one academic practice and may not be generalizable to other settings. The SAQ-A needs to be tested in other outpatient settings. Third, our small sample size made comparisons among provider types difficult. For example, the difference in mean job satisfaction scores for managers (83.0 ± 16.5) compared to physicians (69.9 ± 22.5) and nurses (70.8 ± 22.3) appears to be important, but was not statistically significant. In addition, we could not analyze differences among medical specialties due to small samples. Finally, the validity of the SAQ-A needs evaluation. We are identifying links between inpatient safety attitudes and lengths of stay and nurse turnover rates,¹⁹ but we do not know if ambulatory safety attitudes correlate with other measures of safety, or if changing safety-related attitudes changes provider behavior. Research needs to be done to define the relationship between safety attitudes, error occurrence, and behavior change.

Why would a practice use this survey? In the hospital setting, many leaders find attitudinal data, such as those produced by the SAQ-A, valuable.¹³ Second, measuring these attitudes is one way of estimating the safety culture of a practice, an activity strongly endorsed by the National Quality Forum.²⁰ For example, the Institute for Healthcare Improvement encouraged hospitals to use an expanded version of the safety climate scale from the inpatient version of our measure, the SAQ,²¹ to compare and contrast attitudes among units within hospitals to focus attention on safety. An ambulatory practice could do the same. Third, an ambulatory practice may gain insight into the complexities and needs of different providers and subspecialties. The survey can also be used to measure changes in attitudes before and after initiatives to improve safety. For example, surveys are being used to measure changes in attitudes after executive walkrounds¹³ and comprehensive safety interventions in ICUs.¹⁹ Finally, there is some evidence that health care teams can improve primary care practice.²¹ As more attention is paid to primary care teams,²² our teamwork scale can be used to measure primary care provider attitudes about teamwork.

In conclusion, the SAQ-A is the first survey available to measure ambulatory safety-related attitudes and assess the safety culture of practices. Some providers had low attitude scores and there were marked differences in attitudes among some providers. These findings point to opportunities to improve provider attitudes, and for groups of providers and specialties to learn from one another. For additional information on the SAQ-A and similar surveys, please visit http://www.utpatientsafety.org.

Appendix

Items in the Ambulatory SAQ

Teamwork climate scale (6 items, alpha = 0.82)

In this office, it is difficult to speak up if I perceive a problem with patient care*

The physicians and nurses here work together as a well-coordinated team

Disagreements in this office are appropriately resolved (i.e., not who is right but what is best for patient)

Nurse input is well received in this office

I have the support I need from other personnel to care for patients

It is easy for personnel in this office to ask questions when there is something that they do not understand

Safety climate scale (7 items, alpha = 0.76)

I am encouraged by my colleagues to report any patient safety concerns I may have

The culture in this office makes it easy to learn from errors of others

Medical errors are handled appropriately in this office

I know the proper channels to direct questions regarding patient safety in this office

I receive appropriate feedback about my performance

I would feel safe being treated here as a patient

In this office, it is difficult to discuss errors*

Perception of management scale (4 items, alpha = 0.72)

Senior management of this office is doing a good job

The management of this office supports my daily efforts

I am provided with adequate, timely information about events in the hospital that might affect my work

The levels of staffing in this office are sufficient to handle the number of patients

Job satisfaction scale (5 items, alpha = 0.86)

This office is a good place to work

I am proud to work in this office

Working in this office is like being part of a large family

Morale in this office is high

I like my job

Working conditions scale (4 items, alpha = 0.68)

This office does a good job of training new personnel

This office constructively deals with problem physicians and employees

All the necessary information for diagnostic and therapeutic decisions is routinely available to me

Trainees in my discipline are adequately supervised

Stress recognition scale (4 items, alpha = 0.72)

When my workload becomes excessive, my performance is impaired

I am more likely to make errors in tense or hostile situations

Fatigue impairs my performance during emergency situations (e.g., code or cardiac arrest)

I am less effective at work when I am fatigued

Items not included in the scales (32 items)

Ambulatory Process of Care Items (5 items)

I am satisfied with the current referral process in this office

There is adequate and timely transfer of patient information between primary care physician and specialist

Medications are refilled in a timely manner

Medications are filled correctly

Abnormal test results are frequently lost or overlooked*

Others (27 items)

High levels of workload are common in this office*

Briefing other personnel before a procedure (e.g., biopsy) is important for patient safety

Briefings are common in this office

When I am interrupted, my patients’ safety is not affected*

The management of this office knowingly compromises the safety of patients*

Decision-making in this office should include more input from other personnel than it does now

This office encourages teamwork and cooperation amongst its personnel

The medical equipment in this office is adequate

I have seen others make errors that had the potential to harm patients

Stress from personal problems adversely affects my performance

Disruptions in the continuity of care (e.g., shift changes, patient transfers, etc.) can be detrimental to patient safety

During emergencies, I can predict what other office personnel are going to do next

I am frequently unable to express disagreement with attending physicians/primary care providers in this office*

Very high levels of workload stimulate and improve my performance*

Truly professional personnel can leave personal problems behind when working*

I know the first and last names of all the personnel I worked with during my last shift

I have made errors that had the potential to harm patients

Attending physicians/primary care providers in this office are doing a good job

All the personnel in this office take responsibility for patient safety

If necessary, I know how to report errors that happen in this office

Patient safety is constantly reinforced as the priority in this office

Interactions in this office are collegial, rather than hierarchical

Important issues are well communicated at shift changes

There is widespread adherence to clinical guidelines and evidence-based criteria regarding patient safety here

Personnel are not punished for errors reported through incident reports

During emergency situations (e.g., emergency resuscitations), my performance is not affected by working with inexperienced or less capable personnel*

Personnel frequently disregard rules or guidelines (e.g., handwashing, treatment protocols/clinical pathways, sterile field, etc.) that are established for this office*

*Reverse-scored items