Abstract
Background: The consideration of organisational aspects, such as shared goals and clear communication, within the health care team is important to ensure good quality care. In primary health care, the instrument Survey of Organizational Attributes for Primary Care (SOAPC) is available to measure organisational attributes of care. However, there is no instrument available for dental care. The aim of the present study was to investigate psychometric properties and test–retest reliability of the version of SOAPC adapted for dental care, namely the Survey of Organizational Attributes in Dental Care (SOADC). Methods: The SOADC consists of 21 items in the following four subscales: communication; decision making; stress/chaos; and history of change. Convergent construct validity was measured using the job satisfaction scale. A total of 287 dental-care practices were asked to participate in the validation study. Psychometric properties and test–retest reliability were observed. Results: A total of 43 dental-care practices responded to the survey. At baseline, 178 dental-care staff completed the questionnaire, and 4 weeks later 138 did so. Internal consistency, measured by Cronbach’s alpha, was 0.718 or higher in the subscales. The test–retest reliability for each subscale and the overall SOADC score demonstrated good correlations over the 4-week test–retest interval, except for ‘history of change’. A strong correlation with the aggregated job-satisfaction scale showed high convergent construct validity of SOADC. Conclusions: The consideration of organisational aspects from the perspective of dental-care teams is important for providing good quality of care. The SOADC is a reliable instrument with good psychometric properties and is suitable for the evaluation of organisational attributes in dental-care practices.
Key words: Health services research, oral health care, organisation of care, validation study
INTRODUCTION
Dental-care practices must provide high quality, safe care. It has been shown that high-quality services and high patient satisfaction are core aims of oral health-care but this depends on the organisational culture in each practice1., 2.. Generally, organisational culture means that an organisation has shared attitudes, values and norms of behaviour that influence health-care performance and quality of health care3., 4.. Furthermore, an integral part and important indicator of health-care performance of an organisation is patient safety. Patient safety has different definitions, focused on reduction and prevention of unnecessary harm5. An overview is given by Bailey et al.5. However, within the dental-care setting the consideration of patient safety and the exploration of potential errors is a relatively new concept6. Despite this underdeveloped area in dental care there are common essential elements for producing a safe organisational culture in medical and dental care, such as clear communication, shared goals and effective problem solving, including good co-operation within the team7. It has been shown that the safety culture varies between different medical profession groups8. Therefore, it is important and recommended to adapt measurements on safety culture in relation to each medical professional group and organisational structure. It was found that key elements of patient safety are patient-centred and focus on organisational aspects, such as teamwork, communication and leadership9.
One instrument that measures different aspects concerning organisational culture in health care is the Survey of Organizational Attributes for Primary Care (SOAPC)10. A version for general medical practices in Germany is already available but there is no version for dental practices11. A nationwide oral-health survey published in 2006 showed that over 70% of the German population visits their dentist regularly12. Oral health-care in Germany is mostly ambulatory care and a high proportion of practices are single-dentist practices. However, a projection by the National Association of Statutory Health Insurance Dentists showed that group dental practices are becoming more popular13. Therefore, it is important, especially for good quality care and patient safety, to consider organisational aspects within care teams. To our knowledge there is currently no instrument available that measures organisational aspects from the perspective of dental-care teams. The aim of this study was to develop, adapt and test a German version of the SOAPC instrument for dental-care teams.
METHODS
Recruitment and data collection
In one region of Baden-Wuerttemberg, Germany, 287 dental practices were invited (with an information letter) to participate in the validation study. Five questionnaires for each practice were handed out because the numbers of staff in the practice were unknown. Written consent was obtained from all participants, including dental practices and their individual staff members. Data collection took place between June and July 2012. At baseline (T0 = test), 178 dental-care staff completed the questionnaire, and 138 completed the questionnaire again 4 weeks later (T1 = retest). Therefore, a total of 138 data sets were available for reliability analysis. The protocol is presented in Figure 1.
Figure 1.
Flowchart of participating practices and their staff.
Measures
The German version of the SOAPC11 was adapted for dental care and renamed the Survey of Organizational Attributes in Dental Care (SOADC). Modification of the SOADC for the dental-care setting was conducted by a team of researchers and clinicians, including a sociologist, a health services researcher and dentists. The SOADC was piloted in two dental practices using the think aloud technique14. These dental practices were chosen through personal contact. Two dentists and four dental-care staff were invited to comment on the items in the SOADC. This offers insights into participants’ thoughts and beliefs about the content-related meaning of the items. The think aloud technique presents a method for developing and testing questionnaires15. All items from SOAPC were transferred, although the term ‘medical’ was changed to ‘dental’ and ‘physician’ was changed to ‘dentist’. The SOADC consists of 21 items that are scored using a 5-point Likert scale, ranging from strongly disagree (score = 1) to strongly agree (score = 5). The SOADC consists of four predefined subscales: communication (four items); decision making (eight items); stress/chaos (six items); and history of change (three items). Job satisfaction was measured using the 10-item job-satisfaction scale developed by Warr et al.16 and was used to determine convergent construct validity. The items were scored on a 7-point Likert scale, ranging from extreme dissatisfaction (score = 1) to extreme satisfaction (score = 7). Following a previous validation study, the construct of job satisfaction was used to measure convergent validity11. Job satisfaction consists of intrinsic and extrinsic factors, such as physical working condition, recognition of work, satisfaction with colleagues, remuneration and working hours. Moreover, it was shown that these different aspects are associated with different organisational attributes, such as ‘communication’, ‘decision-making’ and ‘stress’17. The internal consistency of the job satisfaction scale was 0.873. Practice characteristics were determined using a separate, short questionnaire that included mode and location of practice, documentation (paper based, electronic based or mixed), frequency of team meetings and implementation of a quality management system. Additionally, age, gender and time period of employment of dentists and dental staff were evaluated.
Statistical analysis
All analyses were carried out using SPSS 20.0 (SPSS Inc., Chicago, IL, USA). The aim was to assess the psychometric properties and test–retest reliability of the SOADC. Means, standard deviations of means and statistical moments were calculated for each item of the SOADC. Regarding published research, each subscale was calculated10., 11.. Internal consistency was assessed using Cronbach’s alpha, which indicates whether an item of a scale is appropriate for assessing the underlying concept of the scale18. Values for Cronbach’s alpha range from 0 to 1. The closer they are to 0, the less related the items are to one another. Values above 0.60 are generally considered to indicate satisfactory internal consistency, and those above 0.80 indicate high internal consistency. Furthermore, to examine the convergent construct validity, the aggregated score of the job satisfaction scale was correlated with each scale of SOADC using the non-parametric Spearman rank correlation (rrho). Test–retest reliability was assessed using the intraclass correlation coefficient (ICC) to determine the stability of the questionnaire. This criterion refers to the likelihood that a test will yield the same description of a phenomenon if the test is repeated and the phenomenon is unchanged19. Retest reliability is defined as correlation between the two test ratings. ICC scores range from −1 to 1, for which a score of 1 indicates the highest correspondence. SOADC scores showing r > 0.50 are considered to be very reliable. However, their reliability also depends on the expected stability of the investigated construct. The non-parametric Wilcoxon matched pairs test was used to test for differences between T0 and T1. If no significant differences were detected, the stability of the construct could be assumed. Moreover, the effect of practice group was evaluated by regression analyses. An alpha level of P ≤ 0.05 was considered to indicate statistical significance.
Ethical approval
Our research was conducted in full accordance with the World Medical Association Declaration of Helsinki. The study was fully approved by the Ethics Committee of the medical faculty of the University of Heidelberg, Germany (S-155/2012).
RESULTS
Of 287 practices invited to participate in the validation study, 50 responded (response rate = 17.4%). However, of these 50 practices, seven were no longer active. Therefore, our study sample consisted of 43 dental-care practices (see Figure 1). Four or five questionnaires were returned from each dental practice (mean = 4.5).
The dental practices were mainly in urban areas (62.8%) and were mostly single-dentist practices (53.5%). The practice characteristics are presented in Table 1.
Table 1.
Practice characteristics (n = 43)
| Practice characteristics | Value |
|---|---|
| Location | |
| Rural | 0 |
| Urban | 27 (62.8) |
| Suburban | 16 (37.2) |
| Mode of practice* | |
| Single-dentist | 23 (53.5) |
| Group practice | 17 (39.6) |
| Documentation | |
| Only paper-based | 0 |
| Only electronic | 10 (23.2) |
| Mixed | 33 (76.7) |
| Quality management | |
| Quality management system implemented | 28 (65.1) |
| Team meetings (yearly) | 12.9 ± 15.2 |
Values are given as n (%) or mean ± standard deviation.
Values do not sum to 43 because of missing data.
At baseline, 178 dental-care staff (43 dentists and 135 dental staff) completed the questionnaire. The mean age of dentists was 49 years and of dental staff was 34.4 years. Descriptions of dentist and dental staff characteristics are given in Table 2.
Table 2.
Dentist and dental staff characteristics, n = 178
| Characteristics | Dentists | Dental staff |
|---|---|---|
| Number | 43 (24.2) | 135 (75.8) |
| Age (years) | 49.0 ± 12.6 | 34.4 ± 11.8 |
| Gender* | ||
| Female | 24 (55.8) | 132 (97.8) |
| Male | 18 (41.9) | 0 |
| Time period of employment (years) | 21.8 ± 10.4 | 9.4 ± 8.9 |
Values are given as n (%) or mean ± standard deviation. *Values do not sum to 178 because of missing data.
Table 3 shows the distribution and moment for each item of SOADC at baseline. The skewness of most of the items was within tolerable levels, tending to be fairly close to zero. However, only the item ‘This practice encourages nursing and clinical staff input for making changes and improvements’ was symmetrically distributed (skew = 0.02). A majority of the kurtosis values were removed substantially from zero, which is acceptable except for the two items: ‘When there is a conflict in this practice, the people involved usually resolve the problem successfully’ (kurtosis = −0.02); and ‘The leadership in this practice is available for consultation on problems’ (kurtosis = −0.06).
Table 3.
Distribution and moment for each item of the Survey of Organizational Attributes in Dental Care (SOADC)
| Items and subscales of SOADC | Mean (SD) | Skew | Kurtosis |
|---|---|---|---|
| Communication | |||
| 1 When there is a conflict in this practice, the people involved usually resolve the problem successfully* | 3.62 (0.96) | −0.64 | −0.02 |
| 2 Our staff has constructive work relationships* | 3.77 (0.79) | −0.35 | −0.18 |
| 3 There is often tension between people in this practice† | 3.56 (1.05) | −0.64 | −0.25 |
| 4 The dental staff and the dentists in this practice operate as a real team* | 3.84 (0.93) | −0.78 | 0.42 |
| Decision making | |||
| 5 This practice encourages staff input for making changes and improvements* | 3.72 (0.99) | −0.75 | 0.24 |
| 6 This practice encourages nursing and clinical staff input for making changes and improvements* | 3.65 (0.88) | 0.02 | −0.51 |
| 7 All of the staff participate in important decisions about the clinical operation* | 3.34 (1.19) | 0.34 | 1.69 |
| 8 Practice leadership discourages dental staff from taking initiative† | 4.30 (0.77) | −1.24 | 1.91 |
| 9 This is a very hierarchical organization; decisions are made at the top with little input from those doing the work† | 3.53 (1.01) | −0.44 | −0.46 |
| 10 The leadership in this practice is available for consultation on problems* | 4.11 (0.85) | −0.73 | −0.06 |
| 11 The practice defines success as a teamwork and concern for people* | 3.68 (1.12) | −0.66 | −0.33 |
| 12 Dental staff are involved in developing plans for improving quality* | 3.89 (0.81) | −0.76 | 0.77 |
| Stress/chaos | |||
| 13 It is hard to make any changes in this practice because we are so busy seeing patients† | 3.26 (0.97) | −0.28 | −0.63 |
| 14 The dental staff members of this practice very frequently feel overwhelmed by the work demands† | 3.57 (0.95) | −0.60 | 0.11 |
| 15 The dentists in this practice very frequently feel overwhelmed by the work demands† | 3.66 (0.88) | −0.23 | −0.38 |
| 16 Practice experienced as ‘stressful’† | 3.52 (0.98) | −0.68 | 0.13 |
| 17 This practice is almost always in chaos† | 4.21 (0.90) | −1.05 | 0.61 |
| 18 Things have been changing so fast in our practice that is hard to keep up with what is going on† | 4.00 (0.90) | −0.70 | 0.23 |
| History of change | |||
| 19 Our practice has changed in how it takes initiative to improve patient care* | 3.46 (0.80) | −0.35 | 0.19 |
| 20 Our practice has changed in how it does business* | 3.39 (0.85) | −0.45 | −0.10 |
| 21 Our practice has changed in how everyone relates* | 3.16 (0.90) | −0.17 | −0.16 |
SD standard deviation. Different direction of measurements should be considered within the analysis. *Five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). †Five-point scale ranging from 1 (strongly agree) to 5 (strongly disagree).
Of the 178 dental-care staff who completed the questionnaire at baseline, 138 completed the questionnaire again 4 weeks later (response rate = 77.5%) and thus those results were available for reliability analysis. The internal consistency, external validity and test–retest reliability for each subscale and overall scale of SOADC are presented in Table 4. The internal consistency between the four subscales ranged from α = 0.718 (‘history of change’) to α = 0.838 (‘decision making’). For the overall SOADC score the internal consistency was α = 0.775. The convergent construct validity was measured using the aggregated job-satisfaction scale and showed a positive correlation except for the subscale ‘history of change’ (r = −0.090). The matched pair tests identified no significant differences, except for the subscale stress/chaos (P = 0.022). On the level of items there was a significant difference in the item ‘the practice is almost always in chaos’. The ICC used to measure test–retest reliability ranged from 0.373 (for the subscale ‘history of change’) to 0.737 (for the subscale ‘decision making’). For the overall scale of SOADC, the test–retest reliability was ICC = 0.732.
Table 4.
Internal consistency, convergent construct validity and test–retest reliability for each subscale and overall scale of the Survey of Organizational Attributes in Dental Care (SOADC)*
| Internal consistency (Cronbach’s alpha) | Convergent construct validity with aggregated job satisfaction scale | T0 (n = 178) mean (SD) | T1 (n = 138) mean (SD) | T2 (n = 138) mean (SD) | Wilcoxon matched pairs test, P-value* | Test–retest reliability |
||
|---|---|---|---|---|---|---|---|---|
| rrho | ICC | P-value | ||||||
| Overall SOADC score | 0.775 | 0.589** | 3.65 (0.51) | 3.71 (0.47) | 3.64 (0.50) | 0.059 | 0.732 | <0.01 |
| Communication | 0.728 | 0.555** | 3.70 (0.68) | 3.77 (0.60) | 3.70 (0.72) | 0.237 | 0.649 | <0.01 |
| Decision making | 0.838 | 0.517** | 3.71 (0.73) | 3.79 (0.70) | 3.77 (0.65) | 0.572 | 0.737 | <0.01 |
| Stress/chaos | 0.807 | 0.467** | 3.71 (0.66) | 3.74 (0.61) | 3.62 (0.63) | 0.022 | 0.654 | <0.01 |
| History of change | 0.718 | −0.090 | 3.33 (0.69) | 3.38 (0.71) | 3.27 (0.68) | 0.108 | 0.373 | <0.01 |
ICC, intraclass correlation coefficient; SD, standard deviation; T1, T2 measurement points for retest.*Statistical significance at P < 0.05; **statistical significance at P < 0.01.
The regression analyses showed no effect of practice group on responses.
DISCUSSION
This validation study shows that the German version of the SOADC can be used as an instrument for the measurement of organisational attributes in dental-care teams. The results presented describe good psychometric properties and reliable measures for organisational attributes. Moreover, high internal consistency for each subscale and the overall score of the SOADC was observed. The significant correlation with the corresponding job-satisfaction scale indicates sufficient convergent construct validity with the exception of ‘history of change’, which was also found in a validation study for primary-care practices11. Moreover, the validation study for primary-care practices shows higher mean values in each subscale of the instrument11. It can be concluded that organisational attributes could be improved in dental-care teams.
Stability, as assessed using the Wilcoxon matched pairs test, was very good. Only for the subscale ‘stress/chaos’ was a change over time observed and this relates particularly to the item ‘this practice is almost always in chaos’. The test–retest reliability for each subscale and overall score of SOADC demonstrated good correlations over the 4-week test–retest interval, except for ‘history of change’.
The subscale ‘stress/chaos’ depends mainly on organisational aspects as well as on occupational demands. It was found that the opportunity to use one’s ability leads to better perception of the working condition20. Therefore, it could be assumed that clear responsibilities are helpful for distribution of work and for reducing stress. Moreover, to invest in a clear structure and a process of care with clear responsibilities reduces adverse events and increases patient safety5. Furthermore, a survey for the members of National Dental Association showed that, amongst other topics, team work, quality of care and quality assurance are important for patient safety and risk management6. The SOADC evaluates not only the aspects that are important for teamwork but also those which ensure a good structure and process of care. It has already been demonstrated that implementation of a quality-management system could improve quality of care21. Additional to a quality-management system is the description of organisational attributes, such as with SOADC, which seems important for achieving good quality of care. However, to our knowledge there is currently only one study that has used the instrument in primary care and the findings showed that job satisfaction is associated with ‘communication’, ‘decision making’ and ‘stress’17. Further research in the field of organisational attributes using SOAPC or SOADC is needed, not only for primary care but also for dental care.
Strengths and limitations
The validation of an instrument that measures organisational attributes is important for providing good quality, safe care. The study design allows test–retest reliability and responsiveness to change to be determined. We included a convenient sample of dental-care practices from one region in Germany. Our results have to be interpreted against the background of potential selection bias because of a moderate participation rate. Therefore, we cannot examine the levels of organisational culture between the types of practices. Moreover, we did not complete a power analysis to determine the necessary sample size. The results of the study are only explorative. However, in Germany, health-services research in oral health care is underdeveloped, and only a few studies exist on the structure and process of care in an ambulatory care setting. Future research should include follow-up to increase the response rate. Finally, most of the practices (n = 28) had implemented a quality management system and frequent team meetings, which could have an impact on the results presented.
CONCLUSIONS
Overall, the SOADC is a reliable instrument with good psychometric properties and is suitable for the evaluation of organisational attributes in dental-care practices. Our study is an important step for the consideration of organisational aspects from the perspective of dental care teams. It is hoped that the results will lead to more research in this area. Moreover, the availability of this instrument encourages further research in the field of organisational aspects within dental-care teams in other countries.
Acknowledgements
The authors would like to thank the practices and their staff for participating in the validation study.
Source of funding
The study was funded by the Department of General Practice and Health Services Research.
Conflict of interest
The authors declare that they have no conflict of interest.
References
- 1.Dewi FD, Sudjana G, Marty Oesman Y. Patient satisfaction analysis on service quality of dental health care based on empathy and responsiveness. Dent Res J. 2011;8:172–177. doi: 10.4103/1735-3327.86032. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Poorterman JHG, Van Weert CM, Eijkman MAJ. Quality assurance in dentistry: the Dutch approach. Int J Qual Health Care. 1998;10:345–350. doi: 10.1093/intqhc/10.4.345. [DOI] [PubMed] [Google Scholar]
- 3.Davies HTO, Nutley SM, Mannion R. Organisational culture and quality of health care. Qual Health Care. 2000;9:111–119. doi: 10.1136/qhc.9.2.111. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Scott T, Mannion R, Marshall M, et al. Does organisational culture influence health care performance? A review of the evidence. J Health Serv Res Policy. 2003;8:105–117. doi: 10.1258/135581903321466085. [DOI] [PubMed] [Google Scholar]
- 5.Bailey E, Tickle M, Campbell S. Patient safety in primary care dentistry: where are we now? Br Dent J. 2014;217:339–344. doi: 10.1038/sj.bdj.2014.857. [DOI] [PubMed] [Google Scholar]
- 6.Yamalik N, van Dijk W. Analysis of the attitudes and needs/demands of dental practitioners in the field of patient safety and risk management. Int Dent J. 2013;63:291–297. doi: 10.1111/idj.12081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Ramanujam R, Rousseau DM. The challenges are organizational not just clinical. J Organiz Behav. 2006;27:811–827. [Google Scholar]
- 8.Verbakel NJ, Van Melle M, Langelaan M, et al. Exploring patient safety culture in primary care. Int J Qual Health Care. 2014;26:585–591. doi: 10.1093/intqhc/mzu074. [DOI] [PubMed] [Google Scholar]
- 9.Sammer CE, Lykens K, Singh KP, et al. What is patient safety culture? A review of the literature. J Nurs Scholarsh. 2010;42:156–165. doi: 10.1111/j.1547-5069.2009.01330.x. [DOI] [PubMed] [Google Scholar]
- 10.Ohman-Strickland PA, Orzano AJ, Nutting PA, et al. Measuring organizational attributes of primary care practices: development of a new instrument. Health Serv Res. 2007;42:1257–1273. doi: 10.1111/j.1475-6773.2006.00644.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Ose D, Freund T, Kunz CU, et al. Measuring organizational attributes in primary care: a validation study in Germany. J Eval Clin Pract. 2010;16:1289–1294. doi: 10.1111/j.1365-2753.2009.01330.x. [DOI] [PubMed] [Google Scholar]
- 12.Micheelis W, Schiffner U. Deutscher Zahnärzte Verlag; Kurzfassung. Köln: 2006. Vierte Deutsche Mundgesundheitsstudie (DSMIV) [Google Scholar]
- 13.Bauer J, Neumann T, Saekel R. Verlag Hans Huber; Bern: 2009. Zahnmedizinische Versorgung in Deutschland; pp. 168–183. [Google Scholar]
- 14.Koro-Ljungberg M, Douglas PE, Therriault D, et al. Reconceptualizing and decentering think-aloud methodology in qualitative research. Qualitative Res. 2012;13:735–753. [Google Scholar]
- 15.Pohontsch N, Meyer T. Cognitive interviewing – a tool to develop and validate questionnaires. Rehabilitation. 2015;54:53–59. doi: 10.1055/s-0034-1394443. [DOI] [PubMed] [Google Scholar]
- 16.Warr PJ, Cook J, Wall T. Scales for the measurement of some work attitudes and aspects of psychological well-being. J Occup Psychol. 1979;52:129–148. [Google Scholar]
- 17.Gavartina A, Zaroti S, Szecsenyi J, et al. Practice assistants in primary care in Germany – associations with organizational attributes on job satisfaction. BMC Fam Pract. 2013;14:110. doi: 10.1186/1471-2296-14-110. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cronbach LJ. Coefficient Alpha and the internal structure of tests. Psychometrika. 1951;16:297–334. [Google Scholar]
- 19.Roussin V, Gasser T, Seifert B. Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat Med. 2002;21:3431–3446. doi: 10.1002/sim.1253. [DOI] [PubMed] [Google Scholar]
- 20.Goetz K, Campbell SM, Broge B, et al. The impact of intrinsic and extrinsic factors on the job satisfaction of dentists. Community Dent Oral Epidemiol. 2012;40:474–480. doi: 10.1111/j.1600-0528.2012.00693.x. [DOI] [PubMed] [Google Scholar]
- 21.Goetz K, Campbell SM, Broge B, et al. Effectiveness of a quality management program in dental care practices. BMC Oral Health. 2014;14:41. doi: 10.1186/1472-6831-14-41. [DOI] [PMC free article] [PubMed] [Google Scholar]