Validation of Dimensions of Learning Organization Questionnaire (DLOQ) in health care setting in Greece

Abstract

Background: In health care systems the organizational learning is a continuous process to improve actions through better knowledge and detect or correct errors. This study examines the validity and reliability of the Dimensions of Learning Organization Questionnaire in addition to the applicability of the instrument in a Greek health care cultural context.

Design and methods: A cross-sectional study was carried out in 6 general hospitals of Attica and the sampling scheme was the stratified sampling. Questionnaires were distributed to 487 healthcare professionals and 380 valid questionnaires were returned. The research tool used in this survey is the extensive form of DLOQ, which has been adapted and translated into Greek language. Data analysis was carried out with SPSS 25.

Results: Three leading experts of the health sector examined the face validity assessment of the translated DLOQ and stated that it is characterized by high face validity. As regard construct validity of the DOLQ throughout Multitrait-Multimethod Matrix, was proved that all the variables of the same factors are statistically significantly correlated (p<0.001), and their correlation coefficients have moderate to high power ranging between 0.563 and 0.798. Moreover, the discriminant validity was demonstrated as certain correlation coefficients between variables of different factors were found to be higher than of certain correlation coefficients between variables of the same factors. The internal consistency among the items of the DOLQ range between 0.842 and 0.977 and they are considered to be good to excellent.

Conclusions: Our results confirmed that DLOQ is a valuable tool in measuring Learning Organizational in Greek public hospitals.

Significance for public health.

Organizational learning process is not only necessary but also essential for patient safety and healthcare quality of Public Health. The importance of organizational learning in health care systems is to provide the framework for complex interconnected dynamic systems where all operational units must learn and execute their assigned functions to collectively improve safe patient care. Learning organization comprises the modern organization model that allows its members to continuously ameliorate their knowledge in order to make them equipped and adapt to any changes within a continuously changing environment. Policies and procedures are developed in healthcare organizations to reduce errors and improve patient safety. Healthcare professionals are expected to engage in continuing education to maintain and update knowledge and skills to provide safe patient healthcare as continuing education of health care professionals has shown to be related to improved patient outcomes. This research provides a valuable tool in measuring Learning Organizational in Greek public hospitals.

Introduction

Historically, during periods of recession or development, and especially in periods of financial instability and uncertainty, labor market and society are rapidly changing. Organizations/enterprises as well as the systems related to them are called upon to respond to, cope with and anticipate these changes. Many researchers, such as Senge¹ and Argyris², have suggested that organizations have succeeded in these economic and social challenges, only through the process of learning.

A learning organization inspires, enhances and facilitates the continuous learning process of its members, aiming at its own improvement, change and transformation so as not only to challenge but also to overcome. It is basically a process through in which the members of an organization learn from their interaction with others, (such as colleagues, customers, etc.) through the exchange of sharing experiences and information and the problems solving. In addition to the development and management of the employees’ competencies and skills, other factors are of essence, such as team and cooperation spirit, communication capacity and creativity, responsiveness to change, as well as employees’ commitment to achieve the goals of the organization.³ The fundamental tools of the process of learning are dialogue and discussion.

According to Senge⁴ the concept of learning in an organization is a process through which its employees are constantly developing their abilities, competences and skills, they cultivate new ways of thinking and learn how to coexist collectively. There are five main components of an organization that learns:^4-5 i) Systems Thinking; ii) Personal Mastery; iii) Team Learning; iv) Mental Models; and v) Building Shared Vision.

Pedler et al.⁶ define an Organization that Learns as an organization that teaches and facilitates to all of its members the way of learning and which is constantly transforming. Authors have suggested the following eleven key features of an organization that learns: i) learning approach to strategy, ii) participative policy making iii) continuous information (informating/informing), iv) formative accounting and control, v) internal exchange, vi) reward flexibility, vii) enabling structures, viii) boundary workers as environmental scanners, ix) inter-company learning, x) learning climate, xi) self-development for all.

This study has adapted Watkin’s and Marsick’s⁷ definition of learning organization. The researchers argue that “a learning organization is an organization that has an enhanced capacity to learn and change. In order to learn and change, an organization has to both works with people at the individual and group level, as well as creates facilitative structures to support and capture learning”. With regard to health services, the same authors argue that organizational learning refers to “organizations’ capacity to learn and to change to meet current health demands”.⁸

The main responsibility of organizational learning, in health care systems, is to provide the operational scheme and the requirements to complex interconnected dynamic systems, to provide better and higher quality health care services regarding their roles and their responsibilities on how to carry out their assigned functions, so that all the operational units can be fully informed.⁷ It is worth noting that organizational learning in health care is a continuous organizational process and not a onetime intervention. Continuous education through formal and informal learning methods and techniques, sustain and update the knowledge and skills of health professionals, which are associated with organizational change, leading to an improvement in patient outcomes.⁸ Organizational change enhances organizational learning by implementing better knowledge practices that lead to better decision-making and increased efficiency of health care systems.

In Greece, public health units are distinguished by internal orientation, compliance with rules and processes, as well as an emphasis on control hierarchy, predictability and stability, with the absence of decision-making involvement of employees. A centralized, bureaucratic structure is an obstacle to the transformation of health care units into learning organizations.³

Validation background of DLOQ

Decision-making uses learning organization to describe existing practices^11-13 to analyze a situation to guide actions and to change practices within a health service organization.^14-16 The learning organization can be a powerful tool to promote learning within the health sector.

In this sense, a practical and validated tool for effectively measuring the learning culture as a supportive system of the organizational learning process is not only necessary but also essential. Marsick and Watkins⁷ have developed the Dimensions of the Learning Organization Questionnaire (DLOQ) which is a scale that assesses the organizational ability to adapt to change by measuring employees’ perceptions. The constituent questions of the scale make up seven dimensions that measure the positive impact and the cultural features of a supportive learning organization. These dimensions are the following: i) continuous learning; ii) inquiry and dialogue; iii) team learning; iv) embedded systems; v) empowerment; vi) system connection; and vii) strategic leadership. The DLOQ has been translated into more than 15 languages during last years and has been used in several countries around the world.¹⁷ The seven dimensions of the DLOQ were originally assessed with 43 items.¹⁷ Two shorter versions of the DLOQ have also been created. One that consists of 21 questions which compose the same seven dimensions and another one even shorter, which consists of 7 questions that compose the same, subscales as well.¹⁸ Both of these versions of the questionnaire have been tested and validated.

Many different sectors of economic activity and in various countries with different cultural context such as USA, China, Colombia, Taiwan, Korea, Germany, Romania^18-24 have been used, tested and validated the DLOQ. These studies have been able to check the applicability of DLOQ in culturally diverse cultures by providing the internal consistency of the reliability of each item (the alpha coefficient ranges from 0.71 to 0.91).²² Regarding the use of the tool in health care services, research in different countries and cultures, recorded its reliability and validity too. These researches also demonstrated that the tool could offer a guide on where a health organization leader should focus his efforts in order to achieve strategically changes.^8-9,25,26

In Greece, the extensive form of DLOQ has been translated, validated and published regarding enterprises engaging in Advertising and Media Industry,²⁷ while the 21-item short form has been found to be more suitable for the education sector.²⁸ Regarding the health care services in Greece and the uniqueness of product “health”, there is a lack of translated and standardized version of DLOQ, which captures the needs of health care professionals and therefore the diversity of the Greek health care system, comparing free market economy. Thus, in order to use an instrument universally, testing it in different cultural contexts is crucial.

This study can therefore help and contribute to organizational learning research in Greece by evaluating the validity and reliability of the DLOQ, in addition to the instrument’s applicability in the cultural context of Greece. The findings of this study will assist decision-makers in health care organization in effectively implementing learning organization strategies with confirmed learning frameworks.

Design and Methods

Participants and procedure

The survey was conducted in 6 general hospitals of Attica. The selection criterion of these hospitals was the wide range of the healthcare services they offer and the large number of patients they accommodate and serve on a daily basis. The study design was cross-sectional, and the sampling scheme was the stratified sampling. The stratification process was based on the specialty of the participants, so, our population was divided into mutually exclusive sub-groups (doctors, nurses, administrative and paramedical staff) and then a simple random sampling was applied in each stratum in order to select a random sample of each subgroup. The objective of the selected method was to improve the precision of the sample by reducing the sampling error. A written consent form was filled in by each one of the respondents and they were informed that all the information was confidential and would be used for research purposes only.

Questionnaires were distributed to 487 healthcare professionals in the above-mentioned hospitals. 380 valid questionnaires were returned (this corresponds to a response rate of 78%). The study was carried out from 2019 December 17 to 2020 March 10.

Development of the research instrument

The research tool used in this survey is the extensive form of DLOQ (Dimensions of Learning Organization Questionnaire), which consists of 43 questions that compose the seven subscales of organizational learning as it has already mentioned. These subscales are the following: i) continuous learning (CL - 7 items); ii) inquiry and dialogue (I&D - 6 items); iii) team learning (TL - 6 items); iv) embedded system (ED - 6 items); v) empowerment (Em -6 items); vi) system connection (SC - 6 items); vii) strategic leadership (SL -6 items). The items included in each dimension were measured on 5-point Likert scales, ranging from 1-totally disagree to 5-totally agree.⁷ Through four stages of scrupulous translation, the questionnaire was adapted and translated into Greek: forward translation, evaluation, backward translation, and evaluation based on the requirements of consistency, common language, and cultural adequacy.^29,30

More specifically, the translation process is as follows. At first, three experts (one professor and two practitioners) in the field of Greek human resources reviewed the instrument in English. Then, the researchers assigned to a bilingual expert the translation of the instrument into Greek and to another bilingual expert its translation back into English so as to ensure and confirm the accuracy and the correctness of the translation. The initial version of the questionnaire resulted, after the thorough review of the differences that may have existed.

Subsequently, a pilot testing of the DLOQ was conducted on a number of 25 healthcare professionals. Respondents’ comments were incorporated into the second version of the questionnaire, which was distributed again on a sample of 10 healthcare professionals. The results indicated that the items in each dimension showed satisfactory clarity and coherence and thus, no further changes were made on the DLOQ instrument. The final version of the questionnaire consisted of two sections: the first one that gathers demographic information about the healthcare professionals such as gender, age, position level, specialty and education level and the second one which consists of the 43 questions that compose the seven dimensions of DLOQ.

Statistical analysis

Data analysis was carried out with SPSS 25. The seven dimensions of organizational learning were calculated as mean values of the variables/questions that compose each one of them. The correlations between the variables of each dimension were assessed with the use of the non-parametric Spearman’s Rho correlation coefficient, since all the items were measured in 6-point Likert scale. Spearman’s Rho correlation coefficient was also used to assess the relationships between the seven dimensions of OL, since they are not normally distributed. The level of statistical significance was set to α=0.05. As the main objective of this research was the adaptation and implementation of the DLOQ in the Greek cultural context, the assessment of its validity and reliability followed its translation in Greek. Two forms of validity were assessed, face validity and construct validity. Face validity is the most subjective one but just as important as the other validity forms. This type of validity is very difficult, if not impossible; to be measured since it is not based on concrete statistical or other scientific criteria but on the subjective evaluation of experts.^31,32 In our case in order to assess the face validity of the translated DLOQ, we asked from three leading experts of the health sector to evaluate it and we got their feedback of whether it appears to measure what we intend to measure.

Construct validity consists of two validity subsets: convergent validity and discriminant validity. If one can demonstrate that there is evidence of both convergent and discriminant validity, then there is evidence for construct validity.³³ In our case in order to confirm the construct validity of the translated DOLQ, the simplified form of Multitrait-Multimethod Matrix (MTMM) was used, which is a method developed by Campbell and Fiske (1959) that provides a rigorous and thorough framework for assessing/confirming the existence of convergent and discriminant validity of a questionnaire. ³⁴ For the reliability analysis of the questionnaire, Cronbach’s alpha coefficient was used, which is the most common internal consistency measure with values that range between 0 and 1. A scale, demonstrates internal consistency when Cronbach’s alpha coefficient is greater than 0.7.^35-37

Results

Descriptive analysis

DLOQ was distributed to 380 health professionals of the 6 Greek Public Hospitals in the region of Attica. Regarding the sampling frame, 70.5% were females and 29.5% were males. In terms of specialty, the majority of the participants (76.3%) were employees, 15.0% were heads of offices and the remaining 7.6% were heads of departments and Directors. As it was anticipated most of the health professionals that participated in the research (40.0%) are nurses, 31.8% are administrative staff, 21.6% are doctors and the rest 6.6% are paramedical personnel. It is worth noting that more than half of the participants are university graduates and they also hold a postgraduate title (a M.Sc. and/or a Ph.D.). Concerning their age distribution, most of them (43.4%) belong to the age group of (45-54) and 31.3% to the age group of (35-44). Finally, their average professional experience is 17.16±9.48, with a median value of 16.0 years (Table 1).

Validity of DOQL

As regards face validity assessment of the translated DLOQ, the questionnaire was thoroughly examined by three leading experts of the health sector. These experts evaluated the questionnaire and they found that it is characterized by high face validity, which is the desired outcome, i.e. it is evident from the questions, that the test measures what it is intended to measure.

Then in order to confirm construct validity of the DOLQ, as it is already mentioned, the simplified form of Multitrait- Multimethod Matrix (MTMM) was used, so as to assess convergent and discriminant validity. The statistical analysis showed that convergent validity is supported since it was found that all the variables of the same factors are statistically significantly correlated (p<0.001), and their correlation coefficients have moderate to high power. More specifically the values of Spearman’s Rho correlation coefficients among the variables/traits of each of the seven factors range: i) continuous learning between 0.245 and 0.626; ii) inquiry and dialogue between 0.432 and 0.714; iii) team learning between 0.467 and 0.732; iv) embedded systems between 0.464 and 0.719; v) empowerment between 0.596 and 0.798; vi) system connection from 0.596 to 0.798; and vii) strategic leadership between 0.633 and 0.812.

Table 1.

Sampling frame description.

		Frequency	%
Gender	Male	112	29.5
	Female	268	70.5
Position level	Employee	290	76.3
	Head of office	57	15.0
	Head of Depaartment	14	3.7
	Director	15	3.9
Specialty	Doctor	82	21.6
	Nurse	152	40.0
	Administrative staff	121	31.8
	Paramedical staff	25	6.6
Education level	Secondary education	83	21.8
	Technological education	91	23.9
	Higher education (university degree)	68	17.9
	M.Sc.	91	23.9
	Ph.D.	32	8.4
Age group	25-34	46	12.1
	35-44	119	31.3
	45-54	165	43.4
	55-64	46	12.1

Table 2.

Correlations between the factors of OL.

	Spearman’s Rho	CL	I&D	TL	ES	Em	SC	SL
CL	Correlation Coefficient	1
	Sig. (2-tailed)
I&D	Correlation Coefficient	0.752**	1
	Sig. (2-tailed)	0.000	.
TL	Correlation Coefficient	0.685**	0.728**	1
	Sig. (2-tailed)	0.000	0.000	.
ES	Correlation Coefficient	0.656**	0.615**	0.714**	1
	Sig. (2-tailed)	0.000	0.000	0.000	.
Em	Correlation Coefficient	0.579**	0.563**	0.724**	0.777**	1
	Sig. (2-tailed)	0.000	0.000	0.000	0.000	.
SC	Correlation Coefficient	0.604**	0.636**	0.722**	0.739**	0.796**	1
	Sig. (2-tailed)	0.000	0.000	0.000	0.000	0.000	.
SL	Correlation Coefficient	0.628**	0.638**	0.714**	0.754**	0.758**	0.798**	1
	Sig. (2-tailed)	0.000	0.000	0.000	0.000	0.000	0.000	.

CL, continuous learning; I&C, inquiry and dialogue; TL, team learning; ES, embedded system; Em, empowerment; SC, system connection; SL, strategic leadership

^**p<0.001.

Furthermore, as it is shown in Table 2, statistically significant correlations (p<0.001) exist between the hypothesized factors/dimensions ranging between 0.563 and 0.798, a result that according to Song²¹ and Yang¹⁷, imply adequate convergent validity. We highlight that there are no extremely high correlation coefficients between them, which might be an indication of discrimination restriction of the Organizational Learning’s factors.^38-40

As regards discriminant validity, statistical analysis showed that certain correlation coefficients between variables of different factors were found to be higher than of certain correlation coefficients between variables of the same factors. This is a fact that demonstrates moderate discriminant validity^41-43 of DLOQ in the selected sample.

Reliability of DLOQ

As previously mentioned, in order to measure internal consistency among the items of the DOLQ, Cronbach’s alpha coefficient was calculated separately for each section of the questionnaire that composes OL subscales, as well as for all the questions as a whole. Its values, as it is shown in Table 3, range between 0.842 and 0.977 and they are considered to be good to excellent. This result is a proof of the test’s internal consistency, meaning that the Greek version of DLOQ is reliable.

Discussion

It is important to note that the advantages of applying the learning organization in health services have been demonstrated by a plethora of international studies. Learning organization may cause organizational reform and promote learning in health sector, as long as health organizations are willing to adapt themselves and to respond to their changing environments. In this sense the Learning Organization Questionnaire (DLOQ) was developed by Marsick and Watkins⁷ in order to assess the organizational ability to adapt to change and to measure the learning culture of an organization, through seven dimensions. In Greece although the DLOQ has been translated, validated and used in different sectors, there is a lack of a translated and standardized version of DLOQ in the health sector.

Table 3.

Internal consistency results.

	Cronbach's alpha	N of items
CL	0.842	7
I&D	0.878	6
TL	0.893	6
ES	0.905	6
Em	0.932	6
SC	0.925	6
SL	0.938	6
OL total	0.977	43

CL, continuous learning; I&C, inquiry and dialogue; TL, team learning; ES, embedded system; Em, empowerment; SC, system connection; SL, strategic leadership; OL, organizational learning.

The main objective of this research was the adaptation and the implementation of the DLOQ in the Greek cultural context. The questionnaire has been adapted and translated into Greek language through the four steps of scrupulous translation: forward translation, assessment, backward translation, and final assessment. Its pilot testing was followed by two groups of healthcare experts and their comments were incorporated. The final version on the DLOQ was shared to the sample under study and the assessment of its validity and reliability was followed. Two forms of validity were assessed: 1) face validity and 2) construct validity. Regarding face validity assessment, the questionnaire was thoroughly examined by a group of leading health experts and they decided that it is characterized by high face validity, which was the desired outcome. As regards construct validity the simplified form of Multitrait-Multimethod Matrix (MTMM), was used so as to assess convergent and discriminant validity. It was found that convergent validity was supported since all the variables of the same factors were statistically significantly correlated, and their correlation coefficients had moderate to high power. Also, it was found that statistically significant correlations existed between the hypothesized factors/dimensions. As for discriminant validity, statistical analysis showed that certain correlation coefficients between variables of different factors were found to be higher than certain correlation coefficients between variables of the same factors. This fact demonstrates moderate discriminant validity.^41-43 of DLOQ in the selected sample.

From the above, we assumed that the Greek version of the DLOQ demonstrates moderate to satisfactory construct validity and it could be used to evaluate the learning culture in Greek health sector, since it is characterized by sufficient convergent validity and moderate discriminant validity.

Conclusions

Selected data confirmed that DLOQ is a valuable tool in measuring learning organizational in Greek public hospitals. The research concerns 6 general hospitals in the Attica basin. Further research could be done in provincial hospitals or private hospitals to see if there are differences in the instrument variables. Future academic studies should investigate how learning organizational principles can be effectively connected in practice to other basic concepts, such as job satisfaction, job performance or organizational commitment.