Translation and psychometric evaluation of the German version of the Organisational Readiness for Implementing Change measure (ORIC): a cross-sectional study

Abstract

Objectives

To translate the Organisational Readiness for Implementing Change measure into German and assess its psychometric properties.

Design

Cross-sectional psychometric study based on secondary analysis of baseline data from a shared decision-making implementation study.

Setting

Three departments within one academic cancer centre in Hamburg, Germany.

Participants

For comprehensibility assessment of the translated ORIC version, we conducted cognitive interviews with healthcare professionals (HCPs, n=11). Afterwards, HCPs (n=230) filled out the measure.

Primary and secondary outcome measures

The original English version of the ORIC was translated into German using a team translation protocol. Based on comprehensibility assessment via cognitive interviews with HCPs, the translated version was revised. We analysed acceptance (completion rate), factorial structure (exploratory factor analysis (EFA), confirmatory factor analysis (CFA), model fit), item characteristics (item difficulties, corrected item-total correlations, inter-item correlations) and internal consistency (Cronbach’s α).

Results

Translation and cognitive testing of the German ORIC was successful except for item 10, which showed low comprehensibility as part of content validity in cognitive interviews. Completion rate was >97%. EFA and CFA provided a one-factorial structure. Item difficulties ranged between 55.98 and 65.32, corrected item-total-correlation ranged between 0.665 and 0.774, inter-item correlations ranged between 0.434 and 0.723 and Cronbach’s α was 0.93.

Conclusions

The German ORIC is a reliable measure with high completion rates and satisfying psychometric properties. A one-factorial structure of the German ORIC was confirmed. Item 10 showed limited comprehensibility and therefore reduces content validity of the measure. The German ORIC can be used to analyse organisational readiness for change as a precursor for implementation success of various interventions.

Strengths and limitations of this study.

• Appropriate qualitative methods were used to provide a German version of the Organisational Readiness for Implementing Change (ORIC) to be assessed for healthcare professionals.

• The sample size was large enough to robustly perform the psychometric analyses on the German version of the ORIC measure.

• Due to the design of the study as a secondary analysis, it was not possible to calculate psychometric parameters like convergent and divergent validity.

• Data were collected at a single academic cancer centre in the context of a shared decision-making implementation study. Thus, psychometric properties of the German ORIC need replication in other settings.

Introduction

Implementing interventions in healthcare systems is an important and widely discussed topic^1–3 and often mediated by public policies, market forces or new technologies.⁴ The intention to implement new interventions might be to reduce costs, improve quality, increase efficacy or patient satisfaction.⁵ Nevertheless, implementing change in healthcare organisations can be challenging.^6–10 In the German healthcare system, the implementation of shared decision-making (SDM) has received much attention.¹¹ SDM can be described as an interactional process on the basis of information exchange. Patients and healthcare professionals (HCPs) are equally and actively involved and jointly responsible for the decision.^12–14 This is especially important in situations with complex treatment options and high impact on patients’ quality of life.¹⁵ Patients want to be actively involved in decision-making¹⁶ and benefit from SDM by developing better knowledge about their disease and treatment options, better risk perception and less insecurity and decisional conflict.^{17 18} SDM has been supported by health policy^19–21 and research^22–25 over the last decades. However, SDM is currently not routinely implemented in the German healthcare system.^{11 26–28}

When implementing SDM or other interventions in organisations, several barriers on different levels of the organisation (i.e. individual level, group level, organisational and system level) need to be considered.1 5–7 10 29–32 Barriers for implementing SDM in the clinical setting often address both the organisational setting (e.g. lack of resources and lack of management support) and the individual level (e.g. resistance to change or negative attitudes towards SDM).7 10 11 33–35 When implementing SDM or other interventions in healthcare systems, the clinical employees’ perspective on organisational readiness for change is a critical precursor for successful implementation.5 32 36–38 Armenakis et al. ³⁷ describe organisational readiness for change as the degree to which organisational members are prepared to participate in change processes. This is characterised by the belief that the change is needed and that the organisation is capable of changing. In his theory of organisational readiness for change, Weiner³⁹ differentiate between change commitment (i.e. organisational members’ attitudes towards implementing a change) and change efficacy (i.e. organisational members’ belief in their capability to implement a change). If readiness for change is high, organisational members invest more in the change effort and exhibit greater persistence to overcome barriers and setbacks.^{39 40}

To analyse effects of organisational readiness on implementation success, specific measures for assessing organisational readiness for change are needed.⁵ However, only a few validated measures exist^{5 41–43} and none were available in German. One of those measures is the Organisational Readiness for Implementing Change (ORIC).⁴² The ORIC is brief, easy to administer and theoretically and psychometrically well-grounded.³⁹ It was previously translated into Danish and French.^{44 45} The ORIC has been psychometrically tested, revealing a completion rate of more than 72%, a Cronbach’s α of above 0.80 and two correlating factors.^{42 44 45} Due to the described properties, the ORIC seemed well-suited to measure organisational readiness for implementing SDM in Germany.^{5 42 46}

Therefore, the aim of the study was to translate the ORIC measure into German and assess its psychometric properties.

Methods

Measure

The ORIC measures organisational readiness for implementing change. It uses a 5-point Likert scale ranging from ‘disagree’ to ‘agree’.⁴² In the original English version, two subscales were described based on Weiner³⁹: ‘change commitment’ (items 1 to 5) and ‘change efficacy’ (items 6 to 10). Sum scores were calculated for both subscales separately with higher scores indicating higher organisational readiness for change. By using the phrases ‘to implement this change’ or ‘implementing this change’, the original scale does not specify which change is addressed. The items can be specified to adapt to a specific research question and a survey instruction can be added.^{39 44} The English items are displayed in the results section.

Translation

Translation followed the team translation protocol TRAPD (Translation, Review, Adjudication, Pretesting and Documentation), a method with growing recognition within translation research.^47–51 Thereby an optimal translation is facilitated by discussions between members of the translation team with different expertise in translation. First, two team members (AL, cp. list of abbreviations), SZ, cp. list of abbreviations) proficient in German and English, but little experienced in survey translation, independently translated the original ORIC into German. Second, a third bilingual team member (IS, cp. list of abbreviations) with experience in survey translation, reviewed both versions and suggested a third version based on the first two translations. Finally, IS, AL and SZ discussed all versions until reaching consensus on a final version. To find consensus on item 10 we additionally consulted an official translator (MM, cp. list of abbreviations) and an additional team member (PH, cp. list of abbreviations), who is proficient in German and English and experienced in translation. During the translation process, we changed the phrases ‘to implement this change’ and ‘implementing this change’ into ‘to implement shared decision-making’ and ‘implementing shared decision-making’ to address our specific research question. Additionally, we added a survey instruction in German which motivated participants to think about the clinic, they are working in, when answering the item. As a next step we pretested the translated measure by conducting cognitive interviews and thereby assessed comprehensibility as part of content validity.

Assessment of comprehensibility as part of content validity and subsequent adaptation of the scale

Content validity is the degree to which the content of the measure and its items adequately reflect the measured construct.⁵² According to the COSMIN criteria (consensus-based standards for the selection of health measurement instruments),⁵³ content validity includes the relevance of the items and scales, their comprehensiveness and comprehensibility. As this study aimed to evaluate the translation of an existing measure, we focussed on the assessment of comprehensibility (i.e. items being appropriately worded and understood by participants as intended).

To do so, we conducted two rounds of cognitive interviews with a convenience sample of HCPs (nurses, physicians and psycho-oncologists), working in a comprehensive cancer centre in Germany. Two female researchers and psychologists experienced in interviewing (AL, PH) conducted the interviews. We developed an interview guide based on recommendations by Willis et al.⁵⁴ We used verbal probing techniques like comprehension probes (e.g, ‘What does the term ‘organization’ mean to you?’) and paraphrasing (e.g. ‘Can you repeat this sentence in your own words?’). We conducted interviews until reaching theoretical saturation. Interviews were audio-recorded and transcribed verbatim. After the first round of cognitive interviews, we extracted and discussed comments and suggestions from the transcripts (AL, PH, IS). As a further step to enhance comprehensibility, we discussed the items with the original author (CS, cp. list of abbreviations) as well as French (MR, cp. list of abbreviations) and Norwegian researchers (AH, cp. list of abbreviations), who translated the ORIC into their languages. We adapted items of the German ORIC, which were not well understood by participants of the first round of cognitive interviews, according to these discussions. We tested these items in a second round of cognitive interviews. After the second round, we discussed further adaptations of the items and involved another bilingual researcher in the field (DF, cp. list of abbreviations).

We calculated descriptive statistics of participants’ demographic characteristics using SPSS (IBM SPSS Statistics, V.23).

Psychometric evaluation

Data collection

For psychometric evaluation of the ORIC measure we conducted a secondary analysis of cross-sectional data gathered in a SDM implementation study.⁵⁵ Data from baseline assessment of the SDM implementation study were included. The ORIC was the last questionnaire of a three-page survey measuring HCPs’ attitudes regarding SDM and its implementation. Besides the ORIC, it contained the Control Preference Scale⁵⁶ and the IcanSDM,⁵⁷ demographic questions (e.g. gender, age, profession, work experience) as well as several questions that have been used in previous studies in cancer care.⁵⁸ Results of these additional measures will be published as part of the primary evaluation of the SDM implementation study.⁵⁵

Participants were part of a convenience sample of physicians and nurses. Since this is a secondary analysis, inclusion criteria were identical to inclusion criteria of the SDM implementation study.⁵⁵ We included physicians and nurses who worked at one of three departments within the University Cancer Center Hamburg at the University Medical Center Hamburg-Eppendorf during baseline evaluation of the SDM implementation study.⁵⁵ Eligible HPCs were identified through employee lists provided by department managers. The measure was handed out to eligible HCPs either (1) by a member of our study team (e.g. during a regular physician meeting), (2) by the supervising nurses or (3) via employees’ mailboxes. Participants returned the questionnaire personally to a study team member or by mail.

Data were entered into SPSS (IBM SPSS Statistics, V.23) including blinded double entry of 20% of the data for quality control.

Patient and public involvement

The ORIC measure preliminary addresses HCPs. Physicians, nurses and psycho-oncologists were involved in the adaptation of the measure by taking part in cognitive interviews. Patients were not involved in this study.

Data analyses

Descriptive statistics were calculated for demographic characteristics. Cases were excluded if more than 30% of the ORIC items were missing.⁵⁹ For all other cases, missing data were replaced with item means. We evaluated the completion rate and therefore the acceptance of the measure by calculating frequencies of missing data per item as well as for the overall measure. For this analysis, we also included cases with more than 30% of ORIC items missing because these values are part of completion rate and relevant for interpretation of acceptance.

We a priori hypothesised to replicate the theory-based two-dimensional structure of the original English ORIC version. Two correlating factors ‘change commitment’ (item 1 to 5) and ‘change efficacy’ (item 6 to 10) were postulated. Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and the Bartlett’s test of sphericity were performed to test prerequisites for factor analysis.^{60 61} A confirmatory factor analysis (CFA) with Maximum Likelihood Estimates and two factors was applied for the whole data set as a first step. Because the two-factor model could not be confirmed, we decided to calculate an exploratory factor analysis (EFA) and afterwards an additional CFA to check for model fit. It is recommended to not calculate EFA and CFA with the same data set so the data set was randomised by AL and split into two subsets.^{62 63} The first 115 randomised cases including all data of participants were added to EFA, the second 115 cases were added to CFA. An EFA with oblique rotation was calculated for the first subset. The non-orthogonal rotation was chosen according to Weiner.³⁹ In his theory, organisational readiness for change consists of two interrelated dimensions, therefore the two factors are expected to be correlated. Analogue to analyses done by authors of the English ORIC,^{42 64} we extracted components based on parallel analysis. The criterion of parallel analysis was shown to be superior to other statistic criteria like the Kaiser criterion.^{65 66} It compares the eigenvalues of the data to the eigenvalues based on random data with equivalent sample size and number of variables and chooses only factors with eigenvalues higher than for random data.^{64 65} A CFA was calculated for the second subset. A range of global goodness of fit indices were used to assess the degree to which observed data were accounted for by the proposed models: discrepancy χ² statistic (χ²), degree of freedom (df), normed χ² statistic (χ²/df), Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root mean square error of approximation (RMSEA) as well as Akaike Information Criterion (AIC) and Parsimonious Normed Fit Index (PNFI) for analysing model complexity. Established rules to estimate the model fit were used.^67–70

Item analyses was performed for the one-factor model. It included calculation of item means and standart deviation (SD) as well as observation of floor and ceiling effects,⁷¹ calculation of corrected item-total correlations,^{60 72} inter-item correlations^{60 72} and item difficulties.⁶¹ Internal consistency of the scale was assessed by Cronbach’s alpha coefficient (α).^{61 72 73} For a detailed overview on performed data analyses, see table 1.

Table 1.

Psychometric analyses conducted

Psychometric measure	Criteria
Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett’s test of sphericity	These tests ensure that correlations between variables can be accounted for by a smaller set of factors.60 KMO value should be higher than 0.05 and Bartlett’s test value should be less than 0.05 to fulfil the criteria for calculating a factor analysis.60 61
Normed χ2 statistic (χ2/df)	χ2/df is an indicator for model fit, dependent on sample size and should be as small as possible. A ratio between 2 and 3 indicate a good data fit.67
Comparative Fit Indexes (CFI)	CFIs is an indicator for model fit. It ranges from 0 to 1 and higher values indicate better fit. Values above 0.95 indicate a good model fit.70 74
Tucker-Lewis Index (TLI)	TLI is an indicator for model fit. It corrects for complexity of the model and is sensitive to small sample sizes. Values above 0.95 indicate good fit.68
Root mean square error of approximation (RMSEA)	RMSEA is an absolute index which describes closeness to fit. Values below 0.05 indicate a good fit, values between 0.05 and 0.08 indicate an adequate fit, values between 0.08 and 1 indicate a moderate fit and values above 1 are unacceptable.75
Akaike Information Criterion (AIC)	AIC is a parsimony model fit index. It can be used to compare fit of competing models with smaller values indicating better fit.67 70
Parsimonious Normed Fit Index (PNFI)	PNFI is a parsimony model fit index. It ranges between 0 and 1 and higher values indicate a more parsimonious fit.67 No threshold levels are recommended and it has to be analysed in combination with other goodness of fit indices.70
Analysis of frequencies for item response distributions	Floor and ceiling effects were assumed present if more than 15% of participants choose the lowest or highest possible score.71
Corrected item-total correlations	If items correlate with the total score of above 0.30, they measure the same underlying concept. Items with lower correlations should be removed because they do not add exploratory power to the measure.60 72
Item difficulties	Item difficulties are calculated by dividing item means by the maximal value of the answer range (0–4) and multiplying it with 100. Item difficulty should be near to 50%, and items should not differ much in their difficulty level.61
Inter-item correlations	Inter-item correlations ensure association between items. High inter-item correlations of above 0.80 indicate that items ask the same questions and might be redundant.60 72
Cronbach’s α	Cronbach’s α is a measure for reliability and internal consistency. A value of at least 0.70 is acceptable and higher coefficients indicate a more stable measure.60 61 73

During the translation process and cognitive interviews we found low content validity for item 10 (see results section). Thus, the use of item 10 for the German ORIC needs to be evaluated. Accordingly, we also conducted psychometric analyses (EFA with oblique rotation and extraction of components based on parallel analysis, corrected item-total correlations, Cronbach’s α and goodness of fit indices) for the 9-item version of the ORIC.

Analysis of demographic data, analysis of completion rate, item analysis and EFA were performed using SPSS (IBM SPSS Statistics, V.23). CFA and calculation of model fit indices were performed using Amos (IBM SPSS Amos 22.0.0).

Results

To report the results of this validation study, we used the Authors’ Guidelines for Reporting Scale Development and Validation Results by Cabrera-Nguyen (see online supplementary file 1).

Translation

Both translators (AL and SZ) and the reviewer (IS) did not differ much in their translations of items 2 to 5 and 8 as well as the response scale. For these items and the response scale, only the choice of single words differed without differences in meaning. Greater translation differences were found for items 1, 6, 7, 9 and 10. For item 1, the word ‘committed’ was differently translated. For items 6 and 7, differences were found in the translation of the phrase ‘feel confident’ and the sentence structure. For item 9, differences mainly addressed translation of the phrases ‘feel confident’ and ‘adjust to this change’. For item 10, differences occurred in the translation of the term ‘manage the politics’ and the sentence structure. Within the first round of team discussion, we reached consensus for items 2 to 9, the translation of the response scale and the survey introduction. For item 1 we suggested two versions to be further tested in subsequent cognitive interviews. We struggled to translate the phrase ‘manage the politics’ in item 10 into German. Therefore, we discussed item 10 with additional colleagues (cp. Methods section) until consensus was found.

Assessment of comprehensibility as part of content validity and subsequent adaptation of the scale

To test the German ORIC for comprehensibility, cognitive interviews with n=11 participants (nurses, physicians and psycho-oncologists) were conducted. Cognitive interviews lasted about 1 hour. For demographic data of participants see online supplementary table A in online supplementary file 2.

After the first round of cognitive interviews (n=7), no changes have to be made to the response scale as well as for items 2 to 5 and 8 because these items were already well understood by participants. Participants made some minor suggestions for modifications for the introductory description and items 1 and 6. Additionally, some participants did not understand the correct meaning of items 7, 9 and 10 in general or of single words or phrases of these items. After discussions and modifications of these items, we tested alternative versions of the survey introduction, for items 1, 6, 9 and 10 as well as two alternative versions of item 7. After the second round of cognitive interviews (n=4), items 1, 6 and 9 were now understood well by all participants. We had to slightly modify the survey introduction again and decided to use the version of item 7 which was understood best. After all, item 10 could not be translated successfully. Both rounds of cognitive interviews showed that comprehension of the German translation of the phrase ‘manage the politics’ did not picture the correct English meaning. Thus, in a next step we consulted with DF (cp. list of abbreviations) and reached consensus on a final version. Nevertheless, the final version of item 10 was still not satisfying from the study team and experts view. Item 10 was found to have low comprehensibility as part of content validity according to COSMIN criteria.⁵³ The final German ORIC measure, used in this study, is presented in online supplementary file 3.

During cognitive interviews some nurses reported that they had not heard about the term ‘shared decision-making’ (German: ‘Partizipative Entscheidungsfindung’) prior to participation. Thus, we provided a definition of SDM in the introduction part of the questionnaire within the SDM implementation study.⁵⁵

Psychometric evaluation

Sample characteristics

Data of 235 HCPs were available for this secondary analysis. In line with recommendations of Bannon,⁵⁹ five cases (0.02% of all cases) were excluded (except for assessment of completion rate), because all items of the ORIC were missing. Missing values were replaced by means and data of 230 HCPs could be included into analyses.

Table 2 provides an overview of participants’ demographic characteristics. Most of the 230 HCPs were between 31 and 40 years old (37.0%), female (70.4%), worked as a nurse (57.0%) and had a work experience of <5 years (43.9%).

Table 2.

Demographic characteristics of participants (n=230)

	N	%
Age
<30 years	72	31.3
31–40 years	85	37.0
41–50 years	42	18.3
>50 years	26	11.3
Missings	5	2.2
Gender
Female	162	70.4
Male	59	25.7
Different gender or preferred not to answer this question	5	2.1
Missings	4	1.7
Profession
Nurse	131	57.0
Junior physician	69	30.0
Senior physician	27	11.7
Missings	3	1.3
Work experience in healthcare
<5 years	101	43.9
5–10 years	48	20.9
11–20 years	46	20.0
>20 years	28	12.2
Missings	7	3.0

Factor analysis

Requirements for factor analysis were met.⁵³ Sample size was large enough (>100), even for a split data set with n=115. Furthermore, no outliers were found and data values were approximately normally distributed. KMO measure was 0.933 and Bartlett‘s test of sphericity yielded X²=1485.11, p<0.001. This indicates that a factor analysis of the data was appropriate.^{60 61} CFA for the hypothesised two-factor model showed a high correlation of 0.87 between the two components (see online supplementary file 4). Therefore, we postulated a one-factorial structure and conducted a post hoc EFA. As shown in table 3, only the first component had an eigenvalue higher than 95% percentile of the eigenvalues of corresponding random data and the main component explains 67.23% of the variance. Thus, according to parallel analysis, a one-factor model was assumed. The factor loadings for the first component were above 0.754 for all items (see online supplementary table B of online supplementary file 2).

Table 3.

Results of EFA with oblique rotation and parallel analysis: eigenvalues of the ten components of the German ORIC and eigenvalues for corresponding random data

	Eigenvalues of the ORIC			Eigenvalues for random data
	Total	% of variance	Cumulative %	Means	95% percentile
Component 1	6.72	67.23	67.23	1.49	1.65
Component 2	0.83	8.30	75.53	1.33	1.44
Component 3	0.47	4.75	80.28	1.21	1.30
Component 4	0.41	4.08	84.36	1.11	1.19
Component 5	0.39	3.91	88.28	1.02	1.08
Component 6	0.32	3.24	91.52	0.93	1.00
Component 7	0.27	2.74	94.26	0.85	0.92
Component 8	0.23	2.34	96.60	0.77	0.84
Component 9	0.17	1.74	98.35	0.68	0.75
Component 10	0.16	1.65	100.00	0.58	0.66

For EFA, half of the data set (n=115) was used.

EFA, exploratory factor analysis; ORIC, Organisational Readiness for Implementing Change.

A second CFA was performed with the one-factor model to analyse its fit indices. Indices of the two-factor model and the one-factor model are compared in table 4.

Table 4.

Fit indices of two calculated models for factor analysis of the German ORIC

	χ2†	df‡	χ2/df§	CFI¶	TLI**	RMSEA††	AIC‡‡	PNFI§§
Two-factor model	81.71*	34	2.40	0.968	0.947	0.078	143.71	0.585
One-factor model	77.19*	35	2.20	0.928	0.907	0.103	117.19	0.682

Two-factor model was calculated for the whole data set (n=230): factor 1 includes item 1 to 5, factor 2 includes item 6 to 10; one-factor model was calculated for half of the data set (n=115):includes items 1 to 10.

*p=0.000.

†discrepancy χ² statistic.

‡degree of freedom.

§normed χ² statistic.

¶Comparative Fit Index.

**Tucker-Lewis Index.

††Root mean square error of approximation.

‡‡Akaike Information Criterion.

§§Parsimonious Normed Fit Index

AIC, Akaike Information Criterion; CFI, Comparative Fit Index; df, degree of freedom; ORIC, Organisational Readiness for Implementing Change; PNFI, Parsimonious Normed Fit Index; RMSEA, root mean square error of approximation; TLI, Tucker-Lewis Index.

Results of factor analysis for the 9-item version of the ORIC (without item 10) were similar. Also for the 9-item version, a one-factor model was assumed by EFA. Only the first component had an eigenvalue higher than 95% percentile of the eigenvalues of corresponding random data and the main component explains 66.85% of the variance. Factor loadings of the first component are above 0.739 for all items (see online supplementary file 5, online supplementary tables A and B). Goodness of fit indices of the one-factor model of the 9-item ORIC version showed similar values compared with the 10-item ORIC version (see online supplementary file 5, online supplementary table C).

Analysis of the ORIC items and internal consistency

Table 5 shows response distribution, means, SD, acceptance, corrected item-total correlation and item difficulty of the 10 items. Means ranged between 2.24 (item 9) and 2.61 (item 5). Most participants responded in the middle of the scale with a slight shift to more agreement. For items 1 to 9, between four and six missing values could be detected. For item 10, nine missing values were found. Taking all items into account, more than 97% of the measure were answered. Corrected item-total correlations ranged from 0.665 (item 9) to 0.774 (item 3), item difficulties from 55.98 (item 9) to 65.32 (item 5) and inter-item correlations from 0.434 (item 2 and item 9) to 0.723 (item 3 and item 5) (see online supplementary table C of online supplementary file 2). Internal consistency yielded a Cronbach’s α of 0.93.

Table 5.

Response distribution, means, stansard deviation, acceptance, item discrimination and item difficulty of the German ORIC

	Items	Disagree N (%)	Somewhat disagree N (%)	Neither agree nor disagree N (%)	Somewhat agree N (%)	Agree N (%)	Mean (SD)	Acceptance (completion rate in %)*	Item discrimination (corrected item-total correlation)	Item difficulty
1	People who work here are committed to implementing shared decision-making.	1 (0.4)	22 (9.6)	109 (47.0)	73 (30.9)	25 (10.9)	2.42 (.826)	97.43	0.744	60.58
2	People who work here will do whatever it takes to implement shared decision-making.	4 (1.7)	37 (16.1)	103 (44.8)	68 (29.6)	18 (7.4)	2.25 (.878)	98.29	0.689	56.36
3	People who work here want to implement shared decision-making.	0 (0.0)	15 (6.5)	107 (46.5)	84 (35.7)	24 (10.4)	2.50 (.768)	97.43	0.774	62.61
4	People who work here are determined to implement shared decision-making.	2 (9.0)	38 (16.5)	107 (46.5)	67 (29.1)	16 (7.0)	2.25 (.843)	98.29	0.758	56.19
5	People who work here are motivated to implement shared decision-making.	1 (0.4)	16 (7.0)	85 (37.0)	97 (42.2)	31 (13.5)	2.61 (.821)	98.29	0.764	65.32
6	People who work here feel confident that they can handle the challenges that might arise in implementing shared decision-making.	2 (0.9)	20 (8.7)	93 (40.4)	93 (40.4)	22 (9.6)	2.49 (.819)	98.29	0.760	62.28
7	People who work here feel confident that they can keep track of progress in implementing shared decision-making.	1 (0.4)	26 (11.3)	93 (40.4)	92 (40.0)	18 (7.8)	2.43 (.811)	98.29	0.725	60.87
8	People who work here feel confident that they can coordinate tasks so that implementation goes smoothly.	5 (2.2)	24 (10.4)	107 (46.5)	78 (33.5)	16 (6.5)	2.32 (.833)	99.56	0.697	58.13
9	People who work here feel confident that the organisation can support people as they adjust to shared decision-making.	6 (2.6)	43 (18.7)	89 (38.7)	74 (32.2)	18 (7.8)	2.24 (.934)	97.86	0.665	55.98
10	People who work here feel confident that they can manage the politics of implementing shared decision-making.	3 (1.3)	24 (10.04)	122 (50.9)	65 (28.3)	16 (7.0)	2.29 (.796)	96.15	0.714	57.44

Items could be answered on a 5-step Likert scale rating from 0 ‘disagree’ to 4 ‘agree’.

*For calculation of completion rate, five additional cases were included because these participants only skipped items of the ORIC but filled out the rest of the survey.

ORIC, Organisational Readiness for Implementing Change.

Additionally, corrected item-total correlations and internal consistency were calculated for the 9-item ORIC version (see online supplementary file 5, online supplementary table D). They were similar to the results for the 10-item version with corrected item-total correlations between 0.638 (item 9) and 0.777 (item 3) and a Cronbach’s α of 0.92.

Discussion

The original English ORIC measure is a brief measure with good psychometric properties,⁴² which were confirmed in Danish⁴⁴ and French⁴⁵ validation studies. The study at hand aimed to translate the ORIC into German and assess its psychometric properties.

Translation and assessment of comprehensibility as part of content validity

Items 1 to 9 were translated and adapted successfully after two rounds of cognitive interviews and several rounds of discussions within the study team and with external experts. The translation team quickly reached consensus for items 2 to 5 and 8. These items were also well understood by all participants within the first round of cognitive interviews. For items 1, 6, 7 and 9, the translation process was more complex and several adaptations and discussions were necessary. Feedback by participants, members of the study team, and external experts as well as completion rates suggest that comprehensibility of item 10 remains low.⁵³ This might be due to the translation of the phrase ‘manage the politics’ into ‘Machenschaften’. The term ‘manage the politics’ seems to have a strong cultural connotation and no equivalent phrase in German language exists. The German term ‘Machenschaften’ might have a different connotation as the English phrase and might lead to skipping the item. Ruest et al,⁴⁵ who translated the English ORIC into French, also identified several differences in cultural concepts during their adaptation process, but could translate all items successfully. They concluded that limitations in linguistic validation could decrease comparability of psychometric results of the translated measure. However, item 10 showed similar and inconspicuous item characteristics compared with other items in our sample. When repeating factor and item analyses for the 9-item ORIC version including only item 1 to 9, very similar results were observed compared with the 10-item version. To increase comprehensibility and thereby content validity of the scale, the use of the 9-item German ORIC might be a solution and should be evaluated in future studies.

Factor analysis

We a priori hypothesised a two-factorial structure of the German ORIC, because Shea et al ⁴² described correlations between the two theory-based factors ‘change commitment’ and ‘change efficacy’ of 0.56 to 0.60. However, we found much higher factor correlations of 0.87. Results of the subsequent EFA clearly indicated a one-factorial structure. Thus, we could not confirm the two-factor structure of the English and the translated Danish and French versions of ORIC.^{42 44 45} When comparing the two models, both models have acceptable values for χ²/df⁶⁷ and CFI,^{70 74} but only the two-factor model has acceptable values for TLI⁶⁸ and RMSEA.⁷⁵ When involving parsimony of the models by calculating AIC and PNFI, the one-factor model fits better to the data.^{67 70} Therefore, we prefer the more parsimonious one-factor model. These differences to previous validation studies might be a consequence of diverse cultural connotations of the ORIC items in different languages, caused by the adaptation to the context of SDM, or due to specific characteristics of the participating clinics.

Analysis of ORIC items and internal consistency

Since the ORIC was presented as the last measure in a three-page survey, missing values might indicate respondent fatigue. However, missing value rates for single items and the overall measure were quite low and the German ORIC was found to be a well-accepted measure. There were no floor or ceiling effects. Corrected item-total correlations of above 0.66 indicate that all items measure the same underlying concept.^{60 72} Criteria for good item difficulties are met since item difficulties are near to 50% and do not differ much from each other.⁶¹ Inter-item correlations are below 0.80, indicating that items add additional information and are not redundant.^{60 72} Cronbach’s α (α=0.93) suggest excellent internal consistency.^{60 61 73} In summary, item analysis and internal consistency of the German ORIC suggest good quality of the measure.

Nevertheless, according to Streiner and Norman⁶¹ a Cronbachs’ α above 0.90 might also indicate item redundancy. On the other hand, inter-item correlations and corrected item-total correlations are in an acceptable range.^{60 72} In implementation research there is a need for preferably brief measures, which can be applied in diverse settings with high work-load. Thus, future research could investigate the possibility to further reduce the number of items.

Strengths and limitations

This study has some limitations. First, several psychometric parameters are not analysable because this study was a secondary analysis of cross-sectional data. It was not possible to calculate, for example, convergent or divergent validity yet. Second, we applied the ORIC only in three departments of one University Medical Center in Germany. Further validation in different organisational settings is needed to ensure generalisability. Third, for this psychometric evaluation we used a German ORIC, which we adapted and specified for the context of SDM implementation. Our results might not be generalisable for other interventions in other organisations. Fourth, although SDM was not implemented to the participating clinics before, there might be participants who were more familiar with the concept of SDM than others. Fifth, item 10 was again slightly changed after finishing cognitive interviews. This item was not finally tested for comprehensibility.

A major strength of this study is that we provided the first measure to assess organisational readiness for change in German language for use in implementation studies. We conducted an elaborated translation procedure, which was recommended for survey translations. We furthermore used a qualitative approach to explore comprehensibility including discussions with international colleagues and experts outside of the study team. Furthermore, we assessed the ORIC in a sample including physicians and nurses which was large enough to robustly perform the psychometric analysis on the German version of the ORIC measure.

Conclusion

Organisational readiness is a crucial indicator to successfully implement change and a possible barrier if missing. For implementation studies, it is essential to measure organisational readiness with valid and reliable measures. We provide the first German measure for organisational readiness for implementing change and validated it for the context of SDM implementation. The German ORIC is a brief measure with a high completion rate. We found satisfying psychometric properties in a German hospital setting. To increase content validity of the measure, the use of a 9-item German ORIC (without item 10) should be evaluated in future studies. As the ORIC targets the attitude of organisational members, it can detect reduced or missing readiness for implementing a change on the individuals’ level. Therefore, the German ORIC can be used to analyse organisational readiness as a possible barrier for implementing various interventions in organisations.