Skip to main content
NCBI home page
Nursing Open logoLink to Nursing Open
. 2025 Feb 6;12(2):e70151. doi: 10.1002/nop2.70151

Development and Validation of the EBP‐Affinity Questionnaire: A Mixed‐Method Study

Lenny J Tange 1,2,✉, Maud Heinen 3, Raymond TCM Koopmans 1,4, Dirk F de Korne 2,5, Anke Persoon 1, Robert Lindeboom 6, Debby L Gerritsen 1, Marleen H Lovink 1
PMCID: PMC12365660  PMID: 39913361

ABSTRACT

Background

Implementation of evidence‐based practice (EBP) in nursing homes is important to improve quality of care and reduce costs. These facilities primarily employ certified nurse assistants (CNAs) and vocationally‐trained registered nurses (VTRNs). Although EBP is scarcely addressed in their education, these caregivers have an important role in EBP. Insight into their affinity with EBP could provide a good starting point for working according to EBP.

Aims

  1. To develop a questionnaire that measures affinity with EBP amongst CNAs and VTRNs in nursing homes.

  2. To assess the reliability and validity of the questionnaire.

Design

Mixed‐method design according to the COSMIN‐checklist.

Methods

Phase 1: the questionnaire was developed in three steps: (1) gathering items from existing questionnaires, occupational profiles and interviews; (2) compression through consensus rounds and (3) a pilot study.

Phase 2: psychometric properties were assessed amongst CNAs and VTRNs in nine nursing homes. The factors extracted through principal component analysis were tested for measurement invariance using the one‐parameter Rasch model.

Results

Phase 1: all experts interviewed (N = 5) indicated CNAs and VTRNs can play specific and significant roles in EBP. A 26‐item questionnaire was created through two consensus rounds with experts (N = 7) and a pilot study (N = 7).

Phase 2: Principal component analysis revealed a two‐factor structure with good internal reliability (N = 428 questionnaires). All items exhibited good fit with the Rasch model and measurement invariance for CNAs and VTRNs.

Conclusion

CNAs and VTRNs can fulfil important roles in EBP in nursing homes. The ‘Evidence‐Based Practice‐Affinity Questionnaire’ is reliable and valid for assessing affinity with EBP amongst CNAs and VTRNs to improve practice and research, and can guide tailored training programs for CNAs and VTRNs, enhancing evidence‐based practices in nursing homes.

Impact

The research provides insight into the roles CNAs and VTRNs can play in EBP and how to measure their affinity with EBP, which contributes to implementation of EBP in nursing homes.

Patient or Public Contribution

No patient or public contribution in the study.

1. Introduction

In European countries, frail older people for whom home‐based care is no longer sufficient are often admitted to nursing homes (NH). The combined trends of population ageing, governmental policies promoting home care (OECD 2013) and an increase in comorbidity resulting in high complexity of care are placing NHs under increasing pressure (Alders and Schut 2019; Conroy et al. 2009). As a result, healthcare costs are rising, and effectiveness is being increasingly emphasised within the context of healthcare (Robertson‐Preidler, Biller‐Andorno, and Johnson 2017).

The implementation of evidence‐based practice (EBP) is crucial to improving patient outcomes and patient safety, as well as to reducing costs and adverse events in healthcare (Rudman et al. 2020; Teixeira, Nogueira, and Barbieri‐Figueiredo 2023). In addition, EBP empowers care providers and leads to higher levels of engagement, teamwork and job satisfaction (Kim et al. 2017; Melnyk et al. 2010). At present, EBP unquestionably adds value to the increasing complexity of long‐term care for NH residents (Diehl et al. 2016; Specht 2013). Although this fact is well known, EBP is still insufficiently embedded within daily NH practice (Specht 2013).

2. Background

In the Netherlands, more than 46% of all NH staff consist of certified nurse assistants (CNAs) (37%) and vocationally‐trained registered nurses (VTRNs, European Qualifications Framework (EQF) level 4) (9%) (Backhaus et al. 2018; European Commission 2018; Rommets and Roelvink 2023; Tuinman et al. 2016). The primary educational programme for CNAs comprises 3 years of vocational training, and it is comparable to Level 3 of the EQF (Backhaus et al. 2018; European Commission 2018). The education of VTRNs—a function corresponding to Level 4 of the EQF—consists of 4 years of vocational education training (Backhaus et al. 2018). The education qualification files of CNAs and VTRNs do mention EBP in the description of ‘reflective EBP professionals’. For CNAs and VTRNs, reflective EBP professionals are individuals who are able to develop their own expertise and those of their co‐workers, to apply techniques and evidence‐based interventions, and to supervise students and trainees (SBB 2020). Given their pivotal role in direct patient care, CNAs and VTRNs are uniquely positioned to implement EBP strategies effectively, yet are often overlooked in training programs tailored for higher‐educated professionals.

Aside from the reference to reflective EBP professionals, as mentioned above, programmes of nursing training and education devote little attention to EBP (Brink and Joldersma 2016). In addition, the specific roles of nursing professionals in EBP have yet to be established. Three roles can be identified in actual EBP: (1) the ‘doer’, who formulates clinical questions and assesses the evidence; (2) the ‘user’, who only searches evidence that has already been appraised; and (3) the ‘replicator’, who formulates clinical questions and follows the recommendations of experts and guidelines (Straus et al. 2018). In practice, CNAs and VTRNs are likely to provide the most effective care when acting as ‘replicators’ and ‘users’ of EBP, as opposed to ‘doers’, as this role demands at least a Bachelor's level education (Straus et al. 2018). It is not realistic to expect CNAs and VTRNs to search scientific articles for the best research evidence and appraise it critically (Warren et al. 2016). They are nevertheless able to use evidence‐based information (e.g., guidelines and protocols), which they can implement in practice (Warren et al. 2016).

One good starting point for the implementation of EBP in NHs could be to investigate the affinity for working according to EBP (i.e., liking, understanding and applying EBP). Although validated EBP instruments (Haavisto et al. 2023; Hoegen et al. 2021) are available (e.g., the EBPAS and EBPQ), they have several limitations. Firstly, they are directed towards nurses with Bachelor's degrees or professionals with higher levels of education. Therefore, the EBPAS (Aarons et al. 2010) and the EBPQ (Upton, Upton, and Scurlock‐Evans 2014) have been translated into Dutch, with language and terminology adapted through cross‐cultural validation for the assessment of CNAs and VTRNs working in NHs (Maessen et al. 2019). However, secondly, the EBPAS and EBPQ are limited in capturing ‘affinity’. The EBPAS‐ve measures attitudes towards the use of new working methods in professional practice. The EBPQ‐ve focuses on the application of EBP by measuring the use of insights from various sources of knowledge in daily care. The use of both questionnaires can provide insight into attitudes towards and the use of EBP in NHs, although both questionnaires do not yet provide a complete understanding of the affinity with EBP amongst CNAs and VTRNs. In practice, however, the use of two EBP questionnaires with a diverging focus and different response scales is not desirable (Lovink et al. 2022). To date, there is no questionnaire that measures ‘affinity with EBP’ amongst CNAs and VTRNs, based on their specific roles in EBP.

3. The Study

The aim of this study was (1) to develop a new questionnaire—the ‘Evidence‐Based Practice‐Affinity (EBP‐Affinity) Questionnaire’—which measures affinity with working according to EBP amongst CNAs and VTRNs in NHs, based on their specific roles in EBP; and (2) to assess the reliability and validity of the EBP‐Affinity Questionnaire.

4. Methods

4.1. Design

A mixed‐method design covering two phases was used. In Phase 1, a new questionnaire—the ‘EBP‐Affinity Questionnaire’—was developed. In Phase 2, the reliability and validity of this questionnaire were assessed for CNAs and VTRNs working in NHs. Both phases were conducted according to the COnsensus‐based Standards for the selection of health Measurement INstruments (COSMIN) checklist (Mokkink, Terwee, Knol, et al. 2010).

The underlying concepts of this study are the definition of and steps involved in EBP, which consist of the conscientious, explicit and judicious use of recent evidence in making decisions about the care of individual patients in light of their personal values and beliefs. It integrates the best research evidence with clinical expertise and includes resident values when making decisions in practice (Sackett et al. 1996). To integrate evidence from research into clinical decision‐making, EBP follows a cyclical form of five consecutive steps: (1) asking a question in response to a clinical query, (2) acquiring the best available evidence from various sources, (3) appraising the strength of the evidence, (4) applying the evidence alongside the best values and interest of patients and (5) assessing the outcome (Dawes et al. 2005). To these steps, Johnston and Fineout‐Overholt and Johnston (2005) add ‘reflection’ as an important prerequisite for initiating EBP, referring to it as Step 0 in the EBP process.

4.2. Phase 1: Development

Development took place between September 2020 and January 2021. The questionnaire was generated in three steps, in order to ensure both face and content validity (Mokkink, Terwee, Knol, et al. 2010): (1) collecting items from literature and interviews with experts, (2) consensus rounds with an expert group and (3) pilot testing of the questionnaire with users in practice (Hak, Veer van der, and Jansen 2008) (Figure 1).

FIGURE 1.

FIGURE 1

Open in a new tab

Schematic representation of questionnaire development.

4.2.1. Step 1: Collecting Items From Literature and Interviews With Experts

We used the reference list of the article by Maessen et al. (2019) as a starting point for a survey of relevant articles and existing EBP‐measurement questionnaires. In addition, an open search was performed in PubMed for articles on EBP questionnaires measuring affinity with EBP amongst CNAs and VTRNs. The search strategy consisted of the following terms: ‘Evidence Based Practice’, ‘nursing home(s)’, ‘nurs*’, ‘questionnaire design’ and ‘questionnaire development’, along with their synonyms, keywords and MeSH terms. The Dutch professional profiles and educational qualification files of CNAs and VTRNs were also reviewed for activities and responsibilities within the role of EBP. Items were selected from existing questionnaires and compiled from professional profiles and educational qualification files. Items that were similar in content were merged or one was removed by the first author (L.T.). For example: item (a) ‘I look at where the care I provide for the client could be improved’ was almost identical to item (b) ‘I look for ways in which I can improve the quality of care for the client’. Item ‘b’ was deleted, based on the item which align with the target group. Negatively formulated items were converted to positively formulated items. To avoid the use of jargon that is not applied by CNAs, the term ‘EBP’ was not used in the items (Wolf et al. 2014).

Experts involved with EBP projects (N = 5) from the central and southern regions of the Netherlands were selected, including two lecturers from the Bachelor of Nursing programme at a university of applied sciences, one who also had experience in intermediate vocational education of nursing and one who had fulfilled the role of an EBP coach in the NH (as applied in Lovink et al. 2022), a lecturer‐researcher from a university medical centre, a bachelor educated registered nurse who also had fulfilled the role of an EBP coach in the NH (as applied in Lovink et al. (2022)) and M.L., based on her experience as a researcher in the study by Lovink et al. (2022). The participants were interviewed by either telephone or video call. The central question was as follows: ‘What can be expected from CNAs and VTRNs with regard to working according to EBP in daily practice’. All steps of the EBP cycle were discussed (Dawes et al. 2005). The most suitable response scale for the questionnaire was discussed as well. Notes were taken during the calls and later summarised. The summaries of the interviews were analysed to generate potential items for the questionnaire. The items were classified according to the steps of EBP, and new topics were generated for items not covered by the EBP steps.

4.2.2. Step 2: Consensus Rounds

A new group of experts was selected from the central and southern regions of the Netherlands (N = 7) with at least one of the following criteria: sufficient expertise in EBP, professionally active in a NH or older adult care, a didactic or testing expert with an eye for the language and terminology used by both CNAs and VTRNs. The group consisted of two scientific practitioners (Barry et al. 2021), one district nurse with a degree in nursing sciences, two senior researchers (A.P. and M.L.), one lecturer from the Bachelor of Nursing programme at a university of applied sciences with a special interest in EBP, and a lecturer from an intermediate vocational education of nursing. The last member participated only in the first round. Two experts participated in both the interviews and the consensus rounds. Two rounds were necessary to reach consensus.

The consensus rounds were conducted through an online survey distributed by email. The experts rated each item according to its adequacy in reflecting the construct of ‘affinity with EBP’ on a scale from 1 (No, this item clearly does not measure affinity with EBP) to 4 (Yes, this item clearly measures affinity with EBP) within the target population. Each item was also assessed in terms of comprehensibility and unambiguity, and items were re‐articulated as needed. Missing items could be added for each topic. Questions were also included concerning whether items fit adequately within the assigned EBP topic. The inclusion of an item was based on an average score of at least 3.6, as determined by the researchers. Items with an average score lower than 3.0 were deleted from the questionnaire. Items scoring between 3.0 and 3.5 were discussed by the researchers (A.P., M.L. and L.T.), taking into account suggestions from the experts. The modified questionnaire was re‐submitted to the experts for a second digital consensus round. For each item, the professionals were asked ‘whether the item is appropriate for the construct “affinity with EBP”’, based on the following response options: ‘yes, without further adjustments’, ‘yes, but with the following adjustments: …’ and ‘no, because: …’.

4.2.3. Step 3: Pilot Testing

The pilot group consisted of CNAs (N = 5) and VTRNs (N = 2), ensuring representation of diverse practice backgrounds. Besides their function of CNA or VTRN, they fulfil the role of first responsible carer (N = 4), post‐bachelor wound nurse specialist (N = 1) or clinical educator (N = 1). The participants were approached by email and invited for an individual meeting. Pilot testing was conducted by presenting the draft questionnaire to the participants. They were asked to complete the draft questionnaire using the ‘thinking aloud’ method in a face‐to‐face setting with the researcher (L.T.) (Hak, Veer van der, and Jansen 2008). The questionnaire was improved by M.L. and L.T. based on their suggestions.

4.3. Phase 2: Psychometric Evaluation

In Phase 2 (February—April 2021), we assessed the measurement properties of the EBP‐Affinity Questionnaire in terms of (a) procedure, (b) dimensionality, (c) Rasch model fit and measurement invariance, (d) test–retest reliability and (e) hypothesis testing for construct validity. All analyses were performed with R Statistical Software, version 3.5.2 using the packages ‘eRm’ (Mair and Hatzinger 2007) and ‘psych’ (Revelle 2024).

4.4. Procedure

The data required for the psychometric assessment of the EBP‐Affinity Questionnaire were obtained by conducting an online survey (LimeSurvey program, version 2.06+) amongst CNAs and VTRNs employed within NHs affiliated with the University Knowledge Network for Older Adult Care Nijmegen (UKON).

Nine of the 18 NHs approached consented to participate. An email containing a link to the online survey was sent to one point of contact in each of the nine NHs, with a request for further distribution amongst CNAs and VTRNs within their organisations. They were instructed to send a reminder two to 4 weeks after the initial invitation. The survey ended by asking respondents whether they would be willing to complete the EBP‐Affinity Questionnaire a second time. To test reliability, the survey was sent a second time by personal email to consenting participants about 2 weeks after the first round.

4.5. Dimensionality (Structural Validity)

Principal Component Analysis (PCA) with oblique rotation was used to assess the dimensionality of the questionnaire (Field et al. 2012, 765). The number of factors to be extracted was obtained through parallel analysis (Timmerman and Lorenzo‐Seva 2011) and an associated scree plot (Field et al. 2012, 762–763). The cut‐off point for crossloading items was set to a factor loading of > 0.35 on extracted factors (Field et al. 2012, 767). The absence of excessively correlated items was confirmed according to singularity index threshold values exceeding 0.00001 (Field et al. 2012, 770–771).

The internal consistency reliability of the principal components was calculated using Cronbach's alpha, with scores of > 0.7 considered acceptable (Field et al. 2012, 799).

4.6. Unidimensionality and Measurement Invariance

The one‐parameter Rasch model was used to test individual items for unidimensionality and measurement invariance. To estimate the Rasch model, the item scores were dichotomised. A score of 0 was assigned to the response options ‘I never do this’, and ‘I sometimes do this’. A score of 1 was assigned to the response options ‘I do this regularly’, ‘I do this often’ and ‘I always do this’. The Rasch model was tested for the expected number of 1‐scores on each item, based on the summed‐score level of EBP affinity. The proportion of 1‐scores was expected to increase with EBP‐affinity level. With Rasch modelling, measures of the ‘difficulty’ of items and the ‘ability’ of persons are estimated from the response patterns of the participants to the items. Both measures are expressed on a common logit unit scale, ranging from −4 to +4 in most practical situations. The fit of the individual items was evaluated with infit and outfit mean‐square effect sizes (with an expected range between 0.5 and 1.5) (Linacre 2002), as well as by inspection of item‐fit plots showing the observed and expected proportions of 1‐scores as a function of EBP‐affinity level (see Figure 2 for an example).

FIGURE 2.

FIGURE 2

Open in a new tab

Rasch model fit plot, for example: Item 2 ‘request feedback from colleagues to improve care’. The solid line represent Rasch model's expected proportions of a positive score, dots indicate observed proportions of positive scores within increasing EBP‐affinity level groups.

For a good measurement instrument, item‐difficulty measures should be invariant from the sample used to estimate them (and person‐ability measures should be invariant from the set of items used to produce the EBP ‘ability’ measures). The measurement invariance of the items in the EBP‐Affinity Questionnaire was examined according to the extent of agreement between estimated item‐difficulty measures for VTRN (EQF 4) and for CNA (EQF 3) respondents. Measurement invariance was examined visually by plotting these measures along a diagonal identity line with 95% confidence bands.

4.7. Test–Retest Reliability

The intraclass correlation (ICC) for a two‐way‐mixed‐effects model was calculated to evaluate the test–retest reliability of the questionnaire. Coefficients of < 0.5 reflect poor agreement, with those between 0.5 and 0.75 indicating moderate agreement, those between 0.75 and 0.9 indicating good agreement and those of > 0.9 reflecting excellent agreement (Koo and Li 2016).

4.8. Construct Validity

The construct validity of the EBP‐Affinity Questionnaire was evaluated by testing hypotheses based on the expected mean‐sum score differences between subgroups of respondents (Mokkink, Terwee, Patrick, et al. 2010). The validity variables considered were ‘educational level’ (Aarons et al. 2010; Maessen et al. 2019) and ‘first responsible carer (FRC)’ or ‘not FRC’ (Haterd and Zwikker 2009). Higher mean‐sum scores were expected for the groups ‘highest level of education (VTRNs)’ and ‘FRC’ subgroups. The mean score differences (tested according to ANOVA) are expressed as eta‐squared effect sizes (ղ 2), with values of ղ 2 = 0.01 indicating a small difference, values of ղ 2 = 0.06 indicating a medium difference and values of ղ 2 = 0.14 indicating a large difference (Cohen 1988).

5. Results

5.1. Phase 1: Development

5.1.1. Step 1: Collecting Items From Literature and Interviews With Experts

All EBP experts interviewed indicated that CNAs and VTRNs are quite capable of applying the steps of the EBP cycle, with the exception of Step 3 (‘Critically assessing the evidence found for validity, relevance and applicability’). For Step 2 (‘Systematic search for the best evidence’), CNAs and VTRNs are able to base their evidence on protocols, guidelines and professional journals. By noting that CNAs and VTRNs are able to reflect on the provision of healthcare provision, the experts confirmed the appropriateness of including Step 0 (Fineout‐Overholt and Johnston 2005).

In all, 77 items were selected. More specifically, all items from the EBPAS‐36 (Rye et al. 2017), the EBPAS‐50 (Aarons et al. 2012), the EBPQ (Upton, Upton, and Scurlock‐Evans 2014), the EBPAS‐ve and the EBPQ‐ve (Maessen et al. 2019) were included, as were the items derived from the interviews. Three items from the professional profiles and educational qualification files were added as well.

The items were divided into 10 topics representing Steps 0, 1, 2, 4 and 5 of the EBP cycle, with a two‐part division for Steps four and five, along with three additional topics: ‘expertise development’, ‘motivation/attitude’ and ‘participation in research’ (Cook, Jaeschke, and Guyatt 1992).

5.1.2. Steps 2 and 3: Consensus Rounds and Pilot Testing

In the first consensus round, 46 of the 77 items were removed based on relevance scores of < 3.0 and after discussion between M.L. and L.T. concerning items with relevance scores between 3.0 and 3.5. In the second round, another 3 items were removed based on formulation and overlap, thus leaving 28 items, divided into five topics. Another 2 items were removed after pilot testing. The final questionnaire thus consisted of 26 items covering five topics: ‘Taking a critical view of care’ (Items 1–3), ‘Gathering information’ (Items 4–12), ‘Applying new knowledge’ (Items 13 and 14), ‘Evaluating care’ (Items 15–18) and ‘Promoting expertise’ (Items 19–26) (see Figure 1). A five‐point response scale ranging from ‘never’ (1) to ‘always’ (5) was chosen for all items. This scale was adopted based on suggestions from the experts interviewed, as well as in the literature that it would be less confusing and therefore likely to increase the response rate (Babakus and Mangold 1992). Scores on the questionnaire can range between 26 and 130 points, with higher scores reflecting greater affinity for working according to EBP. For information on the content of the items in the EBP‐Affinity Questionnaire, see Table 1.

TABLE 1.

Item score distribution, eigenvalues, factor loadings and infit and outfit values for the EBP‐Affinity Questionnaire (n = 428).

No item Item content EBP‐topic Eigen‐value* Factor loadings** Mean item score SD Distribution answer categories % Infit*** Outfit***
1 2 1 2 3 4 5
Factor 1: Collecting practical information (α = 0.88)
1. Question suitability of care provided 1 5.63 (18.5%) 0.39 3.1 1.07 5 28 31 25 11 1.20 1.49
2. Request feedback from colleagues to improve care 1 0.43 3.2 0.84 0 22 43 30 5 1.08 1.21
3. Daily reflection on care provided 1 0.41 3.7 1.05 1 15 26 31 27 1.17 1.23
4. Formulate question in case of problems with care 1 0.57 3.6 1.00 1 14 32 32 21 1.04 1.07
5. Apply own knowledge in case of problems with care 2 0.62 4.1 0.79 0 2 22 42 34 1.02 1.04
6. Inquire about client preferences in case of problems with care 2 0.71 4.0 0.88 0 5 23 38 34 0.94 0.87
7. Request advice from colleagues in case of problems with care 2 0.76 4.0 0.84 0 3 25 39 33 0.98 0.94
8. Request advice from others outside the team in case of problems with care 2 0.70 3.5 0.99 0 17 34 30 19 0.95 0.94
13. Make a thorough assessment before changing working method 3 0.59 3.1 1.13 6 24 33 22 15 0.94 0.86
14. Change method based on information collected in case of problems with care 3 0.66 3.4 0.97 1 17 37 30 15 0.94 0.96
15. Make own evaluation of change in method 4 0.77 3.8 0.95 0 8 32 33 27 0.84 0.77
16. Evaluate change in method with the client 4 0.55 3.3 1.15 4 23 31 22 20 1.03 1.07
17. Evaluate change in method with colleagues 4 0.72 4.0 0.91 0 5 26 34 35 0.74 0.64
18. Evaluate change in method with practitioners 4 0.61 3.4 1.17 5 20 28 25 22 0.87 0.77
Factor 2: Collecting theoretical information (α = 0.88)
9. Search for protocols in case of problems with care 2 4.76 (21.3%) 0.49 2.9 1.02 3 39 29 21 8 0.82 0.77
10. Search for solutions online in case of problems with care 2 0.61 2.4 1.01 18 48 20 11 3 1.02 1.04
11. Search for solutions in journals or textbooks in case of problems with care 2 0.76 1.8 0.90 49 33 13 4 1 0.93 0.87
12. Request scientific literature within organisation in case of problems with care 2 0.63 1.7 0.91 56 30 8 5 1 1.06 1.10
19. Discuss ideas for improving care from symposia, training and journals with colleagues and practitioners 5 0.59 2.7 1.05 10 42 26 16 6 0.94 0.90
20. Request new experiences or ideas for improving care from colleagues or practitioners 5 0.48 2.8 0.98 5 36 34 19 6 1.06 1.06
21. Read literature or follow relevant professional groups on social media 5 0.74 2.2 1.15 31 35 18 11 5 1.04 1.06
22. Discuss for training with supervisor 5 0.58 2.2 1.04 23 47 17 8 5 1.01 0.96
23. Stay abreast of new protocols to improve care 5 0.54 2.9 1.04 6 33 35 18 8 1.10 1.03
24. Browse internet for interesting care‐related topics 5 0.78 2.6 1.08 14 37 24 21 4 0.95 0.98
25. Inquire about the application of new methods within organisation 5 0.60 2.7 1.01 7 42 28 18 5 0.90 0.86
26. Contribute to research to improve care 5 0.57 2.4 1.17 25 37 19 12 7 1.04 1.09
Open in a new tab
*

Explained variance between brackets.

**

Only loadings > 0.35 are shown, there were no cross‐loading items.

***

Infit and outfit mean‐square effect sizes. Values between 0.5 and 1.5 indicate acceptable fit to the unidimensional Rasch model.

5.2. Phase 2: Psychometric Evaluation

5.2.1. Characteristics and Score Distribution of the Respondents

Of the approximately 2000 healthcare workers in NHs who were invited to participate in the survey, 514 (26%) completed the questionnaire. In all, 86 respondents were excluded from the sample, 60 because they were employed in positions at levels other than EQF 3 or EQF 4, and 26 because their education at a university of applied sciences had not been related to healthcare. In all, 76 respondents agreed to complete the questionnaire a second time. After deleting incomplete questionnaires (n = 4) and questionnaires from respondents working in positions at levels other than EQF 3 or EQF 4 (n = 14), the responses from 58 questionnaires were included in the analysis.

Of the 428 total respondents, 93.7% were female, with the greatest proportion (32.9%) being between the ages of 50 and 59 years, and the majority (76.9%) being employed as CNAs, as compared to 23.1% who were working as VTRNs. The majority (n = 257; 60%) of the respondents were FRCs. Most respondents were working in dementia special care units (n = 233) or units for residents with somatic diseases (n = 185), and some were active in both types of units (Table 2).

TABLE 2.

Characteristics of the sample.

N (%)
Total sample 428 (100)
Female 401 (93.7)
Age (in years)
< 20 3 (0.7)
20–29 58 (13.6)
30–39 78 (18.2)
40–49 67 (15.7)
50–59 141 (32.9)
> 59 45 (10.5)
Unknown 36 (8.4)
Current position
Trainee CNA 16 (3.7)
CNA 313 (73.1)
Trainee VTRN 10 (2.3)
VTRN 89 (20.8)
FRC
Yes 257 (60.1)
No 171 (40.0)
Years active in care
< 5 37 (8.7)
6 t/m 16 147 (34.4)
> 16 243 (56.9)
Average hours per week
< 25 180 (42.1)
> 24 248 (57.9)
Shift time popularity
Day shift 350 (81.7)
Evening shift 37 (8.6)
Night shift 40 (9.4)
Department type (> 1 answer possible)
Psychogeriatrics 233 (54.4)
Somatics 185 (43.2)
Geriatric Rehabilitation Care 55 (12,9)
Misunderstood behaviour 44 (10.3)
Korsakov 16 (3.7)
Huntington 10 (2.3)
Multiple sclerosis 15 (3.5)
Young people with dementia 13 (3.0)
Gerontopsychiatry 13 (3.0)
Palliative 17 (4.0)
Acquired brain injury 31 (7.2)
Other 6 (1.4)
Open in a new tab

Abbreviation: FRC, first responsible caretaker.

5.3. Dimensionality (Structural Validity)

The minimum total score of the respondents was 43 points, with a maximum score of 124 points and a mean of 79.5 points. The sum‐score distribution was approximately normal, with no apparent floor or ceiling effects. There was a good spread for the responses to each item, with the exception of Item 5 (‘Apply own knowledge in case of problems with care’) and Item 7 (‘Request advice from colleagues in case of problems with care’), which had low numbers in Response Categories 1 and 2, and Item 11 (‘Search for solutions in Use journals or textbooks in case of problems with care’) and Item 19 (‘Discuss ideas for improving care from symposia, training and journals with colleagues and practitioners’), which had low numbers in Response Categories 4 and 5. As shown in the scree plot from the parallel analysis, two factors scored above the level of chance, with 14 and 12 items explaining 42% of the total variance: ‘Collecting practical information’ (eigenvalue = 5.6, 21.5%) and ‘Collecting theoretical information’ (eigenvalue = 4.8, 18.3%). All items had factor loadings > 0.37, and there were no crossloading items. Factor 1 covered EBP Topics 1–4, and Factor 2 covered only EBP Topics 2 and 5. The internal‐consistency reliability was 0.88 for both factors (Table 1).

5.4. Unidimensionality and Measurement Invariance

The infit and outfit values of all items were between the reference values of 0.5 to 1.5, thereby indicating a good fit to the unidimensional Rasch model. Figure 2 displays one example of the accompanying fit plot for the Item 2 (‘Request feedback from colleagues to improve care’). The plot indicates good agreement between the expected probability of a 1‐score (the solid curve) and the observed proportions within increasing EBP‐affinity‐level groups (the dots). All items displayed similar fit‐plot patterns.

The level of agreement between the estimated EBP item‐difficulty measures for respondents trained as VTRNs (x‐axis) and as CNAs (y‐axis) is depicted in Figure 3a,b. For Subscale 1 ‘Collecting practical information’, invariance was observed for all items except Item 5 (‘Apply own knowledge in case of problems with care’)—which was easier for CNAs (estimated difficulty B = −2.32) than it was for VTRNs (B = −3.06)—and Item 2 (‘Request feedback from colleagues to improve care’), which was easier for VTRNs (estimated difficulty B = −2.16) than it was for CNAs (B = −0.97). For Subscale 2 (‘Collecting theoretical information’), this was the case for Item 11 (‘Search for solutions in Use journals or textbooks in case of problems with care’)—which was easier for CNAs (B = 1.90) than it was for VTRNs (B = 1.02)—and Item 19 (‘Discuss ideas for improving care from symposia, training and journals with colleagues and practitioners’), which was easier for VTRNs (B = 0.01) than it was for CNAs (B = −0.65).

FIGURE 3.

FIGURE 3

Open in a new tab

(a) Item difficulty invariance VTRN versus CNA subscale 1 'collecting practical information'. (b) Item difficulty invariance VTRN versus CNA subscale 2 'collecting theoretical information'. (a) and (b) Dots indicate estimated item difficulty measures for CNA participants (x‐axis) and VTRN participants (y‐axis) on the logit unit scale. Solid diagonal line indicates perfect agreement between estimates. Dotted lines indicate 95% confidence bands for the item difficulty estimates.

5.5. Test–Retest Reliability

The test–retest ICC value was 0.76 (95% CI: 0.63–0.85) for the entire EBP‐Affinity Questionnaire, thereby reflecting good agreement, with values of 0.74 (95% CI: 0.60–0.84) for Subscale 1 and 0.79 (95% CI: 0.67–0.87) for Subscale 2.

5.6. Construct Validity

The results for the construct validity of the EBP‐Affinity Questionnaire are presented in Table 3. The hypothesis that ‘FRC’ would score higher on average (81.41, SD 14.7), as compared to ‘not FRC’ (76.64, SD 14.6) was confirmed: ղ 2 = 0.02 (p < 0.01). This was also to the case for the mean sum‐score differences between ‘VTRN’ (83.3, SD 15.5) and ‘CNA’ (78.36, SD 14.5): ղ 2 = 0.02 (p < 0.01).

TABLE 3.

Construct validity of EBP‐Affinity: Effect sizes for hypothesis testing.

Factor Subgroup n Mean sum score (SD) ղ 2 effect size p
Education level 1 CNA 329 50.0 (8.5) < 0.01 0.26
VTRN 99 51.1 (9.1)
2 CNA 329 28.4 (7.9) 0.04 < 0.01
VTRN 99 32.2 (8.1)
Total score CNA 329 78.4 (14.5) 0.02 < 0.01
VTRN 99 83.3 (15.5)
First responsible carer (FRC) 1 Yes 257 51.3 (8.3) 0.02 < 0.01
No 171 48.6 (8.9)
2 Yes 257 30.1 (8.4) 0.02 0.01
No 171 28.1 (7.4)
Total score Yes 257 81.4 (14.7) 0.02 < 0.01
No 171 76.6 (14.6)
Open in a new tab

Abbreviations: CNA, certified nurse assistant; VTRN, vocationally trained registered nurse.

6. Discussion

To our knowledge, the EBP‐Affinity Questionnaire is the first questionnaire specially developed to measure affinity with EBP amongst CNAs and VTRNs working in NHs. The final questionnaire consisted of 26 items remained, divided into five topics in order of the four steps of EBP: asking, acquiring, applying and assessing, with addition of the step ‘promoting expertise’. The questionnaire exhibited good psychometric properties, based on five qualities. First, the score distributions indicate that the EBP‐Affinity Questionnaire was well suited to the sample, with no apparent floor or ceiling effects, for the total scale or for the individual items. Second, two unidimensional factors were extracted, named: ‘Collecting practical information’ and ‘Collecting theoretical information’, with high internal consistency (α = 0.88) for both factors, which indicates that the items effectively measure underlying constructs of EBP‐affinity, supporting their reliability across diverse NH settings. This was confirmed by the scores on items for ability groups, which were consistent with those expected for the unidimensional Rasch model. Third, invariant measurement was observed for most items (with the exception of Items 5, 7, 11 and 19), thus indicating that the EBP‐Affinity Questionnaire items provide the same information for both CNAs and VTRNs. There was hardly any item bias related to background of nursing education. Fourth, the test–retest reliability of the total EBP‐Affinity Questionnaire was ‘good’, and even higher than the results reported for the EBPQ‐ve and the EBPAS‐ve questionnaires (Maessen et al. 2019). Fifth, and finally, construct validity was supported by the expected score difference between subgroups. Overall, the questionnaire appears to be reliable and valid for use with the intended target population within the context of NHs in the Netherlands.

We would like to highlight two points of interest based on the results of the psychometric validity. Firstly, the eta‐squared effect size of construct validity was small, based on the difference between ‘EQF 3’ and ‘EQF 4’ and between ‘FRC’ and ‘Not FRC’, with significantly higher average sum scores for ‘EQF 4’ and ‘FRC’. It should be noted that the groups based on ‘educational level’ and ‘FRC’/‘Not FRC’ overlapped with and influenced each other. The FRC is responsible for ensuring that the care provided to the assigned residents is up to date. Given that motivation, responsibility and self‐confidence are fundamental to ensuring that professional care is properly aligned with the wishes and needs of the residents (Bandura 1997), it would be logical to expect the FRC to be associated with more affinity with EBP. This expectation was confirmed by our results. The difference between ‘FRC’ and ‘Not FRC’ may have been influenced by the fact that both CNAs and VTRNs are allowed to execute the function of FRC. In addition to job level, a healthcare provider's role in practice and, possibly, level of education could influence affinity with EBP. It would therefore be interesting to investigate whether job level or the role that a healthcare provider fulfils in practice has a greater influence on affinity with EBP. Other well‐known barriers to the adoption of EBP in practice include workload, lack of autonomy to change practices and limited support from management (Melnyk et al. 2012). The organisation thus also plays an important role in promoting affinity with EBP and creating favourable conditions for the application of EBP. Secondly, the invariant measurement indicates that the EBP‐Affinity Questionnaire items provide the same information for both CNAs and VTRNs. However, Figure 3 exhibit some responder bias within each subscale, respectively item 5, 7 11 and 19. Therefore, it is certainly interesting to look how some items could be reframed to enhance relevance. For example, item 5 could be reframed to specify contexts of application, such as ‘Apply knowledge in addressing care discrepancies’. Item 19 could be reframed to broader contexts than scientific knowledge, such as ‘Discuss ideas from other work settings than my own’. Nevertheless, we do not expect this to be problematic on the sumscore level. We therefore choose to maintain these items in the instrument in view of their content.

The results of this study provide insight into the roles that CNAs and VTRNs can play in the various steps of EBP. Based on these results, it could be expected that CNAs and VTRNs can work according to EBP in the provision of care to NH residents. When providing care, they can reflect on it, ask themselves whether the care plans are well suited to the residents, and search for ways to improve care for residents in protocols, professional journals and textbooks. They are also capable of applying more suitable care, in addition to assessing and evaluating these changes after they have been applied. Along with the steps of EBP, Straus et al. (2018) identifies three specific roles that can be assumed when practicing EBP: (1) the doer, (2) the user and (3) the replicator. The assessments of the experts interviewed for this study confirm that the role of ‘doer’ (i.e., appraising the strength of the evidence) is not suitable for the job profiles of CNAs and VTRNs. At the same time, however, the roles (i.e., ‘user’ and ‘replicator’) are highly appropriate for CNAs and VTRNs, as also confirmed by the study conducted by Maessen et al. (2019). Having become an indispensable part of care for older adults, EBP offers a solution to the growing complexity of care, which is increasingly coming under pressure (Alders and Schut 2019). In the future, it will be important to ensure that CNAs and VTRNs actually fulfil the role of ‘users’ and ‘replicators’ of EBP, in order to ensure the provision of appropriate care within a complex environment to improve outcomes and safety for NH residents, in addition to reducing costs and adverse events (Rudman et al. 2020; Teixeira, Nogueira, and Barbieri‐Figueiredo 2023).

6.1. Strengths and Weaknesses of the Study

One strength of this study is the involvement of CNAs, VTRNs and experts in the development of the questionnaire. In addition, an extensive psychometric assessment was performed based on a large sample (N = 428), which allowed a PCA and Rasch analysis to test the unidimensionality of the EBP‐Affinity Questionnaire. Despite achieving the minimum sample size according to the COSMIN checklist (Mokkink et al. 2019), the survey results may reflect selection bias due to distribution and response. Of the large target population (about 2000) invited to participate in the survey, only 428 respondents completed questionnaires that could be used for the analysis. We did not ask non‐responders for information regarding gender, age or current position. It is therefore possible that the respondents consisted primarily of employees who were already motivated to use EBP. Given the normal distribution of the sum scores, however, selection bias is likely to have been low. This assumption was confirmed by the spread of characteristics within the composition of the target population.

The timeframe within which the study was conducted did not allow for extensive consensus rounds with focus‐group discussions on items on which no consensus is achieved. These items were discussed with the researchers L.T., M.L. and A.P. In addition to the role of an expert, M.L. participated in the discussion of the items on which the expert group. Although the researchers were aware of this dual role and always reflected on her opinion in the light of it, this double role may have influenced the items to include or not to include in the questionnaire.

6.2. Recommendations for Future Research

Although CNAs and VTRNs constitute the largest group of healthcare providers working in NHs, a shift is taking place, in which the average function level is gradually decreasing (European Commission 2018; Rommets and Roelvink 2023). The reliability of the EBP‐Affinity Questionnaire has not been investigated amongst care providers with educational levels lower than EQF 3. For this reason, it is also important to adjust and validate the EBP‐Affinity Questionnaire for this target group in order to investigate affinity with EBP amongst nursing assistants at the level of EQF 2. According to their professional profile, nursing assistants can adequately signal changes in the needs, behaviour and state of mind of their patients and clients, in addition to being able to act effectively according to protocols and instructions (Morgan et al. 2016), thus corresponding to Steps 1 and 5 of EBP.

Besides that, we recommend other research for further studies. In first place, the responsiveness of the questionnaire was not examined in this study. Future research should investigate the sensitivity of the questionnaire to measure changes over time. In second place, it is interesting to investigate the validity of the questionnaire in international settings to confirm its cross‐cultural usefulness.

6.3. Implications for Practice and Education

Assessing affinity with EBP amongst CNAs and VTRNs in NHs is an important step towards identifying problems in healthcare and making evidence‐based choices in order to improve the care for frail elderly residents. Such assessment is also an important step towards embedding EBP in practice, thereby cultivating a culture of EBP in NHs. Within such a culture, the EBP‐Affinity Questionnaire could be used to map affinity with EBP, as expressed by VTRNs and CNAs. The EBP‐Affinity Questionnaire also provides insight into factors that could be addressed in order to enhance affinity with EBP. This could increase the likelihood that success in practice and research will find its way back into practice.

It is important for CNAs and VTRNs to incorporate EBP into their work. The EBP‐Affinity Questionnaire can be used to measure their affinity with EBP by assessing whether they do or do not reflect on and evaluate the care they provide, search for suitable care in protocols or professional journals, and promote their expertise on their own initiative. Feedback on the results of the questionnaire could encourage reflection on the part of care teams, and they could provide additional input regarding actions that are needed in order to enhance affinity with EBP and begin to implement EBP interventions in practice.

Greater attention to EBP is needed during the initial and continuing education of CNAs and VRTNS, with focus on applying evidence collected by others, independently searching databased for guidelines and other available evidence, developing an attentive attitude when proving care, and asking critical questions. The questionnaire could be used as a practical continuous improvement and human resource tool administered annually to assess training needs, with results integrated into team evaluations or professional development plans.

Despite the creation of favourable conditions in practice and devoting attention to EBP in training, it is not feasible in practice for CNAs and VTRNs to apply Step 3 of the EPB cycle (‘appraising the strength of the evidence’). This step is nevertheless necessary to the proper application of EBP in practice. The appointment of Bachelor's‐level registered nurses (Backhaus et al. 2018) are therefore crucial in NHs. Because they are able to fulfil the role of ‘doer’, these healthcare providers are indispensable within the NH setting.

7. Conclusion

The Evidence‐Based Practice‐Affinity Questionnaire is a reliable and valid questionnaire for assessing affinity with EBP amongst CNAs and VTRNs. As such, it can serve as a vantage point for improving practice, in addition to its utility in research, vocational education and tailored training programs for CNAs and VTRNs. Future research should validate the tool across international settings to ensure its cross‐cultural applicability and explore its responsiveness to interventions. Instead of being regarded as a concept that does not belong in NHs, EBP should have a meaningful place in the daily activities performed by CNAs and VTRNs. These healthcare providers that work in various workplace settings in the global nursing landscape are capable of taking a critical view of care, gathering information, applying new knowledge, evaluating care and promoting their expertise.

Author Contributions

L.J.T., M.H., R.T.C.M.K., D.F.K., A.P., R.L., D.L.G. and M.H.L.: made substantial contributions to conception and design, or acquisition of data, analysis, or analysis and interpretation of data; involved in drafting the manuscript or revising it critically for important intellectual content; given final approval of the version to be published. Each author should have participated sufficiently in the work to take public responsibility for appropriate portions of the content and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Ethics Statement

The Medical Ethical Review Committee (METC) deemed that the study did not fall within the scope of the Dutch Medical Research Involving Human Subjects Act (WMO). The participants in both expert groups were informed by email about the study and the processing of confidential data. Thereafter, the participants provided consent by email to participate in the study. The pilot group also provided consent by email. A consent form for the participation of certified nurse assistants and vocationally‐trained registered nurses was included in the online questionnaire. The data were gathered anonymously.

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgements

This research received no specific grant from any funding agency in the public, commercial, or not‐for‐profit sectors. A special thanks from the authors to the experts, the vocationally trained registered nurses and certified nurse assistants working at the nine participating organisations for their important contribution to this study. Also a special thanks to SVRZ, which offered the researcher the opportunity to conduct this research in the context of the Master Evidence‐Based Practice, and in addition the Master‐EBP who conveyed the knowledge to properly conduct this research.

Funding: The authors received no specific funding for this work.

Statistician: Robert Lindeboom.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

  1. Aarons, G. A. , Cafri G., Lugo L., and Sawitzky A.. 2012. “Expanding the Domains of Attitudes Towards Evidence‐Based Practice: The Evidence Based Practice Attitude Scale‐50.” Administration and Policy in Mental Health 39, no. 5: 331–340. 10.1007/s10488-010-0302-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Aarons, G. A. , Glisson C., Hoagwood K., Kelleher K., Landsverk J., and Cafri G.. 2010. “Psychometric Properties and U.S. National Norms of the Evidence‐Based Practice Attitude Scale (EBPAS).” Psychological Assessment 22, no. 2: 356–365. 10.1037/a0019188. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Alders, P. , and Schut F. T.. 2019. “Trends in Ageing and Ageing‐In‐Place and the Future Market for Institutional Care: Scenarios and Policy Implications.” Health Economics, Policy, and Law 14, no. 1: 82–100. 10.1017/s1744133118000129. [DOI] [PubMed] [Google Scholar]
  4. Babakus, E. , and Mangold W. G.. 1992. “Adapting the SERVQUAL Scale to Hospital Services: An Empirical Investigation.” Health Services Research 26, no. 6: 767–786. https://www.ncbi.nlm.nih.gov/pubmed/1737708. [PMC free article] [PubMed] [Google Scholar]
  5. Backhaus, R. , Verbeek H., van Rossum E., Capezuti E., and Hamers J. P. H.. 2018. “Baccalaureate‐Educated Registered Nurses in Nursing Homes: Experiences and Opinions of Administrators and Nursing Staff.” Journal of Advanced Nursing 74, no. 1: 75–88. 10.1111/jan.13391. [DOI] [PubMed] [Google Scholar]
  6. Bandura, A. 1997. Self‐Efficacy: The Exercise of Control. New York, NY: W.H. Freeman. [Google Scholar]
  7. Barry, M. , Kuijer W., Persoon A., Nieuwenhuis L., and Scherpbier N.. 2021. “Enabling Visibility of the Clinician‐Scientists' Knowledge Broker Role: A Participatory Design Research in the Dutch Nursing‐Home Sector.” Health Research Policy and Systems 19, no. 1: 61. 10.1186/s12961-021-00715-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Brink, M. , and Joldersma C.. 2016. “Personnel and Its Education in Nursing Homes.” https://www.rijksoverheid.nl/documenten/rapporten/2016/12/01/personeel‐en‐opleiden‐in‐verpleeghuizen.
  9. Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences. Mahwah, New Jersey: Lawrence Erlbaum Associates. https://www.utstat.toronto.edu/~brunner/oldclass/378f16/readings/CohenPower.pdf. [Google Scholar]
  10. Conroy, S. , Van Der Cammen T., Schols J., Van Balen R., Peteroff P., and Luxton T.. 2009. “Medical Services for Older People in Nursing Homes—Comparing Services in England and the Netherlands.” Journal of Nutrition, Health & Aging 13, no. 6: 559–563. 10.1007/s12603-009-0107-9. [DOI] [PubMed] [Google Scholar]
  11. Cook, D. J. , Jaeschke R., and Guyatt G. H.. 1992. “Critical Appraisal of Therapeutic Interventions in the Intensive Care Unit: Human Monoclonal Antibody Treatment in Sepsis. Journal Club of the Hamilton Regional Critical Care Group.” Journal of Intensive Care Medicine 7, no. 6: 275–282. 10.1177/088506669200700601. [DOI] [PubMed] [Google Scholar]
  12. Dawes, M. , Summerskill W., Glasziou P., et al. 2005. “Sicily Statement on Evidence‐Based Practice.” BMC Medical Education 5, no. 1: 1. 10.1186/1472-6920-5-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Diehl, H. , Graverholt B., Espehaug B., and Lund H.. 2016. “Implementing Guidelines in Nursing Homes: A Systematic Review.” BMC Health Services Research 16: 298. 10.1186/s12913-016-1550-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. European Commission , ed. 2018. The European Qualifications Framework: Supporting Learning, Work and Cross‐Border Mobility. Luxembourg: Publications Office of the European Union. [Google Scholar]
  15. Field, A. , Miles J., and Field Z.. 2012. Discovering Statistics Using R. California, New Delhi: SAGE Publications London. [Google Scholar]
  16. Fineout‐Overholt, E. , and Johnston L.. 2005. “Teaching EBP: ‘Getting From Zero to One’.Moving From Recognizing and Admitting Uncertainties to Asking Searchable, Answerable Questions.” Worldviews on Evidence‐Based Nursing 2, no. 2: 98–102. 10.1111/j.1741-6787.2005.05006.x. [DOI] [PubMed] [Google Scholar]
  17. Haavisto, E. , Siltanen H., Tolvanen A., and Holopainen A.. 2023. “Instruments for Assessing Healthcare Professionals' Knowledge and Skills of Evidence‐Based Practice: A Scoping Review.” Journal of Clinical Nursing 32, no. 15–16: 4391–4407. 10.1111/jocn.16506. [DOI] [PubMed] [Google Scholar]
  18. Hak, T. , Veer van der K., and Jansen H.. 2008. “The Three‐Step Test‐Interview (TSTI): An Observation‐Based Method for Pretesting Self‐Completion Questionnaires [Survey Research Methods].” European Survey Research Association 2, no. 3: 143–150. http://hdl.handle.net/1765/1265. [Google Scholar]
  19. Haterd, J. , and Zwikker N.. 2009. “Profiel Eerst Verantwoordelijk Verzorgende. 3e Herziene Versie op Basis Van Het Profiel Eerst Verantwoordelijk Verzorgende uit 2003.” https://www.aovvt.nl/wp‐content/uploads/2020/09/Profiel_Eerst_Verantwoordelijk_Verzorgende__2009_‐1.pdf.
  20. Hoegen, P. , Bot de C. M. A., Echteld M. A., and Vermeulen H.. 2021. “Measuring Self‐Efficacy and Outcome Expectancy in Evidence‐Based Practice: A Systematic Review on Sychometric Properties [Research Article].” International Journal of Nursing Studies Advances 12: 100024. 10.1016/j.ijnsa.2021.100024. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Kim, S. C. , Ecoff L., Brown C. E., Gallo A. M., Stichler J. F., and Davidson J. E.. 2017. “Benefits of a Regional Evidence‐Based Practice Fellowship Program: A Test of the ARCC Model.” Worldviews on Evidence‐Based Nursing 14, no. 2: 90–98. 10.1111/wvn.12199. [DOI] [PubMed] [Google Scholar]
  22. Koo, T. K. , and Li M. Y.. 2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15, no. 2: 155–163. 10.1016/j.jcm.2016.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Linacre, J. M. 2002. “Rasch Measurement Transactions: What Do Infit and Outfit, Mean‐Square and Standardized Mean?” https://www.rasch.org/rmt/rmt162f.htm.
  24. Lovink, M. H. , Verbeek F., Persoon A., et al. 2022. “Developing an Evidence‐Based Nursing Culture in Nursing Homes: An Action Research Study.” International Journal of Environmental Research and Public Health 19, no. 3: 19031733. 10.3390/ijerph19031733. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Maessen, K. , van Vught A., Gerritsen D. L., Lovink M. H., Vermeulen H., and Persoon A.. 2019. “Development and Validation of the Dutch EBPAS‐Ve and EBPQ‐Ve for Nursing Assistants and Nurses With a Vocational Education.” Worldviews on Evidence‐Based Nursing 16, no. 5: 371–380. 10.1111/wvn.12389. [DOI] [PubMed] [Google Scholar]
  26. Mair, P. , and Hatzinger R.. 2007. “Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R.” Journal of Statistical Software 20, no. 9: 1–20. 10.18637/jss.v020.i09. [DOI] [Google Scholar]
  27. Melnyk, B. M. , Fineout‐Overholt E., Gallagher‐Ford L., and Kaplan L.. 2012. “The State of Evidence‐Based Practice in US Nurses: Critical Implications for Nurse Leaders and Educators.” Journal of Nursing Administration 42, no. 9: 410–417. 10.1097/NNA.0b013e3182664e0a. [DOI] [PubMed] [Google Scholar]
  28. Melnyk, B. M. , Fineout‐Overholt E., Giggleman M., and Cruz R.. 2010. “Correlates Among Cognitive Beliefs, EBP Implementation, Organizational Culture, Cohesion and Job Satisfaction in Evidence‐Based Practice Mentors From a Community Hospital System.” Nursing Outlook 58, no. 6: 301–308. 10.1016/j.outlook.2010.06.002. [DOI] [PubMed] [Google Scholar]
  29. Mokkink, L. B. , Prinsen C. A. C., Patrick D. L., et al. 2019. “COSMIN Study Design Checklist for Patient‐Report Outcome Measurement Instruments.” https://www.cosmin.nl/wp‐content/uploads/COSMIN‐study‐designing‐checklist_final.pdf.
  30. Mokkink, L. B. , Terwee C. B., Knol D. L., et al. 2010. “The COSMIN Checklist for Evaluating the Methodological Quality of Studies on Measurement Properties: A Clarification of Its Content.” BMC Medical Research Methodology 10: 22. 10.1186/1471-2288-10-22. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Mokkink, L. B. , Terwee C. B., Patrick D. L., et al. 2010. “The COSMIN Study Reached International Consensus on Taxonomy, Terminology, and Definitions of Measurement Properties for Health‐Related Patient‐Reported Outcomes.” Journal of Clinical Epidemiology 63, no. 7: 737–745. 10.1016/j.jclinepi.2010.02.006. [DOI] [PubMed] [Google Scholar]
  32. Morgan, D. G. , Kosteniuk J. G., O'Connell M. E., Dal Bello‐Haas V., Stewart N. J., and Karunanayake C.. 2016. “Dementia‐Related Work Activities of Home Care Nurses and Aides: Frequency, Perceived Competence, and Continuing Education Priorities.” Educational Gerontology 42, no. 2: 16. 10.1080/03601277.2015.1083390. [DOI] [Google Scholar]
  33. OECD . 2013. Health at a Glance 2013: OECD Indicators. Paris, France: OECD Publishing. 10.1787/health_glance-2013-en. [DOI] [Google Scholar]
  34. Revelle, W. 2024. “Package ‘Psych’: Procedures for Psychological, Psychometric, and Personality Research.” https://personality‐project.org/r/psych‐manual.pdf.
  35. Robertson‐Preidler, J. , Biller‐Andorno N., and Johnson T. J.. 2017. “What Is Appropriate Care? An Integrative Review of Emerging Themes in the Literature.” BMC Health Services Research 17, no. 1: 452. 10.1186/s12913-017-2357-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Rommets, J. , and Roelvink R.. 2023. “Algemene Personele Kengetallen Kwaliteitskader Verpleeghuiszorg 2022.” Dageraad. https://www.actiz.nl/sites/default/files/inline‐files/Rapport%20Dageraad%20%E2%80%98Algemene%20Personele%20Kengetallen%20Kwaliteitskader%20Verpleeghuiszorg%202021%E2%80%99.pdf.
  37. Rudman, A. , Bostrom A. M., Wallin L., Gustavsson P., and Ehrenberg A.. 2020. “Registered Nurses' Evidence‐Based Practice Revisited: A Longitudinal Study in Mid‐Career.” Worldviews on Evidence‐Based Nursing 17, no. 5: 348–355. 10.1111/wvn.12468. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Rye, M. , Torres E. M., Friborg O., Skre I., and Aarons G. A.. 2017. “The Evidence‐Based Practice Attitude Scale‐36 (EBPAS‐36): A Brief and Pragmatic Measure of Attitudes to Evidence‐Based Practice Validated in US and Norwegian Samples.” Implementation Science 12, no. 1: 44. 10.1186/s13012-017-0573-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Sackett, D. L. , Rosenberg W. M., Gray J. A., Haynes R. B., and Richardson W. S.. 1996. “Evidence Based Medicine: What It Is and What It Isn't.” British Medical Journal 312, no. 7023: 71–72. 10.1136/bmj.312.7023.71. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Samenwerkingsorganisatie Beroepsonderwijs Bedrijfsleven . 2020. https://kwalificatie‐mijn.s‐bb.nl/Home/Zoekresultaat?rf=True&pi=10&p=1&s=&sd=&q=&t=5&fck=&fn=3&fn=4&fv=false&fw=ja&fs=5674.
  41. Specht, J. K. 2013. “Evidence Based Practice in Long Term Care Settings.” Journal of Korean Academy of Nursing 43, no. 2: 145–153. 10.4040/jkan.2013.43.2.145. [DOI] [PubMed] [Google Scholar]
  42. Straus, S. E. , Glasziou P., Richardson S., and Haynes R. B.. 2018. Evidence‐Based Medicine How to Practice and Teach EBM. Edinburgh, Scotland: Elsevier. [Google Scholar]
  43. Teixeira, A. C. , Nogueira A., and Barbieri‐Figueiredo M. D. C.. 2023. “Professional Empowerment and Evidence‐Based Nursing: A Mixed‐Method Systematic Review.” Journal of Clinical Nursing 32, no. 13–14: 3046–3057. 10.1111/jocn.16507. [DOI] [PubMed] [Google Scholar]
  44. Timmerman, M. E. , and Lorenzo‐Seva U.. 2011. “Dimensionality Assessment of Ordered Polytomous Items With Parallel Analysis.” Psychological Methods 16, no. 2: 209–220. 10.1037/a0023353. [DOI] [PubMed] [Google Scholar]
  45. Tuinman, A. , de Greef M. H., Krijnen W. P., Nieweg R. M., and Roodbol P. F.. 2016. “Examining Time Use of Dutch Nursing Staff in Long‐Term Institutional Care: A Time‐Motion Study.” Journal of the American Medical Directors Association 17, no. 2: 148–154. 10.1016/j.jamda.2015.09.002. [DOI] [PubMed] [Google Scholar]
  46. Upton, D. , Upton P., and Scurlock‐Evans L.. 2014. “The Reach, Transferability, and Impact of the Evidence‐Based Practice Questionnaire: A Methodological and Narrative Literature Review.” Worldviews on Evidence‐Based Nursing 11, no. 1: 46–54. 10.1111/wvn.12019. [DOI] [PubMed] [Google Scholar]
  47. Warren, J. I. , McLaughlin M., Bardsley J., et al. 2016. “The Strengths and Challenges of Implementing EBP in Healthcare Systems.” Worldviews on Evidence‐Based Nursing 13, no. 1: 15–24. 10.1111/wvn.12149. [DOI] [PubMed] [Google Scholar]
  48. Wolf, D. A. P. S. , Dulmus C. N., Maguin E., and Fava N.. 2014. “Refining the Evidence‐Based Practice Attitude Scale: An Alternative Confirmatory Factor Analysis.” Social Work Research 38, no. 1: 47–58. 10.1093/swr/svu006. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Articles from Nursing Open are provided here courtesy of Wiley

RESOURCES