Skip to main content
NCBI home page
Nordisk Alkohol- & Narkotikatidskrift : NAT logoLink to Nordisk Alkohol- & Narkotikatidskrift : NAT
. 2023 Aug 10;41(1):97–110. doi: 10.1177/14550725231175373

Validation of the Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department (NSCAIP-ED) instrument

Kaisu-Leena Mäkelä 1,✉, Jari Kylmä 2, Anna-Maija Koivisto 3, Tiina Hakala 4, Marita Koivunen 5
PMCID: PMC10863552  PMID: 38356789

Abstract

Aim To report the development and validation of the Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department instrument, shortened NSCAIP-ED. Methods: A mixed-methods design was used to develop the instrument. It was used to conduct a survey where ED nurses self-evaluated their skills to care for acutely ill alcohol-intoxicated patients (N = 1220, n = 252). The data were utilised to perform instrument validation using confirmatory factor analysis (CFA) and Cronbach's alpha. Results: The construct validity was tested statistically. The CFA model fit indicators showed mostly acceptable fit (chi-square test p < 0.001; RMSEA 0.079; CFI 0.923; TLI 0.918; SRMR 0.084) and instruments’ scales had well acceptable Cronbach's alpha values (all alphas were in the range of 0.866–0.912). Conclusions: The NSCAIP-ED is a feasible and reliable instrument that can be used when measuring nurses’ skills to care for alcohol-intoxicated patients in the ED. This instrument could be useful for nursing managers in EDs for evaluating their nursing staff's skills in the care area in question, but also for designing continuing education based on the results.

Keywords: alcohol, emergency department, instrument, nursing, skills, validation

Introduction

Alcohol is the cause of many visits to hospital and emergency departments (ED) (Bakke et al., 2016; Blow et al., 2010; Xx., 2021). Caring for alcohol-intoxicated patients requires different types of skills from nurses who work in this context (Clarke et al., 2015, Xx., 2021; Hellum et al., 2016; Warren et al., 2012). When it comes to this patient group, EDs around the world deal with the same kind of situations. The somatic symptoms of ED patients are usually managed, whereas dealing with alcohol use disorder is often missing. When a patient is alcohol-intoxicated, it is not known whether the person is a user, abuser or alcohol dependent before they are evaluated in some form of intervention by health care professional. Nursing staff working in the ED do not necessarily have sufficient skills to care for acutely ill alcohol-intoxicated patients, to evaluate the alcohol use of these patients, and provide help for their possible risk use, rin ordr to educe it or even stop it altogether (Iqbal et al., 2015; Russell et al., 2017; Mäkelä et al., 2020). Based on earlier research, the skills that ED nursing staff should have are as follows: evaluation of alcohol risk use skills; interaction skills; ethical skills; counselling and education skills; security and safety skills; teamwork and networking skills; and follow-up treatment skills (Hakala et al., 2020; Mäkelä et al., 2020). To ensure high-quality and appropriate care for patients, the nursing competence of healthcare workers needs to be at a sufficient level. The skills of nursing staff must be continously evaluated to gauge their level of competence (Russell et al., 2017).

Different kinds of nursing skills or nursing staff's competencies have been assessed in various care areas by creating specific instruments. Measuring these skills is a way of determining the level of preparedness and competence and assessing whether training or further education is needed (Araújo et al., 2016; Bing-Jonsson et al., 2015; Lin et al., 2009; Meretoja et al., 2004). However, no instrument measuring the skills to care for acutely ill alcohol-intoxicated patients in ED settings was found to be available for use. That is why development and validation of an instrument of this kind is important. It also corresponds to the aims of the World Health Organization (WHO) as one objective of the WHO's alcohol strategy is to increase awareness of the complexity of alcohol-related problems and promote the development of effective interventions to reduce and prevent problems exacerbated by alcohol use (World Health Organization, 2022). The development and validation of the Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department (NSCAIP-ED) instrument was initiated to meet this need.

The aim of the study was to report the development and validation of the NSCAIP-ED instrument.

Methods

A mixed-methods design was used in the development process of the NSCAIP-ED. It included both deductive and inductive research methods, such as email surveys, group interviews, an integrative literature review and personal interviews. After a pilot study of the instrument, a descriptive quantitative survey was conducted with the instrument to measure the self-evaluated skills of Finnish ED nursing staff concerning the care of acutely ill alcohol-intoxicated patients in the year 2016. The survey was sent to all Finnish central hospitals and university hospitals EDs. The target group was nursing staff (N = 1220) in the EDs. A total of 252 individuals responded. The descriptive data of this survey have been reported in a separate article (Mäkelä et al., 2020).

For the instrument validation, a psychometric analysis was executed. The evaluation of content validity was based on the views of substance specialised experts in the research group considering instruments’ scales and items. Usability of the NSCAIP-ED was assessed based on earlier research (Mäkelä et al., 2020). Construct validity was evaluated using factor analysis. Cronbach's alpha values were calculated when evaluating internal consistency.

Instrument development

Stages of the instrument development were as follows. First, three Delphi rounds were arranged to obtain professionals’ opinions. Round 1 was an email survey (n = 6) and round 2 was group interviews (n = 6). Five nurses and a social worker working in an ED participated in the group interviews. Round 3 was a group interview of the expert panel, which included two nurses and a nursing manager of one ED, a team leader of sobering unit, a project designer of the special skills project and a nursing director of development and evaluation. The aim of the second phase was to conduct an integrative literature review (Hakala et al., 2017). Then, six personal interviews were performed with nurses working in the ED. After every phase, the findings were processed and the instrument was generated based on them. The instrument consisted of seven scales and 49 items. The scales and items were based on competence requirements that were defined at the beginning of the instrument development with the expert panel, and they were also commented on by physician specialising in substance abuse care. Finally, the research group that was established for the instrument’s content evaluation agreed that all the items were adequate to the instrument. This research group consisted of a substance abuse treatment specialised nurse and physician, a nursing manager of the ED and a director of the research group who represented scientific nursing research competence (Figure 1).

Figure 1.

Figure 1.

Open in a new tab

Development of the NSCAIP-ED instrument.

The NSCAIP-ED instrument contained seven scales: (1) alcohol risk use evaluation skills (11 items); (2) interaction skills (6 items); (3) ethical skills (7 items); (4) counselling and education skills (7 items); (5) security and safety skills (9 items); (6) teamwork and network skills (5 items); and (7) follow-up treatment skills (4 items). The instrument consisted of a total of 49 5-point Likert-type items (Table 1). The participants self-evaluated their skills using the following scale: 1 = extremely poor; 2 = reasonably poor; 3 = reasonably good; 4 = good; and 5 = extremely good. The background factors included in the instrument were nursing staff's organisation, position, work experience in healthcare, emergency care, psychiatric nursing and intoxicant care, and participation in alcohol use-related training and substance abuse-related training (Hakala et al., 2020; Mäkelä et al., 2020).

Table 1.

The Nurses’ Skills to Care for Alcohol-Intoxicated Patients in Emergency Department survey.

Descriptive statistics
Item Cronbach’s
alphaa 		n Extremely
poor		 Reasonably
poor		 Reasonably
good		 good Extremely
good		 Median
(Q1-Q3b)
Alcohol risk use evaluation skills  0.912
1. I know how to use the AUDIT-test to identify risky alcohol consumption 251 58 (23.1) 50 (19.9) 64 (25.5) 56 (22.3) 23 (9.2) 3.0 (2.0–4.0)
2. I know how to assess the severity of alcohol problem 251 10 (4.0) 68 (27.1) 110 (43.8) 53 (21.1) 10 (4.0) 3.0 (2.0–3.0)
3. I understand the mechanisms of alcohol dependence 249 8 (3.2) 74 (29.7) 92 (36.9) 63 (25.3) 12 (4.8) 3.0 (2.0–4.0)
4. I know the pattern of the change phase of alcohol dependence 250 77 (30.8) 110 (44.0) 37 (14.8) 18 (7.2) 8 (3.2) 2.0 (1.0–2.0)
5. I am aware of the effects of alcohol dependence on a person’s physical well-being 249 3 (1.2) 20 (8.0) 68 (27.3) 110 (44.2) 48 (19.3) 4.0 (3.0–4.0)
6. I am aware of the effects of alcohol dependence on a person’s mental well-being 251 1 (0.4) 28 (11.2) 81 (32.3) 96 (38.2) 45 (17.9) 4.0 (3.0–4.0)
7. I am aware of the effects of alcohol dependence on a person’s social well-being 252 1 (0.4) 18 (7.1) 82 (32.5) 106 (42.1) 45 (17.9) 4.0 (3.0–4.0)
8. I know how to bring up alcohol use 251 9 (3.6) 48 (19.1) 90 (35.9) 81 (32.3) 23 (9.2) 3.0 (3.0–4.0)
9. I know how to choose a suitable moment to discuss alcohol use 250 7 (2.8) 62 (24.8) 102 (40.8) 64 (25.6) 15 (6.0) 3.0 (2.0–4.0)
10. I know how to map the patient’s social network 249 18 (7.2) 83 (33.3) 95 (38.2) 37 (14.9) 16 (6.4) 3.0 (2.0–3.0)
11. I know how to take the patient’s life situation into account when planning and  implementing patient care 252 11 (4.4) 67 (26.6) 107 (42.5) 54 (21.4) 13 (5.2) 3.0 (2.0–4.0)
Interaction skills 0.877
12. I know how to consider the person with problematic alcohol use as an individual 251 4 (1.6) 40 (15.9) 120 (47.8) 76 (30.3) 11 (4.4) 3.0 (3.0–4.0)
13. I have good command of professional interaction skills 252 0 (0.0) 9 (3.6) 76 (30.2) 126 (50.0) 41 (16.3) 4.0 (3.0–4.0)
14. I recognize the content of nonverbal communication 250 1 (0.4) 15 (6.0) 89 (35.6) 111 (44.4) 34 (13.6) 4.0 (3.0–4.0)
15. I can convey a feeling to the patient that she/he has been heard 251 0 (0.0) 16 (6.4) 113 (45.0) 97 (38.6) 25 (10.0) 3.0 (3.0–4.0)
16. I know how to use comprehensible concepts in conversation with a patient 252 0 (0.0) 10 (4.0) 79 (31.3) 121 (48.0) 42 (16.7) 4.0 (3.0–4.0)
17. I know how to take patient’s relatives and loved ones into account 250 0 (0.0) 17 (6.8) 93 (37.2) 115 (46.0) 25 (10.0) 4.0 (3.0–4.0)
Ethical skills 0.866
18. I know how to use ethical guidelines in patient care 250 9 (3.6) 28 (11.2) 113 (45.2) 82 (32.8) 18 (7.2) 3.0 (3.0–4.0)
19. I am able to observe ethical challenges in my work 248 3 (1.2) 14 (5.6) 104 (41.9) 99 (39.9) 28 (11.3) 4.0 (3.0–4.0)
20. I am able to face the patient with respect 251 1 (0.4) 4 (1.6) 53 (21.1) 134 (53.4) 59 (23.5) 4.0 (4.0–4.0)
21. I am able to face the patient as an individual 249 2 (0.8) 6 (2.4) 57 (22.9) 134 (53.8) 50 (20.1) 4.0 (4.0–4.0)
22. I am able to face the patient in a confidential manner 249 1 (0.4) 1 (0.4) 39 (15.7) 116 (46.6) 92 (36.9) 4.0 (4.0–5.0)
23. I am aware of my own attitudes and relations towards intoxicants 250 1 (0.4) 1 (0.4) 33 (13.2) 111 (44.4) 104 (41.6) 4.0 (4.0–5.0)
24. I understand how my own attitude is reflected on the patient 251 1 (0.4) 2 (0.8) 44 (17.5) 119 (47.4) 85 (33.9) 4.0 (4.0–5.0)
Counseling and education skills 0.880
25. I understand the importance of counseling and education in the care of an intoxicated patient 249 0 (0.0) 15 (6.0) 74 (29.7) 126 (50.6) 34 (13.7) 4.0 (3.0–4.0)
26. I know how to use the information I receive from the patient and relatives to educate the patient 250 2 (0.8) 22 (8.8) 82 (32.8) 123 (49.2) 21 (8.4) 4.0 (3.0–4.0)
27. I know how to educate the patient based on the patient’s individual needs 249 2 (0.8) 47 (18.9) 96 (38.6) 93 (37.3) 11 (4.4) 3.0 (3.0–4.0)
28. I know how to utilize existing written material in the patient education 248 16 (6.5) 87 (35.1) 91 (36.7) 48 (19.4) 6 (2.4) 3.0 (2.0–3.0)
29. I know how to ensure the patient’s understanding in the education situation 249 2 (0.8) 34 (13.7) 106 (42.6) 89 (35.7) 18 (7.2) 3.0 (3.0–4.0)
30. I know how to assess the patient’s need for follow-up treatment 248 6 (2.4) 39 (15.7) 70 (28.2) 111 (44.8) 22 (8.9) 4.0 (3.0–4.0)
31. I know how to guide patient’s relatives and loved ones to find different types of support services 248 19 (7.7) 93 (37.5) 75 (30.2) 51 (20.6) 10 (4.0) 3.0 (2.0–3.0)
Security and safety skills 0.878
32. I know how to use a security alarm devices 249 0 (0.0) 8 (3.2) 21 (8.4) 88 (35.3) 132 (53.0) 5.0 (4.0–5.0)
33. I know how to use security services 247 0 (0.0) 1 (0.4) 10 (4.0) 77 (31.2) 159 (64.4) 5.0 (4.0–5.0)
34. I know how to take care of the safety of the care environment 248 1 (0.4) 0 (0.0) 45 (18.1) 134 (54.0) 68 (2.4) 4.0 (4.0–5.0)
35. I know how to identify threat of violence 248 0 (0.0) 8 (3.2) 55 (22.2) 130 (52.4) 55 (22.2) 4.0 (3.0–4.0)
36. I know how to mitigate the threat of violence 248 1 (0.4) 12 (4.8) 103 (41.5) 95 (38.3) 37 (14.9) 4.0 (3.0–4.0)
37. I know how to act in a violent situation 247 2 (0.8) 27 (10.9) 98 (39.7) 91 (36.8) 29 (11.7) 3.0 (3.0–4.0)
38. I am familiar with the instructions related to infection risks 248 0 (0.0) 4 (1.6) 50 (20.2) 130 (52.4) 64 (25.8) 4.0 (4.0–5.0)
39. I am familiar with the instructions related to infections isolation actions 248 1 (0.4) 8 (3.2) 52 (21.0) 131 (52.8) 56 (22.6) 4.0 (4.0–4.0)
40. I am familiar with the instructions for restricting freedom 248 2 (0.8) 19 (7.7) 69 (27.8) 120 (48.4) 38 (15.3) 4.0 (3.0–4.0)
Teamwork and network skills 0.896
41. I know how to share and receive expertise in a multi-professional team 250 2 (0.8) 11 (4.4) 71 (28.4) 125 (50.0) 41 (16.4) 4.0 (3.0–4.0)
42. I know how to share my skills across team boundaries 250 3 (1.2) 21 (8.4) 92 (36.8) 101 (40.4) 33 (13.2) 4.0 (3.0–4.0)
43. I know how to collaborate with partners outside my work unit 249 4 (1.6) 21 (8.4) 77 (30.9) 106 (42.6) 41 (16.5) 4.0 (3.0–4.0)
44. I know how to consult different experts 248 2 (0.8) 13 (5.2) 70 (28.2) 115 (46.4) 48 (19.4) 4.0 (3.0–4.0)
45. I know how to act as a consultant sharing my own professional knowledge 249 8 (3.2) 44 (17.7) 95 (38.2) 75 (30.1) 27 (10.8) 3.0 (3.0–4.0)
Follow-up treatment skills 0.907
46. I am familiar with the public sectors follow-up treatment services related to the care of alcohol problems 250 13 (5.2) 57 (22.8) 105 (42.0) 61 (24.4) 14 (5.6) 3.0 (2.0–4.0)
47. I am familiar with the private sectors follow-up treatment services related to the care of alcohol problems 250 48 (19.2) 113 (45.2) 64 (25.6) 20 (8.0) 5 (2.0) 2.0 (2.0–3.0)
48. I am familiar with the third sectors follow-up treatment services related to the care of alcohol problems 250 61 (24.4) 110 (44.0) 58 (23.2) 16 (6.4) 5 (2.0) 2.0 (1.0–3.0)
49. I know how to refer the patient to appropriate follow-up treatment 249 19 (7.6) 82 (32.9) 98 (39.4) 42 (16.9) 8 (3.2) 3.0 (2.0–3.0)
Open in a new tab

Note. Values are given as n (%) unless otherwise indicated.

a

Cronbach’s alpha values have been published before (Mäkelä et al., 2020). bThe lower and upper quartiles (Q1–Q3).

*The article has been updated to reflect the changes.

ED nurses (n = 11) working in a district hospital that did not belong to the target group of the final survey participated in a pilot study. The pilot study was conducted to ensure the feasibility of the instrument. Pre-testing the questions helps to ensure that the statements are meaningful when answered by the target population before the development of the instrument is completed. The purpose of pre-testing is to remove incorrect phrasing and inconsistencies from the instrument and thus make it as comprehensible as possible (Boateng et al., 2018). In this phase, feedback was collected to find out whether the participants understood the statements in the same way and find out their thoughts around answering the questionnaire. The participants suggested a few improvements, but they mainly felt that the NSCAIP-ED was easy to use: it took a reasonable 20 min to complete and the layout was suitable. No items were removed and no new ones were added after the pre-testing phase; a few corrections were made to the structures of the existing scales based on the feedback received (Hakala et al., 2020). After the pilot study, the final survey with the NSCAIP-ED instrument was conducted (Mäkelä et al., 2020).

Sample/participants

In this validation study, we used the data collected with the NSCAIP-ED instrument in 2016. It was a cross-sectional survey that was conducted on all Finnish ED nursing staff who worked in a central hospital or university hospital (N = 1220, n = 252). These hospitals were chosen as the target group because of the similarity of their EDs. The target organisations covered the whole of Finland geographically. The aim of that study was to describe nursing staff's self-assessed skills concerning the care of acutely ill alcohol-intoxicated patients in EDs (Mäkelä et al., 2020). According to “the rule of thumb”, at least five participants would be sufficient for each item on the scale for the statistical results to be reliable (Bryman & Cramer, 2005; Rattray & Jones, 2007). The NSCAIP-ED instrument has 49 items; therefore, the number of participants must be at least 245. This criterion was met in this study because 252 ED nursing staff members responded to the survey.

Instrument validation and data analysis

The validity and reliability of the NSCAIP-ED instrument were examined as comprehensively as possible and are reported in this article. The purpose of the validity examination was to find out whether the instrument measured what it was intended to measure (De Vet et al., 2011). Reliability means stability, where the instrument always reliably measures the same thing (DeVellis, 2017). The examination of content validity began with the observation of face validity; then, construct validity was clarified using confirmatory factor analysis (CFA). CFA is a factor analysis that is commonly used when the instrument structure is based on a theoretical framework. With CFA, it is possible to find out how well the data fit this framework (De Vet et al., 2011). The model fit indicators used were the chi-square test, the root mean square error of approximation (RMSEA < 0.08), comparative fit index (CFI > 0.95), Tucker-Lewis index (TLI > 0.90) (Xia & Yang, 2019) and the standardised root mean square residual (SRMR < 0.08). In this study, Cronbach alphas were considered when evaluating internal consistency. Acceptable alpha values are in the range of 0.70–0.90, and values <0.60 are not accepted (DeVellis, 2017).

The data that were collected to assess the NSCAIP-ED instrument's validity were analysed statistically. To describe the data, frequencies, percentages, medians and quartiles were given. CFA and Cronbach's alpha values were used to evaluate the validity and internal consistency of the instrument. The analyses were performed using SPSS version 28 for Windows (IBM, Armonk, NY, USA) and CFA was carried out using Mplus version 8.7 (Muthén & Muthén, Los Angeles, CA, USA).

Ethical considerations

Ethical guidelines and legislations were followed throughout the study (World Medical Association, 2018). The study was approved by the Ethics Committee for Human Sciences of the University of Turku, Finland (Decision 30/2015, 8 September 2015). Permission to conduct the survey was asked accordingly from the target organisations (Mäkelä et al., 2020). Answering the questionnaire was optional for the health professionals who participated in the study, and they were informed about the research by email before they decided to take part. Responding to the survey was considered as consent to participate in the study (Finnish National Board on Research Integrity TENK, 2019). The data were treated in confidence and by using reliable statistical methods. All the stages of instrument development and validation have been reported as comprehensively as possible.

Results

Descriptive statistics

The participants of the NSCAIP-ED survey (N = 1220, n = 252) were nursing professionals who worked in a Finnish ED. The total response rate for the study was 20.7%. Most of them (85.3%) were nurses with a mean working experience more than 10 years (49.2%). Around two-thirds (88.0%) of the participants did not have experience from intoxicant care and over half (59.0%) had not had substance abuse-related training during their career (Mäkelä et al., 2020).

Almost all participants (n = 252) responded to all items of the NSCAIP-ED instrument. The items with the most missing answers were “37. I know how to act in a violent situation” (n = 247) and “33. I know how to use security services” (n = 247), which also had the most “extremely good” answers (n = 159). Medians were in the range of 2.00–5.00. The sections “Ethical skills” and “Security and safety skills” both included three items that had the highest quartiles of the results (Q1–Q3 = 4.00–5.00). The lowest quartiles (Q1–Q3 = 1.00–2.00) and the most “extremely poor” responses were in the sections “Alcohol risk use evaluation skills” and “Follow-up treatment skills” (Table 1).

Evaluation of internal consistency was based on Cronbach's alpha values (Mäkelä et al., 2020). All scales of the NSCAIP-ED instrument had acceptable Cronbach's alpha values (range = 0.866–0.912) (Table 1).

Confirmatory factor analysis

The factor structure was examined using CFA considering 49 items of the instrument divided into seven factors. Model fit indicators showed that this factor model can be applied to the data of this survey with mostly acceptable fit (chi-square test p < 0.001; RMSEA 0.079; CFI 0.923; TLI 0.918; SRMR 0.084).

Discussion

The purpose of the NSCAIP-ED instrument is to measure nursing staff's self-evaluated skills concerning the care of acutely ill alcohol-intoxicated patients in EDs. In line with this purpose it was tested in the ED environment by the nursing staff working there. It has been reported earlier that ED nursing staff's skills to care for alcohol-intoxicated patients may be incomplete (Hakala et al., 2021; Iqbal et al., 2015; Warren et al., 2012). No similar instrument was found that could produce knowledge about the skills needed in caring for alcohol-intoxicated patients, thus pointing out the need for an instrument of this kind. Validation was necessary to establish if the NSCAIP-ED instrument could be commonly used. Based on the results of validation, this instrument measures well what it is meant to measure, and the results also showed good reliability.

Face validity evaluation was based on the opinions of the pilot group and the research group that were established for the instrument’s content evaluation as well as on the results of an earlier study in which the research group stated that NSCAIP-ED instrument is usable and reliable in measuring nursing staff's skills concerning the care mentioned above (Mäkelä et al., 2020). The aim of face validity is to determine the usability of the instrument and the functionality of the layout (De Vet et al., 2011; Udina, 2012). As an indication of the usability of the NSCAIP-ED instrument, earlier use, for example, pointed out that results of instruments’ scales were consistent with the background information of the respondents. Based on the background information, almost half of the participants had more than 10 years of work experience in emergency care, but 88% of the respondents had not worked in intoxicant care. The basic nursing skills, such as ethical skills, were at a better level than alcohol risk use evaluation skills or follow-up treatment skills (Mäkelä et al., 2020). This is relevant because even if there has been work experience as an emergency nurse, there is not necessarily competence on intoxicant care specific skills. These results support the usability of the instrument and show that the content is functional.

The quality of the NSCAIP-ED instrument was investigated by evaluating the implementation of the Delphi method, a literature review and interviews with professionals that were used in the development process of the instrument. Therefore, the opinions and views of various experts were taken into account as fully as possible when developing the instrument. After a pilot study, minor changes to the NSCAIP-ED were made to ensure that the structure of the instrument was feasible, relevant to the context and credible.

The response scale consisted of a Likert-type scale throughout the instrument, which facilitates responding and could reduce erroneous responses. This increased the value of the instrument’s validity. A Likert-type scale has been found to work well, but even when using a scale of this kind the respondent must be clearly informed of what their answer means at any point on the scale, and the same scale should be used throughout the instrument from start to finish (Krosnick & Presser, 2009). When using NSCAIP-ED, the respondent was left with no ambiguity as to what it meant to answer the statements because the answer options were the same from beginning to end (1 = extremely poor, 2 = reasonably poor, 3 = reasonably good, 4 = good and 5 = extremely good).

Responding to the NSCAIP-ED survey was executed electronically, which enabled respondents to reply at the time that was most convenient for them, and it was easy for them to return the answers. The good features of electronic data collection are that the data are often in an immediate format, data collection is inexpensive, participants are easy to reach and respondents prefer to answer questions related to sensitive topics online compared to face-to-face interviews (Boateng et al., 2018).

To verify content validity that is as inclusive as possible, two different scientific methods were used in the instrument development process. A literature review was conducted to build an evidence-based instrument and to define all concepts properly. Interviews with experts were used to ensure that the content and structure of the instrument was adequate and usable. Hence, the approach was both deductive and inductive; such a combination of both approaches is considered clearer and more comprehensive than development processes utilising just one method. Theoretical knowledge can be collected deductively, and practical and contextually relevant information can be collected inductively (Boateng et al., 2018).

Construct validity should be assessed to find out how consistent the results of the instrument are in relation to the phenomenon under study. The results obtained with the instrument should be an accurate description of the subject under investigation (De Vet et al., 2011). Construct validity was tested statistically with the CFA and it mostly showed acceptable fit. These results support the notion that the factor structure of the NSCAIP-ED instrument is the ability to detect ED nursing staff's skills to care for alcohol-intoxicated patients, but it should be tested further with larger sample sizes. The size of the sample participating in a study may affect the results of CFA. Despite this, CFA should be chosen if the instrument is based on theory (Prudon, 2015). The instrument in this study was produced based on scientific literature and professional knowledge. That is why CFA was an undisputed choice as a method to give reliable information about the instrument.

Internal consistency was examined by calculating Cronbach's alpha values. The core idea of Cronbach's alpha test is that instruments are reliable when all their sections are congruent and the variables correlate proportionately with each other (Boateng et al., 2018; DeVellis, 2017). The NSCAIP-ED instrument’s scales had acceptable Cronbach's alpha values, which indicates good internal consistency. The results of this study show that all items were answered evenly, supporting the notion that respondents did not get frustrated with the number of items when answering the questionnaire.

The reliability of the instrument can be improved and the incidence of measurement error reduced by limiting the variation due to the measurement situation; for example, by measuring a certain thing from all participants at the same time in the same way and by making sure that the correct use of the instrument is understood (De Vet et al., 2011). The originally planned sample of respondents was quite large because it was a national survey and covered all Finnish hospital districts. Thus, because all participants could not be contacted individually by the researchers, a cover letter with clear instructions for responding was attached to the survey. This means that all participants received the same instructions. Responding took place over a period of time predetermined by the researchers, so the timing of responses coincided fairly well. The aim was to avoid scenarios such as a change in participants’ responses due to various significant events, seasons or longer-term training, and thus reduce the incidence of measurement error.

The instrument and the survey results were only handled by members of the research group who had the same knowledge and skills to use it. The interpretability of the results means that the quantitative results obtained by the instrument can be converted into a qualitative form that can be generally understood (De Vet et al., 2011; Streiner & Kottner, 2014). Based on the NSCAIP-ED survey that was conducted, all the results could be analysed and discussed in an article (Mäkelä et al., 2020).

Alcohol is a costly societal problem. It is crucial to be able to measure the skills to care for alcohol-intoxicated patients so that ED managers know when the staff needs more training and education (Mäkelä et al., 2022). These skills include different kinds of competencies that are included in the structure of the NSCAIP-ED instrument. ED nursing staff's positive attitude, knowledge and proper use of the skills mentioned above provide better care for patients who are alcohol abusers or at risk of problematic alcohol use (Clarke et al., 2015; Hakala et al., 2021). Patient care where these skills are implemented can be very effective and reduce patients’ alcohol use or even get them to give up alcohol altogether (Koivunen et al., 2017).

Limitations

There are some limitations in our study because the NSCAIP-ED survey was conducted in one country and the number of participants was limited (N = 1220, n = 252). With a larger sample of participants, the data could have been split and different kinds of statistical analyses could have been conducted to verify the construct validity of this instrument. After all, we were able to execute relevant statistical testing and analyses gave mostly reliable results. The NSCAIP-ED survey that was conducted was in Finnish, which limited the use of the instrument at the time; however, translation of the instrument into English for further use was started in this article (Table 1), and the instrument should be tested internationally.

A gold standard that could have been compared to the instrument was not found, so criterion validity could not be clarified. In the future, with test–retest study design, reliability and responsiveness could be tested more comprehensively. The missing gold standard also affected the evaluation of the instrument’s content validity because there was no other instrument to parallel it. However, members of the pilot group and the research group that were established for the instrument’s content evaluation and earlier literature supported the assumption that the NSCAIP-ED instrument’s content validity was good (Mäkelä et al., 2020; Hakala et al., 2021; Mäkelä et al., 2022).

The low participation rate might influence the validation and, in the future, the NSCAIP-ED instrument must be further tested with larger data. This may be due to the length of the survey but there was no mention in the pre-testing phase that the survey would be too long. There is a difference between the measurements of skills and the measurement of self-evaluated skills. The instrument is now based on participants’ self-evaluation but there is a possibility of developing a knowledge test to measure nurses’ skills to care for alcohol-intoxicated patients in the ED. Comparing the results of the knowledge test with the self-evaluation would be interesting and complements the information about the NSCAIP-ED instrument’s validity. This will be taken into account when planning further studies.

Conclusion

The NSCAIP-ED is a feasible and reliable instrument that can be used when measuring nurses’ skills to care for alcohol-intoxicated patients in the ED context. This instrument could be useful for nursing managers in EDs for evaluating their nursing staff's self-evaluated skills in the care area mentioned and for designing continuing education based on the results. It is important to have interventions that can improve skills of this kind. The NSCAIP-ED could also be tested in other nursing environments or with nursing students when there is a need to find out their self-evaluated skills to care for patients influenced by alcohol. This way, necessary training could be offered at the right time and it could raise the standard of care in the long run. Nursing staff's relevant caring skills will offer comprehensive and good quality treatment for patients.

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study is funded by the Satakunta Hospital District (Government Research Financing 81902, Decision 57/2016, 82003 Decision 135/2017, 84012 Decision 158/2019, 850 Decision 412/2020) and the Hospital District of Southwest Finland (Government Research Financing 81902, Decisions 7/2015 and 15/2016).

ORCID iD: Kaisu-Leena Mäkelä https://orcid.org/0000-0002-0913-2339

Correction (November 2023): Table 1 has been corrected in the article.

Contributor Information

Kaisu-Leena Mäkelä, Faculty of Social Sciences, Unit of Health Sciences, Nursing Science, Tampere University, Tampere, Finland; Emergency Department, The Wellbeing Services County of Satakunta, Pori, Finland.

Jari Kylmä, Faculty of Social Sciences, Unit of Health Sciences, Nursing Science, Tampere University, Tampere, Finland.

Anna-Maija Koivisto, Faculty of Social Sciences, Unit of Health Sciences, Tampere University, Tampere, Finland.

Tiina Hakala, The Wellbeing Services County of Satakunta, Pori, Finland.

Marita Koivunen, The Wellbeing Services County of Satakunta, Pori, Finland; Department of Nursing Science, University of Turku, Turku, Finland.

References

  1. Araújo O., Lage I., Cabrita J., Teixeira L. (2016). Development and psychometric properties of ECPICID-AVC to measure informal caregivers’ skills when caring for older stroke survivors at home. Scandinavian Journal of Caring Sciences, 30(4), 821–829. 10.1111/scs.12291 [DOI] [PubMed] [Google Scholar]
  2. Bakke E., Bogstrand S. T., Normann P. T., Ekeberg Ø, Bachs L. (2016). Influence of alcohol and other substances of abuse at the time of injury among patients in a Norwegian emergency department. BMC Emergency Medicine, 16(1), 1–9. 10.1186/s12873-016-0085-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bing-Jonsson P., Bjørk I., Hofoss D., Kirkevold M., Foss C. (2015). Competence in advanced older people nursing: Development of ‘nursing older people – competence evaluation tool’. International Journal of Older People Nursing, 10(1), 59–72. 10.1111/opn.12057 [DOI] [PubMed] [Google Scholar]
  4. Blow F. C., Walton M. A., Barry K. L., Murray R. L., Cunningham R. M., Massey L. S., Chermack S. T., Booth B. M. (2010). Alcohol and drug use among patients presenting to an inner-city emergency department: A latent class analysis. Addictive Behaviors, 36(8), 793–800. 10.1016/j.addbeh.2010.12.028 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Boateng G. O., Neilands T. B., Frongillo E. A., Melgar-Quiñonez H. R., Young S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, 149. 10.3389/fpubh.2018.00149 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bryman A., Cramer D. (2005). Quantitative data analysis with SPSS 12 and 13. A guide for social scientists (pp. 3–126). Routledge. [Google Scholar]
  7. Clarke D. E., Gonzalez M., Pereira A., Boyce-Gaudreau K., Waldman C., Demczuk L. (2015). The impact of knowledge on attitudes of emergency department staff towards patients with substance related presentations: A quantitative systematic review protocol. JBI Database of Systematic Reviews and Implementation Reports, 13(10), 133–145. 10.11124/jbisrir-2015-2203 [DOI] [PubMed] [Google Scholar]
  8. DeVellis R. (2017). Scale development theory and applications (4th ed., pp. 39–44). Sage Publications. [Google Scholar]
  9. de Vet H. C. W., Terwee C. B., Mokkink L. B., Knol D. L. (2011). Measurement in medicine. Practical guides to biostatistics and epidemiology (pp. 70–85, 150–196). Cambridge University Press. [Google Scholar]
  10. Finnish National Board on Research Integrity TENK. (2019). (28.11.2022). The ethical principles of research with human participants and ethical review in the human sciences in Finland. https://tenk.fi/sites/tenk.fi/files/Ihmistieteiden_eettisen_ennakkoarvioinnin_ohje_2019.pdf.
  11. Hakala T., Koivunen M., Åstedt-Kurki P. (2017). Henkilökunnan asennoituminen ja osaaminen päihtyneen potilaan hoitotyössä päivystyksessä - integroitu kirjallisuuskatsaus. [Staff's attitudes and knowledge in the care of intoxicated patients in the emergency department - Integrative literature review]. Tutkiva Hoitotyö [Nursing Evidence], 15(2), 30–40. [Google Scholar]
  12. Hakala, T., Kylmä, J., Mäkelä, K-L., Löyttyniemi, E., & Koivunen, M. (2020). Alkoholista päihtynyt potilas päivystyksessä – hoitohenkilökunnan näkemyksiä hoitoon liittyvän osaamisen tärkeydestä. [Alcohol intoxicated patients in the emergency department – nursing staff’s views on the importance of care-related skills] Yhteiskuntapolitiikka [Social politics], 85(3), 271–282.
  13. Hakala, T., Kylmä, J., Mäkelä, K-L., Noppari, E., & Koivunen M. (2021). Caring for alcohol-intoxicated patients in an emergency department from the nurses’ point of view – focus on attitudes and skills. Scandinavian Journal of Caring Sciences, 35(1), 115–22. doi: 10.1111/scs.12825 [DOI] [PubMed]
  14. Hellum R., Bjerregaard L., Nielsen N. S. (2016). Factors influencing whether nurses talk to somatic patients about their alcohol consumption. Nordic Studies on Alcohol and Drugs, 33(4), 319–448. 10.1515/nsad-2016-0034 [DOI] [Google Scholar]
  15. Iqbal N., McCambridge O., Edgar L., Young C., Shorter G. W. (2015). Health-care professionals’ attitudes across different hospital departments regarding alcohol-related presentations. Drug and Alcohol Review, 34(5), 487–494. 10.1111/dar.12243 [DOI] [PubMed] [Google Scholar]
  16. Koivunen M., Harju S., Kauko T., Välimäki M. (2017). Alcohol risk drinking, quality of life and health state among patients treated at the sobering unit in the emergency department: One year follow-up study. International Emergency Nursing, 31, 22–29. 10.1016/j.ienj.2016.06.003 [DOI] [PubMed] [Google Scholar]
  17. Krosnick J. A., Presser S. (2009). Question and questionnaire design. Handbook of Survey Research (2nd Edition, pp. 263–314). Elsevier. [Google Scholar]
  18. Lin C.-J., Hsu C.-H., Li T.-C., Mathers N., Huang Y.-C. (2009). Measuring professional competency of public health nurses: Development of a scale and psychometric evaluation. Journal of Clinical Nursing, 9(21–22), 3161–3170. 10.1111/j.1365-2702.2009.03149.x [DOI] [PubMed] [Google Scholar]
  19. Mäkelä, K-L., Kylmä, J., Hakala, T., Löyttyniemi, E., Puolakka, K., & Koivunen, M. (2020). Nursing staff’s self-assessed skills concerning the care of acutely ill alcohol-intoxicated patients in emergency departments. Nordic Journal of Nursing Research, 40, 25–32. doi: 10.1177/2057158519864856.
  20. Mäkelä, K-L., Kylmä, J., Noppari, E., Hakala, T., & Koivunen, M. (2022). The effectiveness of a web-based education program for nurses’ self-assessed skills concerning the care of alcohol-intoxicated patients in an emergency department. Finnish Journal of eHealth and eWelfare, 14(2), 243-255. https://doi.org/10.23996/fjhw.113450. [DOI] [PubMed]
  21. Meretoja R., Isoaho H., Leino-Kilpi H. (2004). Nurse competence scale: Development and psychometric testing. Journal of Advanced Nursing, 47(2), 124–133. 10.1111/j.1365-2648.2004.03071.x [DOI] [PubMed] [Google Scholar]
  22. Prudon P. (2015). Confirmatory factor analysis as a tool in research using questionnaires: A critique. Comprehensive Psychology (Missoula, MT), 4, 1–19. 10.2466/03.CP.4.10 [DOI] [Google Scholar]
  23. Rattray J., Jones M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. 10.1111/j.1365-2702.2006.01573.x [DOI] [PubMed] [Google Scholar]
  24. Russell R., Ojeda M. M., Ames B. (2017). Increasing RN perceived competency with substance use disorder patients. Journal of Continuing Education in Nursing, 48(4), 175–183. 10.3928/00220124-20170321-08 [DOI] [PubMed] [Google Scholar]
  25. Streiner D. L., Kottner J. (2014). Recommendations for reporting the results of studies of instrument and scale development and testing. Journal of Advanced Nursing, 70(9), 1970–1979. 10.1111/jan.12402 [DOI] [PubMed] [Google Scholar]
  26. The World Health Organization (WHO) (2022). (28.11.2022). Global strategy to reduce the harmful use of alcohol. https://www.who.int/teams/mental-health-and-substance-use/alcohol-drugs-and-addictive-behaviours/alcohol/governance/global-alcohol-strategy.
  27. Udina M. E. J. (2012). A nursing interface terminology: Evaluation of face validity. Open Journal of Nursing, 2(3), 196–203. 10.4236/ojn.2012.23030 [DOI] [Google Scholar]
  28. Warren O. U., Sena V., Choo E., Machan J. (2012). Emergency physicians’ and nurses’ attitudes towards alcohol intoxicated patients. Journal of Emergency Medicine, 43(6), 1167–1174. 10.1016/j.jemermed.2012.02.018 [DOI] [PubMed] [Google Scholar]
  29. World Medical Association (2018). World Medical Association Declaration of Helsinki. (28.11.2022). Ethical Principles for Medical Research Involving Human Subjects. https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/.
  30. Xia Y., Yang Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. 10.3758/s13428-018-1055-2 [DOI] [PubMed] [Google Scholar]

Articles from Nordisk Alkohol- & Narkotikatidskrift : NAT are provided here courtesy of Nordic Welfare Centre

RESOURCES