Validation of the 30-item nurses’ observation scale for inpatient evaluation and mental health-care promotion

Abstract

CONTEXT:

The 30-item nurses' observation scale for inpatient evaluation (NOSIE-30) developed for the behavioral and observational rating of psychiatric inpatients.

AIMS:

The purpose of this study was to evaluate the validation of the NOSIE-30 and mental health-care promotion.

MATERIALS AND METHODS:

The participants with psychiatric disorders were selected from 310 inpatients referred to the military educational hospital in Iran. This study was carried out in two phases: First, translation of the NOSIE-30 into Persian followed the stepwise, iterative procedures developed by the International Quality of Life Assessment project approach. Second, face validity, criterion-related validity and construct validity, and reliability of the Persian version were determined. The concurrent validity was assessed by using the global assessment of functioning (GAF) instrument. Data were analyzed with the SPSS software of version 22. Exploratory factor analysis, the Cronbach's alpha coefficient, and interrater agreement with intraclass correlation coefficient and Pearson's correlation coefficient were used to data analysis.

RESULTS:

Pearson's correlation coefficient between the NOSIE and GAF was 0.75. The interrater reliability for subscale scores and structure of the NOSIE were ranging from 0.70 to 0.94 and Cohen's kappa = 0.74. Furthermore, the internal consistency of the scale's total scores was estimated by Cronbach's alpha = 0.85.

CONCLUSIONS:

The finding indicated that NOSIE-30 Persian version scale has the desirable validity and reliability for evaluating the nursing care of inpatients. Thus, nurses can promote mental health care by applying this scale in the psychiatric setting.

Introduction

Mental disorder is the most common health problem worldwide. In according to the global estimates of disease burden, a mental disorder is located on the top of three reasons of life years lost related to disability.[1] The prevalence of mental disorders varies in different countries due to cultural, social, and geographical differences.[2] For example, in the United States is estimated to an average of 25%.[3] Furthermore, in Iranian adult society is reported 23.4% in 2015.[4] The process of psychiatric disorders imposes severe damage to the patient's personal, social, and family life. Although many patients are kept in the health center, unfortunately, some of them left wandering in the community due to the lack of organized care.[5] Uncoordinated, nonsystemic treatment of health-care team, pretreatment discharge, and the other hand, unnecessary rehospitalization cause so many costs and consequences for the patient, family, health system, and ultimately for the society.[6]

Nursing care is one of the essential parts of the health-care system, and it is very important; also the health and improvement of patients depends on the provision of quality care.[7]

Nurses play a major role in following therapeutic goals and assessing behavioral changes in hospitalized patients.[8] Nursing observations are well integrated with the nursing profession because of the needed information is directly available during patient care.[9] Indeed, the process of observation of patients is carried out through the day and night on the basis of informal interactions between the nurse and patients at different times of bathing, eating, transporting of the patient or speaking time, or relationship the patients to each other. These observations are not threatening, annoying, or stressful to the patient. It can be implemented even if the patient is a severe illness or underdoing neuropsychological investigation and experiments.[10] Since the results of behavioral observations are completely related to the daily life of patients, there is no necessity for any special laboratory tests and it has a high biological validity.[11]

Given that, the psychiatric ward is the most eventual care environment,[12] psychiatric nurses among the other nurses has been reported the highest rate of violence victimization.[13] Hesketh et al., in Canada reported that 55% of psychiatric nurses are offended by verbal abuse and 20.3% injured of physical abuse.[13,14] Furthermore, Gooshi et al. during the period of study in Iran reported of 162 psychiatric patients showed that aggressive behaviors and 319 aggression events were witnessed and registered by the nurses. Some of the patients's committed aggressive behaviors more than once (from one to eight times).[15] High-risk behaviors of patients have just no physical or psychological consequences, but it can also lead to financial casualties. Therefore, preventive plans and control of these behaviors of inpatients have a high priority.[16]

In spite of fulfilling of observation and investigation of patient behavior by nurses in most of the daily care plan, rather it is done in the lack of instrument or scientific foundation which can accomplish a complementary role of these observations.[11] Many instruments are used to measure and assess the health-care staff's attitudes and judgments about the behavior of psychiatric patients that are not complete instruments. That's why those do not have any titles that specifically represent the role of psychiatric nurses in this domain.[17] In fact, it is not used the particular instruments with good reliability during the recording of behavioral observations in psychiatric hospitals, which is further rooted in the thought that “the use of these instruments is complex and time-consuming.”[18] Today, standardizing is the problem in the process of observing the patients' behavior and the number of observational instruments that have a desirable validity is still relatively rare.[19]

The 30-item nurses' observation scale for inpatient evaluation (NOSIE-30) is one of the instruments made to activate psychiatric nurses in the process of evaluating the inpatients who suffer from a mental disorder. This scale is a 30-item instrument that is highly sensitive to the behavioral changes of patients admitted to psychiatric departments, which was conducted to assess these patients.[20,21] This scale and similar instruments express the essential role of nurses in care planning. The psychiatric nurses have the highest degree of competency to provide an opinion about the presence or absence of symptoms indicating mental disorders during the hospitalization because they are in full-time contact with inpatients.[22] So far, many studies have been carried out on the feasibility of this scale in various countries, including the Netherlands,[23] the United States,[24] Britain,[16] Singapore,[25] and Italy,[26] which has gained generally acceptable validity. This instrument was used in the variety of psychiatric disorder for evaluate of behaviors in schizopherenia inpatient,[27,28,29,30,31,32] bipolar disorder,[33] and suicide.[34] Furthermore, the NOSIE-30 has been applicable for the measurment of addaptive behaviour,[35] prediction of aggressive behavior,[36] and evaluate of behavioral and functional capacity[37] in psychiaric disorders.

The making native of this scale is due to the lack of standard instruments for evaluating the behavioral changes of patients admitted to the psychiatric ward in Iran. The advantage of the NOSIE-30 is being exclusive and particular to observing nursing staff in the psychiatric setting. Indeed, this scale performs a comprehensive and complete assessment of the change in the patient's behavioral patterns of hospitalization to discharge, so that can lead to accurate and organized recording of patients' behavioral changes. It is also easy and quick to use. In addition, this scale is effective in patients who have not even been able to communicate verbally related to their deterioration.[22]

Considering that, the lack of a reliable and suitable instrument for assessing the NOSIE in Iranian psychiatric nurses, the validity and reliability of the NOSIE instrument was measured for the Persian version. It has been established as a valid and reliable scale to help nurse for providing qualified nursing care. Therefore, the aim of the present study is to create the first Persian translation of the NOSIE-30 and carry out psychometric testing of the validity and reliability of the Persian version for promotion of mental health care.

Materials and Methods

Study design

This methodological and cross-sectional descriptive study was evaluated in the psychometric properties of the NOSIE-30.

Participants and settings

A convenience sample of 310 psychiatric inpatients in the educational hospital in Tehran, in 2016. sample size was calculated 10 times greater than the number of the scale 30 items. Furthermore, in order to control of attrition, we selected 10 more than calculated samples in our study. The inclusion criteria were having a definitive diagnosis of a known mental illness and a good deal of mental retardation. This questionnaire was just fulfilled by of psychiatric nurse in the psychiatric setting.

Instruments

The first part of the questionnaire collected demographic data that comprises questions regarding gender, age, marital status, education, occupation, and history of mental illness.

The NOSIE comprises 30 items which seven subscales (social competence, social interest, personal neatness, irritability, manifest psychosis, retardation, and depression) based on the observation of the behavior of patients during the hospitalized in a psychiatric ward. The response option is scored on a 5-point Likert Scale (0–5), which has a range of “never: Zero” to “always: 5.” The high score of behavior indicates its more frequent and lower score, indicating less repetition or nonoccurrence of a particular behavior. The scale is a highly sensitive psychiatric ward behavior rating scale.

Forward and back translations

After obtaining the required permission from the Ethics Committee, a standard translation of the NOSIE-30 into Persian was performed by two native translators according to the WHO guidelines. The second step was to combine and integrate the initial translations into one unified entity. During this step, the first translated versions were carefully revised by another translator who was an expert in both the Persian and English languages. Then, the initial translated versions were compared with one another and the existing differences and contradictions were corrected. Ultimately, the final version of the scale in Persian was obtained by integrating the initial ones. The third step was to translate the final version from Persian to English, and then, as the fourth step, the English version was submitted to the scale designer for confirmation. Finally, the Persian version of the NOSIE was revised grammatically by expert translators and presented for the evaluation of psychometric properties [Figure 1].

Figure 1.

Forward and back translations of the nurses' observation scale for inpatient evaluation

Face and content validity of the 30-item nurses' observation scale for inpatient evaluation

In this step, face validity was determined qualitatively, and the Persian version of NOSIE-30 was offered to 5 psychiatrists and 15 psychiatric nurses. Furthermore, the content validity of this scale in a qualitative manner was specified by the expert specialists. The items of the scale that were difficult to understand were revised by the translators. The information obtained from the two groups was then analyzed, evaluated, and finally implemented in the translated version.

Criterion-related validity of the 30-item nurses' observation scale for inpatient evaluation

The concurrent validity of the NOSIE-30 was evaluated by determining the correlation between the scores of the NOSIE-30 and global assessment of functioning (GAF) on the admission and discharge time from the hospital with the Pearson test.

Data collection

Subjects were adult psychiatric patients admitted to the Educational and Therapeutic hospital. Considered criteria were age (>18), gender (both male and female), and past history of mental illness. Data were collected during the period of 2016–2017. To assess the concurrent validity, convenience sampling was conducted for 310 people by using the GAF instrument. The GAF scale is a scoring rule for the intensity of disorder in psychiatry. Interrater reliability was performed to assess the reliability of this scale. The NOSIE-30 was applied by six psychiatric nurses aged on average 37.7 ± 6.3 years. All raters had at least 6-year work experience in the psychiatric ward and voluntarily agreed to participate in the study. The psychiatric nurses were trained in the NOSIE-30 by a researcher with experience in the scale application. The training was completed within approximately 20 h (usually distributed over 4 shifts) and calculated the aspects such as reading and discussing the registered protocol and the scale manual with the trainer and joint observation of 1, 2, and 3 patients in three successive shifts by the raters and independent filling in of the scale.

Data analysis

After the data were collected, the descriptive analysis, validity, and reliability were analyzed with the SPSS software version x7(2001-2010) distributed by Thomson Reuters under license, U.S.patent. In the construct validity study, exploratory factor analysis was used with the main components. The sample size was 310 inpatients for performing factor analysis (10 times greater than the number of the scale items). Initially, the Kaiser-Meyer-Olkin sampling test was used and the Bartlett test was employed to assess the justification of the factor analysis. The number of tool constructors was determined using a scree plot and special value. Factor and varimax rotation were used to simplify and interpret the factor structures of the tool, with factor loadings >0.4 employed. Participants were military nurses. The internal consistency of the NOSIE was used to calculate the Cronbach's alpha coefficient, and the reliability was determined through evaluating interrater agreement with intra-class correlation coefficient (ICC). In this study, Pearson's correlation coefficient was used to determine for the concurrent validity of the relationship between the NOSIE and GAF scales.

Ethical considerations

The study was carried out under tight supervision and reviewing bodies in the institution and all clinical and ethical standards set forth in the Helsinki Declaration of 1975. Furthermore, psychiatric nurses were informed of the purpose and method of study. The maintenance of participant's rights and the principle of confidentiality were respected in the dissemination of information.

Results

The mean ± standard deviation age of the 310 patients was (33.7 ± 9.8 years). A total of 160 patients (51.6%) were men, and 150 (48.4%) were women. The majority of inpatients had the education of primary education. The diagnoses were schizophrenia (119 patients or 38.4%), bipolar disorder (66 patients or 21.3%), Major depression (102 patients or 32.9%), schizoaffective disorder (80 patients or 25.8%), and posttraumatic stress disorder (23 patients or 7.4%) [Table 1].

Table 1.

Distribution of demographic variables among the participants (n=310)

Demographic variables	Frequency (%)
Age (years)
18-20	17 (5.5)
21-30	115 (37.1)
31-40	97 (31.3)
41-50	69 (22.3)
51-60	12 (3.8)
Gender
Male	160 (51.6)
Female	150 (48.4)
Marital status
Unmarried	164 (52.9)
Married	122 (39.4)
Divorced	24 (7.7)
Education
Primary	177 (57.1)
Secondary	118 (38.1)
Tertiary	15 (4.8)
Mental illness diagnosis
Schizophrenia	119 (38.4)
Bipolar disorder	66 (21.3)
Depressive disorder	102 (32.9)
Schizoaffective disorder	80 (25.8)
Posttraumatic stress disorder	23 (7.4)

Face validity of this scale was determined suitably because total scores of the translation quality, based on the clarity and transparency, common language, conceptual equivalence was evaluated and nurses faced no difficulty with the scale items. The content validity was qualitatively assessed by several specialists and found to be at an appropriate level. The content validity analysis revealed that the item-level content validity index (CVI) was 0.90, and the scale level CVI (S-CVI/average) was 0.92. The ICC value for the subscales of the NOSIE-30 social competence, social interest, personal neatness, irritability, manifest psychosis, depression and retardation, it was 0.75, 0.76, 0.74, 0.78, 0.77, 0.74, and 0.73, respectively, and Pearson's correlation coefficient between the NOSIE and GAF was 0.75 [Table 2].

Table 2.

Pearson’s r between global assessment of functioning and 30-item nurses’ observation scale for inpatient evaluation subscales

NOSIE subscales	r	P
NOSIE-30 domains
Social competence	0.75	0.0001
Social interest	0.76	0.0001
Personal neatness	0.74	0.0001
Irritability	0.78	0.0001
Manifest psychosis	0.77	0.0001
Depression	0.74	0.0001
Retardation	0.73	0.0001
GAF scale	1
Overall NOSIE-30 Score	0.75

GAF=Global assessment of functioning, NOSIE=Nurses’ Observation Scale for Inpatient Evaluation

The inter-rater reliability for subscale scores and structure of the NOSIE were ranging from 0.70 to 0.94 and cohen's kappa = 0.74 [Table 3]. Furthermore, the internal consistency of the scale's total scores was estimated by Cronbach's alpha (α) = 0.85 [Table 4].

Table 3.

Interrater reliability for subscale scores and structure of the nurses’ observation scale for inpatient evaluation

NOSIE subscales	Items of the NOSIE	r*	κ	P
Positive factors
Social competence	11. Refuses to do the ordinary things expected of him	0.72	0.75	<0.0001
	13. Has trouble remembering
	21. Has to be reminded what to do
	24. Has to be told to follow hospital routine
	25. Has difficulty completing simple tasks on his own
Social interest	4. Shows interest in activities around him	0.80	0.74	<0.0001
	9. Tries to be friendly with others
	15. Laughs or smiles at funny comments or events
	17. Starts a conversation with others
	19. Talks about his interests
Personal neatness	1. Is sloppy	0.82	0.76	<0.0001
	8. Keeps his clothes neat
	16. Is messy in his eating habits
	30. Keeps himself clean
Negative factors
Irritability	2. Is impatient	0.78	0.73	<0.0001
	6. Gets angry or annoyed easily
	10. Becomes upset easily if something does not suit him
	12. Is irritable or grouchy
	29. Is quick to fly off the handle
Manifest psychosis	7. Hears things that are not there	0.94	0.78	<0.0001
	20. Sees things that are not there
	26. Talks, mutters, or mumbles to himself
	28. Giggles or smiles to himself for no apparent reason
Retardation	5. Sits, unless directed into activity	0.94	0.72	<0.0001
	22. Sleeps, unless directed into activity
	27. Is slow-moving or sluggish
Depression	3. Cries	0.84	0.71	<0.0001
	14. Refuses to speak
	18. Says he feels blue or depressed
	23. Says that he is not good
Overall NOSIE-30		0.80

*ICC. ICC=Intra-class correlation coefficient, NOSIE=Nurses’ Observation Scale for Inpatient Evaluation

Table 4.

Intercorrelations and Cronbach’s alpha for 30-item nurses’ observation scale for inpatient evaluation subscales

NOSIE subscales	Social competence	Social interest	Personal neatness	Irritability	Manifest psychosis	Retardation	Depression
Social competence	1.00	0.49	0.78	−0.57	−0.57	−0.68	−0.24
Social interest		1.00	0.42	−0.22	−0.25	−0.56	−0.09
Personal neatness			1.00	−0.50	−0.58	−0.58	−0.14
Irritability				1.00	0.37	0.22	0.21
Manifest psychosis					1.00	0.34	0.09
Retardation						1.00	0.30
Depression							1.00
Cronbach’s alpha (α)	0.78	0.88	0.90	0.92	0.90	0.81	0.73

Cronbach‘s alpha overall scale=0.85

Discussion

The results of this study deal with the psychometric characteristic of a NOISE-30 in Iran. The demographic distribution of samples shows that the majority of them were educated that it can be due to being younger of inpatients. Furthermore, most of the evaluated inpatients had a history of hospitalization that of course needs to be investigated. According to the result, it shows the evaluation of experts about the quality and difficulty of the Persian-translated version of NOSIE-30. According to, it can be noted that this scale has good quality and psychiatric nurses can use easily to evaluation inpatients in psychiatric wards. In accordance with this study, many studies in various countries have confirmed the feasibility of the NOSIE-30,[16,23,24,25,26,38] and frequently, have been used in psychiaric and psychology studies.[34,39,40,41] Considering that, the reason for less using some instruments in clinical a setting is being time-consuming and complicated,[18] the strength of the Persian translated version of NOSIE-30 is ability to fullfill so easily and quickly.

Also is represented content validity of the Persian translated version of NOSIE-30 based on CVI. Given that the scale's total scores were over 0.80, it can be said that the Persian version NOSIE-30 has very good content validity. In accordance with our result, Cook et al. in America,[17] Thirthalli et al. in India,[42] Lyall et al. in British,[16] and Hafkenscheid in Netherland,[23] in their studies reported the similar findings. Therefore, in our research, the goals of evaluation, the face and content validity are to have occurred.

In this study, the data indicate the high criteria validity of the Persian version NOSIE-30 with the GAF scale in all of the columns, except 5 that is the same as other studies is done in different countries.[43] Hence, concurrent validity of this scale is to be confirmed.

The reliability of the Persian version of the NOSIE-30 was obtained well. Internal correlation coefficients and Cronbach's alpha coefficient were estimated desirably. In agreement with our work, the studies in America, Italy, British, and the Netherlands reported similar results.[17,23,44]

Since that, observational instrument of patients' behavior by nurses in the psychiatric ward has not been validated in our country so far, the finding of this study can be made available to the care team to promote mental health care. Good validity and reliability of Persian version of the NOSIE-30 are the strength of this study so that psychiatric nurses by using this scale in the clinical setting can be able to provide a comprehensive and complete assessment of the patient's behavior patterns from hospitalization to discharge. Furthermore, based on our result, Persian version of NOSIE-30 is so applicable and effective in patients who have not even been able to communicate verbally related to their deterioration.

One of the limitations of this research was the lack of willingness and motivation of some nurses to complete the NOSIE-30 scale, which sought to cooperate with the nurses' explanations and justifications. Furthermore, because of the patient's scores based on their 72-h behavior, thus patients who entered in the monitoring phase, if they left before 72 h, should be excluded from the list of patients under their control, which was a factor in being time consuming of the study process.

It is suggested the validation of other instruments for monitoring behavioral changes in psychiatric patients as well as other patients and comparing them with the NOISE-30 instrument can be done that to be helpful in monitoring the more accurate and intrinsic behaviors.

Conclusions

This scale for inpatient evaluation is a reliable, feasible to use an instrument for measuring patient's progress in the hospital. This study showed that the Persian version NOSIE-30 retains satisfactory interrater reliability with current clinical populations, and we recommend it for everyday clinical practice and a basis for meaningful communication between staff about patient status. Thus, nurses have access to this scale to promote mental health care in a psychiatric setting.

Financial support and sponsorship

This study was financially supported by the Baqiyatallah University of Medical Sciences.

Conflicts of interest

There are no conflicts of interest.