Abstract
The COMFORT Behavior Scale (CBS) is an instrument that has been validated in several languages to assess the levels of sedation in children. This project was implemented to identify the validity and reliability of CBS. The design used was an analytic descriptive cross-sectional approach. Fifty-one children aged 1 month to 18 years who received analgesic and/or sedation therapy were purposively selected and assessed for their sedation levels using CBS and the Nurse Interpretation of Sedation Score (NISS). The data were analyzed using the Mann-Whitney U test to measure the concurrent validity; each item was analyzed using Pearson correlation; inter-rater reliability was measured by the Kappa coefficient; and the internal consistency was measured by Cronbach’s alpha. The results showed there was no significant correlation between the levels of sedation assessed using CBS and NISS (P= 0.118; α= 0.05). Six items in the CBS instrument were found to be valid (r= 0.348–0.813). The Cronbach’s alpha for CBS was 0.873. Thus, the Indonesian version of CBS is valid and reliable in assessing sedation levels in children.
Key words: COMFORT Behavior Scale, level of sedation, reliability, validity
Introduction
Pediatric intensive care unit (PICU) is a ward for pediatric patients who require invasive procedures and intensive monitoring. Conditions such as pain, inconvenience, disturbed sleep, anxiety, and fear can continuously affect the patients physically and psychologically. 1,2 Health workers, including the nurses working in PICUs, are responsible for minimizing the physical and psychological effects of pain on patients through sedation and analgesic administration.1
Analgesics and sedation are administered to optimize treatments, provide convenience for patients, control pain, and reduce anxiety and agitation in patients undergoing treatment or painful procedures. The administration of sedation and analgesics for critical patients can be very helpful, but their overuse can increase the need for mechanical ventilation and the length of stay.1,2 Inadequate sedation can lead to agitation and cause unplanned extubation.2,3
A systematic review of pediatric patients found that only 57.6% of them were optimally sedated, while 31.8% were oversedated, and 10.6% were undersedated.3 This data shows that oversedation is more frequent than undersedation.
The sedation level is the level of sedation, agitation, and sedation depth in critical patients.4 Effective analgesic and sedation management have been measured using objective sedation evaluations and validated instruments.2 Results have shown that only 42% of PICUs regularly perform an objective sedation assessment.5 The COMFORT Behavior Scale (CBS) is an instrument that has been validated in Portuguese, Swedish, Chinese, Norwegian, and Italian to assess the level of sedation and analyze distress in children who require mechanical ventilation or spontaneous breathing.6-10 Referring to the aforementioned condition, an evidence-based practice project is needed regarding the CBS instrument to monitor the level of sedation in PICUs in order to improve the effectiveness of sedation and analgesic management in pediatric patients.
This evidence-based practice project was implemented to identify the validity and reliability of the CBS instrument in assessing the level of sedation in children. This project was started through explanation the project plan to all nurses and physicians in the PICU then started to collect the data by assessing the level of sedation. The data of sedation level were analyzed to measure the validity and reliability of the instrument. The results of the evidence-based practice were presented in a seminar in order to suggest its implementation as a standard operational procedure for monitoring sedation levels in pediatric patients in the PICU.
Materials and Methods
The design used was an analytic descriptive cross-sectional approach. The population consisted of children ages 0–18 years being treated in the PICU at Dr. Cipto Mangunkusumo Hospital from January to April 2019, who received analgesic and/or sedation therapy. The purposive sampling technique was applied and resulted in 51 respondents. The intervention was the use of the CBS instrument to objectively assess patients’ levels of sedation. Back translation and inter-rater reliability were done before the data was collected. The comparison was the Nurse Interpretation of Sedation Score (NISS) instrument because NISS has been used before as silver standard.
Before assessing the level of sedation, the respondents’ characteristics were identified based on the date of data collection, date of PICU admission, personal identity (age and sex), PICU priority, the use of mechanical ventilation, and the number of analgesic or sedative medications. After initial data collection, the patients were measured for their level of sedation using the CBS instrument for 2± minutes. The CBS instrument is comprised of alertness, calmness/ agitation, respiratory response or crying, physical movement, muscle tone, and facial tension. The assessment was conducted once for each respondent. Subsequently, the levels of sedation were measured based on the nurse’s perception using NISS. The implementation of this evidence-based practice received ethical approval from the ethical research committee of Nursing Faculty, Universitas Indonesia, no. 09/UN2.F12.D/HKP.02.04/2019.
The expected outcomes were the validity and reliability of the CBS instrument in assessing patients’ levels of sedation. Statistical analysis was applied to measure the validity and reliability of the instrument. Concurrent validity was measured using the Mann-Whitney U test to check the correlation or trend between variables, and each item was assessed using Pearson’s correlation and then by comparing the r coefficient with the r table. Inter-rater reliability was measured by the Kappa coefficient, and internal consistency was measured by calculating Cronbach’s alpha.
Results
The implementation of this evidencebased practice involved 51 respondents. The following is the distribution of children who received sedation and/or analgesic during hospitalization in the PICU (Table 1).
Table 1 shows that over half of the children who received sedation and/or analgesic while hospitalized in the PICU were male (n = 30; 58.8%). In terms of age, most of them were under 1 year (n = 20; 41.2%). The reason for PICU admission was dominated by Priority 2 (n = 32; 62.7%) that was a child who is critically ill with a basic disease that is medically unable to be overcome, but with intensive therapy can overcome the critical condition completely until the child returns to his condition before being treated in the PICU then followed by Priority 1 (n = 17; 33.3%). Most of them were on a ventilator (n = 44; 86.3%) and received only sedation (n = 25; 49.0%). Table 2 shows the sedation levels of pediatric patients in the PICU.
Most of the children received adequate levels of sedation (n = 39; 76.5%). However, 11 children (21.6%) were oversedated. The congruence validity is presented in Table 3, and the validity of each item is presented in Table 4.
Table 3 shows no correlation between the levels of sedation measured by CBS and NISS (P=0.118; α=0.05). The Mann– Whitney U test was applied to check the correlation or trend of sedation levels between variables measured by CBS and NISS. Children who were oversedated had higher ratings compared to those who had adequate and insufficient levels of sedation (i.e., 30.59 and 24.74, respectively). Therefore, children who were deemed to be oversedated, according to CBS tended to also be categorized as oversedated according to NISS.
The validity results show that all items in the CBS instrument were valid (r=0.348– 0.813). The result of the interrater reliability test showed good conformity between the observer and nurses (Kappa=0.741; P=0.001). The CBS instrument yielded a Cronbach’s alpha of 0.873, which indicated that this instrument was reliable and had high internal consistency.
Discussion
In this study, there was no significant correlation between levels of sedation measured by CBS and NISS, which means that the levels of sedation measured with CBS and NISS were almost the same. This result contradicts with previous studies that stated that NISS and CBS had low conformity. 10 CBS and NISS showed different outcomes for sedation levels, and this was related to the cultural bias of literacy discrepancy between oversedation and insufficient sedation. Therefore, the assessment using NISS was not as accurate as CBS.10
Some previous studies have identified the use of NISS as the silver standard for assessing the level of sedation in patients.11 This is because there is no gold standard that is comparable to CBS for assessing the level of sedation. Is NISS adequate to assess the level of sedation in patients? Does the use of it as a subjective assessment need to be confirmed? NISS and CBS are two instruments that confirm each other’s results. The use of NISS in assessing the level of sedation in children integrates nurses’ knowledge of disease, medication, behavior, ventilation, and other aspects related to intensive care for children. NISS could be valid if the interpretation was based on the related perception of sedation levels from experienced PICU nurses. In addition, NISS can be valid if it is performed by the same nurse during a single shift.11
Every item in the CBS instrumentwhich is comprised of alertness, calmness/agitation, respiratory response or crying, physical movement, muscle tone, and facial tension-has been validated. However, the item respiratory response still indicates low validity. This result is in line with another study that showed that the correlation coefficient for respiratory response in CBS was lower (0.67) than the overall correlation coefficient (0.79).12 The low validity of respiratory response could be caused by the slight scoring discrepancy between Statements 3, 4, and 5 regarding mechanical ventilation, which could cause the observer to experience difficulty in determining the score.13
Table 1.
Distribution of respondents’ characteristics (N= 51).
| Variable | Frequency (n) | Percent (%) |
|---|---|---|
| Sex | ||
| - Male | 30 | 58.8 |
| - Female | 21 | 41.2 |
| Age | ||
| - < 1 year | 21 | 41.2 |
| - 1-3 years | 7 | 13.7 |
| - 4-6 years | 7 | 13.7 |
| - 7-12 years | 5 | 9.8 |
| - >12 years | 11 | 21.6 |
| Reason for PICU admission | ||
| - Priority 1 | 17 | 33.4 |
| - Priority 2 | 32 | 62.7 |
| - Priority 3 | 2 | 3.9 |
| Ventilator use | ||
| - With ventilator | 44 | 86.3 |
| - Without ventilator | 7 | 13.7 |
| Medication | ||
| - Sedation | 25 | 49.0 |
| - Analgesic | 10 | 19.6 |
| - Sedation and analgesic | 16 | 31.4 |
Table 2.
Distribution of sedation levels using CBS (N= 51).
| Level of sedation | Frequency (n) | Percent (%) |
|---|---|---|
| Oversedation | 11 | 21.5 |
| Adequate sedation | 39 | 76.5 |
| Insufficient sedation | 1 | 2.0 |
Table 3.
Concurrent validity of CBS compared to NISS (N= 51).
| CBS | NISS | P-value | |||||
|---|---|---|---|---|---|---|---|
| Insufficient sedation | Adequate sedation | Oversedation | |||||
| n | % | n | % | n | % | ||
| Oversedation | 1 | 9.0 | 6 | 54.6 | 4 | 36.4 | 0.118 |
| Adequate and insufficient sedation | 3 | 7.5 | 33 | 82.5 | 4 | 10.0 |
Note: The mean rating for the Mann-Whitney U test was 30.59 for oversedation and 24.74 for adequate and insufficient sedation.
Table 4.
The validity of each item of the CBS instrument (N= 51).
| No | Instrument item | r coefficient | r table | Validity criteria |
|---|---|---|---|---|
| 1 | Alertness | 0.709 | Valid | |
| 2 | Calmness/agitation | 0.813 | Valid | |
| 3 | Respiratory response or crying | 0.348 | 0.2759 | Valid |
| 4 | Physical movement | 0.686 | Valid | |
| 5 | Muscle tone | 0.748 | Valid | |
| 6 | Facial tension | 0.793 | Valid |
The result of the inter-rater reliability test showed that the observer and the nurses had good conformity. The nurses made different decisions when scoring each item in the CBS instrument. These differences could have been influenced by the fact that the assessment was conducted once and did not reflect the patients’ level of sedation at all times.11 The reliability of CBS was also influenced by pediatric nurses’ education backgrounds and experience.7
The CBS instrument is considered to be reliable and to indicate high internal consistency. A study of the CBS instrument translated into Portuguese showed a Cronbach’s alpha of 0.81 and 0.92 from the first and second observers, respectively.6 This result proves that this instrument is adequate for assessing the sedation and analgesic administration for pediatric patients in the PICU who are on mechanical ventilation.6 It is concluded that the CBS instrument shows high internal consistency in various languages.
Conclusions
In conclusion, this evidence-based practice proved that the Indonesian version of the CBS instrument is valid and reliable for assessing the level of sedation in children ages 1 month to 18 years. Based on the aspects of the instrument, nurse, and patient, the CBS instrument could be applied in the PICU as an objective measurement to evaluate the level of sedation in patients who require mechanical ventilation or spontaneous breathing.
Acknowledgments
The writer expresses gratitude to all respondents and their families, the PICU nurses at Dr. Cipto Mangunkusumo Hospital Jakarta.
Funding Statement
Funding: This study was supported by Hibah PITTA B 2019, funded by DRPM Universitas Indonesia (No. NKB-0496/UN2.R3.1/HKP.05. 00/2019).
References
- 1.Soares MZL, Mohallem AGC, Brandi S, Cunha MLR. Comparison between Comfort-Behavior and Ramsay scales in a pediatric intensive care unit. Rev Dor 2014;15:25-9. [Google Scholar]
- 2.Dreyfus L, Javouhey E, Denis A, et al. Implementation and evaluation of a paediatric nurse driven sedation protocol in a paediatric intensive care unit. Ann. Intensive Car 2017;7:1-13. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Vet NJ, Ista E, de Wildt SN, et al. Optimal sedation in pediatric intensive care patients: a systematic review. Intensive Care Med 2013;39:1524-34. [DOI] [PubMed] [Google Scholar]
- 4.Suhandoko Pradian E, Maskoen TT. Reliabilitas dan validitas penilaian skala sedasi Richmond Agitation Sedation Scale (RASS) dan Ramsay pada pasien kritis dengan ventilasi mekanik di ruang perawatan intensif. Jurnal Anestesi Perioperatif 2014;2:186-93. [Google Scholar]
- 5.Kudchadkar SR, Yater M, Punjabi NM. Sedation, sleep promotion, and delirium screening practices in the care of mechanically ventilated children: A wake-up call for the pediatric critical care community. Crit Care Med 2014;42,1592-1600. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Amoretti CF, Rodrigues GO, Carvalho PRA, Trotta EA. Validation of sedation scores in mechanically ventilated children admitted to a tertiary pediatric intensive care unit. Rev Bras Ter Intensiva 2008;20:325-30. [PubMed] [Google Scholar]
- 7.Johansson M, Kokinsky E. The COMFORT behavioural scale and the modified FLACC scale in paediatric intensive care. British Association of Critical Care Nurses 2009;14:122-31. [DOI] [PubMed] [Google Scholar]
- 8.Bai J, Hsu L, Tang Y, van Dijk M. Validation of the comfort behavior scale and the FLACC scale for pain assessment in Chinese children after cardiac surgery. Pain Management Nursing 2012;13:18-26. [DOI] [PubMed] [Google Scholar]
- 9.Boerlage AA, Ista E, de Jong M. The COMFORT behavior scale: Is a shorter observation period feasible? Pediatr Crit Care Med 2012;13:124-5. [DOI] [PubMed] [Google Scholar]
- 10.Fagioli D, Evangelista C, Gawronski O. Pain assessment in paediatric intensive care: The Italian COMFORT behaviour scale. Nursing Children and Young People 2018;30:27-33. [DOI] [PubMed] [Google Scholar]
- 11.Ista E, van Dijk M, Tibboel D, de Hoog M. Assessment of sedation levels in pediatric intensive care patientscan be improved by using the COMFORT “behavior” scale. Pediatr Crit Care Med 2005;6:58-63. [DOI] [PubMed] [Google Scholar]
- 12.Maaskant J, Raymakers-Janssen P, Veldhoen E, et al. The clinimetric properties of the COMFORT scale: A systematic review. European Journal Pain 2016;20:1587-611. [DOI] [PubMed] [Google Scholar]
- 13.Van Dijk M, Ista E. Comfort behaviour scale-training module. Available on https://www.comfortassessment.nl/web/index.php/instruments/comfort/; 2015. [Accessed April 18, 2019]. [Google Scholar]