A first contribution to the validation of the Italian version of the Behavioral Pain Scale in sedated, intubated, and mechanically ventilated paediatric patients

Abstract

Background and aim of the work: Numerous negative outcomes of inadequate pain management among children have been cited in the literature. Inadequate pain management may be particularly detrimental to children and adolescents facing life-threatening injury or illness on a Paediatric Intensive Care Unit (PICU). It is therefore absolutely necessary that professionals utilize effective and efficient tools in order to evaluate a person’s sensations of pain in the most objective way possible. The COMFORT-B scale is recognised as the gold standard in such patients. However, the use of this instrument in the clinical PICU setting is disputed. It requires long periods of observation to ensure an adequate utilization. Boerlage et al. noted that nurses are often impatient and do not always observe the patient for the recommended 2 minutes period. The Behavioral Pain Scale (BPS), instead, is considered to be the gold standard for pain assessment in deeply sedated, mechanically ventilated adult patients. This observational pain scale requires shorter observation time compared to the COMFORT-B. Moreover, BPS three subscales are included in other observational pain scales for paediatric patients. Therefore, the objective of this study was to assess the applicability of the BPS for use with paediatric patients. Methods: Firstly, a questionnaire was administered to physicians and nursing staff that work in the units where the study was conducted in order to investigate the actual use of observational pain scales in their units. A second questionnaire was administered to a group of experts regarding the BPS, to assess both face validity and content validity, and to gain opinions on the relative appropriateness of each item. A descriptive, comparative design was used. A convenience sample of non-verbal, sedated and mechanically ventilated critical care paediatric patients was included. 39 observations were collected from 9 patients, all in their first year of age. Patient pain was assessed concurrently with the three observational scales, before, during and after routine procedures that are considered painful and non-painful. Results: The data collected through questionnaires for professionals gave a useful insight into pain assessment in the investigated units: only 46% of respondents stated that they assessed patients’ pain levels, with an average of 2.8 times per shift; 60% of respondents declared to be unhappy with the observational scales that they utilise. Regarding the observations, internal consistency was α = .865. Correlations between BPS and the other instruments were high, demonstrating a good concurrent validity of the test. T test and ROC curves demonstrated a good discriminant validity as well. Conclusions: Although the current study is based on a small sample of participants, these first results encourage us to continue working in the validation of the BPS in paediatric patients.

Introduction

The International Association for the Study of Pain (1; 2) in 1979 defined pain as “An unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage.” (p. 3). Pain is a frequently experienced problem in critically ill patients in the intensive care unit (ICU). Its assessment in critical patients may be complicated by decreased consciousness, severity of illness, mechanical ventilation, and the use of sedatives in these patients, particularly when high doses of sedatives are administered (3). Although self-report is still the gold standard in pain measurement according to the guidelines of the International Association for the Study of Pain (2), a number of ICU patients are unable to communicate effectively. In these cases, the gold standard (that is, the pain intensity reported by the patient) is not possible or is potentially unreliable. This is also a common problem in neonates and children, who are not able to report pain in a reliable manner. However, as IASP reports: “The inability to communicate verbally does not negate the possibility that an individual is experiencing pain and is in need of appropriate pain-relieving treatment.” (1; p. 3). Indeed, the absence of an adequate pain assessment in such patients interferes with their optimal care management and may provoke grave physiological and psychological damage. Numerous negative outcomes of inadequate pain management among children have been cited in the literature, including long-term behavioural changes, pain-perception impairment, pain-tolerance reduction, physical disability, and emotional disability (4). Inadequate pain management may be particularly detrimental to children and adolescents facing life-threatening injury or illness on a Paediatric Intensive Care Unit (PICU), as well as to the parents of such children and adolescents (4). It is therefore absolutely necessary that professionals utilize effective and efficient tools in order to evaluate a person’s sensations of pain in the most objective way possible. Several observational pain assessment scales are available, however, the literature points to an absence of an effective and simple method to evaluate pain in intubated and ventilated children (5). The COMFORT-B scale is recognised as the gold standard in such patients. It can be used in both ventilated and nonventilated paediatric patients and showed to be reliable in pain assessment of 0 to 3-year-old infants allowing professionals to manage the antalgic therapy correctly (6). However, the use of this instrument in the clinical PICU setting is disputed. Studies have demonstrated insufficient correlation between physiologic and behavioural COMFORT items (6, 7). Moreover, it requires long periods of observation to ensure an adequate utilization. In the original validation (6) cut point values for post-operative pain and distress were determined based on a 2-min observation period. Therefore nurses are instructed to observe patients for 2 minutes preceding the actual scoring. However, Boerlage et al. (8) noted that nurses are often impatient and do not always observe the patient for 2 minutes. They tend to reduce the recommended 2-min period – even to 30 secs - this might be understandable in view of the nurses’ heavy workload. Thanks to an experimental study, Boerlage et al. (8) concluded that observation for 30 seconds rather than the recommended 2 minutes creates a greater risk of underscoring pain. Therefore, the research suggests an emphasis on the lack of adherence to the requested observation times by professionals, which results in assessments of limited therapeutic use (8).

The Behavioral Pain Scale (BPS; 9), instead, is considered to be the gold standard for pain assessment in deeply sedated, mechanically ventilated adult patients. This observational pain scale requires shorter observation time compared to the COMFORT-B. Moreover, BPS three subscales (i.e., facial expression, upper limb movements, and compliance with mechanical ventilation) are included in other observational pain scales for paediatric patients (e.g., COMFORT-B, and Children’s Hospital of Eastern Ontario Pain Scale, CHEOPS; 10).

Therefore, the objective of this study was to assess the applicability of the BPS for use with paediatric patients.

Methods

Approval from the Institutional Review Board (IRB) at the University of Parma and written consent and assent from caregivers was obtained before study recruitment.

Preliminary, a questionnaire was administered to physicians and nursing staff that work in the units where the study was conducted in order to investigate the actual use of observational pain scales in their units. A second questionnaire was administered to a group of experts regarding the BPS, to assess both face validity and content validity, and to gain opinions on the relative appropriateness of each item.

Design and sample

A descriptive, comparative design was used. A convenience sample of non-verbal, sedated and mechanically ventilated critical care paediatric patients was included. Patients excluded from the study were those who were on medication with neuromuscular blockers (Curaro®); patients on muscle relaxants (Dandrolene®); patients with neurological pathologies or head injuries or encephalopathies and patients with drug-resistant epilepsy. Premature babies were also excluded because of their specific and complex characteristics. Refusals to take part in the study on the part of even one parent was also considered an exclusion criterion.

An a priori power analysis, conducted with the software G*Power (11), indicated that we needed to have at least 8 subjects to have 80% power for detecting a medium-sized effect when employing the traditional .05 criterion of statistical significance (12).

39 observations were collected from 9 patients, 5 boys and 4 girls, all in their first year of age.

Instruments

Behavioral Pain Scale (BPS). The BPS is a behaviour rating scale that evaluates three behavioural domains (i.e., facial expression, movements of upper limbs and compliance with ventilator). Each domain contains four descriptors that are rated on a 1-4 scale, and the total BPS value can range from 3 (no pain) to 12 (most pain) (13).

COMFORT-B. The COMFORT-B scale (6) asks observers to consider intensity of six behavioural manifestations: Alertness, Calmness, Respiratory response (for ventilated children) or Crying (for spontaneously breathing children), Body movements, Facial tension, and Muscle tone. For each of these items, five descriptions are provided reflecting increasing intensity of the behaviour in question; these are rated from 1 to 5. Summating the six ratings leads to a total score ranging from 6 to 30. Scores from 17 to 30 are thought to suggest pain or distress.

Numerical Rating Scale (NRS). The NRS is a global pain rating scale that asks to rate pain intensity by number (0 no pain and 10 worst pain) (17). NRS is a commonly used clinical measure of pain, and the gold standard of self-report measures - Patients are asked to indicate the intensity of pain by reporting a number that best represents it. In the use with non-verbal patients, the NRS assessment is intrinsically linked to the COMFORT-B scale and expresses the expert opinion of the nurse to complement the behavioural observation with the COMFORT-B scale. This expert opinion can take patient-related, environmental characteristics into account.

Procedure

This study was conducted over a 3-months period at the neonatal intensive care unit of Parma University Hospital and neonatal and paediatric intensive care unit of Genova Paediatric Hospital “G. Gaslini”, two hospitals in the north of Italy.

Patient pain was assessed concurrently with the three observational scales, before (T0), during (T1) and after (T2) routine procedures that are considered painful (mobilization and tracheobronchial aspiration) and non-painful (hematic withdrawal of CVC, CVC medication). The types of procedures to be observed were selected beforehand according to direction provided in the literature (1) but were observed only if they actually occurred during the time the observers were in the unit.

In order to guarantee reliability and objectivity, two independent observers carried out each observation simultaneously. Data were recorded on a data collection form that included a description of the patient and, in addition to the BPS, the COMFORT-B, and the NRS.

Data analysis

All data were analysed with SPSS version 20 (SPSS, Chicago, IL). Internal consistency was assessed with Cronbach’s coefficient α using the scores during nursing procedures when the patient was most likely to be experiencing pain. Inter-rater reliability was assessed using Cohen’s kappa for two raters. Spearman correlation coefficient was also used to examine the relationship between the BPS, COMFORT-B and NRS scores in order to test construct validity. We hypothesized that a significant correlation would be found between the three scales scores seeing that they were supposed to measure the same concept (pain). The discriminant validation was examined by calculating within-patient differences in scores between the assessments on T0, T1, and T2, using a t-test. We hypothesised that if the BPS reliably measures pain, the BPS scores should be much higher during painful procedures than while the patient is at rest. Receiver operating characteristic (ROC) curves and the area under the ROC curves were calculated to illustrate the relationship between sensitivity and specificity of the BPS and - as a further measure of discriminatory validity - to evaluate the probability of the BPS in correctly identifying patients with controlled and non-controlled pain, as defined by the NRS score.

Results

Questionnaire for professionals

In order to investigate the actual use of observational pain scales in their units a questionnaire was administered to physicians and nursing staff. 146 professionals replied to the questionnaire: 83% of the participants were nurses; 16% were physicians. 50% of those who replied to the questionnaire were from an emergency department; the majority (44%) reported work experience ranging from 0 to 5 years.

The data gave a useful insight into pain assessment in the investigated units: only 46% of respondents stated that they assessed patients’ pain levels, with an average of 2.8 times per shift; 60% of respondents declared to be unhappy with the observational scales that they utilise (mainly COMFORT-B and NRS).

In order to assess both face validity and content validity, a group of experts’ opinion was also requested regarding the BPS. 40 professionals replied to the questionnaire. 64% of the respondents believed that the BPS is “very clear”, compared to 36% (“sufficiently clear”). The scale was also judged to be “easy to use” by 72% of participants whilst 28% that stated it was “sufficiently easy”. 64% of the respondents considered the BPS to be very pertinent and 56% were positive towards the idea of adapting it for the use with paediatric patients. It should be noted that no negative responses (“little/none”) were registered. With respect to the detailed analysis of the subscales, all the three subscales were greatly appreciated. Only 4% of respondents were unhappy with “upper limb movements” and “compliance with mechanical ventilation” subscales. In the opinion of experts, these last two parameters should require long training and experience for nursing staff to be able to assess them. Moreover, the relationship between compliance with mechanical ventilation and pain may not be clear to the professional, since this is only recently receiving study (9).

Observations

Internal consistency for the BPS was α=.865. The agreement among raters was high (k=.86). Correlations between BPS and the other instruments were high, demonstrating a good concurrent validity of the test: BPS vs COMFORT-B (ρ=.93; p<.01); BPS vs NRS (ρ=.911; p<.01); NRS vs COMFORT-B (ρ=.880; P<.01). There was a statistically significant difference between BPS scores when the patients were at rest (T0: M=3.33; SD=.68), and during painful procedures (T1: M=7.35; SD=2.55) (t₍₃₈₎=-11.778, p<.001); a significant difference was also found between T1 and post procedure measurements (T2: M=3.31; SD=.71) (t₍₃₈₎=11.857; p<.001). The discriminant validation was further examined by calculating Receiver Operating Characteristic (ROC) curves. The area under the ROC curve was .995 (p<.001; e.s.=.007; I.C. 95% [.982;1.009]), including good diagnostic accuracy of the BPS for the identification of critical pain (Figure 1) – an AUC range between .9 and 1 is classified as “Excellent” (13). For a BPS cut-off score between 3 and 4, sensitivity and specificity were 1 and .929, respectively.

Figure 1.

Receiver operating characterisics curves for BPS scores

Discussion

Pain management is an essential component in the provision of high-quality health care to patients. Nurses play an important role in controlling and activating timely pain management interventions in collaboration with other professionals. The scientific literature has chosen the Behavioral Pain Scale as the gold standard for the evaluation of pain in intubated and non-intubated adult patients. This study aimed to demonstrate the applicability, efficacy and efficiency of the BPS for paediatric patients.

The first questionnaire was designed to investigate the actual use of observational pain scales in the investigated units. The majority of respondents were nurses, confirming what is reported in the literature (e.g., 14): nursing staff provide the most frequent and ongoing assessment of pain and are responsible for reporting any problems or concerns to the physician-led services.

Another interesting finding is that the majority of respondents reported to have a work experience between 0 and 5 years. The emphasis placed on pain management in the last ten years may have influenced professionals whose training is more recent, and thus resulting in a greater awareness on the subject of pain assessment.

However, a less positive element was revealed from questionnaire responses: 54% of nurses working in emergency units state that they do not evaluate pain during their shifts, confirming what was already evidenced in the guidelines issued in 2013 (1). The scant adherence to this practice may be due, amongst other reasons, to the fact that a specific assessment tool for intubated paediatric patients does not exist at present (15). Within the responses to the first questionnaire, the rate of dissatisfaction with the tools that are presently in use in the investigated units amounted to 63%.

The 2013 guidelines on pain (1) outlined the characteristics of a good pain scale. Similar results were confirmed by the questionnaire given to the experts, with greater weight given to efficiency and reliability. The respondents stated that they considered the BPS easy to understand and user friendly: this was reinforced by the fact that no respondent gave a negative opinion of these items. This confirms what has been reported by American and Italian authors in their validation of this scale, in the USA and in Italy, respectively (9, 16). Moreover, more than half of those interviewed believed that that the BPS can be easily used in a PICU setting.

A limitations to this study was the fact that the BPS was administered by the researchers. This decision was inevitable seeing that an increase in their working load would have probably make the nurses to quit the study. However, because of this, no data about the compliance of the professionals to the use of the scale were collected.

Moreover, in the investigated units was common practice for the nurses to concentrate all the assistance activities (e.g., blood sampling, tracheobronchial aspiration, patch change, etc.) in the shortest possible time, usually during the morning shift, in order to reduce the young patients discomfort as much as possible. Most likely, pain evaluation was made at the very end of the assistance procedures. Because of this, the post procedure evaluation of pain was not possible for every single procedure. However, in the original validation of the COMFORT B scale (6) cut point values for distress are determined based on a 2-min observation period before and after the actual evaluation of each potentially painful procedure. Again, this gives an account of the fact that the complexity in the use of this scale does not necessary meet the needs of the operational reality.

On the other hand, a high Cronbach’s α index revealed a very good internal consistency, demonstrating a good reliability of the BPS. Further, a good consistency among observational ratings provided by the two independent observers demonstrated a good inter-rater reliability. High correlations between BPS scores and the scores of the COMFORT-B (the current gold standard in PICUs) and NRS (the most utilized self-report tool) demonstrated good construct validity and concurrent validity. The BPS also showed a great capacity to discriminate pain in this sample; it showed high specificity and sensitivity; that is, it showed a good capacity to identify potential false positives and negatives.

Although the current study is based on a small sample of participants, these first results encourage us to continue working in the validation of the BPS in paediatric patients.