Use of scores to calculate the nursing workload in a pediatric intensive care unit

Abstract

Objective

To assess the performance of the Nursing Activities Score in a pediatric intensive care unit, compare its scores expressed as time spent on nursing activities to the corresponding ones calculated using the Simplified Therapeutic Intervention Scoring System, and correlate the results obtained by both instruments with severity, morbidity and mortality.

Methods

Prospective, observational, and analytical cohort study conducted at a type III general pediatric intensive care unit. The study participants were all the children aged 29 days to 12 years admitted to the investigated pediatric intensive care unit from August 2008 to February 2009.

Results

A total of 545 patients were studied, which corresponded to 2,951 assessments. The average score of the Simplified Therapeutic Intervention Scoring System was 28.79±10.37 (915±330 minutes), and that of the Nursing Activities Score was 55.6±11.82 (802±161 minutes). The number of minutes that resulted from the conversion of the Simplified Therapeutic Intervention Scoring System score was higher compared to that resulting from the Nursing Activities Score for all the assessments (p<0.001). The correlation between the instruments was significant, direct, positive, and moderate (R=0.564).

Conclusions

The agreement between the investigated instruments was satisfactory, and both instruments also exhibited satisfactory discrimination of mortality; for that purpose, the best cutoff point was 16 nursing hours/patient day.

INTRODUCTION

Intensive care units (ICU) provide care to high-risk patients and demand commitment, attention, and qualified skills from healthcare professionals. However, everyday practice imposes an excessive workload on nurses, which interferes with the quality of the care they provide.

Indicators of the severity of patients' conditions and nursing workload are becoming increasingly necessary to ensure the patients' safety and to improve the quality of care, especially within the pediatric setting, in which care includes not only the children but also their mothers and relatives.⁽¹⁾

Several instruments were suggested for that purpose, including the Therapeutic Intervention Scoring System (TISS), whose original formulation in 1974⁽²⁾ included 57 items corresponding to therapeutic interventions. TISS was first revised and updated in 1983,⁽³⁾ when the number of items was increased to 76 (TISS-76). It was then simplified, resulting in a validated instrument comprising 28 items (TISS-28) in 1996.⁽⁴⁾ TISS-28 is a simpler and faster scoring system that allows for a better description of nursing activities; however, its effectiveness in measuring nursing workload in the pediatric setting has not yet been validated.

TISS-28 comprises the following categories: basic activities, ventilator support, cardiovascular support, renal support, neurological support, metabolic support, and specific interventions. Each category comprises specific items that are scored from one to eight. Each point corresponds to 10.6 minutes of nursing work devoted to direct patient care, and a typical nurse is able to deliver work equal to 46 TISS-28 points per eight-hour shift.⁽⁴⁾ TISS-28 was translated and validated in Brazil in 2000.⁽⁵⁾

In 2003, TISS-28 was reformulated seeking to identify the nursing activities that best represent the nurses' workload. The result was the Nursing Activities Score (NAS), which includes several activities omitted in the earlier versions, such as hygiene procedures, patient mobilization and positioning, support and care of relatives and patients, and administrative and managerial tasks.⁽⁶⁾

NAS may be used as a managerial tool. This score proved to be useful for the estimation of nursing workload in pediatric ICU (PICU) based on the analysis of 23 items, including sub-items and is expressed as the percentage of time spent by a nurse on care-related tasks per ICU patient per 24 hours.⁽⁶⁾ Therefore, the NAS scores represent the time spent on patient care, and each point corresponds to 14.4 minutes.⁽⁷⁾ This instrument was translated and validated in Brazil in 2002.⁽⁸⁾

On these grounds, we believe that both TISS-28 and NAS are useful for the estimation of nursing workload in ICU and that their use should be more thoroughly investigated in the pediatric setting. The overall aim of the present study was to assess the performance of NAS by comparing its scores expressed as the time spent on nursing activities with therapeutic interventions and PICU outcomes. In particular, we sought to compare the time spent on nursing activities as indicated by the NAS and TISS-28 and to investigate the correlations of those results with severity, morbidity, and mortality in PICU.

METHODS

The present study was a prospective, observational, analytical, and quantitative cohort study conducted at the PICU of the Hospital da Criança Santo Antônio - HCSA, Porto Alegre (RS), Brazil. This PICU is classified as type III (according to the Brazilian system) and has 25 beds that tend to patients covered by social security and insurance companies, as well as private patients. The study population comprised all the children aged 29 days to 12 years who were admitted to the PICU from August 25, 2008 to February 25, 2009. Only the children who stayed more than eight hours in the PICU were included in the study, whereas in the cases of the children who died, a hospital stay ≥4 hours was considered the inclusion criterion. Moreover, readmissions were considered as new admissions. The children diagnosed with brain death at admission were excluded from the study. The cases without an indication for intensive care (workload too low) as well as patients admitted to the PICU near death (workload too high) were not included in the statistical analysis.⁽⁹⁾ The sample comprised clinical and surgical patients referred by the hospital emergency department, surgery department, clinical wards, and other hospitals.

The data were collected using an ad hoc instrument based on TISS-28 and NAS to facilitate data collection and comparison. The investigators first generated a table with the items covered by each score to identify the categories shared. No semantic changes or changes in the description, content, order, format, use, or application were made in either scoring system. In the table, similar items were grouped together, which resulted in an ad hoc instrument comprising 27 items relevant to therapeutic interventions. The instrument was applied by the investigator, who was assisted by 15 nurses who were given previous training to standardize the responses. The Kappa test⁽¹⁰⁾ was used to assess inter-examiner agreement, for which one single nurse from each shift was randomly selected, as the group was homogenous. The Kappa index value among those three nurses and the researcher was 0.95.

To compare the two instruments, each score point was transformed into minutes as follows: one point in TISS-28 corresponded to 10.6 minutes in an 8 hour period,⁽⁴⁾ and one point in NAS corresponded to 14.4 minutes in a 24 hour period.⁽⁷⁾ To adjust for the difference in the periods of time assessed, i.e., 8 versus 24 hours, the TISS-28 points were multiplied by three, and the linear variation of TISS-28 was considered for the statistical analysis.

The Pediatric Index of Mortality 2 (PIM 2)⁽¹¹⁾ was applied at admission by the researcher and two previously trained nurses. The ad hoc TISS-28/NAS instrument and the demographic data were prospectively collected during the children's stay in the PICU.

The main variables analyzed were TISS-28 and NAS scores, which were used to measure the nursing workload. The following TISS-28 and NAS scores were separately entered in an electronic spreadsheet: the ones calculated each day of stay at the PICU, the ones calculated at admission and discharge, the total and average scores, and the worst scores. For the purpose of correlation, the percent mortality risk in PIM 2 was used. Two outcome variables were established, duration of stay and reason for discharge.

The data were analyzed using the Statistical Package for the Social Sciences (SPSS) 17.0. A simple descriptive analysis was applied to the investigated groups and subgroups. The chi-square test was used for the analysis of qualitative variables; means and medians were calculated for numerical variables, and the t-test was used in inter-group comparisons. The Z-statistic, along with the Hosmer-Lemeshow goodness-of-fit test were used to analyze differences in the outcome variables. The model's discriminatory performance was analyzed using a receiver operating characteristic (ROC) curve. The mortality rate was analyzed based on the standardized mortality ratio (SMR), which was calculated using the PIM 2 data. The results of the analysis are shown in the tables and graphics. The significance level was established as p≤0.05. The correlation between TISS28 and NAS was assessed by Pearson's linear correlation. For interpretation, the results were categorized as follows: r=0.0-0.3, weak correlation; 0.3-0.7, moderate correlation; >0.7, strong correlation.⁽¹²⁾

An analysis of agreement was performed by a Bland-Altman plot⁽¹³⁾ for score variation. The presence of satisfactory agreement was established when more than 95% of the sample was contained within its limits (standard deviation of the mean: ±1.96).

The study was approved by the Research Ethics Committee (Comitê de Ética em Pesquisa - CEP) of the Irmandade da Santa Casa de Misericórdia de Porto Alegre, number 1887/08 with a waiver of informed consent. No clinical or laboratory test was performed that had the exclusive purpose of elucidating or contributing additional data to the study.

RESULTS

A total of 662 admissions to the PICU occurred along the study period. Fifty-five newborn infants were excluded, as were 50 children ≥13 years old. Four children who died within less than four hours after admission, four patients who were discharged from the PICU before eight hours of stay, and four children diagnosed with brain death at admission were also excluded. Therefore, the final sample comprised 545 admissions corresponding to 448 patients.

The general characteristics of the sample are described in table 1. The participants' age, body weight, and height are expressed as median and interquartile range (IQR), and the remainder of the data are expressed as absolute and percent frequencies.

Table 1.

General characteristics of the total sample

General data of the total sample (N = 545)	Results
Age (months)	27.3 (8.1-81.2)
Weight (kg)	12.0 (6.4-20.0)
Height (cm)	87.0 (66.0-117.5)
Infants (< 1 year old)	189 (34.7)
Male gender	303 (55.6)
Clinical patients	203 (37.2)
Acute patients	104 (19.1)
Duration of stay (days)	3 (2-5)
Stay ≤7 days	449 (82.4)
Insurance/social security	312 (58.0)
PIM 2 (%)	1.1 (0.4-2.5)
PIM 2 >10	36 (6.6)
Mechanical ventilation	236 (43.3)
Provenance
Surgery department	319 (58.5)
Hospital wards	80 (14.7)
Emergency department	82 (15.0)
Other hospital	64 (11.7)
Organ dysfunction
Cardiovascular	182 (33.4)
Hematological	19 (3.5)
Liver	4 (0.7)
Neurological	121 (22.2)
Renal	30 (5.5)
Respiratory	154 (28.3)
Gastrointestinal tract	35 (6.4)

PIM 2 - Pediatric Index of Mortality 2. Results are expressed as the median, IQR (interquartile range); absolute and percent frequencies.

The TISS-28 scores at admission varied from 12.0 to 68.0, with an average score of 22.0±12.0 and a median of 30.0. The maximum TISS-28 score varied from 12.0 to 68.0, with an average of 34.0±13.0 and a median of 32.0. The NAS scores at admission varied from 40.0 to 130.0, with an average of 59.0±12.0 and a median of 59.0. The maximum NAS score varied from 40.0 to 130.0, with an average of 63.0±15.0 and a median of 61.0.

The 545 admissions lasted from one to 71 days, with an average of 5.4±8.3 days and a median of three days. A total of 2,951 assessments were performed along the study period. Taking all the measurements into consideration, the TISS-28 score varied from 10.0 to 68.0, with an average of 29.0±10.0 and a median of 27.0 (IQR: 21-5), and the NAS score varied from 31.0 to 132.0, with an average of 56.0 ± 11.0 and a median of 53.0 (IQR: 48-62).

For comparison, the scores were transformed into minutes. The number of minutes corresponding to the TISS-28 score was higher than that of the NAS in all the assessments (p<0.001), as described in table 2.

Table 2.

Therapeutic Intervention Scoring System-28 and Nursing Activities Score values at admission and maximum and total values in points and minutes

	Calculated value	Value in minutes
Value at admission (N=545)
TISS	32.66 (±12.18)	1,039 (±397)
NAS	58.80 (±11.89)	847 (±171)
Maximum value (N=545)
TISS	33.93 (±12.71)	1,079 (±404)
NAS	63.40 (±12.71)	913 (±218)
Total value (N=2.951
TISS	28.79 (±10.37)	915 (±330)
NAS	55.67 (±11.82)	802 (±161)

TISS - Therapeutic Intervention Scoring System-28; NAS - Nursing Activities Score. Results are expressed as mean±standard deviation.

The expected mortality as assessed by PIM 2 was 3.5%, and the observed mortality was 6.2%. The SMR was 1.71 (95% CI).

The scores transformed into minutes were also used for the assessment of severity. The number of minutes resulting from the conversion of the TISS-28 and NAS scores exhibited good discrimination of mortality. Upon admission, the area under the ROC curve (AUROC) corresponding to TISS-28 was 0.79 (95% CI: 0.72-0.87), and that of NAS was 0.84 (95% CI: 0.75-0.92). Relative to the maximum scores, the AUROC of TISS-28 was 0.87 (95% CI: 0.81-0.2), and that of NAS was 0.99 (95% CI: 0.99-1.00). Furthermore, PIM 2 exhibited satisfactory performance, with the AUROC at 0.91 (95% CI: 0.85-0.96).

Based on the ROC curve, the best cutoff point of the assessed instruments by which to define the mortality risk was investigated, and a value of 960 minutes was selected, which corresponds to 16 nursing hours/patient day. Table 3 describes the sensitivity and specificity of this cutoff point for TISS-28 and NAS, as well as the mortality odds ratio (OR).

Table 3.

Therapeutic Intervention Scoring System-28 and Nursing Activities Score performance

Cutoff point	Sensitivity (%)	Specificity (%)	OR	95%CI
TISS at admission >960 minutes	81.8	52.5	4.9	2.0-12.2
TISS maximum >960 minutes	93.9	57.6	30.3	4.1-223.4
NAS at admission >960 minutes	66.7	90.6	19.3	8.8-42.3
NAS maximum >960 minutes	97.0	79.5	124.0	16.7-918.2

OR - odds ratio; TISS - Therapeutic Intervention Scoring System-28; NAS - Nursing Activities Score; 95%CI - 95% confidence interval.

According to assessment by Pearson's correlation, the TIS-28 and NAS scores at admission (R=0.680) and the maximum scores (R=0.685) exhibited significant, direct, positive, and moderate correlation.

Based on the analysis of the 2,951 assessments, the correlation between TISS-28 and NAS assessed by Pearson's correlation relative to the scores converted to minutes was also significant, direct, positive, and moderate (R=0.564).

The difference between the TISS-28 and NAS scores was 114 ± 273 (95% CI: 104123). The limit of agreement for two standard deviations (SD) was -433 and 661, as depicted in figure 1. Only 120 (4.1%) measurements exhibited differences greater than or less than two SD between TISS-28 and NAS.

Figure 1.

Bland & Altman plot representing the agreement between Nursing Activities Score and Therapeutic Intervention Scoring System-28.

DISCUSSION

In the present study, the average TISS-28 and NAS scores corresponding to all assessments were 29.0 and 56.0, respectively. Following the conversion of the scores to minutes, the average time was higher in TISS-28 compared to NAS in all the assessments. TISS-28 and NAS exhibited a significant, direct, positive, and moderate correlation (R=0.564).

The sample comprised 545 admissions to the PICU over six months, and 2,951 assessments were performed. Several studies were conducted in Brazilian adult ICUs, but they had limited durations and, thus, also limited sample sizes. One study conducted in 2007⁽¹⁴⁾ assessed admissions over 14 days, resulting in a sample of 33 patients. One of the same authors assessed admissions over one month periods in 2008⁽¹⁵⁾ and 2010,⁽¹⁾ resulting in samples with 200 and 68 participants, respectively. The samples of international studies conducted on adults for more than four months in 2005⁽¹⁶⁾ and 11 months in 2008⁽¹⁷⁾ comprised 350 and 250 participants, respectively. Furthermore, the duration and sample size of a study conducted at a neonatal ICU (NICU) were lower compared to these values in the present study at one month and 11 participants,⁽¹⁸⁾ respectively, while another study conducted at a NICU assessed 141 newborn infants over 29 days.⁽¹⁹⁾ Therefore, the sample size in the present study stands out, and it supports the relevance of the results found for the TISS-28 and NAS scores.

According to the results of the present study, the admission of surgical patients was predominant, which contradicts the findings of other studies in which admissions for clinical reasons prevailed.^(14,15,20) This discrepancy is due to the profile of the hospital in which the study was conducted, which is characterized by the performance of many surgical procedures. The nursing workload was greater for the surgical patients, with TISS28 scores >60.0 and NAS scores corresponding to >100% of the time spent on nursing care. Those results are indicative of the nursing workload corresponding to the support level needed.

One study conducted at three ICUs (pediatric, surgical, and general) in a university hospital sought to investigate the differences in nursing workload per shift using NAS. The highest NAS scores corresponded to the day shift and the surgical patients,⁽²¹⁾ thus corroborating the findings of a Brazilian study in which the average NAS score was 10.4, whereas the odds of the surgical patients demanding greater nursing workload in the first 24 hours since admission was almost threefold (2.79) higher compared to the clinical patients.⁽²²⁾

In the hospital where the present study was conducted, immediate postoperative care is routinely provided at the PICU. The first 24 hours after a surgical procedure are critical and thus demand a high nursing workload. This fact accounts for the predominance of surgical patients in the present sample, as well as the high TISS-28 and NAS scores associated with their care. Therefore, the TISS-28 and NAS scores at admission of such patients, 33.0 and 59.0, respectively, are worth mentioning. The TISS-28 and NAS scores tend to be higher at admission due to the high dedication of the staff, the performance of the admission standard procedures, and the precise and quick interventions and care upon admission, which demand a high workload from the nursing staff.

One study compared the nursing workloads and interventions related to patients admitted to public or private ICUs. The results showed that the average NAS score at admission was 61.9 and that the nursing workload demanded by the patients admitted to public ICUs was greater in the first day of ICU stay.⁽²³⁾

The average TISS-28 score in the present study was 29.0, and these scores varied from 10.0 to 68.0; converted into minutes, the average score corresponded to 915 minutes. In Brazilian studies conducted on adult patients, the average TISS-28 score was lower than the present study. In studies conducted in 2008⁽¹⁵⁾ and 2002,⁽⁸⁾ these values were 24.2 and 24.1, respectively. In international studies also conducted on adults, the average TISS-28 was 28.8,⁽⁴⁾ and thus, it was closer to that found in the present study. In other studies, the average scores varied from 26.2 to 29.8.^(20,24,25) In one study conducted at a PICU in 2009,⁽²⁶⁾ the average score was 19.3.

In the present study, the average NAS score was 56.0, and these scores varied from 31.0 to 132.0; converted into minutes, the average score corresponded to 802 minutes. In Brazilian studies conducted in general and specialized adult ICUs, the average NAS score found was higher than the present study, ranging from 66.5 to 73.7.^(8,27,28) In two studies conducted by the same author in 2008⁽¹⁵⁾ and 2010,⁽¹⁾ the average scores were 67.2 and 63.7, respectively. Other studies found lower average values of 51.5, for instance.⁽²⁹⁾ In international studies conducted on adults, the average NAS scores were either higher, 76.17,⁽¹⁷⁾ or lower, 41.27,⁽¹⁶⁾ than those in the present study. In one Brazilian study conducted at a NICU in 2007 and involving 11 newborn infants, the average score was 91.9,⁽¹⁸⁾ and 62.29 was the average score in another study conducted in 2011.⁽³⁰⁾ In one international study conducted at three ICUs (pediatric, surgical, and general), the average score was 54.7.⁽²¹⁾

One relevant finding of the present study is that the average TISS-28 score was higher than the average NAS score, which contradicts the findings reported by other studies. The reason for this discrepancy might be that the NAS score became significantly overestimated in the category "basic activities", while the TISS-28 score was almost always higher in the remainder of the categories. The investigated ICU is located at a hospital considered to be a reference for pediatric care, with many patients in severe conditions who demand approximately 16 nursing hours/patient day.

In that hospital, immediate postoperative care is routinely performed at the PICU. The high proportion of cardiology patients in this population results in an excessive workload for the staff, as such patients are in critical condition, are hemodynamically unstable, and exhibit high odds of complications. Those facts were corroborated by the increase in the "cardiovascular support" scores for both the TISS-28 and NAS systems.

We would like to call the attention to a study conducted to assess the TISS-28 scores in a surgical ICU that sought to correlate those results with the type of surgery, the severity of disease, and the outcomes of the ICU patients. The results showed that the highest TISS-28 score (47.7) corresponded to the patients subjected to cardiothoracic surgery.⁽³¹⁾

That result should be taken into consideration for appropriate staff sizing, as small staffs exhibit no true, essential, and fundamental availability to devote themselves and provide emotional support to patients and their relatives. This idea is reflected in the reduction in the values in the NAS category "basic activities".

We assessed each instrument and category separately to investigate the maximum TISS-28 and NAS values. That approach allowed us to identify high NAS scores in the category "basic activities", which represented 73.7% (130.3 points) of the total score, while that category contributed to only 17.9% (14.0) of the total TIFF-28 score. We believe that TISS-28 is not a poor instrument by which to assess the basic nursing activities, but that NAS includes several activities that TISS-28 lacks, such as hygiene procedures, patient mobilization and positioning, support and care of relatives and patients, and administrative and managerial tasks.

We speculate that in the present study, the basic activities were not sufficiently scored to overcome the TISS-28 interventions, especially when the fact that cardiovascular support exhibited a proportionally higher score in TISS-28 is taken into account.

The score of the category "basic activities" was considerably higher in NAS, while the scores of the categories "cardiovascular support", "neurological support", "metabolic support", and "specific interventions" were higher in TISS-28.

The Pearson's coefficient demonstrated a significant, direct, positive, and moderate correlation between TISS-28 and NAS (R=0.564) and between the score at admission and the maximum score, R=0.680 and R=0.685, respectively. Furthermore, one study conducted in 2002⁽⁸⁾ found a significant, positive, and moderate correlation between both instruments, R=0.67.

The results of the present study indicated satisfactory agreement between the instruments assessed. The scores of both instruments at admission and the maximum scores exhibited satisfactory discrimination of the mortality outcomes as shown by the ROC curve. One study conducted in 2009⁽²⁶⁾ found that the scores of both Nine Equivalents of Nursing Manpower Use Score (NEMS) and TISS-28 at admission and at their maximum levels exhibited satisfactory discrimination of mortality outcomes during the stay at the ICU as assessed by the ROC curve (AUROC of TISS-28 at admission 0.68; AUROC of the maximum score 0.76).

The present study found that the performance of PIM 2 was satisfactory, with AUROC 0.91. One study conducted on adults in Brazil⁽²⁹⁾ investigated the correlation between NAS and Acute Physiology and Chronic Health Evaluation II (APACHE II) by Pearson's coefficient and found a positive correlation. In that study, the average NAS score was 51.5. Another author⁽⁸⁾ compared NAS to the severity index Simplified Acute Physiology Score II (SAPS II) using Pearson's coefficient and found a weak liner correlation between the instruments.

TISS-28 and NAS are extremely relevant and useful, which was indicated by their satisfactory correlation, agreement, and performance.

One of the relevant findings of the present study is that the best cutoff point for the mortality risk of TISS-28 and NAS is 16 nursing hours/patient day. However, additional studies are needed to establish further associations with the pediatric population.

Among the limitations of the present study, it should be observed that the data were collected at a single PICU and that the TISS-28 and NAS assessments were not distributed per shift or between weekdays and weekends. In this regard, we call the attention to one study that used NAS to analyze differences in nursing workload per shift.⁽²¹⁾ The data were collected four years ago, although the authors believe that the sample still reflects the present-day conditions of Brazilian PICUs. Although TISS-28 and NAS differ in their application and measurement criteria, the conversion of their scores to minutes of nursing allowed comparisons, while the bias due to that extrapolation was not taken into account. In the present study, severity criteria were applied,⁽⁹⁾ according to which patients who could have demanded excessively high or low nursing workloads were excluded (four patients who died within less than four hours after admission, four patients who were discharged from the PICU before eight hours of stay, and four children who were diagnosed with brain death at admission). None of those patients was included in the study sample. Finally, as only a few Brazilian or international studies used NAS in pediatric populations, comparisons of the findings of the present study were hindered.

To summarize, the results of the present study are a relevant contribution, as they describe the demands for nursing workload and thus point to the need to reflect on the nursing workload associated with pediatric care. The systematic application of NAS makes it an excellent indicator, and sizing is indispensable in future studies.

CONCLUSION

The results of the present study indicated a satisfactory correlation between Therapeutic Intervention Scoring System-28 and Nursing Activities Score in a pediatric population. Both instruments exhibited accurate discrimination by which to predict mortality. The best cutoff point for mortality risk of both instruments was 16 nursing hours/patient day.