Validity and reliability analysis of the Turkish version of pediatric nutritional risk score scale

Abstract

Background/Aims

We aimed to perform the validity and reliability analysis of the Turkish version of the Pediatric Nutritional Risk Score (PNRS).

Materials and Methods

The study group consisted of 149 patients aged between 1 month and 18 years who were admitted to the hospital for at least 48 h. The patients’ age, gender, anthropometric measurements, length of stay, admission diagnosis, daily body weights, food consumption, and pain status were recorded. Backward and forward translations into Turkish were done. PNRS was performed by two different physicians. The consistency of the PNRS results was evaluated to determine the validity of PNRS. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated.

Results

Of all patients, 69 (46.3%) were female and 80 (53.7%) were male. The mean length of the stay was 7.3±4.0 days. The mean age of the patients was 51.9±63.6 months. The Kappa coefficient between the two physicians was 0.66. Weight loss was observed in 65.2% of the patients in the high-risk group and 25.4% in the low-risk group. The hospital malnutrition rate was 31.5%. A higher risk was identified in those with <50% food intake and more severe disease. The specificity, sensitivity, NPV, and PPV of PNRS were 82.1%, 77.8%, 92.0%, and 58.3%, respectively.

Conclusion

A good consistency suggests that the Turkish validation was achieved successfully. The power of PNRS to discriminate the patients with moderate-low risk of developing malnutrition is higher than the patients with high risk. PNRS is considered a valid and reliable tool to establish the risk of malnutrition in the hospitalized patients.

INTRODUCTION

Malnutrition is a clinical condition resulting from the insufficient or unbalanced intake of one or more nutritional elements. This may appear due to protein insufficiency, energy insufficiency or both (1). Malnutrition contributes to 45% of the child mortality under 5 years of age and is still an important public health concern in the underdeveloped and developing countries (2). In Turkey, 1% of the children under 5 years of age are wasted and 12% are stunted (3). Hospital malnutrition is the malnutrition that develops during hospitalization even when a person is not malnourished at the time of hospitalization. The rate of hospital malnutrition is also reported to be very high and is approximately 20–50% (4–7). The nutritional situation of every patient should be assessed and it should be part of the routine physical examination to notice malnutrition. Some screening tools have been developed for this purpose (8,9). The aim of the screening tools is to calculate the nutritional risk and provide sufficient nutritional support during patient’s stay in the hospital (10). The Subjective Global Nutritional Assessment (SGNA), Pediatric Yorkhill Malnutrition Score (PYMS), Screening Tool Risk on Nutritional Status and Growth (STRONGkids), and Pediatric Nutrition Screening Tool (PNST) have been developed based on the rules and principles that are existent while the Nutritional Risk Score (NRS) is developed through comparison with the nutritional risk index (NRI) that was created for the adults (11,12). As for the Pediatric Nutritional Risk Score (PNRS) and Screening Tool for the Assessment of Malnutrition in Pediatrics (STAMP), these are finalized after the multivariate analysis of the structured questionnaires that define a number of factors, which foresee the nutritional risk (13).

When the importance of malnutrition in inpatients is considered, the effect of the scales in identifying this situation is quite high. We aimed to adopt the PNRS scale to the Turkish version, and to assess its validity and reliability.

MATERIALS AND METHODS

This study was conducted at the inpatients departments of Izmir Dr. Behcet Uz Children’s Hospital between August 2015 and October 2015. The study was approved by the local ethics committee of Dr. Behcet Uz Children’s Hospital (No. 2015/07-05). Prof. Dr. Claude Ricour’s permission was received via e-mail to adopt the scale to the Turkish version. As the first step, the forward and backward translations of the original scale were done. Following the consensus on translation, the scale went through the cognitive debriefing process. The last stage was using the scale on a group of patients.

One month to 18 years old patients (with at least 48 h of stay at the hospital) were divided into groups as 1–3 months, 3–12 months, 12–72 months, and 72 months and older, based on the age groups of the original scale. A total of 158 patients were enrolled in the study with at least 35 patients for each subgroup (80% power, 5–20% impact size).

The study team consisted of two pediatricians who were responsible for performing the scale and data collection, one dietician calculating the daily food intake, and two senior pediatricians supervising the process. The following variables were assessed through the scale: decrease in food intake, ability to consume food, and the intensity of pain and disease (Figure 1). The food consumption forms were handed over to the families on admission to assess 48 h of the food intake. The data from the food consumption form were assessed by the dietician. The daily food intake and calorie requirements were calculated through BEBIS 7.1 (Nutrition-Information System 7.1) program. The patients consuming less than 50% of their daily nutritional need received 1 point according to the scale. For the pain assessment, 1 point was given if there was pain while no pain received 0 point. For patients under 6 years old, the pain indicators (restlessness, abnormal behavior, and crying relentlessly) observed by the family or nurses were considered. The patients older than 6 years were assessed based on the visual pain scale and a cut-off value was accepted as pain over 40%. The current situation of the illness was assessed between 1 and 3 points based on the table in the scale. The patients with the liver, heart, and renal failure were excluded from the study based on the assumption that their fluid loss would be confusing. The scale was applied to the same patients separately by two physicians. The malnutrition risk in the patients was scaled between 0 and 5 points; 0 point meaning low risk, 1–2 points meaning medium risk, and ≥3 points meaning high risk. The coherence between the risk scores of the two physicians was calculated by the Kappa statistics (Chi-square test). The results from the first practitioner were used in assessing if the level of coherence between the two was well according to the kappa statistics.

Figure 1.

Pediatric nutritional risk score scale.

The anthropometric measurements [weight-for-age, height-for-age, body-mass index (BMI), triceps skin fold thickness (TSF), and mid-upper arm circumference (MUAC)] were noted at the time of hospitalization. The weight and height were measured as follows: for patients over 2 years old, the weight was without shoes and with a thin robe by using a 100-gr sensitive scale, and height was measured when standing and with 0.1-cm sensitive stadiometer; for patients younger than 2 years old, the weight was measured by 10-g sensitive scale, and height was measured by a plastic measuring tape while the patients were lying down. MUAC was measured around the midpoint of acromion and olecranon of the left arm with plastic measuring tape, TSF at the same point by Holtain caliper that applied a 10-gr pressure on each centimeter, and the mean values from three measurements were taken into account.

The weight-for-age, height-for-age, weight-for-height, BMI values, and z scores of 1–3 months old patients were calculated using the World Health Organization (WHO) Anthro program. The weight-for-age, height-for-age, weight-for-height, BMI, TSF, MUAC values, and z scores of the 3–60 months old patients were assessed based on the WHO Anthro program. The weight-for-age, height-for-age, BMI values, and z scores of the 6–19 years old patients were assessed based on the WHO Anthro Plus program. The weight-for-height in the patients older than 6 years, and TSF and MUAC percentiles in 2 months and older patients were calculated using the National Center for Health Statistics (NCHS) data (14). The weight of the patients was measured daily. When the difference between the lowest weight measured during hospitalization or at the time of discharge from the hospital and the weight at the time of hospitalization was more than 2%, the patient was accepted as malnourished. Thus, a comparison between the malnutrition risk identified by the PNRS scale and the patients with more than 2% of weight loss was possible. When it comes to the level of malnutrition at the time of hospitalization, the weight-for-height between 80 and 90% was defined as mild, 70–79% moderate, and <70% as severe malnutrition (15). The wasting and stunting differentiation was interpreted according to the height-for-age and weight-for-height Standard Deviation Score (SDS) values (16). Those whose height-for-age SDS values were below −2, were accepted as stunted while the weight-for-height values under 90% as wasted (14, 17, 18).

To evaluate the validation of the PNRS scale, the consistency of the PNRS scores between the two different physicians were assessed. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated. The endpoint criteria accepted by the scale developers was the weight loss of more than 2% of the reference weight.

The anthropometric measurements, food consumption, existence of pain in the patients were registered in the patient form.

Statistical analysis

The Statistical Packages for the Social Sciences (SPSS) 22.0 Microsoft for Windows program (IBM Corp.; Armonk, NY, USA) was used for the statistical analysis. Mean, standard deviation, median, minimum–maximum, and percentage distribution data were used for descriptive variables. For validity analyses, the criteria validity (predictive validity), the correlation between >2% weight loss and z score, initial weight, and weight loss in the patients were assessed by the t test, Chi-square test, and odds ratio in independent groups. Inter-rater consistency was used for the reliability analyses and kappa coefficient was calculated for the inter-rater observer correlations and categorical variables.

RESULTS

Of 158 patients, 9 were excluded from the study (6 left hospital before 48 h and 3 needed intensive care). As a result, 149 patients, of whom 69 (46.3%) were females and 80 (53.7%) were males (Table 1), participated in this study.

Table 1.

General characteristics of the patients included in the study.

	Male 80 (53.7)	Female 69 (46.3)
Gender (n (%))	Mean±SD (median)	Minimum	Maximum
Length of stay at the hospital (day)	7.3±4.0 (7.0)	2.0	25.0
Age (months)	51.9±63.6 (16.0)	1.0	213.0
Height (cm)	94.7±38.0 (79.0)	49.0	182.0
Height SDS	−0.1±1.5 (−0.01)	−6.2	3.8
Weight (kg)	10.8±18.7 (10.8)	2.8	89.7
Weight SDS	−0.3±1.4 (−0.23)	−4.3	3.4
Mid-upper arm circumference (cm)	16.6±4.8 (14.8)	8.8	33.5
Triceps skinfold (mm)	10.0±4.6 (8.5)	4.5	31.0
Body mass index (kg/m2)	16.6±3.2 (15.9)	11.5	29.9

The baseline characteristics of the study subjects.

SDS: Standard Deviation Score

The average length of the hospital stay was 7.3±4.0 days. The mean age of the patients was 51.9±63.6 months and median age was 16.0 months. The urinary tract infection was the most frequently seen diagnosis with 10.7% patients while cardiopathy and pneumonia were the second most frequent (7.4%), and acute bronchiolitis was the third frequently seen diagnosis (6.7%). Diseases, such as gastroenteritis, arthritis, convulsion, and asthma attack were among the other reasons for hospitalization.

According to the weight-for-height SDS, 30 patients were wasted on admission. One patient had severe, 5 patients had moderate, and rest of the 30 patients had mild malnutrition.

The MUAC percentiles data were calculated for 128 patients and one patient had >95p, 31 patients had <5p. The TSF percentiles data were calculated for 127 patients and 2 patients had >95p, 14 patients had <5p according to the NCHS data.

The PNRS scores identified by the first and second practitioners were found to be similar (Table 2).

Table 2.

The distribution of the Pediatric Nutritional Risk Score (PNRS).

	1st practitioner		2nd practitioner
PNRS score	Number (n=149)	Percent (%)	Number (n=149)	Percent (%)
0 points	25	16.8	24	16.1
1 point	56	37.6	53	35.6
2 points	45	30.2	47	31.5
3 points	12	8.1	14	9.4
4 points	9	6.0	9	6.0
5 points	2	1.3	2	1.3

Comparison of the PNRS scores between the two blinded practitioners

In this study, 23 (15.4%) patients had high malnutrition risk according to the 1^st practitioner and 25 (16.7%) patients had high malnutrition risk according to the 2^nd practitioner (score≥3). The Kappa coefficient between the two practitioners was 0.66, which was a good level of consistency.

There was no difference between the high and middle-low risk groups in terms of pain. However, the risk was higher for those with <50% food intake (31.0% vs 39.1%, p=0.006). Similarly, those whose intensity of disease was high also had high risk of developing malnutrition (p<0.001).

The weight loss of more than 2% occurred in 65.2% of the high-risk group patients. In the moderate-low risk group, 25.4% of the patients lost weight (Table 3). The rate of malnutrition that developed at the hospital was 31.5%.

Table 3.

The relationship between the PNRS scores and results.

Weight loss	Moderate-low risk group (score<3)** Number (percent)	High-risk group (score ≥3)** Number (percent)	Total Number (percent)
Yes*	32 (25.4%)	15 (65.2%)	47 (31.5%)
No	94 (76.4%)	8 (34.8%)	102 (68.5%)
Total	126 (100.0%)	23 (100.0%)	149 (100.0%)

Low-moderate and high-risk categories grouped

^*Weight loss more than 2% of the reference weight,

^**Pediatric Nutritional Risk Scores (PNRS scores) of the 1st practitioner

When the specificity and sensitivity values were compared, ‘point 3’ which has the closest specificity and sensitivity, was accepted as the threshold. Based on this, the specificity, sensitivity, NPV, and PPV of PNRS were 82.1%, 77.8% 92.0%, and 58.3%, respectively (Table 4).

Table 4.

Specificity, sensitivity, positive, and negative predictive values of the PNRS scores.

Score*	Specificity (%) 95% CI**	Sensitivity (%) 95% CI**	Positive predictive value (%) 95% CI**	Negative predictive value (%) 95% CI**
1 point	34.8	86.7
2 points	45.0	89.5
3 points	82.1	77.8	58.3	92.0
4 points	92.0	77.8
5 points	95.8	33.3

Sensitivity and specificity of the nutritional screening tool in identifying different categories of the undernutrition risk at its best cut-off score

^*PNRS scores of the 1^st practitioner, CI: Confidence Interval

DISCUSSION

Malnutrition that develops during hospitalization is common worldwide. Factors, such as decreased food intake due to the disease, dysfunctions of the gastrointestinal system, increases in the metabolic rate, and insufficient nutritional support might lead to hospital malnutrition. The nutritional screening tests are the important tools for the identification of the malnutrition and of patients that have a malnutrition risk. The nutritional screening tools must identify the patients with hospital malnutrition risk in a simple and quick manner (15, 17). PNRS is one of the tools that has been developed for the children. PNRS is composed of three parameters: pain, decrease in food intake, and intensity of disease (13, 19). PNRS is recognized as a usable scale in the literature (20). In Turkey, no studies have been conducted on the validation, validity, and reliability of PNRS. Therefore, this study aims to validate PNRS for the Turkish version and focuses on the validity and reliability when it comes to its application on the inpatients. PNRS was applied by two different physicians to the same patient. A kappa coefficient of 0.66 was seen in the PNRS scores between the two practitioners. This good level of consistency shows that the application of PNRS validation was successful.

The median age was found to be similar to the other validation studies (20). In our study, the urinary tract infection was the most common diagnosis of the hospitalization while in the other studies, it was acute gastroenteritis and acute bronchiolitis (6, 8). The reason for this difference in our data might be the difference between the patient profiles and socio-economic differences between the countries.

In this study, 10.1% of the patients were diagnosed as stunted, 20.1% as wasted, and overall, 30.2% of patients had malnutrition at the time of admission. This is similar to the other studies in which the malnutrition rate was between 22.9% and 40.9% (21). In Turkey, we have similar ratios as between 27.0% and 40.9% (22). The two studies conducted by Öztürk et al. (8, 23) ten years after one another, 31.8% and 30.2% acute malnutrition was reported. Different ratios might be related to the heterogeneity in the patient population and the fact that the assessments take place in different clinics (surgery, internal diseases). The ratios in our study were similar to the data shown in the literature.

The prevalence of hospital malnutrition is up to 12–96% in the world while it is 32–55.7% in Turkey (8, 23–25).

This difference might stem from the heterogeneity of the patient population and different classifications. In our study, the rate of malnutrition is similar to the data in Turkey.

No difference was found between the high and middle-low risk groups in terms of the existence of pain. However, for those with the food intake of <50%, the risk for developing malnutrition was higher. Similarly, the risk for developing malnutrition was also higher for those with a higher intensity of the disease. This might be linked to hypermetabolism, swallowing difficulty, loss of appetite, and the need for longer fasting periods for certain practices, such as bloodletting. As expected, high intensity of a disease might lead to consumption of less than 50% of the daily need, and therefore, when the patient gets higher points from the two parameters in the scoring system, he is included in the high-risk group. This result shows how important it is to follow up and support the food intake.

The specificity and sensitivity were calculated separately for each point. When the values were compared, ‘point 3’ which has the closest specificity and sensitivity, was identified as the threshold. Based on this, the specificity of PNRS was 82.1%, and its sensitivity was 77.8%; NPV was 92.0% while PPV was 58.3%. In an Indonesian study, the threshold was identified as “point 2” and the sensitivity and specificity were found to be 79.0% and 71.0%, respectively, and PPV as 47.0% and NPV as 92.0% (20). The reason for this difference might be the use of different thresholds.

The aim of the study is to determine acute malnutrition using original scale. The main objective is to predict the significant weight loss risk from admission to discharge. The chronic malnutrition is characterized by height-for-age z scores of <−2 for ≥3-months period (26). However, the information about the 3-month period of patients could not be collected, and therefore, the percentage of chronic malnutrition may not be as accurate as the real data. We may lose a significant proportion of the patients with chronic malnutrition who did not reduce the weight-for-age z-scores to −2 within 3 months from starting with a higher z-score. This situation is a limitation of our study. Another limitation of this study is that the chronic malnutrition (stunted) is based on a single z score, so the patients with idiopathic short stature or structural growth delay may be over-diagnosed with chronic malnutrition.

In conclusion, malnutrition is an important and commonly seen problem in inpatients. It is recommended to use the screening tests for these patients to identify the risks of developing malnutrition. PNRS is one such screening test. In this study, the good consistency between different practitioners points to a successful implementation of the PNRS validation. When specificity, sensitivity, PPV, and NPV values are considered, it is seen that PNRS is stronger when it comes to differentiation of the patients with low-middle risk of developing malnutrition as compared to those with the high risk of developing malnutrition. Considering this, PNRS is accepted as a valid and reliable tool to identify the risk of inpatient malnutrition.

MAIN POINTS.

• Malnutrition is an important and commonly seen problem in inpatients.

• Hospital malnutrition can be easily foreseen with nutritional risk score scales.

• PNRS is accepted as a valid and reliable tool to identify the risk of inpatient malnutrition.